Limitations of Subjective and Objective Quality Tests

Although MOS studies have served as the basis for analyzing many aspects of signal processing, they present several limitations:

$\bullet$

(a)

stringent environments are required;

(b)

the process cannot be automated;

(c)

they are very costly and time consuming to repeat it frequently;

(d)

they are impossible to use in real-time quality assessment.

On the other hand, the disadvantages of objective methods are:

$\bullet$

(a)

they do not correlate well with human perception;

(b)

they require high calculation power, and are then time consuming;

(c)

they cannot be used for real-time quality assessment, as they work on both the original video sequence or speech sample and the transmitted/distorted one;

(d)

it is difficult to build a model that takes into account the effect of many quality-affecting parameters at the same time, especially network parameters.

The requirements of the kind of good objective quality measure we are looking for are: the evaluation must be possible in real time, it must be computationally simple (not time consuming), there is no need to access the original signal, all the quality-affecting parameters including network factors can be considered (that is the test must not be limited to a specific kind of parameters) and the obtained results correlate well with human perception. It is important to mention that, so far, there is no objective measure that can fulfill these requirements. Thus, one of our objectives is to propose a new objective measure that can satisfy all the stated requirements.