Next: On the Use of
Up: Descriptions of Our New
Previous: Comparison of the Methods
  Contents
  Index
MOS Calculation and Statistical Analysis
Let us denote by
the number of subjects in the chosen subjective method and by
the evaluation of sequence
made by user
. The set of values
will probably present variations due to the differences in judgment between subjects. Moreover, it is possible that some subjects do not pay enough attention during the experiment, or behave in some unusual way face to the sequences; this can lead to inconsistent data for the training phase. Some statistical filtering is thus necessary on the set of raw data.
The most widely used reference to deal with this topic is the ITU-R BT.500-10 recommendation, [67]. The described procedure allows to remove the ratings of those subjects who could not conduct consistent scores.
First, denote by
the mean of the evaluations of sequence
on the set of subjects, that is,
![\begin{displaymath}
\bar{u}_s=\frac{1}{N}\displaystyle \sum_{i=1}^{N}u_{is}.
\end{displaymath}](img17.gif) |
(41) |
Denote by
the 95%-confidence
interval obtained from the
, that is,
, where
As stated in [67], it must be ascertained whether this distribution of scores is normal or not using the
test (by calculating the ``kurtosis'' coefficient of the function, i.e. the ratio of the fourth order moment to the square of the second order moment). If
is between 2 and 4, the distribution may be taken to be normal. In symbols, denoting
where
if
then the distribution
can be assumed to be normal. For each subject
, we must compute two integer values
and
, following the following procedure:
ifff yyyy xxxx zzzz uuuu
set
and
for each sequence
if
, then
if
then
if
then
else
if
then
if
then
Finally, if
and
then the scores of subject
must be deleted. For more details about this topic and the other methods of subjective tests see [67].
After eliminating the scores of those subjects who could not conduct coherent ratings using the above technique, the mean score should be recomputed using Eq. 4.1. This will constitute the MOS database that we will use to train and test the NN.
Next: On the Use of
Up: Descriptions of Our New
Previous: Comparison of the Methods
  Contents
  Index
Samir Mohamed
2003-01-08