The Test Sessions

Following the ITU recommendations, overall subjective tests should be divided into multiple sessions and each session should not last more than 30 minutes. For each session, we should add several dummy sequences (about four or five) at the beginning. These sequences should not be taken into account in the calculation. Their aim is to be used as training samples for the subjects to learn how to give meaningful rates. Furthermore, the reliability of subjects can be qualitatively evaluated by checking their behavior when reference/reference pairs are given. In these cases, reliable subjects are expected to give evaluations very close to the maximum point in the quality scale. Hence, in these dummy samples, at least one of samples should be the reference. More advanced technique to check for the reliability of subjects is given in the next Section.