A Mean Opinion Score (MOS) Database for Different Languages

Here, we collected a database conformed by a set of samples of distorted speech signals, according to the quality-affecting parameters variations chosen as described in Section 5.2.1 and as shown in Table 5.2. Distorted speech signals were generated by using an IP network testbed, comprised by a sender, a router, and a receiver. The sender controlled the packetization interval and the selection of the encoding algorithm. The router controlled the loss rate, and the loss pattern distribution. Finally, the receiver stored the received signals, decoded them, substituted lost packets by silence periods, and calculated the loss rate and number of consecutively lost packets. Of course, one can substitute lost packets by comfort noise, waveform substitution, etc. (See Section 3.3.2, page  for more details). For simplicity we chose silence insertion. Each sample consists of speech material sent from the sender to the receiver according to a given combination of network parameters values obtained in Section 5.2.1 and in data from the literature. In addition, it has been established that encoders' performance shows language dependency [79]. Thus, language type is another variable included in our database. We conducted MOS tests in three different languages: Spanish, Arabic and French. The quality scores were obtained through a series of MOS experiments, which were carried out following an ITU-recommended method [70] named ACR (See Section 4.3 for more details about how to carry out subjective quality tests). During the tests, groups of 15, 15 and 7 subjects (for Arabic, Spanish, and French langauges respectively) were asked to indicate the quality of the speech they heard on a 5-point quality scale for each sample (a total of 96, 96 and 65 for Arabic, Spanish and French languages respectively). Then, a specific statistical analysis was carried out to identify the subjects who were not able to provide reliable ratings as described in Section 4.4. As a result, we removed all the rates of 1 and 3 subjects from the Spanish and Arabic languages respectively. After that, the MOS for each sample were calculated. In this way, we generated the database of distorted speech signals and their corresponding quality scores. The results are shown in Table 5.2.

Table 5.2: The quality databases for both Arabic and Spanish languages to train and test the NN. We omitted the data of French language as the subjective test were carried out by only 7 subjects. In addition, we did not consider nor ADPCM codec nor the same combinations of the parameters as Arabic and Spanish languages.

Codec	Packetization	Loss	# Consecutive	MOS for Arabic		MOS for Spanish
Name	Interval	Rate	Lost	Actual	Predicted	Actual	Predicted
	PI (ms)	LR (%)	Packets CLP
GSM	40	10	1	2.92	2.72	3.13	2.79
GSM	20	10	2	3.17	2.80	2.40	2.75
PCM	40	20	1	2.08	2.40	2.53	2.51
GSM	80	40	2	1.33	1.41	1.40	1.46
PCM	80	10	2	3.08	2.79	3.33	3.20
ADPCM	40	10	5	3.00	2.97	3.40	3.15
PCM	60	5	2	3.50	3.69	3.60	3.80
PCM	60	10	2	3.28	3.02	3.33	3.19
PCM	20	20	2	2.50	2.58	2.53	2.49
ADPCM	80	10	3	3.08	2.86	3.00	3.03
ADPCM	20	10	1	2.25	2.53	2.13	2.70
PCM	40	10	5	3.00	2.90	3.73	3.67
PCM	40	10	3	3.42	3.20	3.00	3.34
PCM	60	5	3	3.58	3.67	3.87	3.99
PCM	40	20	2	2.50	2.59	2.40	2.64
GSM	80	5	3	3.67	3.50	3.67	3.64
PCM	40	40	2	2.00	1.88	1.80	1.67
PCM	40	10	4	3.17	3.08	3.47	3.50
ADPCM	80	5	3	3.33	3.56	3.27	3.67
PCM	40	20	3	2.25	2.54	2.40	2.78
PCM	40	5	3	3.75	3.75	4.00	3.93
PCM	80	5	2	3.58	3.52	3.93	3.85
PCM	20	5	3	3.67	3.63	3.67	3.85
PCM	40	40	2	1.75	1.88	1.73	1.67
GSM	60	5	3	3.29	3.54	3.40	3.59
ADPCM	80	10	5	2.58	2.80	2.93	3.25
PCM	40	40	1	1.42	1.56	1.67	1.61
PCM	40	20	1	2.58	2.40	2.60	2.51
ADPCM	60	5	3	3.34	3.70	3.47	3.62
PCM	40	5	2	3.42	3.71	4.13	3.75
PCM	20	40	2	1.67	1.67	1.40	1.70
GSM	40	10	2	3.00	2.88	2.80	2.84
ADPCM	60	10	3	3.00	3.11	3.13	2.98
ADPCM	60	10	5	2.75	2.82	3.13	3.21
PCM	60	20	2	2.67	2.30	2.60	2.65
ADPCM	-	0	-	4.25	4.23	4.07	4.04
ADPCM	40	5	3	3.67	3.67	3.60	3.55
GSM	60	40	2	1.50	1.47	1.40	1.38
GSM	60	20	2	2.50	2.39	2.20	2.24
ADPCM	40	40	1	1.75	1.67	1.27	1.44
GSM	40	20	2	2.08	2.49	2.13	2.23
ADPCM	40	10	1	2.92	2.83	2.73	2.77
ADPCM	80	10	4	2.67	2.87	2.93	3.13
ADPCM	80	10	2	2.75	2.76	3.07	2.93
GSM	80	20	2	2.08	2.12	2.33	2.23
GSM	20	5	2	3.50	3.15	3.20	3.34
ADPCM	40	10	1	2.67	2.83	2.93	2.77
GSM	80	5	2	3.67	3.39	3.27	3.58
GSM	40	40	2	1.67	1.66	1.53	1.40
GSM	20	40	2	1.50	1.60	1.27	1.41
PCM	40	5	3	3.92	3.75	3.87	3.93
ADPCM	20	10	5	3.28	2.98	3.27	3.08
ADPCM	60	10	4	3.00	3.02	3.13	3.09
ADPCM	60	10	2	2.92	3.01	3.00	2.89
ADPCM	20	10	3	3.08	3.16	2.80	2.87
PCM	40	5	2	3.67	3.71	3.73	3.75
PCM	40	40	5	1.50	1.58	1.80	1.92
PCM	20	5	2	3.33	3.48	3.53	3.68
PCM	80	5	3	3.58	3.59	3.93	4.03
GSM	-	0	-	3.75	3.90	3.80	3.94
ADPCM	40	20	1	2.58	2.41	2.53	2.24
ADPCM	40	10	2	3.17	3.10	2.80	2.85
PCM	-	0	-	4.67	4.49	4.60	4.45
GSM	40	5	3	3.58	3.48	3.87	3.52
PCM	40	5	1	3.42	3.47	3.67	3.59
GSM	40	5	1	3.00	3.08	3.13	3.39
ADPCM	20	10	4	3.00	3.13	2.73	2.97
PCM	40	20	4	2.17	2.39	3.27	2.92
ADPCM	20	10	2	3.00	2.98	2.67	2.78
PCM	40	40	3	2.08	1.91	2.13	1.74
PCM	40	20	5	2.17	2.21	2.80	3.07
PCM	40	5	4	3.92	3.67	4.27	4.12
PCM	40	40	4	1.83	1.76	1.93	1.83
PCM	40	5	1	3.58	3.47	3.60	3.59
ADPCM	40	5	1	2.75	3.25	3.13	3.34
PCM	80	20	2	2.25	2.13	2.53	2.62
ADPCM	40	10	3	2.92	3.22	3.00	2.94
PCM	80	40	2	1.83	1.84	1.53	1.65
ADPCM	40	10	4	3.17	3.17	3.33	3.04
PCM	20	10	2	3.42	3.05	3.27	3.15
GSM	40	20	1	2.17	2.38	2.30	2.19
PCM	40	10	2	3.33	3.20	3.33	3.19
GSM	60	5	2	3.50	3.34	3.73	3.52
PCM	40	5	5	3.75	3.53	4.00	4.31
GSM	80	10	2	2.57	2.81	3.20	2.98
GSM	60	10	2	2.83	2.89	2.60	2.91
PCM	40	10	1	2.75	2.97	2.73	3.05
PCM	60	40	2	1.83	1.90	1.93	1.62
ADPCM	60	10	1	2.75	2.88	2.73	2.80
GSM	20	5	3	3.33	3.35	3.20	3.41
ADPCM	20	5	3	3.25	3.53	3.17	3.45
GSM	20	20	2	2.08	2.38	2.13	2.10
ADPCM	80	10	1	2.92	2.77	2.73	2.85
PCM	40	10	2	3.25	3.20	3.13	3.19
GSM	40	40	1	1.58	1.76	1.27	1.39
GSM	40	5	2	3.17	3.27	3.53	3.45