2011 Signal Separation Evaluation Campaign
Under-determined speech and music mixtures
Dev3 database

For details about each algorithm, click on "Algorithm".
Performance are evaluated using SIR/SDR/SAR metrics defined by BSS_EVAL and OPS/TPS/IPS/APS defined in PEASS 2.0.

The audio files below are made available under the terms of the Creative Commons Attribution-NonC ommercial-ShareAlike 2.0 license. The authors is Shoko Araki for mixture signals.

AVERAGE RESULTS

Total average for each conditions

DETAILED RESULTS


Instantaneous mixtures


female4
mix
src1 src2 src3 src4 
sim1 sim2 sim3 sim4 		 male4
mix
src1 src2 src3 src4 
sim1 sim2 sim3 sim4 
Algorithm
B. Makkiabadi(*1)
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
10.4 15.6  7.8 12.4

18.5 22.5 15.0 20.2
11.2 16.7  8.8 13.3

44.8 35.7 36.8 50.6

61.9 69.8 59.4 66.4
25.4 81.2  8.3 44.0 		sim1 sim2 sim3 sim4 
15.2  4.6  7.4 20.0

22.0 11.2 13.9 30.2
16.2  6.0  8.7 20.5

51.2 31.3 35.4 95.5

69.9 47.1 53.2 95.5
67.2  6.8 10.7 86.7
Algorithm
M. Cobos
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 src1 src2 src3 src4 
10.2  8.0 10.7 10.3

23.9 23.1 23.1 24.1
10.4  8.1 11.0 10.5

39.7 31.6 43.3 59.9

77.7 76.9 75.0 79.1
 7.5  3.0 11.1 29.9 		src1 src2 src3 src4 
 5.6  7.4  3.4  5.3

22.4 18.8 14.2 16.4
 5.7  7.8  4.0  5.8

15.4 35.6 32.6 28.6

77.1 76.3 69.6 75.2
 0.1  4.8  2.3  1.7
(*1) The submission (and evaluated) duration is 9.99 sec.


Simulated recordings with 50cm microphone spacing and 130ms reverberation time


female4
mix
src1 src2 src3 src4 
sim1 sim2 sim3 sim4 		 male4
mix
src1 src2 src3 src4 
sim1 sim2 sim3 sim4 
Algorithm
Y.Hirasawa
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
 4.0  2.9  4.2  4.0
 8.4  4.5  5.6  6.6
 7.2  7.1 13.1  8.8
 5.6  4.1  5.8  5.9

27.8 28.4 29.8 23.8
52.6 54.9 47.6 56.6
62.0 65.2 62.7 58.5
36.4 32.7 37.7 35.0 		sim1 sim2 sim3 sim4 
 1.9  3.3  2.5  3.0
 6.0  4.8  3.8  5.4
 1.7  8.2  5.7  6.3
 4.8  4.8  3.7  4.8

25.6 29.3 30.4 26.6
57.8 48.6 53.4 53.7
55.7 58.5 64.5 58.9
34.6 35.2 36.5 39.4
Algorithm
J.Cho
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
 2.2  3.0  4.2  2.2
 3.2  4.2  5.3  3.9
 5.2  6.8  9.1  6.6
 6.4  6.7  8.0  5.7

33.0 36.2 36.8 27.0
68.1 77.7 72.3 64.8
65.1 66.9 67.7 63.7
31.2 23.2 30.5 18.1 		sim1 sim2 sim3 sim4 
 1.9  3.5  3.0  2.6
 3.7  4.8  4.1  4.4
 4.4  7.5  8.2  4.8
 4.9  6.9  7.5  5.9

31.4 32.4 36.8 30.5
77.6 74.4 71.8 72.8
64.3 68.5 68.7 68.0
18.0 20.0 30.6 18.1
Algorithm
H.Sawada
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
 6.1  9.2  7.9  4.4
11.6 15.0 11.9 10.2
11.5 17.2 15.2  8.0
 7.5 10.3  9.8  6.9

32.3 38.3 40.8 28.7
62.7 74.2 67.5 63.4
65.1 69.1 66.6 57.2
36.2 28.1 42.7 31.5 		sim1 sim2 sim3 sim4 
 7.8  5.3  4.4  7.4
13.1 11.8  9.1 11.6
14.9  8.7  8.5 16.2
 8.9  7.0  6.3  8.1

37.0 31.9 34.5 33.4
71.5 77.5 68.6 68.1
69.8 67.8 66.9 72.2
30.1 16.5 32.7 26.4


Simulated recordings with 5cm microphone spacing and 130ms reverberation time


female4
mix
src1 src2 src3 src4 
sim1 sim2 sim3 sim4 		 male4
mix
src1 src2 src3 src4 
sim1 sim2 sim3 sim4 
Algorithm
J.Cho
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
 2.4  3.9  1.6  0.8
 4.2  4.9  3.3  2.6
 3.9  8.6  2.8  2.1
 8.5  9.5  6.1  8.5

40.6 46.3 42.8 38.9
70.6 70.2 63.4 62.3
61.1 64.2 57.4 51.7
43.4 47.1 61.1 56.1 		sim1 sim2 sim3 sim4 
 2.2  2.7  2.6  2.3
 4.4  4.8  4.4  4.0
 4.6  6.8  6.8  5.3
 5.8  5.4  7.4  6.5

41.7 44.6 40.8 37.9
71.5 68.1 64.5 74.4
60.1 61.1 56.2 62.9
45.9 52.5 63.3 33.9
Algorithm
H.Sawada
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
 9.0  8.9  7.8  7.5
14.6 15.8 14.0 12.4
17.1 16.0 14.4 14.7
10.2  9.9  9.0  8.6

32.7 39.5 39.7 41.7
63.5 66.5 68.2 67.0
71.8 69.8 59.7 68.1
30.2 39.2 48.1 42.9 		sim1 sim2 sim3 sim4 
 6.1  6.9  7.2  7.2
12.5 12.9 11.7 11.8
10.9 12.7 15.5 14.1
 7.3  8.1  8.3  8.6

35.0 39.5 28.5 37.6
67.2 67.1 65.9 70.3
69.9 65.9 71.3 70.7
32.0 42.6 17.8 31.1


Simulated recordings with 50cm microphone spacing and 380ms reverberation time


female4
mix
src1 src2 src3 src4 
sim1 sim2 sim3 sim4 		 male4
mix
src1 src2 src3 src4 
sim1 sim2 sim3 sim4 
Algorithm
Y.Hirasawa
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
 0.9  0.1  1.5  0.3
 2.6  0.8  3.2  0.9
 0.4 -3.9  2.1 -3.3
 2.9  0.2  2.7  0.0

15.6 11.9 17.5 13.5
29.3 13.6 23.8 10.4
54.7 49.5 53.9 47.2
33.0 24.6 29.8 22.1 		sim1 sim2 sim3 sim4 
 0.3  0.5  0.4  0.4
 2.1  1.3  1.1  2.8
-2.0 -2.5 -2.0 -1.3
 2.9  0.6  1.5  2.8

12.2 17.6 18.4 13.0
30.8 15.5 20.8 27.7
41.5 52.5 50.4 46.8
37.6 25.7 30.2 34.8
Algorithm
J.Cho
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
-0.8 -0.1 -0.1 -0.1
 0.9  1.3  2.2  1.1
-4.5 -1.9 -2.2 -1.6
 5.6  5.6  6.5  5.9

23.6 23.4 22.5 21.4
43.5 54.5 52.1 23.3
47.1 58.0 58.9 51.1
27.8 24.0 20.9 25.2 		sim1 sim2 sim3 sim4 
-0.2  0.1  2.0 -0.2
 1.8  1.4  3.4  1.7
 0.1 -0.9  4.6 -2.8
 5.0  4.2  6.4  4.5

20.5 22.7 27.4 16.8
41.9 21.5 61.5 31.8
53.4 54.3 69.2 43.4
20.0 19.0 20.6 26.4
Algorithm
H.Sawada
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
 4.2  4.4  2.5  4.6
 8.1  8.6  5.3  7.6
 7.7  7.4  4.2  8.1
 6.9  7.3  5.5  7.6

30.1 31.4 31.4 29.0
68.2 69.3 63.0 61.7
62.1 64.3 57.5 63.6
33.2 29.6 40.2 31.3 		sim1 sim2 sim3 sim4 
 3.4  5.0  4.5  3.5
 7.0  9.5  7.2  8.1
 6.8  8.2  9.5  6.7
 4.9  7.6  6.6  5.8

26.0 25.3 24.6 24.9
65.8 61.0 38.2 58.0
59.4 43.3 54.6 57.4
22.1 42.1 34.4 26.1


Simulated recordings with 5cm microphone spacing and 380ms reverberation time


female4
mix
src1 src2 src3 src4 
sim1 sim2 sim3 sim4 		 male4
mix
src1 src2 src3 src4 
sim1 sim2 sim3 sim4 
Algorithm
J.Cho
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
-0.1 -1.0 -0.5 -0.4
 1.0  1.8  0.8  1.9
-2.6 -0.4 -2.4  0.6
 5.3  4.3  4.3  4.8

25.2 23.0 26.4 19.8
41.4 59.6 53.3 42.6
60.7 54.9 50.7 53.3
27.2 19.3 33.8 23.0 		sim1 sim2 sim3 sim4 
-0.1 -0.0 -0.5  0.2
 1.9  1.7  2.3  1.9
 1.9 -0.6  0.3  1.6
 4.5  3.1  4.3  4.8

25.5 26.1 29.3 21.9
48.8 49.5 56.7 40.5
59.0 62.7 63.1 56.0
25.2 24.5 30.6 21.2
Algorithm
H.Sawada
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
 4.2  3.3  4.0  3.5
 5.6  7.9  9.6  7.5
10.9  5.6  5.6  5.9
 7.3  6.4  7.6  6.5

36.2 26.8 20.7 15.3
64.2 63.7 42.5 53.4
61.8 56.7 42.3 33.5
44.7 37.1 43.3 41.6 		sim1 sim2 sim3 sim4 
 3.2  4.1  3.5  4.4
 6.5  8.0  8.1  7.7
 5.7  7.9  5.6  8.6
 5.0  6.5  6.2  6.4

32.8 23.6 27.9 30.0
62.1 51.8 53.2 65.3
62.0 44.1 55.5 63.5
41.6 44.5 40.5 30.1