2011 Signal Separation Evaluation Campaign
Under-determined speech and music mixtures
Dev2 database

For details about each algorithm, click on "Algorithm".
Performance are evaluated using SIR/SDR/SAR metrics defined by BSS_EVAL and OPS/TPS/IPS/APS defined in PEASS 2.0. The development data are distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 2.0 license. They can be freely used for non-commercial purposes and redistributed under the same license, provided the authors are acknowledged (Another Dreamer and Alex Q for music sources, Emmanuel Vincent and Hiroshi Sawada for other data).

AVERAGE RESULTS

Average for 4 speech mixtures (male4&female4)
Average for music mixtures (nodrums&wdrums)

DETAILED RESULTS


Instantaneous mixtures




female4
mix
src1 src2 src3 src4 
sim1 sim2 sim3 sim4 		 male4
mix
src1 src2 src3 src4 
sim1 sim2 sim3 sim4 		 nodrums
mix
src1 src2 src3 
sim1 sim2 sim3 		 wdrums
mix
src1 src2 src3 
sim1 sim2 sim3 
Algorithm
A. Ozerov and E. Vincent
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
 8.1  5.1  7.2 13.8
13.6 13.5 14.6 26.1
16.0  9.5 15.5 19.4
 9.1  5.9  7.3 15.2

49.2 34.8 37.5 41.1
70.0 54.4 60.5 57.4
67.0 51.9 59.7 69.1
50.1 45.1 43.3 78.0 		sim1 sim2 sim3 sim4 
 3.4  8.0 16.7  6.1
 8.1 17.2 29.6 11.4
 9.2 12.6 20.9 12.5
 2.8  9.4 19.0  6.4

27.4 43.3 42.9 31.1
47.2 70.6 55.4 55.9
58.5 60.4 77.1 58.2
28.7 48.1 82.0 38.5 		sim1 sim2 sim3 
27.0 21.4  7.5
33.3 31.7 18.0
35.8 27.0 12.9
28.9 23.3  9.4

45.2 98.9 33.0
64.3 92.8 59.7
61.5 96.8 58.0
63.0 86.7 73.6 		sim1 sim2 sim3 
 4.0 32.2  7.5
13.1 51.6  9.8
 7.1 34.6 20.8
 8.9 36.0  8.9

46.4 36.6 45.4
73.8 76.8 63.5
75.4 74.4 67.2
32.5 86.7 50.8
Algorithm
B. Makkiabadi(*1)
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 src1 src2 src3 src4 
 9.2 -4.9  0.7 -4.2

13.2 -3.5  4.6  3.4
11.6  5.8  4.2 -1.7

39.9 20.0 27.3 22.9

49.7 16.2 38.8 55.6
52.7  8.9  2.0  0.1 		src1 src2 src3 src4 
-6.1  3.0  0.7 13.1

-0.1  9.9  3.8 16.8
-1.8  4.5  5.1 15.6

19.9 30.0 23.0 34.4

47.5 59.7 31.6 52.4
 0.1  1.2  2.7 70.2 		src1 src2 src3 
 3.1  9.8 -13.4

 6.4 14.9 -12.6
 6.7 11.6  7.5

40.7 35.1 43.3

79.5 84.5 78.7
 0.0  0.0  0.0 		src1 src2 src3 
10.2  3.4 -11.4

22.5  4.0 -5.1
10.5 13.8 -4.0

40.8 41.1 48.4

88.9 81.0 92.1
 0.0  0.0  0.0
(*1) The submission (and evaluated) duration is 9.99 sec.


Live recordings with 1m microphone spacing and 250ms reverberation time




female4
mix
src1 src2 src3 src4 
sim1 sim2 sim3 sim4 		 male4
mix
src1 src2 src3 src4 
sim1 sim2 sim3 sim4 		 nodrums
mix
src1 src2 src3 
sim1 sim2 sim3 		 wdrums
mix
src1 src2 src3 
sim1 sim2 sim3 
Algorithm
Y.Hirasawa
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
 1.2  1.1  3.4  0.5
 3.7  3.2  5.3  2.2
 1.6  0.7  8.0 -1.8
 4.4  2.9  5.0  1.9

20.1 19.4 17.7 20.9
31.3 21.4 23.8 25.8
44.3 39.8 39.8 48.0
36.3 33.6 34.9 33.6 		sim1 sim2 sim3 sim4 
 1.1  0.9  2.0  1.1
 3.7  3.4  3.3  2.9
 1.0 -0.1  4.7  1.0
 3.0  2.4  2.4  2.1

16.9 15.1 28.4 22.0
19.1 22.1 38.8 27.2
34.8 36.4 54.0 45.1
32.9 33.8 37.9 34.2



















Algorithm
K.Iso
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
 3.7  7.8  3.6  4.5
 6.6 12.0  5.3  8.2
 5.1 11.0  6.8  6.3
 7.7 12.2  7.3  8.7

34.0 29.9 34.7 24.2
54.3 39.7 52.2 40.3
34.2 33.2 42.0 21.7
55.9 46.9 54.8 52.1 		sim1 sim2 sim3 sim4 
 3.6  2.1  1.6  5.7
 6.7  5.4  4.0  8.6
 5.7  2.5  1.6  8.5
 7.6  6.5  5.0  9.0

37.4 22.1 16.4 33.5
59.8 17.4 17.7 50.3
44.7 18.7 13.2 38.4
54.9 38.1 41.7 54.7



















Algorithm
J.Cho
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
-0.2 -0.6  2.6 -0.1
 1.7  2.2  6.5  2.6
-4.2 -3.6  3.5 -2.4
 4.5  4.5  9.1  4.5

15.6 14.4 16.5 21.1
 8.3  7.7 18.0 17.9
 8.5  7.5 11.2 20.4
34.1 34.5 43.7 35.2 		sim1 sim2 sim3 sim4 
 1.8  0.2  3.7  2.3
 5.8  2.7  7.0  5.4
 2.0 -2.2  5.4  3.7
 5.4  3.0  7.7  5.6

25.1 16.5 34.5 35.8
30.6 16.1 55.9 53.8
22.4 18.7 43.7 40.6
47.8 41.2 53.9 53.5 		sim1 sim2 sim3 
-4.3 -2.9  1.8
 2.8  5.6  2.7
-7.7 -3.0  5.1
 4.9  7.0  5.3

 9.1 32.6 30.0
12.7 52.7 28.4
 7.6 31.1 33.0
46.6 54.1 32.8 		sim1 sim2 sim3 
 2.6  2.5  3.6
 6.1  8.8  5.9
 5.2  4.4  7.3
 8.1  8.2  6.4

33.8 28.0 33.6
51.1 49.3 62.4
43.2 23.7 37.1
51.0 50.4 49.3
Algorithm
F.Nesta (1)
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
 2.1  0.6  2.2  2.4
 2.9  4.5  5.9  4.1
 6.1 -1.3  2.3  4.8
 4.3  5.9  6.1  5.5

33.1 33.0 38.4 32.6
47.6 60.0 64.0 49.9
49.7 45.9 50.1 46.1
42.7 44.9 52.0 44.9 		sim1 sim2 sim3 sim4 
 3.7  2.8  2.5  1.5
 5.6  5.8  5.3  4.0
 6.9  4.4  3.8  1.1
 6.0  5.5  4.4  3.3

40.7 36.9 30.1 26.3
61.8 62.5 51.7 34.4
59.9 49.8 45.6 43.7
48.2 46.7 37.5 33.2 		sim1 sim2 sim3 
 7.0  0.6  5.1
 7.9  10.0  6.7
19.2  0.1 10.9
12.4  7.6  8.3

39.6 15.6 40.0
54.0 44.9 59.1
52.2 24.9 53.5
53.6 38.2 52.8 		sim1 sim2 sim3 
 5.0  5.5  4.8
 6.6  9.7  7.6
12.4  7.5  6.3
 9.4  9.0  9.6

39.5 34.3 39.1
62.2 65.8 68.1
52.5 45.7 55.7
57.0 45.5 44.6
Algorithm
F.Nesta (2)
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
 3.1 -0.4  2.6  2.9
 4.1  5.4  7.4  4.9
 8.0 -2.8  3.5  6.2
 5.4  7.3  6.9  6.3

31.2 31.4 34.3 30.0
43.8 48.9 56.9 45.1
43.8 37.0 41.8 38.7
46.1 51.8 56.8 48.2 		sim1 sim2 sim3 sim4 
 4.8  3.3  3.1  1.9
 7.8  7.0  6.2  4.7
 8.5  5.3  4.8  1.9
 7.0  6.6  5.4  4.2

41.0 35.5 31.1 26.0
58.7 58.2 53.7 31.5
59.8 47.6 44.6 36.8
50.3 51.6 46.2 41.4 		sim1 sim2 sim3 
 8.6 -0.1  5.6
10.3 14.0  7.5
17.0  0.1 15.4
14.1  9.7  8.0

22.9 14.9 38.4
54.2 40.0 55.0
39.9 22.9 58.7
49.6 47.3 47.1 		sim1 sim2 sim3 
 5.3  6.2  5.5
 6.9 13.1 10.3
13.1  8.5  7.1
 9.8  9.6 10.9

27.6 34.3 36.3
58.8 56.8 59.8
38.4 42.1 51.5
60.2 54.5 50.5
Algorithm
A.Ozerov and E. Vincent
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
 2.2  3.7  5.9  1.0
 5.2  7.4 11.3  4.0
 4.6  6.6  8.6  1.0
 5.4  6.7  9.8  4.3

24.5 36.8 33.1 34.1
30.7 56.7 47.3 48.8
31.8 44.1 37.5 40.1
41.7 53.7 50.9 51.5 		sim1 sim2 sim3 sim4 
 2.1  0.0  4.2  1.2
 6.5  2.7  8.0  4.5
 2.9 -2.1  6.7  2.2
 5.4  3.4  7.2  5.0

33.2 22.0 41.7 35.2
53.9 15.7 58.2 56.4
43.4 33.9 54.2 45.7
47.5 31.8 54.5 47.8 		sim1 sim2 sim3 
-6.7  5.0  2.7
 4.5  6.8  3.5
-8.4 11.9 10.2
 9.4  9.6  8.2

 8.5 38.9 14.7
 2.4 60.4  3.7
 0.4 50.0 16.3
35.6 60.6 11.0 		sim1 sim2 sim3 
 5.6  4.6  4.7
11.9  8.4  7.8
 7.3  8.7  7.8
10.8  7.3  8.7

27.2 36.1 35.7
62.7 66.8 65.4
44.6 44.3 51.6
66.9 50.9 43.6


Live recordings with 5cm microphone spacing and 250ms reverberation time




female4
mix
src1 src2 src3 src4 
sim1 sim2 sim3 sim4 		 male4
mix
src1 src2 src3 src4 
sim1 sim2 sim3 sim4 		 nodrums
mix
src1 src2 src3 
sim1 sim2 sim3 		 wdrums
mix
src1 src2 src3 
sim1 sim2 sim3 
Algorithm
K.Iso
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
 1.2  4.8  4.0  2.9
 4.1  8.2  5.6  7.2
 0.5  7.3  7.8  3.5
 6.6  7.8  9.2  7.6

28.4 24.4 33.5 32.7
48.1 55.2 41.9 51.5
29.2 37.0 36.4 35.9
59.4 69.6 47.9 54.5 		sim1 sim2 sim3 sim4 
 4.2  0.5  2.8  4.1
 8.6  1.7  4.9  9.9
 5.6 -1.3  4.8  4.3
 8.8  4.2  5.3  9.3

31.6 26.0 35.6 31.8
60.7 25.7 52.0 57.3
35.0 26.6 43.5 41.8
62.0 42.0 51.0 64.3



















Algorithm
J.Cho
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
-1.3 -0.2  2.7  2.0
 2.0  2.5  5.5  4.4
-4.5 -3.1  3.5  3.1
 5.6  5.1  6.8  5.0

12.7 31.5 17.5 19.8
 9.8 41.9 16.6 12.0
 4.2 30.2 10.7 12.6
40.0 51.2 42.5 23.0 		sim1 sim2 sim3 sim4 
 2.1 -0.0  2.5  1.7
 7.6  1.7  6.5  4.5
 3.0 -4.4  2.7  2.4
 6.5  2.5  7.9  5.9

34.5 19.2 27.0 34.2
63.5 13.0 46.6 51.7
32.8 18.6 25.9 37.0
54.6 38.8 57.8 54.1 		sim1 sim2 sim3 
 2.3 -1.1 -9.4
 2.8  8.3  5.2
 9.6 -1.1 -11.5
 6.1  6.5  8.6

33.6 37.5 10.6
36.7 61.4 20.7
37.9 44.3  6.8
37.0 57.5 51.7 		sim1 sim2 sim3 
 2.1  0.4  3.1
 9.3  5.6  7.8
 2.9  1.9  4.5
 7.0  6.7  7.1

33.9 35.7 33.4
55.7 63.9 55.8
43.3 40.9 32.0
56.0 58.5 53.6
Algorithm
F. Nesta
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
 4.0  1.0  3.4  2.5
 7.8  4.5  5.6  7.0
 7.1  1.4  5.8  3.8
 5.8  4.2  5.2  5.6

39.3 37.6 34.9 33.7
64.4 62.1 50.5 53.8
49.7 47.5 45.7 46.9
55.5 49.7 47.9 46.3 		sim1 sim2 sim3 sim4 
 4.2  3.7  0.5  2.8
 7.1  8.2  3.8  5.1
 8.5  4.6 -1.3  6.0
 6.2  6.9  3.8  4.3

39.2 41.3 24.8 34.3
64.5 63.2 25.2 58.0
54.7 52.8 31.0 52.5
47.9 51.3 38.5 41.9 		sim1 sim2 sim3 
 5.8  4.0 -5.7
 6.6  9.8  10.0
19.3  5.5 -6.4
11.3  8.1  7.4

34.0 36.1 18.0
63.6 59.8 49.2
58.3 48.1 26.8
65.4 63.9 39.3 		sim1 sim2 sim3 
 4.4  4.8  3.9
 7.1  8.5  7.5
 8.4  6.8  5.5
 7.9  8.6  7.6

40.4 35.7 38.1
68.5 59.1 62.2
50.7 50.8 51.3
53.2 46.4 60.2
Algorithm
F.Nesta
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
 4.6  1.3  4.0  2.9
 8.7  4.7  6.5  7.4
 8.4  2.8  7.1  4.3
 6.2  4.3  5.7  6.1

33.8 35.0 33.4 32.3
58.1 51.8 44.3 48.0
42.8 41.1 45.1 44.2
59.9 53.8 45.5 48.5 		sim1 sim2 sim3 sim4 
 5.0  3.9  0.9  3.3
 8.2  9.1  4.0  5.2
10.2  5.0 -1.0  8.4
 7.1  7.2  4.8  4.7

39.1 39.2 23.9 33.7
61.4 59.8 28.2 52.9
54.6 51.4 28.8 53.9
51.7 54.8 41.5 45.0 		sim1 sim2 sim3 
 6.2  5.0 -6.1
 7.1  9.2 10.4
17.7  8.7 -6.3
12.4  7.6  8.7

26.5 41.7 18.7
68.0 60.1 40.6
49.2 53.2 28.8
62.0 55.4 45.4 		sim1 sim2 sim3 
 5.0  5.2  4.1
 7.9  9.8  7.8
 9.7  7.0  6.0
 8.3  9.5  7.5

40.8 34.5 28.6
62.6 50.0 58.8
53.3 49.3 41.0
54.1 48.1 58.9
Algorithm
A.Ozerov and E. Vincent
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 sim3 sim4 
 0.4 -0.9  0.9  1.8
 2.9  8.4  1.7  7.8
 0.1 -1.6  2.3  2.4
 2.3  6.7  3.3  5.8

27.4 32.6 33.2 35.3
35.8 55.5 48.1 59.8
33.3 34.0 41.0 41.8
45.5 57.8 49.8 59.2 		sim1 sim2 sim3 sim4 
 1.3 -0.4  2.2  0.4
 5.4  3.3  3.5  5.8
 0.6 -3.1  6.7  1.0
 4.8  4.9  4.1  7.0

27.9 21.6 30.6 16.6
26.4 21.4 47.1 18.1
35.3 30.8 30.5  7.9
39.2 36.2 54.2 44.6 		sim1 sim2 sim3 
-6.0  5.1  6.7
 0.7  7.8  9.4
-13.3 11.0 11.2
 5.5  8.2 11.6

 5.7 40.1 14.8
 2.5 58.1  5.8
 1.1 49.9 11.7
37.5 55.1  7.1 		sim1 sim2 sim3 
 3.7  4.7  3.8
 6.4  6.8 12.3
 7.5 10.7  4.9
 6.5  9.6  9.1

32.8 39.2 32.3
58.6 57.3 74.1
45.0 51.7 39.3
58.1 58.7 48.0