2010 Signal Separation Evaluation Campaign
Determined and over-determined speech and music mixtures

For details about each algorithm, click on "Algorithm".

The evaluation with OPS/TPS/IPS/APS were evaluated just for stereo (2 microphone) mixtures.

The mixture audio files are licensed for research use only by their authors Kenneth Hild (Iliad), Hiroshi Sawada (rooms 4 and 5), Mads Dyrholm (rooms 1, 2, 3, C and O), Benedikt Loesch (room L), and Lucas Parra.

The music used for recordings in rooms 1, 2 and 3 was taken from "Germ Germ" by Das Bose Ding and has kindly been approved for public presentation by Jan Klare of Das Bose Ding in the name of research. See Lucas Parra's BSS page for the original dataset. The music signals for roomL are songs from danosongs.com.
The speech sources of roomL come from the CHAINS database. We thank to Fred Cummins.

All other audio files are made available under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 2.0 license.

The results obtaind in the last SiSEC (SiSEC2008) and the ideal binary masks can be found in this page.

Total average for each conditions
Iliad
Rooms4&5
Room1
Rooms2&C
Rooms3&O
RoomL(speech mix.)
Algorithm
D.H.Tran Vu		 SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)



 0.9

 6.1
 3.9












 2.4

 9.2
 4.2
Algorithm
H.Sawada		 SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)



















 6.7
11.2
10.9
10.4
Algorithm
N.Ono		 SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
  9.2
14.0
15.0
12.1
 0.1
 4.1
 0.3
 5.6
 0.8
 4.0
 2.5
 6.2
-0.6
 2.3
-0.5
 5.2
-0.4
 1.8
 1.1
 3.1
 1.6
 5.8
 2.6
 6.7
Algorithm
N.Ono2		 SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
  4.4
 8.9
 7.8
 9.4
 1.1
 6.5
 4.0
 7.5
 1.8
 4.9
 4.6
 7.7
 0.2
 2.9
 0.4
 6.0
-0.0
 2.0
 2.2
 3.9
 3.5
 7.8
 6.1
 7.8


DETAILED RESULTS


Iliad (lab room)
iliad
mix
Algorithm
N.Ono2
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 
 3.5  5.3
10.0  7.8
 4.8 10.8
 9.1  9.7

12.1 20.7
35.6 36.8
21.6 41.3
30.6 39.6
Algorithm
N.Ono
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 
 8.2 10.2
13.0 15.0
13.6 16.3
11.2 13.1

21.4 32.2
43.5 50.9
46.0 60.7
36.2 48.9


Rooms 4 and 5 (chamber with cushion walls)
room4, 2sources
mix		room4, 3sources
mix		room4, 4sources
mix		room5, 2sources
mix		room5, 3sources
mix		room5, 4sources
mix
Algorithm
D.Vu
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)		 src1 src2 
-2.7 -0.3

 7.2  6.9
-1.4  1.4 		src1 src2 src3 
 0.6  4.1 -0.7

 7.9  7.9  6.7
 2.2  7.1  1.1 		src1 src2 src3 src4 
 2.9 -1.5  2.7  0.9

 7.4  0.7  6.2  4.5
 5.6  5.2  6.2  4.7 		src1 src2 
 0.8  0.7

10.6  7.3
 1.7  2.6 		src1 src2 src3 
 1.8  2.9  1.1

 5.8  6.1  6.3
 4.9  6.7  3.6 		src1 src2 src3 src4 
 1.7 -1.7  1.8  1.0

 5.8  0.7  5.9  5.0
 4.9  4.9  4.9  4.4
Algorithm
N.Ono2
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)		 sim1 sim2 
-0.2  3.3
 6.6  5.2
 0.0  8.1
 8.7  9.0 		sim1 sim2 sim3 
-2.3  1.0  2.0
 3.2 12.3  2.8
-3.4  1.6 10.7
 7.4  8.7  5.2 		sim1 sim2 sim3 sim4 
 0.4  0.2 -0.2  1.4
 3.6 10.5  8.5  2.4
 2.6  0.8  0.4  7.2
 4.8  7.4  6.2  4.5 		sim1 sim2 
-1.8  3.5
 7.8  4.6
-2.0 10.4
 9.5  9.2 		sim1 sim2 sim3 
-1.2  1.7  2.4
 5.0 13.1  3.2
-0.5  2.7 11.4
 7.3  8.8  6.8 		sim1 sim2 sim3 sim4 
 1.2  2.5  3.6  2.5
 4.9 10.9  5.1  7.1
 4.0  3.7 10.1  4.4
 6.3  8.8  8.2  8.2
Algorithm
N.Ono
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)		 sim1 sim2 
-0.9  2.6
 5.6  4.6
-1.5  6.5
 8.0  8.2 		sim1 sim2 sim3 
 2.8 -1.0 -0.9
 5.0  4.2  4.5
 6.4 -2.3 -1.2
 5.8  4.9  4.9 		sim1 sim2 sim3 sim4 
-1.9 -0.5  2.3 -0.8
 2.2  2.4  5.3  3.8
-5.9 -2.8  4.8 -1.4
 3.0  3.5  5.8  5.0 		sim1 sim2 
-1.4  3.9
 6.6  5.4
-1.5  9.4
 8.8  9.7 		sim1 sim2 sim3 
 2.0 -0.1 -3.6
 3.9  4.8  1.8
 4.6  0.2 -8.2
 5.5  4.7  4.7 		sim1 sim2 sim3 sim4 
-2.6 -0.0  2.5 -0.3
 1.8  3.8  5.0  3.2
-6.3 -0.6  5.8 -0.5
 3.3  4.0  6.6  4.7


Room 1 (chamber with cushion walls)
room1, 2sources
mix		room1, 3sources
mix		room1, 4sources
mix
Algorithm
N.Ono2
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)		 sim1 sim2 
 0.0  3.7
 4.0  6.2
 1.6  6.9
 5.6 10.1 		sim1 sim2 sim3 
-0.1  2.8  2.6
 4.2  6.5  4.5
 1.1  6.8  5.4
 6.8  8.9  7.3 		sim1 sim2 sim3 sim4 
-1.4  2.9  2.8  2.8
 2.5  4.9  5.9  5.2
-1.0  8.6  6.5  5.6
 6.0  7.6  8.0  9.3
Algorithm
N.Ono
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)		 sim1 sim2 
-0.6  3.1
 3.4  5.9
 0.3  6.0
 4.8  8.2 		sim1 sim2 sim3 
-0.8  1.1  2.5
 3.1  4.9  4.8
-1.0  4.2  5.2
 4.9  6.9  8.0 		sim1 sim2 sim3 sim4 
-1.9  0.8  1.1  2.1
 1.0  2.7  5.2  5.2
-2.1  3.5  3.3  3.4
 4.5  3.6  6.1  8.8


Rooms 2 and C (medium size conference room)
room2, 2sources
mix		room2, 3sources
mix		room2, 4sources
mix		roomC, 2sources
mix		roomC, 3sources
mix		roomC, 4sources
mix
Algorithm
N.Ono2
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)		 sim1 sim2 
 1.2  1.4
 6.2  4.1
 2.6  2.6
 7.7  6.1 		sim1 sim2 sim3 
 1.9  0.1 -0.5
 5.1  3.1  3.1
 3.1  0.4 -1.9
 6.9  6.9  6.3 		sim1 sim2 sim3 sim4 
-0.6 -5.0  0.7  0.2
 1.9  1.6  3.4  2.5
-0.7 -8.8  1.6 -2.1
 4.0  4.6  5.6  5.1 		sim1 sim2 
 3.7 -0.3
 5.1  2.5
13.2  1.3
 8.5  6.0 		sim1 sim2 sim3 
 2.6 -1.6  0.4
 4.0  0.7  3.0
10.1 -6.7  0.5
 7.6  4.5  7.6 		sim1 sim2 sim3 sim4 
-3.0 -1.5  2.2  1.4
 0.5  0.6  3.1  2.5
-14.5 -8.6  7.8  6.8
 4.6  4.4  7.0  4.6
Algorithm
N.Ono
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)		 sim1 sim2 
 0.2 -0.1
 3.3  4.3
-0.5  0.9
 6.0  6.2 		sim1 sim2 sim3 
-1.1 -0.1 -0.9
 2.6  2.2  2.7
-3.0 -0.5 -2.5
 5.2  5.7  6.6 		sim1 sim2 sim3 sim4 
-1.1 -0.3 -0.3 -6.4
 1.2  2.1  1.8  1.8
-3.9 -1.5 -2.1 -8.2
 1.2  5.1  4.4  6.0 		sim1 sim2 
 3.7 -0.5
 5.7  1.6
10.7  6.3
 8.2  4.6 		sim1 sim2 sim3 
-3.2  1.7  0.9
 0.3  3.0  2.8
-8.5  6.3  6.4
 4.6  7.6  4.6 		sim1 sim2 sim3 sim4 
-1.6  1.1 -4.7  1.0
 0.6  1.7  0.7  2.7
-6.6  6.0 -14.9  6.5
 3.0  5.3  4.2  4.6


Rooms 3 and O (medium size office room)
room3, 2sources
mix		room3, 3sources
mix		room3, 4sources
mix		roomO, 2sources
mix		roomO, 3sources
mix		roomO, 4sources
mix
Algorithm
N.Ono2
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)		 sim1 sim2 
-0.2  0.6
 2.2  3.0
 1.6  3.9
 3.8  3.4 		sim1 sim2 sim3 
-0.2  1.5  1.2
 2.2  2.9  3.2
 0.9  7.8  3.6
 5.0  4.9  5.1 		sim1 sim2 sim3 sim4 
-0.2 -1.3  0.5 -1.1
 1.6  0.6  3.0  0.9
-0.6 -3.9  4.2 -4.9
 3.7  3.8  2.4  5.3 		sim1 sim2 
 0.5  0.7
 2.4  3.2
 9.1  7.9
 2.7  2.2 		sim1 sim2 sim3 
-1.4  0.2  2.4
 1.6  1.2  3.5
-4.6  2.6 16.1
 7.3  2.7  6.0 		sim1 sim2 sim3 sim4 
 0.6 -1.3  1.6 -4.4
 1.2  0.8  3.1  0.1
 7.4 -4.7  7.5 -14.7
 4.7  1.3  4.7  0.5
Algorithm
N.Ono
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)		 sim1 sim2 
-0.4 -0.0
 1.8  2.8
 0.5  2.5
 4.2  2.6 		sim1 sim2 sim3 
-0.5  0.3 -0.8
 1.2  1.8  2.7
 0.7  1.4 -0.9
 2.4  5.3  5.7 		sim1 sim2 sim3 sim4 
-0.4 -0.7  0.2 -3.0
 1.1  1.1  2.0  0.8
-1.9 -2.7  0.5 -6.2
 4.7  0.1  5.8  3.3 		sim1 sim2 
 1.0  0.6
 2.7  3.5
10.6 10.6
 3.4  1.2 		sim1 sim2 sim3 
-1.5  0.3  1.8
 1.0  1.5  3.2
-3.9  2.5 12.2
 3.6  4.2  4.5 		sim1 sim2 sim3 sim4 
 0.3 -2.1  1.3 -3.8
 1.2  1.1  2.7  0.5
 1.7 -3.1  7.1 -11.5
 2.2  0.6  3.7 -1.8


Room L(2, 3 and 4 speech sources)
roomL, set1,2sources
mix		roomL, set2,2sources
mix		roomL, set1,3sources
mix		roomL, set2,3sources
mix		roomL, set1,4sources
mix		roomL, set2,4sources
mix
Algorithm
D.Vu
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 src1 src2 
 3.5  3.3

10.5  9.3
 4.8  5.1

20.3 21.9
26.9 55.8
70.6 72.7
10.8 11.2 		src1 src2 
 5.0  3.0

14.5 12.2
 5.7  3.8

28.5 13.2
49.2 30.1
75.8 64.4
18.0  5.2 		src1 src2 src3 
-1.5  2.8  3.5

 2.3  8.4 13.7
 2.9  4.8  4.1





src1 src2 src3 
 4.6  0.6 -2.3

15.4  4.8  2.2
 5.1  3.9  1.7





src1 src2 src3 src4 
 4.3  2.6  2.6  0.6

11.7  8.1 10.7  5.4
 5.4  4.6  3.6  3.4





src1 src2 src3 src4 
 2.8  3.2  3.5  0.8

 9.0 12.1  9.4  5.5
 4.4  4.1  5.2  3.6





Algorithm
H.Sawada
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 
 8.9  9.4
14.3 14.6
13.3 14.6
12.1 12.2

34.7 44.2
56.1 61.3
63.2 71.1
51.7 60.8 		sim1 sim2 
11.2  5.6
15.1 12.6
18.4  8.4
14.1  9.1

49.4 22.1
59.3 46.8
76.5 44.7
59.6 40.1 		sim1 sim2 sim3 
 6.4  7.2  4.5
10.2 13.1  8.3
11.0 10.5  7.7
 9.8 10.9  9.0





sim1 sim2 sim3 
 6.4  8.0  4.4
 9.6 12.0 11.2
11.7 14.8  6.6
10.1 10.2  8.6





sim1 sim2 sim3 sim4 
 7.5  6.0  7.0  4.1
12.0 10.2 10.4  8.1
11.5  9.7 11.4  7.2
11.2  9.3 11.8  8.4





sim1 sim2 sim3 sim4 
 7.3  5.1  7.1  4.1
10.3 11.2  9.5  8.3
12.0  7.5 13.9  6.8
12.5  9.6 11.3  8.1





Algorithm
N.Ono2
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 
 5.5  6.5
11.8 10.6
 8.3 11.4
 9.0  9.7

12.1 20.7
35.6 36.8
21.6 41.3
30.6 39.6 		sim1 sim2 
 1.9  8.0
 8.7 11.2
 3.7 13.8
 6.5 11.5

40.2 28.4
57.1 54.9
69.7 56.1
53.5 45.1 		sim1 sim2 sim3 
 1.1  3.0  5.8
 6.1  5.5 11.0
 2.1  6.0  8.6
 6.2  6.9 10.1





sim1 sim2 sim3 
 3.4  0.4  6.5
 6.7  7.0  9.7
 6.6  0.7 11.7
 7.7  5.1  10.0





sim1 sim2 sim3 sim4 
 1.3  3.2  5.2  3.4
 6.0  7.1  8.8  6.5
 2.8  4.8  8.9  6.5
 5.9  7.7  9.6  8.3





sim1 sim2 sim3 sim4 
 4.2  0.1  3.2  0.1
 6.4  5.6  6.3  5.1
 8.8  0.5  6.1 -0.6
 7.7  5.5  7.8  5.7





Algorithm
N.Ono
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)

OPS 
TPS 
IPS 
APS 		 sim1 sim2 
 3.5  4.5
 9.5  8.0
 5.3  8.4
 8.2  8.2

21.4 32.2
43.5 50.9
46.0 60.7
36.2 48.9 		sim1 sim2 
-1.1  5.0
 4.3  8.3
-1.5  8.4
 4.8  9.4

36.6 25.8
53.0 51.3
65.1 50.9
53.4 44.7 		sim1 sim2 sim3 
 1.7  3.5  5.8
 6.9  6.3 11.5
 4.2  6.6  8.5
 6.8  6.9 10.1





sim1 sim2 sim3 
-1.5 -0.5  0.9
 5.0  3.2  3.2
-2.5 -1.9  0.6
 5.6  4.9  5.1





sim1 sim2 sim3 sim4 
-0.8  2.9  4.7  2.5
 2.6  7.5  8.3  5.6
-2.2  4.3  8.1  5.0
 4.3  8.2  9.4  7.4





sim1 sim2 sim3 sim4 
 0.9 -1.1 -0.1 -1.8
 2.2  5.2  3.9  3.3
 0.8 -0.4 -0.6 -3.4
 4.3  4.9  6.5  5.3







Room L(2 speech & 2 music)
roomL, set1,4sources
mix		roomL, set2,4sources
mix
Algorithm
H.Sawada
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)		 sim1 sim2 
 6.4  7.9
10.0  9.9
11.1 15.0
 9.4 13.1 		sim1 sim2 
 6.5  6.2
 8.7  8.4
12.8 14.6
11.1  9.4
Algorithm
N.Ono2
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)		 sim1 sim2 sim3 sim4 
-1.7  2.6  3.2 -1.3
 1.4  5.4  5.2  1.8
-4.5  4.9  7.1  3.1
 4.3  6.8  7.4  6.0 		sim1 sim2 sim3 sim4 
-0.4  4.3 -0.0  5.9
 1.5  7.2  2.6  8.2
 5.4  8.2  2.8 11.4
 6.0  8.4  6.2 10.1
Algorithm
N.Ono
SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)		 sim1 sim2 sim3 sim4 
-0.5  3.0  4.0 -2.1
 1.3  6.2  6.3  1.6
-0.6  5.8  8.2  2.5
 2.7  6.9  8.1  6.4 		sim1 sim2 sim3 sim4 
-0.3  3.6  4.4 -1.3
 2.0  6.7  6.6  1.5
 1.1  8.3  9.3  4.0
 4.3  7.4  7.7  6.5