2010 Signal Separation Evaluation Campaign
Determined convolutive mixtures under dynamic conditions
(TASK 2)

In this evaluation, a filter length for the BSS_EVAL decomposition was set at 1024, because the reverberation is quite high.

The task proposer proposed to evaluate the SIR/SDR by filtering the components estimated by the BSS_EVAL with an "A-weighting" filter, in order to give measures which are close to the human perception. Therefore, the following result tables include performance with and without the A-weighting.

For this task, the evaluation is performed only considering the segments where two sources overlap, where the reference start-end points are given. Since it may happen that there are very small segments, we evaluate the performance only for those >= 1 sec. because for smaller segments the BSS_EVAL performance are very unstable.
In the following, the mean values for the segments are listed.

For details about each algorithm, click on "Algorithm".

The audio files below are made available under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 2.0 license. The author of the mixtures is F. Nesta.

2cm
mix		6cm
mix		10cm
mix
Algorithm
S. Araki		 w/o A-weighting
SDR (dB)
SIR (dB)
SAR (dB)

with A-weighting
SDR (dB)
SIR (dB)
SAR (dB)		 src1 src2 
mean(variance)
 2.4 (1.5)
 8.7 (2.4)
 4.6 (1.0)


 3.3 (1.5)
 9.7 (2.5)
 5.3 (1.0)

src1 src2 
mean(variance)
 2.2 (1.2)
 9.5 (1.3)
 3.9 (1.1)


 2.6 (1.3)
 10.0 (1.5)
 4.2 (1.2)

src1 src2 
mean(variance)
 1.6 (1.3)
 9.4 (2.0)
 3.2 (1.1)


 1.5 (1.3)
 9.2 (1.9)
 3.1 (1.2)

Algorithm
J. Malek		 w/o A-weighting
SDR (dB)
SIR (dB)
SAR (dB)

with A-weighting
SDR (dB)
SIR (dB)
SAR (dB)		 src1 src2 
mean(variance)
 4.2 (1.6)
 9.4 (1.3)
 6.8 (1.4)


 4.6 (1.6)
 9.9 (1.3)
 7.0 (1.5)

src1 src2 
mean(variance)
 2.2 (2.1)
 8.4 (1.6)
 4.8 (1.9)


 2.3 (2.2)
 8.8 (1.6)
 4.7 (2.1)

src1 src2 
mean(variance)
 1.1 (1.7)
 7.9 (1.5)
 3.4 (1.7)


 0.8 (2.0)
 8.0 (1.8)
 2.9 (2.0)

Algorithm
B. Loesch		 w/o A-weighting
SDR (dB)
SIR (dB)
SAR (dB)

with A-weighting
SDR (dB)
SIR (dB)
SAR (dB)		 src1 src2 
mean(variance)
 3.0 (2.0)
 6.6 (2.6)
 7.4 (1.4)


 4.7 (2.3)
 8.8 (2.9)
 8.0 (1.7)

src1 src2 
mean(variance)
 5.5 (2.1)
11.9 (1.9)
 7.4 (2.1)


 6.4 (2.5)
13.4 (2.2)
 7.9 (2.5)

src1 src2 
mean(variance)
 5.0 (2.6)
11.3 (3.2)
 6.9 (1.9)


 5.0 (2.8)
11.2 (3.4)
 6.9 (2.2)

Algorithm
F. Nesta		 w/o A-weighting
SDR (dB)
SIR (dB)
SAR (dB)

with A-weighting
SDR (dB)
SIR (dB)
SAR (dB)		 src1 src2 
mean(variance)
 5.1 (1.6)
11.3 (1.8)
 7.1 (1.2)


 5.8 (1.6)
12.2 (1.8)
 7.5 (1.2)

src1 src2 
mean(variance)
 4.8 (1.6)
10.9 (1.7)
 7.1 (1.3)


 5.8 (1.5)
12.2 (1.7)
 7.6 (1.3)

src1 src2 
mean(variance)
 4.0 (1.6)
 10.0 (2.0)
 6.1 (1.0)


 4.3 (1.6)
10.2 (2.0)
 6.4 (1.1)