Demonstrations

Multichannel nonnegative matrix factorization for convolutive blind source separation

Factorial scaled hidden Markov model for single channel speech / music separation

SARAH project istrument extraction demos:

One Microphone Singing Voice Separation

Sound examples for the paper: "One microphone singing voice separation using source-adapted models", A. Ozerov, P. Philippe, R. Gribonval and F. Bimbot, In IEEE Worksh. on Apps. of Signal Processing to Audio and Acoustics (WASPAA'05), pages 90-93, Mohonk, NY, Oct. 2005.

Example 1 (in English)	Mixture (= Music + Voice): WAV	Estimated Music: WAV	Estimated Voice: WAV
Example 2 (in Russian)	Mixture (= Music + Voice): WAV	Estimated Music: WAV	Estimated Voice: WAV

Sound examples for the paper: "Séparation voix / musique à partir d'enregistrements mono : quelques remarques sur le choix et l'adaptation des modèles", A. Ozerov, R. Gribonval, P. Philippe and F. Bimbot, In GRETSI'05 Symposium on Signal and Image Processing, Louvain-la-Neuve, Belgique, Sept. 2005.

Example 3 (in French)

Mixture (= Music + Voice): WAV

Estimated Music: WAV

Estimated Voice: WAV

Sound examples for the paper: "Adaptation of Bayesian models for single channel source separation and its application to voice / music separation in popular songs", A. Ozerov, P. Philippe, F. Bimbot and R. Gribonval, IEEE Trans. on Audio, Speech and Lang. Proc., special issue on Blind Signal Proc. for Speech and Audio Applications, vol. 15, no. 5, pp. 1564-1578, July 2007.

Note that in contrast to the examples 1 - 3, which are obtained using manual vocal/non vocal segmentaions, the examples below are obtained in a completely automatic manner.

-	Language	Style	Mixture (= Music + Voice)	Estimated Music	Estimated Voice
Example 4	English	pop	WAV	WAV	WAV
Example 5	Spanish	pop/rock	WAV	WAV	WAV
Example 6	French	rock	WAV	WAV	WAV
Example 7	English	rock	WAV	WAV	WAV
Example 8	English	rock	WAV	WAV	WAV
Example 9	Spanish	pop/rock	WAV	WAV	WAV
Example 10	Russian	rock	WAV	WAV	WAV
Example 11	Russian	rock	WAV	WAV	WAV
Example 12	Spanish	rock	WAV	WAV	WAV
Example 13	Spanish	ska/rock	WAV	WAV	WAV
Example 14	English	rap/rock	WAV	WAV	WAV
Example 15	English	techno/rock	WAV	WAV	WAV

Back to Alexey Ozerov's Home Page

One Microphone Source Separation

Sound examples for the paper: "Single channel source separation using static and dynamic features in the power domain", I. Potamitis and A. Ozerov, In EUSIPCO, 16th European Signal Processing Conference, Laussane, Switzerland, August 25-29, 2008.

Example	Separation method	Estimated Source 1	Estimated Source 2
Example 1 Cicada / Cricket Mixture (= Cicada + Cricket): WAV	Spectral	Estimated Cicada: WAV	Estimated Cricket: WAV
	Power Static	Estimated Cicada: WAV	Estimated Cricket: WAV
	Power Static+Dynamic	Estimated Cicada: WAV	Estimated Cricket: WAV
Example 2 Speech 1 / Speech 2 Mixture (= Speech 1 + Speech 2): WAV	Spectral	Estimated Speech 1: WAV	Estimated Speech 2: WAV
	Power Static	Estimated Speech 1: WAV	Estimated Speech 2: WAV
	Power Static+Dynamic	Estimated Speech 1: WAV	Estimated Speech 2: WAV
Example 3 Speech / Bubble noise Mixture (= Speech + Bubble): WAV	Spectral	Estimated Speech: WAV	Estimated Bubble: WAV
	Power Static	Estimated Speech: WAV	Estimated Bubble: WAV
	Power Static+Dynamic	Estimated Speech: WAV	Estimated Bubble: WAV
Example 4 Speech / Music Mixture (= Speech + Music): WAV	Spectral	Estimated Speech: WAV	Estimated Music: WAV
	Power Static	Estimated Speech: WAV	Estimated Music: WAV
	Power Static+Dynamic	Estimated Speech: WAV	Estimated Music: WAV

Back to Alexey Ozerov's Home Page

Multichannel Nonnegative Matrix Factorization for Convolutive Blind Source Separation

Sound examples for the paper: "Multichannel nonnegative matrix factorization in convolutive mixtures for audio source separation", A. Ozerov and C. Févotte, IEEE Trans. on Audio, Speech and Lang. Proc., Special Issue on Signal Models and Representations of Musical and Environmental Sounds, vol. 18, no. 3, pp. 550-563, March 2010.

Convergence and Separation Performance (with Perturbed Oracle Initialization)

Note that this experiment was performed in some laboratory (non-realistic) conditions, since perturbed oracle initialization is obtained from the original separated sources (see the paper).

Musical sources are from Shannon Hurley

Mixing filters are from Source Separation Evaluation Campaign SiSEC 2008 (Under-determined speech and music mixtures task)

Mixture type

Linear instantaneous mixture

Convolutive mixture

Test material

Sources (mono)

Mixing system

Source images (stereo)

Mix (stereo)

src1

src2

src3

mixing matrix (.mat file)

mixing filters (.mat file)

Condition

perturbed oracle init.

after convergence

perturbed oracle init.

after convergence

EM algorithm

Estimated images

SDR (dB)

ISR (dB)

SIR (dB)

SAR (dB)

esim1	esim2	esim3
6.06	1.87	3.62
13.19	5.41	6.47
9.89	1.30	7.32
10.28	8.03	9.74

esim1	esim2	esim3
14.81	12.12	21.81
19.85	23.62	40.43
25.81	16.79	26.04
16.59	13.99	23.90

esim1	esim2	esim3
2.61	6.00	4.59
5.13	8.43	11.10
7.24	10.80	5.52
8.87	14.47	13.09

esim1	esim2	esim3
8.38	10.38	8.56
10.33	14.37	16.75
18.58	15.07	10.59
12.35	14.74	13.65

MU rules

Estimated images

SDR (dB)

ISR (dB)

SIR (dB)

SAR (dB)

esim1	esim2	esim3
9.29	4.46	4.98
13.45	8.40	7.82
15.10	5.24	7.20
12.03	9.88	10.68

esim1	esim2	esim3
10.51	2.94	3.49
15.28	4.17	15.57
19.02	11.32	4.06
12.41	5.14	10.34

esim1	esim2	esim3
6.38	5.25	4.61
9.23	7.42	8.70
9.74	7.44	5.67
12.50	11.47	11.74

esim1	esim2	esim3
9.62	3.80	3.05
13.03	4.99	13.97
16.88	12.42	3.46
12.92	7.35	10.60

Blind separation of under-determined speech and music mixtures

Test material is from Source Separation Evaluation Campaign SiSEC 2008 (Under-determined speech and music mixtures task)

Linear instantaneous mixtures

Mixture

female4

male4

nodrums

wdrums

Test material

Source images (stereo)

Mix (stereo)

l_0 minim.

Estimated images

SDR (dB)

ISR (dB)

SIR (dB)

SAR (dB)

esim1	esim2	esim3	esim4
12.60	6.19	4.76	7.38
26.59	10.99	11.27	16.87
17.30	16.22	10.59	12.98
14.33	6.21	4.79	8.52

esim1	esim2	esim3	esim4
15.64	2.72	5.34	6.94
29.89	7.31	10.96	19.50
20.18	9.31	13.48	11.72
17.59	1.44	5.16	8.19

esim1	esim2	esim3
21.22	1.79	15.85
32.17	10.12	28.13
27.96	8.04	21.96
22.63	1.69	17.11

esim1	esim2	esim3
-0.57	3.15	28.41
9.17	4.84	47.52
2.11	17.39	30.13
6.87	2.07	33.47

EM algorithm

Estimated images

SDR (dB)

ISR (dB)

SIR (dB)

SAR (dB)

esim1	esim2	esim3	esim4
14.21	7.88	5.98	8.60
27.20	13.70	12.78	15.56
19.12	16.56	11.22	14.95
15.86	8.39	6.65	9.81

esim1	esim2	esim3	esim4
16.89	3.59	8.20	9.63
29.01	8.20	13.46	21.54
21.05	9.48	16.56	14.22
19.29	3.54	9.21	11.28

esim1	esim2	esim3
27.15	7.61	21.41
31.02	14.00	35.48
36.70	12.97	24.51
30.33	10.99	24.53

esim1	esim2	esim3
0.95	4.66	29.89
10.51	6.54	48.04
3.77	19.99	31.85
7.40	5.01	34.53

MU rules

Estimated images

SDR (dB)

ISR (dB)

SIR (dB)

SAR (dB)

esim1	esim2	esim3	esim4
3.92	0.90	0.18	2.29
7.11	6.07	5.64	4.10
8.75	1.75	-0.10	8.43
4.88	3.87	2.70	3.02

esim1	esim2	esim3	esim4
8.68	-0.76	2.82	2.93
13.77	3.59	7.22	7.56
15.16	-1.50	6.24	6.15
10.31	0.60	4.44	4.03

esim1	esim2	esim3
8.82	-6.49	3.36
12.16	7.42	4.69
15.29	-5.82	9.88
11.28	1.83	4.38

esim1	esim2	esim3
10.00	2.97	19.33
16.59	4.21	35.51
16.52	14.52	20.69
12.12	3.08	25.38

Synthetic convolutive mixtures (1m)

Mixture

female4

male4

nodrums

wdrums

Test material

Source images (stereo)

Mix (stereo)

Sawada

Estimated images

SDR (dB)

ISR (dB)

SIR (dB)

SAR (dB)

esim1	esim2	esim3	esim4
5.23	5.36	3.21	2.61
7.61	8.70	7.75	5.10
11.38	10.00	5.37	7.52
7.88	7.57	6.04	4.68

esim1	esim2	esim3	esim4
4.57	0.61	4.99	2.36
6.74	5.59	7.65	5.22
10.85	0.25	9.53	4.76
6.34	3.67	7.69	4.07

esim1	esim2	esim3
3.00	1.03	-1.65
5.03	2.87	3.36
8.51	2.46	-1.65
5.02	9.27	10.36

esim1	esim2	esim3
4.44	-12.71	0.63
4.88	-3.08	0.90
14.76	-10.66	4.20
16.28	13.83	6.99

EM algorithm

Estimated images

SDR (dB)

ISR (dB)

SIR (dB)

SAR (dB)

esim1	esim2	esim3	esim4
7.72	6.48	4.10	3.29
10.15	11.77	10.48	6.20
16.24	9.86	5.59	7.79
11.30	10.45	8.76	6.78

esim1	esim2	esim3	esim4
6.26	0.42	5.56	2.78
9.45	8.63	8.50	6.57
12.00	0.48	10.30	5.06
8.89	6.22	8.79	6.43

esim1	esim2	esim3
4.18	1.02	-1.80
6.32	3.12	3.48
10.19	2.37	-2.18
8.14	10.63	14.59

esim1	esim2	esim3
3.93	-12.46	1.32
4.51	-1.55	1.65
14.08	-12.94	5.86
14.12	12.86	8.77

MU rules

Estimated images

SDR (dB)

ISR (dB)

SIR (dB)

SAR (dB)

esim1	esim2	esim3	esim4
5.29	3.30	2.79	1.49
9.32	9.50	5.96	3.97
8.84	5.90	6.18	3.15
10.08	7.83	6.04	4.57

esim1	esim2	esim3	esim4
3.47	-0.99	3.09	1.72
7.08	5.06	6.20	5.33
6.47	-0.77	6.71	4.71
5.85	2.39	5.62	2.83

esim1	esim2	esim3
2.89	1.04	-2.09
5.98	2.58	4.48
7.48	2.26	-2.15
4.85	7.77	10.24

esim1	esim2	esim3
5.98	-10.93	1.93
9.33	-6.15	2.96
9.90	-1.30	8.88
9.58	3.28	1.40

Live-recorded convolutive mixtures (1m)

Mixture

female4

male4

nodrums

wdrums

Test material

Source images (stereo)

Mix (stereo)

Sawada

Estimated images

SDR (dB)

ISR (dB)

SIR (dB)

SAR (dB)

esim1	esim2	esim3	esim4
4.16	3.85	6.09	3.31
7.35	7.04	9.61	5.38
8.02	7.47	11.19	8.46
6.56	5.93	8.35	4.98

esim1	esim2	esim3	esim4
3.00	1.67	4.87	2.42
6.90	4.23	7.65	5.71
5.19	2.81	11.20	5.06
5.08	2.94	6.00	5.08

esim1	esim2	esim3
4.13	5.18	-3.89
5.34	10.95	3.45
10.06	8.10	-6.50
10.33	8.17	7.71

esim1	esim2	esim3
4.18	4.59	6.03
6.38	10.28	9.09
9.31	7.23	12.35
7.91	8.28	7.99

EM algorithm

Estimated images

SDR (dB)

ISR (dB)

SIR (dB)

SAR (dB)

esim1	esim2	esim3	esim4
5.33	3.60	7.23	4.38
9.84	8.36	12.52	7.51
9.22	6.49	11.19	9.63
7.71	6.90	10.61	6.48

esim1	esim2	esim3	esim4
3.59	2.19	5.65	3.13
9.34	6.56	9.18	5.88
5.30	3.49	10.33	7.08
7.34	5.27	8.55	6.75

esim1	esim2	esim3
4.57	7.34	-4.51
5.81	12.13	4.16
10.90	13.34	-6.24
11.64	9.60	8.40

esim1	esim2	esim3
4.96	5.55	8.03
8.03	11.40	11.86
8.89	8.17	13.27
10.34	10.33	11.89

MU rules

Estimated images

SDR (dB)

ISR (dB)

SIR (dB)

SAR (dB)

esim1	esim2	esim3	esim4
1.60	-0.22	4.38	1.85
5.31	5.50	7.58	3.30
3.01	-0.44	8.67	7.24
5.29	4.79	8.46	2.99

esim1	esim2	esim3	esim4
1.17	0.04	2.85	2.19
4.32	6.88	5.58	4.61
2.21	-0.09	6.73	7.18
2.93	5.07	4.22	3.00

esim1	esim2	esim3
3.94	3.67	-4.93
5.89	7.12	4.41
9.06	11.38	-5.49
6.50	4.79	3.79

esim1	esim2	esim3
4.19	4.50	7.58
7.45	8.33	13.50
7.95	8.41	12.06
7.92	8.42	10.09

Supervised separation of professionally produced music recordings

Test material is from Source Separation Evaluation Campaign SiSEC 2008 ( Professionally produced music recordings task)

Recording

"Que pena tanto faz" by Tamy

"Roads" by Bearlin

Training source images
(development data of SiSEC)

Mix to separate
(test data of SiSEC)

mix

Source images separated with
EM algorithm

Source images separated with
MU rules

Blind separation of professionally produced music recordings (and remixes)

EM algorithm

MU rules

Example 1

Original stereo mix

Blindly estimated images

Remix

Manually regrouped images

Remix

Example 2

Original stereo mix

Blindly estimated images

Remix

Manually regrouped images

Remix

Example 3

Original stereo mix

Blindly estimated images

Remix

Manually regrouped images

Remix

Sound examples for the paper: "Multichannel nonnegative matrix factorization in convolutive mixtures. With application to blind audio source separation", A. Ozerov and C. Févotte, In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP'09), pages 3137-3140, Taipei, Taiwan, April 19-24, 2009.

Separation of a Synthetic Convolutive Mixture

Test Material

Musical sources are from Shannon Hurley

Mixing filters are from Source Separation Evaluation Campaign SiSEC 2008 (Under-determined speech and music mixtures task)

Original (mono) sources

src1

src2

src3

Convolutive mixing filters

Matlab .mat file

Original (stereo) source images

sim1

sim2

sim3

Stereo mix (to separate)

mix

Separation Results

Algorithm

EM algorithm

MU rules

Condition

best SDR

best cost

best SDR

best cost

Estimated images

SDR

ISR

SIR

SAR

1.91	4.92	6.11
6.41	7.20	10.78
2.52	8.77	8.31
7.95	9.79	12.51

0.12	-0.23	0.83
5.70	2.87	2.88
-0.26	-1.16	1.15
8.14	11.52	8.09

8.72	1.48	0.66
11.98	3.47	8.53
16.22	4.41	0.22
11.90	3.13	6.89

1.44	-0.36	0.19
1.86	1.86	10.34
11.21	-5.69	0.08
6.22	6.07	10.71

Separation of Original CD Recordings

Saparation into 3 sources was performed with 500 iterations of the EM/MU algorithm, using 18 NMF components (in total), and some non-random initialization.

Note that the convolutive mixing system model was used for all recordings, even if some recordings are in fact mixed using an instantaneous mixing system.

Original stereo mix

Stereo images estimated with EM algorithm

Stereo images estimated with MU rules

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Back to Alexey Ozerov's Home Page

Factorial scaled hidden Markov model for single channel speech / music separation

Sound examples for the paper: "Factorial scaled hidden Markov model for polyphonic audio representation and source separation", A. Ozerov, C. Févotte and M. Charbit, In IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA'09), Mohonk, NY, Oct. 18-21, 2009.

Speech model				S-HMM		S-HMM		NMF
Music model				S-HMM		NMF		NMF
Example	Speech	Music	Mixtures	Est. speech	Est. music	Est. speech	Est. music	Est. speech	Est. music
Example 1 (male)	speech	music	SMR = +3 dB	espeech	emusic	espeech	emusic	espeech	emusic
Example 1 (male)	speech	music	SMR = -3 dB	espeech	emusic	espeech	emusic	espeech	emusic
Example 2 (female)	speech	music	SMR = +3 dB	espeech	emusic	espeech	emusic	espeech	emusic
Example 2 (female)	speech	music	SMR = -3 dB	espeech	emusic	espeech	emusic	espeech	emusic
Example 3 (male)	speech	music	SMR = +3 dB	espeech	emusic	espeech	emusic	espeech	emusic
Example 3 (male)	speech	music	SMR = -3 dB	espeech	emusic	espeech	emusic	espeech	emusic
Example 4 (female)	speech	music	SMR = +3 dB	espeech	emusic	espeech	emusic	espeech	emusic
Example 4 (female)	speech	music	SMR = -3 dB	espeech	emusic	espeech	emusic	espeech	emusic

Back to Alexey Ozerov's Home Page