Abstract: We propose a novel approach for semi-supervised extraction of a moving audio source of interest (SOI) applicable in reverberant and noisy environments. The blind part of the method is based on independent vector extraction (IVE) and uses the recently proposed constant separating vector (CSV) mixing model. This model allows for changes of mixing parameters within the processed interval of the mixture, which potentially leads to higher accuracy of SOI estimation. The supervised part of the method concerns a pilot signal which is related to the SOI and ensures the convergence of the blind method towards the SOI. The pilot is based on robust detection of frames where SOI is dominant via speaker embeddings called X-vectors. Robustness of the detection is achieved through augmentation of the data for the supervised training of the X-vectors. The pilot-supported extraction yields significantly better performance compared to its unsupervised counterpart identifying SOI solely using the initialization.
Examples in different acoustic environments:
The listener should hear that the extracted speaker is sometimes vanishing, depending on the performance of the given method. This is mainly caused by the speaker’s movements and by the limited ability of blind source extraction algorithms to keep convergence to the desired speaker, in particular, in the online block-by-block processing regime (which is essential for practical deployment). The CSV mixing model brings new flexibility to this approach compared to the conventional block-by-block method where standard static (time-invariant) mixing model is assumed on the blocks.