Adversarially Trained Multi-Singer Sequence-To-Sequence Singing Synthesizer

Authors:

Jie Wu, Jian Luan

Abstract:

This paper presents a high quality singing synthesizer that isable to model a voice with limited available recordings. Basedon the sequence-to-sequence singing model, we design a multi-singer framework to leverage all the existing singing data of different singers. To attenuate the issue of musical score unbalance among singers, we incorporate an adversarial task of singer classification to make encoder output less singer dependent. Furthermore, we apply multiple random window discriminators (MRWDs) on the generated acoustic features to make the network be a GAN. Both objective and subjective evaluations indicate that the proposed synthesizer can generate higher quality singing voice than baseline (4.12 vs 3.53 in MOS). Especially, the articulation of high-pitched vowels is significantly enhanced.

Note: Please ensure your web browser supports wav audio format. You can choose Microsoft Edge, Google Chrome, Firefox, Opera and Safari.

MOS Test on singing voice quality

A/B Tests on the articulation of high-pitched vowels