Publications

Zhisheng Zheng, Zhifeng Kong, Sang-Gil Lee, Jaehyeon Kim, Arushi Goel, David Harwath, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping (2026). Reinforcement Learning for Music Generation with Pairwise Reward Modeling. Preprint 2026.

Cite Demo Audio Samples

Zhisheng Zheng, David Harwath (2026). TransDubber: Learning Timeline-Preserving Speech-to-Speech Translation. arXiv 2026.

PDF Code Demo Model Testset

Zhisheng Zheng, Xiaohang Sun, Zhu Liu, Caren Chen, Rohith Kumar, Manoj Aggarwal, Gerard Medioni, David Harwath (2026). CtrlSpeech: Coarse-to-Fine Control for Expressive Speech Synthesis. Interspeech 2026.

PDF Code Demo Model

Zhisheng Zheng, Puyuan Peng, Ziyang Ma, Xie Chen, Eunsol Choi, David Harwath (2024). BAT: Learning to Reason about Spatial Sounds with Large Language Models. ICML 2024.

PDF Demo

Ziyang Ma, Zhisheng Zheng, Jiaxin Ye, Jinchao Li, Zhifu Gao, Shiliang Zhang, Xie Chen (2023). emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation.

PDF Cite Code Slides

Ziyang Ma, Wen Wu, Zhisheng Zheng, Yiwei Guo, Qian Chen, Shiliang Zhang, Xie Chen (2023). Leveraging Speech PTM, Text LLM, and Emotional TTS for Speech Emotion Recognition.

PDF Cite Slides

Guanrou Yang, Ziyang Ma, Zhisheng Zheng, Yakun Song, Zhikang Niu, Xie Chen (2023). Fast-HuBERT: An Efficient Training Framework for Self-Supervised Speech Representation Learning. In ASRU 2023.

PDF Cite Code Slides

Zhisheng Zheng, Ziyang Ma, Yu Wang, Xie Chen (2023). Unsupervised Active Learning: Optimizing Labeling Cost-Effectiveness for Automatic Speech Recognition. In INTERSPEECH 2023.

PDF Cite Dataset Poster Slides

Ziyang Ma, Zhisheng Zheng, Guanrou Yang, Yu Wang, Chao Zhang, Xie Chen (2023). Pushing the Limits of Unsupervised Unit Discovery for SSL Speech Representation. In INTERSPEECH 2023.

PDF Cite Slides

Xie Chen, Ziyang Ma, Changli Tang, Yujin Wang, Zhisheng Zheng (2023). FRONT-END ADAPTER: ADAPTING FRONT-END INPUT OF SPEECH BASED SELF-SUPERVISED LEARNING FOR SPEECH RECOGNITION. In ICASSP 2023.

PDF Cite Dataset Slides

Ziyang Ma, Zhisheng Zheng, Changli Tang, Yujin Wang, Xie Chen (2022). MT4SSL: Boosting Self-Supervised Speech Representation Learning by Integrating Multiple Targets. In INTERSPEECH 2023.

PDF Cite Code Dataset Slides

Yujin Wang, Changli Tang, Ziyang Ma, Zhisheng Zheng, Xie Chen, Wei-Qiang Zhang (2022). EXPLORING EFFECTIVE DISTILLATION OF SELF-SUPERVISED SPEECH MODELS FOR AUTOMATIC SPEECH RECOGNITION. ASRU 2023.

PDF Cite Slides