Yossi Adi

yossiadidrum at gmail dot com ; adios at lucillecrew dot com

My name is Yossi Adi, I'm a Senior Lecturer (Assistant Professor) at the School of Computer Science and Engineering at the Hebrew University of Jerusalem and a Research Scientist (part time) at Meta AI Research, FAIR. I'm also a professional drummer at Lucille Crew. Before joining the Hebrew University, between 2019-2022, I was a full time Research Scientist at the FAIR team, Meta AI Research. I completed my Ph.D in computer science at Bar-Ilan University under the supervision of Professor Joseph Keshet.

My research interests are focused on developing and analyzing machine learning and deep learning algorithms for speech and language applications. My complete CV can be found here.

As a musician, I'm the drummer and one of the band members of Lucille Crew, an international Groove, Hip Hop and Soul collective based in Tel Aviv.

News:

Five research papers got accepted to Interspeech 2022! More details in the publication section.
Two research papers got accepted to ACL 2022! More details in the publication section.
I had a pleasure to chat about my research at the F2F podcast (in Hebrew): [podcast].
Our research paper Learning Discrete Structured Variational Auto-Encoder Using Natural Evolution Strategies was accepted to ICLR 2022!
Our research paper Continual Self-Training with Bootstrapped Remixing For Speech Enhancement was accepted to ICASSP 2022!
Our research paper Generative Spoken Language Modeling from Raw Audio was accepted to TACL!
I gave a talk at the TECHINOVATiON, The Marker conference [Talk].
I was honored to be a keynote speaker at the International Conference on Artificial Intelligence - Academy and Industry.
Speech Resynthesis from Discrete Disentangled Self-Supervised Representations was accepted to Interspeech 2021!
Fairness in the Eyes of the Data: Certifying Machine-Learning Models was accepted to AIES 2021!
Two research papers got accepted to ICASSP 2021! More details in the publication section.
SAGRNN: Self-Attentive Gated RNN for Binaural Speaker Separation with Interaural Cue Preservation was accepted to IEEE Signal Processing Letters (2020)!
I am humbled to be part of TheMarker Magazine's 40 Under 40 list (in Herbrew): [The Marker].
An article about my music, AI research, and everything in between (in Hebrew): [Mako].
Our work on single channel speaker source separation using deep nets got attention from the media: [Globes (Hebrew), Venture Beat, Digital Trends].
A research in a brief blog post about our work on single channel source separation using deep nets: [FB].
I am grateful to be the recipient of the Best Doctoral Dissertation Award in 2020 given by the Israeli Association for Artificial Intelligence (IAAI)!
Four research papers got accepted to INTERSPEECH 2020! More details in the publication section.
Voice Separation with an Unknown Number of Multiple Speakers was accepted to ICML 2020!

Publications

Conferences and Workshops Proceedings

Arnon Turetzky, Tzvi Michelson, Yossi Adi, Shmuel Peleg. Deep Audio Waveform Prior. The 23rd Annual Conference of the International Speech Communication Association (Interspeech), 2022, [PDF].
Or Tal, Moshe Mandel, Felix Kreuk, Yossi Adi. A Systematic Comparison of Phonetic Aware Techniques for Speech Enhancement. The 23rd Annual Conference of the International Speech Communication Association (Interspeech), 2022, [PDF].
Shahaf Bassan, Yossi Adi, Jeffrey S. Rosenschein. Unsupervised Symbolic Music Segmentation using Ensemble Temporal Prediction Errors. The 23rd Annual Conference of the International Speech Communication Association (Interspeech), 2022, [PDF].
Sravya Popuri, Peng-Jen Chen, Changhan Wang, Juan Pino, Yossi Adi, Jiatao Gu, Wei-Ning Hsu, Ann Lee. Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation. The 23rd Annual Conference of the International Speech Communication Association (Interspeech), 2022, [PDF].
Maureen de Seyssel, Marvin Lavechin, Yossi Adi, Emmanuel Dupoux, Guillaume Wisniewski. Probing Phoneme, Language and Speaker Information in Unsupervised Speech Representations. The 23rd Annual Conference of the International Speech Communication Association (Interspeech), 2022, [PDF].
Eugene Kharitonov, Jade Copet, Kushal Lakhotia, Tu-Anh Nguyen, Paden Tomasello, Ann Lee, Ali Elkahky, Wei-Ning Hsu, Abdelrahman Mohamed Emmanuel Dupoux, Yossi Adi. textless-lib: a Library for Textless Spoken Language Processing. Conference of the North American Chapter of the Association for Computational Linguistics (NAACL): System Demonstrations, [PDF, Code].
Eugene Kharitonov∗, Ann Lee∗, Adam Polyak, Yossi Adi, Jade Copet, Kushal Lakhotia, Tu-Anh Nguyen, Morgane Rivière, Abdelrahman Mohamed, Emmanuel Dupoux, Wei-Ning Hsu. Text-Free Prosody-Aware Generative Spoken Language Modeling. The 60th Annual Meeting of the Association for Computational Linguistics (ACL), 2022, [PDF].
Ann Lee, Peng-Jen Chen, Changhan Wang, Jiatao Gu, Xutai Ma, Adam Polyak, Yossi Adi, Qing He, Yun Tang, Juan Pino, Wei-Ning Hsu. Direct Speech-to-Speech Translation with Discrete Units. The 60th Annual Meeting of the Association for Computational Linguistics (ACL), 2022, [PDF].
Alon Berliner, Guy Rotman, Yossi Adi, Roi Reichart, Tamir Hazan. Learning Discrete Structured Variational Auto-Encoder Using Natural Evolution Strategies. International Conference on Learning Representations (ICLR), 2022, [PDF, Code].
Efthymios Tzinis, Yossi Adi, Vamsi K. Ithapu, Buye Xu, Anurag Kumar. Continual Self-Training with Bootstrapped Remixing For Speech Enhancement. The 47th IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), 2022, [PDF].
Changhan Wang*, Wei-Ning Hsu*, Yossi Adi, Adam Polyak, Ann Lee, Peng-Jen Chen, Jiatao Gu, Juan Pino. FAIRSEQ S^2: A Scalable and Integrable Speech Synthesis Toolkit. Conference on Empirical Methods in Natural Language Processing (EMNLP): System Demonstrations, 2021, [PDF, Code].
Kushal Lakhotia*, Eugene Kharitonov*, Wei-Ning Hsu, Yossi Adi, Adam Polyak, Benjamin Bolte, Tu-Anh Nguyen, Jade Copet, Alexei Baevski, Michael Auli, Alexis Conneau, Adelrahman Mohamed, Emmanuel Dupoux. On Generative Spoken Language Modeling from Raw Audio. Transactions of the Association for Computational Linguistics (TACL), 2021, [PDF, Samples, Code, Blog].
*Ori Kabeli, *Yossi Adi, Zhenyu Tang, Buye Xu, Anurag Kumar. Online Self-Attentive Gated RNNs for Real-Time Speaker Separation. Proceedings of Machine Learning in Speech and Language Processing Workshop, 2021, [PDF, Samples].
Adam Polyak, Yossi Adi, Jade Copet, Eugene Kharitonov, Kushal Lakhotia, Wei-Ning Hsu, Adelrahman Mohamed, Emmanuel Dupoux. Speech Resynthesis from Discrete Disentangled Self-Supervised Representations. The 22nd Annual Conference of the International Speech Communication Association (Interspeech), 2021, [PDF, Samples, Code].
Shahar Segal, Yossi Adi, Benny Pinkas, Carsten Baum, Chaya Ganesh, Joseph Keshet. Fairness in the Eyes of the Data: Certifying Machine-Learning Models. The Forth AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES), 2021, [PDF].
Adam Polyak, Lior Wolf, Yossi Adi, Ori Kabeli, Yaniv Taigman. High Fidelity Speech Regeneration with Application to Speech Enhancement. The 46th IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), 2021, [PDF, Samples].
Shlomo E. Chazan, Lior Wolf, Eliya Nachmani, Yossi Adi. Single Channel Voice Separation for Unknown Number of Speakers Under Reverberant and Noisy Settings. The 46th IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), 2021, [PDF, Samples].
Felix Kreuk, Joseph Keshet, Yossi Adi. Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentation. The 21st Annual Conference of the International Speech Communication Association (Interspeech), 2020, [PDF, Code].
Adam Polyak, Lior Wolf, Yossi Adi, Yaniv Taigman. Unsupervised Cross-Domain Singing Voice Conversion. The 21st Annual Conference of the International Speech Communication Association (Interspeech), 2020, [PDF, Samples].
Alexandre Défossez, Gabriel Synnaeve, Yossi Adi. Real Time Speech Enhancement in the Waveform Domain. The 21st Annual Conference of the International Speech Communication Association (Interspeech), 2020, [PDF, Samples, Code].
Felix Kreuk, Yossi Adi, Bhiksha Raj, Rita Singh, Joseph Keshet. Hide and Speak: Towards Deep Neural Networks for Speech Steganography. The 21st Annual Conference of the International Speech Communication Association (Interspeech), 2020, [PDF, Samples, Code].
Eliya Nachmani, Yossi Adi, Lior Wolf. Voice Separation with an Unknown Number of Multiple Speakers. The 37th International Conference on Machine Learning (ICML), 2020, [PDF, Samples, Code, Blog].
Ben Goldberger, Yossi Adi, Joseph Keshet, Guy Katz. Minimal Modifications of Deep Neural Networks using Verification. The 23rd International Conference on Logic for Programming, Artificial Intelligence and Reasoning (LPAR), Alicante, Spain, 2020, [PDF, Code].
Felix Kreuk, Yaniv Sheena, Joseph Keshet, Yossi Adi. Phoneme Boundary Detection using Learnable Segmental Features. The 45th IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), Barcelona, Spain, 2020, [PDF, Code].
Yossi Adi, Neil Zeghidour, Ronan Collobert, Nicolas Usunier, Vitaliy Liptchinsky and Gabriel Synnaeve. To Reverse The Gradient or Not: An Empirical Comparison of Adversarial and Multi-Task Learning in Speech Recognition. The 44th IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), Brighton, UK, 2019, [PDF].
Gabi Shalev, Yossi Adi, and Joseph Keshet. Out-of-Distribution Detection using Multiple Semantic Label Representations. The 32nd Annual Conference on Neural Information Processing Systems (NeurIPS), Montréal, Canada, 2018, [PDF].
Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring. Usenix Security, 2018, [PDF, Blog].
Felix Kreuk, Yossi Adi, Moustapha Cisse, and Joseph Keshet. Fooling End-to-End Speaker Verification With Adversarial Examples. The 43nd IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), Alberta, Calgary, Canada, 2018, [PDF].
Moustapha Cisse, Yossi Adi, Natalia Neverova, and Joseph Keshet. Houdini: Fooling Deep Structured Visual and Speech Recognition Models with Adversarial Examples. The 31st Annual Conference on Neural Information Processing Systems (NeurIPS), Long Beach, CA, USA, 2017, [PDF].
Einat Naaman, Yossi Adi, and Joseph Keshet. Learning Similarity Function for Pronunciation Variations. The 18th Annual Conference of the International Speech Communication Association (Interspeech), Stockholm, Sweden, 2017, [PDF].
Yaniv Sheena, Misha Hejna, Yossi Adi, Joseph Keshet. Automatic Measurement of Pre-aspiration. The 18th Annual Conference of the International Speech Communication Association (Interspeech), Stockholm, Sweden, 2017, [PDF].
Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi and Yoav Goldberg. Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks. International Conference on Learning Representations (ICLR), Toulon, France, 2017, [PDF].
Yossi Adi, Joseph Keshet, Emily Cibelli, and Matt Goldrick. Sequence Segmentation Using Joint RNN and Structured Prediction Models. The 42nd IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), New Orleans, LA, USA, 2017, [PDF].
Yossi Adi, Joseph Keshet, Olga Dmitrieva and Matt Goldrick. Automatic Measurement of Voice Onset Time and Prevoicing using Recurrent Neural Networks. The 17th Annual Conference of the International Speech Communication Association (Interspeech), San Francisco, CA, USA, 2016, [PDF, Code].
Yossi Adi, Joseph Keshet and Matt Goldrick. Vowel Duration Measurement Using Deep Neural Networks. The 25th IEEE International Workshop on Machine Learning for Signal Processing (MLSP), Boston, MA, USA, 2015, [PDF].

Journals

Ke Tan, Buye Xu, Anurag Kumar, Eliya Nachmani, Yossi Adi. SAGRNN: Self-Attentive Gated RNN for Binaural Speaker Separation with Interaural Cue Preservation. IEEE Signal Processing Letters, 2020, (accepted for publication), [PDF, Samples].
Jacob T. Cohen, Alma Cohen, Limor Benyamini, Yossi Adi, Joseph Keshet. Predicting Glottal Closure Insufficiency using Fundamental Frequency Contour Analysis. Head & Neck, Journal of the Sciences and Specialities of the Head and Neck, 41 (7), 2019: 2324-2331. [PDF].
Matthew Goldrick, Rhonda McClain, Emily Cibelli, Yossi Adi, Erin Gustafson, Cornelia Moers, and Joseph Keshet. The Influence of Lexical Selection Disruptions on Articulation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 45 (6), 2018: 1107-1141, [PDF].
Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi and Yoav Goldberg. Analysis of Sentence Embedding Models using Prediction Tasks in Natural Language Processing. IBM Journal of Research and Development, 61 (4/5), 2017: 31-39, [PDF].
Yossi Adi, Joseph Keshet, Emily Cibelli, Erin Gustafson, Cynthia Clopper and Matt Goldrick. Automatic Measurement of Vowel Duration via Structured Prediction. Journal of the Acoustical Society of America, 140 (6), 2016: 4517-4527, [PDF, Code].
Yossi Adi and Joseph Keshet. StructED : Risk Minimization in Structured Prediction. Journal of Machine Learning Research, 17 (1), 2016: 2282-2286, [PDF, Website, Code].

Preprints

Tu Anh Nguyen, Eugene Kharitonov, Jade Copet, Yossi Adi, Wei-Ning Hsu, Ali Elkahky, Paden Tomasello, Robin Algayres, Benoit Sagot, Abdelrahman Mohamed, Emmanuel Dupoux. Generative Spoken Dialogue Language Modeling. arXiv preprint arXiv:2203.16502 (2022), [PDF, Samples].
Efthymios Tzinis, Yossi Adi, Vamsi K. Ithapu, Buye Xu, Anurag Kumar. RemixIT: Continual Self-Training of Speech Enhancement Models via Bootstrapped Remixing. arXiv preprint arXiv:2202.08862 (2022), [PDF].
Felix Kreuk, Adam Polyak, Jade Copet, Eugene Kharitonov, Tu-Anh Nguyen, Morgane Rivière, Wei-Ning Hsu, Abdelrahman Mohamed Emmanuel Dupoux, Yossi Adi. Textless Speech Emotion Conversion using Decomposed and Discrete Representations. arXiv preprint arXiv:2111.07402 (2021), [PDF, Samples].
*Alexandre Défossez, *Yossi Adi, Gabriel Synnaeve. Differentiable Model Compression via Pseudo Quantization Noise. arXiv preprint arXiv:2104.09987 (2021), [PDF, Code].
Yossi Adi, Yaniv Nemcovsky, Alex Schwing, Tamir Hazan. On The Generalization of Bayesian Deep Nets for Multi-class Classification. arXiv preprint arXiv:2002.09866 (2020), [PDF].

Talks

May. 2022 - Towards Textless NLP: Modeling Spoken Language From Raw Audio: [The 9th AI Data Science Summit, Israel].
Apr. 2022 - Towards Textless NLP: Modeling Spoken Language the Raw Audio: [Speech Recognition Israel Meetup].
Jan. 2022 - A non-formal presentation of my research at F2F podcast (in Hebrew): [podcast].
Nov. 2021 - A Textless Approach For Generative Spoken Language Modeling. [HUJI vision seminar, BIU machine learning seminar,Weizman vision seminar].
June 2021 - The Future Of Audio: Towards Super Human Hearing And Audio Presence. The Marker, TECHNOVATiON conference [Talk, Article].
May 2021 - Generative Spoken Language Modeling From The Raw Waveform. Keynote talk at the International Conference on Artificial Intelligence - Academy and Industry.

Awards & Scholarships

Best Doctoral Dissertation Award - The Israeli Association for Artificial Intelligence (IAAI) - 2020.
Rector's Award for Excellence in Academic Studies - Bar-Ilan University - 2017.
President Award for Excellence (Ph.D Students) - Bar-Ilan University - 2016.
Zvulun Hammer Scholarship - 2015.
President Award for Excellence - College of Management Academic Studies - 2012.

From the press

I had a pleasure to chat about my research at the F2F podcast (in Hebrew): [podcast].
An article about my research on speech enhancement and source separation was published in The Marker magazine: [article].
I am humbled to be part of TheMarker Magazine's 40 Under 40 list (in Herbrew): [The Marker].
An article about my music, AI research, and everything in between (in Hebrew): [Mako].
Our work on single channel speaker source separation using deep nets got attention from the media: [Globes (Hebrew), Venture Beat, Digital Trends].
A research in a brief blog post about our work on single channel source separation using deep nets: [FB].
[ A medium blog post about our joint work with Carsten Baum, Moustapha Cissé, Benny Pinkas and Joseph Keshet about watermarking deep neural networks: Medium].
Our joint work with Moustapha Cissé, Natalia Neverova, and Joseph Keshet about fooling structured deep learning models got attention from the media: [New Scientist, MIT Technology Review].