Cross Lingual Understanding: Scaling NLP Across Languages

Friday, September 25, 2020, 11:00 am - 12:00 pm PSTiCal
This event is open to the public.
AI Seminar
Ves Stoyanov, Research Scientist Manager at Facebook AI focusing on NLP
Video Recording:


Billions of people around the world use Facebook in over one hundred languages. This linguistic diversity is wonderful, but it presents challenges for Natural Language Processing (NLP) systems. It is simply impossible to annotate data and train a new system for each language. Instead, we rely on Cross-Lingual Understanding (XLU) to learn NLP systems in one language and apply them in languages that are not a part of the original training data.

In the last two years, self-training methods have enabled significant progress in XLU. I will give a brief overview of self-training methods such as BERT, XLNet and RoBERTa. I will then talk about how we have been able to use self-training advance the state-of-the-art in XLU including the recent cross-lingual language model (XLM) and XLM-R. I will cover recent work that shows that cross-lingual structure emerges naturally in self-trained models such as BERT. I will finish by discussing exciting ongoing work in my group.


Ves is a Research Scientist Manager at Facebook AI focusing on Natural Language Processing (NLP). Before Facebook AI, Ves was a Research Scientist on the Search team at Facebook working on NLP uses for Search. Before Facebook, Ves spent three wonderful years as a PostDoc at the Center for Language and Speech Processing at Johns Hopkins University. There he worked with Jason Eisner on Machine Learning for Structured Prediction and was supported by a Computing Innovation Fellowship from the CRA. Ves graduated with a PhD from Cornell University, where he worked with his advisor, Claire Cardie, on opinion analysis. His thesis title is “Opinion Summarization: Automatically Creating Useful Representations Of The Opinions Expressed In Text.” During his PhD, Ves was supported by an NSF Graduate Research Fellowship.


« Return to Upcoming Events