1.) Improving Dialogue Agents with a Social Dimension 2.) Understanding Monolingual Pre-Training for Bilingual Models

Thursday, August 13, 2020, 11:00 am - 12:00 pm PDTiCal
This event is open to the public.
NL Seminar
Naitian Zhou and Omar Shaikh (ISI Interns)
Video Recording:

Abstract 1:

"Improving Dialogue Agents with a Social Dimension"

The dialogue problem is challenging because a proper response must be conditioned on many different factors: knowledge about the language, knowledge about the world, knowledge about self, and knowledge about the speaker, to name a few. Prior research has focused on language modeling and "persona" modeling, encoding facts about the dialogue agent. In this project, I try to focus on an alternative dimension to dialogue: how do we condition the way we converse, based on our understanding of ourselves and our social relationship with our dialogue partner?


Naitian Zhou is a rising junior at the University of Michigan studying computer science and data science. His interests include computational social science and natural language processing.

Abstract 2:

"Understanding Monolingual Pre-Training for Bilingual Models"

Monolingual embeddings (from models like BERT) are known to help on a variety of downstream tasks in a straightforward way. Usually, these embeddings are plug-and-play — initializing models with BERT embeddings or using them as input representations result in increased model performance. However, supervised NMT tasks don’t appear to benefit equally from traditional pretraining methods. We explore what makes NMT (bilingual) and BERT/LM (monolingual) representations different on several probing tasks, and why certain training methods succeed in extracting performance from BERT embeddings from NMT tasks.


Omar is a Summer 2020 intern with Dr. Jon May. He’s also a rising junior at Georgia Institute of Technology.

« Return to Upcoming Events