What do (and should) language models know about language?

Thursday, February 27, 2020, 11:00 am - 12:00 pm PDTiCal
*This talk will be given from the ISI Boston office. It will be live-streamed from ISI-MDR in 6th floor conference room: #689
This event is open to the public.
NL Seminar
Ellie Pavlick (Brown University)
Video Recording:

Natural language processing has become indisputably good over the past few years. We can perform retrieval and question answering with purported super-human accuracy, and can generate full documents of text that seem good enough to pass the Turing test. In light of these successes, it is tempting to attribute the empirical performance to a deeper "understanding" of language that the models have acquired. Measuring natural language "understanding", however, is itself an unsolved research problem. In this talk, I will discuss several studies which attempt to illuminate what it is that state-of-the-art models of language are capturing. I will argue that current SOTA models have made significant progress in modeling linguistic form, but have completely failed to capture linguistic meaning. I will discuss recent work which investigates the effect of dataset skew on representation learning, as well as work investigating inconsistencies in human's own representations of "meaning".

Ellie Pavlick is an Assistant Professor of Computer Science at Brown University. She received her PhD from University of Pennsylvania in 2017, where her focus was on paraphrasing and lexical semantics. Ellie's current research is on cognitively-inspired approaches to language acquisition, focusing on grounded language learning and on the emergence of structure (or lack thereof) in neural language models. Ellie leads the language understanding and representation (LUNAR) lab, which collaborates with Brown's Robotics and Visual Computing labs and with the Department of Cognitive, Linguistic, and Psychological Sciences.

« Return to Upcoming Events