Chenguang Zhu (Microsoft Research)

Feb 9, 2022

Title and Abstract

How We Achieved Human Parity in CommonsenseQA – Fusing Knowledge into Language Models

Large-scale language models (LM) have achieved great results in many NLP applications. However, there is still a non-negligible gap compared with human's capability. One of the key reasons is the lack of external knowledge integration. We argue that language models should be equipped with knowledge to better understand world common sense and relations. In this talk, I will introduce how to represent and fuse knowledge into language models, which includes three steps: 1) Ground language into related knowledge, 2) Represent knowledge, and 3) Fuse knowledge representation into language model. We demonstrate our proposed knowledge-boosted LM in the following work: i) achieving human parity in Commonsense Q&A, ii) Dictionary-boosted Language Model, and iii) Knowledge-text Co-pretraining.

Bio

Dr. Chenguang Zhu is a Principal Research Manager in Microsoft Cognitive Services Research Group, where he leads the Knowledge & Language Team. His research covers knowledge integration, text summarization and few-shot learning. Dr. Zhu has led teams to achieve human parity in CommonsenseQA and CoQA, and first places in CommonGen, FEVER, ARC and SQuAD v1.0. He holds a Ph.D. degree in Computer Science from Stanford University