PhD Dissertation 2023 - Liyan Xu

Facilitate Document Understanding by Leveraging Language Structures

Liyan Xu

Abstract

Towards resolving a variety of Natural Language Processing (NLP) tasks, pretrained language models (PLMs) have been incredibly successful by simply modeling language sequences, backed by their powerful sequence encoding capabilities. However, for document understanding tasks involving multi-sentence or multi-paragraph inputs, the model still needs to overcome the inherent challenge of processing scattered information across the entire document context, such as resolving pronouns or recognizing relations among multiple sentences. 

To address the motivation of effectively understanding document context beyond sequence modeling, this dissertation focuses on the utilization of certain intrinsic language structures to facilitate this process. Four distinct works within this line of research are presented in this dissertation, where different language structures, including syntactic, discourse, and knowledge-specific structures, are investigated on how they could be incorporated towards stronger context understanding, described in details through Chapter 3 to Chapter 6. 

mperical results in each experiment suggest that these structures can enhance the performance on document-oriented tasks, primarily for machine reading comprehension, coreference resolution, and information extraction. Auxiliary information is provided by modeling these structures, which complements the sequence modeling of PLMs. Overall, this dissertation contributes to the research community’s understanding on the potentials and benefits of leveraging language structures in advancing the state-of-the-art in natural language understanding.

Department / School

Computer Science and Informatics / Emory University

Degree / Year

PhD / Spring 2023

Committee

Jinho D. Choi, Computer Science and QTM, Emory University (Chair)
Liang Zhao, Computer Science, Emory University
Joyce Ho, Computer Science, Emory University
Chenwei Zhang, Amazon Inc.

Links

Anthology | Paper | Presentation

Liang Zhao, Jinho Choi, Liyan Xu, Chenwei Zhang (screen), Joyce Ho