Improving Performance of Clinical Text Classification Models with Attention Mechanisms and Rationales

AI is revolutionizing the technological landscape in medicine. A key application is the AI-driven summarization of clinical text, which facilitates the harmonization and curation of clinical data elements for common data models leading to improved understanding of population-level health. Population-level health is derived from aggregating patient-level information stored in unstructured electronic health records, often in the form of free-text clinical notes. As clinical text documents are information dense and written in highly complex clinical language, a model’s ability to discern signal from noise becomes exceedingly more crucial. To enable models to identify relevant information in text documents, previous research has shown attention mechanisms and non-medical human-based rationales to be effective. Building on this foundation, this dissertation evaluated methods to optimize attention mechanisms and to effectively use human-based clinical rationales as additional supervision to improve the performance and interpretability of clinical text classification models. In particular, this dissertation shows the following: (i) the effective utilization of the reference information—initialization with external text code descriptions or encoding of code hierarchy of medical coding systems—contained by an attention mechanism’s query matrix can improve model performance; (ii) extending an atten- tion mechanism’s receptive field with a flexible context window at the phrase-level leads to improved understanding of local linguistic information in clinical text; and (iii) utilizing human-based clinical rationales as additional supplementary training data can improve model performance and positively impacts model interpretability.