MS Thesis 2024 - Catherine Baker

Read with Emora: Revolutionizing Children's Reading Comprehension Through An LLM-Powered Personalized Intelligent Tutoring System for Vocabulary, Grammar, and Context-Based Learning

Catherine Baker

Abstract

Reading comprehension is critical for educational success, yet many students in the United States struggle with low proficiency. Decades of research on reading comprehension techniques show depth-based in-context approaches for practical learning components like grammar and vocabulary, spaced repetition and practice for long-term retention, and personalized learning to ensure all students learn at the same rate. However, many educators have yet to incorporate these findings into classroom practice opting instead for core or standardized curricula, which can contribute to suboptimal student reading progression. This thesis presents the development of Read with Emora, an Intelligent Tutoring System designed to enhance current reading comprehension-focused tutoring systems by incorporating LLM-powered personalized learning. Read with Emora integrates large language models to generate AI-driven reading passages, context-based questions, and adaptive feedback tailored to each student’s grade level, vocabulary list, grammar rules, and chosen topics. By integrating LLMs for personalized instruction, Read with Emora aims to improve content generation and personalized learning features present in current reading comprehension-focused ITSs.

This project demonstrates the potential for LLM-driven tutoring systems to provide accessible and free reading comprehension tools for parents, teachers, and students. Read with Emora generates personalized reading materials and assessments that would otherwise require purchasing, making reading practice more available to students regardless of background or resources. The evaluation of Read with Emora focuses on three key areas: (1) the integration of critical components such as learning objectives, grammar rules, and vocabulary words into generated passages through assigned integration scores, (2) the accuracy of the system's grading for context-based comprehension questions compared to human evaluators (analyzed through loss between human and LLM-assigned scores), and (3) the variety of questions generated by assigning questions and analyzing their distribution across six categories. The results show that while the system excels in integrating personalized topics and generating factual recall questions, it faces challenges in more context-dependent areas such as complex grammar rules, factual content integration, and Cause and Effect questions. The system also exhibited leniency in grading, often assigning partial credit for incomplete or incorrect responses, highlighting the need for refinement in the grading prompt.

This project offers a promising innovation for improving reading comprehension systems through LLMs. Future directions include refining Read with Emora's question generation and grading systems, particularly in generating nuanced question types and improving grading accuracy, incorporating second language learning benefits, and expanding the system's feature set with tools like speech-to-text functionality and image generation. Ultimately, this thesis underscores the importance of integrating LLMs into comprehension-focused educational tools to provide children with personalized, effective learning experiences.