Modeling demands-resources fit in teacher education using open-ended data: a methodological-substantive synergy

Abstract

This study explores the effectiveness of large language models (LLMs) in automatically encoding a large set of open-ended responses to obtain data for use in applied statistics. As a case study, we focus on demands-resources fit processes and engagement in teacher education. To probe the validity of LLMs in investigating these processes, we compare results from measures obtained via ordinary Likert-type items (scale measures), and measures obtained from automatically encoding open-ended questions (LLM measures) for the same sample of student teachers (N = 499, 82% female, Mage=23.5 years). Results demonstrate the reliability of LLMs in processing and quantifying large amounts of open-ended data quickly and as accurately as scale measures. Moreover, results concur to reveal an "optimal margin" of demands-resources fit in student teacher engagement. Accordingly, study resources surpassing study demands maximizes engagement, whereas insufficient resources minimize it, and moderate levels of both demands and resources lead to intermediate engagement. By contrast, high or low levels of both demands and resources are suboptimal for engagement. Taken together, these findings demonstrate that LLM-derived statistics offer an efficient and reliable approach to extracting data from open-ended responses, enabling the large-scale analysis of qualitative insights while preserving their richness. This method facilitates the integration of qualitative and quantitative approaches, enhancing the study of individual behavior, and holds significant potential for enhancing digital education frameworks by supporting adaptive learning systems and digital assessment practices.