Measuring and optimising latent variables in technology-enhanced learning contexts

This dissertation sheds more light onto the measurement of latent variables, through the exploration of use cases on engagement and cognitive load. Different data sources are monitored and at the same time, self-reports are collected. In this way, the studies verify which data sources are related to the latent variable. Results indicate that although some data sources are significantly associated to the self-reported scores, the measurement model cannot approximate these scores very well, since effect sizes are small. Although these two use cases focus on the measurement of engagement and cognitive load in particular, they also result in a step-by-step plan that can serve as general guidelines for researchers that aim to measure any latent variable. Besides, in order to support researchers that struggle to analyse complex multimodal data, this dissertation also elaborates on the use of different statistical and machine learning techniques, and explains their general working principle as well as their specific application to measure latent variables. Although self-reports are not perfect, they serve as an interesting golden standard to find out which proxies are related to the latent variable at hand. Still, random deviations and bias in self-reporting respectively affect the model's reliability and validity. Researchers are advised to design the conditions as to induce an as high as possible variance in the latent variable of interest and improve the validity of self-reports (e.g. through confidentiality, anonymity, clear and unambiguous questions, see Morsbach & Prinz, 2006). Besides measuring, this dissertation also addresses the optimisation of latent variables in technology-enhanced learning contexts. One study optimises the engagement of both face-to-face and remote learners in a virtual classroom. The results enable to improve the learning and teaching experience within this learning space both from a technical and pedagogical perspective. Another study investigates the optimisation of Augmented Reality (AR) instructions as cognitive support during assembly work. Results indicate that AR can entail interesting benefits, especially for novice learners. The effect of AR instructions with a different level of detail is also studied. These support levels are not perceived very differently, except a small differences in perceived complexity, which yields only limited evidence for adaptivity. Presumably, tasks also need to be sufficiently complex in order for adaptive instructions to be able to make a difference (Ginns, 2006; Reiser, 2004). The data-driven approaches that this dissertation takes also provide more insights on the challenging but interesting question on what AI can and cannot do in terms of measuring latent variables. Although the question remains difficult to answer, this dissertation comes up with three criteria that determine how likely AI is to be successful. These are 1) the ability to capture relevant data, 2) the quantity of training data and 3) the degree of logicalness of the task at hand. The increasing importance of lifelong learning (OECD, 2019) and the ongoing digitalisation have made learning and technology strongly intertwined. This intertwining has entailed several challenges and opportunities, such as revealing indicators of learning processes and optimising technology-enhanced learning contexts. These topics will undoubtedly continue to provide many interesting research and development opportunities in the future.

Publication year:2021

Accessibility:Closed