Psychometrics of MOOCs: How to measure proficiency?

The popularity of online courses with open access and unlimited learner participation (MOOCs) has been growing intensively. Certificates of MOOC completion are becoming a significant element in learners' portfolios as well as an additional source of academic credits. That is why learners, professors, universities, and employers have an interest in accurate measures of learners' proficiency in MOOCs. Existing psychometric approaches do not serve perfectly as measurement tools for MOOCs. However, item response theory (IRT) is a flexible and well-elaborated framework, in which models could be tuned up to be applied for MOOCs. Thus, the general purpose of the doctoral dissertation is to propose model extensions that make IRT able to accurately measure learners' proficiency and to describe constructs closely linked with learners' proficiency in MOOCs. The doctoral thesis consists of five chapters following the introductory chapter - four empirical and one conceptual. In the four empirical chapters, we have proposed Rasch model extensions aimed at solving the following problems. In Chapter 2, we have improved the proficiency measures by modeling the effect of attempts and by involving non-assessment data such as learners' interaction with video lectures and practical tasks. In Chapter 3, we have modeled individual growth in proficiency through the MOOC as an effect of the cumulative sum of video lectures a learner has watched before responding on a summative assessment item. In Chapter 4, we have established a more nuanced insight on the role of proficiency on the learners' performance by involving one extra latent effect, the effect of learners' interest. In Chapter 5, we have proposed a way to measure learners' activity (e.g., watching videos, reading texts) as influenced by a latent learner characteristic and a latent content characteristic. The four studies have been conducted using the single psychometric framework of cross-classification multilevel logistic models and real datasets from the Coursera platform. In the final, reflective and conceptual chapter (Chapter 6), we have summarized a connection between psychometrics, as a scientific discipline, and MOOCs, as an industry, and have sketched ideas for the future development of psychometrics of MOOCs.

Publication year:2019

Accessibility:Closed