Elimination scoring versus correction for guessing: A simulation study

Correct for guessing is a commonly used scoring method in multiple-choice questions. A penalty is used for incorrect responses to discourage guessing. However, combining item response theory (IRT) and prospect theory of decision making under uncertainty, Budescu and Bo (2015) showed that a penalty has detrimental effects for examinees, especially for those who are risk averse. Another drawback of correct for guessing is its insensitivity to the differences between various knowledge levels. This can be dealt with a scoring method that credits partial knowledge, such as elimination scoring (Coombs, Milholland, & Womer, 1956). It requires examinees to eliminate alternatives that they consider to be incorrect. This study investigates the combined effect of ability and risk aversion on expected scores on multiple-choice items, and compares these effects under two testing instructions, namely, correction for guessing and elimination scoring. A model is proposed to simulate expected answering patterns on multiple-choice items, combining the IRT and prospect theory. It consists of two steps: (1) probabilities of a correct response to each of the alternatives in a multiple-choice question are modeled using Rasch model based on ability; (2) the decision making of giving a particular answering pattern is modeled using the prospect theory taking into account risk aversion. The results from the simulation revealed that overall ability had a predominant effect on expected scores, while risk aversion had a decisive impact on expected answering patterns especially for examinees with lower abilities. Examinees with medium ability levels benefited from using elimination scoring.

ISBN:978-3-319-77249-3

Jaar van publicatie:2018

BOF-keylabel:ja

IOF-keylabel:ja

Authors from:Higher Education

Toegankelijkheid:Closed