An investigation of the information provided by the distractors of multiple-choice test items

Previous studies have indicated that the knowledge or random guessing principle is often contradicted by distractor analysis of multiple-choice test data. This study attempted to analyze item content to better describe those distractors providing differential information with regard to achievement level. Item Response Theory (IRT) methods and approaches were applied to response data for the Vocabulary subtest of the Iowa Tests of Basic Skills. Informative distractors were identified from the resultant plausibility functions, which express the relationship between achievement level and choice of distractor. Although considerable agreement in the prediction of informative distractors was found among educators asked to review item content, there was only modest success in the accuracy of these predictions. Content-based rules or reasons that would lead to the selection of a particular distractor were identified, and indications of a relationship between these rules and achievement level were found. However, much more research is needed to understand this link between distractor content and selection. Implications for item writers and test developers were discussed.