Improving multiple-choice questions to better assess dental student knowledge: Distractor utilization in oral and maxillofacial pathology course examinations

C. Alex McMahan, R. N Pinckard, Thomas J. Prihoda, William D. Hendricson, Anne C Jones

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

How many incorrect response options (known as distractors) to use in multiple-choice questions has been the source of considerable debate in the assessment literature, especially relative to influence on the likelihood of students' guessing the correct answer. This study compared distractor use by second-year dental students in three successive oral and maxillofacial pathology classes that had three different examination question formats and scoring resulting in different levels of academic performance. One class was given all multiple-choice questions; the two other were given half multiple-choice questions, with and without formula scoring, and half un-cued short-answer questions. Use by at least 1 percent of the students was found to better identify functioning distractors than higher cutoffs. The average number of functioning distractors differed among the three classes and did not always correspond to differences in class scores. Increased numbers of functioning distractors were associated with higher question discrimination and greater question difficulty. Fewer functioning distractors fostered more effective student guessing and overestimation of academic achievement. Appropriate identification of functioning distractors is essential for improving examination quality and better estimating actual student knowledge through retrospective use of formula scoring, where the amount subtracted for incorrect answers is based on the harmonic mean number of functioning distractors.

Original languageEnglish (US)
Pages (from-to)1593-1608
Number of pages16
JournalJournal of Dental Education
Volume77
Issue number12
StatePublished - Dec 1 2013

Fingerprint

Oral Pathology
Dental Students
pathology
utilization
Students
examination
student
academic achievement
discrimination
performance

Keywords

  • Assessment
  • Correction for guessing
  • Dental education
  • Educational assessment
  • Formula scoring
  • Functioning distractors
  • Multiple-choice questions
  • Oral and maxillofacial pathology

ASJC Scopus subject areas

  • Dentistry(all)
  • Education
  • Medicine(all)

Cite this

Improving multiple-choice questions to better assess dental student knowledge : Distractor utilization in oral and maxillofacial pathology course examinations. / McMahan, C. Alex; Pinckard, R. N; Prihoda, Thomas J.; Hendricson, William D.; Jones, Anne C.

In: Journal of Dental Education, Vol. 77, No. 12, 01.12.2013, p. 1593-1608.

Research output: Contribution to journalArticle

@article{949880bf1274469a8c702b213dc71bf2,
title = "Improving multiple-choice questions to better assess dental student knowledge: Distractor utilization in oral and maxillofacial pathology course examinations",
abstract = "How many incorrect response options (known as distractors) to use in multiple-choice questions has been the source of considerable debate in the assessment literature, especially relative to influence on the likelihood of students' guessing the correct answer. This study compared distractor use by second-year dental students in three successive oral and maxillofacial pathology classes that had three different examination question formats and scoring resulting in different levels of academic performance. One class was given all multiple-choice questions; the two other were given half multiple-choice questions, with and without formula scoring, and half un-cued short-answer questions. Use by at least 1 percent of the students was found to better identify functioning distractors than higher cutoffs. The average number of functioning distractors differed among the three classes and did not always correspond to differences in class scores. Increased numbers of functioning distractors were associated with higher question discrimination and greater question difficulty. Fewer functioning distractors fostered more effective student guessing and overestimation of academic achievement. Appropriate identification of functioning distractors is essential for improving examination quality and better estimating actual student knowledge through retrospective use of formula scoring, where the amount subtracted for incorrect answers is based on the harmonic mean number of functioning distractors.",
keywords = "Assessment, Correction for guessing, Dental education, Educational assessment, Formula scoring, Functioning distractors, Multiple-choice questions, Oral and maxillofacial pathology",
author = "McMahan, {C. Alex} and Pinckard, {R. N} and Prihoda, {Thomas J.} and Hendricson, {William D.} and Jones, {Anne C}",
year = "2013",
month = "12",
day = "1",
language = "English (US)",
volume = "77",
pages = "1593--1608",
journal = "Journal of Dental Education",
issn = "0022-0337",
publisher = "American Dental Education Association",
number = "12",

}

TY - JOUR

T1 - Improving multiple-choice questions to better assess dental student knowledge

T2 - Distractor utilization in oral and maxillofacial pathology course examinations

AU - McMahan, C. Alex

AU - Pinckard, R. N

AU - Prihoda, Thomas J.

AU - Hendricson, William D.

AU - Jones, Anne C

PY - 2013/12/1

Y1 - 2013/12/1

N2 - How many incorrect response options (known as distractors) to use in multiple-choice questions has been the source of considerable debate in the assessment literature, especially relative to influence on the likelihood of students' guessing the correct answer. This study compared distractor use by second-year dental students in three successive oral and maxillofacial pathology classes that had three different examination question formats and scoring resulting in different levels of academic performance. One class was given all multiple-choice questions; the two other were given half multiple-choice questions, with and without formula scoring, and half un-cued short-answer questions. Use by at least 1 percent of the students was found to better identify functioning distractors than higher cutoffs. The average number of functioning distractors differed among the three classes and did not always correspond to differences in class scores. Increased numbers of functioning distractors were associated with higher question discrimination and greater question difficulty. Fewer functioning distractors fostered more effective student guessing and overestimation of academic achievement. Appropriate identification of functioning distractors is essential for improving examination quality and better estimating actual student knowledge through retrospective use of formula scoring, where the amount subtracted for incorrect answers is based on the harmonic mean number of functioning distractors.

AB - How many incorrect response options (known as distractors) to use in multiple-choice questions has been the source of considerable debate in the assessment literature, especially relative to influence on the likelihood of students' guessing the correct answer. This study compared distractor use by second-year dental students in three successive oral and maxillofacial pathology classes that had three different examination question formats and scoring resulting in different levels of academic performance. One class was given all multiple-choice questions; the two other were given half multiple-choice questions, with and without formula scoring, and half un-cued short-answer questions. Use by at least 1 percent of the students was found to better identify functioning distractors than higher cutoffs. The average number of functioning distractors differed among the three classes and did not always correspond to differences in class scores. Increased numbers of functioning distractors were associated with higher question discrimination and greater question difficulty. Fewer functioning distractors fostered more effective student guessing and overestimation of academic achievement. Appropriate identification of functioning distractors is essential for improving examination quality and better estimating actual student knowledge through retrospective use of formula scoring, where the amount subtracted for incorrect answers is based on the harmonic mean number of functioning distractors.

KW - Assessment

KW - Correction for guessing

KW - Dental education

KW - Educational assessment

KW - Formula scoring

KW - Functioning distractors

KW - Multiple-choice questions

KW - Oral and maxillofacial pathology

UR - http://www.scopus.com/inward/record.url?scp=84891883082&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84891883082&partnerID=8YFLogxK

M3 - Article

C2 - 24319131

AN - SCOPUS:84891883082

VL - 77

SP - 1593

EP - 1608

JO - Journal of Dental Education

JF - Journal of Dental Education

SN - 0022-0337

IS - 12

ER -