Falsifiable Distractor

A falsifiable distractor is an incorrect option in a multiple-choice item that a candidate with partial understanding can plausibly select. The term names the operational target of distractor design: an option must be wrong, but wrong in a way that requires the target sub-skill to detect. Distractors that fall too far from plausible — random unrelated content, grammatical mismatches, obvious nonsense — degrade the item to a 3-choice or 2-choice format and reduce discrimination.

The framing comes out of the diagnostic-distractor literature. To build a falsifiable distractor, the writer must model the specific reasoning error a candidate at the target proficiency would make and turn that error into the option. The test then discriminates between the candidate who completes the inference and the candidate who completes most but not all of it.

The design rule

Three properties define a falsifiable distractor:

• Surface coherence with the passage. The distractor uses words, ideas, or structures actually present in the text. Candidates without the target skill see superficial overlap and select.
• Reasoning-error alignment. The distractor lands exactly where a candidate's reasoning would stop one step short of the key. For paraphrase items, the distractor matches the surface form of a wrong sentence in the passage; for inference items, it captures a literal reading the writer's argument actually overrides.
• Definitive falseness on full comprehension. A candidate who fully grasps the passage rejects the distractor unambiguously. Plausible-but-wrong, never plausible-and-defensible.

The third property is what separates a falsifiable distractor from an ambiguous one. Ambiguous options reduce reliability and invite candidate appeals; falsifiable options sharpen the test.

Empirical signal

Item analysis on falsifiable distractors shows a characteristic pattern: low-scoring candidates select the distractor at meaningful rates, high-scoring candidates do not, and the discrimination index is positive. A distractor that no one chooses is dead weight; one that high-scorers choose is mis-keyed or genuinely ambiguous. The 5Ps Distractor Typology (Sun, Yang & Liu 2026) gives a finer-grained vocabulary for the kinds of reasoning errors falsifiable distractors can target.

Implications for AI-assisted distractor generation

Falsifiability is the property AI distractor generators struggle most with. Out-of-the-box LLM generation produces distractors that are surface-coherent but reasoning-misaligned: they look plausible to the model and are obviously wrong to a competent candidate. Closing the gap requires explicit modelling of the target candidate's reasoning errors in the prompt, retrieval of grounded passage spans the distractor must trace back to, and post-hoc filtering against an item-analysis prior built from human-authored distractors.

Key References

• Haladyna, T. M., Downing, S. M. & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309–334.
• Gierl, M. J., Bulut, O., Guo, Q. & Zhang, X. (2017). Developing, analyzing, and using distractors for multiple-choice tests in education. Review of Educational Research, 87(6), 1082–1116.
• Sun, Y., Yang, Y. & Liu, X. (2026). Proposing the 5Ps typology of distractors for EFL multiple-choice reading comprehension tests. Higher Education Studies, 16(1).

See Also

• Distractor: the parent concept
• 5Ps Distractor Typology: a recent typology of falsifiable distractor kinds
• Item Discrimination: the empirical signature falsifiable distractors produce
• Item Analysis: the diagnostic loop that validates distractor quality