Test-retest Reliability
Test-retest reliability is the consistency of scores when the same test is administered to the same examinees on two occasions. The Pearson correlation between the two score sets is termed the coefficient of stability and indexes the proportion of score variance attributable to a stable trait rather than occasion-specific fluctuation.
Estimation
Examinees take the test at time 1 and again at time 2 under conditions as similar as practicable. The correlation between paired scores estimates stability over the chosen interval. In classical test theory the coefficient is interpreted as a lower bound on the proportion of true-score variance, on the assumption that the underlying trait is itself stable across the interval and that conditions of measurement do not change.
Methodological tensions
The interval is critical and contested. Short intervals — days or a week — risk practice and memory effects, inflating the coefficient because examinees recall items or strategies. Long intervals — months — allow genuine learning, maturation, or attrition, depressing the coefficient even when the instrument is sound. Two-week to one-month windows are common compromises in language testing research, but the choice is always defensible only with reference to the construct's expected stability.
The procedure is poorly suited to constructs that change rapidly under instruction, which describes much of language proficiency. For developing skills, alternate-form or internal-consistency estimates are usually more informative. Carryover effects — fatigue, motivation, item exposure — also complicate interpretation, and counterbalanced designs or alternate forms across occasions are sometimes used to mitigate them.
Reported coefficients are sample-dependent: reliability is a property of scores in a population, not of the test in the abstract.
References
- AERA, APA, & NCME (2014). Standards for Educational and Psychological Testing. American Educational Research Association.
- Bachman, L. F. (1990). Fundamental Considerations in Language Testing. Oxford University Press.
- McNamara, T. (2000). Language Testing. Oxford University Press.