Replication
research-methodology
Replication is the practice of repeating a study to verify whether its findings hold. It is a cornerstone of scientific knowledge — a finding that cannot be reproduced is not trustworthy. Yet replication is strikingly rare in SLA and applied linguistics research.
Types of Replication
Language Teaching Replication Research (Porte, 2012; Marsden et al., 2018) distinguishes three types:
| Type | What changes | Purpose |
|---|---|---|
| Exact (direct) replication | Nothing — same procedures, instruments, population | Tests whether original results can be reproduced |
| Approximate (close) replication | Minor changes (different participants, setting) | Tests robustness across similar contexts |
| Conceptual replication | Different method, same research question | Tests whether the finding holds regardless of methodology |
Exact replications are the strongest test of a finding's reliability but are vanishingly rare in SLA.
The Replication Crisis in SLA
Marsden, Morgan-Short, Thompson & Abugaber (2018) conducted a systematic review and found that fewer than 1 in 400 published SLA articles was a self-labelled replication. Moreover:
- Most "replications" made so many changes that they were effectively new studies
- Direct replications were almost non-existent
- Journals historically favoured novel findings, creating a publication bias against replication
- Many original studies lacked sufficient methodological detail to be replicated at all
This mirrors the broader replication crisis in psychology and the social sciences.
Why Replication Matters for SLA
- Statistical Significance thresholds (p < .05) mean that 1 in 20 "significant" results may be a false positive — only replication can distinguish real effects from statistical noise
- Small samples in classroom research produce unstable effect sizes that may not replicate
- Meta-analyses synthesise findings, but they are only as good as the primary studies — if those studies are unreplicated, the synthesis may amplify unreliable findings
- Pedagogical decisions based on single, unreplicated studies are risky
Initiatives to Promote Replication
- IRIS database (Marsden & colleagues) — open repository of research instruments, enabling exact replication
- Registered Reports — journals (e.g., Language Learning) accept study protocols before data collection, reducing publication bias against null results
- Open data and materials — sharing datasets allows re-analysis and verification
- Multi-site replication projects — Morgan-Short, Marsden et al. (2018) conducted a coordinated multi-site replication of attention to form research
Key References
- Porte (2012) — Replication Research in Applied Linguistics
- Marsden, Morgan-Short, Thompson & Abugaber (2018) — systematic review of replication in SLA
- Language Teaching Replication Research project — coordinated replication initiatives
- Plonsky (2013) — methodological transparency as a prerequisite for replication