Programme Evaluation

The systematic appraisal of a specific language programme or course (its design, delivery, and effects) to inform decisions about its continuation, revision, or expansion. Where curriculum evaluation often targets the framework, programme evaluation targets a bounded instance: a particular EAP course at a particular institution, a national initiative, an in-company training scheme.

Canonical Treatments

Lynch (1996, Language Program Evaluation: Theory and Practice, Cambridge University Press) is the first book-length applied-linguistics treatment. Lynch proposes a context-adaptive model: evaluation design is not chosen off the shelf but built from the programme's audience, purpose, and constraints. He argues for shifting beyond pre/post test gains toward investigation of programme process through qualitative, naturalistic inquiry alongside quantitative measures.

Kiely and Rea-Dickins (2005, Program Evaluation in Language Education, Palgrave Macmillan) consolidates the field a decade later. Drawing on case studies across school, tertiary, and workplace contexts, they map the evaluator's stance (insider, outsider, commissioned), the political economy of evaluation, and the methodological repertoire available: questionnaires, observation, interviews, document analysis, test data, classroom recordings.

Norris (2009, "Language program evaluation", in Doughty & Long eds., The Handbook of Language Teaching, Wiley-Blackwell) and the special issue of Language Teaching Research 13(1) he edited the same year argue that programme evaluation in ELT remained underdeveloped relative to assessment research, and call for evaluation embedded in routine programme operation rather than commissioned only at moments of crisis or funding review.

Formative vs Summative Purposes

Two purposes drive evaluation design and have been distinguished in the wider evaluation literature since Scriven (1967):

• Formative evaluation gathers information during a programme to improve it. Findings feed back into teacher development, materials revision, or syllabus adjustment while the programme runs.
• Summative evaluation judges a programme at completion against its stated goals, typically to support decisions about continuation, scaling, or termination.

The same instrument can serve either purpose; what differs is timing and audience. A mid-course questionnaire treated as formative becomes summative if its results are filed for the funder rather than used to revise teaching.

Designing a Programme Evaluation

A defensible evaluation specifies, in advance:

• Audience and purpose: who will read the report and what decisions it informs
• Evaluation questions: a small number of focused, answerable questions, not a sweep of everything
• Data sources and methods: matched to questions, triangulated across stakeholders
• Criteria and standards: how "good enough" will be defined, agreed before data collection
• Reporting format: what will be shared, with whom, and on what timeline

Lynch and Norris both emphasise that utilisation — whether findings actually feed decisions — is the strongest test of an evaluation, not its methodological elegance.

Stakeholders and Politics

Sponsors, administrators, teachers, learners, and external regulators bring incompatible criteria. A programme rated successful on completion rates may be judged costly by funders, narrow by employers, or stressful by teachers. Kiely and Rea-Dickins treat this incompatibility as the substance of evaluation rather than noise to be averaged out: surfacing whose criteria count, and why, is part of the work.

References

• Lynch, B. K. (1996). Language Program Evaluation: Theory and Practice. Cambridge University Press.
• Kiely, R., & Rea-Dickins, P. (2005). Program Evaluation in Language Education. Palgrave Macmillan.
• Norris, J. M. (2009). Language program evaluation. In C. J. Doughty & M. H. Long (Eds.), The Handbook of Language Teaching (pp. 578–594). Wiley-Blackwell.
• Norris, J. M. (Ed.). (2009). Understanding and improving language education through program evaluation [Special issue]. Language Teaching Research, 13(1).