A critical revisiting of research methods with multilingual learners raises issues of validity that are often unaddressed in current discussions of research quality. Though validity is conceptualized and assessed differently across research traditions, validity broadly describes the production of findings that are credible and trustworthy. The…

"Decolonising our approaches to educational research-- Learning from our shared experiences at Rangahau Matauranga o Aotearoa (NZCER)" shares insights from a recent workshop at NZCER focused on the questions, tensions and challenges our kairangahau are grappling with in research and evaluation. It includes a description of the workshop…

Our understanding of programs is enhanced when trained, skilled, and observant evaluators go "into the field"--the real world where programs are conducted--paying attention to what's going on, systematically documenting what they see, and reporting what they learn. The article opens by presenting and illustrating twelve reasons for…

Qualitative interviewing is a flexible and powerful tool to capture the voices and the ways people make meaning of their experience Learning to conduct semi-structure interviews requires the following six stages: (a) selecting the type of interview; (b) establishing ethical guidelines, (c) crafting the interview protocol; (d) conducting and…

This paper provides practical guidance for researchers who are designing and analyzing studies that randomize schools--which comprise three levels of clustering (students in classrooms in schools)--to measure intervention effects on student academic outcomes when information on the middle level (classrooms) is missing. This situation arises…

Some researchers have argued for the use of or have attempted to make use of randomization tests in single-subject research. To address this tide of interest, the authors of this article describe randomization tests, discuss the theoretical rationale for applying them to single-subject research, and provide an overview of the methodological…

Good evaluation requires that evaluation efforts themselves be evaluated. Many things can and often do go wrong in evaluation work. Accordingly, it is necessary to check evaluations for problems such as bias, technical error, administrative difficulties, and misuse. Such checks are needed both to improve ongoing evaluation activities and to assess…

Evaluations that combine social science and law have tremendous potential to illuminate the effects of governmental policies and yield insights into how effectively policy makers' efforts achieve their aims. This potential is infrequently achieved, however, because such interdisciplinary research contains often overlooked substantive and…

Higher education experts tout learning outcomes assessment as a vehicle for program improvement. To this end the authors share a rubric designed explicitly to evaluate the quality of assessment and how it leads to program improvement. The rubric contains six general assessment areas, which are further broken down into 14 elements. Embedded within…

A large body of literature documents the central importance of fidelity of program implementation in creating an internally valid research design and considering such fidelity in judgments of research quality. The What Works Clearinghouse (WWC) provides web-based summary ratings of educational innovations and is the only rating group that is…

The field of evaluation faces a number of serious challenges in light of No Child Left Behind legislation, among them feasibility, resources, and blurring lines among research, evaluation, and assessment. At the same time, these challenges open the door for opportunities in evaluation. Now more than ever, the expertise of evaluators is needed and…

Donald W. Compton, the first director of evaluation services at the National Home Office (Atlanta) of the American Cancer Society, tells the story of building the unit in conditions of high demand and a limited budget. Along the way, evaluation was brought to regional divisions and to local offices in part as a response to United Way and to his…

This article describes a research-driven heuristic for the scholarly evaluation of teaching and learning interventions, which is systematic, collaborative, and discipline focused. We offer this guide to educational developers and other instructional support staff who are tracking the impact of interventions in teaching and learning with academic…

In its most basic form, strategic planning is a process of anticipating change, identifying new opportunities, and executing strategy. The use of mixed methods, blending quantitative and qualitative analytical techniques and data, in the process of assembling a strategic plan can help to ensure a successful outcome. In this article, the author…

The last major review of the achievement outcomes of comprehensive school reform (CSR) models was conducted in 2003. Despite the growing evidence base supporting CSR, the program was discontinued by the federal government in 2007. Now, six years after the 2003 meta-analysis, the study's lead author, Geoffrey Borman, revisits the results and…