In this commentary, Rebecca Kopriva examines the articles in this special issue by drawing on her experience from three series of investigations examining how English language learners (ELLs) and other students perceive what test items ask and how they can successfully represent what they know. The first series examined the effect of different…

The primary focus of this article is ''advocating for the results of a specific evaluation.'' To some extent, all evaluators are advocates, however the author notes, "there is no great mystery in how to use evaluations for advocacy, especially with respect to influencing policy making within federal and state…

As another year is approaching, the time seems appropriate to look back and reflect on all the things that have been done, and more importantly learned during 2014. Along the same lines, and in order to offer further food for thought, the IBE is happy to share with you its latest Thematic alert on the topic of assessment in education. More…

The present article considers a fundamental question in evaluation research: "By how much do program effects vary across sites?" The article first presents a theoretical model of cross-site impact variation and a related estimation model with a random treatment coefficient and fixed site-specific intercepts. This approach eliminates…

Objectives: This study examined how evaluation and intervention research (IR) studies assessed statistical change to ascertain effectiveness. Methods: Studies from six core social work journals (2009-2013) were reviewed (N = 1,380). Fifty-two evaluation (n= 27) and intervention (n = 25) studies met the inclusion criteria. These studies were…

In educational research that calls itself empirical, the relationship between validity and reliability is that of trade-off: the stronger the bases for validity, the weaker the bases for reliability (and vice versa). Validity and reliability are widely regarded as basic criteria for evaluating research; however, there are ethical implications of…

Without deliberate and resourced translation, research evidence is unlikely to inform policy and practice. This paper describes the processes and practical solutions used to translate evaluation research findings to improve the readability of print materials in a large scale worksite health programme. It is argued that a knowledge brokering and…

This study examines the effectiveness of three approaches for maintaining equivalent performance standards across test forms with small samples: (1) common-item equating, (2) resetting the standard, and (3) rescaling the standard. Rescaling the standard (i.e., applying common-item equating methodology to standard setting ratings to account for…

No area of language assessment research in the past 20 years has received a greater increase in attention than washback research. Beginning with the seminal work of Alderson & Wall (Alderson & Wall 1993; Wall & Alderson 1993), an evolving body of empirical washback studies has been conducted worldwide, especially in countries where…

In this paper, we focus on participatory evaluation in the context of international development and specifically on the emerging empirical knowledge base. In a prior review and critique of research on participatory evaluation (Cousins and Chouinard 2012), we examined 121 studies, with only 21 (17%) situated in development contexts. However, the…

Evaluation theories can be tested in various ways. One approach, the experimental analogue study, is described and illustrated in this article. The approach is presented as a method worthy to use in the pursuit of what Alkin and others have called descriptive evaluation theory. Drawing on analogue studies conducted by the first author, we…

Psychometrics has recently undergone extensive criticism within the medical education literature. The use of quantitative measurement using psychometric instruments such as response scales is thought to emphasize a narrow range of relevant learner skills and competencies. Recent reviews and commentaries suggest that a paradigm shift might be…

Evaluation research conducted in agencies that sanction law violators is often challenging and due process may preclude evaluators from using experimental methods in traditional criminal justice agencies such as police, courts, and corrections. However, administrative agencies often deal with the same population but are not bound by due process…

This article discusses a range of current issues and future research possibilities in Communicative Language Testing (CLT) using, as its departure point, the key questions which emerged during the CLT symposium at the 2010 Language Testing Forum. The article begins with a summary of the 2010 symposium discussion in which three main issues related…

School choice programs continue to be controversial, spurring a number of researchers into evaluating them. When possible, researchers evaluate the effect of attending a school of choice using randomized designs to eliminate possible selection bias. Randomized designs are often thought of as the gold standard for research, but many circumstances…