The present article considers a fundamental question in evaluation research: "By how much do program effects vary across sites?" The article first presents a theoretical model of cross-site impact variation and a related estimation model with a random treatment coefficient and fixed site-specific intercepts. This approach eliminates…

Elicited imitation (EI) has been widely used to examine second language (L2) proficiency and development and was an especially popular method in the 1970s and early 1980s. However, as the field embraced more communicative approaches to both instruction and assessment, the use of EI diminished, and the construct-related validity of EI scores as a…

The examination of cross-classified category data is common in evaluation and research, with Karl Pearson's family of chi-square tests representing one of the most utilized statistical analyses for answering questions about the association or difference between categorical variables. Unfortunately, these tests are also among the more commonly…

The analysis of differential item functioning (DIF) examines whether item responses differ according to characteristics such as language and ethnicity, when people with matching ability levels respond differently to the items. This analysis can be performed by calculating various statistics, one of the most important being the Mantel-Haenszel,…

Autism Diagnostic Interview-Revised (Rutter et al. in "Autism diagnostic interview-revised." Western Psychological Services, Los Angeles, 2003) diagnostic algorithms specific to toddlers and young preschoolers were created using 829 assessments of children aged from 12 to 47 months with ASD, nonspectrum disorders, and typical development. The…

The study assesses the psychometric properties of the Italian version of the Norwegian Teacher Self-Efficacy Scale--NTSES. Multiple group confirmatory factor analysis was used to explore the measurement invariance of the scale across two countries. Analyses performed on Italian and Norwegian samples confirmed a six-factor structure of the scale…

Recently, there have been advances in the development of multidimensional poverty measures. Work is needed however on how to implement such measures. This paper deals with the process of selecting dimensions and setting weights in multidimensional poverty measurement using qualitative and quantitative methods in a participatory framework. We…

This article investigates three types of stage-sequential growth mixture models in the structural equation modeling framework for the analysis of multiple-phase longitudinal data. These models can be important tools for situations in which a single-phase growth mixture model produces distorted results and can allow researchers to better understand…

There has been a steady increase in the use of mixed-format tests, that is, tests consisting of both multiple-choice items and constructed-response items in both classroom and large-scale assessments. This calls for appropriate equating methods for such tests. As Item Response Theory (IRT) has rapidly become mainstream as the theoretical basis for…

When comparing latent variables among groups, it is important to first establish the equivalence or invariance of the measurement model across groups. Confirmatory factor analysis (CFA) is a commonly used methodological approach to examine measurement equivalence/invariance (ME/I). Within the CFA framework, the chi-square goodness-of-fit test and…

Characterising the relationship between participants' scores on two different questionnaires is a common problem in educational research. The complement of the statistic known as Wilks' lambda measures the amount of variance shared between the scores obtained by the same group of participants on two sets of variables. (1 - lambda) is symmetric, in…

There has been much research concerning functional assessment over the past 20 years, but several important research considerations have yet to be explained. One is the comparison of different types of functional assessment (e.g., experimental functional analysis and non-experimental functional assessment). The current study aims to compare the…

Education researchers, practitioners, and policymakers alike are committed to identifying interventions that teach students more effectively. Increased emphasis on evaluation and accountability has increased desire for sound evaluations of these interventions; and at the same time, school-level data have become increasingly available. This article…

This monograph critiques the many nonexperimental impact estimation approaches that have been based on external comparison groups. It proposes an approach to evaluating employment and training (E&T) programs that calls for using the group of individuals who apply to a program but then choose not to participate in that program as an…

Describes a study of the retrieval results of World Wide Web search engines. Research quantified accurate matches versus matches of arguable quality for 200 subjects relevant to undergraduate curricula. Both "evaluative" engines (Magellan, Point Communications) and "nonevaluative" engines (Lycos, InfoSeek, AltaVista) were…