Purpose: To assess strengths and weaknesses of the reading level (RL) match approach and its potential to generate insights regarding the cognitive foundations of reading ability and disability. Method: We applied RL-match design to a sample of 2nd-6th graders reading a consistent orthography, Russian, using an "extreme phenotype"…

This article explores the viability of online crowdsourcing for creating matched-comparison groups. This exploratory study compares survey results from a randomized control group to survey results from a matched-comparison group created from Amazon.com's MTurk crowdsourcing service to determine their comparability. Study findings indicate…

In this paper, we examine the validity and precision of two nonexperimental study designs (NXDs) that can be used in educational evaluation: the comparative interrupted time series (CITS) design and the difference-in-difference (DD) design. In a CITS design, program impacts are evaluated by looking at whether the treatment group deviates from its…

This study examined the impact of the 2+2 Alternative Teacher Performance Appraisal System that has been implemented in Shanxi province in China. A mixed research design was used to evaluate the program. Six high schools and a total of 78 teachers (13 teachers in each school) in Shanxi province were selected. Three of the schools participated in…

This review and critique of the use of comparison groups in 57 studies of mental retardation family research addresses such issues as: matching on age versus competency of the mentally retarded family member, demographic equivalence of groups, research involving siblings, research involving mentally retarded parents, and appropriateness of…

Despite its apparent simplicity, random assignment is an abstract concept for many students. This article describes a classroom demonstration that uses a standard deck of playing cards. In small groups, students randomly assign playing cards (i.e., participants) to 2 treatment groups. Following randomization, students compare the relative…

This article discusses research design problems peculiar to work with intellectually disabled subjects, such as discrepancies in uses of control groups and mental age matching, definitions of subject groups and variability within and between subject groups. Suggestions for minimizing these problems are made and alternative methodologies offered.…

Evidence is presented that strengthens the argument against comparison group research strategies. Results from a study of youth attitudes related to the California Conservation Corps involving 528 participants and 668 comparisons indicate that the comparison group may not adequately represent the nontreatment state. (SLD)

A study was conducted to obtain pilot evidence that could serve as a basis for developing convincing methods for individual Australian companies to use in determining their returns from investment in training. The study attempted to replicate survey results from significant overseas surveys by using information collected on more than 90 Australian…

The National Job Corps Study was conducted in 1994-1996 to provide a thorough and rigorous assessment of the impacts of the Job Corps on key participant outcomes. To ensure that the study was well implemented, a study team from Mathematica Policy Research, Inc., (MPR) investigated outcome and admission (OA) procedures in each Job Corps region and…