An Angoff standard setting study generally yields judgments on a number of items by a number of judges (who may or may not be nested in panels). Variability associated with judges (and possibly panels) contributes error to the resulting cut score. The variability associated with items plays a more complicated role. To the extent that the mean item…

Educational materials focused on spatial data analysis often feature mathematical descriptions of methods and step-by-step instructions of software tools, but infrequently discuss the set of decisions involved in specifying a statistical model. Failing to consider model specification may lead to specification searching, or the process of repeating…

This paper critically explores how survey and routinely collected data could aid in assessing the generalisability of public health evidence. We propose developing approaches that could be employed in understanding the relevance of public health evidence, and investigate ways of producing meta-analytic estimates tailored to reflect local…

The University of Reading Learning Gain project is a three-year longitudinal project to test and evaluate a range of available methodologies and to draw conclusions on what might be the right combination of instruments for the measurement of Learning Gain in higher education. This paper analyses the validity of a measure of critical thinking…

Classroom observations are increasingly common in education policies as a means to assess the quality of teachers and/or education programs for purposes of making high-stakes decisions. This article considers one policy, the Head Start Designation Renewal System (DRS), which involves classroom observations to assess the quality of Head Start…

Bipolar disorder is frequently clinically diagnosed in youths who do not actually satisfy Diagnostic and Statistical Manual of Mental Disorders (4th ed., text revision; DSM-IV-TR; American Psychiatric Association, 1994) criteria, yet cases that would satisfy full DSM-IV-TR criteria are often undetected clinically. Evidence-based assessment methods…

The present study sought to evaluate (a) the effectiveness of an intervention in developing adolescents' argument skills regarding a decision on a topic of high potential personal relevance (teen pregnancy) or one of general social relevance (capital punishment), and (b) differential effects of the two topics in promoting the generalization of…

Literature on the generalizability of program effects focuses on the a priori development of evaluation designs which enable certain generalizations to be made. Secondary analysis procedures which can be employed using existing evaluation data to estimate a program's generalizability when follow-up field studies are not feasible are suggested.…

A formal model is presented of the statistical properties of molecular behavioral matching. The model indicates that aggregation effects should be controlled in analysis of matching and that comparison of molecular theories of matching performance affords ideographic analysis of choice behavior. (SLD)

The paper examines "Decision Rules" in the evaluation of intervention techniques to promote skill generalization with mentally retarded persons. Two studies are reported which demonstrated that use of Decision Rules to select training strategies resulted in 80 percent or above skill generalization. (DB)