Evidence-based assessment (EBA) requires that investigators employ scientific theories and research findings to guide decisions about what domains to measure, how and when to measure them, and how to make decisions and interpret results. To implement EBA, investigators need high-quality assessment tools along with evidence-based processes. We…

Background/Context: Principles of evidential reasoning have often been discussed in the context of educational and psychological measurement with respect to construct validity and validity arguments. More recently, Mislevy proposed the metaphor of assessment as an evidentiary argument about students' learning and abilities given their…

The motivation for this paper is the authors' recent work on several randomized control trials in which they found the primary result, which averaged across subgroups or sites, to be moderated by demographic or site characteristics. They are led to examine a distinction that the Institute of Education Sciences (IES) makes between "confirmatory"…

Lecoutre, Lecoutre, and Poitevineau (2010) have provided sophisticated grounding for "p[subscript rep]." Computing it precisely appears, fortunately, no more difficult than doing so approximately. Their analysis will help move predictive inference into the mainstream. Iverson, Wagenmakers, and Lee (2010) have also validated…

P. R. Killeen's (2005a) probability of replication ("p[subscript rep]") of an experimental result is the fiducial Bayesian predictive probability of finding a same-sign effect in a replication of an experiment. "p[subscript rep]" is now routinely reported in "Psychological Science" and has also begun to appear in…

The sense that replicability is an important aspect of empirical science led Killeen (2005a) to define "p[subscript rep]," the probability that a replication will result in an outcome in the same direction as that found in a current experiment. Since then, several authors have praised and criticized 'p[subscript rep]," culminating…

Group projects are an important component of higher education, and the use of peer assessment of students' individual contributions to group projects has increased. The researchers employed an expectancy theory approach and an experimental design in a field setting to investigate conditions that influence students' motivation to rate their peers'…

Three critical issues in the design and analysis of evaluation studies suggested at the conference are (1) the univariate versus multivariate dependent variable studies, (2) the choice of a response surface design over the conventional fixed model, and (3) the tendency to interpret every study as if it were being done for the first time. Taking…

Purpose: The purpose of this study was to review research published by Educational Administration Quarterly (EAQ) during the past 10 years to determine if confidence intervals and effect sizes were being reported as recommended by the American Psychological Association (APA) Publication Manual. Research Design: The author examined 49 volumes of…

A method is presented for analyzing data collected in a multiple group time-series design. This consists of testing linear hypotheses about the experimental and control group-means. Both a multivariate and a univariate procedure are described. (Author/GK)

The objective of this study was to adapt the ideas of "strong inference" in developing a design procedure which can be used in the evaluation of an instructional system in such a way as to identify and correct specific weaknesses within a system. This method allows the evaluator to consider many hypotheses as possible causes of system…

The validity of the equipercentile hypothesis of the Title I Evaluation and Reporting System norm-referenced evaluation model was examined using 3,224 seventh- and ninth-grade students. Findings from confidence interval procedures contradicted the equipercentile hypothesis. There was a pattern of large gains for students not receiving any special…

The degree to which the chosen units of analysis are likely to produce spurious findings in staged combinations of multiple linear regression procedures are examined. The effects of grouping variables (e.g., classroom, school, and school district) on Procedures such as Coleman's semipartial regression and Mayeske's commonalities, in light of…

Presents a general framework for designing sampling programs that ensure cost effectiveness, and managed errors kept within known and acceptable limits. (LZ)