Internationally, the political appetite for educational measurement capable of capturing a metric of value for money and effectiveness has momentum. While most would agree with the need to assess costs relevant to quality to help support better governmental policy decisions about public spending, poorly understood measurement comes with unintended…

This article proposes that sampling design effects have potentially huge unrecognized impacts on the results reported by large-scale district and state assessments in the United States. When design effects are unrecognized and unaccounted for they lead to underestimating the sampling error in item and test statistics. Underestimating the sampling…

Background: The Thinking Styles Inventory--developed by Sternberg and Wagner based on Sternberg's (1988, 1997) earlier theory of mental self-government--was selected for the research in order to assess thinking styles of student teachers. Another reason is that the theoretical constructs, as well as the inventory generated from the theory, have…

It is argued in this paper that generalizability theory provides a uniquely useful framework for defining and quantifying the dependability of data for decision making. It does so by requiring careful specification of the conditions of measurement and the anticipated sources of variation in the results of the measurement procedure. A distinction…

An examination of typical situations concerning the status of a variable is followed by a discussion of the measurement and empirical problems involved in measuring current status. Arguments for more consideration of variability, particularly individual variation, are presented. Among the problems discussed are (1) describing large samples through…

This report describes a statistically designed experiment that was carried out in an operational setting to determine the contributions of different sources of variation to the unreliability of scoring. The experiment made novel use of partially balanced incomplete block designs that facilitated the unbiased estimation of certain main effects…

The pretest-posttest design referred to as Design 2 by Campbell and Stanley (1963) is commonly used in educational research and evaluation. The tenability of the assumption of a zero population difference commonly used with this design is questioned. A nonzero population estimate based on the mean difference observed in test-retest reliability…

Textbook calculations of statistical power or sample size follow from formulas that assume that the variables under consideration are measured without error. However, in the real world of behavioral research, errors of measurement cannot be neglected. The determination of sample size is discussed, and an example illustrates blocking strategy.…

Four different procedures were used for estimating the proportion of persons who would be classified consistently as either passing both of two parallel tests or failing both. These four methods were applied at each of four different mastery level scores for each of three different length tests. Data were based on 50 replications of each procedure…

This paper examines issues in designing post-school follow-up studies in special education. The examination focuses on survey research techniques, which are widely used in the investigation of post-school adjustment of former students with handicaps. In special education, survey research studies are used commonly to address many important…

The statistical properties of two methods of estimating gain scores for groups in quasi-experiments are compared: (1) gains in scores standardized separately for each group; and (2) analysis of covariance with estimated true pretest scores. The fan spread hypothesis is assumed for groups but not necessarily assumed for members of the groups.…

The most popular activities from APA's successful "Activities Handbooks for the Teaching of Psychology" are gathered together and updated in this book of teachers' favorites. The lesson plans, which encourage active learning and involve the whole class, have stood the test of time and proven themselves to be entertaining, effective, and easy to…