The measurement error in principal components extracted from a set of fallible measures is discussed and evaluated. It is shown that as long as one or more measures in a given set of observed variables contains error of measurement, so also does any principal component obtained from the set. The error variance in any principal component is shown…

This discussion constructs a survey data quality strategy for institutional researchers in higher education in light of total survey error theory. It starts with describing the characteristics of institutional research and identifying the gaps in literature regarding survey data quality issues in institutional research and then introduces the…

Reliability captures the influence of error on a measurement and, in the classical setting, is defined as one minus the ratio of the error variance to the total variance. Laenen, Alonso, and Molenberghs ("Psychometrika" 73:443-448, 2007) proposed an axiomatic definition of reliability and introduced the R[subscript T] coefficient, a measure of…

We are currently at an exciting juncture in developing effective means for assessing so-called 21st-century skills in an innovative yet reliable fashion. One of these avenues leads through the world of "epistemic games" (Shaffer, 2006a), which are games designed to give learners the rich experience of professional practica within a discipline.…

This paper intends to construct a survey data quality strategy for institutional researchers in higher education in light of total survey error theory. It starts with describing the characteristics of institutional research and identifying the gaps in literature regarding survey data quality issues in institutional research. Then it is followed by…

This report summarizes research findings and implementation practices for teacher and principal value-added models (VAMs), as a first step in the Team Pennsylvania Foundation's (Team PA) pilot project to inform the development of a full, statewide model evaluation system. We have selected 21 studies that represent key issues and findings in the…

The Center on Education Policy (CEP) report, "Has Progress Been Made in Raising Achievement for English Language Learners?", finds that some states have seen increases in the number of English language learners (ELLs) meeting proficiency standards under No Child Left Behind (NCLB), while others have seen decreases. The report notes some…

This article provides a framework for reporting internal consistency reliability in counseling research and other related social science fields, including guidelines relative to score reliability coefficients and associated confidence intervals for both full sample and subgroups. Follow-up techniques for investigating low score reliability are…

Although often ignored, reliability is critical when interpreting study effects and test results. Accordingly, this article focuses on the most commonly used estimate of reliability, internal consistency coefficients, with emphasis on coefficient alpha. An interpretive framework is provided for applied researchers and others seeking a conceptual…

The purpose of this study was to examine the potential impact of selected methodological factors on the validity of conclusions from reliability generalization (RG) studies. The study focused on four factors; (1) missing data in the primary studies; (2) transformation of sample reliability estimates; (3) use of sample weights for estimating mean…

Focuses on regression effects that arise from measurement error, and examines methods used to evaluate pre- to post-therapy score changes that attempt to take these regression effects into account. Discusses statistical and practical problems, identifies and analyzes specific examples, and distinguishes objectives of these methods from other…

The purpose of this survey was to identify the statistics texts most frequently used by psychology departments and to see if a typical survey of textbooks used by a partial sample of psychology departments would be accurate in describing the entire sample of departments. Partial returns resulted in a meaningful error. (RM)

Seven features of valid classroom observations procedures, a five-step measurement schemata, and a systematic classroom observation procedure designed to reduce measurement error are presented. The following features are addressed: (1) the inference of the observation is established a priori and confirmed post hoc; (2) craft knowledge and…

Statistics are an essential tool for making proper judgement decisions. It is concerned with probability distribution models, testing of hypotheses, significance tests and other means of determining the correctness of deductions and the most likely outcome of decisions. Measures of central tendency include the mean, median and mode. A second…

The author conducted a cross national analysis of sociological research reported in leading journals at two points in time over a ten year period to determine if sociologists' ability to produce valid social generalizations had improved significantly over the recent past. The official journals of the United States (American Sociological Review),…