There is increasing concern among researchers about collecting data on sex and gender variables, yet many researchers are unsure of how to deal meaningfully with these variables. Drawing on literature that tests the psychometric properties of sex and gender demographic questions, we present considerations for collecting sex and gender demographic…

The measurement error in principal components extracted from a set of fallible measures is discussed and evaluated. It is shown that as long as one or more measures in a given set of observed variables contains error of measurement, so also does any principal component obtained from the set. The error variance in any principal component is shown…

It is common in regression discontinuity analysis to control for high order (third, fourth, or higher) polynomials of the forcing variable. We argue that estimators for causal effects based on such methods can be misleading, and we recommend researchers do not use them, and instead use estimators based on local linear or quadratic polynomials or…

Missing data are a common occurrence in survey-based research studies in education, and the way missing values are handled can significantly affect the results of analyses based on such data. Despite known problems with performance of some missing data handling methods, such as mean imputation, many researchers in education continue to use those…

The purpose of this paper is to review major statistical and psychometric issues impacting the study of psychophysiological reactivity and discuss their implications for applied developmental researchers. We first cover traditional approaches such as the observed difference score (DS) and the observed residual score (RS), including a review of…

This article provides an overview of the available evidence on psychological misconceptions, including key findings, current directions and emerging issues for investigation. We begin by defining misconceptions and then examine their prevalence and persistence, discuss their implications for student learning and highlight potential strategies to…

Latent state-trait (LST) analysis is frequently applied in psychological research to determine the degree to which observed scores reflect stable person-specific effects, effects of situations and/or person-situation interactions, and random measurement error. Most LST applications use multiple repeatedly measured observed variables as indicators…

Classroom context and climate are inherently classroom-level (L2) constructs, but applied researchers sometimes--inappropriately--represent them by student-level (L1) responses in single-level models rather than more appropriate multilevel models. Here we focus on important conceptual issues (distinctions between climate and contextual variables;…

Background: Return on investment (ROI) is one of the most popular evaluation metrics. ROI analysis (when applied correctly) is a powerful tool of evaluating existing information systems and making informed decisions on the acquisitions. However, practical use of the ROI is complicated by a number of uncertainties and controversies. The article…

Reliability captures the influence of error on a measurement and, in the classical setting, is defined as one minus the ratio of the error variance to the total variance. Laenen, Alonso, and Molenberghs ("Psychometrika" 73:443-448, 2007) proposed an axiomatic definition of reliability and introduced the R[subscript T] coefficient, a measure of…

The current main world university rankings broadly group the leading research universities of nations. Australia's Go8 universities are generally within the top 250 ranked universities, with several institutions in the top 50-100 on some measures. This recognition is commendable, however imperfect the individual rankings may be. Use is made of…

This paper considers the model of school effectiveness (SE) currently dominant in research, policy and practice in England (although the concerns it raises are international). It shows, principally through consideration of initial and propagated error, that SE results cannot be relied upon. By considering the residual difference between the…

This paper intends to construct a survey data quality strategy for institutional researchers in higher education in light of total survey error theory. It starts with describing the characteristics of institutional research and identifying the gaps in literature regarding survey data quality issues in institutional research. Then it is followed by…

Schools and school systems throughout the nation are increasingly experimenting with using various instructional technologies to improve productivity and decrease costs, but evidence on both the effectiveness and the costs of education technology is limited. A recent report published by the Thomas B. Fordham Institute sets out to describe "the…

The Center on Education Policy (CEP) report, "Has Progress Been Made in Raising Achievement for English Language Learners?", finds that some states have seen increases in the number of English language learners (ELLs) meeting proficiency standards under No Child Left Behind (NCLB), while others have seen decreases. The report notes some…