Despite the rising popularity of big data, there is speculation that evaluators have been slow adopters of these new statistical approaches. Several possible reasons have been offered for why this is the case: ethical concerns, institutional capacity, and evaluator capacity and values. In this method note, we address one of these barriers and aim…

Premised on the idea that evaluators should be familiar with a range of approaches to program modifications, I review several existing approaches and then describe another, less well-recognized option. In this newer option, evaluators work with others to identify potentially needed adaptations for select program aspects "in advance." In…

In the United States, but not only here, the movement to evaluate teachers based on student test scores has received powerful political and parental support. The logic is simple. From one testing occasion to another students should show growth in their knowledge and skill. Similar types of students should show similar patterns of growth. Those…

This commentary examines the role of the unit from the perspective of the definition of measurement in physics as the ratio of two magnitudes, one of which is defined as the unit; it is an important and timely contribution to measurement in the social sciences. There are many different points that could be commented upon, but the author will…

Compared to traditional test theory, where person measures are typically referenced to the distribution of a population, item response theory allows for a much more meaningful interpretation of measures as they can be directly compared to item locations. However, Stephen Humphry shows that the crucial role of the unit of measurement has been…

Humphry (this issue) deserves credit for drawing attention to the long-neglected fact that differences in item discrimination parameters are often due to empirical factors and not the product of random error components. In doing so, Humphry offers a psychometrically elegant, coherent, and practically important new model that is more flexible while…

Humphry (this issue) has written a thought-provoking piece on the interpretation of item discrimination parameters as scale units in item response theory. One of the key features of his work is the description of an item response theory (IRT) model that he calls the logistic measurement function that combines aspects of two traditions in IRT that…

Behavioral scientists have struggled with units of measurement for as long as they have struggled with measurement itself. Psychology's sole attempt at an explicit unit of measurement--the Lexile Framework for Reading (Stenner, Burdick, Sanford, & Burdick, 2006)--has been and continues to be ignored by the psychometric "cognoscenti."…

The purpose of this article is to examine the role of the unit in physics in order to clarify the role of the unit in psychometrics. Based on this examination, metrological conventions are used to formulate the relationship between discrimination and the unit of a scale in item response theory. Seminal literature in two lines of item response…

This article presents Stephen Humphry's response to the commentaries for his article "The Role of the Unit in Physics and Psychometrics." The commentaries covered a range of important considerations and implications. Given that the author fully agrees with the majority of the content, attention will be confined mainly to points that call…

The purpose of the recently proposed "p[subscript rep]" statistic is to estimate the probability of concurrence, that is, the probability that a replicate experiment yields an effect of the same sign (Killeen, 2005a). The influential journal "Psychological Science" endorses "p[subscript rep]" and recommends its use…

As the largest international study ever taken in history, the Trend in Mathematics and Science Study (TIMSS) has been held as a benchmark to measure U.S. student performance in the global context. In-depth analyses of the TIMSS project are conducted in this study to examine key issues of the comparative investigation: (1) item flaws in mathematics…

The assessment procedures within Response to Intervention (RTI) models have begun to supplant the use of traditional, discrepancy-based frameworks for identifying students with specific learning disabilities (SLD). Many RTI proponents applaud this shift because of perceived shortcomings in utilizing discrepancy as an indicator of SLD. However,…

This paper addresses a key methodological challenge in the modeling of individual poverty dynamics--the influence of measurement error. Taking the US and Britain as case studies and building on recent research that uses latent Markov models to reduce bias, we examine how measurement error can affect a range of important poverty estimates. Our data…

This special series focused on treatment integrity in the child mental health and education field is timely. The articles do a laudable job of reviewing (a) the current status of treatment integrity research and measurement, (b) existing conceptual models of treatment integrity, and (c) the limitations of prior research. Overall, this thoughtful…