We explored three approaches to resolving or re-scoring constructed-response items in mixed-format assessments: rater agreement, person fit, and targeted double scoring (TDS). We used a simulation study to consider how the three approaches impact the psychometric properties of student achievement estimates, with an emphasis on person fit. We found…

Pearson product-moment correlation coefficient between item g and test score X, known as item-test or item-total correlation ("Rit"), and item-rest correlation ("Rir") are two of the most used classical estimators for item discrimination power (IDP). Both "Rit" and "Rir" underestimate IDP caused by the…

There is much discussion about and many policies to address achievement gaps in education among groups of students. The focus here is on a different gap and it is argued that it also should be of concern. Speed gaps are differences in how quickly different groups of students answer the questions on academic assessments. To investigate some speed…

The present study explored the dimensionality of cognitive structure from two approaches. The first approach used a famous relation between Visual Spatial Working Memory (VSWM) and calculation to demonstrate the multidimensional item response analyses when true dimensions are unknown. The second approach explored the detectability of dimensions by…

Student testing and knowledge assessment is a significant aspect of the learning process. In a number of cases, it is expedient not to present the exact same test to all learners all the time (Pritchett, 1999). This may be desired so that cheating in the exam is made harder to carry out or so that the learners can take several practice tests on…

Educational and psychological constructs are normally measured by multifaceted dimensions. The measured construct is defined and measured by a set of related subdomains. A bifactor model can accurately describe such data with both the measured construct and the related subdomains. However, a limitation of the bifactor model is the orthogonality…

This paper provides a history of ETS's role in developing assessment instruments and psychometric procedures for measuring change in large-scale national assessments funded by the Longitudinal Studies branch of the National Center for Education Statistics. It documents the innovations developed during more than 30 years of working with…

The purpose of the current study was to examine the relationship between Advanced Placement (AP) exam participation and enrollment in a 4-year postsecondary institution. A positive relationship was expected given that the primary purpose of offering AP courses is to allow students to engage in college-level academic work while in high school, and…

Developmental score scales represent the performance of students along a continuum, where as students learn more they move higher along that continuum. Unidimensional item response theory (UIRT) vertical scaling has become a commonly used method to create developmental score scales. Research has shown that UIRT vertical scaling methods can be…

By applying multilevel models with random effects, the authors reviewed and synthesized findings from 30 studies that were published in the last 20 years exploring the relationship between the Educational Testing Service Major Field Test for a Bachelor's Degree in Business (MFTB) and related factors. The results suggest that MFTB scores correlated…

Young learners with severe visual impairments are restricted in many ways, and psychologists and special needs teachers require information about the nature and extent of the possible educationally handicapping effects. This article, written by Michael Tobin, Emeritus Professor of Special Education within the School of Education at the University…

This paper re-considers the widespread use of value-added approaches to estimate school "effects", and shows the results to be very unstable over time. The paper uses as an example the contextualised value-added scores of all secondary schools in England. The study asks how many schools with at least 99% of their pupils included in the…

The vertical scales of large-scale achievement tests created by using item response theory (IRT) models are mostly based on cluster (or correlated) educational data in which students usually are clustered in certain groups or settings (classrooms or schools). While such application directly violated assumption of independent sample of person in…

Many states have adopted the Common Core State Standards (CCSS), which emphasize "deeper learning" skills, such as mastery of core academic content, critical thinking, problem-solving, and collaboration. But can the new tests being developed to align with the CCSS be used to promote deeper learning and high-quality classroom instruction?…

There are good reasons for re-thinking teacher evaluation. Evaluation systems in most school districts appear to be far from rigorous. A recent study showed that more than 99 percent of teachers in a number of districts were rated "satisfactory," which does not comport with empirical evidence that teachers differ substantially from each…