In preparation for online administration of the ACT® test, ACT conducted studies to examine the comparability of scores between online and paper administrations, including a timing study in fall 2013, a mode comparability study in spring 2014, and a second mode comparability study in spring 2015. This report presents major findings from these…

Does student-teacher match quality exist? Prior work has documented large disparities in teachers' impacts across student types but has not distinguished between sorting and causal effects as the drivers of these disparities. I propose a disparate value-added model and derive a novel measure of teacher quality--revealed comparative advantage--that…

The aim of the research reported here was to get some idea of the accuracy of grade boundaries (cut-scores) obtained by applying the 'similar items method' described in Bramley & Wilson (2016). In this method experts identify items on the current version of a test that are sufficiently similar to items on previous versions for them to be…

Each year the National Reference Test (NRT) reports the proportions of students achieving at or above three key GCSE grades in English and mathematics. The NRT is therefore uniquely placed as a Key Stage 4 assessment which compares performance directly at ability level, to contribute to the evidence on the impact of disruption to education due to…

Because of concerns about test security, different test forms are typically used across different testing occasions. As a result, equating is necessary in order to get scores from the different test forms that can be used interchangeably. In order to assure the quality of equating, multiple equating methods are often examined. Various equity…

One major issue with large-scale assessments is that the respondents might give no responses to many items, resulting in less accurate estimations of both assessed abilities and item parameters. This report studies how the types of items affect the item-level nonresponse rates and how different methods of treating item-level nonresponses have an…

This paper investigates four methods of linear equating under the common item nonequivalent groups design. Three of the methods are well known: Tucker, Angoff-Levine, and Congeneric-Levine. A fourth method is presented as a variant of the Congeneric-Levine method. Using simulation data generated from the three-parameter logistic IRT model we…

The first operational administration of the Partnership for Assessment of Readiness for College and Careers (PARCC) took place during the 2014-2015 school year. In addition to the traditional paper-and-pencil format, the assessments were available for administration on a variety of electronic devices, including desktop computers, laptop computers,…

Student responses to the West Virginia Educational Standards Test (WESTEST) 2 Online Writing Assessment are scored by a computer-scoring engine. The scoring method is not widely understood among educators, and there exists a misperception that it is not comparable to hand scoring. To address these issues, the West Virginia Department of Education…

The Student Outcomes Survey is an annual national survey of vocational education and training (VET) students. Since 1995, participants have been asked to rate their satisfaction with different aspects of their training, grouped under three main themes: teaching, assessment, and generic skills and learning experiences. While the composition of the…

The most critical feature of a common-item nonequivalent groups equating design is that the average score difference between the new and old groups can be accurately decomposed into a group ability difference and a form difficulty difference. Two widely used observed-score linear equating methods, the Tucker and the Levine observed-score methods,…

Math experts developed Formative Assessment Lessons that teachers can incorporate throughout the year's curriculum. Both tools target the "instructional core." A study by Research for Action (RFA) examining the first year of piloting the Literacy Design Collaborative (LDC) and Math Design Collaborative (MDC) tools (2010-11) found…

In the reporting of individual student results from standardized tests in educational assessments, the percentile rank of the individual student is a major numerical indicator. This paper develops a formulation and presents calculations to examine the accuracy of the individual percentile rank score. Here, accuracy follows the common-sense…

In the reporting of individual student results from standardized tests in educational assessments, the percentile rank of the individual student is a major numerical indicator. For example, in the 1998 and 1999 California Standardized Testing and Reporting (STAR) program using the Stanford Achievement Test Series, Ninth Edition, Form T (Stanford…

Four equating methods were compared using four equating criteria: first-order equity (FOE), second-order equity (SOE), conditional mean squared error (CMSE) difference, and the equipercentile equating property. The four methods were: (1) three parameter logistic (3PL) model true score equating; (2) 3PL observed score equating; (3) beta 4 true…