With scant research to draw upon with respect to the maintenance of vertical scales over time, decisions around the creation and performance of vertical scales over time necessarily suffers due to the lack of information. Undetected item parameter drift (IPD) presents one of the greatest threats to scale maintenance within an item response theory…

In their article "Innovations in Setting Performance Standards for K-12 Test-Based Accountability," Kristen Huff and Barbara S. Plake (2010) lay out three preconditions for continued investment in standard-setting methodology and practice, all focused on the sound development and use of achievement level descriptors (ALDs). Among these…

This paper presents a framework to provide a structured approach for developing score reports for cognitive diagnostic assessments ("CDAs"). Guidelines for reporting and presenting diagnostic scores are based on a review of current educational test score reporting practices and literature from the area of information design. A sample diagnostic…

This report evaluates the construct coverage of the e-rater[R[ scoring engine. The matter of construct coverage depends on whether one defines writing skill, in terms of process or product. Originally, the e-rater engine consisted of a large set of components with a proven ability to predict human holistic scores. By organizing these capabilities…

Large data sets from a state reading assessment for third and fifth graders were analyzed to examine differential item functioning (DIF), differential distractor functioning (DDF), and differential omission frequency (DOF) between students with particular categories of disabilities (speech/language impairments, learning disabilities, and emotional…

The myth equating high-stakes testing with rigor and difficulty is one that can be debunked given the empirical work that has been conducted in this area. To completely debunk this myth in gifted education, the field must centralize efforts. Educators need to consider alternatives to the current system of assessment and the delivery of…

As the recession crimps education budgets, states are beginning to pare the number of standardized tests they give, particularly those that no longer factor into state or federal accountability decisions. At the district level, though, it's a different story. Despite pressure not to cut staffing and programs, many districts are preserving local…

Background: Political devolution occurred in the UK in 1998-99, following many years in which some degree of policy administration had been devolved to the four nations. Since devolution, all four countries of the UK have pursued increasingly divergent education policies. This is true in England in particular, where diversity, choice and…

In this study, we describe a framework for monitoring rater performance over time. We present several statistical indices to identify raters whose standards drift and explain how to use those indices operationally. To illustrate the use of the framework, we analyzed rating data from the 2002 Advanced Placement English Literature and Composition…

In this study, the effectiveness of detection of differential item functioning (DIF) and testlet DIF using SIBTEST and Poly-SIBTEST were examined in tests composed of testlets. An example using data from a reading comprehension test showed that results from SIBTEST and Poly-SIBTEST were not completely consistent in the detection of DIF and testlet…

If a person were to ask an educator to identify the two most important attributes of an education test, the response most certainly would be "validity and reliability." These two tightly wedded concepts have become icons in the field of education assessment. As far as validity is concerned, the term doesn't refer to the accuracy of a test. Rather,…

As an extension of the effort devoted to updating the questionnaires for TIMSS and PIRLS 2011, this dissertation explored a new reporting strategy for contextual questionnaire data. The study investigated the feasibility of constructing "global indicators" from a large number of diverse background variables, which could provide policy…

"Harvard Education Letter" is published bimonthly at the Harvard Graduate School of Education. This issue of "Harvard Education Letter" contains the following articles: (1) Hybrid Schools for the iGeneration: New Schools Combine "Bricks" and "Clicks" (Brigid Schulte); (2) Dual Language Programs on the Rise: "Enrichment" Model Puts Content Learning…

Achievement goal theory is considered to be a well-researched field. However, this research has been primarily through surveys, and not enough attention has been paid to the cognitive aspects of how children perceive goals. Additionally, the mastery-avoidance construct is relatively new to the achievement goal literature, with little research to…

This dissertation illuminates relationships between micro-level practices of schools and macro-level structures of society through the socio-historical lens of New York State Regents mathematics examinations, which were administered to public school students throughout the State of New York between 1866 and 2009, inclusive. Fundamental research…