In standardized educational testing, test items are reused in multiple test administrations. To ensure the validity of test scores, the psychometric properties of items should remain unchanged over time. In this article, we consider the sequential monitoring of test items, in particular, the detection of abrupt changes to their psychometric…

David Thissen, a professor in the Department of Psychology and Neuroscience, Quantitative Program at the University of North Carolina, has consulted and served on technical advisory committees for assessment programs that use item response theory (IRT) over the past couple decades. He has come to the conclusion that there are usually two purposes…

Since the passage of No Child Left Behind (NCLB), the education system has seen immense shifts in its approach to schooling. Previously, students were taught using an extant curriculum with the instructional methods of the teachers at the school; there was little systematic modification to curriculum and methods; and the variable underlying…

While measurement precision is relatively easy to establish for single tests and assessments, it is much more difficult to determine for decision making with multiple tests on different subjects. This latter is the situation in the system of final examinations for secondary education in the Netherlands and is used as an example in this paper. This…

The Academic Performance Index (API) is the centerpiece of California's state assessment and accountability system. With the recent passage of SB1458 and the pending reauthorization of both state and federal accountability legislation, there is now an unprecedented opportunity to improve the API for next generation accountability in California. In…

Perhaps no topic has as thoroughly vexed officials who oversee the nation's leading test of academic progress as the wide variation among states and cities in the proportion of students with disabilities and limited English proficiency whom they exclude from taking the exam or provide with special accommodations for it. The board that sets policy…

The importance of using the standard error of measurement (SEm) in determining reliability in test scores is emphasized. The SEm is compared to the hypothetical true score for standardized tests, and procedures for calculation of the SEm are explained. (JDD)

This paper discusses the fact that reliability is about scores and not tests and how reliability limits effect sizes. The paper also explores the classical reliability coefficients of stability, equivalence, and internal consistency. Stability is concerned with how stable test scores will be over time, while equivalence addresses the relationship…

Research says the school-improvement mechanisms favored by policymakers-more certification tests (like the Massachusetts Educator Certification Test that 59 percent of candidates failed in 1998), higher cut scores, and severe penalties for institutions not meeting pass rates-are unlikely to deliver increased accountability and better teachers.…

The purposes of this paper are five-fold to discuss: (1) when item response theory (IRT) equating methods should provide better results than traditional methods; (2) which IRT model, the three-parameter logistic or the one-parameter logistic (Rasch), is the most reasonable to use; (3) what unique contributions IRT methods can offer the equating…

Considers the situation in which content or administrative considerations limit the way in which a test can be partitioned to estimate the internal consistency reliability of the total test score. Demonstrates that a single-valued estimate of the total score reliability is possible only if an assumption is made about the comparative size of the…

Reviews a dozen basic questions about passing scores on state-mandated competency testing programs for students, teachers, administrators, and other education personnel. Condemns the "cardiac" approach (or traditonal 80 percent standard) in favor of procedures that systematically incorporate judgment with a variety of performance data.…

The increase in large-scale standardized testing in the nation's schools is one major result of demands for accountability. Such testing also impacts many aspects of school and family life and culture. For some, certain unintended impacts may be like side effects of medicines that need to be tolerated; for others they raise larger questions about…

This paper looks at some of the problems arising from the procedures used by the Queensland, Australia, Board of Secondary School Studies to compile percentile scores known as Tertiary Entrance Scores. Although efforts to equate across schools and across subjects are being challenged, at present there is no alternative. (LMO)

Develops a conceptual framework that addresses score comparability for performance assessments, adaptive tests, paper-and-pencil tests, and alternate item pools for computerized tests. Outlines testing situation aspects that might threaten score comparability and describes procedures for evaluating the degree of score comparability. Suggests ways…