Educational tests are standardized so that all examinees are tested on the same material, under the same testing conditions, and with the same scoring protocols. This uniformity is designed to provide a level "playing field" for all examinees so that the test is "the same" for everyone. Thus, standardization is designed to…

The goal in personalized assessment is to best fit the needs of each individual test taker, given the assessment purposes. Design-in-Real-Time (DIRTy) assessment reflects the progressive evolution in testing from a single test, to an adaptive test, to an adaptive assessment "system." In this article, we lay the foundation for DIRTy…

Selection decisions have a major impact on our education, occupation, and quality of life, and the role of standardized tests in selection has always been a source of controversy. Here, I consider various definitions of fairness in measurement and selection--those emerging from within educational measurement and statistics, those from philosophy,…

In this digital ITEMS module, Dr. Robert [Bob] Mislevy and Dr. Maria Elena Oliveri introduce and illustrate a sociocognitive perspective on educational measurement, which focuses on a variety of design and implementation considerations for creating fair and valid assessments for learners from diverse populations with diverse sociocultural…

One of the substantive changes in the 2014 Standards for Educational and Psychological Testing was the elevation of fairness in testing as a foundational element of practice in addition to validity and reliability. Previous research indicates that testing practices often do not align with professional standards and guidelines. Therefore, to raise…

Research indicates that the performance-gap between English Language Learners (ELLs) and their non-ELL peers is partly due to ELLs' difficulty in understanding assessment language. Accommodations have been shown to narrow this performance-gap, but many accommodations studies have not used a randomized design and are based on relatively small…

Differences in test scores can be attributed to various causes, including genuine knowledge differences, test-taking abilities, and irrelevant and biased questions. The Golden Rule reform is a safeguard to ensure that standardized tests measure relevant knowledge differences between test takers and not irrelevant, culturally specific factors. (JAZ)

A methodology for developing multiple-choice critical thinking tests is described, which attempts to overcome certain problems of validity and fairness associated with such tests. It is concluded that direct evidence on test validity should be gathered using verbal reports of students' thinking on trial items. (TJH)

Test scores are being used for a variety of inappropriate purposes, often to the detriment of minorities. This article recounts recent evidence on misuse of testing, offers strategies for preventing abuse, and urges professionals to take on active roles in preventing misuse of their wares. (LMO)

The extent to which performance assessments can have unintended and undesirable consequences is not yet clear, but preliminary evidence suggests several issues of bias and fairness. Among these is the sheer difficulty of scoring complex performance assessments, as Vermont's experiences with portfolio assessment illustrate. (SLD)