Background: Digital-first assessments leverage the affordances of technology in all elements of the assessment process--from design and development to score reporting and evaluation to create test taker-centric assessments. Objectives: The goal of this paper is to describe the engineering, machine learning, and psychometric processes and…

As validity pertains to test use rather than the test itself, using a test for unintended purposes requires a new validation program using additional evidence from relevant sources. This small-scale study contributes to the validation of the use of originally academic language tests--the International English Language Testing System and the Test…

This book discusses how to obtain test scores and, in particular, how to obtain test scores from tests that consist of a combination of multiple choice and open-ended questions. The strength of the book is that scoring solutions are presented for a diversity of real world scoring problems. (SLD)

While most school districts believe they understand grade equivalent scores, teachers, parents, and measurement specialists frequently misinterpret this apparently simple statistical expression. Echternacht's article describes the construction, application, and interpretation of grade equivalent scores from the test publisher's perspective.…

In some testing programs an early item analysis is performed before final scoring in order to validate the intended keys. As a result, some items which are flawed and do not discriminate well may be keyed so as to give credit to examinees no matter which answer was chosen. This is referred to as allkeying. This research examined how varying the…

Normal Curve Equivalents (NCEs), a new score system for standardized tests, are used by school districts in reporting results to federal funding agencies. The author uses a quiz format to answer questions on the use of NCE scores. (EGS)

While standard setting procedures are typically discussed in terms of deriving a reasonable cutting score for a given form of a test, the situation may be structured such that the standard has been mandated without regard to the test form itself. This situation may result either through legislative or policy actions and may be a fait accompli by…

A common problem in practical educational research is that of perfect scores which result when latent trait models are used. A simple procedure for managing the perfect and zero response problem encountered in converting test scores into measures is presented. It allows the test user to chose among two or three reasonable finite representations of…

During early attempts to interpret factors represented in scores on the Gumpgookies test, an instrument designed to tap motivation to achieve in young children, the factors identified by ordinary factor-analytic techniques were found to be confounded by the subjects' response sets. This paper proposes a method for defining objectively irrelevant…

This study explores the use of the modified caution index (MCI) for identifying judges whose patterns of recommendations suggest that their judgments might be based on incomplete information, flawed reasoning, or inattention to their standard-setting tasks. It also examines the effect on test standards and passing rates when the test standards of…

When individuals are given tests that are too hard or too easy, the resulting scores are likely to be poor estimates of their performance. To get valid and accurate test scores that provide meaningful results, one should use functional-level testing (FLT). FLT is the practice of administering to an individual a version of a test with a difficulty…

A solution is offered to problems associated with the inequality in the manipulability of probabilities of classification errors of masters versus nonmasters, based on competency test results. Eschewing the typical arbitrary establishment of observed-score standards below 100 percent, the solution incorporates a self-correction of wrong answers.…

Three pilot studies--effects of writing prompt modality on writing performance; effect of topic familiarity; and effect of topic, sample and rater group membership on the stability of scoring criteria application--are used to identify the variability in student writing performance. A writing competency test aims to assess writing-specific skills…

Students in an introductory college course (n=275) responded to equivalent 20-item halves of a test under number-right and formula-scoring instructions. Formula scores of those who omitted items overaged about one point lower than their comparable (formula adjusted) scores on the test half administered under number-right instructions. In contrast,…

Procedures for conducting functional level testing have been available for use by practitioners for some time. However, the Title I Evaluation and Reporting System (TIERS), developed in response to the educational amendments of 1974 to the Elementary and Secondary Education Act (ESEA), has provided the impetus for widespread adoption of this…