The educational testing landscape is changing in many significant ways as evidence-based, principled assessment design (PAD) approaches are formally adopted. This article discusses the challenges and presents some score scale- and task-focused strategies for developing useful performance-level descriptors (PLDs) under a PAD approach. Details of…

Assessment engineering is a new way to design and implement scalable, sustainable and ideally lower-cost solutions to the complexities of designing and developing tests. It represents a merger of sorts between cognitive task modeling and engineering design principles--a merger that requires some new thinking about the nature of score scales, item…

Over the past four decades, there has been incremental growth in computer-based testing (CBT) as a viable alternative to paper-and-pencil testing. However, the transition to CBT is neither easy nor inexpensive. As Drasgow, Luecht, and Bennett (2006) noted, many design engineering, test development, operations/logistics, and psychometric changes…

Adaptive multistage testlet (MST) designs appear to be gaining popularity for many large-scale computer-based testing programs. These adaptive MST designs use a modularized configuration of preconstructed testlets and embedded score-routing schemes to prepackage different forms of an adaptive test. The conditional information targeting (CIT)…

Computerized testing has created new challenges for the production and administration of test forms. This paper describes a multi-stage, testlet-based framework for test design, assembly, and administration called computer-adaptive sequential testing (CAST). CAST is a structured testing approach that is amenable to both adaptive and mastery…

Describes an integrated approach to test development and administration called computer-adaptive sequential testing (CAST). CAST incorporates adaptive testing methods with automated test assembly. Describes the CAST framework and demonstrates several applications using a medical-licensure example. (SLD)

This paper presents a multistage adaptive testing test development paradigm that promises to handle content balancing and other test development needs, psychometric reliability concerns, and item exposure. The bundled multistage adaptive testing (BMAT) framework is a modification of the computer-adaptive sequential testing framework introduced by…

A set of linear conditions on the item response functions is derived that guarantees identical observed-score distributions on two test forms. The conditions can be added as constraints to a linear programming model for test assembly that assembles a new test form to have an observed-score distribution optimally equated to the distribution of the…

Derives a set of linear conditions of item-response functions that guarantees identical observed-score distributions on two test forms. The conditions can be added as constraints to a linear programming model for test assembly. An example illustrates the use of the model for an item pool from the Law School Admissions Test (LSAT). (SLD)

Comments on the application of a proposed automated test assembly (ATA) to the problem of reducing potential performance differential among population subgroups and points out some pitfalls. Presents a rejoinder by M. Stocking and others. (SLD)

This article contends that the necessary links between constructs and test scores/decisions in language assessment must be established through principled design procedures that align three models: (1) a theoretical construct model; (2) a test development model; and (3) a psychometric scoring model. The theoretical construct model articulates the…

An optimization model is presented that allows test assemblers to control the shape of the observed-score distribution on a test for a population with a known ability distribution. An obvious application is for item response theory-based test assembly in programs where observed scores are reported and operational test forms are required to produce…

The example of a medical licensure test is used to demonstrate situations in which complex, integrated content must be balanced at the total test level for validity reasons, but items assigned to reportable subscore categories may be used under a multidimensional item response theory adaptive paradigm to improve subscore reliability. (SLD)

This paper discusses six challenges relevant to Web-based testing. Some of these challenges are not specific to Web-based testing, but generalize to all computer-based testing. The challenges are: (1) security and using test centers for Web-based testing; (2) measuring complex skills and problem-solving tasks on the Web; (3) integrating modern…

Derivations of several item selection algorithms for use in fitting test items to target information functions (IFs) are described. These algorithms, which use an average growth approximation of target IFs, were tested by generating six test forms and were found to provide reliable fit. (SLD)