The designing of tests has been a source of concern for test developers over the past decade. Various kinds of test forms have been applied. Among these are the fixed-form test, the adaptive test, and the testlet. Each of these forms has its own design. In this paper, the construction of test forms is placed within the general framework of optimal…

In studies investigating the power of person-fit statistics it is often assumed that the item parameters that are used to calculate the statistics can be estimated in a sample without aberrant persons. However, in practical test applications calibration samples most likely will contain aberrant persons. In the present study, the influence of the…

Educational reform efforts have led to increased use of alternatives to the traditional binary-scored multiple choice item. Many stimuli employed by these alternative assessments yield complex responses that require complex scoring rules. Some of these new item types can be polytomously-scored. Differential item functioning (DIF) assessment is a…

When items are selected during a computerized adaptive test (CAT) solely with regard to their measurement properties, it is commonly found that certain items are administered to nearly every examinee, and that a small number of the available items will account for a large proportion of the item administrations. This presents a clear security risk…

In adaptive testing, item selection is sequentially optimized during the test. Since the optimization takes place over a pool of items calibrated with estimation error, capitalization on these errors is likely to occur. How serious the consequences of this phenomenon are depends not only on the distribution of the estimation errors in the pool or…

The purpose of many certification or licensure tests is to identify candidates who possess some level of minimum competence to practice their profession. In general, this type of test is referred to as classification testing. When this type of test is administered with a computer, the test is a computerized classification test (CCT). This paper…

The Mantel-Haenszel (MH) procedure has become one of the most popular procedures for detecting differential item functioning (DIF). One of the most troublesome criticisms of this procedure is that while detection rates for uniform DIF are very good, the procedure is not sensitive to non-uniform DIF. In this study, examinee responses were generated…

The purpose of this paper is to demonstrate how graphical analyses can enhance the interpretation and understanding of multidimensional item-response theory (IRT) analyses. Conceptually many of the unidimensional IRT concepts such as item characteristic curves, information, etc., can be extended to multiple dimensions. However, as the…

A definition of essential independence is proposed for sequences of polytomous items. For items which satisfy the assumption that the expected amount of credit awarded increases with examinee ability, a theory of essential unidimensionality is developed that closely parallels that of W. F. Stout (1987, 1990). Essentially unidimensional item…

A rating scale can be expressed as a chain of dichotomous items. The relationship between the dichotomies depends on the manner in which the rating scale is presented to the test taker. Three models for ordered scales are discussed. In the success model, which represents growth, the lowest or easiest category is presented first. If the test taker…

This paper presents a promising new item bias detection technique that derives practical appeal from its simplicity and economy, and provides data relating to the method's consistency across samples. The proposed regression on item characteristics (RIC) method is a straightforward and inexpensive approach to the study of item bias, based on…

This paper describes an approach to item analysis that is based on the estimation of a set of response curves for each item. The response curves show, at a glance, the difficulty and the discriminating power of the item and the popularity of each distractor, at any level of the criterion variable (e.g., total score). The curves are estimated by…

In educational tests, subscores are often generated from a portion of the items in a larger test. Guidelines based on mean-squared error are proposed to indicate whether subscores are worth reporting. Alternatives considered are direct reports of subscores, estimates of subscores based on total score, combined estimates based on subscores and…

Chang and Zhang (2002, 2003) proposed several baseline criteria for assessing the severity of possible test security violations for computerized tests with high-stakes outcomes. However, these criteria were obtained from theoretical derivations that assumed uniformly randomized item selection. The current study investigated potential damage caused…

The problem of obtaining designs that result in the most precise parameter estimates is encountered in at least two situations where item response theory (IRT) models are used. In so-called two-stage testing procedures, certain designs that match difficulty levels of the test items with the ability of the examinees may be located. Such designs…