Fit analysis is widely performed in item response theory (IRT) based test development to assess the fit of individual items to the IRT model being used. The paper explores a step fit analysis procedure that is an extension of IRT-based item fit diagnostics applied to the response categories present in popular performance-based tasks. The step fit…

The problem of response set is important for questionnaire designers and interpreters, but the public is affected as well if policy is determined on the basis of unsupported conclusions. This study focused on one of the most researched response sets, extreme responding (ER), or extreme checking styles, and its relationship to one dimension of…

An item-selection algorithm to neutralize the differential effects of time limits on scores on computerized adaptive tests is proposed. The method is based on a statistical model for the response-time distributions of the examinees on items in the pool that is updated each time a new item has been administered. Predictions from the model are used…

This study investigated and compared the properties of five methods of item exposure control within the purview of estimating examinees' abilities in a computerized adaptive testing (CAT) context. Each of the exposure control algorithms was incorporated into the item selection procedure and the adaptive testing progressed based on the CAT design…

Items of Altemeyer's 1986 version of the "Right-Wing Authoritarianism Scale" (RWA Scale) were analyzed as philosophical propositions in an effort to establish each item's suggestive connotation and denotation. The guiding principle of the analysis was the way in which the statements reflected authoritarianism's defining characteristics…

This paper presents a study on the generation of mathematics test items using algorithmic methods. The history of this approach is briefly reviewed and is followed by a survey of the research to date on the statistical parallelism of algorithmically generated mathematics items. Results are presented for 8 parallel test forms generated using 16…

The purpose of this study was to investigate whether systematic, non-zero differences between pairs of item bank b-values have occurred in the recent history of two licensure examinations. Licensing examinations were studied for two related health care professions (Program 1 and Program 2). A series of analysis of covariance models was fit to the…

A project is reported that developed a test for students in foreign language in the elementary school (FLES) programs. Relevant tests in Spanish and English as a Second Language (ESL) were reviewed in order to develop a listening and reading test that could determine achievement in a typical FLES curriculum. Pilot testing was conducted with 121…

A new index for assessing the dimensionality underlying a set of test items was investigated. The incremental fit index (IFI) is based on the sum of squares of the residual covariances. Purposes of the study were to: (1) examine the distribution of the IFI in the null situation, with truly unidimensional data; (2) examine the rejection rate of the…

Three methods of factor analyzing dichotomously scored item performance data were compared using two raw score data sets of 20-item tests, one reflecting normally distributed latent traits and the other reflecting uniformly distributed latent traits. This comparison was accomplished by using phi and tetrachoric correlations among dichotomous data…

Multiple group factor analysis is described and illustrated through a simulation involving 5,000 examinees. The estimation process of the group factors were implemented using the TESTFACT program of Wilson and others (1987). Group factor analysis is described as a special case of confirmatory factor analysis. Group factors can be computed based on…

Many researchers have suggested that the main cause of item bias is the misspecification of the latent ability space. That is, items that measure multiple abilities are scored as though they are measuring a single ability. If two different groups of examinees have different underlying multidimensional ability distributions and the test items are…

Item analysis procedures appropriate for domain-referenced classroom testing are described. A conceptual framework within which item statistics can be considered and promising statistics in light of this framework are presented. The sampling fluctuations of the more promising item statistics for sample sizes comparable to the typical classroom…

Educational item banking presents observers with a considerable paradox. The development of test items from scratch is viewed as wasteful, a luxury in times of declining resources. On the other hand, item banking has failed to become a mature technology despite large amounts of money and the efforts of talented professionals. The question of which…

Item parameter estimates vary for a variety of reasons, including estimation error, characteristics of the examinee samples, and context effects (e.g., item location effects, section location effects, etc.). Although we expect variation based on theory, there is reason to believe that observed variation in item parameter estimates exceeds what…