Suggestions for software design in computer managed testing (CMT) cover instructions to testees, their physical format, provision of practice items, and time limit information; test item presentation, physical format, discussion of task demands, review capabilities, and rate of presentation; pedagogically helpful utilities; typefonts; vocabulary;…

Compared items from psychologically naive vs. psychologically sophisticated item-writers vs. a standardized test (N=552). Results showed that nonpsychologists with no formal definition of the construct they were to measure were able to write items that were as valid as those elicited from psychologists. (BH)

Two strategies for assessing item bias are discussed: methods comparing item difficulties unconditional on ability and methods comparing probabilities of response conditional on ability. Results suggest that the iterative logit method is an improvement on the noniterative one and is efficient in detecting biased and unbiased items. (Author/DWH)

A paradoxical phenomenon of decreases in reliability as the number of elements averaged over increases is shown to be possible in multifacet reliability procedures (intraclass correlations or generalizability coefficients). Conditions governing this phenomenon are presented along with implications and cautions. (Author)

This document contains the framework and a set of recommendations for the 2005 NAEP mathematics assessment. It includes descriptions of the mathematical content of the test, the types of test questions, and recommendations for administration of the test. In broad terms, this framework attempts to answer the question: What mathematics should be…

Continuation ratio logits are used to model the possibilities of obtaining ordered categories in a polytomously scored item. This model is an alternative to other models for ordered category items such as the graded response model and the generalized partial credit model. The discussion includes a theoretical development of the model, a…

This Digest describes matrix sampling of test items as an approach to achieving broad coverage while minimizing testing time per student. Matrix sampling involves developing a complete set of items judged to cover the curriculum, then dividing the items into subsets and administering one subset to each student. Matrix sampling, by limiting the…

The purpose of this study was to determine if the patterning of the responses of English as a Second Language (ESL) students to a reading comprehension test would change over time due to the restructuring of the subjects ESL reading comprehension competence as they increased their overall ESL proficiency. In this context, restructuring refers to…

This study compared various equating models and procedures for a sample of data from the Medical College Admission Test(MCAT), considering how item response theory (IRT) equating results compare with classical equipercentile results and how the results based on use of various IRT models, observed score versus true score, direct versus linked…

This study represented a first attempt to evaluate the impact of local item dependence (LID) for Item Response Theory (IRT) scoring in computerized adaptive testing (CAT). The most basic CAT design and a simplified design for simulating CAT item pools with varying degrees of LID were applied. A data generation method that allows the LID among…

The writer comments on the issue of high-stakes testing and the pressures on teachers to "teach to the test." Although many view teaching to the test as an all or none issue, in practice it is actually a continuum. At one end, some teachers examine the achievement objectives as described in their curriculum and then design instructional activities…

The Mantel Haenszel (MH; 1959) approach of Holland and Thayer (1988) is a well-established method for assessing differential item functioning (DIF). The formula for the variance of the MH DIF statistic is based on work by Phillips and Holland (1987) and Robins, Breslow, and Greenland (1986). Recent simulation studies showed that the MH variances…

This paper discusses alternatives to R-technique factor analysis that are applicable to counseling and psychotherapy. The traditional R-technique involves correlating columns of a data matrix. O, P, Q, S, and T techniques are discussed with particular emphasis on Q-technique. In Q-technique, people are factored across items or variables with the…

The purpose of the study was to compare reliability estimates for a test composed of stimulus-dependent testlets as derived from item scores, testlet scores, and under the univariate generalizability theory and multivariate generalizability theory designs, as well as to determine the influence of the number of testlets and the number of items per…

The use of grid-in formats, such as those requiring students to solve problems and fill in bubbles, is common on large-scale standardized assessments, but little is known about the use of this format with a more general population of students than high school students taking college entrance examinations, including those attending public schools…