This study examined variations of a nonequivalent groups equating design used with constructed-response (CR) tests to determine which design was most effective in producing equivalent scores across the two tests to be equated. Using data from a large-scale exam, the study investigated the use of anchor CR item rescoring in the context of classical…

In assessment, the ability to construct test items that measure a targeted skill is fundamental to validity and alignment. The ability to do the reverse is also important: determining what skill an existing test item measures. This paper presents a model for classifying test items that builds on procedures developed by others, including Bloom…

The influence of general and verbal ability on each of 72 verbal analogy test items were investigated with new factor analytical techniques. The analogy items together with the Computerized Swedish Enlistment Battery (CAT-SEB) were given randomly to two samples of 18-year-old male conscripts (n = 8566 and n = 5289). Thirty-two of the 72 items had…

Background: To establish credible, defensible and acceptable passing scores for written tests is a challenge for health profession educators. Angoff procedures are often used to establish pass/fail decisions for written and performance tests. In an Angoff procedure judges' expertise and professional skills are assumed to influence their ratings of…

In the course of screening a form of a medical licensing exam for items that function differentially (DIF) between men and women, the authors used the traditional Mantel-Haenszel (MH) statistic for initial screening and a Bayesian method for deeper analysis. For very easy items, the MH statistic unexpectedly often found DIF where there was none.…

The examination of measurement invariance in polytomous items is complicated by the possibility that the magnitude and sign of lack of invariance may vary across the steps underlying the set of polytomous response options, a concept referred to as differential step functioning (DSF). This article describes three classes of nonparametric DSF effect…

A new method for analyzing differential item functioning is proposed to investigate the relative strengths and weaknesses of multiple groups of examinees. Accordingly, the notion of a conditional measure of difference between two groups (Reference and Focal) is generalized to a conditional variance. The objective of this article is to present and…

We describe the results of an experiment conducted to test predictions about student responses to questions about motion based on an explicit model of student thinking in terms of the cuing of a variety of different physical intuitions or conceptual resources. This particular model allows us to account for observed variations in patterns of…

Methods are presented for comparing grades obtained in a situation where students can choose between different subjects. It must be expected that the comparison between the grades is complicated by the interaction between the students' pattern and level of proficiency on one hand, and the choice of the subjects on the other hand. Three methods…

When expectations and stereotypes are activated at retrieval, they spontaneously create distorted and illusory recollections that are consistent with these expectations. Participants studied doctor (physician)-related and lawyer-related statements that were presented by 2 different people. When informed, on a subsequent source memory test, (i.e.,…

Statewide physical fitness testing is gaining popularity in the United States because of increased childhood obesity levels, the relations between physical fitness and academic performance, and the hypothesized relations between adult characteristics and childhood physical activity, physical fitness, and health behaviors. Large-scale physical…

A profile is a vector of scores for one examinee. The mean score in the vector can be interpreted as a measure of overall profile height, the variance can be interpreted as a measure of within person variation, and the ipsatized vector of score deviations about the mean can be said to describe the pattern in the score profile. A within person…

Within the general context of the attention being given to vocational education in China and imminent major reform in the English language vocational education curriculum, this paper describes the construction of a set of calibrated tests of reading skills, grammar and vocabulary knowledge to inform curriculum planners about levels of ability in…

Many exam programs have begun to include innovative item types in their operational assessments. While innovative item types appear to have great promise for expanding measurement, there can also be genuine challenges to their successful implementation. In this paper we present a set of four activities that can be beneficially incorporated into…

Differential item functioning (DIF) analysis is a statistical technique used for ensuring the equity and fairness of educational assessments. This study formulates a new DIF analysis method using the information similarity index (ISI). ISI compares item information functions when data fits the Rasch model. Through simulations and an international…