The importance of stress in the understanding of adolescent health and well-being is widely documented. The measurement of adolescent stress has however been subjected to sufficient methodological and conceptual criticism in recent times to warrant a concerted re-evaluation of the exercise. This study sought information on the nature of adolescent…

This study sheds light on the achievement gap between two culturally diverse populations in Israel by employing a diagnostic model for analysing responses of a representative sample of Jewish and Arab fourth graders on a national mathematics test. The results indicated large significant differences, in favour of the Jewish group, on most…

The purpose of this study was to examine the effect of pretest items on response time in an operational, fixed-length, time-limited computerized adaptive test (CAT). These pretest items are embedded within the CAT, but unlike the operational items, are not tailored to the examinee's ability level. If examinees with higher ability levels need less…

The current study, carried out jointly by Test Development and Statistical Analysis staff at Educational Testing Service investigated the feasibility of the Automated Item Selection (AIS) procedure for the Test of English as a Foreign Language (TOEFL). Item-response theory (IRT)-based statistical specifications were developed. Two TOEFL test forms…

Standard item response theory (IRT) models posit latent variables to account for regularities in students' performance on test items. They can accommodate learning only if the expected changes in performance are smooth, and, in an appropriate metric, uniform over items. Wilson's "Saltus" model extends the ideas of IRT to development that…

As adaptive testing moves toward operational implementation in large scale testing programs, where it is important that adaptive tests be as parallel as possible to existing linear tests, a number of practical issues arise. This paper concerns three such issues. First, optimum item pool size is difficult to determine in advance of pool…

This study evaluated the use of bivariate matching as a solution to the problem of studying differential item functioning (DIF) with formula scored tests. Using Scholastic Aptitude Test verbal data with large samples, both male/female and black/white group comparisons were investigated. Mantel-Haenszel (MH) delta-(D) DIF values and DIF category…

The term differential item functioning (DIF) refers to whether or not the same psychological constructs are measured across different groups. If an item does not measure the same skills or subskills in different populations, it is said to function differentially or to display item bias. A multilevel approach to DIF is proposed. In such a model,…

Several exploratory analyses of the fifths data generated by Test of English as a Foreign Language (TOEFL) item analyses were developed in order to evaluate the effects of options on the discriminability of difficult items and to identify difficult items with low, unreliable biserials that had been rejected by test developers, but for which…

The influence of anchor length on the accuracy of test equating was studied using Tucker's linear method and two Item-Response-Theory (IRT) based methods, focusing on whether equating accuracy improved with more anchor items, whether the anchor effect depended on the equating method used, and the adequacy of the inclusion of the guessing parameter…

The Mantel-Haenszel procedure (N. Mantel and W. Haenszel, 1959) and its extension to constructed response items, the Generalized Mantel Haenszel (A. Agresti, 1990), compare performance of subgroups across different score groups to determine differential item functioning (DIF). At each level of comparison, or score group, the subgroups are…

Analyses of questionnaire data from a program evaluation indicate that the two dichotomous items "Would you recommend this to a friend?" and "Would you choose to do this again?" are not as interchangeable as might be expected from the survey literature. As part of the evaluation of a university program, a survey of graduates…

This study investigated the intrajudge consistency of Angoff-based item performance estimates. The examination used was a certification examination in an emergency medicine specialty. Ten expert panelists rated the same 24 items twice during an operational standard setting study. Results indicate that the panelists were highly consistent, in terms…

A confirmatory approach to assessing test structure using multidimensional item response theory (MIRT) was developed and evaluated. The approach involved adding to the exponent of the MIRT model an item structure matrix that allows the user to specify the ability dimensions measured by an item. Various combinations of item structures were fit to…

Binary or graded disagree-agree responses to attitude items are often collected for the purpose of attitude measurement. Although such data are sometimes analyzed with cumulative measurement models, recent investigations suggest that unfolding models are more appropriate (J. S. Roberts, 1995; W. H. Van Schuur and H. A. L. Kiers, 1994). Advances in…