This article reviews the evidence on the extent to which experts' perceptions of item difficulties, captured using comparative judgement, can predict empirical item difficulties. This evidence is drawn from existing published studies on this topic and also from statistical analysis of data held by Cambridge Assessment. Having reviewed the…

This paper reviews the International English Language Testing System's speaking sub-test in the East Asia region with reference to theoretical and practice-based perspectives and identifies future research opportunities to enhance the measures of test qualities found. The test's construct validity was seen to accurately measure the abilities…

This paper explored the authors' concerns about students enrolled in their introductory accounting course. Anecdotal evidence suggested that students struggle with basic arithmetic concepts that underlie basic business transactions even though their math placement and ACT scores are high. A survey of 125 students in a first accounting course was…

In Philip Pullman's dark matter sci-fi trilogy, there is a golden compass that in the hands of the right person is predictively powerful; the same was supposed to be true of the SAT/ACT--the statistically indistinguishable standardized tests for college admissions. They were intended to be reliable mechanisms for identifying future trajectories,…

Predictive validity, generally taken to be the prime validity that occupationally normed interest inventories should demonstrate, is dependent on the capacity of an instrument to differentiate between occupations. A comparison of two methods of differentiation shows that a method using proportions of each occupational group to assign item-scoring…

This paper outlines a technique for differentially weighting options of a multiple choice test in a fashion that maximizes the item predictive validity. The rule can be applied with different number of categories and the "optimal" number of categories can be determined by significance tests and/or through the R2 criterion. Our theoretical analysis…

Many districts and schools across the U. S. have begun to develop and administer assessments to complement state testing systems and provide additional information to monitor curriculum, instruction and schools. In advance of this trend, the Jackson Public Schools (JPS) district has had a district benchmark testing system in place for many years.…

From 13 objective interview items, five with adequate response variability were studied to determine if they would improve the validity of a verbal ability selection test in predicting work performance of 181 psychiatric aide trainees. In a multiple regression analysis, the verbal test correlated .27 with the weighted composite rating score.…

The idea that validity should be considered a property of inferences, rather than of assessments, has developed slowly over the past century. In early writings about the validity of educational assessments, validity was defined as a property of an assessment. The most common definition was that an assessment was valid to the extent that it…

The agreement between the Educational Testing Service (ETS) and the Golden Rule Insurance Company of Illinois is interpreted as setting the general principles on which items must be selected to be included in a licensure test. These principles put a limit to the difficulty level of any item, and they also limit the size of the difference in…

Questions on the Scholastic Aptitude Test (SAT) with the largest score differences between women and men of all racial and ethnic groups were identified. Patterns of difficulty that would explain the SAT's continuing underprediction of female first-year college performance were studied. An item analysis of one form of the June 1986 SAT for 1,112…

The aim of this study was to develop and validate a team-referent attribution scale. Conducted over three studies, Study 1 modified items from McAuley, Duncan, and Russell's (1992) Causal Dimension Scale II by rewording items to reflect team attributions and adding one item per factor. This led to the development of a 16-item scale (Causal…

The composite of Armed Services Vocational Aptitude Battery (ASVAB) subtests used to select applicants for entry-level training in Army clerical schools was evaluated by correlating composite scores with training performance scores. Comparisons were made between the multiple R for this optimal set of predictors and that for the composite of…