This study examined the influence of prompt characteristics on the averages of all scores given to test taker responses on the TOEFL iBT[TM] integrated Read-Listen-Write (RLW) writing tasks for multiple administrations from 2005 to 2009. In the context of TOEFL iBT RLW tasks, the prompt consists of a reading passage and a lecture. To understand…

Providing information to test takers and test score users about the abilities of test takers at different score levels has been a persistent problem in educational and psychological measurement (Carroll, 1993). Scale anchoring (Beaton & Allen, 1992), a technique that describes what students at different points on a score scale know and can do,…

The Non-Equivalent groups with Anchor Test equating (NEAT) design is a widely used equating design in large scale testing that involves two groups that do not have to be of equal ability. One group P gets form X and a group of items A and the other group Q gets form Y and the same group of items A. One of the most commonly used equating methods in…

The factorial structure of the Time Management (TM) scale of the Student 360: Insight Program (S360) was evaluated based on a national sample. A general procedure with a variety of methods was introduced and implemented, including the computation of descriptive statistics, exploratory factor analysis (EFA), and confirmatory factor analysis (CFA).…

This document contains released items used in the ICCS 2009 main survey. Seven clusters of items were used in the study in a rotated booklet design with three clusters per test booklet. Clusters 1 and 7 comprise the released item set. Cluster 1 comprises items that were newly developed for ICCS 2009 and Cluster 7 is drawn from previously secure…

There are at least two procedures to assess item difficulty stability in the Rasch model: robust z procedure and "0.3 Logit Difference" procedure. The robust z procedure is a variation of the z statistic that reduces dependency on outliers. The "0.3 Logit Difference" procedure is based on experiences in Rasch linking for tests…

In automated test assembly (ATA), the methodology of mixed-integer programming is used to select test items from an item bank to meet the specifications for a desired test form and optimize its measurement accuracy. The same methodology can be used to automate the formatting of the set of selected items into the actual test form. Three different…

A practical concern for many existing tests is that subscore test lengths are too short to provide reliable and meaningful measurement. A possible method of improving the subscale reliability and validity would be to make use of collateral information provided by items from other subscales of the same test. To this end, the purpose of this article…

This paper proposes two new item selection methods for cognitive diagnostic computerized adaptive testing: the restrictive progressive method and the restrictive threshold method. They are built upon the posterior weighted Kullback-Leibler (KL) information index but include additional stochastic components either in the item selection index or in…

We propose three latent scales within the framework of nonparametric item response theory for polytomously scored items. Latent scales are models that imply an invariant item ordering, meaning that the order of the items is the same for each measurement value on the latent scale. This ordering property may be important in, for example,…

Based on Maryland's 2017-2018 Kindergarten Readiness Assessment (KRA) results, nearly half of all entering kindergarten children show foundational skills indicating they are fully ready for kindergarten, more than a third are approaching readiness, and 18% have emerging readiness skills. Results for the 2017-2018 school year show a slight increase…

In this study, we investigated gender and minority achievement gaps on 8th-grade science items employing a multilevel item response methodology. Both gaps were wider on physics and earth science items than on biology and chemistry items. Larger gender gaps were found on items with specific topics favoring male students than other items, for…

This study compared classroom interactive behaviors of science pre-service teachers and pre-service teachers of other subjects. Participants included pre-service teachers enrolled in a general methods course for secondary educators and its school-based fieldwork counterpart. Statistical tests found that science pre-service teachers had fewer…

Reviewing the test specifications to improve the quality of language tests may be a routine process in professional testing systems. However, there is a paucity of research about the effect of specifications review on improving the quality of small-scale tests. The purpose of the present study was twofold: how specifications review could help…

This article describes a hybrid standard setting method that combines characteristics of the Angoff (1971) and Bookmark (Mitzel, Lewis, Patz & Green, 2001) methods. The proposed approach utilizes strengths of each method while addressing weaknesses. An ordered item booklet, with items sorted based on item difficulty, is used in combination…