Identifies and evaluates alternative methods for weighting tests. Presents formulas for composite reliability and validity as a function of component weights and suggests a rational process that identifies and considers trade-offs in determining weights. Discusses drawbacks to implicit weighting and explicit weighting and the difficulty of…

Presents six computational methods based on mixed-integer programming for assembling tests from a bank with an item-set structure and evaluated these methods using mathematical programming feasibiity and expected solution times. Illustrates these methods with two data sets from the Law School Admission Test. Discusses the best approximations to…

Investigated the usefulness of multidimensional scaling (MDS) for assessing the dimensionality of dichotomous test data. Focused on two MDS proximity measures, one based on the PC statistic (T. Chen and M. Davidson, 1996) and other, on interitem Euclidean distances. Simulation results show that both MDS procedures correctly identify…

The purpose of this note is twofold: (a) to present the formula for the item information function (IIF) in any direction for the Multidimensional 3-Parameter Logistic (M3-PL) model and (b) to give the equation for the location of maximum item information (theta[sub max]) in the direction of the item discrimination vector. Several corollaries are…

In an Angoff standard-setting procedure, judges estimate the probability that a hypothetical randomly selected minimally competent candidate will answer correctly each item constituting the test. In many cases, these item performance estimates are made twice, with information shared with the judges between estimates. Especially for long tests,…

Recent literature on parcel indicators in measurement models used in covariance structural modelling has mainly been concerned with statistical properties of parameter estimates. Less attention has been paid to measurement properties for inferring the assumed latent construct. The present study illustrates a two-facet measurement model that…

The Possible Word Constraint limits the number of lexical candidates considered in speech recognition by stipulating that input should be parsed into a string of lexically viable chunks. For instance, an isolated single consonant is not a feasible word candidate. Any segmentation containing such a chunk is disfavored. Five experiments using the…

In this article, a distinction is made between absolute and relative measurement. Absolute measurement refers to the measurement of traits on a group-invariant scale, and relative measurement refers to the within-group measurement of traits, where the scale of measurement is expressed in terms of the within-group position on a trait. Relative…

The purpose of this study was to develop a test of academic readiness for first grade instruction in Thailand. Test of Academic Readiness (TAR) consists of six domains: verbal, visual, memory, math, logical, and general knowledge. Two pilot studies were carried out and a main study tested items in those domains. Rasch model was used to assess the…

The purposes of this study were to investigate preservice mathematics teachers' orientations, content knowledge, and pedagogical content knowledge of probability; the relationships among these three aspects; and the usefulness of tasks with respect to examining these aspects of knowledge. The design of the study was a multi-case study of five…

The ultimate goal of physics education research (PER) is to develop a theoretical framework to understand and improve the learning process. In this journey of discovery, assessment serves as our headlamp and alpenstock. It sometimes detects signals in student mental structures, and sometimes presents the difference between expert understanding and…

As the ongoing national indicator of what American students know and can do, the National Assessment of Educational Progress (NAEP) in Reading regularly collects achievement information on representative samples of students in grades 4, 8, and 12. The information that NAEP provides about student achievement helps the public, educators, and…

This study examined the percentage of task-sampling variability in performance assessment via a meta-analysis. In total, 50 studies containing 130 independent data sets were analyzed. Overall results indicate that the percentage of variance for (a) differential difficulty of task was roughly 12% and (b) examinee's differential performance of the…

With recent statistical advances in cognitive diagnostic assessment (CDA), the CDA approach has been increasingly applied to non-diagnostic tests partly to meet accountability demands for student achievement. The study aimed to evaluate critically the validity of the CDA application to an existing non-diagnostic L2 reading comprehension test and…

This study investigated the relationship between students' actual performance (accuracy) and their subjective judgments of accuracy (confidence) on selected English language proficiency tests. The unidimensional and multidimensional IRT Rasch approaches were used to model the discrepancy between confidence and accuracy at the item and test level…