In selected-response assessments such as attitude surveys with Likert-type rating scales, examinees often select from rating scale categories to reflect their locations on a construct. Researchers have observed that some examinees exhibit "response styles," which are systematic patterns of responses in which examinees are more likely to…

What models of governance are effective in complex education systems? In all systems an increasing number of stakeholders are involved in designing, delivering, and monitoring education. Like our societies, education systems are increasingly diverse regarding students, teachers, and communities, as well as the values and identities we expect…

This paper reports on a study on the relationship between cognitive task complexity and linguistic performance in L2 writing. In the study, two models proposed to explain the influence of cognitive task complexity on linguistic performance in L2 are tested and compared: Skehan and Foster's Limited Attentional Capacity Model (Skehan, 1998; Skehan…

A model of test item dependency is presented and used to illustrate the effect that violations of local independence have on the behavior of item characteristic curves. The dependency model is flexible enough to simulate the interaction of a number of factors including item difficulty and item discrimination, varying degrees of item dependence,…

Concern has been expressed over the item response theory (IRT) assumption that a person's ability can be estimated in a unidimensional latent space. To examine whether or not the response to an item requires only a single latent ability, unidimensional ability estimates were compared for data generated from the multidimensional item response…

An alternative definition has been developed of the delta scale of item difficulty used at Educational Testing Service. The traditional delta scale uses an inverse normal transformation based on normal ogive models developed years ago. However, no use is made of this fact in typical uses of item deltas. It is simply one way to make the probability…

The purpose of this paper is to present a generalization of the concept of item difficulty to test items that measure more than one dimension. Three common definitions of item difficulty were considered: the proportion of correct responses for a group of individuals; the probability of a correct response to an item for a specific person; and the…

This study investigates the influence of changes in the wording of simple addition and subtraction problems without affecting their semantic structure on the level of difficulty of those problems for first and second graders and on the nature of their errors. The objective is to contribute to a better understanding of the process of constructing a…

In most item response theory models a particular mathematical form is assumed for all item characteristic curves, e.g., a logistic function. It could be desirable, however, to estimate the shape of the item characteristic curves without prior restrictive assumptions about its mathematical form. We have developed a practical method of estimating…

The purpose of this study was to examine the effect that large within-examinee item difficulty variability had on estimates of the proportion of consistent classification of examinees into mastery categories over two test administrations. The classification consistency estimate was based on a single test administration from an estimation procedure…

Factor analysis is the traditional method for studying the dimensionality of test data. However, under common conditions, the factor analysis of tetrachoric correlations does not recover the underlying structure of dichotomous data. The purpose of this paper is to demonstrate that the factor analyses of tetrachoric correlations is unlikely to…

The effects of a multidimensional latent trait space on estimation of item and person parameters by the computer program LOGIST are examined. Several item pools were simulated that ranged from truly unidimensional to an inconsequential general latent trait. Item pools with intermediate levels of prepotency of the general latent trait were also…

Latent trait models are presented that can be used for test design in the context of a theory about the variables that underlie task performance. Examples of methods for decomposing and testing hypotheses about the theoretical variables in task performance are given. The methods can be used to determine the processing components that are involved…

In order to determine the effectiveness of multidimensional scaling (MDS) in recovering the dimensionality of a set of dichotomously-scored items, data were simulated in one, two, and three dimensions for a variety of correlations with the underlying latent trait. Similarity matrices were constructed from these data using three margin-sensitive…

This paper deals with test fairness regarding a test consisting of two parts: (1) a "common" section, taken by all students; and (2) a "variable" section, in which some students may answer a different set of questions from other students. For example, a test taken by several thousand students each year contains a common multiple-choice portion and…