The purpose of this study was to develop and gather validity evidence for silent reading fluency passages. A number of passages were written following a traditional story grammar structure (character, setting, events) and placed on a computer for students to read silently. We describe in detail, the manner in which content-related evidence was…

The purpose of this study was to document the instrument development of maze measures for grades 3-8. Each maze passage contained twelve omitted words that students filled in by choosing the best-fit word from among the provided options. In this technical report, we describe the process of creating, reviewing, and pilot testing the maze measures.…

In today's knowledge economy, technology is utilized more than ever to deliver instructional material to the learner. Nonetheless, information may not always be presented in a manner that maximizes the learning experience, resulting in a negative impact on learning outcomes. Drawing on the Task-Technology Fit model, a research framework was…

The purpose of this Item Response Theory study was to investigate how the expected reduction in item information, due to the collapsing of response categories in performance assessment data, was affected by varying testing conditions: item difficulty, item discrimination, inter-rater reliability, and direction of collapsing. The investigation used…

The main goal of this study was to assess how sensitive unidimensional parameter estimates derived from BILOG were when the unidimensionality assumption was violated and the underlying ability distribution was not multivariate normal. A multidimensional three-parameter logistic distribution that was a straightforward generalization of the…

The Mantel-Haenszel (MH) statistic for identifying differential item functioning (DIF) commonly conditions on the observed test score as a surrogate for conditioning on latent ability. When the comparison group distributions are not completely overlapping (i.e., are incongruent), the observed score represents different levels of latent ability…

When running out of time on a multiple-choice test, some examinees are likely to respond rapidly to the remaining unanswered items in an attempt to get some items right by chance. Because these responses will tend to be incorrect, the presence of "rapid-guessing behavior" could cause these items to appear to be more difficult than they…

A method of estimating Rasch-model difficulty calibrations from judges' ratings of item difficulty is described. The ability of judges to estimate item difficulty was assessed by correlating estimated and empirical calibrations on each of four examinations offered by the American Association of State Social Work Boards. Thirteen members of the…

This paper presents six steps in test construction generally recommended by measurement textbook authors. The focus is primarily on paper-and-pencil achievement tests as used by class instructions, although the discussion touches on the construction of other types of assessment. The six steps are: (1) determine the test purpose; (2) determine the…

Fit statistics provide a direct measure of assessment accuracy by analyzing the fit of measurement models to an individual's (or group's) response pattern. Students that lose interest during the assessment, for example, will miss exercises that are within their abilities. Such students will respond correctly to some more difficult items and…

A method is presented for the simultaneous analysis of differential item functioning (DIF) in multi-factor situations. The method is unique in that it combines item response theory (IRT) and analysis of variance (ANOVA), takes a simultaneous approach to multifactor DIF analysis, and is capable of capturing interaction and controlling for possible…

The Hindi language proficiency guidelines are based on the 1986 generic ACTFL (American Council on the Teaching of Foreign Languages) proficiency guidelines. They represent a hierarchy of global characterizations of integrated performance in four skill areas (speaking, listening, reading, and writing) in Hindi. The guidelines for each skill area…

Six series of experiments (11 individual experiments involving over 150 subjects) were conducted to further extend automatic/controlled processing research to command and control mission-specific training. The issues examined in these experiments were related to retention of task-component skills of amount of practice, component training for…

This research investigated developmental shifts in the character of children's tool using activity in the domain of scale drawing. Fifty-five children from three grade levels (grades 3, 5, and 7) were individually interviewed as they participated in both enlarging and reducing a one-dimensional object, the letter "F," to a scale of four.…

A two-level regression model is imposed on the ability parameters in an item response theory (IRT) model. The advantage of using latent rather than observed scores as dependent variables of a multilevel model is that this offers the possibility of separating the influence of item difficulty and ability level and modeling response variation and…