Large gains in scores have been observed over the first years of the Kentucky Instructional Results Information System (KIRIS) program. The extent to which these gains in scores indicate that student learning improved was evaluated. Previous studies have suggested that KIRIS score gains might be appreciably inflated, something that might result…

This study investigated the effect of writing task topic on learner performance in a second-language writing test, in this case the Michigan English Language Assessment Battery designed to test proficiency in English as a Second Language. The 64 topics or "prompts" used in the test (offered as pairs of options) were categorized according…

Modified Parallel Analysis (MPA) is a heuristic method for assessing "approximate unidimensionality" of item pools. It compares the second eigenvalue of the observed correlation matrix with the corresponding eigenvalue extracted from a "parallel" matrix generated by a unidimensional and locally independent model. Revised…

The first purpose of this study was to investigate the stability of two essential dimensionality measures across 10 random samples within a particular assessment item (AT1) selection. Other purposes were to investigate the discrepancy of the essential unidimensionality estimates for a test across different AT1 selections and sample sizes and to…

A didactic example is provided, using a Monte Carlo method, of how differential item functioning (DIF) can be eliminated (and thus better understood) when the complete latent space is used. The main source of DIF is that the matching single criterion used in some DIF procedures, Mantel Haenszel or Simultaneous Item Bias (SIBTEST), does not account…

The Iowa Test of Educational Development (ITED), a test of general abilities given to high school students, is intended as a tool for teachers and administrators to measure the progress of their students and plan accordingly; it also purports to assess the use by students of critical and evaluative abilities. Because such tests as the ITED carry…

Performance in assessing the unidimensionality of tests was examined for four methods: (1) W. F. Stout's procedure (1987); (2) the approach of P. W. Holland and P. R. Rosenbaum (1986); (3) linear factor analysis; and (4) non-linear factor analysis. Each method was examined and compared with the others using simulated and real test data. Seven data…

A model for summarizing test scores and using them to modify instructional programs is presented. The proposed model consists of two types of summaries of the data gathered through standardized tests. The first summary contains individual and single class results. Information in a "Class Item Response Record" chart provides individual student…

Enough evidence has accumulated on Bloom's "Taxonomy of Educational Objectives" for the cognitive domain to justify a review of its communicability. This article covers both published and unpublished studies as well as certain informal reports that bear on this property. It also examines possibilities for improving agreement among…

The major purpose of the three studies reported here was to investigate possible differences in agreement/disagreement with attitude statements as a function of their type (with regard to positivity/negativity) and personalism. In the first study, 90 students completed scales on energy conservation and on having good study habits. Agreement varied…

In the Spring 2003 issue of "Harvard Educational Review," Roy Freedle stated that the SAT® is both culturally and statistically biased, and he proposed a solution to ameliorate this bias. His claims, which garnered national attention, were based on serious errors in his analysis. We begin our analyses by assessing the psychometric…

It is a widely held belief that anchor tests should be miniature versions (i.e., minitests), with respect to content and statistical characteristics of the tests being equated. This paper examines the foundations for this belief. It examines the requirement of statistical representativeness of anchor tests that are content representative. The…

The similarity of item and ability parameter estimations was investigated using two numerical analysis techniques via marginal maximum likelihood estimation (MMLE) with a large simulated data set (n=1,000 examinees) and changing the number of quadrature points. MMLE estimation uses a numerical analysis technique to integrate examinees' abilities…

A joint maximum likelihood estimation algorithm, based on the partial compensatory multidimensional logistic model (PCML) proposed by L. Zeng (1989), is presented. The algorithm simultaneously estimates item difficulty parameters, the strength of each dimension, and individuals' abilities on each of the dimensions involved in arriving at a correct…

Research has shown that multiple-choice questions formed by transforming or paraphrasing a reading passage provide a measure of student comprehension. It is argued that similar transformation and paraphrasing of lesson questions is an appropriate way to form parallel multiple-choice items to be used as a posttest measure of student comprehension.…