This article proposes that sampling design effects have potentially huge unrecognized impacts on the results reported by large-scale district and state assessments in the United States. When design effects are unrecognized and unaccounted for they lead to underestimating the sampling error in item and test statistics. Underestimating the sampling…

Sampling errors limit the accuracy with which forms can be linked. Limitations on accuracy are especially important in testing programs in which a very large number of forms are employed. Standard inequalities in mathematical statistics may be used to establish lower bounds on the achievable inking accuracy. To illustrate results, a variety of…

The Los Angeles Unified School District has been concerned over the excess testing of elementary school pupils, particularly those who are in special programs. When the Board of Education decided to have yearly testing of pupil progress using a locally developed criterion-referenced test, the intention was to replace a previously used…

This study considers the use of repeaters when test equating. The subjects consist of five groups of applicants to a professional certification program. Each group comprises first time examinees and repeaters. The procedures include a common item linear equating with nonrandom groups, use of equating chains, and the use of total examinee group…

If annual performance assessments are to yield results that can be compared from year to year, many technical problems must be addressed. It is essential that tests to be equated measure the same construct. Methods of equating performance assessment scores, ways of equating system assessments, and standard setting are discussed. (SLD)

Specific recommendations are made concerning the circumstances under which the benefits of out-of-level testing outweigh the problems associated with it. Topics explored are: various methods for deciding when a set of test scores is invalid and the utility of these methods for local evaluators, the accuracy of vertical scaling, and the usefulness…

Criticisms of norm referenced standardized tests are contrasted with the assumption and claims of proponents of Rasch model testing and item banking. The Rasch test theory model is based on the assumption of equal interval curriculum referenced scales for both students' abilities and test item difficulties. The Portland school district has…

The research leading to the decisions to revise the Graduate Record Examination Aptitude Test (GRE) (beginning in October 1981) is reviewed. The issues discussed include the format of the test (the timing of each section and the number of sections, the content of the sections--especially the analytical section), the scoring procedure for the GRE,…

Item parameter estimates derived through item response theory methods have been considered relatively robust to changes in item position and context, but the anomaly in reading scores from the 1986 National Assessment of Educational Progress (NAEP) illustrates problems with common population equating procedures when there are test form changes.…

Although few question the positive impacts alternative forms of assessment can have on instruction, concerns about the psychometric quality of data obtained from such assessments are taking their toll. Scoring issues are at the heart of many of these concerns. This paper addresses the causes of these concerns: misinformation about psychometric…

Two major issues are addressed in this commentary on the National Assessment of Educational Progress (NAEP) reading assessment: the design of the reading proficiency scale and interpretation of test data; and the dimensionality of this scale and the possibility of supplying data on specific skill dimensions. An examination of NAEP data and the…

The Council of Chief State School Officers has raised some issues pertinent to the implementation of a national project to collect statistical indicators of academic achievement and to compare them across states. The United States Department of Education shares this concern, with an emphasis on sampling schools for a school level analysis which…

A study involving all students taking the California Achievement Test (CAT) in the Cleveland City School District (Ohio) during the fall of 1989 was conducted to assess the effects of the use of parallel test forms. Each spring, all students in grades 4, 6, and 8 are selected to take the appropriate level of the CAT reading test, Form E. As a…

This paper focuses on the questions which arise as test practitioners monitor score scales derived from latent trait theory. Large scale assessment programs are dynamic and constantly challenge the assumptions and limits of latent trait models. Even though testing programs evolve, test scores must remain reliable indicators of progress.…

In spring 1986, new citywide testing programs in both reading and mathematics were implemented by the New York City Board of Education. In those cases where the test had changed, it was necessary to statistically equate the new tests to the old tests, so that comparisons of equated rather than actual scores could be made during the transitional…