This article proposes that sampling design effects have potentially huge unrecognized impacts on the results reported by large-scale district and state assessments in the United States. When design effects are unrecognized and unaccounted for they lead to underestimating the sampling error in item and test statistics. Underestimating the sampling…

This study is an evaluation of the psychometric issues associated with estimating objective level scores, often referred to as "subscores." The article begins by introducing the concepts of reliability and validity for subscores from statewide achievement tests. These issues are discussed with reference to popular scaling techniques, classical…

Pennsylvania's Educational Quality Assessment Program provides each participating school with a building level report in which state percentiles are a prominent part. Multiple matrix sampling was being considered as a technique to reduce testing time. However, there was great concern that the error associated with estimating the school mean might…

The Registered Holistic Scoring Method, which has been used for one year to score the ninth grade writing test of the New Jersey High School Proficiency Test, is described. Registered Holistic Scoring was developed from the previous holistic approach in order to provide more reliable scoring guidelines year after year. Two trained evaluators…

Two attempts to adjust state mean Scholastic Aptitude Test (SAT) scores for differential participation rates are examined. Both attempts are rejected, and five rules for performing adjustments are outlined to foster follow-up checks on untested assumptions. National Assessment of Educational Progress state data are determined to be more accurate.…

Political and legislative pressures have posed a number of measurement issues and challenges to the development of sound, valid voluntary national tests (VNTs). This paper focuses on what appear to be the most difficult technical issues related to the VNT proposed by President Clinton in 1997. Technical issues refer to psychometric issues, as…

Portfolios are the centerpiece of Vermont's statewide assessment program in mathematics. Portfolio scores in the first two years were not reliable enough to permit the reporting of student-level results, but increasing the number of readers or the number of portfolio pieces is not operationally feasible. (SLD)

The adequacy of linking statewide standardized test results to the National Assessment of Educational Progress by using equipercentile equating procedures was investigated using statewide mathematics data from four states. Results suggest that the linkings are not sufficiently trustworthy to make comparisons based on the tails of the distribution.…

Key findings of a pilot study of the Alaska Instructional Diagnostic System (AIDS) are summarized. The AIDS pilot test served to verify the appropriateness of the skills survey as well as the validity and reliability of the items. The AIDS testing system includes three components: (1) upper level skills surveys (grades 3-8); (2) lower level skill…

Part I of this report attempts to describe the system that was developed for local educational agencies by the Delaware State Department of Public Instruction to support classroom and curricular improvement in mathematics through the administration of an objective-referenced test in mathematics to grade four students. This system includes the…

Most states have expanded their statewide testing programs to include alternative educational assessments, and two (Kentucky and Maine) have completely abandoned the multiple-choice format. However, over half of states designing alternative assessments are encountering major difficulties related to the high cost of performance-based assessments,…

The statewide teaching performance assessment instruments being used in Georgia, North Carolina, and Florida were examined. Forty-one reliability and validity studies regarding the instruments in use in each state were collected from state departments and universities. Georgia uses the Georgia Teacher Performance Assessment Instrument. North…

Since the approval of the Basic Skills Improvement Policy in 1978, the Massachusetts Department of Education has been developing tests and alternative forms for the assessment of student achievement in five basic skills content areas: reading, writing, mathematics, listening, and speaking. Because of the lack of previous research on which to draw…

The psychometric properties of the testing tools of the Michigan Educational Assessment Program (MEAP), the state standardized testing program, are examined. Reliability studies have indicated that the scores from the MEAP, ranging from 0.654 to 0.949, are generally acceptable. The State Department of Education offered supporting evidence for the…

This digest is designed to provide information to Texas testing coordinators, other educators, and interested citizens about the development procedures and technical attributes of the state-mandated criterion-referenced assessment program. The chapters are: (1) "Background"; (2) "Test Development"; (3) "Test…