This National Council on Measurement in Education (NCME) module is designed to help teachers understand test reliability requirements. Focus is on the reliability of scores from teacher-made achievement tests used for interpreting norm-referenced scores. Reliability, factors affecting scores, reliability estimation methods, reliability's relation…

A Friends for Education (FFE) survey revealed that no state is below the norm at the elementary school level on six nationally normed commercially available achievement tests. Tests use a norm group from the past for comparison, but FFE suspects that inaccurate initial norms and teaching the test may cause high scores. (SLD)

Officials of the United States Department of Education conclude that the pervasive "above average" achievement reported by J. J. Cannell (1988) results from a combination of testing practices and procedures among school testing programs. Causative factors of this paradox, dubbed the "Lake Wobegon Phenomenon," are discussed.…

Representatives of the Riverside Publishing Company discuss the "Lake Wobegon Phenomenon," the reported "above average" achievement of most elementary schools nationwide that was discussed by J. J. Cannell (1988) of the Friends for Education. More appropriate use of normative comparisons and more complete reporting of test…

The time-bound nature of test norms is discussed in light of the report of J. J. Cannell (1988) that a large majority of elementary schools report above average achievement on national achievement tests. Generally increasing levels of achievement are deemed to be responsible for this phenomenon. (SLD)

Recognizing the concerns raised by J. J. Cannell (1988), Science Research Associates (SRA) defends its norms. Trends in curriculum need to be monitored in addition to students' achievement levels. New norms are needed only when there is a significant shift in curriculum or student performance. (SLD)

Despite the skepticism of J. J. Cannell (1988) that commercial test scores reveal much about the real achievement level of students, there is ample evidence that at least some of the gains observed on standardized tests are legitimate. Standardized scores should be greeted with healthy skepticism when high, but taken very seriously when low. (SLD)

The instructional impact of criterion-referenced testing during the past three decades is reviewed, and a method of devising criterion-referenced tests is suggested that departs from the usual model. Adding verification along the lines of panel review that newly constructed test items do elicit appropriate information is suggested. (SLD)

In response to J. J. Cannell's (1988) discussion of the reported "above average" achievement of the vast majority of public school systems nationwide, the nature of national norms is discussed. The public must be better informed about the interpretation of test results. (SLD)

The developer of the Comprehensive Tests of Basic Skills scaling responds to Hoover's attacks on Thurstone and item response theory scales (TM 510 173). While agreeing that grade equivalent (GE) scores are presently the most appropriate developmental score for reporting elementary grade achievement tests, limitations and inconsistencies in the GE…

Grade equivalent (GE) scores for elementary school achievement tests are defended in response to efforts to ban their use. Equal interval developmental scoring methods for the Sequential Tests of Educational Progress and the Comprehensive Tests of Basic Skills are criticized. (BS)

Some unfinished issues relating to achievement test theory that seemed implicit in the basic idea of criterion-referenced testing are reviewed, recognizing their importance in current studies of authentic assessment and performance-based tests. The future of performance-based evaluation is explored. (SLD)

The author addresses issues raised by Burket (TM 510 174) about the Iowa Test of Basic Skills scaling procedures. Further reasons for his criticism of Thurstone scale scores and item response theory scale scores for elementary school achievement tests are given. (BS)

This paper discusses how to maintain the integrity of national nomative information for achievement tests when the test that is administered has been customized to satisfy local needs and is not a test that has been nationally normed. Alternative procedures for item selection and calibration are examined. (Author/LMO)

One means of establishing the validity of performance-based assessments is to determine whether the criteria for basic competencies on performance tests are reasonable in relation to expected (nationally normed) age-level performance. Links between performances on three assessments in Michigan and the Comprehensive Tests of Basic Skills illustrate…