This article proposes that sampling design effects have potentially huge unrecognized impacts on the results reported by large-scale district and state assessments in the United States. When design effects are unrecognized and unaccounted for they lead to underestimating the sampling error in item and test statistics. Underestimating the sampling…

State-level testing programs continue to grow, and the challenge of validation does not wane. Although more than a decade has passed since the 1999 Joint Standards for Educational and Psychological Testing set out a call for the organization of validity evidence into validity arguments, practical examples of such arguments are not readily…

This study is an evaluation of the psychometric issues associated with estimating objective level scores, often referred to as "subscores." The article begins by introducing the concepts of reliability and validity for subscores from statewide achievement tests. These issues are discussed with reference to popular scaling techniques, classical…

It is argued that publishers of achievement tests, especially those who publish tests intended for use in many parts of the United States, are for the most part not in a position to obtain any decent evidence about the consequences of the uses that are made of their tests. What responsibilities and actions publishers can reasonably be expected to…

This paper was presented with other papers in a forum dealing with statewide testing programs. The primary purpose of the paper is to address practical considerations and methods of resolution for large districts or states who are planning on conducting large scale testing or assessment programs with criterion or performance referenced measures.…

The Arizona Tax Research Association analyzed the Arizona Teacher Proficiency Examination (ATPE). An item analysis of the 150 questions administered to 2,430 applicants from July 1982 through March 1983 was undertaken. A minimum point biserial correlation coefficient of .33 indicated the validity of an item. Nearly two-thirds (64 percent) of the…

In 1979-80, the Arkansas minimum competency tests were administered to a sample of 5,000 students in grades 3, 6, and 8. To determine how well test objectives matched the curriculum, their teachers estimated how many of the four items per objective a randomly selected student would answer correctly. Because chi square test comparisons of teacher…

State directors of teacher certification and testing provided updated data (reports, press clippings, manuals, legislative hearings, and validity studies) in April 1987 describing their testing programs and the latest passing rates. Twenty-seven states have admissions testing programs, and 44 have certification testing programs. It is unclear…

A method to investigate consequential evidence of validity for a state assessment developed to change teacher instructional practices is presented. Survey responses from over 1,000 Kansas teachers were used to construct a path model that allowed effects of the state assessment to be studied at building and teacher levels. (SLD)

Based on 18 months of extensive research and study, the New Mexico State Department of Education developed the Staff Accountability Plan to address the issue of teacher accountability and certification. One of the provisions of the plan (a written assessment of general and professional knowledge for initial certification) will be fulfilled by…

Security procedures for the New Jersey High School Proficiency Test (HSPT) are discussed and evaluated. All New Jersey high school students are required to pass the HSPT, which was administered for the first time in 1984. Generally, security plans are designed to limit access to test questions prior to test administration and to prevent…

Reports on an ongoing evaluation of the Vermont portfolio assessment program. Indicates that the positive news about the instructional effects of the assessment program are in contrast with the empirical findings about the quality of the data the program has yielded. (SLD)

This pamphlet presents answers to the most frequently asked questions about the Michigan Educational Assessment Program (MEAP). They include questions about the history of MEAP, its costs, subject coverage, test validity, the type of tests, administration of the tests, and use of MEAP results. (BW)

The items of the Florida minimal competency test have never been validated as measures of "survival skills," and the pass/fail standards were set mindlessly and capriciously. (Author/IRT)

A Teacher Survey was developed for administration to Illinois teachers whose students take the Illinois Inventory of Educational Progress (IIEP). The IIEP are the achievement tests of the state assessment program. The purposes of the Teacher Survey range from determination of a test's curricular validity to investigations of teachers' abilities to…