Standard setting is a method used to set cut scores on large-scale assessments. One of the most popular standard setting methods is the Bookmark method. In the Bookmark method, panelists are asked to envision a response probability (RP) criterion and move through a booklet of ordered items based on a RP criterion. This study investigates whether…

Scores are the most visible and widely used products of a testing program. The choice of score scale has implications for test specifications, equating, and test reliability and validity, as well as for test interpretation. At the same time, the score scale should be viewed as infrastructure likely to require repair at some point. In this report…

A new approach for examining the quality of judgments from standard-setting judges using a Binomial Trials Model (BTM) is presented and illustrated with 26 judges from the Georgia High School Graduation Test. Results suggest that the BTM provides information not available from other methods. (SLD)

In the context of controversy over the use of high stakes testing, the Pennsylvania State Education Association (PSEA) asked for an evaluation of the performance levels and cut score of the Pennsylvania System of School Assessment (PSSA) mathematics and reading tests. While awaiting technical documentation from the Pennsylvania Department of…

These five papers describe the methods, results, and implications of a multimethod approach to the establishment of standards for the three subject areas of the Alabama High School Graduation Examination (AHSGE). James E. McLean introduces the problems and methods involved by identifying three general approaches to scoring procedures found in the…

In most judgmental methods for setting performance standards, content experts evaluate test items based on a conceptualization of a borderline examinee or a group of examinees. An alternative method is proposed by which judges evaluate items based on the degree of criticality regardless of content area. This classification of items serves to link…

Standard-setting practices in states using the National Teachers Examination (NTE) were examined. Passing score-setting procedures, recommended study scores, passing scores established by the states, and the implications of the passing scores were studied. States have typically adopted standards lower than cut scores recommended by advisory…

Evidence of the validity of the National Teacher Examinations' (NTEs') Early Childhood Education Test (ECET) as it is used in the New York City School System is presented. The NTE Core Battery and Specialty Area Tests are used as part of the alternative process for teacher licensure in New York City. The ECET measures knowledge and skills required…

Focusing on the experiences of five southern states using teacher testing policies, this survey provides evidence of shared experiences and lessons learned despite the existence of five different sets of certification requirements. The first section of the report concentrates on the national context for the movement toward teacher testing.…

The primary objective of this research was to examine the effect of scoring items known to be multidimensional using a unidimensional model. Although several simulation studies have examined this, few studies have been conducted using data obtained from actual test administrations. In this study, open-ended mathematics items from a mandated state…

This report presents the results of an initial, iterative performance standards-setting (SS) task of a comprehensive on-the-job statewide teacher assessment system--the System for Teaching and Learning Assessment and Review (STAR). The 1990-91 STAR assesses and makes inferences about the quality of teaching and learning on sets of assessment…

Several tests available from the Educational Testing Service (Princeton, New Jersey) have been used or are candidates for use by the Tennessee State Department of Education as initial public school personnel licensure endorsement area tests. The first of the two reports contained in this document (i.e., the Executive Report) describes a study of…

Memphis State University conducted a study for the Tennessee Department of Education to determine the validity of 11 previously unvalidated, extensively revised, or new Educational Testing Service (ETS) tests as measures of the skills and knowledge required for specific initial certification endorsements of public school personnel in Tennessee.…

A study conducted for the Tennessee State Department of Education by Memphis State University addressed two objectives with respect to the Professional School Service Personnel endorsements for initial licensure for non-teaching professional personnel: to determine the validity of the National Teacher Examination (NTE) Core Battery tests for…

Comparative data and results of validation and standard-setting studies are presented for five subject-matter tests for teacher certification in two states. The five tests evaluated were available from the Educational Testing Service (ETS): one National Teacher Examination specialty area test (special education); and four tests from a consortium…