The term measurement disturbance has been used to describe systematic conditions that affect a measurement process, resulting in a compromised interpretation of person or item estimates. Measurement disturbances have been discussed in relation to systematic response patterns associated with items and persons, such as start-up, plodding, boredom,…

A number of brief papers are presented to provide guidelines for test directors of school systems. This collection is intended for both newly appointed and experienced directors. Contributions were solicited from practicing directors of testing; the authors include Anthony J. Allen, Margaret Backman, Joan Bollenbacker, Gerald Hanna, James Lawson;…

Traditionally, most rehabilitation centers have evaluated a clients' vocational skills on an individual basis. Individually administered work samples, however, may not provide as much individual attention to the assessment process as is usually assumed. Two types of group work sample evaluation systems have been developed: the Valpar Work Sample…

Several issues are related to the use of educational tests. First, test users must be able to choose appropriate tests, interpret scores, and make decisions based on scores. In the field of educational testing, few test users have adequate training in these areas. Second, test makers must clearly specify directions for administration, allowable…

The Maryland School Performance Assessment Program (MSPAP) is a relatively new, statewide performance assessment of students in grades 3, 5, and 8. When first administered in May of 1991, the MSPAP included a battery of performance assessment tasks designed to generate written or drawn responses to reading texts. This study evaluated selected…

A standard-setting procedure was developed for the Georgia Teacher Certification Testing Program as tests in 30 teaching fields were revised. A list of important characteristics of a standard-setting procedure was derived, drawing on the work of R. A. Berk (1986). The best method was found to be a highly formalized judgmental, empirical Angoff…

Policy-making problems being faced in Louisiana regarding the testing of handicapped students within a state mandated minimum competency testing program are dealt with. The decision-making process is complicated by the fact that two groups (staff administering the test and staff administering programs for the handicapped) with differing…

Possible sources of intrajudge inconsistency in standard setting are reviewed, and approaches are presented to improve the accuracy of rating. Procedures for providing judges with feedback through discussion or computerized communication are discussed. Monitoring and maintaining judges' consistency throughout the rating process are essential. (SLD)

Although much has been written about educational evaluation, few guidelines exist for consumers--project directors, school administrators, curriculum developers, legislators, teachers, parents, and boards of education. Several cautions surface from a review of the literature. First, tests that are based on program objectives are most useful to…

Robert Stake presents his views on responsive evaluation, naturalistic approaches to evaluation, the role of testing in evaluation, and the training of evaluators. (BW)

The methods used by the General Educational Development Testing Service (GEDTS) to establish and maintain score stability and reading reliability on its direct assessment of writing are described. Using the 1988 site certification and monitoring results of several scoring sites, the focus is on describing how the score scale was established and…

An accepted criterion for gauging the fairness of examinees' scores, derived from judge-awarded ratings, has been the size of the correlation between the judges and the inter-rater reliability. Various means of achieving inter-rater reliability were reviewed, and a model to measure inter-rater reliability is forwarded. Both theoretical and…

Ways to use standard-setting data to adjust cutoff scores on examinations are reviewed. Ten sources of information to be used in determining standards are listed. The decision to modify passing scores should be based on these types of information and consideration of adverse impact or rating process irregularities. (SLD)

Describes some forces that have shaped current attitudes toward evaluation of education, presents five guidelines to help in discussions of assessment, and charges teachers with the responsibility of making evaluation effective. (DF)

As part of a study of teacher evaluation of student replies to open-ended questions, a second question--the best method of determining interrater reliability--was examined. The standard method, the Pearson Product-Moment correlation, overestimated the degree of match between researchers' and teachers' scoring of tests. The simpler percent…