Researchers have documented the impact of rater effects, or raters' tendencies to give different ratings than would be expected given examinee achievement levels, in performance assessments. However, the degree to which rater effects influence person fit, or the reasonableness of test-takers' achievement estimates given their response patterns,…

At the University of California, Los Angeles, the Test of Oral Proficiency (TOP), an internally developed oral proficiency test, is administered to international teaching assistant (ITA) candidates to ensure an appropriate level of academic oral English proficiency. Test taker performances are rated live by two raters according to four subscales.…

Assessing a high school's effectiveness is not straightforward. Comparing a school's standardized test scores to those of other schools is one approach to measuring effectiveness, but a major objection to this method is that students' test scores tend to be related to students' "socioeconomic" status--family household income, for…

Much of the argument about comparability of examination standards is at cross-purposes; contradictory positions are in fact often both defensible, but they are using the same words to mean different things. To clarify this, two broad conceptualisations of standards can be identified. One sees the standard in the observed phenomena of performance…

Robert Coe has claimed that three broad conceptions of comparability can be identified from the literature: performance, statistical and conventional. Each of these he rejected, in favour of a single, integrated conception which relies upon the notion of a "linking construct" and which he termed "construct comparability".…

This handbook is designed as a quick reference for first-year teachers who find themselves in an assessment driven environment with little experience to help make sense of the language, underlying philosophy, or organizational structure of the assessment system. The handbook begins with advice on developing and evaluating effective learning…

"Classroom Assessment in Action" clarifies the multi-faceted roles of measurement and assessment and their applications in a classroom setting. Comprehensive in scope, Shermis and Di Vesta explain basic measurement concepts and show students how to interpret the results of standardized tests. From these basic concepts, the authors then…

This paper is based on guidelines developed in 1989 for training workshops for state and local educators to demonstrate the processes by which performance assessments could be created, validated, and used in statewide assessment programs. These guidelines are based on work with the National Assessment of Educational Progress and several statewide…

Describes four criteria that can be used to evaluate methods of assessment: (1) "What are the consequences of using the test?" (2) "Is this assessment fair?" (3) "Do the skills and knowledge of this assessment transfer or generalize?" and (4) "Does this assessment cover cognitively complex task?" (KDP)

Describes the addition of a writing performance assessment to the Graduate Management Admission Test (GMAT), and how development of the Analysis of Argument measure and the Persuasive Adaptiveness measure helps explain the holistic writing score given during grading of the GMAT. Correlates holistic and analytic scores, revealing a positive…

A quiz is presented that may call the attention of school personnel to regression effects in educational measurement, and clarify the meaning of statistical regression for educators in schools. Ten questions and their answers explore regression in the contexts of classroom testing, grading, and performance assessment. (SLD)

Based on her study of the assessment and validation of final year projects in two academic departments--one located in a humanities faculty and the other in an engineering faculty of a South African university--Suellen Shay argues that the assessment of complex tasks is a socially situated interpretive act. Her argument centers on three questions.…

The concept of developmental appropriateness can be applied to the assessment of young children. Clarifying the main purpose for which children are assessed can help determine what kinds of assessments would be most appropriate. Assessment of individual children might serve one of the following purposes: (1) to determine progress on significant…

Several common sources of error in assessment that depends on the use of judges are identified, and ways to reduce the impact of rating errors are examined. Numerous threats to the validity of scores based on ratings exist. These threats include: (1) the halo effect; (2) stereotyping; (3) perception differences; (4) leniency/stringency error; and…

California has provided other states with an example of how not to assess student writing and has initiated a new direction in the statewide assessment of student writing achievement, a direction designed to influence writing instruction positively. Before 1987, the California Assessment Program (CAP) limited English skills assessment to multiple…