In this speech, the author covers a lot of ground. In the first half of his speech, the author gives a brief summary of the last 40 years of the history of language testing, from his perspective. The author reviews these years more or less by decade. Additionally, he discusses the evolution of the profession of language testing during this period,…

Student responses to a large number of constructed response items in three Math and three Reading tests were scored on two occasions using three ways of assigning raters: single reader scoring, a different reader for each response (item-specific), and three readers each scoring a rater item block (RIB) containing approximately one-third of a…

This is a mixed methods investigation of consistency in PhD examination. At its core is the quantification of the content and conceptual analysis of examiner reports for 804 Australian theses. First, the level of consistency between what examiners say in their reports and the recommendation they provide for a thesis is explored, followed by an…

In this essay, we describe our experiences working with a rebellious primary stakeholder, Sylvia, as evaluators of a district-wide safe schools program. Given the breadth of the program and its multiple target constituencies, we were confronted with the challenges of managing a large number of stakeholders, or those individuals and groups that…

Developing and using a design for teacher portfolio assessment is a complex process including several components: the domain to be assessed (the teacher competences), the content standards or criteria, the portfolio format, the completion of the format (by teachers) with content, and the scoring of the portfolios (by raters). For a portfolio…

A district evaluator in a large Texas district examines new challenges arising since implementation of No Child Left Behind, relating to (1) navigating competing requirements in state and federal accountability systems; (2) evaluating effectiveness of sanctions districts are required to address; (3) using scientifically based research (SBR) to…

This study investigates the effect of reporting the unadjusted raw scores in a high-stakes language exam when raters differ significantly in severity and self-selected questions differ significantly in difficulty. More sophisticated models, introducing meaningful facets and parameters, are successively used to investigate the characteristics of…

Rigorous research provides information that will allow ECE (Early Childhood Educator) programs to select interventions that have a scientifically based track record of effectiveness in increasing teachers' skills and teaching quality. This paper shares implementation and impact results as well as the lessons learned in conducting an evaluation of…

There has been an increase in the use of partnerships as a policy prescription for improving education since the mid 1980's. This trend builds on nearly a century of reform movements in education, has been noted in the professional literature and can also be seen in the growing trend to add partnership requirements to federal grant programs in a…

The definition and operationalization of interactional competence in speaking tests that entail co-construction of discourse is an area of language testing requiring further research. This article explores the reactions of four trained raters to paired candidates who oriented to asymmetric patterns of interaction in a discussion task. Through an…

Racial and ethnic identity (REI) measures are in danger of becoming conceptually meaningless because of evaluators' insistence that they conform to measurement models intended to assess unidimensional constructs, rather than the multidimensional constructs necessary to capture the complexity of internalized racial or cultural socialization. Some…

Four experiments show that gender differences in the propensity to initiate negotiations may be explained by differential treatment of men and women when they attempt to negotiate. In Experiments 1 and 2, participants evaluated written accounts of candidates who did or did not initiate negotiations for higher compensation. Evaluators penalized…

Increasing calls for accountability in education have promoted improvements in quantitative evaluation approaches that measure student performance; however, this has often been to the detriment of qualitative approaches, reducing the richness of educational evaluation as an enterprise. In this article the authors assert that it is not merely…

In this article, the authors provide their analyses on a scenario wherein Dr. Luanda, an evaluator, facilitated an empowerment evaluation with youth. In this commentary, the authors review what they think is missing in Dr. Luanda's participatory evaluation effort to empower youth and highlight the knowledge and skills they believe are essential…

By centralizing the issue of test fairness in language proficiency assessments, this study responds to a call by researchers for developing greater social responsibility in the language testing agenda. As inquiries into language attitude and psychology indicate, there is an underlying uncertainty pertaining to the validity of test use and score…