In the context of policy-driven language testing for citizenship, a growing body of research examines the political justifications and ethical implications of language requirements and test use. However, virtually no studies have looked at the role that language testers play in the evolution of language requirements. Critical gaps remain in our…

Researchers have documented the impact of rater effects, or raters' tendencies to give different ratings than would be expected given examinee achievement levels, in performance assessments. However, the degree to which rater effects influence person fit, or the reasonableness of test-takers' achievement estimates given their response patterns,…

Rater training is an important part of developing and conducting large-scale constructed-response assessments. As part of this process, candidate raters have to pass a certification test to confirm that they are able to score consistently and accurately before they begin scoring operationally. Moreover, many assessment programs require raters to…

Elicited Imitation Tests (EITs) are commonly used in second language acquisition (SLA)/bilingualism research contexts to assess the general oral proficiency of study participants. While previous studies have provided valuable EIT construct-related validity evidence, some key gaps remain. This study uses an integrative data analysis to further…

Drawing on Glenn Fulcher's extensive work in performance-based language assessment of speaking, this paper explores the assessment of L2 speaking ability in local language testing contexts. For that purpose, I review Fulcher's influential work that highlights the relationship between the speaking construct, the task, the performance, and the…

This study examined the use of a popular Automatic Speech Recognition (ASR), Google Voice Typing (GVT), to automatically assess English as second language pronunciation. It aimed to answer the following question: What is the relationship between GVT-rated scores and human-rated scores? To answer this question, we compared audio recordings of 56…

Assessment lies at the centre of PhD degree quality standards, with quality assurance relying on independent external examiners. This study investigates the role of the viva and the selection of external examiners from within the accounting and finance discipline across UK institutions. A questionnaire survey and follow-up interviews with…

This study employed Bachman and Palmer's (2010) Assessment Use Argument framework to investigate to what extent the use of a second language oral test as an exit test in a Hong Kong university can be justified. It also aimed to help test developers of this oral test identify the most critical areas in the current test design that might need…

The first article in this section, "Rating Scales and Shared Meaning," by Winifred A. Lopez, discusses the analysis of rating scale data. The second article, "Rating Scale Categories: Dichotomy, Double Dichotomy, and the Number Two," by Mark H. Stone, argues that dichotomies in rating scales are more useful than multiple…

Developing scoring rubrics to evaluate student work was studied, concentrating on the use of intermediate points in rating scales. How scales that allow for intermediate points between defined categories should be constructed and used was explored. In the recent National Assessment of Educational Progress (NAEP) visual arts field test, researchers…

Most human gradings of essays are holistic, or "overall." Therefore, Project Essay Grade (PEG), an attempt to develop computerized grading of essays, has concentrated most of its research on overall grading. It has successfully simulated human judges. However, since computer grading is less expensive than human grading, PEG has also…

This paper reports the results of an analysis of the relationship between scorer behaviors and score variability. Thirty-six essay scorers were interviewed and asked to perform a think-aloud task as they scored 24 essays. Each comment made by a scorer was coded according to its content focus (i.e. appearance, assignment, mechanics, communication,…

Describes some forces that have shaped current attitudes toward evaluation of education, presents five guidelines to help in discussions of assessment, and charges teachers with the responsibility of making evaluation effective. (DF)

Discusses the role and consistency of judges' interpretations of teacher performance as part of an evaluative scheme for complex performance, with reference to the ideological framework of professional standards. The tension between assessment decisions and the recognition that assessment involves interpretation is explored. (SLD)

The "Standards for Educational and Psychological Testing" (1985) and the "Uniform Guidelines on Employee Selection Procedures" (1978) provide no clear guidance on choosing methods for setting standards on teacher certification tests, although they require that practices be well-documented and produce reasonable results, and…