Due to subjectivity in oral assessment, much concentration has been put on obtaining a satisfactory measure of consistency among raters. However, the process for obtaining more consistency might not result in valid decisions. One matter that is at the core of both reliability and validity in oral assessment is rater training. Recently,…

Despite the increased use of integrated tasks in high-stakes academic writing assessment, research on rating criteria which reflect the unique construct of integrated summary writing skills is comparatively rare. Using a mixed-method approach of expert judgement, text analysis, and statistical analysis, this study examines writing features that…

This study examined the effect of rater training on promoting inter-rater reliability in oral language assessment. It also investigated whether rater training and the consideration of the examinees' expectations by the examiners have any effect on test-takers' perceptions of being fairly evaluated. To this end, four raters scored 31 Iranian…

Oral narrative assessments are important for diagnosis of language disorders in school-age children so scoring needs to be reliable and consistent. This study explored the impact of training on the variability of story grammar scores in children's oral narrative assessments scored by multiple raters. Fifty-one speech pathologists and 19 final-year…

A large data set of L1 psycholinguistic norms (Balota "et al." 2007) was used to assess spoken L2 English lexical proficiency in cross-sectional and longitudinal learner corpora. Behavioral norms included lexical decision and word naming latencies (i.e. reaction times) and accuracies for 40,481 English words. A frequency measure was…

Salient in an EFL teaching context is students' dissatisfaction with their final scores especially in oral courses. This study tried to bridge the gap between students' and teachers' rating system through alternatives to existing measurement methods. Task-based language assessment has stimulated language teachers to question the way through which…

Speech act theory-based, second language pragmatics testing (SLPT) poses problems for validation due to a lack of correspondence with empirical conversational data. Since conversation analysis (CA) provides a richer and more accurate account of language behavior, it may be preferred as a basis for SLPT development. However, applying CA methodology…

This study examined the effects of training on peer assessment and comments provided regarding oral presentations in EFL (English as a Foreign Language) classrooms. In Study 1, both the treatment and control groups received instruction on skill aspects, but only the treatment group was given an additional 40-minute training on how to rate…

Issues surrounding the testing and evaluation of speaking in Singapore secondary schools are examined, including the questions of what skills and knowledge a speech evaluator should possess and what model of spoken knowledge--Received Pronunciation (RP) or Singapore English--should be provided. Oral tests, developed for the English language…

The increasing availability and performance of computer-based testing has prompted more research on the automatic assessment of language and speaking proficiency. In this investigation, we evaluated the feasibility of using an off-the-shelf speech-recognition system for scoring speaking prompts from the LanguEdge field test of 2002. We first…

Differences in procedures used by academic institutions and government agencies in administering the American Council on the Teaching of Foreign Languages' Oral Proficiency Interview test are examined, and results and implications of two studies of interrater reliability are discussed. (MSE)