This paper explores the reliability of using ChatGPT in evaluating EFL writing by assessing its intra- and inter-rater reliability. Eighty-two compositions were randomly sampled from the Written English Corpus of Chinese Learners. These compositions were rated by three experienced raters with regard to 'language', 'content', and 'organization'.…

Teacher agency is a pivotal element of professionalism and second/foreign language (L2) education. However, its role in L2 assessment has remained under-researched. Part of this negligence is due to the absence of a validated questionnaire to measure the construct and its underlying components. To address this gap, drawing on the ecological…

Comparative Judgement (CJ) is an evaluation method, typically conducted online, whereby a rank order is constructed, and scores calculated, from judges' pairwise comparisons of performances. CJ has been researched in various educational contexts, though only rarely in English as a Foreign Language (EFL) writing settings, and is generally agreed to…

The study aimed to elicit information from Vietnamese EFL university instructors about their knowledge and skills regarding the principles, theory, and practices of language assessment by means of revision and validation of the Language Assessment Literacy--Revised Vietnam (LAL-RV), which was previously developed by Kremmel and Harding (2020). A…

Whereas many scholars have emphasized the relative importance of "comprehensibility" as an ecologically valid goal for L2 speech training, testing, and development, eliciting listeners' judgments is time-consuming. Following calls for research on more efficient L2 speech rating methods in applied linguistics, and growing attention toward…

This study examined the effect of rater training on promoting inter-rater reliability in oral language assessment. It also investigated whether rater training and the consideration of the examinees' expectations by the examiners have any effect on test-takers' perceptions of being fairly evaluated. To this end, four raters scored 31 Iranian…

Sight translation has been widely used in aptitude testing to screen prospective trainee interpreters at leading interpreter training schools, including ESIT, ISIT, and EMCI. However, it has also been criticised for its lack of validity and reliability. No empirical study has thus far been conducted to explore its power to predict interpreting…

Panahi and Mohebbi review James Dean Brown's 50-years of research in language testing, curriculum development and research statistics with reference to an impressionistic framework for analysis containing two components with their subcomponents: Annotations (i.e., briefing and implications) and main concepts and themes (i.e., testing and teaching…

In language performance tests, raters are important as their scoring decisions determine which aspects of performance the scores represent; however, raters are considered as one of the potential sources contributing to unwanted variability in scores (Davis, 2012). Although a great number of studies have been conducted to unpack how rater…

This study measures the perceptions of English language teachers at Japanese universities. It combines the results of online questionnaires on assessment practices and another one on teacher autonomy. Results suggest that according to teachers' self-reports of assessment practices, certain measures of assessment quality are being affected by the…

In this investigation we propose new machine learning methods for automated scoring models that predict the vocabulary acquisition in science and social studies of second grade English language learners, based upon free-form spoken responses. We evaluate performance on an existing dataset and use transfer learning from a large pre-trained language…

English teachers' assessment literacy has always been considered as an important factor in their performance. However, no instrument has ever been developed to assess this construct among Iranian EFL teachers. To fill this gap, in the first phase of the present study, a theoretical framework for the main four components of teacher assessment…

Common European Framework of Reference (CEFR) is an international standard to measure learners' language abilities on a six-point scale, A1 for beginners up to C2 for those who have mastered a language. this study attempted to examine the implementation of the CEFR policy in learning Arabic in Indonesia at al levels, beginning form curriculum…

This study reports on the development and assessment of a new 30-item Multidimensional Language Class Anxiety Scale which is designed to assess foreign language learners' anxiety regarding four language skills (listening, reading, writing, and speaking) and testing. In Study 1, the initial items were piloted with 323 students studying English as a…

Background: Translation quality assessment (TQA) suffers from subjectivity in both neighboring disciplines: 'TEFL' and 'Translation Studies, and more empirical studies are required to get closer to objectivity in this domain. The present study evaluated the quality of the written translation of TEFL students through three different approaches to…