Rating scales that language testers design should be tailored to the specific test purpose and score use as well as reflect the target construct. Researchers have long argued for the value of data-driven scales for classroom performance assessment, because they are specific to pedagogical tasks and objectives, have rich descriptors to offer useful…

Recent research on rater effects in performance assessments has increasingly focused on rater centrality, the tendency to assign scores clustering around the rating scale's middle categories. In the present paper, we adopted Jin and Wang's (2018) extended facets modeling approach and constructed a centrality continuum, ranging from raters…

Pearson's Learning Foundations describe the optimal conditions for learning and reflect the learner experience Pearson hopes their products will create. Pearson does this by incorporating the Learning Design Principles. Each of the Learning Design Principles goes into detail about a key principle, supporting product design and marketing by…

This research investigates the effects of rater experience on performance ratings in language testing using a systematic review of studies published from 1985 to 2017. Based on a comprehensive literature search of 14 databases, we identified sixteen relevant papers. With these we conducted a narrative review to conceptualize a theoretical…

Accomplishing a communication task in the real world requires the ability not only to do the task per se but also to manage aspects of the context in which it occurs. For this reason, simulations of target language use contexts have been incorporated into the design of communicative language tests as a way of enhancing the authenticity of…

Performance assessments heavily rely on human ratings. These ratings are typically subject to various forms of error and bias, threatening the assessment outcomes' validity and fairness. Differential rater functioning (DRF) is a special kind of threat to fairness manifesting itself in unwanted interactions between raters and performance- or…

This study introduces a novel differential item functioning (DIF) method based on propensity score matching that tackles two challenges in analyzing performance assessment data, that is, continuous task scores and lack of a reliable internal variable as a proxy for ability or aptitude. The proposed DIF method consists of two main stages. First,…

Many university students in Taiwan have complained that the general English class is not very exciting nor useful for the workplace due to its exam-oriented and teacher-centered instruction focus. Such negative impressions often lead to a low motivation toward learning English and affect students' learning outcomes. This study aimed to establish…

This qualitative study, conducted in Hokkaido, Japan, concerns the investigation of the teacher cognition and practice of English as a Foreign Language (EFL) assessment and the impact of a professional development (PD) program on the participants. The PD program was carried out as a series of seven in-service workshops with five native speakers of…

This study represents the inaugural attempt at assessing the authenticity of the tasks encompassed in the IELTS Speaking Module. The evaluation is conducted from the vantage points of applied linguistics and general education, and serves to enhance comprehension of authenticity and authentic assessment. In order to achieve this objective, an…

Researchers frequently evaluate rater judgments in performance assessments for evidence of differential rater functioning (DRF), which occurs when rater severity is systematically related to construct-irrelevant student characteristics after controlling for student achievement levels. However, researchers have observed that methods for detecting…

Summative assessments measure students' mastery of grade-level academic standards and skills in specific content areas after learning. State summative assessment systems provide students and families, school and district leaders, and state policymakers with valuable data to help understand student progress and school quality. This Policy Guide…

Assessment is one component of a learning process that cannot be excluded by an English teacher in the teaching-learning process. The form and type of assessment applied in the teaching-learning process are adapted to the orientation of learners' target outcomes. It is a space of reflection for teachers and students in the awareness of…

Task-Based Language Teaching has been developed in response to the teacher-dominated, focus-on-forms methods such as Present, Practice, Produce (PPP). The body of literature is replete with studies examining the learning efficacy of the PPP approach versus TBLT; however, these studies did not use assessment tasks in comparing these two methods. To…

There are several student performances assessed in Intensive English Programs (IEPs) worldwide in each academic year. These student performances are mostly graded by human raters with a certain degree of error. However, the accuracy of these performance assessments is of utmost importance because they feed data into some high stakes decisions…