This paper proposes to use depth perception to represent raters' decision in holistic evaluation of ESL essays, as an alternative medium to conventional form of numerical scores. The researchers verified the new method's accuracy and inter/intra-rater reliability by inviting 24 ESL teachers to perform different representations when rating 60…

It is widely believed that human rating performance is influenced by an array of different factors. Among these, rater-related variables such as experience, language background, perceptions, and attitudes have been mentioned. One of the important rater-related factors is the way the raters interact with the rating scales. In particular, how raters…

College and university admissions officers are the gatekeepers who decide which individuals may join the "country club" of higher education. As such, the decisions they make have a significant impact on the lives of individuals, especially those who are willing to travel across the globe and risk living in an unfamiliar country in order…

The paper evaluates the ELT textbook "English for Today" (Classes Nine--Ten) which is used as a dominant instructional material in secondary level schools and madrasahs in Bangladesh. The National Curriculum and Textbook Board (NCTB) of Bangladesh published the revised edition of the textbook series "English for Today" in the…

The current assessment in language classrooms prevailingly utilizes the criteria provided by instructors, regarding learners as passive recipients of assessment. The current study drew upon sustainable assessment and the community of practice to highlight the importance of involving learners in co-constructing the assessment criteria and argued…

This literature review attempts to survey representative studies within the context of L2 speaking assessment that have contributed to the conceptualization of rater cognition. Two types of studies are looked at: 1) studies that examine "how" raters differ (and sometimes agree) in their cognitive processes and rating behaviors, in terms…

Summative assessment of interpretation is widely conducted in interpreting courses/programs to inform high-stakes decision making, such as the selection, certification, and conferral of academic degrees. Yet there has been very limited empirical research to investigate the score dependability of summative interpretation assessment. The present…

An assumption underlying speaking tests is that scores reflect the ability to produce online, non-rehearsed speech. Speech produced in testing situations may, however, be less spontaneous if extensive test preparation takes place, resulting in memorized or rehearsed responses. If raters detect these patterns, they may conceptualize speech as…

Criterion referenced tests of second language speaking performance are administered in different institutions using different procedures. The present study reports raters' practices of second language speaking tests, in particular the correspondence between test-takers' grades when assessed individually and in groups. Data derived from…

This study investigated teachers' grading decision making, focusing on their beliefs underlying their grading decision making, their grading practices and assessment types, and factors they considered in grading decision making. Two teachers from two junior high schools applying different curriculum policies in grade reporting in Indonesian…

Criticism of specific-purpose language (LSP) tests is often directed at their limited ability to represent fully the demands of the target language use situation. Such criticisms extend to the criteria used to assess test performance, which may fail to capture what matters to participants in the domain of interest. This paper reports on the…

The growing recognition of the plurality of English underlying the World Englishes (WE) paradigm has problematised the conventional second language acquisition (SLA) views of errors. If English use in emerging English-speaking contexts is to be judged by local norms, as argued by WE scholars, applying exocentric norms in these contexts can be…

This study demonstrates the application of the analytic hierarchy process (AHP) in English language teaching materials evaluation, focusing in particular on its potential for systematically integrating different components of evaluation criteria in a variety of teaching contexts. AHP is a measurement procedure wherein pairwise comparisons are made…

This paper, fundamentally a procedural paper, outlines pedagogical practices which others may want to adapt for their own purposes. We demonstrate how a strong form of task-based language learning and teaching (TBLT), classroom implementation of the CEFR, cyclical learning, and assessment fulfilling roles, such as informing the next learning…

Various factors contribute to variability in English as a second language (ESL) essay scores and rating processes. Most previous research, however, has focused on score variability in relation to task, rater, and essay characteristics. A few studies have examined variability in essay rating processes. The current study used think-aloud protocols…