This article explores the issue of identifying guessers -- with a specific focus on multiple-choice tests. Guessing has long been considered a problem due to the fact that it compromises validity. A test taker scoring higher than they should through guessing does not provide a picture of their actual ability. After an initial description of issues…

Bots are destined to dominate how humans interact with the internet of things that continues to grow around them. Despite their still budding intellectual capacity, major companies (e.g., Apple, Google and Amazon) have already placed (chat)bots at the centre of their flagship devices. (Chat)Bots currently fill the internet acting as guides,…

The calibration of test materials generally involves the interaction between empirical analysis and expert judgement. This paper explores the extent to which scale familiarity might affect expert judgement as a component of test validation in the calibration process. It forms part of a larger study that investigates the alignment of the…

This paper reports reactions by candidates to the use of online proctoring (OLP), 'invigilation', in the delivery of high-stakes English language examinations. The paper first sets the scene in terms of the move from face-to-face to online modes of delivery. It explores the challenges and benefits that both modes offer, in terms of accessibility,…

Over the past two decades awareness has been growing that English language teachers need good English language proficiency and professional training to teach effectively. To address this issue, a benchmark--minimum standards assessment test--was initiated, developed, and introduced in Hong Kong in 2001. Because more than a decade and a half has…

Background: The Common European Framework of Reference (CEFR) (Council of Europe 2001) has, over the past decade, come to be widely used as a reference tool for teaching, learning and assessment (Alderson 2002; North 2014). The focus of the current study is on scaling the China Standards of English (CSE) vocabulary descriptors for College English…

This study investigates the perceptions of stakeholders on the impact of a high-stakes assessment of English language teachers' proficiency--the minimum language standards Language Proficiency Assessment for Teachers (English) [LPATE], which was introduced in 2000. Given that the test has now been in place for 17 years, the study investigates the…

While there has been a proliferation of research on assessment for learning (AFL) over the past two decades, L2 writing assessment has tended to focus much more on assessment of learning (AOL) than AFL. This study seeks to investigate the implementation of AFL for EFL writing within an examination-driven AOL system in Hong Kong, its possible…

Background: This article reports a study into the double marking of Liberal Studies in Hong Kong. This is now a compulsory subject in Hong Kong's Years 10-12 curriculum which, when first examined in the new Hong Kong Diploma of Secondary Education in 2012, will increase its candidature from its current 3,300 to 80,000. Aims: To examine the…

This paper investigates the linguistic worth of current "chatbot" programs--software programs which attempt to hold a conversation, or interact, in English--as a precursor to their potential as an ESL (English as a second language) learning resource. After some initial background to the development of chatbots, and a discussion of the Loebner…

This paper describes a study comparing paper-based marking (PBM) and onscreen marking (OSM) in Hong Kong utilising English language essay scripts drawn from the live 2007 Hong Kong Certificate of Education Examination (HKCEE) Year 11 English Language Writing Paper. In the study, 30 raters from the 2007 HKCEE Writing Paper marked on paper 100…

This paper describes a study of the computer essay-scoring program BETSY. While the use of computers in rating written scripts has been criticised in some quarters for lacking transparency or lack of fit with how human raters rate written scripts, a number of essay rating programs are available commercially, many of which claim to offer comparable…

This paper investigates authentic writing through the use of wikis by Year 7 ESL learners in a secondary school in Hong Kong. The wikis were used as a collaborative writing platform to produce--with minimal input and support from their teachers--wiki content that describes the different facilities and features of their school. Over a period of two…

This article looks at Year 9 (age 13) ESL learners in a secondary school in Hong Kong producing--with minimal input and support from their teachers--their own story books, these being the final task outcome in a series of lessons focusing on creativity. Over a period of two months, as an integral part of their ESL lessons, groups of students…

Describes the design and implementation of a computer-based listening test--the "Text dictation." The article first discusses the importance of listening centering on a coherent text, rather than short listening fragments, and discusses drawbacks of listening programs wherein the required responses consist of noninteractive test types, such as…