In this research report, we describe the design and empirical findings for a large-scale study of essay writing ability with approximately 2,500 high school students in Germany and Switzerland on the basis of 2 tasks with 2 associated prompts, each from a standardized writing assessment whose scoring involved both human and automated components.…

Many assessments of writing proficiency that aid in making high-stakes decisions consist of several essay tasks evaluated by a combination of human holistic scores and computer-generated scores for essay features such as the rate of grammatical errors per word. Under typical conditions, a summary writing score is provided by a linear combination…

The paper reports the results of a phenomenographic study on academics' conceptions of e-assessment. A cohort of 21 academics from 17 disciplines participated in semi-structured interviews exploring their experiences of using web-based technologies for formative and summative assessment purposes. Through iterative analysis of the interview…

Critical thinking, problem solving and communication are fundamental elements of undergraduate education, but methods for assessing these skills across an institution are susceptible to logistical, motivational and financial issues. Queen's University conducted two research studies investigating the use of standardised tests to assess cognitive…

The interest in, and use of, computers and software for assessment is reported to be increasingly popular via electronic examinations (e-exams). We deepen our understanding of the design, reception, and effectiveness of e-exams for history and philosophy of science modules, undertaken by first-year advanced science and medical science students at…

Assessing job applicants' general mental ability online poses psychometric challenges due to the necessity of having brief but accurate tests. Recent research (Myszkowski & Storme, 2018) suggests that recovering distractor information through Nested Logit Models (NLM; Suh & Bolt, 2010) increases the reliability of ability estimates in…

This research proposes the development of a web-based assessment system to identify students' misconception. The system, named WAS (web-based assessment system), can identify students' misconception profile on linear kinematics automatically after the student has finished the test. The test instrument was developed and validated. Items were…

This article provides an overview and evaluation of the uses--actual and potential--of automatic speech recognition (ASR) and spoken dialogue systems (SDS), related technologies that can be applied to second language speaking assessment, given particular definitions of the construct. Both technologies have only gradually moved in the direction of…

The Discrete Option Multiple Choice (DOMC) item format was introduced by Foster and Miller (2009) with the intent of improving the security of test content. However, by changing the amount and order of the content presented, the test taking experience varies by test taker, thereby introducing potential fairness issues. In this paper we…

We compared students' performance on a paper-based test (PBT) and three computer-based tests (CBTs). The three computer-based tests used different test navigation and answer selection features, allowing us to examine how these features affect student performance. The study sample consisted of 9,698 fourth through twelfth grade students from across…

The Common Core State Standards (CCSS) for ELA/Literacy, and the standardized assessments that test students on these standards, focus on students' ability to navigate online text and require students to comprehend, analyze, and make inferences about complex texts in both print and digital format. While the ability to read digital text with a high…

Substantial progress has been made toward applying technology enhanced conversation-based assessments (CBAs) to measure the English-language proficiency of English learners (ELs). CBAs are conversation-based systems that use conversations among computer-animated agents and a test taker. We expanded the design and capability of prior…

English is required as the official working language for oceangoing seafarers across the world. In China, much time and effort has been dedicated to improving maritime students' Maritime English (ME). However, low English proficiency is still one of the main barriers for Chinese seafarers to compete in the international maritime labor market.…

The study focuses on the process of assessing (a micro-skill) goal attainment in EFL pronunciation course via measures which can foster different students' attitudes and self-perceptions. Standard (pen-and-paper) tests offer immediate evidence of success but they put heavy demands on students' cognitive, performance and stress-controlling skills.…

This action research study presents the findings of using a formative assessment strategy in an online mathematic course during the world-wide outbreak of COVID-19 at the University of Passau, Germany. The main goals of this study were: (1) to enhance students' self-regulated learning by shifting the direction of assessment from instructors to the…