In this paper, we describe subskills of visual communication based on the skill structure outlined in the Common European Framework of Visual Literacy. We have developed this Framework further through assessing the development of subskills related to visual communication in the "produce" and "respond" domains of CEFR-VC in…

We used multivariate generalizability theory to examine the reliability of hand-scoring and automated essay scoring (AES) and to identify how these scoring methods could be used in conjunction to optimize writing assessment. Students (n = 113) included subsamples of struggling writers and non-struggling writers in Grades 3-5 drawn from a larger…

Unlike the traditional multiple-choice (MC) format, the discrete-option multiple-choice (DOMC) format does not necessarily reveal all answer options to an examinee. The purpose of this study was to determine whether the reduced exposure of item content affects test security. We conducted an experiment in which participants were allowed to view…

Much psychological research depends on participants' diligence in filling out materials such as surveys. However, not all participants are motivated to respond attentively, which leads to unintended issues with data quality, known as careless responding. Our question is: how do different modes of data collection--paper/pencil, computer/web-based,…

With the use of computerized testing, ordinary assessments can capture both answer accuracy and answer response time. For the Canadian Programme for the International Assessment of Adult Competencies (PIAAC) numeracy and literacy subtests, person ability, person speed, question difficulty, question time intensity, fluency (rate), person fluency…

Although educators are using various ways in evaluating students' learning, traditional paper pencil final exams are usually held at the end of the school year to conclude the course. During the school suspension due to the COVID-19 outbreak, this traditional approach is no longer applicable to some schools. In this article, alternative final…

Administering exams in large enrollment courses is challenging and systems in place for accomplishing this task were upended in the spring of 2020 when a sudden transformation to online instruction and testing occurred due to the COVID-19 pandemic. In the following year, when courses remained online, approaches to improve exam security included…

Mobile-assisted Language Learning (MALL) has been widely adopted in teaching and learning, yet there has been scant research concerning speaking test adaptation. An emerging type of mobile application is designed to facilitate test takers' performance in a high-stakes speaking test (e.g., the International English Language Testing System (IELTS)).…

Classical automated test assembly (ATA) methods assume fixed and known coefficients for the constraints and the objective function. This hypothesis is not true for the estimates of item response theory parameters, which are crucial elements in test assembly classical models. To account for uncertainty in ATA, we propose a chance-constrained…

Teaching physiology can be challenging as students are initially required to understand basic and abstract concepts. Thus students typically view physiology as a "difficult" subject and place an emphasis on rote learning and memorization. Here, we attempted to address this knowledge gap by introducing a pedagogical intervention into the…

Forced-choice (FC) measures are becoming increasingly common in the assessment of personality for high-stakes testing purposes in both educational and organizational settings. Despite this, there has been relatively little research into the reliability of scores obtained from these measures, particularly when administered as a computerized…

Calibration of an item bank for computer adaptive testing requires substantial resources. In this study, we investigated whether the efficiency of calibration under the Rasch model could be enhanced by improving the match between item difficulty and student ability. We introduced targeted multistage calibration designs, a design type that…

This article explores the comparability of assessment tools under different format conditions. Prior studies have not considered the interaction of format and device on time to complete an assessment and have instead treated each of them separately with conflicting results. This study assesses, by linear regressions using web-based data, the…

The rapid shift to an online instructional mode necessitated by COVID-19 has been a formidable challenge for most institutions but particularly for those in which online learning was not widely implemented before the pandemic. Assessment has been an additional challenge, coupled with the accreditation requirements that higher educational…

Recent research has found that maintaining an item in working memory improves subsequent long-term memory performance. The present study explored the role of working memory consolidation in long-term recognition. Participants completed a stimuli-identification task followed by a surprise delayed recognition task. Participants first identified…