This article explores the comparability of assessment tools under different format conditions. Prior studies have not considered the interaction of format and device on time to complete an assessment and have instead treated each of them separately with conflicting results. This study assesses, by linear regressions using web-based data, the…

As more states move to universal computer-based assessments, an emergent issue concerns the effect that device type might have on student results. Although, several research studies have explored device effects, most of these studies focused on the differences between tablets and desktops/laptops. In the current study, we distinguish between…

Any body of research evolves over time. Previous understandings become more nuanced, ideas are supported or rebuked, and, eventually we arrive at a clearer view of the issue. The research on score comparability across computerized devices is no exception. CCSSO [Council of Chief State School Officers] and the Center for Assessment have published…

In this study, we investigate whether mobile device use in surveys can be predicted. We aim to identify possible motives for device use and build a model by drawing on theory from technology acceptance research and survey research. We then test this model with a Structural Equation Modeling approach using data of seven waves of the GESIS panel. We…

The purpose of this document is to provide information and advice to support states in meeting USED Peer Review Requirements related to demonstrating the comparability of test scores across various devices used for technology-based testing. The document is divided into four main sections. In the first section, we discuss and define the…

Evidence of comparability is generally needed whenever there are variations in the conditions of an assessment administration, including variations introduced by the administration of an assessment on multiple digital devices (e.g., tablet, laptop, desktop). This article is meant to provide a comprehensive examination of issues relevant to the…

The International Computer and Information Literacy Study (ICILS) is the first international comparative study that examines students' acquisition of computer and information literacy (CIL). The study assessed student CIL achievement in 18 countries and three benchmarking participants through a computer-based assessment administered to students in…

The definition of what it means to take a test online continues to evolve with the inclusion of a broader range of item types and a wide array of devices used by students to access test content. To assure the validity and reliability of test scores for all students, device comparability research should be conducted to evaluate the impact of…

To investigate possible iPad related mode effect, we tested 403 8th graders in Indiana, Maryland, and New Jersey under three mode conditions through random assignment: a desktop computer, an iPad alone, and an iPad with an external keyboard. All students had used an iPad or computer for six months or longer. The 2-hour test included reading, math,…

A dialog system or a conversational agent provides a means for a human to interact with a computer system. Dialog systems use text, voice and other means to carry out conversations with humans in order to achieve some objective. Most dialog systems are created with specific objectives in mind and consist of preprogrammed conversations. The primary…

When our focus is on assessment educators should work to value the nature of assessment. This paper presents a new approach to multiple-choice competency testing in mathematics education. The instrument discussed here reflects student competence, encourages self-regulatory learning behaviours and links content with current curriculum documents and…

Some brand names are used generically to describe an entire class of products that perform the same function. "Kleenex," "Xerox," "Thermos," and "Band-Aid" are good examples. The term "computerized adaptive testing" (CAT) is similar in that it is often applied uniformly across a diverse family of testing methods. Although the various members of…

This study investigated the online reading performances and the level of visual fatigue from the perspectives of non-native speaking students (NNSs). Reading on a computer screen is more visually more demanding than reading printed text. Online reading requires frequent saccadic eye movements and imposes continuous focusing and alignment demand.…

In addition to the Common Core State Standards (CCSS), California is implementing a new, online assessment system: the California Assessment of Student Performance and Progress (CAASPP). Field tests were conducted last spring and the system is being rolled out this year, amid concerns about whether schools are technologically prepared. Using…

Purpose: This study investigated whether computers are a useful tool in the assessment of people with aphasia (PWA). Computerized and traditionally administered versions of tasks were compared to determine whether (a) the scores were equivalent, (b) the administration was comparable, (c) variables such as age affected performance, and (d) the…