This article studied two methods to detect mode effects in two credentialing exams. In Study 1, we used a "modal scale comparison approach," where the same pool of items was calibrated separately, without transformation, within two TC cohorts (TC1 and TC2) and one OP cohort (OP1) matched on their pool-based scale score distributions. The…

Critical thinking, problem solving and communication are fundamental elements of undergraduate education, but methods for assessing these skills across an institution are susceptible to logistical, motivational and financial issues. Queen's University conducted two research studies investigating the use of standardised tests to assess cognitive…

The Programme for International Student Assessment (PISA) is an important cross-national study of 15-year-olds' academic knowledge and skills. Educationalists and public policymakers eagerly await the tri-annual results, with particular interest in whether their country has moved up or slid down the international rankings, as compared to earlier…

The Programme for International Assessment (PISA) is an important cross-national study of 15-year olds academic achievement. Although it has traditionally been conducted using paper-and-pencil tests, the vast majority of countries will use computer-based assessment from 2015. In this paper, we consider how cross-country comparisons of children's…

Online quizzes have been shown to be effective learning and assessment approaches. However, if scenario-based online construction safety quizzes do not include time pressure similar to real-world situations, they reflect situations too ideally. The purpose of this paper is to compare engineering students' performance when carrying out an online…

As technology has become more advanced and accessible in instructional settings, there has been an upward trend in computer-based testing in the last decades. The present experimental study examines students' behaviors during computer-based testing in two different conditions and explores how these conditions affect the test results. Results…

This study investigates the use of pragmatic markers (PMs) by learners of English at varying proficiency levels. The study analyzes data from a university-level oral proficiency exam that categorized Chinese and Korean English-as-a-second-language (ESL) speakers into four proficiency levels and compares data with those of native speakers taking…

This article describes three experiments comparing paper-and-pencil tests (PPTs) to computer-based tests (CBTs) in terms of test method preferences and student performance. In Experiment 1, students took tests using three methods: PPT in class, CBT in class, and CBT at the time and place of their choosing. Results indicate that test method did not…

The interpretation of retest scores is problematic because they are potentially affected by measurement and predictive bias, which impact construct validity, and because their size differs as a function of various factors. This paper investigates the construct stability of scores on a figural matrices test and models retest effects at the level of…

This article investigates several questions regarding the impact of different item formats on measurement characteristics. Constructed response (CR) items and multiple choice (MC) items obviously differ in their formats and in the resources needed to score them. As such, they have been the subject of considerable discussion regarding the impact of…

An increasing number of K-12 school teachers have been using handheld, or palmtop, computers in the classroom as an integral means of facilitating education due to the flexibility, mobility, interactive learning capability, and comparatively inexpensive cost. This study involved two experiments in handheld computers: (a) a comparison of the…

A college administered the computer-adaptive ACCUPLACER (College Board, 1995) reading placement test to 399 entering students and its paper-and-pencil version, COMPANION, to 481 students. When the age of the two groups was held constant, no differences were found between the groups. (SLD)

The construction of parallel editions of conventional tests for purposes of test security while maintaining score comparability has always been a recognized and difficult problem in psychometrics and test construction. The introduction of new modes of test construction, e.g., adaptive testing, changes the nature of the problem, but does not make…

The degree to which laptop and standard-size desktop computers are likely to produce comparable test results for the Graduate Record Examination (GRE) General Test was studied. Verbal, quantitative, and writing sections of a retired version of the GRE were used, since it was expected that performance on reading passages or mathematics items might…

Three test administration procedures for making placement decisions in adult education were compared (paper-based, computer-based, and computerized-adaptive tests) with 90 adult-education students. Test performance was not differentially affected by the mode of administration, but the computerized adaptive test always yielded more precise ability…