This article studied two methods to detect mode effects in two credentialing exams. In Study 1, we used a "modal scale comparison approach," where the same pool of items was calibrated separately, without transformation, within two TC cohorts (TC1 and TC2) and one OP cohort (OP1) matched on their pool-based scale score distributions. The…

Some schools use results from the MAP® Growth™ interim assessments from the Northwest Evaluation Association (NWEA®) in a number of high-stakes ways. These include as a component of their teacher evaluation systems; to determine whether a student advances to the next grade; or as an indicator for student readiness for certain programs or…

Critical thinking, problem solving and communication are fundamental elements of undergraduate education, but methods for assessing these skills across an institution are susceptible to logistical, motivational and financial issues. Queen's University conducted two research studies investigating the use of standardised tests to assess cognitive…

The Programme for International Student Assessment (PISA) is an important cross-national study of 15-year-olds' academic knowledge and skills. Educationalists and public policymakers eagerly await the tri-annual results, with particular interest in whether their country has moved up or slid down the international rankings, as compared to earlier…

The Programme for International Assessment (PISA) is an important cross-national study of 15-year olds academic achievement. Although it has traditionally been conducted using paper-and-pencil tests, the vast majority of countries will use computer-based assessment from 2015. In this paper, we consider how cross-country comparisons of children's…

This article investigates the persistent and change elements of educational testing and assessment from 1920 to the present day. I show by examining the addresses and texts of American Educational Research Association presidents a continuing focus on schools, from early experiments and development up through applications in accountability systems.…

This article describes three experiments comparing paper-and-pencil tests (PPTs) to computer-based tests (CBTs) in terms of test method preferences and student performance. In Experiment 1, students took tests using three methods: PPT in class, CBT in class, and CBT at the time and place of their choosing. Results indicate that test method did not…

Online quizzes have been shown to be effective learning and assessment approaches. However, if scenario-based online construction safety quizzes do not include time pressure similar to real-world situations, they reflect situations too ideally. The purpose of this paper is to compare engineering students' performance when carrying out an online…

The interpretation of retest scores is problematic because they are potentially affected by measurement and predictive bias, which impact construct validity, and because their size differs as a function of various factors. This paper investigates the construct stability of scores on a figural matrices test and models retest effects at the level of…

As technology has become more advanced and accessible in instructional settings, there has been an upward trend in computer-based testing in the last decades. The present experimental study examines students' behaviors during computer-based testing in two different conditions and explores how these conditions affect the test results. Results…

Assessing the comparability of different groups is an issue facing many researchers and evaluators in a variety of settings. Commonly, null hypothesis significance testing (NHST) is incorrectly used to demonstrate comparability when a non-significant result is found. This is problematic because a failure to find a difference between groups is not…

This dissertation research examined the changes in item RIT calibration that occurred when adding audio to a set of currently calibrated RIT items and then placing these new items as field test items in the modified assessments on the NWEA MAP test platform. The researcher used test results from over 600 students in the Poway School District in…

This article investigates several questions regarding the impact of different item formats on measurement characteristics. Constructed response (CR) items and multiple choice (MC) items obviously differ in their formats and in the resources needed to score them. As such, they have been the subject of considerable discussion regarding the impact of…

Many postsecondary institutions currently administer standardized tests of general college outcomes; more than a quarter of Association of American Colleges and Universities (AAC&U) member institutions do so. Using standardized tests for accountability purposes has been contentious mainly because these tests do not measure every important…

This study investigates the use of pragmatic markers (PMs) by learners of English at varying proficiency levels. The study analyzes data from a university-level oral proficiency exam that categorized Chinese and Korean English-as-a-second-language (ESL) speakers into four proficiency levels and compares data with those of native speakers taking…