Background: Digital-first assessments leverage the affordances of technology in all elements of the assessment process--from design and development to score reporting and evaluation to create test taker-centric assessments. Objectives: The goal of this paper is to describe the engineering, machine learning, and psychometric processes and…

Many studies have shown that using a computer-aided detection (CAD) system does not significantly improve diagnostic accuracy in radiology, possibly because radiologists fail to interpret the CAD results properly. We tested this possibility using screening mammography as an illustrative example. We carried out two experiments, one using 28…

With the iPad-mediated cognitive assessment gaining popularity with school districts and the need for alternative modes for training and instruction during this COVID-19 pandemic, school psychology training programs will need to adapt to effectively train their students to be competent in administering, scoring, an interpreting cognitive…

The Wechsler Individual Achievement Test (WIAT-4) is the latest iteration of a popular instrument that psychologists employ to assess academic achievement. The WIAT-4 authors make both pragmatic and measurement claims about the instrument. The pragmatic claims involve being useful for identifying individuals in certain academic achievement-related…

We investigated the ecological validity of performance measures from a computer-based assessment tool that utilises scripted video vignettes. The intended purpose of this tool is to assess the maintenance and repair skills of automotive technician apprentices, complementing traditional hands-on assessment formats from the German journeymen's…

Scholars have lamented that current methods of assessing student performance do not align with contemporary views of learning as situated within students, contexts, and time. Here, we introduce and describe one theoretical--psychometric paradigm--termed "dynamic measurement"--designed to provide a valid representation of the way students…

Given significant changes to legislation, practice, research, and instrumentation, the purpose of this study was to examine the course on cognitive assessment in school psychology programs and to describe the (a) structure, (b) instructional strategies, (c) content, and (d) interpretative strategies taught to school psychology graduate students.…

To help ensure that assessments are properly administered, states have developed policies about who can administer or proctor assessments, and the training they must have. This report presents the findings of our analysis of states' 2021 policies on the qualifications and requirements for test administrators, proctors, and accommodations providers…

The Woodcock-Johnson Tests of Achievement, Fourth Edition (WJ IV ACH) is purported to align with Cattell-Horn-Carroll (CHC) theory and offers upward of 20 scores within its interpretive and scoring system. The Technical Manual does not furnish validity evidence for the scores reported by the scoring system, suggesting that evidentiary support may…

Test anxiety has proliferated in the era of test-based accountability; however, there are limited tools that allow for consistent identification of students at risk. The present study reports on the psychometric evidence and continued development of the Multidimensional Test Anxiety Scale (MTAS). Evidence is presented to support both the…

Background/Context: Pass/fail standardized exams frequently selectively rescore failing exams and retest failing examinees. This practice distorts the test score distribution and can confuse those who do analysis on these distributions. In 2011, the Wall Street Journal showed large discontinuities in the New York City Regent test score…

As a replication in the Chinese context of previous aptitude testing validation studies, this research aims at exploring the validity of an interpreting aptitude test battery, composed of the Logical Memory Test (LMT), the Scale of Language Anxiety (SLA) and the Achievement Motivation Test (AMT), in the prediction of consecutive interpreting…

When the Differential Ability Scales--Second Edition (DAS-II) was developed, the instrument's content, structure, and theoretical orientation were amended. Despite these changes, the "Technical Handbook" did not report results from exploratory factor analytic investigations, and confirmatory factor analyses were implemented using…

High school chemistry teachers will often establish goals that guide assessment design and interpretation of assessment results. However, little is known about how these goals and the assessment design collectively guide the interpretation of results. This study seeks to better understand what teachers notice when interpreting assessment results…

We are developing the Equitable Screening to Support Youth (ESSY) Whole Child Screener to address concerns prevalent in existing school-based screenings that impede goals to advance educational equity using universal screeners. Traditional assessment development does not include end users in the early development phases, instead relying on a…