In recent years, Automatic Item Generation (AIG) has increasingly shifted from theoretical research to operational implementation, a shift raising some unforeseen practical challenges. Specifically, generating high-quality answer choices presents several challenges such as ensuring that answer choices blend in nicely together for all possible item…

When the coronavirus pandemic was declared a public health crisis in March 2020, school psychologists were forced into situations where face-to-face interaction with their students was discouraged and in some cases, prohibited. Consequently, the traditional practice of school psychology abruptly ended. Individualized Education Plans (IEP) and…

U.S. Supreme Court justices and other federal judges are, effectively, appointed for life, with no built-in check on their cognitive functioning as they approach old age. There is about a century of research on aging and intelligence that shows the vulnerability of processing speed, fluid reasoning, visual-spatial processing, and working memory to…

This article discusses critical methodological design decisions for collecting, interpreting, and synthesizing empirical evidence during the design, deployment, and operational quality-control phases for automated scoring systems. The discussion is inspired by work on operational large-scale systems for automated essay scoring but many of the…

The Partnership for Assessment of Readiness for College and Careers (PARCC) is a group of states working together to develop a set of assessments that measure whether students are on track to be successful in college and their careers. These high-quality, computer-based K-12 assessments in mathematics and English language arts/literacy (ELA/L)…

The Partnership for Assessment of Readiness for College and Careers (PARCC) is a group of states working together to develop a set of assessments that measure whether students are on track to be successful in college and their careers. These high-quality, computer-based K-12 assessments in mathematics and English language arts/literacy (ELA/L)…

The COVID-19 pandemic has given the entire country a crash course in virtual education and digital education platforms. School leaders have been forced to consider a set of very imperfect options as they struggle to reopen their schools. This three-brief collection identifies key issues for school leaders to consider before adopting a digital…

The Partnership for Assessment of Readiness for College and Careers (PARCC) is a group of states working together to develop a set of assessments that measure whether students are on track to be successful in college and their careers. These high-quality, computer-based K-12 assessments in mathematics and English language arts (ELA)/literacy give…

This report describes how big data and an evidence framework can align across five contexts of educational improvement. It explains that before working with big data, there is an important prerequisite: the proposed innovation should align with deeper learning objectives and should incorporate sound learning sciences principles. New curriculum…

Accommodations are an important part of paper and pencil testing. They give students with disabilities and English language learners (ELLs) access to the assessment and provide results that have greater validity for those students. This Brief addresses the need to think carefully about accommodations when moving from paper-based assessments to…

Cognitive readiness (CR) and performance for operational time-critical environments are continuing points of focus for military and academic communities. In response to this need, we designed an open source interactive CR assessment application as a highly adaptive and efficient open source testing administration and analysis tool. It is capable…

Although the term data-driven decision making (DDDM) is relatively new (Moss, 2007), the underlying concept of DDDM is not. For example, the practices of formative assessment and computer-managed instruction have historically involved the use of student performance data to guide what happens next in the instructional sequence (Morrison, Kemp, &…

The article examines decision-making expert systems and discusses their implications for diagnosis and prescription of reading difficulties. A detailed description of how a reading diagnostic expert system might operate to aid classroom teachers is followed by a discussion of advantages and limitations of expert systems for educational use.…

Describes a computer-based method of confidence-testing, available in batch processing and interactive form, which improves a student's ability to assess probabilities during clinical diagnosis. The methods and results of three experiments are presented. (EAO)

Three developments over the past 15 years--our understanding of cognition, new techniques in technical psychometrics, and the computer explosion--form the basis for a proposal for new intelligence tests. These developments are reviewed, and both leaps and steps in intellectual assessment are proposed. (Author/RD)