In response to the COVID-19 pandemic, the American Board of Surgery (ABS) shifted from in-person to remote administrations of the oral certifying exam (CE). Although the overall exam architecture remains the same, there are a number of differences in administration and staffing costs, exam content, security concerns, and the tools used to give the…

In this digital ITEMS module, Dr. Sue Lottridge, Amy Burkhardt, and Dr. Michelle Boyer provide an overview of automated scoring. Automated scoring is the use of computer algorithms to score unconstrained open-ended test items by mimicking human scoring. The use of automated scoring is increasing in educational assessment programs because it allows…

Testing organization needs large numbers of high-quality items due to the proliferation of alternative test administration methods and modern test designs. But the current demand for items far exceeds the supply. Test items, as they are currently written, evoke a process that is both time-consuming and expensive because each item is written,…

Research on psychometric methods is heavily dependent on software. The quality, availability, and documentation of such software are critical to the advancement of the field. In 2000, an ad hoc committee of NCME recommended that NCME adopt policies that promote greater availability and better documentation of software. This article follows the ad…

Though the most widely used "Intelligence" tests have changed little in the last half century, the author predicts a variety of new influences of future intelligence testing, ranging from the ways we conceptualize intelligence, to the manner in which we pose problems to assess it. (Author/JAZ)

The attractiveness of computer-based tests (CBTs) is due largely to their capability to expand the ways we conduct testing. A relatively unexplored application, however, is actively using the computer to reduce construct-irrelevant variance while a test is being administered. This investigation introduces the effort-monitoring CBT, in which the…