At the Orange County Public School District in Orlando, Florida, assessing reading skills among the youngest students used to be quite a process. Relying on rudimentary products such as paper and pencils, the strategy hinged on the bubble sheets teachers administered to students once a year. After teachers scored the exams, they sent them to the…

Research on computer-based writing evaluation has only recently focused on the potential for providing formative feedback rather than summative assessment. This study tests the impact of an automated essay scorer (AES) that provides formative feedback on essay drafts written as part of a series of online teacher education case studies. Seventy…

The attractiveness of computer-based tests (CBTs) is due largely to their capability to expand the ways we conduct testing. A relatively unexplored application, however, is actively using the computer to reduce construct-irrelevant variance while a test is being administered. This investigation introduces the effort-monitoring CBT, in which the…

Two local methods for observed-score equating are applied to the problem of equating an adaptive test to a linear test. In an empirical study, the methods were evaluated against a method based on the test characteristic function (TCF) of the linear test and traditional equipercentile equating applied to the ability estimates on the adaptive test…

Researchers studying monitoring accuracy currently use two different indexes to estimate accuracy: relative accuracy and absolute accuracy. The authors compared the distributional properties of two measures of monitoring accuracy using Monte Carlo procedures that fit within these categories. They manipulated the accuracy of judgments (i.e., chance…

Objective: The Accreditation Council for Graduate Medical Education and the Liaison Committee on Medical Education have mandated the transition from written global evaluation methods to competency-based assessments in resident and medical student training. Assessment of competency requires analysis of performance data from numerous sources. This…

The College of Business and Economics at the University of Idaho conducted a pilot study that used commercially available encryption software called Securexam to deliver computer-based examinations. A multi-step implementation procedure was developed, implemented, and then evaluated on the basis of what students viewed as valuable. Two key aspects…

Much has been written in recent years about advances in assessment technologies designed to aid decision making in the juvenile justice system. Adoption and implementation of this latest generation of actuarial tools, however, have lagged behind their development. Assessment in juvenile justice exemplifies the "science-practice gap" that…

Simulated data were used to investigate the performance of modified versions of the Mantel-Haenszel and standardization methods of differential item functioning (DIF) analysis in computer-adaptive tests (CATs). Each "examinee" received 25 items out of a 75-item pool. A three-parameter logistic item response model was assumed, and…

The General Computer Adaptive Test (CAT) of the Graduate Record Examinations (GRE) includes three operational sections that are separately timed and scored. A "no score" is reported if the examinee answers fewer than 80% of the items or if the examinee does not answer all of the items and leaves the section before time expires. The 80%…

A study was conducted to evaluate the effects of choice of item response theory (IRT) model, parameter calibration group, starting ability estimate, and stopping criterion on the conversion of an 80-item vocabulary test to computer adaptive format. Three parameter calibration groups were tested: (1) a group of 1,000 high school seniors, (2) a…

The use of indices and statistics based on nonlinear factor analysis (NLFA) has become increasingly popular as a means of assessing the dimensionality of an item response matrix. Although the indices and statistics currently available to the practitioner have been shown to be useful and accurate in many testing situations, few studies have…

The Office of Personnel Management (OPM) decided to explore alternative examining procedures for positions covered by the Administrative Careers with America (ACWA) examination. One requirement for new procedures was that they be automated for use with OPM's recently developed Microcomputer Assisted Rating System (MARS), a highly efficient system…

Technology has provided the public with the ability to steal the contents of multiple examinations in a short time period and at minimal cost. No examination is safe from compromise, and the only variable is the extent and sophistication used to bring about the compromise or theft. With the easy availability of micro-video equipment, audio…

Does a drawing embody the form and focus of what the artist actually sees, or instead, is it only after seeing the finished drawing that the artist knows the true meaning of his or her visual experience? It is the knowledge of the visual experience that drives the representation of it. Knowledge of the visual experience is present in varying…