This study evaluated a "substantively driven" method for scoring NAEP writing assessments automatically. The study used variations of an existing commercial program, e-rater[R], to compare the performance of three approaches to automated essay scoring: a "brute-empirical" approach in which variables are selected and weighted solely according to…

From 2005 to 2008, the International Development Research Centre (IDRC) is funding a series of distance education (DE) studies in its Pan Asia Networking (PAN) region. Eight collaborative projects are under way in Bhutan, Cambodia, China, Hong Kong, India, Indonesia, Laos, Mongolia, Pakistan, the Philippines, Sri Lanka, Thailand, and Viet Nam. The…

In the chapters presented in this volume, leading experts describe some of the assessment challenges in greater detail and provide federal recommendations on how to address them. In "College and Work Readiness as a Goal of High Schools: The Role of Standards, Assessments, and Accountability," John Tanner of the Center for Innovative Measures at…

Many students who are deaf or hard-of-hearing are eligible for a signing accommodation for state and other standardized tests. The signing accommodation, however, presents several challenges for testing programs that attempt to administer tests under standardized conditions. One potential solution for many of these challenges is the use of…

Background: This study examines the use of the Brief Child and Family Phone Interview (BCFPI) to screen for childhood psychiatric disorder based on Diagnostic Interview Schedule for Children Version IV (DISC-IV) classifications of attention-deficit hyperactivity disorder (ADHD), oppositional defiant disorder (ODD), conduct disorder (CD),…

Simulated data were used to investigate the performance of modified versions of the Mantel-Haenszel and standardization methods of differential item functioning (DIF) analysis in computer-adaptive tests (CATs). Each "examinee" received 25 items out of a 75-item pool. A three-parameter logistic item response model was assumed, and…

The General Computer Adaptive Test (CAT) of the Graduate Record Examinations (GRE) includes three operational sections that are separately timed and scored. A "no score" is reported if the examinee answers fewer than 80% of the items or if the examinee does not answer all of the items and leaves the section before time expires. The 80%…

A study was conducted to evaluate the effects of choice of item response theory (IRT) model, parameter calibration group, starting ability estimate, and stopping criterion on the conversion of an 80-item vocabulary test to computer adaptive format. Three parameter calibration groups were tested: (1) a group of 1,000 high school seniors, (2) a…

The use of indices and statistics based on nonlinear factor analysis (NLFA) has become increasingly popular as a means of assessing the dimensionality of an item response matrix. Although the indices and statistics currently available to the practitioner have been shown to be useful and accurate in many testing situations, few studies have…

The Office of Personnel Management (OPM) decided to explore alternative examining procedures for positions covered by the Administrative Careers with America (ACWA) examination. One requirement for new procedures was that they be automated for use with OPM's recently developed Microcomputer Assisted Rating System (MARS), a highly efficient system…

Technology has provided the public with the ability to steal the contents of multiple examinations in a short time period and at minimal cost. No examination is safe from compromise, and the only variable is the extent and sophistication used to bring about the compromise or theft. With the easy availability of micro-video equipment, audio…

Does a drawing embody the form and focus of what the artist actually sees, or instead, is it only after seeing the finished drawing that the artist knows the true meaning of his or her visual experience? It is the knowledge of the visual experience that drives the representation of it. Knowledge of the visual experience is present in varying…

Students taking the paper-based Scholastic Assessment Test (SAT) mathematics test are permitted to bring and use their own hand-held calculators, and this policy was continued for the computer-adaptive tests (CAT) designed for use in talent search programs. An on-screen calculator may also be used with the CAT. The bring-your-own option has raised…

The objectives of this study were to develop, implement, and evaluate the year long project, Microcomputer Adaptive Testing High-Risk Urban Students (MATH-R-US). The project produced diagnostic software to meet the following criteria: (1) help students obtain high school mathematics credit needed for graduation; (2) motivate students to learn…

The Work Personality Profile-Self Report (WPP-SR) is a self-report work behavior instrument for use in situational assessment in work centers, comprehensive facilities, and employment settings. The WPP-SR assesses work attitudes, values, habits, and behaviors that are essential to the achievement and maintenance of employment. It consists of 58…