National Assessment of Educational Progress writing sample essays from 1988 and 1990 (495 and 599 essays) were subjected to computerized grading and human ratings. Cross-validation suggests that computer scoring is superior to a two-judge panel, a finding encouraging for large programs of essay evaluation. (SLD)

Item response theory (IRT) has become one of the most popular scoring frameworks for measurement data. IRT models are used frequently in computerized adaptive testing, cognitively diagnostic assessment, and test equating. This article reviews two of the most popular software packages for IRT model estimation, BILOG-MG (Zimowski, Muraki, Mislevy, &…

In computer-based interactive environments meant to support learning, students must bring a wide range of relevant knowledge, skills, and abilities to bear jointly as they solve meaningful problems in a learning domain. To function effectively as an assessment, a computer system must additionally be able to evoke and interpret observable evidence…

A new sharing item response theory (SIRT) model is presented that explicitly models the effects of sharing item content between informants and test takers. This model is used to construct adaptive item selection and scoring rules that provide increased precision and reduced score gains in instances where sharing occurs. The adaptive item selection…

Modern applications of computerized adaptive testing (CAT) are typically grounded in item response theory (IRT; Lord, 1980). While the IRT foundations of adaptive testing provide a number of approaches to adaptive test scoring that may seem natural and efficient to psychometricians, these approaches may be more demanding for test takers, test…

This paper describes the implementation of a computer-based approach to scoring open-ended problem lists constructed to evaluate student and practitioner clinical judgment from real or simulated records. Based on 62 previously administered and scored problem lists, the program was written in BASIC for a Heathkit H11A computer (equivalent to DEC…

FITNESSGRAM, a computerized program for scoring and reporting students' physical fitness status based on specific tests, has been a resounding success in the Tulsa, Oklahoma, public schools. The program provides feedback to students and permits evaluation of physical fitness efforts on a school-by-school basis. (PP)

Research on automated scoring is extended by comparing alternative automated systems for scoring a computer simulation of physicians' patient management skills. A regression-based system is more highly correlated with experts' evaluations than a system that uses complex rules to map performances into score levels, but both approaches are feasible.…

Comparison of special education teacher (n=18) and computer-scored curriculum-based measurements (CBM) of spelling found that computer scoring accuracy was significantly higher and more stable. Additionally, high correlations were found between the CBM spelling scores and a standardized test of spelling achievement. (DB)

This article reports the results of an investigation of the convergent criterion-related validity of two computer-based tools for scoring concept maps and essays as part of the ongoing formative evaluation of these tools. In pairs, participants researched a science topic online and created a concept map of the topic. Later, participants…

This paper reviews the role of ICT-based assessment in the light of the growing use of virtual learning environments in universities. Issues of validity, efficiency, type of response, and scoring are addressed. A major area of research is the automated scoring of text. Claims for automated formative assessment are queried, since the feedback of…

Because technology is more prevalent and accessible for use in assessment, this article highlights what counselors and educators need to know when considering the use of computers and the Internet for that purpose. The article concludes with some predictions on how technology might influence assessment and accountability in the future. This…

The current research was conducted to investigate the validity of automated essay scoring (AES) by comparing group mean scores assigned by an AES tool, IntelliMetric [TM] and human raters. Data collection included administering the Texas version of the WriterPlacer "Plus" test and obtaining scores assigned by IntelliMetric [TM] and by…

Following the publication of the third edition Wechsler scales (i.e., WAIS-III and WMS-III), demographically corrected norms were made available in the form of a computerized scoring program (i.e., WAIS-III/WMS-III/WIAT-II Scoring Assistant). These norms correct for age, gender, ethnicity, and education. Since then, four new indexes have been…

Issues concerning the cost and feasibility of using Formulating Hypotheses (FH) test item types for the Graduate Record Examinations have slowed research into their use. This project focused on two major issues that need to be addressed in considering FH items for operational use: the costs of scoring and the assignment of scores along a range of…