Scoring essays is generally an exhausting and time-consuming task for teachers. Automated Essay Scoring (AES) facilitates the scoring process to be faster and more consistent. The most logical way to assess the performance of an automated scorer is by measuring the score agreement with the human raters. However, we provide empirical evidence that…

As we move into the 21st century, educationalists are exploring the myriad of possibilities associated with Computer Based Assessment (CBA). At first glance this mode of assessment seems to provide many exciting opportunities in the mathematics domain, yet one must question the validity of CBA and whether our school systems, students and teachers…

This study developed a prototype occupational interest inventory for academically disadvantaged/functionally illiterate adults. The reliability and validity of the prototype were determined by comparing the results with those of a standardized commercially-available occupational interest inventory. A counterbalance design was employed with 30…

Olae is a computer system for assessing student knowledge of physics, and Newtonian mechanics in particular, using performance data collected while students solve complex problems. Although originally designed as a stand-alone system, it has also been used as part of the Andes intelligent tutoring system. Like many other performance assessment…

Describes the comparative approach to system evaluation used in a project which administered an online retrieval test to an experimental online catalog to produce data for evaluating effectiveness of a new subject access design. Discusses efforts used to ensure data reliability, and strategies the researchers can use to improve on the data…

A study determined the viability and effectiveness of the Computerized Competent Speaker Evaluation System. A total of 659 comments made by undergraduate students about the paper version of the Competent Speaker Speech Evaluation Form were subjected to two Q sorts. The 635 comments that emerged were incorporated into the evaluation system during…

Problems of validity and reliability of concept mapping are addressed by using item-response theory (IRT) models for scoring. In this study, the overall structure of students' concept maps are defined by the number of links, the number of hierarchies, the number of cross-links, and the number of examples. The study was conducted with 92 students…

In the spring of 1991 the first full-scale National Household Education Survey (NHES:91) was conducted for the National Center for Education Statistics (NCES). The NHES:91 was a national random digit dial telephone survey of about 14,000 parents of 3- to 8-year-old children concerning the educational experiences of young children. A reinterview…