Scoring essays is generally an exhausting and time-consuming task for teachers. Automated Essay Scoring (AES) facilitates the scoring process to be faster and more consistent. The most logical way to assess the performance of an automated scorer is by measuring the score agreement with the human raters. However, we provide empirical evidence that…

The continuous measurement of learning behavior or learning outcome by appropriate sensors is fundamental for the implementation of technology-based adaptive learning courses. An adaptive system needs such learning data to adapt the instruction to the needs of students. Depending on the learning objective, the sensors use information generated…

For the past few years, the authors have been working on the design and development of an online First Certificate in English (FCE) preparatory course and exam simulator in an attempt to provide a supplementary tool and resources for students aiming to achieve a B2 level of English. These materials have been designed in such a way that learners…

Technology has been part of the language learning process, both inside and outside the classroom, for decades, helping to bridge the gap between different contexts of learning. At the same time, the concept of lifelong-learning has widened our understanding of what learning is, what different forms it can take, and of the importance of considering…

Discusses the potential positive and negative effects computer-based testing could have on validity, reviews the literature on validation perspectives in computer-based testing, and suggests ways to evaluate the contributions of computer-based testing to more valid measurement practices. (SLD)

Today's college ACCUPLACER System's Computerized Placement Test (CPT) plays a major role for determining whether post-secondary students are ready for placement into Arithmetic, Elementary Algebra, Intermediate Algebra, or College Mathematics courses. It is widely used in most college systems in the U.S. This paper discusses some of the issues to…

The purpose of this project was to develop valid evaluation procedures for the selection of Container Equipment Operators (CEOs) in the shipping industry. A job analysis was conducted to identify the essential tasks of the CEO job. Site visits, a task inventory, and the determination of essential tasks were used in the job analysis. The skills and…

Recently, the Test of English as a Foreign Language (TOEFL) changed by including a writing section that gives the examinee an option between computer and handwritten formats to compose their responses. Unfortunately, this may introduce several potential sources of error that might reduce the reliability and validity of the scores. The seriousness…

Recently, the Test of English as a Foreign Language (TOEFL) changed by including a direct writing assessment where examinees choose between computer and handwritten composition formats. Unfortunately, examinees may have differential access to and comfort with computers; as a result, scores across these formats may not be comparable. Analysis of…

The purpose of this research was to establish, within the constraints of the methods presented, whether the computer is capable of scoring essays in much the same way that human experts rate essays. The investigation attempted to establish what was actually going on within the computer and within the mind of the rater and to describe the degree to…

This paper describes the implementation of a computer-based approach to scoring open-ended problem lists constructed to evaluate student and practitioner clinical judgment from real or simulated records. Based on 62 previously administered and scored problem lists, the program was written in BASIC for a Heathkit H11A computer (equivalent to DEC…

This paper describes the investigation of a computerized measure of reading rate as measured by the new Nelson-Denny Reading Test CD-ROM (1993). This study addressed three aspects of validity: (1) score comparability between reading rate measured by the computer version and the paper-and-pencil version; (2) concurrent validity with reading…

This study developed a prototype occupational interest inventory for academically disadvantaged/functionally illiterate adults. The reliability and validity of the prototype were determined by comparing the results with those of a standardized commercially-available occupational interest inventory. A counterbalance design was employed with 30…

Olae is a computer system for assessing student knowledge of physics, and Newtonian mechanics in particular, using performance data collected while students solve complex problems. Although originally designed as a stand-alone system, it has also been used as part of the Andes intelligent tutoring system. Like many other performance assessment…

To evaluate student learning in a computer-supported environment known as "GenScope," a system was developed for assessing students' understanding and learning of introductory genetics material presented in two developed GenScope instruments. Both quantitative and qualitative methods were used to address traditional evidential validity…