In this paper, we provide an overview of psychometric procedures and guidelines Educational Testing Service (ETS) uses to evaluate automated essay scoring for operational use. We briefly describe the e-rater system, the procedures and criteria used to evaluate e-rater, implications for a range of potential uses of e-rater, and directions for…

Automated Essay Scoring (AES) has garnered a great deal of attention from the rhetoric and composition/writing studies community since the Educational Testing Service began using e-rater[R] and the "Criterion"[R] Online Writing Evaluation Service as products in scoring writing tests, and most of the responses have been negative. While the…

This study investigated the attitudes of primary school students towards computer based testing and assessment in terms of different variables. The sample for this research is primary school students attending a computer based testing and assessment application via CITO-OIS. The "Scale on Attitudes towards Computer Based Testing and…

David Foster is the founder of Kryterion, an Internet test administration company, and currently serves there as chief scientist and executive vice president. He is the author of numerous articles for industry trade journals and textbooks and sits on the Council for the International Test Commission. In this interview, Foster talks about his…

Analysis of Qatar's standards-based student assessment system, the first in the region, offers several lessons for other nations instituting similar reforms. These include the need to coordinate on standards and assessment development, allow sufficient time for a fully aligned assessment, and communicate about the purposes and uses of testing.…

Pupil monitoring systems support the teacher in tailoring teaching to the individual level of a student and in comparing the progress and results of teaching with national standards. The systems are based on the availability of an item bank calibrated using item response theory. The assessment of the students' progress and results can be further…

This report evaluates the construct coverage of the e-rater[R[ scoring engine. The matter of construct coverage depends on whether one defines writing skill, in terms of process or product. Originally, the e-rater engine consisted of a large set of components with a proven ability to predict human holistic scores. By organizing these capabilities…

Recent advances in educational technologies and the wide-spread use of computers in schools have fueled innovations in test construction and analysis. As the measurement accuracy of a test depends on the quality of the items it includes, item selection procedures play a central role in this process. Mathematical programming and the item response…

Assessment has long been at the center of education policy debates, and for good reason. The goal of schooling is to maximize student learning, and assessments provide a picture of what students know and are able to do. Assessments also have a strong influence on what goes on in classrooms. The United States is now poised to make the most dramatic…

The world in which learning and assessment must take place is rapidly changing. The digital revolution has created a vast space of interconnected information, communication, and interaction. Functioning effectively in this environment requires so-called 21st century skills such as technological fluency, complex problem solving, and the ability to…

Psychometricians continue to introduce new approaches to setting cut scores for educational assessments in an attempt to improve on current methods. In this paper we describe the Item-Descriptor (ID) Matching method, a method based on IRT item mapping. In ID Matching, test content area experts match items (i.e., their judgments about the knowledge…

Conventional testing systems usually give students a score as their test result, but do not show them how to improve their learning performance. Researchers have indicated that students would benefit more if individual learning guidance could be provided. However, most of the existing learning diagnosis models ignore the fact that one concept…

The way in which examinations influence teaching and learning is called washback. This study examined the attitudes of 100 teachers towards the computerization of the English section of the Spanish University Entrance Examination paying special attention to the nature and scope of these changes, and the constraints and benefits. A 10 item…

Within large scale educational assessment agencies in the UK, there has been a shift towards assessors marking digitally scanned copies rather than the original paper scripts that were traditionally used. This project uses extended essay examination scripts to consider whether the mode in which an essay is read potentially influences the…

This report summarizes the twelfth survey of states by the National Center on Educational Outcomes (NCEO) at the University of Minnesota. Results are presented for all 50 states and 8 of the 11 unique states. The purpose of this report is to provide a snapshot of the new initiatives, trends, accomplishments, and emerging issues during this…