In this paper, we provide an overview of psychometric procedures and guidelines Educational Testing Service (ETS) uses to evaluate automated essay scoring for operational use. We briefly describe the e-rater system, the procedures and criteria used to evaluate e-rater, implications for a range of potential uses of e-rater, and directions for…

The purpose of this study was to investigate pre-service science teachers' understanding of surface tension, cohesion and adhesion forces by using computer-mediated predict-observe-explain tasks. 22 third-year pre-service science teachers participated in this study. Three computer-mediated predict-observe-explain tasks were developed and applied…

Automated Essay Scoring (AES) has garnered a great deal of attention from the rhetoric and composition/writing studies community since the Educational Testing Service began using e-rater[R] and the "Criterion"[R] Online Writing Evaluation Service as products in scoring writing tests, and most of the responses have been negative. While the…

A simultaneous equating of four new test forms to each other and to one previous form was accomplished through a complex design incorporating seven separate equating links. Each new form was linked to the reference form by four different paths, and each path produced a different score conversion. The procedure used to resolve these inconsistencies…

This study compared higher education dental undergraduate student performance in online assessments with performance in traditional paper-based tests and investigated students' perceptions of the fairness and acceptability of online tests, and showed performance to be comparable. The project design involved two parallel cross-over trials, one in…

This brief research report gives an overview of a visually based methodology that has recently been used to shed light on situated decision-making in a high stakes educational assessment environment. Looking carefully at the feedback interactions taking place between senior and less senior examiners, the project aim was to gain insight into the…

David Foster is the founder of Kryterion, an Internet test administration company, and currently serves there as chief scientist and executive vice president. He is the author of numerous articles for industry trade journals and textbooks and sits on the Council for the International Test Commission. In this interview, Foster talks about his…

Online learning has been rapidly developing in the last decade. However, there is very little literature available about the actual adoption of online synchronous assessment approaches and any guidelines for effective assessment design and implementation. This paper aims at designing and evaluating the possibility of applying online synchronous…

This study aims to determine the problems faced by students taking an online exam during a graduate degree program administered by the Distance Education Research and Application Center, KTU (Karadeniz Technical University). The research was conducted with 106 students during the spring semester of the 2009-2010 academic years in the department of…

With the rapidly increasing entrance examinations for the School of Continuing education of CUGB, there is an urgent need of the examination papers that have the parallel testing ability for the same academic program. Based on the analysis of parallel factors and the comparison of the manual generated mode and the computer generated mode, the most…

The aim of the present research was to compare the use of multiple-choice questions (MCQs) as an examination method, to the examination based on constructed-response questions (CRQs). Despite that MCQs have an advantage concerning objectivity in the grading process and speed in production of results, they also introduce an error in the final…

After surveying 1,827 students in their final year at eighty randomly selected two-year and four-year public and private institutions, American Institutes for Research (2006) reported that approximately 30 percent of students in two-year institutions and nearly 20 percent of students in four-year institutions have only basic quantitative…

Although much has been said about ergonomics in interface and in computer tools and interface design, very few articles in major journals have addressed this topic in relation to language testing. This article describes an experiment carried out at the Polytechnic University of Valencia, Spain, in which 27 Media and Communication students provided…

This report evaluates the construct coverage of the e-rater[R[ scoring engine. The matter of construct coverage depends on whether one defines writing skill, in terms of process or product. Originally, the e-rater engine consisted of a large set of components with a proven ability to predict human holistic scores. By organizing these capabilities…

The national certification tests and entrance examinations are the most important tests for proving the ability or knowledge level of a person. To accurately evaluate the professional skills or knowledge level, the composed test sheets must meet multiple assessment criteria such as the ratio of relevant concepts to be evaluated and the estimated…