While automated essay scoring (AES) can reliably grade essays at scale, automated writing evaluation (AWE) additionally provides formative feedback to guide essay revision. However, a neural AES typically does not provide useful feature representations for supporting AWE. This paper presents a method for linking AWE and neural AES, by extracting…

We present an empirical study on the use of keystroke analytics to capture and understand how writers manage their time and make inferences on how they allocate their cognitive resources during essay writing. The results suggest three distinct longitudinal patterns of writing process that describe how writers approach an essay task in a writing…

This study examines the effectiveness of an automated translation chatbot used in online interactions which consequently could enhance second/foreign language (L2) competence. Based on the sociocultural perspectives of learning, such as communication to recognize the difference from others and to be involved in sense-making processes, this study…

The vast majority of second language teachers feels confident about their instructional performance and does not usually have much difficulty with their teaching thanks to their professional training and accumulated classroom experience. Nonetheless, many second language teachers may not have received sufficient training in test development to…

This paper presents the results of writing processing among 260 high school senior students, their degree of satisfaction using the new trial version of the Computer Based University Entrance Examination in Spain and their degree of motivation towards written online test tasks. Currently, this is one of the closing studies to verify whether…

A new method for evaluating rater accuracy within the context of performance assessments is described. It uses an extended Rasch measurement model, FACETS, which is illustrated with 373 benchmark papers from the Georgia High School Graduation Writing Test rated by 20 operational raters and an expert panel. (SLD)

A procedure for creating a numerical index which can be used to score and rank summaries of prose passages is described. The quality of a summary is assumed to be a function of two variables: the number of important ideas mentioned, and the number of words used. Scoring involves multiplying two ratios: a power ratio and an efficiency ratio. The…

A study examined the reaction of college freshmen to 30 essay test prompts to determine if rhetorical factors in the prompt would affect a student's choice of that prompt. The subjects, 40 college freshman enrolled in an introductory English course, rated eight rhetorical qualities of the essay prompts: (1) whether the essay would be easy to…

A central aim in any assessment program is to ensure fair and stable scoring from administration to administration. When administrations are decentralized, not only in location, but in frequency and in logistical configuration, it is imperative to construct training, certifying, and monitoring systems that provide continuity between the original…

Two variables directly related to understanding and improving notetaking practices and instructional strategies were investigated. The first variable investigated was the different levels of pretraining: pretraining vs. no pretraining; and the second was the influence of three types of test mode expectations: multiple choice, essay, or…

To ensure fairness, it is of critical importance that testing programs make sure that essay items given to examinees are equivalent in difficulty. The purpose of this study was to evaluate the stability and accuracy of a logistic regression based polytomous essay difficulty index. Preliminary results from a simulation study (9 conditions with a…

The assumptions behind secondary school literature course tests--whether asking students to recall aspects of literary works, to relate literary works to each other, or to analyze unfamiliar literary works--are open to question. They fail to acknowledge some of the most important aspects of literature which, if properly taught, should provide a…

This study investigated the practice of weighting a type of test item, such as constructed response, more than other types of items, such as selected response, to compute student scores for a mixed-item type of test. The study used data from statewide writing field tests in grades 3, 5, and 8 and considered two contexts, that in which a single…

Unstable, and potentially invalid, variance component estimates may result from using only a limited portion of available data from operational performance assessments. However, missing observations are common in these settings because of the nature of the assessment design. This paper describes a procedure for overcoming the computational and…

Free-response (FR) item formats, such as essay questions, are popular in educational assessment. The criticisms against FR items are that they are more expensive to score, take up more testing time, provide less content coverage, and are less reliable than multiple-choice (MC) items. For these reasons, FR items are often combined with MC items.…