Efficient constructed response (CR) scoring requires both accuracy and speed from human raters. This study was designed to determine if setting scoring rate expectations would encourage raters to score at a faster pace, and if so, if there would be differential effects on scoring accuracy for raters who score at different rates. Three rater groups…

In this research report, we present an empirical argument for the use of a contributory scoring approach for the 2-essay writing assessment of the analytical writing section of the "GRE"® test in which human and machine scores are combined for score creation at the task and section levels. The approach was designed to replace a currently…

Essay scores generated by machine and by human raters are generally comparable; that is, they can produce scores with similar means and standard deviations, and machine scores generally correlate as highly with human scores as scores from one human correlate with scores from another human. Although human and machine essay scores are highly related…

With the pressing need for accountability in higher education, standardized outcomes assessments have been widely used to evaluate learning and inform policy. However, the critical question on how scores are influenced by students' motivation has been insufficiently addressed. Using random assignment, we administered a multiple-choice test and an…

Test results of 915 high school students taking a history examination with a choice of topics show that students were generally able to pick the topic on which they could get the highest score. Implications for fair scoring when topic choice is allowed are discussed. (SLD)

If a student writes two essays, the score reliability can be estimated from the correlation between essays. However, if the essays are in different modes or require different skills, the reliability may be underestimated from the correlation. In Advanced Placement history examinations, students wrote one standard essay and one essay that required…

Students from 38 colleges with high scores on an advanced placement examination essay and low scores on the multiple-choice portion were compared with students with the opposite pattern. The pattern was not related to college grades, but was related to other test performance. (SLD)

This study at 21 colleges found that having entering freshmen write a short holistically scored essay added nothing to the prediction of freshman grade point average compared to predictions made from high school grade point average, Scholastic Aptitude Test scores, and a multiple-choice test of writing-related skills. (Author/DB)

This study assessed the ability of high school advanced placement history students to choose the essay topic on which they can get the highest score. A second, equally important, question was whether the score on the chosen topic was more highly related to other indicators of proficiency in history than the score on the unchosen topic. Overall,…

Grades in college freshman English composition courses were predicted from high school rank in class, multiple-choice writing scores, essays, current Scholastic Aptitude Test (SAT) Verbal scores, and scores from a revised version of the SAT-Verbal. Data were obtained from 21 English courses at 17 different colleges with some supplementary data…

Essays for the Graduate Management Admissions Test must be written with a word processor (except in some foreign countries). The test sponsors, the Graduate Management Admissions Council, believed that this is fair because some word processing skill is a prerequisite for advanced management education. Furthermore, it might also be unfair to…

Examination of the correlation between multiple-choice and essay portions of the College Board Advanced Placement (AP) examinations with grades of first-year students from 32 colleges (largest sample=6,243) shows the best correlation for multiple choice tests for 2 examinations, with multiple choice and essay performing nearly equally for the…

The Advanced Placement Program reports grades to students and colleges on a 1 to 5 point scale derived from combining the separate scores on the multiple-choice and free-response sections of Advanced Placement Examinations (ADPEs). In this study, predictions of grades in first-semester college biology courses were made based on students' separate…

Essay and multiple-choice scores from Advanced Placement (AP) examinations in American History, European History, English Language and Composition, and Biology were matched with freshman grades in a sample of 32 colleges. Multiple-choice scores from the American History and Biology examinations were superior to essays for predicting overall grade…

Students with high scores (top third) on the essay portion of an Advanced Placement Examination (AP) (College Board) and low scores (bottom third) on the multiple-choice portion of the same examination were compared with students whose performance showed the opposite pattern. Across examinations in different subject areas (history, English, and…