Two aspects of rater accuracy in performance assessment were analyzed: rater stringency/leniency, and rater consistency. Data were obtained from three administrations of an oral certification examination in a health profession. The examination consists of clinical cases in four content areas or subspecialities. A total of 364 candidates were…

Theory and implications for methods of assessing administrative performance in simulated exercises are presented. The rationale is given for the following: (1) developing simulated exercises; (2) measuring behaviors exhibited during the exercises; (3) training evaluators; (4) combining information across exercises; and (5) storing and retrieving…

This handbook, intended for English language arts teachers, principals, curriculum and testing specialists, superintendents, and all educators interested in preparing students for the grade 8 California Assessment Program (CAP) writing assessment, provides the practitioner with information about the content of the test, the rationale underlying…

The results of classifying test items on the basis of their Mantel-Haenszel (MH) alpha estimates were compared to the results of classifying these items using an item response theory (IRT) based procedure involving the comparison of item difficulties in the interest of identifying the alpha value that maximized the decision concordance between the…

Information is provided regarding major learning styles and other factors important to student learning. Several typically asked questions are presented regarding different learning styles (visual, auditory, tactile and kinesthetic, and multisensory learning), associated considerations, determining individuals' learning styles, and appropriate…

The Center for the Study of Evaluation's (CSE) expository and narrative rating scales have been developed to meet the need for instructionally relevant methods for assessing students' writing competence. Research indicates that large numbers of raters can be trained in the use of these scales and that, during training and independent rating, they…

This study investigates the strength of relationships between direct measures (holistic scores for free writing samples) and indirect measures (the Test of Ability to Subordinate) of attained writing proficiency in English as a second language. Subjects were 67 undergraduate foreign language students enrolled in Basic Composition for Foreign…

The Registered Holistic Scoring Method, which has been used for one year to score the ninth grade writing test of the New Jersey High School Proficiency Test, is described. Registered Holistic Scoring was developed from the previous holistic approach in order to provide more reliable scoring guidelines year after year. Two trained evaluators…

The most appropriate statistical model for the small-scale (n<100) studies common in language testing research is the Rasch one-parameter logistic model. The Rasch model provides a wide range of options for conducting research, refining existing examinations, and developing tailored (computerized adaptive) language tests. Three investigations…

This study introduced a probabilistic model utilizing item response theory (IRT) for dealing with a variety of misconceptions. The model can be used for evaluating the transition behavior of error types, advancement of learning stages, or the stability and persistence of particular misconceptions. Moreover, it apparently can be used for relating…

A standard-setting experiment was conducted in which one group of participants was instructed to refrain from discussing their recommended test standards and another group was requested to report the reasons underlying their recommendations. The procedures used were designed to induce convergence of recommendations, but did not require consensus.…

These five papers describe the methods, results, and implications of a multimethod approach to the establishment of standards for the three subject areas of the Alabama High School Graduation Examination (AHSGE). James E. McLean introduces the problems and methods involved by identifying three general approaches to scoring procedures found in the…

Major efforts of this project fall into four categories: (1) investigations were performed on the relationship between the dimensionality of a dataset and its underlying cognitive processes; (2) two approaches for diagnosing erroneous rules of operation were developed (an "error vector" system for constructing error diagnostic programs…

This booklet provides Grade 6 teachers, administrators, and students with samples of students' writing that exemplify the criteria used to score students' written responses on the June 1988 Grade 6 English Language Arts Achievement Test. The writing assignment required students to read the beginning of a story and use their imaginations to…

The graded response (GR) model of Samejima (1969) and the partial credit model (PC) of Masters (1982) were fitted to identical writing samples that were holistically scored. The performance and relative benefits of each model were then evaluated. Writing samples were both expository and narrative. Data were from statewide assessments of secondary…