Rater scoring has an impact on writing test reliability and validity. Thus, there has been a continued call for researchers to investigate issues related to rating (Crusan, 2015). Investigating the scoring process and understanding how raters arrive at particular scores are critical "because the score is ultimately what will be used in making…

This article reports on the development and validation of a rubric for assessing students' written responses to artworks. Since the implementation of the Hong Kong New Senior Secondary Curriculum in 2009, art educators have seen responding to artworks as increasingly important. In this context, the Art Criticism Assessment Rubric (ACAR) was…

Western Governors University (WGU) is an online competency-based university in which students demonstrate content competence through a series of assessments. Assessments most often are performance-based or objective assessments that are developed in accordance with specific content objectives. Objective assessments generally assess lower-level…

An interjudge reliability test was conducted to evaluate questionnaires used in the surveillance of residential care institutions in Israel. Results from 32 institutions (evaluated by two surveyor teams--one social worker and 1 nurse per team) and the variance in reliability were used to improve the questionnaires and their administration. (SLD)

The role of sample papers (i.e., anchor papers, prototypes, range-finders) in holistic evaluation of writing is discussed. When, where, and how many sample papers are to be selected, and who should perform the selection are covered. The process of sample selection should proceed as follows: (1) a general reading of papers by committee members to…

Eleven judges representing 11 different geographic regions in the United States participated in a standard-setting session designed to determine the possibility of obtaining interjudge consensus and intrajudge consistency simultaneously. Each judge had experience in the field for which standards were being set. The judges rated 65 multiple-choice…

The present study examines the influence of judges' item-related knowledge on setting standards for competency tests. Seventeen judges from different professions took a 122-item teacher-certification test in economics while setting competency standards for the test using the Angoff procedure. Judges tended to set higher standards for items they…

An instrument for to assessing the extent to which postsecondary vocational education programs are competency-based consisted of 159 items grouped into 8 subscales corresponding to 4 standards: (1) identification and validation of competencies; (2) procedures for assessing student competencies; (3) instructional procedures; and (4) administrative…

A method is presented for collecting information about the match between students' learning issues in problem-based learning and teachers' objectives. Subjects were 82 second-year medical students at the University of Limburg in Maastricht (Netherlands) in a problem-based curriculum. During a unit on pregnancy, childbirth, and child development,…

This study was undertaken to test the hypothesis that items of the Test of English as a Foreign Language (TOEFL) containing reference to American people, places, customs, etc., tend to favor examinees who have spent some time living in the United States. Two samples of examinees were drawn from the March 1987 TOEFL administration, one tested in…

The Educational Testing Service is developing a new generation of teacher assessments--the Praxis Series: Professional Assessments for Beginning Teachers. The assessment series consists of three components. An academic skills component will assess the candidate's basic academic and enabling skills. Subject assessments will test the candidate's…

The design, administration, scoring, and results of the 1993 New York State Correctional Captain Examination are described. The examination was administered to 405 candidates. As in previous Sergeant and Lieutenant examinations, candidates also completed latent image written simulation problems and open/closed book multiple choice test components.…

The validation of key contextual features of algebra word problems was studied in two phases. In the first phase, five experts were asked to assess the appropriateness of the concepts in the problems and the adequacy of the assignment of the contextual features to the problems. In the second phase, construct validity was established by having 6…

This study involved the selection and adaptation of a writing assessment procedure for teachers and researchers in the Duval County Public Schools (Florida) to use in assessing changes in writing ability among elementary grade students. Through a review of the literature, four writing assessment procedures (analytic, holistic, focused holistic,…

Construction of a work sample test, the Investigator Planning Exercise (IPE), for the job of detective in the Chicago (Illinois) Police Department is described. Simulated crime scenarios, a mock crime scene, and five checklists of necessary skills (i.e., ability to summarize and communicate facts, identify inconsistencies, and determine the next…