The design, administration, scoring, and results of the 1993 New York State Correctional Captain Examination are described. The examination was administered to 405 candidates. As in previous Sergeant and Lieutenant examinations, candidates also completed latent image written simulation problems and open/closed book multiple choice test components.…

The development and validation of the General Management In-Basket (GMIB) is described. The GMIB is a theory-based generic in-basket simulation, designed to assess supervisory and management skills independent of any job classification. Three of the 15 in-basket items in the GMIB are critical and are scored on a 0-5 scale. The remaining 12 items…

Earlier content analyses showed that the match between content covered by textbooks and tests varied as a function of the particular textbook and test a teacher was asked to use. This study tried to determine if the congruity in textbook-test content varied as a function of different styles of textbook use. Using year-long case studies of seven…

The investigation examined relationships among scales for observing and rating teacher performance. Beginning teachers with varying levels of professional experience (2, 9, and 16 months) were rated by pairs of observers on two occasions. Intercorrelations across occasions fell between .5 and .8. Interrater agreement ranged between .5 and .9.…

The Association of American Medical Colleges is conducting research to develop, implement, and evaluate a Medical College Admission Test (MCAT) essay testing program. Essay administration in the spring and fall of 1985 and 1986 suggested that additional research was needed on the development of topics which elicit similar skills and meet standard…

This study is an attempt at a cohesive characterization of the concept of essay reliability. As such, it takes as a basic premise that previous and current practices in reporting reliability estimates for essay tests have certain shortcomings. The study provides an analysis of these shortcomings--partly to encourage a fuller understanding of the…

Even though several national testing firms have developed measures to evaluate the effectiveness of baccalaureate education, there continues to be a general reluctance on the part of faculty in colleges and universities to accept these measures as criteria on which to evaluate educational programs. Some of the resistance appears to lie in the lack…

In an assessment of the adequacy of the Gesell screening examination as a test instrument, a Gesell Screening Evaluation was given to 400 children semi-annually from their 4th to 6th year. The sample, which was stratified by parent occupation, included 40 girls and 40 boys at 5 age levels. The test battery corresponded with the Gesell Preschool…

The necessity of controlling the order in which trained essay raters for a statewide writing assessment program receive student essays was studied. The underlying theoretical question concerns possible rater bias caused by raters reading long strings of essays of homogeneous quality; this problem is usually referred to as context effect or…

The purpose of this study was to compare the relative magnitude of the variance components and generalizability coefficients derived from the Teacher Performance Assessment Instruments (TPAI) data using two different methods of data collection: (1) occasions when observers were in the classroom for simultaneous observation and (2) occasions when…

Six university English departments collaborated in this examination of the differences between multiple-choice and essay tests in evaluating writing skills. The study also investigated ways the two tools can complement one another, ways to improve cost effectiveness of essay testing, and ways to integrate assessment and the educational process.…

A study investigated the use of reading proficiency scales developed by the American Council on the Teaching of Foreign Languages (ACTFL), Educational Testing Service (ETS), and Interagency Language Roundtable (ILR) for meaningful rank-ordering and assigning levels of second language competence to reading passages. In a proficiency test writing…

The purposes of this study were to establish the interrater reliability, dimensionality, and internal consistency of an instruction evaluation instrument used at The University of Michigan Medical School. Using the nine-item rating scale, 1,758 student ratings and 88 staff ratings were gathered on 61 faculty. Interrater agreement ranged from .28…

This study addressed seven questions regarding the methods used in setting passing scores on the essay subtest of the National Teacher Examinations (NTE) Communication Skills test for the North Carolina State Board of Education. North Carolina uses these tests to screen prospective applicants to teacher education programs. The judges (five college…

A latent trait method is presented to investigate the possibility that Angoff or Nedelsky judges specify inconsistent probabilities in standard setting techniques for objectives-based instructional programs. It is suggested that judges frequently specify a low probability of success for an easy item but a large probability for a hard item. The…