The ECFMG[R] Clinical Skills Assessment (CSA[R]) was developed to evaluate whether graduates of international medical schools (IMGs) are ready to enter graduate training programs in the United States. The patient note (PN) exercise, conducted after a 15-minute interview with a standardized patient (SP), is specifically used to assess a candidate's…

Increased demands for colleges and universities to engage in outcomes assessment for accountability purposes have accelerated the need to bridge the gap between higher education practice and the fields of measurement, assessment, and evaluation. The "Handbook on Measurement, Assessment, and Evaluation in Higher Education" provides higher…

Classical test theory, item response theory, and generalizability theory all treat the abilities to be measured as continuous variables, and the items of a test as independent probes of underlying continua. These models are well-suited to measuring the broad, diffuse traits of traditional differential psychology, but not for measuring the outcomes…

Tables of specifications are used to guide test developers in sampling items and maintaining consistency from form to form. This paper is a generalizability study of the American College Testing Program (ACT) Achievement Program Mathematics Test (AAP), with the content areas of the table of specifications representing multiple dependent variables.…

The Association of American Medical Colleges is conducting research to develop, implement, and evaluate a Medical College Admission Test (MCAT) essay testing program. Essay administration in the spring and fall of 1985 and 1986 suggested that additional research was needed on the development of topics which elicit similar skills and meet standard…

This study is an attempt at a cohesive characterization of the concept of essay reliability. As such, it takes as a basic premise that previous and current practices in reporting reliability estimates for essay tests have certain shortcomings. The study provides an analysis of these shortcomings--partly to encourage a fuller understanding of the…

Comments are made on the review papers presented by six Dutch psychometricians: Ivo Molenaar, Wim van der Linden, Ed Roskam, Arnold Van den Wollenberg, Gideon Mellenbergh, and Dato de Gruijter. Molenaar has embraced a pragmatic viewpoint on Bayesian methods, using both empirical and pure approaches to solve educational research problems. Molenaar…

The purpose of this study was to compare the relative magnitude of the variance components and generalizability coefficients derived from the Teacher Performance Assessment Instruments (TPAI) data using two different methods of data collection: (1) occasions when observers were in the classroom for simultaneous observation and (2) occasions when…

The percentage of students at/above a cut point (PAAC) is one of the most common measures used for reporting school-level performance relative to a proficiency standard (L. Cronbach, N. Bradburn, and D. Horvitz, 1994). The two purposes of this study were to introduce procedures for estimating standard errors for school PAACs under a…

Natural statistical models based on renewal processes for individuals' Behavior Stream are developed. For continuous observation, statistical and psychometric properties are obtained for empirical: (1) rates of behavior; (2) proportions of behavior type; (3) prevalence; and (4) event duration. Results provide a guide for design and analysis. (SLD)

The stability of scores on 2 types of performance assessments, an observed hands-on investigation and a notebook surrogate, was investigated for 29 sixth graders on 2 occasions. Results indicate that student performance and procedures changed and that generalizability across occasions was moderate. Implications for assessment are discussed. (SLD)

In this article, we discuss the history behind efforts to transfer school psychology research into practice and review the literature pertaining to treatment acceptability, participatory action research, organizational change, and generalization programming. We then present a model for the systematic programming of this transfer and propose a…

The authors present results of a generalizability study of running record assessment. They conducted 2 decision studies to ascertain the number of raters and passages necessary to obtain a reliable estimate of a student's reading ability on the basis of a running record assessment. Ten teachers completed running record assessments of 10…

Issues surrounding accurate assessment of depression in children have received much attention. However, the stability of scores from depression measures has generally been estimated using only classical test score theory, rather than the more powerful generalizability theory. The dependability of scores from the Children's Depression Inventory…

The meta-analysis by A. C. Willig of bilingual education evaluation literature was not successful in comparison with and did not replicate the Baker and de Kanter narrative review. Willig's conclusion that bilingual education is effective is not supported by the analysis. (SLD)