This digest is designed to provide information to Texas testing coordinators, other educators, and interested citizens about the development procedures and technical attributes of the state-mandated criterion-referenced assessment program. The chapters are: (1) "Background"; (2) "Test Development"; (3) "Test…

Whether cutting score equivalents (CSEs) based on examinee performance are the same as CSEs based on expert judgment was examined using data from 3,262 examinees taking an internal medicine certification examination. CSEs produced by 40 physicians/experts were closer to the criteria than were standards derived from examinee performance. (SLD)

The analytically derived asymptotic standard errors (SEs) of maximum likelihood (ML) item estimates can be approximated by a mathematical function without examinees' responses to test items, and the empirically determined SEs of marginal maximum likelihood estimation (MMLE)/Bayesian item estimates can be obtained when the same set of items is…

Two local methods for observed-score equating are applied to the problem of equating an adaptive test to a linear test. In an empirical study, the methods were evaluated against a method based on the test characteristic function (TCF) of the linear test and traditional equipercentile equating applied to the ability estimates on the adaptive test…

Understanding that the behavior of people takes place within a context, over the past 20 years research in education and the sport sciences has witnessed an increasing development of multilevel frameworks that are both conceptually and methodologically sound. Despite these advances, the use of multilevel models and research designs in education…

This paper presents and evaluates a new procedure to classify examinees into meaningful categories based on their responses to (performance on) items comprising a test instrument. The proposed procedure uses the data analytic technique of cluster analysis to partition the examinee population into homogeneous groupings. This procedure was applied…

Three papers and a sample outline of Requests for Proposals (RFPs) represent the work of the Monitoring and Improving Testing and Evaluation Innovations (MITEI) Project during 1988. The skeleton of a sample RFP outline for large-scale assessment was expanded to provide a checklist of the type of information that should be included in a RFP. The…

The gap-reduction model has been identified as a potential alternative to or extension of the Title I Evaluation and Reporting System (TIERS). The gap-reduction model has been recommended for the evaluation of bilingual programs, but has only recently been given consideration for evaluating local Chapter 1 programs. This report recommends that…

This paper overviews technical developments in data analysis procedures for the National Assessment of Educational Progress (NAEP) reading data during 1984. The highlight of the reshaping of the NAEP data has been the scaling using item response theory (IRT). AT this point in the data analysis, an IRT-based scale appears appropriate for reading…

In this study, simulated and actual Air Force test data were used to compare the different procedures for equating mental tests: conventional (equipercentile and linear), Item Response Theory (IRT), and strong true-score theory (STST); data collection designs used were single-group, equivalent-groups, and anchor test. Equating transformations were…

For Texas's Career Ladder System of rewarding good teachers, teachers' performance evaluations from 1981 to 1984 were used to rank teachers in the Austin Independent School District. Significant biases were noted between raters, between years, and between elementary and secondary teacher ratings. To adjust for these biases, each teacher's raw…

Assumptions have been made about the growth and development of reasoning and communicating skills in college. This study describes a standardized set of assessments of reasoning, writing and speaking with norms for freshmen and seniors and multiple equated forms. The study provides evidence of the validity and reliability of these assessments.…

The use of a three-parameter logistic model for applying latent trait theory has become more popular because of the availability of computer programs. The program Logist 5 can be used not only for the item parameter estimation in the three-parameter logistic model, but also in the two parameter logistic model by setting the third parameter equal…

Guidelines are described for setting up an item bank under latent trait theory which may be applied to the achievement testing system of multi-section, large-enrollment, college survey courses. The enrollment for the course is typically heterogeneous: students may be majors or non-majors, any one section may contain honors college students and…

The problems and issues involved in the conduct of educational-developmental research are examined within the perspective of longitudinal research methodology. Chapters 2 and 3 examine contemporary research designs and procedures implemented for the selection of subjects and testing of behavior over time. Particular attention is given to the…