In educational testing, subscores may be provided based on a portion of the items from a larger test. One consideration in evaluation of such subscores is their ability to predict a criterion score. Two limitations on prediction exist. The first, which is well known, is that the coefficient of determination for linear prediction of the criterion…

The relationship between classical discrimination indices (CDI) and criterion-referenced discrimination indices (CRDI) and the appropriateness of each for use on criterion-referenced tests are investigated. A CRDI is proposed that attempts to separate those who master material from those who do not master material. A 26 item multiple-choice…

This paper evaluates the magnitude of the total error in estimates of the difference between an examinee's domain score and the cutoff score. An observed score based on a random sample of items from the domain, and an estimated cutoff score derived from a judgmental standard setting procedure are assumed. The work of Brennan and Lockwood (1980) is…

A general issue related to language program development involves the empirical rationalization of cut score decisions in criterion-referenced language tests. Cut score dependability focuses on the consistency of the decisions in repeated testing or the assessment of language learner performances. In this case, the issue is to determine the optimal…

Because testing, in many different forms, currently plays such an important role in education, Elementary Secondary Education Act, Title I, the Division of Federal Assistance in the Vermont State Department of Education, prepared this brochure to present a general introduction to terms and phrases commonly used in testing and to highlight some of…

Standards, passing scores, or cut scores have been seen as an element of criterion-referenced tests since their introduction. This paper discusses at least two issues surrounding the establishment of cut scores which appear to need clarification: (1) the theoretical definition of a cut score; and (2) decisions which must be made in selecting a…

This paper presents an introduction to the use of latent trait models for the estimation of domain scores. It was shown that these models provided an advantage over classical test theory and binomial error models in that unbiased estimates of true domain scores could be obtained even when items were not randomly selected from a universe of items.…

Four discrimination indices were compared, using score distributions which were normal, bimodal, and negatively skewed. The score distributions were systematically varied to represent the common circumstances of a military training situation using criterion-referenced mastery tests. Three 20-item tests were administered to 110 simulated subjects.…

Increasing numbers of state-mandated testing programs have resulted in the selection of norm-referenced tests (NRTs) for statewide assessment. Adaptation or customization of existing tests has become a viable solution for individual programs. Changes in test instruments have arisen from the behaviorist approach and mastery learning, minimum…

The family of Weibull distributions was investigated as a model for the distributions of response times for items in computer-based criterion-referenced tests. The fit of these distributions were, with a few exceptions, good to excellent according to the Kolmogorov-Smirnov test. For a few relatively simple items, the two-parameter gamma…

A procedure is demonstrated for constructing tables showing, for each score on a commercial reading achievement test, the percentage of real-world materials that the testee is likely to comprehend with at least a criterion level of proficiency, the percentages of students in a local or national sample who can competently comprehend a given…

This report was designed to respond to two major methodological shortcomings in the item bias literature: (1) misfitting test models; and (2) the use of significance tests. Specifically, the goals of the research were to describe a newly developed method known as the "plot method" for identifying potentially biased test items and to…

The 1976 Educational Testing Service (ETS) Invitational Conference served as a platform for individuals who have been prominent in educational measurement and research to present their views on issues surrounding the testing controversy. The 1976 ETS "The Testing Scene: Chaos and Controversy," presents a historical review of events surrounding the…

This book, which is intended for adult educators and human resource developers, presents guidelines for assessing adult learning. The following are among the topics covered in the book's eight chapters: (1) basic principles of informal assessment (relationship between learning and assessment activities; sequencing learning and assessment…