While standard setting procedures are typically discussed in terms of deriving a reasonable cutting score for a given form of a test, the situation may be structured such that the standard has been mandated without regard to the test form itself. This situation may result either through legislative or policy actions and may be a fait accompli by…

This paper details how memory protocols obtained in individual interviews may be translated to scores on several dimensions suggested by White. White's nine dimensions are: (1) extent, (2) precision, (3) internal consistency, (4) accord with reality, (5) variety of types of element, (6) variety of topics, (7) shape, (8) ratio of internal to…

The booklet was designed to help each recipient of the California Achievement Test (CAT) reports to understand the general format of the reports, the abbreviations and symbols used, and the types of scores presented. It was also intended to assist in interpreting CAT results and using these results at each level of the educational process to best…

A common problem in practical educational research is that of perfect scores which result when latent trait models are used. A simple procedure for managing the perfect and zero response problem encountered in converting test scores into measures is presented. It allows the test user to chose among two or three reasonable finite representations of…

A description of the difficulties encountered in constructing a criterion-referenced test to assess end-of-year skills as part of a program evaluation is presented. The problems encountered in preparing the customized test are described in humorous detail. The first problem involved the listening test obtained from a publisher without…

Research indicating that different cut-off points result from the use of different standard-setting techniques leaves decision makers with a disturbing dilemma: Which standard-setting method is best? This investigation of the reliability and validity of 10 different standard-setting approaches was designed to provide information that might help…

In the standard scoring procedure for multiple-choice exams, students must choose exactly one response as correct. Often students may be unable to identify the correct response, but can determine that some of the options are incorrect. This partial knowledge is not captured in the standard scoring format. The Coombs elimination procedure is an…

The ways in which standard-setting procedures are carried out in the medical specialty certification area are described. Three experiments that have been conducted with alternatives to currently used normative standards are examined, and reasons why these experiments have had limited success are suggested in this speech. The defensibility of…

It is suggested that for checking purposes, "Questions and Answers" should accompany, in the form of a class work blank, the study of a given text. Justification is given, possible objections refuted. An example shows how such work can be quickly corrected and evaluated. (Text is in German.) (IFS/WGA)

The four scoring strategies compared were: lamda coefficients, chi-square weights, and two applications of multiple discriminant analysis. No significant differences were found when applied to the Kuder Occupational Interest Survey. (RC)

This paper outlines a technique for differentially weighting options of a multiple choice test in a fashion that maximizes the item predictive validity. The rule can be applied with different number of categories and the "optimal" number of categories can be determined by significance tests and/or through the R2 criterion. Our theoretical analysis…

Seventeen statistics for measuring the reliability of criterion-referenced tests were critically reviewed. The review was organized into two sections: (1) a discussion of preliminary considerations to provide a foundation for choosing the appropriate category of "reliability" (threshold loss function, squared-error loss-function, or…

The subject of constructing criterion-referenced tests is often researched, but many technical problems remain to be satisfactorily resolved. Foremost, criterion-referenced test developers need a comprehensive set of steps for construction. In this paper, 14 logical steps for building criterion-referenced tests that refer to several different…

In order to determine whether students transcribe the same number of words regardless of the speed of the dictation, dictation tests were administered to students enrolled in beginning Gregg shorthand at the secondary level. Test batteries were administered at three different times throughout the first semester of shorthand instruction. The…