We provide evidence of validity for a newly developed diagnostic competence model of operation sense, by both (a) describing the theoretically substantiated development of the competence model in close association with its use within a large-scale formative assessment and (b) providing empirical evidence for the theoretically described cognitive…

A decision model designed to determine the reliability and validity of criterion referenced measures (CRMs) is presented. General procedures which pertain to the model are discussed as to: Measures of relationship, Reliability, Validity (content, criterion-oriented, and construct validation), and Item Analysis. The decision model is presented in…

New strategies for evaluation criterion referenced measures (CRM) are discussed. These strategies examine the following issues: (1) the use of normed referenced measures (NRM) as CRM and then estimating the reliability and validity of such measures in terms of variance from an arbitrarily specified criterion score, (2) estimation of the…

A unified framework for validating criterion-referenced test (CRT) interpretation is proposed. Using functional literacy as an illustration, the instructional outcomes assessed are called achievement constructs, and described in psychological and behavioral terms. The proposed construct validation methods can yield tests more closely linked to…

Attempts to select empirically the optimal cutting score or criterion level for a test based on response data from validation samples of instructed and uninstructed students. This score maximizes the probability of correct mastery-nonmastery decisions (or minimizes the probability of incorrect decisions). (Author/RK)

Developing good criterion measures of professional performance is difficult. If criterion-related validity evidence for certification requirements could not be generalized beyond the specific context in which it was obtained, gathering that evidence would probably not be worth the effort. This paper examines two possible approaches to the…

In this paper, an attempt has been made to synthesize some of the current thinking in the area of criterion-referenced testing as well as to provide the beginning of an integration of theory and method for such testing. (Editor)

Two ESEA Title I evaluation models developed by the Resource Management Corporation (RMC), were field tested simultaneously with 560 Title I reading students, grades 2-8. Measuring instruments for models 1 and 2 were, respectively, the California Achievement Test (reading vocabulary section), a norm-referenced test; and the Tarmac Reading…

The job of the 4-H extension agent involves fairly complex levels of performance. The curriculum for the extension agent program should produce youth workers who have the ability to perform competently and who possess the basic concepts and values required to function effectively. Performance objectives were written for each competency considered…

While the usefulness of branched testing over conventional paper-and-pencil testing has been in doubt, particularly for the student of average ability, this has been with reference to normative measures rather than the criterion-referenced measures characteristic of individualized instruction. A computer-assisted test model for assessing an…

This paper presents an evaluation of Army job training development and testing practices, with a focus on Advanced Individual Testing. Information comes from intensive interviews with school instructors and from observations in the schools. Results indicate that some aspects of the Instructional Systems Development (ISD) model have been…

In light of the widespread use of competency testing, the authors consider that it is important to determine ways of developing and using competency testing to insure that it achieves its full potential. The paper, in three parts, introduces a model for the development and validation of competency tests, reviews several methods for setting…

Technical considerations associated with item selection and reliability assessment are considered in relation to criterion-referenced tests constructed to provide group information. The purpose is to emphasize test building and the evaluation of test scores in program evaluation studies. It is stressed that an evaluator employ a performance or…

This resource paper is a guide for planning and developing instructionally relevant tests of student learning at the classroom, building, or district level. It is based on a model of instruction and testing which systematically uses assessment information to support and facilitate instructional improvement. Course goals and objectives are first…