We provide evidence of validity for a newly developed diagnostic competence model of operation sense, by both (a) describing the theoretically substantiated development of the competence model in close association with its use within a large-scale formative assessment and (b) providing empirical evidence for the theoretically described cognitive…

Work integrated learning (WIL) educators using reflective practice to facilitate student learning require a set of standards that works within the traditional assessment frame of Higher Education, to ascertain the level at which reflective practice has been demonstrated. However, there is a paucity of tested assessment instruments that provide…

In this commentary, the authors discuss some of the issues regarding the use of diagnostic classification models that practitioners should keep in mind. In the authors experience, these issues are not as well known as they should be. The authors then provide recommendations on diagnostic scoring.

Developing good criterion measures of professional performance is difficult. If criterion-related validity evidence for certification requirements could not be generalized beyond the specific context in which it was obtained, gathering that evidence would probably not be worth the effort. This paper examines two possible approaches to the…

This report was written to provide guidance in the development and evaluation of alternative instructional approaches that hold promise of improving instructional effectiveness. The main focus of the report is on how to identify and test interactive relationships between individual differences among learners and instructional conditions or…

This paper describes a model for the evaluation and approval of a test battery for compliance with a midwestern state law mandating criterion-referenced testing of specific objectives. Standards specifying that the test scores must demonstrate content validity and criterion-related validity form the foundation of the model. The model also…

A decision model designed to determine the reliability and validity of criterion referenced measures (CRMs) is presented. General procedures which pertain to the model are discussed as to: Measures of relationship, Reliability, Validity (content, criterion-oriented, and construct validation), and Item Analysis. The decision model is presented in…

This paper discusses the validity of test preparation practices in the context of large-scale norm-referenced and criterion-referenced tests. It presents a framework for conceptualizing the various aspects of test preparation that includes the basic premises underlying the validity of the practices. Five areas of test preparation are outlined: (1)…

New strategies for evaluation criterion referenced measures (CRM) are discussed. These strategies examine the following issues: (1) the use of normed referenced measures (NRM) as CRM and then estimating the reliability and validity of such measures in terms of variance from an arbitrarily specified criterion score, (2) estimation of the…

A unified framework for validating criterion-referenced test (CRT) interpretation is proposed. Using functional literacy as an illustration, the instructional outcomes assessed are called achievement constructs, and described in psychological and behavioral terms. The proposed construct validation methods can yield tests more closely linked to…

Attempts to select empirically the optimal cutting score or criterion level for a test based on response data from validation samples of instructed and uninstructed students. This score maximizes the probability of correct mastery-nonmastery decisions (or minimizes the probability of incorrect decisions). (Author/RK)

In this paper, an attempt has been made to synthesize some of the current thinking in the area of criterion-referenced testing as well as to provide the beginning of an integration of theory and method for such testing. (Editor)

On the basis of the findings it would appear that mathematics instructors should consider seriously the use of Structure-of-Intellect factor tests for selection and placement of students. (Authors)

Validation in curriculum development is the "check-and-balance" dimension of any instructional system, in the broadest sense almost synonymous with evaluation and accountability. This paper relates validation to individual, formative, and summative evaluation. Validation measures to be applied to instructional systems are outlined according to a…