This article provides an extensive overview of the recent literature on student evaluation of teaching (SET) in higher education. The review is based on the SET meta-validation model, drawing upon research reports published in peer-reviewed journals since 2000. Through the lens of validity, we consider both the more traditional research themes in…

Criterion-referencing is a way to enhance the interpretation of test scores by referencing them to well-defined behavior domains. Behavior domains may be ordered or unordered; several varieties of criterion-referenced tests within each of these types are discussed. (Author/RL)

Seventeen statistics for measuring the reliability of criterion-referenced tests were critically reviewed. The review was organized into two sections: (1) a discussion of preliminary considerations to provide a foundation for choosing the appropriate category of "reliability" (threshold loss function, squared-error loss-function, or…

A review of alternative methodologies and a conceptual framework for the study of reliability of criterion-referenced tests are presented. The possibility of aptitude-x-assessment interactions is considered and implications are discussed. (Author)

Presents simplified methods for deriving estimates of the consistency of criterion-referenced, English-as-a-Second-Language tests, including (1) the threshold loss agreement approach using agreement or kappa coefficients, (2) the squared-error loss agreement approach using the phi(lambda) dependability approach, and (3) the domain score…

This literature review summarizes what is currently known about the agreement among six measures of writing skills. Three of these methods involve the application of human judgment in scoring or rating a piece of writing: holistic, analytical, and primary trait scoring. Two methods involve anatomical or taxonomic analysis of a piece of writing:…

A dozen different approaches that yield 13 reliability indices for criterion-referenced tests were identified and grouped into three categories: threshold loss function, squared-error loss function, and domain score estimation. Indices were evaluated within each category. (Author/RL)

Intended to help educators select diagnostic and criterion referenced reading tests for their specific purposes, this book contains reviews of nine individual and three group tests. The reviewed tests were selected for their "importance" from a list of 34 and were reviewed by experts according to their descriptions of content, test development,…

A behavioral domain is well-defined when it is clear to both test developers and test users which categories of performance should or should not be considered for potential test items. Only those tests that are keyed to well-defined domains meet the definition of criterion-referenced tests. The greatest proliferation of criterion-referenced tests…

This booklet is intended to help school personnel, parents, students, and members of the community understand concepts and research relating to achievement testing in public schools. The paper's sections include: (1) test use with direct effects on students (test of certification, selection, and placement); (2) test use with indirect effects on…

In light of the widespread use of competency testing, the authors consider that it is important to determine ways of developing and using competency testing to insure that it achieves its full potential. The paper, in three parts, introduces a model for the development and validation of competency tests, reviews several methods for setting…

Three basic areas of definition, measurement, and research, voiced as concerns by members of the National Council of Measurement in Education (NCME) to its Task Force on Measurement Problems in Competency-Based Programs, are used as an organizer for a sponsored symposia and the papers in this monograph. The problem of the definition of competency…

A review of literature serves as the basis for a discussion of various aspects of criterion-referenced tests. The aspects discussed are: teaching and evaluation objectives, criterion- and norm-referenced measurement, stages in construction of criterion-referenced tests, construction and selection of items, test validity, and test reliability.…

A discussion of evaluating informal logic competence centers on the identification of currently available tests and on a description of test theory, to aid in test selection. The following concepts in test theory are discussed: norm-referenced and criterion-referenced tests, true scores, reliability, validity, test selection and evaluation, test…

Noting that the last 40 years have witnessed an enormous increase in the number of psychological tests designed for the assessment of competencies in very young children, this review summarizes the quantitative and qualitative characteristics of psychological tests and other assessment instruments used to evaluate the cognitive functioning of…