The characteristics of three threshold loss agreement indices which reflect the agreement or consistency in assignment to mastery-nonmastery status are reviewed. These are proportion of agreement, coefficient kappa, and modified kappa. (Author/MT)

This study compared the reliability and validity indexes of randomly parallel tests administered under inclusion, exclusion, and correction for guessing directions, using 54 graduate students. It also compared the criterion-referenced grading decisions based on the different scoring methods. (TJH)

Subkoviak suggested a technique for estimating both group reliability and the reliability associated with assigning a given individual to a mastery or non-mastery category based on a single test administration. Two assumptions underlie this model. First, it is assumed that had successive test administrations occurred, scores for each individual…

The reliability of criterion-referenced tests (CRTs) used in health program evaluation can be conceptualized in different ways. Formulas are presented for estimating appropriate standard error of measurement (SEM) for CRTs. The SEM can be used in computing confidence intervals for domain score estimates and for a cut-score. (Author/LMO)

This study was designed to develop and standardize parallel forms of a criterion-referenced test for measuring the proficiency of undergraduate college students in the application of syllabication skills. Twenty-seven classes in 14 colleges and universities participated in the final standardization of the tests. Analysis of results indicated that…

Since competency-based instruction is based on the testing process, this study suggests that careful attention to the construction of tests can produce a reliable, valid measure. The next steps to be taken include developing larger banks of test items, increasingly accurate tests, and instruction to meet specified objectives of the curriculum.…

A general issue related to language program development involves the empirical rationalization of cut score decisions in criterion-referenced language tests. Cut score dependability focuses on the consistency of the decisions in repeated testing or the assessment of language learner performances. In this case, the issue is to determine the optimal…

The Assessment Resource Center (ARC) at Florida State University provides computer assisted testing (CAT) for approximately 4,000 students each term. Computer capabilities permit a small proctoring staff to administer tests simultaneously to large numbers of students. Programs provide immediate feedback for students and generate a variety of…

This document is a training module designed to familiarize practicing school psychologists and school psychology students with the field of criterion-referenced measurement and recent developments within the field. The introduction gives reasons for the need for criterion-referenced testing as opposed to norm-referenced tests which include legal…

The standard growth expectations for various levels and categories of the Quarterly Profile Examination were reported in this study as an approach to assessing edumetric validity. The results demonstrated the usefulness of computing standard growth expectations for tests used in the measurement of within-individual growth. (Author/JKS)

The reliabilities of two types of measurement plans were compared across six hypothetical distributions of true scores or abilities. The measurement plans were: (1) fixed-length, where the number of trials for all examinees is set in advance; and (2) trials-to-criterion, where examinees must keep trying until they complete a given number of trials…

Higher education institutions have difficulty providing stakeholders with reliable, valid information about student achievement. High stakes assessment data are ambiguous in communicating such information. It is important to distinguish criterion-referenced assessment from norm-referenced assessment. This paper proposes a system that goes beyond…

A study involving 234 Texas college students was undertaken to develop and test the validity and reliability of an instrument designed to measure college students' attitudes toward alcohol. Special attention was given to the content and construct validity of the instrument. Subjects ranged in age from 18 to 24 years. The mapping sentence served as…

The development of a simulation technique to evaluate performance of psychomotor skills in an undergraduate nursing program is described. This method is used as one admission requirement to an alternate route nursing program. With modifications, any health profession could use this technique where psychomotor skills performance is important.…

Implicit in most of the recommendations for teaching reading is the inclusion of structural analysis as a part of the instructional program for developing skills in word recognition. This study focused on the development and standardization of a criterion referenced syllabication skills test designed to evaluate the individual proficiency of…