This handbook treats a restricted set of statistical procedures for addressing some of the most prevalent technical issues that arise in domain-referenced testing. The procedures discussed here were chosen because they do not necessitate extensive computations. The five major sections of the paper cover: (1) item analysis procedures for using data…

Basic issues in criterion-referenced measurement are addressed. In section II, issues involved in determining what a person does and can do are considered. A preliminary analysis of "can" is given which shows that there are several important senses of "can". In section III, results of an analysis of "ability" are…

Four points are made and discussed concerning the correction for guessing on tests: (1) the correction could more properly be called a penalty for answering wrong; (2) the basic assumption underlying the correction is the concept of the "chance score"--one expects a proportion of the number of items to be answered correctly on the basis of chance;…

A distinction is made between reliability of measurement and reliability of classification; the "criterion-referenced reliability coefficient" describes the former. Application of this coefficient to the probability distribution of possible scores for a single student yields a meaningful way to describe the reliability of a single score. (Author)

Using a linear loss function, a procedure is described for computing a cutting score that minimizes the risk for a given decision rule. The procedure is demonstrated with a criterion-referenced achievement test of elementary statistics administered to 167 students. (Author/CTM)

A scoring technique for true-false tests is presented. The technique, paired item scoring, involves combining two statements and having the student select one of the four resultants possible: true-true, false-true, true-false, and false-false. The combined item is treated as a multiple choice item. (Author/JKS)

Given known item parameters for a test, unbiased estimators are derived for an examinee's ability parameter, his or her proportion correct true score for the variances of these parameters and for the parallel forms reliability of the maximum likelihood estimator of the ability parameter. (Author/JKS)

No test, however well designed, can measure a student's true achievement because numerous factors interfere with the ability to measure achievement. These factors are sources of measurement error, and the goal in creating tests is to have as little measurement error as possible. Error can result from the test design, factors related to individual…

This paper traces the course of the consequences of viewing test responses as simply providing dichotomous data concerning ordinal relations. It begins by proposing that the score matrix is best considered to be items-plus-persons by items-plus-persons, and recording the wrongs as well as the rights. (Author/CTM)

Methods are described for obtaining upper and lower bounds to both false-positive and false-negative decisions with a mastery test. These methods make no assumptions about the form of the true score distribution. (CTM)

A structural equation model for attitude-behavior relationships is presented which conceptualizes attitude scales as congeneric measurements. The model represents a re-parameterization of an earlier one. (Author/RC)

A method of estimating conditional standard error of measurement at specific score/ability levels is described that avoids theoretical problems identified for previous methods. The method focuses on variance of observed scores conditional on a fixed value of an observed parallel measurement, decomposing these variances into true and error parts.…

The importance of using the standard error of measurement (SEm) in determining reliability in test scores is emphasized. The SEm is compared to the hypothetical true score for standardized tests, and procedures for calculation of the SEm are explained. (JDD)

An equation derived by W. A. Nicewander and J. M. Price relating statistical power to reliability of dependent variable measures when true score regression is homogeneous across treatment conditions is enhanced through overcoming the problem of directly estimating the squared linear correlation between true scores for X and Y. (SLD)

An extension is described to a product-testing model to account for misinformation among subjects that would lead them to perform incorrectly on "pick one of two" tests. The model is applied to a data set of 367 subjects picking 1 of 2 colas. Misinformation does exist. (SLD)