This document contains the Phase III report from the "What Works in Oklahoma Schools" study. As opposed to describing the findings from the study that was conducted, it provides a tool-kit that can be used by Oklahoma principals and teachers to determine the best courses of action for their schools and classrooms. The tools provided in…

The study of human performance on discrete optimization problems has a considerable history that spans various disciplines. The two most widely studied problems are the Euclidean traveling salesperson problem and the quadratic assignment problem. The purpose of this paper is to outline a program of study for the measurement of human performance on…

In an Angoff standard setting procedure, judges estimate the probability that a hypothetical randomly selected minimally competent candidate will answer correctly each item in the test. In many cases, these item performance estimates are made twice, with information shared with the panelists between estimates. Especially for long tests, this…

Scores on a vocabulary test given at the beginning of two semesters in a large entry-level course predicted performance on multiple-choice exams more strongly than pre-course knowledge and critical thinking. Words on the vocabulary instrument were derived from multiple-choice exam items in the course. Although commonly used in the course, these…

This article describes the process by which one study utilized qualitative methods to create items for a multi dimensional scale to measure twelve step program affiliation. The process included interviewing fourteen addicted persons while in twelve step focused treatment about specific pros (things they like or would miss out on by not being…

Mixture modeling is a widely applied data analysis technique used to identify unobserved heterogeneity in a population. Despite mixture models' usefulness in practice, one unresolved issue in the application of mixture models is that there is not one commonly accepted statistical indicator for deciding on the number of classes in a study…

Recently, R. Shealy and W. Stout (1993) proposed a procedure for detecting differential item functioning (DIF) called SIBTEST. Current versions of SIBTEST can only be used for dichotomously scored items, but this paper presents an extension to handle polytomous items. The paper presents: (1) a discussion of an appropriate definition of DIF for…

A problem frequently confronted in item response theory (IRT) applications is that the item parameters calibrated using more than two independent samples of subjects must be expressed on the same scale. The existing methods were developed for a pairwise transformation, that is, from one scale to the other. The purpose of this study is to introduce…

The interest in the application of large-scale adaptive testing for secure tests has served to focus attention on issues that arise when theoretical advances are made operational. Many such issues in the application of large-scale adaptive testing for secure tests have more to do with changes in testing conditions than with testing paradigms. One…

In many testing situations, differential item functioning (DIF) is a potentially serious problem. It occurs when a test item appears to be easier for one group of examinees than another even after controlling for overall skill level. Differential person functioning (DPF) can occur when "items" can be considered raters and the persons are the…

This paper presents three different ways of computing the internal consistency coefficient alpha for a same set of data. The main objective of the paper is the illustration of a method for maximizing coefficient alpha. The maximization of alpha can be achieved with the aid of a principal component analysis. The relation between alpha max. and the…

Modern test construction strategies in the areas of personality and psychopathology differ in the use of disguise within test stimulus material. Previous research on the validity of using disguised test item content has favored the rational strategy of test construction which views disguise as a liability under normal test-taking circumstances.…

The requirements for adaptive testing are reviewed, and the question of why implementation has taken so long is examined. The concept of a testing procedure that selects items to match the level of performance of an examinee during the administration of a test had to wait for the technology necessary to apply the idea. Current procedures were…

The background and results of an effort to use dBASE II, a microcomputer database management package, to establish, maintain, and update an item bank useful in a complex test development process are presented. The paper explores some of the perspectives and considerations in designing such a database which make the test development process easier,…

A neglected element in designing tests is that of publicness, that is, the extent to which test specifications are understandable and usable by all interested parties. Issues related to content validity, such as test bias and instructional sensitivity, become accessible to these parties once content validity and design have been adequately…