This paper points out that there are certain generalizations about directions for guessing and methods of scoring that require that data be derived from random groups design. It supports the viewpoint that it is neither sufficient nor appropriate to make such generalizations on the basis of an analysis of scores obtained from the answer sheets of…

One of the important underlying assumptions of all item response theory (IRT) models is that of local independence. This assumption requires that the response to an item on a test not be influenced by the response to any other items. This assumption is often taken for granted, with little or no scrutiny of the response process required to answer…

This paper discusses cognitive and noncognitive variables, and their relationship with each other, in learning and in educational evaluation. The nature of noncognitive learning environment variables is examined. The following instruments that have been widely used in assessing classroom environment are described: (1) the Learning Environment…

To assess the effects of correlated abilities on test characteristics, and to explore the effects of correlated abilities on the use of a multidimensional item response theory model which does not explicitly account for such a correlation, two tests were constructed. One had two relatively unidimensional subsets of items, the other had all…

The purpose of this paper is to compare the precision of direct and indirect measures of writing assessment using the test information functions from a graded response Item Response Theory (IRT) model. Subjects were 192 sophomore English students from a parochial high school in Wisconsin. Both direct and indirect measures of writing ability were…

This was a replicative study that was initiated with a journeyman level certification instrument for an international union, when industry monitors were observed suggesting to examinees to "go with your first response." The question arose whether this was a researched-based practice. If not, wouldn't this practice inject constant error…

This paper discusses distribution theory and power computations for four common "tests of combined significance." These tests are calculated using one-sided sample probabilities or p values from independent studies (or hypothesis tests), and provide an overall significance level for the series of results. Noncentral asymptotic sampling…

The goal of this research was to predict from a recruit's responses to the Armed Services Vocational Aptitude Battery (ASVAB) items whether the recruit would pass the Armed Forces Qualification Test (AFQT). The data consisted of the responses (correct/incorrect) of 1,020 Navy recruits to 200 items of the ASVAB together with the scores of these…

Attitude measures frequently produce distributions of item scores that attenuate interitem correlations and thus also distort findings regarding the factor structure underlying the items. An actual data set involving 260 adult subjects' responses to 55 items on the Love Relationships Scale is employed to illustrate empirical methods for…

The reliability of criterion referenced tests, which are often used to evaluate health education programs, may be conceptualized in different ways. Classical conceptualizations of test reliability have limited usefulness when applied to health-related criterion referenced tests. When a cutting score is set, test reliability can be represented as…

The relation between Bayley Scale and Stanford-Binet measures in preschool and scholastic achievement at pre-adolescence is examined in the context of an early age intervention. The program sought to normalize the socio-cognitive development of disadvantaged 2-year-olds directly and through a training program for mothers. The follow-up includes a…