A common interpretation of achievement test results is that they provide measures of achievement that are much like other measures we commonly use for height, weight, or the cost of goods. In a limited sense, such interpretations are correct, but some nuances of these interpretations have important implications for the use of achievement test…

Differential item functioning (DIF) statistics were computed using items from the Peabody Individual Achievement Test (PIAT)-Reading Comprehension subtest for children of the same age group (ages 7 through 12 respectively). The pattern of observed DIF items was determined by comparing each cohort across age groups. Differences related to race and…

It is understandable that parents, policy makers, educators, etc. want to know how schools are functioning. Extensive resources are expended on the educational enterprise and it is only reasonable that the impact of those resources be determined. However, determining the amount of change in students' skills and knowledge is not easy. Further,…

Definition of the issues to the use of latent trait models, specifically one- and three-parameter logistic models, in conjunction with multi-level achievement batteries, forms the basis of this paper. Research results related to these issues are also documented in an attempt to provide a rational basis for model selection. The application of the…

The feasibility of implementing self-paced computerized tailored testing evaluation methods in an undergraduate measurement and evaluation course, and possible differences in achievement levels under a paced versus self-paced testing schedule were investigated. A maximum likelihood tailored testing procedure based on the simple logistic model had…

The process used to identify the provisional locations of points that would inform the National Assessment Governing Board in setting achievement levels for the science National Assessment of Educational Progress (NAEP) is described. This process includes identification of panelists for achievement-level setting and the method for converting…

Because latent trait models require that large numbers of items be calibrated or that testing of the same large group be repeated, item parameter estimates are often obtained by administering separate tests to different groups and "linking" the results to construct an adequate item pool. Four issues were studied, based upon the analysis…

The purpose of this paper is to identify and discuss some of the problems presented by the use of computerized adaptive testing (CAT) in an instructional programs environment versus large scale testing applications, and to describe an actual implementation of CAT in an instructional programs setting. This particular application is in the…

This report summarizes the research findings of a four year contract investigating the applicability of item response theory and tailored testing to criterion-referenced measurement. Six major areas were studied on the project: (1) techniques for forming unidimensional item sets; (2) techniques for calibrating items; (3) item parameter linking…

Since all commonly used latent trait models assume a unidimensional test, the applicability of the procedure to obviously multidimensional tests is questionable. This paper presents the results of the application of latent trait, traditional, and factor analyses to a series of actual and hypothetical tests that vary in factoral complexity.…

The purpose of the paper is to determine whether test forms of the Mathematics Usage Test (AAP Math) of the American College Testing Program are parallel in a multidimensional sense. The AAP Math is an achievement test of mathematics concepts acquired by high school students by the end of their third year. To determine the dimensionality of the…

This paper has shown the advantages of tailored testing over traditional methods, the need for latent trait theory in the application of tailored testing, and some of the results available to show the quality of estimates obtained by tailored testing procedures. Since the tailored testing procedures overcome many of the problems with traditional…

The reliability and validity of a tailored testing procedure based on the simple logistic model was determined for an achievement test in statistics and measurement. The test was administered on a CRT terminal to students from graduate and undergraduate measurement courses. Equivalent form reliability over a one-week interval was found to be 0.595…

The current technology of computerized testing is discussed, and a few comments are made on how such technology might be used for assessing school-related skills as part of the National Assessment of Educational progress (NAEP). The critical feature of computerized assessment procedures is that the test items are presented in interactive fashion,…

This paper explores the effects of item choice on ability estimation when using a tailored testing procedure based on the Rasch simple logistic model. Most studies of the simple logistic model imply that ability estimates are totally independent of the items used, regardless of the testing procedure. This paper shows that the ability estimate is…