We study the properties of two specification tests that have been applied to a variety of estimators in the context of value-added measures (VAMs) of teacher and school quality: the Hausman test for choosing between student-level random and fixed effects, and a test for feedback (sometimes called a "falsification test"). We discuss…

Empirical Bayes' (EB) estimation is a widely used procedure to calculate teacher value-added. It is primarily viewed as a way to make imprecise estimates more reliable. In this paper we review the theory of EB estimation and use simulated data to study its ability to properly rank teachers. We compare the performance of EB estimators with that of…

This study investigates how different rounding rules and ways of providing Angoff standard-setting judgments affect cut-scores. A simulation design based on data from the National Assessment of Education Progress was used to investigate how rounding judgments to the nearest whole number (e.g., 0, 1, 2, etc.), nearest 0.05, or nearest two decimal…

We investigate whether commonly used value-added estimation strategies produce accurate estimates of teacher effects under a variety of scenarios. We estimate teacher effects in simulated student achievement data sets that mimic plausible types of student grouping and teacher assignment scenarios. We find that no one method accurately captures…

We study the properties of two specification tests that have been applied to a variety of estimators in the context of value-added measures (VAMs) of teacher and school quality: the Hausman test for choosing between random and fixed effects and a test for feedback (sometimes called a "falsification test"). We discuss theoretical…

School districts and state departments of education frequently must choose between a variety of methods to estimating teacher quality. This paper examines under what circumstances the decision between estimators of teacher quality is important. We examine estimates derived from student growth percentile measures and estimates derived from commonly…

Because of the increased popularity of computerized adaptive testing (CAT), many admissions tests, as well as certification and licensure examinations, have been transformed from their paper-and-pencil versions to computerized adaptive versions. A major difference between paper-and-pencil tests and CAT from an examinee's point of view is that in…

This paper addresses the problem of whether two-dimensional solutions with different apparent meanings and different implied interpretations of the number-correct scale could be produced from the same test data set by simply shifting the orientation of the two-dimensional projection plane. An artificial data set of 3,000 response vectors and a…

This paper reviews the existing multidimensional item response theory (IRT) models and demonstrates how one of the models can be applied to estimation of abilities from a test measuring more than one dimension. The purposes of this paper were threefold. First, the fundamental concepts required when considering multidimensional models for the…

Real test data of unknown structure were analyzed using both a unidimensional and a multidimensional latent trait model in an attempt to determine the underlying components of the test. The models used were the three-parameter logistic model and a multidimensional extension of the two-parameter logistic model. The basic design for the analysis of…

A two-stage study was conducted to compare the ability estimates yielded by tailored testing procedures based on the one-parameter logistic (1PL) and three-parameter logistic (3PL) models. The first stage of the study employed real data, while the second stage employed simulated data. In the first stage, response data for 3,000 examinees were…

Five comparisons were made relative to the quality of estimates of ability parameters and item calibrations obtained from the one-parameter and three-parameter logistic models. The results indicate: (1) The three-parameter model fit the test data better in all cases than did the one-parameter model. For simulation data sets, multi-factor data were…

Since all commonly used latent trait models assume a unidimensional test, the applicability of the procedure to obviously multidimensional tests is questionable. This paper presents the results of the application of latent trait, traditional, and factor analyses to a series of actual and hypothetical tests that vary in factoral complexity.…

One of the major assumptions of latent trait theory is that the items in a test measure a single dimension. This report describes an investigation of procedures for forming a set of items that meet this assumption. Factor analysis, nonmetric multidimensional scaling, cluster analysis and latent trait analysis were applied to simulated and real…

This paper explores the effects of item choice on ability estimation when using a tailored testing procedure based on the Rasch simple logistic model. Most studies of the simple logistic model imply that ability estimates are totally independent of the items used, regardless of the testing procedure. This paper shows that the ability estimate is…