Investigated incorporating a testlet definition into the estimation of the conditional standard error of measurement (SEM) for tests composed of testlets using five conditional SEM estimation methods. Results from 3,876 tests from the Iowa Tests of Basic Skills and 1,000 simulated responses show that item-based methods provide lower conditional…

Investigated the impact of categorization on confirmatory factor analysis parameter estimates, standard errors, and five ad hoc fit indexes through simulation studies. Results replicate some previous studies but also suggest that tests of parameter estimates will be underestimated and the amount of underestimation will increase as saturation…

Proposes a method for obtaining standard errors and confidence intervals of composite reliability coefficients based on bootstrap methods and using a structural-equation-modeling framework for estimating the composite reliability of congeneric measures (T. Raykov, 1997). Demonstrates the approach with simulated data. (SLD)

The purpose of this study was to examine the effect of test dimensionality on the stability of examinee ability estimates and item response theory (IRT) based score reports. A simulation procedure based on W. F. Stout's Essential Unidimensionality was used to generate test data with one dominant trait for the whole test and three minor traits…

This paper reviews various procedures for constructing an interval for an individual's true score given the assumption that errors of measurement are distributed as binomial. This paper also presents two general interval estimation procedures (i.e., normal approximation and endpoints conversion methods) for an individual's true scale score;…

Describes four procedures previously developed for estimating conditional standard errors of measurement for scale scores and compares them in a simulation study. All four procedures appear viable. Recommends that test users select a procedure based on various factors such as the type of scale score of concern, test characteristics, assumptions…

A nonmetric algorithm, pairwise partitioning, is developed to identify feature-based similarity structures. Presents theorems about the validity of the features identified by the algorithm, and reports results of Monte Carlo simulations that estimate the probabilities of identifying valid features for different feature structures and amounts of…

Restrictions due to loss of freedom in estimation, the targeting of the instrument, and the presence of misfit in the data were studied through simulation as factors that influence the asymptotic standard errors for person measures. The underestimation of the observed standard deviation of ability in simulated data is discussed. (SLD)

Standard statistical methods are used to analyze data that is assumed to be collected using a simple random sampling scheme. These methods, however, tend to underestimate variance when the data is collected with a cluster design, which is often found in educational survey research. The purposes of this paper are to demonstrate how a cluster design…

Analytical solutions for point and variance estimators of the mediated effect, the ratio of mediated to direct effect, and the proportion of the total effect mediated were determined through simulation for different samples. The sample sizes needed for accuracy and stability are discussed with implications for mediated effects estimates. (SLD)

Provides asymptotic standard errors of the estimates of equated scores from several types of item response theory (IRT) true score equatings. Equating designs considered cover those with internal or external common items and separate or simultaneous estimation. Uses marginal maximum likelihood estimation for the estimation of item parameters. (SLD)

The analytically derived expected asymptotic standard errors (SEs) of maximum likelihood (ML) item estimates can be predicted by a mathematical function without examinees' responses to test items. The empirically determined SEs of marginal maximum likelihood estimation/Bayesian item estimates can be obtained when the same set of items is…

Large scale surveys usually employ a complex sampling design and as a consequence, no standard methods for estimation of the standard errors associated with the estimates of population means are available. Resampling methods, such as jackknife or bootstrap, are often used, with reference to their properties of robustness and reduction of bias. A…

This study examined the robustness of the estimation of fixed-effects in multilevel analysis, as might occur in the conduct of school-effects studies with outlying schools. Outlying values for both intercepts and slopes for individual schools were modeled separately to determine the effects of extreme values of second-level variables on the…

Item response theory (IRT) model parameter estimates have considerable merit and open up new directions for test development, but misleading results are often obtained because of errors in the item parameter estimates. The problem of the effects of item parameter estimation errors on the test development process is discussed, and the seriousness…