In assessment programs where scores are reported for individual examinees, it is desirable to have responses to performance exercises graded by more than one rater. If more than one item on each test form is so graded, it is also desirable that different raters grade the responses of any one examinee. This gives rise to sampling designs in which…

Rater bias has long been considered a source of error in observer ratings but has been ignored by process researchers using participant ratings. In particular, rater variance, or differences in generalized favorable or unfavorable perceptions of others, represents a neglected source of error in studies using participant ratings. The author…

The present study compared item and ability invariance as well as model-data fit between the one-parameter (1PL) and three-parameter (3PL) Item Response Theory (IRT) models utilizing real data across five grades; second through sixth as well as simulated data at second, fourth and sixth grade. At each grade, the 1PL and 3PL IRT models were run…

The major aim of educational effectiveness research is to examine and explain school, class, and teacher differences with respect to relevant educational criteria. Until now, in the large majority of studies, language and mathematics scores were used as a criterion. In the present study, the educational track students choose at the start of…

R. D. Howell, E. Breivik, and J. B. Wilcox (2007) have argued that causal (formative) indicators are inherently subject to interpretational confounding. That is, they have argued that using causal (formative) indicators leads the empirical meaning of a latent variable to be other than that assigned to it by a researcher. Their critique of causal…

The constraints of conducting evaluations in real-world settings often necessitate the implementation of less than ideal designs. Unfortunately, the standard method for estimating the precision of a result (i.e., confidence intervals [CI]) cannot be used for evaluative conclusions that are derived from multiple indicators, measures, and data…

Because persons with mental retardation cannot be executed for murder, the diagnosis becomes a life and death matter. The American Association on Mental Retardation (now the American Association on Intellectual and Developmental Disabilities) and other associations agree that IQ alone is an insufficient criterion and adaptive functioning also…

This study empirically investigates the effectiveness of Distributed Leadership Teacher Training (DLT) program on improving student's academic achievement. In addition, it both tests the assumption that the year 1 impacts are stable across calendar years and examines the importance of properly accounting for the fact that the standard error of the…

A procedure for checking the score equivalence of nearly identical editions of a test is described. The procedure employs the standard error of equating (SEE) and utilizes graphical representation of score conversion deviation from the identity function in standard error units. Two illustrations of the procedure involving Scholastic Aptitude Test…

The purpose of this study was to examine the effect of test dimensionality on the stability of examinee ability estimates and item response theory (IRT) based score reports. A simulation procedure based on W. F. Stout's Essential Unidimensionality was used to generate test data with one dominant trait for the whole test and three minor traits…

The use of a covariate for randomized response (RRT) research has been shown to reduce standard errors of sensitive trait proportion estimates. At the same time, the model has been shown to be subject to serious misspecification when the relationship between the covariate and the sensitive trait is non-monotonic. The RRT covariate model is adapted…

Logistic regression was used for modeling the observation-to-indicator ratio needed for the standard error scree procedure (SEscree) to correctly identify the number of factors existing in generated sample correlation matrices. The created correlation matrices were manipulated along the number of factors (4,6), sample size (250, 500), magnitude of…

Educational researchers often use multiple statistical tests in their research studies and program evaluations. When multiple statistical tests are conducted, the chance that Type I errors may be committed increases. Thus, the researchers are faced with the task of adjusting the alpha levels for their individual statistical tests in order to keep…

A probability distribution was developed for the Andrews-Pregibon (AP) statistic. The statistic, developed by D. F. Andrews and D. Pregibon (1978), identifies multivariate outliers. It is a ratio of the determinant of the data matrix with an observation deleted to the determinant of the entire data matrix. Although the AP statistic has been used…

Simulations were performed to verify the accuracy with which the Mantel-Haenszel (MH) and Rasch PROX procedures recover simulated item bias. Several standard error estimators for the MH procedure were evaluated. Item bias is recovered satisfactorily by both techniques under all simulated conditions. The proposed MH standard error estimators have…