Two studies involving a total of 99 experts examined the reproducibility of standards for 2 medical certifying examinations set under different conditions. Together, results of both studies provide evidence that a modified version of the Angoff method is quite reliable and produces stable results under varying conditions. (SLD)

J. L. Rury (1968) presented evidence that environmental influences are important determinants of interstate black-white score differentials on the Army alpha examination. This article generalizes and extends these findings by constructing a series of educational-quality indices that test the inferential sturdiness of Rury's findings across a…

Research over the past 15 years concerning parametric and nonparametric statistics and 10 recently developed nonparametric techniques for testing interactions in experimental design are reviewed. The new nonparametric techniques are robust, powerful, versatile, and easy to compute. A fabricated dataset is used to illustrate the technique. (TJH)

Data in social and behavioral sciences typically possess heavy tails. Structural equation modeling is commonly used in analyzing interrelations among variables of such data. Classical methods for structural equation modeling fit a proposed model to the sample covariance matrix, which can lead to very inefficient parameter estimates. By fitting a…

To deal with nonnormal and heterogeneous data for the one-way fixed effect analysis of variance model, the authors adopted a trimmed means method in conjunction with Hall's invertible transformation into a heteroscedastic test statistic (Alexander-Govern test or Welch test). The results of simulation experiments showed that the proposed technique…

Original formulations of the Work Locus of Control Scale (WLCS) proposed a unidimensional structure of this measure; however, more recently, evidence for a two-dimensional structure has been reported, with separate subscales for internal and external loci of control. The current study evaluates the one- and two-factor models with confirmatory…

This paper presents the development of the fusion model skills diagnosis system (fusion model system), which can help integrate standardized testing into the learning process with both skills-level examinee parameters for modeling examinee skill mastery and skills-level item parameters, giving information about the diagnostic power of the test.…

Because 3rd Grade Florida Comprehensive Assessment Test (FCAT) scores have a direct impact on promotion, the results for that grade level are released early by the State. When the FCAT results for 3rd Grade were released in May 2007, many people were troubled. Over 80% of the elementary schools in the Miami-Dade School District showed a decrease…

In this paper, the author looks at whether teachers give better subjective assessments to students of their own race and/or gender, conditionally on test scores. Subjective assessments are pervasive in schools; most teachers fill school records that include comments on the child's ability or behavior. And important decisions such as tracking,…

Validity generalization meta-analysis (VG) examines the extent to which the validity of an instrument can be transported across settings. VG offers correction and summarization procedures designed in part to remove the effects of statistical artifacts on estimates of association between criterion and predictor. By employing a random effects model,…

C. L. Olson (1976, 1979) suggests the Pillai-Bartlett trace (V) as an omnibus multivariate analysis of variance (MANOVA) test statistic for its superior robustness to heterogeneous variances. J. Stevens (1979, 1980) contends that the robustness of V, Wilk's lambda (W) and the Hotelling-Lawley trace (T) are similar, and that their power functions…

The Type I error and power properties of the 2x2x2 analysis of variance (ANOVA) and tests developed by McSweeney (1967), Bradley (1979), Harwell-Serlin (1989; Harwell, 1991), and Blair-Sawilowsky (1990) were compared using Monte Carlo methods. The ANOVA was superior under the Gaussian and uniform distributions. The Blair-Sawilowsky test was…

This study investigated the robustness of the James second-order test (James 1951; Wilcox, 1989) and the univariate F test under a two-factor fixed-effect analysis of variance (ANOVA) model in which cell variances were heterogeneous and/or distributions were nonnormal. With computer-simulated data, Type I error rates and statistical power for the…

Whether anchor item sets varying in difficulty and discrimination affect precision of cutting score equivalents generated through judge rescaling as much as equivalents from score equating was studied with 4 groups of experts and 250 and 1,000 examinees. Results indicate the robustness of judge rescaling and its superiority over equating. (SLD)

Argues that the loss-function analysis used to adjust the census of 1990 depends on estimating probable errors in census and adjustment and that data from field-validation projects does not provide estimates accurate enough for the purpose. The loss-function analysis is actually driven by assumptions, not data. (SLD)