Impact evaluations of school-based management (SBM) programs, or any other kind of program, are important because they can demonstrate whether or not the program has accomplished its objectives. Furthermore, these evaluations can identify ways to improve the design of the program. These evaluations can also make successful interventions…

Classical methods for detecting outliers deal with continuous variables. These methods are not readily applicable to categorical data, such as incorrect/correct scores (0/1) and ordered rating scale scores (e.g., 0,..., 4) typical of multi-item tests and questionnaires. This study proposes two definitions of outlier scores suited for categorical…

In this article we examine empirical evidence on the criterion, predictive, transfer, and fairness aspects of validity of a large-scale language arts performance assessment, referred to as the Performance Assignment (PA). We use multilevel models to avoid biased inferences that might result from the naturally nested data. Specifically, we examine…

Assessing students' aptitude and educational performance and grading them on a hierarchical scale is a universal feature of the schooling process. In light of grades' instrumental, motivational, and symbolic saliency in students' school experience, it is not surprising they are highly "valued goods," and the process of their fair or…

Increasingly, government, foundations, and others are asking community-based organizations for more evaluation information. Although the demand for this information may be increasing, the field knows very little about the way organizations are responding to this demand. Using data collected from interviews and a mail survey of community-based…

This paper describes a comprehensive model of alignment that provides a foundation for meaningful reporting of students' academic progress over time. The model includes both horizontal and vertical alignment as integral parts of the development of content standards, test blueprints, items, item pools, instruments, performance level descriptors,…

This study focused on the robustness of unidimensional item response theory (UIRT) models in computerized classification testing against violation of the unidimensionality assumption. The study addressed whether UIRT models remain acceptable under various testing conditions and dimensionality strengths. Monte Carlo simulation techniques were used…

A large number of pairwise multiple comparisons (P-MCPs) have been introduced recently to the educational research community. The use of these P-MCPs with single group repeated measures data was studied through an exploratory Monte Carlo study of P-MCPs that have been shown to control different types of Type 2 error and Type 1 familywise error…

Ordinary least-squares regression treats the variables asymmetrically, designating a dependent variable and one or more independent variables. When it is not obvious how to make this distinction, a researcher may prefer to use orthogonal regression, which treats the variables symmetrically. However, the usual procedure for orthogonal regression is…

Correlation coefficients are frequently reported in educational and psychological research. The robustness properties and optimality among practical approximations when phi does not equal 0 with moderate sample sizes are not well documented. Three major approximations and their variations are examined: (1) a normal approximation of Fisher's Z,…

This LISREL study examines the robustness of the maximum likelihood estimates under varying degrees of measurement model misspecification. A true model containing five latent variables (two endogenous and three exogenous) and two indicator variables per latent variable was used. Measurement model misspecification considered included errors of…

Two estimators of effect size that are based on the sample quartiles are proposed and studied. One is for the situation where a treatment effect is evaluated against a control group, and the other is for a situation in which two parallel treatments are compared. Both are illustrated and evaluated. (SLD)

A selection model for meta-analysis is proposed that models the selection process and corrects for the consequences of selection by publication on estimates of the mean and variance of the effect parameters. Simulation studies show that the model substantially reduces bias when the model specification is correct. (SLD)

Recently, C. E. McCulloch (1987) suggested a modification of the Morgan-Pitman test for comparing the variances of two dependent groups. This paper demonstrates that there are situations where the procedure is not robust. A subsample approach, similar to the Box-Scheffe test, and the Sandvik-Olsson procedure are also assessed. (TJH)

This paper explains how to obtain generally robust and powerful analyses with any of the recommended nonorthogonal solutions by adapting a modification of the Welch-James procedure for comparing means when population variances are heterogeneous. Results from a Monte Carlo study support use of the procedure. (SLD)