Conceptual replications have received increased coverage in the educational research agenda. This article argues for clarity in, and justification of, the definition, scope, and boundaries of a conceptual replication and what it can and cannot do. It argues for clear justifications when changing components from those of the original study. The…

Experimental studies of educational interventions are rarely based on representative samples of the target population. This simulation study tests two formal sampling strategies for selecting districts and schools from within strata when they may not agree to participate if selected: (1) balanced selection of the most typical district or school…

Students often believe that statistical significance is the only determinant of whether a quantitative result is "important." In this paper, I review traditional null hypothesis statistical testing to identify what questions inferential statistics can and cannot answer, including statistical significance, effect size and direction,…

Special education researchers conduct studies that can be considered replications. However, they do not often refer to them as replication studies. The purpose of this article is to consider the potential benefits of conceptualizing special education intervention research within a framework of systematic, conceptual replication. Specifically, we…

I define a treatment effect in terms of a comparison of outcomes and provide a typology of all possible comparisons that can be used to estimate treatment effects, including comparisons that are relatively unknown in both the literature and practice. I then assess the relative merit, worth, and value of all possible comparisons based on the…

Written by a social worker for social work students, this is a nuts and bolts guide to statistics that presents complex calculations and concepts in clear, easy-to-understand language. It includes numerous examples, data sets, and issues that students will encounter in social work practice. The first section introduces basic concepts and terms to…

Quantitative research methods produce factual, reliable, and generalizable data. Qualitative methods generate rich, detailed, valid process data. Ways to integrate them include (1) using qualitative methods to develop quantitative instruments; (2) using qualitative methods to explain quantitative findings; (3) using quantitative methods to…

Much of the current rhetoric in areas of child and family research and in early childhood education emphasizes the importance of listening to children in research that has a direct impact on them. Despite this, there remain qualms in some research contexts and amongst some researchers about the reliability, validity and generalizability of…

Develops basic concepts of generalizability theory using hypothetical study of reliability of scores of vocational interest inventory. Asserts that generalizability theory makes it possible to consider all possible sources of unreliability simultaneously and to design optimal decision-making study. (NB)

In education research, a polar distinction is frequently made to describe and produce different kinds of research: "quantitative" versus "qualitative." In this article, the authors argue against that polarization and the associated polarization of the "subjective" and the "objective," and they question the attribution of generalizability to only…

Addresses generalizability theory (GT), which offers a flexible framework for assessing dependability of measurement. GT allows for consideration of multiple sources of error, allowing investigators to assess the overall impact of measurement error. Illustrative analyses demonstrate the special advantages of GT for planning studies in which…

This paper presents a review of the literature on reliability in qualitative studies. Reliability is defined as the extent to which studies can be replicated, using the same methods, and getting the same results. It is the degree to which data are independent of the accidental circumstances of the research. The review includes the following three…

A methodology is presented for minimizing mean error variance in generalizability studies when resource constraints are imposed. The optimal number of observations and conditions of facets for random model, fully crossed one- and two-facet designs can be decided. Parallel closed form formulas can be determined for other designs. (SLD)

A measurement approach called generalizability theory (G-theory) is an important alternative to the more familiar classical measurement theory that yields less useful coefficients such as alpha or the KR-20 coefficient. G-theory is a theory about the dependability of behavioral measurements that allows the simultaneous estimation of multiple…

Selected examples using the statistical packages Statistical Package for the Social Sciences (SPSS), the Statistical Analysis System (SAS), and BMDP are presented to facilitate their use and encourage appropriate uses in: (1) a hierarchical design; (2) a confounded factorial design; and (3) variance component estimation procedures. To illustrate…