Recent studies show cluster randomized trials may be well powered to detect mediation or indirect effects in multilevel settings. However, literature has rarely provided guidance on designing cluster-randomized trials aim to assess indirect effects. In this study, we developed closed-form expression to estimate the variance of and the statistical…

Methods for meta-analyzing single-case designs (SCDs) are needed in order to inform evidence based practice in special education and to draw broader and more defensible generalizations in areas where SCDs comprise a large part of the research base. The most widely used outcomes in single-case research are measures of behavior collected using…

The popularity of sample surveys in evaluation and research makes it necessary for consumers to tell a good survey from a poor one. Several sources were identified that gave advice on how to evaluate a sample design used in a survey study. The sources are either too limited or too extensive to be useful practically. The purpose of this paper is to…

Examined the effect of complex sampling of items on the measurement of differential item functioning (DIF) using the Mantel-Haenszel procedure through a Monte Carlo study. Suggests the superiority of the pooled booklet method when items are selected for examinees according to a balanced incomplete block design. Discusses implications for other DIF…

This paper helps researchers develop criteria for conducting surveys. Cost considerations frame the paper's suggestions. Specifically, the text advises that researchers consider the human and material resources needed, type of survey instrument, medium for conducting the survey (mail, telephone, personal interview, etc.), data analysis…

The problems addressed, and solutions devised, by educational ethnographers in selection of and sampling from phenomena for internal representativeness external comparison are examined. The process of sampling is designated as a specialized form of the more general process of focusing and choosing in research, which is called selection. Common…

A brief, practical overview of "design effects" (DEFFs) is presented for users of the results of sample surveys. The overview is intended to help such users to determine how and when to use DEFFs and to compute them correctly. DEFFs are needed only for inferential statistics, not for descriptive statistics. When the selections for…

There are two general methods of cross-validation: empirical estimation, and formula estimation. In choosing a specific cross-validation procedure, one should consider both costs (e.g., inefficient use of available data in estimating regression parameters) and benefits (e.g., accuracy in estimating population cross-validity). Empirical…

Two relatively new tools for analysis of data compiled in evaluation studies are presented. The National Test-Equating Study in Reading, known as the Anchor Test Study, produced tables of score-correspondence between the eight reading comprehension and vocabulary tests most widely used in the United States. Two types of tables from this report…

The problem of blocking on a status variable was investigated. The one-way fixed-effects analysis of variance, analysis of covariance, and generalized randomized block designs each treat the blocking problem in a different way. In order to compare these designs, it is necessary to restrict attention to experimental situations in which observations…

Against an historical backdrop, this paper summarizes four uses of intraclass correlation of importance to contemporary researchers in the behavioral sciences. First, it shows how the intraclass correlation coefficient can be used to adjust confidence intervals for statistical significance testing when data are intracorrelated and the independence…

Standard statistical methods are used to analyze data that is assumed to be collected using a simple random sampling scheme. These methods, however, tend to underestimate variance when the data is collected with a cluster design, which is often found in educational survey research. The purposes of this paper are to demonstrate how a cluster design…

Present research connected with computer-based instruction indicates that many investigators rely on conventional experimental design and data analysis to address the research question. Although this is appropriate in many cases, the limits of hard and software resources and sample size in other cases have made conventional approaches a poor…

The zeitgeist for reform in education precipitated a number of changes in assessment. Among these are performance assessments, sometimes linked to "high stakes" accountability decisions. In some instances, the trustworthiness of these decisions is based on variance components and error variances derived through generalizability theory.…

Research with older persons suffering from physical illness presents numerous challenges to gerontologists. Issues of conceptualization pertaining to the definition of illness, its location in the research paradigm, and the context in which illness occurs must be addressed prior to dealing with methodological problems. Access to physically ill…