Monte Carlo methods are used to explore the accuracy of a method for establishing confidence intervals for variance component estimates in generalizability studies. Previous research has shown that variance component estimation errors due to sampling are often larger than suspected. (Author/CM)

Conceptual rationales from five research contexts are presented, which all posit unmeasured variables that interact with observed independent variables to produce a complete model of the dependent variable. Strategies for overcoming related difficulties are outlined, including increased longitudinal assessment, oversampling of levels of…

If the quantitative result of a study is observed only when the mean difference is statistically significant, the observed mean difference, variance, and effect size are biased estimators of corresponding population parameters. The exact distribution of sample effect size and the maximum likelihood estimator of effect size are derived. (Author/BW)

Reliability and validity of self-report data provided by 178 mentally ill homeless persons were generally favorable. Self-reports of service use also generally agreed with treatment staff estimates, providing further validity evidence. Researchers and administrators can be relatively confident in using such data. (SLD)

Monte Carlo studies explored the sampling characteristics of Cohen's d and three approximations to Cohen's d when used as average effect size measures in meta-analysis. Reviews of 10, 100, and 500 studies (M) were simulated, with degrees of freedom (df) varied in seven steps from 8 to 58. In a two independent groups design, samples were obtained…

Three criticisms of overreliance on results from statistical significance tests are noted. It is suggested that: (1) statistical significance tests are often tautological; (2) some uses can involve comparisons that are not completely sensible; and (3) using statistical significance tests to evaluate both methodological assumptions (e.g., the…

Problems of definition in estimation in evaluation research were studied through a survey of homeless persons in Cook County (Illinois) based on 481 interviews. Differences arising from the following four population definitions are presented: (1) traditional homeless; (2) marginally housed; (3) social isolates; and (4) total of these categories.…

Several issues related to the design and reporting of results from the National Assessment of Educational Progress (NAEP) are discussed in the context of current expectations for the NAEP and its origins. These issues include: (1) content coverage and format; (2) estimation procedures; and (3) reporting problems. (SLD)

The efficacy of the random digit dialing method in telephone surveys was examined. Random digit dialing (RDD) generates a pure random sample and provides the advantage of including unlisted phone numbers, as well as numbers which are too new to be listed. Its disadvantage is that it generates a major proportion of nonworking and business…

Tracking and two statistical methods (probability weighting and sample selection modeling) were studied as ways to minimize bias attributable to sample attrition in school-based studies. Data on student smoking from 30 middle schools illustrate that sample weighting yields the best results, with estimates superior to sample selection and much less…

The Integrated Postsecondary Education Data System (IPEDS) of the National Center for Education Statistics collects data from all sections of postsecondary education. For less-than-two-year institutions, the IPEDS collects data on fall enrollment, completions (earned awards and certificates), etc., from all public institutions in that category and…

The National Household Education Survey (NHES) was conducted for the first time in 1991 to collect data on the early childhood education (ECE) experiences of young children and participation in adult education. Because the NHES methodology is relatively new, field tests were necessary. A large field test of approximately 15,000 households was…