Could the research you read be fundamentally flawed? Could critical defects in methodology slip by you undetected? To become informed consumers of research, students need to thoughtfully evaluate the research they read rather than accept it without question. This second edition of a classic text gives students the tools they need to apply critical…

Students may need explicit training in informal statistical reasoning in order to design experiments or use formal statistical tests effectively. By using scientific scandals and media misinterpretation, we can explore the need for good experimental design in an informal way. This article describes the use of a paper that reviews the measles mumps…

In several subdomains of the social, behavioral, health, and human sciences, research questions are increasingly answered through mixed methods studies, combining qualitative and quantitative evidence and research elements. Accordingly, the importance of including those primary mixed methods research articles in systematic reviews grows. It is…

Conceptualizes the process of conducting integrative research reviews as a data-gathering, empirically based procedure and offers guidelines for conducting such reviews. Introduces specific suggestions that were heretofore unaddressed. Maintains that efforts to standardize integrative review process and to ensure its scientific rigor in counselor…

Institutional researchers frequently use national datasets such as those provided by the National Center for Education Statistics (NCES). The authors of this chapter explore the adjustments required when analyzing NCES data collected using complex sample designs. (Contains 8 tables.)

Expresses the importance of scientifically based research in education reform and explains how to judge the validity of educational research. Describes control groups, randomized and matched experiments, statistical educational significance, sample size, and the difference between scientifically based and rigorously evaluated research. (Contains…

A method for drawing inferences from complex samples is based on Rubin's approach to missing data in survey research. Standard procedures for drawing such inferences do not apply when the variables of interest are not observed directly, but must be inferred from secondary random variables which depend on the variables of interest stochastically.…

A discussion of survey methodology for investigating second language programs and instruction examines two methods: oral interviews and written questionnaires. Each method is defined, and variations are explored. For interviews, this includes individual, group, and telephone interviews. For questionnaires, this includes self-administered and…

In 1982, the Educational Testing Service (ETS) proposed to implement a new, complex design for the National Assessment of Educational Progress (NAEP). The major features of this design are described in "A New Design for a New Era" (Messick, Beaton, and Lord, 1983). The purpose of this document is to describe the actual implementation of…

The findings of the Cost, Quality, and Child Outcomes (CQO) study in Child Care Centers, the largest and most visible child care research conducted in the 1990s, were widely publicized and used to promote increased spending on initiatives to improve child care quality, the redesign of subsidy systems to provide quality incentives, more stringent…