This guide offers a comprehensive handbook to scientific research methodology and experimental design, specifically for novice MA and PhD researchers in Education and Language Learning (TESOL/TEFL). It establishes scientific research as a systematic, objective inquiry focused on identifying cause-and-effect relationships through empirical data.…

Background: For decades, education researchers have relied on the work of Campbell, Cook, and Shadish to help guide their thinking about valid impact estimates in the social sciences (Campbell & Stanley, 1963; Shadish et al., 2002). The foundation of this work is the "validity typology" and its associated "threats to…

There are usually limited user evaluation of resources on a recommender system, which caused an extremely sparse user rating matrix, and this greatly reduce the accuracy of personalized recommendation, especially for new users or new items. This paper presents a recommendation method based on rating prediction using causal association rules.…

Donald Campbell's approach to causal inference (D. T. Campbell, 1957; W. R. Shadish, T. D. Cook, & D. T. Campbell, 2002) is widely used in psychology and education, whereas Donald Rubin's causal model (P. W. Holland, 1986; D. B. Rubin, 1974, 2005) is widely used in economics, statistics, medicine, and public health. Campbell's approach focuses on…

In an effort to expand the pool of scientific evidence available for review, the What Works Clearinghouse (WWC) assembled a panel of national experts in single-case design (SCD) and analysis to draft SCD Standards. SCDs are adaptations of interrupted time-series designs and can provide a rigorous experimental evaluation of intervention effects.…

In Shadish (2010) and West and Thoemmes (2010), the authors contrasted 2 approaches to causality. The first originated in the psychology literature and is associated with work by Campbell (e.g., Shadish, Cook, & Campbell, 2002), and the second has its roots in the statistics literature and is associated with work by Rubin (e.g., Rubin, 2006). In…

This article compares Donald Campbell's and Donald Rubin's work on causal inference in field settings on issues of epistemology, theories of cause and effect, methodology, statistics, generalization, and terminology. The two approaches are quite different but compatible, differing mostly in matters of bandwidth versus fidelity. Campbell's work…

Ethics is a complicated field and much has been written about its application to educational research. In this paper we introduce a way of planning for and dealing with situations that arise in the course of research that promotes detailed ethical analysis. We present a framework based on the work of Seedhouse and Flinders and describe a…

Experimental designs are distinguished as the best method to respond to questions involving causality. The purpose of the present paper is to explicate the logic of experimental design and why it is so vital to questions that demand causal conclusions. In addition, types of internal and external validity threats are discussed. To emphasize the…

A rich narrative-style bibliography of concept mapping (reviewing six articles published between 1992-2005). Articles reviewed include: (1) Cognitive mapping: A qualitative research method for social work (C. Bitoni); (2) Collaborative concept mapping: Provoking and supporting meaningful discourse (C. Boxtel, J. Linden, E. Roelofs, and G. Erkens);…

Explains causal path modeling, addressing common misconceptions. Emphasizes the real-world validity of causal relationships depicted in models. Uses examples from research on the antecedents of career success. (Contains 37 references.) (SK)

This paper describes several common types of research studies in special education transition literature and the threats to their validity. It then describes how the evidential base may be broadened, how diverse sources of evidence can be combined to strengthen causal inferences, and the role of judgment within quasi-experimentation. The paper…

This paper contains a consideration of the nature and role of warrants for research conclusions in educational research. The paper argues the need for an explicit warrant in the form of a logical and persuasive link between the evidence produced and the conclusions drawn (with appropriate qualifications and caveats). It describes social scientific…

Production-function analysis, a correlation-based analysis, has been used for years to evaluate whether resources deposited into the educational process yielded a definitive result. For example, it has been used in several state cases as a measure of equity in educational funding. This paper is of the opinion that correlation-based analyses are…

Abuses and misuses of statistics are frequent. This digest attempts to warn against these in three broad classes of pitfalls: sources of bias, errors of methodology, and misinterpretation of results. Sources of bias are conditions or circumstances that affect the external validity of statistical results. In order for a researcher to make…