This guide offers a comprehensive handbook to scientific research methodology and experimental design, specifically for novice MA and PhD researchers in Education and Language Learning (TESOL/TEFL). It establishes scientific research as a systematic, objective inquiry focused on identifying cause-and-effect relationships through empirical data.…

Aim/Purpose: The purpose of this paper is to address the challenges posed by disinformation in an educational context. The paper aims to review existing information assessment techniques, highlight their limitations, and propose a conceptual design for a multimodal, explainable information assessment system for higher education. The ultimate goal…

This report summarizes the complex research literature on quantitative methods for assessing how impacts of educational interventions on instructional practices and student learning differ across students, educators, and schools. It also provides technical guidance about the use and interpretation of these methods. The research topics addressed…

Education systems are increasingly creating rich, longitudinal data sets with frequent, and even real-time, data updates of many student measures, including daily attendance, homework submissions, and exam scores. These data sets provide an opportunity for district and school staff members to move beyond an indicators-based approach and instead…

As lines between research paradigms continue to blur with the ever-increasing popularity of mixed methods research, there are useful, and occasionally oxymoronic, opportunities for educational researchers to juxtapose tools from opposing methods. The gold standard is just not possible in so much of what we do with small-scale research, nor is it…

In this article, the author challenges the validity and usefulness of the concept of "paradigm," as this term has been used in the social sciences generally, and specifically in the debates over research methods. He emphasizes that in criticizing what he sees as the misuse of the paradigm concept, he is not arguing for dismissing or ignoring…

Donald Campbell's approach to causal inference (D. T. Campbell, 1957; W. R. Shadish, T. D. Cook, & D. T. Campbell, 2002) is widely used in psychology and education, whereas Donald Rubin's causal model (P. W. Holland, 1986; D. B. Rubin, 1974, 2005) is widely used in economics, statistics, medicine, and public health. Campbell's approach focuses on…

In Shadish (2010) and West and Thoemmes (2010), the authors contrasted 2 approaches to causality. The first originated in the psychology literature and is associated with work by Campbell (e.g., Shadish, Cook, & Campbell, 2002), and the second has its roots in the statistics literature and is associated with work by Rubin (e.g., Rubin, 2006). In…

Researchers using structural equation modeling (SEM) aspire to learn about the world by seeking models with causal specifications that match the causal forces extant in the world. This quest for a model matching existing worldly causal forces constitutes an ontology that orients, or perhaps reorients, thinking about measurement validity. This…

The purpose of this article is to demonstrate application of a new framework, the validation framework (VF), to assist researchers in evaluating mixed research studies. Based on an earlier work by Dellinger and Leech, a description of the VF is delineated. Using the VF, three studies from education, health care, and counseling fields are…

This article discusses how methodological practices can shape and limit how mixed methods is practiced and makes visible the current methodological assumptions embedded in mixed methods practice that can shut down a range of social inquiry. The article argues that there is a "methodological orthodoxy" in how mixed methods is practiced…

A sequential design utilizing identical samples was used to classify mixed methods studies via a two-dimensional model, wherein sampling designs were grouped according to the time orientation of each study's components and the relationship of the qualitative and quantitative samples. A quantitative analysis of 121 studies representing nine fields…

Empirical literature on two types of hearings, and their applicability in evaluating educational programs, is reviewed. An introduction describes the use and nature of hearings and distinguishes adversary hearings, in which two parties argue opposing sides of an issue, from committee hearings, in which a variety of positions are presented to a…

This paper examines representative research in information science and compares its designs to an abstract ideal of the experimental research design. Threats to validity of conclusions are highlighted, including history, maturation, testing, instrumentation, statistical regression, selection biases, experimental mortality, and selection-maturation…

This article describes the strengths and weaknesses of different models of classification (primarily monothetic and polythetic) and then focuses on cluster analysis as a method of pursuing a polythetic model. Design features that need to be incorporated in a cluster analytic study are discussed in relation to theory formulation, internal validity,…