Evaluators are often called upon to assess the effects of programs. To assess a program effect, evaluators need a clear understanding of how a program effect is defined. Arguably, the most widely used definition of a program effect is the counterfactual one. According to the counterfactual definition, a program effect is the difference between…

This guide offers a comprehensive handbook to scientific research methodology and experimental design, specifically for novice MA and PhD researchers in Education and Language Learning (TESOL/TEFL). It establishes scientific research as a systematic, objective inquiry focused on identifying cause-and-effect relationships through empirical data.…

In this commentary from Issue 14, n3, authors John Cadogan and Nick Lee applaud the paper by Aguirre-Urreta, Rönkkö, and Marakas "Measurement: Interdisciplinary Research and Perspectives", 14(3), 75-97 (2016), since their explanations and simulations work toward demystifying causal indicator models, which are often used by scholars…

Fisher randomization tests for Neyman's null hypothesis of no average treatment effects are considered in a finite population setting associated with completely randomized experiments with more than two treatments. The consequences of using the F statistic to conduct such a test are examined both theoretically and computationally, and it is argued…

In this article, the authors extend the results of Aguirre-Urreta, Rönkkö, and Marakas (2016) concerning the omission of a relevant causal indicator by testing the validity of the assumption that causal indicators are entirely superfluous to the measurement model and discuss the implications for measurement theory. Contrary to common wisdom…

Regression discontinuity design (RD) has been widely used to produce reliable causal estimates. Researchers have validated the accuracy of RD design using within study comparisons (Cook, Shadish & Wong, 2008; Cook & Steiner, 2010; Shadish et al, 2011). Within study comparisons examines the validity of a quasi-experiment by comparing its…

This study assessed students' attitudes toward science in Qatar. A cross-sectional, nationwide probability sample representing all students enrolled in grades 3 through 12 in the various types of schools in Qatar completed the "Arabic Speaking Students' Attitudes toward Science Survey" (ASSASS). The validity and reliability of the…

There are usually limited user evaluation of resources on a recommender system, which caused an extremely sparse user rating matrix, and this greatly reduce the accuracy of personalized recommendation, especially for new users or new items. This paper presents a recommendation method based on rating prediction using causal association rules.…

The authors propose and test a causal model theory of reasoning about conditional arguments with causal content. According to the theory, the acceptability of modus ponens (MP) and affirming the consequent (AC) reflect the conditional likelihood of causes and effects based on a probabilistic causal model of the scenario being judged. Acceptability…

In Shadish (2010) and West and Thoemmes (2010), the authors contrasted 2 approaches to causality. The first originated in the psychology literature and is associated with work by Campbell (e.g., Shadish, Cook, & Campbell, 2002), and the second has its roots in the statistics literature and is associated with work by Rubin (e.g., Rubin, 2006). In…

This report summarizes the complex research literature on quantitative methods for assessing how impacts of educational interventions on instructional practices and student learning differ across students, educators, and schools. It also provides technical guidance about the use and interpretation of these methods. The research topics addressed…

Production-function analysis, a correlation-based analysis, has been used for years to evaluate whether resources deposited into the educational process yielded a definitive result. For example, it has been used in several state cases as a measure of equity in educational funding. This paper is of the opinion that correlation-based analyses are…

Rectifies the unsatisfactory textbook treatment of the finite-sample proprieties of estimators of regression models with a lagged dependent variable and autocorrelated disturbances. Maintains that the bias of the ordinary least squares estimator is determined by the dynamic and correlation effects. (MJP)