This paper scrutinizes the increasing trend of using international large-scale assessment (ILSA) data for causal inferences in educational research, arguing that such inferences are often tenuous. We explore the complexities of causality within ILSAs, highlighting the methodological constraints that challenge the validity of causal claims derived…

In the article "Rethinking Traditional Methods of Survey Validation" Andrew Maul describes a minimalist validation methodology for survey instruments, which he suggests is widely used in some areas of psychology and then critiques this methodology empirically and conceptually. He provides a reduction ad absurdum argument by showing that…

Within evidence-based education, results from randomised controlled trials (RCTs), and meta-analyses of them, are taken as reliable evidence for effectiveness -- they speak to "what works". Extending RCT results requires establishing that study samples and settings are representative of the intended target. Although widely recognised as…

Quasi-experiments are common in studies that estimate the effect of instructional interventions on student performance outcomes. In this type of research, the nature of the experimental design, the choice in assessment, the selection of comparison groups, and the statistical methods used to analyze the comparison data dictate the validity of…

In this article, the authors extend the results of Aguirre-Urreta, Rönkkö, and Marakas (2016) concerning the omission of a relevant causal indicator by testing the validity of the assumption that causal indicators are entirely superfluous to the measurement model and discuss the implications for measurement theory. Contrary to common wisdom…

Although randomized studies have high internal validity, generalizability of the estimated causal effect from randomized clinical trials to real-world clinical or educational practice may be limited. We consider the implication of randomized assignment to treatment, as compared with choice of preferred treatment as it occurs in real-world…

The purpose of the present paper is to critically examine causal inferences and internal validity as defined by Campbell and co-workers. Several arguments are given against their counterfactual effect definition, and this effect definition should be considered inadequate for causal research in general. Moreover, their defined independence between…

This article compares Donald Campbell's and Donald Rubin's work on causal inference in field settings on issues of epistemology, theories of cause and effect, methodology, statistics, generalization, and terminology. The two approaches are quite different but compatible, differing mostly in matters of bandwidth versus fidelity. Campbell's work…

The Maslach Burnout Inventory--General Survey (MBI-GS) is designed to measure the three subdimensions (exhaustion, cynicism, and professional efficacy) of burnout in a wide range of occupations. This article examines the factorial validity of the MBI-GS across eight different occupational groups in Norway: lawyers, physicians, nurses, teachers,…

A supports classification and assessment system is required in residential settings that enables accurate classification and resource allocation in the context of limited resources. In developing such a system, the American Association on Mental Retardation intensities of supports framework was used and consultation with all stakeholders resulted…

Regression-based adjustment of student outcomes for the assessment of the merit of schools is considered. First, the basics of causal modeling and multiple regression are briefly reviewed. Then, two common regression-based adjustment procedures are described, pointing out that the validity of the final assessments depends on: (1) the degree to…