This paper argues for three aspects of tolerance with respect to QCA research: tolerance with respect to different approaches to QCA; producing QCA research with tolerance (work that is resistant to criticism); and for QCA researchers to be clear about the tolerance of the solutions they present -- especially in terms of calibration and truth…

This three-article dissertation explores Learning by Evaluating (LbE), an educational approach that enhances student learning through the process of assessing and providing feedback on peer work. Leveraging principles of comparative judgement, students evaluate pairs of exemplars side-by-side to determine which is better and provide a rational for…

The distinction between "Big-C" and "little-c" creativity implies that the generative process of celebrated creators is of a special type or degree. Arguments for and against such a hierarchy of creativity are found in the literature, primarily built on rhetorical argumentation. The aim of this work is to examine the rationale…

In this paper we explore the concept of the middle tier in education systems, outlining how it is a crucial element that links high-level education policy to the practices that are carried out in schools. Reflecting on the similarities and differences in the profiles of the middle tiers of the four nations of the United Kingdom (UK), we observe…

As a research field geared toward understanding and improving learning, Learning Analytics (LA) must be able to provide empirical support for causal claims. However, as a highly applied field, tightly controlled randomized experiments are not always feasible nor desirable. Instead, researchers often rely on observational data, based on which they…

Treatment effect estimates from a "regression discontinuity design" (RDD) have high internal validity. However, the arguments that support the design apply to a subpopulation that is narrower and usually different from the population of substantive interest in evaluation research. The disconnect between RDD population and the…

Inquiry-based laboratories were implemented into a General Chemistry Laboratory sequence, and the impact of these exercises on students' experimental design skills was assessed using a four-part assessment developed for this study. This assessment contained a multiple-choice section, a section asking students to explain their reasoning behind a…

The objective of this paper is to analyze the impact of the factors that stimulate inspiration in the design process. An empirical study is proposed in this paper. Three factors were summarized, including knowledge, knowledge relations and innovative strategies. Representations of these three factors that were extracted from the international…

Adaptive comparative judgment (ACJ) has proven to be a valid, reliable, and feasible method for assessing student performance in open-ended design scenarios. In addition to the use of ACJ for purely assessment and evaluation, research has demonstrated an opportunity to identify the design values of judges involved with the ACJ process. The…

Studies combined in a meta-analysis often have differences in their design and conduct that can lead to heterogeneous results. A random-effects model accounts for these differences in the underlying study effects, which includes a heterogeneity variance parameter. The DerSimonian-Laird method is often used to estimate the heterogeneity variance,…

Given the widespread use of non-experimental (NE) methods for assessing program impacts, there is a strong need to know whether NE approaches yield causally valid results in field settings. In within-study comparison (WSC) designs, the researcher compares treatment effects from an NE with those obtained from a randomized experiment that shares the…

Research design is a critical topic that is central to research studies in science, social science, and many other disciplines. After identifying the research topic and formulating questions, selecting the appropriate design is perhaps the most important decision a researcher makes. Currently, there is a plethora of literature presenting multiple…

Purpose: This paper aims to compare two types of prompts, encouraging participants to think about real-world examples or engineering principles to show how these two approaches can result in vastly different design practices. Design/methodology/approach: Two studies (N = 20, N = 40) examine the impact of two different prompts. Non-expert students,…

Tools for evaluating the quality and rigor of single case research designs (SCD) are often used when conducting SCD syntheses. Preferred components include evaluations of design features related to the internal validity of SCD to obtain quality and/or rigor ratings. Three tools for evaluating the quality and rigor of SCD (Council for Exceptional…

Varying methods for evaluating the outcomes of single case research designs (SCD) are currently used in reviews and meta-analyses of interventions. Quantitative effect size measures are often presented alongside visual analysis conclusions. Six measures across two classes--overlap measures (percentage non-overlapping data, improvement rate…