The purpose of this paper is to analyze the use of the exemplar methodology (ExM) as a method for selecting exemplars in education research. ExM is a systematic approach to selecting outliers that can be used to education researchers who investigate outliers to better understand phenomena among students, teachers, schools, and communities. While…

With growing social science interest in new materialist and posthuman ontologies, it is timely to explore how these may translate practically into social research methodologies. This task is complicated by differing interpretations of how new materialist precepts should shape research. This paper aims to fill a gap in the literature by setting out…

Teacher agency is a set of actions that a teacher takes beyond what is generally expected of them. The concept merits examination, as agency can bolster teachers' ability to set and achieve professional development goals. To better understand how to study, and use, this relatively new concept in the academic literature, a systematic review of 164…

Randomized A/B tests allow causal estimation without confounding but are often under-powered. This paper uses a new dataset, including over 250 randomized comparisons conducted in an online learning platform, to illustrate a method combining data from A/B tests with log data from users who were not in the experiment. Inference remains exact and…

Participatory modeling (PM) is an engaged research methodology for creating analog or computer-based models of complex systems, such as socio-environmental systems. Used across a range of fields, PM centers stakeholder knowledge and participation to create more internally valid models that can inform policy and increase engagement and trust…

Facebook is widely used and researched. However, though the data generated by educational technology tools and social media platforms other than Facebook have been used for research purposes, very little research has used Facebook posts as a data source--with most studies relying on self-report studies. While it has historically been impractical…

Qualitative studies in the domain of physics education research have become more common in the last several decades. Methodologically, this has been marked by an expansion of the types of data collected in physics education research (PER): namely, in the use of individual and group interviews, problem-solving sessions, and classroom…

OpenStax Kinetic is an innovative research infrastructure that aims to transform education and learning research in the digital age. With its access to large sample sizes, authentic learning environments, experimental control, scalability, security and privacy protection, Kinetic provides an unparalleled opportunity for researchers to study the…

While conflict event data sets are increasingly used in contemporary conflict research, important concerns persist regarding the quality of the collected data. Such concerns are not necessarily new. Yet, because the methodological debate and evidence on potential errors remains scattered across different subdisciplines of social sciences, there is…

Although educational research and evaluation generally occur in multilevel settings, many analyses ignore cluster effects. Neglecting the nature of data from educational settings, especially in non-randomized experiments, can result in biased estimates with long-term consequences. Our manuscript improves the availability and understanding of…

Despite the rising popularity of big data, there is speculation that evaluators have been slow adopters of these new statistical approaches. Several possible reasons have been offered for why this is the case: ethical concerns, institutional capacity, and evaluator capacity and values. In this method note, we address one of these barriers and aim…

Purpose: The purpose of this dissertation is to investigate the feasibility of using tests of robustness in peer review. This study involved selecting three high-impact papers which featured open data and utilized bioinformatic analyses but provided no source code and refactoring these to allow external survey participants to swap tools,…

This paper presents an ethical framework designed to support the development of critical data literacy for research methods courses and data training programmes in higher education. The framework we present draws upon our reviews of literature, course syllabi and existing frameworks on data ethics. For this research we reviewed 250 research…

Changing criterion designs (CCD) are single-case experimental designs that entail a step-by-step approximation of the final level desired for a target behavior. Following a recent review on the desirable methodological features of CCDs, the current text focuses on an analytical challenge: the definition of an objective rule for assessing the…

This conceptual article considers data used in the humanities and the human sciences, which are fundamentally different from data in other disciplines, such as the sciences or medicine. Data in the humanities are, however, equally important to study and understand. Humanists and others studying human artifacts often face the dual challenge of both…