The use of inter-rater reliability (IRR) methods may provide an opportunity to improve the transparency and consistency of qualitative case study data analysis in terms of the rigor of how codes and constructs have been developed from the raw data. Few articles on qualitative research methods in the literature conduct IRR assessments or neglect to…

Single-subjects design has the advantage of providing detailed, individualized analyses of intervention. However, creating a tutorial for single-subject design (SSD) is challenging, and an effective version for online delivery is currently unavailable. Here, we developed an experiential SSD tutorial designed for online delivery and evaluated its…

Transcription is an integral part of much qualitative data analysis, yet rarely has it received close attention in debates over the use (or non-use) of "computer assisted qualitative data analysis software" (CAQDAS). This article draws upon a mixed-methods study that involved transcribing conversational interviews with carers, third…

Undergraduate economics programs prepare students for future careers by developing competency working with data, or "data literacy." Our research examined the data literacy components of undergraduate economics programs at R1 and R2 universities in the United States (N = 190). We developed a protocol with core data skills and coded…

Despite the rising popularity of big data, there is speculation that evaluators have been slow adopters of these new statistical approaches. Several possible reasons have been offered for why this is the case: ethical concerns, institutional capacity, and evaluator capacity and values. In this method note, we address one of these barriers and aim…

One part of COVID-19's staggering impact on education has been to suspend or fundamentally alter ongoing education research projects. This article addresses how to analyze the simple but fundamental example of a multi-cohort study in which student assessment data for the final cohort are missing because schools were closed, learning was virtual,…

To ensure validity in survey research, it is imperative that we properly educate doctoral students on best practices in data quality procedures. A 14-year analysis of 679 studies in the AIS "Basket of 8" journals noted undercommunication in the most pertinent procedures, consistent across journals and time. Given recent calls for…

Interest in applied educational research methodology is growing as educators and researchers strive to seek empirical evidence about what is effective teaching within distinctive contexts. However, for beginner researchers who are interested in conducting case studies within educational settings and are looking for an appropriate starting point,…

In this commentary, we propose a framework for applying the Complex Dynamical Systems (CDS) approach in educational research. Drawing on the conceptual articles in the special issue for ontological, theoretical, and methodological principles, and on the empirical articles for examples of these principles' application, we suggest six interdependent…

In undergraduate research settings, students are likely to encounter anomalous data, that is, data that do not meet their expectations. Most of the research that directly or indirectly captures the role of anomalous data in research settings uses post-hoc reflective interviews or surveys. These data collection approaches focus on recall of past…

In education settings, treatments are often non-randomly assigned to clusters, such as schools or classrooms, while outcomes are measured for students. This research design is called the clustered observational study (COS). We examine the consequences of common support violations in the COS context. Common support violations occur when the…

When a psychologist announces a new research finding, it is often based on a rejected null hypothesis. However, if that hypothesis is true, the claim is a false alarm. Many students mistakenly believe that the probability of committing a false alarm equals alpha, the criterion for statistical significance, which is typically set at 5%. Instructors…

Close reading has long been heralded as a humanities-specific methodology with significant potential for SoTL. This essay fills a gap in SoTL literature with a full case study demonstrating what, exactly, close reading shows us about our data that social science-based quantitative and qualitative analyses may not. Close reading-based analysis of…

Physics education research (PER) has long used concept inventories to investigate student learning over time and to compare performance across various student subpopulations. PER has traditionally used normalized gain to explore these questions but has begun to use established methods from other fields, including Cohen's "d," multiple…

Activity Theory (AT) is increasingly employed as a lens to guide data analysis in information systems (IS) studies. The theory is also applied to assess and evaluate information systems and technologies (IS/IT) in organisations. Even though its popularity continues to increase in both business and academic domains, there is no formal or assessment…