The use of inter-rater reliability (IRR) methods may provide an opportunity to improve the transparency and consistency of qualitative case study data analysis in terms of the rigor of how codes and constructs have been developed from the raw data. Few articles on qualitative research methods in the literature conduct IRR assessments or neglect to…

This paper argues for three aspects of tolerance with respect to QCA research: tolerance with respect to different approaches to QCA; producing QCA research with tolerance (work that is resistant to criticism); and for QCA researchers to be clear about the tolerance of the solutions they present -- especially in terms of calibration and truth…

Single-subjects design has the advantage of providing detailed, individualized analyses of intervention. However, creating a tutorial for single-subject design (SSD) is challenging, and an effective version for online delivery is currently unavailable. Here, we developed an experiential SSD tutorial designed for online delivery and evaluated its…

In the past decade, moderated mediation analysis has been extensively and increasingly employed in social and behavioral sciences. With its widespread use, it is particularly important to ensure the moderated mediation analysis will not bring spurious results. Spurious effects have been studied in both mediation and moderation analysis, but this…

Transcription is an integral part of much qualitative data analysis, yet rarely has it received close attention in debates over the use (or non-use) of "computer assisted qualitative data analysis software" (CAQDAS). This article draws upon a mixed-methods study that involved transcribing conversational interviews with carers, third…

Exploratory graph analysis (EGA) is a commonly applied technique intended to help social scientists discover latent variables. Yet, the results can be influenced by the methodological decisions the researcher makes along the way. In this article, we focus on the choice regarding the number of factors to retain: We compare the performance of the…

Undergraduate economics programs prepare students for future careers by developing competency working with data, or "data literacy." Our research examined the data literacy components of undergraduate economics programs at R1 and R2 universities in the United States (N = 190). We developed a protocol with core data skills and coded…

Ethnographic and interview research have made significant contributions to cumulative social science and influenced the public conversation around important social issues. However, debates rage over whether the standards of positivistic social science can or should be used to judge the rigor of interpretive methods. I begin this essay by briefly…

This article examines the use of the repertory grid technique as a method to investigate learner perceptions in language education research. An important issue raised in this article concerns how far a researcher's agenda may be unintentionally imposed onto a research study which is investigating learners' perceptions, and how far the ensuing data…

Despite the rising popularity of big data, there is speculation that evaluators have been slow adopters of these new statistical approaches. Several possible reasons have been offered for why this is the case: ethical concerns, institutional capacity, and evaluator capacity and values. In this method note, we address one of these barriers and aim…

High-quality systematic literature reviews provide a systematic process for identifying, synthesizing, and critiquing multiple studies and, in turn, inform theory, research, practice, and policy. With a focus on special education systematic reviews, we propose four core principles (i.e., coherence, contextualization, generativity, and…

For research evidence to inform decision making, an appraisal needs to be made of whether the claims are justified and whether they are useful to the decisions being made. This chapter provides a high level framework of core issues relevant to appraising the "fitness for purpose" of evidence claims. The framework includes (I) the…

One part of COVID-19's staggering impact on education has been to suspend or fundamentally alter ongoing education research projects. This article addresses how to analyze the simple but fundamental example of a multi-cohort study in which student assessment data for the final cohort are missing because schools were closed, learning was virtual,…

This article draws on a new materialist paradigm to explore bricolaging data from an early childhood research project through an immanent ethical lens. This lens enables the researcher to stretch towards non-hierarchical relationships in between subjects and objects, thinking and doing. A bricoleur explores and builds different…

To ensure validity in survey research, it is imperative that we properly educate doctoral students on best practices in data quality procedures. A 14-year analysis of 679 studies in the AIS "Basket of 8" journals noted undercommunication in the most pertinent procedures, consistent across journals and time. Given recent calls for…