This paper argues for three aspects of tolerance with respect to QCA research: tolerance with respect to different approaches to QCA; producing QCA research with tolerance (work that is resistant to criticism); and for QCA researchers to be clear about the tolerance of the solutions they present -- especially in terms of calibration and truth…

Purpose: This paper illustrates how Guba and Lincoln's parallel criteria for establishing trustworthiness, can be adapted and applied to qualitative research on indigenous social protection systems. It provides insights for social protection researchers, exploring plausible qualitative research rigor evaluation criteria, on plausible alternatives.…

The purpose of this paper is to introduce the Five-Level Qualitative Data Analysis (5LQDA) method for ATLAS.ti as a way to intentionally design methodological approaches applicable to the field of evaluation. To demonstrate my analytical process using ATLAS.ti, I use examples from an existing evaluation of a STEM Peer Learning Assistant program.…

Psycholinguistics is the research of language in the mind: The architecture of storing the different components of language in our memory, and the ways we access that knowledge when processing and producing language. Psycholinguists usually conduct quantitative experimental research in which certain standards of experimental design, data…

A reproducible analysis is one in which an independent entity, using the same data and the same statistical code, would obtain the exact same result as the previous analyst. Reproducible analyses utilize script-based analyses and open data to aid in the reproduction of the analysis. A reproducible analysis does not ensure the same results are…

This article develops a set-theoretic approach to Bayes's theorem and Bayesian process tracing. In the approach, hypothesis testing is the procedure whereby one updates beliefs by narrowing the range of states of the world that are regarded as possible, thus diminishing the domain in which the actual world can reside. By explicitly connecting…

Background and context: The variables that researchers measure and how they measure them are central in any area of research, including computing education. Which research questions can be asked and how they are answered depends on measurement. Objective: To summarize the commonly used variables and measurements in computing education and to…

This article provides insight into how an adequate approach to selecting methods can establish credible and actionable evidence. The authors offer strategies to effectively support Extension professionals, including program developers and evaluators, in being more deliberate when selecting appropriate qualitative and quantitative methods. In…

Purpose: The purpose of this paper is to present a framework intended to guide students and novice researchers in learning about the necessary dimensions for assessing the rigour of qualitative research studies. The framework has four dimensions -- (T)rustworthiness, (A)uditability, (C)redibility and (T)ransferability. The development of TACT is…

With today's technology, Extension professionals have a variety of tools available for program evaluation. This article describes an innovative platform called VoiceThread that has been used in many classrooms but also is useful for conducting virtual focus group research. I explain how this tool can be used to collect qualitative participant…

The search for necessary and sufficient causes of some outcome of interest, referred to as "configurational comparative research," has long been one of the main preoccupations of evaluation scholars and practitioners. However, only the last three decades have witnessed the evolution of a set of formal methods that are sufficiently…

Purpose: Participatory extension programmes are widely used to promote change in the agricultural sector, and an important question is how best to measure the effectiveness of such programmes after implementation. This study seeks to understand the current state of practice through a review of ex post evaluations of participatory extension…

Vignettes, brief descriptions of fictional characters and situations, serve as a tool to study people's lives, perceptions, beliefs, and attitudes about specific situations. Although not widely used in library and information science (LIS) research, vignettes can depersonalize responses to controversial situations or behavioral responses related…

Our research examines the effectiveness of intake variables used by a Canadian post-degree teacher education program over the period of 3 years to select candidates for entry into the program. Using a mixed-methods approach, we compared intake variables (grade point average, written response, work experience, reference letters, academic…

Understanding the reason(s) for Differential Item Functioning (DIF) in the context of measurement is difficult. Although identifying potential DIF items is typically a statistical endeavor, understanding the reasons for DIF (and item repair or replacement) might require investigations that can be informed by qualitative work. Such work is…