We empirically examined the effectiveness of how the Analysis of Competing Hypotheses (ACH) technique structures task information to help reduce confirmation bias (Study 1) and the portrayal of intelligence analysts as suffering from such bias (Study 2). Study 1 (N = 161) showed that individuals presented with hypotheses in rows and evidence items…

A necessary step in applying bi-factor models is to evaluate the need for domain factors with a general factor in place. The conventional null hypothesis testing (NHT) was commonly used for such a purpose. However, the conventional NHT meets challenges when the domain loadings are weak or the sample size is insufficient. This article proposes…

Humans actively seek information to reduce uncertainty, providing insight on how our decisions causally affect the world. While we know that episodic memories can help support future goal-oriented behaviors, little is known about how hypothesis testing during exploration influences episodic memory. To investigate this question, we designed a…

Therapeutic reasoning is when the purpose, task, or goal for engaging in reasoning is to determine the patient's management plan. As the field's understanding of the process of therapeutic reasoning is less well understood, we focused on studies that collected data on the process of therapeutic reasoning. To synthesize previous studies of…

The analysis of competing hypotheses (ACH) has been suggested to be a method that can protect against confirmation bias in the context of intelligence analysis. In the current study, we aimed to determine whether ACH could counter confirmation bias in the reasoning with evidence in the context of criminal law proceedings. Law students (N = 191)…

This study explored the decision-making processes of soldiers as they evaluated scenarios containing information offering various levels of certainty. Soldiers with different levels of experience read scenarios that presented potential threats and then identified the priority threat, their confidence in this identification, and the cues they used…

Evidence-based assessment (EBA) requires that investigators employ scientific theories and research findings to guide decisions about what domains to measure, how and when to measure them, and how to make decisions and interpret results. To implement EBA, investigators need high-quality assessment tools along with evidence-based processes. We…

Education researchers frequently have to choose between statistical models for their data, and in many cases the candidate models or parameters can be listed in a sequence, m=1,...,M, from less preferable choices to more. For instance, in choosing a bandwidth for regression discontinuity designs, researchers would favor the largest possible…

The intelligence community uses "structured analytic techniques" to help analysts think critically and avoid cognitive bias. However, little evidence exists of how techniques are applied and whether they are effective. We examined the use of the analysis of competing hypotheses (ACH)--a technique designed to reduce "confirmation…

Background: A clinical swallow examination (CSE) provides integral information that informs the diagnostic decision-making process within dysphagia management. However, multiple studies have highlighted a high degree of reported variability within the CSE process. It has been hypothesized that such variability may be the result of the clinical…

How do scientists generate and weight candidate queries for hypothesis testing, and how does learning from observations or experimental data impact query selection? Field sciences offer a compelling context to ask these questions because query selection and adaptation involves consideration of the spatiotemporal arrangement of data, and therefore…

When comparing social science phenomena through a time perspective, absolute and relative difference (RD) are the two typical presentation formats used to communicate interpretations to the audience, while time distance (TD) is the least frequently used of such formats. This article argues that the chosen presentation format is extremely important…

Here, we present a strong test of the hypothesis that sacrificial moral dilemmas are solved using the same value-based decision mechanism that operates on decisions concerning economic goods. To test this hypothesis, we developed Psychological Value Theory. Psychological Value Theory is an expansion and generalization of Cohen and Ahn's (2016)…

We tested the associations among the career decision-making difficulties, the career decision status, and either (a) the career decision-making profiles of 575 young adults, or (b) the coping strategies of 379 young adults. As hypothesized, a more advanced decision status was negatively associated with both career decision-making difficulties…

Automatic machine learning (AutoML) methods automate the time-consuming, feature-engineering process so that researchers produce accurate student models more quickly and easily. In this paper, we compare two AutoML feature engineering methods in the context of the National Assessment of Educational Progress (NAEP) data mining competition. The…