The intelligence community uses "structured analytic techniques" to help analysts think critically and avoid cognitive bias. However, little evidence exists of how techniques are applied and whether they are effective. We examined the use of the analysis of competing hypotheses (ACH)--a technique designed to reduce "confirmation…

Individuals trained in the use of cognitive tests should be able to complete an assessment without making administrative, scoring, or recording errors. However, an examination of 295 Wechsler protocols completed by graduate students and practicing school psychologists revealed that errors are the norm, not the exception. The most common errors…

Test score users often demand the reporting of subscores due to their potential diagnostic, remedial, and instructional benefits. Therefore, there is substantial pressure on testing programs to report subscores. However, professional standards require that subscores have to satisfy minimum quality standards before they can be reported. In this…

The natural experiment proposed in this article extracts three stories from boxes of "100 paper clips". The activity requires students to apply three lessons from inferential statistics, starting with a hypothesis test and including confidence intervals as well as tolerance intervals.

Given the importance of developing civically engaged and knowledgeable citizens, many have sought to improve the quality of students' civic education experiences. This article examines one particular effort spearheaded by the Joe Foss Institute, the Civic Education Initiative (CEI). Since the CEI was announced, 18 states have passed legislation…

This study developed a model of team mental models' influence on team adaptive performance at different stages. It proposed that mental models have different relationships with adaptive performance as a function of the performance stage when teams experience changes: early on (i.e., during situation assessment) more divergent mental models are…

The testing effect occurs when students, given information to learn and then practice during a test, perform better on a subsequent content post-test than students who restudy the information as a substitute for the practice test. The effect is often weaker or reversed if immediate rather than delayed post-tests are used. The weakening may be due…

Developer-friendly professional authoring tools have greatly simplified entertainment videogame development. However, this simplification had a limited impact on serious games, which require the active collaboration of developers with educators and other stakeholders. To address this issue, we present uAdventure, an easy-to-use game development…

Scoring of teachers' noticing responses is typically burdened with rater bias and reliance upon interrater consensus. The authors sought to make the scoring process more objective, equitable, and generalizable. The development process began with a description of response characteristics for each professional noticing component disconnected from…

Benefiting from item preknowledge (e.g., McLeod, Lewis, & Thissen, 2003) is a major type of fraudulent behavior during educational assessments. This paper suggests a new statistic that can be used for detecting the examinees who may have benefitted from item preknowledge using their response times. The statistic quantifies the difference in…

Intelligence Tutoring Systems (ITSs) focus on promoting knowledge acquisition, while providing relevant feedback during students' practice. Self-explanation practice is an effective method used to help students understand complex texts by leveraging comprehension. Our aim is to introduce a deep learning neural model for automatically scoring…

The Regulations of the Commissioner of Education provide that an elementary-level science test is to be administered in Grade 4 to serve as a basis for determining students' needs for academic intervention services in science. The New York State Grade 4 Elementary-Level Science Test consists of two required components: a Written Test and a…

It has been suggested that tacit knowledge may be a good predictor of performance in college. The purpose of this study was to investigate the extent to which a situational judgment test developed to measure tacit knowledge correlates with predictors and indicators of college performance. This situational judgment test includes eight situations…

It is now widely accepted that the standard inferential toolkit used by the scientific research community--null-hypothesis significance testing (NHST)--is not fit for purpose. Yet despite the threat posed to the scientific enterprise, there is no agreement concerning alternative approaches for evidence assessment. This lack of consensus reflects…

Children have traditionally been viewed as less reliable witnesses than are adults. More recently, a concept known as developmental reversals, has brought this view into question. Developmental reversals have demonstrated that in certain contexts, children produce fewer false memories than adults. The primary paradigm used to demonstrate…