In this digital ITEMS module, Dr. Jacqueline Leighton and Dr. Blair Lehman review differences between think-aloud interviews to measure problem-solving processes and cognitive labs to measure comprehension processes. Learners are introduced to historical, theoretical, and procedural differences between these methods and how to use and analyze…

This article illustrates and differentiates the unique role cognitive interviews and think-aloud interviews play in developing and validating assessments. Specifically, we describe the use of (a) cognitive interviews to gather empirical evidence to support claims about the intended construct being measured and (b) think-aloud interviews to gather…

Rater cognition or "think-aloud" studies have historically been used to enhance rater accuracy and consistency in writing and language assessments. As assessments are developed for new, complex constructs from the "Next Generation Science Standards (NGSS)," the present study illustrates the utility of extending…

Cognitive interviewing (CI) provides a method of systematically collecting validity evidence of response processes for questionnaire items. CI involves a range of techniques for prompting individuals to verbalise their responses to items. One such technique is concurrent verbalisation, as developed in Think Aloud Protocol (TAP). This article…

Sequence of contextual information refers to the order of descriptions involved in the context component of items, such as sequence of events, sequence of intention and action, and sequence of effect and cause. To build on prior findings on how different sequences of contextual information could influence student test scores, we used eye tracking…

Over the last three decades, there has been increased attention on the collection and interpretation of "response processing data" to inform claims of learners' knowledge and skills (e.g., see Ercikan et al., 2010; Kobrin & Young, 2003; see also, Leighton, 2004). Response processing data are perhaps most consequential in the…

The purpose of this research is describing and comparing the listening-comprehension strategies used by the students who are identified as successful and less successful about listening. In the research, the case study pattern was used among the qualitative research approaches. In the study, different situations of each student within the same…

The purpose of this article is to describe the process of a proof construction. It is more specific on the failure of the process. Piaget's frameworks, assimilation and accommodation, were used to analyze it. Method of this research was qualitative method. Data were collected by asking five students working on problems of proof using think aloud…

Stimulating thinking related to mathematical content is the focus of many tasks in the mathematics classroom. Beyond such content-related thinking, promoting forms of higher order thinking is among the goals of mathematics instruction as well. So-called hybrid tasks focus on combining both goals: they aim at fostering mathematical thinking and…

The main goal of this study was to investigate how readers' visual attention distribution during reading of conflicting science information is related to their scientific reasoning behavior. A total of 25 university students voluntarily participated in the study. They were given conflicting science information about earthquake predictions to read…

The primary goal of this study was to explore the changes that four Korean university students made in their regulation of cognition during reading processes. The students were trained using explicit reading strategy instruction based on the CALLA model. To this end, first, metacognition was framed and categorized by the definition from Baker and…

The paper aims to demonstrate the value of cognitive interviewing (CI) as a survey pretesting method in comparative education research. Although rarely used by education researchers, CI has been successfully applied in different disciplines to evaluate and improve question performance. The method assumes that observing people's thought processes…

Purposes: This study uses think-alouds and cognitive interviews to provide validity evidence for an online formative assessment--i-Ready Standards Mastery (iSM) mini-assessments--which involves a heavy use of innovative items. iSM mini-assessments are intended to help teachers determine student understanding of each of the on-grade-level Common…

The purpose of this exploratory study was to investigate the cognitive processes used by individuals who read a refutational text about physics and demonstrated conceptual change learning. Four high school readers whose initial conceptions differed from the scientific conception of Newton's first law thought aloud while reading a refutational…

Williams's (2005) study on "learning without awareness" and three subsequent extensions (Faretta-Stutenberg & Morgan-Short, 2011; Hama & Leow, 2010; Rebuschat, Hamrick, Sachs, Riestenberg, & Ziegler, 2013) have reported conflicting results, perhaps in part due to differences in how awareness has been measured. The present…