Response process validity evidence provides a window into a respondent's cognitive processing. The purpose of this study is to describe a new data collection tool called a whole-class think aloud (WCTA). This work is performed as part of test development for a series of problem-solving measures to be used in elementary and middle grades. Data from…

We describe the use of think alouds to examine substantive processes involved in performance on a formative assessment of computational thinking (CT) designed to support self-regulated learning (SRL). Our task design model included three phases of work on a computational thinking problem: forethought, performance, and reflection. The cognitive…

This paper marks the second in a series of five reports detailing the work of the Optimizing Assessment for All (OAA) project at Brookings to strengthen education systems capacity to integrate 21st century skills into teaching and learning, using assessment as a lever for changing classroom practices. Twenty-first century skills (21CS) are now…

Pondering a math problem while she swings her sneakered feet from a chair, 12-year-old Andrea Guevara is helping researchers design an assessment that will shape the learning of 19 million students. The 8th grader, who came to the United States from Ecuador three years ago, is trying out two ways of providing English-language support on a…

A study was conducted to determine whether competently reasoning clinicians (clinical instructors in medical instruction) could identify reasons competently reasoning students would eliminate distractors, and explain how students would reason to select the keyed response when solving multiple-choice items. The think-aloud protocols of clinicians…

The domination of the information processing approach has shifted research from problem solving strategies to the structure and organization of knowledge that characterizes expertise. The purpose of this study was to compare the reasoning processes of 12 clinicians and 40 medical students as they responded to 6 positively stated multiple choice…

Information about how Graduate Record Examination (GRE) examinees solve verbal analogy problems was obtained in this study through protocol analysis. High- and low-ability subjects who had recently taken the GRE General Test were asked to "think aloud" as they worked through eight analogy items. These items varied factorially on the…

Analyses of the cognitive activity displayed by students on existing innovative assessments in science were conducted. The project has worked with pilot alternative assessment programs in Connecticut and California to document the match or mismatch between the skills and processes that the assessment is designed to tap and those actually elicited.…