Performance-based assessment is a common approach to assess the development and acquisition of practice competencies among health professions students. Judgments related to the quality of performance are typically operationalized as ratings against success criteria specified within a rubric. The extent to which the rubric is understood,…

This article illustrates and differentiates the unique role cognitive interviews and think-aloud interviews play in developing and validating assessments. Specifically, we describe the use of (a) cognitive interviews to gather empirical evidence to support claims about the intended construct being measured and (b) think-aloud interviews to gather…

Models of physical systems are used to explain and predict experimental results and observations. The Modeling Framework for Experimental Physics describes the process by which physicists revise their models to account for the newly acquired observations, or change their apparatus to better represent their models when they encounter discrepancies…

Classroom-based assessments have the potential to enhance validity by facilitating the assessment of important skills that are difficult to assess in written examinations. Such assessments tend to be marked by teachers. To ensure consistent marking standards, quality assurance procedures are needed. In the context of continued debate over the…

Purposes: This study uses think-alouds and cognitive interviews to provide validity evidence for an online formative assessment--i-Ready Standards Mastery (iSM) mini-assessments--which involves a heavy use of innovative items. iSM mini-assessments are intended to help teachers determine student understanding of each of the on-grade-level Common…

The purpose of this paper is to assess the validity of multiple-choice questions in measuring fourth grade students' ability to interpret graphs related to physical science topics such as motion and temperature. We administered a test including 6 multiple-choice questions to 28 fourth grade students. Students were asked to explain their thinking…

Problem solving is not simply a process that ends when an answer is found; it is a scientific process that evolves from understanding the problem to evaluating the solution. This process is affected by several factors. Among these, one of the most substantial is belief. The purpose of this study was to evaluate the beliefs of high school students…

This study investigated the effects of the use of scaffolds in written classroom assessments through the voices of both native English speakers and English language learners from two middle schools. Students responded to assessment tasks in writing, by speaking aloud using think aloud protocols, and by reflecting in a post-assessment interview.…

We report on the development, validation, and implementation of a collection of test items designed to detect misconceptions related to first-year computer science courses. To this end, we reworked the development scheme proposed by Almstrum et al. ("SIGCSE Bulletin" 38(4):132-145, 2006) to include students' artifacts and to…

The Berge et al. article in this volume presents the functional construct of writing that underlies summative and formative assessment of writing as a key competency in Norway. A functional construct implies that specific acts of writing and their purposes constrain what is a relevant selection among the semiotic resources that writing generally…

This article considers the design process for new formative history assessments. Over the course of 3 years, my colleagues from the Stanford History Education Group and I designed, piloted, and revised dozens of "History Assessments of Thinking" (HATs). As we created HATs, we sought to gather information about their cognitive validity,…

Integrated second-language writing tasks elicit writing performances that involve other abilities such as reading or listening. Thus, understanding the role of these other abilities is necessary for interpreting performance on such tasks. This study used an inductive analysis of think-aloud protocol data and interviews to uncover the reading…

This article investigates whether verbal protocol analysis is a sound methodology in the context of researching examination marking processes. As part of a larger study, six examiners marked fifty scripts at home and some further scripts during meetings (some silently and some whilst thinking aloud) for two A level examinations. Some scripts were…

This article reports on a study aimed at testing the hypothesis that, because of strategic and temporal variables, composing rate and text quantity may not be valid measures of writing fluency. A second objective was to validate the mean length of writers' translating episodes as a process-based indicator that mirrors their fluent written…

Instructors in electrical and computer engineering and in computer science have developed innovative methods to teach digital logic circuits. These methods attempt to increase student learning, satisfaction, and retention. Although there are readily accessible and accepted means for measuring satisfaction and retention, there are no widely…