Objective To develop an online discrete choice experiment (DCE) to elicit preferences for the format and content of tobacco treatment and determine the feasibility of the survey in face-to-face online interviews among college students. format content Participants A convenience sample of 28 college students. Methods A pilot online DCE survey with…

This article considers the design process for new formative history assessments. Over the course of 3 years, my colleagues from the Stanford History Education Group and I designed, piloted, and revised dozens of "History Assessments of Thinking" (HATs). As we created HATs, we sought to gather information about their cognitive validity,…

Findings derived from self-reported, structured survey questionnaires are commonly used in evaluation and applied research to inform policy-making and program decisions. Although there are a variety of issues related to the quality of survey evidence (e.g., sampling precision), the validity of response processes--how respondents process thoughts…

Developing a cognitive model of task performance is an important and often overlooked phase in assessment design; failing to establish such a model can threaten the validity of the inferences made from the scores produced by an assessment (e.g., Leighton, 2004). Conducting think aloud interviews (TAIs), where students think aloud while completing…

Instructors in electrical and computer engineering and in computer science have developed innovative methods to teach digital logic circuits. These methods attempt to increase student learning, satisfaction, and retention. Although there are readily accessible and accepted means for measuring satisfaction and retention, there are no widely…

Analyses of 10 raters' reasoning during think-aloud interviews provided evidence to support a model of the fundamental processes involved in rating standards-based, nonprescriptive portfolios. This process model provides a framework within which to conceptualize sound-rater reasoning and to identify reasoning that distorts the meaning of scores.…