Probes are follow-ups to survey questions used to gain insights on respondents' understanding of and responses to these questions. They are usually administered as open-ended questions, primarily in the context of questionnaire pretesting. Due to the decreased cost of data collection for open-ended questions in web surveys, researchers have argued…

In this study, we investigated whether incorporating eye tracking into cognitive interviewing is effective when pretesting survey questions. In the control condition, a cognitive interview was conducted using a standardized interview protocol that included pre-defined probing questions for about one-quarter of the questions in a 52-item…

Achievement goal theory is one of the most popular theories of achievement motivation. Techniques researchers have used to assess goals include standardized questionnaires and interviews. One curious finding is that participants whose self-report questionnaire responses strongly indicate they operate with a performance goal do not make performance…

This paper describes a method aimed at pointing out the quality of the mental models undergraduate engineering students deploy when asked to create explanations for phenomena or processes and/or use a given model in the same context. Student responses to a specially designed written questionnaire are quantitatively analyzed using…

Assessing pragmatic knowledge in the instructed setting is seen as a complex but necessary task, which requires the design of appropriate research methodologies to examine pragmatic performance. This study discusses the use of two different research methodologies, namely those of Discourse Completion Tests/Tasks (DCTs) and verbal reports. Research…

Examined how attitudes about the elderly are influenced by the procedural context in which attitude expression is assessed. Administered questionnaire evaluative attitude statements to 263 adults in two judgmental contexts. Found adults in the comparative context expressed more extreme negative attitudes than in the isolated context. (Author/RC)

The purpose of the "International Civic and Citizenship Education Study" (ICCS) is to investigate the ways in which young people are prepared to undertake their roles as citizens in a range of countries. The study will report on student achievement on a test of conceptual understandings and competencies in civic and citizenship education. It will…

The present experiment attempts to manipulate the saliency of subjects' recollection of their behavior by giving them a task to do which requires them to generate cognitions resulting in one of two sets of cognitive associations. (Author)

1,242 subjects, in five experiments plus a pilot study, saw a series of slides depicting a single auto-pedestrian accident. These experiments investigate how information supplied after an event influences a witness's memory for that event. Results suggest that information supplied a witness after an event, whether inconsistent or misleading, is…

The fundamental question in creating coding categories for an open-ended questionnaire is how to transform a complete transcript into manageable pieces of data. This paper describes the methodology involved in coding qualitative data derived from an evaluation of the Cognitive Approaches to School Leadership (CASL) program. The first task was to…

Examines issues associated with the design of and statistical inferences from embedded experiments which vary the questionnaire content into sample surveys of broad populations. Provides an example of context effects in attitude surveys, a summary, and conclusions. (RT)

Twelve linear algebra students were interviewed about the concept of a Solution of a System of Equations. The interviews were analyzed using APOS tools, in particular the ideas of Action, Process, Object and Schema, and Genetic Decomposition. The analysis of the interviews revealed several misconceptions of Solution. The analysis also revealed…

Cognitive interviews were conducted with 12 teachers and 66 of their students to learn how to improve National Assessment of Educational Progress (NAEP) questionnaire items asking about instructional practices and teacher development experiences. Items were purposively selected from those used in the 1996 and 1998 NAEP student and teacher…

This paper describes the task analysis of performance-based science tasks that were designed for the 1994 National Assessment of Educational Progress (NAEP) science assessment, now postponed until 1996, and field tested in 1993. A brief description of the science performance tasks is followed by a description of the task analyses performed and a…

A detailed description is provided of the variable, constructivism versus behaviorism (CVSB), which is part of the analysis in phase I of the Evaluation of Practices in Individualized Schooling. The variable is defined; the questionnaires on which the variable is based are listed; the procedures used to scale the variable are explained; the…