NotesFAQContact Us
Collection
Advanced
Search Tips
Showing all 6 results Save | Export
Rethinking Think-Alouds: The Often-Problematic Collection of Response Process Data
Peer reviewed Peer reviewed
Direct linkDirect link
Leighton, Jacqueline P. – Applied Measurement in Education, 2021
The objective of this paper is to comment on the think-aloud methods presented in the three papers included in this special issue. The commentary offered stems from the author's own psychological investigations of unobservable information processes and the conditions under which the most defensible claims can be advanced. The structure of this…
Descriptors: Protocol Analysis, Data Collection, Test Construction, Test Validity
Determining Reliability of Daily Measures: An Illustration with Data on Teacher Stress
Peer reviewed Peer reviewed
Direct linkDirect link
van Alphen, Thijmen; Jak, Suzanne; Jansen in de Wal, Joost; Schuitema, Jaap; Peetsma, Thea – Applied Measurement in Education, 2022
Intensive longitudinal data is increasingly used to study state-like processes such as changes in daily stress. Measures aimed at collecting such data require the same level of scrutiny regarding scale reliability as traditional questionnaires. The most prevalent methods used to assess reliability of intensive longitudinal measures are based on…
Descriptors: Test Reliability, Measures (Individuals), Anxiety, Data Collection
Analyzing Student Response Processes to Evaluate Success on a Technology-Based Problem-Solving Task
Peer reviewed Peer reviewed
Direct linkDirect link
Han, Yuting; Wilson, Mark – Applied Measurement in Education, 2022
A technology-based problem-solving test can automatically capture all the actions of students when they complete tasks and save them as process data. Response sequences are the external manifestations of the latent intellectual activities of the students, and it contains rich information about students' abilities and different problem-solving…
Descriptors: Technology Uses in Education, Problem Solving, 21st Century Skills, Evaluation Methods
Think Alouds: Informing Scholarship and Broadening Partnerships through Assessment
Peer reviewed Peer reviewed
Direct linkDirect link
Bostic, Jonathan David – Applied Measurement in Education, 2021
Think alouds are valuable tools for academicians, test developers, and practitioners as they provide a unique window into a respondent's thinking during an assessment. The purpose of this special issue is to highlight novel ways to use think alouds as a means to gather evidence about respondents' thinking. An intended outcome from this special…
Descriptors: Protocol Analysis, Cognitive Processes, Data Collection, STEM Education
Gathering Response Process Data for a Problem-Solving Measure through Whole-Class Think Alouds
Peer reviewed Peer reviewed
Direct linkDirect link
Bostic, Jonathan David; Sondergeld, Toni A.; Matney, Gabriel; Stone, Gregory; Hicks, Tiara – Applied Measurement in Education, 2021
Response process validity evidence provides a window into a respondent's cognitive processing. The purpose of this study is to describe a new data collection tool called a whole-class think aloud (WCTA). This work is performed as part of test development for a series of problem-solving measures to be used in elementary and middle grades. Data from…
Descriptors: Data Collection, Protocol Analysis, Problem Solving, Cognitive Processes
Designing, Evaluating, and Deploying Automated Scoring Systems with Validity in Mind: Methodological Design Decisions
Peer reviewed Peer reviewed
Direct linkDirect link
Rupp, André A. – Applied Measurement in Education, 2018
This article discusses critical methodological design decisions for collecting, interpreting, and synthesizing empirical evidence during the design, deployment, and operational quality-control phases for automated scoring systems. The discussion is inspired by work on operational large-scale systems for automated essay scoring but many of the…
Descriptors: Design, Automation, Scoring, Test Scoring Machines