We report on and validate a system for ranking the cognitive demand of mathematical tasks. In our framework, task rankings are determined by the sequences of units and unit transformations students might use to solve each task. Using this framework, we ranked a set of 10 fractions tasks. We then interviewed 12 pre-service teachers to assess the…

Background & Context: We describe the rationale, design, and initial validation of computational thinking (CT) assessments to pair with curricular lessons that integrate fractions and CT. Objective: We used cognitive models of CT (learning trajectories; LTs) to design assessments and obtained evidence to support a validity argument. Method: We…

"Technology-enhanced learning" (TEL) is a broad term to denote the incorporation of digital technology to mediate activities that support education. Educational researchers need to validate critical assumptions about any new system that involves TEL early and throughout its development course to make quick, informed, de-risked decisions…

This article illustrates and differentiates the unique role cognitive interviews and think-aloud interviews play in developing and validating assessments. Specifically, we describe the use of (a) cognitive interviews to gather empirical evidence to support claims about the intended construct being measured and (b) think-aloud interviews to gather…

Since the development of the technological pedagogical content knowledge (TPACK) framework, researchers have been developing a variety of instruments to measure the TPACK of pre-service and in-service teachers. The task of developing an efficient, reliable, and valid instrument is difficult. Even validated instruments require guidance for…

Mathematical calculations are an important part of chemistry. Those problems are difficult for students, especially if the task is set with a limiting reactant. The aim of this study was development of a Procedure for evaluation of cognitive complexity of the Stoichiometric Tasks with a Limiting Reactant. The procedure created included an…

Developmental science demonstrates that younger children (ages 4 to 8) are capable of making meaning of their contexts and the self. Yet, younger children's meaning making is largely absent from intervention and implementation research on early childhood policies. This absence is notable given the rise in investment in early childhood policies and…

This article reports a validity study of History Assessments of Thinking (HATs), which are short, constructed-response assessments of historical thinking. In particular, this study focuses on aspects of cognitive validity, which is an examination of whether assessments tap the intended constructs. Think-aloud interviews with 26 high school…

This article describes the results of Action Research conducted in an ESP classroom of Dhofar University located in Oman. Following the call of Oman Vision 2040 to emphasize educational practices that promote the development of higher-order cognitive processes, this study raises the following question: Can an online multiple choice question (MCQ)…

Standardised educational assessments are now widespread, yet their development has given comparatively more consideration to what to assess than how to optimally assess students' competencies. Existing evidence from behavioural studies with children and neuroscience studies with adults suggest that the method of assessment may affect neural…

In this Brief Report, we share the main findings from our line of research into embodied cognition and proof activities. First, attending to students' gestures during proving activities can reveal aspects of mathematics thinking not apparent in their speech, and analyzing gestures after proof production can contribute significantly to our…

Relational reasoning is the foundational cognitive ability to discern meaningful patterns within an informational stream, but its reliable and valid measurement remains problematic. In this investigation, the measurement of relational reasoning unfolded in three stages. Stage 1 entailed the establishment of a research-based conceptualization of…

Validity is a central principle of assessment relating to the appropriateness of the uses and interpretations of test results. Usually, one of the inferences that we wish to make is that the score reflects the extent of a student's learning in a given domain. Thus, it is important to establish that the assessment tasks elicit performances that…

International large-scale assessments of achievement often have a large degree of differential item functioning (DIF) between countries, which can threaten score equivalence and reduce the validity of inferences based on comparisons of group performances. It is important to understand potential sources of DIF to improve the validity of future…

Recent advances in cognitive science and psychometrics have expanded the possibilities for the next generation of literacy assessment as an integrated domain (Bennett, 2011a; Deane, Sabatini, & O'Reilly, 2011; Leighton & Gierl, 2011; Sabatini, Albro, & O'Reilly, 2012). In this paper, we discuss four key areas supporting innovations in…