In this study, we examine "what went wrong" in our professional development program for encouraging cognitively demanding instruction, focusing on the difficulties we encountered in using an observational tool for evaluating this type of instruction and reaching inter-rater reliability. We do so through the lens of a discursive theory of…

In this digital ITEMS module, Dr. Jacqueline Leighton and Dr. Blair Lehman review differences between think-aloud interviews to measure problem-solving processes and cognitive labs to measure comprehension processes. Learners are introduced to historical, theoretical, and procedural differences between these methods and how to use and analyze…

Taxonomies of cognitive demand are frequently used to ensure that assessment tasks include questions ranging from low to high cognitive demand. This paper investigates inter-rater agreement among four evaluators on the cognitive demand of the South African National Senior Certificate Life Sciences examinations after training, practice and…

Critical thinking is recognised as instrumental for positive, personal and professional, long-term outlooks. It is also widely accepted that the development of students' critical thinking skills can be achieved through explicit interventions. This paper documents the outcomes of a pilot study that investigated the value and impact of an…

Recent education reforms including the "Next Generation Science Standards" emphasize integrating engineering into K-12 science instruction. Although prior studies have characterized engineering design (ED) in K-12 settings, no validated protocols attempt to characterize ED integration in secondary science classrooms. The present study…

Computer-based testing is an expanding use of technology offering advantages to teachers and students. We studied Calculus II classes for science, technology, engineering, and mathematics majors using different testing modes. Three sections with 324 students employed: paper-and-pencil testing, computer-based testing, and both. Computer tests gave…

This study provides preliminary evidence for reliability and validity of the Hawaii Early Learning Profile Strands 0-3 (HELP Strands 0-3), an assessment instrument for young children. First, the degree of interobserver agreement for a sample of representative HELP items was examined; results indicated that HELP scoring was dependable and…

This study compared students with specific learning disorders (SLDs) and attention-deficit/hyperactivity disorder (ADHD) based on the Formal Elements Art Therapy Scale (FEATS). Thirty students with SLDs, 30 students with ADHD, and 30 students without SLDs/ADHD in Babol (Iran) completed the Person Picking an Apple from a Tree (PPAT) assessment.…

The Next Generation Science Standards propose a multidimensional model of science learning, comprised of Core Disciplinary Ideas, Science and Engineering Practices, and Crosscutting Concepts (NGSS Lead States, 2013). Accordingly, there is a need for student assessment aligned with the new standards. Creating assessments that validly and reliably…

Rater cognition or "think-aloud" studies have historically been used to enhance rater accuracy and consistency in writing and language assessments. As assessments are developed for new, complex constructs from the "Next Generation Science Standards (NGSS)," the present study illustrates the utility of extending…

The purpose of this meta-analysis was to examine the relation between mathematics anxiety (MA) and mathematics performance among school-aged students, and to identify potential moderators and underlying mechanisms of such relation, including grade level, temporal relations, difficulty of mathematical tasks, dimensions of MA measures, effects on…

Medical education relies heavily on assessment formats that require raters to assess the competence and skills of learners. Unfortunately, there are often inconsistencies and variability in the scores raters assign. To ensure the scores from these assessment tools have validity, it is important to understand the underlying cognitive processes that…

This study aims at analyzing assessment questions included in an Arabic language textbook (Communication Skills) for eight grade in the Jordanian schools according to the levels of knowledge in Bloom's Taxonomy to reveal the percentages of distributing of these questions against this taxonomy. In order to achieve the purposes of this study, the…

The researchers in this study examined the influence of questions designed with the Practical Inquiry Model (PIM), compared with the regular (playground) questions, on students' levels of cognitive presence in online discussions. Students' discussion postings were collected and categorized according to the four levels of cognitive presence:…

Students approach learning in different ways, depending on the experienced learning situation. A deep approach is geared toward long-term retention and conceptual change while a surface approach focuses on quickly acquiring knowledge for immediate use. These approaches ultimately affect the students' academic outcomes. This study takes a…