The objective in the current paper is to examine the processes of how our research team negotiated meaning using an iterative design approach as we established, developed, and refined a rubric to capture comprehension processes and strategies evident in students' verbal protocols. The overarching project comprises multiple data sets, multiple…

Performance-based assessment is a common approach to assess the development and acquisition of practice competencies among health professions students. Judgments related to the quality of performance are typically operationalized as ratings against success criteria specified within a rubric. The extent to which the rubric is understood,…

Despite problem solving being a core skill in chemistry, students often struggle to solve chemistry problems. This difficulty may arise from students trying to solve problems through memorising algorithms. Goldilocks Help serves as a problem-solving scaffold that supports students through structured problem solving and its elements, such as…

This study touches upon three major themes in the field of information literacy (IL): the assessment of IL, the association between IL knowledge and skills, and the dimensionality of the IL construct. Three quantitative measures were developed and tested with several samples of university students to assess knowledge and skills for core facets of…

In this study, we investigated factors that influence raters' application of the scoring criteria of an Explicit Instruction (EI) observation protocol using many-faceted Rasch measurement (MFRM) and think aloud analysis. Specifically, we investigated the extent to which raters are able to consistently represent the scoring criteria in the EI…

The aspects of inquiry based learning (IBL) are vigorously and frequently in the focus of recent studies. With the use of inquiry in mathematics in the daily school practice, some further questions are arising there: What kind of problems can be useful for an analysis of students' competencies in the field of IBL and how to assess the performed…

We studied the impact of metacognitive reflections on recently-completed work as a way to improve the retention of newly learned problem-solving techniques. Students video recorded themselves talking through problems immediately after finishing them, completed ongoing problem-solving strategy maps or problem-sorting exercises, and filled out…

Rater cognition or "think-aloud" studies have historically been used to enhance rater accuracy and consistency in writing and language assessments. As assessments are developed for new, complex constructs from the "Next Generation Science Standards (NGSS)," the present study illustrates the utility of extending…

We introduce a template to (i) scaffold the problem solving process for students in the physics 1 course, and (ii) serve as a generic rubric for measuring how expertlike students are in their problem solving. This template is based on empirical studies of the problem solving practices of expert scientists and engineers, unlike most existing…

This study examines the decision-making behaviors of raters with varying levels of experience while assessing EFL essays of distinct qualities. The data were collected from 28 raters with varying levels of rating experience and working at the English language departments of different universities in Turkey. Using a 10-point analytic rubric, each…

The aim of this study is to examine the variability in and reliability of scores assigned to different quality EFL compositions by EFL instructors and their rating behaviors. Using a mixed research design, quantitative data were collected from EFL instructors' ratings of 30 compositions of three different qualities using a holistic scoring rubric.…

The purpose of the District Capacity Assessment (DCA) Technical Manual is to provide background information on the technical adequacy of the DCA (Ward et al., 2015). This current version draws upon a rich history and background of previous work assessing district capacity. Notably, the current version includes significant modifications from…

Purpose: In light of growing interest in the maker movement and electronic textiles (e-textiles) as an educational technology, the purpose of this paper is to characterize competence change in undergraduate students who participated in a semester-length course that used e-textiles. Design/methodology/approach: This qualitative and exploratory…

The think-aloud approach to classroom writing assessment is designed to expand teachers' perspectives on adolescent students as writers and help them integrate instruction and assessment in a timely way. Emphasizing learning over evaluation, it is especially well-suited to revealing students' strengths and helping them overcome common challenges…

Rater variation in performance assessment can impact test-takers' scores and compromise assessments' fairness and validity (Crooks, Kane, & Cohen, 1996). Rater variation can also undermine a test's validity and fairness; therefore, it is important to investigate raters' scoring patterns in order to inform rater training. Substantial work has…