There is increasing interest in using factor scores in structural equation models and there have been numerous methodological papers on the topic. Nevertheless, sum scores, which are computed from adding up item responses, continue to be ubiquitous in practice. It is therefore important to compare simulation results involving factor scores to…

Trend scoring constructed response items (i.e. rescoring Time A responses at Time B) gives rise to two-way data that follow a product multinomial distribution rather than the multinomial distribution that is usually assumed. Recent work has shown that the difference in sampling model can have profound negative effects on statistics usually used to…

In evaluating the learning achievement of programming-thinking skills, the method of using a rubric that describes evaluation items and evaluation stages is widely employed. However, few studies have evaluated the reliability, validity, and consistency of the rubrics themselves. In this study, we introduced a statistical method for evaluating the…

The analysis of exemplars of different quality is a potentially powerful tool in enabling students to understand assessment expectations and appreciate academic standards. Through a systematic review methodology, this paper synthesises exemplar-based research designs, exemplar implementation and the educational effects of exemplars. The review of…

The aim of the study was to examine the effects of rubric co-creation on peer feedback. From a social constructivist perspective, rubric co-creation might have the ability to promote the quality of feedback messages, interactivities within feedback processes, and uptake of feedback information in a peer-based collaborative setting of higher…

California's Commission on Teacher Credentialing (Commission) requires all programs of preliminary multiple and single subject teacher preparation to use a Commission-approved Teaching Performance Assessment (TPA) as one of the program completion requirements for prospective teacher candidates. Three TPA models were approved by the Commission: (1)…

While marking rubrics offer a range of potential benefits for students and staff, educators are working to develop a richer understanding of the most effective ways to unlock these benefits. This study contributes by examining the link between rubrics and performance through the lens of student engagement. The work introduced an assessment rubric…

This paper presents assessment data from a first-year writing library partnership to examine the relationship between student source use and written synthesis. It finds that first-year students could locate peer-reviewed, scholarly sources but that these sources were poorly integrated in their arguments--if they were used at all. In contrast, it…

Rubrics are widely used in higher education to assess performance in project-based learning environments. To date, the sources of error that may affect their reliability have not been studied in depth. Using generalisability theory as its starting-point, this article analyses the influence of the assessors and the criteria of the rubrics on the…

Drawing appropriate diagrams is a useful problem solving heuristic that can transform a problem into a representation that is easier to exploit for solving it. One major focus while helping introductory physics students learn effective problem solving is to help them understand that drawing diagrams can facilitate problem solution. We conducted an…

The purpose of this study was to describe the development of content and performance standards for a rubric to evaluate secondary-level solo instrumental music performance using a modified bookmark standard setting procedure. The research questions that guided this study include (1) What are the psychometric qualities of a rubric to evaluate…

Herein, we describe an intensive, week-long course design institute (CDI) designed to introduce participants to the scholarly and evidence-driven process of learning-focused course design. Impact of this intervention is demonstrated using a multifaceted approach: (a) post-CDI satisfaction and perception surveys, (b) pre-/post-CDI surveys probing…

Statistical literacy is essential to an informed citizenry; and two emerging trends highlight a growing need for training that achieves this literacy. The first trend is towards "big" data: while automated analyses can exploit massive amounts of data, the interpretation--and possibly more importantly, the replication--of results are…

In this paper, we propose a conceptual framework to guide the design of a computer-supported collaborative learning intervention to help students learn how to improve collaborative knowledge building discourse at the level of the small group. The framework focuses on scripting individual and collective regulatory processes following collaboration.…

Effective teamwork skills are essential for success in an increasingly team-based workplace. However, research suggests that there is often confusion concerning how teamwork is measured and assessed, making it difficult to develop these skills in undergraduate curricula. The goal of the present study was to develop a sustainable tool for assessing…