This article presents the insights from an evaluation of a transformative policy experiment, in the food domain, conducted at the Swedish Agency of Innovation (Vinnova). To be consistent with the principles and objectives of these policies, it was necessary to implement a formative evaluation approach developed in the Transformative Innovation…

This research aimed to develop formative assessment rubrics for enhancing students' performance on Thai percussion instruments using the Many-Facet Rasch Measurement Partial Credit model (MFRM-PCM). Samples were high-school students playing four types of Thai instruments and the raters who were qualified and properly trained. The research…

The award of a PhD degree is based on a rigorous examination process which is unique in that examiners can recommend that the degree is awarded subject to minor or major amendments to the thesis. In their examiner reports, examiners also include formative feedback that is intended to help the doctoral researchers improve their work. Using data…

The term "meta-evaluation"--referring to the "evaluation of evaluations"--has been in the evaluation lexicon for a half-century. Despite this longevity, research on meta-evaluation is sparse and even more so for internal formative types of meta-evaluation. This exploratory study builds on our understanding of meta-evaluative…

Teacher evaluation has evolved from a task used for administrative decisions to an activity whose main goal is the enhancement of student learning and well-being through the improvement of instruction. The actual implementation of teacher evaluation by school principals will determine greatly the extent to which it can achieve this goal. An…

Background: Multiple-choice item (MCI) assessments are burdensome for instructors to develop. Artificial intelligence (AI, e.g., ChatGPT) can streamline the process without sacrificing quality. The quality of AI-generated MCIs and human experts is comparable. However, whether the quality of AI-generated MCIs is equally good across various domain-…

Adult learning involves the analysis and synthesis of knowledge to become competent, which cannot be assessed only by traditional assessment tool and didactic learning methods. Stimulation of higher domains of cognitive learning needs to be inculcated to reach a better understanding of the subject rather than traditional assessment tools that…

To understand the persistent social class achievement gap, researchers have investigated how educational settings affect lower versus higher socioeconomic status (SES) students' performance. We move beyond the question of actual performance to study its assessment by evaluators. We hypothesized that even in the absence of performance differences,…

Evaluation of interdisciplinary, team science research initiatives is an evolving and challenging field. This descriptive, longitudinal, mixed methods case study examined how an embedded, formative evaluation approach contributed to team science in the interdisciplinary Research into Policy to Enhance Physical Activity (REPOPA) project, which…

In an era of widespread automation--from grocery store self-checkout machines to selfdriving cars--it is not outrageous to wonder: can teachers be automated? And more specifically, can automated computer teachers instruct students how to write? Automated computer programs have long been used in summative writing evaluation efforts, such as scoring…

Collective impact (CI) initiatives provide a promising collaborative community-based approach for addressing health disparities, but they are challenging to evaluate. This paper provides a detailed strategy for addressing these challenges through a case study, with the goal of not only of evaluating the CI, but also putting organizations in a…

Teacher effectiveness has a powerful impact on student performance and a teacher evaluation process that supports professional growth can be a key lever for improving teaching quality. The purpose of this study was to examine teacher perspectives on the use of the Australian Professional Standards for Teachers, when used as part of their…

Although much research has compared the functioning between analytic and holistic rating scales, little research has compared the functioning of binary rating scales with other types of rating scales. This quantitative study set out to preliminarily and comparatively validate binary and analytic rating scales intended for use in formative…

This article investigates the role of emotions of inspectors while inspecting schools as reported by inspectors themselves within an education context of increased accountability that arguably privileges rationality over emotions. The study is built on an emotion management framework that regards emotions not only as unavoidably natural and…

Contribution: This paper shows how significantly computer science instructors can disagree when grading code writing questions in introductory computer science (CS1) exams. Certain solutions are considered "very good" by some instructors and "very bad" by others. The study identifies four factors as possible contributors to…