All students at Norwegian universities and colleges have the right to complain about ordinary grading decisions. When an appeal is made, two new examiners are appointed, at least one of whom should be external. The handling of appeals in the current system is to be blind, meaning that the examiners handling the complaint should not be aware of the…

This article presents the insights from an evaluation of a transformative policy experiment, in the food domain, conducted at the Swedish Agency of Innovation (Vinnova). To be consistent with the principles and objectives of these policies, it was necessary to implement a formative evaluation approach developed in the Transformative Innovation…

Higher education accreditation has spread internationally as a vehicle for quality assurance and improvement but is strongly influenced by accreditation practices in the United States. The organisational structure and processes of seven United States health professions accreditors were analysed to identify common characteristics that reflect…

Editors of academic journals play a key part in the production of knowledge and the continuation of scholarly conversations about practical and intellectual issues of the day. While peer review, an essential part of journal processes, has received considerable attention in the scholarly and the popular press, much of it critical, the role of the…

Social justice evaluation approaches advance antioppressive and culturally responsive methods to mitigate societal inequalities. However, social justice efforts will fall short if they focus solely on one side of inequality: oppression. Accordingly, this article argues that the other side of inequality--privilege--needs to be scrutinized. This…

RobotReviewer is a tool for automatically assessing the risk of bias in randomized controlled trials, but there is limited evidence of its reliability. We evaluated the agreement between RobotReviewer and humans regarding the risk of bias assessment based on 1955 randomized controlled trials. The risk of bias in these trials was assessed via two…

Assessment practitioners in higher education follow a variety of paths to their roles. Diverse preparation supports creative problem-solving in a changing educational landscape. However, it can also lead to inconsistency in language, preparation, and background knowledge. Further, the chasms between assessment practitioners' paths can lead to…

This article explores the utility of mindfulness in the field of evaluation. Mindfulness is a translation of the ancient Indian word, "Sati," which means awareness, attention, and remembering. While definitions vary, a practical definition of mindfulness is present-moment awareness in an open and nonjudgmental manner. Mindfulness-based…

Maintaining the quality and comparability of qualifications has become a major priority for vocational education and training (VET) systems world-wide, especially for those countries which have decentralised assessment and reporting systems. In the absence of external examinations, one policy solution to align the assessment standards of different…

Recent times have seen the growth in the number and scope of interacting professional reform movements in science, centered on themes such as open research, research integrity, responsible research assessment, and responsible metrics. The responsible metrics movement identifies the growing influence of quantitative performance indicators as a…

Comparative judgment is gaining popularity as an assessment tool, including for high-stakes testing purposes, despite relatively little research on the use of the technique. Advocates claim two main rationales for its use: that comparative judgment is valid because humans are better at comparative than absolute judgment, and because it distils the…

Automated Essay Scoring (AES) tools aim to improve the efficiency and consistency of essay scoring by using machine learning algorithms. In the existing research work on this topic, most researchers agree that human-automated score agreement remains the benchmark for assessing the accuracy of machine-generated scores. To measure the performance of…

The George Mason University (Mason) offices responsible for institutional reporting and assessment were merged to strengthen and solidify their functions and expertise. The organizational structure creating the Office of Institutional Effectiveness and Planning (OIEP) was intended to engineer a new paradigm where assessment would inform the…

As part of my work as an educator, I see the need to surface for discussion what might indeed be considered as acts of oppression on the part of peer reviewers when certain aspects of knowing and meaning are misrecognized, obscured, or suppressed. Drawing on observations concerning coercive and oppressive relational and educational practices found…

Innovation agencies, that is organizations with the primary focus of funding applied research and technological development, evaluate project proposals to select the most promising proposals for funding. At the moment, there is only little verified knowledge available on project evaluation and selection processes of innovation agencies. We want to…