Higher education accreditation has spread internationally as a vehicle for quality assurance and improvement but is strongly influenced by accreditation practices in the United States. The organisational structure and processes of seven United States health professions accreditors were analysed to identify common characteristics that reflect…

RobotReviewer is a tool for automatically assessing the risk of bias in randomized controlled trials, but there is limited evidence of its reliability. We evaluated the agreement between RobotReviewer and humans regarding the risk of bias assessment based on 1955 randomized controlled trials. The risk of bias in these trials was assessed via two…

Maintaining the quality and comparability of qualifications has become a major priority for vocational education and training (VET) systems world-wide, especially for those countries which have decentralised assessment and reporting systems. In the absence of external examinations, one policy solution to align the assessment standards of different…

Recent times have seen the growth in the number and scope of interacting professional reform movements in science, centered on themes such as open research, research integrity, responsible research assessment, and responsible metrics. The responsible metrics movement identifies the growing influence of quantitative performance indicators as a…

Comparative judgment is gaining popularity as an assessment tool, including for high-stakes testing purposes, despite relatively little research on the use of the technique. Advocates claim two main rationales for its use: that comparative judgment is valid because humans are better at comparative than absolute judgment, and because it distils the…

The George Mason University (Mason) offices responsible for institutional reporting and assessment were merged to strengthen and solidify their functions and expertise. The organizational structure creating the Office of Institutional Effectiveness and Planning (OIEP) was intended to engineer a new paradigm where assessment would inform the…

Innovation agencies, that is organizations with the primary focus of funding applied research and technological development, evaluate project proposals to select the most promising proposals for funding. At the moment, there is only little verified knowledge available on project evaluation and selection processes of innovation agencies. We want to…

Evaluation has been expanding as an important discipline in Latin America, yet there are significant challenges regarding capacity development through educational and training initiatives in the region. This paper first analyzes the evolution and state of the field in terms of the teaching of, and training in evaluation in Latin America, with a…

Recent literature places more emphasis on assessment comments rather than relying solely on scores. Both are variable, however, emanating from assessment judgements. One established source of variability is "contrast effects": scores are shifted away from the depicted level of competence in a preceding encounter. The shift could arise…

George Grob presented the fifth and final Eleanor Chelimsky Forum address at the 2017 annual meeting of the Eastern Evaluation Research Society. In this commentary, I respond to several points that George raises in the "American Journal of Evaluation" paper based on that address. An overarching theme of my comments involves the potential…

Drawing on Glenn Fulcher's extensive work in performance-based language assessment of speaking, this paper explores the assessment of L2 speaking ability in local language testing contexts. For that purpose, I review Fulcher's influential work that highlights the relationship between the speaking construct, the task, the performance, and the…

As global discussions of evaluation standards become more contextually nuanced, culturally responsive conceptions of ethics have not been sufficiently discussed. In academic social research, ethical clearance processes have been designed to protect vulnerable people from harm related to participation in a research project. This article expands the…

The Culture of Assessment Matrix is a one-page rubric intended to provide institutions of higher education with a simplistic way to quickly gauge the status of their institution's assessment culture and the degree to which the culture is an integral part of the institution's operations, expectations, methods of accountability, and reward systems.…

In an era of ever-deepening budget cuts and a concomitant demand for substantiated programs, many organizations have elected to conduct internal program evaluations. Internal evaluations offer advantages (e.g., enhanced evaluator program knowledge and ease of data collection) but may confront important challenges, including credibility threats,…

In many of Germany's states the procedures for school inspections were amended following completion of a first evaluation cycle. This chapter examines the way changes carried out in the course of reform were configured with respect to content and process. It is based on the assumption that school inspections represent an institution whose…