Over the last decade, the idea of societal impact resulting from publicly funded research has changed from being a relatively fringe concern related to high-technology entrepreneurship and spin-off companies to becoming an increasingly important public policy concern. This has stimulated academic science policy research to investigate the impact…

Despite the increasing recognition for the scientific and societal potential of interdisciplinary research, selection committees struggle with the evaluation of interdisciplinary proposals. Interdisciplinary proposals include a wider range of theories and methods, involve a more diverse team, pose a higher level of uncertainty, and their…

Peer review has developed over time to become the established procedure for assessing and assuring the scientific quality of research. Nevertheless, the procedure has also been variously criticized as conservative, biased, and unfair, among other things. Do scientists regard all these flaws as equally problematic? Do they have the same opinions on…

As part of the shift towards a more equitable research culture, funders are reconsidering traditional approaches to peer review. In doing so, they seek to minimize bias towards certain research ideas and researcher profiles, to ensure greater inclusion of disadvantaged groups, to improve review quality, to reduce burden, and to enable more…

This article addresses the debate on pre-evaluative choices of impact depictions and the forms of responses between applicants and funders. By adopting a reflexive perspective on the social impact of social sciences, this article explores researchers' vocabularies in the research proposals and mid-term reports of consortiums during the Strategic…

We have limited understanding of why reviewers tend to strongly disagree when scoring the same research proposal. Thus far, research that explored disagreement has focused on the characteristics of the proposal or the applicants, while ignoring the characteristics of the reviewers themselves. This article aims to address this gap by exploring…

The European Research Council (ERC) receives many high-quality applications, but funds only a few. We analyze how members of ERC review panels assess applications in the first, highly competitive step of evaluations for ERC Starting and Consolidator Grants. Drawing on interviews with ERC panel members in different fields, we show that they adopt a…

Impact statements are increasingly required and assessed in grant applications. In this study, we used content analysis to examine the 'comments on impact' section of the postal reviews and related documents of Science Foundation Ireland's Investigators' Programme to understand reviewers' "ex ante" impact assessment. We found three key…

In this study, we analyze the two-phase bottom-up procedure applied by the Future and Emerging Technologies Program (FET-Open) at the Research Executive Agency (REA) of the European Commission (EC), for the evaluation of highly interdisciplinary, multi-beneficiary research proposals which request funding. In the first phase, remote experts assess…

Although many studies have shown that reviewers particularly value the feasibility of a proposed project, very little attention has gone to how applicants try to establish the plausibility of their proposal's realization. With a sample of 335 proposals, we examined the ways applicants reason the feasibility of their projects and the kinds of…

Evaluation for the allocation of project-funding schemes devoted to sustain academic research often undergoes changes of the rules for the ex-ante selection, which are supposed to improve the capability of peer review to select the best proposals. How modifications of the rules realize a more accountable evaluation result? Do the changes suggest…

To blind or not researcher's identity has often been a topic of debate in the context of peer-review process for scientific publication and research grant application. This article reports on how knowing the name and experience of researchers/institutions influences the qualification of a proposal. We present our experience of managing the…