Review of peer review
Publication date: 4 July 2023 | Report language: EN
This report presents the findings of a study commissioned by UKRI to review interventions to the peer review processes used in R&I award funding. It is intended as a resource for R&I funders across the globe looking to optimise and innovate in their award-making processes. The study assessed 38 interventions, which range from small process ‘tweaks’ such as increasing/decreasing the number of reviewers per application and shortening application sections, to more fundamental changes such as partial randomisation and complete bypass of peer review.
We find that all interventions we considered here are typically intended to fulfil at least one (or sometimes several) of the following seven aims:
- To save time, i.e. to speed up time-to-grant
- To optimise the relevance of applications and funded awards to the aims of the funding scheme
- To increase the ability to identify and fund high-risk / high-reward projects (sometimes known as ‘frontier’, ‘transformative’ or ‘breakthrough’ research)
- To reduce burden (on applicants, reviewers, panellists and/or administrators)
- To manage application volume (often a subset of reducing burden, but may also occur for other reasons)
- To reduce bias and ensure greater inclusion of disadvantaged groups, including along lines of gender, career stage, institution, or any other category
- To improve the overall quality of reviews (for instance, by ensuring optimally tailored expertise of reviewers or increased levels of transparency and feedback)
Our study highlights that there is a critical need to coordinate use of the interventions with the context and aims of each specific funding opportunity in question. Based on our findings, creating bespoke funding processes tailored to the needs and aims of each funding opportunity is a clear ‘direction of travel’ for the future of R&I funding. Our headline recommendation is that process design should always be a constituent part of scheme design. Every funding scheme has specific aims and characteristics, and so the design of the application, review and decision-making process should be considered for each individual funding opportunity.
