Why this project on peer review?
As researchers, we spend a significant amount of time writing papers and refereeing for peer-reviewed journals. These activities are closely interconnected: if we want to improve how we produce research, it is essential that we better understand how we evaluate research.
Despite the central role that peer review plays in the production and evaluation of research, systematic efforts to understand the current system and improve it have remained limited:
- There is very little community discussion about what should be expected from the peer review process and how each group of stakeholders (authors, referees, editors, readers) might best benefit from it.
- Institutions in peer review provide very little systematic guidance or training about how to write constructive referee reports.
- Finally, statistics about the current volume and distribution of peer review activity across the global research community are generally unavailable.
The aim of this project is to take steps towards filling the gaps in our knowledge, with the hope that this will be the beginning of a dynamic conversation about peer review that enables the community to:
- Identify and discuss potential deficiencies and frictions in our current system.
- Brainstorm and evaluate practical proposals aimed at improving it.
What did we do?
To create the foundation for productive conversations about peer review, we undertook a comprehensive effort at documenting the current state of our system:
1) Survey:
- We designed and disseminated a survey taken by over 1,400 economists in order to (i) understand their experiences with peer review and (ii) collect opinions about potential proposals to improve the system.
- We also offered respondents the opportunity to voice their concerns and suggest their own proposals for improving the system.
- The curated dataset, survey material, and codes can be accessed on our OSF page.
2) Literature review:
- We reviewed the existing literature about peer review, drawing on sources from inside and outside of economics.
- We then built a (non-comprehensive) themed bibliography, which we make available under the "Bibliography" tab of this website.
- The bibliography will be periodically updated as additional resources relating to peer review become known to us.
3) Report:
- Based on our survey data, a review of peer review literature, and research about current journal policies, we wrote a report to document what we learned about the system and to present a list of over 160 proposals that could improve it.
- The full report can be found here, while a summary of results can be found under the "Report" tab.