Cross-funder action to improve the assessment of researchers for grant funding

Submitted by cfoley on Wed, 01/19/2022 - 18:11

Funding organisations signal what is valued within research ecosystems by defining the criteria upon which proposals and applicants are assessed. In doing so, they help to shape the culture of research through the projects and people they support. Focusing on a limited set of quantitative criteria favours a narrow view of research and disincentivises creativity and innovation. To counter this, many funding organisations are shifting to a more holistic interpretation of research outputs and achievements that can be recognised in the grant evaluation process through the use of narrative CV formats that change what is visible and valued within the research ecosystem. As communities of practice in research and innovation funding, the San Francisco Declaration on Research Assessment (DORA) funder’s group and Funding Organisations for Gender Equality Community of Practice (FORGEN CoP) partnered to organise a workshop focused on optimising the use of narrative CVs, which is an emerging method used by research funders to widen the research outputs that can be recognised in research assessment.

DORA’s funder discussion group was launched in March 2020 to enable communication about research assessment reform. By learning from each other, funding organisations can accelerate the development of new policies and practices that lead to positive changes in research culture.

The focus on implementing narrative CV formats for grant funding grew organically within the discussion group. In 2019, the Royal Society in the United Kingdom introduced the Résumé for Researchers as an alternative to the traditional CV to recognise the diversity of research contributions through a concise, structured narrative. At the same time, a handful of research funders began to experiment with the use of narrative CVs in grant evaluation. During the first meeting of the DORA’s funders group, the Dutch Research Council and the Swiss National Science Foundation shared information about their pilot projects to implement narrative CVs. At subsequent meetings in 2020 and 2021, the Health Research Board Ireland, Luxembourg National Research Fund, and Science Foundation Ireland also shared updates on their implementation of narrative CVs.

FORGEN CoP, led by Science Foundation Ireland, was established in October 2019 and aims to share knowledge and best practice on gender equality in research and innovation funding. Mitigating gender bias within the grant evaluation process was identified as a primary area of focus.

Inequalities in the grant evaluation processes are more prevalent when assessments are focused on the researcher, as opposed to the research. With the conversation shifting to the assessment of people, the discussion naturally led to the use of narrative CVs and their assessment. Narrative CVs can be used as a method to move away from assessing productivity (quantity over time) to research achievements, focusing on the quality and relevance of these achievements in relation to the research proposed. They may also support researchers with non-linear career paths or career breaks by removing the focus on productivity which can be affected by periods of leave for care-giving responsibilities. During subsequent meetings, members shared their experiences implementing narrative CVs and their assessment methods. In a special FORGEN CoP seminar focused on how funders could mitigate the long-term gendered impacts of the COVID-19 pandemic, narrative CVs were highlighted as a method to address potential gender inequalities and/or a reduction in productivity that some researchers experienced throughout the pandemic.

During September 2021, DORA and the FORGEN CoP joined forces to address current knowledge gaps for the use of narrative CVs in grant evaluation and to improve their usefulness as a tool for responsible research assessment. We jointly organised a workshop, facilitated by Dr Claartje Vinkenburg, which was convened as two events to enable global attendance. More than 120 participants from over 40 funding organisations and 22 countries joined us to explore strategies to mitigate potential biases of narrative CVs and monitor their effectiveness in the grant funding process. During these events, we heard from organisations implementing narrative CVs and experts who spoke about the influence of gender bias and language on research assessment. Researchers spoke to us about decision-making and process optimisation for grant funding. In breakout sessions, we were able to identify useful areas of alignment for funding organisations and what studies are needed to gather evidence to improve the implementation of narrative CVs.

We are delighted to share a short report that captures our learnings from the workshop and identifies a course of action to improve narrative CVs as a tool for assessment. The workshop and report provide a foundation for the optimisation of narrative CV formats upon which we plan to build.

Using Narrative CVs: Process optimisation and bias mitigation

We compiled a list of resources leading up to and during the workshop to help us understand the opportunities and challenges of using narrative CVs for grant funding, such as how bias might influence their evaluation (see below for the list of resources). As narrative CVs can be used for purposes other than grant funding, including the hiring and promotion of research and academic staff, we foresee that this report and the collated resources will also be useful to the wider academic community.

Narrative CVs have been the focus of other international funders’ fora in tandem with the work of DORA and FORGEN CoP. The Swiss National Science Foundation and the Open Researcher and Contributor ID (ORCID) organisation established the CV Harmonisation Group (H-group) in 2019 to improve academic CVs as a tool for responsible research assessment. In 2021, UK Research and Innovation (UKRI) launched a Joint Funders Group, an international community of practice of research funders, to develop a shared approach for the adoption of the Resume for Researchers, which UKRI also plans to implement.

Research assessment reform requires collective action, which is why DORA and FORGEN CoP have now partnered with the Swiss National Science Foundation and UKRI to build on this work. We are excited to announce that a second workshop for research funders is planned for February 2022 that aims to identify shared objectives for the use of narrative CVs in grant funding and determine how funding organisations can work collaboratively to monitor their effectiveness. If you work at a public or private research funding organisation and would like to participate, please email diversity@sfi.ie.

Narrative CVs reduce emphasis on journal-based indicators, allow for the recognition of a variety of research contributions, and help to shift the focus from metricised research productivity to research achievements. Through the activities already completed and those planned, we aim to develop a path to collective action on the optimisation and application of narrative CVs as a tool for responsible research assessment.