Following the extension of the REF timeline to 2029, I’ve taken a look at what we know so far, what to look out for in 2024, and what institutions can do now to lay the foundations for a successful submission in 2028.
I’ve mostly focused on research impact but also touch on research culture as part of the new People, Culture and Environment element of REF2029.
Introduction – what do we know?
The Rocky Road to REF
With the news at the end of 2023 that the REF timeline is being extended to REF2029, there was a collective sigh of relief across the university sector. The extended timeline will give the funding bodies time to test out some of the new elements planned for the next REF, and to work through some of the complexities that have been highlighted through the consultation on the Initial Decisions, published in Summer 2023. With Rebecca Fairbairn appointed as the new REF Director and additional REF team members being recruited, we can expect the pace to pick up on activities and planning. The draft guidance was due in Summer 2024, although with the extension this is likely to be delayed. Once released we will get to see the ‘rules of the game’ fleshed out in more detail.
What to look out for over the next 12 months
Is Culture the new Impact?
The decision to expand the environment element of REF to include ‘People, Culture and Environment’ (PCE) feels a bit like the decision to include Impact back in REF2014. There is general agreement that the idea is laudable but there are questions about how this will be assessed and what unintended consequences might arise for the sector. As with impact in REF2014, there will likely be pilot exercises to test how research culture can be assessed. The funding bodies have commissioned a piece of work to thoroughly look at the PCE element and how it can be measured and appropriately assessed – expect to see a lot of engagement with the university sector to gauge views, opinions and test out options. We’re seeing suggestions that this element of REF will involve some form of metrics or indicators, however as with impact, is research culture really something that can be boiled down to a few numbers? I very much doubt it having heard about the experiences of universities trying to use metrics to measure the success of their research culture strategies at the Warwick Research Culture Conference in September. One thing that everyone was clear about is that research culture should not be seen as a ‘competition’ with the community keen to share their experiences and learn from each other. There was a general view that any evaluation of research culture should focus more on how universities are tackling the root causes of obstructive practices, rather than whether their research culture is better or worse than their peers.
The REF is known to have considerable influence over cultures, behaviours and practices across the university sector. The inclusion of impact in REF2014 led to a positive shift in the understanding and awareness of research impact, alongside increased recognition and reward of impactful activity. REF2029 has the opportunity to lead to similar positive outcomes in relation to research culture, leading to more inclusive and collaborative research environments, so there is a lot resting on the funding bodies to make sure they get this part right.
Engagement and Impact
REF2029 will reintroduce an additional statement on wider impact and engagement activities for each UOA, in a similar approach to REF2014. This is a positive change that will recognise and reward a wider range of impact and engagement activities than the impact case studies alone, and will help to generate a more inclusive impact culture. Exactly how the additional statement will be formatted and weighted for assessment remains to be seen, but the Initial Decisions document suggests a combination of quantitative and qualitative data. This means tracking of impact and engagement activities will need to be stepped up, beyond just the impact case study pipeline. Perhaps the long awaited HEBCIS review will help with tracking these wider activities? Expect a consultation on the new proposals in Spring 2024.
The big news from the December REF update for impact was the confirmation that the minimum number of impact case studies required for a UOA submission would be reduced from two to one. This is welcome news for those very small departments, many within the Arts and Humanities, who previously had to produce a disproportionate number of case studies per researcher than their larger counterparts. Also welcome was confirmation of the removal of the 2* research threshold. This will allow universities to be more confident of submitting some of the more practice led impact case studies, which had previously been perceived as ‘risky’ choices due the perception that unconventional research outputs (often seen in practice-based research) were viewed as lower quality.
Interdisciplinary collaborations
The December REF update confirmed that the UOA structure will remain unchanged for REF2029. While this allows continuity and the ability to benchmark against previous REF results, the recognition of interdisciplinary research still feels secondary to the individual disciplines. We are seeing more collaboration across disciplines as it becomes evident that having an impact on the world’s most complex challenges, such as climate change and food sustainability, requires engagement across disciplines and also across stakeholders.
Despite this widespread acknowledgement, the barriers to working across disciplines persist – limited funding opportunities, peer review issues, university (and REF) structures, publishing disparities. This was a theme running through the AESIS Societal Impact of Social Sciences, Humanities and Arts I attended last autumn, with many sessions describing the value of the social sciences within wider collaborations, alongside discussions on mechanisms for influencing policy. UKRI are currently piloting a cross-council responsive mode funding call, so perhaps this could be a starting point to breaking down these barriers and making changes to the research ecosystem to recognise and reward cross disciplinary research. Maybe we’ll see more from REF2029 in this space too, as the guidelines start to develop.
What do you need to do/what might help?
Systems – internal and external – can help
Many universities are now using research information management systems to support their end-to-end research management. Many of these systems include a function to ‘track’ or ‘monitor’ impact, and while often fairly basic, they allow the monitoring of impact related activity and provide wider reporting functionality to understand what is in your impact pipeline. Knowing what is in your impact pipeline is crucial at this stage in the REF cycle so that you know how much additional resource will be needed to nurture your pipeline into a set of highly impactful case studies.
External systems can also help with tracking engagement and impact. Altmetric and Plum X Metrics are probably the most well-known applications for tracking engagement with research papers e.g. citations, social media mentions, and in some cases references in patents and policy documents. When it comes to tracking policy influence, Overton appear to have cornered the market with their own software (overton.io) and partnerships such as BMJ Impact Analytics and SagePolicyProfiles. Having used Overton in a previous role, I found it really useful for tracking the policy influence of a researcher, rather than that of a specific paper which Altmetric and Plum X provide.
Impact support
When it comes to impact, never underestimate how time consuming it can be to track and monitor activities and source evidence. Unless you are providing time in workload models for this, many academics just won’t be able to keep up (especially if they are particularly impactful). This is where your impact officers/coordinators/managers come into their own. These are crucial members of staff (I’ve been one and was repeatedly told how crucial I was!) who can support your academics to plan, develop, track, monitor and evidence impact and, when the time comes, help to write those REF impact case studies. They will also be the ones that will know all the details of the engagement activities that will now form part of the impact element of REF, making them even more indispensable.
I can see from the ARMA mailing lists that there are many impact roles currently being recruited, and it’s good to see that lessons have been learnt – impact officers are needed throughout the REF cycle, not just as we get closer to submission. What is also clear is that Research Culture Manager roles are being created across the sector and these will come into their own once the rocky road to REF produces some guidance on what those PCE indicators will look like.
Key take home message…
The key take home message is – don’t leave your REF planning too late. REF2029 will be upon us faster than you think. For impact and the new research culture elements, if you’ve not starting planning now, you’re already on the back foot.
This article first appeared on LinkedIn on 19 January 2024.
