Liz Allen on what DORA’s Practical Guide series has learned from funders, research organisations and publishers.
The case for responsible research assessment is increasingly well established. The harder question is how to put it into practice – without creating unnecessary burden or introducing new incentives that make things worse.
That’s the challenge at the heart of the Declaration on Research Assessment’s (DORA) Practical Guide series, which aims to help organisations move from principles to practical change. The first guide, for research performing organisations, was published in 2025, followed by a guide for research funders in 2026. A third, focused on scholarly communication organisations, is now in development.
Liz Allen has spent the past two years working with DORA to develop its Practical Guide series. Liz, who is also a Research Consulting Associate Director, has been working with DORA as a Project Coordinator and brings experience spanning Wellcome, F1000, Taylor & Francis, ORCID and CRediT.
That cross-sector perspective matters. All actors in the research system – funder, publisher or institution – have a role to play in how research gets assessed; each shapes, and is influenced by, what the others do.
We sat down with Liz – speaking here in her DORA capacity – to talk about what the guides reveal about putting responsible research assessment into practice – and what comes next.
What’s the biggest shift you’ve seen in how research gets assessed during your time working in this area?
“Clearly one of the most fundamental shifts in research assessment policy in most parts of the world over the last decade is that research shouldn’t be assessed on published outputs alone.” Liz says.
“Recognising a more holistic view of the value of research and its impact is definitely a direction of travel for many universities, research performing organisations and funders. In addition, there’s increasing recognition of the importance of good scientific governance to ensure trust in research, and this should also be a component of research and researcher assessment. ”
“The challenge is how to do that systematically and introduce assessment approaches that keep the burden light for everyone involved, and not introduce perverse incentives that make things worse. ”
And changes in one part of the research system inevitably affect others.
“The incentives in one area drive behaviours in other areas”, Liz says. “For example, many scholarly publishers haven’t traditionally seen themselves as having a role in research and researcher assessment, seeing themselves largely as service organisations supporting research communities to disseminate and share their findings. But, as part of reform to research assessment, and through initiatives like DORA, publishers have become much more aware of the impact that journal prestige and other incentives have on researchers.”
What problem were the Practical Guides trying to solve?
“The focus of the guides has very much been about supporting real-world application – providing resources, toolkits, and guidance on how different organisations can develop responsible research assessment approaches within their own context.” Liz says.
“They’re full of examples: new case studies in each guide show how an organisation has actually done something in practice. That’s where the demand has been. People are asking for examples – how can I introduce responsible research assessment aligned approaches in my organisation, where do I start?”
“The guides aren’t prescriptive. It’s very much: ‘Here are things that matter in this space; use whichever bits are useful to you, wherever you are on your journey.’”
That emphasis has shaped how the guides themselves have been developed, with DORA’s communities involved from the early stages.
“DORA have been really proactive about making sure these guides are genuinely co-created with the communities they’re for,” Liz explains. We created five new case studies for the RPO guide and five for the funder guide, with more planned for the scholarly comms one – and we’re running a satellite workshop at the OASPA conference in September to help shape that one too.”
How useful are case studies when research assessment is changing so quickly?
Real-world examples make the guides practical, but inevitably capture a particular moment.
“Case studies are time-bound by nature – they have a shelf life.” Liz says. “Some describe a process that was introduced and is still working years later; others are a snapshot of something an organisation is currently trying, without the follow-up on how it went.”
That challenge is particularly acute in fast-moving areas such as AI.
“One of the issues that’s massively of interest, both to funders and specifically to scholarly comms, is how to use AI responsibly, and how AI is impacting on responsible research assessment. That’s something that’s evolving so fast, it’s just hard to capture with longevity in a guide like this. So we’re not trying to be the expert, definitive voice on it – what we want to do is point people to good resources that they can use now. And as far as possible we provide guidance that is fully evidence-based.”
What’s genuinely different about implementing responsible assessment at a funding organisation versus a research institution?
“There are many close synergies in how different actors in the research ecosystem work. For example, the guidance around how to provide responsible peer review practices in a funder and in a publisher can be very similar – it’s just that the context is different.”
“In an RPO, that might mean making sure a promotion and tenure committee is properly briefed on what the organisation is looking for, that there are no unnecessary barriers stopping certain people from applying, and that there is appropriate diversity and expertise on that committee.”
“In a funder, the equivalent is a funding panel – so defining funding scheme or programme eligibility criteria carefully to avoid creating inappropriate barriers for applying, and again, making sure there is appropriate diversity and expertise. In publishing, the equivalent would be how editorial boards are appointed and briefed.”
“So some of the themes are similar, but the context is different – which is why we have different guides. The language and the approach need to fit the audience that are reading them, even where the underlying principles are the same.”
The principles may travel across the research system, but implementation has to reflect who is making decisions, what is being assessed and the incentives operating in that context.
If an organisation only has time to act on one thing after reading a guide, what would you point them to first?
“It really depends where they are – each guide includes a way for organisations to think about what they are already doing and where specific actions could make the most difference.” Liz says.
“I’d say: don’t treat the guides as prescriptive. There’s no judgement involved. The guides don’t suggest you must do everything to be considered responsible – there are different research contexts, disciplines and norms around the world.”
“Look at the content, see which areas resonate with where you’re trying to go, and start there. And the case studies are genuinely useful to see what others are actually doing.”
What has bringing the guides together taught you about where responsible research assessment goes next?
For Liz, one of the clearest lessons has been how difficult it is to separate one part of the research system from another.
“The next guide is for scholarly communication organisations, including publishers. But I think what we’ve learned as we’ve developed the guides is just how connected everything is.”
“Even though we’ve had to package each guide as something standalone, the underlying issues don’t sit in isolation, especially as AI and other technologies change how we do research.”
“There are a lot of initiatives seeking to reform and enable good research practices, and this makes it complicated for researchers, funders and publishers to keep track and know how best to respond. DORA is very conscious of the importance of collaboration, and how it can add value without duplicating what others are doing – which is exactly why guides like these try to bring together what’s already out there, not just focus on resources that DORA itself has created.”
That interconnectedness may be one of the most important lessons from the series. Responsible research assessment is often discussed within individual parts of the research system, but incentives don’t respect organisational boundaries.
The challenge now is increasingly not whether responsible research assessment is desirable, but how funders, institutions and publishers can turn those principles into workable practice that works for the research ecosystem as a whole.
