We are pleased to announce a new study commissioned by UK Research and Innovation (UKRI) to develop a monitoring and evaluation (M&E) framework for their Open Access policy. This framework will seek to enable UKRI and the broader sector to assess progress towards Open Access, levels of compliance with the UKRI policy and its overall effectiveness. The M&E framework will also aim to establish how Open Access publication trends affect academic practices and society, where possible.
Our approach
Working alongside our associates Bianca Kramer and Cameron Neylon, we will be identifying a series of potential M&E questions, approaches to answer them, their feasibility and expected resource intensity. We anticipate that the M&E framework will employ a mix of quantitative and qualitative measures, to account for the diversity of practices and stakeholders across the research landscape. We will also be considering information sources and datasets that can yield useful insights while minimising reporting burdens.
UKRI wishes to ensure that the M&E framework is co-designed with the research and publishing community. To achieve this, Research Consulting and UKRI will be conducting interviews and workshops with a diversity of experts and stakeholders.
What happens next?
The project kicked off in July 2022 and aims to complete in the autumn. UKRI will consider the outcomes of this work with a view to developing a final monitoring and evaluation framework for the Open Access policy.
Regular updates will be available on this page over the course of the project, and feel free get in touch with Andrea Chiarelli (Senior Consultant, Research Consulting) for more information on this work. To find out about UKRI’s monitoring and compliance efforts, please see the UKRI Open Access policy supplementary FAQs.
Project updates
16 September 2022
Over the course of August and the first part of September, we have been inviting funders, sector bodies, higher education institutions and publishers to join interviews and focus group to provide their perspectives on the creation of the a monitoring and evaluation framework for the UKRI Open Access policy.
We have primarily engaged UK-based contributors, but a range of international views are also being captured, including from the Netherlands, Germany and Austria. To date, we have engaged over 60 stakeholders, and we are planning to reach a total of over 80 individuals by mid-October via further interviews, focus groups and a validation workshop.
Meanwhile, we have been reviewing a wide range of existing and planned assessment mechanisms for Open Access policies, to build a longlist of potential M&E questions for UKRI’s consideration. This is quickly taking shape in the form of a spreadsheet, which we will then use to inform the prioritisation of recommended M&E questions, in combination with stakeholder views.
We will post an update in early October to outline next steps towards the conclusion of this project.
6 October 2022
We have now conducted the majority of interviews and focus groups, and a discussion workshop took place on 5 October with over 30 attendees from a range of stakeholder groups. This helped us validate our findings to date and gather valuable feedback from the community.
During the workshop, we discussed a shortlist of prioritised M&E questions, which we will now sharpen and take forward in the form of a data specification for UKRI’s review. Over the course of October and early November, we will be working on a final project report, a recommended methodology to answer M&E questions and a summary of our findings.
30 November 2022
The project has now reached its end, and we are in the process of finalising all outputs suitable for public dissemination. We expect a final report and methodology annex to be released in the summer of 2023. You can find an overview of findings to date on our blog.
UKRI will include an update on the M&E approach in their upcoming OA communication and, once outputs are published, build on our findings to operationalise the M&E approach.
24 May 2023
A final project blog post has been published in our News section. We note that this is the last project announcement to take place via Research Consulting’s website.
