The second edition of Connecting the Dots brings together representatives from CoARA, the Barcelona Declaration on Open Research Information and GraspOS to discuss ways in which open research information is needed for research assessment reform. 

Bianca Kramer - Barcelona Declaration Organising Team
Thanasis Vergoulis - GraspOS Lead Technical Manager
Janne Pölönen - CoARA Steering Board member

Following the GraspOS Community of Practice meeting held on 13 November 2024, Bianca Kramer, Thanasis Vergoulis and Janne Pölönen have accepted to answer our questions on ways in which open research information can support and strengthen reform in research assessment practices. 

Why IS the transition from closed to open research information important, and who will benefit from it?

 Janne: There are various benefits of transitioning to open research information. Closed commercial databases and tools are expensive and exclusive, so open research information is more equitable if organisations and researchers around the world have free access to research information needed for research assessment and for determining research impacts. The most frequently used commercial databases, Web of Science (WoS) and Scopus, are also very exclusive about literature from which publications and citations are indexed. Therefore, open research information may also promote inclusiveness, if it covers a broader range of contributions irrespective of communication language, format or context, than WoS and Scopus, which prioritise peer-reviewed articles in international English language journals.

 Thanasis: All the previous points are accurate and to the point. Furthermore, it is clear that applications, analyses, and processes relying on proprietary data sources locked behind paywalls suffer from a lack of transparency. This restricts the ability of the research community to independently validate the findings or provide alternative interpretations of the data.  

 Bianca: Indeed, for researchers, these biases and lack of transparency affect them when they are being evaluated or assessed, and also when they want to find scientific information using bibliographic databases and search engines. Open research information will thus benefit researchers and research performing and funding organisations directly.   

What does the signing of the Barcelona Declaration mean? In concrete terms, what will it imply for signatories?

 Janne: Signing the Barcelona Declaration signals recognition of the importance of open research information, and a commitment to making strategic choices that in the long run contribute to quality, development and use of open research information in research assessment.

 Thanasis: Yes, I agree. By signing the Barcelona Declaration research performing organisations make it clear that they recognise the challenges with the current status quo and acknowledge that change is needed. In addition, each organisation that signed the Declaration commits to actively supporting open research information providers in various ways and contributing to the broader dialogue on how to effectively realise the shared vision of the Declaration.  

 Bianca: It is also important to emphasise that the commitments in the Declaration are intended to be aspirational - we do not expect signatories to already meet all commitments before signing the Declaration, or to meet them in a specific time period. There is, however, the expectation for signatories to actively work towards the commitments within their organisation. One thing we want to facilitate is for organisations (both signatories and supporting providers of data, services and infrastructure) to work together to make this transition easier and quicker. 

In what ways can open infrastructures support an Open Science-aware research assessment system?

 Bianca: Open research information can make research assessment more transparent and fair. It allows for all information used to be openly accessible, and it also allows research assessment to be performed in a more equitable way - not being dependent on selective, closed databases, but with more agency over the data being used (from a number of different sources). Open infrastructures can support this by making sure that the research information they collect and provide access to is both open and FAIR - for instance, by using persistent identifiers, allowing full access (and reuse rights) to underlying data, and making the data interoperable. 

 Janne: One of the main goals of the responsible research assessment movement is to recognise and reward Open Science practices. This includes transparency – following the principle “As open as possible, as closed as necessary” – of the research processes, and by extension, assessment processes. Another aspect, already highlighted in the first question, is that open research information may potentially support recognition of the broader range and diversity of Open Science practices and contributions. 

 Thanasis: Yes, exactly. Implementing research assessment practices based on open infrastructures and guided by Open Science principles, is increasingly recognised as a prerequisite for achieving the goal of making research assessment practices more responsible and inclusive. At the same time, it is important to emphasise the reciprocal relationship: an Open-Science-aware research assessment framework can, in turn, benefit Open Infrastructures and the broader Open Science ecosystem by encouraging and rewarding the adoption of Open Science practices.  

What are the next steps in your organisation regarding the joint roadmap^*?

^* A joint roadmap for open research information was recently developed at the Paris Conference on Open Research Information (23-24 September 2024)

 Thanasis: From the GraspOS perspective, the vision of the project is well-aligned with the Declaration’s vision for a transition from closed to open research information. Furthermore, various project partners have already signed or supported the Barcelona Declaration, with their representatives willing to actively participate in the discussions within the newly forming working groups. These partners are also working to develop internal action plans that align with the key commitments outlined in the Declaration. 

 Janne: CoARA has welcomed but has not yet formally endorsed or signed the Barcelona Declaration. However, it is one of the principles of the Agreement on Reforming Research Assessment to “Ensure independence and transparency of the data, infrastructure and criteria necessary for research assessment and for determining research impacts”. To facilitate implementation of this principle, CoARA has a thematic Working Group “Towards Open Infrastructure for Responsible Research Assessment” (OIRRA). One further step for the CoARA community could be to support the Barcelona Declaration, which indicates a clear pathway to open research information.

 Bianca: For the Barcelona Declaration itself, the most important next step is that we are currently setting up working groups for signatories and supporters to work together on a number of topics on the joint roadmap. The roadmap contains a set of actions that were formulated by signatories and supporters at the Paris Conference on Open Research Information in September 2024. These are mostly actions where organisations together can do more than each individual organisation (like sharing information and expertise, formulating minimal standards, and acting together rather than individually). We will provide working groups with organisational support, and also want to work closely together with other related initiatives, to reinforce each other and prevent duplicate efforts. 

In your opinion, what are some of the challenges that infrastructures for open research information could face in the near future?

 Janne: There are several challenges. Information on research outputs, activities and impacts produced by researchers, institutions and infrastructures is scattered across various platforms and difficult to use and reuse systematically in assessments. Therefore, we need to improve interoperability. Another important issue is how can we achieve sufficient quality, structure and robustness of open research information needed for research assessment purposes. In addition, one important barrier for relying on open research information is that access and analytics require relatively high technical competence compared to the ready-made commercial solutions. It is important to dedicate effort to develop analysis tools and dashboards to support use of open research information.

 Thanasis: I believe that Janne provided an excellent overview of this topic. If I were to add anything, it would be that open infrastructures should find proper ways to implement curation mechanisms to achieve improved coverage and quality for certain types of metadata, like affiliations and topics, potentially leveraging the networks of volunteer experts, such as those involved in initiatives like the Barcelona Declaration. 

 Bianca: I would add that with all these requirements and expectations, infrastructures for open research information need to be actively used as well as financially supported to be able to continue to exist and further develop. This also puts a responsibility on research performing and funding organisations. By engaging with infrastructures, they can actively contribute to the continued improvement of data quality and functionality, and by reinvesting money currently spent on closed, proprietary systems, they can contribute to their sustainability. 

Thank you very much for your time Bianca, Janne and Thanasis!  

To learn more about the Barcelona Declaration and openness of research information for research assessment reform, we invite you to take a look at the slides from the Community of Practice meeting which have been made available here.