In this news piece, GraspOS project member Fotis Mystakopoulos (OPERAS) reflects on research assessment and open infrastructures, sharing takeaways from the PUBMET 2024 Conference and the Paris Conference on Open Research Information.
In September 2024, I attended two conferences that both had flavours of research assessment within their content. The first conference was the 2024 PUBMET, in Zadar, Croatia. The second conference was the Paris Conference on Open Research Information, an event associated with the Barcelona Declaration.
The PUBMET conference explored core issues in scholarly communications through the research assessment lens. I was invited to join as a panellist for Day 1, in Session 2 – Integrity and Ethics in Research Assessment, which was chaired by Matt Hodgkinson, and I was joined by Iva Grabarić Andonovski, Lynne Bowker, and Svjetlana Kalanj Bognar.
Image: Panel on Integrity and Ethics in Research Assessment
(from Left to Right: Svjetlana Kalanj Bognar, Fotis Mystakopoulos, Matt Hodgkinson, Lynne Bowker, Iva Grabarić Andonovski)
In the opening keynote speech, Bianca Kramer talked about openness in research assessment and how we need to “broaden and widen” our lens. The idea of broadening and widening what we include in research assessment resonates with the findings of the UNESCO Outlook Report (2023). It is essential that we monitor a wider range of outputs, and not just scholarly publications to ensure that we capture and use advice from a diverse range of activities.
GraspOS, through its technical partners and associated pilots, is demonstrating its value by offering an alternative path to research monitoring. Tools like OpenAIRE Graph, OPERAS Metrics, and the Openness Profile, along with GraspOS's integrated monitoring services, play a crucial role in this.
During the panel session in PUBMET, key topics emerged, highlighting a bi-directional link between GraspOS, research assessment, and research integrity. The conversation touched on the ongoing 'publish or perish' culture in academia, which resonated with the panel. A way out of this could be by alleviating the pressure on researchers to publish so many papers by offering them alternatives and more diverse outlets for communicating the value of their research.
The notion of widening the publishing available output is linked directly to research integrity which involves practices like pre-registering research methods, openly sharing code, workflows, and software to ensure reproducibility and verifiability of research outcomes. Yet, these activities are undervalued in the current research assessment process, and without proper incentives, researchers are reluctant to engage with these practices. Luckily, there is already a statement in this direction, the Hong Kong Principles for assessing researchers but more needs to be done at the policy level for these principles to become part of research assessment processes.
In any case, GraspOS is filling a gap by providing a space and tools that allow researchers to track, trace, and monitor their workflows for assessment. The infrastructure fosters behaviour and culture change by offering visibility and accountability for published workflows, thus enhancing research integrity. This vision of infrastructure as an enabler of research assessment is why several GraspOS partners participated in the Paris Conference on Open Research Information in September 2024.
The Barcelona Declaration, in short, is advocating for the creation and use of infrastructures that share their information openly without restrictions, for a number of uses (you can see a full list of the commitments here). Individuals from organisations from all around the globe gathered in Paris to discuss and brainstorm how the initiative can move forward by setting concrete actions for the next 12 to 18 months. A big portion of this event was dedicated to metadata, and how it can be standardised, and how different organisations managing metadata can work together to use and reuse the information for a range of causes including research evaluation.
GraspOS representatives were in attendance. Ludo Waltman, from CWTS, is one of coordinators of this activity, and heavily involved in shaping the vision of Barcelona Declaration. Our scientific coordinator, Thanasis Vergoulis (Athena Research Center), was in attendance alongside Angeliki Tzouganatou (OpenAIRE) to present the CoARA working group on Open Infrastructures; and Silvio Peroni presented the latest developments from the University of Bologna and how they meet the commitments of Barcelona Declaration highlighting the value of OpenCitations in this process (Full conference Presentations available here). There was also a mention of OPERAS during one of the presentations as a relevant infrastructure to work on metadata for open access books to be taken into account as they are often underrepresented in those discussions.
GraspOS is fully aligned with the commitments of the Barcelona Declaration, and is actively involved in the development of open infrastructures through its array of data, tools, services and guidelines via the Open Science Assessment Framework.
