Hybrid Indicator for Responsible Research Assessment and Open Science Evaluation

13 November 2025, 12:00-12:15

Novel methods are needed to improve research assessment and open science monitoring. Higher education institutions are expected to interact with the surrounding society and promote the societal impact of research; nevertheless, it is difficult to quantify this impact. Despite challenges of assessing societal impact, there is an urgent need to explore and develop novel assessment metrics. To acknowledge the level of effort in achieving societal impact, novel indicators that consider the diverse encounters of science and society must be discovered.

The SIAMPI project showed that interaction is prerequisite for impact. It identified three categories of interactions that are involved in the impact: 1) direct interactions between people, 2) indirect interactions through physical objects such as publications, and 3) financial interactions such as research contracts and grants.

Societal impact is like interaction or exchange of knowledge between scientists and society. These interactions can be utilised in formulating an intermediary or a "level of effort" for the societal impact. We developed a semi-quantitative hybrid indicator model for the level of effort by examining both direct interactions between people and indirect interactions in the form of professional and popularised publications. The level of effort makes the extent of societal interactions visible.

To create the indicator model we curated, preprocessed and examined two Current Research Information System (CRIS) datasets: University of Eastern Finland’s UEF-CRIS and VIRTA (Finnish national publication information service that integrates institutional CRIS data at the national level). From UEF-CRIS data, we processed research activities, and from VIRTA data, we included popular and professional publications. In developing the hybrid indicator, we used the main disciplines as the unifying factor. Summations and normalisations were made in order to achieve a numerical value that indicates the level of effort. We illustrated the numerical values of the indicator using a star rating system. The main disciplines were given one to four stars based on normalized numbers of interaction efforts. One star refers to a minor effort and four stars refer to significant effort.

The indicators in our model are prone to biases. We do not encourage assessments, evaluations or comparisons between units, such as disciplines or individuals, without sufficient knowledge on the target’s data collection practices and datasets as well as applicability of the hybrid indicator model with its limitations. The hybrid indicator model is, however, functional and can be applied and further developed. It visualises the importance of noticing the different efforts researchers make to achieve societal impact.

The hybrid indicator model advances to support recognising merits in academic activities, assessing research, and enhancing open science. We will offer the hybrid indicator model openly available and encourage open conversation around the diverse topics of societal activities.

Anni Tarkiainen, University of Eastern Finland, Library 
ORCID: 0009-0003-2238-405X
Heikki Laitinen, University of Eastern Finland, Library
ORCID: 0009-0009-9008-9515
Katri Rintamäki, University of Eastern Finland, Library
ORCID: 0000-0001-9729-4647

Heikki Laitinen is an information specialist with over 41 years of experience working in higher education. Throughout his career, he has been involved in numerous library and information-related projects. These include designing and implementing one of the first academic library websites and e-journal services in Finland.

Heikki has also conducted bibliometric analyses for research evaluation and developed distance learning courses for teaching information skills. His current responsibilities include the GraspOS UEF pilot, providing guidance and consulting on information skills, conducting literature searches for systematic reviews, and teaching the use of reference management tools such as Zotero and Covidence.

Heikki holds a Master of Science degree in Environmental Health Science from the University of Kuopio, Finland.

Katri Rintamäki serves as the Head of Services for Training and Information Services at the University of Eastern Finland Library. In her current role, she is responsible for coordinating and developing library services that support teaching, learning, and research.

Her previous experience includes working as an Information Specialist in the university library, focusing on teaching, information services, and research support, as well as serving as a full-time Lecturer in the Library and Information Services degree programme.

She holds a Master of Administrative Sciences with a specialization in Public Law, a Master of Social Sciences with a specialization in Information Science, and a Doctor of Social Sciences with a specialization in Economic Sociology.