In this second blogpost of the series describing the activities of the SUPER MoRRI project, we expand on the broad and global networks that the project has developed. Beyond the Country Correspondents, that were introduced in the previous blogpost, the project planned for engaging with partners working in RRI as well, the so-called Science with and for Society (SWAFS) – Ecosystem.
RRI permeated European research and innovation policy from 2013-2022, where it was reflected in (mainly) European policy documents, funding schemes, and a list of projects which was expanding until the end of H2020 (2021). The many actors who were working in this SwafS-funded RRI environment generated a wealth of information and tacit knowledge (meaning the many contexts that cannot be made visible) related to co-creation, responsibility, (citizen) engagement, and all other RRI keys as conceptualized by the European Commission such as gender equality, ethics, open science, and science education. They all benefitted from the SwafS-ecosystem for sharing and communication and acknowledging the challenges and opportunities that these interactions brought about. The Ecosystem hosted virtual meetings every month to discuss topics related to responsible research and innovation (RRI), science and society ad monitoring and evaluation. This meeting space ran since October 2019 and generated a lively space for fruitful discussion among participants and representatives from the European Commission. In particular, the subset of ten territorial RRI-projects was productive in their conversations on Monitoring and Evaluation of RRI.
SwafS Ecosystem Conversations
The territorial RRI projects held four specific sessions on identifying how each of them approaches the monitoring and evaluation of the RRI-indicators that the projects were supposed to collect according to the H2020 call-text. Nhien Nguyen, Felicitas Schmittinger and Tjitske Holtrop address in their blogpost that questionnaires often seem simple and straightforward means to monitor and assess ongoing changes and engage with people. It is eye opening to learn how complex this evaluation practice turns out to be in practice when trying to assess pilot experimentations conducted within a project! An important lesson is to consider the nuances of co-creation, stakeholder participation or transnational learning. One size doesn’t fit all, and diversity cannot be grasped solely by focusing on long and complex surveys, or short and sweet open questions, checking boxes, using alternate responses, or scaling the feedback.
In the next blogpost, Anestis Amanatidis, Norbert Steinhaus, and René Wintjes elaborate further on differences (in viewpoints, capabilities, or objectives) between stakeholders. Without difference, there would be no benefit of including new participants, and limited opportunity to learn from each other, or benefit from each other’s ways. In this respect, differences should be celebrated and actively searched for. On the other hand, differences can also make it more difficult to learn from each other, e.g., in case of cognitive and language differences. In case of quadruple helix stakeholder encounters, cultural differences across the actors’ social worlds are crucial to acknowledge and discuss. Translational work is therefore essential in engagements with diverse stakeholders.
SUPER MoRRI’s global network
Aligning to the EC’s wish to take a global perspective on responsibility too, SUPER MoRRI has implemented the Global Satellite Partner Network. Carolina Llorente describes in her blogpost this international dimension, with the aim of promoting responsibility in research and innovation globally, enabling mutual learning about RRI and RRI-like activities beyond Europe.
Not only were the Global partners involved as sounding board to the project’s insights, SUPER MoRRI’s second annual event in 2021 actually revolved around the globe! The COVID pandemic created the need for the event to take place in a virtual setting, which led to an experience that would not have been possible otherwise: Instead of a full-day seminar as originally planned, four webinars were held across different continents, exploring responsible evaluation from a global perspective. The first three webinars focused on the Americas, Asia/Pacific and Africa/Middle East. Each one of these consisting of a plenary session with three country-based panel statements about responsible research evaluation practices from a local perspective. Those presentations contributed to inspire subsequent discussions in small breakout sessions with invited participants from research funding organisations, research performing organisations and policymakers. The fourth reflection webinar brought together the ideas, and the debate was enriched with the contribution of representatives from DORA, ENRESSH, OECD and UNESCO. A total of 30 different countries were represented in the four SUPER MoRRI seminars on responsible evaluation practices. André Brasil comments in his first blogpost on different realities in social, economic, political and geographical contexts which is reflected in the specificities of their science systems. At the same time, recurrent converging perspectives and themes appear when responsible evaluation is concerned globally: Move away from metrics but beware of subjectivity; change takes time and is expensive; distinct realities, different purposes confirming credible contextualization. Three further posts dive deeper into these topics.
In the first post by André Brasil three issues were discussed regarding evaluation of research: how indicators are usually unable to capture the complexity of regionally relevant research in Latin America and the Global South; the benefit of peer review to consider solutions for regional problems that are not necessarily the focus of the type of research traditional indicators; and to address the pressure from evaluation practices (SCI-worship) that has led to an increase in research integrity problems and the quantity over quality focus. While many are braving parallel paths towards better perspectives on metrics and the value of qualitative approaches to assess research outputs, different science systems exist in distinct levels of maturity and development.
The second blog post by Carolina Llorente argues that implementing responsible practices of evaluation in research requires a considerable investment in terms of financial and human resources. Funding is a very important part of this movement especially when it comes to training high-level evaluators and research management for science systems over the world. The concept of evidence-based policy applied to the scientific environment only makes sense if people are qualified to produce and consume the evidence, and there are substantial costs of both time and money to make that real. There is a common denominator in the need to invest more efforts (both human and financial) to ensure a responsible evaluation of teaching. New criteria are needed in addition to a greater effort to homogenize the conceptualization of “responsibility” among the different countries and regions.
In the third post, Ingeborg Meijer reflects on whether transition in science systems and evaluation practices should be evolutionary or revolutionary. A dedicated line of support for connecting science to the values and interests of European citizens has run like a red thread through two decades of European framework programmes (FPs). Whilst creating impact may be a policy goal, in the science system it is often seen as an additional requirement, to be delivered on top of the academic disciplinary traditions, aims and outputs. Concomitantly, the evaluation of the science system has been revolving, and still does, around traditional ways of valuing science, unintentionally creating a science-society divide. In a sense, RRI is providing the (policy)conditions for creating societal impact through changing the science system at the researcher, institutional and national level through engagement with stakeholders. Most countries participating in the SUPER MoRRI annual event agreed that, when it comes to responsible practices in research, it is not particularly difficult to change policy; the problem is changing the culture. While new legislation and regulations can impose change, they must be aligned with people’s motivations towards responsibility, and the transition needs to be incremental.
Of course, all topics would benefit from a much deeper reflection than the ones that can be developed in this blog. But what is clear is that this series of events has allowed the SUPER MoRRI consortium to initiate a more global reflection on the subject and open our minds to obtain much more international, effective, and useful outputs. Without a doubt, the message we take home after these sessions is that countries have similar concerns despite being in very different situations.
Now that the last months of the SUPER MoRRI project are there, all the reflections, data collections, analyses, conversations, and writings are getting together. This, as well as our interactive portal, and dashboard will be presented during the final event (30th of November and 1st of December 2023) where we will hand over the legacy of the project to successors and share it with our community and others!
- September 25, 2023
