Financial and human resources to advance responsible practices of research
As described in a previous post, four meetings were held on “Responsibility in research evaluation practices”, three with a regional orientation (Americas, Asia/Pacific, Africa/Middle East), ending with a global reflection webinar. One of the concerns that appeared repeatedly throughout the events was that the investment of both financial and human resources is required to advance responsible practices of research.
Implementing responsible practices of evaluation in research requires a considerable investment in terms of financial and human resources. Funding is a very important part of this movement especially when it comes to training high-level evaluators and research managements for science systems over the world. The concept of evidence-based policy applied to the scientific environment only makes sense if people are qualified to produce and consume the evidence, and there are substantial costs of both time and money to make that real.
One of the concepts that emerged in the Americas event discussion panel is that in the Brazilian context it is important to have an overview of society and how funding is distributed to understand where resources are lacking. The issue reappeared in group discussions where some participants commented on the difficulty of maintaining a consistent and systematic evaluation system when there is a discontinuity in funding processes due to changes in government or decision-makers.
In general, it seems that in the American context, the Research Evaluation System is closely linked to its funding. Therefore, it can be considered that the funding agencies have the key to improving the evaluation of research, if they begin to include indicators of responsibility it will be reflected in the way of conducting research.
On the other hand, something that appeared also in both the Asia/Pacific event and the Africa/Middle East event discussions was the need to include different dimensions of research in order to be able to evaluate it responsibly and not fall into a merely bibliometric evaluation. To this end, many of the comments from the group sessions in the different regions made reference to the need to establish basic guidelines or training to establish more responsible evaluation systems. Both things require, on the one hand, economic investment and, on the other, an investment in human resources to train specialists who can implement this evaluation.
Moreover, the need to include other stakeholders as well as representatives of the academy in the evaluation process of both researchers, as well as entities producing science or research projects, was also commented on. Which also requires a considerable investment of resources and training.
In the Asia/Pacific event, Yang Yun presented a top-down, government-commissioned, reform of the science and technology evaluation system to move it from a system heavily based on publications to a system that is results-oriented and less burdensome. The dissatisfaction around the evaluation systems and the reliance on publications was common in all regions and also was the feeling that currently there is a focus on excellence. Moving from measuring excellence to responsibility means a conceptual change of the whole evaluation system starting with having a common understanding of what responsibility is.
The concern about financial and human resources to promote responsible research practices was also the subject of fruitful discussions in the Africa/Middle East event. One of the things that was repeated a lot in the group discussions was how rare the term “responsible research and innovation” is in the African context and how this implies that resources must be allocated to promote the concept and thus improve in this type of practice and, therefore, its evaluation.
Another interesting thing to take into account in this regard is the diversity of science-producing entities on the African continent. As Nelius Boshoff commented in his speech, the reality of South Africa is not the same as that of other African countries. Something similar was commented in the group sessions of this event, where some of the participants shared that there are many African universities where the university is mainly dedicated to teaching and not to research. In this case, perhaps the evaluation indicators should be reviewed since the same ones cannot be applied as in a context where the main activity is research. This means that research and evaluation are contextual and we have to keep this in mind when offering a monitoring system for RRI.
We have seen that each country and each region has different perceptions on the subject. However, there is a common denominator in the need to invest more efforts (both human and financial) to ensure a responsible evaluation of teaching. New criteria are needed in addition to a greater effort to homogenize the conceptualization of “responsibility” among the different countries and regions.
Of course, all this topic gives for a much deeper reflection than the one that can be developed in this blog. But what is clear is that this series of events has allowed the SUPER_MoRRi consortium to initiate a more global reflection on the subject and open our minds to obtain much more international, effective and useful outputs. Without a doubt, the message we take home after these sessions is that countries have similar concerns despite being in very different situations.
- December 9, 2021