Photo: Jitendra Raj Bajracharya/ICIMOD.

The COVID-19 pandemic and its impacts

The world is reeling under the impact of the COVID-19 pandemic, which has affected life in multifarious ways. Countries have resorted to various measures – like imposing physical distancing and travel restrictions – to check the spread of this virus. In our context here in the HKH, a region where poverty is rife, the efforts to combat the pandemic have been particularly challenging. As an institution, we have taken various aspects of situation into account and published a policy paper about the short-, intermediate, and long-term priorities for the region around the issues on which our work focuses.

The economy of the region is mostly dependent on the following sources: agriculture; trade in high-value mountain products; tourism; and remittances from migrant workers. All these sources have been heavily impacted by the virus and the subsequent responses from governments across the globe. The market chain has been disrupted; the farmers are neither able to sell their products nor get seeds and fertilizers on time; the migrant workers are returning home after losing their jobs abroad; and the service and informal sectors are almost in a state of paralysis. This has led to acute issues of hunger, unemployment, suicides, and gender-based violence.

But even so, there has been a silver lining. Various experts and agencies point out that as a result of contingency measures such as lockdowns, there has been a reduction in air and noise pollution, as well as in the emission of greenhouse gases. To address all of the issues now in ways that point us in directions of a green and resilient recovery, measures must effectively reach out to the most vulnerable by following a compassionate trajectory that recognizes the diverse situations and peoples of the region.

What can evaluators learn from this pandemic?

In the face of this pandemic, the role of the evaluation community has become more important than ever. Moving forward, it is pertinent for them to reflect on the past and ask these two major questions of itself:

1. What was the state of preparedness of the evaluation experts to continue their role in such a crisis?
2. How have evaluation experts been able to contribute to develop a more sustainable, equitable, and resilient world?

Reflecting on the first question, it has to be noted that the evaluation society has been exploring the parameters of moving forward, not in a business-as-usual way, but by adapting to a new set of global and national contexts, priorities, programme management modalities, and ICT constraints and opportunities. With social distancing being a necessity, the use of the virtual platform and online tools is now more common than ever. This has also fuelled discussions on the significance of big data as well as of GIS and remote-sensing data in evaluation practice. And in the area of data analysis, the potential of tools such as artificial intelligence, machine learning, R, Python, Microsoft Power BI, Tableau, and Qlik are being studied.

The second question reflects on the value and performance of the evaluators in the past so that it helps in future learning. With an increased focus on development, evaluation science can help decision makers and managers understand complex issues and trade-offs so as to navigate change towards impact. Indeed, monitoring and evaluation (M&E) has become one of the bedrocks of shaping a more sustainable, equitable, and resilient world.

That said, despite its strengths, many people think that the evaluation practices of the past were too project-focused and did not sufficiently address issues such as complexity, trade-offs, equity, and integration of human rights and gender equality. Reacting to this, The Organisation for Economic Co-operation and Development (OECD) Development Assistance Committee (DAC) have recently revised the evaluation criteria which was first laid out in the 1991 OECD DAC Principles for Evaluation of Development Assistance, and is widely used in development programmes and policies. The revised criteria consider gaps in current practices in terms of evaluation concept and current policy priorities. Effective implementation of the criteria is still a challenge for which capacity needs to be strengthened.

In many cases, M&E practices are still limited to administrative purposes; for example, in the case of a project, the information that is collected is used by the programme manager to simply report on the activities and expenditure so as to justify the release of funds. A broader use of such information is still not in play to cut a path towards better policies and practices. So, in order to enable experimentation and adaptation, M&E systems need to support managers and decision makers so that they understand what works and what doesn’t, and how and when interventions are to be made.

Post-COVID monitoring and evaluation

The COVID-19 pandemic is an opportunity for the evaluation community to evolve and work towards achieving the Sustainable Development Goals (SDGs). Given the crisis at hand, we argue that M&E practices in the post-COVID world should be reformed and improved along the following lines:

Increase the M&E focus on tacit knowledge and learning: Studies show that people rely more on tacit knowledge for learning than on written information from self-evaluation systems. There are cases where clearly defined objectives, targets, and indicators are not specified at the design stage and remain unspecified throughout the programme/project cycle; so, post-COVID M&E has to be used for developing such aspects like the clarity of a programme/project and enhancing the understanding about issues of complexity, trade-offs, and learning, besides executing reporting and accountability functions.

Set up operational ties for shared responsibility and responsive behaviour: M&E becomes relevant only when management decisions are supported by enough evidence. Therefore, M&E has to build a strong connection with the decision-making system at different levels for which responsibilities have to be shared, and challenges collectively addressed. A robust M&E mechanism led by a team of experts is pivotal in terms of coordination, collaboration, and overall support activities.

Enhance the use of advanced technology: The evaluation community has realized the importance of real-time monitoring – and the need for enhanced capacity in this regard – to bolster effective response mechanisms, especially in complex situations such as the one we find ourselves in at the moment. In the face of the COVID-19 pandemic, the use of virtual platforms and online tools has risen exponentially. So, despite constraints and opportunities, M&E professionals need to strengthen their technological capacities and innovative skills.

Promote a safe culture by recognizing negative effects despite programme or policy goals: Environmental scientists warn of a bigger crisis than the COVID-19 pandemic if humankind continues to destroy nature for short-term benefits.  In this context, the evaluation of unintended and negative effects of any development work has become more important than ever. Most of us know about the havoc wreaked on agriculture by the belated detection of the harmful effects of chemical fertilizers. There has also been a report on the adverse impact of commercial cropping on the nutrient intake of households in underdeveloped and developing countries. These examples show glaring gaps in evaluation practices. Meanwhile, in many cases, greater effort has gone into proving achievement than in learning and improving. This will continue if we don’t promote a culture of identifying, sharing, and learning from the negative and unintended consequences of development efforts.

Use evaluation methodologies beyond borders: It has become more important than ever for evaluation practices to adapt to and consider the substantial influences that globalization and regionalization have at a local level. Like in the case of this pandemic, problems or actions in one part of the world can affect other parts as well, and as badly. It is also apposite here to point out that pollution, especially of air and water, recognizes no borders. So, here’s where collaboration, coordination, and sharing of learning beyond borders become critically important.

Dispel the trend of misleading data: Analysing the data on COVID-19, the public affairs consultant Keith Naughton concludes that all predictions about the future course of the pandemic are highly suspect because of misleading data. This case of misleading data is also a huge issue in development evaluation. In order to understand the quality of any data, we need to have an in-depth understanding of the data source, the methods used to collect and validate it, and how it’s analysed and interpreted. However, these are aspects that have not been accorded enough importance in current evaluation and research practices. But, as they say, a time of crisis is also a time of opportunity – COVID-19 is a strong motivator to strengthen evaluation and research practices, thereby helping communities make the right decisions at the right time.

Lalu Maya Kadel

Farid Ahmad

Nisheeth Basnet

Share