By Chris Thompson, Technical Director and Paige Mason, Technical Director at Social Impact
Virtual data collection has become common in evaluations with the shift to remote work caused by the COVID-19 pandemic, yet the management of those remote evaluators conducting research is an often-overlooked and less well-resourced aspect of such research.
Remote evaluation technical management requires advanced approaches to team oversight, support, communication, and collaboration. Applying these approaches has both challenges and benefits, which our colleagues at Social Impact, Inc. (SI) have explored in internal knowledge exchanges and an external Remote Evaluation Technical Management Blog Series.
Although the worst of COVID-19 may have passed, the pandemic has led to a global shift in expectations around where evaluation is performed. In addition to remote data collection, the “behind the scenes” activities of evaluation planning, team management, and technical oversight also occur virtually – in some cases team members never meet in person. The likelihood that this work modality remains a part of at least some monitoring, evaluation, research, and learning (MERL) activities encouraged us to capture and formalize the evidence about what works when managing evaluations remotely by publishing this Remote Evaluation Technical Management Guide.
This guide provides information for managers on how to remotely administer the technical aspects of evaluations and presents options and recommendations for innovative, responsible, and safe approaches to assess programs without field-based travel. Drawing from experience conducting field-based and (now) remote evaluation for the US Agency for International Development (USAID), the US Department of State, the Millennium Challenge Corporation, Mastercard Foundation, and the Bill and Melinda Gates Foundation, this guide compiles our challenges, lessons learned, and promising practices for the wider MERL community.
The guide is designed to be immediately useful to evaluation practitioners and funders because it:
- Is organized according to well-established stages of the evaluation process. It provides guidance on the technical aspects of managing a remote evaluation’s planning, startup, design, fieldwork, analysis, reporting, and use. By encompassing each stage of the evaluation process, it hopes to provide solutions to challenges that could occur at any point.
- Specifies who should be involved at each stage. The guide identifies the key actors involved from the commissioning organization, evaluation team, program being evaluated, stakeholders, and counterparts throughout the evaluation process. This clarifies who should contribute to implementing solutions to obstacles in remote evaluation management as they arise.
- Presents learning in terms of challenges and solutions. The guide enumerates the common challenges experienced by remote evaluators based on SI’s experience and then offers the best available solutions that its evaluation teams have “road tested” to help make the lessons learned immediately useful to those involved in a remote evaluation.
- Takes an asset-based approach to team dynamics. The solutions address not only the technicalities of working remotely, but also the relational aspects of evaluation team management. The guide encourages evaluation team leaders and managers to consider the strengths that the evaluation team can bring and offers suggestions for how to leverage these strengths and individual advantages in a remote environment.
- Concludes with a summative yet concise checklist of those challenges and solutions. The guide’s annex is a two-page tool summarizing all the lesson learned by evaluation cycle stage designed for evaluators to apply with their teams to determine what innovations they can adopt to immediately make a difference in their remote management.
The recent and potentially enduring emphasis on remote work in evaluation has influenced the technical methods used by evaluators and the management, communication, and collaboration approaches applied by those who support them. We hope this guide has offered our fellow evaluation managers, team leaders, and funders a few solutions to their challenges of administering the technical aspects of evaluations remotely and safely when field-based travel is not possible, for whatever reason, whether in our present pandemic or the future.
Cover Photo Credit: Chris Montgomery