By: Rebecca Herrington, Social Impact, Embedded Evaluator at the USAID’s Global Development Lab
Nine months ago I started work on a developmental evaluation (DE)—one of the first to be conducted at USAID under the Developmental Evaluation Pilot Activity (DEPA-MERL). In just this short period of time, I’ve learned more about DE, embedding, making long term integration and data management easier, and of course the programs I’m examining. Because DE is relatively unknown even within the evaluation field and there are few practical resources available, I’d like to share some of my insights thus far.
The most important piece of advice I can give for the first few weeks is to calm down and enjoy the ride. It is important that to slow down and sit with the ambiguity. Know that everything is going to take longer than you think it will. The first part of the DE is really about relationship building and making sure you integrate yourself enough with the DE stakeholders so that they routinely share information and honest thoughts with you.
As you get organized and dig into the learnings of the DE, here are an additional few words of wisdom from my experience:
1. Be Systematic. Build the foundations and habits of the evaluation from Day One by a) deciding on a data management system/software, b) committing to write a protocol for any data collection effort, and c) start capturing everything as data from emails to hallway conversations. Capture, upload, code, repeat! In DEs that are qualitatively dominant, it can be easy to let the data amass around you. Use technology where possible to make things easier, like setting up a basic codebook and coding process in Dedoose and using a Smart Notebook to make uploading notes more efficient. It will be invaluable when it comes time to share actionable, evidence-based feedback with the DE stakeholders.
2. Be Physically Present. The value of being face to face, building personal relationships, and being in the room when decisions are made cannot be overemphasized. Being with the people you’re working with and serving in the DE makes a huge difference. The opportunity to get to know you in person, even seeing you furiously taking notes during a meeting, helps to build trust with the DE stakeholders. This translates directly to how often and consistently they include you, share data, and provide honest feedback, both through formal and informal conversations. This means better data and better recommendations. It also builds legitimacy of the DE deliverables and willingness of the stakeholders to take time to adapt.
3. Create and Demonstrate ‘Value Add’ Regularly. This advice is a little more nuanced than the previous points. It may take a long time to answer a particular DE learning objective that the DE stakeholders have set forth, so make sure they know what a DE can do for them by sharing smaller deliverables along the way (that are within scope, of course). Some examples include stakeholder mapping, data-driven timeline analysis, SWOT analysis, discrete feedback, and suggested adaptations on team norms, just to name a few. Remember, an activity you see as a part of a larger data collection effort may be of distinct value to the DE stakeholders! Make sure you keep in touch with what their needs are and find ways to meet them when objectives align.
4. Make Use of your Resources and Support Network. Bounce ideas off other evaluators, your project team, and whoever else might be able to help you out. They will enrich your own approaches with their experiences, new ideas, and different perspectives.
There is certainly a lot more to learn, but the above will get you started on the right foot!
Want more in depth information for doing a DE? Check out the Developmental Evaluation in Practice: Tips, Tools, and Templates—a guide I developed with my fellow DEPA-MERL team members at SI, Search for Common Ground, and the William Davidson Institute at the University of Michigan.
