In conducting an evaluation of the A Ganar youth development project in Central America, Social Impact worked closely with our local partner CIEN Centro de Investigaciones Económicas Nacionales. We recently interviewed Pablo Pastor of CIEN to get his views on the project and evaluation in Guatemala.
What are your thoughts on the A Ganar evaluation and its impact on youth development work in Guatemala?
We could really see the difference between the youth in the control group and those in the program. I could easily see positive changes in them in the youth, even 18 months after the project ended. Changes were simple, but important. I witnessed improvements in their self-confidence, style of dress, and social skills.
Many of the youth that attended the program were eagerly seeking jobs, going back to school, attending church, or participating in other social activities that were important to them. Their relationships with their families had also improved.
There was so much data collected and so many achievements made by the implementing organizations in the project that might have not been taken into account without SI’s evaluation.
This would not have been possible to see if we didn’t have the control group. On the other hand, many of the youth from the control group were part of gangs and, even attended the interviews with other ‘gang members’; you could hear, in their words, resentment and a life without much change during the past few years.
I have seen that USAID is supporting many other projects in Guatemala with the same intention of helping at risk youth. This project’s approach is quite different. There was so much data collected and so many achievements made by the implementing organizations in the project that might have not been taken into account without SI’s evaluation. I believe that the experience learned during this evaluation might lead to the implementation of better interventions for youth Guatemala.
How can this evaluation help improve other evaluations and programming in Guatemala?
I believe that the design of the evaluation was complex and well designed. It was created as few projects I have seen before and, could provide results that are quite important to formulate new programs for the youth in Guatemala. I believe that SI created a very important and interesting set of questions to understand the youth and their environment. We used innovative ways to evaluate that could be valuable in future evaluations. Many organizations in Guatemala need to revise their impact evaluation designs so that they can be useful for policymaking. Lessons learned from this project should be shared with the community so that we can learn ways to improve evaluations in Guatemala.
Evaluations are not cheap, but without them it is impossible to effectively correct or change a program. In Guatemala, it’s difficult to view an evaluation as a necessary cost to improve a program. Therefore, I believe we could improve evaluations by increasing funding for them.
What is the benefit of partnering on evaluations?
I believe that it’s a win- win situation. CIEN learned from SI’s knowledge of evaluation design. While CIEN’s field experience helped SI collect quality data in local environments that can be difficult to access. This helps us provide better service for customers and helps us improve our knowledge. It has been a rewarding partnership. We have incorporated some of SI’s survey design into our other projects. We used similar questions with the indigenous youth of Guatemala aiming to understand their behavior and way of thinking.