Like many international development organizations, Social Impact (SI) has produced a vast body of knowledge over decades, including evaluation reports, proposals, performance reports, learning briefs, and technical studies spanning sectors, countries, and donors. While this depth of experience is one of SI’s greatest strengths, it also presents a familiar challenge: how to make institutional knowledge easily discoverable, usable, and reusable across teams, particularly in a time of rapid change.

To address this, SI developed an internal, AI-enabled knowledge management tool that allows staff to query a curated corpus of past work using natural language. Rather than relying on folder structures, filenames, or individual institutional memory, the system uses modern retrieval-augmented generation (RAG) techniques to surface relevant excerpts from SI documents and synthesize clear, evidence-based responses. Users can ask questions such as “How has SI dealt with attrition in longitudinal studies with at-risk youth?” or “What lessons have we learned in evaluating youth employment programs?” and receive answers grounded explicitly in cited source material.

Behind the scenes, the system combines document chunking, semantic embeddings, and large language models to retrieve and synthesize relevant excerpts from SI reports and performance documentation. Importantly, the tool is designed with transparency in mind: every response is accompanied by citations and source excerpts so users can easily verify, contextualize, and build upon the information provided. Further, metadata supports intelligent retrieval and prioritization without requiring users to manually filter or know exactly where information is stored. In this way, the tool functions less as a “black box AI” and more as an intelligent research assistant that accelerates access to institutional knowledge.

What makes this approach particularly valuable as a knowledge management solution is its flexibility. The system does not simply return documents, it surfaces patterns, examples, and lessons across projects, geographies, and clients, while allowing our staff to emphasize recency, relevance, or specific thematic priorities. This helps us move more quickly from “What do we know?” to “How can we apply it?”

While the tool can support proposal development, its broader value lies in strengthening institutional memory, supporting learning, and making SI’s collective experience more accessible across teams. By investing in AI as an enabler of knowledge management, rather than a replacement for human judgment, SI is demonstrating how technology can help organizations better capture, share, and use what they already know.

Lessons Learned

  • Knowledge quality matters as much as model quality. The system allows users to compare results across LLMs (including different models from ChatGPT, Claude, and Gemini), yet we have found the quality of results is less dependent on the LLM and more on the tuning of the system to ensure the best context data is included in the query.
  • Transparency builds trust. Providing source citations, diagnostic information, and the ability to view underlying excerpts has been essential for user confidence.
  • Human workflows still matter. AI does not replace expert judgment; it augments it by reducing search costs and surfacing relevant evidence more quickly.
  • Flexibility is key. Allowing users to adjust retrieval parameters and compare model outputs helped accommodate different analytical and learning needs.