A new tool in the Outcome Harvesting toolbox: How Pact is collaborating to improve an essential evaluation method

At Pact, we aim to improve the impact and process of global development programming, especially in ways that bolster community engagement. This is at the core of our engaged communities approach. 

One way we have done so is by using Outcome Harvesting as a monitoring, evaluation and learning approach in our programs. Over the past few years, Pact has been a leader in implementing this in our MEL systems and in improving the operationalization of Outcome Harvesting to better meet the needs of those we serve.

Outcome Harvesting (OH) is a qualitative methodology that centers project participants in evaluation by empowering them to tell their own stories of change. OH is different from other qualitative and empowering methodologies in its use of external substantiation, which adds rigor to the stories of change. Substantiation engages third-party, independent individuals who are knowledgeable about the program and context but far enough away to validate the outcome and contribution statements made by project participants. However, in the spirt of the flexible methodology, existing OH guidance does not give prescriptive requirements for how to do so. 

Pact and others have seen the immense utility of OH, yet many practitioners have expressed a need for more guidance on how to execute the substantiation step in a manner that is faithful to the spirit of OH, and in a matter that maintains rigor within the limitations of their study. During a session on tailoring the Outcome Harvesting method at the American Evaluation Association 2023 conference, which Pact co-led with Oxfam Novib, there was a lively discussion among practitioners that indicated a desire for supplementary guidance on the substantiation step to aid in determining which approach is appropriate in a given study. 

For this reason, Pact and two external colleagues co-developed a new tool to assist with substantiation in OH. Building on Pact’s experience, the new tool will supplement existing Outcome Harvesting guidance and will be the first to dive deeper into how practitioners can carry out substantiation. The tool has three components:

1. It first guides users through a decision tree to determine whether and how they may substantiate.
2. They then move to a short set of decision point questions considering any constraints they may be facing.
3. Together this allows users to design a substantiation approach tailored to a study’s context and limitations.

The guidance document also includes worked examples of the tools from real-life case studies. As part of the tool’s development, Pact and Oxfam are presenting the tool this week at the 15th European Evaluation Biennial Conference, “Better Together 2024: Collaborative Thought and Action for Better Evaluation.” At our session we are presenting a prototype of the new tool, and participants will have the opportunity to apply it to a real Pact case study. In the spirit of collaborative thought and action, we will facilitate a Solution Focused/Solution Finding session for participants to provide inputs to the prototype, update the tool and reshare with the global OutcomeHarvesting.net community for wider use.

Our hope is that the tool will allow practitioners to better understand the various ways to substantiate harvested data. We are confident that it will advance evaluation practitioners’ ability to conduct high quality, useful OH studies and expand our ability to learn from and communicate about their results. Most important, it will ensure communities’ voices remain central to not only providing harvested outcomes but also in their validation.  

 To learn more, watch Pact’s blog for updates, or reach out to @email.