Rowing hard in the right direction – how to assess the impact of philanthropic foundations

“Rowing harder doesn’t help if the boat is headed in the wrong direction” Kenichi Ohmae

In the field of social change, it is easy to engage in good work. The scale of injustice, discrimination and inequality means that, wherever we look, there is work to be done. But knowing how to differentiate between “good work” and “work that makes a lasting difference” is complex.

That is why we were delighted to be asked by Carnegie UK to look at how best to assess impact. Having just developed a new strategy, a refreshed approach to impact and learning was a good next step.

The first lesson learnt from the interviews and literature review we undertook is that, to understand impact, it is critical to have a full understanding of the environment in which you operate. In other words, “context is queen”. Other Foundations we spoke to and those working to influence the policy environment, pointed to the importance of one big trend: the widening gap between government, the public and civil society.

In a world where there is division, distrust and defensiveness, Foundations have a freedom to create space and time for civil society organisations and policy makers to come together, learn together, share intelligence and improve practice. Another consequence of falling trust in government is that Foundations need to ask themselves whether it is always right to target government policy as the end point for change and to recognise the moments when there is a need to engage the public or influence public opinion, before pushing for policy change.

The second lesson was that any thinking about impact needs to move away from “proving” impact to “improving” practice. This shift in mindset can free organisations from a heavy reliance on measurement using what can be quite narrow, quantitative measures to adopting an assessment of qualitative and quantitative evidence to build a more rounded picture of impact.

Strongly linked to this, both the literature and interviews pointed to the importance of any work on impact being part of a broader learning culture. Important here is keeping your approach to impact assessment proportionate and adaptable – recognising that different programmes may require different assessments. Across the piece, using the lightest touch measures possible will be important to avoid the risk of the assessment process itself becoming an “industry” or a “beast that has to be fed”.

Ultimately, good impact assessment needs to change decisions made and inform future ways of working.  Keeping an eye on this end goal of improving impact is critical.

Looking ahead, when a new impact assessment framework is in place, Carnegie UK will want to think about areas of further stretch.

We identified three. First, Foundations need to think beyond programmes or grant streams to consider what the Association of Charitable Foundations calls their “total impact”. Whether that is use of endowment, operations, grantee experience, campaigning or other direct activity, understanding impact across the totality of a Foundation’s work is critical. Second, there is considerable potential to use co-design and participatory methods within impact frameworks to make sure that, for example, affected communities can themselves specify the change that matters to them. Finally, there is recognition that Carnegie UK  is only one part of making change happen. Understanding how the organisation contributes to change alongside other people and organisations will be an important part of developing its thinking about impact in the future.

And for all foundations, we think there are some questions that will help kickstart new thinking on impact:

1. How much learning and impact work has been focussed on reassuring ourselves we made good decisions vs. changing our practice?
2. How are you reflecting on your own power in the way you approach your impact and learning work? Are you sharing power with partners ? Who gets to determine what is valuable?
3. How can you think and act as part of a network of other social change organisations to make collaborative efforts on the issues most important to you?
4. What indicators are you using to understand the context you work in? What opportunities are your stakeholders, staff and Trustees being given to make sense of this context with you?

We hope Carnegie UK’s own learning has stimulated you to look again at how you measure impact and in the next few weeks we will be publishing a longer write-up of our work together on these issues. Please just drop a line to [email protected] or [email protected] if you want to be part of this ongoing conversation.