Julia Grant, chief Executive of Pro Bono Economics, looks at the importance of impact evaluations and why it should become integral to the entire philanthropic model.
“What is impact evaluation for and what’s the benefit for the people I want to help?” Charity leaders often ask me those two big questions. They are right to consider their priorities: resources these days are tightly stretched and impact questions continue to generate headlines.
“I think there needs to be a national conversation around the complexity [of calculating impact],” said Lee Middleton in Civil Society last year. Managing director of the National Youth Agency and a former local government commissioner, he pointed out that charities are reluctant to commit to demanding bespoke assessments. Meanwhile, Amy Cross, a Blackpool councillor, explained how local charities that lack a full impact rationale are losing contracts to organisations which are larger and better resourced, but also far less expert at handling local problems.
As public funds get further squeezed, the winners will be the charities that can show they make a difference. But smaller charities, in particular, find it hard to prove how they generate value with each pound they receive and spend. This reflects the discussions that Pro Bono Economics has with charity leaders. The vast majority of them – 86 per cent, in fact – admit to struggling with demonstrating impact. They accept its importance, but their immediate focus is on helping clients, raising money and paying salaries. When it comes to impact evaluation, they often do not know how best to start.
Fortunately, over its nine years of existence, Pro Bono Economics (PBE) has learned a few things that might prove useful. As our name suggests, we harness the potential of economics to help charities understand impact. Our project teams are built around professional economists working on a voluntary basis, and the emphasis is on efficient allocation of charities’ resources.
Economists often draw the analogy of cutting up a cake, so if you’re looking to get your fair share of the funding cake, what are the basic ingredients for impact evaluation? And how will you put them together to produce a successful case for support?
Nail your objectives
Impact evaluation is a business activity and, before you invest in it, you need to be clear on your objectives. It plays a role in delivering strategy and thus needs to be seen in terms of management rather than merely measurement and reporting to funders. Successful charities develop and refine their services continually, so impact evaluation must further your efforts to achieve the best results for the people you serve. It is vital to align your impact objectives with your charity’s declared mission; they will guide you in selecting data to calculate the value of your desired outcomes .
Find hard data to evaluate soft outcomes
Charity leaders often assume that formal measures don’t apply to the process of helping people deal with complex, subjective, sometimes traumatic issues. As it happens, PBE has a lot of experience here. One strand of economic impact evaluation is the cost-benefit analysis: in other words, what do you get out for the money you put in? When conducting an analysis of this kind, we generally follow HM Treasury guidelines. They explicitly cover “items for which the market does not provide a satisfactory measure of economic value”.
PBE’s expert volunteers – highly-qualified professional economists – analyse the available data and often contextualise it with relevant data from external sources. This approach made a major difference to our work for Tavistock Relationships, which helps couples and families. Our project evaluated three intended outcomes of the charity’s ‘Parents as Partners’ intervention: i) reduced levels of clinical depression, ii) improved child conduct and behavior, and iii) reduced risk of domestic violence. A volunteer from the Department for Education calculated social returns of up to £7 for every £1 spent. The chief executive of Tavistock, Honor Rhodes, tells us that the charity cites the findings every time it talks to local authorities. It has also shared the report with academics and DWP Policy Analysts as an example of best practice in the sector.
Remember there is no silver bullet
We take care to manage our client charities’ expectations for impact evaluation. Initial findings won’t necessarily confirm resounding success for your charity, but you need to be ready to draw insights from bad news and to acknowledge that small positive changes can make a big difference. When the British Science Association learned that its programme for GCSE students was not reaching certain disadvantaged pupils, it developed a bespoke support package for schools where a high proportion of children faced educational barriers. Success lies in capitalising on insights as you build your charity’s future strategy.
Pro Bono Economics believes that charities need to embed impact management into their DNA. As we help charities navigate through the entire process of evaluation, we draw on a strong base of practical experience. Pro Bono Economics is itself a charity, run by a team with a deep knowledge of the not-for-profit sector and advised by some of this country’s most distinguished economists.This expertise is channelled into a totally impartial assessment of the social value of a charity’s work, today and in the years to come. Since our foundation in 2009, we have engaged with well over 300 organisations, advising on data collection and producing economic impact assessments.
Our client charities are active in clearly defined areas, centered around wellbeing. They are at a stage where they want to heighten their impact by understanding what drives their performance. Our volunteer economists bring rigour to the evaluation process with an approach that is disciplined and pragmatic, but not lacking in empathy. Their focus is on producing robust conclusions and practical recommendations. We also want as many charities as possible to benefit from our findings so we publish all our impact reports on our website. Since they are independent and externally reviewed, charities and funders know they can trust the results and conclusions.
There is compelling evidence that charities can build long-term sustainability through a solid understanding of their impact. For instance, a 2016 survey by City consultancy FTI showed that 88 per cent of the public felt that charities would attract more donations by proving their impact on society. 81 per cent said they would prioritise donations to charities that could prove their economic impact. Donors of every kind gain assurance from understanding how their money is allocated and invested.
Despite all this, charitable investment in impact evaluation remains minimal. Pro Bono Economics can only offer its service to other charities because it receives philanthropic support from a small group of funders. Impact evaluation is also, of course, available from specialist consultancies – but it is often beyond the budgets of smaller charities.
The fact is that there is no logical reason why impact evaluation – or at least impact evaluation that can transform results – should come cheap. It is not a simple process and each charity has different needs, functioning in different circumstances and under different pressures. Specialist input enhances the immediate benefits of evaluation and generates long-term advantages.
Essentially, impact evaluation is too important to get wrong – yet charities are unlikely to adopt it as best practice if they have to fund it by moving money and resource away from frontline services. This is why PBE strongly believes that impact management needs to become integral to the entire philanthropic model. The time has come for grant-makers and other funders to provide ringfenced budget for the accurate and meaningful evaluation of a charity’s service. It will constitute a wise investment in that charity’s future sustainability.
Julia Grant is the chief executive of Pro Bono Economics
Related articles
