Key elements of effective impact measurement are the telling of the story and fitting measurements to an organisation's intervention, says a new report

The report, Measuring social impact in social enterprise, summarises findings from an event held this year, which saw a range of those involved in measurement, including providers, funders and commissioners, discuss issues around impact measurement.  This was designed to find the common threads and necessary differences in impact measurement , and feed into the GECES advisory group supporting the European Commission in developing legislation and policy under the EU Social Business Initiative.

The report says the day identified common factors of effective impact measurement including a clearly enunciated story, a clarity of beneficiary perspective and the demonstration of change over time. It also identified considerable difference in what is demanded of measurement by different commissioning environments, with factors such as centralised or localised commissioning, the complexity of the interventions being delivered, and the need for buy-in by service-users to achieve effectiveness all playing a part.

The overall conclusion is that much of existing measurement fits within a range of alternatives, many of which are useful, but the choice of which will depend on the needs of the user, rather than driven by funder or commissioner need.

A new aspect that is developed in the report is how presentation of measurement, indeed the act of measurement itself,  may influence the behaviour of the various parties (commissioner, providers, funders, service users and communities) to make interventions more possible and more effective.  It suggests that more thought needs to be given to how the measurement is presented, and what measures are chosen, in order to appeal to the various key parties’ level of interests and understanding.

One of several differences that is discussed is the different timescales that commissioners, providers and beneficiaries may take.  Commissioners can be expected to focus on shorter term gains (in cashable savings, as well as in response to wider policy imperatives), which may conflict with the longer-term outcomes being sought by providers and beneficiaries.

Social investors relying less on SROI than commissioners and providers

Whilst recognising considerable value in monetising, and the use of ratios in simplifying presentations, the report also suggested that there was a shift away among social investors from putting a ‘pound value’ on social impact because of the difficulty of comparing outcomes for different services using the conventional social return on investment method. The shift was towards the use of the underlying approaches implicit in SROI and Social Accounting to tell the story.  Funders expressed a strong preference for the measurement to follow, and reflect the needs of the intervention.

The report says that  funders are not always attempting to put a ‘pound value’ on social impact because of the twin issues of "ratio inflation" - overstating results - and the difficulty of comparing the services that different types of charities and social enterprises deliver with social return on investment methods. 

It adds: “It is worth noting that the issue of quantifying social impact was scarcely focused on by social investors in both care and education groups, even though this is often cited as the ‘holy grail’ of impact measurement…from the non-funders’ point of view it was interesting to see funders giving a lower priority than they expected to monetising the benefits.”

In response, Jeremy Nicholls, chief executive of the SROI Network, said that inflation or overstating results was an issue for any type of impact reporting. “I imagine if financial accounts were not audited we’d see a lot of companies' profits rise,” he said. All social impact measurement is constructed around judgement so appropriate assurance processes were necessary, he added. 

Nicholls did however agree that there were issues around inter-comparability of SROI ratios as opposed to analyses. "Nonethless it is a useful measure for organisations to compare internally. The strength of a standardised approach is comparability of the process and valuation in SROI also helps ensure that analyses are easy to compare," he said 

Nicholls also said that whether valuation is useful depends on the audience and the purpose behind the analysis of  impact.

Social investment influencing impact

The report also says there was wide consensus within the groups that the developing social investment market was influencing the impact measurement agenda, as funders needed to prove the impact of their portfolio to their own investors. 

“The funders were key participants in the drive towards common measurement frameworks and benchmarks,” it says. “But, while funders wanted to develop their thinking when it came to impact measurement, they did not seem to focus on quantifying impact or picking a particular methodology for impact measurement.” 

The report also warns that there is a perception among service providers that even if a commissioner is seeking measures that are not best for monitoring or securing outcomes, their view holds sway and is not to be challenged.

And, some service providers expressed concern that the government was driving the outcomes agenda, and there was the danger of social value being seen in the context of cost-cutting rather than systematic redesign and innovation. 

“One impact of this could be the disjoint between achieving long-term social outcomes over short-term commissioning (or political) periods,” the report warns. 

The report was written by Jim Clifford of Baker Tilly and Cass; Kate Markey, managing director of CAN Invest and Natasha Malpani, who leads Big Society Capital’s social impact strategy.