We still have a long way to go in ensuring that good quality evidence is used to guide decision-making in public policy, says Genevieve Maitland Hudson.

Last week’s Public Administration and Constitutional Affairs Committee hearings took evidence from two MPs who had contrasting understandings of evidence. Tim Loughton used the language of impact and outcomes and touched on the need for a positive matrix of shared outcomes for the youth sector. Oliver Letwin used the language of belief. He “believed” that Kids Company was doing good work, he continued to believe this despite the charity’s sudden closure, and despite, in his own words, “its gross financial mismanagement”. Asked by the Chair where he drew this belief, he mentioned a short visit to the charity at some point between 2001 and 2003.

These two very different accounts, given on the same day, demonstrate the particular problems that still face the voluntary sector in general – and the youth sector in particular – in thinking about, and demonstrating, social value.

It is notable, for instance, that Tim Loughton is no longer a minister while Oliver Letwin now runs a department. This suggests that despite the investment in the What Works Centres and some rhetorical commitment to the use of better quality evidence in commissioning and grant-making, this is not the basis upon which important decisions are made by the current government. To judge by the hearing, Mr Letwin was influenced far more by the excellent marketing and salesmanship of Kids Company’s chief executive than he was by the kind of evidence that Mr Loughton touched upon, the kind of evidence which this particular charity never produced.

Separate evidence given to the Committee by Richard Heaton, Permanent Secretary to the Cabinet Office at the time of the final two grants to Kids Company, showed a better grasp of the use of outcomes, but still fell short of the kind of familiarity with the process of robust evaluation that would have reassured those of us who work in this area.

The Kids Company inquiry tends to show that the problems outlined in the Institute for Government’s 2012 report on ‘Evidence and Evaluation in Policy Making’ still stand. The demand barriers identified in that report were:

• Problems with timeliness and helpfulness of evidence
• The fact that many political decisions were driven by values rather than outcomes
• The lack of culture and skills for using rigorous evidence in the civil service
• A need to create openness to feedback among service providers

The Kids Company inquiry shows all of these. The research produced by Kids Company itself was of mixed quality and did not answer the question ‘what works?’. Government decision-making was – at best – driven by belief rather than evidence, and perhaps equally by even less rigorous motives. The ministers in question and their civil servants did not have the necessary understanding of evidence to support the charity to put a proper evaluation framework in place. The charity itself was actively resistant to assessing the quality of its interventions.

This leaves the youth sector in a bind. Even those committed to better quality evidence might reasonably ask themselves if investing scarce resources in generating it is worthwhile in this kind of climate. We can, and must, make sure that it is by learning from it and actively improving the quality of our programmes in the light of it. This will put those we aim to support at the heart of our evidence gathering and cut through the mesh of poorly aligned incentives from commissioners and grant-makers. It will mean making significant changes to current approaches to social value measurement. It will mean, as Tim Loughton noted, shared frameworks and, importantly, shared data. Policy-makers can’t make this change on our behalf. Time then, to do it for ourselves and bring them along with us.