The IISaM project started out by focusing on the information sharing issues arising from the Troubled Families programme. Information is needed at each stage in that process: first of all, to identify families who meet the nationally set criteria, or any criteria which are a priority locally; to refer those families into whatever programme of support has been developed; and then, importantly, to monitor the success of the intervention. Like many current Government programmes, Troubled Families is funded on a ‘payment by results’ basis, where local authorities receive a payment once certain criteria are achieved – children are regularly attending school, anti-social behaviour has ceased, or adults are on the path to employment. Of course, it makes sense to try and focus our attention on the long term outcomes that Government wants to achieve, whether that’s a reduction in offending behaviour, improved mental health or sustainable employment. And in the current financial climate, it’s more important than ever to demonstrate the impact of public funding. To say anything otherwise would be heresy – like saying kittens are evil.
But in October, a conference called “Kittens are evil” will be exploring the pitfalls of outcome-based performance measurement. I attended an earlier edition of the conference, up in Manchester, and found it a thought-provoking day. Dr Toby Lowe from Helix Arts discussed his research, which explored how difficult it is to attribute outcomes to a single intervention. For example, the Work Programme works with the long term unemployed to try and address the issues which might be stopping them getting into work. Individuals might get help with writing a CV, or with interview skills. But there are so many factors beyond the control of the Work Programme, which influence whether an individual is able to get a job. An article in the Guardian back in May highlighted one potential problem – the current economic climate. A regional operations director for a Work Programme supplier is quoted as saying,
“In order to make the Work Programme work, we need more jobs. We can do our work to prepare people for work, but sometimes there are just not the right jobs for our client base”.
The difficulty of attributing a change in an outcome to a single intervention means that you end up using an alternative measurement – something that IS easy to tie down. If you attach payments to that measurement, you then risk creating a new de facto purpose – substituting out the “good idea, but hard to measure or attribute”, and substituting in “subtly different but actually possible to measure”. The Work and Pensions Select Committee’s report into The Work Programme noted that the programme has been designed to eliminate the practice known as “creaming and parking” – focusing on participants who are easiest to work with (and more likely to trigger a payment) at the expense of more complex cases. But the Committee concluded that,
There is growing evidence that the Work Programme is failing to reach jobseekers with the most severe barriers to employment… We recommend that DWP assess how a needs-based differential pricing structure might determine the level of up-front funding and the types of services required by jobseekers referred to the Work Programme and whether alternative funding models, which reward providers for achieving milestones along the way to employment, should apply to jobseekers who are furthest from the labour market. (Paragraph 104)
So how do we resolve this issue? Nobody wants to return to a focus on outputs rather than outcomes. The other speakers at the day offered some insights into alternative approaches; the chaps at Perfect Flow use realtime performance data in order to radically change how housing repairs are managed, for example. Simon Guilfoyle from West Midlands Police has even written a book about how to measure the success of your organisational or service purpose. If you want to spend some time thinking about how information can be used to improve public services, you might want to consider taking a closer look at those kittens.