National Evaluation of the Troubled Families Programme
Resource Pack for Local Troubled families Teams
Use the links below to view additional information
How can local authorities support the evaluation and what will you get in return?
The four elements are further outlined below.
1: Who are the troubled families?
In September all local authorities were asked to collect monitoring data on at least 10% of their families in each year of the programme (ie, 10% of families in year 1, then year 2, and so on), aligned with a template that was circulated. That set of indicators was purposely kept to the minimum you told us was workable but essential to monitor progress.
We would now like to start gathering what data you have using a secure online system we are developing, and will be telephoning you in the next four weeks to talk about how we can make this work for you. We will in future aim to align the collection of the monitoring data with the management information to provide to DCLG.
There will be two stages involved for setting-up the system at a national level -
• First, the evaluation team will develop an online version of the monitoring data template provided by DCLG, and talk to each local authority to gather and input Year 1 data. Each of these ‘family records’ should include a Unique Reference Number so it remains anonymous for the evaluation team, with no personally identifying information in the online system.
• Second, we need to establish a more sustainable model of monitoring data collection for the rest of the evaluation. This will either involve supporting local authorities inputting to the online MI database directly, or to explore other ways to align local data systems with the national system. We don’t want to cut across existing local systems but build on them, and avoid anything like the NatCen system for FIPs, which was far too time consuming and complicated.
LAs will be able to access and use the data themselves. The monitoring data system will be set up to provide a reporting tab for local authorities to use, so that they can run tables and extract their own data . The aggregate data will provide a rich source of information for the annual national evaluation reports.
2: What outcomes does the troubled families programme achieve?
This, the “impact assessment”, will track families as they progress through the programme (and beyond). It will use data from two main sources to measure outcomes achieved beyond the PBR results:
1. National datasets, such as the National Pupil Database, benefit and employment records, the Police National Computer and Hospital Episode Statistics.
2. A survey of around 1,000 troubled families in early 2014, 500 of whom will have been on the programme for a number of months, and 500 of whom will be about to start the programme. (The survey will take place in 10 ‘level 1’ Local Authorities – see section 4 below.)
Depending on local data collection plans, we may also use data on other outcomes (e.g. housing) that is routinely collected on families and that can be made available to the evaluation team.
The data will be used to measure the impact of the programme by comparing the outcomes of families who have gone through the programme with the outcomes of families who have been identified as eligible but have not yet started the programme.
Both the national datasets and the survey will allow us to track families in the period between their identification as eligible for the programme and the point at which local authorities start working them as part of the programme. This will provide a comparison group, against which the outcomes of the families going through the TF programme can be measured.
This is where we would need your help, over and above your collection of family monitoring data. We can access the national datasets but we would need you to provide “identifiers” for individual family members that should have benefited from the intervention, if you have them already or could get them as part of providing the services. That could be a Unique Pupil Reference Number, a National Insurance Number, NHS number, etc. However, having provided these data, it will be possible for the evaluation team to monitor outcomes without the need for further input from local authorities. Our work with Government Departments so far suggests consent would not be needed since families would not be identified – all the results are pooled together and anonymised.
3: How is the TF programme delivered?
This, the “process evaluation”, will involve in-depth work with at least 20 TF ‘case study’ areas (Level 1 and 2 local authorities – see section 4 below). Its focus will be on system and service redesign and reform. It will be based on in-depth interviews and focus groups with staff, in depth interviews with families, and workshops with local partners, both strategic and delivery.
4: How much does the troubled families programme save?
This, the “economic evaluation”, will draw on evidence from the outcomes achieved by families. This will be combined with information on unit costs (using national and, where possible, local data) to result in an estimate of financial savings. Information on the costs associated with delivery of the programme will be obtained through the “how is the programme being delivered” (process) evaluation and the family monitoring data. Information on costs and benefits (in the form of savings/costs avoided) will be used to produce a cost benefit analysis (CBA) of the programme.
We would like to work with five of our Level 1 local authorities (see below) to build up detailed local CBAs. We are currently working on a prototype ‘cashable savings calculator’, and this will be available as soon as possible this Summer. In addition, we will produce CBA guidance and self-completion tools for other local authorities to use, and we will be inviting any interested authorities to submit their data to the evaluation team to feed into the economic evaluation (more details on this to be provided later).
(1) We are also looking into another way of measuring the real impact on families - i.e. allowing for what would have happened to the families without the programme. We are currently testing the feasibility of comparing the outcomes of families who went through the programme with the outcomes of those who were ‘almost eligible’ for the programme. Tracking these families could provide a second comparison group against which to measure longer-term impacts, but we would like to hear your views on whether this is feasible and whether it is data that would help you locally before deciding whether to take this analysis forward