National Evaluation of the Troubled Families Programme

Resource Pack for Local Troubled families Team

Level 1

authorities: enhanced case studies

 

Overview of activities:

The 10 local authorities that become Level 1 authorities will be involved in the following evaluation activities:

 

• How the programme is being delivered (the process evaluation)

• Who are the families, what problems do they face, what progress do they make (monitoring)

• What outcomes do the families achieve (the impact assessment - national dataset tracking)

• What other outcomes families achieve and their views (impact assessment – survey)

• What savings the programme makes (the economic assessment)

 

Evaluation activity 1: Process evaluation

Evaluation activity 2: Monitoring

Summary of activities

Implications for local teams

Summary of activities:

The process evaluation for Level 1 authorities will involve:

• Interviews with the LA/service team

• In-depth interviews with a small number of families

• Workshops with local partners. 

 

In Level 1, families involved in the in-depth interviews, and staff involved in their case will be given digital cameras and asked to keep a photographic record of their experiences throughout the evaluation. 

 

There will be three ‘waves’ of data collection, one per year of the evaluation: the start of their intervention  in Summer or Autumn 2013; at the end of their intervention (and we need to talk to you about what that means, at least for the purpose of the evaluation); and 6-9 months later

 

Implications for local teams:

The key implications for Local TF teams are that:

• Your key staff are available for interview.

• Some of those staff will need to be willing to keep a photographic record of their experiences.

• You will help with the recruitment of families for the in-depth interviews.  

• Your help in identifying the local partners and stakeholders that you work closely with to take part in the workshops.     

 

In between the fieldwork ‘waves’ we will ask TF teams to stay in touch with us via an online forum. This will operate as an online diary for each TF service to record their experiences and views at key points during the evaluation.

Summary of activities

Implications for local teams

Summary of activities:

Phase 1: We will consult with local authorities about how best to gather and input the data that you hold to date (Years 1 and part of Year 2) on your troubled families, in order to provide aggregate figures for whole of the programme.  Monitoring data will be needed for at least 10% of troubled families per authority, but up to 100% where feasible.

 

Phase 2: We will consult with local authorities on the development of the new online monitoring data collection system.  Local authorities will be asked to provide quarterly or six monthly data returns (TBC) once this system is operational, and we will aim to align this with the Management Information returns.

 

Implications for local teams:

Your help in providing access to your existing data on troubled families and work with the national evaluators to enable the secure transfer of this information, using Unique Reference Numbers (URNs) in place of personal identifying information.

 

Your support of the work of the national evaluators to identify the optimal approach for inputting to the national MI system when this is operational, including either data entry to the secure online database, or an arrangement to extract and securely transfer data-files from your existing case management system. .

 

 

For more details on this Activity: Click Here

For more details on this Activity: Click Here

 

Evaluation activity 3: Impact evaluation

(national dataset tracking)

Evaluation activity 4: Impact evaluation 

(surveys of families)

Summary of activities

Implications for local teams

Summary of activities:

Local authorities will be asked to supply “identifiers”, like Unique Property Reference Numbers,  for individual family members (from troubled families and the comparison families) to allow them to be tracked via national datasets such as the National Pupil Database, benefit and employment records, the Police National Computer and Hospital Episode Statistics.

 

Implications for local teams:

We will work with the local teams to support them in providing some basic information on  individual family members to allow them to be identified in the national datasets, linked to the type of eligibility criteria they met (crime, ASB, education, benefits or local) and the date of first contact with the TF programme.

 

We are looking into the consent process needed for the comparison group of families, as it is possible that we will need local authorities to collect consent from families for the linking of datasets.

 

 

Summary of activities

Implications for local teams

Summary of activities:

For each Level 1 local authority, we want to carry out face-to-face interviews with 100 families in early Spring 2014. The surveys will be carried out by trained interviewers from Ipsos-MORI.

 

50 of these families will have started the programme in early 2013.  The other 50 families will be families who were identified as eligible for the programme but who are due to enter onto the programme in spring 2014. These will act as ‘comparison families’. For the purpose of the evaluation, the waiting period should be as long as possible.

 

 

 

Implications for local teams:

Level 1 authorities are the only authorities within which this survey will take place. There are a number of implications of taking part:

 

• Local teams will help in recruiting families to the survey. We will provide materials and guidance on this, but the initial approach to families may need to be done by your teams.

• Recruiting the ‘comparison families’ is likely to be particularly difficult. We will need to discuss with each

Level 1 authority the best way to do this.

• We need to conduct the interviews with the comparison families before the programme starts with these families. This may affect the timing and nature of your early work with these families.

 

 

For more details on this Activity: Click Here

For more details on this Activity: Click Here

Evaluation activity 5: Economic assessment

Summary of activities

Implications for local teams

Summary of activities:

We will draw on data collected as part of the ‘how’ (process evaluation), ‘who’ (monitoring) and ‘outcomes’ -  the findings of the impact assessment (survey and national dataset tracking) -  to compile the economic assessment.  The aim, as always, is not to duplicate existing tools but to build upon them, e.g .in aligning collection of monitoring data and management information

 

We will also work with up to five authorities to develop a detailed local Cost Benefit Assessment (CBA).

 

CBA guidance and self assessment tools will be made available to enable other local authorities to estimate savings.

 

Implications for local teams:

In addition to the implications set out for the process, monitoring and impact strands, those authorities which agree to undertake a detailed local CBA will be asked to help collect some additional data, on delivery costs and potential cost savings. Guidance and tools to inform this data collection will be provided along with support from a member of the research team.

 

CBA guidance and tools will also be provided to those not undertaking detailed CBAs to support self evaluation activity.

 

 

 

 

 

For more details on this Activity: Click Here

What you would receive back?

 

All Level 1 authorities will receive a bespoke local evaluation report for their area that includes summaries of the results across the different components of the evaluation (including CBA where relevant). The report will include a ‘digital story’ or case illustration comprising an edited collection of the photographs (with narration or soundtrack) captured by each TF service and the families who took part in the evaluation. 

 

Is your local authority suitable?

 

Not all local authorities will be suitable to be a Level 1 authority. The most crucial requirement is that the families joining the programme in spring 2014 (the survey comparison families) have similar characteristics and problems as the families starting in spring 2013. That is, the order in which families start the programme needs to be close to random.  This is because any process of selecting families that it as close to random as possible will mean that the sample of families is genuinely representative of all the families supported. We can handle some variation on this (you may, for instance, have prioritised families with the very greatest levels of need), but as long as the process for the majority of families is fairly random you may be eligible to be a Level 1 authority.

 

However, there can only be 10 Level 1 authorities in total, and we need to ensure that the 10 chosen cover, between them, a range in terms of delivery models, areas and eligibility criteria (including local eligibility criteria). So unfortunately it is possible that not all those local authorities eligible and  volunteering for Level 1 will be selected for this level.