Search:
Our latest evaluation blog post from Professor Neil Humphrey and Dr Qiqi Cheng
Does participation in Power 2 Rediscover improve the mental wellbeing of at-risk young people?
What is Power 2 Rediscover?
Power 2 Rediscover is a targeted intervention for young people who are considered at-risk because they struggling with their wellbeing, feeling isolated, and/or disengaged with school and learning. The intervention aims to support them with these challenges, via ten weekly 1:1 sessions delivered by a trained facilitator. These sessions include conversations and practical tools to help young people develop positive routines, healthy habits and sleeping patterns in support of their mental health; raising awareness of their social support options to address feelings of loneliness and isolation; provision of practical and emotional support to help them reengage with school and learning; and, support to set goals and aspirations for both the long and short term. All young people earn a certificate of achievement upon completion of the programme. You can read more about Power 2 Rediscover here.
What did we do?
Our aim was to provide a preliminary independent evaluation of Power 2 Rediscover. To do so, we used propensity score matching (PSM), a statistical technique that can be used to ensure that an intervention and comparison group in an evaluation are well matched. PSM enabled us to test whether there are differences in mental wellbeing between young people who have participated in Power 2 Rediscover and a matched comparison group drawn from the #BeeWell dataset. We have previously used this approach to assess the impact of the Football Beyond Borders intervention (read about this here).
Power2 securely shared mental wellbeing data (assessed using the SWEMWBS measure, which we also use in #BeeWell) for 70 at-risk young people, collected at the end of their participation in the programme, in addition to socio-demographic information (e.g. gender). Using PSM, we identified a well-matched comparison sample from within the #BeeWell dataset. Having done so, we were able to assess differences in mental wellbeing among young people who had participated in Power 2 Rediscover and young people with very similar characteristics who had not participated in the programme. We matched our two samples on gender, special educational needs, free school meal eligibility, year of survey completion, year group/age, and disengagement from school. The latter is a key selection criterion in Power 2 Rediscover, so we used a composite school connection score derived from responses on our school belonging and relationships with school staff measures to identify young people in the #BeeWell dataset who were disengaged.
What did we find?
We found a statistically significant difference in mental wellbeing between Power 2 Rediscover participants and our matched comparison sample, favouring the intervention. This difference was equivalent to about four and a half points on the SWEMWBS scale (a standardised effect size of d=0.8). To check the robustness of this effect, we ran a series of additional ‘sensitivity’ analyses, in which we change something in the analysis to see if it makes a difference to our main finding. In most sensitivity analyses, this was not the case, leading us to conclude that there is a clear difference in mental wellbeing scores of young people who have participated in Power 2 Rediscover and those who have very similar characteristics but have not participated in the programme.
Building on the above, our colleagues at Pro Bono Economics performed an evaluation designed to assess the economic benefits of Power 2 Rediscover. Using #BeeWell data, we provided them with estimates of how the observed differences in mental wellbeing would translate into differences in life satisfaction, and thus, ‘wellbeing years’ (known as WELLBYS), which can be monetized (see here). Pro Bono Economics performed analyses using these data, alongside information about the costs of delivering Power 2 Rediscover, ultimately determining that for every £1 spent, the programme could provide more than £5 of wellbeing benefits. You can read their report here.
What does this mean?
Our analysis provides preliminary independent evidence that Power 2 Rediscover may improve the mental wellbeing of at-risk young people, and the accompanying work by Pro Bono Economics indicates that the programme likely provides a good social return on investment. However, it is very important to note a substantial limitation of this research. Due to the relatively short length of the intervention (10 weeks, compared to the 12-month lag between surveys in #BeeWell), we only able to assess differences in mental wellbeing between our groups at a single point in time, as opposed to assessing differences in changes in mental wellbeing over time. Accordingly, our findings must be treated with caution.
Several next steps are recommended. First, we need to determine if the promising findings outlined above are replicated in a study where we can assess actual changes in mental wellbeing. Second, we also need to assess whether Power 2 Rediscover reduces feelings of loneliness, increases engagement with school, improves self-confidence, and other key outcomes in the programme’s theory of change. Third, we need to better understand how and why the programme works, through rigorous evaluation of potential ‘treatment effect moderators’ (e.g., the number of sessions attended by young people). Finally, we need to determine if any impact of Power 2 Rediscover on wellbeing and other outcomes for at-risk young people are maintained in the longer term.
This evaluation showcases the utility of the #BeeWell dataset in helping to understand how best to support young people’s wellbeing. We hope to undertake similar evaluations of other interventions in the future.
* The estimated economic value assumes the effect lasts for six months. If the effect duration differs, the economic impact may vary accordingly.
