Research Library

This peer-reviewed, prize-winning article describes the theoretical underpinnings of the Outcomes Star and the development process used for all versions of the Star. This process, is driven by the needs of service delivery organisations to have practical tools for supporting and measuring change. Each new Star undergoes a very thorough process of testing and revision involving front-line staff, managers and service users.

This working paper emphasises that the Outcomes Star ‘ideally facilitates the interaction between counsellor and client’ and ‘not only measures outcomes, but is instrumental in helping the desired outcomes to be achieved’. It also discusses the values underlying the Outcomes Star, including ‘the reversal of roles underlined by power and knowledge, usually represented in evaluations by the powerful funder, the mediating evaluator, and a less powerful service user.’

This report documents a seminar in which six organisations using different versions of the Outcomes Star to share their experience of analysing Star data.  The report sets out the benefits the organisations had found in using the data and the challenges they faced.

This peer-reviewed paper describes Triangle’s case study approach to demonstrating inter-rater reliability and provides initial findings for the Family Star (first edition) using a sample of 24 trained workers. The inter-rater reliability reached the accepted threshold of 0.8 for the five-point Journey of Change when three outlying workers were excluded. Following the publication of this article, Triangle found very similar results when re-analysing the data using a chance-corrected reliability statistic.

York Consulting evaluated the implementation of the Family Star within Family Action involving the analysis of more than 3000 Stars, interviews with service users and staff across four locations, and interviews with commissioners and Family Action’s management.  They concluded that the Star helped develop service users’ resilience and helped workers reflect on their practice, particularly in terms of improving outcomes and that Family Star data can provide valuable insights into the extent and nature of changes occurring, as well as highlighting areas for further investigation.

A hierarchical criterion based approach was used to evaluate which of 22 instruments of individual’s mental health recovery might be candidates for measuring recovery in the Australian context. Only four tools were identified as promising candidates for routine use in Australian public sector mental health services: Recovery Assessment Scale; Illness Management and Recovery Scales; Stages of Recovery Instrument; and the Recovery Process Inventory. At the time of this study, The Recovery Star was classified as ‘never subject to scientific scrutiny in a peer-reviewed article’, so it was not considered on the remaining criteria.

This study explored the psychometric properties of the Recovery Star using data from 203 adults attending mental health services. The Recovery Star was found to have high internal consistency, a two-factor structure and appeared to measure underlying recovery-oriented construct. The items were responsive to change with statistically significant improvement over time. Correlations between items confirmed little item redundancy. The authors conclude that the ‘Recovery Star has been received enthusiastically by both mental health service providers and service users. This study provides further evidence for its adoption in recovery-focused mental health services’.

This article describes the benefits of implementing the Recovery Star approach in a secure learning disabilities service. Key workers found that the tool ‘opened up avenues for discussing topics covered in the domains of the Recovery Star tool which may otherwise have not been discussed as fully’. They concluded that ‘the Recovery Star tool, embedded in a care programme approach process, equips patients and staff for measuring the recovery journey”.

This study demonstrates the responsiveness of the Recovery Star in a transitional intervention service facilitating return to community following mental health crisis. Recovery Star and personal goal achievement data was collected at service entry and exit (181 service users), at four sites in England. There was a significant increase in overall Recovery Star scores with a large effect size, and significant increases in eight of the ten Recovery Star life domains. There were significant increases in the goal scores linked to “Managing mental health”, “Self-care” and “Living skills”.

The Recovery Star had good acceptability, test-retest reliability and convergent validity between social domains of the Star and a well-established, standardised measure of social functioning. Service user only readings did not converge with the Mental Health Recovery Measure (MHRM; Young & Bullock, 2003), however the MHRM has been described as ‘containing items that are not consistent with the definition of psychological recovery’ (Anderson et al. 2006) and it is recommended that the Star completed collaboratively with service users. There was relatively poor convergence between worker-only Star readings, which emphasises the importance of staff and service-users collaborating to produce accurate scores. Several critiques of this article have been published.

This study showed significant increases in Recovery Star scores and achievement of personal goals from entry to exit of a crisis house. Recovery Star and personal goal scoring data were for 722 services users. There were statistically significant increases in all Recovery Star domains. The authors note that the findings ‘highlight the value of using recovery-oriented support planning and outcome capture tools in routine practice’.

This is a review of the utility of the Mental Health Recovery Star as a clinical outcome measure. Service users at a Mental Health Recovery Service in Queensland, Australia completed the Recovery Star. Service users identified that they needed assistance in work, social networks, managing mental health and identity and self-esteem. The review concluded that the Recovery Star was ‘useful in service mapping and assisting recovery clinicians to identify areas that they needed to focus on when providing treatment and following service user's progress. It complemented other outcome measures used by the service.’

A short history of routine clinical outcomes measurement (RCOM) in UK mental health services. The authors note that ‘one of the issues of patient reported outcome measures is the degree with which, although completed by service users, they address actual concerns of service users…as opposed to those of clinicians, managers, service providers and governments. The Mental Health Recovery Star is designed with this in mind’. The review also states that ‘experience by United Kingdom Routine Clinical Outcomes Measurement in Mental Health Network (UKRCOM) members suggests that it is a useful tool for a collaborative approach to care planning and perhaps outcomes measurement.’

This study demonstrates that the Recovery Star is responsive to change. Service users completed the Recovery Star before and after sessions with a Psychological Wellbeing Practitioner. For the three biggest areas of change, service users were asked, ‘‘What difference has this change made to your life?’’ There was improvement across all areas of the Star, with the biggest improvement in the managing mental health area.

A description of the origin, development and increasing application of the Recovery Star within the UK. It was found that the Recovery Star has been instrumental in promoting social inclusion for many service users, their carers and families. The Recovery Star has rapidly established itself as the recovery tool of choice for many service users and providers and fits in well with the personalisation agenda.

This is a review of instruments of mental health recovery. The results of a single article (Killapsy et al., 2012) was used as evidence that the Recovery Star does not converge with some other tools, and the Star was criticised on the basis of it taking more than 30 minutes to complete. The authors falsely state that ‘no service user involvement was reported in the development of the MHRS’.

This paper recognises that the Mental Health Recovery Star (MHRS) has received focus at a national policy level and reports the views of 12 mental health professionals about its use within clinical practice. The tool was considered useful for informing care planning and gathering new information (e.g. the wider context of mental health difficulties). The collaborative nature of completing the Recovery Star was valued and service users had given positive feedback about their experience of the Star. It was also emphasised that it is beneficial to have a good relationship with service users before completing the first Star.

The Recovery Star model was used as a guide for discussion in group work sessions in residential programmes for anorexia nervosa. Patients brainstormed skills and issues associated with each area of the Recovery Star and also self-completed the Star. Group therapy based on the Recovery Star was an important component of treatment. The authors ethos of the residential treatment programme for Anorexia Nervosa was ‘epitomised in the Recovery Star model’.

In this doctoral theses, a literature review examined the utility of the Recovery Star and similar instruments in recovery oriented mental health services. Preliminary findings were that the Outcome Star is effective in monitoring and facilitating change. The Outcome Star was found to possess many of the aspects of recovery model: empowering clients to make change, seek supportive environments, promote inclusion, meaning and importance in relationships. The author concluded that “it is possible that the Outcome Star will become adopted by many recovery orientated mental health services’.

This report documents a seminar in which six UK academics, three service providers and one service user came together to share ideas and expertise around the academic studies that have been carried out on the Recovery Star and set the research agenda going forward. The report sets out key conclusions on the validity of the tool and actions to take knowledge forward.

Southside Partnership and the Mental Health Providers Forum (MHPF) undertook a piece of action research in which a Recovery Star Project Coordinator worked with 15 services within Southside to support implementation of the Recovery Star and evaluate the impact of using the Star. The report was published by MHPF in 2009 and found positive impacts on service users, frontline staff and managers.

This multi-component service evaluation project focused on questions about the importance of the Recovery Star domains for care planning, and the perceived impact of current treatment practices on these recovery domains. The researchers concluded that ‘the ten domains extracted from the Mental Health Recovery Star appear to provide a useful basis for examining recovery (at both an individual and a service level)’.

An evaluation of the effectiveness of the Recovery Star for people from black and minority ethnic backgrounds. The Recovery Star was considered to be a valuable part of key-working sessions by the majority of service users, and most areas on the Star were seen to have cross-cultural relevance. The research showed that improvements could be made by providing further acknowledgement of the cultural practices that shape and impact on the lived experience of mental ill health and as a result of this study a second edition of the Recovery Star was published which addressed the specific issues highlighted in the study.

A peer-reviewed article. The Homelessness Star was completed by prisoners at beginning and end of a training programme delivered by Health Trainers. There was improvement in all areas except alcohol and family relations. Attitudes to alcohol use stayed positive and constant because they were unable to obtain alcohol. Family relations were reported more negatively after the training because it helped them understand that their criminal behaviour had been influenced by damaging family relationships. To conclude, this study demonstrates that the Homelessness Star is responsive to change in key areas.

This peer-reviewed study looked at changes in wellbeing and use and cost of wider local services (police, health, housing etc.) within the Making Every Adult Matter (MEAM) service pilots. The Homelessness Star was completed by 39 clients, with an average period of nine months between initial and final measurements. There were significant improvements in wellbeing for nearly all clients across three quantitative measures (including the Homelessness Star). The evaluation also recorded changes in the use and cost of local services. In Cambridgeshire, the reduction in crime costs (£100,000 or 31 percent) was large enough to lead to an overall cost reduction.

A peer-reviewed article. An evaluation of a multi-agency project aimed at improving services from the third sector to support health and social care needs. The majority of the 100 patients saw significant improvements in most areas of the Homelessness Star Outcomes Star. The only areas with little improvement were drug and alcohol (which was not an area of great need in this client group), and accommodation.

A peer-reviewed article. The Homelessness Star was used with 15 men in a homeless shelter over a 6-week period. Each participant chose two domains on which to focus their change efforts. The study found that participants demonstrated statistically significant change.  The authors conclude that the tool is effective in guiding homeless people towards self-reliance and could “change the face of the way that shelters provide and share care with the homeless population.”

A peer-reviewed article. An evaluation of a life coaching programme (Coaching Inside and Out; (CIAO) for offenders using the Outcomes Star. The change on the Outcomes Star was ‘impressive, showing positive movement in the average scores across all areas, most pronounced in motivation/ responsibility and emotional and mental health’. The movement in the lowest scores was particularly good - in all areas bar one (accommodation), this was 1 or 2 at the start of coaching, rising to at least 5 (conscious desire for change).

All 25 organisations included in this evaluation reported that key-work had improved as a result of using the Outcomes Star; that key-work had become more focused on service user change, more systematic and consistent and covered a wider range of issues than before the Outcomes Star was used; and those organisations that analysed the data that they collected found that the data was helpful in assessing the effectiveness of the service and identifying areas for improvement.

This article reports on an in-depth action research study of the implementation of the Homelessness, Drug & Alcohol and Empowerment Star within St Kilda’s Crisis Support Network in Australia.  The report identifies 20 benefits to the project of using the Star at organisation, programme, practitioner and client level and concludes that the Star provides a unique innovation in human service delivery.

This article supports the person-centred approach of the Homelessness Star. The authors state that: ‘current advice suggests that supportive and collaborative relations with the client are more effective than technical or coercive interventions. This supports the experiences of many housing resettlement staff that user-defined outcomes are the strongest basis on which to build. Use of person-centred approaches such as the ‘Outcomes Star’ can give this a specific focus’.

This article reviews measures of member engagement. In relation to the Outcomes Star, the author notes that ‘expectations at each of the points on each scale are clearly defined… The flexibility of this approach allows different stakeholders to contribute their own perspectives to the evaluation, which, in turn, allows different questions to be answered for different people….’ They conclude that ‘The Outcomes Star is a useful tool for self-evaluation.’

A peer-reviewed article. The responsiveness of the Well-being Star is demonstrated in this evaluation of 3DFD (3 Dimensions of Care for Diabetes) – a service for improving glycaemic control and reducing diabetes complications in South London’s multi-ethnic, disadvantaged diabetes population. The service produced improvements in glycaemic control, psychological status (including significant change on the Well-being Star) and health service use.

This peer-reviewed article describes how a youth homelessness service (‘1625’) implemented the conceptual ideas of the psychologically informed environment (PIE) into a practical and beneficial service for very challenging young people who have been homeless, are leaving care or have left custody. The author’s note that the Young Person’s Outcomes Star is ‘embedded in the organization and provides a valuable tool for evaluation and measurement of outcomes’.

This report concluded that the Spectrum Star is "a very useful tool to use with young people with ASD, given its holistic and visual approach to the assessment" and that it "empowers young people to make decisions with regards to intervention, while still including parents/carers at the action planning stage." The Star was recommended as a "valuable tool to implement into service delivery within the Paediatric ASD Service.”

A study of 12 pairs of keyworker and service user experience of using the Life Star found that "clients enjoyed the experience of talking about various areas of their life", and that the "Journey of Change appeared to resonate with the experience of participants." Workers found the Life Star conversations helpful in understanding their clients and that it complimented and accentuated the existing Person-Centred approach. In addition, the study found that "the Life Star's emphasis on collaboration promotes more choice and control for the client, which ultimately leads to empowerment." There was a degree of scepticism around data privacy and use, highlighting the importance of clear communication and engagement with staff and service users when implementing the Star. 

The final report of a project evaluating the use of the Work Star with vulnerable 16+ young adults across 6 services including the Connexions team and local, partner organisations.   The study found that feedback from partners involved the pilot was “overwhelmingly positive,” with particularly strong endorsements for the way the Work Star created “a consistent approach within and between organisations in how we describe where particular clients are on their journey towards employment” and the opportunities it gives for “line managers to systematically review case work and engage in outcome-focused discussions during supervisions linked to specific casework.”

The Spectrum Star was used as a keywork and outcomes tool within a 12-week specialist Enablement intervention. The majority (83%) of participants made progress in every area and all participants progressed in at least one area. The Star readings suggested that improvements in some areas can increase after finishing the intervention. It was concluded that Spectrum Star is useful in helping people with autism to understand their condition, and in informing decisions about the outcomes individuals want to achieve. The authors recommend the Spectrum Star to others, and state that it provides ‘a fuller and more personalised assessment than any Standard Care Act 2014 template’.

Triangle conducted additional analyses on data from this intervention and found correlations between Star readings, the Rosenberg self-esteem scale and baseline costs to the service.

This study evaluated the acceptability and psychometric properties (convergent validity, test-retest reliability and inter-rater reliability) of the Italian version of the Mental Health Recovery Star (MHRS).Convergent validity was assessed by examining readings on MHRS scales and the relevant scales of other validated tools: the Health of the Nation Outcome Scales (HoNOS), World Health Organisation Quality of Life-BREF (WHOQoL-B) and Global Assessment of Functioning (GAF). Temporal stability was tested by 117 service users completing the MHRS two months apart. Finally, inter-rater reliability was examined by 42 keyworkers giving readings for a clinical case, which was described and discussed at the end of a training session.

Service users and keyworkers perceived the Recovery Star as acceptable and easy to use. Convergent validity was demonstrated and there was good test-retest reliability for all areas of the Star. Cohen's kappa coefficient was calculated as a measure of inter-rater reliability and all pairs reached acceptable levels of at least 0.70. The authors note that using a larger number of case studies would have made the finding more generalisable*.

* Placentino and colleagues refer to Killaspy et al. (2012), in which keyworkers assessed 85 service users, but since these readings were taken without collaboration with service users or using a written case study, we cannot know what information was being used by each keyworker and it is likely that they were missing important information upon which to base their readings. These and other limitations of Killapsy et al.’s method are discussed in letters published in the British Journal of Psychiatry (e.g. MacKeith, 2012).

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Acceptability
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Acceptability
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Acceptability
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Acceptability
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Acceptability
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Acceptability
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Acceptability
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Acceptability
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Acceptability
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness
· Inter-rater reliability

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Acceptability
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness
· Inter-rater reliability

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness
· Inter-rater reliability

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness
· Inter-rater reliability
Predictive/convergent validity

This factsheet shows the psychometric tests completed by Triangle for this version of the Outcomes Star. These tests include:
· Factor analysis
· Internal consistency
· Item redundancy
· Responsiveness
· Inter-rater reliability
Predictive/convergent validity

The Preparation Star has been developed to empower patients to share what is important to them
when approaching end of life.

The Parent and Baby Star covers the critical first 1001 days of life, from conception to age 2, during which the foundations are laid for each baby’s physical, emotional and social development and long-term life chances – promoting the earliest of early intervention and prevention for babies and infants.

The New Mum Star has been developed for women as they prepare to be mums and after their baby is born. It is primarily designed to be used with first time young mums but can be used more broadly.

Family Star (Relationships) has been developed for parents, and others with a key parenting role, who need support to tackle disagreements in a healthy way.

Family Star Plus – validity as an outcomes tool: Researchers at Queen’s University Belfast published an evaluation of the Early Intervention Support Service (EISS) in Northern Ireland (Winter, Neeson, Sweet & Connolly, 2018). This service uses several versions of the Outcomes Star, and the report confirms previous findings that practitioners and families are very positive about the tool. The evaluation reports a good level of change shown using the Star despite the relatively short duration of the intervention. However, we wrote a response to address serious concerns that readers will be misled by the way in which the Outcomes Star has been described.