Equality in Evaluation

It is an exciting time to be part of the world of measurement and evaluation. Having attended three conferences this autumn, it is clear that those with a critique of the traditional ways of doing things are finding a voice, and being given a platform. In the wake of Black Lives Matter everyone seems more open to looking deeper into the implicit assumptions that we make about each other, and along with that, into the power dynamics of measurement and evaluation. 

NPC ignites was one of these events and it was the session “Rebalancing data for the 21st century” that really captured my attention. Jara Dean Coffey, Director of the Equitable Evaluation Initiative presented a five-year plan she is leading to change the way funders in the United States think about evaluation. Bonnie Chui of The Social Investment Consultancy is leading an initiative bringing together people of colour working in evaluation. Here were some of their key messages:

Co-create knowledge rather than extract data

Traditional approaches to the evaluation involve experts collecting data and taking it away to analyse and draw conclusions. The subjects of the evaluation are passive in the process. Bonnie described this as like using research as tool of ‘command and control’. Jara argued, like several others I have heard this year, that we learn more when knowledge is co-created – researcher and subject bringing together their very different expertise to build a more complete and informed picture. This is one way to challenge the power relationships in evaluation and promote greater equity. The Outcomes Star’s collaborative approach to measurement brings these ideas alive in day-to-day service delivery. 

What is the Outcomes Star

The Star is underpinned by three values – empowerment, collaboration and integration

Get comfortable with complexity

“We need to let go of causality and be OK with contribution”

Star Data

The Star collects an innovative and holistic dataset

Jara made the case that although funders who commission evaluations want certainty and yes/no answers, the complex reality of service provision can’t be reduced to a few numbers.  Funders and evaluators need to embrace the complexity that comes from working in open systems where it isn’t possible to control all the variables and come up with answers that are always true no matter what the context. Bonnie also made the point that top down funder-driven monitoring and evaluation frameworks can perpetuate power imbalances. It is difficult for funded organisations to raise these issues because of their dependence on the funders so it is important that evaluators use their influence. This very much echoes points we have been raising at Triangle for some time. Data is helpful but must be interpreted in context. The numbers help to focus our questions rather than providing definitive yes/no answers. 

De-colonise evidence

Bonnie Chui argued that we need to ‘decolonise’ evidence and ensure that people of colour are both reached by research and represented in the research and evaluation community.   Jara is promoting multi-cultural validity alongside statistical validity, a point which chimes with issues Triangle has raised about moving beyond traditional formulations of what is a ‘good’ tool (keep an eye on our homepage for a blog on this coming out soon).

Both presenters made the case that evaluation is a human process. Those doing the evaluation have to do their own personal work to understand their own implicit biases as well as those that are hardwired into the context in which they are working. The biases identified were racial ones as well as foundational ideas such as the preference for doing over being and our belief in scarcity rather than abundance.

I found it very inspiring to hear an analysis connecting up racism, core orientations towards life and the way services are valued and measured. I can’t do it all justice here, so I recommend that you take a look at the recording of the session.

***

Triangle is the social enterprise behind the Outcomes Star™. Triangle exists to help service providers transform lives by creating engaging tools and promoting enabling approaches. To talk to Joy MacKeith or another member of the Triangle team, or for any other information, please email info@triangleconsulting.co.uk.

The Refugee Council reaches for the Star

A major new tool to help refugees settle successfully in the UK is to be made available from this week – the Integration Star.

Created by Triangle, the social enterprise behind the Outcomes Star, in collaboration with the Refugee Council and others, the Integration Star focuses on eight key areas on which successful integration depends.

Triangle director Sara Burns said: “The Integration Star primarily focuses on the means and markers of successful integration, from employment and housing to education and health. It highlights refugees’ strengths, recognises the areas they need more information or support, and records their progress. Most importantly it makes them the centre of their own integration process.”

In a 7-8-month pilot of the new Star, caseworkers from a variety of organisations found that it opened up conversations with refugees on important subjects and gave workers a way in to providing crucial support.

“It really encouraged the client to have more autonomy in the conversation – it brought them to the centre of the discussion”

Case Worker

“The Star provoked conversations we wouldn’t normally have – like social isolation issues”

Case Worker
Man, woman and child walking down the road

Getting to the heart of these issues means refugees can integrate faster and more successfully,.which will help them to rebuild their lives in their new home.

The Refugee Council approached Triangle in 2018 to develop a new version of the Outcomes Star that would support the refugees they work with. They and several community refugee organisations worked closely with Triangle throughout the process, making sure that former refugees were involved all the way along.

Refugee Council Logo

“This is such an important piece of work that feeds into the national strategy for integrating refugees. The Integration Star has already proved itself in a really rigorous pilot – now it can help in supporting refugees throughout the UK to build new lives.”

Sara Burns

"The Star gives organisations that work with refugees an extremely useful tool that can help to shape the support they offer as well as measure progress and impact. Successful refugee integration means that people are enabled to rebuild their lives and become active members of their new communities”

Refugee Council's Head of Integration Andrew Lawton

The Integration Star was published in September 2020 after 18 months of development and launched in October 2020 via the Star Online – the online home of the Outcomes Star.

In addition to the materials created to use the Integration Star, Triangle is producing a range of resources to support its launch, including a webinar to share more information on the development process and pilot.

****

Triangle Consulting Social Enterprise is an innovative, mission-led organisation that exists to help people reach their highest potential and live meaningful and fulfilling lives, often in the context of social disadvantage, trauma, disability or illness.

The Refugee Council works with individuals and families to make sure they can live safe, fulfilling lives in the UK after being forced to seek refuge from persecution and human rights abuses overseas.

The development of this star was part funded by the EU Asylum, Migration and Integration Fund.

Other organisations taking part in the development of the Integration Star included: RETAS (Refugee Education Training Advice Service) Leeds, Leeds Refugee Forum, Path Yorkshire, Goodwin Development Trust, Humber Community Advice Services (H-CAS), Refugee Action Kingston, Iranian Association, Diversity Living Services, Bahar Women’s Association, Action for Community Development, West Yorkshire Somali Association, DAMASQ, Stepping Stone 4, Leeds Swahili Community.

To participate in the Triangle webinar on the development and application of the Integration Star, organisations should contact info@triangleconsulting.co.uk to register their interest.

Triangle developed and published a separate Star for unaccompanied asylum-seeking children in 2019 – the Planning Star.

Please contact info@triangleconsulting.co.uk in the first instance to discuss the Integration Star, Planning Star or other aspects of Triangle’s work.