SIMNA NSW IMPACT MEASUREMENT BREAKFAST EVENT 2018

WESTIR Limited attended a breakfast event held by SIMNA NSW (Social Impact Measurement Network Australia) at the Urbis Sydney offices in July 2018. The event was called ‘A 360 degree view of collecting outcomes data: reflections from impact measurement specialists and their frontline colleagues’

The event showcased two case studies from Mission Australia and The Salvation Army on how they are implementing impact measurement in their programs and organisations. The Mission Australia presentation was presented by Maria Berry (Common Ground Program Manager, Mission Australia) and Rachel Christie (Impact Measurement Specialist, Mission Australia) and The Salvation Army presentation was presented by Garima Misra (Research Analyst, The Salvation Army).

MISSION AUSTRALIA PRESENTATION: IMPACT MEASUREMENT AND THE COMMON GROUND PROGRAM

The Mission Australia presentation focused on the implementation of an impact measurement (IM) framework for the Common Ground program in Sydney, as well as other sites around Australia. Common Ground is an inner city service run by Mission Australia that uses a Street to Home model to house inner city homeless and provide individualised, tailored support to their clients. The aim of the initiative is to establish a national approach to impact measurement starting with grassroot grounding.

Mission Australia’s IM framework was developed using Social Venture Australia’s Golden Thread Program Logic and Deakin University’s Personal Wellbeing Index (PWI). Surveys are administered to clients when entering the Mission Australia service, during the program and once they have left the Mission Australia service. Mission Australia uses the Microsoft Power BI software platform to feed survey results back to frontline staff. There is a mechanism in the software that flags low scores on PWI questions, which can be immediately fed back to front line staff for immediate attention.

Recent analysis on the survey data has shown that Mission Australia’s work is making a statistically significant impact on all the PWI domains of their clients. Regression analysis was used to determine what domains are most likely to drive overall client wellbeing for Mission Australia Children and Families Services. Their findings indicate that the PWI domains with the most impact are standard of living, achieving in life and personal health. The aim moving forward is to integrate the IM framework across the organisation to improve service delivery and eventually contribute to social change.

Some critical lessons from the Mission Australia experience included:

• The need for leadership buy in and a culture of accountability
• Alignment of collected data to minimum datasets (e.g. FACS is increasingly using PWI)
• Flexibility
• Consistency in language use and definition
• The need to share data and insights
• The use of technology, automation and IT solutions.

THE SALVATION ARMY PRESENTATION: MONEYCARE OUTCOMES MEASUREMENT PILOT

The Salvation Army presentation focused on the Moneycare program, which is a program run by The Salvation Army to improve the financial resilience of clients through a range of short, medium and long term interventions such as financial counselling, financial capability work and no interest loans.

The outcomes measurement pilot was a 3 month pilot study. The pilot was undertaken across New South Wales, Australian Capital Territory and Queensland with 67 Moneycare workers, 39 centres and over 200 clients.

The guiding outcomes framework used was ‘The 3Ps for Achieving Impact’ through the Centre for Social Impact (purpose, process and progress). The implementation of the framework included several activities such as review of internal documents and existing datasets, review of funding requirements, site observations, front line staff conversations, outcomes prioritisation, pilot implementation and collective learning, and a continuous improvement/feedback loop.

Some important insights from The Salvation Army pilot experience included:

• The need to co-design the pilot with frontline staff: this opens the gateways for innovation and ensures that data collected is useful and relevant.
• The need for the pilot to be integrated and holistic: the recognition that we can’t work in silos to address client wellbeing in its entirety.
• The need to share data and findings with frontline staff: the pilot’s reports are accessible to front line staff, so they can see in real time how their clients are progressing between the 3 assessments (beginning, middle and end of program). The interaction is also important for researchers to clarify results that they themselves can’t explain.

The Moneycare outcomes measurement pilot will be assessed over time to gauge the long term sustainability and effectiveness of the program and highlight where more intensive support and resources are needed.

INTERESTED IN SIMILAR EVENTS?

SIMNA NSW runs a variety of social impact measurement events throughout the year. Visit www.simna.com.au for more information.