AES SEMINAR 2019 – AMPLIFY A CASE STUDY

WESTIR Limited attended an Australasian Evaluation Society’s (AES) seminar exploring how outcomes and impact can be measured validly and reliably. The seminar focused on the Amplify Social Impact Initiative which is being driven by the Centre for Social Impact (CSI).

The seminar began with Lena Etuk, CSI’s Amplify Project Manager at the University of New South Wales (UNSW), outlining the current state of social impact in Australia:

• Approximately, $510 billion is spent annually on social purpose programs, with many funders asking programs to show evidence and demonstrate accountability.
• There is an increasing demand for programs to measure outcomes rather than outputs, however, the majority do not have the resources to undertake this measurement correctly.
• There is no mechanism to benchmark and invest in programs that work, while social disadvantage persists for many groups.

It is in this context that the Amplify Social Impact Initiative is being developed. Amplify is a suite of online resources and tools which are designed to support organisations to improve their evidence-based decision-making, program evaluation and, ultimately, their social impact. The Amplify online platform is working towards having the following features:

• Australia’s Social Pulse: a rigorous and ongoing measurement of the nation’s social progress to show where social problems are most acute and amongst which populations.
• Social Progress Index: an index exploring whether we are meeting the basic needs of people and whether we have the foundations of wellbeing.
• Indicator Engine: a database of over 700 indicators which will help organisations find the right indicators to measure their impact.
• Yardstick: a programs, outcomes and impact database, which will benchmark programs across the country, providing opportunities for replication and improvement.

Amplify’s online tools will be rolled out from January 2020. The initiative also incorporates a set of research reports (called Amplify Insights) and a series of collaboration projects (such as the Constellation Project) that aim to improve social outcomes in five areas: housing and homelessness, education, work, social inclusion and financial wellbeing. It has received $6.2 million funding to date, mainly coming from UNSW, PwC and the QBE Foundation.

The second part of the seminar focused on small group discussions around the challenges of the Amplify Project, specifically issues around indicators and benchmarking. Some issues that were outlined include:

• The need for new online tools to work with existing outcome frameworks so that frontline staff don’t have to waste time re-entering their data into a different platform.
• The use of existing tools that are not validated but are still widely used across social purpose programs.
• The inherent difficulties with measuring change over time.
• The difficulty of picking appropriate benchmarks as all organisations are different and have different caseloads. Benchmarking is also particularly difficult for populations such as the Aboriginal and Torres Strait Islander community where they need to be compared against Aboriginal programs or populations rather than the overall benchmark.
•  The pros and cons of making the results public. Some saw the benefits of publicly releasing the data to learn from other best practice examples, while others thought that it would naturally favour large organisations, with money and resources to undertake more rigorous measurement practices.

Overall, the session highlighted the complexity of measuring outcomes and impact and the need to consider the principles of validity, reliability and rigour in this process. For more information about the Amplify Social Impact Initiative, please visit its website.