Evaluate and share outcomes

Overview

Local governments should evaluate the effectiveness of their efforts against initial measures and benchmarks. It is good practice to evaluate not just the outputs (what you did) but also the outcomes (what change occurred). Evaluation helps local governments maximise public value by allocating resources to programs and activities which support effective outcomes.

Given that building social cohesion is a long-term goal and process, local governments should collect and collate data over time to continually monitor progress. This section covers: 

A review of international literature of interventions to address locational disadvantage found:

  • Multi-level government policies and interventions should be designed to produce positive outcomes at the neighbourhood level
  • Mechanisms should be both people-based and place-based
  • Interventions should be conducted at the macro and micro levels
  • The aim should be for genuine community empowerment and involvement 
  • Interventions require partnerships between the public, private and community sectors.

Elements to avoid include:

  • Tokenism in forming partnerships and building community involvement
  • Short-term quick fixes
  • Discourses which entrench the problem by identifying an area as dysfunctional
  • Investing too quickly and beyond the capacity of the community to participate fully
  • Interventions which merely displace the problem.

Source: Ware, V-A., Gronda, H. & Vitis, L. (2010). Addressing locational disadvantage effectively: Research Synthesis.

What evaluation is

Evaluation is a systematic process of collecting data and using it to make judgements about the appropriateness, effectiveness and efficiency of an initiative. Evaluation should occur in partnership with internal and external stakeholders, especially where programs aim to change behaviours and attitudes.

To assess the outcomes of change, think about what information sources can provide data at the start (in order to establish a baseline measure), during the project and at the end of the project (to measure progress), such as:

  • Community surveys (about attitudes towards different cultures, experiences of racism)
  • Audits of policies and procedures
  • Focus groups with specific communities or organisations
  • Key informant interviews with community leaders.

Evaluation occurs at various stages in the implementation of an initiative, for example:

  • In the planning stage to ensure the desired initiative is appropriate and will address the issue
  • When taking action to strengthen organisational culture and community capacity when establishing partnerships to ensure that the right people and organisations have been engaged 
  • During the implementation of new policies or programs to ensure they are on track and achieving interim results
  • After implementation to reflect on lessons learned and outcomes achieved.

Other resources about evaluation include:

Does Your Project Make a Difference?: A resource from the then NSW Department of Environment and Conservation (now Office of Environment and Heritage). NSW Department of Environment and Conservation (2004). Does Your Project Make a Difference? A guide to evaluation environmental projects and programs.

VicHealth evaluation framework and performance measures: VicHealth (2014). Localities Embracing and Accepting Diversity (LEAD) program: summary report. LEAD program evaluation logic. p.30. 

Queensland Government evaluation frameworks: Queensland Government (2011). Developing an evaluation framework.

Why local governments should evaluate

Investing in building social cohesion over the long term can be resource intensive so it is vital to evaluate whether the investment is achieving the intended outcomes. In addition, because evaluation is a participative process, it can help build strong partnerships with the community. 

Evaluation is also important for the reasons outlined below.

Evaluation is important for: Example questions to consider
Measuring change and success (or otherwise) Did the activities undertaken lead to the expected outcomes?
Were we able to track progress and adjust activities if necessary?
Did reporting help accountability with our stakeholders?
Did we quantify the benefits and costs (tangible and intangible)?
Continuous improvement How could processes be improved?
What could be done differently?
Were there any barriers to implementation?
How can these results be communicated to our stakeholders?
Providing evidence which builds a shared knowledge base of good practice What worked and what didn’t work?
What data needs to be collected to make a judgment about success?
What can we usefully share with others?
Building strong partnerships and networks Were strong partnerships and networks built?
Can these be reused proactively and reactively over time?
If so, were they built with key influencers?
How will they benefit social cohesion in the future?

Since 2010, Brimbank City Council, an outer suburban local government area, has designed an interactive cross-council program for CALD communities, in particular new and emerging communities. The program aims to create a welcoming environment that encourages participation in civic and community life. It creates awareness of council services and increases settling communities' understanding of the Australian government system. The program includes bus tours, interactive presentations and handouts. It assists in supporting culturally and linguistically diverse communities to understand council governance structures and to physically locate and familiarise themselves with key council services.

Council officers worked in collaboration with language teachers to develop this program to ensure that Council facilitators present material which aligns with classroom learning competencies based on the participants' formal English classrooms. The program also enriches the learning programs of the language training providers by using practical day-to-day living information for participants enrolled in the language programs. 

Formal feedback from the participants has shown that there is much support for the program. Participants stated that: 

  • They learnt about council services that they did not know about previously
  • The practical demonstrations helped them to understand the required processes
  • The guided tours increased the confidence of the participants returning to the venues in the future
  • Positive relationships were formed between staff and residents. 

Source: Personal communication with Community Planning & Engagement Officer, Brimbank City Council (2014).

Key elements of an evaluation framework

A strong evaluation framework is one which:

  • Is developed through a participatory process
  • Describes the purpose of the evaluation and its intended use
  • Describes the methodological approach of the evaluation including: 
    • The evaluation questions
    • The performance criteria and indicators
    • The type of data to be collected
    • How data will be collected, analysed and interpreted
    • Who will be involved in the evaluation process
  • Includes a clear timetable, starting at the planning stage
  • Includes performance criteria which reflect the agreed outcomes
  • Uses data collection methods that are appropriate to the evaluation objectives and the participants
  • Includes a plan for sharing and using the learnings from the evaluation.

The City of Stirling ran a CALD Youth Sport and Recreation Project from 2006-2009. The project aimed to engage young people, particularly those from African and Aboriginal backgrounds, in organised sport, recreation and leisure activities. It was based at a council-run recreation centre at Mirrabooka and staffed by a full-time Special Project Officer. The Department of Sport and Recreation, the Office of Multicultural Interests in Western Australia and the council funded this position.

Some of the objectives of this program were to:

  • Enable sporting and community groups the opportunity to showcase their club and recruit new members through a wider range of community events and activities 
  • Provide training to build inclusive participation in sport and recreational activities in sporting clubs 
  • Increase opportunities to collaborate with peak ethnic community organisations 
  • Provide additional support to assist with engaging multicultural youth into your sporting club 
  • Deliver coaching workshops for multicultural youth 
  • Create new multicultural youth participation programs through sport and recreational activities 
  • Provide education and training around engaging CALD youth into sport, recreation and leisure activities 
  • Drive club development education specifically for CALD youth 
  • Deliver coaching and officiating courses for target groups to be role models. 

The project was evaluated by the Social Justice Research Centre at Edith Cowan University. Baseline data was collated at the start of the program via:

  • A questionnaire sent to all sporting clubs
  • Interviews with key stakeholders (government agencies, NGOs and ethnic organisations)
  • An audit of record keeping processes for attendance and participation.

During the project data was collected via:

  • Re-interviewing the NGO and ethnic organisations and government agencies
  • Interviewing a sample of the target group utilising the recreation centre
  • Interviewing a sample of the target group in leadership roles
  • Interviewing a sample of sport and recreation clubs.

However, the evaluation did come across problems such as:

  • Low response rate from clubs to the questionnaire (only 27 out of 151 responded)
  • NGOs, ethnic organisations and government agencies were hard to engage
  • Few response levels from the target group in leadership roles.

The evaluators concluded that may be problems in evaluating refugee settlement programs due to:

  • Conflict between NGOs and government agencies
  • A lack of interest in outside research from refugees
  • Significant difficulties in making contact with stakeholders
  • Language barriers.

The evaluation found that whilst the project achieved increased participation in sport, recreation and leisure amongst the targeted groups, barriers to participation remained, especially for young women. Barriers for women included lack of childcare, few gender-specific activities and cost.

Sources: Dandy, J. & Pe-Pua, R. (2013). Research into the Current and Emerging Drivers for Social Cohesion, Social Division and Conflict in Multicultural Australia. pp.62-63. Report prepared for: Joint Commonwealth, State and Territory Research Advisory Committee

Hancock, P., Cooper, T., & Bahn, S. (2009). 'Evaluation of a Youth CALD (Cultural and Linguistically Diverse) Sports Program in Western Australia: Resettling refugees using sport as a conduit to integration'. Tamara: Journal of Critical Postmodern Organization Science, 8(8.2), 159-172.

Sharing outcomes

Evaluation can help local governments understand how to adjust or modify future projects and programs and can be a way to share valuable lessons with others to avoid 'reinventing the wheel'. Key lessons learned through evaluation are:

  • Change is a long-term process
  • Resilience is needed when things do not go to plan and adjustments are required 
  • A strengths based approach is important
  • Multiple strategies across multiple levels can deliver more positive outcomes
  • It is important to know your community and engage with the right people.

Key findings from a review of anti-racism strategies suggest that these strategies should:

  • Seek to eliminate false beliefs by providing accurate information
  • Avoid one-way communication as people are unlikely to engage with the topic of anti-racism if they are not given the opportunity to contribute their views
  • Aim to provide the practical skills to empower people to speak out against racism
  • Invoke empathy for others
  • Incorporate longitudinal strategies. initially emphasise similarities amongst rather than differences between groups but subsequently emphasise diversity and plurality
  • Focus on changing racist behaviours and actions, rather than on changing racist attitudes and/or beliefs, which are remarkably resistant to change 
  • Offer practical solutions to changing behaviour
  • Recognise that ethnic groups are not homogenous
  • Have the support of clear unambiguous political leadership
  • Draw from a coalition of leaders from academia, sport, police, public life etc. who are committed to delivering a consistent message
  • Be supported by sustained and substantial funding 
  • Be supported by collaboration across agencies and full consultation/involvement of ethnic communities
  • Develop long-term plans rather than one-off interventions as change takes time. 

Source: Centre for Social Change & Social Equity Murdoch University for the Western Australia Office of Multicultural Interests (2003). Anti-Racism – What Works? An evaluation of the effectiveness of anti-racism strategies.

The Building Bridges program was developed in 2006 to add value to community activities which bring people from many different cultural backgrounds together such as festivals, food and artistic and cultural pursuits. 

Findings from an evaluation of 203 projects involving over 90,000 participants showed these types of activities should be underpinned by:

  • Contact around a collaborative activity where all participants are working towards a common goal
  • Specific opportunities for friendships to be built between people from different groups
  • Participants being engaged on equal terms
  • Contact supported by relevant authorities within a safe and inclusive environment.

Source: Victorian Health Promotion Foundation (2013). Building bridges to wellbeing and cultural diversity: Key evaluation findings of Phase Two of the Building Bridges program.

Recommendations from the evaluation of the LEAD program include: 

  • Work with local government as a central partner
  • Assess community experiences of race-based discrimination, including places where race-based discrimination occurs, before implementing interventions so that actions can be targeted appropriately
  • Assess the numbers of suitable target organisations in each setting to ascertain the appropriateness of the place-based approach
  • Employ a program coordinator supported by senior management
  • Target organisations with experience in working with cultural diversity
  • Phase implementation in order to harness new resources at each stage of the intervention
  • Maximise the level of intervention within organisations
  • Evaluate the results. 

Source: VicHealth (2014). Localities Embracing and Accepting Diversity (LEAD) program: Summary report. p.28.

An evaluation of success factors to enhance social cohesion in Mirrabooka and Balga in the City of Stirling found:

  • Programs related to arts, sport, employment, English language and community capacity (amongst which there was considerable overlap) are key for social cohesion
  • Activities and events relating to food and music were the most successful in bringing people in the community together. Accessible public spaces were identified as contributing to the affordability and non-exclusivity of such activities
  • There was a need for further education and awareness, specifically amongst newer migrants and refugees, of Aboriginal Australians' cultures and history in recognition that many intergroup tensions stemmed from a lack of understanding and ignorance
  • Political leadership at the local, state and federal levels, as well as strong and effective leadership of minority ethnic groups, was recognised as a significant contributor toward social cohesion, particularly in the dimensions of participation and recognition
  • Short-term and piecemeal approaches to program funding were recognised as an important challenge, especially because it was difficult to secure longer-term funding once the pilot program was complete.

Source: Dandy, J. & Pe-Pau, R. (2013). Social cohesion, social division and conflict in Mirrabooka and Balga (Western Australia).

  • Facebook logo
  • LinkedIn logo
  • Twitter logo