closing the loop in evaluation of safety in remote nt communities using an ethical, robust and...
TRANSCRIPT
Closing the loop in Evaluation of safety
in remote NT communities
using an ethical, robust and participatory
approach
Department of Families, Housing, Community Services and Indigenous Affairs
NTER whole-of-government evaluation
Evaluation Principles:• ‘person centred’ approach• Light footprint• Long term assessment of
change• Learning partnership• Systems focus
Local Research Projects
• Provides a deeper understanding of the context where data is being collected
• Train and mentor local Indigenous people in research skills
• Develop and strengthen existing local research knowledge, skills and competencies
• Develop a pool of skilled participatory researchers (participatory panel)
Components of the NTER evaluation
Secondary
data analysis
Community
impact
studies –
Community Safety
and Wellbeing
Research Study
Monitoring Reports and
analysis
Evaluation of ' bundled measures’
NTER Revie
w
NTER Redesig
n
WoG Evaluation
Report
Evaluation of separate
measures
Other
intelligence
Top Down
NTE
R W
hole
of G
over
nmen
t
Eval
uatio
n re
port
GAPS
Bottom up
Service Provider Survey
Community Safety & Wellbeing Research Study
Coordination&
Engagement Study
Program evaluations Program admin data
Evaluation Design
Community Safety and Wellbeing Research Study
• Provided a voice for Indigenous Australians affected by the NTER
• Explored how the health, welfare and safety initiatives as part of the NTER impacted on
community members lives.
FaHCSIA team
• Kim Grey – psychology and sociology, mixed methods background
• Sue Sutton - ABS and survey experience, quantitative data collection and analysis
• Nathalie Baxter – International Development background including Participatory Rural Appraisal (PRA) techniques
Methodology
• Research was designed to be both consistent and flexible• Structured quantitative survey• Participatory approaches using methods that
suit individual communities
• Allowed triangulation of monitoring and evaluation data with the views of people directly affected by the NTER
Survey Questionnaire• Original survey developed by Australian Institute of
Criminology (AIC)
• Questions comparable with other research e.g. National Aboriginal and Torres Strait Islander Social Survey (NATSISS)
• Pretested by Experienced Aboriginal Researchers
• Piloted in one community
• Modified questions and language
• Rolled out in 16 other NTER communities
Communities
• 17 communities chosen to cover a cross section of NTER communities – size, location, police presence
• 1,323 people participated in the quantitative survey
• A stratified quota sample of community members aged 15 and over were surveyed based on the sex and age profiles of each community from the 2006
Census of Population and Housing.
Research Consultants
• Four consultants engaged through departmental panel of consultants with expertise in participatory research methods
• Consultants had prior relationships with communities and participatory research experience
• Most consultants had experienced Aboriginal researchers as part of the team
Consultancy team
• Bowchung Consulting
• Colmar Brunton Social Research
• N-Carta Group
• Dr Sithole and ARPNet
Participatory research cycle
Training local researchers
Ownership of report by community
Consent
Capturing ‘community voice’
Giving report back
Employment opportunities e.g. ABS census
Community builds on research e.g. community safety planning
Ethics Board
Consent
• Research did not go ahead until there was signed consent from each community
• Differed from community to community
• Appropriate time should be allocated in all projects to do this right
Training local researchers
• Consultants engaged and trained local researchers
• Over 60 Indigenous researchers were employed
• Around 10 experienced Indigenous Researchers
• 55 local researchers from the participating communities
Capturing the community voice – Quantitative Survey
Local Researchers played a number of roles: • Project leaders • Interpreters• ‘Brokers’ to encourage people to participate• Conducted surveys and interviews• Organised groups and conducted
participatory research• Entered data
Capturing the community voice – participatory qualitative research
• Ranking of changes and challenges
• Different methods were used to achieve the same outcome
• Changes and challenges were identified from the responses/comments received while conducting the survey
• And/or from discussion groups
• Ranking was conducted by individuals or groups
Difficulties working in remote communities
Issues that researchers had to work around included:• Sorry business• Court dates• Meetings in communities• Ceremonies• Sporting events• Gambling• Pay days/Grog Days• School holidays – leaving community• Apathy about being over-researched• Mistrust of Government
Difficulties working in remote communities
• Difficulty in getting to communities• Cheeky dogs• Persistent rain / heatEngaging local researchers• Local researchers hard to attract if they have
other income sources e.g. mining royalties• Lack of understanding of importance of
research• Feel they don’t have the right to ask people
questions• Low literacy levels within the community
Researchers working in their own communities presents challenges
• Working across difficult family/community relationships
• Getting caught up in family/community related issues
• Researchers knowing something about the respondent
• Living situation can impact on energy levels of researchers
• Home environment (project material gets lost or is destroyed) and
• Issues of “jealousing” from members of the community
Reporting back to communities
• Results of the survey and participatory comments written up in a ‘Community Analysis Report’
• The report is jointly owned by FaHCSIA and the community
• Community can use the report for future planning
Handing report back
• All consultants went back to communities to present the report to them
• Differed from community to community• Formal meetings with local reference group• Community BBQ• Meetings with major stakeholders• Separate presentations with groups e.g.
women’s group, men’s group etc. • Walk through the community and discuss
with community members
“Getting Balanda people from outside might feel hard to express our feelings to them, countryman feel comfortable and relaxed with our own people”
Comment from community member about using Local Researchers….
• “It’s good to have u mob come talk, its people’s point of view, how they feel before and now, yes I do, yes I did and it help us understand what’s bad and what’s good”
Comment from community member about the research…
Comment from research team member…
“As we were doing our work some people were interested in working with us and getting involved. They said it was good we were doing the work”
Feedback from Communities
"One of the best research projects ever done for us...the way the guys went about it, the way the guys engaged with the community and Yalu and got them involved and the timeliness of reporting back."
Thank you