Evaluating a citizen science research

programme:

Understanding the people who make it possible.Dale Wright

MSc Conservation Biology

Supervisors: Prof. Les Underhill,

Dr. Andrew Knight, Mr. Matt Keane.

And now for something completely

different!

Environmental Evaluation

Citizen science

Psychometric research

Background

A relatively old enterprise which has been increasing recently.

Research technique which enlists the public in gathering scientific information.

Leads to the formation of a coordinated network of volunteers- carrying out various tasks.

Allows for integrating public outreach and scientific data collection over variable geographic scales.

Multiple scientific & educational outcomes.

1)Identifying the programme and evaluation needs.

2)Designing a conceptual model of the programme.

3)Outlining evaluation questions

4)Designing instruments for data collection.

5)Collecting & analysing data

6)Disseminating findings

SABAP 2: How successful is it?

Logic model of SABAP 2Research objectives

Stakeholder questionnaire, Focus groups, Atlaser questionnaire.

Evaluation Procedure

Research objectives

1)Design a logic model for SABAP 2.2)Design an instrument to assess

different psychological constructs of the volunteers

•Describe major motivations of atlasers & their level of satisfaction.•How do these link with level of participation?•“Ambassadors for biodiversity”? •Assessing the level of burnout.•Understanding the constraints on atlasers

Methodology

Evaluation:

Held 1 hr long focus groups with ADU management

12 item stakeholder questionnaire.

Elicit information for logic model.

Understand how prog. Management relates to the atlasers.

75 item online questionnaire:

5 sections:

•Introductory- 10 items•Motivations & Satisfaction-27 items•Ambassador potential- 10 items•Burnout-13 items•Demographic questions

Used Tests of internal consistency, Factor analysis, Correspondence analysis & Simple summary stats.

Southern African Bird Atlas Project 2

Dale Wright 

Welcome to the SABAP 2 atlaser questionnaire. The following questionnaire has been designed to gather important information which will help us to improve your experiences as an atlaser with SABAP 2.

Throughout the questionnaire SABAP 2 may also be referred to as 'the programme'. Many of the questions follow an easy to answer format in which you estimate your level of agreement with certain statements by simply checking the appropriate box. Thus after the initial open ended questions the survey will move much quicker. These question types allow us to gather important information whilst reducing the time you spend answering.

All answers to this survey are completely anonymous. Although we will ask for your observer number, this is done so that our database can retrieve information about participation without you having to calculate this information. Your observer number will then be deleted and excluded from the data which we capture. Our analysis of this information will also occur at the collective level. So even though we are hoping to improve the atlasing experience for each of our individual atlasers, there are no individual analyses undertaken and all your comments will remain anonymous.

This survey should take no more than 15 minutes of your time, and we hope that it is interesting and enjoyable.

Thank you in advance for supplying this critical information.  

Top of FormPlease take the survey    Go!

Bottom of Form

4 of 19

Please indicate your level of agreement with the following items, or how accurately each item represents you.

Next

I feel it is important for people to volunteer their time to a worthy cause.

 Strongly disagree    Disagree    Neutral    Agree    Strongly agree    Unsure  

I enjoy being in the outdoors.

 Strongly disagree    Disagree    Neutral    Agree    Strongly agree    Unsure  

I want to learn more about birds.

 Strongly disagree    Disagree    Neutral    Agree    Strongly agree    Unsure  

Developing the Environmental Volunteer Functions Inventory (EVFI)

Inventory Sub-motives include:

Recreational/ nature basedValuesPersonal learningSocialProject organization

Also an index for assessing satisfaction- useful for other programmes and investigations.

SABAP 2 Atlasers are Highly satisfied with the programme!

Atlaser questionnaire outcomes

Sub-motive & Items Factor 1 scores

Factor 2 scores Cronbachs Alpha

Mean score

Enhancement 0.74 4.02Q13 0.511Q23 0.795Q24 0.861Q25 0.504Q31 0.776Q41 0.641Q43 0.813

Values 0.66 4.47Q11 0.730Q14 0.760Q16 0.691Q18 0.750Q36 0.813Q40 0.725

Social 0.75 3.64Q19 -0.512*Q21 -0.723Q22 -0.727Q26 -0.809Q42 -0.782

Recreation/Nature 0.76 4.46Q12 0.436**Q15 0.682Q17 0.770Q20 0.772Q28 0.632Q29 0.600Q38 0.687Q45 0.650

Project organization/ Career***

0.71 3.78

Q27 -0.662Q30 -0.702Q32 -0.755Q44 -0.751Q46 -0.572

I feel it is important for people to volunteer their time to a worthy cause.

2D P lot of Colum n Coordinates ; Dim ens ion: 1 x 2

Input Table (Rows x Colum ns ): 217 x 18

S tandardiz ation: Row and c olum n profiles

Q 47+

Q 48+

Q 49+

Q 50+

Q 51+

Q 52+

Q 53+

Q 54+

Q 55+

Q 47-

Q 48-

Q 49-

Q 50-

Q 51-

Q 52-

Q 53-

Q 54-

Q 55-

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4

Dim ens ion 1; E igenvalue: .03673 (36.73% of Inert ia)

-0.5

-0.4

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

Dim

ensi

on 2

; E

igen

valu

e: .

0165

7 (1

6.58

% o

fIn

ertia

)

Q 47+

Q 48+

Q 49+

Q 50+

Q 51+

Q 52+

Q 53+

Q 54+

Q 55+

Q 47-

Q 48-

Q 49-

Q 50-

Q 51-

Q 52-

Q 53-

Q 54-

Q 55-

Further outcomes from atlaser questionnaireWhat are the demographics of our atlaser group:

How they heard about the programme?

Who do they see as champions?

What are the constraints limiting participation?

How would THEY improve the programme?

Assessing levels of burnout in the programme.

Correlations with participation levels

Notification typeNumber of

respondentsPercentage of total

(n=283)

Word of mouth 63 22.26%

Birdlife SA 46 16.25%

Internet & email 44 15.55%

Local bird clubs 39 13.78%

Via SABAP 1 29 10.25%

ADU 28 9.89%

Birds & Birding magazine 14 4.95%

Other media 11 3.89%

Unsure 9 3.18%

0102030405060708090

100

Nu

mb

er

of

men

tion

s

Programme functioning in relation to the

atlasersUnderstanding the needs which drive the goals of the programme.

Successful at achieving these goals?

Mobilizing citizen scientists?

Self-sustaining social networks.

A complete logic model.

Multiple outcomes for the participants and the programme.

Online technology, updated distribution maps for birding

IT resources: Software, website and DBMS, Hardware & servers, Online discussion forums

Participants: active/ inactive

Human resources:

ADU management team

SteerCo & RACs

BLSA Birdclub network

Atlasers

Financial: Donor/ Govt funding; Travel subsidies?

Workshops: Training courses; Bird ID courses

IT admin: Automated replies, Support for atlasing

Operational admin: Feedback with atlasers, Facilitating interactions, Mentoring

Field surveys: Individuals, Group outings, Small challenge projects

Formal PR: Newsletters, Radio & Public talks, Published articles, SABAP2 website.

Informal PR: Online blogs, ‘word of mouth’,

Website & associated resources

Database: Checklists

Resources Activities Outputs

Birding mindset changes to atlasing mindset

Meeting fellow birders, social networking & learning amongst atlasers.

Skills developmentIncreasing knowledge of birds/birding areas/routes

Feeling of contribution to research & ownership thereof.

Holistic citizen scientists

Submitting data

Ambassador for biodiversity & programme.

Scientific & other publications from data

Access to information

Online technology, updated distribution maps for birding

Participants: active/ inactive

Mechanism for people to contribute to research

Website & associated resources

Database: Checklists

Outputs Outcomes

Short-term Mid-term Long-term

A few interesting outcomes:

•Citizens are the cornerstone of the research- shift in focus of projects & research is essential.

•Providing an adaptive framework for management.

•Bridging the research-implementation gap.

•Practicing evidence-based conservation.

Two pronged approach to conservation=

Generating environmental awareness and stewardship,Collecting the biological data necessary to understand changes.

Transdisciplinary conservation science?

Citizen science

Biological Data collection

Information systems technologySocietal engagement & Environmental/ Scientific education

Current study

Evaluation science

Psychometric research

Aspects of conservation biology

Acknowledgements

•Supervisors for feedback and guidance.

•Percy FitzPatrick Institute’s CoE for funding this research & a CB CoE bursary.

•Animal Demography Unit’s Michael Brooks, Sue Kuyper , Doug Harebottle & Les for providing input and the project which could be evaluated.

•All members of SABAP 2, from RACs to SteerCo and especially the atlasers for contributing the data!

•My class mates for such a stimulating & interesting year!