collaboration and evaluation morgan braganza, m.s.w., ph.d. student rethink research group

Collaboration and Evaluation

Morgan Braganza, M.S.W., Ph.D. Student

ReThink Research Group

Who Am I?

Collaboration

Enter Collaboration

“Collaboration shows promise for solving organizational and societal problems” (Gray & Wood, 1991, p. 3-4)

One “strategy” for building community capacity is “fostering collaborative relations” (Chaskin, 2001, p. 299)

Build community capacity = build relationships

Enter Collaboration Social services provide support to those in need

Agencies often work in silos (Braganza, 2009)

Individuals fall through the cracks Too challenging to navigate the system (Leviten-Reid, 2007)

Comprehensive community initiatives bring agencies together Coordinate services Inter-organizational collaboration Community member participation (Chaskin, 2001)

Collaboration is important in improving community building and improving society

“All sectors are responsible for addressing these problems, preferably through a planned and coordinated approach that combines resources and expertise in new and sustainable ways” (Torjman, 1998, p. 25)

Why Collaboration?

Working collaboratively “just makes sense” for addressing community issues It raises awareness of other agencies It brings attention to existing services and gaps in service Results in finding new and creative ways to develop high quality

sustainable services Better use of scarce resources Combined efforts to address issues Combat larger (rather than smaller) issues Minimize the duplication of services Shared knowledge and information exchange The development of best practices

But…What IS Collaboration?

According to the Literature…

a way of bringing together agencies to address social issues

to design/offer programs to address these complex issues

exchange information which has the potential for action in the interests of a common purpose

“…when a group of autonomous stakeholders of a problem domain engage in an interactive process, using shared rules, norms, and structures, to act or decide to act on issues related to that domain'”

true collaboration involves being willing to address issues using methods different than would typically be used

“more than simply sharing knowledge and information (communication) and more than a relationship that helps each party achieve its own goals (cooperation and coordination). The purpose is to create a shared vision and joint strategies to address concerns that go beyond the purview of any particular party”

The “Continuum of Partnerships”

But…What IS Collaboration?

…In Practice

Collaboration is understood and practiced differently

Commonalities in definition but no two definitions were the same

What does this Mean?

If people are working from their own definition they may be disappointed when their experience doesn’t match their expectations

Establishing a common understanding most important

Stumbling Blocks to Successful Collaboration

Stumbling Blocks

Relationships

“Missing” people collaborations often begin through informal conversations

may be hard for smaller organizations who are not connected well to others or have few staff resources

Staff turnover Not having a “leader” Conflict between members Competing - particularly to acquire funds Members being unable to contribute equally to the collaborative Involvement in collaboratives solely to meet personal goals (i.e. career

advancement) Make others aware of these agendas (Winer & Ray, 1994)

Stumbling Blocks

Communication and Conflict

Not having a plan for how to resolve conflicts

Not having rules for communication Not having frequent enough

communication

Stumbling Blocks

Direction

Having unclear goals, expectations and limitations Having different values or beliefs Not having timelines established Not having written protocols and documents in

place (i.e. logic models, written agreements and contracts outlining rules and plans)

Strategies for Success


Direction

Define “collaboration” and the level of collaboration desired

Be clear about the purpose of the collaborative Be clear about the issue(s) being addressed Be clear about goals Be clear about what each member should contribute and

roles – define formal “contracts” whenever possible Determine timelines for activities but also for the

collaborative (when and why will it be terminated)


Planning for Action

Learn the gaps in services Learn what relevant services are already being offered Determine appropriate collaborative members – ensure

multisector viewpoints Determine who has the power within the community to

influence change


Meeting Management objectives for the meetings should be stated time lines for each objective established rules should be established for how to participate and to

make decisions progress of the meeting should be evaluated at the end summaries of each meeting should be developed contributions of members should be given positive

reception (Winer & Ray, 1994)


Conflict Management

Conflict should be expected Conflict resolution strategies should be designed (Winer & Ray, 1994)

Conflicts within collaboratives can be beneficial engage in “self-reflection” investigate “basic assumptions” - correct faulty assumptions glean greater understanding of the perceptions of others (Mai, et al., 2005,

p. 108)

Determine (with regular check-ins) whether member needs are being met


Freedom To share ideas To engage in free thinking and brainstorming To ask questions To challenge To take risks To clarify the collaborative

Have a good balance between planning and action Limit the number of activities

SO…

Collaboration is risky and can be challenging

Collaboration is Risky and can be Challenging

To engage in collaboration: let go of own ideas and values

come to collective agreement (Chaskin, 2001)

let go of power differentials (Chaskin, 2001)

sacrifice time and resources accept constructive criticism be honest and transparent

Collaborative projects work best when: project itself is clear responsibilities of each organization

are clear (Chaskin, 2001)

So where does evaluation fit in?


Plans for projects make sense Figure out

progress made how well everything is going

Evaluation offers evidence of successes Evaluation offers tangible information to

inform policy decisions


How is it related to collaboration? Strengthens collaborative relationships

uncovers strengths, opportunities for growth, best practices and unexpected learnings

Greater likelihood of successful projects

stronger programs

stronger community

Evaluation

Be Critical

Ask QuestionsChallenge

Periodically Reflect

Sounds easy enough… Social change can get muddy and challenging

Challenges to community capacity building: An unclear understanding of the population(s) of interest Who will be involved and role (enter collaboration) An unclear idea of the outcomes

short or long term? individuals, agencies or the community? what are the goals? what are your indicators of success

Outcomes realistic and achievable? Tracking data?

Recommendation = get some help if possible

Levels of Evaluation

Level 1 Aspects of the Partnership

Level 2 Outcomes of Activities

Level 3 Impacts on the Community

Ideal: do a little of each Reality: Level 3 is very hard to measure – most do levels

1 and 2

Evaluation Activities and Updates from Evaluation Sub-Committee





Ideal: do a little of each Reality: Level 3 is very hard to measure – most do levels 1 and 2

Level 1: Aspects of the Partnership

Member Survey

Purpose: Learn what activities, initiatives or efforts are already available in

the community and to begin with Alliance members What has brought members to the Alliance What may help members feel value in continuing to be part of

the Alliance Feedback from Members about Alliance activities


Member Survey

Survey conducted by telephone 6 participants

Level 1: Aspects of the PartnershipMember Survey

Preliminary Findings: Various reasons for becoming an Alliance Member (personal,

community focused, Alliance focused) Recommendation: Build on these reasons to ensure Members

are receiving the benefits they hoped for

Numerous ways time on Alliance would be considered a wise investment (more members, awareness, progress on activities, etc.)

Recommendation: Establish short term and process goals for the Alliance, for activities and the community


Member Survey

Preliminary Findings: Members would eager to contribute tangle supports and

resources (i.e. financial, data, equipment, etc.) Recommendation: Assign roles and responsibilities to

leverage contributions

Every Alliance member spoke about activities the Alliance could implement beyond current activities

Recommendation: consider ideas for other activities and determine how current or potential activities align with Alliance goals


Preliminary Findings: Variable feedback about activities offered by the Alliance but

buy in overall Recommendations:

Ensure all Members know about and have buy in for Alliance activities Ensure activities aim to achieve the goals of the Alliance Clearly conceptualize each activity (target audience, logic and theory of

change, etc.) Establish evaluation/measurement criteria Assign roles, responsibilities and timelines Have consensus


Next Steps: Survey put into an online survey format Each Alliance member will be emailed the survey for their

feedback

You remember this? The “Continuum of Partnerships”

How to Make progress:

1) Know where you are today2) Agree about where you want to be 3) Evaluate!4) Consider, as a group, how to move “up one notch”5) Measure again at a later point - determine progress6) Make collaboration an important activity - dedicate time to making it successful

Level 1: Aspects of the PartnershipMapping Exercise

Purpose: To capture current partnership information To strengthen relationships Consensus on purpose of the Alliance

Next Steps: All Alliance members will be asked to complete this exercise

(paper copy or email)

Level 1: Aspects of the PartnershipMeeting Checkup

Purpose: To capture feedback about Alliance meetings from members To strengthen meetings Ultimate goal: members will feel their time has been a wise

investment

Next Steps: All Alliance members will be asked to complete this evaluation

activity at some or all meetings (to be determined)





Ideal: do a little of each Reality: Level 3 is very hard to measure – most do levels 1 and 2

Level 2: Outcomes of Activities

Lifelong Learning Week Champions Campaign Inventory

Collaboration and Evaluation Together…In practice

Questions?

Works Cited Beckley, T.M., Martz, D., Nadeau, S., Wall, E. & Reimer, B. (2008). Journal of Rural and Community Development, 3 (3), 56-75. Braganza, M. (2009). “Exploring Collaboration between Organizations Assisting Persons Experiencing Poverty”. Opportunities Waterloo

Region. Chaskin, R.J. (1999). Defining community capacity: a framework and implications from a comprehensive community initiative. Retrieved

January 15 2012 from www.chapinhall.org/sites/default/files/old_reports/41.pdf. Chaskin, R.J. (2001). Building community capacity: a definitional framework and case studies from a comprehensive community initiative.

Urban Affairs Review, 36 (3), 291-323. Gray, B. & Wood, D.J. (1991). Collaborative alliances: Moving from practice to theory. Journal of Applied Behavioural Science, 21 (1), 3-

22. Jakes, S. (2003). Community capacity building. Retrieved February 3 2012 from http://www.google.ca/url?sa=t&rct=j&q=collaboration

%20and%20community%20capacity%20building&source=web&cd=7&sqi=2&ved=0CFgQFjAG&url=http%3A%2F%2Fwww.ces.ncsu.edu%2Fdepts%2Ffcs%2Fpdfs%2Fcap.pdf&ei=jo8sT4eWLqLs0gGBnICRCw&usg=AFQjCNG7ESbpY7zKZTR_Y_uGuKBQf7ZWHw&sig2=fuZ2PnkgyojHyqZrww7gkw.

Leake, D. & Black, R. (2005). Cultural and linguistic diversity: implications for transition personnel. Retrieved February 4 2012 from http://www.ncset.org/publications/essentialtools/diversity/partIII.asp.

Leviten-Reid, E. (2007). Reflecting on vibrant communities (2002-2006). Caledon Institute of Social Policy. Retrieved June 6, 2007 from http://www.caledoninst.org/Publications/PDF/612ENG.pdf.

No Author. (2012). Dystopia: George Orwell web source. Retrieved February 3 2012 from http://www.netcharles.com/orwell/articles/col-dystopia.htm

The Associated Press. (2012). Davos financial group warns of a ‘dystopian future’. Retrieved February 3 2012 from http://www.cbc.ca/news/world/story/2012/01/11/davos-economy-future.html.

Torjman, S. (1998). Community-based poverty reduction. Caledon Institute of Social Policy. Retrieved June 6, 2007 from http://www.caledoninst.org/Publications/PDF/260ENG%2Epdf.

Westley, F., Zimmerman, B. & Patton, M.Q. (2006). Getting to maybe: How the world is changed. Toronto: Vintage Canada.

collaboration and evaluation morgan braganza, m.s.w., ph.d. student rethink research group

Documents

collaboration collaboration

successful collaboration

practice collaboration

important slide

collaboration social

community issues

domain true collaboration

particular party slide