archee workshop day 1: 17 june, 2015 washington, dc 17-18 june 2015archee workshop 1
TRANSCRIPT
ARCHEE WORKSHOP 1
ArchEE WorkshopDAY 1: 17 JUNE, 2015
WASHINGTON, DC
17-18 June 2015
ARCHEE WORKSHOP 2
Welcome!
17-18 June 2015
ARCHEE WORKSHOP 3
Who is ArchEE?
17-18 June 2015
ARCHEE WORKSHOP 4
Who is in the room? What is your . . .
Name?Affiliation?
What would ArchEE look like to engage you totally?
What is one burning question?
17-18 June 2015
ARCHEE WORKSHOP 5
Why a Workshop? We will create a foundation for collective action to design, develop, implement, and sustain ArchEE.
17-18 June 2015
ARCHEE WORKSHOP 6
What questions will we consider?
Who are we and what are we about?
What is ArchEE and what will its success look like?
What is ArchEE’s Theory of Change?
Who will use ArchEE and how will they use it?
How will we will support ArchEE development and implementation?
How will we sustain ArchEE into the future?
What needs to be done and by whom and when?
What is next?
17-18 June 2015
ARCHEE WORKSHOP 7
WednesdayWhat? So what?
17-18 June 2015
8:00 – 8:30 Arrive and set up8:30 – 10:45 Get started10:45 – 11:00 Break11:00 – 12:30 Build foundation12:30 – 1:30 Lunch1:30 – 3:00 Define success criteria3:00 – 4:30
With breakHow will it work?
4:30 –5:00 Close the day5:00 – 5:30 Prepare for tomorrow7:00 – ??? Take me out to the ballgame!
ARCHEE WORKSHOP 8
ThursdayNow what?8:00 – 8:30 Arrive and set up8:30 – 9:30 Gather and settle9:30 – 10:45 Define opportunities arising from use cases10:45 – 11:00 Break
11:00 – 12:30 Plan for managing ArchEE in the short- and long-term
12:30 – 2:00 Lunch2:00 – 3:45 With break
Plan for action
3:45 – 4:30 Integrate plans and commit4:30 – 5:00 Close the meeting5:00 – 5:30 Clean up and exit7:00 – ??? Dinner at Matt’s house
17-18 June 2015
ARCHEE WORKSHOP 9
What will we produce? Technical and project requirements
Conceptual design
Draft Theory of Change
Project structure and building blocks for a project plan
Commitment from partners
17-18 June 2015
ARCHEE WORKSHOP 10
Patterns of Hope and Excitement
What do you hope for and what excites you about:Your own work with ArchEE?This workshop?ArchEE’s long-term impact?
17-18 June 2015
ARCHEE WORKSHOP 11
What structures the systems work?
Human Systems DynamicsComplex Adaptive SystemsAdaptive ActionPattern LogicRules of InquiryThree kinds of questions
17-18 June 2015
Complex Adaptive System (CAS)Self-Organizing System
Those system-wide patterns, in turn, influence the behaviors of the agents
Agents interact
System-wide patterns emerge
Adaptive Action
What? So what?
Now what?
ARCHEE WORKSHOP 14
Pattern Logic Method of reasoning based on conditions for self-organizing:
Similarities (C), differences (D), and connections (E)Patterns
Works under conditions of:Open boundariesMultiple interdependent variablesNonlinear causality
Supports best fit action under conditions of uncertainty
17-18 June 2015
ARCHEE WORKSHOP 15
Simple Rules of Inquiry Turn judgment into curiosity
Turn conflict into shared exploration
Turn defensiveness into self-reflection
Turn assumptions into questions
17-18 June 2015
ARCHEE WORKSHOP 16
Kinds of Questions Answers are:
Known by someone, we just have to find the right expertUnknown by anyone, but we have a method to find outUnknowable here and now, we watch patterns emerge over time
What is a question you have about ArchEE?
17-18 June 2015
ARCHEE WORKSHOP 17
What Is ArchEE? . . . an open-access repository of evaluations and other evaluative knowledge that can be analyzed and synthesized to support evidence-based practice and improved transfer and use of evaluative knowledge for environmental management.
In general . . . In general . . . , exceptOn one hand . . ., on the other . . . I am surprised that . . . I wonder . . .
17-18 June 2015
MaterialsAluminum
Energy
ManufactureFactory
ProductCanned Soda
UseThirsty
ReturnCollection
DisassemblySeparate
DisposeLandfill
MaterialsLifecycle
Cradle to Cradle Management
A Model to Emulate
AccessUser Interface
EvaluateCollect Data
Analyze
ReportMethods Findings
UseEvidence-Based Archive
Cabinet Shelf
Website
Cradle to GraveEvaluation
Local Learning
One Evaluation
ArchEELifecycleArchEE
Lifecycle
Cradle to Cradle Evaluation
Global Learning
Some products skip use and go directly into the database.
ResourcesUser Interface
ArchEE and associated repositories are now added to the pool of resources available to evaluators, managers, policy makers, researchers who may choose to access ArchEE to evaluate . . .
Materials
Evaluate Collect Data
AnalyzeSynthesizeM
anuf
actu
re
This is the process of using resources to create something new and useful to be seen as products and services
UseEvidence-Based
Just as the can is for use, the products and services that come out of the evaluation process are for use in: decisions, understanding, communication, learning, building evidence, design . . .
Use
CollectDatabase
Return
Just as the can is returned to a collections facility, evaluation products are returned to a repository/database. This can happen in many ways, but once it is returned, it is available for analysis etc . . .
The information is analyzed, disaggregated, disassembled, tagged, sometimes synthesized. After it is in the database and on the platform, that evaluative, evaluation knowledge is available and accessible as additional resources . . .
AnalysisTag
Disaggregate
Disassem
bly
Products & ServicesMethods Findings
Prod
uct
A report, paper, findings, questions, recommendations, methods, knowledge, information, websites, tools, evaluators, expertise, experience, topics and so on are then, hopefully for use . . .
The user can customize analysis to be fit for purpose.
ARCHEE WORKSHOP 21
What is a Theory of Change?
A theory of change (TOC) is a tool for developing solutions to complex social problems. A basic TOC explains how a group of early and intermediate accomplishments sets the stage for producing long-range results. A more complete TOC articulates the assumptions about the process through which change will occur and specifies the ways in which all of the required early and intermediate outcomes related to achieving the desired long-term change will be brought about and documented as they occur.
Adapted from Anderson, A. (2005). The community builder's approach to theory of change: A practical guide to theory and development. New York: The Aspen Institute Roundtable on Community Change.
17-18 June 2015
ArchEELifecycleArchEE
Lifecycle
1
5
4 3
2
6
ResourcesUser Interface
Evaluate Collect Data
AnalyzeSynthesize
Products & ServicesMethods Findings
UseEvidence-Based
CollectDatabase
AnalysisTag
Disaggregate
What happens here?
So what makes it successful?
Now what are we sure about and what questions do we have?
ARCHEE DESIGN NOTES 23
What Requirements?ArchEE will succeed if we . . . 1.0 Attract and retain the right user base
2.0 Include the right stuff
3.0 Use state-of-the-art methods
4.0 Keep it simple
5.0 Make it a learning system
6.0 Sustain it
7.0 And . . .
28APR15
ARCHEE DESIGN NOTES 24
What Requirements?Setting the Foundation
Dennis Bours & Anna Viggh
Kara Chrohn
Gabriela Fitz
Marc Hockings
Andrew Pullin
David Widawsky
28APR15
ARCHEE DESIGN NOTES 25
What Requirements?ArchEE will succeed if we . . . 1.0 Attract and retain the right user base
2.0 Include the right stuff
3.0 Use state-of-the-art methods
4.0 Keep it simple
5.0 Make it a learning system
6.0 Sustain it
7.0 And . . .
28APR15
What is the most exciting and challenging?
So what is missing, unnecessary, wrong headed?
Now what change will you make?Now what questions do you have?
ARCHEE DESIGN NOTES 26
1.0 Attract and retain the right user base Participatory engagement
Make it as bottom-up as possible.Transparent process.Develop broad community investment; use network; seed small experiments throughout to work on different aspects.Funding source for seed grants.Distribute small, defined tasks that build engagement and investment. Seed the community with opportunities to beta test, usability test, etc. – maybe give small seed grants for stakeholders to become involved, perhaps do their own “use case” studies, etc. Provide small ways people and organizations can contribute and engage.
Effective communicationsPlan marketing, visibility – where will it be highlighted?Target specific audiences.Who is your audience? Who do you want to engage? If all inclusive is that the best strategy?Identify what value you will add for what audiences. What are users doing now? Google? How would ArchEE help? The energy industry, public health and other sectors already have repositories of studies, so what do you add, or how do you connect or relate to other systems?
Manage power and politicsDepend on network structures and processes to reduce hierarchyBe careful of hierarchies within user organizations – people at different levels of stakeholder organizations will have different perspectives on use, etc.Who are the stakeholders? Buy-in? Investment?
28APR15
ARCHEE DESIGN NOTES 27
2.0 Include the right stuff Collect and retain the “best”
Establish clear inclusion criteria, e.g. is it environmental evaluation or public health?Minimum quality requirements.Make use of existing sources like Better Evaluation.Standards, boundaries, and transparent decisions about what’Establish clear criteria for inclusion; what crosses over the border from environmental evaluation to other sectors such as health?Single decision-maker with clear criteria, guidance.Keep it fresh with new materials
Recycle, reuseConnect to other clearinghouse sites.Link to other repositories rather than duplicating their work.Use groups like the Environmental Education Association of Oregon to get feedback.
Follow the rulesCopyrightOwnershipPrivacyClearance and review
28APR15
ARCHEE DESIGN NOTES 28
3.0 Use state-of-the-art methods Prototype:
Build a RAPID prototype showing end-to-end functionality. Find the audience. Find the money. Make it cheap. Make users work.Build simple prototype for testing and adding to.Make an end-to-end prototype.
Learn from others:ViVo-digital CV – health field.HUB Zero – open source.Ask developers of other repositories for advice.First step: survey users to find out what they need most, first.
Take one step at a time:Start small, follow staged, phased-in development process.Do beta testing.Start small and expand.Start small and focused, then add components.
28APR15
ARCHEE DESIGN NOTES 29
Language:Avoid jargon, acronyms, exclusionary languageExplain all jargon, terminology clearly.
Ease of use:Usability testingExpert reviewPilot test each phase.Usability; if the interface of this repository can’t be used intuitively people will not like using it.Easy entry; Easy navigation, easy searchQuick response time
Common standards.Wiki-styleADA compliant
Effective information architectureUse the best of library scienceSimple, transparent categoriesThere must be some kind of taxonomy for querying tables in advance of artificial intelligence.Establish a template and/or protocol for use cases and recruit lots of folks to contribute these.
4.0 Keep it simple
28APR15
ARCHEE DESIGN NOTES 30
5.0 Make it a learning system Use a login system so you know who’s using it; perhaps a username with no password?
Do “use cases” in the beginning to understand what users will actually do and how they will do it and what they need.
User feedback.A safe place (anonymous) for posting lessons learned and failures and problems and challenges.Make ArchEE an open source siteArchEE creates competition in the education market.
28APR15
ARCHEE DESIGN NOTES 31
6.0 Sustain It Technical environment
Develop a place and a model for long term maintenance.Arrange sustained funding for long-term maintenance and curation. Software becomes vulnerable and requires patching;
FundingFind out if a subscription model would work – is there willingness to pay? Or a donation model like Wikipedia?Establish a funding mechanism.Fundraising for administration, development, maintenance of the server, tech support, management of content, etc.
Collective decision makingThe community needs cultivation and outreach Collaborative ownership to ensure diverse voices in the development and in the gatekeeper role. Don’t start something that requires consensus that dictates whether the project continues.
Accountability:You need a lead person or organization to make it happen; there should be a clear leader.An organization that hosts responsibly.A clear organizational structure – who’s responsible for what – paid positions or rotating responsibilities among groups or something similar.Establish a home institution that can organize funding and maintenance/technical support.These tasks must be somebody’s job who is responsible and accountable.Complete annual assessment and adaptation
28APR15
ARCHEE DESIGN NOTES 32
7.0 What is missing?
28APR15
ARCHEE WORKSHOP 33
Who will use ArchEE?Users Uses
Evaluators • •
• •
Funders • •
• •
Managers • •
• •
Planners • •
• •
Policy Makers • •
• •
Researchers • •
• •
Teachers • •
• •
17-18 June 2015
ARCHEE WORKSHOP 34
So what? Complete Use Cases0.0 Submitting to ArchEE1.0 Evidence-Based Decisions2.0 Strategic Investment3.0 Policy Design4.0 Evaluation Design5.0 Collaboration
6.0 Recruiting7.0 Quality8.0 Evaluation9.0 Resource Allocation10.0 Research11.0 Capacity Development
17-18 June 2015
ARCHEE WORKSHOP 35
Close Day 1 What have we found?
What do you want more of, less of tomorrow?
What changes do we need to make to our questions?
What is our agenda for tomorrow?
What else for the good of the group?
17-18 June 2015
ARCHEE WORKSHOP 36
ArchEE WorkshopDAY 2: 18 JUNE, 2015
WASHINGTON, DC
17-18 June 2015
ARCHEE WORKSHOP 37
Get Started Overnight thoughts
Questions from yesterday—more? Different?
Revisit purpose and deliverables
Rules for Inquiry
And . . .
17-18 June 2015
ARCHEE WORKSHOP 38
So what? Complete Use Cases0.0 Submitting to ArchEE1.0 Evidence-Based Decisions2.0 Strategic Investment3.0 Policy Design4.0 Evaluation Design5.0 Collaboration
6.0 Recruiting7.0 Quality8.0 Evaluation9.0 Resource Allocation10.0 Research11.0 Capacity Development
17-18 June 2015
ARCHEE WORKSHOP 3917-18 June 2015
ARCHEE WORKSHOP 40
Now what about ArchEE Management?
What functions? What resources?Accumulating documentsEvaluating, coding and taggingStoring and maintainingAccessingCollecting and processing feedback about specific documents and processesResourcing and resource management
17-18 June 2015
ARCHEE WORKSHOP 41
Now what helps ArchEE learn?
What functions need to be in place to ensure on-going adaptation and growth?
17-18 June 2015
ARCHEE WORKSHOP 42
Now what will we do?Planning the Work
What tasks need to be done?
What are interdependencies?
What are resources?
What are task clusters?
17-18 June 2015
ARCHEE WORKSHOP 43
For each cluster, define . . .
Purpose
Deliverables
Assumptions
Tasks
Schedules and milestones
Resources
Responsibilities
Interdependencies with other clusters
17-18 June 2015
ARCHEE WORKSHOP 44
What questions will we consider?
Who are we and what are we about?
What is ArchEE and what will its success look like?
What is ArchEE’s Theory of Change?
Who will use ArchEE and how will they use it?
How will we will support ArchEE development and implementation?
How will we sustain ArchEE into the future?
What needs to be done and by whom and when?
What is next?
17-18 June 2015
ARCHEE WORKSHOP 45
What will we produce? Technical and project requirements
Conceptual design
Draft Theory of Change
Project structure and building blocks for a project plan
Commitment from partners
17-18 June 2015
ARCHEE WORKSHOP 46
Now what is next? What do we know for sure about ArchEE and its future?
So what do we wonder about? So what are the remaining questions?
Now what will we commit to do to move forward?
Now what are our next steps?
17-18 June 2015