jennifer schwarz ballard (chicago botanic garden), rik panganiban (california academy of science),...
TRANSCRIPT
Jennifer Schwarz Ballard (Chicago Botanic Garden), Rik Panganiban (California Academy of Science), Simeon Schnapper (Youtopia), Krystal Meisel (Badging Consultant/After School Matters/Project Exploration), Nathalie Rayter (Adler Planetarium)
And you will know us by the trail of badges…2.0
Lessons learned and ways forward using a community STEM digital
badging ecosystem
DML 2015Los Angeles, CA
Agenda
Background• Why a community STEM badging ecosystem?• Overview of the development process• Current status
Cross-Cutting Themes• Institutional challenges• Technology• Implementation• Data gathering & evaluation• Youth voice
Q & A
Backchannel: #DMLBadges
Share your questions and ideas by tweeting with hashtag #DMLBadges
A Community STEM Badging Ecosystem
What? How?Why?
Why community badging?
SharedVettedExpandable/TransferablePowerful
What should the ecosystem look like?
What framework would we use to create the actual badges and what should they be?
What skills should they recognize? How should those skills be assessed?
What technology should we use? What functionality would we need?
How would they be integrated into different learning contexts from both educator and youth perspective?
How would they be used in the future and what for?
Building the framework
Lyon, G. H., Jafri, J., & St Louis, K.(2012). Beyond the Pipeline: STEMPathways for YouthDevelopment. Afterschool Matters, 16,48-57.
Project Exploration 2012
Testing and refining
Cross cutting themes
Institutional challengesTechnology
ImplementationData & evaluation
Youth voice
Institutional Challenges
• Securing buy-in• Challenges of implementing an
outside system
Institutional Challenges
Securing Institutional Buy-in:
➢ Leadership ➢ Key Stakeholders
Cal Academy Stakeholder Consultations
Staff we included:➢ Teacher Education / Professional Development➢ Youth Education➢ Docents / Volunteers➢ Membership➢ Web Team➢ Development
Results of Consultations
Results:➢ Stronger support for
digital badging➢ Key areas to start
work➢ Creates internal
evangelists
Technology
• Theory of design• Choosing a platform• Different functionality for different
purposes
Technology
+Motivational Psychology
+Behavioral Economics
+Neurobiology
+Badging (Credentialing)
Technology
Phase #1 Badging Air Force Academy High School @ Adler Planetarium
C-STEMM Badges issued: 8 (Discover: Communication)
Content Badges: 72 (Sketch Artist, Sales Pitch, Recon
Badge, Conferencing Badge, etc.)
Technology
Technology
Phase #2 Badging with After School Matters:
Varied CSTEMBE badges (over a dozen) coupled with Content badges specific to each program were issued.
5 Programs 106 YouthFree Spirit Media 15
Chicago Architecture Foundation 16
Choices Robotics 33
Columbia Jr. Research Scientists 19
Project Syncere 23
Implementation
Meets someone in a STEM Career, field or with similar interests.
“During the Regionals Science Fair Project, I met the engineering Dean of CSU, who took a great
interest in my personal project and even offered me a STEM summer programming job.” - Vontrice B.
Data Gathering & Evaluation
• Goals• Pilot Testing• Data
What did we want to know?
Usability
Integration
Value
Usefulness
Pilot Testing
Data: Analytics, Observation, Interviews, Focus Groups, Surveys
Phase 1 (2014) Phase 2 (Spring 2015)
9 classes, 3 orgs 5 classes, 5 orgs (through ASM)
120 youth 106 youth
83 surveys 49 surveys
5 youth focus groups NA
19 program observations NA
3 instructor focus groups(1 per org)
3 instructor interviews
Data
Coming Soon…
Analytics Badge accessing
Badge sharing
Third party accessing
Variables Badge
Badge type
Organization
Program
Student
Evidence
User type
Youth Voice
• Buy-in• Access & Equity
So, are students interested in badges?
Well, it depends…
on program format One-off programs (Adler field trip
experience)
Was earning the digital badge fun?
62% Yes 32% No
Extended programs (ASM pilot)
Do you think badges were an important part of what you did in the program?
37.2% Yes 62.8% No
Student: “Yes because badging was a big part of the program and it taught us how we could use them on our resume.”
Student: “While the concept seems intriguing, I don't know if they fit in with what we were doing. We had to set time aside to work on our badges, which meant our usual work was disrupted. If there were improvements made on how to earn the badges or the process of earning them, I think that would help.”
Instructor buy-in student buy-in 5 different ASM pilot programs w/ varying
badge integration
How did earning badges fit into the other activities in your program?
They interrupted what we were doing20.9% overall, 60% of students in JRS program
More positive student feedback came from group with more positive instructor feedback
How did earning badges fit into the other activities in your program?
They fit in smoothly with the other activities34.9% overall, 80% of students in CAF program
So, are students interested in badges?
JRS Instructor: “On the teen side – they don’t care. The only reason they [earned badges] is because we asked them to. The badge has no value.”
CAF Instructor: “We already had the program curriculum so we integrated badges into that. They’re a great way to have something tangible to show for your work.”
Usefulness
While some students can identify opportunities to use their badges in the future, the majority still don’t see the point.
Are you going to do anything with the digital badges you earned in this program?
Yes - 37.5%
Student: I would add these badges to my resume to help me in both college and the real world.
No - 63.5%
Student: I don't know what to do
Student: I don't think I will have any use for them.
(Source: After School Matters C-STEMBE Pilot survey)
Usefulness
Yes: I think badges helped me to feel like the work I was putting into the program was being recorded and looked at.
Yes: There are some skills that one doesn't really take into consideration when involved in a certain activity. It's hard to notice certain things that we do while we are doing them. I realized that many of the skills and tools used in the activities we did also applied to many other things.
No: Because I don't feel that badges should be used in a hands on program
No: The badges for me were just a waste of time. They didn't really represent what we did in the program and they just took away from our time.
Do you think badges were an important part of what you did in the program? Explain.
(Source: After School Matters C-STEMBE Pilot survey)
Which technology do you use most often when you’re NOT in the program?
Access
Technology Percentage
Computer with Internet 12.2%
Phone with texting 18.4%
Smart Phone 65.3%
Tablet/iPad/iPod 2.0%
None 2.0%
When in programs, students have varying access to technology.
Student: It felt a bit like a chore having to apply and type so much.
(Source: After School Matters C-STEMBE Pilot survey)
Youth still not all equipped with internet access
What do we take from this?
Student feedback and observation should influence the way forward.
Don’t put the cart in front of the horse
Ease of use
Development of badge value
Wrap up
Institutional challengesTechnologyImplementationData gathering & evaluationYouth voice
Jennifer Schwarz BallardChicago Botanic [email protected]@jschwarzballard
Q & A
Simeon [email protected] @simeonschnapper
Krystal MeiselBadging [email protected] @euhedral
Rik PanganibanCalifornia Academy of [email protected] @riktheranger
Nathalie RayterAdler [email protected] @nathalie_bfr