jennifer schwarz ballard (chicago botanic garden), rik panganiban (california academy of science),...

Jennifer Schwarz Ballard (Chicago Botanic Garden), Rik Panganiban (California Academy of Science), Simeon Schnapper (Youtopia), Krystal Meisel (Badging Consultant/After School Matters/Project Exploration), Nathalie Rayter (Adler Planetarium)

And you will know us by the trail of badges…2.0

Lessons learned and ways forward using a community STEM digital

badging ecosystem

DML 2015Los Angeles, CA

Agenda

Background• Why a community STEM badging ecosystem?• Overview of the development process• Current status

Cross-Cutting Themes• Institutional challenges• Technology• Implementation• Data gathering & evaluation• Youth voice

Q & A

Backchannel: #DMLBadges

Share your questions and ideas by tweeting with hashtag #DMLBadges

A Community STEM Badging Ecosystem

What? How?Why?

Why community badging?

SharedVettedExpandable/TransferablePowerful

What should the ecosystem look like?

What framework would we use to create the actual badges and what should they be?

What skills should they recognize? How should those skills be assessed?

What technology should we use? What functionality would we need?

How would they be integrated into different learning contexts from both educator and youth perspective?

How would they be used in the future and what for?

Building the framework

Lyon, G. H., Jafri, J., & St Louis, K.(2012). Beyond the Pipeline: STEMPathways for YouthDevelopment. Afterschool Matters, 16,48-57.

Project Exploration 2012

Testing and refining

Cross cutting themes

Institutional challengesTechnology

ImplementationData & evaluation

Youth voice

Institutional Challenges

• Securing buy-in• Challenges of implementing an

outside system

Institutional Challenges

Securing Institutional Buy-in:

➢ Leadership ➢ Key Stakeholders

Cal Academy Stakeholder Consultations

Staff we included:➢ Teacher Education / Professional Development➢ Youth Education➢ Docents / Volunteers➢ Membership➢ Web Team➢ Development

Results of Consultations

Results:➢ Stronger support for

digital badging➢ Key areas to start

work➢ Creates internal

evangelists

Technology

• Theory of design• Choosing a platform• Different functionality for different

purposes

Technology

+Motivational Psychology

+Behavioral Economics

+Neurobiology

+Badging (Credentialing)

Technology

Phase #1 Badging Air Force Academy High School @ Adler Planetarium

C-STEMM Badges issued: 8 (Discover: Communication)

Content Badges: 72 (Sketch Artist, Sales Pitch, Recon

Badge, Conferencing Badge, etc.)

Technology

Technology

Phase #2 Badging with After School Matters:

Varied CSTEMBE badges (over a dozen) coupled with Content badges specific to each program were issued.

5 Programs 106 YouthFree Spirit Media 15

Chicago Architecture Foundation 16

Choices Robotics 33

Columbia Jr. Research Scientists 19

Project Syncere 23

Implementation

Meets someone in a STEM Career, field or with similar interests.

“During the Regionals Science Fair Project, I met the engineering Dean of CSU, who took a great

interest in my personal project and even offered me a STEM summer programming job.” - Vontrice B.

Data Gathering & Evaluation

• Goals• Pilot Testing• Data

What did we want to know?

Usability

Integration

Value

Usefulness

Pilot Testing

Data: Analytics, Observation, Interviews, Focus Groups, Surveys

Phase 1 (2014) Phase 2 (Spring 2015)

9 classes, 3 orgs 5 classes, 5 orgs (through ASM)

120 youth 106 youth

83 surveys 49 surveys

5 youth focus groups NA

19 program observations NA

3 instructor focus groups(1 per org)

3 instructor interviews

Data

Coming Soon…

Analytics Badge accessing

Badge sharing

Third party accessing

Variables Badge

Badge type

Organization

Program

Student

Evidence

User type

Youth Voice

• Buy-in• Access & Equity

So, are students interested in badges?

Well, it depends…

on program format One-off programs (Adler field trip

experience)

Was earning the digital badge fun?

62% Yes 32% No

Extended programs (ASM pilot)

Do you think badges were an important part of what you did in the program?

37.2% Yes 62.8% No

Student: “Yes because badging was a big part of the program and it taught us how we could use them on our resume.”

Student: “While the concept seems intriguing, I don't know if they fit in with what we were doing. We had to set time aside to work on our badges, which meant our usual work was disrupted. If there were improvements made on how to earn the badges or the process of earning them, I think that would help.”

Instructor buy-in student buy-in 5 different ASM pilot programs w/ varying

badge integration

How did earning badges fit into the other activities in your program?

They interrupted what we were doing20.9% overall, 60% of students in JRS program

More positive student feedback came from group with more positive instructor feedback

How did earning badges fit into the other activities in your program?

They fit in smoothly with the other activities34.9% overall, 80% of students in CAF program

So, are students interested in badges?

JRS Instructor: “On the teen side – they don’t care. The only reason they [earned badges] is because we asked them to. The badge has no value.”

CAF Instructor: “We already had the program curriculum so we integrated badges into that. They’re a great way to have something tangible to show for your work.”

Usefulness

While some students can identify opportunities to use their badges in the future, the majority still don’t see the point.

Are you going to do anything with the digital badges you earned in this program?

Yes - 37.5%

Student: I would add these badges to my resume to help me in both college and the real world.

No - 63.5%

Student: I don't know what to do

Student: I don't think I will have any use for them.

(Source: After School Matters C-STEMBE Pilot survey)

Usefulness

Yes: I think badges helped me to feel like the work I was putting into the program was being recorded and looked at.

Yes: There are some skills that one doesn't really take into consideration when involved in a certain activity. It's hard to notice certain things that we do while we are doing them. I realized that many of the skills and tools used in the activities we did also applied to many other things.

No: Because I don't feel that badges should be used in a hands on program

No: The badges for me were just a waste of time. They didn't really represent what we did in the program and they just took away from our time.

Do you think badges were an important part of what you did in the program? Explain.

(Source: After School Matters C-STEMBE Pilot survey)

Which technology do you use most often when you’re NOT in the program?

Access

Technology Percentage

Computer with Internet 12.2%

Phone with texting 18.4%

Smart Phone 65.3%

Tablet/iPad/iPod 2.0%

None 2.0%

When in programs, students have varying access to technology.

Student: It felt a bit like a chore having to apply and type so much.

(Source: After School Matters C-STEMBE Pilot survey)

Youth still not all equipped with internet access

What do we take from this?

Student feedback and observation should influence the way forward.

Don’t put the cart in front of the horse

Ease of use

Development of badge value

Wrap up

Institutional challengesTechnologyImplementationData gathering & evaluationYouth voice

Jennifer Schwarz BallardChicago Botanic Gardenjschwarz@chicagobotanic.org@jschwarzballard

Q & A

Simeon SchnapperYoutopiasimeon@youtopia.com @simeonschnapper

Krystal MeiselBadging Consultantkrystalmeisel@gmail.com @euhedral

Rik PanganibanCalifornia Academy of SciencesRPanganiban@calacademy.org @riktheranger

Nathalie RayterAdler Planetariumnrayter@adlerplanetarium.org @nathalie_bfr