aaeebl regional meeting 2016 keynote
TRANSCRIPT
![Page 1: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/1.jpg)
1
Open Digital Badges + ePortfolios:Searching for and Supporting Synergy
Daniel T. HickeyProfessor and Program CoordinatorLearning Sciences ProgramDirector, Participatory Assessment LabIndiana University
![Page 2: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/2.jpg)
2
![Page 3: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/3.jpg)
3
IU Open Badges Team Members
RebeccaItow
CathyTran
KaterinaSchenke
JamesWillis
Nate Otto
ChristineChow
JoshuaQuick
Cindy Cogswell
KirstinHelström
Suraj Uttamchandani
GinaHoward
![Page 4: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/4.jpg)
4
Jim P., Jim G. & Jim Gee(My Three Uncles)
Jim Pellegrino
JimGreeno
JimGee
![Page 5: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/5.jpg)
5
My Three Cousins
LorrieShepard
PamelaMoss
Randi Engle
![Page 6: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/6.jpg)
6
![Page 7: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/7.jpg)
7
Why Badges?• Badges contain specific claims of competency, achievement,
completion, etc.• Badges can contain evidence supporting those claims.
– The optional evidence field can contain links to additional evidence (e.g., artifacts) supporting those claims
• These claims and evidence can readily circulate in social networks, email, etc.– Thus they gain additional information including endorsements and
(potential) rejection.– Thus they can help potential earners find opportunities.
• Badges are interoperable and extensible.– Earners can curate and annotate their collections.– OBI Badges should be displayable in all future platforms
![Page 8: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/8.jpg)
8
• MacArthur launches DML initiative in 2006
• P2PU and Mozilla define badges in 2010
• 2012-2014 Badges for Lifelong Learning Initiative
• 2012-2015 Badges DPD Project
• Credly and other startups start emerging ~2013
• Badge Alliance launched in 2014
• Open Badges in Higher Ed Project 2014-2016
![Page 9: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/9.jpg)
9
The Promise of Digital Badges
![Page 10: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/10.jpg)
10
Challenges for Digital Badges
![Page 11: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/11.jpg)
11
The Evolving Badgescape• MacArthur’s $25M start up
& Project LRNG 2015• Badge Alliance
reorganization in 2016• LTI-compliant badges for
major LMSs• IMS Global Open Badge in
Ed Extensions• New JSON-LD (linked data)
standards• DML 2016 Competition
![Page 12: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/12.jpg)
12
![Page 13: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/13.jpg)
13
![Page 14: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/14.jpg)
14
![Page 15: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/15.jpg)
15
• Project Zero launches in 1967, portfolios in 1980s
• High-stakes K-12 portfolios rise and fall 1990-1995
• ePortfolios emerge in higher ed late 1990s
• Commercial platforms start emerging ~2000
• INCEPR in 2003, AAEEBL in 2009
• Cambridge et. al, 2009, Penny Light and Chen, 2011
• MCNRC/Catalyst in 2014
![Page 16: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/16.jpg)
16
• Project Zero launches in 1967, portfolios in 1980s
• High-stakes K-12 portfolios rise and fall 1990-1995
• ePortfolios emerge in higher ed late 1990s
• Commercial platforms start emerging ~2000
• INCEPR in 2003, AAEEBL in 2009
• Cambridge et. al, 2009, Penny Light and Chen, 2011
• MCNRC/Catalyst in 2014
![Page 17: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/17.jpg)
17
The Promise of ePortfolios• Puts learners in charge of learning and displaying• ePortfolio is the “common denominator” in the move
from teaching to learning– Abundance of knowledge– Knowledge is rapidly changing– Economy that demands documented competency– Dismal evidence of learning from legacy methods– Changing nature of college students
• Complements other trends in higher education– CBE, SRL, authentic assessment, personalization, self-pacing,
service learning, etc.
![Page 18: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/18.jpg)
18
The Promise of e-Portfolios
![Page 19: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/19.jpg)
19
Challenges for ePortfolios
![Page 20: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/20.jpg)
20
![Page 21: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/21.jpg)
21
Why Add Badges to ePortfolios?• Allow credit outside of formal credential context
– Give credit for things that are hard to grade• Can provide additional useful information
– Specific claims and detailed evidence – Context in which content was created– Standardize the inclusion of additional information without cluttering
eportfolios• May simplify the process of defining competencies• May offload summative credentialing functions
– Allows more formative and transformative functions• Can connect eportfolio content to competencies and grades
– Learners stack badge URLs in maps or gradebook• Can increase value of portfolio content by circulating independently
![Page 22: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/22.jpg)
22
Claimsand
Evidence
![Page 23: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/23.jpg)
23
ev·i·dence /ˈevədəns/
• the available body of facts or information indicating whether a belief or proposition is true or valid
• In education, evidence usually concerns claims about what someone learned and/or what they will be able to do in the future.
• Validity is a property of claims not assessments– Assessments can be reliable (but not valid)– Validity is an argument
![Page 24: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/24.jpg)
24
Standards for Educational and Psychological Testing (NCME/APA/AERA)
1999
19851966
2014
• Different functions of assessment– Summative vs. formative vs.
transformative.• Different types of evidence
– Behavior vs. cognition vs. participation.
• Different units of analysis– Practices vs. people vs. programs vs.
institutions.• Competing purposes create tensions
– Many unintended consequences– Focus on functions instead
![Page 25: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/25.jpg)
25
Validity is Contentious
• Traditional Conception– Criterion, content, and construct-related evidence
• Modern “aspects of construct validity” from Messick– Content, substantive, structural, generalizability, external,
and consequential– Shepard, Moss, Hickey, and others argue that
consequences should be considered first.• Primary threats to validity– Construct under-representation– Construct-irrelevant variance
![Page 26: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/26.jpg)
26
Badges as Boundary Objects• Boundary objects are particular kinds of
cultural tools (Star and Grisemer, 1989)– “both plastic enough to adapt to local needs
and constraints of the several parties employing them, yet robust enough to maintain a common identity across sites.
– “weakly structured in common use, and become strongly structured in individual-site use.
– “They may be abstract or concrete. They have different meanings in different social worlds but their structure is common enough to more than one world to make them recognizable, a means of translation.”
![Page 27: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/27.jpg)
27
Credentials that Cross Boundaries(Moss, Girard, & Haniford, 2006)
• Six dimensions in which evidence crosses boundaries:1. Types of evidence (artifacts vs. scores)2. Explicitness of evidence (systematic?)3. Extent to which actors from the local context are
available in the new context4. Comprehensiveness of representation of local
context (foreground vs. background)5. The interpretive norms and routines that accompany
the evidence6. Roles of local and external actors in shaping
representations of practice.
![Page 28: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/28.jpg)
28
Credentials that Cross Boundaries(Moss, Girard, & Haniford, 2006)
• “When evidence crosses boundaries, it brings far more than information.”– Cultural tools (e.g., artifacts & concepts).– Norms and routines mediate understanding and
(inter)action in sending and receiving contexts. • “Positions the different actors with different
authority, accountability, and agency for making decisions about how their practices are represented and how those representations should be interpreted and used”
![Page 29: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/29.jpg)
29
• Students do meaningful work in ePortfolios– Including curricular and co-curricular– Artifacts are shared, discussed and endorsed.
• Students earn badges for the work– Highlights competencies and presents context– Competencies are shared, discussed, and endorsed
• Students “stack” badges into competency maps or gradebooks by simply pasting URLs– Instructors assign grades and private feedback– Information is FERPA-protected– Same badge can be in a gradebook or a competency map
The Ultimate Synergy:Badges + Portfolios + Competencies
![Page 30: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/30.jpg)
30
![Page 31: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/31.jpg)
31
Transforming ePortfolio Credentialing with Digital Badges
(Quick & Willis, 10:00)
In development
Credly integration
Pearson Acclaim integration
Interested
Open Badge Factory default
Credly integration
Interested
![Page 32: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/32.jpg)
32
Balancing, Summative, Formative and Transformative ePortfolio Functions
(Hickey & Helstrom, 3:20)PLA DESIGN PRINCIPLES
Use public contexts give meaning to knowledge toolsPublically support and reward productive disciplinary engagementGrade artifacts through local reflectionsLet individuals assess their understanding privatelyMeasure aggregated achievement discreetly
![Page 33: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/33.jpg)
33
2014 2016
Suspended None
Suspended None
Partial Existing
Implemented Existing
Implemented Thriving
Implemented Thriving
Status of 6 (of 29) Projects from 2012 Badges for Lifelong Learning Initiative
Where ePortfolios and Open Digital Badges Work Better (Hickey & Willis, 4:20)
![Page 34: AAEEBL Regional Meeting 2016 Keynote](https://reader034.vdocument.in/reader034/viewer/2022052706/58a7e9811a28abd7248b59cd/html5/thumbnails/34.jpg)
34
Open Questions and Discussion• Are badges even going to endure?
– If it does not what micro-credentialing practice will endure• Is carpetbadging really a problem?
– Maybe “bro-badges” have a useful function– Democratization of credentialing?
• Are badges redundant with ePortfolio functions?• Might the OBI standards be “IMS-OBEE lite”• What is the financial model for badges?
– e.g., Achievery shut down in 2015– Registrars charge for copies, platforms and assessment firms charge for licenses.
• Sustainability of badges• Unintended consequences?• Other issues?