caliper: tapping into the learn activity stream
TRANSCRIPT
Caliper: tapping into the Learn Activity StreamDaniel Knight Product Development @ BlackboardDan Rinzel Product Manager for Analytics @ Blackboard
Agenda – Q’s we will hope to A today
• What is IMS Global, Caliper Learning Analytics, why does it matter and how is Blackboard involved?
• What is the Learn activity stream?
• What do stream events look like?
• How can schools use the stream?
• How can a tool provider participate in the stream?
Statements regarding our product development initiatives, including new products and future product upgrades, updates or enhancements represent our current intentions, but may be modified, delayed or abandoned without prior notice and there is no assurance that such offering, upgrades, updates or functionality will become available unless and until they have been made generally available to our customers.
Blackboard & CaliperWork in Progress
Caliper Learning Analytics
• IMS Global as an organization
• Interoperability Standards – LTI, LIS, Common Cartridge, Extended Transcript, Caliper
• Create a viable tool ecosystem for .edu that avoids vendor lock-in
• LTI focuses on launch semantics and SSO, includes grade post-backs
• Caliper focuses on learning event interoperability, to get away from the data silo problem
Activity Stream: Service Architecture
Old School Enterprise approach
Record activity in the transactional database and run reports directly, or execute an expensive nightly ETL
Activity Stream: Service Architecture
Cloud approach
Send atomic events to a separately elastic telemetry service dedicated to collection and rollup
Activity Stream: Event Instrumentation
• More events, with better context, reducing ”noise”
• Compliant with & extends initial IMS Caliper standard
• Available to Learn SaaS deployments and planned for Enterprise 9.1 if hosted with Blackboard
• Exploring ways around technical constraints for self-hosted customers
The Goal
Integrated insights for teachers into what students are doing across tools and experiences
Usage data first, progress data to come
So what does that really mean?A look behind the curtain
Caliper Metric Profiles
• Learn SaaS was the first LMS certified against the Caliper 1.0 specification in October 2015, for 5 metric profiles: Session, Assessment, AssessmentItem, Assignable, Outcome
• Other metric profiles exist in the spec, specific to types of tools (e.g. MediaEvent, ReaderEvent)
The goal is to encapsulate events & context in a shared semantic model so that there is consistency in what can be expected in each event
Caliper Event Syntax
• JSON-LD (JSON for Linked Data)
• Defines a context and the relationships between involved entities (e.g between a user and an assessment)
• Transport-compliant custom events (e.g ForumEvent)
• Extensions allow BB to include extra specific data in related entities
Example Stream Event Syntax
{"@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type": "http://purl.imsglobal.org/caliper/v1/AssignableEvent", "actor":{ …Person…}, "action": "http://purl.imsglobal.org/vocab/caliper/v1/action#Completed", "object":{ …AssignableDigitalResource…}, "generated”:{ … Attempt…}, "eventTime":"2016-06-02T15:25:47.815Z", "edApp":{ …SoftwareApplication…}, "group":{ …CourseOffering…}, "membership":{…Membership…}, "federatedSession": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/sessions/21622015C7998FEA7A7824F053D514C8"}
Example Stream Event Syntax
{"@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type": "http://purl.imsglobal.org/caliper/v1/AssignableEvent", "actor":{ …Person…}, "action": "http://purl.imsglobal.org/vocab/caliper/v1/action#Completed", "object":{ …AssignableDigitalResource…}, "generated”:{ … Attempt…}, "eventTime":"2016-06-02T15:25:47.815Z", "edApp":{ …SoftwareApplication…}, "group":{ …CourseOffering…}, "membership":{…Membership…}, "federatedSession": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/sessions/21622015C7998FEA7A7824F053D514C8"}
{ "@id": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/users/fd6adc5678514c25bd2569c266c9bf65", "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type":"http://purl.imsglobal.org/caliper/v1/lis/Person", "extensions":{ "bb:user.id":"_50_1", "bb:user.externalId":"demo-39" },}
Example Stream Event Syntax
{"@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type": "http://purl.imsglobal.org/caliper/v1/AssignableEvent", "actor":{ …Person…}, "action": "http://purl.imsglobal.org/vocab/caliper/v1/action#Completed", "object":{ …AssignableDigitalResource…}, "generated”:{ … Attempt…}, "eventTime":"2016-06-02T15:25:47.815Z", "edApp":{ …SoftwareApplication…}, "group":{ …CourseOffering…}, "membership":{…Membership…}, "federatedSession": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/sessions/21622015C7998FEA7A7824F053D514C8"}
{ "@id": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/gradableItems/_7219_1", "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type”: "http://purl.imsglobal.org/caliper/v1/AssignableDigitalResource", "name": ”Chapter 1 Test", "dateToSubmit":"2016-07-28T13:04:50.963Z", "maxAttempts":0, "maxSubmits":0, "maxScore":25}
Example Stream Event Syntax
{"@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type": "http://purl.imsglobal.org/caliper/v1/AssignableEvent", "actor":{ …Person…}, "action": "http://purl.imsglobal.org/vocab/caliper/v1/action#Completed", "object":{ …AssignableDigitalResource…}, "generated”:{ … Attempt…}, "eventTime":"2016-06-02T15:25:47.815Z", "edApp":{ …SoftwareApplication…}, "group":{ …CourseOffering…}, "membership":{…Membership…}, "federatedSession": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/sessions/21622015C7998FEA7A7824F053D514C8"}
{ "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@id": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/gradableItems/_7219_1/attempts/_13000_1", "@type":"http://purl.imsglobal.org/caliper/v1/Attempt", "assignable": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/gradableItems/_7219_1", "actor":"https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/users/fd6adc5678514c25bd2569c266c9bf65", "count":1, "startedAtTime":"2016-06-02T15:25:47.815Z",}
Example Stream Event Syntax
{"@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type": "http://purl.imsglobal.org/caliper/v1/AssignableEvent", "actor":{ …Person…}, "action": "http://purl.imsglobal.org/vocab/caliper/v1/action#Completed", "object":{ …AssignableDigitalResource…}, "generated”:{ … Attempt…}, "eventTime":"2016-06-02T15:25:47.815Z", "edApp":{ …SoftwareApplication…}, "group":{ …CourseOffering…}, "membership":{…Membership…}, "federatedSession": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/sessions/21622015C7998FEA7A7824F053D514C8"}
{ "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@id": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/applications/learn", "@type": "http://purl.imsglobal.org/caliper/v1/SoftwareApplication”}
Example Stream Event Syntax
{"@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type": "http://purl.imsglobal.org/caliper/v1/AssignableEvent", "actor":{ …Person…}, "action": "http://purl.imsglobal.org/vocab/caliper/v1/action#Completed", "object":{ …AssignableDigitalResource…}, "generated”:{ … Attempt…}, "eventTime":"2016-06-02T15:25:47.815Z", "edApp":{ …SoftwareApplication…}, "group":{ …CourseOffering…}, "membership":{…Membership…}, "federatedSession": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/sessions/21622015C7998FEA7A7824F053D514C8"}
{ "@id": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/courses/f85ce1b9dfc740968fd91a38d4b20817", "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type": "http://purl.imsglobal.org/caliper/v1/lis/CourseOffering", "extensions":{ "bb:course.id":"_2972_1" }, "courseNumber":"GAT-1",}
Example Stream Event Syntax
{"@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type": "http://purl.imsglobal.org/caliper/v1/AssignableEvent", "actor":{ …Person…}, "action": "http://purl.imsglobal.org/vocab/caliper/v1/action#Completed", "object":{ …AssignableDigitalResource…}, "generated”:{ … Attempt…}, "eventTime":"2016-06-02T15:25:47.815Z", "edApp":{ …SoftwareApplication…}, "group":{ …CourseOffering…}, "membership":{…Membership…}, "federatedSession": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/sessions/21622015C7998FEA7A7824F053D514C8"}
{ "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@id": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/courses/f85ce1b9dfc740968fd91a38d4b20817/members/fd6adc5678514c25bd2569c266c9bf65", "@type":"http://purl.imsglobal.org/caliper/v1/lis/Membership", "extensions":{ "bb:user.externalId":"demo-39", "bb:user.id":"_50_1", "bb:course.id":"_2972_1", "bb:course.externalId":"GAT-1" }, "member": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/users/fd6adc5678514c25bd2569c266c9bf65", "organization": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/courses/f85ce1b9dfc740968fd91a38d4b20817", "roles": [http://purl.imsglobal.org/vocab/lis/v2/membership#Learner], "status":"http://purl.imsglobal.org/vocab/lis/v2/status#Active"}
How do we join the ecosystem?
Configuring Learn SaaS to relay events to an external Caliper record store
Enabling the Caliper “handshake” between Learn and an LTI Tool
New bb-manifest.xml permission block<permission type="blackboard.platform.blti.CaliperProfileCallbackPermission" name="*" actions="*"/></permissions>
Event Instrumentation
If your system already has event instrumentation:
• Map existing event instrumentation to Caliper metric profiles• Transform events to be compliant with the Caliper standard event syntax
If you do not currently record events:
• Determine what events are meaningful• Investigate the available Caliper Metric Profiles
Caliper Sensor Implementations
Standard implementations of the Caliper Sensor are available on GitHub:
• Java: https://github.com/IMSGlobal/caliper-java-public• Node.js: https://github.com/IMSGlobal/caliper-js-public• PHP: https://github.com/IMSGlobal/caliper-php-public• Python: https://github.com/IMSGlobal/caliper-python-public• Ruby: https://github.com/IMSGlobal/caliper-ruby-public• .NET: https://github.com/IMSGlobal/caliper-net-public
xAPI & the future
There is a similar specification in the world of SCORM – Blackboard is working with both IMS & ADL to drive towards a convergence that we hope will make it easier on everyone in the ecosystem.
Other goals for Blackboard & Caliper
• Support for Caliper 1.1 Discussion profiles
• Certification as a record store when that is available
• Convergence or “ambidexterity”• Support for 9.1 Enterprise releases
Questions?
[email protected] Product Development @ Blackboard
[email protected] Product Manager for Analytics @ Blackboard