3 june 2014 university of zürich sef kloninger open edx at...

Open edX at StanfordOverview With A Few Technical Bits

Sef Kloninger3 June 2014 University of Zürich

Agenda: http://tinyurl.com/openedx-zurichSlides: http://tinyurl.com/openedx-zurich-sefShared Notes: http://tinyurl.com/openedx-zurich-shared-notes

or: https://etherpad.mozilla.org/eJQtMMcPRe

Today

● MOOCs at Stanford; why Open edX

● Engineering and Operations

● Looking Forward: Improving Community

Engineer

Manager

Internet Software● Infrastructure● Dev/Ops

me !me

Scientist / Researcher

Open Source Guru

Educational Software

MOOCs At Stanford

As of Winter 2014, more than 145 Stanford faculty from all seven schools

171 of 246 offerings are distinct course offerings or course components; 75 are repeated courses or course components

94 of 171 distinct offerings were flipped or blended classes on campus, and 51 were free to the public as MOOCs. The remainder (26) were course components, Continuing / Professional Education or other specialized audiences.

More than 1.9 million people have registered for one or more free public online courses taught by Stanford faculty offered through NovoEd, Coursera, Class2Go (Stanford’s initial platform), or Stanford OpenEdX.

Since the fall of 2012, more than 4 million hours of interaction.

Platform Strategy“Control Our Destiny” via multiple platforms● Too important to outsource to a vendor● Our brand; no limitations on terms (use, sharing...)

Open Source● It’s about the content, not platform. Get it pervasive.● Not just more resources, it’s about more use cases.

Faculty choose the platform that suits them best

Where We Started

JUNE2013

FeaturefulGreat video experience

Rich assessments

Peer Evaluation

ExpressiveBalance between standard layouts and customization

Studio is a good option for non-nerds

Stable XML format

Extendable

External Graders

JSInput●

Frontend● xBlocks● “JS Input” (!)● LTI

Backend● External Graders

Engineering

Features We’ve Built1. Shibboleth integration2. Chat for on-campus courses3. Shopping cart / Cybersource

payments for paid courses (CME, edx.org)

4. Bulk email5. Authoring tool improvements (e.

g. view this unit in Studio, check all captions)

6. Basic analytics (metrics tab)7. Theming8. Targeted feedback

9. Option shuffling10. LTI 2.0 (multiple submissions)11. Send anonymized user_id to

external tools (e.g. Qualtrics)12. Time delay between problem set

attempts13. Assist with new peer assessment

system14. Incremental cert generation *15. Unauthenticated, deep linking *16. Stanford-specific checklist *

Be Careful Of The Fork!

How we manage ● Very easy to quickly build up tech debt● Keep tests working

Release path● Best: dev => master PR => stanford● Usual: dev => stanford => master PR

A Word About Theming

“Stanford Theming” is kind of hacky● Templating: mako, includes● Explicit look-aside code

Microsites is nice but specific

Opportunity for cleanup new “null theme”

Stanford Roadmap1. Offline analytics pipeline2. Analytic reports (esp in-line)3. Better support for interactive exercises4. Hinting5. Semantic labelling6. Reporting tools7. Timed exams8. Authoring improvements, eg. bulk course-checking tools9. Update our Cybersource integration

10. Coursera -> edX XML converter11. Always: Small features for Stanford courses

Wish List1. On-campus LMS features: manual

grading, late days, registrar integration, gradebook

2. download/edit/upload flow3. Hosting for others4. Richer unauthenticated access: per-

item access control, read-only view for forum, embeddable content

5. Authenticated access for videos6. Targeted email, e.g. emailing students

who haven't submitted yet

Operations

1. Presentation LayerLoad Balancer● SSL termination● Session affinity

NginxApp processesSpecial case: shibboleth

Load Balancer

nginx

gunicornLMS (x8)

gunicornCMS (x4)

shib

S3

YouTube

memcache

Fat Processes● 200 MB per LMS● 150 MB per CMS● 150 MB for Shib

Mitigations● Nginx protections● Static offload

Oppty for Cleanup!

2. Services Layer

App

Workers do long-running tasks: mail, grading, reports, notifications, certs

Push or pull to graders

Not much SOA support

XqueueApp

RabbitMQ UtilUtil

Comment Service

App

Srch

UtilExtGrader

3. Data LayerMySQL & MongoDBRead-only replicas Hosted

New M/R Pipeline● Teachers want insights● Researchers want data

Tracking Logs

LMSCMS

M/R Pipeline

LMSCMS

OLAP DB Reports Data Dumps

Concept: In-Line Reporting

Our FlowReleases● Manual merge from master 2-4 weeks● Staging, privateprod● Deploy via Ansible

Operations● “Hotseat” duty● Other services: Pingdom, Datadog

Service Use Requirements

EC2 Compute

RDS Replicated MySQL M/S, backups, read replica

(MongoHQ) Replicated MongoDB M/S, backups

S3 Objects, logs Cheap, serve direct

EMR Map/Reduce on Logs, DB Bursty

ELB Terminate SSL, session affinity

ElastiCache Memcache

SES SMTP Large fanout (MOOC)

Our Amazon Soup

Many others: Access Control (IAM), DNS (Route53), Monitoring & Alerting (CloudWatch), Network Security (VPC)

Careful, You Might Need...

Deploy tooling Jenkins for ops and CI

Test databases Test environments

Monitoring Secrets management

Log management Content delivery

Asset management Hostname mgmt

Costs: Dedicated Platform Team

● Engineering Manager, Product Manager, Tech Support (2), Engineers (5)

● Half time on ops, “keeping current”, tactical support issues

Does not count other Stanford groups, e.g. instructional design, production

Costs: Amazon, etc.

Class2Go Open edX

Cost Breakdown

Community

Open edX AdoptersUse Case Need

Contributors Extend features Dev tools and support, community

Researchers Experiments A/B/N, API’s, raw data(some core changes?)

Operators On prem, data separation Install & ops tools, community

Extenders Modules, mobile clients API’s, dev/test environment, docs

Teachers Content and brand Hosted solution, reports & insights

StudyBreadth of interviewsBenchmark against other projectsConcrete recommendations

http://bit.ly/stanford-openedx-report

Governance

● Clarify and communicate the mission of Open edX

● Establish clear guidelines for contributors● Expand governance to involve

community in technical and product decisions

RecommendationsCommunity Building

● Hire a full time Open edX community manager● Establish, measure, and communicate Key

Performance Indicators (KPIs)● Create forums to engage platform

users (developers, hosting providers, researchers), e.g. user group meetings and office hours

Technical Improvements

● Open up the development process: public wiki’s, public bug tracking● Move to 2-4 stable releases per year: release notes, upgrade scripts, improved packaging

and testing● Provide more ways to extend and modify the platform without having to change the core: content interfaces and API’

s● Improve the Open edX documentation● Create a more informative website targeted at platform adopters● Establish an ecosystem of commercial vendors and hosting providers

Thank You!