3 june 2014 university of zürich sef kloninger open edx at...
TRANSCRIPT
Open edX at StanfordOverview With A Few Technical Bits
Sef Kloninger3 June 2014 University of Zürich
Agenda: http://tinyurl.com/openedx-zurichSlides: http://tinyurl.com/openedx-zurich-sefShared Notes: http://tinyurl.com/openedx-zurich-shared-notes
or: https://etherpad.mozilla.org/eJQtMMcPRe
Today
● MOOCs at Stanford; why Open edX
● Engineering and Operations
● Looking Forward: Improving Community
Engineer
Manager
Internet Software● Infrastructure● Dev/Ops
me !me
Scientist / Researcher
Open Source Guru
Educational Software
MOOCs At Stanford
As of Winter 2014, more than 145 Stanford faculty from all seven schools
171 of 246 offerings are distinct course offerings or course components; 75 are repeated courses or course components
94 of 171 distinct offerings were flipped or blended classes on campus, and 51 were free to the public as MOOCs. The remainder (26) were course components, Continuing / Professional Education or other specialized audiences.
More than 1.9 million people have registered for one or more free public online courses taught by Stanford faculty offered through NovoEd, Coursera, Class2Go (Stanford’s initial platform), or Stanford OpenEdX.
Since the fall of 2012, more than 4 million hours of interaction.
Platform Strategy“Control Our Destiny” via multiple platforms● Too important to outsource to a vendor● Our brand; no limitations on terms (use, sharing...)
Open Source● It’s about the content, not platform. Get it pervasive.● Not just more resources, it’s about more use cases.
Faculty choose the platform that suits them best
Where We Started
JUNE2013
FeaturefulGreat video experience
Rich assessments
Peer Evaluation
ExpressiveBalance between standard layouts and customization
Studio is a good option for non-nerds
Stable XML format
Extendable
External Graders
JSInput●
Frontend● xBlocks● “JS Input” (!)● LTI
Backend● External Graders
Engineering
Features We’ve Built1. Shibboleth integration2. Chat for on-campus courses3. Shopping cart / Cybersource
payments for paid courses (CME, edx.org)
4. Bulk email5. Authoring tool improvements (e.
g. view this unit in Studio, check all captions)
6. Basic analytics (metrics tab)7. Theming8. Targeted feedback
9. Option shuffling10. LTI 2.0 (multiple submissions)11. Send anonymized user_id to
external tools (e.g. Qualtrics)12. Time delay between problem set
attempts13. Assist with new peer assessment
system14. Incremental cert generation *15. Unauthenticated, deep linking *16. Stanford-specific checklist *
Be Careful Of The Fork!
How we manage ● Very easy to quickly build up tech debt● Keep tests working
Release path● Best: dev => master PR => stanford● Usual: dev => stanford => master PR
A Word About Theming
“Stanford Theming” is kind of hacky● Templating: mako, includes● Explicit look-aside code
Microsites is nice but specific
Opportunity for cleanup new “null theme”
Stanford Roadmap1. Offline analytics pipeline2. Analytic reports (esp in-line)3. Better support for interactive exercises4. Hinting5. Semantic labelling6. Reporting tools7. Timed exams8. Authoring improvements, eg. bulk course-checking tools9. Update our Cybersource integration
10. Coursera -> edX XML converter11. Always: Small features for Stanford courses
Wish List1. On-campus LMS features: manual
grading, late days, registrar integration, gradebook
2. download/edit/upload flow3. Hosting for others4. Richer unauthenticated access: per-
item access control, read-only view for forum, embeddable content
5. Authenticated access for videos6. Targeted email, e.g. emailing students
who haven't submitted yet
Operations
1. Presentation LayerLoad Balancer● SSL termination● Session affinity
NginxApp processesSpecial case: shibboleth
Load Balancer
nginx
gunicornLMS (x8)
gunicornCMS (x4)
shib
S3
YouTube
memcache
Fat Processes● 200 MB per LMS● 150 MB per CMS● 150 MB for Shib
Mitigations● Nginx protections● Static offload
Oppty for Cleanup!
2. Services Layer
App
Workers do long-running tasks: mail, grading, reports, notifications, certs
Push or pull to graders
Not much SOA support
XqueueApp
RabbitMQ UtilUtil
Comment Service
App
Srch
UtilExtGrader
3. Data LayerMySQL & MongoDBRead-only replicas Hosted
New M/R Pipeline● Teachers want insights● Researchers want data
Tracking Logs
LMSCMS
M/R Pipeline
LMSCMS
OLAP DB Reports Data Dumps
Concept: In-Line Reporting
Our FlowReleases● Manual merge from master 2-4 weeks● Staging, privateprod● Deploy via Ansible
Operations● “Hotseat” duty● Other services: Pingdom, Datadog
Service Use Requirements
EC2 Compute
RDS Replicated MySQL M/S, backups, read replica
(MongoHQ) Replicated MongoDB M/S, backups
S3 Objects, logs Cheap, serve direct
EMR Map/Reduce on Logs, DB Bursty
ELB Terminate SSL, session affinity
ElastiCache Memcache
SES SMTP Large fanout (MOOC)
Our Amazon Soup
Many others: Access Control (IAM), DNS (Route53), Monitoring & Alerting (CloudWatch), Network Security (VPC)
Careful, You Might Need...
Deploy tooling Jenkins for ops and CI
Test databases Test environments
Monitoring Secrets management
Log management Content delivery
Asset management Hostname mgmt
Costs: Dedicated Platform Team
● Engineering Manager, Product Manager, Tech Support (2), Engineers (5)
● Half time on ops, “keeping current”, tactical support issues
Does not count other Stanford groups, e.g. instructional design, production
Costs: Amazon, etc.
Class2Go Open edX
Cost Breakdown
Community
Open edX AdoptersUse Case Need
Contributors Extend features Dev tools and support, community
Researchers Experiments A/B/N, API’s, raw data(some core changes?)
Operators On prem, data separation Install & ops tools, community
Extenders Modules, mobile clients API’s, dev/test environment, docs
Teachers Content and brand Hosted solution, reports & insights
StudyBreadth of interviewsBenchmark against other projectsConcrete recommendations
http://bit.ly/stanford-openedx-report
Governance
● Clarify and communicate the mission of Open edX
● Establish clear guidelines for contributors● Expand governance to involve
community in technical and product decisions
RecommendationsCommunity Building
● Hire a full time Open edX community manager● Establish, measure, and communicate Key
Performance Indicators (KPIs)● Create forums to engage platform
users (developers, hosting providers, researchers), e.g. user group meetings and office hours
Technical Improvements
● Open up the development process: public wiki’s, public bug tracking● Move to 2-4 stable releases per year: release notes, upgrade scripts, improved packaging
and testing● Provide more ways to extend and modify the platform without having to change the core: content interfaces and API’
s● Improve the Open edX documentation● Create a more informative website targeted at platform adopters● Establish an ecosystem of commercial vendors and hosting providers
Thank You!