version 2 to do upgrade logos to high res add in “demo” slides add in “backup” slides
TRANSCRIPT
![Page 1: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/1.jpg)
VERSION 2TO DO
• Upgrade logos to high res
• Add in “demo” slides
• Add in “backup” slides
![Page 2: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/2.jpg)
Richard Baraniuk Founder and DirectorDaniel Williamson Managing DirectorDavid Harris Editor-in-ChiefKathi Fletcher Product Manager
OpenStax Courseware
![Page 3: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/3.jpg)
GOALS of OpenStax Courseware
1. broader access to high-quality courseware2. new tools to improve learning experiences
(machine learning, cognitive science models)3. validation in real classrooms + research
![Page 4: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/4.jpg)
1. broader access to high-quality courseware2. new tools to improve learning experiences
(machine learning, cognitive science models)3. validation in real classrooms + research
![Page 5: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/5.jpg)
today’s agenda
• OpenStax Courseware building blocks– digital content: Connexions + OpenStax College– digital assessment: OpenStax Tutor– cognitive science– machine learning
• key technology components
• content focus areas
• go-to-market strategy
• discussion and closing statement
![Page 6: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/6.jpg)
digital content• open ed publishing platform
established in 1999• 25000 learning objects
in 40 languages• millions of users per month
• library of 25 free and open college textbooks
• professionally authored and peer reviewed
• 875 adoptions, saving 140,000 students over $14M
![Page 7: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/7.jpg)
digital assessment
• in use at 12 colleges(Rice, Georgia Tech, Duke, UT El Paso, …)
• built-in research infrastructure
• integrated cognitive science principles(collaborators at Duke, UT-Austin, WashU)
• flexible platform for computer-based assessment and research
![Page 8: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/8.jpg)
learning principles
retrieval practice– retrieving information from memory
is not a neutral event; rather it changes memory
spacing– distributing practice over time produces
better long-term retention than massing practice
feedback– closes the learning feedback loop– must be timely
learn
erscontent
data
![Page 9: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/9.jpg)
digital assessment
• experiment at Rice 2012
• findings: Students using cognitive science principles in OST scored ½-1 GPA point better than those using standard practice homework
• flexible platform for computer-based assessment and research
![Page 10: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/10.jpg)
learninganalytics content
analytics
![Page 11: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/11.jpg)
learning/content analytics
classical approach – “knowledge engineering”– domain experts pore over content, assessments,
data, tagging and building rules– fragile, expensive, not scalable, not transferable
modern approach – “machine learning”– learn directly from data– automatic– robust, inexpensive, scalable, transferable
![Page 12: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/12.jpg)
standard practice
Johnny
Eve
Patty
Neelsh
Nora
Nicholas
Barbara
Agnes
Vivek
Bob
Fernando
Sarah
Hillary
JudyJanet
![Page 13: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/13.jpg)
standard practice
Johnny
Eve
Patty
Neelsh
Nora
Nicholas
Barbara
Agnes
Vivek
Bob
Fernando
Sarah
Hillary
JudyJanet
Goal: using only “grade book” data, infer:
1. the concepts underlying the questions (content analytics)
2. each student’s “knowledge” of each underlying concept (learning analytics)
![Page 14: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/14.jpg)
students
pro
ble
ms
sparse factor analysis
• Goal: using only “grade book” data
white: correct responseblack: incorrect responsegrey: unobserved
infer:
1. the concepts underlying the questions (content analytics)
2. each student’s “knowledge” of each underlying concept (learning analytics)
![Page 15: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/15.jpg)
students
pro
ble
ms
+students
concepts
each problem involves a combination of a small number of key “concepts”
each student’s knowledge of each “concept”
each problem’s intrinsic “difficulty”
~ Ber
Sparse Factor Analysis
![Page 16: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/16.jpg)
questions(w/ estimated inherent difficulty)
concepts
studentknowledge
profile
87
55
23
93
62
Patty
![Page 17: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/17.jpg)
![Page 18: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/18.jpg)
DEMO SLIDES
![Page 19: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/19.jpg)
technology architecture
![Page 20: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/20.jpg)
content
• Principles of AccountingPrinciples of ManagementAmerican GovernmentMicrobiology
• content development in partnership with Words & Numbers (9 texts published to date, X in production)
• quality control via extensive peer review and classroom testing at partner colleges
• ROOM FOR ONE MORE BULLET
1.65 million students/year
![Page 21: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/21.jpg)
go-to-market strategy• research partners will co-develop
– Salt Lake Community College, University of Georgia
• pilot partners will field test– The Ohio State University, Auburn University, University System
of Georgia-Online Courses, Central New Mexico College, South Florida State College, Maricopa CC District, Tarrant County CC
• key elements– fit into existing faculty/student workflow– build an ecosystem of affiliate partners– execute advertising and marketing campaigns– employ viral new media approaches– employ direct marketing and customer relationship
management system
• proven success 2012-2014
![Page 22: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/22.jpg)
driving adoption: workflow principles
• Scope/sequence: content is available in complete discrete units of 100/200 level courses
• flexible: we provide for a blended learning experience
• part of the grade: assignable with metrics given
• APIs: interoperable across multiple platforms
![Page 23: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/23.jpg)
access to drive adoption
• Institutional partners and pilots1. Salt Lake Community College2. University of Georgia3. Georgia University System4. Auburn University5. College of South Florida6. The Ohio State University7. Mariposa Community College District8. Shasta College District
• Large Base of OpenStax College Adopters1. Approaching 1,000 adoptions2. Over 135,000 student seats
![Page 24: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/24.jpg)
Lumen Learning LogoCCOER Logo
Students and Faculty
Administrators and Faculty
Ecosystem Partners
![Page 25: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/25.jpg)
summary – 1 What makes your proposed courseware “exemplary?”
– strong research base in machine learning, cog science
– 15 years of experience in digital education network of administrators/educators who already
use our content/tools
– once proven, can expand at minimal cost into comprehensive library of highest enrollment college courses
– strong backing of Rice University
– Not sure about this one: Flexible: works in multiple modes to meet various workflow requirements
![Page 26: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/26.jpg)
summary – 2
How will your proposed courseware enable a “great leap forward” in improving the learning outcomes for low income, disadvantaged learners?
– platform integrates cognitive science principles that have been proven to improve knowledge retention and transfer large literature of laboratory studies Rice 2013 experiment
– machine learning learning/content analytics scale across courses dramatically lower cost/prices will result as compared to
courseware based on hand-coded ontologies
![Page 27: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/27.jpg)
summary – 3
Why do you believe your team can develop your proposed courseware?
– experienced team has built
OpenStax College 140,000 students in 2 yearsPhysics textbook displacing market leaders
OpenStax Tutor 50 years of experience at IBM, Microsoft,JP Morgan Chase, Northrop Grumman,Texas Instruments, Cengage, Pearson, …
machine learning 20 years of research in Rice DSP group
– $72M in research, development, and deployment funding from 15 foundations and government agencies
![Page 28: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/28.jpg)
summary – 4
How will you achieve wide adoption of your proposed courseware?
– proven go-to-market strategy most successful launch of a physics text
in 30 years (17.5% market share in 2 years)
– WHAT ELSE (remember this is a summary)
![Page 29: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/29.jpg)
summary – 5 How does this project align to the charitable purpose set forth by the Foundation?
– Gates Foundation “guiding principles of Global Access”– The technology and products developed with grant funds be made
available and accessible at an affordable price to people most in need
Because of the relatively low cost (due to scalability enabled by machine learning), we can sustainably make OpenStax Courseware affordable to those most in need
– Knowledge and information gained from the project be promptly and broadly disseminated
As a university project dissemination is core to our mission; we have already published a number of papers in machine learning and cog sci
![Page 30: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/30.jpg)
closing statement
![Page 31: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/31.jpg)
curriculum(re)design
personalizedlearning pathways
cognitive science research
machine learning
cycles ofinnovation
![Page 32: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/32.jpg)
closingAndrew Carnegie: “personalized courseware library” of the future
• Eric j appeal to carnegie – this is the “personalized courseware library” of the future
![Page 33: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/33.jpg)
backup slides
![Page 34: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/34.jpg)
backup slides
• More on tech• More on ecosystem/marketing?
![Page 35: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/35.jpg)
budget
• Overview of the $5M budget and key categories
![Page 36: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/36.jpg)
privacy
![Page 37: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/37.jpg)
sparfa
![Page 38: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/38.jpg)
from grades to concepts
students
pro
ble
ms
data– graded student responses
to unlabeled questions– large matrix with entries:
white: correct responseblack: incorrect responsegrey: unobserved
standard practice– instructor’s “grade book”
= sum/average over each column
goal– infer underlying concepts and
student understanding without question-level metadata
![Page 39: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/39.jpg)
students
pro
ble
ms
data– graded student responses
to unlabeled questions– large matrix with entries:
white: correct responseblack: incorrect responsegrey: unobserved
goal– infer underlying concepts and
student understanding without question-level metadata
key observation– each question involves only
a small number of “concepts” (low rank)
from grades to concepts
![Page 40: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/40.jpg)
students
pro
ble
ms
~ Ber
statistical model
converts to 0/1(probit or logisticcoin flip transformation)
estimate of each student’s ability to solve each problem(even unsolved problems)
red = strong ability
blue = weak ability
![Page 41: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/41.jpg)
students
pro
ble
ms
+
SPARse Factor Analysis
~ Ber
![Page 42: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/42.jpg)
students
pro
ble
ms
+students
concepts
SPARFA
each problem involves a combination of a small number of key “concepts”
each student’s knowledge of each “concept”
each problem’s intrinsic “difficulty”
~ Ber
![Page 43: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/43.jpg)
students
pro
ble
ms
solving SPARFA
factor analyzing the grade book matrix is a severely ill-posed problem
significant recent progress in relaxation-based optimization for sparse/low-rank problems
– matrix based methods (SPARFA-M)– Bayesian methods (SPARFA-B)
similar to compressive sensing
![Page 44: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/44.jpg)
standard practice
Johnny
Eve
Patty
Neelsh
Nora
Nicholas
Barbara
Agnes
Vivek
Bob
Fernando
Sarah
Hillary
JudyJanet
Grade 8 science
• 80 questions• 145 students• 1353 problems
solved (sparsely) • learned 5 concepts
![Page 45: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/45.jpg)
Grade 8 science
• 80 questions• 145 students• 1353 problems
solved (sparsely) • 5 concepts
![Page 46: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/46.jpg)
questions(w/ estimated inherent difficulty)
concepts
studentknowledge
profile
87
55
23
93
62
![Page 47: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/47.jpg)
marketing
![Page 48: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/48.jpg)
driving adoption: workflow principles
• Scope/sequence: content is available in complete discrete units of 100/200 level courses
• flexible: we provide for a blended learning experience
• part of the grade: assignable with metrics given
• APIs: interoperable across multiple platforms
![Page 49: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/49.jpg)
access to drive adoption
• Institutional partners and pilots1. Salt Lake Community College2. University of Georgia3. Georgia University System4. Auburn University5. College of South Florida6. The Ohio State University7. Mariposa Community College District8. Shasta College District
• Large Base of OpenStax College Adopters1. Approaching 1,000 adoptions2. Over 135,000 student seats
![Page 50: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/50.jpg)
Lumen Learning LogoCCOER Logo
Students and Faculty
Administrators and Faculty
Ecosystem Partners
![Page 51: VERSION 2 TO DO Upgrade logos to high res Add in “demo” slides Add in “backup” slides](https://reader030.vdocument.in/reader030/viewer/2022032707/56649e525503460f94b47ec6/html5/thumbnails/51.jpg)
charitable purpose
• mission of providing free access since 1999• free access for tutoring and review• anticipate a low cost, analytic driven, version
for classroom use, approx $10/student to maintain system