web lecture archiving, robotic tracking systems, and the...

Web Lecture Archiving,Web Lecture Archiving,Robotic Tracking Systems,Robotic Tracking Systems,

and the Lecture Objectand the Lecture Object

Jeremy HerrJeremy Herr

University of MichiganUniversity of Michigan

Shaping Collaboration 2006, GenevaShaping Collaboration 2006, Geneva

12 December 2006 2Jeremy Herr

Shaping Collaboration 2006, Geneva

What is a Web Lecture?What is a Web Lecture?

Media-rich presentation viewable by anyone in the world with:Media-rich presentation viewable by anyone in the world with: any web browserany web browser RealPlayer plug-inRealPlayer plug-in

Media streams:Media streams: lecturerlecturer’’s audios audio lecturerlecturer’’s video (low bandwidth)s video (low bandwidth) high-high-resres slide images slide images high-high-resres chalkboard images chalkboard images

FeaturesFeatures slide indexslide index ability to ability to ““jump aroundjump around”” platform independenceplatform independence low bandwidthlow bandwidth ability to evaluate usageability to evaluate usage



Web Lecture RecordingWeb Lecture Recording

Charles Severance wrote initial software in 1997Charles Severance wrote initial software in 1997 Michigan based WLAP Archive started recording CERNMichigan based WLAP Archive started recording CERN

Summer Student Lectures in 1999Summer Student Lectures in 1999 Our archive now has 1300+ lecturesOur archive now has 1300+ lectures Talks recorded from:Talks recorded from:

The ATLAS experimentThe ATLAS experiment CERN eventsCERN events University of Michigan EventsUniversity of Michigan Events U-M Saturday Morning PhysicsU-M Saturday Morning Physics American Physical Society (APS)American Physical Society (APS) IntInt’’l Conference on Systems Biology 2005 at Harvardl Conference on Systems Biology 2005 at Harvard MScribeMScribe classroom recordings classroom recordings



Recordings for ATLASRecordings for ATLAS

ATLAS is funding ATLAS is funding UMUM’’ss team to record several priority events team to record several priority eventsper year for the Collaboration.per year for the Collaboration. ATLAS Week Plenary SessionsATLAS Week Plenary Sessions Software TutorialsSoftware Tutorials WorkshopsWorkshops

All ATLAS Talks Recorded by UM-WLAP

0

100

200

300

400

500

600

Jan-03

Jan-04

Jan-05

Jan-06

Jan-07

ATLAS Week Plenary Sessions: 5+ minute sessions

0

10

20

30

40

50

60

70

80

4/6/2005

5/6/2005

6/6/2005

7/6/2005

8/6/2005

9/6/2005

10/6/2005

11/6/2005

12/6/2005

1/6/2006

2/6/2006

3/6/2006

4/6/2006

5/6/2006

6/6/2006

7/6/2006

8/6/2006

9/6/2006

10/6/2006

11/6/2006

Jun-05

Feb-06

Jul-06

Oct-06



MScribeMScribe Pilot Project Pilot Project

Ambitious new project to completely automateAmbitious new project to completely automatethe recording of classroom lecturesthe recording of classroom lectures

2006 Spring: tested system in Physics 140 lecture2006 Spring: tested system in Physics 140 lecture 2006 Fall: Pilot project began2006 Fall: Pilot project began

4 courses being recorded4 courses being recorded courses and instructors selected based oncourses and instructors selected based on

requirements of pedagogical researchersrequirements of pedagogical researchers media server, integrated with media server, integrated with CToolsCTools (Sakai) (Sakai) about 200 hours of talks recordedabout 200 hours of talks recorded



MScribeMScribe Recordings Fall 2006 Recordings Fall 2006

Entire courses (each 2 lectures/week)Entire courses (each 2 lectures/week) AMCULT 100: Rethinking American Culture (Judy AMCULT 100: Rethinking American Culture (Judy DaubenmierDaubenmier)) AMCULT 219: American Folklore (Bruce AMCULT 219: American Folklore (Bruce ConforthConforth)) HISTART 112: The History of Photography (Matt Biro)HISTART 112: The History of Photography (Matt Biro) PSYCH 111: Introduction to Psychology (Brian PSYCH 111: Introduction to Psychology (Brian MalleyMalley))

Selected sessions of coursesSelected sessions of courses STATS 350: Introduction to Statistics and Data Analysis (BrendaSTATS 350: Introduction to Statistics and Data Analysis (Brenda

Gunderson)Gunderson) ENVIRON 110: Introduction to Global Change I (Ben van ENVIRON 110: Introduction to Global Change I (Ben van derder PluijmPluijm))

Other eventsOther events School of Education CSHPE TalksSchool of Education CSHPE Talks Various Bioinformatics TalksVarious Bioinformatics Talks Saturday Morning PhysicsSaturday Morning Physics



MScribeMScribe System System

4 recording carts (roughly $10K each)4 recording carts (roughly $10K each) one one linuxlinux PC for media capture and tracking control PC for media capture and tracking control tracking camera systemtracking camera system high quality audio equipmenthigh quality audio equipment additional additional ““chalkboardchalkboard”” camera camera

Archive server (3 TB; each L.O. is about 2GB)Archive server (3 TB; each L.O. is about 2GB) Media serverMedia server Lecture Object processing serverLecture Object processing server iPodiPod processing server processing server Manpower:Manpower:

1 engineer (tracking research and development, cart design)1 engineer (tracking research and development, cart design) 1 web developer/manager (software development, schedule management, liaison1 web developer/manager (software development, schedule management, liaison

with University support staff)with University support staff) 2-3 part time work study students (pushing carts around, pressing START and2-3 part time work study students (pushing carts around, pressing START and

STOP, reporting problems)STOP, reporting problems)



MScribeMScribe Pilot Components Pilot Components

Automated recording, archiving, processing andAutomated recording, archiving, processing andpublication systemspublication systems

Integrated pedagogical studies to evaluate theIntegrated pedagogical studies to evaluate theimpact of this revolutionary technology onimpact of this revolutionary technology onclassroom learningclassroom learning

Dissemination:Dissemination: Sakai (Sakai (CToolsCTools))

iTunesUiTunesU podcastingpodcasting



Usage InformationUsage Information

MScribeMScribe pedagogical researchers are: pedagogical researchers are: conducting student surveys conducting student surveys evaluating demographic dataevaluating demographic data performing comparisons to server logperforming comparisons to server log

data.data.

1797

116

36 39

1422

10 8 10 9

71

0

40

80

120

nu

mb

er

of

sessio

ns

0% 20% 40% 60% 80% 100%

percent of lecture viewed

UM-WLAP: Geant4 Workshop

Histogram: percent of lecture viewed

UM-WLAP: Visits/week

0

50

100

150

200

250

300

350

400

450

6/1

7/0

1

7/1

7/0

1

8/1

7/0

1

9/1

7/0

1

10/1

7/0

1

11/1

7/0

1

12/1

7/0

1

1/1

7/0

2

2/1

7/0

2

visits

visits >

1 min

visits >

5 minserver down

Geant4 weblectures announced

network down

Christmas

server down

USA Today

article



Robotic TrackingRobotic Tracking

To scale up web lecture recording, all aspects of recording,To scale up web lecture recording, all aspects of recording,archiving and formatting must be automatedarchiving and formatting must be automated

Human camera operator must be replaced with a robotHuman camera operator must be replaced with a robot We received a $250,000 grant from the NSF National ScienceWe received a $250,000 grant from the NSF National Science

Digital Library in 2003 to begin work on tracking.Digital Library in 2003 to begin work on tracking. System RequirementsSystem Requirements

PortablePortable RobustRobust AffordableAffordable Runs without expert interventionRuns without expert intervention Little setup or calibrationLittle setup or calibration



Survey of Existing TrackingSurvey of Existing TrackingTechnologiesTechnologies

We have investigated and ruled out the followingWe have investigated and ruled out the followingtracking technologiestracking technologies

Very easily distracted by anyreflective objects in the room

Inexpensive (4000 USD),simple and very accurate

“passive” IRUsed in Boeing factories

Requires extensive set-up andcalibration

Almost reasonably priced(10,000 USD), more accuratethan RF

Ultra Wide Band (UWB)UbiSense

Slow and insufficient accuracy,especially indoors

Almost reasonably priced(6,000 USD)

Radio Frequency (RF)BlueSoft

Requires extensive permanentinstallation

Probably accurate enough fortracking a lecturer

UltrasonicDeveloped by AT&T labs

Very expensive, gets tired afterseveral hours

Intelligent tracking thatgenerates pleasing video

Human camera operator



Current Tracking SystemCurrent Tracking System

““ActiveActive”” Infrared Infrared necklace chain of bright IR LEDnecklace chain of bright IR LED’’ss CCD camera follows itCCD camera follows it PTZ commands sent to video cameraPTZ commands sent to video camera

This system satisfies our criteriaThis system satisfies our criteria Portable: sits on a cartPortable: sits on a cart Robust: simple design makes it very robustRobust: simple design makes it very robust Affordable: currently under 4 000 USDAffordable: currently under 4 000 USD No expert intervention: start it and it just worksNo expert intervention: start it and it just works Little setup: almost no calibration requiredLittle setup: almost no calibration required Accurate to within centimetersAccurate to within centimeters



Celebrate Invention 2005Celebrate Invention 2005

March 2005March 2005 Provisional patent filed, describing the current dual-camera with active IRProvisional patent filed, describing the current dual-camera with active IR

necklace.necklace. October 2005October 2005

University of Michigan researchers filed 287 invention disclosures in 2005University of Michigan researchers filed 287 invention disclosures in 2005 Eight of these were selected to be featured at the 2005 Tech Transfer InventorEight of these were selected to be featured at the 2005 Tech Transfer Inventor’’ss

Reception, as the most commercially promising and world-changing.Reception, as the most commercially promising and world-changing. Our tracking system was one of these eight.Our tracking system was one of these eight. Most importantly, Most importantly, I won an I won an iPodiPod!!

March 2006March 2006 Permanent patent filed.Permanent patent filed.



Current Tracking ResearchCurrent Tracking Research

CCD with synchronized pulsing LEDCCD with synchronized pulsing LED’’ss Quad detectorQuad detector Position Sensing Detector (PSD)Position Sensing Detector (PSD) Ultrasonic phase differenceUltrasonic phase difference RF phase arrayRF phase array



Lecture ObjectLecture Object

Originally proposed by Originally proposed by Giosue Vitaglione Giosue Vitaglione atatPontignano Pontignano 2000 conference2000 conference

A standardized data object containingA standardized data object containing high-resolution, open format media fileshigh-resolution, open format media files timing datatiming data standardized metadata (Dublin Core, IEEE LOM)standardized metadata (Dublin Core, IEEE LOM)

Designed forDesigned for LongevityLongevity Sharing among multiple institutionsSharing among multiple institutions Flexibility in viewing formatsFlexibility in viewing formats



Lecture ObjectLecture Object

Creation of the Lecture Object sets us apart from mostCreation of the Lecture Object sets us apart from mostother media recording other media recording activitesactivites, since it creates an, since it creates anadditional step in the Web Lecture production process,additional step in the Web Lecture production process,but it is the right way to do it.but it is the right way to do it.

mpeg-4

jpeg

XMLDescription

meta data

Lecture Object

Transformations



Users of Lecture ObjectUsers of Lecture Object

Using a simple, open, well-defined global archivalUsing a simple, open, well-defined global archivalstandard will:standard will: encourage multiple institutions to share their archivesencourage multiple institutions to share their archives enable shoe-string operations as well as well-funded groupsenable shoe-string operations as well as well-funded groups

to easily produce compatible contentto easily produce compatible content

Archives using UM-WLAP technology and LectureArchives using UM-WLAP technology and LectureObject maintained by:Object maintained by: University of Michigan WLAP / University of Michigan WLAP / MScribeMScribe CERNCERN American Physical Society (APS)American Physical Society (APS) FermilabFermilab



Lecture Object DevelopmentLecture Object Development

MScribeMScribe technical advisory committee is working on technical advisory committee is working onrefining the standardrefining the standard

We are collaborating with CERN in this development.We are collaborating with CERN in this development.Gregory Gregory FavreFavre (CERN IT) is working to make SMAC (CERN IT) is working to make SMACsupport it.support it.

Next versions of the Lecture Object will:Next versions of the Lecture Object will: support arbitrary numbers of streamssupport arbitrary numbers of streams support access control, authorization, copyrightsupport access control, authorization, copyright remain simple, minimal and easy to useremain simple, minimal and easy to use be targeted to be targeted to lectureslectures, not generalized , not generalized ““learning objectslearning objects””



BlueStreamBlueStreamA deeply disturbing image...



Advanced Indexing and SearchAdvanced Indexing and Search

BlueStreamBlueStream is an online environment at the University of is an online environment at the University ofMichigan with powerful features for using digital video, audio,Michigan with powerful features for using digital video, audio,images, and documents in higher education.images, and documents in higher education.

Each course recorded by Each course recorded by MScribeMScribe will have at least 30 hours of will have at least 30 hours ofrecorded video available to students.recorded video available to students.

Clearly, powerful search and indexing is needed.Clearly, powerful search and indexing is needed. BlueStreamBlueStream has tools that ingest video, images and metadata and has tools that ingest video, images and metadata and

transcodetranscode it to multiple formats it to multiple formats convert speech to text and index the video streamconvert speech to text and index the video stream perform OCR on the slide imagesperform OCR on the slide images provide search functions that take the user directly to a point in the videoprovide search functions that take the user directly to a point in the video

BlueStreamBlueStream also supports sophisticated access control schemes also supports sophisticated access control schemes



PeoplePeople

Homer A. NealHomer A. Neal Steven GoldfarbSteven Goldfarb Jeremy HerrJeremy Herr Jim Jim IrrerIrrer CangCang Ye Ye Mitch Mitch McLachlanMcLachlan Charles SeveranceCharles Severance



InfoInfo

http://www.wlap.orghttp://www.wlap.org http://www.umich.edu/~mscribehttp://www.umich.edu/~mscribe

web lecture archiving, robotic tracking systems, and the...

Documents