From Virtual Classroom to the Virtual Streets

Emmanuel Velasco

City College of New York

Capstone 1

Fall 2004

CCNY Virtual Classroom Objectives

Automatic Data Collection Intelligent Post-Processing Natural Interaction between students,

teachers and computers.

Automatic Data Collection

Mimio Virtual Ink

Omnidirectional Camera

PowerPoint Slide Capture

Automatic Data Collection

PowerPoint Slide Capture The start date and time is added when

recording, to synchronize the recorded PowerPoint slides with the whiteboard pages.

Mimio Virtual Ink Everything the instructor writes on the

whiteboard is captured by the Mimio hardware and software.

Automatic Data Collection

Omnidirectional Lens Captures a 360° view of the classroom. Captures both video and audio.

Intelligent Post-Processing

Video Processing Media Synchronization Connecting Powerpoint Slides and

Whiteboard Images Connecting Powerpoint Slides and

Instructor Images

Intelligent Post-Processing

Video Processing 360° field of view is converted to a panoramic

view. (currently cylindrical panoramic)

Intelligent Post-Processing

Media Synchronization The PowerPoint slides, whiteboard pages, and

video/audio are synchronized using the time information in the PowerPoint log files, whiteboard log files, and the video/audio streams.

Intelligent Post-Processing

Connecting PowerPoint Slides and Whiteboard Images

To connect the PowerPoint slides, the instructor marks the four corners of the PowerPoint projection on the whiteboard. Since they lie on the same plane, we use a projective transformation to connect the two.

When a portable presentation device may be moved, a video camera is used for dynamic registration of the two sources.

Intelligent Post-Processing

Connecting PowerPoint Slides and Instructor Images

Actual video of the instructor is merged to the digital PowerPoint slides. The instructor is extracted from the video using computer vision techniques, and then merged to the slides.

Gives the students the relation between the instructor and the visual aids used.

Natural Interaction

Virtualized Classroom Presentation System• VCPS Creator• VCPS Player

Natural Interaction

Virtualized Classroom Presentation Creator The Virtual Classroom Presentation Creator is fully

customizable. The user can decide which presentation components should be included.

Natural Interaction

Virtualized Classroom Presentation Player The Virtual Classroom Presentation Player

allows the user to load the presentation file. The user can access the Table of Contents by right-clicking the mouse.

How Can We Take It To The Streets

Automatic Data Collection Intelligent Post-Processing Natural Interaction between traffic

management and the traffic

Traffic: Automatic Data Collection

Currently:

Traffic: Automatic Data Collection

By using an omnidirectional camera, we can get live traffic view in all directions.

Pros: 360° point of view. More information is available from one source.

Cons: Requires physical installation of camera. Pictures may have some distortion.

Traffic: Intelligent Post-Processing

After data is collected, we should be able to determine a busy street or highway. Using background subtraction, we should be able to obtain similar results.

heavytraffic

lighttraffic

Traffic: Natural Interaction

Like the Virtualized Classroom Presentation Player, traffic management should be able to easily view traffic information that is received.

Table of Contents(possibly show mapand camera locations)

New York City Map

Camera View

From Virtual Classroom to the Virtual Streets

Emmanuel Velasco

City College of New York

Capstone 1

Fall 2004