ist-1999-29017 – omniviews omnidirectional visual system final review september 27-28, 2001 lisbon...

IST-1999-29017 – OMNIVIEWSIST-1999-29017 – OMNIVIEWSOmnidirectional Visual SystemOmnidirectional Visual System

Final Review

September 27-28, 2001

Lisbon

Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open scheme - Assessment ProjectFET-Open scheme - Assessment Project


Agenda of the meetingAgenda of the meetingFirst DayFirst Day

Agenda of the meetingAgenda of the meetingFirst DayFirst Day

14:30 Reviewers’ private meeting15:00 Welcome Jose' Santos-Victor15:10 Goal of Review Pekka Karp15:20 Omniviews Main Achievements Giulio Sandini16:00 Coffee Break16:30 Mirror design principles Branislav Micusik16:50 Mirror design tools Jose' Santos-Victor, Claudia Decco17:10 Demos Introduction17:20 Surveillance Demo Tomas Pajdla, Alex Bernardino18:00 Transmission Demo Pedro Soares, Giulio Sandini18:30 End of first Day 20:30 Dinner

14:30 Reviewers’ private meeting15:00 Welcome Jose' Santos-Victor15:10 Goal of Review Pekka Karp15:20 Omniviews Main Achievements Giulio Sandini16:00 Coffee Break16:30 Mirror design principles Branislav Micusik16:50 Mirror design tools Jose' Santos-Victor, Claudia Decco17:10 Demos Introduction17:20 Surveillance Demo Tomas Pajdla, Alex Bernardino18:00 Transmission Demo Pedro Soares, Giulio Sandini18:30 End of first Day 20:30 Dinner



Agenda of the meetingAgenda of the meetingSecond DaySecond Day

Agenda of the meetingAgenda of the meetingSecond DaySecond Day

9:00 Navigation Demo José Santos-Victor 9:40 Future Outlook and General Discussion Giulio Sandini10:10 Reviewer’s Private Meeting (with coffee)11:10 Preliminary Evaluation Report11:40 End of meeting

9:00 Navigation Demo José Santos-Victor 9:40 Future Outlook and General Discussion Giulio Sandini10:10 Reviewer’s Private Meeting (with coffee)11:10 Preliminary Evaluation Report11:40 End of meeting



Main FactsMain FactsMain FactsMain Facts

Consortium:DIST - University of Genova - GenovaCMP - Czech Technical University in PragueVISLAB - Instituto Superior Técnico - Lisbon

Consortium:DIST - University of Genova - GenovaCMP - Czech Technical University in PragueVISLAB - Instituto Superior Técnico - Lisbon

Project Start and Duration:September 1st 2000 – One year

Project Start and Duration:September 1st 2000 – One year

Funding: 100 K€Funding: 100 K€

Open-scheme, Assessment Phase ProjectOpen-scheme, Assessment Phase Project



Project’s Main ObjectiveProject’s Main ObjectiveProject’s Main ObjectiveProject’s Main Objective

The main objective of the project is to integrate optical, hardware, and software technology for the realization of a smart visual sensor, and to demonstrate its utility in key application areas.

In particular our intention is to design and realize a low-cost, digital camera acquiring panoramic (360°) images and performing a useful low-level processing on the incoming stream of images.

The main objective of the project is to integrate optical, hardware, and software technology for the realization of a smart visual sensor, and to demonstrate its utility in key application areas.

In particular our intention is to design and realize a low-cost, digital camera acquiring panoramic (360°) images and performing a useful low-level processing on the incoming stream of images.



Key TechnologiesKey TechnologiesKey TechnologiesKey Technologies

Retina-like visual sensorRetina-like visual sensor

Omnidirectional MirrorsOmnidirectional Mirrors



Specific Objectives of Specific Objectives of Assessment PhaseAssessment Phase

Specific Objectives of Specific Objectives of Assessment PhaseAssessment Phase

Define the optimal profile of a mirror matching a retina-like visual sensor. Optimal in the sense that direct read-out of panoramic images is obtained.

Demonstrate its utility in key application areas If successful present a follow-up proposal

Define the optimal profile of a mirror matching a retina-like visual sensor. Optimal in the sense that direct read-out of panoramic images is obtained.

Demonstrate its utility in key application areas If successful present a follow-up proposal



MethodologiesMethodologiesMethodologiesMethodologies

1. Use the currently available SVAVISCA camera for initial experiments

2. Design and simulate mirror using SVAVISCA camera

3. Realize the OMNIVIEWS mirror for the current sensor

4. Demonstrate the mirror in key applications

1. Use the currently available SVAVISCA camera for initial experiments

2. Design and simulate mirror using SVAVISCA camera

3. Realize the OMNIVIEWS mirror for the current sensor

4. Demonstrate the mirror in key applications

Simulated image:SVAVISCA cameraHyperbolic mirror

Simulated image:SVAVISCA cameraHyperbolic mirror

SVAVISCAPixel layout

SVAVISCAPixel layout



Mirror’s design principleMirror’s design principleMirror’s design principleMirror’s design principle

1. Uniform Cylindrical Projection

2. Direct read-out through log-polar mapping

1. Uniform Cylindrical Projection

2. Direct read-out through log-polar mapping



OMNIVIEWS MirrorOMNIVIEWS MirrorOMNIVIEWS MirrorOMNIVIEWS Mirror

Mirror’s ProfileMirror’s Profile

Experimental Set-up and test imagesExperimental Set-up and test images



Assessment CriteriaAssessment CriteriaAssessment CriteriaAssessment Criteria

1. Direct read-out of panoramic images

2. Frame rate

3. Resolution and layout of the sensor

4. Mirror profile and size

5. Lens characteristics

6. Camera cost

7. Image quality

1. Direct read-out of panoramic images

2. Frame rate

3. Resolution and layout of the sensor

4. Mirror profile and size

5. Lens characteristics

6. Camera cost

7. Image quality



AC1: Direct read-out of AC1: Direct read-out of panoramic imagespanoramic images

AC1: Direct read-out of AC1: Direct read-out of panoramic imagespanoramic images

Direct read-out from OMNIVIEWS:About 30,000 read-out operations

Direct read-out from OMNIVIEWS:About 30,000 read-out operations

Image Obtained from a conventional camera:About 1.8 M operations required:882,000 read-outs (30 times more)882,000 additions30,000 divisions

Image Obtained from a conventional camera:About 1.8 M operations required:882,000 read-outs (30 times more)882,000 additions30,000 divisions



AC2: Frame rateAC2: Frame rateAC2: Frame rateAC2: Frame rate

Currently the maximum read-out frequency is fixed by the camera’s interface (PC Parallel port) limiting the frame rate to about 12 frames/s.

More than 25 frames/s is achievable with a faster interface (e.g. USB or PCMCIA)

Currently the maximum read-out frequency is fixed by the camera’s interface (PC Parallel port) limiting the frame rate to about 12 frames/s.

More than 25 frames/s is achievable with a faster interface (e.g. USB or PCMCIA)



AC3: Resolution and Layout of AC3: Resolution and Layout of the sensorthe sensor

AC3: Resolution and Layout of AC3: Resolution and Layout of the sensorthe sensor



AC4: Mirror Profile and SizeAC4: Mirror Profile and SizeAC4: Mirror Profile and SizeAC4: Mirror Profile and Size

Mirror profile and size meets the original plan (6 cm.).Mirror profile and size meets the original plan (6 cm.).

Furthermore:1) New technology for mirror realization2) Mirror’s design tool of general utility3) Design and realization of “mixed-mirror”4) Overall size can be reduced

Furthermore:1) New technology for mirror realization2) Mirror’s design tool of general utility3) Design and realization of “mixed-mirror”4) Overall size can be reduced



AC5: Lens characteristicsAC5: Lens characteristicsAC5: Lens characteristicsAC5: Lens characteristics

Standard C-Mount lenses have been used for all experiments and demos

Standard C-Mount lenses have been used for all experiments and demos

No difficulties in principle are envisaged for the design of smaller size lenses (possibly including the mirror).

No difficulties in principle are envisaged for the design of smaller size lenses (possibly including the mirror).



AC6: Camera CostsAC6: Camera CostsAC6: Camera CostsAC6: Camera Costs

Cost of obtaining panoramic images is zero in our case

Compared to conventional solutions no extra-cost for the mirror is required. Lower cost is possible with the new “glass-based” technology.

The cost of the sensor is equivalent to the cost of conventional sensors realized with the same technology and with the same size.

Cost of obtaining panoramic images is zero in our case

Compared to conventional solutions no extra-cost for the mirror is required. Lower cost is possible with the new “glass-based” technology.

The cost of the sensor is equivalent to the cost of conventional sensors realized with the same technology and with the same size.



AC7: Image qualityAC7: Image qualityAC7: Image qualityAC7: Image quality

Topology of images meets the quality criteria Evaluation of numerical approximations Three demonstrations:

1) Surveillance (two parts)2) Navigation3) Image Transmission

Further processing experiments:1) Localization using Agam fiducials2) 3D reconstruction

Topology of images meets the quality criteria Evaluation of numerical approximations Three demonstrations:

1) Surveillance (two parts)2) Navigation3) Image Transmission

Further processing experiments:1) Localization using Agam fiducials2) 3D reconstruction

The project will be successful if we demonstrate that it is possible to create virtual images by simple reading out the pixels from the proposed sensor and to use such images in the aimed applications ….

The project will be successful if we demonstrate that it is possible to create virtual images by simple reading out the pixels from the proposed sensor and to use such images in the aimed applications ….



Additional remarksAdditional remarksAdditional remarksAdditional remarks

1. Software mirror-design tools have been developed

2. New kind of mirror have been proposed extending the original plan (i.e. the “mixed mirror”)

3. 10 scientific papers have been published

4. Plans for the future are clearer.

1. Software mirror-design tools have been developed

2. New kind of mirror have been proposed extending the original plan (i.e. the “mixed mirror”)

3. 10 scientific papers have been published

4. Plans for the future are clearer.

ist-1999-29017 – omniviews omnidirectional visual system final review september 27-28, 2001 lisbon...

Documents

year project

assessment phase project

end of meeting slide

reviewers private meeting

welcomejose santosvictor

lisbon slide

emerging technologies

future outlook