ecen631 1 - class introductionece631web.groups.et.byu.net/lectures/ecen631 1 - class...
TRANSCRIPT
1/5/12
Copyright©2012 Robo3c Vision Lab Brigham Young University
Class Introduction
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Course Description
Textbook
Topics
Class Format
Grading
Projects
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Not an image processing or computer vision course
Emphasis on real-time performance Emphasis on engineering aspect Review mathematical concepts Algorithm study, review, and implementation Assignments, team projects, team semester
project, final report, and exams
Course Overview
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
• Machine Vision & Applications
• 3-D Vision
Geometry
Motion
• Human vision is nature and seems easy
• Single view or multiple views
• Camera model, image processing, geometry
• Image noise removal, feature detection, feature matching, reconstruction, and applications
Course Description
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
• Machine Vision – Theory, Algorithms, & Applications • Compute 3-D information from one or multiple views in
real time with hardware or software implementation. • Study methods of using passive sensors for – Stereo vision – Motion analysis to achieve – 3-D information extraction – Static obstacle avoidance – Moving obstacle avoidance – Short-range Motion Estimation – Long-range Motion Estimation – Object recognition – Object localization
Course Objectives
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Machine Vision E.R. Davies
Elsevier, 2005, 3rd Edition
Introductory Techniques for 3-D Computer Vision Emanuele Trucco & Alessandro Verri
Prentice Hall, 1998
An Invitation to 3-D Vision - From Images to Geometric Models
Yi Ma, Stefano Soatto, Jana Kosecka, Shankar Sastry Springer, 2004
References
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
References
Multiple View Geometry in Computer Vision Richard Hartley & Andrew Zisserman
Cambridge University Press, 2003
An Introduction to 3D Computer Vision Techniques and Algorithms
Boguslaw Cyganek & J. Paul Siebert Wiley, 2009
3D Computer Vision: Efficient Methods and
Applications Christian Wšhler
Springer-Verlag, 2009
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Introduction to Machine Vision & 3-D Vision Introduction Theory, Algorithms, & Applications Features and Feature Detection Edge, Corner, Line, & Color Segmentation Imaging and Image Representation Imaging Devices Image Digitization Digital Image Properties Problems in Digital Images CCD vs. CMOS Camera Model & Calibration Camera Model and Geometry 3D Transformation Perspective Transformation Matrix Camera Calibration Camera Calibration Method
Topics
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Stereo Vision Introduction Two Cameras – Stereo & Geometry Epipolar Geometry Stereo Correspondence Other Methods Motion Introduction Optical Flow Differential Method Feature-based Kalman filters
Topics
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Image Acquisition
Noise Attenuation
Feature Extraction Differential Motion Analysis
Calibration
Recognition
Feature-Based Stereo
Intensity-Based Stereo
Features Optical Flow Shape from
Single Image
Feature-Based Motion Analysis
Optical Flow Analysis
3-D Motion 3-D Structure Object Localization
Object Identification
Camera/System Parameters
Image
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Vision Algorithm Implementation
System on a Chip using FPGA Real-time vision processing Visual C++, OpenCV
“A rough, quickly calculated motion estimation is arguably more useful for robotic vision than a more
accurate, but slowly calculated estimate”
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Class Format
• January-February: 2 lectures a week & project discussion
• March: 2 lectures a week, literature review, progress report, presentation
• April: semester project
final report: conference proceeding format
demonstration and presentation
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Assignments (30%): 6 assignments
Three team projects:
Machine vision inspection (5%)
Tennis ball catcher (15%)
Structure from Motion (10%)
One midterm exam (10%): in-class closed-book exam
One final exam (10%): in-class closed-book exam (04/17 7:00AM)
Semester Project (20%): quality (10%) demonstration & presentation (5%), final report (5%)
Grading
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
>=95: A 90: A- 85: B+ 80: B 75: B- 65: C <65:E
Historically: A(1/2), A-(1/4), B+ or below (1/4)
Grading
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Assignments
OpenCV
Feature detection
Feature tracking
Camera calibration & distortion correction
Stereo calibration & rectification
Optical flow & time to impact
Motion field & structure from motion
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Tennis Ball Catcher
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Tennis Ball Catcher
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Tennis Ball Catcher
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
2~3 per team 25 shots 3 points each catch Presenta3on 25 points (evaluated by the class)
Presenta3on include algorithm, calibra3on, trajectory es3ma3on, etc.
Tennis Ball Catcher
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Semester Project
Preferably 3‐D Camera live input Real‐3me performance Real‐3me response Quality of project (10%) Demonstra3on and presenta3on 5% (evaluated by the class) Presenta3on in technical conference format Final report 5% (technical paper format)
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Past projects – Rock, paper, scissors – Small ground robot – obstacle avoidance – Stereo vision – Gaze‐direc3on Input device for the PC – Es3ma3on of op3mal landing area – Structure from Mo3on
Semester Project
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Schedule Six assignments: follow the lecture schedule Semester project proposal and presenta3on: 01/31 Real‐3me visual inspec3on project: 02/16 First exam: 03/06 Tennis ball catcher demonstra3on: 03/15 Tennis ball catcher presenta3on: 03/22 Structure from mo3on project: 03/29 Final exam: 04/17 Semester project demonstra3on and presenta3on: 04/05, 04/10
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Smart Vehicle
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Vision-Guided Mobile Robot
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Gaze-direction Input device
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
3-D Face Model
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Rock, paper, scissors
50 100 150 200 250 300 350 400 450 500
50
100
150
200
250
300
350
400
100 150 200 250 300 350 400 450
50
100
150
200
250
300
350
40050 100 150 200 250 300 350 400 450
100
150
200
250
300
50 100 150 200 250 300 350 400 450 500 550
50
100
150
200
250
300
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Vision-guided Path Planning
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Threat Assessment
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Stereo Vision
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
3-D Modeling
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Structure from Motion
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Structure from Motion
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Optical Flow
3-D Pose Estimation
Roll = 0.3° Pitch = 10.8° Yaw = -36.4°
X Trans. = 1.3m Y Trans. = -0.7m Z Trans. = 3.6m
(0,0) (1,0)
(0,1)
(1,1)
• Calculate the Pose of aircraft by solving the Orthogonal Procrustes Problem. • Pose Estimation Scheme will be implemented on a Gumstix Embedded System
allowing for on-board vision processing.
1/5/12
Copyright©2012 Robo3c Vision Lab Brigham Young University
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Light Saber
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Body Tracking
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Magic Mirror
1/5/12 Copyright©2012 Robo3c Vision Lab Brigham Young University
Finger Paint