an introduction to real-time machine vision in mechatronics dr. onur toker
Post on 21-Dec-2015
236 Views
Preview:
TRANSCRIPT
Dr. Onur TOKER 2
Outline
• RT Machine Vision ? Mechatronics ?
• Review of previous experiments
• Image sensors (CMOS versus CCD)
• CMUCam, and cwCAM
• Interfacing a CCD camera to an 8-bit uC
• Difficulties in real-time machine vision
• Conclusion
Dr. Onur TOKER 3
RT Machine Vision ? Mechatronics ?
Machine vision is the ability of a computation machine to "see."
•Visual object tracking
•Object recognition
•Automated inspection, sorting
•Pattern recognition, etc.
RT: There is no strict real-time system. There are systems with very short event response latency times.
Dr. Onur TOKER 4
Experiment #1
• 1-D tracking system
• Analog video camera & PCI grabber
• VB 6 & VFW based
• Simple algorithm
• PID control
• Pentium 2/350MHz
Dr. Onur TOKER 5
Experiment #2
• Line following
• Wireless video camera and ToyCar
• Processing on a remote PC
• VC++ & DirectX based
• Simple algorithm
• Pentium 3/1GHz
Dr. Onur TOKER 6
Experiment #3
• Intel 8051
• Very primitive machine vision
• Rapid prototyping board
• LDR sensor
• MOSFET driver
Dr. Onur TOKER 8
BOE-BOT kit
Simple kit
PBASIC
Not very flexible
Very small RAM
IR LEDs &
photo transistors
Dr. Onur TOKER 10
Other demos
CMUCam demo
(Color tracking) WAM demo (MIT 1995)
(Tracking by stereo machine vision)
Dr. Onur TOKER 11
CMOS image sensors
A CMOS sensor (OV7620)
CUMCam uses such a sensor 2nd PCB has a Scenix
uC
Digital output
Easy to interface
DALSA CMOS Sensor
Dr. Onur TOKER 12
CCD image sensors
DALSA CCD Sensor
Analog output
Difficult to interface
Require several support chips
Dr. Onur TOKER 13
CMOS versus CCD
CMOS sensor
640x480 mode
CCD sensor
640x480 (NTSC output)
Under same lightning,
same distance,
comparable budget,
CCD image is better.
Dr. Onur TOKER 14
CMUCam architecture
CMOS sensor
SX28
uCuC/DSP
•“User device” issues high level commands
•SX28 does the processing (Limited built-in functions)
•SX28 replies
Serial I/O
CMUCam
Dr. Onur TOKER 15
What is wrong with CMUCam ?
• Serial I/O (Low bandwidth)• Low frame rate (Max. 17fps)• CMOS sensor• Processing done by SX28• Limited to built in functions• Not much flexibility• Instead of FPGA, uses SX28• Very compact design
Dr. Onur TOKER 16
Proposed cwCam architecture
CCD camera
Video ADC
FPGA
uC/DSP
uC/DSP
uC/DSP
•Co-operating windowing approach (Discussed later)
•Parallel processors
•Parallel application specific digital architectures in the FPGA
•ASIC CPU cores in FPGA
cwCam
Dr. Onur TOKER 20
Machine Vision with an Analog Industrial camera
• NTSC/30fps or PAL/25fps
• Even/odd field interlacing: 60fips/50fips rate
• 31ms VSYNC, 4.7us HSYNC for NTSC
• Needs a high speed ADC (AD9048 is 35 MHz)
• Most 8-bit uCs are too slow for this task
• Scenix SX28AC/DP 13.3 ns instruction cycle
• FPGA for accurate and high resolution capture
Dr. Onur TOKER 21
Scenix SX28AC/DP
• 13.3 ns instruction cycle (75MHz clock)
• 10MHz video sampling = 100 ns loop time
• 1 Branch=3 cycles
• 4 instruction loop OK, but int. RAM too small
• 8051 too slow !
• PIC16F877 too slow !
• USE AN FPGA !
Dr. Onur TOKER 23
Our ADC (AD9048)
•35MSPS, 8-bit Flash ADC, Bipolar, 550mW, DIP 28 available
•AD9203, 40MSPS, 10-bit,CMOS, 74mW, No DIP available
Actual photo of AD9048 used in our video digitizer
Dr. Onur TOKER 24
Cortex-I approach
•Bederson, 1992
•Logarithmic structured space variant pixel geometry
•Based on human vision system
•For real-time machine vision, reduce data to < 1500 pixels
Dr. Onur TOKER 25
Co-operating windowing (1)
•Nassif & Capson, 1997
•2 Watch windows (200x20)
•1 Peripheral window (40x40 … 200x200)
•1 Foveal window (20x20)
•Object tracking at 113Hz
Dr. Onur TOKER 27
Where we are at cwCAM ?
•AD9048 Video ADC board design completed (PCB layout !)
•AD9048 interfaced to 8051 prototyping board and tested
•Logic design is being done by Xilinx ISE software
•Mixed VHDL and graphical logic designs
•Tedious and long taskCCD camera
Video ADC
FPGA
cwCam
Dr. Onur TOKER 29
Conclusion
•Real time machine vision requires innovative use of software and hardware techniques.
•Cortex-I (Human Eye), Co-operating windowing, etc.
•Innovative use of FPGAs and uC/DSPs.
•High frame rate CCD sensors.
•Optimum designs likely to be an application specific one.
•cwCAM is based on co-operating windowing approach and innovative hardware/software techniques.
Dr. Onur TOKER 32
Prototyping / Final product
Prototyping board Serial download, EEPROM based, 9V battery
Final design EPROM based minimum size PCB
Dr. Onur TOKER 34
Image sensor types
1. Charged coupled devices (CCD)
2. Charge injection devices (CID)
3. CMOS Active Pixel Sensors (CMOS)
•They all convert incident light (photons) into electronic charge (electrons) by a photo-conversion process.
•Color sensors can be made by coating each individual pixel with a filter color (e.g. red, green, and blue).
•Beyond that point, everything is different.
top related