embedded vision on fpgas€¦ · – edge detection, median filter " conversions – chroma...
TRANSCRIPT
1 © 2015 The MathWorks, Inc.
Embedded Vision on FPGAs
2
Enhanced Edge Detection in MATLAB Test bench
Read Image from File
Add noise
Median Filter
Edge Detect
Video Display
Frame To
Pixel
Pixel To
Frame
Design
3
Computer Vision Ø Feature matching, and extraction Ø Object detection and recognition Ø Object Tracking and motion estimation Ø Dynamic resolution scaling Ø Focus assessment
Video Processing Ø Video in and out Ø Gamma correction Ø Color balancing Ø Noise removal Ø Image sharpening
Video Processing and Computer Vision
4
A Workflow for Video Image Processing
Concept Development
Algorithm Development Prototyping Architecture
design Prototyping Chip design
Image/Video Engineer
Pixel based
HW Engineer
Frame based
5
Vision HDL Toolbox, R2015a New Product
§ Customer Problems – Prototyping video and image processing algorithms on FPGA/SoC
§ Our Solution – Pixel-streaming algorithms for the design and implementation of vision systems on
FPGAs and ASICs – A diverse set of frame sizes and frame rates
§ Product Detail – Price: €5000/$5500 – Platforms: MATLAB and Simulink – Dependency: MATLAB for modeling, HDL Coder for code generation
6
Vision HDL Toolbox, Key Benefits
§ Modeling bit-true and cycle-accurate behavior of video image processing algorithms – Pixel-based functions and blocks – Optimized for HDL code generation – Conversion between frame and pixel – Standard and custom frame sizes and frame rates
§ Prototyping video and image processing algorithms on FPGA/SoC – (With HDL Coder) Efficient and readable HDL code – (With HDL Verifier) FPGA-in-the-loop testing and acceleration
7
Pixel Based Video Image Algorithms
§ Analysis & Enhancement – Edge Detection, Median Filter
§ Conversions – Chroma Resampling, Color-Space Converter – Demosaic Interpolator, Gamma Corrector, Look-up Table
§ Filter – Image Filter, Median Filter
§ Morphological Operations – Dilation, Erosion, – Opening, Closing
§ Statistics – Histogram – Image Statistics
§ I/O Interfaces – Frame to Pixels, Pixels to
Frame, FIL versions
§ Utilities – Pixel Control Bus Creator – Pixel Control Bus Selector
8
A broader solution for embedded vision
Product Capabilities Vision HDL Toolbox (“VHT”)
Design and simulate image processing, video, and computer vision systems for FPGAs and ASICs
HDL Coder Provide code generation capability for the functions and blocks in Vision HDL Toolbox
HDL Verifier Provide FPGA-in-the-loop capability for Vision HDL Toolbox
Computer Vision System Toolbox (“CVST”)
Provide frame based computer vision functions and blocks as well as image and video I/O capability
Image Processing Toolbox
Provide image processing and analysis functions
9
CVST and VHT, complimentary products Scopes
Vision algorithms Ø Feature matching, and extraction Ø Object detection and recognition Ø Object Tracking and motion estimation Ø Dynamic resolution scaling Ø Focus assessment
CVST
10
CVST and VHT, complimentary products Scopes
Camera pipeline Ø Video in and out Ø Gamma correction Ø Color balancing Ø Noise removal Ø Image sharpening
11
CVST and VHT, complimentary products Scopes
CVST
VHT 1.0 VHT future releases
Camera Pipeline
Vision Algorithm
image image information
12
CVST and VHT, complementary products Product details
VHT CVST Image Processing Pixel based Frame based Workflow System prototyping Algorithm design
Algorithms Image filtering Color space conversion Edge detection Statistics & histogram Morphological operations
(Most of the VHT algorithms, plus) Object detection & tracking Feature extraction and matching Stereo vision Camera calibration Image registration
I/O Frame-to-pixel Pixel-to-frame FIL
File I/O Video display
Code gen HDL (with HDL Coder) C (with ML Coder or SL Coder)
13
CVST and VHT, complementary products MBD workflow for embedded vision
MBD workflow 1. Build behavior model with CVST blocks and simulate 2. Build prototyping model with VHT blocks and simulate 3. Generate HDL code using HDL Coder 4. Run HDL code on FPGA and run testbench in SL (FIL)
14
CVST and VHT, complementary products MBD workflow for embedded vision
15
Enhanced Edge Detection: Simulink Demo
16
Streaming Pixel Interface Full Frame vs Pixel Stream
full-frame source
full-frame sink
à
à
pixel-stream processing
17
Streaming Pixel Interface Full Frame vs Pixel Stream
full-frame source
full-frame sink
à
à
pixel-stream processing
18
Streaming Pixel Interface Pixel control signals
pixelcontrol bus Ø hStart, hEnd, vStart, vEnd, and valid Ø Consistent block interface
19
Frame To Pixels and Pixels To Frame
20
Frame To Pixels and Pixels To Frame
21
Examples: Starting Points for Your Models
22
Image Filtering Keywords: Median Filter, Image Filter, PSNR
23
Gamma Correction Example Keywords: Gamma, PSNR
24
Histogram Equalization Keywords: Histogram, Linear Equalization, External Frame Delay
0 255 0 255
25
Edge Detection and Image Overlay Keywords: Sobel, Align Video, Alpha Mix, PSNR
26
Edge Detection with Impaired Frame Keywords: Sobel, Align Video, Alpha Mix, PSNR
27
Multi-Zone Metering Keywords: ROI, Image Statistics, Mean
Source Mask Selected ROI
28
Multi-Zone Metering Keywords: ROI, Image Statistics, Mean
Source Mask Selected ROI
29
Challenge Accelerate the implementation of advanced thermal imaging filters and algorithms on FPGA hardware Solution Use MATLAB to develop, simulate, and evaluate algorithms, and use HDL Coder to implement the best algorithms on FPGAs Results § Time from concept to field-testable prototype
reduced by 60% § Enhancements completed in hours, not weeks § Code reuse increased from zero to 30%
“With MATLAB and HDL Coder we are much more responsive to marketplace needs. We now embrace change, because we can take a new idea to a real-time-capable hardware prototype in just a few weeks. There is more joy in engineering, so we’ve increased job satisfaction as well as customer satisfaction.”
Nicholas Hogasten
FLIR Systems Link to user story
Raw image (left) and image after applying filter developed with HDL Coder (right).
FLIR Accelerates Development of Thermal Imaging FPGA
30
From Algorithm to FPGA Implementation
Implement Design
Map
Place & Route
Synthesis
Verification
Static Timing Analysis
Timing Simulation
Functional Simulation
Back Annotation
HDL Verifier QuestaSim
HDL Co-Simulation
HDL Coder
RTL Creation
Behavioral Simulation
MATLAB® and Simulink®
Algorithm and System Design Model Refinement for Hardware
HDL Verifier
FPGA-in-the-Loop
31
A complete solution
Frame based
Concept Development
Algorithm Development Prototyping Architecture
design Prototyping Chip design
Pixel based
MATLAB
Vision HDL Toolbox
MATLAB Coder
HDL Coder
Fixed Pt Designer HDL Verifier
Image Processing
Toolbox
Computer Vision System Toolbox
32
ROI: Customer Adoption Of Model-Based Design Time spent on FPGA implementation
§ Shorter implementation time by 48% (total project 33%) § Reduced FPGA prototype development schedule by 47% § Shorter design iteration cycle by 80% 1st FPGA Prototype 2nd FPGA Prototype
1st FPGA Prototype