"what's new in tools for vision application design and development?," a presentation...
TRANSCRIPT
Copyright © 2014 BDTI 1
Jeff Bier, President, BDTI
May 29, 2014
What’s New in Tools for Vision
Application Design and Development?
Copyright © 2014 BDTI 2
About BDTI
• BDTI provides:
• Product development engineering services
• Emphasis on optimization for performance, cost and power
• Benchmarking and evaluation
• For technology selection, feasibility studies, competitive
analysis and proof points
• Focused on:
• Algorithm-intensive applications: vision, video, audio, wireless
• Embedded processors, tools and techniques
“BDTI’s deliverables were orders of magnitude better than other
vendors’. BDTI has set a new standard for quality.”
– Group Program Manager, Fortune 500 Systems & Software Co.
See our fun vision demos in the Technology Showcase!
Copyright © 2014 BDTI 3
The Right Tool for the Job
Concept Proof of
Concept Prototype Product
Priority on rapid experimentation Priority on efficient implementation
Which tools you use, and how you use them, depends on where you
are in the product development process
Copyright © 2014 BDTI 4
Fundamental Trade-offs
Breadth: Applications,
Algorithms, Data Types
Maturity,
Quality
Portability Across
Processor Types,
Brands, Models
Developer
Productivity
Copyright © 2014 BDTI 5
• OpenCV is a popular open source computer vision software component
library
• Supports Windows, Linux, Android, iOS
• Advantages:
• Large library; free
• Easy to use; good abstraction from details
• Open source with liberal (BSD) license
• Large, active community
• Disadvantages:
• Not all algorithms are best-in-class
• Not a standard
• OpenCV 3.0 release due in June
• Most significant enhancement: automatic acceleration via OpenCL
• Complements existing optimizations for CUDA, ARM NEON, x86
OpenCV 3.0
Copyright © 2014 BDTI 6
• A Khronos standard API and language for implementing data-parallel
kernels
• Provides portability across different types of platforms (GPU, FPGA,
other)
• Most efficient when code is customized for a specific target
platform
• OpenCL 2.0 (spec released in November) moves towards Heterogeneous
System Architecture (HSA)
• Expands the utility of OpenCL for applications requiring CPU-GPU
interaction and task-based parallelism
• Key features:
• Shared Virtual Memory (SVM)
• Dynamic Parallelism
• Generic Address Space
OpenCL 2.0
Copyright © 2014 BDTI 7
• Strengths:
• Portability
• Beginning to see strong traction in industry and academia
• Limitations/Weaknesses:
• Single Program, Multiple Data paradigm can be difficult for
developers
• And not all algorithms naturally exhibit data parallelism
• Relative immaturity compared to other parallel programming
frameworks or APIs (e.g. CUDA)
• Vendor-specific extensions can provide valuable functionality but break
portability
• E.g. FPGA extensions allow threads to communicate across
workgroups
OpenCL Strengths and Weaknesses
Copyright © 2014 BDTI 8
• The HSA Foundation is developing a standards-based unifying framework
for heterogeneous computing
• Goal: “Write once, run anywhere for heterogeneous systems”
• Encompasses system architecture spec and software stack
• A unified memory space and programming methodology for
programming CPU and associated compute engines (GPU, DSP, FPGA, …)
• Initiated by AMD, which has been most active in implementing it
• Overlaps with OpenCL; OpenCL can be implemented on top of HSA
• Developers will likely use HSA indirectly via
optimized OpenCL drivers
• Breadth and speed of adoption unclear
Heterogeneous System Architecture (HSA)
Foundation
Copyright © 2014 BDTI 9
• An emerging Khronos standard API providing a
vision hardware acceleration (abstraction) layer
• Focus on enabling real-time vision on mobile and
embedded systems
• For use by high-level libraries or apps
• Promises to relax the trade-off between
portability and efficiency
• Targets support for heterogeneous systems,
including CPU, GPU, DSP, FPGA, etc.
• Developer specifies data flow graph of vision
algorithm kernels
• OpenVX framework partitions the graph
among processing engines, handles
memory transfers, etc.
• Provisional specification released November 2013
• Breadth and speed of adoption TBD
OpenVX
Open source sample implementation?
Hardware vendor implementations
OpenCV open source library
Other higher-level CV libraries
Application
Copyright © 2014 BDTI 10
• Not a new concept (see, e.g., TI’s VLIB, released in 2008), but the
number and scope of offerings is increasing rapidly
• Multiple levels of abstraction:
• Building-block function libraries (e.g., HOG, optical flow)
• NVIDIA VisionWorks Toolkit for Tegra K1
• CEVA-CV library for CEVA-MM3101 IP processor core
• Qualcomm FastCV: Supports all ARM-based application
processors; accelerated using co-processors on Qualcomm chips
• Various accelerated OpenCV subsets (ARM, AMD, …)
• Function-specific
• SoftKinetic iisu for gesture recognition
• Application-domain-specific
• Qualcomm Vuforia for mobile augmented reality
• LabView Vision for machine vision
Optimized Vision Software Libraries for
Specific Processors
Copyright © 2014 BDTI 11
• Vision-oriented development boards and kits are increasingly
available for embedded processors; many at very low prices
• Avnet FinBoard (BF609)
• NVIDIA Jetson (TK1)
• Xilinx Zynq-7000 SoC Video and Imaging Kit
• Qualcomm DragonBoard (SnapDragon)
• Enable getting a sense for the capabilities of the
processor and its tools
• Facilitate early processor-aware algorithm and software design
• Enable testing algorithms in real-world conditions
• Often used for prototyping, and sometimes initial production
Development Kits
Copyright © 2014 BDTI 12
• Many tools of many kinds; they’re all moving (some quickly)
• Choosing tools has big implications for programming style, choice of
processor, efficiency of implementation, and other important factors.
• Ideally, think about processor choice early, so initial development
on a PC can use tools and techniques that make it easier to
implement on the target processor
• Choice of tools has big implications for portability and can lock you into
a hardware platform
• Source code is key for libraries: enables learning, debugging,
customization
• With many developers tackling vision for the first time, easy on-ramps
are very valuable:
• Development kits, application examples, tool flow tutorials, etc.
Lessons Learned
Copyright © 2014 BDTI 13
• Tools and libraries that relax the trade-offs between productivity,
portability, and flexibility
• Tools for exploring, evaluating, and
selecting algorithm options
• More complete, higher-quality libraries
• More complete reference designs
• Better guidance for developers on tool selection (and co-selection of
tools, algorithms and processors)
Wish List
Copyright © 2014 BDTI 14
• "Tools for 'Democratizing' Computer Vision,” http://bit.ly/1tpD9ty
• “Start Developing OpenCV Applications Immediately Using the BDTI
Quick-Start OpenCV Kit,” http://bit.ly/1pmn8a4
• “Designing Visionary Mobile Apps Using the Tegra Android Development
Pack,” http://bit.ly/1jvwbgV
• “Getting Started With GPU-Accelerated Computer Vision Using OpenCV
and CUDA,” http://bit.ly/1oMwJEG
• “The open standard for parallel programming of heterogeneous
systems,” https://www.khronos.org/opencl/
• “Hardware acceleration API for Computer Vision applications and
libraries,” https://www.khronos.org/openvx
Resources