embedded software for video wayne wolf princeton university and mediaworks technology
Post on 22-Dec-2015
228 views
TRANSCRIPT
Outline
Initial observations.Sample application: smart cameras.Levels of abstraction in software:
process; data; instructions.
Lessons.
Characteristics of embedded computing
Functional and non-functional requirements: performance, power, cost.
Some aspects of design methodology are similar to hardware design.
Embedded software delivers more complex functions than HW models.
Will often be delivered in chips---that’s where the money is.
Ozer et al: human activity recognition
JPEG IMAGE/MPEG VIDEO
UncompressedIMAGE/VIDEO
EIGENSPACE MATCHING
PRINCIPALCOMPONENTS
RELATIONALGRAPH MATCHING
Motion Vectors
DCTExtracted Area
Extracted Area
Pixel Values
OBJECT DETECTIONOBJECT DETECTION
OBJECT DETECTIONOBJECT DETECTION
Previous work
Watlington et al.: Dataflow architecture for real-time video processing.
Wandell et al.: Programmable sensor array.
Foote et al.: Panaromic camera system.
Nicolescu et al.: Algorithms to pan, tilt, zoom array of cameras.
MIT Media Lab.: Smart rooms for several applications.
Keck Lab.: Multiple cameras on networked PCs.
Our hardware platform
TriMedia BoardTM32 (VLIW)
PC
I Bu
s Host PC
CameraNTSC Shared
Memory
TriMedia BoardTM32 (VLIW)
SharedMemory
CameraNTSC
Background elimination
Skin color detection
contour following
Superellipse fitting
graph matching
Background elimination
Skin color detection
Contour following
Superellipse fitting
Graph matching
Characteristics of current video computing
Streaming data (with control).
A wide range of types of algorithms.
Data volum
eA
lgorithm com
plexity
Challenges in embedded video software
Video is more than pushing pixels.Programs run a long time---harder to
verify.Must derive programs through many
stages from functional to real-time.Must effectively use a deep memory
hierarchy: registers through off-chip.
Characteristics of next-generation video systems
Distributed computation and networking: Distributed analysis of video. Data fusion.
Design methodology for this system
Algorithm design: Matlab.C implementation on existing
platform.Software optimization.Design of custom platform and
software.
Levels of abstraction and optimization
Instruction: opcode selection, register allocation.
Data: loop transformations, data layout, buffer optimization.
Module: inlining, outlining.Process: decomposition,
communication, etc.
Questions from smart camera
Instructions: Which loops are predictable and which
are not?Data:
How does data layout change at each stage?
How do we buffer and lay out data?Process:
How do we manage multiple processes?
View from a start-up
Standards body software sucks.Configurable CPUs: offer some
advantages, but harder to characterize.Embedded OS verification: WinCE, Linux.Must manage multiple versions of
software.Can’t buy many useful components.Verification, verification, verification.