high performance video processing for military...
Post on 07-Mar-2018
212 Views
Preview:
TRANSCRIPT
1
High Performance Video Processing for Military Applications
2
MotionDSP – introduction
• Sean Varah, Founder/CEO MotionDSP• San Mateo software company, founded 2005• Leading developer of real‐time image processing
software• Customers include US Air Force, US Secret Service,
Navy NCIS, Scotland Yard, DOD/IC customers
3
Video in the US Air Force
• Market Size– Systems: The US DoD expects to spend $64 BILLION on unmanned
systems by 20211
– Analysts: The US Air Force has 5000 people devoted to “ISR” data analysis and dissemination. Add to that: Navy, Army, DIA, CIA, NGA, NRO . . .
• 3‐vector expansion in video– More missions, more vehicles, more cameras– 330% growth in Combat Air Patrols of UAVs since 2007– Growing number UAVs and ISR vehicles – Number of cameras per vehicle going up 10x
1) Strategy Analytics
4
The Problem
Lieutenant General David DeptulaFormer Deputy Chief of Staff for ISR, Headquarters, US Air Force“We’re going to find ourselves in the not too distant future swimming in sensors and drowning in data”
General Norton A. SchwartzAir Force’s chief of staff “If automation can provide a cue for our people that would make better use of their time, that would help us significantly,” NY Times.
5
$64 Billion of Systems
The following slides are excerpts from a presentation by Gen Paul Dettmer, Asst Deputy Chief of Staff for Intelligence, Surveillance, and
Reconnaissance (US Air Force)
66
NanoNavigate / communicate
inside buildings
MicroClose-in reconnaissance & situational awareness
Man-portable- ISR
- Time-Sensitive- Lethal
Family of Expendables- Close-In ISR- Expendable Jammers- Lethal- Counter Air- Precision Clandestine Resupply- Cyber attack
“SUAS Family of Transformers”- Personal ISR- Lethal- SIGINT- Cyber/EW- Counter-UAV- AutoSentries
Multi-Mission- ISR
- Force protection- FID
Air-Launched- Close-in ISR
- Lethal- SIGINT/DF
Irregular Warfare
Anti-Access Support
Increasing across all mission sets
Tier II Joint- ISR- Comm Relay- Lethal- SIGINT
Bio-Mechanicals- Indoor Reconnaissance
- Indoor Lethal- Indoor Comm- Cyber attack
Next Gen Multi-Mission- ISR- Communications Relay- Lethal / Non-lethal- Electronic/Cyber Attack/SEAD- SIGINT/Low Altitude Pseudo-Sats- = New Mission areas
Now Future
Nano
Raven B
Wasp III
Switchblade SUASTechnical Demonstration
Scan Eagle
FinderSUAS
Voyeur SUASTechnical Demonstration
Artist Conception
Future AL-SUAS
Bio-Mechanicals- Indoor Reconnaissance- Indoor Lethal/Non-lethal- Indoor Comm- Cyber attack- Swarming
Lite Machine’sConceptual SUAS
GT AeroConceptual Bandit SUAS
6
AF RPA Flight Plan:Small “Family of Systems”
7
Specialized ISR
CollectionDissemination
ISR
Comm Relay
EW
CAS
AAR-TSEAD
Air InterdictionAeromedical EvacuationPersonnel Recovery
Mod
ular
Pay
load
s
MQ-Xa
7
MQ-1- Collection – SIGINT/FMV
- CAS
MQ-9- Collection – SIGINT/FMV
- CASMQ-9- Collection
- SIGINT/FMV- Wide Area Airborne ISR- SAR/GMTI
- CAS
MQ-Xb
MQ-Xc
Now Future
Specialized ISR
CollectionDissemination
ISR
Comm Relay
EW
AAR-R
Mod
ular
Pay
load
s
CAS
AAR-R&T
DisseminationSpecialized ISR
CAS
Collection
EW/ISR
Comm Relay
SEADAir Interdiction
Counter AirCSAR
Strategic Attack
Mod
ular
Pay
load
s
Missile Defense
AF RPA Flight Plan:Medium Systems
8
Ground Moving Target IndicatorInformation Integration
ISR
Airborne Moving Target Indicator
EW
Airlift
Command and Control
AAR-R & TAirlift
Humanitarian AssistanceStrategic Attack
Global StrikeCAS
Air Mobility
Mod
ular
Pay
load
s8
MQ-Lc
RQ-4 (Blk 20/30/40)- Collection:
- Block 20- Enh SAR- Enh EO/IR
- Block 30- Adv SIGINT
- Block 40- MP-RTIP Radar- GMTI and concurrent SAR- High Range Resolution- No EO/IR or SIGINT
MQ-La
RQ-4 (Blk10)- Collection – ISR
- Basic SAR- Basic EO/IR
MQ-Lb
Ground Moving Target IndicatorInformation Integration
ISR
Airborne Moving Target Indicator
EW
AAR-RMod
ular
Pay
load
s
Command and Control
Now Future
Ground Moving Target IndicatorInformation Integration
ISR
Airborne Moving Target Indicator
EW
Humanitarian Assistance
Command and Control
AAR-R & TAirliftM
odul
ar P
aylo
ads
AF RPA Flight Plan:Large Systems
9
10
11
The increasing number of cameras on each platform
12
13
Where the video goes
Analysts
14
15
16
Challenges for analysis
17
• Analysts overwhelmed by “data crush”• Analysis and exploitation tools not keeping pace with
sensor development and deployment• “Copy & Paste” approach to ISR fusion becoming the
“norm”…revealing new ISR information becoming rare• “Hunt and not Gather” mentality pervasive in ISR
operations…few resources and tools available to “cast-the-net” approach to analysis and fusion
21st Century Challenges:A new emerging problem set
Swimming in sensors… Drowning in Data…
17
18
The “Big Picture” – everything is a sensor, thousands of analysts
19
Viewers/Analysts of FMV (as of 2007)
• Army + EUCOM, Southcom– 537,000 desktops
• Air Force + STRATCOM, NORTHCOM, TRANS, JCS, OSD– 525,000 desktops
• Intelligence Community– About 190,000 desktops– DHS: 170,000 desktops
• Navy– NMCI 330,000 desktops (Navy Marine Corps Intranet)– OCONUS 50,000 desktops (Outside the Continental US)
1.6 million desktops
20
21
The Problem
Video is rarely recorded in ideal conditions
Poor resolution Noise Camera shake
• Poor cameras– Low‐resolution sources like cell phones capture videos with poor detail, shakiness, and noise
artifacts
• Poor Conditions affect capture quality for all sensors– Medium/Small UAVs don’t have the payload for stabilized camera pods – extreme shake– Even large UAVs and ISR platforms suffer from atmospheric distortion
22
“Traditional” military image processing
Custom hardware
• Pro: small/low power consumption• Con: limited capabilities “one task” devices
– Lighting/contrast– Color– Rudimentary stabilization
• Con: difficult to integrate and upgrade
23
MotionDSP’s GPU image processing software: Ikena ISR
• Real‐time image processing with GPU‐acceleration• Runs on off‐the‐shelf Windows hardware• Superior image processing to hardware‐based solutions• Easy to integrate and upgrade• Lots of memory, lots of compute• Capabilities scale with Moores Law
24
Demo Video
• Med/Large UAV source• Real‐time stabilization, super‐resolution enhancement, noise removal,
dynamic contrast
25
Cessna 206
Android
Enhanced Imagery Distributed to Disadvantaged Users Via IP
• IP Video and Still Images sent to Rover5 and Android/iPad
• Higher NIIRS quality, Higher Resolution
Rover 5
Shore
R/T Stabilization/De‐JItter
R/T Super‐Resolution Re‐construction
COTS Embedded Windows PC (GE MAGIC1) with GPU runs Ikena ISR software for R/T
image processing and change detection
Stiletto
On‐board Real‐time FMV Image Processing on Scalable Cloud• COTS Rack‐mount servers with GPUs• Multi‐channel video input and output
• Live, Real‐time Image Processing• Live, Real‐time automatic change detection
FMV Image Processing
IP Video
Stiletto Demonstration
26
Ikena ISR WorkFlow
EZ TV System
Live IP Video
Outputs
Flat Video Files
Still Imagery
JPEGNTIF
COTS Workstation(Windows XP/7)
Inputs
Archived Video
High‐Resolution Still Images
JPEGNTIF
• Includes metadata• JPEG, PNG, BMP, TIF • NTIF (coming soon)
Flat Video Files
JPEGNTIF
• Includes metadata• Windows Media (WMV) format
• Also H.264, MPEG2 formats
IP Video• Real‐time enhanced results
• Includes metadata• H.264 or MPEG2
Real‐Time FMV Processing• Super‐resolution enhancement
• De‐noise; de‐jitter; de‐interlace; light, color, and haze correction• Real‐time mosaics
• Ground‐mover detection• View embedded metadata Metadata: Output products include
embedded metadata from the original input
27
Air Force ESC/XR Project Roadmap Capabilities
Detecting motion from video •Combine motion detection with tracking•Fuse video content information with metadata•Adapt algorithms for very large pixel dimensions of WAS•Fuse with other layers of INT (GMTI, SigInt)•Expected testing in spring 2011
28
The Future
• Trends– Sensors looking at everything.
• "ability to survey any spot on earth, in any weather, at any time"– Tools to analyze data automatically will be key
• Tools we need– Multi‐GPU virtualization
• Run multiple GPU algorithms dynamically across GPUs– Cloud GPU virtualization
• High‐performance cloud environments and virtualization software which allow full access to GPU compute
29
Questions/Thank you
MotionDSP Inc.
700 Airport Blvd, Suite 270Burlingame, CA 94010sales@motiondsp.com(650) 288‐1164 (o)
30
Disclaimer & Attribution
The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors.
The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. There is no obligation to update or otherwise correct or revise this information. However, we reserve the right to revise this information and to make changes from time to time to the content hereof without obligation to notify any person of such revisions or changes.
NO REPRESENTATIONS OR WARRANTIES ARE MADE WITH RESPECT TO THE CONTENTS HEREOF AND NO RESPONSIBILITY IS ASSUMED FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION.
ALL IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE ARE EXPRESSLY DISCLAIMED. IN NO EVENT WILL ANY LIABILITY TO ANY PERSON BE INCURRED FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
AMD, the AMD arrow logo, and combinations thereof are trademarks of Advanced Micro Devices, Inc. All other names used in this presentation are for informational purposes only and may be trademarks of their respective owners.
The contents of this presentation were provided by individual(s) and/or company listed on the title page. The information and opinions presented in this presentation may not represent AMD’s positions, strategies or opinions. Unless explicitly stated, AMD is not responsible for the content herein and no endorsements are implied.
top related