real-time scene understanding - amazon s3 · real-time scene understanding ... {t, t, t, t, t, t,...

1

Real-Time Scene Understanding

Dynamic Data‐Driven Applications SystemsDr. Frederica Darema

Dr. Alex AvedResearch Computer ScientistAnalytical Systems [email protected] Statement A. Approved for Public Release; Distribution Unlimited Case Number: 88ABW‐2016‐0367

2Distribution Statement A. Approved for Public Release; Distribution UnlimitedCase Number: 88ABW‐2016‐0367

Problem StatementFocus• full motion video exploitation to support decision-making; event

specification, detection, situational awareness.

Problem• we collect more data than we can process• data is complex, fragmented; increasing in quantity & complexity• at-a-glance processing, exploitation and dissemination infeasible

without software.

Goal• break the linear relationship (staffing) between collection and

analysis, • Leverage analysts for synthesis and insight; computers for

calculations and memory.

14 February 2016

• Many sensors• Need to quickly leverage data for decisions

• Integrated informationarchitecture

“Simply stated, the need for accurate intelligence and prescient analysis[…] has never been greater in 2013‐‐or than it will be in the coming

years”John Brennan, Director, Central Intelligence Agency


Approaches

Business usersIdentify questions

SPBuilds database structure

SPPlatform

Business usersExplore correlations

Structured and repeatableQuestions (queries) drive insightData at restRelational data model, warehouses

Iterative and exploratoryInsight drives answersData in motionHadoop (MapReduce), LVDBMS

• Talent matters as much as technology• SP – Solution Providers; researchers…etc.

Note


Conventional Solution

Background

14 February 2016

Network-connected video cameras leveraged for monitoring (bridges,malls, parks…). Currently monitored by humans who can becomefatigued, get interrupted, attention lapses, work in shifts…

The Live Video Database Management System[1,2] (LVDBMS)prototype will be leveraged to develop and apply machine learning andfusion algorithms, and associated data structures and communicationprotocols.

[1] A. J. Aved, K. A. Hua, and V. Gurappa, “An Informatics‐Based Approach to Object Tracking for Distributed Live Video Computing,” in Multimedia Communications, Services and Security, 2011, pp. 120–128.[2] A. J. Aved and K. A. Hua, “A general framework for managing and processing live video data with privacy protection,” Multimedia Systems., vol. 18, no. 2, pp. 123–143, Feb. 2012.[3] Anwar, Fahad, et al. "An Efficient Event Definition Framework for Retail Sector Surveillance Systems." MMEDIA 2014, The Sixth International Conferences on Advances in Multimedia. 2014.

HP: http://www.autonomy.com/products/surveillance• Generates “events”. Events can be analyzed.• Face detection• Recognize models of automobiles• Integration with Cyber (via event model)

IBM: http://www‐03.ibm.com/software/products/en/intelligent‐video‐analytics

• CBIR• Alerting (tripwires; boxes)• Size correlation (3d scene reconstruction)

[3]

Example applicationTraffic monitoring

Academia

Industry


Domain-Specific Systems

FP 1

A 1

FP 2 FP 3

A 2 A 3

FP: Fusion ProcessorA: Application

Application specific

programming

A specific set of known cameras

Sharing similar

underlying software


The LVC Approach


Query Language LVQL

++22 33

55

ACTION <action>ON EVENT <composite event> <window>

Query Optimizer

Mathematical example


Event Model – Spatial Event

Spatial event stream– Stream of spatial events

on two objects

Es(t1) = Disjoint(Oi(t1), Oj(t1))

Es = {T, T, T, T, T, T, F, F, F}o

Es(t1) = T Es(t2) = T Es(t3) = T

Es(t4) = T Es(t5) = T

Es(t7) = F

Es(t6) = T

Es(t8) = F Es(t9) = F

t


Stream Processing

Server

…...CameraServer

CameraServer

CameraServer

A query is decomposed intoThree sub‐queries

Subqueries

Results

Query Translation

Subquery resultsSubquery results

Query

Query Flow


Video Analytics

14 February 2016

• The LVDBMS is a scalable, distributed video database management system designed for processing continuous queries over live video streams in near real time.

“Creating an effective analytical environment can substantially improve operational visibility for public sector organizations and help them execute on their mission of cost [improvement].”2012 Aberdeen Group, BI in the Public Sector: Enhanced Efficiency with Data Discovery

Video analytics

Multi‐cameraVideo characterization

Object matching

Data flow


• Increasing value and reducing costs• Growing demand for improved end‐user experience, quality

and functionality• Vision (of solution) and analytics/metrics to measure• Expands research and business/operations communication,

collaboration and representation• Improved service and value to stakeholders• Measurement, results and analytics – Creating an

environment of analytical engagement and assessment

“Creating an effective analytical environment can substantially improve operational visibility for public sector organizations and help them execute on their mission of cost [improvement].”2012 Aberdeen Group, BI in the Public Sector: Enhanced Efficiency with Data Discovery

Challenges and Opportunities

Thank You….Questions…. ?


Planned Future Research

14 February 2016

• Pattern Search, Fusion and Prediction• POL for Anomaly Discovery and Prediction• Historical Pattern Search• Federated LVC• PaaS Cloud Architecture• Policy-based Information Dissemination• Extend Fusion to Remote Platforms


14 February 2016

real-time scene understanding - amazon s3 · real-time scene understanding ... {t, t, t, t, t, t,...

Documents