data aggregation, curation and analytics for security and situational awareness

36
Copyright 2016 Prescient Holdings, LLC Bespoke Solutions for Data Curation, Analysis, Dissemination, and Visualization PrescientTraveler™ leverages decades of success in high-stakes intelligence operations, irregular warfare, crisis management, threat analysis, and complex systems design to keep international travelers safe. Purpose-built software and advanced analytic systems are used to aggregate, evaluate, distribute, and visualize threat and safety-related information. DATA AGGREGATION, CURATION, AND ANALYSIS FOR SAFETY SITUATIONAL AWARENESS SECURITY & Emergent Threat Detection and Incident Alerting Continuous Monitoring of Asset-Threat Proximities Venue-specific Threat and Business Continuity Assessments Low-bandwidth, Secure Communications

Upload: dataworks-summithadoop-summit

Post on 24-Jan-2018

391 views

Category:

Technology


1 download

TRANSCRIPT

Page 1: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Bespoke Solutions for Data Curation, Analysis, Dissemination, and Visualization

PrescientTraveler™ leverages decades of success in high-stakes intelligence operations,irregular warfare, crisis management, threat analysis, and complex systems design to keepinternational travelers safe. Purpose-built software and advanced analytic systems are used toaggregate, evaluate, distribute, and visualize threat and safety-related information.

DATA AGGREGATION, CURATION, AND ANALYSIS FOR

SAFETY SITUATIONAL AWARENESSSECURITY & Emergent Threat Detection and Incident Alerting

Continuous Monitoring of Asset-Threat Proximities Venue-specific Threat and Business Continuity Assessments Low-bandwidth, Secure Communications

Page 2: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Bespoke Solutions for Data Curation, Analysis, Dissemination, and Visualization

DATA AGGREGATION, CURATION, AND ANALYSIS FOR

SAFETY SITUATIONAL AWARENESSSECURITY &Flexible Models and Processes

ScalabilityHigh Availability (DR/HA)

DEMANDS

Page 3: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Bespoke Solutions for Data Curation, Analysis, Dissemination, and Visualization

DATA AGGREGATION, CURATION, AND ANALYSIS FOR

SAFETY SITUATIONAL AWARENESSSECURITY &Flexible Models and Processes

ScalabilityHigh Availability (DR/HA)

DEMANDS

TECH-HUMAN BALANCE (FOR BUSINESS PERFORMANCE)

Page 4: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 5: Data Aggregation, Curation and analytics for security and situational awareness

Mobile App Features and Capabilities

• Real-time threat alerts for your location & demographics• Real-Time tracking and monitoring of travelers• Country and City based reporting• Selected points of interest and safe sites

Prescient Traveler Mobile Application

Copyright 2016 Prescient Holdings, LLC

Page 6: Data Aggregation, Curation and analytics for security and situational awareness

While production analytics and operationaldata are managed on premises,autonomous failover to a remote datacenter (on a different segment of thenational power grid) provides disasterresilience. High Performance, Availability,and Scalability were core designrequirements.

Copyright 2016 Prescient Holdings, LLC

Failover Sites

Real-time Worldview for Safety and Security Stakeholders

sys·tem /ˈsistəm/ noun

1. a set of connected things or parts forming a complex whole.

Page 7: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

A Few Basic Questions that we all face

What is the problem to be solved?

Will the solution fit within a viable business model?

What tools are best suited for the work to be done?

How much is known, unknown but knowable, and unknowable? I.e. How complete is your model?

Page 8: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

A Few Basic Questions

What is the problem to be solved?Improve Traveler Safety and Business Continuity

Will the solution fit within a viable business model?

Yes, with sufficient automation

What tools are best suited for the work to be done?

For Prescient: Hadoop, SAP HANA, MongoDB, NiFi

How much is known, unknown but knowable, and unknowable? I.e. How complete is your model?

Guestimate: 5%, 70%, 25%

Page 9: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

A Few Basic Questions

What is the problem to be solved?Improve Traveler Safety and Business Continuity

Will the solution fit within a viable business model?

Yes, with sufficient automation

What tools are best suited for the work to be done?

For Prescient: Hadoop, SAP HANA, MongoDB, NiFi

How much is known, unknown but knowable, and unknowable? I.e. How complete is your model?

Guestimate: 5%, 70%, 25%

Project Phase 1 2 3

We Are Here

Page 10: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Deeper Questions

How do you figure out what you know? Model → Ingest/Collect → Test → Refine → Repeat

What must you do to convert unknowns in to knowns?

Extract Signal from Noise

What can you do to mitigate the risks created by the unknowable?

Interpolate + Infer + Hypothesize

How much is known, unknown but knowable, and unknowable? I.e. How complete is your model?

Guestimate: 5%, 70%, 25%

Inference

PredictiveProject Phase 1 2 3

We Are Here

Page 11: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Deeper Questions

How do you figure out what you know? Model → Ingest/Collect → Test → Refine → Repeat

What must you do to convert unknowns in to knowns?

Extract Signal from Noise

What can you do to mitigate the risks created by the unknowable?

Interpolate + Infer + Hypothesize

How much is known, unknown but knowable, and unknowable? I.e. How complete is your model?

Guestimate: 5%, 70%, 25%

Inference

PredictiveProject Phase 1 2 3

We Are Here

Page 12: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Entities of Special Interest & Foundational Concepts• Location – Geographic area characterized by cultural norms,

taboos, and chronic or emergent threats

• Traveler – Person with exploitable attributes, proximities to threats, and an associated risk profile

• Itinerary – Traveler’s scheduled exposures to threats

• Threat Parameterization – Proprietary process by which threat vectors are correlated with traveler attributes

Partially expanded data models upon which linguistic, geospatial, and multi-variable analyses are performed using SAP HANA, Hadoop, and a variety of purpose-built tools.

Hundreds of defined entities and the thousands of relationships between them permit, among other things:• Automated Entity Extraction for

Threat Detection and Identification• Sentiment Analysis on Social Media

and News Feeds• Threat-Proximity Alerting• Quantification of Exposure Across

Threat Domains, Locations, and User Populations

A small region of the PrescientTraveler™ Ontology

Data Modeling for Advanced Analytics

Page 13: Data Aggregation, Curation and analytics for security and situational awareness

Locate, Curate, Organize, Analyze, and Produce Information that Helps Users stay safe and remain productive

Asset profiles and itineraries are correlated with parameterized threats. Cultural and Safety Parity Assessments are performed for Home and Destination Locations.

Myriad sources are curated then translated into risk-mitigating recommendations.

Copyright 2016 Prescient Holdings, LLC

... + RSS Feeds + Social Media + Breaking News + Crime, Health, and Disaster Statistics + Economic Stability Indicators + ...

Proprietary Interfaces, Applications, and Methods

Copyright 2016 Prescient Holdings, LLC

Page 14: Data Aggregation, Curation and analytics for security and situational awareness

Below The Surface

Advanced Tools for Big Data Aggregation and Analysis

Event-DrivenRealtime

Copyright 2016 Prescient Holdings, LLC

As of 08:00 CST on 24 JUNE 2016, a total of 48,217 distinct threat data sources had been indexed and ingested by the Prescient Platform

PT API DB

A BA BQGIS

Proprietary Parsers

PostGISArcMap

SMART DATA ACCESS

Locations of Interest, Profiles, Timelines

REST API

PrescientTraveler Applications

Content Builder

Dashboard

Mobile

MongoDB

Data Curation Toolset

(DCT with NiFi)REST API

Client Authentication

Examples of Realtime Alerts• Significant Environmental Events• Infrastructure Shortfalls• Civil Unrest• Terror Attacks• Emerging Pandemics• Geopolitical Instabilities

RSS

Social Media

Local Sources

Crime Stats

Economic Data

News

Forums

...

...

Public Bulletins

Case Studies

...

Page 15: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 16: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 17: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 18: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 19: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 20: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 21: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 22: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 23: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 24: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 25: Data Aggregation, Curation and analytics for security and situational awareness
Page 26: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 27: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 28: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 29: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 30: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 31: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 32: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 33: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 34: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 35: Data Aggregation, Curation and analytics for security and situational awareness

Copyright 2016 Prescient Holdings, LLC

Page 36: Data Aggregation, Curation and analytics for security and situational awareness

Protection of international travelers requires real-time, actionable data that is analyzed and simplified by human experts.We’re Prescient, and we’re humanizing

intelligence.

Copyright 2016 Prescient Holdings, LLC