Intelligence in National Security

Association of Former Intelligence OfficersBanquet 2 May 2014

Dr. John M. Poindexterjohn@jmpconsultant.com

2

Agenda

• My goal -- Improving product to help decision makers

• National Security Equation

• Big Data

• Cognitive Computing

• Privacy Problems and Potential Solution

3

National Security Equation

Where operators (functions) are: Collection = human and sensor (e.g. Internet) collection of data-> Analysis = selects data-in-context to produce information=> SenseMaking= understanding what the information meansp> PathFinding = deciding what to do about it in policy contexte> Execution = “operational forces” carry out decision Iteration = many steps are often repeated (e.g. Action

changes the world and thus new collection is required.)• Simplified but basic non-linear process that is essential to understand.• Analysis is an over-used term.• This provides a working definition of Sensemaking and Pathfinding.• Process carried out in a collaborative environment with relevant agencies.

• Collaboration is essential to bring diversity to problem of uncertain data.• Need competitive SenseMaking to give decision makers range of understanding.

• Great deal of confusion amongst the terms data, information and knowledge.• “Operational Forces” – military, diplomatic, economic, public diplomacy, law enforcement, covert.

Involves All of National Security Community Not Just Intelligence…

Data -> Information => Knowledge p> Options e> Action

Extensive Automation More Cognitive

c>

c>

Goal: Develop information technology components to aid process.

4

Improvement Requires Co-Evolution

One• Technology• Largely exists but needs system integration

Two• Process• Needs more automated systematic approach

Three• Policy• Needs to be clearly defined and supported

Four• Culture• Understanding of technology by all concerned

Incr

easi

ng D

iffic

ulty

Of:

5

Big Data – Major Problem but Opportunity

Characterized by: • Volume• Velocity• Variety• Veracity

----------------data

6

DataBases

• DataBases are designed for storage – not analysis– Great for storage of collection– Originally designed for back office operations

• Personnel, inventory and accounting– Ok if queries are of static form– Tables are designed to answer these queries promptly

• With intelligence, complex query forms are dynamic– Can’t predict a priori what needs to be asked– In this case table joins are usually required – With Big Data these joins are very time consuming

• Typically Hours to Days

• Often said about DataBases – “Write Once Read Never”

For analysis there are problems…

7

Person:

Poindexter

Matrix

Organization:

White House

Matrix

MemoryBase – A New Technology

Who/What is similar?How similar/different?

Who/What is related? How? Where? When?

What could happen? Where?

When?

What has been done before? Did it

work?

Sense-M

aking D

ecision Support

A matrix for every person, place, and thingA matrix for every situation, action, and outcome

Design influenced by analogy to human memory…

Multiple Contexts

8

MemoryBase Characteristics

• Does not replace databases, but is an adjunct• Ingests distributed data in heterogeneous formats

– Static and streaming – structured and unstructured text• Incoming schemas are translated to generic schema• Scales to Big Data• Standard off-the-shelf servers• Dynamic query response time in sub-second to seconds

independent of MemoryBase size• Now moving to more cognitive functions• Produced by Saffron Technology, Inc.• Intel has made a multi-million dollar investment recently

Works like the human brain, but never forgets…

9

Privacy Appliance Concept

• Recent revelations about access to Big Data by the USG have raised concerns again about privacy – Section 215.

• Government agencies in the national security domain work diligently IAW the law to protect the privacy of innocent individuals while protecting the US from various threats.– The people want this protection, but are concerned about privacy.

• The problem is the people don’t trust the government.• Maybe technology can help with this.

– Complicated, but possible.• When I was at DARPA after 911, we came up with a concept for a

Privacy Appliance and began research.– It was the only part of the TIA program that was not transferred to the IC and

work on it stopped.

Access to Big Data has privacy implications…

10

Concept for Controlled Data Access

Transactions

Collaborative, Multi-Agency

AnalyticalEnvironment

AutomatedData

RepositoriesWorld WideDistributedData Bases

PrivacyAppliance

Leave data distributed, identify critical data bases…

Red teams simulating threat organizations plan attacks and developpatterns of transactions that are indicative of attack planning.

Pattern-basedQuery

FilteredResults

Patterns are important to search for data-in-context to avoid 6-degrees of separation problem.

11

Finding Relevant Information -- AnalysisWhile protecting the privacy of innocents, sources & methods…

data source

data source

data source

privacy appliance

user query cross-

source privacy

appliance

privacy appliance

privacy appliance

Government owned

Commercial or Government owned

Independently operated

response

• Authentication • Authorization• Anonymization• Immutable audit trail• Inference checking• Selective revelation• Data transformation• Policy is embeded• Create MB Index

• Contains MemoryBase (MB) Index• Updated in real time

12

• Search Patterns are authorized by a judicial authority (e.g. FISA court).• Selective Revelation to limit response details depending on level of authorization.• Inference Control to identify queries that would allow defeat of anonymization.• Access Control to return identifying data only to appropriately authorized,

authenticated users.• Immutable Audit Trail for accountability – must have way of analyzing routinely.• Masking to hide analyst intent – especially for non-government data bases.• MemoryBase index created to home in on relevant data bases.

Authorization tables

Inference control knowledge base

Immutable audit trail

User query

Query blocked

or allowed

Masking

Selective Revelation&

Anonymization

Policy & BusinessRules Embedded (machine readable)

The Privacy Appliance ConceptAll functions highly automated to reduce time late…

Transparent, cryptographicprotected shell

(much like network guards)

MemoryBaseProcessing

Publish source code for appliance. Need to avoid Clipper Chip problem.