sentinel week 9 h4d stanford 2016

Team Sentinel

● Team members:

○ Jared Dunnmon

○ Darren Hau

○ Atsu Kobashi

○ Rachel Moore

● Cumulative # of interviews: 100 + 12

○ Users: 4 MDA SMEs: 3 Buyer SMEs: 5

● What we do: Enable rapid, well-informed decisions for anti-IUU and future C2 by combining effective UI with high data density, local storage, and analytics

○ Open and automated data aggregation (i.e. incorporate open source data)

○ Flexible layering and filtering with improved UI/UX

○ Enhanced intel through contextualization and easily accessible, common database

○ Identifying deviations from baseline by utilizing historical data

○ Shareable, socializable conclusions visible up and down chains of command

● Why it matters:

○ Outdated platforms not built for current generation--21st century C2 needed

○ A2/AD prevents timely deployment of traditional ISR--need rapid conclusions from available data

○ Data aggregation platforms and database access in PACOM are extremely manual

● Military Liaisons

○ --- (Colonel, US Army)

○ --- (Commander, US Navy)

● Problem Sponsor

○ --- (Lieutenant, US Navy 7th Fleet)

● Tech Mentors include:

○ --- (BAH)

QUOTE OF THE WEEK

“There are solutions out there that may work, but the biggest problem is implementation.”

Contents

1. MVP

2. Key Partners, Resources, and Activities

3. Learning about Costs

4. Experiments for Fixed Cost, Variable Cost

5. Mission Model Canvas

6. Cost Flows

7. Finance and Operations Timeline

MVP

CIC PACOM

Surface radar contact but no AIS… This is odd. Let me ALERT others.

MVP

CIC PACOM

Surface radar contact but no AIS… This is odd. Let me ALERT others.

I see an ALERT from DDG102. Lets share the C2 screen and take a look

MVP

CIC PACOM

Customer Discovery DeploymentProduct Development Navy Testing

Initial Testing

Information Assurance

Maintenance & Support

Key Partners, Resources, Activities

TRL 1 TRL 2 TRL 3 TRL 4 TRL 5 TRL 6 TRL 7 TRL 8 TRL 9

Partners Resources Incentives Risks & Obstacles

DIUx $50-100K (?), network Create + validate innovation transfer model

Spending authority is not as broad as we expect

NPS Network, access to data sources, SME on analytics, Navy systems

Inter-university collaboration Their research thrust might differ from ours

SeaVision, SkyTruth, Sea Scout

Exposure to existing tools, data sources Outside team addressing capability gaps in IUU fishing

They have stake in their own tool, proprietary or classified tools and data

Palantir Access to tools, technical advice, B2G experience Leads to Navy as potential customer

They might decide to create own tool

Lawyers IP and contracting expertise $150K (?) Finding lawyers with the right experience and investment in our problem

Sub-contractor (i.e. Lockheed Martin, OGSystems)

Expertise in contracting work with government $50K (?), potential future work with Navy

Finding the right company with solid track record and willingness to work with non-traditional

GSA/18F Highest-level information assurance Fulfill their purpose People unfamiliar w/ process = slowGSA IA route actually takes longer

OPNAV/SPAWAR/NWDC Explicit requirements statementsExpertise on procurement routesTesting in relevant environment

Improving function of Navy tools Might not be open to working with non-traditional groups

Coast Guard, 7th Fleet, 3rd Fleet, PACOM

Understanding of pain points + use casesUser feedback on MVP and productBuy-in from top levels

Bringing in C2 tools built for millennial sailors

Complexity of navigating large orgLow bandwidth / availability

Key Partners, Resources, Activities

Key Cost Experiments

Hypotheses Experiments Results Action

There are different kinds of contract vehicles worth considering (cost plus award, cost plus fixed fee, fixed fee, T&M)

-Interview with General Partner of mid-size defense (OGSystems)-Will do more validation!

Validated with caveat that, at least within IC community, there is a shift towards cost plus fixed fee b/c validating performance for award is too cumbersome (more experiments will be run for further validation)

- Talk with people within Navy who are familiar with Navy contracts to determine appropriate cost structure

Necessary to expend resources setting up quality process (i.e. AS-9100, ISO-9000)

-Interview with General Partner of mid-size defense (OGSystems)-Will do more validation!

Invalidated! These are nice-to-haves at least for signing the first contract (more experiments will be run for further validation)

- Talk with people within Navy who are familiar with Navy contracts to determine appropriate quality process for first contract

Information assurance process is avoidable and therefore we don’t have to worry about its associated costs

- Interview with problem sponsor Jason-Interview with NWDC

Invalidated, but there are potential shortcuts like the Office of Digital Services, 18F, and GSA that would hold us to a higher standard yet allow us to circumvent the long and expensive typical IA process; NWDC personnel can give us excruciating detail

- Talk with Office of Digital, 18F, and GSA or people who are familiar with these avenues as acquisition pathways

Experiments for FC and VC Validation

Experiments What we will learn Who we want to speak with

Interview commercial companies with a similar software product

- General cost breakdowns (i.e. personnel, overhead) for company with a similar product- Rough estimates for actual costs associated with each bucket

- Global Fishing Watch, Skytruth, Pew Charitable Trust- Software companies who sell C2-like tool to customers like FedEx, UPS, etc

Interview defense contractors with similar software in terms of user base and/or deployment

- General cost breakdowns (i.e. personnel, overhead) for company with a similar product- Rough estimates for actual costs associated with each bucket

- Palantir, Northrop Grumman, Scitor, SAIC

Interview experienced VC personnel with experience in developing software companies

-general approach to reasonable cost assumptions- ways to approach project management to reduce variability and mitigate unexpected costs

-BVP, etc.

Try for ourselves! Where we went wrong, what we did right

Our future selves

Data- Compile existing data resources/scope out future ones

Defining C2-F- Brainstorming what “Command and Control of the Future” would be by interviewing younger sailors

Software Development- Develop flexible data fusion/analytics algorithms, and an intuitive UI for millennials


Prototype Testing/Procurement

Contracting, Acquisitions

Maintenance and Support

USN Strategic Decision Makers

USN Analysts (N/J2)

USN Operators (N/J3)

Anti-IUU Fishing Enforcers (USCG, Partner Nations, etc.)

Anti-IUU Fishing Stakeholders (NGOs, Legal Fishing)

(Commercial entities that use/would benefit from enhanced C2-type systems)

USN- Timely, accurate operational decisions- Decreased time to predict hot spots, ID & differentiate threats- Increased engagement and effectiveness of younger sailors - Up-to-date, reliable info in frontline environment

Anti-IUU Fishing- Reduction in IUU fishing worldwide due to better deterrence- Better allocation of scarce / expensive interdiction resources- Widespread engagement of operators, governments, and the public

USN- Work with fleet sponsor to get C2-F system on fleet needs list- Ensure C2-F makes it into FIMS database, engage S&T bridge personnel to talk with key decision makers- Work with NWDC, ONR S&T, PACFLT LOEs to test solution - Engage PACFLT N8/N9 shops to implement modular operational deployment & update pathways

Anti IUU Fishing- Work with NGOs, gov’t departments, USCG, operators, etc. to find key influencers/stakeholders- Deploy solution where possible,

Fixed- Existing Software tools/APIs, Data- IA process steps- Travel for site visits, pilots, interviews with sailors- R&D personnel- AWS & Distributed Computing- Overhead

Data & AnalyticsAPIs for accessing data (e.g. API for Global Fishing Watch, AIS), $$$ needed to access this

Defining C2-FIdeas/feedback from young sailorsHackathon w/ Navy and DIUx support

Software DevelopmentAWS, programmers, $$$ for both, SME on phenomenology of ships, activities

- Need commanding officer to confirm decision-making benefits

- Need intelligence officers from ONI / N2 and operators from N3 to confirm effectiveness of insights

- Need IT approvals to integrate into systems

- Need support of commercial partners if we want to leverage their platforms

-Need support of existing PMOs/S&T personnel to make sure we’re not duplicating work

Beneficiaries

Mission Achievement

Mission Budget/Costs

Buy-In

Deployment

Value Proposition

Key Activities

Key Resources

Key PartnersDataSkytruth, Pew, GFW, TerraSAR

Defining C2-F7th,3rd Fleet junior officers, sailors

Software developmentPalantir Skytruth, NPS/ONR, SeaVision, Sea Scout, Universities (e.g. Vanderbilt), NGOs

Information AssuranceGSA, NWDC

Prototype Testing/ProcurementUSFF (NWDC), NAVSEA, SPAWAR, C7F CIG, PACFLT CSIG, IA contact

Contracting, Acquisitions-IP Lawyer, subs with gov experience-DIUx, C3F N8/9, PACFLT N8/N9

Mission: Creating C2-F--Enabling Rapid Decisions from Heterogeneous Data

Information Assurance Access to personnel to provide certification / approval

Prototype Testing/AcquisitionNavy testing venue and exercise (e.g. Trident Warrior), Military Sealift Command ships, 7th Fleet experimentation ships and personnel

Contracting, AcquisitionsDomain knowledge of software contracting and IP from lawyers, subsVariable- Maintenance and Support- Integration with existing systems and processes

IMPROVE USN DECISIONS & OPS VIA C2-F WITH

IMPROVED DATA HANDLING, UI/UX,

COMMS, AND HARDWARE

(1) Rapid Strategic Decisionmaking via Improved Reporting, Coordination, Visibility

(2) Improved Tactical Decision Making via Timely, Accurate Information Sharing

(3) More Effective Analysis via Searchable, Visualizable, Source-Flexible Data Integration (Layering & Filtering)

(4) Increased Analyst Bandwidth via Predictive Intel and Alerts (e.g. Machine Learning) Flexibly Applied to Available Data

(5) Improved Collection of Existing Data Streams

(6) Increasing Morale & Engagement for Millenial Sailors

ENHANCE ANTI-IUU FISHING CAPABILITIES

(1) Improved Detection Using Data Fusion/Analytics

(2) Enhanced Enforcement via Improved Communication

(3) Lower Barriers to Engaging Civilians in Reporting IUU Fishing Activities

Cost Flows

Database ($80k)

Analytics Engine ($120k)

Translation (ETLs) ($100k)

AIS VMS Radar SAR Sat

UI ($80k)


($240k)

Testing ($480k)

Maintenance and Support

(VC)

Assume 10 data streams, need cost validation on streams

$380K $240K $480K $???

Total: $1.1 MM + Var Costs

3 Year Financial/Ops/Funding Timeline

2016 2017 2018 2019

Q3 Q4 Q1 Q2

Cas

h R

eser

ves

Phase

Prod

uct

Gov

’tC

om’l

Mile

ston

es

Q1 Q2Q3 Q4 Q1 Q2Q3 Q4

TRL 1

TRL 2

TRL 3

TRL 4

TRL 5

TRL 6

TRL 7

TRL 8

TRL 9

POC

Wireframe

Prototype BetaPrototype

Marketable Product

BetaPrototype

Released to first customers (<3)

Commercial Product Launch;

2 contracts

Test in Navy env Navy-wide Deployment Maintenance and Support

V2.0 Commercial Product Launch;

5 contracts signed

Initialize System Development/Customer Relationship Development Launch at scale

Hea

dco

unt

410

20 50

15 customers; V2.5 launch

CONTRACT SIGNED CONTRACT

RENEWED

SBIR/DIUx

Series A (In-Q-Tel, Impact Investor(s))

2 com’l contracts

$250k$0

$1.25M DoD Contract

$2M

$5M

Thank you!

Any questions?

Products& Services- Timely data- Good UI/UX

for presenting data

- Streamlined reporting process

- Improved coordination across ranks

- Utilizes current tool pipeline

Customer Jobs

Gains

Pains

Gain Creators

Pain Relievers

- Good UI/UX- Platform

incorporates more data streams

- Platform is robust and can handle drop out of data streams

- Allocate assets- Identify, eliminate

threats- Predict hot spots- Safety and morale

of team- Projecting peace,

stability in region

- More informed decisions

- Faster decisions- Decisions made

on most up-to-date info

- Poor quality/lack of data

- Latency of data -> insight

Admiral/Strategic Decision Maker

Value Proposition Canvas

Customer persona:

● 3 or 4 star admiral● Born in late 1950’s● Have their own office on-base● Gives out challenge coins● Responsibility for well-being of

their entire organization (e.g. 7th Fleet)

● 30,000 ft view thinker● Spent entire professional career

in Navy (deeply ingrained culture)

Products& Services

- Contextualized, object-oriented database

- Algorithms for processing, analyzing data

- Ability to search for trends across database

- Integration of disparate data sources

- Automation of data analysis- Improved UX/UI enabling

greater engagement- Predictive notifications- Filtering and layering

features- Tool architecture is flexible

and intelligent

Customer Jobs

Gains

Pains

Gain Creators

Pain Relievers

- Compatible data format

- Incorporate multiple data streams with existing object-oriented database

- Integration into current processes is simple

- Collect & analyze data

- Communicate findings

- Piece together contextualized awareness

- More actionable insights- Faster identification & response times- Easy-to-use- Information continues to be processed

and visualized even if data streams are added/dropped (no Christmas Light effect)

- Incorporation of context is manual/mental

- Poor quality / lack of data- Latency of data -> insight- Long onboarding processes

Analyst (N2)


Customer persona:● 18-35 years old, described as “19-year old from Oklahoma”● Sits in front of computer 12 hours per day● Job usually boring with bursts of activity● Regimented daily process, but schedules could change drastically

day-to-day

Products& Services

- N/A

- Actually a common operating picture!

- Data is actually synced across databases

- Tool architecture is flexible and intelligent

- UI/UX that is much more intuitive for the millennial sailor

Customer Jobs

Gains

Pains

Gain Creators

Pain Relievers

- No hardware to deploy so no risk of asset or personnel loss

- Fewer change orders- Training and integration

with current processes is simple

- Utilize assets and human capital in order to obtain ISR data on adversary or regions of interest

- Timely and enhanced allocation and deployment of assets

- Information continues to be processed and visualized even if data streams are added/dropped (no Christmas Light effect)

- Increased engagement

- High manpower, time- Operator error- Safety concern for deploying

in unfriendly territory- Struggle to redeploy systems

on short notice (<12 hours) = frustration

- Long onboarding processes

Operations (N3)


Customer persona:On Ship

● 18-35 years old● Follow preset “if-then” procedures● When passing through high-risk

area, very jumpy

Customer persona:Head of N3

● 40-55 years old● General sense that N2 and N6

“work” for them● Frustrated by changing demands

- Crowd-sourced data collection

- Better COP with partner nations

- Enable fast response with better allocation of assets

Customer Jobs

Gains

Pains

Gain Creators

Pain Relievers

- Better intel = more effective use of assets

- Locate IUU fishing activity

- Apprehend the individuals involved

- Easier to communicate insights

- Increased buy-in from partner nations

- Improved deterrence

- Inability to detect “dark” targets

- Expensive interdiction vehicles sent to verify IUU fishing

Anti-IUU Fishing Enforcers (USCG)


Customer persona:Intel (works in MIFC)

● 25-40 years old● Deep domain expertise - interfaces with

partner nations, NGOs● Develops internal tools● May be re-assigned by HR to unrelated roles

Products& Services

- Contextualized, object-oriented database

- Algorithms for processing, analyzing data

Customer persona:Operator (on ship)

● 20-30 years old● Is law-enforcement presence - can direct

entire Navy ship for related tasks● Domain expertise or not???

Anti-IUU Fishing Stakeholders


● PLACEHOLDER● We are gathering more info on:

○ NGO’s○ Fisheries○ Partner nations○ +other potential Anti-IUU Fishing Stakeholder beneficiaries

Commercial Entities● PLACEHOLDER● We are gathering more info on:

○ Fisheries (as a customer)○ Amazon (also as they look to develop their drone delivery

capabilities)○ Commercial Shipping companies○ O&G companies

Mission Achievement

Beneficiaries Mission Achievement

Strategic Decision Makers (Admirals)

- Better, faster decisions despite uncertain information- Improved COP with analysts + operators = save

manpower/bandwidth resources associated with preparing PPT- Better understanding of what data contributed to analysis in CUB,

and reliability of that information

Analysts (N/J2) - Predictive analytics based on historical baselines eliminates tedious elements of workflow → more engagement

- Data aggregation into common database decreases time to insights

Ship CIC Operators (N3) - Up-to-date, reliable information in frontline environment- Easy to share COP with analysts + strategic decision makers- Next-generation user interface which is intuitive + leverages

familiarity with commercial tech → more engagement

Anti-IUU Fishing Enforcers (USCG, w/ support from USN)

- Better allocation of scarce / expensive interdiction resources- Faster response = better deterrent

Anti-IUU Fishing Stakeholders (Partner nations, NGOs, legal fishers)

- Ensure economic + food security- Reduce # of illegal fishers + associated IUU catch- Improve working conditions / reduce human trafficking

MVP

http://youtube.com/v/Mz9xysVIrUU

http://youtube.com/v/fn2JXmCUo30

MVP: Modular Intake, Algorithm, and Display

Customer Discovery - Get/Keep/Grow Diagram

Awareness Interest Consideration Purchase Keep Unbundling Up-sell Cross-sell Referral

Activity & People

- Evangelist & advocate from originator Flt- ???

Corey Hesselberg, CDR Jason Schwarzkopf, MIOC watch standers

- Buy-in from flag officers- ADM Swift, VADM Aucoin, RADM Piersey

- N8/9- Dave Yoshihara (PacFlt N9)- 7th Fleet ???

- Maintainers (N6)- Bob Stevenson (PacFlt N6)- 7th Fleet ???

N/A Expanding COP & intel extensions / functionality within 7th Fleet

Expanding user base within 7th Fleet

Expanding tool set to other fleets

Metrics % people who have heard of program before vs after *how to reassess?

# people who say “we want this”

Seems binary… any recommendations?

# Systems outfitted

?? ?? ?? # users within 7th Fleet using tool

# fleets using tool

Map of System Functions and Needs

QUELLFIRE

GCCS (1)

FOBM

STORAGE/COMMS

CST

GCCS (3)GCCS (2)

STORAGE/COMMS

STORAGE/COMMS

Sensors Sensors Sensors

.oth-.json Translator

Visualization

Analytics

Ship-to-Ship Sharing

Long-Term Storage

KEY NEEDSFUNCTIONS

& PROGRAMS

SHIP 2 SHIP 3SHIP 1

MVP: Software Domain Awareness

Program POC OrganizationFunction & Goals

To be used by whom?

Security Level Status Contract History Inputs

Technical Details

CSII

Insight

MTC2

Quellfire

DCGS-N Increment 2

C2PC

HAMDD

SeaVision

GCCS

EWBMRC2 (Resilient C2)

Sample In-Development Product: ONR/CTI EWBM Tool

MVP (3 weeks ago)

MVP (4 weeks ago)

AIS Weather

Customer Workflow

N2

N3

N2(“owns”

the intel)

N3(“owns”

the assets)

Ready-To-Use DataDeployment

Data Acquisition

Data Analysis

Data

Order/Decision

Customer Workflow

Key Acquisition Paths

● Several potential deployment strategies

○ Linking in with an existing POR (PMW-150, etc.)

■ Pros: Allocated funding, long-term integration plans

■ Cons: Long timescale, getting in the door

■ POCs: ONI, SPAWAR (Stan Kowalski), Primes

■ Source of Excitement: TBD

○ Rapid Acquisition Pathways (Limited Objective Experiments, Rapid Reaction Technology Office)

■ Pros: Speed, Close to user, Don’t have to go through Navy (other services work)

■ Cons: Limited spending authority

■ POCs: 7th Fleet (Jason Knudson), DHS (Chuck Wolf)

■ Source of Excitement: Rapid deployment, changed acquisition model

○ DARPA

■ Pros: Development mindset, existing programs (Insight) that are well-aligned, deployment authority/capability to pay for deployment to end-users

■ Cons: stepping on toes, limited number of PMs

■ POCs: Craig Lawrence (ADAPT)

■ Source of Excitement: Directly solving a problem as opposed to many-year process

Sample Deployment Path (Software, POR Path)

1. Operational testing to make sure meets military specs (engage SPAWAR for this)a. Ensure NSA-standard Information Assurance (IA)

i. Lock down system and codeii. Make sure no category 1,2,3 in code - backdoors, exceptions, etc.

b. Observe appropriate NIST protocols (TBD)2. First, limited deployment to evaluate functionality (on testbed system or specific asset)3. Then, if integrated into a POR:

a. Deployed on whatever platform is neededb. Moves into sustainment phasec. Think about disposal & replacement--we want continuous improvement!

4. IT installs where requireda. Technical support install software and make sure up and runningb. Maintains communications systems and networks

5. Personnel training for system operation and maintenancea. CTMs focus on maintaining classified systems & special collections abilities

WE WILL BE GETTING MORE DETAIL ON THIS GOING FORWARD!

Procurement + Deployment Schedule

Activity Timeline Funds People + Resources

Continue Customer Discovery w/ Gov’t May - Sept 2016

$0 Sentinel, H4D cohort, Navy, USCG, OSD

Customer Discovery with Commercial Entities May - Dec 2016

$0 Sentinel, Teaching Team, Chris Robinette, GSB network

Continue Technical Development of MVP May - Dec 2016

$0 Sentinel, AWS(?)

6 month Period of Performance SBIR/BAA/etc tailored to our MVP is released and awarded to us

Sept - Dec 2016

$0 Sentinel, Rich LeBron, NWDC(?), DHS(?)

Work on SBIR/other contract and commercial product in parallel

Jan - June 2017

$250k - $70k Sentinel

Lay framework for DoD acquisition (i.e. build relationships, iron out process, LOE, etc)

Jan - June 2017

$0 Sentinel, Rich LeBron, influencers and acquisition DoD people

Identify and develop relationships/partnerships with key early adopter/early supporter entities (i.e. commercial companies, NGO’s)

Jan - June 2017

$0 Sentinel, TBD

Close acquisition and sales deals with DoD and early adopter commercial companies!

June - August 2017

$1MM - $3MM, for each contract

Sentinel, DoD (specifics TBD), TBD

Procurement + Deployment Methods

● This references the italicized line in the schedule● For Navy, one potential deployment avenue is latch onto a program of record

already in the Fleet Experimentation Info Management System● We can also leverage SBIR contract as a way to set up for eventual acquisition● We know that the Coast Guard brings in technology for evaluation (on the path

to acquisition) relatively frequently throughout the year○ We just need to determine how we insert ourselves into that funnel

● In terms of implementation, we propose an external hard drive that has historical data pre-loaded in addition to our software solution○ In addition to the solution, this is beneficial because of bandwidth issues

and memory issues (and our customer discovery has intensely validated these needs)

○ This avoids having to deal with deploying our software on their hardware while maintaining modularity and flexibility of our solution

○ It also gives us a fully vetted avenue by which to test and roll out updates in modular fashion

sentinel week 9 h4d stanford 2016

Education