fishreel lessons learned h4d stanford 2016

The NSA has neither reviewed nor confirmed these slides.

110

Many interviews later...

Hacking for Defense

National Security Agency


Gabbi Fisher

B.S. Computer Science ‘17

Maya Israni

B.S. Computer Science ‘17

Chris Gomes

M.B.A. ‘17

Travis Noll

M.B.A. ‘17

Our new problem statement:

Fishreel uses publicly available social media

data to provide information about

potentially anomalous personas to government and commercial entities.

Our first problem statement:

Fishreel automates the detection of catfishing attempts to produce

insights useful to catfishing targets and

entire IC entities.

Who is team fishreel?fishreel

Team Members Gabbi Fisher Chris Gomes Maya Israni Travis Noll

Degree program and Department/Major

B.S. Computer Science 2017 M.B.A 2017 B.S. Computer Science 2017 M.B.A 2017

LinkedIn www.linkedin.com/in/gabbifish

https://www.linkedin.com/in/gomeschris

https://www.linkedin.com/in/mayaisrani

https://www.linkedin.com/in/travisalexandernoll

Are you the subject matter expert (SME) for this team?

Yes(Engineering, Data Analysis)

No(Customer Success, Product Strategy

No(Machine Learning, Artificial Intelligence, Product Design)

No(Market research, Simulation software)

How does your expertise fit the problem?

+ Experience with data scraping and analytics+ Previous Executive Branch experience in national security policy/impl

+ 3 years at DoJ+ Statistical programming+ Management consulting

+ Software engineering ML/AI experience at Google+ Product/user testing experience at GoldieBlox

+ Conducted 100+ MR interviews+ Built simulation software as a consultant

Experience Solving a Problem that seemed Impossible

+ Coordinating rollout of insider threat detection tool, still in minimum viable product (MVP) state, among international clients over 13 week timeframe.

+ Led 120 NGO staff members in 6 month workshops to identify pain points and solicit input for 5 year strategic plan

+ At GoldieBlox, helped develop a prototype to become a mass-produced toy on the shelves of Toys ‘R Us; saw the company from a one-person startup to a well established company.

+ Turned around a small retail store that was bleeding cash and achieved most profitable quarter on record

Professional affiliations


http://www.linkedin.com/in/gabbifish














What is the national security concern we’re helping to solve?

Bad people make heavy use of social media.

Sometimes they make themselves obvious.


Sometimes not.


Our original hypothesis was that the NSA wanted to send us data on ISIS and help them mitigate recruitment

Original Mission Model Canvas heavily emphasized ISIS, recruitment, accessing NSA data, and mitigation


Through 100+ interviews, came to understand broader problem for both the NSA and other beneficiaries: a need to better understand who or what is behind various social media accounts

New Mission Model Canvas identifies multiple beneficiaries and NSA mission achievement around entity insight


Tracing (1) problem understanding, (2) proposed solution, and (3) agency relationship over 10 weeks


Pushed agency for ISIS data, won our first award

1



“If you haven’t gotten thrown out yet, you’re not trying hard enough”: We came out swinging

~Spring Break


“If you haven’t gotten thrown out yet, you’re not trying hard enough”: We came out swinging

Week ZeroOur initial hypothesis was that we would get a supervised training set of sensitive data on which to build a model

Instead, we would be pulling our own unsupervised data from publicly available sources (meaning if we wanted to train a model on recorded instances of catfishing, we’d have to code them ourselves…)

Steve dropping a major clue before day one of the class about where our product would go…

...But it would be a while before we came around to this point ourselves.

~Spring Break




Push to build solution before understanding problem, AKA #1 way to get on Steve’s hit list

1

2


In week one, we were holding out hope that NSA would still send us *some* data (spoiler alert: we’re still waiting)

Week 1

We had only heard that we were not getting *supervised* data (i.e., data that is pre-coded for instances of catfishing). We were still holding out hope that our sponsor would send us at least an unsupervised data set...

Slide from week 1 presentation

We hadn’t yet learned the difference between hearing and listening… … More importantly to Steve, we were worried

about data and technical solutions before we had even understood the problem.


In week one, we were holding out hope that NSA would still send us *some* data (spoiler alert: we’re still waiting)

We had only heard that we were not getting *supervised* data (i.e., data that is pre-coded for instances of catfishing). We were still holding out hope that our sponsor would send us at least an unsupervised data set...

Slide from week 1 presentation

We hadn’t yet learned the difference between hearing and listening… … More importantly to Steve, we were worried

about data and technical solutions before we had even understood the problem.

STEVE BLANK: “FULL STOP!”

Week 1




Hit it off early with sponsors, they quickly lined up analysts to help us understand problem. But it took a while to understand how to have productive convos...


1

2

3


Sponsors were immediately helpful in setting up calls with analysts, though communication was hard at first

Weeks 2-3

What is the setup of your work station?

Is it bigger than a breadbox?

Do you, maybe, like, have a computer?

. . .

I would describe that as a not unreasonable assumption.

The challenge:


Sponsors encouraged us to ask lots of questions, constantly improving our understanding

Weeks 2-3

The process:

INTERVIEWS. Over 50 interviews with NSA employees and many more with members of intel community

DRAWINGS. “Is this what it looks like?”

TRIAL AND ERROR. Mapping sponsor org and roles, getting it wrong, going back to drawing board

QUESTIONS. Sponsors joined our slack channel, constantly available to help

A day in the life of an agency analyst: major pain point is manually digging into “robot-human-liar”

+ Receive docket on desk as part of broader investigation+ Part of the docket is the “Robot-human-liar” test: Is this identity tag tied to

a bot, an individual telling the truth, or an individual manipulating you?+ Begin manually investigating userIDs

+ Search through one data set+ Print / read all data+ Search through second data set+ Print / read all data

+ Continue until confidence is extremely high: “If we get it wrong in the report, it’s on us. It would be a big problem. Like a doctor diagnosing the wrong illness.”

+ Identity analysis takes the form of a section in a report going back to the inquiring organization

+ LARP on weekends / Magic the Gathering at lunch

Repeat for2 days - 4 months

(all userIDs, redundant searches)

Repeat for all data sources

Takeaways● Monotonous● Manual● Slow● Sisyphean Task● ML learning can add insight in addition to speed / automation

The NSA has neither reviewed nor confirmed these slides.Weeks 2-3

The results:


Products& Services

+Account consistency tools+Account linking

tools+Author

classification

+ Identity matching across platforms

+ “Consistency” scoring on each platform and across the platforms

+ Classify catfishing accurately (4 categories) Customer

Jobs

Produce intelligence for

reports (internally or on behalf of other agencies)

- Using publicly available data, work/ confirmation tends to be manual- False positives (short-run bad)- False negatives (long-run bad)- Data overload - doesn’t have real-time solution- Information overload: can’t understand model output-Timeliness: Manual nature makes “responsive” research more difficult

Beneficiary: A/B Intel Analysts

Gains

Pains

Gain Creators

Pain Relievers

+ Target research+ Incremental progress on understanding entities+ Understanding adversary strategy+ Detecting false identities & bots

- Make efficient use of new data for benefit of rapid updates

-Synthesize output into easily-understood, non-static reports

- Weigh false positives, false negatives appropriately

Value Proposition

Demographics● Average 3-10 years at the Agency● Majority male (70/30 split)● Technical background

Agency

Developed understanding of the value proposition for intelligence analysts after getting in their heads

Note: Specific agency orgs have been masked

Weeks 2-3


Began iterating on our wireframe MVP to test the hypotheses we were developing

Our first MVP Several iterations later...

Weeks 2-3






Realized we were just scratching the surface of necessary buy-in at agency...

1

24

3


We had plateaued in our understanding of the problem - in part because “deployment” is part of the landscape

A/B Analysts+ Grades 13/14+ Technical and non-

technical focuses

+ Mostly male, late 20s-30s, technical degrees, 5-12 years @ Agency+ Want better tools to use at work

This information is based on student

inferences and public sources, and

has not been reviewed or

endorsed by the NSA.

Integrated Office+ Mandate: bridge the gap between A/B/C (etc.), and innovators outside of agency

where we were initially focused


Weeks 3-4




technical focuses


A/B DirectorateSeniors+ Control funding+ Indirect knowledge of tech being used/desired+ Mostly male, 40s-50s, 15+ years @ agency








Weeks 3-4




technical focuses


A/B Directorate C DirectorateSeniors+ Control funding+ Indirect knowledge of tech being used/desired+ Mostly male, 40s-50s, 15+ years @ agency

C Seniors+ Controls funding+ Indirect knowledge of tech

being used/built

C Developers+ Grades 13/14+ Technical focus

+ Mostly male, (demographics TBD)+Advance the cutting edge research that helps analysts do their job (algo focused)








Weeks 3-4




technical focuses




being used/built




D Directorate+ Developers+ Build tools to make analyst life easier / more effective







Weeks 3-4




technical focuses




being used/built




D Directorate

Team E (exact location unknown)

+ Renders pre-existing external tools on high-side (e.g. Google)

+ Developers+ Build tools to make analyst life easier / more effective







Weeks 3-4


Several weeks of desk research and interviews to arrive at current understanding of org and key players...


Weeks 3-4


Products& Services

Publicly available social media search

engine and classifier

+ Develop in capabilities currently underserved by Agency technology (publicly available information, social media)+ Create algos to directly fulfill mandate Customer

JobsHigh level guidance to

org, cutting edge researching,

developing new algos- Trying to balance differentorganizational priorities- Difficulty keeping pace with technology industry and academia-Convincing analysts to use latest methods and algos

C Seniors @ Agency

Gains

Pains

Gain Creators

Pain Relievers

+ Keep Agency closer to cutting edge of social media technology+ Augmenting tools and processes with breadth of cutting-edge research

+ Algos that directly meet the demands of analysts will reduce time spent by C researching and developing in-house

Primary beneficiary:4 of 10

Demographics● 15+ years at Agency● Very technical background

Agency

... And value proposition for all agency stakeholders involved

Weeks 3-4





Began exploring beneficiaries at similar agencies, branched out to beneficiaries outside gov’t: first dual-use hypothesis



1

2

5

4

3


Came back to Steve’s early clue in week ~4-5, realizing the need was far greater than just NSA, and a dual-use solution could be better for all customers & fundraising

Mapped landscape of all possible beneficiaries, comparing the needs and value-proposition for each.

Ultimate goal is to find customers (a) with high need and (b) whose data could make our classifier more powerful

Began deep dive into several of these customers

Weeks 4-5


Product Beneficiary Mission Achievement

Open-source

agency tool

A/B Analysts Automation, insight, and timeliness in understanding entities

A/B Seniors Minimize pain in bringing in new tools, increase robustness of reports

C Dev Code base that can be adopted painlessly, pass security check

C Seniors Bolster existing algos with publicly available data

E Successful, painless rendering of low-end tool on high side

Agency Director Improve understanding of signal intelligence, information assurance

Classified agency tool

v1.0 Beneficiaries In addition to above - integrating analysis/insights with classified data

D Directorate Developing easily deployable high-end tool that builds off the success of user experience in v1.0, managing transition from 1.0 to 2.0

Dual-use tool

Consumers Notified when interacting with a potentially fake account, or blocked

Commercial IA No employees lose credentials to catfishing attempt. Notified of attempts

Gov’t IA teams All fake account interactions with personnel is flagged and/or stopped

Social media co Fake accounts are flagged and removed before harm is done

Tricky part is that “Mission Achievement” varies by the beneficiaries - still figuring out how to balance...


Weeks 4-5






Whoops: Security concern. Big setback



1

2

5

4

3

6







Picked up momentum in developing solution for agency



1

2

5

4

3

7

6


Reviewed academic literature on anomaly detection in social media; conferred with sponsor and mentors

Weeks 8-9


The last several weeks have been spent sprinting on a non-powerpoint MVP to use with the NSA

Weeks 8-9










1

2

5

4

3

7

6

Weeks 8-9










1

2

5

4

3

7

6

So where are we now, in week 10?


We’ve identified needs for both government and non-government, proactive and defensive use cases

The Hypothesized World of our Addressable Market

Proactive Needs Defensive Needs

Government

● Entity Enrichment: Understanding online monikers, the authors behind them, and their behavior and identities across various social media platforms

● Traditionally invested in classified data, need to better leverage open-source data

Anything to replace ineffective spear phishing training

“Currently orgs give their employees training so they can recognize incoming spear phishing attacks. Well I bet you can guess how well that works. I mean, how many people keep it in their pants after taking sex ed?” -Interview with VC investor

Non-government

● Offering a service to users that is unmuddied by fake accounts

● Typically make effective use of proprietary data and some open-source; unclear how well they take advantage of data off their respective platforms

Week 10


We believe the dual-use option can pick up significant momentum relative to a bespoke solution

Week 10


We believe the dual-use option can pick up significant momentum relative to a bespoke solution

...But the major, immediate need is verifying hypotheses outside of the NSA

Week 10

IRL 1

IRL 4

IRL 3

IRL 2

IRL 7

IRL 6

IRL 5

IRL 8

IRL 9

First pass on MMC w/Problem Sponsor

Complete ecosystem analysis petal diagram

Validate mission achievement (Right side of canvas)

Problem validated through initial interviews

Prototype low-fidelity Minimum Viable Product

Value proposition/mission fit (Value Proposition Canvas)

Validate resource strategy (Left side of canvas)

Prototype high-fidelity Minimum Viable Product

Establish mission achievement metrics that matterTeam Assessment: IRL 5

Post H4D Course Actions

Team fishreel validated relevant needs for intel analysts at the NSA and mocked up a “Beta” solution - however, the next step is validating the dual-use case by continuing to test hypotheses with beneficiaries outside the NSA

Week 10


We’ve sketched out the minimum resource needs if we were to continue beyond Week 10

Week 10


Thank you!


Fishreel is four Stanford student standing on the shoulders of several giants:

Contact Role Org

SPONSORS - Two really great public servants in particular, 50+ others

Sponsor

Brandon Johns DIUX Liaison

Leora Morgenstern Mentor

Tushar Tambay Mentor

Matt Johnson Mentor

Guy Mordecai Mentor

Contact Role Org

Eric Smyth Mentor

Aaron Sander Mentor

Wayne E Chen Mentor

Brad Dispensa Technical SME

Matt Jamieson Technical SME

Lieutenant Colonel Scott Maytan

Military Liaison

Lieutenant Colonel Ed Sumangil

Military Liaison

And many more...


Backup: MMCS

- Data (training and validation sets)- Processing power- Communication with customer (DoD analyst needs, existing infrastructure, etc)

Mission Model Canvas: Week 1

- Software Engineering- Automated ML model building- API integrations

- UI/UX Design- Information produced must be easily interpreted by analyst

- Continued sponsorship by defense beneficiary

- Government entities with established catfishing detection algorithms, databases, infrastructure

- Public streaming data sources (e.g. Twitter, Facebook, YouTube)

- Possibly, data sources for private/encrypted communications (encrypted consumer applications like Kik and Telegram, popular among ISIS recruiters)

- Predictive modeling/data analysis companies

-Technical partners w/ processing power

- Scholars in behavioral psychology and interdisciplinary fields relevant to catfishing

- Primary: Intelligence analysts trying to find catfishing attempt underway so they can intervene and prevent security breaches.

- Secondary: Catfishing targets pursued by non-state, insurgent actors. This includes intelligence analysts who may be subject to social engineering attacks by sophisticated/state actors, as well as young westerners being targeted by ISIS.

- Secondary: Users interacting with intelligence analysts, e.g., superiors, finance officers

- Secondary: Private companies that provide profile / social component with incentive to bolster existing technology

Intelligence Orgs- Understand adversary behavior and strategy: gain knowledge about channels used by adversaries

- Provide predictive insight: based on target’s historic activity, predict likely future activity

- Improve existing model for detecting catfishing and authenticating identity

Catfishing victims- Prevent recruiting under false identities: e.g. ISIS uses twitter to recruit westerners

- Prevent transmission of sensitive data to bad actors:

- Improve speed and accuracy of identity authentication- Reduce occurrences of catfishing-Better communicate predictive power and logic of machine learning

- Pilot test: Use models to predict likelihood of catfishing in historical data - Initial deployment with specific focus on ISIS within the DoD - Initial focus on publicly available data to make case to DoD- Broader deployment: more use cases within DoD- Combine findings with private sector, in-house solutions

Fixed:- Processing costs (AWS servers, virtual machine space for data processing)

Variable:- Electricity to power servers

- Need access to DoD data sources

- Support from both public and encrypted social media and data service sites

- Need understanding of existing DoD catfishing prevention intelligence and infrastructure

Beneficiaries

Mission AchievementMission Budget/Costs

Buy-In/Support

Deployment

Value PropositionKey Activities

Key Resources

Key Partners


- Data Scraping- Pull publicly available catfishing data- Software Engineering- Automated ML model building- API integrations

- UI/UX Design- Information produced must be easily interpreted by analyst- Results should be able to be tweaked and further explored


- Continued sponsorship by defense beneficiary- Other Government agencies with or without established catfishing detection / defense- Public streaming data sources (e.g. Twitter, Facebook, YouTube)- Possibly, data sources for private/encrypted communications (encrypted consumer applications like Kik and Telegram, popular among ISIS recruiters)- Predictive modeling/data analysis companies-Technical partners w/ processing power- Scholars in CS, behavioral psychology, and interdisciplinary fields relevant to catfishing

- Primary: Intelligence analysts trying to proactively find catfishing attempt outside the Agency network so they can intervene and prevent security breaches.- Secondary: Agencies who need to augment network defense against catfishing- Secondary: Catfishing targets pursued by non-state, insurgent actors. This includes intelligence analysts who may be subject to social engineering attacks by sophisticated/state actors, as well as young westerners- Secondary: Users interacting with intelligence analysts, e.g., superiors, finance officers- Tertiary: Private companies that provide profile / social component with incentive to bolster existing technology

- Provide users investigating catfishing with better speed & accuracy, and flexibility in classifying online activity- Better communicate underlying logic of machine learning- Provide dynamic solution to analysts

Primary users:- Classify behavior as “catfishing” or normal - Understand adversary behavior and strategy: gain knowledge about channels used by adversaries- Improve existing model for detecting catfishing and authenticating identity- Reduce burden on analysts by automating review of catfishing input data

Secondary users: Improve existing defenses against catfishing

Tertiary users: Augment private companies’ ability to detect catfishing within walled gardens

- Pilot test: Use models to predict likelihood of catfishing in historical data- Initial focus on publicly avail. data, make case to DoD- Broader deployment: more use cases within DoD use cases w/ publically avail. data- Combine findings with other agencies and private sector, in-house solutions



-Agency sponsors/legal team- Need access to DoD data sources- Support from both public and encrypted social media and data service sites- Understanding of existing DoD catfishing prevention intelligence and infrastructure

Beneficiaries


Buy-In/Support

Deployment


Key Resources

Key Partners

- Data (training and validation sets)- Processing power- Communication with customer (Agency agencies’ analyst needs, existing infrastructure, etc)


dfd

sf

- Data- Processing power- Communication with customer (analyst needs, existing infrastructure, etc)

- Data Scraping- Pull publicly available catfishing data- Software Engineering- Automated ML model building- API integrations



- Continued sponsorship by defense beneficiary- Other Government agencies with or without established catfishing detection / defense- Public streaming data sources (e.g. Twitter, Facebook, YouTube)- Predictive modeling/data analysis companies-Technical partners w/ processing power- Scholars in CS, behavioral psychology, and interdisciplinary fields relevant to catfishing

- Primary: Intelligence analysts trying to proactively find catfishing attempt outside the Agency network so they can intervene and prevent security breaches.

- Secondary: Agencies who need to augment network defense against catfishing spearfishing. This includes the FBI, DoD, and others within The Agency

- Tertiary: Private companies (high priority: banks, utilities, critical infrastructure)

- Users want to grab a *working model* from FIshreel hands- Provide users investigating catfishing with better speed & accuracy, and flexibility in classifying online activity- Better communicate underlying logic of machine learning- Provide dynamic solution to analysts

Primary users:- Classify behavior userIDs as “catfishing” or normal: 1) Real, 2) Bot, 3) Impersonator, 4) Spoof

- Understand adversary identity / archetypes, behavior, and strategy- Reduce manual burden on analysts by automating review of catfishing input data

Secondary users: Improve existing defenses against catfishing spearfishing


- Pilot test: Use models to predict likelihood of catfishing in historical data- Focus on publicly avail. data, make case to DoD- Broader deployment: use cases w/ publically avail. data- Combine findings with other agencies and private sector, in-house solutions



- Understanding of existing DoD catfishing prevention intelligence and infrastructure- Technical support for MVP backend infrastructure

Beneficiaries


Buy-In/Support

Deployment


Key Resources

Key Partners


dfd

sf


- Data Scraping- Pull publicly available catfishing social media data- Software Engineering- Automated ML model building- API integrations




- Primary: Intelligence analysts trying to proactively find catfishing attempt outside the Agency network so they can intervene and prevent security breaches.

- Secondary: Agencies who need to augment network defense against spearfishing. This includes the cybercrimes agents in the FBI, DoD, and others within The Agency

- Tertiary: Cybersecurity analysis in private companies (high priority: banks, utilities, critical infrastructure)

- Users want to grab a *working model* from FIshreel hands- Provide users investigating catfishing with better speed & accuracy, and flexibility in classifying online activity- Better communicate underlying logic of machine learning- Provide dynamic solution to analysts

Primary users:- Classify userIDs as “catfishing” or normal: 1) Real, 2) Bot, 3) Impersonator, 4) Spoof

- Understand adversary identity / archetypes, behavior, and strategy- Reduce manual burden on analysts by automating review of catfishing input data

Secondary users: Improve existing defenses against catfishing spearfishing


+ 2 routes: 1) mandated by top leaders (rare)2) Small teams pilot, if useful & good UI word of mouthBoth take ~2 years+ Dist: Standalone tool, purchased directly by small team (hypothesis)- Pilot test: Use models to predict likelihood of catfishing in historical data - Focus on publicly avail. data, make case to DoD- Broader deployment: use cases w/ publically avail. data

- Combine findings with other agencies and private sector, in-house solutions



- Understanding of existing DoD catfishing prevention intelligence and infrastructure

- Technical support for MVP backend infrastructure: daily maintenance of data scraping- Approval of security/legal depts and buy-in from team lead / manager

Beneficiaries


Buy-In/Support

Deployment


Key Resources

Key Partners


Sponsor agency: → S2/S3 Analysts→ S2/S3 Chiefs→ R6 Developers→ R6 Chiefs→ DoDDIR

Other agencies: → FBI - SSA→ DOD - ?

- Commercial:→ Banks→ Utilities→ Critical infrastructure→ Social Media

dfd

sf


- Data Scraping- Pull publicly available social media data- Software Engineering- Automated ML model building- API integrations




- Users want to grab a *working model* from FIshreel hands- MINIMIZE FALSE POSITIVES AT START, FALSE NEGATIVES LONG RUN- Provide users investigating catfishing with better speed & accuracy, and flexibility in classifying online activity- Better communicate underlying logic of machine learning- Provide dynamic solution to analysts

S2/S3 Analysts: Use ML techniques to quickly, effectively classify userIDs as “catfishing” or normal: 1) Real, 2) Bot, 3) Impersonator, 4) Spoof. S2/S3 Chiefs: Improved speed of reports emanating from agency. Increased credibility by relying on latest methods.R6 Developers: “version 1.0” is quick and painless in deploy.R6 Chiefs: Fulfill mandate to create / identify pain-relieving tools

Other agencies: Improve existing defenses against spearfishing

Commercial: Augment private companies’ ability to detect catfishing within walled gardens

+ 2 routes: 1) mandated by top leaders (rare)2) Small teams pilot, if useful & good UI word of mouthBoth take ~2 years+ Dist: Standalone tool, purchased directly by small team (hypothesis)- Combine findings with other agencies and private sector, in-house solutions




Beneficiaries


Buy-In/Support

Deployment


Key Resources

Key Partners


Sponsor agency: → S2/S3 Analysts→ S2/S3 Chiefs→ R6 Developers→ R6 Chiefs→ DoDDIR

Other agencies: → FBI - SSA→ DOD - ?

- Commercial:→ Banks→ Utilities→ Critical infrastructure→ Social Media→ Dating Sites

dfd

sf






- Users want to grab a *working model* from FIshreel hands- ASSIGNING COSTS TO FALSE POSITIVES V. FALSE NEGATIVES- Provide users investigating catfishing with better speed & accuracy, and flexibility in classifying online activity- Better communicate underlying logic of machine learning- Provide dynamic solution to analysts

S2/S3 Analysts: Use ML techniques to quickly, effectively classify userIDs as “catfishing” or normal: 1) Real, 2) Bot, 3) Impersonator, 4) Spoof. S2/S3 Chiefs: Improved speed of reports emanating from agency. Increased credibility by relying on latest methods.R6 Developers: “version 1.0” is quick and painless in deploy.R6 Chiefs: Fulfill mandate to create / identify pain-relieving tools

Other agencies: Improve existing defenses against spearfishing

Commercial: Augment private companies’ ability to detect catfishing within walled gardens





Beneficiaries


Buy-In/Support

Deployment


Key Resources

Key Partners



Version 1.0:Sponsor agency: 1. A/B Analysts2. A/B Seniors3. C Developers4. C Seniors5. E6. DoDDIR

Version 2.0:→ All of v1.07. D

Version 3.0:8. Consumers9. Other agencies: → FBI - SSA→ DOD - ?10. Commercial:→ Banks→ Utilities→ Critical infrastructure→ Social Media→ Dating Sites

dfd

sf






A/B Analysts: Use ML techniques to better understand online personas using publicly available data.A/B Seniors: Improved speed of reports emanating from agency. Increased credibility by relying on latest methods.C Developers: “version 1.0” is quick and painless in deploy.C Seniors: Mandate to create / identify pain-relieving toolsE: Minimum technical difficulty in rendering tool, traffic from other usersDoDDIR: Enhance understanding of signal intelligence, better defense against cyber attacksD: Seamless transition to 2.0Consumer: Protect selfOther agencies: Improve existing defenses against spearfishingCommercial: Augment private companies’ ability to detect catfishing within walled gardens





Beneficiaries


Buy-In/Support

Deployment


Key Resources

Key Partners

Consumers: Notified when interacting with a potentially fake account, or blockedGov’t IA teams: All fake account interactions with personnel is flagged and/or stoppedCommercial IA: No employees lose credentials to catfishing attempt. Notified of attemptsSocial media co: Fake accounts are flagged and removed before harm is done

-A/B Analysts:: Automation, insight, and timeliness in understanding entities -A/B Seniors: Minimize pain in bringing in tools, up robustness of reports-C Developers: Code base that can be adopted painlessly, clear security-C Seniors Bolster existing tools w/ tool oriented toward publicly available data-E: Successful, painless rendering of low-end tool on high side -DoDDIR: Improve understanding of signal intelligence- D: Developing easily deployable high-end tool that builds off the success of user experience in v1.0, managing transition from 1.0 to 2.0




Version 3.0:8. Consumers9. Other agencies: → FBI - SSA→ DOD - ?10. Commercial:→ Banks→ Utilities→ Critical infrastructure→ Social Media→ Dating Sites→ Municipalities

dfd

sf

+Time+Connections+Publicly available data+Execution, Product Dev, Research Talent+Sponsor connection+Killer val-prop+Encryption / 3.0 security+Money+Onboarding/HR process+Servers, architecture+Dual-use pitch

+3.0 Beneficiaries/Value Prop+Gather/process ortho data+Build model on ortho data+Push 1.0 backend code +Dev, host, render 1.0+ID early third-party adopters +Dev SaaS version+ID next wave of adopters+Raise money, build 3.0 team+Buy addt’l third-party data+Process all new data+Build model on new data+Sell “2.0” as dual-use


+ Channels to more orthogonal data/ creative solutions b4 data partnership

● Product Dev XP, e.g., CTO visionaries

● PhD resources, Researchers, Stanford U

+ Early 3rd party adopters● Smaller co’s that still

have useful data, e.g., schools, munis, critical local infr.

+3.0 data security partners +High-data-value adopters+VCs






Beneficiaries


Buy-In/Support

Deployment


Key Resources

Key Partners







dfd

sf



Mission Model Canvas Week 9








Sources+$100k in first seed funding to cover basic continuation of work beyond class+$400k in winter to cover major variable cost expansion+??? in Series A at 12 months+??? Series B at 24 month

Uses+Almost entirely variable for first 12 months: Founders salary: $42Building model: $48Develop SaaS: $48Non-tech labor: $26Paid data $180Servers: $40Additional eng: $96Total: $480 for first 12 months



Beneficiaries

Mission Achievement

Mission Budget/Costs

Buy-In/Support

Deployment


Key Resources

Key Partners







dfd

sf



Mission Model Canvas Week 10








Sources+$100k in first seed funding to cover basic continuation of work beyond class+$400k in winter to cover major variable cost expansion+??? in Series A at 12 months+??? Series B at 24 month

Uses+Almost entirely variable for first 12 months: Founders salary: $42Building model: $48Develop SaaS: $48Non-tech labor: $26Paid data $180Servers: $40Additional eng: $96Total: $480 for first 12 months



Beneficiaries

Mission Achievement

Mission Budget/Costs

Buy-In/Support

Deployment


Key Resources

Key Partners



fishreel lessons learned h4d stanford 2016

Education