1
Collaborate)with)AI)(HLT))agents:)find)knowledge)you)are)missing)
in)your)tools)and)data.))
Partnered)with))
2
KM)via)Knowledge)Graphs)
Vendors Signal for prioritizing edges
Wikipedia, MusicBrainz, imdb Interests of volunteer hobbyists
Google, Bing, Baidu Web trends
Facebook, Twitter, LinkedIn Social graph, contact graph
Several Fintech startups External events, e.g. iPhone launches
Diffeo (only) You as viewed through your digital work.
The graph of knowledge is staggeringly large. BellNumber(n) of possible associations is combinatorial explosive. e.g. B(26) = 49,631,246,523,618,756,274 All practical approaches to knowledge graphs use some strategy to cut down the combinations of interest.
3
Agenda)
• What is a collaborative agent? • How do we measure collaborative HLT • Architectures & algorithms for agents
3"
4
Agenda)
• What is a collaborative agent? • How do we measure collaborative HLT? • Architectures & algorithms for agents
4"
5
Diffeo is “augmented intelligence … and won the grand prize by a considerable margin,” Col. DiPaolo NGA Disparate Data Challenge
● Users invite the Diffeo agent into familiar tools, like Firefox and Outlook, so it can help you uncover connections.
● Grew out of research at DARPA Memex and NIST’s TREC Knowledge Base Acceleration.
● Operational in multiple environments.
6
Diffeo)CollaboraGve)AI)PlaHorm)Research Assistant
OpsCenter Teammate
Org-wide Tasking
Dynamic Knowledge Graph
Digital Work Sensors Collaborative Agents
40+ more deep Web sources
Multi-tenant knowledge graph fed by the digital lives of users is a platform for a new kind of machine intelligence… AI 2.0
7
Teddy Counterparty
Analysis
Sam Account Manager
Christoph Supply Chain
Analyst
Diffeo)Users
Exploring internal and external content, …
Cogan All-Source
Analyst
8
Gets a task
Report back with action recommendation
Teddy’s)workflow)
Explores internal and external content, looking for connections
Uncovers a story… Is it the whole story?
9
Can you please dig in to Freeport McMoRan, we’re considering a major deal with them. More information to follow shortly.
Freeport McMoRan Info
Teddy’s)workflow)
10
Teddy’s)workflow)
11
Teddy’s)workflow)
12
Teddy’s)workflow)
13
Teddy’s)workflow)
14
Teddy’s)workflow)
15
Teddy’s)workflow)
16
Teddy’s)workflow)
17 Many sources of content
Teddy’s)workflow)
18 Many collaborators, each with their own content
Teddy’s)workflow)
19 Many collaborators, each with their own content
Teddy’s)workflow)
20 20"
What)am)I)missing?)
Humans must act as data switchboard operators.
21 21"
Machine intelligence reads your text while you write and uncovers what you missed
Highlights show knowledge gaps relative to your document
What)am)I)missing?
CollaboraGve)agent)dynamically)highlights)gaps.)
Diffeo connects the windows for you.
22 22"
Recommendations from across the web and your private data recommend evidence that uncovers connections
What)am)I)missing?
23 23"
What)am)I)missing?
24 24"
What)am)I)missing?
25 25"
What)am)I)missing?
26 26"
What)am)I)missing?
28
Your)InteracGve)Knowledge)Graph)
Diffeo builds out your knowledge graph as you explore.
29 Diffeo builds out your knowledge graph as you explore.
Evidence of connections from the Web, your files, and subscription data.
Your)InteracGve)Knowledge)Graph)
30
Agenda)
• What is a collaborative agent? • How do we measure collaborative HLT? • Architectures & algorithms for agents
30"
31
How)deep)in)the)list?)
31"
User skipped to #3 and then #5. Score is average of 1/3 + 1/5
KBAa TREC evaluation
1"
10"
100"
2011%10%05%00"
2011%10%10%17"
2011%10%16%09"
2011%10%22%01"
2011%10%27%17"
2011%11%02%09"
2011%11%08%01"
2011%11%13%17"
2011%11%19%09"
2011%11%25%01"
2011%11%30%17"
2011%12%06%09"
2011%12%12%01"
2011%12%17%17"
2011%12%23%09"
2011%12%29%01"
2012%01%03%17"
2012%01%09%09"
2012%01%15%01"
2012%01%20%17"
2012%01%26%09"
2012%02%01%01"
2012%02%06%17"
2012%02%12%09"
2012%02%18%01"
2012%02%23%17"
2012%02%29%09"
2012%03%06%01"
2012%03%11%17"
2012%03%17%09"
2012%03%23%01"
2012%03%28%17"
2012%04%03%09"
2012%04%09%01"
2012%04%14%17"
2012%04%20%09"
2012%04%26%01"
2012%05%01%17"
2012%05%07%09"
2012%05%13%01"
2012%05%18%17"
2012%05%24%09"
2012%05%30%01"
2012%06%04%17"
2012%06%10%09"
2012%06%16%01"
2012%06%21%17"
2012%06%27%09"
2012%07%03%02"
2012%07%08%20"
2012%07%14%12"
2012%07%20%04"
2012%07%25%20"
2012%07%31%12"
2012%08%06%04"
2012%08%11%20"
2012%08%17%12"
2012%08%23%04"
2012%08%28%20"
2012%09%03%12"
2012%09%09%04"
2012%09%14%20"
2012%09%20%12"
2012%09%26%04"
2012%10%01%20"
2012%10%07%12"
2012%10%13%04"
2012%10%18%20"
2012%10%24%12"
2012%10%30%04"
2012%11%04%20"
2012%11%10%12"
2012%11%16%04"
2012%11%21%20"
2012%11%27%12"
2012%12%03%04"
2012%12%08%20"
2012%12%14%12"
2012%12%20%04"
2012%12%25%20"
2012%12%31%12"
2013%01%06%04"
2013%01%11%20"
2013%01%17%12"
2013%01%23%04"
2013%01%28%20"
2013%02%03%12"
2013%02%09%04"
Corpus"counts"per"hour"
Pre6hoc"assess"all"surface"men;ons"
“vital”"docs"change"en;ty"profile"!"filling"a"slot?"
training"
evalua;o
n"
KBAa TREC evaluation
1"
10"
0" 500" 1000" 1500" 2000" 2500" 3000" 3500" 4000" 4500" 5000"
log(coun
t))
Aharon_Barak" Alex_Kapranos" Alexander_McCall_Smith"Annie_Laurie_Gaylor" Basic_Element_(company)" Basic_Element_(music_group)"Bill_Coen" Boris_Berezovsky_(businessman)" Boris_Berezovsky_(pianist)"Charlie_Savage" Darren_Rowse" Douglas_Carswell"Frederick_M._Lawrence" Ikuhisa_Minowa" James_McCartney"Jim_Steyer" Lisa_Bloom" Lovebug_Starski"Mario_Garnero" Masaru_Emoto" Nassim_Nicholas_Taleb"Rodrigo_Pimentel" Roustam_Tariko" Ruth_Rendell"Satoshi_Ishii" Vladimir_Potanin" William_Cohen"William_D._Cohan" William_H._Gates,_Sr"
35
Agenda)
• What is a collaborative agent? • How do we measure collaborative HLT? • Architectures & algorithms for agents
35"
36 36"
37
Knowledge)OperaGons)Journal)
38
Knowledge)OperaGons)Journal)
39
Knowledge)OperaGons)Journal)
40
Knowledge)OperaGons)Journal)
41
Knowledge)OperaGons)Journal)
42
Knowledge)OperaGons)Journal)
43
Knowledge)OperaGons)Journal)
44 Collaborative Machine Intelligence
Knowledge)OperaGons)Journal)
45
1. Gap analysis of user’s current task
2. Hierarchical Agglomerative Clustering
3. Automatic Query Formulation
4. Deep Web Mining
5. Interactive Knowledge Graph
Forest)graphs)for)enGty)disambiguaGon)
(US Patent 9,275,132)
Instead of seeking similar data, users seek decision boundaries –– the edge of their knowledge.
49
CollaboraGve)AI))amplifies)human)analysts)by:) • Digging in more places & languages • Uncovering connections • Exposing gaps • Testing hypotheses
49"
50
Collaborate)with)AI)(HLT))agents:)find)knowledge)you)are)missing)
in)your)tools)and)data.))
Partnered)with))