03 december 2012 by abe lederman, ceo

© 2012 Deep Web Technologies, Inc.

03 December 2012By Abe Lederman, CEO

Deep Web TechnologiesShow and Tell

Presentation to

© 2012 Deep Web Technologies, Inc. 2

Abe Lederman

Deep Web Technologies was founded by Abe Lederman in 2002.

–BS & MS Degrees in Computer Science from MIT

–A co-founder of Verity, acquired by Autonomy (now HP)

–Developed SciSearch@LANL (part of “Library without Walls”)

–25 years experience in Information Retrieval

© 2012 Deep Web Technologies, Inc. 3

About Deep Web Technologies...

• 20 person company based in Santa Fe, New Mexico

• Over $5M in DOE SBIR Grants (2003-2011)

• Pioneer/trailblazer in federated search

• 100+ solutions in production

© 2012 Deep Web Technologies, Inc.

Customers Include...

Government:• Defense Technical Info

Center (DTIC)• Office of Sci. & Tech. Info

(DOE-OSTI)• UN Economic Comm. for

Africa (UNECA)• European Space Agency

Corporate:• Boeing • BASF• Intel• HP• P&G

Academic:• Stanford University• George Mason

University• Texas Medical Center• University College of

Cork

Public Portals:• WorldWideScience.org• Science.gov• Biznar• Mednar• ScienceResearch.com

© 2012 Deep Web Technologies, Inc. 5

Develop 3 POC’s (Top 10 DB, 5 Catalogs, Digital Repositories

Launch xSearch for Science & Engineering (28 sources)

Expand xSearch to include Social Sciences & Humanities. Also, expanded later in the year for GSB sources (170 sources)

In November 2011 the Charleston Advisor Review was published

Upgrade and Expand xSearch to 200 sources in December 2012

History of Partnership

2007

2010

2011

2011

2012

© 2012 Deep Web Technologies, Inc.

2008 PR

© 2012 Deep Web Technologies, Inc.

© 2012 Deep Web Technologies, Inc. 8

© 2012 Deep Web Technologies, Inc. 9

© 2012 Deep Web Technologies, Inc. 10

Federated Search allows users to submit a real-time search in parallel to multiple information sources and retrieve aggregated, ranked and de-duplicated results.

What Is Federated Search?

© 2012 Deep Web Technologies, Inc. 11

In Other Words…One Search, Many Sources

Internal Sources

Blogs & Wikis

SubscriptionSources

Public Web Sources

Reports

News & Social Media

Begin Search

© 2012 Deep Web Technologies, Inc. 12

xSearch Status

• Upgraded early fall to v. 3.2.2• GSB linked to xSearch• 200 collections in application• 30 new connectors in acceptance

testing (roll-out imminent)

© 2012 Deep Web Technologies, Inc.

Janu

ary

Mar

chMay Ju

ly

Sept

embe

r

Novem

ber

0

1000

2000

3000

4000

5000

6000

7000

User Queries by Month/Year

2012 User Queries2011 User Queries2010 User Queries

© 2012 Deep Web Technologies, Inc.

Janu

ary

Febr

uary

Mar

chAp

ril

May

June

July

Augu

st

Sept

embe

r

Octob

er

Novem

ber

Decem

ber

0100,000200,000300,000400,000500,000600,000700,000

Source Queries

2012 Source Queries 2011 Source Queries2010 Source Queries

© 2012 Deep Web Technologies, Inc.

Web

of S

cien

ce

ABI/I

nfor

m G

loba

l

PubM

ed

Enviro

nmen

tal S

cien

ces & P

ollu

tion

Man

agem

ent

Engi

neer

ing

Villa

ge

Sociol

ogical

Abs

tract

s

Busine

ss S

ourc

e Com

plet

e

Scop

us

JSTO

R

ACS

Publ

icat

ions

Perio

dica

ls A

rchi

ve O

nlin

e

Proj

ect M

use

0

2000

4000

6000

Top click-throughs: Jan 1, 2012 – November 30, 2012

© 2012 Deep Web Technologies, Inc. 16

Explorit Release 3.2.3Starting customer upgrades

• Visual clusters• Full-text filters• Content type/Media

type• Integration with

Zotero, Mendeley

© 2012 Deep Web Technologies, Inc. 17

© 2012 Deep Web Technologies, Inc. 18

© 2012 Deep Web Technologies, Inc. 19

© 2012 Deep Web Technologies, Inc.

© 2012 Deep Web Technologies, Inc. 21

Explorit Release 4.0Coming mid-2013

• Dynamic tab searching• Thesaurus-based searching

(Do you also want to search for?)

• Personal library• Big Data mashups

(Enhanced content)• Faceted navigation

© 2012 Deep Web Technologies, Inc. 22

Big Data Mashups

© 2012 Deep Web Technologies, Inc. 23

Related Content

• Major Science portals • Science News • Patent Databases • Scholar Networks• Subscription Sources• Public Databases• Open Access Journals

© 2012 Deep Web Technologies, Inc.

Article Grouping

© 2012 Deep Web Technologies, Inc.

Non-Invasive imaging

© 2012 Deep Web Technologies, Inc.

Linking Open Data Cloud Diagram by Richard Cyganiak and Anja Jentzsch. http://lod-cloud.net/

© 2012 Deep Web Technologies, Inc.

Nature has 297 million triples.

© 2012 Deep Web Technologies, Inc. 28

Journey to 10,000 sources

© 2012 Deep Web Technologies, Inc. 29

Scalability Challenges

• Source selection • Ranking and organizing of results • Traffic management • System load management • Finding, building, and maintaining

connectors

© 2012 Deep Web Technologies, Inc. 30

Scalability - Divide and Conquer

ScienceResearch.com

WorldWideScience.org

Other Federated Search Engines

ScienceAccelerator

Science.gov

© 2012 Deep Web Technologies, Inc. 31

© 2012 Deep Web Technologies, Inc. 32

© 2012 Deep Web Technologies, Inc. 33

Multilingual WorldWideScience.org

© 2012 Deep Web Technologies, Inc. 34

How Multilingual Federated Search Works

Ranked resultstranslated by Microsoft to user’s language

Results returned to user

EXPLORIT

Microsoft Translator

German

Chinese

Russian

Queryin user’s language

Ranked resultsin user’s language

Queryto be translatedfor each source

Queryin source’slanguage

Foreign language

search engines

Resultsin source’slanguage

Ranking

© 2012 Deep Web Technologies, Inc.

© 2012 Deep Web Technologies, Inc.

Translated

Original

© 2012 Deep Web Technologies, Inc.

© 2012 Deep Web Technologies, Inc.

© 2012 Deep Web Technologies, Inc.

© 2012 Deep Web Technologies, Inc. 40

© 2012 Deep Web Technologies, Inc. 41

ESN – x2

© 2012 Deep Web Technologies, Inc. 42

© 2012 Deep Web Technologies, Inc. 43

© 2012 Deep Web Technologies, Inc. 44

UNECA ASKIA Portal (United Nations – Access Scientific Knowledge in Africa)

© 2012 Deep Web Technologies, Inc. 45

© 2012 Deep Web Technologies, Inc. 46

© 2012 Deep Web Technologies, Inc. 47

© 2012 Deep Web Technologies, Inc.

© 2012 Deep Web Technologies, Inc.

© 2012 Deep Web Technologies, Inc. 50

BASF

© 2012 Deep Web Technologies, Inc. 51

© 2012 Deep Web Technologies, Inc. 52

© 2012 Deep Web Technologies, Inc. 53

© 2012 Deep Web Technologies, Inc. 54

© 2012 Deep Web Technologies, Inc. 55

© 2012 Deep Web Technologies, Inc. 56

Find It!TMC’s Link Resolver

© 2012 Deep Web Technologies, Inc. 57

© 2012 Deep Web Technologies, Inc. 58

© 2012 Deep Web Technologies, Inc. 59

© 2012 Deep Web Technologies, Inc. 60

© 2012 Deep Web Technologies, Inc.

© 2012 Deep Web Technologies, Inc. 62

© 2012 Deep Web Technologies, Inc. 63

Abe’s Stanford ProjectsWISHLIST

• Assist SULAIR in integrating Explorit preview via Web Services into new library portal.

• Integration with SearchWorks

• Develop a stand-alone portal focused on Stanford core-competency (Energy, Environment, …)

© 2012 Deep Web Technologies, Inc. 64

Abe’s Stanford ProjectsWISHLIST (cont.)

• Develop Chinese Explorit in collaboration with Library of Congress and/other library

• xSearch / Explorit for Stanford Medical School

• Mash up Big Data (Link Data, Mendeley, citations) and articles

• Data Portal (expansion of Data searching in WorldWideScience.org)

• Integration with Sakai or other CMS

© 2012 Deep Web Technologies, Inc. 65

Explore our Applications

• xSearch• WorldWideScience.org• Science.gov• Ciencia.Science.gov• DTIC Multisearch

© 2012 Deep Web Technologies, Inc. 66

Thank you!

Abe Ledermanabe@deepwebtech.com

View this presentation online