03 december 2012 by abe lederman, ceo

© 2012 Deep Web Technologies, Inc.

03 December 2012By Abe Lederman, CEO

Deep Web TechnologiesShow and Tell

Presentation to

© 2012 Deep Web Technologies, Inc. 2

Abe Lederman

Deep Web Technologies was founded by Abe Lederman in 2002.

–BS & MS Degrees in Computer Science from MIT

–A co-founder of Verity, acquired by Autonomy (now HP)

–Developed SciSearch@LANL (part of “Library without Walls”)

–25 years experience in Information Retrieval


About Deep Web Technologies...

• 20 person company based in Santa Fe, New Mexico

• Over $5M in DOE SBIR Grants (2003-2011)

• Pioneer/trailblazer in federated search

• 100+ solutions in production


Customers Include...

Government:• Defense Technical Info

Center (DTIC)• Office of Sci. & Tech. Info

(DOE-OSTI)• UN Economic Comm. for

Africa (UNECA)• European Space Agency

Corporate:• Boeing • BASF• Intel• HP• P&G

Academic:• Stanford University• George Mason

University• Texas Medical Center• University College of

Cork

Public Portals:• WorldWideScience.org• Science.gov• Biznar• Mednar• ScienceResearch.com


Develop 3 POC’s (Top 10 DB, 5 Catalogs, Digital Repositories

Launch xSearch for Science & Engineering (28 sources)

Expand xSearch to include Social Sciences & Humanities. Also, expanded later in the year for GSB sources (170 sources)

In November 2011 the Charleston Advisor Review was published

Upgrade and Expand xSearch to 200 sources in December 2012

History of Partnership

2007

2010

2011

2011

2012


2008 PR


Federated Search allows users to submit a real-time search in parallel to multiple information sources and retrieve aggregated, ranked and de-duplicated results.

What Is Federated Search?


In Other Words…One Search, Many Sources

Internal Sources

Blogs & Wikis

SubscriptionSources

Public Web Sources

Reports

News & Social Media

Begin Search


xSearch Status

• Upgraded early fall to v. 3.2.2• GSB linked to xSearch• 200 collections in application• 30 new connectors in acceptance

testing (roll-out imminent)


Janu

ary

Mar

chMay Ju

ly

Sept

embe

r

Novem

ber

0

1000

2000

3000

4000

5000

6000

7000

User Queries by Month/Year

2012 User Queries2011 User Queries2010 User Queries


Janu

ary

Febr

uary

Mar

chAp

ril

May

June

July

Augu

st

Sept

embe

r

Octob

er

Novem

ber

Decem

ber

0100,000200,000300,000400,000500,000600,000700,000

Source Queries

2012 Source Queries 2011 Source Queries2010 Source Queries


Web

of S

cien

ce

ABI/I

nfor

m G

loba

l

PubM

ed

Enviro

nmen

tal S

cien

ces & P

ollu

tion

Man

agem

ent

Engi

neer

ing

Villa

ge

Sociol

ogical

Abs

tract

s

Busine

ss S

ourc

e Com

plet

e

Scop

us

JSTO

R

ACS

Publ

icat

ions

Perio

dica

ls A

rchi

ve O

nlin

e

Proj

ect M

use

0

2000

4000

6000

Top click-throughs: Jan 1, 2012 – November 30, 2012


Explorit Release 3.2.3Starting customer upgrades

• Visual clusters• Full-text filters• Content type/Media

type• Integration with

Zotero, Mendeley


Explorit Release 4.0Coming mid-2013

• Dynamic tab searching• Thesaurus-based searching

(Do you also want to search for?)

• Personal library• Big Data mashups

(Enhanced content)• Faceted navigation


Big Data Mashups


Related Content

• Major Science portals • Science News • Patent Databases • Scholar Networks• Subscription Sources• Public Databases• Open Access Journals


Article Grouping


Non-Invasive imaging


Linking Open Data Cloud Diagram by Richard Cyganiak and Anja Jentzsch. http://lod-cloud.net/


Nature has 297 million triples.


Journey to 10,000 sources


Scalability Challenges

• Source selection • Ranking and organizing of results • Traffic management • System load management • Finding, building, and maintaining

connectors


Scalability - Divide and Conquer

ScienceResearch.com

WorldWideScience.org

Other Federated Search Engines

ScienceAccelerator

Science.gov


Multilingual WorldWideScience.org


How Multilingual Federated Search Works

Ranked resultstranslated by Microsoft to user’s language

Results returned to user

EXPLORIT

Microsoft Translator

German

Chinese

Russian

Queryin user’s language

Ranked resultsin user’s language

Queryto be translatedfor each source

Queryin source’slanguage

Foreign language

search engines

Resultsin source’slanguage

Ranking


Translated

Original


ESN – x2


UNECA ASKIA Portal (United Nations – Access Scientific Knowledge in Africa)


BASF


Find It!TMC’s Link Resolver


Abe’s Stanford ProjectsWISHLIST

• Assist SULAIR in integrating Explorit preview via Web Services into new library portal.

• Integration with SearchWorks

• Develop a stand-alone portal focused on Stanford core-competency (Energy, Environment, …)


Abe’s Stanford ProjectsWISHLIST (cont.)

• Develop Chinese Explorit in collaboration with Library of Congress and/other library

• xSearch / Explorit for Stanford Medical School

• Mash up Big Data (Link Data, Mendeley, citations) and articles

• Data Portal (expansion of Data searching in WorldWideScience.org)

• Integration with Sakai or other CMS


Explore our Applications

• xSearch• WorldWideScience.org• Science.gov• Ciencia.Science.gov• DTIC Multisearch

http://xsearch.stanford.edu/search/

http://worldwidescience.org/

http://www.science.gov/scigov/

http://ciencia.science.gov/

http://multisearch.deepwebaccess.com/multisearch/

http://multisearch.deepwebaccess.com/multisearch/


Thank you!

Abe [email protected]

View this presentation online

mailto:[email protected]

http://www.deepwebtech.com/talks/Stanford-DWT.pptx

http://www.deepwebtech.com/talks/Stanford-DWT.pptx

03 december 2012 by abe lederman, ceo

Documents

ceo deep web technologiesshow

realtime search

wordsone search

computer science

explorit release

federated search100

information retrieval

new mexicoover