leveraging publisher’s search engines to deliver relevant results to users
DESCRIPTION
Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users. Presented by Abe Lederman, President and CTO Deep Web Technologies, LLC. 28 th Annual Scholarly Publishing Meeting – Virginia – June 9, 2006. Abe’s Background. Earned B.S. and M.S. Computer Science degrees, MIT - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/1.jpg)
Leveraging Publisher’s Search Engines to Deliver Relevant
Results to Users
Presented by Abe Lederman, President and CTO
Deep Web Technologies, LLC
28th Annual Scholarly Publishing Meeting – Virginia – June 9, 2006
![Page 2: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/2.jpg)
Abe’s Background• Earned B.S. and M.S. Computer Science degrees, MIT• 18 years experience developing sophisticated
information retrieval applications• Cofounded Verity, 1988• Consulted to LANL, 1994-2000• Deployed first “federated search” portal in the Federal
government, 1999• Founded Deep Web Technologies (DWT), 2002
DWT is a New Mexico based company focused on providing state-of-the-art software solutions which search, retrieve,
aggregate, and analyze content from web-based databases.
![Page 3: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/3.jpg)
The Problem:
Searching a large number of
sources can lead to a flood of
results
![Page 4: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/4.jpg)
Relevance ranking
begins as soon as the user clicks the Search
button
![Page 5: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/5.jpg)
Ranking Recipe
Source Selection
Query Language
Search Conductor
Ranking Algorithms
INGREDIENTS
MIX WELL AND SERVE UP RELEVANT RESULTS
![Page 6: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/6.jpg)
Source Selection Optimizer
Search Conductor
Source Selection Optimizer
Source
Descriptions Previous Results
![Page 7: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/7.jpg)
Powerful Query Language
• Takes advantage of search capabilities of each source
• Supports full Boolean operators where possible
• Supports fielded search
• Translates natural language questions into query syntax
![Page 8: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/8.jpg)
Select sources to search
Can I get more results from “good”
sources?
Enough good
results?
YES
Deliver results to user
YES
NO
NO
Perform Search
Get Next Results
Search Conductor
![Page 9: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/9.jpg)
Challenges in Organizing and Ranking Results
Multi-tier Relevance Ranking
User-driven Ranking
Clustering of Results
![Page 10: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/10.jpg)
Multi-tier Relevance Ranking
• QuickRank – Ranks results based on occurrence of search terms in title, author, and snippet
• MetaRank – Ranks results utilizing custom algorithms applied to meta-data
• DeepRank – Downloads and indexes full-text documents
HEAVY LIFTING REQUIRED!
![Page 11: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/11.jpg)
User-driven Ranking
Credibility of sourceDate rangeDocument lengthDocument type
Geographic proximityPopularity of documentReading levelRelevance
Desired: Blending (weighing) of above criteria
![Page 12: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/12.jpg)
Clustering
![Page 13: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/13.jpg)
Attributes of Successful Federated Search
• Powerful query language that takes advantage of publisher search capabilities
• Source selection optimizer will reduce unnecessary searches
• Search conductor gets more results from sources bringing back good results
• A tool that highlights best search results
• Caching of search results
![Page 14: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/14.jpg)
Advice for Publishers
• Use good search engines with good relevance ranking
• Return 100 or more results at a time
• Return meta-data (author, journal, snippet) as part of result list
• Provide access to your content through XML Gateway or Web Services
• Speed up search time
![Page 15: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/15.jpg)
![Page 16: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/16.jpg)
![Page 17: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/17.jpg)
![Page 18: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/18.jpg)
![Page 19: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/19.jpg)
![Page 20: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/20.jpg)
![Page 21: Leveraging Publisher’s Search Engines to Deliver Relevant Results to Users](https://reader036.vdocument.in/reader036/viewer/2022070417/568153b6550346895dc1b81e/html5/thumbnails/21.jpg)
Abe Lederman
301 N Guadalupe, Ste 201
Santa Fe, NM 87501
www.deepwebtech.com
Thank You!