FederatedFederated Search Engines Search Engines

Paper presented at seminar onPaper presented at seminar on

SEARCH ENGINES AND DATABASES SEARCH ENGINES AND DATABASES

ByByMrs. Shakuntala NighotMrs. Shakuntala Nighot

2009-20102009-2010

SHPT School of Library ScienceS.N.D.T. Women’s University

Mumbai 400 020

Under Guidance of

Dr. Sarika Sawant

OutlineOutlineDefinition- Federated search Definition- Federated search

Need for Search EnginesNeed for Search Engines

What is Deep Web?What is Deep Web?

Google search vs. Federated SearchGoogle search vs. Federated Search

Need For Federated Search EnginesNeed For Federated Search Engines

Features, Limitations, Examples of Federated Search Engines Features, Limitations, Examples of Federated Search Engines

Criteria for Selecting Best Federated Search EngineCriteria for Selecting Best Federated Search Engine

MetaLib Vs. WebFeat- A comparative Study MetaLib Vs. WebFeat- A comparative Study

ConclusionConclusion2204/10/2304/10/23 SHPT School Of Library ScienceSHPT School Of Library Science

Definition - Federated SearchDefinition - Federated Search

““Federated search is the process of performing a Federated search is the process of performing a simultaneous real-time search of multiple diverse simultaneous real-time search of multiple diverse and distributed sources from a single search page, and distributed sources from a single search page, with the federated search engine acting as with the federated search engine acting as intermediary.” intermediary.” (Lederman, n,d,)(Lederman, n,d,)

3304/10/2304/10/23 SHPT School Of Library ScienceSHPT School Of Library Science

Search Engines-NeedSearch Engines-Need

A web search engine is a tool designed to search for A web search engine is a tool designed to search for information on wwwinformation on wwwE.g.. Yahoo, GoogleE.g.. Yahoo, Google

Need- One needs to refer the catalogue to find to Need- One needs to refer the catalogue to find to particular book from the vast collection of the library. particular book from the vast collection of the library. catalogue acts as an important intermediary between catalogue acts as an important intermediary between library sources and user.library sources and user.

Following the same lines, The Search engine helps the Following the same lines, The Search engine helps the user to sift through ocean of knowledge on World Wide user to sift through ocean of knowledge on World Wide Web and to find the specific information needed.Web and to find the specific information needed.

4404/10/2304/10/23 SHPT School Of Library ScienceSHPT School Of Library Science

What is Deep Web? What is Deep Web?

5504/10/2304/10/23 SHPT School Of Library ScienceSHPT School Of Library Science

Deep Web is part of World Wide Web other than surface Deep Web is part of World Wide Web other than surface web which cannot be indexed by common search web which cannot be indexed by common search engines.engines.

It is 500 times surface web, Consists of Scholarly and It is 500 times surface web, Consists of Scholarly and research material research material

Providers of such content Providers of such content ::

Database Vendors, Database Vendors,

Commercial Publishers of full-text material, Commercial Publishers of full-text material,

Libraries Libraries

RepositoriesRepositories

Need for Federated Search EnginesNeed for Federated Search Engines

Libraries/Institutions Procure these databases Libraries/Institutions Procure these databases

Query Language, User interface for each of them is Query Language, User interface for each of them is

differentdifferent

Patrons don’t prefer searching them one by one.Patrons don’t prefer searching them one by one.

Federated Search Engines offers single interface to Federated Search Engines offers single interface to search across all resourcessearch across all resources

They give entry to the deep web while Common Search They give entry to the deep web while Common Search engines Can’tengines Can’t

6604/10/2304/10/23 SHPT School Of Library ScienceSHPT School Of Library Science

Google vs. Fed. Search EngineGoogle vs. Fed. Search EngineGoogle periodically visits the sites on its list and Google periodically visits the sites on its list and identifies the new links at those sites. Following those identifies the new links at those sites. Following those links it arrives at new pages where it find more links. In links it arrives at new pages where it find more links. In doing this, Google discovers sites it didn’t knew till doing this, Google discovers sites it didn’t knew till previous visits. And add it its databases.previous visits. And add it its databases.

Process of going from one page to another and then to Process of going from one page to another and then to another is referred to as “crawling,”another is referred to as “crawling,”

Deep Web content don’t have such links. Google Can’t Deep Web content don’t have such links. Google Can’t retrieve it.retrieve it.

Federated Search Engines are programmed to fill up the Federated Search Engines are programmed to fill up the search form for user queries, submit them to various search form for user queries, submit them to various deep web resources and to read the results from them.deep web resources and to read the results from them.

Google is not designed to fill up search formsGoogle is not designed to fill up search forms7704/10/2304/10/23 SHPT School Of Library ScienceSHPT School Of Library Science

Federated Search Engines-FeaturesFederated Search Engines-FeaturesSaves Time/One Stop ShoppingSaves Time/One Stop Shopping

Quality Results –Authentic sourcesQuality Results –Authentic sources

Most current ContentMost current Content

Aggregation (Helpful Arrangement)Aggregation (Helpful Arrangement)

Relevance RankingRelevance Ranking

De-duplicationDe-duplication

Simple Search, Advance Search/ LimitersSimple Search, Advance Search/ Limiters

Clustering/ Subject GroupingClustering/ Subject Grouping88

04/10/2304/10/23SHPT School Of Library ScienceSHPT School Of Library Science

Federated Search Engines - LimitationsFederated Search Engines - Limitations

Doesn’t offer native searchDoesn’t offer native search

Hard to go deeper in collectionHard to go deeper in collection

Slow in Response TimeSlow in Response Time

Complete de-duping is difficultComplete de-duping is difficult

Configuring For new database- time consumingConfiguring For new database- time consuming

Changed database configuration- unsearchableChanged database configuration- unsearchable

9904/10/2304/10/23 SHPT School Of Library ScienceSHPT School Of Library Science

Some Federated Search ApplicationsSome Federated Search Applications

360 Search360 Search

DeepWebDeepWeb

LiraryFindLiraryFind

MetaLibMetaLib

Scitopeia.orgScitopeia.org

WebFeatWebFeat

WorldWideScienceWorldWideScience

101004/10/2304/10/23 SHPT School Of Library ScienceSHPT School Of Library Science

Criteria -Selecting Best Federated Criteria -Selecting Best Federated Search EngineSearch Engine

HostingHosting Vendor Hosted ModelVendor Hosted Model Locally Hosted ModelLocally Hosted Model

PricingPricing100% database compatibility100% database compatibilityScreen Scrapping Vs. Native InterfaceScreen Scrapping Vs. Native InterfaceAutomatic 24*7 Monitoring and updatesAutomatic 24*7 Monitoring and updatesCustom User InterfaceCustom User InterfaceQuality of ConnectorsQuality of ConnectorsRelevance RankingRelevance RankingTrialsTrials

111104/10/2304/10/23 SHPT School Of Library ScienceSHPT School Of Library Science

MetaLib Vs WebFeat MetaLib Vs WebFeat

WebFeatWebFeat DeepWebDeepWeb

121204/10/2304/10/23 SHPT School Of Library ScienceSHPT School Of Library Science

Criteria MetaLib Webfeat

1 Shows Search Results in

2 Consistency in results returned

3 No of databases searched at a time

4 Option for “Search All Databases”

5 One Stop Shopping

MetaLib Interface

Yes; All databases have same search default operators

Ten

No

No

Native Interface

No; All databases searched has different default operators

No Limit

Yes

Yes

MeaLib Vs. WebFeatMeaLib Vs. WebFeat

WebFeatWebFeat DeepWebDeepWeb

131304/10/2304/10/23 SHPT School Of Library ScienceSHPT School Of Library Science

Criteria MetaLib Webfeat

6 Speed of retrieval

7 Allows Customized Grouping of Databases

8 Type of Hosting offered

9 At a time Searching Capacity of Simple Search

10 Interface Complexity

11Sorting of Records offered

More

No

Vendor, Local both

1 Category of subject

Less

By many ways (Year, relevance etc.)

Comparatively Less

Yes

Vendor

All Categories

More

No sorting

ConclusionConclusion

Federated search engines are powerful tools Federated search engines are powerful tools which can create a single gateway linking to the which can create a single gateway linking to the scattered information resources, lying even in scattered information resources, lying even in the deep web.the deep web.

It helps users to find high-quality, most It helps users to find high-quality, most current ,more specialized information from current ,more specialized information from remote corners of the Internet. Hence it’s a vital remote corners of the Internet. Hence it’s a vital technology in today's information age technology in today's information age

141404/10/2304/10/23 SHPT School Of Library ScienceSHPT School Of Library Science