effective ranking fusion methods for personalized metasearch engines leonidas akritidis dimitrios...

Effective Ranking Fusion Methods Effective Ranking Fusion Methods for Personalized Metasearch for Personalized Metasearch

EnginesEngines

Leonidas AkritidisLeonidas Akritidis

Dimitrios KatsarosDimitrios Katsaros

Panayiotis BozanisPanayiotis Bozanis

12th Pan-Hellenic Conference on Informatics12th Pan-Hellenic Conference on Informatics, Samos Island, Hellas, 28-, Samos Island, Hellas, 28-30/08/200830/08/2008

Department of Computer And Communication EngineeringDepartment of Computer And Communication EngineeringUniversity of Thessaly, Volos, HellasUniversity of Thessaly, Volos, Hellas

IntroductionIntroduction

Single Search EnginesSingle Search Engines

Maintenance of a document database.Maintenance of a document database. Maintenance of an Index that allows fast searching.Maintenance of an Index that allows fast searching.

Metasearch EnginesMetasearch Engines

Invocation of multiple search engines.Invocation of multiple search engines. No document database.No document database. No Index.No Index. Collection of results from component engines.Collection of results from component engines. Use of a Rank Aggregation Algorithm to merge the Use of a Rank Aggregation Algorithm to merge the

results.results.

http://quadsearch.csd.auth.gr

Rank AggregationRank Aggregation

What is Rank Aggregation?What is Rank Aggregation?

The collected data is merged to a final unordered list.The collected data is merged to a final unordered list.

A Rank Aggregation procedure proposes a way to sort this A Rank Aggregation procedure proposes a way to sort this list.list.

Why do we need Rank Aggregation?Why do we need Rank Aggregation?

To provide robust search on the Web.To provide robust search on the Web.

Meta-searching.Meta-searching.

Spam problem.Spam problem.


Existing Rank Aggregation MethodsExisting Rank Aggregation Methods

The most common rank aggregation methods are:The most common rank aggregation methods are:

Spearman’s FootruleSpearman’s Footrule

Kental’s TauKental’s Tau

Borda CountBorda Count

Markov ChainsMarkov Chains

KE Algorithm and its AntiSpam versionKE Algorithm and its AntiSpam version


KE AlgorithmKE Algorithm

The algorithm considers each result as a candidate.The algorithm considers each result as a candidate. Each candidate receives a score (weight), according to the Each candidate receives a score (weight), according to the

formula below:formula below:

m

i 1n

m

r iw

kn 1

10

r(i): The candidate’s rank in the i-th engine.r(i): The candidate’s rank in the i-th engine. n: The number of the candidate’s appearances.n: The number of the candidate’s appearances. m: The number of the invoked search engines.m: The number of the invoked search engines. k: The length of the top-k list.k: The length of the top-k list.


MotivationsMotivations

No public information on the ranking algorithms used by No public information on the ranking algorithms used by metasearch engines.metasearch engines.

Some sporadic works concern the way that search engines Some sporadic works concern the way that search engines should construct their index, by exploiting the geographical should construct their index, by exploiting the geographical location information of Web pages. location information of Web pages.

None of these techniques can be used by metasearch None of these techniques can be used by metasearch enginesengines

The existing Algorithms:The existing Algorithms:

Cannot accept data varying among different users Cannot accept data varying among different users (subjective data) and produce different results respectively.(subjective data) and produce different results respectively.

Output the same results for the same queries, submitted by Output the same results for the same queries, submitted by different users.different users.


Proposed AlgorithmsProposed Algorithms

We introduce four new algorithms:We introduce four new algorithms:

GeoKE AlgorithmGeoKE Algorithm Weighted KE AlgorithmWeighted KE Algorithm URL Aware KE AlgorithmURL Aware KE Algorithm Global KE AlgorithmGlobal KE Algorithm

All proposed algorithmsAll proposed algorithms

Manage to confront the mentioned problems.Manage to confront the mentioned problems. Are score based (that is, assign scores to the collected Are score based (that is, assign scores to the collected

results).results). Can be fine tuned by the user, enabling their usage in Can be fine tuned by the user, enabling their usage in

personalized search systems.personalized search systems. Have been implemented in a fully functional metasearch Have been implemented in a fully functional metasearch

engine, engine, QuadSearchQuadSearch, available at , available at http://quadsearch.csd.auth.gr ..


Geography Aware Ranking (1)Geography Aware Ranking (1)

Users often seek for information that is directly linked to Users often seek for information that is directly linked to a specific geographical region.a specific geographical region.

For instance, travel related queries are usually For instance, travel related queries are usually connected to the travel’s destination.connected to the travel’s destination.

There is strong evidence that a considerable number of There is strong evidence that a considerable number of search engine queries are geographically oriented.search engine queries are geographically oriented.

The two or three final characters of the domain name The two or three final characters of the domain name usually declare the originating country of that page.usually declare the originating country of that page.


Geography Aware Ranking (2)Geography Aware Ranking (2)

Example Query: Example Query: “Hotels in Paris”.“Hotels in Paris”.

Significant possibility that pages hosted under Significant possibility that pages hosted under .fr .fr domain domain extension, contain more valuable information than these extension, contain more valuable information than these hosted under other extensions.hosted under other extensions.

Assumption: a hypothetical user from United Kingdom Assumption: a hypothetical user from United Kingdom searches for searches for “Hotels in Paris“Hotels in Paris”.”.

The pages under The pages under .fr .fr domains are usually written in domains are usually written in French.French.

The hypothetical British user must be familiar to French The hypothetical British user must be familiar to French to understand the provided information.to understand the provided information.

A page with A page with .uk.uk domain extension would probably be best domain extension would probably be best result. Pages with result. Pages with .us,.us, .au.au or or .ca.ca domain extensions would domain extensions would also be good choices.also be good choices.


GeoKE AlgorithmGeoKE Algorithm Each candidate receives a score (weight), according to the formula:Each candidate receives a score (weight), according to the formula:

n

m

m

1iGAke

110k

n

irGW

Where Where GG is the is the GeoKE CoefficientGeoKE Coefficient. Its value depends on a . Its value depends on a variety of cases and can be any integer.variety of cases and can be any integer.

A result with lower score will achieve better ranking than A result with lower score will achieve better ranking than a result with greater weight.a result with greater weight.


GeoKE Algorithm Implementation (1)GeoKE Algorithm Implementation (1)

The user’s locality is automatically obtained by using a The user’s locality is automatically obtained by using a database that is specially designed for matching IP database that is specially designed for matching IP addresses against geographical locations.addresses against geographical locations.

Another local database table is used to store the Another local database table is used to store the relationships between the geographical regions and their relationships between the geographical regions and their respective languages. These relationships are stored as respective languages. These relationships are stored as {region - friendly region} pairs.{region - friendly region} pairs.

The user’s locality is compared to the locality of the page The user’s locality is compared to the locality of the page (revealed by its domain extension). The result of this (revealed by its domain extension). The result of this comparison leads to one of the following four cases.comparison leads to one of the following four cases.


GeoKE Algorithm Implementation (2)GeoKE Algorithm Implementation (2)

The Algorithm considers the following casesThe Algorithm considers the following cases

1.1. The domain extension of the result and the user’s region are The domain extension of the result and the user’s region are the same.the same.

2.2. The user can understand the language that the page is The user can understand the language that the page is written (Example: Brazilian user and Portuguese pages).written (Example: Brazilian user and Portuguese pages).

3.3. The domain extension of the page does not reveal any The domain extension of the page does not reveal any information about its locality (information about its locality (.com, .net, .edu, .org.com, .net, .edu, .org etc). etc).

4.4. The page is written in a language that the user can’t The page is written in a language that the user can’t understand.understand.


Weighted KE Algorithm (1)Weighted KE Algorithm (1)

All methods examined so far, treat all component engines All methods examined so far, treat all component engines equally.equally.

It is a common intuition, that a single search engine can’t It is a common intuition, that a single search engine can’t perform equally well for all types of queries.perform equally well for all types of queries.

Each engine maintains its own document index.Each engine maintains its own document index.

Every time a query is submitted, it searches different documents Every time a query is submitted, it searches different documents and presents different results than another engine does.and presents different results than another engine does.

For such occasions, an effective scoring algorithm must provide For such occasions, an effective scoring algorithm must provide the user with the ability to modify the importance of each the user with the ability to modify the importance of each component engine.component engine.


Weighted KE Algorithm (2)Weighted KE Algorithm (2)

The The Weighted KE AlgorithmWeighted KE Algorithm introduces the weighting of the introduces the weighting of the component search engines. Each candidate receives a component search engines. Each candidate receives a score (weight), according to the formula:score (weight), according to the formula:


n

m

m

1iWke

110k

n

irieW

Where Where e(i)e(i) is the is the Weight Factor of the ith Engine (EWF).Weight Factor of the ith Engine (EWF). EWFEWF can receive integer values. can receive integer values. The equation above leads to the same ranking as the The equation above leads to the same ranking as the

original version of the algorithm, unless a different original version of the algorithm, unless a different EWFEWF is is assigned to at least one engine.assigned to at least one engine.

Weighted KE Algorithm Weighted KE Algorithm ImplementationImplementation

In our implementation in In our implementation in QuadSearchQuadSearch, the user is free to , the user is free to assign integer weights to the four component search assign integer weights to the four component search engines (engines (Google, Yahoo, Live Search Google, Yahoo, Live Search andand Ask.com Ask.com).).

The user selects weights for each engine. The weights can The user selects weights for each engine. The weights can be any integer between 1 and 10.be any integer between 1 and 10.

To increase the importance of the results coming from an To increase the importance of the results coming from an engine with higher weight, we must decrease their score.engine with higher weight, we must decrease their score.

Thus, the user selected weights are being silently Thus, the user selected weights are being silently subtracted by eleven (11), then applied to the subtracted by eleven (11), then applied to the Weighted Weighted KE AlgorithmKE Algorithm..


Domain Name Analysis (1)Domain Name Analysis (1)

The phenomenon of Web pages appearing under different The phenomenon of Web pages appearing under different subdomains of the same domain name is very usual these subdomains of the same domain name is very usual these days.days.

In fact, subdomains are simple folders in a Web server’s In fact, subdomains are simple folders in a Web server’s public directory and can contain multiple pages with public directory and can contain multiple pages with similar or unique informational material.similar or unique informational material.

Examples of such pages are the Departments of a Examples of such pages are the Departments of a University, or the personal pages of the academic staff of a University, or the personal pages of the academic staff of a faculty.faculty.


Domain Name Analysis (2)Domain Name Analysis (2)

There is a danger that many pages with similar contents or There is a danger that many pages with similar contents or many pages from one source would appear in the engine’s many pages from one source would appear in the engine’s result list.result list.

This danger is even greater in metasearch engines, where This danger is even greater in metasearch engines, where more than one component engines are being exploited.more than one component engines are being exploited.

The top-10 list that The top-10 list that GoogleGoogle returns for the query returns for the query “Aristotle “Aristotle University of Thessaloniki”University of Thessaloniki” contains 10 results, all having the contains 10 results, all having the term term “auth.gr”“auth.gr” in their domain name. in their domain name.

Obviously, the limitation of no more than two pages with the Obviously, the limitation of no more than two pages with the same domain name in the same result list does not cover same domain name in the same result list does not cover subdomains.subdomains.


URL Aware KE Algorithm (1)URL Aware KE Algorithm (1)

Solution: Give these pages a lower ranking. Solution: Give these pages a lower ranking. The The URL Aware KE AlgorithmURL Aware KE Algorithm assigns scores to the assigns scores to the

candidates according to the formula: candidates according to the formula:


nm

m

1iUAke

110k

n

irD11W

DD is the is the Domain Awareness Constant (DAC)Domain Awareness Constant (DAC).. A higher value of A higher value of D D will decrease the result’s weight, will decrease the result’s weight,

therefore improve its ranking.therefore improve its ranking. The following four distinct cases determine the value that The following four distinct cases determine the value that

the the Domain Awareness ConstantDomain Awareness Constant receives. receives.

URL Aware KE Algorithm (2)URL Aware KE Algorithm (2)

1.1. The result has a domain name that is not repeated more The result has a domain name that is not repeated more than two times in the result list.than two times in the result list.

2.2. The result has a domain name that is repeated more than The result has a domain name that is repeated more than two times in the result list, but this result is the best among two times in the result list, but this result is the best among the others having the same domain name (it has received the others having the same domain name (it has received the best rankings from component engines).the best rankings from component engines).

3.3. The result has a domain name that is repeated more than The result has a domain name that is repeated more than two times in the result list and it is not the best among the two times in the result list and it is not the best among the results with the same domain name. Its domain is results with the same domain name. Its domain is centralcentral..

4.4. The result has a domain name that is repeated more than The result has a domain name that is repeated more than two times in the result list and it is not the best among the two times in the result list and it is not the best among the results with the same domain name.results with the same domain name. The result appears The result appears under a subdomain of a under a subdomain of a central domaincentral domain..


Global KE Algorithm (1)Global KE Algorithm (1)

The The Global KE Algorithm Global KE Algorithm derives from the combination of derives from the combination of the three scoring methods described above.the three scoring methods described above.

Each result is assigned a score that is determined by the Each result is assigned a score that is determined by the following formula:following formula:


nm

m

1igke

110k

n

irie11GD11W

Global KE Algorithm (2)Global KE Algorithm (2)

The The Global KE Algorithm Global KE Algorithm gives the user the ability to define:gives the user the ability to define:

The engine’s importance for a single query. Not all search The engine’s importance for a single query. Not all search engines are treated equally for all types of queries.engines are treated equally for all types of queries.

How the geographic origin of a page and the language it is How the geographic origin of a page and the language it is written, affect its ranking.written, affect its ranking.

How the domain name structure of a Web page affects its How the domain name structure of a Web page affects its ranking.ranking.


Time ComplexityTime Complexity

Result Classification’s performance (seconds) for the Result Classification’s performance (seconds) for the proposed algorithms, for various numbers of input proposed algorithms, for various numbers of input resultsresults for the query for the query “Athenian Acropolis”.“Athenian Acropolis”.


40 40 resultresult

ss

80 80 resultresult

ss

120 120 resultsresults

Original KEOriginal KE 0.030.03 0.060.06 0.110.11

Weighted KEWeighted KE 0.030.03 0.060.06 0.110.11

GeoKEGeoKE 0.030.03 0.060.06 0.130.13

URL Aware URL Aware KEKE 0.030.03 0.060.06 0.110.11

Global KEGlobal KE 0.030.03 0.060.06 0.140.14

Time ComplexityTime Complexity

None of the introduced algorithms harms the performance None of the introduced algorithms harms the performance of the results classification process significantly. No of the results classification process significantly. No significant overhead comes out of their use.significant overhead comes out of their use.

One might expect that the use of One might expect that the use of GeoKE AlgorithmGeoKE Algorithm would would result a slower query execution, as a connection to a result a slower query execution, as a connection to a database and numerous data transfers are required.database and numerous data transfers are required.

In this occasion, keeping the appropriate data in the In this occasion, keeping the appropriate data in the server’s main memory eliminates this drawback.server’s main memory eliminates this drawback.


Example Query – Test of UsefulnessExample Query – Test of Usefulness

The The GeoKE AlgorithmGeoKE Algorithm proved very useful for travel related proved very useful for travel related queries and generally for cases where the quality of the queries and generally for cases where the quality of the presented results is affected by geographical information.presented results is affected by geographical information.

We compared the top-10 list returned by the We compared the top-10 list returned by the GeoKE Algorithm,GeoKE Algorithm, against the list returned by the original against the list returned by the original KE Algorithm,KE Algorithm, for the for the query “query “Athenian AcropolisAthenian Acropolis”.”.

The query is submitted by a hypothetical user located in The query is submitted by a hypothetical user located in Greece.Greece.

The exploited result resources were the four major commercial The exploited result resources were the four major commercial search engines, search engines, Google, Yahoo, Live SearchGoogle, Yahoo, Live Search and and Ask.comAsk.com. All . All engines were considered to be of equal weight.engines were considered to be of equal weight.


Example Query ResultsExample Query Results


Original KE AlgorithmOriginal KE Algorithm GeoKE AlgorithmGeoKE Algorithm

11 witcombe.sbc.eduwitcombe.sbc.edu witcombe.sbc.eduwitcombe.sbc.edu

22 www.bluffton.eduwww.bluffton.edu www.bluffton.eduwww.bluffton.edu

33 www.metrum.orgwww.metrum.org www.acropolisofathens.www.acropolisofathens.grgr

44 people.hsc.edupeople.hsc.edu www.metrum.orgwww.metrum.org

55 plato-dialogues.orgplato-dialogues.org people.hsc.edupeople.hsc.edu

66 www.wikitravel.orgwww.wikitravel.org plato-dialogues.orgplato-dialogues.org

77 en.wikipedia.orgen.wikipedia.org www.wikitravel.orgwww.wikitravel.org

88 www.acropolisofathens.www.acropolisofathens.grgr

en.wikipedia.orgen.wikipedia.org

99 www.amazon.comwww.amazon.com www.amazon.comwww.amazon.com

1010 www.reconstructions.orwww.reconstructions.orgg

www.culture.grwww.culture.gr

CommentsComments

The The GeoKE Algorithm GeoKE Algorithm assigns scores to the collected assigns scores to the collected results with respect to the user’s and the result’s locality.results with respect to the user’s and the result’s locality.

The algorithm’s scoring formula has given a higher rank to The algorithm’s scoring formula has given a higher rank to the Greek oriented result the Greek oriented result www.acropolisofathens.gr (rank: (rank: 3), than the original 3), than the original KE AlgorithmKE Algorithm did (rank: 8). did (rank: 8).

Moreover, the top-10 list formed by the Moreover, the top-10 list formed by the GeoKE Algorithm GeoKE Algorithm contains one more result with Greek origin, placed in the contains one more result with Greek origin, placed in the tenth position (tenth position (www.culutre.gr).).


ConclusionsConclusions

We presented three innovative ranking fusion methods and We presented three innovative ranking fusion methods and their generalization, the their generalization, the Global KE AlgorithmGlobal KE Algorithm that derives that derives from their combination. from their combination.

All three methods have been implemented and can be All three methods have been implemented and can be tested in tested in QuadSearchQuadSearch, our experimental metasearch engine , our experimental metasearch engine that can be found in that can be found in http://quadsearch.csd.auth.gr..

The open architecture of the proposed algorithms, allows The open architecture of the proposed algorithms, allows user defined modifications and further fine-tuning is user defined modifications and further fine-tuning is possible.possible.

We have tested and proved the usefulness of the algorithms We have tested and proved the usefulness of the algorithms for various types of queries, especially the for various types of queries, especially the GeoKE AlgorithmGeoKE Algorithm..

The computational cost a ranking system suffers from their The computational cost a ranking system suffers from their use is negligible.use is negligible.


effective ranking fusion methods for personalized metasearch engines leonidas akritidis dimitrios...

Documents

rank aggregation algorithm

rank aggregation procedure

rank aggregationwhat

invoked search engines

robust search

different results

ranking algorithms

collected results