social choice and rank aggregation in metasearch/ xiaotie deng lecture 9: rank aggregation in...

Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng

Lecture 9: Rank Aggregation in MetaSearch

MetaSearch Engine Social Choice Rules Rank Aggregation


Choices of Search Engines

Many search engines exist to compete for users The results are not necessarily the same Different users prefer different search

engines Search results may, in the future, be

biased towards paid advertisements.


GOOGLE: City University1. City University London - the University for business and the ...

... The University for business and the professions. Contact Us | About City University| Maps & Directions, AZ Index | Site Map | Help. Prospective Students. ... Description: Official site with information about courses, research, schools, and departments. Includes details...

2. City University HomeA University which believes in forward-thinking business and leadership skills, exposing students to the latest technology in all courses of study. Description: A private, nonprofit institution founded to serve working adults wanting to pursue educational opportunit...

3. Welcome to Dublin City University... here... Dublin City University, Dublin 9, Ireland. Tel. +353 (0) 1700 5000, Fax. +353 (0) 1 836 0830. Page updated: 02/09/03 legal. ... Description: Information on facilities, services, degree courses, research, the campus, student life, the library...

4. The City University of New YorkDescription: The University's Main Website.

5. City University of Hong KongAD Working Group lends staff, students an ear. Several issues stillevoked strong emotions as staff members and students voiced their ... Description: Formerly the City Polytechnic of Hong Kong. Includes information on university, links to learning...

6. Welcome to Oklahoma City University... At Oklahoma City University, our students come first. Our ... Oklahoma CityUniversity offers a quality, values-centered education. United ... Description: Admissions, academic programs and alumni relations, sports, services, news, calendar of events and...


YAHOO1. City University London - the University for business and the ...

... The University for business and the professions. Contact Us | About City University | Maps & Directions, AZ Index | Site Map | Help. Prospective Students. ...

2. City University A University which believes in forward-thinking business and leadership skills, exposing students to the latest technology in all courses of study.www.cityu.edu/ - 3k - Cached - More pages from this site

3. Dublin City University ... here... Dublin City University, Dublin 9, Ireland. Tel. +353 (0) 1 700 5000, Fax. +353 (0) 1 836 0830. Page updated: 02/09/03 legal. ...www.dcu.ie/ - 6k - Cached - More pages from this site

4. City University of New York The University's Main Website.www.cuny.edu/ - More pages from this site

5. City University of Hong Kong AD Working Group lends staff, students an ear. Several issues still evoked strong emotions as staff members and students voiced their ...www.cityu.edu.hk/ - 26k - Cached - More pages from this site

6. Oklahoma City University ... At Oklahoma City University, our students come first. Our ... Oklahoma City University offers a quality, values-centered education. United ...www.okcu.edu/ - 20k - Cached - More pages from this site


AllTheWeb1. Apartment Listings in University City (sponsored)

St. Louis, Missouri area apartment listings at Apartments.com. Free nationwide apartment search with visual rental listings online.http://www.apartments.com

2. City University - Washington (sponsored)Contact information and resources such as yellow page information, phone number, address, maps and directions as provided by QwestDex.http://service.bfast.com

3. City University Apartments - Rent.com (sponsored)Rent.com has millions of free apartment listings nationwide. Get $100 when you sign a lease near your school - it's easy.http://www.rent.com

4. City University London - the University for business and the professions... Contact Us | About City University | Maps & Directions A-Z Index | Site Map | Help ... Description: Official site with information about courses, research, schools, and departments. Includes details of news and events.more hits from: http://www.city.ac.uk/ - 13 KB

5. Flash UpgradeDescription: A University which believes in forward-thinking business and leadership skills, exposing students to the latest technology in all courses of study.http://www.cityu.edu/ - 27 KB

6. Welcome to Dublin City University... KnowledgeWorks, wins DCU Mallin-invent award Full text you can go anywhere in the world from here... Dublin City University, Dublin 9, Ireland. Tel. +353 (0) 1 700 5000, Fax. +353 (0) 1 836 0830. Page updated: 02/09/03 legal search ... Description: Information on facilities, services, degree courses, research, the campus, student life, the library and DCU news.


MSN Search1. City University of New York Details the CUNY system's campuses and admissions

requirements. Eye photos of notable graduates such as actor Judd Hirsch.www.cuny.edu

2. City University Offers prospectus, admissions, academic calendar, courses and degrees, and news.www.cityuniversity.net

3. City University Private, nonprofit institution located in Belleview, Washington, offers higher education opportunities to working professionals and lifelong learners.www.cityu.edu

4. Valley City University Admissions Site covers financial aid and tuition and offers a virtual campus tour. Site also provides access to an online application.www.vcsu.nodak.edu/admissions

5. Oklahoma City University The Oklahoma City University site contains information about admissions, academic departments and student life. The site also provides library resources, faculty information and news.frodo.okcu.edu

6. Dublin City University, Dublin - TripAdvisor Browse reviews of the Dublin City University in Dublin, County Dublin, Ireland. Check consumer reviews, articles and opinions about the Dublin City University.www.tripadvisor.com/Hotel_Review-g186605-d209093-Reviews-Dublin_City_U...


AOL Search1. City University London - the University for business and the ... - ... The University for business

and the professions. Contact Us | About City University | Maps & Directions, AZ Index | Site Map | Help. Prospective Students. ... http://www.city.ac.uk/

2. City University Home - A University which believes in forward-thinking business and leadership skills, exposing students to the latest technology in all courses of study. http://www.cityu.edu/

3. Welcome to Dublin City University - ... here... Dublin City University, Dublin 9, Ireland. Tel. +353 (0) 1 700 5000, Fax. +353 (0) 1 836 0830. Page updated: 02/09/03 legal. ... http://www.dcu.ie/

4. http://www.cuny.edu/ - The University's Main Website.http://www.cuny.edu/

5. City University of Hong Kong - AD Working Group lends staff, students an ear. Several issues still evoked strong emotions as staff members and students voiced their ... http://www.cityu.edu.hk/

6. Welcome to Oklahoma City University - ... At Oklahoma City University, our students come first. Our ... Oklahoma City University offers a quality, values-centered education. United ... http://www.okcu.edu/


Ask Jeeves1. City University of Hong Kong Formerly the City Polytechnic of Hong Kong.

Includes information on university, links to learning resources, research and departmental information.....From:www.cityu.edu.hk/

2. City University Text Version. The University for business and the professions ... Contact Us | About City University | Maps & Directions A-Z Index | Site Map | Help...From:www.city.ac.uk/

3. New Jersey City University Official site includes graduate and undergraduate information, news, events, online learning, calendars, and athletics.From:www.njcu.edu/

4. Elizabeth City State University Admission Application Introduction 2004-2005 APPLICATION FOR ADMISSION. Welcome to the Elizabeth City State University Online Application. New Freshmen...From:www.ncmentor.org/applications/unc/apply/elizabeth_city_state_univ...

5. City University A University which believes in forward-thinking business and leadership skills, exposing students to the latest technology in all courses of study.From:www.cityu.edu/

6. The City College of New York City College of New York (CUNY) A senior college of The City University of New York. Located in Manhattan. 138th Street at Convent Avenue. A Harlem masterpiece in the neo-Gothic...From:www.ccny.cuny.edu/


HotBot1. City University of New York

Details the CUNY system's campuses and admissions requirements. Eye photos of notable graduates such as actor Judd Hirsch.www.cuny.edu/ - August 10, 2003 - 25 KB

2. City UniversityPrivate, nonprofit institution located in Belleview, Washington, offers higher education opportunities to working professionals and lifelong learners.www.cityu.edu/ - August 23, 2003 - 27 KB

3. New Jersey City UniversityExplore the course requirements for this liberal arts institution. Link to admissions and financial aid information. ... New Jersey City University. 2039 Kennedy Boulevard Jersey City, New Jersey 07305-1597 ... www.njcu.edu/ - November 1, 2003 - 20 KB

4. City University London - the University for business and the...... Contact Us | About City University | Maps & Directions. A-Z Index | Site Map | Help ... www.city.ac.uk/ - September 28, 2003 - 14 KB

5. Oklahoma City UniversityTake a tour of the campus, and explore a roster of academic programs. ... At Oklahoma City University, our students come first. ... www.okcu.edu/ - October 26, 2003 - 20 KB

6. University of Missouri, Kansas CityUniversity based in Kansas City, Missouri, presents and overview of its academic programs, and offers campus news. ... UNIVERSITY OF MISSOURI-KANSAS CITY. Bulletin. Winter 2004 Fee Update, more... ... www.umkc.edu/ - October 23, 2003 - 16 KB


Lycos1. City University London - the University for business and the...

… Contact Us | About City University | Maps & Directions A-Z Index | Site Map | Help … More results from: www.city.ac.uk September 16, 2003 - 14 KB

2. Flash UpgradeA University which believes in forward-thinking business and leadership skills, exposing students to the latest technology in all courses of study.www.cityu.edu November 1, 2003 - 27 KB

3. The City University of New YorkThe University's Main Website.www.cuny.edu December 31, 1969 - 121 B

4. Welcome to Dublin City University… KnowledgeWorks, wins DCU Mallin-invent award Full text you can go anywhere in the world from here... Dublin City University , Dublin 9, Ireland. Tel. +353 (0) 1 700 5000, Fax. +353 (0) 1 836...More results from: www.dcu.ie October 29, 2003 - 5 KB

5. City University of Hong KongFormerly the City Polytechnic of Hong Kong. Includes information on university, links to learning resources, research and departmental information and student information.More results from: www.cityu.edu.hk October 29, 2003 - 26 KB

6. HCU HomePage -English-… contact us For any suggestion and requests to this web site, [email protected] Feel free to link this web site. Hiroshima City University More results from: www.hiroshima-cu.ac.jp May 11, 2003 - 26 KB


Teoma1. City University of Hong Kong Formerly the City Polytechnic of Hong Kong. Includes

information on university, links to learning resources, research and departmental information...www.cityu.edu.hk/[Related Pages][More Results from www.cityu.edu.hk]

2. City University Text Version. The University for business and the professions ... Contact Us | About City University | Maps & Directions A-Z Index | Site Map | Help...www.city.ac.uk/[More Results from www.city.ac.uk]

3. New Jersey City University Official site includes graduate and undergraduate information, news, events, online learning, calendars, and athletics.www.njcu.edu/[Related Pages][More Results from www.njcu.edu]

4. Elizabeth City State University Admission Application Introduction 2004-2005 APPLICATION FOR ADMISSION. Welcome to the Elizabeth City State University Online Application. New Freshmen...www.ncmentor.org/applications/unc/apply/el...

5. City University A University which believes in forward-thinking business and leadership skills, exposing students to the latest technology in all courses of study.www.cityu.edu/

6. The City College of New York City College of New York (CUNY) A senior college of The City University of New York. Located in Manhattan. 138th Street at Convent Avenue. A Harlem masterpiece in the neo-Gothic...www.ccny.cuny.edu/[More Results from www.ccny.cuny.edu]


MetaSearch Engine Metasearch Engines are designed to

increase the coverage of web by forwarding users’ queries to multiple search engines Users’ requests are sent to multiple

search engines such as AlltheWeb, Google, MSN.

Then the results from the individual search engine are combined into a single result set to present to users.


Different Forms of MetaSearch

Submit different representations of the same query to the same search engine, then combine the results.

Submit the same query to several search engine adopting different information retrieval models, then combine the results.


How to combine the results retrieved by different source search engines is crucial for the success of a metasearch engine.

And this is the problem that social choice theory has been trying to answer.

Issues


Search Engine Watch

Interesting metat search engines are listed at http://www.searchenginewatch.com/links/

article.php/2156241

Social Choice and Rank Aggregation in MetaSearch/ Xiaotie DengAward Winners Source: www.searchenginewatch.com

VivisimoEnter a search term, and Vivismo will not only pull back matching responses from major search engines but also automatically organize the pages into categories. Slick and easy to use. Winner of Best Meta Search Engine award from Search Engine Watch in 2002.

EZ2WWWThis meta search engine provides results from AllTheWeb, AltaVista, Google, Open Directory, Teoma, Wisenut and Yahoo. Advanced search offers access to more than 1,000 specialized resources, searchable by category. Honorable mention for Best Meta Search Engine award from Search Engine Watch in 2002.

KartooIf you like the idea of seeing your web results visually, this meta search site shows the results with sites being interconnected by keywords. Honorable mention for Best Meta Search Engine award from Search Engine Watch in 2002.

SurfWaxSearches against major engines or provides those who open free accounts the ability to chose from a list of hundreds. Using the "SiteSnaps" feature, you can preview any page in the results and see where your terms appear in the document. Allows results or documents to be saved for future use. Honorable mention for Best Meta Search Engine award from Search Engine Watch in 2002.

FazzleFazzle offers a highly flexible and customizable interface to a wide variety of information sources, ranging from general web results to specialized search resources in a number of subject specific categories. Formerly called SearchOnline.

Social Choice and Rank Aggregation in MetaSearch/ Xiaotie DengOther Top Choices Source: www.searchenginewatch.com

InfoGridIn a compact format, InfoGrid provides direct links to major search sites and topical web sites in different categories. Meta search and news searching is also offered.

Infonetware RealTerm SearchThis site is primarily designed to demonstrate classification technology from Infogistics. It's a meta search engine, and it does topical classification of results, like Vivisimo. However, it is unique in that you can select several different topics, then "drill down" to see results from all of them, rather than being restricted to the results from only one topic.

IthakiIthaki is probably the most "global" of all meta search engines, available in 14 languages and offering more than 35 different categories for limiting your search. In addition, Ithaki offers country specific search, querying only local search engines rather than the regional versions of the major search engines.

IxquickMeta search engine that ranks results based on the number of "top 10" rankings a site receives from the various search engines.

ProFusionBrings back listings from several major search engines as well as "Invisible Web" resources. Formerly based at the University of Kansas, the site was purchased by search company Intelliseek in April 2000.

qbSearchWant to get multiple pages of results from a search engine combined into one single page? QB-Search will quickly join up to 200 pages of listings from major search engines.

Query ServerSearch against major web-wide search engines, as well as major news, health, money and government search services.

Turbo10Turbo10 is a metasearch Engine accesses both traditional web search engines and some invisible web databases, with a very speedy interface.

Social Choice and Rank Aggregation in MetaSearch/ Xiaotie DengPopular Services Source: www.searchenginewatch.com

DogpilePopular metasearch site owned by InfoSpace that sends a search to a customizable list of search engines, directories and specialty search sites, then displays results from each search engine individually.

ExciteFormerly a crawled-based search engine, Excite was acquired by InfoSpace in 2002 and uses the same underlying technology as the other InfoSpace meta search engines, but maintains its own portal features.

MetaCrawlerOne of the oldest meta search services, MetaCrawler began in July 1995 at the University of Washington. MetaCrawler was purchased by InfoSpace, an online content provider, in Feb. 97.

WebcrawlerFormerly a crawled-based search engine owned by Excite, Webcrawler was acquired by InfoSpace in 2002 and uses the same underlying technology as the other InfoSpace meta search engines, but offers a fast and clean, ad-free interface.

MammaFounded in 1996, Mamma.com is one of the oldest meta search engines on the web. Mamma searches against a variety of major crawlers, directories and specialty search sites. The service also provides a paid listings option for advertisers, Mamma Classifieds.

Search.comSearch.com is a meta search engine operated by CNET. It offers both web-wide search and a wide variety of specialty search options. Search.com uses technology from SavvySearch, which was acquired by CNET in October 1999. The SavvySearch site itself no longer operates. SavvySearch was one of the older metasearch services,

Social Choice and Rank Aggregation in MetaSearch/ Xiaotie DengSpecial Services Source: www.searchenginewatch.com

Family Friendly SearchMeta search service that queries major kid-friendly search engines.

Searchy.co.ukSearches 15 U.K. engines. The advanced search form allows you to change the order that results are presented, either by speed or manually to suit your own preferences.

TeRespondoSpanish metacrawler that searches on the more popular search engines. Motor de busqueda que busca tu consulta en los buscadores mas populares en espanol.

Watson for the MacintoshWatson is a "Swiss Army Knife" with nineteen interfaces to web content and services -- an improvement on Sherlock, with nearly twice as many tools, including Google Searching.


Social Choice Theory

Studies on protocols that help a group of people make collective decisions, such as vote.


A Fundamental problem

Given a collection of agents (voters) with preferences over different

alternatives (allocations, outcomes), how should society evaluate these

alternatives and make a decision for all that may be for the will of some voters

but against that of others.


Applications

Voters elect president from several candidates.

National polls for economic or political policy of the government

The procedure or rule of election The rank of metasearch engine

obtained from those of search engines


Group Descisions

How do we make decisions Flip a coin? Dictatorship? Democracy (Majority rule)?in a group with

Heterogeneity of individual preference


Group Decision Rules Majority rule , Condorcet paradox (voting cycle) Borda rule


A set of voters V={v1,v2,v3,…,Vn}

A set of alternatives or outcomes S={s1,s2,s3,…Sm}, with |S|=m; and

A set of preference relation P={R1,R2,R3…Rn}, called a preference profile, the preference relation Ri for each voter i is a

permutation (order) of elements in S.

Mathematical model


Example 1 Majority Rule

3 rational people have rational preferences over 2 alternatives {x,y}

Person

1 2 3

1st X Y X 1 : X>Y

Pref. i.e.Person 2 : Y>X

2nd Y X Y 3 : X>Y

How to Aggregate their preferences? How to choose?


Using majority rule.

Since more than ½ people (two out of three) prefer x to y.

Then the group prefers x to y

Social Choice and Rank Aggregation in MetaSearch/ Xiaotie DengExample 2 Condorcet Paradox

3 rational people have rational preferences over 3 alternatives {x,y,z}

Person

1 2 3

1st X Y Z 1 : X>Y>Z

Pref. 2nd Y Z X i.e. Person 2 : Y>Z>X

3rd Z X Y 3 : Z>X>Y


Person 1 2 3 1st X Y Z 1 : X>YPref. 2nd Y Z X for (x,y) 2 : Y>X

X>Y 3rd Z X Y 3 : X>Y Similarly, for (Y,Z) we can get Y>Z; for (Z,X)

we can get Z>X. Then X>Y>Z>X (cycling) , Intransitive

Not rational

Binary/paired Comparison With Majority rule


It was noted by Condorcet in the 18 century that no alternative can win a majority against all other alternatives.

Pairwise majority is not satisfactory in all cases.


Example 3 Borda Rule

For each voter, associate the number 1 with the most

preferred alternative, 2 with the second and so on,

Assign to each alternative the number equal to the sum of the numbers the individual

voters assigned to the alternative.


Person

1 2 3

1st X(1) Y(1) X(1) X(4) X

Pref. 2nd Y(2) X(2) W(2) Y(7) Y 3rd Z(3) W(3) Z(3) Z(10) W 4th W(4) Z(4) Y(4) W(9) Z

Then We get choice X>Y>W>Z


For above example, if we use binary/paired comparison With majority rule . We can get

X>Y in 2 out of 3, Y>W in 2 out of 3, W>Z in 2 out of 3, X>W in 3 out of 3, X>Z in 3 out of 3, Y>Z in 2 out of 3

Then we can achieve same choice X>Y>W>Z


For the previous example we had trouble with majority rule via binary/paired comparison, we get a tie between all three alternatives with the Borda’s rule: All three alternatives get a

sum of 6.


Some variations 1 with relevant scores available allotting each input system a point p to be

distributed according to relevance scores of the documents.

2 Weighted Borda-rule Each voter may not have equal

effectiveness to the final result. We may set more weight to good quality input systems.


Condorcet winner algorithm

It also comes from social choice theory. The Condorcet algorithm says that any candidate that can beat all other candidates in a head-to-head contest (pair-wise comparison) should win the election.


Step 1, Construct Condorcet Graph. For each candidate pair (x,y), there exists an

edge from x to y if x would receive at least as many votes as y in a head-to-head contest.

In Condorcet graph, there is at least one directed edge between every pair of candidates. ( we call the graph is semi-complete)

It may contains cycles in the graph. This is due to voting paradox of the condorcet voting.


Step 2, We form a new acyclic graph from an old cyclic one by contracting all of the nodes in a cycle into one. It is a strongly connected component graph (SCCG).

Definition of Strongly connected component(SCC): A strongly connected subgraph, S, of a directed

graph, D, such that no vertex or subset of vertices of D can be added to S such that the new subgraph is still strongly connected.


The graph is totally orderable at the level of the SCC’s and each SCC is a “pocket” of cycles, within which each candidate is tied.

Step 3, The condorcet-consistent Hamiltonian path is any Hamiltonian path through Condorcet graph.

Definition Hamiltonian path: A path between two vertices of a graph that visits each vertex exactly once.


Theorem 1. Every semi-complete graph contains a Hamiltonian path.

Theorem 2. Suppose x and y are nodes in a graph g, and that X and Y are nodes of the associated SCCG G such that x X and y Y. If there exists a path from X to Y in G, then every Condorcet path of g has x before y.

Refer to [Javed A. Aslam, Mark Montague 2001]

for proof.


Rank Aggregation in MetaSearch Here we discussed two cases which using

algorithm rooted at social choice theory for MetaSearch rank aggregation.

Data fusion track in TREC [Javed A. Aslam, Mark Montague 2001] Models for Metasearch in SIGIR2001

Rank aggregation for web search engine

[Cynthia Dwork, Ravi Kumar, Moni Naor, D.Sivakumar 2001]

Rank Aggregation Methods for the Web in WWW10


Data fusion track in TREC

TREC (Text Retrieval Conference ,see http://trec.nist.gov/) maintains about 6Gb of SGML tagged text, queries and respective answers for evaluation purposes.

The TREC organizers distribute data sets in advance and 50 new queries each year.

The competing teams then submit ranked lists of documents that their system gave in response to each query. And these retrieval systems will be evaluated.


These ranked lists are available for metasearch researchers to download and fuse.

For each query, every retrieval system will return top 1000 documents and relevant score is available.

Then given these results retrieved by many different retrieval systems, how to aggregate them for better performance?


Previous algorithms

Min, Max and Average Models [Fox and Shaw,1995] Linear Combination Model [Bartell 1995]

Logistic Regression Model


Example Min, Max and Average model The final score of each document d is based on

the scores given to d by each input systems (voters).

Algorithm Final scoreCombMin minimum of individual relevance scoresCombMed median of individual relevance scoresCombMax maximum of individual relevance scoresCombSum sum of individual relevance scoresCombANZ CombSum / num non-zero relevance scoresCombMNZ CombSum * num non-zero relevance scores


Linear Combination Model

The final score of document d is a simply linearly (each weighted differently) combining the normalized relevance scores given to each document.

i

iiLC dsadS )()(


Experiment result on TREC Model

The performance of rank aggregation is evaluated by average precision over the queries

Score-based borda-fuse is usually the best method among several borda variant algorithms.

It is better than best input system over most of data collection. Such as TREC3, TREC5


Experiment result II The performance of rank aggregation is evaluated

by average precision over the queries.

Condorcet-fusion is the only algorithm that , without training data, ever matches the performance of the best input system over TREC 9.

Condorcet-fusion seems particularly sensitive to the dependence of input systems. If the input systems (voters) are too similar, the performance will decrease.


Rank aggregation methods for web

New Challenges: Different from the case in TREC data fusion,

The coverage of various search engine is different

Thus some highly relevant web pages may not be ranked by some search engines.

Therefore, each voter ranks a partial candidate list


Preliminaries Given a universe U, an ordered list with

respect to U is an ordering of a subset S U, i.e., ,with each and is some ordering relation on S.

If contains all the elements in U, then it is said to be a full

list, otherwise it is called partial list.

]...[ 21 dxxx ,Sxi


Distance measures between two full lists with respect to a set S The Kendall tau distance It counts the number of pairwise disagreements

between two lists. The distance is given by

Normalize it by dividing the maximum possible value

)}()(),()(,|),{(),( jibutjijijiK

2/2S


Spearman footrule distance

Given two full lists and , the distance is given by

Normalize it by dividing the maximum value

s

iiiF

1)()(),(

2/2S


Distance measures for more than 2 list

Given several full lists , for instance, the normalized Footrule distance of to is given by

If are partial lists, let U denote the union of elements in and let be a full list with respect to U. Considering the distance between and the projection of with respect to , we have the induced footrule distance

k ,...,,, 21

k ,...,, 21

k

i ik FkF121 ),()/1(),...,,,(

k ,...,, 21

k ,...,, 21

i

i

k

i ik iFF1 |1 ),(),...,,(


Given one full list and a partial list, the scaled footrule distance is given by

In a similar manner, Kendall tau distance can be extended for partial lists and more than two lists.

Example then

)1,3(),3,2(),2,1(),3,2,1( 54321

3),...,,,( 521 K

iiiF /)(/)(),('


Optimal rank aggregation

The question is Given (full or partial) lists , find a such

that is a full list with respect to the union of the elements of

and minimizes

The aggregation obtained by optimizing Kendall distance is called Kemeny optimal aggregation.

k ,...,, 21

k ,...,, 21 ),...,,,( 21 kK


When k>=4,computing the Kemeny optimal aggregation is NP-hard.

(please refer to [Cynthia Dwork, Ravi Kumar,

Moni Naor, D.Sivakumar 2001] for detailed proof )

We can use Spearman footrule distance to approximate the Kendall distance.


Proposition 1 For any two full lists , ,

Proposition 2 if is the Kemeny optimal aggregation of full lists , and optimizes the footrule aggregation, then

),(2),(),( KFK

k ,...,, 21

),...,,(2),...,,( 11 kk KK


Proposition 3 Given full lists if the median positions of the candidates in the lists form a permutation, then this permutation is a footrule optimal aggregation.

Proposition 4 Footrule optimal aggregation of full lists can be computed in polynomial time, specifically, the time to find a minimum cost perfect matching in a bipartite graph.

k ,...,, 21


The footrule optimal aggregation for full lists can be recast as a minimum cost bipartite matching problem.

For partial list, we define the bipartite graph as before, except that the weights are given by scaled footrule distance .

The weight W(c,p) of a ranking that places element c at position p is

k

i ii npcpcW1

//)(),(


As before , we can solve the minimum cost maximum matching problem on this bipartite graph to obtain the footrule aggregation algorithm for partial lists.

We called this method the scaled footrule aggregation (SFO)

Scaled Footrule aggregation


Locally Kemeny optimal aggregation Since Kemeny optimal aggregation is

NP-hard for k>=4, Here we introduce locally Kemeny optimal aggregation.

It ensures satisfaction of the extended Condorcet principle and yet remains computationally tractable.


A full list is locally kemeny optimal aggregation of partial lists if there is no full list that can be obatined from by a single transposition of an adjacent pair of elements and for which

k ,...,, 21

),...,,(),...,,( 11 kk KK


A local Kemenization of full list with respect to is a procedure that computes a locally Kemeny optimal aggregation of that is maximally consistent with .

This approach also preserves the strengths of the initial aggregation

k ,...,, 21

k ,...,, 21


It has following characteristics.1. The Condorcet losers receive low rank, while the

Condorcet winners receive high rank.

2. The result disagrees with on the order of any given pair (i,j) of elements only if a majority of those ‘s expressing opinions disagree with on (i,j)

3. For every ,the length d prefix of the output is a local Kemenization of the top d elements in .

d1

social choice and rank aggregation in metasearch/ xiaotie deng lecture 9: rank aggregation in...

Documents

dublin city university

city university london

city university home

oklahoma city university

city polytechnic of

forwardthinking business

prospective students

metasearch xiaotie deng