Lecture Notes in Computer Science 9626

Commenced Publication in 1973Founding and Former Series Editors:Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen

Editorial Board

David HutchisonLancaster University, Lancaster, UK

Takeo KanadeCarnegie Mellon University, Pittsburgh, PA, USA

Josef KittlerUniversity of Surrey, Guildford, UK

Jon M. KleinbergCornell University, Ithaca, NY, USA

Friedemann MatternETH Zurich, Zürich, Switzerland

John C. MitchellStanford University, Stanford, CA, USA

Moni NaorWeizmann Institute of Science, Rehovot, Israel

C. Pandu RanganIndian Institute of Technology, Madras, India

Bernhard SteffenTU Dortmund University, Dortmund, Germany

Demetri TerzopoulosUniversity of California, Los Angeles, CA, USA

Doug TygarUniversity of California, Berkeley, CA, USA

Gerhard WeikumMax Planck Institute for Informatics, Saarbrücken, Germany

More information about this series at http://www.springer.com/series/7409

Nicola Ferro • Fabio CrestaniMarie-Francine Moens • Josiane MotheFabrizio Silvestri • Giorgio Maria Di NunzioClaudia Hauff • Gianmaria Silvello (Eds.)

Advances inInformation Retrieval38th European Conference on IR Research, ECIR 2016Padua, Italy, March 20–23, 2016Proceedings

123

EditorsNicola FerroDepartment of Information EngineeringUniversity of PaduaPadovaItaly

Fabio CrestaniFaculty of InformaticsUniversity of Lugano (USI)LuganoSwitzerland

Marie-Francine MoensDepartment of Computer ScienceKatholieke Universiteit LeuvenHeverleeBelgium

Josiane MotheSystèmes d’informations, Big Dataet Recherche d’Information

Institut de Recherche en Informatiquede Toulouse IRIT/équipe SIG

Toulouse Cedex 04France

Fabrizio SilvestriYahoo! Labs LondonLondonUK

Giorgio Maria Di NunzioDepartment of Information EngineeringUniversity of PaduaPadovaItaly

Claudia HauffTU Delft - EWI/ST/WISDelftThe Netherlands

Gianmaria SilvelloDepartment of Information EngineeringUniversity of PaduaPadovaItaly

ISSN 0302-9743 ISSN 1611-3349 (electronic)Lecture Notes in Computer ScienceISBN 978-3-319-30670-4 ISBN 978-3-319-30671-1 (eBook)DOI 10.1007/978-3-319-30671-1

Library of Congress Control Number: 2016932329

LNCS Sublibrary: SL3 – Information Systems and Applications, incl. Internet/Web, and HCI

© Springer International Publishing Switzerland 2016This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of thematerial is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,broadcasting, reproduction on microfilms or in any other physical way, and transmission or informationstorage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology nowknown or hereafter developed.The use of general descriptive names, registered names, trademarks, service marks, etc. in this publicationdoes not imply, even in the absence of a specific statement, that such names are exempt from the relevantprotective laws and regulations and therefore free for general use.The publisher, the authors and the editors are safe to assume that the advice and information in this book arebelieved to be true and accurate at the date of publication. Neither the publisher nor the authors or the editorsgive a warranty, express or implied, with respect to the material contained herein or for any errors oromissions that may have been made.

Printed on acid-free paper

This Springer imprint is published by SpringerNatureThe registered company is Springer International Publishing AG Switzerland

Preface

These proceedings contain the full papers, short papers, and demonstrations selectedfor presentation at the 38th European Conference on Information Retrieval (ECIR2016). The event was organized by the Information Management Systems (IMS)research group1 of the Department of Information Engineering2 of the University ofPadua3, Italy. The conference was held during March 20–23 2016, in Padua, Italy.

ECIR 2016 received a total of 284 submissions in three categories: 201 full papersout of which seven papers in the reproducibility track, 66 short papers, and 17demonstrations.

The geographical distribution of the submissions was as follows: 51 % were fromEurope, 21 % from Asia, 19 % from North and South America, 7 % from North Africaand the Middle East, and 2 % from Australasia.

All submissions were reviewed by at least three members of an international two-tierProgram Committee. Of the full papers submitted to the conference, 42 were acceptedfor oral presentation (22 % of the submitted ones) and eight as posters (4 % of thesubmitted ones). Of the short papers submitted to the conference, 20 were accepted forposter presentation (30 % of the submitted ones). In addition, six demonstrations (35 %of the submitted ones) were accepted. The accepted contributions represent the stateof the art in information retrieval, cover a diverse range of topics, propose novelapplications, and indicate promising directions for future research.

We thank all Program Committee members for their time and effort in ensuring thehigh quality of the ECIR 2016 program.

ECIR 2016 continued the reproducibility track introduced at ECIR 2015, whichspecifically invited the submission of papers reproducing a single paper or a group ofpapers from a third party, where the authors were not directly involved in the originalpaper. Authors were requested to emphasize the motivation for selecting the papers tobe reproduced, the process of how results were attempted to be reproduced (success-fully or not), the communication that was necessary to gather all information, thepotential difficulties encountered, and the result of the process. Of the seven paperssubmitted to this track, four were accepted (57 % of the submitted ones).

A panel on “Data-Driven Information Retrieval” was organized at ECIR byMaristella Agosti. The panel stems from the fact that information retrieval has alwaysbeen concerned with finding the “needle in a haystack” to retrieve the most relevantinformation from huge amounts of data, able to best address user information needs.Nevertheless, nowadays we are facing a radical paradigm shift, common also to manyother research fields, and information retrieval is becoming an increasingly data-drivenscience due, for example, to recent developments in machine learning, crowdsourcing,

1 http://ims.dei.unipd.it/2 http://www.dei.unipd.it/3 http://www.unipd.it/

user interaction analysis, and so on. The goal of the panel is to discuss the emergenttrends in this area, their advantages, their pitfalls, and their implications for the futureof the field.

Additionally, ECIR 2016 hosted four tutorials and four workshops covering a rangeof information retrieval topics. These were selected by workshop and tutorialcommittees.

The workshops were:

– Third International Workshop on Bibliometric-Enhanced Information Retrieval(BIR2016)

– First International Workshop on Modeling, Learning and Mining for Cross/Multilinguality (MultiLingMine 2016)

– ProActive Information Retrieval: Anticipating Users’ Information Needs (ProAct IR)– First International Workshop on Recent Trends in News Information Retrieval

(NewsIR 2016)

The following ECIR 2016 tutorials were selected:

– Collaborative Information Retrieval: Concepts, Models and Evaluation– Group Recommender Systems: State of the Art, Emerging Aspects and Techniques,

and Research Challenges– Living Labs for Online Evaluation: From Theory to Practice (LiLa2016)– Real-Time Bidding Based Display Advertising: Mechanisms and Algorithms

(RTBMA 2016)

Short descriptions of these workshops and tutorials are included in the proceedings.We would like to thank our invited speakers for their contributions to the program:

Jordan Boyd-Graber (University of Colorado, USA), Emine Yilmaz (UniversityCollege London, UK), and Domonkos Tikk (Gravity R&D, Hungary). Short descrip-tions of these talks are included in the proceedings.

We are grateful to the panel led by Stefan Rüger for selecting the recipients of the2015 Microsoft BCS/BCS IRSG Karen Spärck Jones Award, and we congratulateJordan Boyd-Graber and Emine Yilmaz on receiving this award (unique to 2015, thepanel decided to make two full awards).

Considering the long history of ECIR, which is now at it 38th edition, ECIR 2016introduced a new award, the Test of Time (ToT) Award, to recognize research that hashad long-lasting influence, including impact on a subarea of information retrievalresearch, across subareas of information retrieval research, and outside of the infor-mation retrieval research community (e.g., non-information retrieval research orindustry).

On the final day of the conference, the Industry Day ran in parallel with the con-ference session with the goal of bringing an exciting program containing a mix ofinvited talks by industry leaders with presentations of novel and innovative ideas fromthe search industry. A short description of the Industry Day is included in theseproceedings.

ECIR 2016 was held under the patronage of: Regione del Veneto (Veneto Region),Comune di Padova (Municipality of Padua), University of Padua, Department ofInformantion Engineering, and Department of Mathematics.

VI Preface

Finally, ECIR 2016 would have not been possible without the generous financialsupport from our sponsors: Google (gold level); Elsevier, Spotify, and Yahoo! Labs(palladium level); Springer (silver level); and Yandex (bronze level). The conferencewas supported by the ELIAS Research Network Program of the European ScienceFoundation, University of Padua, Department of Informantion Engineering andDepartment of Mathematics.

March 2016 Nicola FerroFabio Crestani

Marie-Francine MoensJosiane Mothe

Fabrizio SilvestriGiorgio Maria Di Nunzio

Claudia HauffGianmaria Silvello

Preface VII

Organization

General Chair

Nicola Ferro University of Padua, Italy

Program Chairs

Fabio Crestani University of Lugano (USI), SwitzerlandMarie-Francine Moens KU Leuven, Belgium

Short Paper Chairs

Josiane Mothe ESPE, IRIT, Université de Toulouse, FranceFabrizio Silvestri Yahoo! Labs, London

Student Mentorship Chairs

Jaana Kekäläinen University of Tampere, FinlandPaolo Rosso Universitat Politècnica de València, Spain

Workshop Chairs

Paul Clough University of Sheffield, UKGabriella Pasi University of Milano Bicocca, Italy

Tutorial Chairs

Christina Lioma University of Copenhagen, DenmarkStefano Mizzaro University of Udine, Italy

Demo Chairs

Giorgio Maria Di Nunzio University of Padua, ItalyClaudia Hauff TU Delft, The Netherlands

Industry Day Chairs

Omar Alonso Microsoft Bing, USAPavel Serdyukov Yandex, Russia

Test of Time (ToT) Award Chair

Norbert Fuhr University of Duisburg-Essen, Germany

Best Paper Award Chair

Jaap Kamps University of Amsterdam, The Netherlands

Student Grant Chair

John Tait johntait.net Ltd., UK

Local Organization Chair

Gianmaria Silvello University of Padua, Italy

Sponsorship Chair

Emanuele Di Buccio University of Padua, Italy

Local Organizing Team

Antonio Camporese University of Padua, ItalyLinda Cappellato University of Padua, ItalyMarco Ferrante University of Padua, ItalyDebora Leoncini University of Padua, ItalyMaria Maistro University of Padua, Italy

Website and Communication Material Chair

Ivano Masiero University of Padua, Italy

Program Committee

Full-Paper Meta-Reviewers

Giambattista Amati Fondazione Ugo Bordoni, ItalyLeif Azzopardi University of Glasgow, UKRoberto Basili University of Rome Tor Vergata, ItalyMohand Boughanem IRIT, Paul Sabatier University, FrancePaul Clough University of Sheffield, UKBruce Croft University of Massachusetts Amherst, USAArjen de Vries Radboud University, The NetherlandsNorbert Fuhr University of Duisburg-Essen, GermanyEric Gaussier Université Joseph Fourier, FranceCathal Gurrin Dublin City University, IrelandGareth Jones Dublin City University, Ireland

X Organization

Joemon Jose University of Glasgow, UKGabriella Kazai Lumi.do, UKUdo Kruschwitz University of Essex, UKOren Kurland Technion - Israel Institute of Technology, IsraelHenning Müller University of Applied Sciences Western Switzerland,

SwitzerlandWolfgang Nejdl L3S and University of Hannover, GermanyIadh Ounis University of Glasgow, UKGabriella Pasi University of Milano Bicocca, ItalyPaolo Rosso Universitat Politècnica de València, SpainStefan Rüger Knowledge Media Institute, UK

Full-Paper, Short Paper, and Demonstration Reviewers

Mikhail Ageev Moscow State University, RussiaDirk Ahlers Norwegian University of Science and Technology,

NorwayAhmet Aker University of Sheffield, UKElif Aktolga University of Massachusetts Amherst, USAM-Dyaa Albakour Signal Media, UKOmar Alonso Microsoft, USAIsmail Sengor Altingovde Middle East Technical University, TurkeyRobin Aly University of Twente, The NetherlandsGiambattista Amati Fondazione Ugo Bordoni, ItalyLinda Andersson TU Wien, AustriaAvi Arampatzis Democritus University of Thrace, GreeceJaime Arguello University of North Carolina at Chapel Hill, USASeyed Ali Bahrainian University of Lugano (USI), SwitzerlandKrisztian Balog University of Stavanger, NorwayAlvaro Barreiro University of A Coruña, SpainRoberto Basili University of Rome Tor Vergata, ItalySrikanta Bedathur Jagannath IBM Research, IndiaAlejandro Bellogin Universidad Autonoma de Madrid, SpainPatrice Bellot Université Aix-Marseille, FranceCatherine Berrut LIG, Université Joseph Fourier Grenoble I, FranceRalf Bierig TU Wien, AustriaToine Bogers Aalborg University Copenhagen, DenmarkAlessandro Bozzon Delft University of Technology, The NetherlandsMateusz Budnik University of Grenoble, FrancePaul Buitelaar Insight - National University of Ireland, Galway,

IrelandFidel Cacheda Universidad de A Coruña, SpainPavel Calado INESC-ID, Instituto Superior Técnico, Universidade

de Lisboa, PortugalFazli Can Bilkent University, TurkeyMark Carman Monash University, Australia

Organization XI

Claudio Carpineto Fondazione Ugo Bordoni, ItalyMarc Cartright Google Inc., USAJean-Pierre Chevallet Grenoble Alpes University, FranceLuisa Coheur Luisa Coheur, INESC-ID, Instituto Superior Técnico,

Universidade de Lisboa, PortugalAlfredo Cuzzocrea ICAR-CNR and University of Calabria, ItalyEva D’hondt Laboratoire d’Informatique pour la Mécanique et les

Sciences de l’Ingénieur (LIMSI), FranceWalter Daelemans University of Antwerp, BelgiumMartine De Cock University of Washington Tacoma, USAPablo de la Fuente Universidad de Valladolid, SpainThomas Demeester Ghent University, BelgiumEmanuele Di Buccio University of Padua, ItalyGiorgio Maria Di Nunzio University of Padua, ItalyVladimir Dobrynin Saint-Petersburg State University, RussiaHuizhong Duan WalmartLabs, USACarsten Eickhoff ETH Zurich, SwitzerlandDavid Elsweiler University of Regensburg, GermanyLiana Ermakova Institut de Recherche en Informatique de Toulouse

(IRIT), Perm State National Research University,France

Hui Fang University of Delaware, USAYi Fang Santa Clara University, USAJuan M. Fernández-Luna University of Granada, SpainLuanne Freund University of British Columbia, CanadaKarin Friberg Heppin University of Gothenburg, SwedenIngo Frommholz University of Bedfordshire, UKPatrick Gallinari LIP6 - University of Paris 6, FranceKavita Ganesan University of Illinois at Urbana Champaign, USAAnastasia Giachanou University of Lugano (USI), SwitzerlandGiorgos Giannopoulos Imis Institute, Athena R.C., GreeceLorraine Goeuriot Laboratoire d’informatique de Grenoble, FranceAyse Goker Robert Gordon University, UKMichael Granitzer Universität Passau, GermanyGuillaume Gravier IRISA and Inria Rennes, FranceDavid Grossman Georgetown University, USAAntonino Gulli Elsevier, The NetherlandsMatthias Hagen Bauhaus-Universität Weimar, GermanyAllan Hanbury TU Wien, AustriaPreben Hansen Stockholm University, SwedenDonna Harman NIST, USAMorgan Harvey Northumbria University, UKClaudia Hauff Delft University of Technology, The NetherlandsJer Hayes IBM, IrelandBen He University of Chinese Academy of Sciences, ChinaDaqing He University of Pittsburgh, USA

XII Organization

Jiyin He CWI, The NetherlandsNathalie Hernandez IRIT, FranceKatja Hofmann Microsoft, UKFrank Hopfgartner University of Glasgow, UKAndreas Hotho University of Würzburg, GermanyGilles Hubert IRIT, University of Toulouse, FranceDmitry Ignatov National Research University Higher School

of Economics, RussiaShen Jialie Singapore Management University, SingaporeJiepu Jiang University of Massachusetts Amherst, USAHideo Joho University of Tsukuba, JapanJaap Kamps University of Amsterdam, The NetherlandsNattiya Kanhabua Aalborg University, DenmarkDiane Kelly University of North Carolina at Chapel Hill, USALiadh Kelly Trinity College Dublin, IrelandYiannis Kompatsiaris Information Technologies Institute, CERTH, GreeceAlexander Kotov Wayne State University, USAUdo Kruschwitz University of Essex, UKMonica Landoni University of Lugano (USI), SwitzerlandMartha Larson Delft University of Technology, The NetherlandsKyumin Lee Utah State University, USAWang-Chien Lee Pennsylvania State University, USAJohannes Leveling Elsevier, The NetherlandsLiz Liddy Center for Natural Language Processing,

Syracuse University, USAChristina Lioma University of Copenhagen, DenmarkXiaozhong Liu Indiana University Bloomington, USAElena Lloret University of Alicante, SpainFernando Loizides University of Wolverhampton, UKDavid Losada University of Santiago de Compostela, SpainBernd Ludwig University of Regensburg, GermanyMihai Lupu TU Wien, AustriaYuanhua Lv Microsoft Research, USACraig Macdonald University of Glasgow, UKAndrew Macfarlane City University London, UKWalid Magdy Qatar Computing Research Institute, QatarMarco Maggini University of Siena, ItalyThomas Mandl University of Hildesheim, GermanyStephane Marchand-Maillet University of Geneva, SwitzerlandMiguel Martinez-Alvarez Signal Media, UKBruno Martins NESC-ID, Instituto Superior Técnico, Universidade

de Lisboa, PortugalYosi Mass IBM Haifa Research Lab, IsraelMax Chevalier IRIT, FranceEdgar Meij Yahoo Labs, UKMarcelo Mendoza Universidad Técnica Federico Santa María, Chile

Organization XIII

Alessandro Micarelli Roma Tre University, ItalyDunja Mladenic Jozef Stefan Institute, SloveniaJosiane Mothe Institut de Recherche en Informatique de Toulouse,

FranceHannes Mühleisen Centrum Wiskunde & Informatica (CWI), The

NetherlandsPhilippe Mulhem LIG-CNRS, FranceDong Nguyen University of Twente, The NetherlandsBoris Novikov St. Petersburg University, RussiaAndreas Nürnberger Otto von Guericke University of Magdeburg,

GermanyNeil O’Hare Yahoo Labs, USAMichael O’Mahony University College Dublin, IrelandMichael Oakes University of Wolverhampton, UKIadh Ounis University of Glasgow, UKKaterina Pastra Cognitive Systems Research Institute, GreeceVirgil Pavlu Northeastern University, USAPavel Pecina Charles University in Prague, Czech RepublicVivien Petras HU Berlin, GermanyKaren Pinel-Sauvagnat Institut de Recherche en Informatique de Toulouse

(IRIT), FranceFlorina Piroi TU Wien, AustriaVassilis Plachouras Thomson Reuters, UKBarbara Poblete University of Chile, ChileGeorges Quénot Laboratoire d’Informatique de Grenoble, CNRS,

FranceDmitri Roussinov University of Strathclyde, UKAlan Said Recorded Future, SwedenMichail Salampasis Technological Educational Institute of Thessaloniki,

GreeceRodrygo Santos Universidade Federal de Minas Gerais, BrazilMarkus Schedl Johannes Kepler University Linz, AustriaRalf Schenkel Universität Passau, GermanyPascale Sébillot IRISA/INSA Rennes, FranceFlorence Sedes IRIT, Paul Sabatier University, FranceGiovanni Semeraro University of Bari, ItalyAzadeh Shakery University of Tehran, IranJan Snajder University of Zagreb, CroatiaParikshit Sondhi WalmartLabs, USAYang Song Microsoft Research, USASimone Stumpf City University London, UKL. Venkata Subramaniam IBM Research, IndiaLynda Tamine IRIT University Paul Sabatier, FranceBart Thomee Yahoo Labs, USAIlya Tikhomirov Institute for Systems Analysis, FRC CSC RAS,

Russia

XIV Organization

Marko Tkalcic Johannes Kepler University, AustriaDolf Trieschnigg Mydatafactory, The NetherlandsChristos Tryfonopoulos University of Peloponnese, GreeceMing-Feng Tsai National Chengchi University, TaiwanTheodora Tsikrika Information Technologies Institute, CERTH, GreeceDenis Turdakov Institute for System Programming RAS, RussiaAta Turk Boston University, USAYannis Tzitzikas University of Crete and FORTH-ICS, GreeceMarieke van Erp Vrije Universiteit Amsterdam, The NetherlandsJacco van Ossenbruggen CWI & VU University Amsterdam, The NetherlandsNatalia Vassilieva Hewlett Packard Labs, USASumithra Velupillai Stockholm University, SwedenSuzan Verberne Radboud University, The NetherlandsStefanos Vrochidis Information Technologies Institute, CERTH, GreeceIvan Vulic Cambridge University, UKJeroen Vuurens Delft University of Technology, The NetherlandsV.G. Vinod Vydiswaran University of Michigan, USAXiaojun Wan Peking University, ChinaHongning Wang University of Virginia, USAJun Wang University College London, UKLidan Wang University of Illinois, Urbana-Champaign, USAWouter Weerkamp 904Labs, The NetherlandsChrista Womser-Hacker University of Hildesheim, GermanyTao Yang Ask.com and UCSB, USADavid Zellhoefer Berlin State Library, GermanyDan Zhang Facebook, USALanbo Zhang University of California, Santa Cruz, USADuo Zhang University of Illinois, Urbana-Champaign, USAKe Zhou Yahoo Labs, UKGuido Zuccon Queensland University of Technology, Australia

Reproducible IR Track Reviewers

Ahmet Aker University of Sheffield, UKCatherine Berrut LIG, Université Joseph Fourier Grenoble, FranceFidel Cacheda Universidad de A Coruña, SpainFazli Can Bilkent University, TurkeyLuisa Coheur INESC-ID, Instituto Superior Técnico, Universidade

de Lisboa, PortugalPablo de la Fuente Universidad de Valladolid, SpainThomas Demeester Ghent University, BelgiumNorbert Fuhr University of Duisburg-Essen, GermanyGuillaume Gravier IRISA and Inria Rennes, FranceDonna Harman NIST, USAKatja Hofmann Microsoft, UKAndreas Hotho University of Würzburg, Germany

Organization XV

David Losada University of Santiago de Compostela, SpainCraig Macdonald University of Glasgow, UKEdgar Meij Yahoo Labs, UKPhilippe Mulhem LIG-CNRS, FranceKaren Pinel-Sauvagnat IRIT, FranceMarkus Schedl Johannes Kepler University Linz, AustriaRalf Schenkel Universitaet Passau, GermanyFlorence Sedes IRIT, Paul Sabatier University, FranceSuzan Verberne Radboud University, The NetherlandsWouter Weerkamp 904Labs, The Netherlands

Tutorial Selection Committee

Leif Azzopardi University of Glasgow, UKAlejandro Bellogin Universidad Autonoma de Madrid, SpainRonan Cummins University of Cambridge, UKJulio Gonzalo UNED, SpainDjoerd Hiemstra University of Twente, The NetherlandsEvangelos Kanoulas University of Amsterdam, The NetherlandsDiane Kelly University of North Carolina at Chapel Hill, USAJian-Yun Nie Université de Montréal, CanadaThomas Roelleke Queen Mary University of London, UKFalk Scholer RMIT University, AustraliaFabrizio Sebastiani Qatar Computing Research Institute, QatarTheodora Tsikrika Information Technologies Institute, CERTH, Greece

Additional Reviewers

Aggarwal, NitishAgun, DanielBalaneshin-Kordan, SaeidBasile, PierpaoloBiancalana, ClaudioBoididou, ChristinaBordea, GeorgetaCaputo, AnnalinaChen, Yi-Lingde Gemmis, MarcoFafalios, PavlosFarnadi, GolnooshFreund, LuanneFu, Tao-YangGialampoukidis, IliasGossen, TatianaGrachev, ArtemGrossman, David

Hasibi, FaeghehHerrera, JoseHung, Hui-JuJin, XinKaliciak, LeszekKamateri, EleniKotzyba, MichaelLin, Yu-SanLipani, AldoLoni, BabakLow, ThomasLudwig, PhilippLuo, RuiMota, PedroNarducci, FedelucioNikolaev, FedorOnal, K. DilekPalomino, Marco

XVI Organization

Student Mentors

Paavo Arvola University of Tampere, FinlandRafael E. Banchs I2R SingaporeRafael Berlanga Llavori Universitat Jaume I, SpainPia Borlund University of Copenhagen, DenmarkDavide Buscaldi Université Paris XIII, FranceFidel Cacheda University of A Coruña, SpainMarta Costa-Jussà Instituto Politécnico Nacional México, MexicoWalter Daelemans University of Antwerp, The NetherlandsKareem M. Darwish Qatar Computing Research Institute, QatarMaarten de Rijke University of Amsterdam, The NetherlandsMarcelo Luis Errecalde Universidad Nacional de San Luís, ArgentinaJulio Gonzalo UNED, SpainHugo Jair Escalante INAOE Puebla, MexicoJaap Kamps University of Amsterdam, The NetherlandsHeikki Keskustalo University of Tampere, FinalandGreg Kondrak University of Alberta, CanadaZornitsa Kozareva Yahoo! Labs, USAMandar Mitra Indian Statistical Institute, IndiaManuel Montes y Gómez INAOE Puebla, MexicoAlessandro Moschitti Qatar Computing Research Institute, QatarPreslav Nakov Qatar Computing Research Institute, QatarDoug Oard University of Maryland, USAIadh Ounis University of Glasgow, UKKaren Pinel-Sauvagnat IRIT, Université de Toulouse, FranceIan Ruthven University of Strathclyde, UKGrigori Sidorov Instituto Politécnico Nacional México, MexicoThamar Solorio University of Houston, USAElaine Toms University of Sheffield, UKChrista Womser-Hacker University of Hildesheim, Germany

Papadakos, PanagiotisParapar, JavierPetkos, GeorgiosRaftopoulou, ParaskeviRamanath, MayaRasmussen, EdieRekabsaz, NavidRodrigues, HugoSarwar, Sheikh MuhammadSchinas, ManosSchlötterer, JörgŞimon, Anca-Roxana

Sushmita, ShanuSymeonidis, SymeonThiel, MarcusToraman, CagriValcarce, DanielVergoulis, ThanasisWang, ZhenruiWood, IanXu, TanYu, HangZheng, Lu

Organization XVII

Test of Time (ToT) Award Committee

Maristella Agosti University of Padua, ItalyPia Borlund University of Copenhagen, DenmarkDjoerd Hiemstra University of Twente, The NetherlandsKalervo Järvelin University of Tampere, FinlandGabriella Kazai Lumi.do, UKIadh Ounis University of Glasgow, UKJacques Savoy University of Neuchatel, Switzerland

Patronage

Platinum Sponsors

XVIII Organization

Gold Sponsors

Palladium Sponsors

Silver Sponsors

Bronze Sponsors

Organization XIX

Keynote Talks

Machine Learning Shouldn’t be a Black Box

Jordan Boyd-Graber

University of Colorado, Boulder CO 80309, USA

Machine learning is ubiquitous: detecting spam e-mails, flagging fraudulent purchases,and providing the next movie in a Netflix binge. But few users at the mercy of machinelearning outputs know what’s happening behind the curtain. My research goal is todemystify the black box for non-experts by creating algorithms that can inform, col-laborate with, compete with, and understand users in real-world settings.

This is at odds with mainstream machine learning—take topic models. Topicmodels are sold as a tool for understanding large data collections: lawyers scouringEnron e-mails for a smoking gun, journalists making sense of Wikileaks, or humanistscharacterizing the oeuvre of Lope de Vega. But topic models’ proponents never askedwhat those lawyers, journalists, or humanists needed. Instead, they optimized held-outlikelihood. When my colleagues and I developed the interpretability measure to assesswhether topic models’ users understood their outputs, we found that interpretability andheld-out likelihood were negatively correlated [2]! The topic modeling community(including me) had fetishized complexity at the expense of usability.

Since this humbling discovery, I’ve built topic models that are a collaborationbetween humans and computers. The computer starts by proposing an organizationof the data. The user responds by separating confusing clusters, joining similar clusterstogether, or comparing notes with another user [5]. The model updates and then directsthe user to problematic areas that it knows are wrong. This is a huge improvement overthe “take it or leave it” philosophy of most machine learning algorithms.

This is not only a technical improvement but also an improvement to the socialprocess of machine learning adoption. A program manager who used topic models tocharacterize NIH investments uncovered interesting synergies and trends, but the resultswere unpresentable because of a fatal flaw: one of the 700 clusters lumped urologytogether with the nervous system, anathema to NIH insiders [14]. Our tools allownon-experts to fix such obvious problems (obvious to a human, that is), allowingmachine learning algorithms to overcome the social barriers that often hamperadoption.

Our realization that humans have a lot to teach machines led us to simultaneousmachine interpretation [3]. Because verbs end phrases in many languages, such asGerman and Japanese, existing algorithms must wait until the end of a sentence tobegin translating (since English sentences have verbs near the start). We learned tricksfrom professional human interpreters—passivizing sentences and guessing the verb—to translate sentences sooner [4], letting speakers and algorithms cooperate together andenabling more natural cross-cultural communication.

The reverse of cooperation is competition; it also has much to teach computers. I’veincreasingly looked at language-based games whose clear goals and intrinsic fun speedresearch progress. For example, in Diplomacy, users chat with each other while

marshaling armies for world conquest. Alliances are fluid: friends are betrayed andenemies embraced as the game develops. However, users’ conversations let us predictwhen friendships break: betrayers writing ostensibly friendly messages before abetrayal become more polite, stop talking about the future, and change how much theywrite [13]. Diplomacy may be a nerdy game, but it is a fruitful testbed to teachcomputers to understand messy, emotional human interactions.

A game with higher stakes is politics. However, just like Diplomacy, the words thatpeople use reveal their underlying goals; computational methods can help expose the“moves” political players can use. With collaborators in political science, we’ve builtmodels that: show when politicians in debates strategically change the topic to influ-ence others [9, 11]; frame topics to reflect political leanings [10]; use subtle linguisticphrasing to express their political leaning [7]; or create political subgroups with largerpolitical movements [12].

Conversely, games also teach humans how computers think. Our trivia-playingrobot [1, 6, 8] faced off against four former Jeopardy champions in front of 600 highschool students.1 The computer claimed an early lead, but we foolishly projected thecomputer’s thought process for all to see. The humans learned to read the algorithm’sranked dot products and schemed to answer just before the computer. In five years ofteaching machine learning, I’ve never had students catch on so quickly to how linearclassifiers work. The probing questions from high school students in the audienceshowed they caught on too. (Later, when we played again against Ken Jennings,2 he satin front of the dot products and our system did much better.)

Advancing machine learning requires closer, more natural interactions. However,we still require much of the user—reading distributions or dot products—rather thannatural language interactions. Document exploration tools should describe in wordswhat a cluster is, not just provide inscrutable word clouds; deception detection systemsshould say why a betrayal is imminent; and question answers should explain how itknows Aaron Burr shot Alexander Hamilton. My work will complement machinelearning’s ubiquity with transparent, empathetic, and useful interactions with users.

Bibliography

1. Boyd-Graber, J., Satinoff, B., He, H., Daumé III, H.: Besting the quiz master:crowdsourcing incremental classification games. In: Empirical Methods in NaturalLanguage Processing (2012). http://www.cs.colorado.edu/*jbg/docs/qb_emnlp_2012.pdf

2. Chang, J., Boyd-Graber, J., Wang, C., Gerrish, S., Blei, D.M.: Reading tea leaves:how humans interpret topic models. In: Proceedings of Advances in NeuralInformation Processing Systems (2009). http://www.cs.colorado.edu/*jbg/docs/nips2009-rtl.pdf

XXIV J. Boyd-Graber

1 https://www.youtube.com/watch?v=LqsUaprYMOw2 https://www.youtube.com/watch?v=kTXJCEvCDYk

3. Grissom II, A., He, H., Boyd-Graber, J., Morgan, J., Daumé III, H.: Don’t until thefinal verb wait: reinforcement learning for simultaneous machine translation. In:Proceedings of Empirical Methods in Natural Language Processing (2014). http://www.cs.colorado.edu/*jbg/docs/2014_emnlp_simtrans.pdf

4. He, H., Grissom II, A., Boyd-Graber, J., Daumé III, H.: Syntax-based rewriting forsimultaneous machine translation. In: Empirical Methods in Natural LanguageProcessing (2015). http://www.cs.colorado.edu/*jbg/docs/2015_emnlp_rewrite.pdf

5. Hu, Y., Boyd-Graber, J., Satinoff, B., Smith, A.: Interactive topic modeling. Mach.Learn. 95(3), 423–469 (2014). http://dx.doi.org/10.1007/s10994-013-5413-0

6. Iyyer, M., Boyd-Graber, J., Claudino, L., Socher, R., Daumé III, H.: A neuralnetwork for factoid question answering over paragraphs. In: Proceedings ofEmpirical Methods in Natural Language Processing (2014). http://www.cs.colorado.edu/*jbg/docs/2014_emnlp_qb_rnn.pdf

7. Iyyer, M., Enns, P., Boyd-Graber, J., Resnik, P.: Political ideology detection usingrecursive neural networks. In: Proceedings of the Association for ComputationalLinguistics (2014). http://www.cs.colorado.edu/*jbg/docs/ 2014_acl_rnn_ideology.pdf

8. Iyyer, M., Manjunatha, V., Boyd-Graber, J., Daumé III, H.: Deep unorderedcomposition rivals syntactic methods for text classification. In: Association forComputational Linguistics (2015). http://www.cs.colorado.edu/*jbg/docs/2015_acl_dan.pdf

9. Nguyen, V.A., Boyd-Graber, J., Resnik, P.: SITS: A hierarchical non-parametricmodel using speaker identity for topic segmentation in multiparty conversations.In: Proceedings of the Association for Computational Linguistics (2012). http://www.cs.colorado.edu/*jbg/docs/acl_2012_sitspdf

10. Nguyen, V.A., Boyd-Graber, J., Resnik, P.: Lexical and hierarchical topicregression. In: Proceedings of Advances in Neural Information Processing Sys-tems (2013). http://www.cs.colorado.edu/*jbg/docs/2013_shlda.pdf

11. Nguyen, V.A., Boyd-Graber, J., Resnik, P., Cai, D., Midberry, J., Wang, Y.:Modeling topic control to detect influence in conversations using nonparametrictopic models. Mach. Learn. 95, 381–421 (2014). http://www.cs.colorado.edu/*jbg/docs/mlj_2013_influencer.pdf

12. Nguyen, V.A., Boyd-Graber, J., Resnik, P., Miler, K.: Tea party in the house: ahierarchical ideal point topic model and its application to Republican legislators inthe 112th Congress. In: Association for Computational Linguistics (2015). http://www.cs.colorado.edu/*jbg/docs/2015_acl_teaparty.pdf

13. Niculae, V., Kumar, S., Boyd-Graber, J., Danescu-Niculescu-Mizil, C.: Linguisticharbingers of betrayal: a case study on an online strategy game. In: Association forComputational Linguistics (2015). http://www.cs.colorado.edu/*jbg/docs/2015_acl_diplomacy.pdf

14. Talley, E.M., Newman, D., Mimno, D., Herr, B.W.,Wallach, H.M., Burns, G.A.P.C.,Leenders, A.G.M., McCallum, A.: Database of NIH grants using machine-learnedcategories and graphical clustering. Nat. Methods 8(6), 443–444 (2011)

Machine Learning Shouldn’t be a Black Box XXV

A Task-Based Perspective to InformationRetrieval

Emine Yilmaz

Deptartment of Computer Science, University College Londonemine.yilmaz@ucl.ac.uk

The need for search often arises from a persons need to achieve a goal, or a task such asbooking travels, organizing a wedding, buying a house, etc. [1]. Contemporary searchengines focus on retrieving documents relevant to the query submitted as opposed tounderstanding and supporting the underlying information needs (or tasks) that have leda person to submit the query. Therefore, search engine users often have to submitmultiple queries to the current search engines to achieve a single information need [2].For example, booking travels to a location such as London would require the user tosubmit various different queries such as flights to London, hotels in London, points ofinterest around London as all of these queries are related to possible subtasks the usermight have to perform in order to arrange their travels.

Ideally, an information retrieval (IR) system should be able to understand thereason that caused the user to submit a query and it should help the user achieve theactual task by guiding her through the steps (or subtasks) that need to be completed.Even though designing such systems that can characterize/identify tasks, and canrespond to them efficiently is listed as one of the grant challenges in IR [1], very littleprogress has been made in this direction [3].

Having identified that users often have to reformulate their queries in order toachieve their final goal, most current search engines attempt to assist users towards abetter expression of their needs by suggesting queries to them, other than the currentlyissued query. However, query suggestions mainly focus on helping the user refine theircurrent query, as opposed to helping them identify and explore aspects related to theircurrent complex tasks. For example, when a user issues the query “flights to Barce-lona”, it is clear that the user is planning to travel to Barcelona and it is very likely thatthe user will also need to search for hotels in Barcelona or for shuttles from Barcelonaairport. Since query suggestions mainly focuses on refining the current query, sug-gestions provided commonly used search engines are mostly of the form “flights toBarcelona from <LOCATION>”, or “<FLIGHT CARRIER NAME> flights to Bar-celona” and the result pages provided by these systems do not contain any informationthat could help users book hotels or shuttles from the airport.

For very common tasks such as arranging travels, it may be possible to manuallyidentify and guide the user through a list of (sub)tasks that needs to be achieved toachieve a certain task (booking a flight, finding a hotel, looking for points of interests,etc. when the user trying to arrange her travels). However, given the variety of taskssearch engines are used for, this would only be possible for a very small subset of them.Furthermore, quite often search engines are used to achieve such complex tasks that

often the searcher herself lacks the task knowledge necessary to decide which step totackle next [2]. For example, a searcher looking for information about how to maintaina car with no prior knowledge would first need to use the search engine to identify theparts of the car that need maintenance and issue separate queries to learn aboutmaintaining each part. Hence, retrieval systems that can automatically detect the taskthe user trying to achieve and guide her through the process are needed, where a searchtask has been previously defined as an atomic information need that consists of a set ofrelated (sub)tasks [2].

With the introduction of new types of devices in our everyday lives, search systemsare now being used via very different types of devices. The types of devices searchsystems are used over are becoming increasingly smaller (e.g. mobile phones, smartwatches, smart glasses etc.), which limit the types of interactions users may have withthe systems. Searching over devices with such small interfaces is not easy as it requiresmore effort to type and interact with the system. Hence, building IR systems that canreduce the interactions needed with the device is highly critical for such devices.Therefore, task based information retrieval systems will be even more valuable for suchsmall interfaces, which are increasingly being introduced/used.

Devising task based information retrieval systems have several challenges that haveto be tackled. In this talk, I will start with describing the problems that need to besolved when designing such systems, comparing and contrasting them we the tradi-tional way in building IR systems. In particular, devising such task based systemswould involve tackling several challenges, such as (1) devising methodologies foraccurately extracting and representing tasks, (2) building and designing new interfacesfor task based IR systems, (3) devising methodologies for evaluating the quality of taskbased IR systems, and (4) task based personalization of IR systems. I will talk about theinitial attempts made in tackling in these challenges, as well as the initial method-ologies we have built in order to tackle each of these challenges.

References

1. Belkin, N.: Some(what) grand challenges for IR. ACM SIGIR Forum 42(1), 47–54 (2008)2. Jones, R., Klinkner, K.L.: Beyond the session timeout: automatic hierarchical segmentation of

search topics in query logs. In: Proceedings of ACM CIKM 2008 Conference on Informationand Knowledge Management, pp. 699–708 (2008)

3. Kelly,D., Arguello, J., Capra, R.: NSF workshop on task-based information searchsystems. In:ACM SIGIR Forum, vol. 47, no. 2, December 2013

A Task-Based Perspective to Information Retrieval XXVII

Lessons Learnt at Building RecommendationServices in Industry Scale

Domonkos Tikk

Gravity R&D Zrt, Budapest, Hungarydomonkos.tikk@gravityrd.com

http://gravityrd.com

Abstract. Gravity R&D has been providing recommendation services as SaaSsolutions since 2009. Founded by top contenders in the Netflix Prize, thecompany can be considered as an offspring of the competition. In this talk it isshown how Gravity’s recommendation technology was created from the big pileof task specific program codes to scalable services that serve billions of rec-ommendation requests monthly. Having academic origin with strong researchfocus, the recommendation quality has always been the primary differentiatingfactor at Gravity. But we also learnt that machine learning competitions aredifferent from scalable and robust services. We discuss some lessons learnt onthis road to create a solution that can equally encompass complex algorithms,yet fast and scalable.

Keywords: Recommender systems • Scalability • Real-time • Matrixfactorization • Context-aware recommenders • Neighbor based models

Gravity R&D experienced many challenges while scaling up their services. The sheerquantity of data handled on a daily basis increased exponentially. This presentation willcover how overcoming these challenges permanently shaped our algorithms and systemarchitecture used to generate these recommendations. Serving personalized recom-mendations requires real-time computation and data access for every single request. Togenerate responses in real-time, current user inputs have to be compared against theirhistory in order to deliver accurate recommendations.

We then combine this user information with specific details about available items asthe next step in the recommendation process. It becomes more difficult to provideaccurate recommendations as the number of transactions and items increase. It alsobecomes difficult because this type of analysis requires the combination of multiplecomplex algorithms that all may require heterogeneous inputs.

Initially, the architecture was designed for matrix factorization based models [4]and serving huge numbers of requests but with a limited number of items. Now,Gravity is using MF, neighborhood based models [5], context-aware recommenders[2, 3] and metadata based models to generate recommendations for millions of itemswithin their databases, and now Gravity is experimenting with applying deep learningtechnology for recommendations [1]. This required a shift from a monolithic archi-tecture with in-process caching to a more service oriented architecture with multi-layercaching. As a result of an increase in the number of components and number of clients,managing the infrastructure can be quite difficult.

Even with these challenges, we do not believe that it is worthwhile to use a fullydistributed system. It adds unneeded complexity, resources, and overhead to the sys-tem. We prefer an approach of firstly optimizing current algorithms and architectureand only moving to a distributed system when no other options are left.

References

1. Hidasi, B., Karatzoglou, A., Baltrunas, L., Tikk, D.: Session-based recommendations withrecurrent neural networks. CoRR (Arxiv) abs/1511.06939 (2015). http://arxiv.org/abs/1511.06939

2. Hidasi, B., Tikk, D.: Fast ALS-based tensor factorization for context-aware recommendationfrom implicit feedback. In: Flach, P., et al. (eds.) ECML PKDD 2012. LNCS vol. 7524,pp. 67–82. Springer, Berlin (2012)

3. Hidasi, B., Tikk, D.: General factorization framework for context-aware recommendations.Data Mining and Knowledge Discovery, pp. 1–30 (2015). http://dx.doi.org/10.1007/s10618-015-0417-y

4. Takács, G., Pilászy, I., Németh, B., Tikk, D.: Scalable collaborative filtering approaches forlarge recommender systems. J. Mach. Learn. Res. 10, 623–656 (2009)

5. Takács, G., Pilászy, I., Németh, B., Tikk, D.: Matrix factorization and neighbor basedalgorithms for the Netflix Prize problem. In: 2nd ACM Conference on RecommendationSystems, pp. 267–274. Lausanne, Switzerland, 21–24 October 2008

Lessons Learnt at Building Recommendation Services in Industry Scale XXIX

Contents

Social Context and News

SoRTESum: A Social Context Framework for Single-DocumentSummarization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Minh-Tien Nguyen and Minh-Le Nguyen

A Graph-Based Approach to Topic Clustering for Online Comments toNews. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Ahmet Aker, Emina Kurtic, A.R. Balamurali, Monica Paramita,Emma Barker, Mark Hepple, and Rob Gaizauskas

Leveraging Semantic Annotations to Link Wikipedia and News Archives . . . 30Arunav Mishra and Klaus Berberich

Machine Learning

Deep Learning over Multi-field Categorical Data – A Case Study on UserResponse Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

Weinan Zhang, Tianming Du, and Jun Wang

Supervised Local Contexts Aggregation for Effective Session Search. . . . . . . 58Zhiwei Zhang, Jingang Wang, Tao Wu, Pengjie Ren, Zhumin Chen,and Luo Si

An Empirical Study of Skip-Gram Features and Regularization for Learningon Sentiment Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

Cheng Li, Bingyu Wang, Virgil Pavlu, and Javed A. Aslam

Multi-task Representation Learning for Demographic Prediction . . . . . . . . . . 88Pengfei Wang, Jiafeng Guo, Yanyan Lan, Jun Xu, and Xueqi Cheng

Large-Scale Kernel-Based Language Learning Through the EnsembleNystr€om Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

Danilo Croce and Roberto Basili

Question Answering

Beyond Factoid QA: Effective Methods for Non-factoid Answer SentenceRetrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

Liu Yang, Qingyao Ai, Damiano Spina, Ruey-Cheng Chen, Liang Pang,W. Bruce Croft, Jiafeng Guo, and Falk Scholer

Supporting Human Answers for Advice-Seeking Questions in CQA Sites . . . 129Liora Braunstain, Oren Kurland, David Carmel, Idan Szpektor,and Anna Shtok

Ranking

Does Selective Search Benefit from WAND Optimization? . . . . . . . . . . . . . 145Yubin Kim, Jamie Callan, J. Shane Culpepper, and Alistair Moffat

Efficient AUC Optimization for Information Ranking Applications . . . . . . . . 159Sean J. Welleck

Modeling User Interests for Zero-Query Ranking . . . . . . . . . . . . . . . . . . . . 171Liu Yang, Qi Guo, Yang Song, Sha Meng, Milad Shokouhi,Kieran McDonald, and W. Bruce Croft

Evaluation Methodology

Adaptive Effort for Search Evaluation Metrics . . . . . . . . . . . . . . . . . . . . . . 187Jiepu Jiang and James Allan

Evaluating Memory Efficiency and Robustness of Word Embeddings . . . . . . 200Johannes Jurgovsky, Michael Granitzer, and Christin Seifert

Characterizing Relevance on Mobile and Desktop . . . . . . . . . . . . . . . . . . . . 212Manisha Verma and Emine Yilmaz

Probabilistic Modelling

Probabilistic Local Expert Retrieval. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227Wen Li, Carsten Eickhoff, and Arjen P. de Vries

Probabilistic Topic Modelling with Semantic Graph . . . . . . . . . . . . . . . . . . 240Long Chen, Joemon M. Jose, Haitao Yu, Fajie Yuan, and Huaizhi Zhang

Estimating Probability Density of Content Types for Promoting MedicalRecords Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

Yun He, Qinmin Hu, Yang Song, and Liang He

Evaluation Issues

The Curious Incidence of Bias Corrections in the Pool . . . . . . . . . . . . . . . . 267Aldo Lipani, Mihai Lupu, and Allan Hanbury

Understandability Biased Evaluation for Information Retrieval . . . . . . . . . . . 280Guido Zuccon

XXXII Contents

The Relationship Between User Perception and User Behaviour inInteractive Information Retrieval Evaluation . . . . . . . . . . . . . . . . . . . . . . . . 293

Mengdie Zhuang, Elaine G. Toms, and Gianluca Demartini

Multimedia

Using Query Performance Predictors to Improve Spoken Queries . . . . . . . . . 309Jaime Arguello, Sandeep Avula, and Fernando Diaz

Fusing Web and Audio Predictors to Localize the Origin of Music Piecesfor Geospatial Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322

Markus Schedl and Fang Zhou

Key Estimation in Electronic Dance Music. . . . . . . . . . . . . . . . . . . . . . . . . 335Ángel Faraldo, Emilia Gómez, Sergi Jordà, and Perfecto Herrera

Summarization

Evaluating Text Summarization Systems with a Fair Baseline from MultipleReference Summaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351

Fahmida Hamid, David Haraburda, and Paul Tarau

Multi-document Summarization Based on Atomic Semantic Eventsand Their Temporal Relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366

Yllias Chali and Mohsin Uddin

Tweet Stream Summarization for Online Reputation Management . . . . . . . . . 378Jorge Carrillo-de-Albornoz, Enrique Amigó, Laura Plaza,and Julio Gonzalo

Reproducibility

Who Wrote the Web? Revisiting Influential Author Identification ResearchApplicable to Information Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393

Martin Potthast, Sarah Braun, Tolga Buz, Fabian Duffhauss,Florian Friedrich, Jörg Marvin Gülzow, Jakob Köhler,Winfried Lötzsch, Fabian Müller, Maike Elisa Müller, Robert Paßmann,Bernhard Reinke, Lucas Rettenmeier, Thomas Rometsch, Timo Sommer,Michael Träger, Sebastian Wilhelm, Benno Stein, Efstathios Stamatatos,and Matthias Hagen

Toward Reproducible Baselines: The Open-Source IR ReproducibilityChallenge. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408

Jimmy Lin, Matt Crane, Andrew Trotman, Jamie Callan,Ishan Chattopadhyaya, John Foley, Grant Ingersoll, Craig Macdonald,and Sebastiano Vigna

Contents XXXIII

Experiments in Newswire Summarisation . . . . . . . . . . . . . . . . . . . . . . . . . . 421Stuart Mackie, Richard McCreadie, Craig Macdonald, and Iadh Ounis

On the Reproducibility of the TAGME Entity Linking System . . . . . . . . . . . 436Faegheh Hasibi, Krisztian Balog, and Svein Erik Bratsberg

Twitter

Correlation Analysis of Reader’s Demographics and Tweet CredibilityPerception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453

Shafiza Mohd Shariff, Mark Sanderson, and Xiuzhen Zhang

Topic-Specific Stylistic Variations for Opinion Retrieval on Twitter . . . . . . . 466Anastasia Giachanou, Morgan Harvey, and Fabio Crestani

Inferring Implicit Topical Interests on Twitter . . . . . . . . . . . . . . . . . . . . . . . 479Fattane Zarrinkalam, Hossein Fani, Ebrahim Bagheri,and Mohsen Kahani

Topics in Tweets: A User Study of Topic Coherence Metrics for TwitterData . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492

Anjie Fang, Craig Macdonald, Iadh Ounis, and Philip Habel

Retrieval Models

Supporting Scholarly Search with Keyqueries . . . . . . . . . . . . . . . . . . . . . . . 507Matthias Hagen, Anna Beyer, Tim Gollub, Kristof Komlossy,and Benno Stein

Pseudo-Query Reformulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 521Fernando Diaz

VODUM: A Topic Model Unifying Viewpoint, Topic and OpinionDiscovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533

Thibaut Thonet, Guillaume Cabanac, Mohand Boughanem,and Karen Pinel-Sauvagnat

Applications

Harvesting Training Images for Fine-Grained Object Categories UsingVisual Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 549

Josiah Wang, Katja Markert, and Mark Everingham

Do Your Social Profiles Reveal What Languages You Speak? LanguageInference from Social Media Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 561

Yu Xu, M. Rami Ghorab, Zhongqing Wang, Dong Zhou,and Séamus Lawless

XXXIV Contents

Retrieving Hierarchical Syllabus Items for Exam Question Analysis . . . . . . . 575John Foley and James Allan

Collaborative Filtering

Implicit Look-Alike Modelling in Display Ads – Transfer CollaborativeFiltering to CTR Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 589

Weinan Zhang, Lingxi Chen, and Jun Wang

Efficient Pseudo-Relevance Feedback Methods for Collaborative FilteringRecommendation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 602

Daniel Valcarce, Javier Parapar, and Álvaro Barreiro

Language Models for Collaborative Filtering Neighbourhoods . . . . . . . . . . . 614Daniel Valcarce, Javier Parapar, and Álvaro Barreiro

Adaptive Collaborative Filtering with Extended Kalman Filtersand Multi-armed Bandits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 626

Jean-Michel Renders

Short Papers

A Business Zone Recommender System Based on Facebook and UrbanPlanning Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 641

Jovian Lin, Richard J. Oentaryo, Ee-Peng Lim, Casey Vu, Adrian Vu,Agus T. Kwee, and Philips K. Prasetyo

On the Evaluation of Tweet Timeline Generation Task . . . . . . . . . . . . . . . . 648Walid Magdy, Tamer Elsayed, and Maram Hasanain

Finding Relevant Relations in Relevant Documents . . . . . . . . . . . . . . . . . . . 654Michael Schuhmacher, Benjamin Roth, Simone Paolo Ponzetto,and Laura Dietz

Probabilistic Multileave Gradient Descent . . . . . . . . . . . . . . . . . . . . . . . . . 661Harrie Oosterhuis, Anne Schuth, and Maarten de Rijke

Real-World Expertise Retrieval: The Information Seeking Behaviourof Recruitment Professionals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 669

Tony Russell-Rose and Jon Chamberlain

Compressing and Decoding Term Statistics Time Series. . . . . . . . . . . . . . . . 675Jinfeng Rao, Xing Niu, and Jimmy Lin

Contents XXXV

Feedback or Research: Separating Pre-purchase from Post-purchaseConsumer Reviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 682

Mehedi Hasan, Alexander Kotov, Aravind Mohan, Shiyong Lu,and Paul M. Stieg

Inferring the Socioeconomic Status of Social Media Users Based onBehaviour and Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 689

Vasileios Lampos, Nikolaos Aletras, Jens K. Geyti, Bin Zou,and Ingemar J. Cox

Two Scrolls or One Click: A Cost Model for Browsing Search Results . . . . . 696Leif Azzopardi and Guido Zuccon

Determining the Optimal Session Interval for Transaction Log Analysisof an Online Library Catalogue. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 703

Simon Wakeling and Paul Clough

A Comparison of Deep Learning Based Query Expansionwith Pseudo-Relevance Feedback and Mutual Information . . . . . . . . . . . . . . 709

Mohannad ALMasri, Catherine Berrut, and Jean-Pierre Chevallet

A Full-Text Learning to Rank Dataset for Medical Information Retrieval . . . . 716Vera Boteva, Demian Gholipour, Artem Sokolov, and Stefan Riezler

Multi-label, Multi-class Classification Using Polylingual Embeddings . . . . . . 723Georgios Balikas and Massih-Reza Amini

Learning Word Embeddings from Wikipedia for Content-BasedRecommender Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 729

Cataldo Musto, Giovanni Semeraro, Marco de Gemmis,and Pasquale Lops

Tracking Interactions Across Business News, Social Media, and StockFluctuations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 735

Ossi Karkulahti, Lidia Pivovarova, Mian Du, Jussi Kangasharju,and Roman Yangarber

Subtopic Mining Based on Three-Level Hierarchical Search Intentions . . . . . 741Se-Jong Kim, Jaehun Shin, and Jong-Hyeok Lee

Cold Start Cumulative Citation Recommendation for Knowledge BaseAcceleration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 748

Jingang Wang, Jingtian Jiang, Lejian Liao, Dandan Song,Zhiwei Zhang, and Chin-Yew Lin

Cross Domain User Engagement Evaluation . . . . . . . . . . . . . . . . . . . . . . . . 754Ali Montazeralghaem, Hamed Zamani, and Azadeh Shakery

XXXVI Contents

An Empirical Comparison of Term Association and Knowledge Graphsfor Query Expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 761

Saeid Balaneshinkordan and Alexander Kotov

Deep Learning to Predict Patient Future Diseases from the ElectronicHealth Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 768

Riccardo Miotto, Li Li, and Joel T. Dudley

Improving Document Ranking for Long Queries with Nested QuerySegmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 775

Rishiraj Saha Roy, Anusha Suresh, Niloy Ganguly,and Monojit Choudhury

Sketching Techniques for Very Large Matrix Factorization. . . . . . . . . . . . . . 782Raghavendran Balu, Teddy Furon, and Laurent Amsaleg

Diversifying Search Results Using Time: An Information Retrieval Methodfor Historians . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 789

Dhruv Gupta and Klaus Berberich

On Cross-Script Information Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . 796Nada Naji and James Allan

LExL: A Learning Approach for Local Expert Discovery on Twitter . . . . . . . 803Wei Niu, Zhijiao Liu, and James Caverlee

Clickbait Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 810Martin Potthast, Sebastian Köpsel, Benno Stein, and Matthias Hagen

Informativeness for Adhoc IR Evaluation: A Measure that PreventsAssessing Individual Documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 818

Romain Deveaud, Véronique Moriceau, Josiane Mothe,and Eric SanJuan

What Multimedia Sentiment Analysis Says About City Liveability . . . . . . . . 824Joost Boonzajer Flaes, Stevan Rudinac, and Marcel Worring

Demos

Scenemash: Multimodal Route Summarization for City Exploration. . . . . . . . 833Jorrit van den Berg, Stevan Rudinac, and Marcel Worring

Exactus Like: Plagiarism Detection in Scientific Texts . . . . . . . . . . . . . . . . . 837Ilya Sochenkov, Denis Zubarev, Ilya Tikhomirov, Ivan Smirnov,Artem Shelmanov, Roman Suvorov, and Gennady Osipov

Jitter Search: A News-Based Real-Time Twitter Search Interface . . . . . . . . . 841Flávio Martins, João Magalhães, and Jamie Callan

Contents XXXVII

TimeMachine: Entity-Centric Search and Visualization of News Archives . . . 845Pedro Saleiro, Jorge Teixeira, Carlos Soares, and Eugénio Oliveira

OPMES: A Similarity Search Engine for Mathematical Content . . . . . . . . . . 849Wei Zhong and Hui Fang

SHAMUS: UFAL Search and Hyperlinking Multimedia System . . . . . . . . . . 853Petra Galuščáková, Shadi Saleh, and Pavel Pecina

Industry Day

Industry Day Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 859Omar Alonso and Pavel Serdyukov

Workshops

Bibliometric-Enhanced Information Retrieval: 3rd International BIRWorkshop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 865

Philipp Mayr, Ingo Frommholz, and Guillaume Cabanac

MultiLingMine 2016: Modeling, Learning and Mining forCross/Multilinguality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 869

Dino Ienco, Mathieu Roche, Salvatore Romeo, Paolo Rosso,and Andrea Tagarelli

Proactive Information Retrieval: Anticipating Users’ Information Need . . . . . 874Sumit Bhatia, Debapriyo Majumdar, and Nitish Aggarwal

First International Workshop on Recent Trends in News InformationRetrieval (NewsIR’16) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 878

Miguel Martinez-Alvarez, Udo Kruschwitz, Gabriella Kazai,Frank Hopfgartner, David Corney, Ricardo Campos,and Dyaa Albakour

Tutorials

Collaborative Information Retrieval: Concepts, Models and Evaluation . . . . . 885Lynda Tamine and Laure Soulier

Group Recommender Systems: State of the Art, Emerging Aspectsand Techniques, and Research Challenges . . . . . . . . . . . . . . . . . . . . . . . . . 889

Ludovico Boratto

Living Labs for Online Evaluation: From Theory to Practice . . . . . . . . . . . . 893Anne Schuth and Krisztian Balog

XXXVIII Contents

Real-Time Bidding Based Display Advertising: Mechanismsand Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 897

Jun Wang, Shuai Yuan, and Weinan Zhang

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 903

Contents XXXIX