![Page 1: Presentation at joint PIA workshop at UMAP 2014](https://reader036.vdocument.in/reader036/viewer/2022081513/55626c87d8b42a14048b5619/html5/thumbnails/1.jpg)
Does Personalisation Benefit Everyone in the Same Way?
M. Rami GhorabPostdoc, School of Computer Science & Statistics,
Trinity College Dublin
![Page 2: Presentation at joint PIA workshop at UMAP 2014](https://reader036.vdocument.in/reader036/viewer/2022081513/55626c87d8b42a14048b5619/html5/thumbnails/2.jpg)
Today’s Web
Monolingual & MultilingualUsers
Searching acrossMultilingual Content
• Diverse linguistic backgrounds
• Different language capabilities
• Different language preferences
We want to personalise search, given these characteristics
• Various languages.
• Relevant content – which lang?
![Page 3: Presentation at joint PIA workshop at UMAP 2014](https://reader036.vdocument.in/reader036/viewer/2022081513/55626c87d8b42a14048b5619/html5/thumbnails/3.jpg)
• User Modelling– Search interests (keywords) that span across multiple languages.– Grouped into language fragments.
• Adapting Results in Multilingual Web Search– Merging and Re-ranking the results.– Translating where necessary.
Extending Personalisationinto the Multilingual Dimension
Personalised Multilingual Information Retrieval (PMIR)
![Page 4: Presentation at joint PIA workshop at UMAP 2014](https://reader036.vdocument.in/reader036/viewer/2022081513/55626c87d8b42a14048b5619/html5/thumbnails/4.jpg)
User Modelling
Native Language
Familiar Languages
Preferred Language
Attributes
Structure
![Page 5: Presentation at joint PIA workshop at UMAP 2014](https://reader036.vdocument.in/reader036/viewer/2022081513/55626c87d8b42a14048b5619/html5/thumbnails/5.jpg)
Result Lists(English, French, German)
Ranked separately
against keywords
in User Model fragment
(textual similarity)
Re-ranked Result Lists
(English, French, German)
Merged & Translated List
![Page 6: Presentation at joint PIA workshop at UMAP 2014](https://reader036.vdocument.in/reader036/viewer/2022081513/55626c87d8b42a14048b5619/html5/thumbnails/6.jpg)
Research Question - Revisited
Would multilingual search personalisation algorithms
achieve the same degree of improvements
for all search queries, regardless of query language?
![Page 7: Presentation at joint PIA workshop at UMAP 2014](https://reader036.vdocument.in/reader036/viewer/2022081513/55626c87d8b42a14048b5619/html5/thumbnails/7.jpg)
• Evaluate the retrieval effectiveness of the multilingual search personalisation algorithms (User Modelling and Result Adaptation).
• Determine whether the algorithms achieve the same degree of effectiveness for users who have different language preferences (examine English vs. Non-English users).
Experiment - Objectives
![Page 8: Presentation at joint PIA workshop at UMAP 2014](https://reader036.vdocument.in/reader036/viewer/2022081513/55626c87d8b42a14048b5619/html5/thumbnails/8.jpg)
Experiment - Setup
Phase 2: Result Pooling
• Last query reserved for testing.
• Construct the user models.
• Generate various result lists.Phase 3: Relevance Judgments
• 4-point scale of relevance
(not relevant / somewhat relevant / relevant / very relevant)
Phase 4: Evaluation
• Metric: Mean Average Precision (MAP).
• Measures effectiveness of each algorithm across all test queries
Phase 1: User Participation
• Sign up – language preferences.
• Two search topics.
• Use baseline multilingual Web search.
• Submit findings about topic.
![Page 9: Presentation at joint PIA workshop at UMAP 2014](https://reader036.vdocument.in/reader036/viewer/2022081513/55626c87d8b42a14048b5619/html5/thumbnails/9.jpg)
Experiment - Results
MAP Improvements over Baselinefor various result list positions (cut-off points @5..@20)
![Page 10: Presentation at joint PIA workshop at UMAP 2014](https://reader036.vdocument.in/reader036/viewer/2022081513/55626c87d8b42a14048b5619/html5/thumbnails/10.jpg)
Understanding the Results
List Position
EnglishNon-
English%
English over Non-English
P@5 0.58 0.45 29.15%
P@10 0.55 0.49 11.54%
P@15 0.51 0.45 14.46%
P@20 0.50 0.48 3.71%
Baseline (non-personalised) Precision Scores
![Page 11: Presentation at joint PIA workshop at UMAP 2014](https://reader036.vdocument.in/reader036/viewer/2022081513/55626c87d8b42a14048b5619/html5/thumbnails/11.jpg)
• Does personalisation benefit everyone in the same way?– No.– Multilingual search adaptation algorithms work differently with users of
different language preferences/capabilities.
• Recommendation– Personalised Search systems should adopt different personalisation
strategies for certain languages or groups of languages.
• Future Work– Concept-based user models (multilingual ontology or web taxonomy).
Conclusion & Future Work
![Page 12: Presentation at joint PIA workshop at UMAP 2014](https://reader036.vdocument.in/reader036/viewer/2022081513/55626c87d8b42a14048b5619/html5/thumbnails/12.jpg)
Thank You
This research is supported bythe Science Foundation Ireland (Grant 12/CE/I2267)
as part of the Centre for Next Generation Localisation (www.cngl.ie) at Trinity College, Dublin.