zoe's presentation
Post on 22-Nov-2014
296 Views
Preview:
DESCRIPTION
TRANSCRIPT
Understanding User Goals in Web Search
Daniel E. RoseDanny Levinson
Not only what, but also why
● previous work on information-seeking behavior
● authors' hierarchy of search goals
● how to classify queries
● analysis of results
● applicability of model
LIS precedents● Bates
○ "Information Search Tactics" (1979)
○ "The Design of Browsing and Berrypicking Techniques for the Online Search Interface" (1989)
Diagrams are recreations of figures that appear in "The Design of Browsing and Berrypicking Techniques for the Online Search Interface."
Above: Bates' berrypicking model Left: the traditional model of document recovery
● Belkin, Oddy, and Brooks ○ "ASK for Information Retrieval" (1982)
■ from "Part I. Background and Theory"
■ from "Part II. Results of a Design Study"
● Broder○ "A Taxonomy of Web Search" (2002)
Process
● developed flat list of user goals
● used list to classify queries in test set
● revised categories as necessary--developing hierarchical classification
● manually classified three sets of queries
○ Each set of approximately 500 queries was randomly selected from AltaVista query logs on a different day, and at a different time of the day, than the other two.
Search goal hierarchy
Search goal hierarchy (continued)
Manual query classification
"We need to know the relative prevalence of various goals. And if we hope to infer goals automatically in the future, we need to know that it is possible to do so manually" (16).
Objects of consideration:● ��the query itself● results returned by the
search engine● results clicked on by the
user● further searches or other
actions by the userDo these documents provide "sufficient information for a human to consistently classify queries according to our goal framework" (16)?
"Once we could successfully classify queries manually, we would be able to provide training data for a future automatic classification system" (16).
Final Fantasy example
What kind of a site did the user who searched for "final fantasy" intend to find--a site that sells a version of the game, one that lists the game's "official" rules, or one that provides less exact information about the game?
Time Delta t Event Details
36 36 result click pg 1, pos 1 http://www.ffonline.com
113 77 query pg 1 final fantasy
118 5 result click pg 1, pos 8 http://www.eyesonff.com
147 29 result click pg 1, pos 8 http://www.eyesonff.com
Examining the results returned by two search engines for this query, and the user's subsequent clicks, the authors conclude that the goal of this search was "undirected" information.
Results
Results (continued)
● "One issue is that we have no way of knowing conclusively whether the goal we inferred for a query is in fact the user’s actual goal. In the future, we would like to combine our work with user studies, including qualitative data such as diary reports of user goals" (19).
Applications and limitations
● "If our findings about the relatively small number of navigational queries are accurate, they suggest that much of the attention in the commercial search engine world may be misdirected" (18).
Questions● How does Rose and Levinson's user goals paradigm
compare to the idea of document relevance? Does a result need to meet a search goal in order for the user to perceive it as relevant, or simply convey information related to the search query?
● In what ways does their user goals model take into account query refinement and/or query expansion?
● What is the user's goal in the Aloha/American Airlines scenario (outlined on slides 8 and 9)?
● How would you apply Rose and Levinson's research to the following questions:
○ Why does a search occur?○ What is its purpose?○ How does it proceed?
top related