exploratory computing: designing discovery-driven user experiences

Politecnico di MilanoDepartment of Electronics, Information and

Bioengineering

Exploratory Computing: designing enginesfor discovery-driven user experiences

Luigi [email protected]

December 10, 2012

L. Spagnolo Exploratory Computing 1 / 39

mailto:[email protected]

Outline

Information exploration

The proposed model

The interface

Conclusions

Publications


Information exploration | A different experience

Beyond information seeking: making sense, understanding,investigating.• Example: digital library of scientific publications

Retrieval taskFinding recent, significant publications on a certain research topicand/or by certain authors

Exploration

• (E.g. scholar) acquiring knowledge about a research domain orspecific community: approaches, terminology, main sources ofcontribution, communication style, etc.

• (E.g. university reviewers): Evaluating the contribution of aperson/research group (citations, co-authors, metrics, etc.)


Information exploration | Traditional approaches

• Hypertext navigationI Based on well-engineered information architectures

(hierarchies of hyperlinked pages)I Static taxonomies fail to scale to large bodies of informationI Interesting content buried under levels of navigation and

information overload• Keyword-based search

I Arranges content items dynamically according to user needs(expressed by a query)

I Not very suitable for ill-defined knowledge, non-retrieval goalsI The user is required to know the terminology

• Emerging patterns combine search and browsing →exploratory search

I Faceted navigation: iterative filtering according to multiplecriteria (facets). Becoming a de facto standard.

I Limit: focus is still mainly on object seekingI A more comprehensive approach to the problem is needed


Information exploration | My proposal

FELIS: Faceted Exploration for Large Information SpacesA comprehensive, general framework for exploratory userexperiences, covering:• Knowledge representation according to multiple itemproperties (facets) and querying

• Widgets and interaction patterns for (semantic) facetedexplorations

• information visualization strategies to highlight thecorrelation between item features, supported by explorationmetrics to estimate their relevance


Outline


The proposed model

The interface

Conclusions

Publications


The proposed model | How the exploration works

• Over a structured information baseI Represented according to a more or less complex semanticsI Basic faceted classification: categories grouped by propertyI Semantic faceted classification: also the properties of

related items are used (e.g. browsing artworks by artist’scountry)

• The user iteratively defines the scope of exploration within thewhole space

I The query specifies the features that information items shouldor should not possess

• The user gets the set of items matching the query plusfeedback information about the properties shared by suchresults

• Shared features are also used to refine the query restricting(zoom-in) or enlarging (zoom-out) the scope of theexploration.


The proposed model | FelisQL (1)

FelisQL: knowledge modelling and query language• An extension to LISQL (Prof. Ferré, University of Rennes I)• Item representation and querying, definition of facetedtaxonomies, syntactic query transformations

• Fully translatable into standard RDF and SPARQLWhy yet another language?!• user-centric, high level representation of concepts (forboth item description and querying) in a concise and naturalway (e.g. avoiding unnecessary variables)

• easier graphical representation into interface elements• Understandable by domain experts (designers and powerusers) with lack of expertise in programming

I Easier configuration and fast prototyping of exploratoryapplications (future work: computer-aided design tools)



Examples of representable queries:• Publications authored by foreign PhD students at Politecnicodi Milano, with the support of a professor from a foreignuniversitya publication and autho r :

[ a ’PhD student’ and coun t r y : not Italy ]at a f f i l i a t i o n : ’Politecnico di Milano’ ,

[ a professor ]at a f f i l i a t i o n : coun t r y : not Italy

• Italian painters authors of landscapes in XIX or XX centurya painter

and coun t r y : ’Italy’and autho r of

[ s u b j e c t : [ a landscape ]and p e r i o d :

"XIX century" or "XX century" ]



Example of description:’Monna Lisa’ [ a painting

and s u b j e c t : [ a woman and h a i r : c o l o r : "black" ] ,[ a landscape ]

and autho r : ’Leonardo Da Vinci’ [a painter and an engineer and a scientistand n a t i o n a l i t y : "Italian"and b i r t h : [

p l a c e : Vinci [ a city andpart of ’Province of Florence’ ]

and yea r : 1452 ]and death : [

p l a c e : Amboise [ a city andpart of ’Indre et Loire’ ]

and yea r : 1519 ]] . . .

]



Query transformations: syntactic operations (supported out byhyperlinks in the interface) allowing to move from a stage toanother of the exploration, e.g.:• Focus change: selecting the specific sub-expression (focus) ofthe query at which appending a new restrictiona painting and artist : [a woman]〈focus a woman〉

• And-insertion: adding a new restriction in conjunction at theselected focusa painting and artist : [a woman]〈and country : ‘UK’〉

• Or-insertion adding a new restriction in disjunction at theselected focusa painting and artist : [a woman and country : ‘UK’]〈or country : ‘USA’〉

• Exclusion: selecting items that do not have certain featuresa painting and artist : [a woman and country : ‘UK’ or ‘USA’]〈exclude subject : [a portrait]〉


The proposed model | Relevance measures (1)

Providing feedback about the interestingness of a item feature Cw.r.t. the whole information space Ω or the specific context ofexploration represented by the query qNotation: | C′Ω | is the number of items in Ω matching C′.

• Relative count: most common measure

µcount(C, q) =∣∣∣q〈and C〉

Ω

∣∣∣• Frequency: the percentage of items that match the query qand also the feature C

µfreq(C, q) =µcount(C, q)

|qΩ|

I Corresponds to the confidence of the association rule q → C,i.e. conf(q → C)



• Recall: the percentage of items that match the query qamong those matching the feature C

µrecall(C, q) =µcount(C, q)∣∣Cq

Ω

∣∣I Corresponds to conf(C → q)I Measures peculiarity, i.e. how much C is typical in the context

of q (and less likely to be found elsewhere)• Highlighting significant correlations between a feature Cand the query q

I estimating how much both frequency and recall diverge fromtheir expected values if C and q where statistically independent



Civilization orperiods related toarchaeologicalvenues in Italy.• Font size:frequency/countof terms

• Bar charts:frequency andrecall When nofilter is set, therecall is 100%.

Demo available at: http:

//hoc12.elet.polimi.

it/metrics/archeo/


http://hoc12.elet.polimi.it/metrics/archeo/




Civilization orperiods related toarchaeologicalvenues in NorthernItaly.• Coloured tags:significantpostive/negativefrequency (textcolor) or recall(background)

• Bar charts: Detailand deviancefrom expected.



Civilization orperiods related tomuseums inSouthern Italy.



Joint relevance measure: measuring how much the features Ca

and Cc are correlated in the context of the query q• E.g. Lift:

νlift(Ca, Cb, q) =

∣∣∣q〈and Ca〉〈and Cb〉

Ω

∣∣∣∣∣∣q〈and Ca〉

Ω

∣∣∣ · ∣∣∣q〈and Cb〉

Ω

∣∣∣



Correlations betweenconcepts representingCivilization orperiods related toarchaeologicalvenues in CentralItaly.• Node: count(size) andsignificantfrequency (color)

• Edge size andcolour: Liftmeasure betweenpairs of concepts



Correlations betweenconcepts representingCivilization orperiods related toarchaeologicalvenues in CentralItaly.• Selection of anode (filteringpreview)


The proposed model | Difference metrics (1)

Providing feedback about how much the relevance measure of aitem feature C changes when moving from a new stage of thetexploration, e.g. from query qn−1 to query qn• Difference metrics can be applied to the result of any querytrasformation

• providing additional clues of possible correlations to beinvestigated

• Absolute frequency difference:

∆freq(C, qn) = µfreq(C, qn)− µfreq(C, qn−1)

• Relative frequency difference:

δfreq(C, qn) =∆freq(C, qn)

µfreq(C, qn)



Bipolar bar chart• showing frequency differences for types of archaeological venue• previous query:location : ‘Southern Italy’ or ‘Insular Italy’

• current query: location : ‘Central Italy’



Difference bar chart• showing frequencydifferences forperiods andcivilizationsassociated toarchaeologicalvenues

• previous query:location :

‘Southern Italy’ or‘Insular Italy’

• current query:location :

‘Central Italy’


Outline


The proposed model

The interface

Conclusions

Publications


The interface | Main elements (1)

• Facet widgetsI Represent a specific faceting property ( mono-dimensional)I Allow to apply restrictions over property values (one or more

query transformations are implemented)I Show the distribution of property values (according to one

ore more relevance metrics)I Possible visualizations depend on: type of property (functional

vs. multivalued) and communicative vs. analytical purposes• Canvases

I Provide preview and access to information itemsI Possibly provide aggregate view according to two or more facet

properties (multidimensional) at a given level of granularity• (Interactive) query representation

I Shows the current selection of filters (can be embedded infacet widgets)

I Allows for focus change in semantic faceted exploration


The interface | Main elements (2)

Demo: PoliculturaPortal www.policulturaportal.it: allowsexploring 600+ interactive narratives produced within thePolicultura context for schools.


www.policulturaportal.it

The interface | Canvases (1)

Mosaic canvas for PoliculturaPortal:shapes represent a fixed dimension (school level), colours a secondselectable property (e.g. year)



Mosaic canvas for PoliculturaPortal:items that are no more relevant for the current query disappear.



Pie-chart map for PoliculturaPortal:a selected property (e.g. year) is show at different geographic levelsof granularity (Region, Province)



Pie-chart map for PoliculturaPortal:index providing access to items in specific selected area


The interface | Semantic exploration (1)

Concept of semantic faceted exploration for artworks.

Your exploration

Start a new exploration Exploration history Saved explorations

Main item features:

Canvas: Thumbnails 1 2 3 4 5

subject:

a child a drawing room a flower

a garden a girl a landscape

a person a portrait a self-portrait a theatre a woman

period:

Show by: Decade1801 1992

technique:

Others (show)WatercoloursPrintPastelsOil on canvasCharcoal 7

132331627

5

art movement:

268721162646Others (show)

Surrealism RealismPreraphaelitesImpressionismArt Decò

location:

Others (show)Tate Gallery, London

National Museum of Women in the Arts, Washington D.C.

Musée d'Orsay, ParisMetropolitan, New YorkArt Institute, Chicago 18

3940

28

2372

211418452399

artist:

Others (show)Tamara de Lempicka Mary CassattFrida Kahlo Elizabeth Thompson Berthe Morisot

a painting

period: orXIX Century XX Century

author: gender: female

items of type Artwork matching the query220

ArtExplorer

A

B

C

AQueryrepresentationa painting and

period : [‘XIX century’ or‘XX century’] andartist : [gender : “female”]

BFacets widgets• for the current

(main) focus

• properties ofpaintings (location,subject, artist name,technique, etc.)

CCanvas• Simple thumbnail

view of artworksmatching the query




Your exploration


Main item features:


subject:




period:


technique:


132331627

5

art movement:



location:




3940

28

2372

211418452399

artist:


a painting




ArtExplorer

A

B

C




(main) focus






Concept of semantic faceted exploration for artworks (other focus).

Your exploration


type:an Illustratora Paintera Photographera Sculptor

8202

1038

subject ofa Paintinga Photographya Drawing

1531

birth: year:


death: year:

Show by: Decade1836 alive


Features for this focus (58 items of type Artist ):

a painting



Others (show)Tamara de Lempicka Suzanne Valadon Natalia Goncharova Mary CassattLouise Bourgeois Gwen John Georgia O'Keeffe Frida Kahlo Evelyn De MorganElizabeth Thompson Elizabeth SiddalBerthe Morisot 21

11141218

98

1245

56

2336

art movement:




ArtExplorer

country:

United StatesUnited KingdomPolandMexicoFrance

Others (show)

471823596211

A

B

C

D



B and CFacets widgets• for the current focus

[gender : “female”]

• list of artists names(B), plus

• properties of artists(C): country, type,date of birth/death,etc.

DCanvas• Simple thumbnail

view of artworksmatching the query,the same asprevious one




Your exploration



8202

1038


1531

birth: year:


death: year:




a painting




11141218

98

1245

56

2336

art movement:




ArtExplorer

country:


Others (show)

471823596211

A

B

C

D










Outline


The proposed model

The interface

Conclusions

Publications


Conclusions | Possible applications (1)

• Support to knowledge acquisition and exploratorystorytelling

I “Learning by doing” to acquire knowledge about a specificdomain and its ontology → cultural heritage, edutainment

I Faceted exploratory applications help users focusing on thewhole picture → perceive the forest rather that the treesonly

I Exploratory storytelling: telling sorties with data, to informand persuade → e-journalism, e-commerce, education,advertising

• Exploratory analysis and user-driven feature miningI Investigating underlying phenomena in the information space

(correlations between features) → Support to scholar research,data warehousing (OLAP)

I Discovery arises from guided “trial-and-error” browsing, noneed to have a preliminary hypothesis

I Support to decision making


Conclusions | Possible applications (2)

• Enhanced information architectures and exploratorysearch

I Enhancing findability is not the primary goal of Felis (ratheraimed at supporting the other fundamental requirements ofexploration)

I Creating complex, semantic and full boolean queries bynavigation can however also serve for enhancing the retrieval ofspecific items


Conclusions | Results achieved

• A general framework of design and implementationstrategies for exploratory user experiences,

• User-centred design methodologies have be applied on anumber case studies

I Prototype for the Directorate-General of Antiquities, ItalianMinistry of Culture

I PoliculturaPortal (exploring PoliCultura interactive narratives)I “Twinned” applications: L4All Portal (teaching exoeriences)

and University of Lugano research projects portal• JavaScript APIs allowing for fast prototyping of applications(both client-side only and based on Solr as search server)

I Towards computer-aided tools for design and protyping (futurework)

• Concepts for advanced semantic faceted explorationI Future work: obtaining a prototype based on Sewelis

(University of Rennes 1) as search server


Outline


The proposed model

The interface

Conclusions

Publications


Publications | So far I

JournalsSpagnolo, Luigi, Davide Bolchini, Paolo Paolini, and Nicoletta DiBlas (2010). “Beyond Findability: Search-Enhanced InformationArchitecture for Content-Intensive Rich Internet Applications”. In:Journal of Information Architecture 2.1, pp. 19–36.

Conference proceedingsDi Blas, N., P. Paolini, and L. Spagnolo (2012). “PoliCulturaPortal: 17,000 Students Tell their Stories about Cultural Heritage”.In: Museums and the Web 2012.De Caro, Stefano, Nicoletta Di Blas, and Luigi Spagnolo (2010).“In SEARCH of Novel Ways to Design Large Cultural HeritageWebsites”. In: Museum and the Web 2010 Proceedings. Ed. byJ Trant and D Bearman. Archives and Museum Informatics,Toronto.


Publications | So far II

De Caro, Stefano, Nicoletta Di Blas, Paolo Paolini, andLuigi Spagnolo (2009). “Search-enhanced web informationarchitecture for findability and discovery of archeological heritage:the MiBAC-DGA case study”. In: Congresso Nazionale AICA.


Publications | In progress

• L.Spagnolo , P. Paolini, and F. Ferré“Exploratory computing: a new paradigm for accessing largebodies of information”ACM Trans. on Information Systems

• L.Spagnolo and S. Ferré “A flexible, user-centric language forsupporting faceted explorations”IEEE Trans. On Knowledge and Data Engineering

• L.Spagnolo and P. Paolini“Advanced widgets for faceted visual exploration”ACM Trans. On Computer-Human Interaction


exploratory computing: designing discovery-driven user experiences

Technology