inspectability and control in social recommenders

AND

IN

INSPECTABILITY

SOCIALRECOMMENDERS

CONTROL

ALEX BOSTANDJIEV

BART KNIJNENBURG

ALFR

ED KO

BSA

JOH

N O

’DO

NO

VAN

WEWHY SHOULD

SOCIALUSE

RECOMMENDERS?

DJ’S

VINYL?USE

WHY DO

BETTER INSPECTABILITY

INSPECTABILITY IN NORMAL RECOMMENDERS

BETTER INSPECTABILITYIN SOCIAL RECOMMENDERS

?THINGS I LIKE

VS.

RECOMMENDATIONS

THINGS I LIKE RECOMMENDATIONSFRIENDS

MAGIC

MORE CONTROL

CONTROLIN NORMAL RECOMMENDERS

MORE CONTROLIN SOCIAL RECOMMENDERS

VS.

RECOMMENDATIONS

RECOMMENDATIONS+

INTUITIVE INTERFACE

MORE CONTROL & INSPECTABILITY?MORE COMPLEXITY!

130 L. Chen, P. Pu

Step 4

Fig. 4 System showing a new set of alternatives after the user’s critiques

More specifically, after getting users’ initial preferences via the conversational dia-log, the system translates them into a SQL query and passes it to the database. If toomany matching goods exist, the Navigation by Asking function would calculate theinformation gain of possible questions and then ask appropriate questions to the shop-per for narrowing down the matching goods. After merchandise records are narroweddown to a pre-defined threshold number, the Navigation by Proposing function willshow three significantly different samples. The first sample good is the good recordclosest to the center point of all matching goods. Its selling points directly reflect thecustomer’s request. The second sample good is the record positioned farthest awayfrom the center point, and the third sample good is the one positioned farthest awayfrom the second sample. The explanation of the sample’s selling point is given like“this is twice as expensive as those because it is made of silk and the other two aremade of polyester”. While seeing the explanation, the shopper could easily excludeone of the three proposed goods with a specific reason, such as “this one is too darkfor me compared with the other two”. The system will then modify the sample pickingstrategy accordingly.

2.1.2 Adaptive place advisor (Thompson et al. 2004)

This system employs natural language dialog for giving personalized place recom-mendations. Consider the following conversation between the inquirer (i.e., the user)and the advisor (i.e., the system):

123

Critiquing-based recommenders 133

Fig. 5 The Dynamic Critiquing interface with system suggested compound critiques for users to select(McCarthy et al. 2005c)

2.2.2 MAUT-based compound critiques and visual critiquing (Zhang and Pu 2006)

However, the Dynamic-Critiquing method (including its extension) is still limited,in that it only reveals what the system can provide, but does not take into accountusers’ interest in the suggested critiques. Given this limitation, Zhang and Pu (2006)have proposed an approach with the purpose of adapting the generation of compoundcritiques to user preferences. Formally, they model each user’s preferences based onthe multi-Attribute Utility Theory (MAUT), which is a theory taking into accountof conflicting value preferences and producing a sore for each item to represent itsoverall satisfaction degree with the user preferences (Keeney and Raiffa 1976). Thus,during each recommendation cycle, according to the user’s current preferences, thetop k products with maximal MAUT values are retrieved. The ranked first item is thentaken as the top candidate, and for each of the others, its detailed value differencesfrom the top candidate will be presented as a compound critique (e.g., “same brandwith lower price, but slower CPU speed, smaller screen, smaller memory and smallerhard disk”).

Relative to Dynamic Critiquing methods, these MAUT-based compound critiqueswere proven with significantly higher recommendation quality, inferring that they can

123

144 L. Chen, P. Pu

(a) The preference-based organization interface.

(b) The user-initiated example critiquing interface.

If suggested critiques and products do not interest the user, he/she could switch to create critiques his/herself by clicking the button “Self specify criteria for ‘Better Features’.

Fig. 10 Hybrid critiquing system (version 2): the combination of preference-based organization interface(Pref-ORG) and user-initiated critiquing facility (Chen and Pu 2007b, 2010)

organization algorithm (as described in Sect. 2.2). If the user is interested in one ofthe suggested critiques, she could click “Show All” to see more products under thecritique. Among these products, the user can either choose one as her final choice, or

123

142 L. Chen, P. Pu

The product being critiqued

System-suggested compound critiques

User-initiated critiquing facility

Fig. 9 Hybrid critiquing system (version 1): the combination of system-suggested compound critiques anduser-initiated critiquing facility (Chen and Pu 2007a)

can not only obtain knowledge of the domain and easily perform critiquing via thesuggested critiques, but also have the opportunity to freely compose critiques on theirown with the self-initiated critiquing support.

3.2 Hybrid Critiquing

Two hybrid critiquing systems have been developed so far. One was the combinationof Example Critiquing facilities with Dynamic Critiquing based compound critiqueson a single screen. The second version integrated the preference-based organizationinterface (Chen and Pu 2007c) (which shows system-suggested critiques and theirassociated products on a separate page) with Example Critiquing (the EC interface isevoked only when users activate it). Series of user studies were conducted on the twoversions.

3.2.1 Version 1

Figure 9 shows a sample interface of the first type of hybrid critiquing system, whereDC compound critiques are displayed with EC facilities on the same screen (Chenand Pu 2007a). Specifically, the current recommendation is displayed at the top andfollowed by multiple suggested critiques. The self-initiated critiquing area is placedbelow, which provides functions to facilitate various types of critiquing modality

123

SIMPLECONTROL

SIMPLEINSPECTABILITY

THE POWER OF VISUALIZATION

AND MOREINSPECTABILITY

SATISFACTIONINCREASES

CONTROL

HYPOTHESIS:

BETTER

ONLINE USER EXPERIMENT

SYSTEM

Modified TasteWeights system

Facebook friends as recommenders

Music recommendations (based on “likes”)

Split up control + inspectability

PARTICIPANTS267 participants

Mechanical Turk + Craigslist

At least 5 music “likes” and overlap with at least 5 friends at least 10 recommendations

lists limited to 10 to avoid cognitive overload

Demographics similar to Facebook user population

PROCEDURESTEP 1: Log in to Facebook

System collects your music “likes”

System collects your friends’ music likes

PROCEDURESTEP 2: Control

3 conditions, between subjects

<skip>

NOTHING WEIGH ITEMS WEIGH FRIENDS

VS VS

PROCEDURESTEP 3: Inspection

2 conditions, between subjects

LIST ONLY FULL GRAPH

VS

6 CONDITIONS<skip> -->

-->

-->

<skip> -->

-->

-->

PROCEDURESTEP 4: Evaluation

For each recommendation:

Do you know this band/artist?

How do you rate this band/artist?(link to LastFM page for reference)

PROCEDURESTEP 5: Questionnaires

-understandability

-perceived control

-perceived recommendation quality

-system satisfaction

-music expertise

- trusting propensity

- familiarity with recommender systems

RESULTS

SUBJECTIVE3 items:- The recommendation

process is clear to me

- I understand how TasteWeights came up with the recommendations

- I am unsure how the recommendations were generated*

SUBJECTIVEINSPECTABILITY

full graph list only

CONTROL

SUBJECTIVE4 items:- I had limited control over

the way TasteWeights made recommen-dations*

- TasteWeights restricted me in my choice of music*

- Compared to how I normally get recommendations, TasteWeights was very limited*

- I would like to have more control over the recommendations*


SUBJECTIVE6 items:- I liked the artists/bands

recommended by the TasteWeights system

- The recommended artists/bands fitted my preference

- The recommended artists/bands were well chosen

- The recommended artists/bands were relevant

- TasteWeights recommen-ded too many bad artists/bands*

- I didn't like any of the recommended artists/bands*


SUBJECTIVE7 items:- I would recommend

TasteWeights to others

- TasteWeights is useless*

- TasteWeights makes me more aware of my choice options

- I can make better music choices with TasteWeights

- I can find better music using TasteWeights

- Using TasteWeights is a pleasant experience

- TasteWeights has no real benefit for me*


BEHAVIORTime (min:sec) taken in the inspection phase (step 3)

- Including LastFM visits

-Not including the control phase (step 2)

-Not including the evaluation phase (step 4)


BEHAVIORNumber of artists the participant claims she already knows

Why higher in the full graph condition?

- Link to friends reminds the user how she knows the artist

- Social conformance


BEHAVIORAverage rating of the 10 recommendations

-Lower when rating items than when rating friends

-Slightly higher in full graph condition


User Experience (EXP)Objective System Aspects (OSA)

Subjective System Aspects (SSA)

STRUCTURAL MODEL



+

Satisfaction with the system

(R2 = .696)0.410 (0.092)***

0.955 (0.148)***+

STRUCTURAL MODEL

+Perceived

recommendation quality

(R2 = .512)

0.770(0.094)***

+ Perceived control

(R2 = .311)0.377(0.074)***

+

+

Understandability

(R2 = .153)

Controlitem/friend vs. no control

Inspectabilityfull graph vs. list only

!2(2) = 10.70**item: 0.428 (0.207)*friend: 0.668 (0.206)**

0.459 (0.148)**



Interaction (INT)

+


(R2 = .696)0.410 (0.092)***

0.955 (0.148)***+

STRUCTURAL MODEL

Inspection time (min)(R2 = .092)

+

−

+

0.231(0.114)*

!2(2) = 10.81**item: −0.181 (0.097)1

friend: −0.389 (0.125)**

0.288 (0.091)**

Average rating(R2 = .508)

++

0.067 (0.022)**

0.323 (0.031)***

+

−

+

+

number of known recommendations

(R2 = .044)

−0.152 (0.063)*

0.249(0.049)***

0.695 (0.304)*

0.148(0.051)**

+Perceived


(R2 = .512)

0.770(0.094)***

+ Perceived control

(R2 = .311)0.377(0.074)***

+

+

Understandability

(R2 = .153)



!2(2) = 10.70**item: 0.428 (0.207)*friend: 0.668 (0.206)**

0.459 (0.148)**



Interaction (INT)

+


(R2 = .696)0.410 (0.092)***

0.955 (0.148)***+

STRUCTURAL MODEL


+

−

+

0.231(0.114)*

!2(2) = 10.81**item: −0.181 (0.097)1

friend: −0.389 (0.125)**

0.288 (0.091)**


++

0.067 (0.022)**

0.323 (0.031)***

+

−

+

+


(R2 = .044)

−0.152 (0.063)*

0.249(0.049)***

0.695 (0.304)*

0.148(0.051)**

+Perceived


(R2 = .512)

0.770(0.094)***

+ Perceived control

(R2 = .311)0.377(0.074)***

+

+

Understandability

(R2 = .153)



!2(2) = 10.70**item: 0.428 (0.207)*friend: 0.668 (0.206)**

0.459 (0.148)**

Personal Characteristics (PC)

+

−0.332 (0.088)***

0.205(0.100)*

0.375(0.094)***

+

−

Music expertise

0.257(0.124)*

+

Trusting propensity

0.166 (0.077)*

+

Familiarity with recommenders

CONCLUSION

CONCLUSIONSocial recommenders

- Give users inspectability and control

- Can be done with a simple user interface!

Inspectability:

- Graph increases understandability and perceived control

- Improves recognition of known recommendations

Control:

- Items control: higher novelty (fewer known recs)

- Friend control: higher accuracy

FUTURE WORKInspectability and control work

-Separately

-What about together?

SOCIALRECOMMENDERS

LET YOUBE A

RECOMMENDATION

DJ

THANK YOU!WWW.USABART.NL

[email protected]@USABART

CONCLUSIONSocial recommenders

- Give users inspectability and control

- Can be done with a simple user interface!

Inspectability:

- Increases understandability and perceived control

- Improves recognition of known recommendations

Control:

- Friend control: higher accuracy

- Items control: higher novelty (fewer known recs)



Interaction (INT)

+


(R2 = .696)0.410 (0.092)***

0.955 (0.148)***+

STRUCTURAL MODEL


+

−

+

0.231(0.114)*

!2(2) = 10.81**item: −0.181 (0.097)1

friend: −0.389 (0.125)**

0.288 (0.091)**


++

0.067 (0.022)**

0.323 (0.031)***

+

−

+

+


(R2 = .044)

−0.152 (0.063)*

0.249(0.049)***

0.695 (0.304)*

0.148(0.051)**

+Perceived


(R2 = .512)

0.770(0.094)***

+ Perceived control

(R2 = .311)0.377(0.074)***

+

+

Understandability

(R2 = .153)



!2(2) = 10.70**item: 0.428 (0.207)*friend: 0.668 (0.206)**

0.459 (0.148)**

Personal Characteristics (PC)

+

−0.332 (0.088)***

0.205(0.100)*

0.375(0.094)***

+

−

Music expertise

0.257(0.124)*

+

Trusting propensity

0.166 (0.077)*

+

Familiarity with recommenders

inspectability and control in social recommenders

Technology