11

A User-Guided Cognitive A User-Guided Cognitive Agent for Wireless Service Agent for Wireless Service

Selection in Pervasive Selection in Pervasive ComputingComputing

George LeeGeorge Lee

May 5, 2004May 5, 2004

G. Lee, P. Faratin, S. Bauer, and J. Wroclawski. G. Lee, P. Faratin, S. Bauer, and J. Wroclawski. A User-Guided A User-Guided Cognitive Agent for Network Service Selection in Pervasive Cognitive Agent for Network Service Selection in Pervasive Computing EnvironmentsComputing Environments. In Proceedings of Second IEEE . In Proceedings of Second IEEE International Conference on Pervasive Computing and International Conference on Pervasive Computing and Communications (PerCom ’04), 2004.Communications (PerCom ’04), 2004.

22

PublicationsPublications

G. Lee, S. Bauer, P. Faratin, and J. Wroclawski. G. Lee, S. Bauer, P. Faratin, and J. Wroclawski. An An Agent for Interactively Learning User Preferences On-Agent for Interactively Learning User Preferences On-Line for Wireless Services ProvisioningLine for Wireless Services Provisioning. To appear in . To appear in Proceedings of AAMAS, 2004.Proceedings of AAMAS, 2004.

G. Lee, P. Faratin, S. Bauer, and J. Wroclawski. G. Lee, P. Faratin, S. Bauer, and J. Wroclawski. A User-A User-Guided Cognitive Agent for Network Service Selection Guided Cognitive Agent for Network Service Selection in Pervasive Computing Environmentsin Pervasive Computing Environments. In Proceedings . In Proceedings of IEEE Conference on Pervasive Computing and of IEEE Conference on Pervasive Computing and Communication (PerCom), 2004.Communication (PerCom), 2004.

P. Faratin, G. Lee, J. Wroclawski, and S. Parsons. P. Faratin, G. Lee, J. Wroclawski, and S. Parsons. Social Social User Agents for Dynamic Access to Wireless NetworksUser Agents for Dynamic Access to Wireless Networks. . In Proceedings of AAAI Spring Symposium, 2003. In Proceedings of AAAI Spring Symposium, 2003.

P. Faratin, J. Wroclawski, G. Lee, and S. Parsons. P. Faratin, J. Wroclawski, G. Lee, and S. Parsons. The The Personal Router: An Agent for Wireless AccessPersonal Router: An Agent for Wireless Access. In . In Proceedings of AAAI Fall Symposium, 2002.Proceedings of AAAI Fall Symposium, 2002.

33

OverviewOverview The Personal Router connectivity modelThe Personal Router connectivity model

– An open, competitive market for network accessAn open, competitive market for network access The service selection problemThe service selection problem

– Network and user challengesNetwork and user challenges A learning agent approachA learning agent approach

– Design and implementationDesign and implementation Experimental resultsExperimental results

– Agent is accurate and unobtrusiveAgent is accurate and unobtrusive Solving other systems and networking Solving other systems and networking

problemsproblems– Helping users deal with complexity using learningHelping users deal with complexity using learning

44

The Problems with The Problems with Current Connectivity Current Connectivity ModelsModels

Limited Limited competitioncompetition

Long-term Long-term contractscontracts

Difficult to switchDifficult to switch

Regional Providers

Limited Limited coveragecoverage

No marketNo market

Local Providers

55

A New Connectivity A New Connectivity ModelModel

CompetitionCompetition No long-term No long-term

contractscontracts Bottom-upBottom-up

Advantages:

Verizon

User

MIT

Pizza Shop

66

The Personal RouterThe Personal Router

PersonalRouter

ServiceNegotiation

ApplicationCompositionYour personal

digital accessories

Your home

National provider

Local pizza shop

77

Economic and Economic and Technical ChallengesTechnical Challenges Service discoveryService discovery Network level mobility/handoffNetwork level mobility/handoff Payment mechanismsPayment mechanisms Service provisioningService provisioning SecuritySecurity Service selectionService selection

88

Services

Service SelectionService Selection

UserInput

ServiceInfo

UserActivity

Inputs Output

ServicesSelectionPersonalRouter

99

Incomplete service informationIncomplete service information– Service providers may be unwilling Service providers may be unwilling

or unable to provide detailed or unable to provide detailed service informationservice information

Dynamic environmentDynamic environment– Existing services may become Existing services may become

unavailable and new services may unavailable and new services may become availablebecome available

Network Service Network Service ChallengesChallenges

1010

Dealing with User Dealing with User PreferencesPreferences

Service features Service features includeinclude– BandwidthBandwidth– LatencyLatency– Complex pricing Complex pricing

plansplans User context User context

includesincludes– LocationLocation– ApplicationsApplications– Urgency of their Urgency of their

tasktask

11 Mbps $0.25/min

128 kbps $10/mon

1 Mbps $0.05/MB

Utility depends on:

1111

Service Selection Must Service Selection Must Be UnobtrusiveBe Unobtrusive Manual selection does not workManual selection does not work

– Cognitively demandingCognitively demanding Static rules do not workStatic rules do not work

– Does not accommodate individual user Does not accommodate individual user preferencespreferences

– Difficult for application developersDifficult for application developers Offline preference elicitation (e.g. Offline preference elicitation (e.g.

configuration files) does not workconfiguration files) does not work– Time consumingTime consuming– Users may not be able to specify their Users may not be able to specify their

preferencespreferences

1212

Services

A Three Step Learning A Three Step Learning Service Selection Service Selection ApproachApproach

PerceivedQuality

PerceivedCost

Q/CTradeoff

Utility

UserInput

ServiceInfo

UserActivity

Inputs Output

ServicesSelectionSelectservice

Computeutility

Learnfromuser

User Model User Utility

1313

Services

Agent ArchitectureAgent Architecture

PerceivedQuality

PerceivedCost

Q/CTradeoff

Utility

UserInput

ServiceInfo

UserActivity

Inputs

Output

ServicesSelection

User Model User Utility

ServiceValue

Predictor

ServiceValue

Estimator

UtilityCalculator

ServiceSelector

Q/CTradeoffModeler

InputTranslator

1414

Services

Interpreting User InputInterpreting User Input

PerceivedQuality

PerceivedCost

Q/CTradeoff

Utility

UserInput

ServiceInfo

UserActivity ServicesSelection

ServiceValue

Predictor

ServiceValue

Estimator

UtilityCalculator

ServiceSelector

Q/CTradeoffModeler

InputTranslator

Inputs

OutputUser Model User Utility

1515

Interpreting UserInterpreting UserInputInput

UserInput

ServiceValue

Predictor

ServiceValue

Estimator

Q/CTradeoffModeler

Δw

Δq, Δc

InputTranslator

PreferenceUpdates

Q/CTradeoffUpdates

UserInterface

Translates high level Translates high level user input into specific user input into specific updatesupdates

Provides an intuitive Provides an intuitive and unobtrusive and unobtrusive interfaceinterface

1616

Unobtrusiveness is Unobtrusiveness is EssentialEssential Goal is opposite of traditional user Goal is opposite of traditional user

interfacesinterfaces User interface must not distract or User interface must not distract or

interrupt userinterrupt user Explicit rating is too obtrusive and Explicit rating is too obtrusive and

cognitively demandingcognitively demanding Implicit rating based on simple Implicit rating based on simple

inputinput

1717

An IntuitiveAn IntuitiveInterface Interface High LevelHigh Level Simple & IntuitiveSimple & Intuitive ““better” and better” and

“cheaper”“cheaper” ““No input” implies No input” implies

satisfactionsatisfaction

““Taxi Meter”Taxi Meter”– Shows total paid since Shows total paid since

last reset.last reset. ““Money Speedometer”Money Speedometer”

– Shows derivative of the Shows derivative of the taxi meter.taxi meter.

““Better” buttonBetter” button– Noticeably improves quality. Noticeably improves quality.

““Cheaper” button.Cheaper” button.– Noticeably improves cost.Noticeably improves cost.– Note that “better” and Note that “better” and

“cheaper” actions are not “cheaper” actions are not symmetric.symmetric.

$ 01.14$ 01.14

Cheaper Better

TotalSpending Rate

1818

Services

Learning Quality and Learning Quality and CostCost

PerceivedQuality

PerceivedCost

Q/CTradeoff

Utility

UserInput

ServiceInfo

UserActivity

Output

ServicesSelection

ServiceValue

Predictor

ServiceValue

Estimator

UtilityCalculator

ServiceSelector

Q/CTradeoffModeler

InputTranslator

Inputs

User Model User Utility

1919

Estimating ServiceEstimating ServiceValueValue

ServiceInfo

CostPrediction

QualityPrediction

PerceivedQuality

PerceivedCost

UserActivity

ServiceValue

Predictor

ServiceValue

EstimatorΔq, Δc

PreferenceUpdates

Q(s,a)

C(s,a)

a

Learns user perceived quality and cost based on user Learns user perceived quality and cost based on user feedback:feedback:Q(s,a) Q(s,a) (1 – (1 – αα)Q(s,a) + )Q(s,a) + ααΔqC(s,a) C(s,a) (1 – (1 – αα)C(s,a) + )C(s,a) + ααΔc

Predictor Predictor provides initial provides initial estimatesestimates

2020

Predicting ServicePredicting ServiceValueValue

ServiceInfo

CostPrediction

Prediction by nonlinear Prediction by nonlinear regression: maps from regression: maps from activity and service activity and service features to perceived features to perceived quality and cost using quality and cost using a neural networka neural networkQuality

Prediction

PerceivedQuality

PerceivedCost

UserActivity

ServiceValue

Predictor

ServiceValue

EstimatorPreferenceUpdates

a

Δq, Δc

I

CP(I,a)QP(I,a)

Predictor trained on user feedback

Δq, Δc

Q(s,a)

C(s,a)

a

2121

Services

Calculating Utility and Calculating Utility and Selecting a ServiceSelecting a Service

PerceivedQuality

PerceivedCost

Q/CTradeoff

Utility

UserInput

ServiceInfo

UserActivity

Inputs Output

ServicesSelection

User Model User Utility

ServiceValue

Predictor

ServiceValue

Estimator

UtilityCalculator

ServiceSelector

Q/CTradeoffModeler

InputTranslator

2222

Calculating UtilityCalculating Utility

Services

PerceivedQuality

PerceivedCost

Q/CTradeoff

Utility ServicesSelectionUtility

CalculatorServiceSelector

U(w,q,c) = wq + (1 – U(w,q,c) = wq + (1 – w)cw)c

Q(s,a)

C(s,a)

w

U(w,q,c)

Calculates utility as a Calculates utility as a linear weighted linear weighted average:average:U(w,q,c) = wq + (1 – U(w,q,c) = wq + (1 – w)cw)c

2323

P(s) = P(s) =

((TT: “temperature” controls exploration : “temperature” controls exploration level)level)

Selecting a ServiceSelecting a Service

Services

PerceivedQuality

PerceivedCost

Q/CTradeoff

Utility ServicesSelectionUtility

CalculatorServiceSelector

Q(s,a)

C(s,a)

w

U(w,q,c)

U(s)

Balances Balances exploration and exploration and exploitation by exploitation by selecting services selecting services stochastically stochastically according to a according to a Gibbs softmax Gibbs softmax distributiondistribution

Only switches on Only switches on user input, activity user input, activity change, or change change, or change in set of available in set of available servicesservices

2424

Services

Learning Addresses Learning Addresses Network and User Network and User ComplexityComplexity

PerceivedQuality

PerceivedCost

Q/CTradeoff

Utility

UserInput

ServiceInfo

UserActivity

Inputs Output

ServicesSelection

User Model User Utility

ServiceValue

Predictor

ServiceValue

Estimator

UtilityCalculator

ServiceSelector

Q/CTradeoffModeler

InputTranslator

Handles incomplete service Handles incomplete service information using estimatorinformation using estimator

Handles dynamic Handles dynamic environments using environments using predictorpredictor

User model accommodates User model accommodates complex quality and cost complex quality and cost functionsfunctions

2525

Evaluating the Agent Evaluating the Agent with User Experimentswith User Experiments To answer these questions:To answer these questions:

– Can the agent learn?Can the agent learn?– Is it better than manual selection?Is it better than manual selection?– How can we improve the agent model?How can we improve the agent model?

Dynamic networkDynamic network– 8 services available8 services available– Features: Bandwidth, Price/min, Price/kbFeatures: Bandwidth, Price/min, Price/kb– Available services change during experimentAvailable services change during experiment

17 subjects17 subjects– 9 learning agent users9 learning agent users– 8 manual selection users8 manual selection users

Only 30 minutes longOnly 30 minutes long

2626

Experimental Experimental ProcedureProcedure View 20 pages in 5 View 20 pages in 5

minutesminutes Spend as few Spend as few

credits as possiblecredits as possible– Subjects were paid Subjects were paid

according to how according to how well they didwell they did

Select services Select services using only the using only the given interfacegiven interface

2727

The Agent vs. Manual The Agent vs. Manual SelectionSelection

2828

The Agent Learns The Agent Learns UtilityUtility

2929

SummarySummary Our fundamental goals:Our fundamental goals:

– Catalyze a ubiquitous, competitive Catalyze a ubiquitous, competitive wireless access marketwireless access market

– Make it simple for users and diverse Make it simple for users and diverse digital devices to use itdigital devices to use it

Our approach:Our approach:– A Personal Router that selects A Personal Router that selects

services using a learning agentservices using a learning agent AccurateAccurate UnobtrusiveUnobtrusive

3030

Future WorkFuture Work

Group learning Group learning – Collaborative filteringCollaborative filtering– Clustering based on user preferencesClustering based on user preferences

More experimentsMore experiments– Modeling seller behaviorModeling seller behavior– More sophisticated learning algorithmsMore sophisticated learning algorithms

Other market modelsOther market models– AuctionsAuctions– NegotiationNegotiation

Solving other systems and networking Solving other systems and networking problemsproblems– Helping users deal with complexity using learningHelping users deal with complexity using learning

3131

The Agent vs. Manual The Agent vs. Manual SelectionSelection

3232

The Agent Learns The Agent Learns UtilityUtility