collaborative future event recommendation · event recommendation einat minkov, ben charrow...
TRANSCRIPT
COLLABORATIVE FUTURE EVENT RECOMMENDATION
Einat Minkov, Ben Charrow Jonathan Ledlie, Seth Teller, Tommi Jaakkola
1
CIKM 2010
Motivation: Context Aware Devices 2
■ We are interested in building devices that can make good recommendations with minimal feedback
■ In the future these devices will know a person’s location with incredible accuracy and precision
■ We focus on a recommendation system that can use this type of information
Roadmap 3
■ Event Recommendation ■ Individual vs. Collaborative Recommendation ■ User Study + Evaluation ■ Related Work ■ Active Learning + Future Work ■ Conclusion
Event Recommendations
■ Events are standard in that - they have textual descriptions
(e.g. announcements) - can be extracted
automatically from e-mail
■ Events are special in that - they are time and location
dependent - they have not happened yet
■ We focus on academic talks
4
PHAST: Hardware-Accelerated Shortest Path TreesSpeaker: Daniel DellingSpeaker Affiliation: Microsoft CorporationHost: Prof. Charles E. Leiserson
Date: 10-20-2010Time: 4:15 PM - 5:15 PMLocation: 32-G575 (Theory of Computation Seminar Room)
We present a novel algorithm to solve the nonnegativesingle-source shortest path problem on road networks andother graphs with low highway dimension. After a quickpreprocessing phase, we can compute all distances from agiven source in the graph with essentially a linear sweepover all vertices. Because this sweep is independent of thesource, we are able to reorder vertices in advance toexploit locality. Moreover, our algorithm takes advantageof features of modern CPU architectures, such as SSE andmulti-core. Compared to Dijkstra's algorithm...
5
■ Events require explicit representation in a feature space - pure collaborative filtering is not possible - each future event forms a “cold start”
■ We cast recommendations as a ranking problem - Assume user feedback in the form of pairwise preferences - Given:
Past events EP , future events EF
Preference pairs: (ej, ek) R(u), ej,ek EP , u U xj preferred over xk by user u
Event Recommendations
Roadmap 6
Event Recommendation ■ Individual vs. Collaborative Recommendation ■ User Study + Evaluation ■ Related Work ■ Active Learning + Future work ■ Conclusion
Small scale validation cont’d• We can also directly examine changes due to OR1 knock-out
Preliminary validation• The λ-phage (bacteria infecting virus)!"#$%&'#($)*#+,-./
0#,12(#'
3-142(#
5&1-6#
7&8.6,
OR3 OR1OR2cI cro
cI RNAp2
Arkin et al. 1998
• We can find the equilibrium of the game (binding frequencies)as a function of overall protein concentrations.
50
10!2
100
102
0
0.2
0.4
0.6
0.8
1
Binding in OR
3
frepressor
/fRNA
Bin
din
g F
requency (
tim
e!
avera
ge)
RepressorRNA!polymerase
(a) OR3
10!2
100
102
0
0.2
0.4
0.6
0.8
1
Binding in OR
2
frepressor
/fRNA
Bin
din
g F
requency (
tim
e!
avera
ge)
RepressorRNA!polymerase
(b) OR2
10!2
100
102
0
0.2
0.4
0.6
0.8
1
Binding in OR
1
frepressor
/fRNA
Bin
din
g F
requency (
tim
e!
avera
ge)
RepressorRNA!polymerase
(c) OR1
Figure 3: Predicted protein binding to sites OR3, OR2, and mutated OR1 for increasing amounts of cI2.
became sufficiently high do we find cI2 at the mutatedOR1 as well. Note, however, that cI2 inhibits transcrip-tion at OR3 prior to occupying OR1. Thus the bindingat the mutated OR1 could not be observed without in-terventions.
7 Discussion
We believe the game theoretic approach provides a com-pelling causal abstraction of biological systems with re-source constraints. The model is complete with prov-ably convergent algorithms for finding equilibria on agenome-wide scale.
The results from the small scale application are en-couraging. Our model successfully reproduces knownbehavior of the λ−switch on the basis of molecularlevel competition and resource constraints, without theneed to assume protein-protein interactions between cI2dimers and cI2 and RNA-polymerase. Even in the con-text of this well-known sub-system, however, few quan-titative experimental results are available about bind-ing. Proper validation and use of our model thereforerelies on estimating the game parameters from availableprotein-DNA binding data (in progress). Once the gameparameters are known, the model provides valid pre-dictions for a number of possible perturbations to thesystem, including changing nuclear concentrations andknock-outs.
Acknowledgments
This work was supported in part by NIH grant GM68762and by NSF ITR grant 0428715. Luis Perez-Breva is a“Fundacion Rafael del Pino” Fellow.
References
[1] Adam Arkin, John Ross, and Harley H. McAdams.Stochastic kinetic analysis of developmental path-way bifurcation in phage λ-infected excherichia colicells. Genetics, 149:1633–1648, August 1998.
[2] Kenneth J. Arrow and Gerard Debreu. Existence ofan equilibrium for a competitive economy. Econo-metrica, 22(3):265–290, July 1954.
[3] Z. Bar-Joseph, G. Gerber, T. Lee, N. Rinaldi,J. Yoo, B. Gordon F. Robert, E. Fraenkel,T. Jaakkola, R. Young, and D. Gifford. Compu-tational discovery of gene modules and regulatorynetworks. Nature Biotechnology, 21(11):1337–1342,2003.
[4] Otto G. Berg, Robert B. Winter, and Peter H. vonHippel. Diffusion- driven mechanisms of proteintranslocation on nucleic acids. 1. models and theory.Biochemistry, 20(24):6929–48, November 1981.
[5] Drew Fudenberg and Jean Tirole. Game Theory.The MIT Press, 1991.
10
• Predictions are again qualitatively correct
52
• The goal is to learn user rankings of events (talks) with minimal feedback
• With little feedback about past events, it is difficult to estimate the ranking function separately for each user
Event ranking
event descriptions(e.g., announcements)
relativeranking
Facilitating Complex Scientific Analytics in the CloudSpeaker: Magdalena BalazinskaSpeaker Affiliation: University of WashingtonHost: Sam MaddenHost Affiliation: CSAIL
Date: 9-21-2010Time: 11:00 AM - 12:00 PMRefreshments: 10:50 AMLocation: 32-D463
Abstract: Scientists today have the ability to generate data at anunprecedented scale and rate and, as a result, they must increasingly turn to
Facilitating Complex Scientific Analytics in the CloudSpeaker: Magdalena BalazinskaSpeaker Affiliation: University of WashingtonHost: Sam MaddenHost Affiliation: CSAIL
Date: 9-21-2010Time: 11:00 AM - 12:00 PMRefreshments: 10:50 AMLocation: 32-D463
Abstract: Scientists today have the ability to generate data at anunprecedented scale and rate and, as a result, they must increasingly turn to
Facilitating Complex Scientific Analytics in the CloudSpeaker: Magdalena BalazinskaSpeaker Affiliation: University of WashingtonHost: Sam MaddenHost Affiliation: CSAIL
Date: 9-21-2010Time: 11:00 AM - 12:00 PMRefreshments: 10:50 AMLocation: 32-D463
Abstract: Scientists today have the ability to generate data at anunprecedented scale and rate and, as a result, they must increasingly turn to
acb
1st
2nd
3rd
b)a)
c)user specific
ranking function
Event Ranking 7
■ The goal is to learn user rankings of future events (talks) with minimal feedback
■ With little feedback, it is difficult to estimate a ranking function separately for each user
■ The goal is to learn user rankings of future events (talks) with minimal feedback
■ With little feedback, it is difficult to estimate a ranking function separately for each user
Event Ranking 7
SVMRank for User Specific Ranking Function
Collaborative Event Ranking 8
■ We rank by making: - a feature mapping shared across users - user specific ranking functions
Small scale validation cont’d• We can also directly examine changes due to OR1 knock-out
Preliminary validation• The λ-phage (bacteria infecting virus)!"#$%&'#($)*#+,-./
0#,12(#'
3-142(#
5&1-6#
7&8.6,
OR3 OR1OR2cI cro
cI RNAp2
Arkin et al. 1998
• We can find the equilibrium of the game (binding frequencies)as a function of overall protein concentrations.
50
10!2
100
102
0
0.2
0.4
0.6
0.8
1
Binding in OR
3
frepressor
/fRNA
Bin
din
g F
requency (
tim
e!
avera
ge)
RepressorRNA!polymerase
(a) OR3
10!2
100
102
0
0.2
0.4
0.6
0.8
1
Binding in OR
2
frepressor
/fRNA
Bin
din
g F
requency (
tim
e!
avera
ge)
RepressorRNA!polymerase
(b) OR2
10!2
100
102
0
0.2
0.4
0.6
0.8
1
Binding in OR
1
frepressor
/fRNA
Bin
din
g F
requency (
tim
e!
avera
ge)
RepressorRNA!polymerase
(c) OR1
Figure 3: Predicted protein binding to sites OR3, OR2, and mutated OR1 for increasing amounts of cI2.
became sufficiently high do we find cI2 at the mutatedOR1 as well. Note, however, that cI2 inhibits transcrip-tion at OR3 prior to occupying OR1. Thus the bindingat the mutated OR1 could not be observed without in-terventions.
7 Discussion
We believe the game theoretic approach provides a com-pelling causal abstraction of biological systems with re-source constraints. The model is complete with prov-ably convergent algorithms for finding equilibria on agenome-wide scale.
The results from the small scale application are en-couraging. Our model successfully reproduces knownbehavior of the λ−switch on the basis of molecularlevel competition and resource constraints, without theneed to assume protein-protein interactions between cI2dimers and cI2 and RNA-polymerase. Even in the con-text of this well-known sub-system, however, few quan-titative experimental results are available about bind-ing. Proper validation and use of our model thereforerelies on estimating the game parameters from availableprotein-DNA binding data (in progress). Once the gameparameters are known, the model provides valid pre-dictions for a number of possible perturbations to thesystem, including changing nuclear concentrations andknock-outs.
Acknowledgments
This work was supported in part by NIH grant GM68762and by NSF ITR grant 0428715. Luis Perez-Breva is a“Fundacion Rafael del Pino” Fellow.
References
[1] Adam Arkin, John Ross, and Harley H. McAdams.Stochastic kinetic analysis of developmental path-way bifurcation in phage λ-infected excherichia colicells. Genetics, 149:1633–1648, August 1998.
[2] Kenneth J. Arrow and Gerard Debreu. Existence ofan equilibrium for a competitive economy. Econo-metrica, 22(3):265–290, July 1954.
[3] Z. Bar-Joseph, G. Gerber, T. Lee, N. Rinaldi,J. Yoo, B. Gordon F. Robert, E. Fraenkel,T. Jaakkola, R. Young, and D. Gifford. Compu-tational discovery of gene modules and regulatorynetworks. Nature Biotechnology, 21(11):1337–1342,2003.
[4] Otto G. Berg, Robert B. Winter, and Peter H. vonHippel. Diffusion- driven mechanisms of proteintranslocation on nucleic acids. 1. models and theory.Biochemistry, 20(24):6929–48, November 1981.
[5] Drew Fudenberg and Jean Tirole. Game Theory.The MIT Press, 1991.
10
• Predictions are again qualitatively correct
52
• The goal is to learn user rankings of events (talks) with minimal feedback
• We learn the ranking function in two parts- a feature mapping shared across users- user specific ranking functions based on the features
Collaborative event ranking
event descriptions(e.g., announcements)
“perceptual” featurecoordinates
relativeranking
a
Facilitating Complex Scientific Analytics in the CloudSpeaker: Magdalena BalazinskaSpeaker Affiliation: University of WashingtonHost: Sam MaddenHost Affiliation: CSAIL
Date: 9-21-2010Time: 11:00 AM - 12:00 PMRefreshments: 10:50 AMLocation: 32-D463
Abstract: Scientists today have the ability to generate data at anunprecedented scale and rate and, as a result, they must increasingly turn to
Facilitating Complex Scientific Analytics in the CloudSpeaker: Magdalena BalazinskaSpeaker Affiliation: University of WashingtonHost: Sam MaddenHost Affiliation: CSAIL
Date: 9-21-2010Time: 11:00 AM - 12:00 PMRefreshments: 10:50 AMLocation: 32-D463
Abstract: Scientists today have the ability to generate data at anunprecedented scale and rate and, as a result, they must increasingly turn to
Facilitating Complex Scientific Analytics in the CloudSpeaker: Magdalena BalazinskaSpeaker Affiliation: University of WashingtonHost: Sam MaddenHost Affiliation: CSAIL
Date: 9-21-2010Time: 11:00 AM - 12:00 PMRefreshments: 10:50 AMLocation: 32-D463
Abstract: Scientists today have the ability to generate data at anunprecedented scale and rate and, as a result, they must increasingly turn to
b
c
bac
1st
2nd
3rd
b)a)
c)user specific
ranking functionsharedfeaturemapping
Collaborative Event Ranking 8
■ We rank by making: - a feature mapping shared across users - user specific ranking functions
Comparing Ranking Methods 9
RankSVM LowRank
Feature Vectors
User Parameters
Estimation
Number of Parameters
A seminar announcement 10
A seminar announcement 10
Representing Announcements 11
■ Token Frequency-Inverse Document Frequency (TF-IDF)
■ Latent Dirichlet Allocation (LDA) - generative model where documents are distributions
over “topics” which are distributions over words - trained with 100 topics
■ Both representations use large unlabelled corpus
Roadmap 12
Event Recommendation Individual vs. Collaborative Recommendation ■ User Study + Evaluation ■ Related Work ■ Active Learning + Future work ■ Conclusion
Feedback Elicitation 13
■ Events compete for user’s resources (primarily, time) ■ Assume a week-long time frame in which events can
be meaningfully compared ■ User study: ask users to select events they would
have liked to attend in a given week.
User Study: Interface 14
User Study: Details 15
■ At MIT: - 30 graduate students, researchers and faculty - 15 consecutive weeks of MIT CSAIL seminar announcements
starting from 9/2007
■ At CMU: - 56 CS grad. students - 15 consecutive weeks of MIT CSAIL seminar announcements
starting from 2/2009
■ 7 years of unlabelled MIT CSAIL seminar announcements from 5/2002 – 7/2009
User Study: The Data Set 16
0
5
10
15
20
25
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Nu
mb
er o
f T
alk
s
Week
CMU StudyAdvertised Talks
User Interest
0
5
10
15
20
25
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Nu
mb
er o
f T
alk
s
Week
MIT StudyAdvertised Talks
User Interest
User Study: The Data Set 16
0
5
10
15
20
25
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Nu
mb
er o
f T
alk
s
Week
CMU StudyAdvertised Talks
User Interest
0
5
10
15
20
25
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Nu
mb
er o
f T
alk
s
Week
MIT StudyAdvertised Talks
User Interest
# Announcements
Avg. # Attended talks
Evaluation: Overview 17
■ RankSVM vs. LowRank - TF-IDF or LDA features - Perceptual space and parameter sensitivity
■ Experimental setup - Randomly select k<10 weeks per user to train on - Test on same 5 weeks across users
■ Measuring results - Metric: Mean Average Precision (MAP) - Baseline performance: random recommendation
0.35
0.40
0.45
0.50
0.55
0.60
1 2 3 4 5 6 7 8 9 10
Me
an
Ave
rag
e P
reci
sio
n
Number of Training Weeks
LowRank w/TF-IDFLowRank w/LDA
RankSVM w/TF-IDFRankSVM w/LDA
Evaluation: LowRank vs. RankSVM 18
Random
0.35
0.40
0.45
0.50
0.55
0.60
1 2 3 4 5 6 7 8 9 10
Me
an
Ave
rag
e P
reci
sio
n
Number of Training Weeks
LowRank w/TF-IDFLowRank w/LDA
RankSVM w/TF-IDFRankSVM w/LDA
Evaluation: LowRank vs. RankSVM 18
Week 3 performance
Random
0.40
0.42
0.44
0.46
0.48
0.50
0.52
0.54
0.56
0.58
0.60
1 2 3 4 5 6 7 8 9 10
Mean A
vera
ge P
reci
sion
Number of Training Weeks
TF-IDF k=12, C=2TF-IDF k=10, C=1
LDA k=12, C=2LDA k=10, C=1
Evaluation: Perceptual Space 19
LowRank: Perceptual Spaces and Slack
0.40
0.42
0.44
0.46
0.48
0.50
0.52
0.54
0.56
0.58
0.60
1 2 3 4 5 6 7 8 9 10
Mean A
vera
ge P
reci
sion
Number of Training Weeks
TF-IDF k=12, C=2TF-IDF k=10, C=1
LDA k=12, C=2LDA k=10, C=1
Evaluation: Perceptual Space 19
LowRank: Perceptual Spaces and Slack
LDA
TF-IDF
0.0
0.2
0.4
0.6
0.8
1.0
0.2 0.3 0.4 0.5 0.6 0.7 0.8
CD
F o
f A
ll T
est
ing W
eeks
Average Precision of Testing Week
LDATF-IDFRandom
Evaluation: Individual Users 20
LowRank with 3 weeks of training vs. Random
Roadmap 21
Event Recommendation Individual vs. Collaborative Recommendation User Study + Evaluation ■ Related Work ■ Active Learning + Future work ■ Conclusion
Related Work 22
■ Collaborative filtering via matrix factorization - J. Rennie and N. Srebro. (ICML 2005) - T. Hoffman (TOIS 2004) - S. Tong and D. Koller. (JMLR 2002)
■ Reduced dimensionality recommendations - B. Bai et. Al (NIPS 2009)
■ Content-Based Recommendations for scientific papers - Dumais and Nielsen. (SIGIR 1992) - C. Basu, H. Hirsh, W. W. Cohen, and C. Nevill-Manning. (JAIR 2001) - D. Yarowsky and R. Florian. (SIGDAT 1999)
■ Collaborative recommendation with feature descriptions - W. Chu and S.-T. Park. (WWW 2009)
Active Learning 23
■ Problem: users must rate all announcements ■ Idea: model the uncertainty in ranking functions
- add uncertainty to each user provided label (i.e. add a small flipping probability)
- use information gain to select which seminars to present to a user
■ We want to run another user study which has an active learning component
Future Work: Features + Deployment
■ Include additional features - speaker’s past articles - speaker affiliation - host name - venue information
■ Couple the recommendations with an indoor location system
24
PHAST: Hardware-Accelerated Shortest Path TreesSpeaker: Daniel DellingSpeaker Affiliation: Microsoft CorporationHost: Prof. Charles E. Leiserson
Date: 10-20-2010Time: 4:15 PM - 5:15 PMLocation: 32-G575 (Theory of Computation Seminar Room)
We present a novel algorithm to solve the nonnegativesingle-source shortest path problem on road networks andother graphs with low highway dimension. After a quickpreprocessing phase, we can compute all distances from agiven source in the graph with essentially a linear sweepover all vertices. Because this sweep is independent of thesource, we are able to reorder vertices in advance toexploit locality. Moreover, our algorithm takes advantageof features of modern CPU architectures, such as SSE andmulti-core. Compared to Dijkstra's algorithm...
Roadmap 25
Event Recommendation Individual vs. Collaborative Recommendation User Study + Evaluation Related Work Active Learning + Future work ■ Conclusion
Contributions 26
■ Presented and motivated the problem of recommending future events
■ Gave a low rank collaborative learning approach ■ Empirically evaluated the approach, showing that it
outperformed a content based recommendation system
■ Created a publicly available dataset http://mis.haifa.ac.il/~einatm
http://www.seas.upenn.edu/~bcharrow
Acknowledgments 27
Einat Minkov Jonathan Ledlie Tommi Jaakkola Seth Teller
Thank you!
34