assignment 2 observations mausam © mausam1. the sat encoding induced subgraph isomorphism – np...
TRANSCRIPT
1
Assignment 2 Observations
Mausam
© Mausam
2
The SAT Encoding
• Induced Subgraph Isomorphism– NP complete problem– Applications: similarities between chemical compounds; protein-
interaction networks, social networks, circuit design..
• Variables– X(i,j) denotes that ith node (in smaller graph) mapped to jth node
(in larger graph)– Self edge: If node i has a self-edge and node j doesn’t
• remove X(i,j)
– Self-edge: If node j has a self-edge and node i doesn’t• remove X(i,j)
© Mausam
3
The SAT encoding• Each node maps to exactly one other node– X(i,1) X(i,2) X(i,3)… X(i,n)– Pairs of X(i,1) X(i,2); X(i,1) X(i,3) …
• Each node in larger graph maps to at most one node– Pairs of X(1,j)X(2,j); X(1,j)X(3,j); …
• Disallowed pairs of mappings– If edge(i1,i2) and no edge(j1,j2): X(i1,j1)X(i2,j2); – If no edge(i1,i2) and edge(j1,j2): X(i1,j1)X(i2,j2); – If edge(i1,i2) and edge(j2,j1): X(i1,j1)X(i2,j2);
• Overall – O(nm) vars, O(n2m2) clauses© Mausam
4
Fun optimizations
• Degree trick: Can B ever map to P/S/R?– No: Indegree of B is greater than indegree of P/S/R– Remove X(B,P), X(B,S), X(B,R)
• Common neighbor constraint: Can A-U and C-T?– They both don’t have an edge– They both have outdegrees 1– No: because AC have common neighbor 1; UT 0– X(A,U)X(C,T)
• …
A C
B
P R
Q
SYes. A mapping is: M(A) = S, M(B) = Q, M(C) = R
The edges from P to other nodesdon‘t matter since no node in Ggot mapped to P.
T
U
© Mausam
5
Observations
• People solved ~100+80 node problems…– For an NP hard problem that’s not too bad, is it?
• Paresh’s observations– adding redundant clauses helps miniSAT for tough
problems.
© Mausam
6
Best Competitors
• Kim Wallmark
• Alan Ludwig
• Haroon Barri
© Mausam
7
Assignment 3 Discussion
Mausam
© Mausam
8
BlackJack: State Space
• MinSum: the minimum sum of the player's hand so far.• NumAces: the number of Aces in the player's hand so far.• dMinSum: the minimum sum of the dealer's hand so far.• dNumAces: the number of Aces in the dealer's hand so far.• isTwoCards: a Boolean that represents that the player's hand has
just two cards so far.• isBlackJack: a Boolean that represents that the player got a
BlackJack.• pair: has value between 0 and 10. Zero indicates that the player
doesn't have a pair. Any other value i indicates that the player has two cards of value i.
• turn: a Boolean to indicate whether it is player's turn or dealer's.© Mausam
9
BlackJack: Action Space
• Stand: Turn = 0• Hit: get a new card with prob… update minSum,
numAces, isTwoCard, pair• Double: get a new card, update minSum, numAces,
etc.; – Turn = 1. – Allowed when isTwoCard = 1
• Default: run default dealer policy. – Allowed with turn = 1
• Split: tricky! – Allowed with pair > 0; isTwoCard = 1
© Mausam
10
Equations
© Mausam
'
)'(),|'(),(s
sVHITssPHITsQ
'
)'(),|'(),(s
sVSTANDssPSTANDsQ
'
)'(),|'(),(s
sVDEFssPDEFsQ
'
)'(),|'(2),(s
sVDOUBssPDOUBsQ
11
BlackJack Split (say only three cards)• (c,c)– [(c,1) & (c,1)] p1p1
– [(c,1) & (c,2)] p1p2– [(c,1) & (c,3)] p1p3– [(c,2) & (c,1)] p2p1– [(c,2) & (c,2)] p2p2– [(c,2) & (c,3)] p2p3– [(c,3) & (c,1)] p3p1– [(c,3) & (c,2)] p3p2– [(c,3) & (c,3)] p3p3© Mausam
• Q((c,c), SPLIT)– 2p1p1V(c,1) – p1p2V(c,1) + p1p2V(c,2)– p1p3V(c,1) + p1p3V(c,3)– p2p1V(c,1) + p2p1V(c,2) – 2p2p2V(c,2)– p2p3V(c,3) + p2p3V(c,3)– p1p3V(c,1) + p1p3V(c,3)– p2p3V(c,3) + p2p3V(c,3)– 2p3p3V(c,3)
2p1(p1+p2+p3) = 2p1
12
Equations
© Mausam
'
)'(),|'(),(s
sVHITssPHITsQ
'
)'(),|'(),(s
sVSTANDssPSTANDsQ
'
)'(),|'(),(s
sVDEFssPDEFsQ
'
)'(),|'(2),(s
sVDOUBssPDOUBsQ
'
,...)',()'(2),...),,((c
ccVcPSPLITccQ
13
Value of Terminal States• If isTwoCards = 1, minSum=11, numAces = 1– Return 1.5
• If minSum>21– Return -1
• If minSum < 21 and dMinSum > 21– Return 1
• …• …
© Mausam
14
Can you solve it with Expectimax?
• No– Because there is possibility of infinite loop
• Values are well-formed– Loop of size n becomes less probable as n increases– (see beginning example from lecture on MDPs)– EXCEPTION: Q(1010, split) for p> 0.5
• Solution: use Value Iteration/Policy Iteration© Mausam
15
Optimizations• Dealer’s policy is fixed. So, run the Markov chain ahead of
time to generate a table– Pr(dealer gets 17|first card=c)– Pr(dealer gets 18|first card=c)– Pr(dealer gets 19|first card=c)– Pr(dealer gets 20|first card=c)– Pr(dealer gets 21|first card=c)– Pr(dealer busts|first card=c)
• Remove dMinSum, dNumAces, turn.• Add dFirstCard• Remove Default action• Change the equations for double/stand based on computed
probabilities© Mausam
16
Comments
• numAces – not required. Convert to hasAce– Only first ace is relevant. After that each ace = 1
• Handle infinite splits by allowing say, 10 splits– Should work fine in practice
• Modification: have the bet amount in state space– Needn’t multiply by 2 in equations for doubling– Need to use this info when computing terminal rewards
© Mausam
17
Difference in solutions?
• In real BJ: Dealer and player BJ is a push– In our BJ: player wins
• ???• In real BJ: there is a 4/6/8 card deck– In our BJ: infinite cards
• You can easily modify code to handle 1st case– 2nd case: Monte carlo sampling!
© Mausam
18
Should you play our BlackJack?
• Expected reward before game starts– SUM[Pr(game)*expected reward of the game]
• Deal three cards… and use V(s) to compute exp reward
• My expected reward: $0.058– Yes! We should play the game
© Mausam
19
Discussion
Mausam
© Mausam
20
Followup Courses• 546 Machine Learning/Data Mining
• 515 Probabilistic Graphical Models
• 517 Natural Language Processing
• ??? AI/Statistics for Big Data
• 571 Robotics
• 579 Intelligent control
• 574 Special Topics in AI
• 528 Computational Neuroscience
• 510 Human computer Interaction© Mausam
21
Final Exam• 50% objective– True/false– Multiple choice– Fill in the blanks– Match the following…
• 25% short answer questions– 1 word – 5 line responses…– opinions/strengths/weaknesses/etc…
• 25% Problem solving– Problem modeling; short calculations…
© Mausam
22
Why do an exam?
• Goal: revise and analyze the whole course
• Expected preparation time– ~1 day– Go over lecture slides– Read parts of the notes that are unclear…
© Mausam
23
What worked• Programming Assignments
– You hated them– You loved them– You learned through them
• Written Assignments(?)– Forced us to be regular
• Classes– I absolutely enjoyed the interaction!
• Emails/bulletin boards
• Videos(?)
© Mausam
24
What didn’t work
• Programming environment– People didn’t know unix– People didn’t know Java!
• Packed– Once someone got behind, it got tough to recover
• Workload
© Mausam
25
Thanks
• To all of you– who contributed to helping each other through
newsgroup and other questions
– who asked terrific questions in class
– who gave regular feedback on class, assignments, grading, organization
© Mausam