policy and value transfer in lifelong reinforcement learning poster.pdf · policy and value...

1
Policy and Value Transfer in Lifelong Reinforcement Learning Yuu Jinnai, David Abel, Yue Guo, George Konidaris, Michael L. Littman Goal Jumpstart Policies Informed Transfer for RL Theorem 1. For a distribution of MDPs with R ~ D, Average MDP is an optimal fixed policy. PAC-MDP (Objective) Minimize the overestimate (Our Solution) Initialize with: Proposed Algortihm: MaxQInit Approximate from n sampled MDPs: Theorem 3. For a given δ, after n sampled MDPs, MaxQInit will retain optimism with probability 1 - δ: Lifelong Reinforcement Learning Objective 1: Find the policy that maximizes the expected performance over the distribution: Sample complexity of many PAC-MDP algorithms depends on the of overestimation of the initial value function: Jumpstart Experiments Theorem 2. For a distribution of MDPs with R ~ D, the performance of Average MDP has a lower bound: Delayed Q Q-Learning Transfer Experiments Understand knowledge transfer in lifelong RL. Task E M D h V avg M (s) i max M 2M Pr(M )V M (s) Expected performance of Avg. MDP policy ˜ O 0 @ X (s,a)2S A max {Q init (s, a) - V (s), 0} 1 A MaxQInit(s, a) := max M 2{M 1 ,...,M n } b Q M (s, a) MaxQInit 1. Optimal Fixed Policy 2. Informed Transfer for RL Task 1 Task 2 Task 3 Task 1 Task 2 1 2 Task 1 Task 2 Task 3 M D Tasks arg max 2E M D [V M (s 0 )] Performance of on task M. repeat: 1.Sample a task from a distribution: M ~ D. 2.Solve the sampled task M. R D G D d : S 7! A Avg. MDP Avg. V s : S 7! Pr(A) Avg. MDP Avg. V b : S Pr(M ) 7! A Belief MDP Belief MDP (Deterministic) (Stochastic) (Any policy) Probability the action is optimal: Action Prior prior (a | s) := Pr M D a = arg max a 0 Q M (s, a 0 ) Average MDP Pr(M 1 ) = 0.5 Pr(M 2 ) = 0.5 0.0 1.0 s 0 avg 0.5 0.5 s 0 prior Choose optimal action in averaged MDP: prior avg u b Q init (s, a) Q (s, a) Q max (s, a) := max M 2M Q M (s, a) (Constraint) Do not underestimate: Q max (s, a) 8 s,a : MaxQInit(s, a) max M 2M Q M (s, a) VMax Q max (s, a) References: [1] [1]: Brunskill & Li. Pac-inspired option discovery in lifelong reinforcement learning. 2014. [3] [4] [2] [2] [2]: Ramachandran & Amir. Bayesian inverse reinforcement learning. 2007. [3]: Singh, Jakkola, Jordan. Learning without state-estimation in partially observable markovian decision processes 2003 [4]: Astrom. Optimal control of markov processes with incomplete state information 1965 [5]: Rossman & Ramamoorthy. What good are actions? 2012. [6] [5] [6] [6]: Strehl et al.. Reinforcement learning in finite MDPs: PAC analysis. 2009. Department of Computer Science, Brown University {yuu_jinnai,david_abel,yue_guo,george_konidaris,michael_littman}@brown.edu

Upload: others

Post on 14-Sep-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Policy and Value Transfer in Lifelong Reinforcement Learning poster.pdf · Policy and Value Transfer in Lifelong Reinforcement Learning Yuu Jinnai, David Abel, Yue Guo, George Konidaris,

Policy and Value Transfer in Lifelong Reinforcement LearningYuu Jinnai, David Abel, Yue Guo, George Konidaris, Michael L. Littman

Goal Jumpstart Policies Informed Transfer for RL

Theorem 1. For a distribution of MDPs with R ~ D, Average MDP is an optimal fixed policy.

PAC-MDP

(Objective) Minimize the overestimate

(Our Solution) Initialize with:

Proposed Algortihm: MaxQInitApproximate from n sampled MDPs:

Theorem 3. For a given δ, after n sampled MDPs, MaxQInit will retain optimism with probability 1 - δ:

Lifelong Reinforcement Learning

Objective 1: Find the policy that maximizes the expected performance over the distribution:

Sample complexity of many PAC-MDP algorithms depends on the of overestimation of the initial value function:

Jumpstart Experiments

Theorem 2. For a distribution of MDPs with R ~ D, the performance of Average MDP has a lower bound:

Delayed Q Q-Learning

Transfer Experiments

Understand knowledge transfer in lifelong RL.

Task

EM⇠D

hV

⇡⇤avg

M (s)i� max

M2MPr(M)V ⇤

M (s)<latexit sha1_base64="xADi9rUOL0idl7C43GEdsrw2RUk=">AAAFcHicfVPdahtHFF4lUZuqf057U8hFJxUhklkbrSkUDIKQOLQ3ojLYTqhWFrO7Z1eD5mczM2tHmcyj9En6JH2NPkHPSEoVydC5kM6e7zv/52Q1Z8YOBn+37t1/0P7s84dfdL786utvvj149N2VUY3O4TJXXOk3GTXAmYRLyyyHN7UGKjIOr7PFy4C/vgFtmJIXdlnDVNBKspLl1KJqdvDX07RmM1drprTvUZIKVhDTJ6dDko71zI1IapggZ56kHEqLDASorgR9N3P0mSfns9H1Yc/E9FmfpJpVc9vvpI0sMCbYrblLs1d+5WJyhRZuFZXeVP760PdMf205JWkFbzGH4BxNmQyynZtcu5H3IaPeqE+u1iH7s4Pu4HiweuSukGyEbrR549mj+3+mhcobAdLmnBozSQa1nTqqLcs5+M5TcnR0RMY0X9AKTPjAWgzUa8VEYOFMDgfHJ0xOXQVKgNVL/ynHgUT3mlqISUhdm9LEBCeQYcNFTKgwQb0S7HytMEuRocArpdlax6vaQIOpqgL91KxU0sbENFnJqpiUDec1xopJbt42ygJGKKiZ64ajzrLF+5hkGfrJlFpYmiEcvrZJNJrvJD0pNRWAxcxVMbxgiz+mThQrXeE7nVTCba6EoLJwKdQGZ3lDNQqMKxl6drGscZ844Uwwe7rLl6hEg/8WQm6WhGqtbsOES7tEPLD8nqVpgiX+zhzDrUv8tdyn1FoVyAl/n5IwpxE2OeSEvJWcZbumIveTZOpW64U810324zNZsNyvGdhNd4eAC/0Rda/2QU0R3Fa6h/KAhmPYgLtoUUDpJydT10kzwIXbDmOiFXYyV1qCHiY17kSGE6w2Wrz14e2cWZh20huDswWXCOE7mGpJzqBkkoWj96TXTfqnBEu0pHuCqQFG3Q48pHMGeB8aQud+r8M6K33oVnfPpMd7qdI4SP9HpO8+ElHa6xxDP8Wmexq4S1+w6oP3eNDJ/vneFa5OjpPBcXL+c/f5i81pP4weRz9FvSiJfomeR79F4+gyyltPWr+2xq3zB/+0f2j/2H6ypt5rbWy+j3Ze+/Bfw1nZDQ==</latexit><latexit sha1_base64="xADi9rUOL0idl7C43GEdsrw2RUk=">AAAFcHicfVPdahtHFF4lUZuqf057U8hFJxUhklkbrSkUDIKQOLQ3ojLYTqhWFrO7Z1eD5mczM2tHmcyj9En6JH2NPkHPSEoVydC5kM6e7zv/52Q1Z8YOBn+37t1/0P7s84dfdL786utvvj149N2VUY3O4TJXXOk3GTXAmYRLyyyHN7UGKjIOr7PFy4C/vgFtmJIXdlnDVNBKspLl1KJqdvDX07RmM1drprTvUZIKVhDTJ6dDko71zI1IapggZ56kHEqLDASorgR9N3P0mSfns9H1Yc/E9FmfpJpVc9vvpI0sMCbYrblLs1d+5WJyhRZuFZXeVP760PdMf205JWkFbzGH4BxNmQyynZtcu5H3IaPeqE+u1iH7s4Pu4HiweuSukGyEbrR549mj+3+mhcobAdLmnBozSQa1nTqqLcs5+M5TcnR0RMY0X9AKTPjAWgzUa8VEYOFMDgfHJ0xOXQVKgNVL/ynHgUT3mlqISUhdm9LEBCeQYcNFTKgwQb0S7HytMEuRocArpdlax6vaQIOpqgL91KxU0sbENFnJqpiUDec1xopJbt42ygJGKKiZ64ajzrLF+5hkGfrJlFpYmiEcvrZJNJrvJD0pNRWAxcxVMbxgiz+mThQrXeE7nVTCba6EoLJwKdQGZ3lDNQqMKxl6drGscZ844Uwwe7rLl6hEg/8WQm6WhGqtbsOES7tEPLD8nqVpgiX+zhzDrUv8tdyn1FoVyAl/n5IwpxE2OeSEvJWcZbumIveTZOpW64U810324zNZsNyvGdhNd4eAC/0Rda/2QU0R3Fa6h/KAhmPYgLtoUUDpJydT10kzwIXbDmOiFXYyV1qCHiY17kSGE6w2Wrz14e2cWZh20huDswWXCOE7mGpJzqBkkoWj96TXTfqnBEu0pHuCqQFG3Q48pHMGeB8aQud+r8M6K33oVnfPpMd7qdI4SP9HpO8+ElHa6xxDP8Wmexq4S1+w6oP3eNDJ/vneFa5OjpPBcXL+c/f5i81pP4weRz9FvSiJfomeR79F4+gyyltPWr+2xq3zB/+0f2j/2H6ypt5rbWy+j3Ze+/Bfw1nZDQ==</latexit><latexit sha1_base64="xADi9rUOL0idl7C43GEdsrw2RUk=">AAAFcHicfVPdahtHFF4lUZuqf057U8hFJxUhklkbrSkUDIKQOLQ3ojLYTqhWFrO7Z1eD5mczM2tHmcyj9En6JH2NPkHPSEoVydC5kM6e7zv/52Q1Z8YOBn+37t1/0P7s84dfdL786utvvj149N2VUY3O4TJXXOk3GTXAmYRLyyyHN7UGKjIOr7PFy4C/vgFtmJIXdlnDVNBKspLl1KJqdvDX07RmM1drprTvUZIKVhDTJ6dDko71zI1IapggZ56kHEqLDASorgR9N3P0mSfns9H1Yc/E9FmfpJpVc9vvpI0sMCbYrblLs1d+5WJyhRZuFZXeVP760PdMf205JWkFbzGH4BxNmQyynZtcu5H3IaPeqE+u1iH7s4Pu4HiweuSukGyEbrR549mj+3+mhcobAdLmnBozSQa1nTqqLcs5+M5TcnR0RMY0X9AKTPjAWgzUa8VEYOFMDgfHJ0xOXQVKgNVL/ynHgUT3mlqISUhdm9LEBCeQYcNFTKgwQb0S7HytMEuRocArpdlax6vaQIOpqgL91KxU0sbENFnJqpiUDec1xopJbt42ygJGKKiZ64ajzrLF+5hkGfrJlFpYmiEcvrZJNJrvJD0pNRWAxcxVMbxgiz+mThQrXeE7nVTCba6EoLJwKdQGZ3lDNQqMKxl6drGscZ844Uwwe7rLl6hEg/8WQm6WhGqtbsOES7tEPLD8nqVpgiX+zhzDrUv8tdyn1FoVyAl/n5IwpxE2OeSEvJWcZbumIveTZOpW64U810324zNZsNyvGdhNd4eAC/0Rda/2QU0R3Fa6h/KAhmPYgLtoUUDpJydT10kzwIXbDmOiFXYyV1qCHiY17kSGE6w2Wrz14e2cWZh20huDswWXCOE7mGpJzqBkkoWj96TXTfqnBEu0pHuCqQFG3Q48pHMGeB8aQud+r8M6K33oVnfPpMd7qdI4SP9HpO8+ElHa6xxDP8Wmexq4S1+w6oP3eNDJ/vneFa5OjpPBcXL+c/f5i81pP4weRz9FvSiJfomeR79F4+gyyltPWr+2xq3zB/+0f2j/2H6ypt5rbWy+j3Ze+/Bfw1nZDQ==</latexit><latexit sha1_base64="xADi9rUOL0idl7C43GEdsrw2RUk=">AAAFcHicfVPdahtHFF4lUZuqf057U8hFJxUhklkbrSkUDIKQOLQ3ojLYTqhWFrO7Z1eD5mczM2tHmcyj9En6JH2NPkHPSEoVydC5kM6e7zv/52Q1Z8YOBn+37t1/0P7s84dfdL786utvvj149N2VUY3O4TJXXOk3GTXAmYRLyyyHN7UGKjIOr7PFy4C/vgFtmJIXdlnDVNBKspLl1KJqdvDX07RmM1drprTvUZIKVhDTJ6dDko71zI1IapggZ56kHEqLDASorgR9N3P0mSfns9H1Yc/E9FmfpJpVc9vvpI0sMCbYrblLs1d+5WJyhRZuFZXeVP760PdMf205JWkFbzGH4BxNmQyynZtcu5H3IaPeqE+u1iH7s4Pu4HiweuSukGyEbrR549mj+3+mhcobAdLmnBozSQa1nTqqLcs5+M5TcnR0RMY0X9AKTPjAWgzUa8VEYOFMDgfHJ0xOXQVKgNVL/ynHgUT3mlqISUhdm9LEBCeQYcNFTKgwQb0S7HytMEuRocArpdlax6vaQIOpqgL91KxU0sbENFnJqpiUDec1xopJbt42ygJGKKiZ64ajzrLF+5hkGfrJlFpYmiEcvrZJNJrvJD0pNRWAxcxVMbxgiz+mThQrXeE7nVTCba6EoLJwKdQGZ3lDNQqMKxl6drGscZ844Uwwe7rLl6hEg/8WQm6WhGqtbsOES7tEPLD8nqVpgiX+zhzDrUv8tdyn1FoVyAl/n5IwpxE2OeSEvJWcZbumIveTZOpW64U810324zNZsNyvGdhNd4eAC/0Rda/2QU0R3Fa6h/KAhmPYgLtoUUDpJydT10kzwIXbDmOiFXYyV1qCHiY17kSGE6w2Wrz14e2cWZh20huDswWXCOE7mGpJzqBkkoWj96TXTfqnBEu0pHuCqQFG3Q48pHMGeB8aQud+r8M6K33oVnfPpMd7qdI4SP9HpO8+ElHa6xxDP8Wmexq4S1+w6oP3eNDJ/vneFa5OjpPBcXL+c/f5i81pP4weRz9FvSiJfomeR79F4+gyyltPWr+2xq3zB/+0f2j/2H6ypt5rbWy+j3Ze+/Bfw1nZDQ==</latexit>

Expected performance of Avg. MDP policy

O

0

@X

(s,a)2S⇥A

max {Qinit(s, a)� V⇤(s), 0}

1

A

<latexit sha1_base64="UfWYGir5PS6p+8gu0yE6VZPXMIw=">AAAF0XicfVRNb9tGEKWSqk3Zrzg95rKoYEQyZEM0ChQwICBJHaAXITZiK0G1srAkl9RC+8HsLu2o2wWKXPsbeu+1/Tf9N50llTqSgfLC4cybmbezb5hWnBk7Gv3TuXf/k+6nnz34PP7iy6++/ubh3qOpUbXO6GWmuNJvUmIoZ5JeWmY5fVNpSkTK6et09WOIv76m2jAlL+y6onNBSskKlhELrsVe5/E+rtjCVZop7fsEYcFyZAboZIzwmV64CcKGCXTqEea0sICAANGlIO8Wjjzx6HwxuTromyF5MkBYs3JpB/E+rmUOXam9zXc4feGbGrMppLimLbku/dWB75tBmzpHuKRvgUSoDqlMBtsuTabdxPtAqT8ZoGnbcxBjy3hO3cu2cB+bWixcIDNocl8hAAhq0DPf1GxQ2J0vHJPM+hZ4iKZNsSEatSSw35xj8bA3Oho1D7prJBujF22es8Xe/T9wrrJaUGkzToyZJaPKzh3RlmWc+ngfHR4eojOSrUgJtOAjxrWhVeuYCRgsk+PR0TGTc1dSJajVa/8xxlEJ5TWxdIjCZLQpzBDBFadwo2KIiDDB3Rh22TrMWqRg8FJp1vp4WRlaA1WVQ52KFUraITJ1WrByiIqa8wp6DVFm3tbKUuiQE7PUNQefZatfhihNoU6q1MqSFMLh65ZErfkW6VmhiaBwmKXKxxds9fPcibzx5T6OsaQ3mRKCyNxhWhmQyjXRYDCuZJjZxboCwXLEmWD2ZBsvwQkJ/+lNbkRItFY3QQSFXUM8oPxOJogFIo1kGKg68VdyF1JplQMmvD4GAacJDDlwAlxjp+l2qsj8LJm7Rr2Ac71ktz+TOct8i4BpujsA2JcPUfdiN6gJBG9PuhPlIRrEvgluR/OcFn52PHcxTikI7vYyZlrBJDOlJdXjpAJNpHCD5cYLP5PxzZJZOo/xtYG7pS4RwsdAtUCntAhLBX8Vj/q9ZHCC4IgW9Y6BGoWutxce6JxS2A9Nw+ReVkHOSh+45r/CpId9KfEwWP8HJO8+AMHamRyDOvlmeppyh5+z8lfvYaGT3fW9a0yPj5LRUXL+fe/p881qP4geR99F/SiJfoieRj9FZ9FllHXed/7s/NX5u/uqu+7+1n3fQu91NjnfRltP9/d/AZ+p+9g=</latexit><latexit sha1_base64="UfWYGir5PS6p+8gu0yE6VZPXMIw=">AAAF0XicfVRNb9tGEKWSqk3Zrzg95rKoYEQyZEM0ChQwICBJHaAXITZiK0G1srAkl9RC+8HsLu2o2wWKXPsbeu+1/Tf9N50llTqSgfLC4cybmbezb5hWnBk7Gv3TuXf/k+6nnz34PP7iy6++/ubh3qOpUbXO6GWmuNJvUmIoZ5JeWmY5fVNpSkTK6et09WOIv76m2jAlL+y6onNBSskKlhELrsVe5/E+rtjCVZop7fsEYcFyZAboZIzwmV64CcKGCXTqEea0sICAANGlIO8Wjjzx6HwxuTromyF5MkBYs3JpB/E+rmUOXam9zXc4feGbGrMppLimLbku/dWB75tBmzpHuKRvgUSoDqlMBtsuTabdxPtAqT8ZoGnbcxBjy3hO3cu2cB+bWixcIDNocl8hAAhq0DPf1GxQ2J0vHJPM+hZ4iKZNsSEatSSw35xj8bA3Oho1D7prJBujF22es8Xe/T9wrrJaUGkzToyZJaPKzh3RlmWc+ngfHR4eojOSrUgJtOAjxrWhVeuYCRgsk+PR0TGTc1dSJajVa/8xxlEJ5TWxdIjCZLQpzBDBFadwo2KIiDDB3Rh22TrMWqRg8FJp1vp4WRlaA1WVQ52KFUraITJ1WrByiIqa8wp6DVFm3tbKUuiQE7PUNQefZatfhihNoU6q1MqSFMLh65ZErfkW6VmhiaBwmKXKxxds9fPcibzx5T6OsaQ3mRKCyNxhWhmQyjXRYDCuZJjZxboCwXLEmWD2ZBsvwQkJ/+lNbkRItFY3QQSFXUM8oPxOJogFIo1kGKg68VdyF1JplQMmvD4GAacJDDlwAlxjp+l2qsj8LJm7Rr2Ac71ktz+TOct8i4BpujsA2JcPUfdiN6gJBG9PuhPlIRrEvgluR/OcFn52PHcxTikI7vYyZlrBJDOlJdXjpAJNpHCD5cYLP5PxzZJZOo/xtYG7pS4RwsdAtUCntAhLBX8Vj/q9ZHCC4IgW9Y6BGoWutxce6JxS2A9Nw+ReVkHOSh+45r/CpId9KfEwWP8HJO8+AMHamRyDOvlmeppyh5+z8lfvYaGT3fW9a0yPj5LRUXL+fe/p881qP4geR99F/SiJfoieRj9FZ9FllHXed/7s/NX5u/uqu+7+1n3fQu91NjnfRltP9/d/AZ+p+9g=</latexit><latexit sha1_base64="UfWYGir5PS6p+8gu0yE6VZPXMIw=">AAAF0XicfVRNb9tGEKWSqk3Zrzg95rKoYEQyZEM0ChQwICBJHaAXITZiK0G1srAkl9RC+8HsLu2o2wWKXPsbeu+1/Tf9N50llTqSgfLC4cybmbezb5hWnBk7Gv3TuXf/k+6nnz34PP7iy6++/ubh3qOpUbXO6GWmuNJvUmIoZ5JeWmY5fVNpSkTK6et09WOIv76m2jAlL+y6onNBSskKlhELrsVe5/E+rtjCVZop7fsEYcFyZAboZIzwmV64CcKGCXTqEea0sICAANGlIO8Wjjzx6HwxuTromyF5MkBYs3JpB/E+rmUOXam9zXc4feGbGrMppLimLbku/dWB75tBmzpHuKRvgUSoDqlMBtsuTabdxPtAqT8ZoGnbcxBjy3hO3cu2cB+bWixcIDNocl8hAAhq0DPf1GxQ2J0vHJPM+hZ4iKZNsSEatSSw35xj8bA3Oho1D7prJBujF22es8Xe/T9wrrJaUGkzToyZJaPKzh3RlmWc+ngfHR4eojOSrUgJtOAjxrWhVeuYCRgsk+PR0TGTc1dSJajVa/8xxlEJ5TWxdIjCZLQpzBDBFadwo2KIiDDB3Rh22TrMWqRg8FJp1vp4WRlaA1WVQ52KFUraITJ1WrByiIqa8wp6DVFm3tbKUuiQE7PUNQefZatfhihNoU6q1MqSFMLh65ZErfkW6VmhiaBwmKXKxxds9fPcibzx5T6OsaQ3mRKCyNxhWhmQyjXRYDCuZJjZxboCwXLEmWD2ZBsvwQkJ/+lNbkRItFY3QQSFXUM8oPxOJogFIo1kGKg68VdyF1JplQMmvD4GAacJDDlwAlxjp+l2qsj8LJm7Rr2Ac71ktz+TOct8i4BpujsA2JcPUfdiN6gJBG9PuhPlIRrEvgluR/OcFn52PHcxTikI7vYyZlrBJDOlJdXjpAJNpHCD5cYLP5PxzZJZOo/xtYG7pS4RwsdAtUCntAhLBX8Vj/q9ZHCC4IgW9Y6BGoWutxce6JxS2A9Nw+ReVkHOSh+45r/CpId9KfEwWP8HJO8+AMHamRyDOvlmeppyh5+z8lfvYaGT3fW9a0yPj5LRUXL+fe/p881qP4geR99F/SiJfoieRj9FZ9FllHXed/7s/NX5u/uqu+7+1n3fQu91NjnfRltP9/d/AZ+p+9g=</latexit><latexit sha1_base64="UfWYGir5PS6p+8gu0yE6VZPXMIw=">AAAF0XicfVRNb9tGEKWSqk3Zrzg95rKoYEQyZEM0ChQwICBJHaAXITZiK0G1srAkl9RC+8HsLu2o2wWKXPsbeu+1/Tf9N50llTqSgfLC4cybmbezb5hWnBk7Gv3TuXf/k+6nnz34PP7iy6++/ubh3qOpUbXO6GWmuNJvUmIoZ5JeWmY5fVNpSkTK6et09WOIv76m2jAlL+y6onNBSskKlhELrsVe5/E+rtjCVZop7fsEYcFyZAboZIzwmV64CcKGCXTqEea0sICAANGlIO8Wjjzx6HwxuTromyF5MkBYs3JpB/E+rmUOXam9zXc4feGbGrMppLimLbku/dWB75tBmzpHuKRvgUSoDqlMBtsuTabdxPtAqT8ZoGnbcxBjy3hO3cu2cB+bWixcIDNocl8hAAhq0DPf1GxQ2J0vHJPM+hZ4iKZNsSEatSSw35xj8bA3Oho1D7prJBujF22es8Xe/T9wrrJaUGkzToyZJaPKzh3RlmWc+ngfHR4eojOSrUgJtOAjxrWhVeuYCRgsk+PR0TGTc1dSJajVa/8xxlEJ5TWxdIjCZLQpzBDBFadwo2KIiDDB3Rh22TrMWqRg8FJp1vp4WRlaA1WVQ52KFUraITJ1WrByiIqa8wp6DVFm3tbKUuiQE7PUNQefZatfhihNoU6q1MqSFMLh65ZErfkW6VmhiaBwmKXKxxds9fPcibzx5T6OsaQ3mRKCyNxhWhmQyjXRYDCuZJjZxboCwXLEmWD2ZBsvwQkJ/+lNbkRItFY3QQSFXUM8oPxOJogFIo1kGKg68VdyF1JplQMmvD4GAacJDDlwAlxjp+l2qsj8LJm7Rr2Ac71ktz+TOct8i4BpujsA2JcPUfdiN6gJBG9PuhPlIRrEvgluR/OcFn52PHcxTikI7vYyZlrBJDOlJdXjpAJNpHCD5cYLP5PxzZJZOo/xtYG7pS4RwsdAtUCntAhLBX8Vj/q9ZHCC4IgW9Y6BGoWutxce6JxS2A9Nw+ReVkHOSh+45r/CpId9KfEwWP8HJO8+AMHamRyDOvlmeppyh5+z8lfvYaGT3fW9a0yPj5LRUXL+fe/p881qP4geR99F/SiJfoieRj9FZ9FllHXed/7s/NX5u/uqu+7+1n3fQu91NjnfRltP9/d/AZ+p+9g=</latexit>

MaxQInit(s, a) := maxM2{M1,...,Mn}

bQM (s, a)<latexit sha1_base64="D7cijIxrmR46vujUUkmWJbW3wcM=">AAAGIXicfVRfb9s2EJfTeeu8P226x70QM4LagRJYwYACAQx0XQpsD0ZjLHGLmY5ASZRMWCRVkkricfwoe9qn2dvQt2FfZI87ys4cO8D4otPd7+5+d7xjUpVMm8Hgr9beo4/aH3/y+NPOZ59/8eWTp/vPJlrWKqWXqSylepcQTUsm6KVhpqTvKkUJT0r6Nll87+1vr6nSTIoLs6zojJNCsJylxIAq3m8tDnDFYlspJpXrEYQ5y5Duo9MhwucqtiOENePozCFc0twAAgxEFZzcxpY8d2gcj64Oezokz/sIK1bMTb9zgGuRQVZqNv4WJ69dE2M6ARfbpCXXhbs6dD3dX7nOEC7oeyDho4MrE142c50qO3LOU+qN+miyyukTGVZm1L5ZRe5hXfPYejb9xvknBABONfrONUEbFLbj2DLBjFsBj9CkiRaiwYoFdneFYENvjU7tiNyOf9x4+Obcp2hHcRRCgzJpdIhGscCe6w3L6JwYO3bxqPGLn3YHx4PmoIdCtBa6wfqcx/uPfsOZTGtOhUlLovU0GlRmZokyLC2p6xygo6MjdE7SBSmgSvjp4FrTaqWYcrgoJoaD4xMmZragklOjlu4+xlIB4RUxNES+00rnUAKMTAITwkNEuPbqRjDzlUIveQJCWUjFVrqyqDStgarMIE7FcilMiHSd5KwIUV6XZQW5QpTq97U0FDJkRM9VXYLOsMUvIUoSiJNIuTAkAbP/25CoVblFeporwikUM5fZ8IItfp5ZnjW6zHU6WNCbVHJORGYxrTSM3jVRILBSCt+zi2UFC1CiknFmTrfxApTg8N/8ivVQE6Xkjb/t3CzB7lFuxxNmDyzNBDLYkshdiV1IpWQGGP+5DwJOI2iy5wS4Rk6SbVeeumk0s802AM52o938TGQsdSsEdNM+AMD+3Vnt612jImDcVLpjLb3V787auG3NMpq76cnMdnBCYeA2lzFVEjqZSiWoGkYVzEQCN1istfA4DW/mzNBZB19ruFtqI85dB6jm6IzmfkfhlXKo1436pwhKNKh7AtQoZN1cuKdzRmE/FPWde1P5cZbq0DbvFBMO9qXAoZf+D0hu74Ag7XSOQZxs3T1FS4tfseJX52Cho931fShMTo6jwXE0/rb78tV6tR8HXwffBL0gCl4EL4MfgvPgMkhbH1r/7LX29tq/t/9o/9n+sILutdY+XwVbp/33v7IXGFw=</latexit><latexit sha1_base64="D7cijIxrmR46vujUUkmWJbW3wcM=">AAAGIXicfVRfb9s2EJfTeeu8P226x70QM4LagRJYwYACAQx0XQpsD0ZjLHGLmY5ASZRMWCRVkkricfwoe9qn2dvQt2FfZI87ys4cO8D4otPd7+5+d7xjUpVMm8Hgr9beo4/aH3/y+NPOZ59/8eWTp/vPJlrWKqWXqSylepcQTUsm6KVhpqTvKkUJT0r6Nll87+1vr6nSTIoLs6zojJNCsJylxIAq3m8tDnDFYlspJpXrEYQ5y5Duo9MhwucqtiOENePozCFc0twAAgxEFZzcxpY8d2gcj64Oezokz/sIK1bMTb9zgGuRQVZqNv4WJ69dE2M6ARfbpCXXhbs6dD3dX7nOEC7oeyDho4MrE142c50qO3LOU+qN+miyyukTGVZm1L5ZRe5hXfPYejb9xvknBABONfrONUEbFLbj2DLBjFsBj9CkiRaiwYoFdneFYENvjU7tiNyOf9x4+Obcp2hHcRRCgzJpdIhGscCe6w3L6JwYO3bxqPGLn3YHx4PmoIdCtBa6wfqcx/uPfsOZTGtOhUlLovU0GlRmZokyLC2p6xygo6MjdE7SBSmgSvjp4FrTaqWYcrgoJoaD4xMmZragklOjlu4+xlIB4RUxNES+00rnUAKMTAITwkNEuPbqRjDzlUIveQJCWUjFVrqyqDStgarMIE7FcilMiHSd5KwIUV6XZQW5QpTq97U0FDJkRM9VXYLOsMUvIUoSiJNIuTAkAbP/25CoVblFeporwikUM5fZ8IItfp5ZnjW6zHU6WNCbVHJORGYxrTSM3jVRILBSCt+zi2UFC1CiknFmTrfxApTg8N/8ivVQE6Xkjb/t3CzB7lFuxxNmDyzNBDLYkshdiV1IpWQGGP+5DwJOI2iy5wS4Rk6SbVeeumk0s802AM52o938TGQsdSsEdNM+AMD+3Vnt612jImDcVLpjLb3V787auG3NMpq76cnMdnBCYeA2lzFVEjqZSiWoGkYVzEQCN1istfA4DW/mzNBZB19ruFtqI85dB6jm6IzmfkfhlXKo1436pwhKNKh7AtQoZN1cuKdzRmE/FPWde1P5cZbq0DbvFBMO9qXAoZf+D0hu74Ag7XSOQZxs3T1FS4tfseJX52Cho931fShMTo6jwXE0/rb78tV6tR8HXwffBL0gCl4EL4MfgvPgMkhbH1r/7LX29tq/t/9o/9n+sILutdY+XwVbp/33v7IXGFw=</latexit><latexit sha1_base64="D7cijIxrmR46vujUUkmWJbW3wcM=">AAAGIXicfVRfb9s2EJfTeeu8P226x70QM4LagRJYwYACAQx0XQpsD0ZjLHGLmY5ASZRMWCRVkkricfwoe9qn2dvQt2FfZI87ys4cO8D4otPd7+5+d7xjUpVMm8Hgr9beo4/aH3/y+NPOZ59/8eWTp/vPJlrWKqWXqSylepcQTUsm6KVhpqTvKkUJT0r6Nll87+1vr6nSTIoLs6zojJNCsJylxIAq3m8tDnDFYlspJpXrEYQ5y5Duo9MhwucqtiOENePozCFc0twAAgxEFZzcxpY8d2gcj64Oezokz/sIK1bMTb9zgGuRQVZqNv4WJ69dE2M6ARfbpCXXhbs6dD3dX7nOEC7oeyDho4MrE142c50qO3LOU+qN+miyyukTGVZm1L5ZRe5hXfPYejb9xvknBABONfrONUEbFLbj2DLBjFsBj9CkiRaiwYoFdneFYENvjU7tiNyOf9x4+Obcp2hHcRRCgzJpdIhGscCe6w3L6JwYO3bxqPGLn3YHx4PmoIdCtBa6wfqcx/uPfsOZTGtOhUlLovU0GlRmZokyLC2p6xygo6MjdE7SBSmgSvjp4FrTaqWYcrgoJoaD4xMmZragklOjlu4+xlIB4RUxNES+00rnUAKMTAITwkNEuPbqRjDzlUIveQJCWUjFVrqyqDStgarMIE7FcilMiHSd5KwIUV6XZQW5QpTq97U0FDJkRM9VXYLOsMUvIUoSiJNIuTAkAbP/25CoVblFeporwikUM5fZ8IItfp5ZnjW6zHU6WNCbVHJORGYxrTSM3jVRILBSCt+zi2UFC1CiknFmTrfxApTg8N/8ivVQE6Xkjb/t3CzB7lFuxxNmDyzNBDLYkshdiV1IpWQGGP+5DwJOI2iy5wS4Rk6SbVeeumk0s802AM52o938TGQsdSsEdNM+AMD+3Vnt612jImDcVLpjLb3V787auG3NMpq76cnMdnBCYeA2lzFVEjqZSiWoGkYVzEQCN1istfA4DW/mzNBZB19ruFtqI85dB6jm6IzmfkfhlXKo1436pwhKNKh7AtQoZN1cuKdzRmE/FPWde1P5cZbq0DbvFBMO9qXAoZf+D0hu74Ag7XSOQZxs3T1FS4tfseJX52Cho931fShMTo6jwXE0/rb78tV6tR8HXwffBL0gCl4EL4MfgvPgMkhbH1r/7LX29tq/t/9o/9n+sILutdY+XwVbp/33v7IXGFw=</latexit><latexit sha1_base64="D7cijIxrmR46vujUUkmWJbW3wcM=">AAAGIXicfVRfb9s2EJfTeeu8P226x70QM4LagRJYwYACAQx0XQpsD0ZjLHGLmY5ASZRMWCRVkkricfwoe9qn2dvQt2FfZI87ys4cO8D4otPd7+5+d7xjUpVMm8Hgr9beo4/aH3/y+NPOZ59/8eWTp/vPJlrWKqWXqSylepcQTUsm6KVhpqTvKkUJT0r6Nll87+1vr6nSTIoLs6zojJNCsJylxIAq3m8tDnDFYlspJpXrEYQ5y5Duo9MhwucqtiOENePozCFc0twAAgxEFZzcxpY8d2gcj64Oezokz/sIK1bMTb9zgGuRQVZqNv4WJ69dE2M6ARfbpCXXhbs6dD3dX7nOEC7oeyDho4MrE142c50qO3LOU+qN+miyyukTGVZm1L5ZRe5hXfPYejb9xvknBABONfrONUEbFLbj2DLBjFsBj9CkiRaiwYoFdneFYENvjU7tiNyOf9x4+Obcp2hHcRRCgzJpdIhGscCe6w3L6JwYO3bxqPGLn3YHx4PmoIdCtBa6wfqcx/uPfsOZTGtOhUlLovU0GlRmZokyLC2p6xygo6MjdE7SBSmgSvjp4FrTaqWYcrgoJoaD4xMmZragklOjlu4+xlIB4RUxNES+00rnUAKMTAITwkNEuPbqRjDzlUIveQJCWUjFVrqyqDStgarMIE7FcilMiHSd5KwIUV6XZQW5QpTq97U0FDJkRM9VXYLOsMUvIUoSiJNIuTAkAbP/25CoVblFeporwikUM5fZ8IItfp5ZnjW6zHU6WNCbVHJORGYxrTSM3jVRILBSCt+zi2UFC1CiknFmTrfxApTg8N/8ivVQE6Xkjb/t3CzB7lFuxxNmDyzNBDLYkshdiV1IpWQGGP+5DwJOI2iy5wS4Rk6SbVeeumk0s802AM52o938TGQsdSsEdNM+AMD+3Vnt612jImDcVLpjLb3V787auG3NMpq76cnMdnBCYeA2lzFVEjqZSiWoGkYVzEQCN1istfA4DW/mzNBZB19ruFtqI85dB6jm6IzmfkfhlXKo1436pwhKNKh7AtQoZN1cuKdzRmE/FPWde1P5cZbq0DbvFBMO9qXAoZf+D0hu74Ag7XSOQZxs3T1FS4tfseJX52Cho931fShMTo6jwXE0/rb78tV6tR8HXwffBL0gCl4EL4MfgvPgMkhbH1r/7LX29tq/t/9o/9n+sILutdY+XwVbp/33v7IXGFw=</latexit>

MaxQInit<latexit sha1_base64="DW8Ia6iCuPIoeNK7vIpp0ac1ec0=">AAAGNHicfVRfb9s2EFfceevc/Wna7WkvxIygdqAEVjBgQAADXZcC24PRGEvcYqYjUBIlExZJlaSSeBz3TfY19l0G7G3o6z7DjrI7xw4wvuh097u73x3vmFQl02Yw+HOv9eCD9ocfPfy48+iTTz/7/PH+k4mWtUrpZSpLqd4kRNOSCXppmCnpm0pRwpOSvk4W33v762uqNJPiwiwrOuOkECxnKTGgivf3fjvAFYttpZhUrkcQ5ixDuo9Ohwifq9iOENaMozOHcElzAwgwEFVwchtb8syhcTy6OuzpkDzrI6xYMTf9zgGuRQZZqdn4W5y8dE2M6QRcbJOWXBfu6tD1dH/lOkO4oG+BhI8Orkx42cx1quzIOU+pN+qjySqnT2RYmVH7ahW5h3XNY+vZ9BvnnxAAONXoO9cEbVDYjmPLBDNuBTxCkyZaiAYrFthtCjH01ujUjsjt+MeNi+/OXY52FEchdCiTRodoFAvsyd6wjM6JsWMXjxq/zr1w8ePu4HjQHHRfiNZCN1if83j/we84k2nNqTBpSbSeRoPKzCxRhqUldZ0DdHR0hM5JuiAFFA4/HVxrWq0UUw53x8RwcHzCxMwWVHJq1NLdxVgqILwihobIN1/pHIqCKUpgaHiICNde3QhmvlLoJU9AKAup2EpXFpWmNVCVGcSpWC6FCZGuk5wVIcrrsqwgV4hS/baWhkKGjOi5qkvQGbb4JURJAnESKReGJGD2fxsStSq3SE9zRTiFYuYyG16wxc8zy7NGl7lOBwt6k0rOicgsppWGabwmCgRWSuF7drGsYCdKVDLOzOk2XoASHP4babGec6KUvPH3n5sl2D3K7XjCOIKlGUoGixO5K7ELqZTMAOM/d0HAaQRN9pwA18hJsu3KUzeNZrZZEMDZbrSbn4mMpW6FgG7aewBYyfdW+3LXqAgYN5XuWEtv9eu0Nm5bs4zmbnoysx2cUBi4zWVMlYROplIJqoZRBTORwA0Way28V8ObOTN01sHXGu6W2ohz1wGqOTqjuV9beLgc6nWj/imCEg3qngA1Clk3F+7pnFHYD0V9515VfpylOrTN08WEg30pcOil/wOS2/dAkHY6xyBOtu6eoqXFL1jxq/MLHe2u731hcnIcDY6j8Tfd5y/Wq/0w+Cr4OugFUfBt8Dz4ITgPLoN0713rUeuL1pftP9p/tf9uv1tBW3trn6fB1mn/8y/xUh9E</latexit><latexit sha1_base64="DW8Ia6iCuPIoeNK7vIpp0ac1ec0=">AAAGNHicfVRfb9s2EFfceevc/Wna7WkvxIygdqAEVjBgQAADXZcC24PRGEvcYqYjUBIlExZJlaSSeBz3TfY19l0G7G3o6z7DjrI7xw4wvuh097u73x3vmFQl02Yw+HOv9eCD9ocfPfy48+iTTz/7/PH+k4mWtUrpZSpLqd4kRNOSCXppmCnpm0pRwpOSvk4W33v762uqNJPiwiwrOuOkECxnKTGgivf3fjvAFYttpZhUrkcQ5ixDuo9Ohwifq9iOENaMozOHcElzAwgwEFVwchtb8syhcTy6OuzpkDzrI6xYMTf9zgGuRQZZqdn4W5y8dE2M6QRcbJOWXBfu6tD1dH/lOkO4oG+BhI8Orkx42cx1quzIOU+pN+qjySqnT2RYmVH7ahW5h3XNY+vZ9BvnnxAAONXoO9cEbVDYjmPLBDNuBTxCkyZaiAYrFthtCjH01ujUjsjt+MeNi+/OXY52FEchdCiTRodoFAvsyd6wjM6JsWMXjxq/zr1w8ePu4HjQHHRfiNZCN1if83j/we84k2nNqTBpSbSeRoPKzCxRhqUldZ0DdHR0hM5JuiAFFA4/HVxrWq0UUw53x8RwcHzCxMwWVHJq1NLdxVgqILwihobIN1/pHIqCKUpgaHiICNde3QhmvlLoJU9AKAup2EpXFpWmNVCVGcSpWC6FCZGuk5wVIcrrsqwgV4hS/baWhkKGjOi5qkvQGbb4JURJAnESKReGJGD2fxsStSq3SE9zRTiFYuYyG16wxc8zy7NGl7lOBwt6k0rOicgsppWGabwmCgRWSuF7drGsYCdKVDLOzOk2XoASHP4babGec6KUvPH3n5sl2D3K7XjCOIKlGUoGixO5K7ELqZTMAOM/d0HAaQRN9pwA18hJsu3KUzeNZrZZEMDZbrSbn4mMpW6FgG7aewBYyfdW+3LXqAgYN5XuWEtv9eu0Nm5bs4zmbnoysx2cUBi4zWVMlYROplIJqoZRBTORwA0Way28V8ObOTN01sHXGu6W2ohz1wGqOTqjuV9beLgc6nWj/imCEg3qngA1Clk3F+7pnFHYD0V9515VfpylOrTN08WEg30pcOil/wOS2/dAkHY6xyBOtu6eoqXFL1jxq/MLHe2u731hcnIcDY6j8Tfd5y/Wq/0w+Cr4OugFUfBt8Dz4ITgPLoN0713rUeuL1pftP9p/tf9uv1tBW3trn6fB1mn/8y/xUh9E</latexit><latexit sha1_base64="DW8Ia6iCuPIoeNK7vIpp0ac1ec0=">AAAGNHicfVRfb9s2EFfceevc/Wna7WkvxIygdqAEVjBgQAADXZcC24PRGEvcYqYjUBIlExZJlaSSeBz3TfY19l0G7G3o6z7DjrI7xw4wvuh097u73x3vmFQl02Yw+HOv9eCD9ocfPfy48+iTTz/7/PH+k4mWtUrpZSpLqd4kRNOSCXppmCnpm0pRwpOSvk4W33v762uqNJPiwiwrOuOkECxnKTGgivf3fjvAFYttpZhUrkcQ5ixDuo9Ohwifq9iOENaMozOHcElzAwgwEFVwchtb8syhcTy6OuzpkDzrI6xYMTf9zgGuRQZZqdn4W5y8dE2M6QRcbJOWXBfu6tD1dH/lOkO4oG+BhI8Orkx42cx1quzIOU+pN+qjySqnT2RYmVH7ahW5h3XNY+vZ9BvnnxAAONXoO9cEbVDYjmPLBDNuBTxCkyZaiAYrFthtCjH01ujUjsjt+MeNi+/OXY52FEchdCiTRodoFAvsyd6wjM6JsWMXjxq/zr1w8ePu4HjQHHRfiNZCN1if83j/we84k2nNqTBpSbSeRoPKzCxRhqUldZ0DdHR0hM5JuiAFFA4/HVxrWq0UUw53x8RwcHzCxMwWVHJq1NLdxVgqILwihobIN1/pHIqCKUpgaHiICNde3QhmvlLoJU9AKAup2EpXFpWmNVCVGcSpWC6FCZGuk5wVIcrrsqwgV4hS/baWhkKGjOi5qkvQGbb4JURJAnESKReGJGD2fxsStSq3SE9zRTiFYuYyG16wxc8zy7NGl7lOBwt6k0rOicgsppWGabwmCgRWSuF7drGsYCdKVDLOzOk2XoASHP4babGec6KUvPH3n5sl2D3K7XjCOIKlGUoGixO5K7ELqZTMAOM/d0HAaQRN9pwA18hJsu3KUzeNZrZZEMDZbrSbn4mMpW6FgG7aewBYyfdW+3LXqAgYN5XuWEtv9eu0Nm5bs4zmbnoysx2cUBi4zWVMlYROplIJqoZRBTORwA0Way28V8ObOTN01sHXGu6W2ohz1wGqOTqjuV9beLgc6nWj/imCEg3qngA1Clk3F+7pnFHYD0V9515VfpylOrTN08WEg30pcOil/wOS2/dAkHY6xyBOtu6eoqXFL1jxq/MLHe2u731hcnIcDY6j8Tfd5y/Wq/0w+Cr4OugFUfBt8Dz4ITgPLoN0713rUeuL1pftP9p/tf9uv1tBW3trn6fB1mn/8y/xUh9E</latexit><latexit sha1_base64="DW8Ia6iCuPIoeNK7vIpp0ac1ec0=">AAAGNHicfVRfb9s2EFfceevc/Wna7WkvxIygdqAEVjBgQAADXZcC24PRGEvcYqYjUBIlExZJlaSSeBz3TfY19l0G7G3o6z7DjrI7xw4wvuh097u73x3vmFQl02Yw+HOv9eCD9ocfPfy48+iTTz/7/PH+k4mWtUrpZSpLqd4kRNOSCXppmCnpm0pRwpOSvk4W33v762uqNJPiwiwrOuOkECxnKTGgivf3fjvAFYttpZhUrkcQ5ixDuo9Ohwifq9iOENaMozOHcElzAwgwEFVwchtb8syhcTy6OuzpkDzrI6xYMTf9zgGuRQZZqdn4W5y8dE2M6QRcbJOWXBfu6tD1dH/lOkO4oG+BhI8Orkx42cx1quzIOU+pN+qjySqnT2RYmVH7ahW5h3XNY+vZ9BvnnxAAONXoO9cEbVDYjmPLBDNuBTxCkyZaiAYrFthtCjH01ujUjsjt+MeNi+/OXY52FEchdCiTRodoFAvsyd6wjM6JsWMXjxq/zr1w8ePu4HjQHHRfiNZCN1if83j/we84k2nNqTBpSbSeRoPKzCxRhqUldZ0DdHR0hM5JuiAFFA4/HVxrWq0UUw53x8RwcHzCxMwWVHJq1NLdxVgqILwihobIN1/pHIqCKUpgaHiICNde3QhmvlLoJU9AKAup2EpXFpWmNVCVGcSpWC6FCZGuk5wVIcrrsqwgV4hS/baWhkKGjOi5qkvQGbb4JURJAnESKReGJGD2fxsStSq3SE9zRTiFYuYyG16wxc8zy7NGl7lOBwt6k0rOicgsppWGabwmCgRWSuF7drGsYCdKVDLOzOk2XoASHP4babGec6KUvPH3n5sl2D3K7XjCOIKlGUoGixO5K7ELqZTMAOM/d0HAaQRN9pwA18hJsu3KUzeNZrZZEMDZbrSbn4mMpW6FgG7aewBYyfdW+3LXqAgYN5XuWEtv9eu0Nm5bs4zmbnoysx2cUBi4zWVMlYROplIJqoZRBTORwA0Way28V8ObOTN01sHXGu6W2ohz1wGqOTqjuV9beLgc6nWj/imCEg3qngA1Clk3F+7pnFHYD0V9515VfpylOrTN08WEg30pcOil/wOS2/dAkHY6xyBOtu6eoqXFL1jxq/MLHe2u731hcnIcDY6j8Tfd5y/Wq/0w+Cr4OugFUfBt8Dz4ITgPLoN0713rUeuL1pftP9p/tf9uv1tBW3trn6fB1mn/8y/xUh9E</latexit>

1. Optimal Fixed Policy 2. Informed Transfer for RL

Task 1 Task 2 Task 3

⇡<latexit sha1_base64="bg6iQpcsz52Z46e+zRQZx1hbgxs=">AAAG93icfVVLj9xEEHYCGYJ5JXDk0mKEFNDsajxCihRppCgPiQOIBXaTiPGwarfLnmb64XS3dxka/wSucOSGuPJzuPBbqLa9eMe7wdJMl6u+qv6qqqudVYJbN5//fePma6/fmrxx+834rbffefe9O3fff2Z1bRicMC20eZFRC4IrOHHcCXhRGaAyE/A82z4O9udnYCzX6tjtKlhLWipecEYdqr5NK356Zzo/nLcPuSokvTCN+ufo9O6tf9Jcs1qCckxQa1fJvHJrT43jTEATf0wODg7IEWVbWoINL3FaW6g6xUpSU3K1nB8uuFr7ErQEZ3bNZYwHheENdTAjkrqNsYWdEcwjQ9pyRqi0Qd0KbtMp7E5mKIhSG97pRFlZqJGqzjFOxQut3IzYOit4OSNFLUSFe80Isy9r7QB3yKndmFqgzvHtTzOSZRgn03rraIbm8DaQqI3YI70qDJWAyWx0vjzm2+/WXuatLm/iOFVwzrSUVOU+hco2Pj2jBgUutAo1O95V2BVBBJfcPdjHK1SiQ61ybCU4r0hqeLlx1Bh9TlKuCrdDe0A1I09bB0/8P/WcLEnSfK/GkMroHDFhuQxCTl9ikQMnxLVylu27StaskjWuHc5Pk/H+XOWcNR0Cq+mvALKn/1n907HRUDQOmY6sIlgFFBfGfWueQ9GsFmsfpxnggRuasTIaK8m0UWCWSYVnIsMOlr0WJ2Z5vuEO1nF6ZrG34BMpmxipFuQJFFzxMDoNuTdNPnlAMEVHpgukBrjr0PBA5wngfBgIlfuqCsdZm099isdfctXgvJTpLEj/B6Q/XgBRGlWOY5y8r54B4dNHvPy56evgNqANSM+0+gGYqw00/vEgrywKmMZ6D4w/araN/6Zbrwf1a+OPe+F6GNZXC8xqF/a9EFe9dR8qAFMabP6L9n3Uz7YVYTgx466hDC8gME26uRhbv2BICzva/bUd6UGjqnCFvWe1dVriXTGk0oJAnXGjVbjf/IDBgx4TbLeBgVWK8fZDhRFIR9FbN+RyRf0qSlg6c7lqr2DVwq6lNYo1JtU6XsOq08f4RUjG9/9V4dniMJkfJl9/Nn34qP823I4+jD6K7kVJdD96GH0eHUUnEYvK6Jfo1+i3yW7y++SPyZ8d9OaN3ueDaO+Z/PUv9JeECg==</latexit><latexit sha1_base64="bg6iQpcsz52Z46e+zRQZx1hbgxs=">AAAG93icfVVLj9xEEHYCGYJ5JXDk0mKEFNDsajxCihRppCgPiQOIBXaTiPGwarfLnmb64XS3dxka/wSucOSGuPJzuPBbqLa9eMe7wdJMl6u+qv6qqqudVYJbN5//fePma6/fmrxx+834rbffefe9O3fff2Z1bRicMC20eZFRC4IrOHHcCXhRGaAyE/A82z4O9udnYCzX6tjtKlhLWipecEYdqr5NK356Zzo/nLcPuSokvTCN+ufo9O6tf9Jcs1qCckxQa1fJvHJrT43jTEATf0wODg7IEWVbWoINL3FaW6g6xUpSU3K1nB8uuFr7ErQEZ3bNZYwHheENdTAjkrqNsYWdEcwjQ9pyRqi0Qd0KbtMp7E5mKIhSG97pRFlZqJGqzjFOxQut3IzYOit4OSNFLUSFe80Isy9r7QB3yKndmFqgzvHtTzOSZRgn03rraIbm8DaQqI3YI70qDJWAyWx0vjzm2+/WXuatLm/iOFVwzrSUVOU+hco2Pj2jBgUutAo1O95V2BVBBJfcPdjHK1SiQ61ybCU4r0hqeLlx1Bh9TlKuCrdDe0A1I09bB0/8P/WcLEnSfK/GkMroHDFhuQxCTl9ikQMnxLVylu27StaskjWuHc5Pk/H+XOWcNR0Cq+mvALKn/1n907HRUDQOmY6sIlgFFBfGfWueQ9GsFmsfpxnggRuasTIaK8m0UWCWSYVnIsMOlr0WJ2Z5vuEO1nF6ZrG34BMpmxipFuQJFFzxMDoNuTdNPnlAMEVHpgukBrjr0PBA5wngfBgIlfuqCsdZm099isdfctXgvJTpLEj/B6Q/XgBRGlWOY5y8r54B4dNHvPy56evgNqANSM+0+gGYqw00/vEgrywKmMZ6D4w/araN/6Zbrwf1a+OPe+F6GNZXC8xqF/a9EFe9dR8qAFMabP6L9n3Uz7YVYTgx466hDC8gME26uRhbv2BICzva/bUd6UGjqnCFvWe1dVriXTGk0oJAnXGjVbjf/IDBgx4TbLeBgVWK8fZDhRFIR9FbN+RyRf0qSlg6c7lqr2DVwq6lNYo1JtU6XsOq08f4RUjG9/9V4dniMJkfJl9/Nn34qP823I4+jD6K7kVJdD96GH0eHUUnEYvK6Jfo1+i3yW7y++SPyZ8d9OaN3ueDaO+Z/PUv9JeECg==</latexit><latexit sha1_base64="bg6iQpcsz52Z46e+zRQZx1hbgxs=">AAAG93icfVVLj9xEEHYCGYJ5JXDk0mKEFNDsajxCihRppCgPiQOIBXaTiPGwarfLnmb64XS3dxka/wSucOSGuPJzuPBbqLa9eMe7wdJMl6u+qv6qqqudVYJbN5//fePma6/fmrxx+834rbffefe9O3fff2Z1bRicMC20eZFRC4IrOHHcCXhRGaAyE/A82z4O9udnYCzX6tjtKlhLWipecEYdqr5NK356Zzo/nLcPuSokvTCN+ufo9O6tf9Jcs1qCckxQa1fJvHJrT43jTEATf0wODg7IEWVbWoINL3FaW6g6xUpSU3K1nB8uuFr7ErQEZ3bNZYwHheENdTAjkrqNsYWdEcwjQ9pyRqi0Qd0KbtMp7E5mKIhSG97pRFlZqJGqzjFOxQut3IzYOit4OSNFLUSFe80Isy9r7QB3yKndmFqgzvHtTzOSZRgn03rraIbm8DaQqI3YI70qDJWAyWx0vjzm2+/WXuatLm/iOFVwzrSUVOU+hco2Pj2jBgUutAo1O95V2BVBBJfcPdjHK1SiQ61ybCU4r0hqeLlx1Bh9TlKuCrdDe0A1I09bB0/8P/WcLEnSfK/GkMroHDFhuQxCTl9ikQMnxLVylu27StaskjWuHc5Pk/H+XOWcNR0Cq+mvALKn/1n907HRUDQOmY6sIlgFFBfGfWueQ9GsFmsfpxnggRuasTIaK8m0UWCWSYVnIsMOlr0WJ2Z5vuEO1nF6ZrG34BMpmxipFuQJFFzxMDoNuTdNPnlAMEVHpgukBrjr0PBA5wngfBgIlfuqCsdZm099isdfctXgvJTpLEj/B6Q/XgBRGlWOY5y8r54B4dNHvPy56evgNqANSM+0+gGYqw00/vEgrywKmMZ6D4w/araN/6Zbrwf1a+OPe+F6GNZXC8xqF/a9EFe9dR8qAFMabP6L9n3Uz7YVYTgx466hDC8gME26uRhbv2BICzva/bUd6UGjqnCFvWe1dVriXTGk0oJAnXGjVbjf/IDBgx4TbLeBgVWK8fZDhRFIR9FbN+RyRf0qSlg6c7lqr2DVwq6lNYo1JtU6XsOq08f4RUjG9/9V4dniMJkfJl9/Nn34qP823I4+jD6K7kVJdD96GH0eHUUnEYvK6Jfo1+i3yW7y++SPyZ8d9OaN3ueDaO+Z/PUv9JeECg==</latexit><latexit sha1_base64="bg6iQpcsz52Z46e+zRQZx1hbgxs=">AAAG93icfVVLj9xEEHYCGYJ5JXDk0mKEFNDsajxCihRppCgPiQOIBXaTiPGwarfLnmb64XS3dxka/wSucOSGuPJzuPBbqLa9eMe7wdJMl6u+qv6qqqudVYJbN5//fePma6/fmrxx+834rbffefe9O3fff2Z1bRicMC20eZFRC4IrOHHcCXhRGaAyE/A82z4O9udnYCzX6tjtKlhLWipecEYdqr5NK356Zzo/nLcPuSokvTCN+ufo9O6tf9Jcs1qCckxQa1fJvHJrT43jTEATf0wODg7IEWVbWoINL3FaW6g6xUpSU3K1nB8uuFr7ErQEZ3bNZYwHheENdTAjkrqNsYWdEcwjQ9pyRqi0Qd0KbtMp7E5mKIhSG97pRFlZqJGqzjFOxQut3IzYOit4OSNFLUSFe80Isy9r7QB3yKndmFqgzvHtTzOSZRgn03rraIbm8DaQqI3YI70qDJWAyWx0vjzm2+/WXuatLm/iOFVwzrSUVOU+hco2Pj2jBgUutAo1O95V2BVBBJfcPdjHK1SiQ61ybCU4r0hqeLlx1Bh9TlKuCrdDe0A1I09bB0/8P/WcLEnSfK/GkMroHDFhuQxCTl9ikQMnxLVylu27StaskjWuHc5Pk/H+XOWcNR0Cq+mvALKn/1n907HRUDQOmY6sIlgFFBfGfWueQ9GsFmsfpxnggRuasTIaK8m0UWCWSYVnIsMOlr0WJ2Z5vuEO1nF6ZrG34BMpmxipFuQJFFzxMDoNuTdNPnlAMEVHpgukBrjr0PBA5wngfBgIlfuqCsdZm099isdfctXgvJTpLEj/B6Q/XgBRGlWOY5y8r54B4dNHvPy56evgNqANSM+0+gGYqw00/vEgrywKmMZ6D4w/araN/6Zbrwf1a+OPe+F6GNZXC8xqF/a9EFe9dR8qAFMabP6L9n3Uz7YVYTgx466hDC8gME26uRhbv2BICzva/bUd6UGjqnCFvWe1dVriXTGk0oJAnXGjVbjf/IDBgx4TbLeBgVWK8fZDhRFIR9FbN+RyRf0qSlg6c7lqr2DVwq6lNYo1JtU6XsOq08f4RUjG9/9V4dniMJkfJl9/Nn34qP823I4+jD6K7kVJdD96GH0eHUUnEYvK6Jfo1+i3yW7y++SPyZ8d9OaN3ueDaO+Z/PUv9JeECg==</latexit>

Task 1 Task 2

⇡1<latexit sha1_base64="Ih0XDseZyFFqpaU+2ZQn+0+XBJM=">AAAG+XicfVVLj9xEEHYCGYJ55MGRS4sRUkCzq/EICSnSSFEeEgcQC9pNIsbDqt0ue5rph9Pd3mVo/Bu4wpEb4sqv4cJvodr24h3vBkszXa76qvqrqq52Vglu3Xz+942bb7x5a/LW7bfjd9597/07d+/df251bRicMC20eZlRC4IrOHHcCXhZGaAyE/Ai2z4J9hdnYCzX6tjtKlhLWipecEYdqk7Sip8mp3en88N5+5CrQtIL06h/jk7v3fonzTWrJSjHBLV2lcwrt/bUOM4ENPHH5ODggBxRtqUl2PASp7WFqlOsJDUlV8v54YKrtS9BS3Bm11zGeFAY3lAHMyKp2xhb2BnBTDIkLmeEShvUreA2ncLuZIaCKLXhnU6UlYUaqeoc41S80MrNiK2zgpczUtRCVLjXjDD7qtYOcIec2o2pBeoc3/40I1mGcTKtt45maA5vA4naiD3Sq8JQCZjMRufLY779bu1l3uryJo5TBedMS0lV7lOobOPTM2pQ4EKrULPjXYV9EURwyd3DfbxCJTrUKsdmgvOKpIaXG0eN0eck5apwO7QHVDPytHXwxP9Tz8mSJM33agypjM4RE5bLIOT0FRY5cEJcK2fZvqtkzSpZ49rh/DQZ789VzlnTIbCa/goge/af1T8bGw1F45DpyCqCVUBxYdy35jkUzWqx9nGaAR64oRkro7GSTBsFZplUeCYy7GDZa3Fmlucb7mAdp2cWews+kbKJkWpBnkLBFQ/D05AH0+SThwRTdGS6QGqAuw4ND3SeAs6HgVC5r6twnLX51Kd4/CVXDc5Lmc6C9H9A+uMFEKVR5TjGyfvqGRA+fczLn5u+Dm4D2oD0TKsfgLnaQOOfDPLKooBprPfA+KNm2/hvu/V6UL82/rgXrodhfbXArHZh3wtx1Vv3oQIwpcHmv2zfR/1sWxGGEzPuGsrwAgLTpJuLsfULhrSwo91f25EeNKoKV9h7VlunJd4VQyotCNQZN1qF+80PGDzoMcF2GxhYpRhvP1QYgXQUvXVDLlfUr6OEpTOXq/YaVi3sWlqjWGNSreM1rDp9jF+EZHz/XxWeLw6T+WHyzWfTR4/7b8Pt6MPoo+hBlESfR4+iL6Kj6CRiEY9+iX6Nfpv4ye+TPyZ/dtCbN3qfD6K9Z/LXv/0YhK4=</latexit><latexit sha1_base64="Ih0XDseZyFFqpaU+2ZQn+0+XBJM=">AAAG+XicfVVLj9xEEHYCGYJ55MGRS4sRUkCzq/EICSnSSFEeEgcQC9pNIsbDqt0ue5rph9Pd3mVo/Bu4wpEb4sqv4cJvodr24h3vBkszXa76qvqrqq52Vglu3Xz+942bb7x5a/LW7bfjd9597/07d+/df251bRicMC20eZlRC4IrOHHcCXhZGaAyE/Ai2z4J9hdnYCzX6tjtKlhLWipecEYdqk7Sip8mp3en88N5+5CrQtIL06h/jk7v3fonzTWrJSjHBLV2lcwrt/bUOM4ENPHH5ODggBxRtqUl2PASp7WFqlOsJDUlV8v54YKrtS9BS3Bm11zGeFAY3lAHMyKp2xhb2BnBTDIkLmeEShvUreA2ncLuZIaCKLXhnU6UlYUaqeoc41S80MrNiK2zgpczUtRCVLjXjDD7qtYOcIec2o2pBeoc3/40I1mGcTKtt45maA5vA4naiD3Sq8JQCZjMRufLY779bu1l3uryJo5TBedMS0lV7lOobOPTM2pQ4EKrULPjXYV9EURwyd3DfbxCJTrUKsdmgvOKpIaXG0eN0eck5apwO7QHVDPytHXwxP9Tz8mSJM33agypjM4RE5bLIOT0FRY5cEJcK2fZvqtkzSpZ49rh/DQZ789VzlnTIbCa/goge/af1T8bGw1F45DpyCqCVUBxYdy35jkUzWqx9nGaAR64oRkro7GSTBsFZplUeCYy7GDZa3Fmlucb7mAdp2cWews+kbKJkWpBnkLBFQ/D05AH0+SThwRTdGS6QGqAuw4ND3SeAs6HgVC5r6twnLX51Kd4/CVXDc5Lmc6C9H9A+uMFEKVR5TjGyfvqGRA+fczLn5u+Dm4D2oD0TKsfgLnaQOOfDPLKooBprPfA+KNm2/hvu/V6UL82/rgXrodhfbXArHZh3wtx1Vv3oQIwpcHmv2zfR/1sWxGGEzPuGsrwAgLTpJuLsfULhrSwo91f25EeNKoKV9h7VlunJd4VQyotCNQZN1qF+80PGDzoMcF2GxhYpRhvP1QYgXQUvXVDLlfUr6OEpTOXq/YaVi3sWlqjWGNSreM1rDp9jF+EZHz/XxWeLw6T+WHyzWfTR4/7b8Pt6MPoo+hBlESfR4+iL6Kj6CRiEY9+iX6Nfpv4ye+TPyZ/dtCbN3qfD6K9Z/LXv/0YhK4=</latexit><latexit sha1_base64="Ih0XDseZyFFqpaU+2ZQn+0+XBJM=">AAAG+XicfVVLj9xEEHYCGYJ55MGRS4sRUkCzq/EICSnSSFEeEgcQC9pNIsbDqt0ue5rph9Pd3mVo/Bu4wpEb4sqv4cJvodr24h3vBkszXa76qvqrqq52Vglu3Xz+942bb7x5a/LW7bfjd9597/07d+/df251bRicMC20eZlRC4IrOHHcCXhZGaAyE/Ai2z4J9hdnYCzX6tjtKlhLWipecEYdqk7Sip8mp3en88N5+5CrQtIL06h/jk7v3fonzTWrJSjHBLV2lcwrt/bUOM4ENPHH5ODggBxRtqUl2PASp7WFqlOsJDUlV8v54YKrtS9BS3Bm11zGeFAY3lAHMyKp2xhb2BnBTDIkLmeEShvUreA2ncLuZIaCKLXhnU6UlYUaqeoc41S80MrNiK2zgpczUtRCVLjXjDD7qtYOcIec2o2pBeoc3/40I1mGcTKtt45maA5vA4naiD3Sq8JQCZjMRufLY779bu1l3uryJo5TBedMS0lV7lOobOPTM2pQ4EKrULPjXYV9EURwyd3DfbxCJTrUKsdmgvOKpIaXG0eN0eck5apwO7QHVDPytHXwxP9Tz8mSJM33agypjM4RE5bLIOT0FRY5cEJcK2fZvqtkzSpZ49rh/DQZ789VzlnTIbCa/goge/af1T8bGw1F45DpyCqCVUBxYdy35jkUzWqx9nGaAR64oRkro7GSTBsFZplUeCYy7GDZa3Fmlucb7mAdp2cWews+kbKJkWpBnkLBFQ/D05AH0+SThwRTdGS6QGqAuw4ND3SeAs6HgVC5r6twnLX51Kd4/CVXDc5Lmc6C9H9A+uMFEKVR5TjGyfvqGRA+fczLn5u+Dm4D2oD0TKsfgLnaQOOfDPLKooBprPfA+KNm2/hvu/V6UL82/rgXrodhfbXArHZh3wtx1Vv3oQIwpcHmv2zfR/1sWxGGEzPuGsrwAgLTpJuLsfULhrSwo91f25EeNKoKV9h7VlunJd4VQyotCNQZN1qF+80PGDzoMcF2GxhYpRhvP1QYgXQUvXVDLlfUr6OEpTOXq/YaVi3sWlqjWGNSreM1rDp9jF+EZHz/XxWeLw6T+WHyzWfTR4/7b8Pt6MPoo+hBlESfR4+iL6Kj6CRiEY9+iX6Nfpv4ye+TPyZ/dtCbN3qfD6K9Z/LXv/0YhK4=</latexit><latexit sha1_base64="Ih0XDseZyFFqpaU+2ZQn+0+XBJM=">AAAG+XicfVVLj9xEEHYCGYJ55MGRS4sRUkCzq/EICSnSSFEeEgcQC9pNIsbDqt0ue5rph9Pd3mVo/Bu4wpEb4sqv4cJvodr24h3vBkszXa76qvqrqq52Vglu3Xz+942bb7x5a/LW7bfjd9597/07d+/df251bRicMC20eZlRC4IrOHHcCXhZGaAyE/Ai2z4J9hdnYCzX6tjtKlhLWipecEYdqk7Sip8mp3en88N5+5CrQtIL06h/jk7v3fonzTWrJSjHBLV2lcwrt/bUOM4ENPHH5ODggBxRtqUl2PASp7WFqlOsJDUlV8v54YKrtS9BS3Bm11zGeFAY3lAHMyKp2xhb2BnBTDIkLmeEShvUreA2ncLuZIaCKLXhnU6UlYUaqeoc41S80MrNiK2zgpczUtRCVLjXjDD7qtYOcIec2o2pBeoc3/40I1mGcTKtt45maA5vA4naiD3Sq8JQCZjMRufLY779bu1l3uryJo5TBedMS0lV7lOobOPTM2pQ4EKrULPjXYV9EURwyd3DfbxCJTrUKsdmgvOKpIaXG0eN0eck5apwO7QHVDPytHXwxP9Tz8mSJM33agypjM4RE5bLIOT0FRY5cEJcK2fZvqtkzSpZ49rh/DQZ789VzlnTIbCa/goge/af1T8bGw1F45DpyCqCVUBxYdy35jkUzWqx9nGaAR64oRkro7GSTBsFZplUeCYy7GDZa3Fmlucb7mAdp2cWews+kbKJkWpBnkLBFQ/D05AH0+SThwRTdGS6QGqAuw4ND3SeAs6HgVC5r6twnLX51Kd4/CVXDc5Lmc6C9H9A+uMFEKVR5TjGyfvqGRA+fczLn5u+Dm4D2oD0TKsfgLnaQOOfDPLKooBprPfA+KNm2/hvu/V6UL82/rgXrodhfbXArHZh3wtx1Vv3oQIwpcHmv2zfR/1sWxGGEzPuGsrwAgLTpJuLsfULhrSwo91f25EeNKoKV9h7VlunJd4VQyotCNQZN1qF+80PGDzoMcF2GxhYpRhvP1QYgXQUvXVDLlfUr6OEpTOXq/YaVi3sWlqjWGNSreM1rDp9jF+EZHz/XxWeLw6T+WHyzWfTR4/7b8Pt6MPoo+hBlESfR4+iL6Kj6CRiEY9+iX6Nfpv4ye+TPyZ/dtCbN3qfD6K9Z/LXv/0YhK4=</latexit>

⇡2<latexit sha1_base64="tC+ay56Q/V9J+Aj4ruLoo5U9if8=">AAAG+XicfVVLj9xEEHYCGYJ55MGRS4sRUkCzq/EICSnSSFEeEgcQC9pNIsbDqt0ue5rph9Pd3mVo/Bu4wpEb4sqv4cJvodr24h3vBkszXa76qvqrqq52Vglu3Xz+942bb7x5a/LW7bfjd9597/07d+/df251bRicMC20eZlRC4IrOHHcCXhZGaAyE/Ai2z4J9hdnYCzX6tjtKlhLWipecEYdqk7Sip8uTu9O54fz9iFXhaQXplH/HJ3eu/VPmmtWS1COCWrtKplXbu2pcZwJaOKPycHBATmibEtLsOElTmsLVadYSWpKrpbzwwVXa1+CluDMrrmM8aAwvKEOZkRStzG2sDOCmWRIXM4IlTaoW8FtOoXdyQwFUWrDO50oKws1UtU5xql4oZWbEVtnBS9npKiFqHCvGWH2Va0d4A45tRtTC9Q5vv1pRrIM42Rabx3N0BzeBhK1EXukV4WhEjCZjc6Xx3z73drLvNXlTRynCs6ZlpKq3KdQ2canZ9SgwIVWoWbHuwr7IojgkruH+3iFSnSoVY7NBOcVSQ0vN44ao89JylXhdmgPqGbkaevgif+nnpMlSZrv1RhSGZ0jJiyXQcjpKyxy4IS4Vs6yfVfJmlWyxrXD+Wky3p+rnLOmQ2A1/RVA9uw/q382NhqKxiHTkVUEq4DiwrhvzXMomtVi7eM0AzxwQzNWRmMlmTYKzDKp8Exk2MGy1+LMLM833ME6Ts8s9hZ8ImUTI9WCPIWCKx6GpyEPpsknDwmm6Mh0gdQAdx0aHug8BZwPA6FyX1fhOGvzqU/x+EuuGpyXMp0F6f+A9McLIEqjynGMk/fVMyB8+piXPzd9HdwGtAHpmVY/AHO1gcY/GeSVRQHTWO+B8UfNtvHfduv1oH5t/HEvXA/D+mqBWe3CvhfiqrfuQwVgSoPNf9m+j/rZtiIMJ2bcNZThBQSmSTcXY+sXDGlhR7u/tiM9aFQVrrD3rLZOS7wrhlRaEKgzbrQK95sfMHjQY4LtNjCwSjHefqgwAukoeuuGXK6oX0cJS2cuV+01rFrYtbRGscakWsdrWHX6GL8Iyfj+vyo8Xxwm88Pkm8+mjx7334bb0YfRR9GDKIk+jx5FX0RH0UnEIh79Ev0a/Tbxk98nf0z+7KA3b/Q+H0R7z+SvfwEDuISv</latexit><latexit sha1_base64="tC+ay56Q/V9J+Aj4ruLoo5U9if8=">AAAG+XicfVVLj9xEEHYCGYJ55MGRS4sRUkCzq/EICSnSSFEeEgcQC9pNIsbDqt0ue5rph9Pd3mVo/Bu4wpEb4sqv4cJvodr24h3vBkszXa76qvqrqq52Vglu3Xz+942bb7x5a/LW7bfjd9597/07d+/df251bRicMC20eZlRC4IrOHHcCXhZGaAyE/Ai2z4J9hdnYCzX6tjtKlhLWipecEYdqk7Sip8uTu9O54fz9iFXhaQXplH/HJ3eu/VPmmtWS1COCWrtKplXbu2pcZwJaOKPycHBATmibEtLsOElTmsLVadYSWpKrpbzwwVXa1+CluDMrrmM8aAwvKEOZkRStzG2sDOCmWRIXM4IlTaoW8FtOoXdyQwFUWrDO50oKws1UtU5xql4oZWbEVtnBS9npKiFqHCvGWH2Va0d4A45tRtTC9Q5vv1pRrIM42Rabx3N0BzeBhK1EXukV4WhEjCZjc6Xx3z73drLvNXlTRynCs6ZlpKq3KdQ2canZ9SgwIVWoWbHuwr7IojgkruH+3iFSnSoVY7NBOcVSQ0vN44ao89JylXhdmgPqGbkaevgif+nnpMlSZrv1RhSGZ0jJiyXQcjpKyxy4IS4Vs6yfVfJmlWyxrXD+Wky3p+rnLOmQ2A1/RVA9uw/q382NhqKxiHTkVUEq4DiwrhvzXMomtVi7eM0AzxwQzNWRmMlmTYKzDKp8Exk2MGy1+LMLM833ME6Ts8s9hZ8ImUTI9WCPIWCKx6GpyEPpsknDwmm6Mh0gdQAdx0aHug8BZwPA6FyX1fhOGvzqU/x+EuuGpyXMp0F6f+A9McLIEqjynGMk/fVMyB8+piXPzd9HdwGtAHpmVY/AHO1gcY/GeSVRQHTWO+B8UfNtvHfduv1oH5t/HEvXA/D+mqBWe3CvhfiqrfuQwVgSoPNf9m+j/rZtiIMJ2bcNZThBQSmSTcXY+sXDGlhR7u/tiM9aFQVrrD3rLZOS7wrhlRaEKgzbrQK95sfMHjQY4LtNjCwSjHefqgwAukoeuuGXK6oX0cJS2cuV+01rFrYtbRGscakWsdrWHX6GL8Iyfj+vyo8Xxwm88Pkm8+mjx7334bb0YfRR9GDKIk+jx5FX0RH0UnEIh79Ev0a/Tbxk98nf0z+7KA3b/Q+H0R7z+SvfwEDuISv</latexit><latexit sha1_base64="tC+ay56Q/V9J+Aj4ruLoo5U9if8=">AAAG+XicfVVLj9xEEHYCGYJ55MGRS4sRUkCzq/EICSnSSFEeEgcQC9pNIsbDqt0ue5rph9Pd3mVo/Bu4wpEb4sqv4cJvodr24h3vBkszXa76qvqrqq52Vglu3Xz+942bb7x5a/LW7bfjd9597/07d+/df251bRicMC20eZlRC4IrOHHcCXhZGaAyE/Ai2z4J9hdnYCzX6tjtKlhLWipecEYdqk7Sip8uTu9O54fz9iFXhaQXplH/HJ3eu/VPmmtWS1COCWrtKplXbu2pcZwJaOKPycHBATmibEtLsOElTmsLVadYSWpKrpbzwwVXa1+CluDMrrmM8aAwvKEOZkRStzG2sDOCmWRIXM4IlTaoW8FtOoXdyQwFUWrDO50oKws1UtU5xql4oZWbEVtnBS9npKiFqHCvGWH2Va0d4A45tRtTC9Q5vv1pRrIM42Rabx3N0BzeBhK1EXukV4WhEjCZjc6Xx3z73drLvNXlTRynCs6ZlpKq3KdQ2canZ9SgwIVWoWbHuwr7IojgkruH+3iFSnSoVY7NBOcVSQ0vN44ao89JylXhdmgPqGbkaevgif+nnpMlSZrv1RhSGZ0jJiyXQcjpKyxy4IS4Vs6yfVfJmlWyxrXD+Wky3p+rnLOmQ2A1/RVA9uw/q382NhqKxiHTkVUEq4DiwrhvzXMomtVi7eM0AzxwQzNWRmMlmTYKzDKp8Exk2MGy1+LMLM833ME6Ts8s9hZ8ImUTI9WCPIWCKx6GpyEPpsknDwmm6Mh0gdQAdx0aHug8BZwPA6FyX1fhOGvzqU/x+EuuGpyXMp0F6f+A9McLIEqjynGMk/fVMyB8+piXPzd9HdwGtAHpmVY/AHO1gcY/GeSVRQHTWO+B8UfNtvHfduv1oH5t/HEvXA/D+mqBWe3CvhfiqrfuQwVgSoPNf9m+j/rZtiIMJ2bcNZThBQSmSTcXY+sXDGlhR7u/tiM9aFQVrrD3rLZOS7wrhlRaEKgzbrQK95sfMHjQY4LtNjCwSjHefqgwAukoeuuGXK6oX0cJS2cuV+01rFrYtbRGscakWsdrWHX6GL8Iyfj+vyo8Xxwm88Pkm8+mjx7334bb0YfRR9GDKIk+jx5FX0RH0UnEIh79Ev0a/Tbxk98nf0z+7KA3b/Q+H0R7z+SvfwEDuISv</latexit><latexit sha1_base64="tC+ay56Q/V9J+Aj4ruLoo5U9if8=">AAAG+XicfVVLj9xEEHYCGYJ55MGRS4sRUkCzq/EICSnSSFEeEgcQC9pNIsbDqt0ue5rph9Pd3mVo/Bu4wpEb4sqv4cJvodr24h3vBkszXa76qvqrqq52Vglu3Xz+942bb7x5a/LW7bfjd9597/07d+/df251bRicMC20eZlRC4IrOHHcCXhZGaAyE/Ai2z4J9hdnYCzX6tjtKlhLWipecEYdqk7Sip8uTu9O54fz9iFXhaQXplH/HJ3eu/VPmmtWS1COCWrtKplXbu2pcZwJaOKPycHBATmibEtLsOElTmsLVadYSWpKrpbzwwVXa1+CluDMrrmM8aAwvKEOZkRStzG2sDOCmWRIXM4IlTaoW8FtOoXdyQwFUWrDO50oKws1UtU5xql4oZWbEVtnBS9npKiFqHCvGWH2Va0d4A45tRtTC9Q5vv1pRrIM42Rabx3N0BzeBhK1EXukV4WhEjCZjc6Xx3z73drLvNXlTRynCs6ZlpKq3KdQ2canZ9SgwIVWoWbHuwr7IojgkruH+3iFSnSoVY7NBOcVSQ0vN44ao89JylXhdmgPqGbkaevgif+nnpMlSZrv1RhSGZ0jJiyXQcjpKyxy4IS4Vs6yfVfJmlWyxrXD+Wky3p+rnLOmQ2A1/RVA9uw/q382NhqKxiHTkVUEq4DiwrhvzXMomtVi7eM0AzxwQzNWRmMlmTYKzDKp8Exk2MGy1+LMLM833ME6Ts8s9hZ8ImUTI9WCPIWCKx6GpyEPpsknDwmm6Mh0gdQAdx0aHug8BZwPA6FyX1fhOGvzqU/x+EuuGpyXMp0F6f+A9McLIEqjynGMk/fVMyB8+piXPzd9HdwGtAHpmVY/AHO1gcY/GeSVRQHTWO+B8UfNtvHfduv1oH5t/HEvXA/D+mqBWe3CvhfiqrfuQwVgSoPNf9m+j/rZtiIMJ2bcNZThBQSmSTcXY+sXDGlhR7u/tiM9aFQVrrD3rLZOS7wrhlRaEKgzbrQK95sfMHjQY4LtNjCwSjHefqgwAukoeuuGXK6oX0cJS2cuV+01rFrYtbRGscakWsdrWHX6GL8Iyfj+vyo8Xxwm88Pkm8+mjx7334bb0YfRR9GDKIk+jx5FX0RH0UnEIh79Ev0a/Tbxk98nf0z+7KA3b/Q+H0R7z+SvfwEDuISv</latexit>

Task 1

Task 2

Task 3

M ⇠ D<latexit sha1_base64="OX1ier/XMCCLr88ChMp6EICKr2E=">AAAEQnicZVNLb9NAEN4SHsW8WjhyWRGQipRGdoSEVClSBa3EpVKR+hJxqHbtsbPKPszuulWw/Ce4ws/hT/AXuCGuHBgngTTpHuzZ7/tmZ2ZnhxdSOB+GP9ZutG7eun1n/W5w7/6Dh482Nh+fOFPaBI4TI40948yBFBqOvfASzgoLTHEJp3z8tuFPL8A6YfSRnxQwVCzXIhMJ8widHdDYCUX3zjfaYTecLnrdiOZGm8zX4flm63mcmqRUoH0imXODKCz8sGLWi0RCHbyg29vb9JAlY5aDazZBXDooZsBAMZsL3Q+7PaGHVQ5GgbeT+qqmAo3HW+ahQxXzI+sy16FYDMfcVYcy5Rp4avjRDHATxdGQubFihsm8cFBiqibFcwqRGe071JU8E3mHZqWUBcbq0MR9Ko0HjJAyN7KlRMyL8ecO5RzP4caMPeNIN7tFEqWVS0kPMssUYDEjk/aPxPjDsFLpFEvrIIg1XCZGKabTKobC1VV8wSwaQhrd3NnRpMDWSCqFEn5nWa8RRIdSp9hP8JWmsRX5yDNrzSWNhc78BPlGVa94urLxxO95JWifRvVHvSoprElR0/yuijCnA7zkJifUTW3Ol11VUg+iIf5nuqodrcYXOhVJPVPgbVbXBHz/P1vtr5KWIbmodIWVDSsh+0cus2kKWT3oDasg5oAPbtGMgTV4k4mxGmw/KvBNcOxgPkdxbPqXI+FhGMQXDnsLVaRUHWCqGd2DTGjRzE9Nt9rRyx2KJXra7mFqgFEXDa9xrqLVKbpunPS6UdiN3r9q776ZT9g6eUqekS0Skddkl7wjh+SYJESSL+Qr+db63vrZ+tX6PZPeWJv7PCFLq/XnL0yObpQ=</latexit><latexit sha1_base64="OX1ier/XMCCLr88ChMp6EICKr2E=">AAAEQnicZVNLb9NAEN4SHsW8WjhyWRGQipRGdoSEVClSBa3EpVKR+hJxqHbtsbPKPszuulWw/Ce4ws/hT/AXuCGuHBgngTTpHuzZ7/tmZ2ZnhxdSOB+GP9ZutG7eun1n/W5w7/6Dh482Nh+fOFPaBI4TI40948yBFBqOvfASzgoLTHEJp3z8tuFPL8A6YfSRnxQwVCzXIhMJ8widHdDYCUX3zjfaYTecLnrdiOZGm8zX4flm63mcmqRUoH0imXODKCz8sGLWi0RCHbyg29vb9JAlY5aDazZBXDooZsBAMZsL3Q+7PaGHVQ5GgbeT+qqmAo3HW+ahQxXzI+sy16FYDMfcVYcy5Rp4avjRDHATxdGQubFihsm8cFBiqibFcwqRGe071JU8E3mHZqWUBcbq0MR9Ko0HjJAyN7KlRMyL8ecO5RzP4caMPeNIN7tFEqWVS0kPMssUYDEjk/aPxPjDsFLpFEvrIIg1XCZGKabTKobC1VV8wSwaQhrd3NnRpMDWSCqFEn5nWa8RRIdSp9hP8JWmsRX5yDNrzSWNhc78BPlGVa94urLxxO95JWifRvVHvSoprElR0/yuijCnA7zkJifUTW3Ol11VUg+iIf5nuqodrcYXOhVJPVPgbVbXBHz/P1vtr5KWIbmodIWVDSsh+0cus2kKWT3oDasg5oAPbtGMgTV4k4mxGmw/KvBNcOxgPkdxbPqXI+FhGMQXDnsLVaRUHWCqGd2DTGjRzE9Nt9rRyx2KJXra7mFqgFEXDa9xrqLVKbpunPS6UdiN3r9q776ZT9g6eUqekS0Skddkl7wjh+SYJESSL+Qr+db63vrZ+tX6PZPeWJv7PCFLq/XnL0yObpQ=</latexit><latexit sha1_base64="OX1ier/XMCCLr88ChMp6EICKr2E=">AAAEQnicZVNLb9NAEN4SHsW8WjhyWRGQipRGdoSEVClSBa3EpVKR+hJxqHbtsbPKPszuulWw/Ce4ws/hT/AXuCGuHBgngTTpHuzZ7/tmZ2ZnhxdSOB+GP9ZutG7eun1n/W5w7/6Dh482Nh+fOFPaBI4TI40948yBFBqOvfASzgoLTHEJp3z8tuFPL8A6YfSRnxQwVCzXIhMJ8widHdDYCUX3zjfaYTecLnrdiOZGm8zX4flm63mcmqRUoH0imXODKCz8sGLWi0RCHbyg29vb9JAlY5aDazZBXDooZsBAMZsL3Q+7PaGHVQ5GgbeT+qqmAo3HW+ahQxXzI+sy16FYDMfcVYcy5Rp4avjRDHATxdGQubFihsm8cFBiqibFcwqRGe071JU8E3mHZqWUBcbq0MR9Ko0HjJAyN7KlRMyL8ecO5RzP4caMPeNIN7tFEqWVS0kPMssUYDEjk/aPxPjDsFLpFEvrIIg1XCZGKabTKobC1VV8wSwaQhrd3NnRpMDWSCqFEn5nWa8RRIdSp9hP8JWmsRX5yDNrzSWNhc78BPlGVa94urLxxO95JWifRvVHvSoprElR0/yuijCnA7zkJifUTW3Ol11VUg+iIf5nuqodrcYXOhVJPVPgbVbXBHz/P1vtr5KWIbmodIWVDSsh+0cus2kKWT3oDasg5oAPbtGMgTV4k4mxGmw/KvBNcOxgPkdxbPqXI+FhGMQXDnsLVaRUHWCqGd2DTGjRzE9Nt9rRyx2KJXra7mFqgFEXDa9xrqLVKbpunPS6UdiN3r9q776ZT9g6eUqekS0Skddkl7wjh+SYJESSL+Qr+db63vrZ+tX6PZPeWJv7PCFLq/XnL0yObpQ=</latexit><latexit sha1_base64="OX1ier/XMCCLr88ChMp6EICKr2E=">AAAEQnicZVNLb9NAEN4SHsW8WjhyWRGQipRGdoSEVClSBa3EpVKR+hJxqHbtsbPKPszuulWw/Ce4ws/hT/AXuCGuHBgngTTpHuzZ7/tmZ2ZnhxdSOB+GP9ZutG7eun1n/W5w7/6Dh482Nh+fOFPaBI4TI40948yBFBqOvfASzgoLTHEJp3z8tuFPL8A6YfSRnxQwVCzXIhMJ8widHdDYCUX3zjfaYTecLnrdiOZGm8zX4flm63mcmqRUoH0imXODKCz8sGLWi0RCHbyg29vb9JAlY5aDazZBXDooZsBAMZsL3Q+7PaGHVQ5GgbeT+qqmAo3HW+ahQxXzI+sy16FYDMfcVYcy5Rp4avjRDHATxdGQubFihsm8cFBiqibFcwqRGe071JU8E3mHZqWUBcbq0MR9Ko0HjJAyN7KlRMyL8ecO5RzP4caMPeNIN7tFEqWVS0kPMssUYDEjk/aPxPjDsFLpFEvrIIg1XCZGKabTKobC1VV8wSwaQhrd3NnRpMDWSCqFEn5nWa8RRIdSp9hP8JWmsRX5yDNrzSWNhc78BPlGVa94urLxxO95JWifRvVHvSoprElR0/yuijCnA7zkJifUTW3Ol11VUg+iIf5nuqodrcYXOhVJPVPgbVbXBHz/P1vtr5KWIbmodIWVDSsh+0cus2kKWT3oDasg5oAPbtGMgTV4k4mxGmw/KvBNcOxgPkdxbPqXI+FhGMQXDnsLVaRUHWCqGd2DTGjRzE9Nt9rRyx2KJXra7mFqgFEXDa9xrqLVKbpunPS6UdiN3r9q776ZT9g6eUqekS0Skddkl7wjh+SYJESSL+Qr+db63vrZ+tX6PZPeWJv7PCFLq/XnL0yObpQ=</latexit>

Tasks

argmax⇡2⇧

EM⇠D

[V ⇡M (s0)]

<latexit sha1_base64="VUn+sIozzVaGxmfIAX/n0e4S040=">AAAHPXicfVVdb9xEFHULXYr5SuGRlxErpBRtovUKqVKlSFWbCh6ICChJK3a2q7F97R12PtyZcdJl8E/iN/AjeEIC3nhDvPLKHa/DZp0US7tzfe+ZO+feMzNOK8GtG49/vXX7jTfvDN66+3b8zrvvvf/Bzr0Pz6yuTQanmRbaPE+ZBcEVnDruBDyvDDCZCniWLp+E+LNzMJZrdeJWFcwkKxUveMYcuuY7X1BmSslezT2tOKFcEXrMG0JrleMscP6IUMslOWw8TZ82VEDhpmfzoxcI37Xz8X1qeLlws/nOcLw/bh9y3Ug6Yxh1z/H83p0/aK6zWoJymWDWTpNx5WaeGcczAU38Kdnb2yPHLFuyEmx4iWltoVo7phJ5c3Uw3p9wNfMlaAnOrJqrGA8K0xvmYEQkcwtjCzsi2I0Ui5cjwqQN7tZwi7XDrmSKhii14WufKCsLNVLVOeapeKGVGxFbpwUvR6SohahwrRHJ7MtaO8AVcmYXphboc3z5w4ikKeZJtV46lmI4vG1I1EZskZ4WhknAYhY6Pzjhy+9mXuatL2/imCq4yLSUTOWeQmVRlXNm0OBCq9Czk1WF2goiuOTu4TZeoRMn/Cctat2Kx4zRF0H7wq0wHlBNb6atw0z8n3tODkjSvFB9SGV0jpgwXAUhpyNscuCEuNZO0+2pMmumyQzHNc4Pk/76XOU8a9YI7Ka/BsCteRn1T/tBwzC4qbQXFSEatnUX3I7mORTNdDLzMU0BN9xGjKnR2MlMGwXmIKlwT6SoYNl58dwdXCy4g1lMzy1qCz6RsomRakEOoeCKhwPYkN1hcv8hwRIdGU6QGuCqG8EDnUPA82EgdO7rKmxnbT7z7bHlqsHzUtJRsP4PyF5dAtHqdY5jnrzrngHh6WNe/th0fXAL0Aakz7T6HjJXG2j8k409tWhgGbMtMP6YWTb+2/V4M6gbG3/SGTfDsL9aYFWrsO6lOe2i21ABWNIm5r9q33t6tlKEwxnus1bQDC8gMA1dXB5bP8mQFiq6/msV6UC9rnCF2me1dVriXbEppQWBOudGq3C/+Q0GN3pMUG4DG1YU822nCkeA9rK305DLNffrKGHrzNWuvYZVC7uRVi9Xn1Q78QZWa3+MX4Skf/9fN84m+8l4P/nm8+Gjx9234W70cfRJtBsl0YPoUfRldBydRln0U/RL9Fv0++DnwZ+DvwZ/r6G3b3VzPoq2nsE//wI32KAO</latexit><latexit sha1_base64="VUn+sIozzVaGxmfIAX/n0e4S040=">AAAHPXicfVVdb9xEFHULXYr5SuGRlxErpBRtovUKqVKlSFWbCh6ICChJK3a2q7F97R12PtyZcdJl8E/iN/AjeEIC3nhDvPLKHa/DZp0US7tzfe+ZO+feMzNOK8GtG49/vXX7jTfvDN66+3b8zrvvvf/Bzr0Pz6yuTQanmRbaPE+ZBcEVnDruBDyvDDCZCniWLp+E+LNzMJZrdeJWFcwkKxUveMYcuuY7X1BmSslezT2tOKFcEXrMG0JrleMscP6IUMslOWw8TZ82VEDhpmfzoxcI37Xz8X1qeLlws/nOcLw/bh9y3Ug6Yxh1z/H83p0/aK6zWoJymWDWTpNx5WaeGcczAU38Kdnb2yPHLFuyEmx4iWltoVo7phJ5c3Uw3p9wNfMlaAnOrJqrGA8K0xvmYEQkcwtjCzsi2I0Ui5cjwqQN7tZwi7XDrmSKhii14WufKCsLNVLVOeapeKGVGxFbpwUvR6SohahwrRHJ7MtaO8AVcmYXphboc3z5w4ikKeZJtV46lmI4vG1I1EZskZ4WhknAYhY6Pzjhy+9mXuatL2/imCq4yLSUTOWeQmVRlXNm0OBCq9Czk1WF2goiuOTu4TZeoRMn/Cctat2Kx4zRF0H7wq0wHlBNb6atw0z8n3tODkjSvFB9SGV0jpgwXAUhpyNscuCEuNZO0+2pMmumyQzHNc4Pk/76XOU8a9YI7Ka/BsCteRn1T/tBwzC4qbQXFSEatnUX3I7mORTNdDLzMU0BN9xGjKnR2MlMGwXmIKlwT6SoYNl58dwdXCy4g1lMzy1qCz6RsomRakEOoeCKhwPYkN1hcv8hwRIdGU6QGuCqG8EDnUPA82EgdO7rKmxnbT7z7bHlqsHzUtJRsP4PyF5dAtHqdY5jnrzrngHh6WNe/th0fXAL0Aakz7T6HjJXG2j8k409tWhgGbMtMP6YWTb+2/V4M6gbG3/SGTfDsL9aYFWrsO6lOe2i21ABWNIm5r9q33t6tlKEwxnus1bQDC8gMA1dXB5bP8mQFiq6/msV6UC9rnCF2me1dVriXbEppQWBOudGq3C/+Q0GN3pMUG4DG1YU822nCkeA9rK305DLNffrKGHrzNWuvYZVC7uRVi9Xn1Q78QZWa3+MX4Skf/9fN84m+8l4P/nm8+Gjx9234W70cfRJtBsl0YPoUfRldBydRln0U/RL9Fv0++DnwZ+DvwZ/r6G3b3VzPoq2nsE//wI32KAO</latexit><latexit sha1_base64="VUn+sIozzVaGxmfIAX/n0e4S040=">AAAHPXicfVVdb9xEFHULXYr5SuGRlxErpBRtovUKqVKlSFWbCh6ICChJK3a2q7F97R12PtyZcdJl8E/iN/AjeEIC3nhDvPLKHa/DZp0US7tzfe+ZO+feMzNOK8GtG49/vXX7jTfvDN66+3b8zrvvvf/Bzr0Pz6yuTQanmRbaPE+ZBcEVnDruBDyvDDCZCniWLp+E+LNzMJZrdeJWFcwkKxUveMYcuuY7X1BmSslezT2tOKFcEXrMG0JrleMscP6IUMslOWw8TZ82VEDhpmfzoxcI37Xz8X1qeLlws/nOcLw/bh9y3Ug6Yxh1z/H83p0/aK6zWoJymWDWTpNx5WaeGcczAU38Kdnb2yPHLFuyEmx4iWltoVo7phJ5c3Uw3p9wNfMlaAnOrJqrGA8K0xvmYEQkcwtjCzsi2I0Ui5cjwqQN7tZwi7XDrmSKhii14WufKCsLNVLVOeapeKGVGxFbpwUvR6SohahwrRHJ7MtaO8AVcmYXphboc3z5w4ikKeZJtV46lmI4vG1I1EZskZ4WhknAYhY6Pzjhy+9mXuatL2/imCq4yLSUTOWeQmVRlXNm0OBCq9Czk1WF2goiuOTu4TZeoRMn/Cctat2Kx4zRF0H7wq0wHlBNb6atw0z8n3tODkjSvFB9SGV0jpgwXAUhpyNscuCEuNZO0+2pMmumyQzHNc4Pk/76XOU8a9YI7Ka/BsCteRn1T/tBwzC4qbQXFSEatnUX3I7mORTNdDLzMU0BN9xGjKnR2MlMGwXmIKlwT6SoYNl58dwdXCy4g1lMzy1qCz6RsomRakEOoeCKhwPYkN1hcv8hwRIdGU6QGuCqG8EDnUPA82EgdO7rKmxnbT7z7bHlqsHzUtJRsP4PyF5dAtHqdY5jnrzrngHh6WNe/th0fXAL0Aakz7T6HjJXG2j8k409tWhgGbMtMP6YWTb+2/V4M6gbG3/SGTfDsL9aYFWrsO6lOe2i21ABWNIm5r9q33t6tlKEwxnus1bQDC8gMA1dXB5bP8mQFiq6/msV6UC9rnCF2me1dVriXbEppQWBOudGq3C/+Q0GN3pMUG4DG1YU822nCkeA9rK305DLNffrKGHrzNWuvYZVC7uRVi9Xn1Q78QZWa3+MX4Skf/9fN84m+8l4P/nm8+Gjx9234W70cfRJtBsl0YPoUfRldBydRln0U/RL9Fv0++DnwZ+DvwZ/r6G3b3VzPoq2nsE//wI32KAO</latexit><latexit sha1_base64="VUn+sIozzVaGxmfIAX/n0e4S040=">AAAHPXicfVVdb9xEFHULXYr5SuGRlxErpBRtovUKqVKlSFWbCh6ICChJK3a2q7F97R12PtyZcdJl8E/iN/AjeEIC3nhDvPLKHa/DZp0US7tzfe+ZO+feMzNOK8GtG49/vXX7jTfvDN66+3b8zrvvvf/Bzr0Pz6yuTQanmRbaPE+ZBcEVnDruBDyvDDCZCniWLp+E+LNzMJZrdeJWFcwkKxUveMYcuuY7X1BmSslezT2tOKFcEXrMG0JrleMscP6IUMslOWw8TZ82VEDhpmfzoxcI37Xz8X1qeLlws/nOcLw/bh9y3Ug6Yxh1z/H83p0/aK6zWoJymWDWTpNx5WaeGcczAU38Kdnb2yPHLFuyEmx4iWltoVo7phJ5c3Uw3p9wNfMlaAnOrJqrGA8K0xvmYEQkcwtjCzsi2I0Ui5cjwqQN7tZwi7XDrmSKhii14WufKCsLNVLVOeapeKGVGxFbpwUvR6SohahwrRHJ7MtaO8AVcmYXphboc3z5w4ikKeZJtV46lmI4vG1I1EZskZ4WhknAYhY6Pzjhy+9mXuatL2/imCq4yLSUTOWeQmVRlXNm0OBCq9Czk1WF2goiuOTu4TZeoRMn/Cctat2Kx4zRF0H7wq0wHlBNb6atw0z8n3tODkjSvFB9SGV0jpgwXAUhpyNscuCEuNZO0+2pMmumyQzHNc4Pk/76XOU8a9YI7Ka/BsCteRn1T/tBwzC4qbQXFSEatnUX3I7mORTNdDLzMU0BN9xGjKnR2MlMGwXmIKlwT6SoYNl58dwdXCy4g1lMzy1qCz6RsomRakEOoeCKhwPYkN1hcv8hwRIdGU6QGuCqG8EDnUPA82EgdO7rKmxnbT7z7bHlqsHzUtJRsP4PyF5dAtHqdY5jnrzrngHh6WNe/th0fXAL0Aakz7T6HjJXG2j8k409tWhgGbMtMP6YWTb+2/V4M6gbG3/SGTfDsL9aYFWrsO6lOe2i21ABWNIm5r9q33t6tlKEwxnus1bQDC8gMA1dXB5bP8mQFiq6/msV6UC9rnCF2me1dVriXbEppQWBOudGq3C/+Q0GN3pMUG4DG1YU822nCkeA9rK305DLNffrKGHrzNWuvYZVC7uRVi9Xn1Q78QZWa3+MX4Skf/9fN84m+8l4P/nm8+Gjx9234W70cfRJtBsl0YPoUfRldBydRln0U/RL9Fv0++DnwZ+DvwZ/r6G3b3VzPoq2nsE//wI32KAO</latexit>

Performance of on task M.⇡<latexit sha1_base64="bg6iQpcsz52Z46e+zRQZx1hbgxs=">AAAG93icfVVLj9xEEHYCGYJ5JXDk0mKEFNDsajxCihRppCgPiQOIBXaTiPGwarfLnmb64XS3dxka/wSucOSGuPJzuPBbqLa9eMe7wdJMl6u+qv6qqqudVYJbN5//fePma6/fmrxx+834rbffefe9O3fff2Z1bRicMC20eZFRC4IrOHHcCXhRGaAyE/A82z4O9udnYCzX6tjtKlhLWipecEYdqr5NK356Zzo/nLcPuSokvTCN+ufo9O6tf9Jcs1qCckxQa1fJvHJrT43jTEATf0wODg7IEWVbWoINL3FaW6g6xUpSU3K1nB8uuFr7ErQEZ3bNZYwHheENdTAjkrqNsYWdEcwjQ9pyRqi0Qd0KbtMp7E5mKIhSG97pRFlZqJGqzjFOxQut3IzYOit4OSNFLUSFe80Isy9r7QB3yKndmFqgzvHtTzOSZRgn03rraIbm8DaQqI3YI70qDJWAyWx0vjzm2+/WXuatLm/iOFVwzrSUVOU+hco2Pj2jBgUutAo1O95V2BVBBJfcPdjHK1SiQ61ybCU4r0hqeLlx1Bh9TlKuCrdDe0A1I09bB0/8P/WcLEnSfK/GkMroHDFhuQxCTl9ikQMnxLVylu27StaskjWuHc5Pk/H+XOWcNR0Cq+mvALKn/1n907HRUDQOmY6sIlgFFBfGfWueQ9GsFmsfpxnggRuasTIaK8m0UWCWSYVnIsMOlr0WJ2Z5vuEO1nF6ZrG34BMpmxipFuQJFFzxMDoNuTdNPnlAMEVHpgukBrjr0PBA5wngfBgIlfuqCsdZm099isdfctXgvJTpLEj/B6Q/XgBRGlWOY5y8r54B4dNHvPy56evgNqANSM+0+gGYqw00/vEgrywKmMZ6D4w/araN/6Zbrwf1a+OPe+F6GNZXC8xqF/a9EFe9dR8qAFMabP6L9n3Uz7YVYTgx466hDC8gME26uRhbv2BICzva/bUd6UGjqnCFvWe1dVriXTGk0oJAnXGjVbjf/IDBgx4TbLeBgVWK8fZDhRFIR9FbN+RyRf0qSlg6c7lqr2DVwq6lNYo1JtU6XsOq08f4RUjG9/9V4dniMJkfJl9/Nn34qP823I4+jD6K7kVJdD96GH0eHUUnEYvK6Jfo1+i3yW7y++SPyZ8d9OaN3ueDaO+Z/PUv9JeECg==</latexit><latexit sha1_base64="bg6iQpcsz52Z46e+zRQZx1hbgxs=">AAAG93icfVVLj9xEEHYCGYJ5JXDk0mKEFNDsajxCihRppCgPiQOIBXaTiPGwarfLnmb64XS3dxka/wSucOSGuPJzuPBbqLa9eMe7wdJMl6u+qv6qqqudVYJbN5//fePma6/fmrxx+834rbffefe9O3fff2Z1bRicMC20eZFRC4IrOHHcCXhRGaAyE/A82z4O9udnYCzX6tjtKlhLWipecEYdqr5NK356Zzo/nLcPuSokvTCN+ufo9O6tf9Jcs1qCckxQa1fJvHJrT43jTEATf0wODg7IEWVbWoINL3FaW6g6xUpSU3K1nB8uuFr7ErQEZ3bNZYwHheENdTAjkrqNsYWdEcwjQ9pyRqi0Qd0KbtMp7E5mKIhSG97pRFlZqJGqzjFOxQut3IzYOit4OSNFLUSFe80Isy9r7QB3yKndmFqgzvHtTzOSZRgn03rraIbm8DaQqI3YI70qDJWAyWx0vjzm2+/WXuatLm/iOFVwzrSUVOU+hco2Pj2jBgUutAo1O95V2BVBBJfcPdjHK1SiQ61ybCU4r0hqeLlx1Bh9TlKuCrdDe0A1I09bB0/8P/WcLEnSfK/GkMroHDFhuQxCTl9ikQMnxLVylu27StaskjWuHc5Pk/H+XOWcNR0Cq+mvALKn/1n907HRUDQOmY6sIlgFFBfGfWueQ9GsFmsfpxnggRuasTIaK8m0UWCWSYVnIsMOlr0WJ2Z5vuEO1nF6ZrG34BMpmxipFuQJFFzxMDoNuTdNPnlAMEVHpgukBrjr0PBA5wngfBgIlfuqCsdZm099isdfctXgvJTpLEj/B6Q/XgBRGlWOY5y8r54B4dNHvPy56evgNqANSM+0+gGYqw00/vEgrywKmMZ6D4w/araN/6Zbrwf1a+OPe+F6GNZXC8xqF/a9EFe9dR8qAFMabP6L9n3Uz7YVYTgx466hDC8gME26uRhbv2BICzva/bUd6UGjqnCFvWe1dVriXTGk0oJAnXGjVbjf/IDBgx4TbLeBgVWK8fZDhRFIR9FbN+RyRf0qSlg6c7lqr2DVwq6lNYo1JtU6XsOq08f4RUjG9/9V4dniMJkfJl9/Nn34qP823I4+jD6K7kVJdD96GH0eHUUnEYvK6Jfo1+i3yW7y++SPyZ8d9OaN3ueDaO+Z/PUv9JeECg==</latexit><latexit sha1_base64="bg6iQpcsz52Z46e+zRQZx1hbgxs=">AAAG93icfVVLj9xEEHYCGYJ5JXDk0mKEFNDsajxCihRppCgPiQOIBXaTiPGwarfLnmb64XS3dxka/wSucOSGuPJzuPBbqLa9eMe7wdJMl6u+qv6qqqudVYJbN5//fePma6/fmrxx+834rbffefe9O3fff2Z1bRicMC20eZFRC4IrOHHcCXhRGaAyE/A82z4O9udnYCzX6tjtKlhLWipecEYdqr5NK356Zzo/nLcPuSokvTCN+ufo9O6tf9Jcs1qCckxQa1fJvHJrT43jTEATf0wODg7IEWVbWoINL3FaW6g6xUpSU3K1nB8uuFr7ErQEZ3bNZYwHheENdTAjkrqNsYWdEcwjQ9pyRqi0Qd0KbtMp7E5mKIhSG97pRFlZqJGqzjFOxQut3IzYOit4OSNFLUSFe80Isy9r7QB3yKndmFqgzvHtTzOSZRgn03rraIbm8DaQqI3YI70qDJWAyWx0vjzm2+/WXuatLm/iOFVwzrSUVOU+hco2Pj2jBgUutAo1O95V2BVBBJfcPdjHK1SiQ61ybCU4r0hqeLlx1Bh9TlKuCrdDe0A1I09bB0/8P/WcLEnSfK/GkMroHDFhuQxCTl9ikQMnxLVylu27StaskjWuHc5Pk/H+XOWcNR0Cq+mvALKn/1n907HRUDQOmY6sIlgFFBfGfWueQ9GsFmsfpxnggRuasTIaK8m0UWCWSYVnIsMOlr0WJ2Z5vuEO1nF6ZrG34BMpmxipFuQJFFzxMDoNuTdNPnlAMEVHpgukBrjr0PBA5wngfBgIlfuqCsdZm099isdfctXgvJTpLEj/B6Q/XgBRGlWOY5y8r54B4dNHvPy56evgNqANSM+0+gGYqw00/vEgrywKmMZ6D4w/araN/6Zbrwf1a+OPe+F6GNZXC8xqF/a9EFe9dR8qAFMabP6L9n3Uz7YVYTgx466hDC8gME26uRhbv2BICzva/bUd6UGjqnCFvWe1dVriXTGk0oJAnXGjVbjf/IDBgx4TbLeBgVWK8fZDhRFIR9FbN+RyRf0qSlg6c7lqr2DVwq6lNYo1JtU6XsOq08f4RUjG9/9V4dniMJkfJl9/Nn34qP823I4+jD6K7kVJdD96GH0eHUUnEYvK6Jfo1+i3yW7y++SPyZ8d9OaN3ueDaO+Z/PUv9JeECg==</latexit><latexit sha1_base64="bg6iQpcsz52Z46e+zRQZx1hbgxs=">AAAG93icfVVLj9xEEHYCGYJ5JXDk0mKEFNDsajxCihRppCgPiQOIBXaTiPGwarfLnmb64XS3dxka/wSucOSGuPJzuPBbqLa9eMe7wdJMl6u+qv6qqqudVYJbN5//fePma6/fmrxx+834rbffefe9O3fff2Z1bRicMC20eZFRC4IrOHHcCXhRGaAyE/A82z4O9udnYCzX6tjtKlhLWipecEYdqr5NK356Zzo/nLcPuSokvTCN+ufo9O6tf9Jcs1qCckxQa1fJvHJrT43jTEATf0wODg7IEWVbWoINL3FaW6g6xUpSU3K1nB8uuFr7ErQEZ3bNZYwHheENdTAjkrqNsYWdEcwjQ9pyRqi0Qd0KbtMp7E5mKIhSG97pRFlZqJGqzjFOxQut3IzYOit4OSNFLUSFe80Isy9r7QB3yKndmFqgzvHtTzOSZRgn03rraIbm8DaQqI3YI70qDJWAyWx0vjzm2+/WXuatLm/iOFVwzrSUVOU+hco2Pj2jBgUutAo1O95V2BVBBJfcPdjHK1SiQ61ybCU4r0hqeLlx1Bh9TlKuCrdDe0A1I09bB0/8P/WcLEnSfK/GkMroHDFhuQxCTl9ikQMnxLVylu27StaskjWuHc5Pk/H+XOWcNR0Cq+mvALKn/1n907HRUDQOmY6sIlgFFBfGfWueQ9GsFmsfpxnggRuasTIaK8m0UWCWSYVnIsMOlr0WJ2Z5vuEO1nF6ZrG34BMpmxipFuQJFFzxMDoNuTdNPnlAMEVHpgukBrjr0PBA5wngfBgIlfuqCsdZm099isdfctXgvJTpLEj/B6Q/XgBRGlWOY5y8r54B4dNHvPy56evgNqANSM+0+gGYqw00/vEgrywKmMZ6D4w/araN/6Zbrwf1a+OPe+F6GNZXC8xqF/a9EFe9dR8qAFMabP6L9n3Uz7YVYTgx466hDC8gME26uRhbv2BICzva/bUd6UGjqnCFvWe1dVriXTGk0oJAnXGjVbjf/IDBgx4TbLeBgVWK8fZDhRFIR9FbN+RyRf0qSlg6c7lqr2DVwq6lNYo1JtU6XsOq08f4RUjG9/9V4dniMJkfJl9/Nn34qP823I4+jD6K7kVJdD96GH0eHUUnEYvK6Jfo1+i3yW7y++SPyZ8d9OaN3ueDaO+Z/PUv9JeECg==</latexit>

repeat:

1.Sample a task from a distribution: M ~ D.

2.Solve the sampled task M.

Intuition ⇧ R ⇠ D G ⇠ D

Deterministic ⇧d : S 7! A Avg. MDP Avg. VStochastic ⇧s : S 7! Pr(A) Avg. MDP Avg. VAny policy ⇧b : S ⇥ Pr(M ) 7! A Belief MDP Belief MDP

<latexit sha1_base64="ZoT+FLqM4GOC1VdEvuyCKky7+Z8=">AAAKynicnVZbb9s2FHbbdOu0W7M97oVYkiItnMD2S4EAAZo0xS5osKxJL5jpZRRFyWx40UgqmcdpT/uHe9rLfssOJTm2ZWfoJiDW0TmHHw+/c2HiXHDrer2/bt2+s3b3vffvfRB9+NHHn3x6f/2zV1YXhrKXVAtt3sTEMsEVe+m4E+xNbhiRsWCv44unwf76khnLtTpzk5yNJMkUTzklDlTn63f+3EI7Ozvo20Lm1hHj0GkhJTETdEYAI9giHLOMK5/yrDDsURlhypRjhqtsanIkLgQxpRfwlBFqHux0bgrBrhXfKFfwsPGDTXzCNxF6gDaxpP5FibDlEh1tguKrqYwwnkFJnixAHTEIQXIFFHFawZ0naA8FtFNAkwSOo6vPgxJQDy6zXXR8dIIacfPV5hz6qdN0TGZIdgXSidmu0R6uhpuP9kBNUK4Fp5MaL57Dc1wy28ARN7bU+OPy4XLEh5BTljabzH3MbbO1RPDWO/F6be1W251dey1gtxnfemeSf8eUO+YNkQRoVYkhatDrPY7JhFlOVDnP2moCt/5/Gh6gmbwMeVMmktwy9x+y0E4JjrVzWlaEYaaS65aItjAleah5f4Bs01s6RRp0kgjuJohQo61FghGjoKkQROLgbRFQ15QRooJYy+wuwoLETIRm3DPM7rjQpWVZ7znt0PP7G73dXvWgZaHfCBud5jk5X7/7N040LSQ0drXTsN/L3cjDPOAU8KN6SJwQekEy4KyaCoVlea0YwqFgDOz3dgdcjXzGtGTOTMp5H88UwBviWBeFujc2tV0EoymGSSS7iEgb1JXgxrXCTmQMgsi04bVOZJCnAkLVCeDkPNXKdYHWGI7eRWkhRA57dRG1PxfaMdghIXYcstJFjl/82kVxDDix1hfAHJjD1yyIwoiFoIcp1DCDw4x1sn/GL34YeZlUuqSMIqzYFdWQUaAes9yWHl8SAwIXWgXOziY5DFqBBJfc7S36K1DCgkIlMJ2Z8wphw7MxjGCjrxDmKnUTsAevsrUSqggs8HvuOdpH/fJH1XbJjU7AJ7zmnSCmYyA5xBRqK8hxvLhU0nLYH/lqNoGf3+i39+cq4bSsPYBNv+QQP7u2+mdtoyFgnJ20ZRXBKlg6NS5ak4Sl5XAw8tN75zoZQ6OBSaqNYma/n0NNxJDBrNHCJbh/NYaRNIrwpYXcMt+Xsowg1BQukhRukdCgJdre6D/cQ3BEhzYGTVPNEh7COWLQH4YF5r7LQzlr88hjKH+4ikrolwx3g/RvjuSXqSNILeY44CQNewbaHB/y7Ley4cGNmTZMeqrVW0YdtHrpn87koQUBjjFacIY/Yi5K/6J+r3Zq3qU/a4TVbsCvFnCqSdh3Kg4b66KrYHCkmc0/r75b+axSEZoTTlwntP7PosTjadv6AYWwIKP1T5WRxqnFCleQe1rA9JYwK2ZHqZyYuuRGqzDf/MwHCh2uByhHNosKA94iVGgB3EKvlkEsS+qbQgLqzDxrN0RVua0Mq4XVDqpauCKqWh/BjdBvz/9l4dVgt9/b7X8/2Hhy2NwN9zpfdL7sbHf6ncedJ52vOyedlx26trf20xpfexs9j0w0iXztevtWs+bzzsIT/fEPyQSu8Q==</latexit><latexit sha1_base64="ZoT+FLqM4GOC1VdEvuyCKky7+Z8=">AAAKynicnVZbb9s2FHbbdOu0W7M97oVYkiItnMD2S4EAAZo0xS5osKxJL5jpZRRFyWx40UgqmcdpT/uHe9rLfssOJTm2ZWfoJiDW0TmHHw+/c2HiXHDrer2/bt2+s3b3vffvfRB9+NHHn3x6f/2zV1YXhrKXVAtt3sTEMsEVe+m4E+xNbhiRsWCv44unwf76khnLtTpzk5yNJMkUTzklDlTn63f+3EI7Ozvo20Lm1hHj0GkhJTETdEYAI9giHLOMK5/yrDDsURlhypRjhqtsanIkLgQxpRfwlBFqHux0bgrBrhXfKFfwsPGDTXzCNxF6gDaxpP5FibDlEh1tguKrqYwwnkFJnixAHTEIQXIFFHFawZ0naA8FtFNAkwSOo6vPgxJQDy6zXXR8dIIacfPV5hz6qdN0TGZIdgXSidmu0R6uhpuP9kBNUK4Fp5MaL57Dc1wy28ARN7bU+OPy4XLEh5BTljabzH3MbbO1RPDWO/F6be1W251dey1gtxnfemeSf8eUO+YNkQRoVYkhatDrPY7JhFlOVDnP2moCt/5/Gh6gmbwMeVMmktwy9x+y0E4JjrVzWlaEYaaS65aItjAleah5f4Bs01s6RRp0kgjuJohQo61FghGjoKkQROLgbRFQ15QRooJYy+wuwoLETIRm3DPM7rjQpWVZ7znt0PP7G73dXvWgZaHfCBud5jk5X7/7N040LSQ0drXTsN/L3cjDPOAU8KN6SJwQekEy4KyaCoVlea0YwqFgDOz3dgdcjXzGtGTOTMp5H88UwBviWBeFujc2tV0EoymGSSS7iEgb1JXgxrXCTmQMgsi04bVOZJCnAkLVCeDkPNXKdYHWGI7eRWkhRA57dRG1PxfaMdghIXYcstJFjl/82kVxDDix1hfAHJjD1yyIwoiFoIcp1DCDw4x1sn/GL34YeZlUuqSMIqzYFdWQUaAes9yWHl8SAwIXWgXOziY5DFqBBJfc7S36K1DCgkIlMJ2Z8wphw7MxjGCjrxDmKnUTsAevsrUSqggs8HvuOdpH/fJH1XbJjU7AJ7zmnSCmYyA5xBRqK8hxvLhU0nLYH/lqNoGf3+i39+cq4bSsPYBNv+QQP7u2+mdtoyFgnJ20ZRXBKlg6NS5ak4Sl5XAw8tN75zoZQ6OBSaqNYma/n0NNxJDBrNHCJbh/NYaRNIrwpYXcMt+Xsowg1BQukhRukdCgJdre6D/cQ3BEhzYGTVPNEh7COWLQH4YF5r7LQzlr88hjKH+4ikrolwx3g/RvjuSXqSNILeY44CQNewbaHB/y7Ley4cGNmTZMeqrVW0YdtHrpn87koQUBjjFacIY/Yi5K/6J+r3Zq3qU/a4TVbsCvFnCqSdh3Kg4b66KrYHCkmc0/r75b+axSEZoTTlwntP7PosTjadv6AYWwIKP1T5WRxqnFCleQe1rA9JYwK2ZHqZyYuuRGqzDf/MwHCh2uByhHNosKA94iVGgB3EKvlkEsS+qbQgLqzDxrN0RVua0Mq4XVDqpauCKqWh/BjdBvz/9l4dVgt9/b7X8/2Hhy2NwN9zpfdL7sbHf6ncedJ52vOyedlx26trf20xpfexs9j0w0iXztevtWs+bzzsIT/fEPyQSu8Q==</latexit><latexit sha1_base64="ZoT+FLqM4GOC1VdEvuyCKky7+Z8=">AAAKynicnVZbb9s2FHbbdOu0W7M97oVYkiItnMD2S4EAAZo0xS5osKxJL5jpZRRFyWx40UgqmcdpT/uHe9rLfssOJTm2ZWfoJiDW0TmHHw+/c2HiXHDrer2/bt2+s3b3vffvfRB9+NHHn3x6f/2zV1YXhrKXVAtt3sTEMsEVe+m4E+xNbhiRsWCv44unwf76khnLtTpzk5yNJMkUTzklDlTn63f+3EI7Ozvo20Lm1hHj0GkhJTETdEYAI9giHLOMK5/yrDDsURlhypRjhqtsanIkLgQxpRfwlBFqHux0bgrBrhXfKFfwsPGDTXzCNxF6gDaxpP5FibDlEh1tguKrqYwwnkFJnixAHTEIQXIFFHFawZ0naA8FtFNAkwSOo6vPgxJQDy6zXXR8dIIacfPV5hz6qdN0TGZIdgXSidmu0R6uhpuP9kBNUK4Fp5MaL57Dc1wy28ARN7bU+OPy4XLEh5BTljabzH3MbbO1RPDWO/F6be1W251dey1gtxnfemeSf8eUO+YNkQRoVYkhatDrPY7JhFlOVDnP2moCt/5/Gh6gmbwMeVMmktwy9x+y0E4JjrVzWlaEYaaS65aItjAleah5f4Bs01s6RRp0kgjuJohQo61FghGjoKkQROLgbRFQ15QRooJYy+wuwoLETIRm3DPM7rjQpWVZ7znt0PP7G73dXvWgZaHfCBud5jk5X7/7N040LSQ0drXTsN/L3cjDPOAU8KN6SJwQekEy4KyaCoVlea0YwqFgDOz3dgdcjXzGtGTOTMp5H88UwBviWBeFujc2tV0EoymGSSS7iEgb1JXgxrXCTmQMgsi04bVOZJCnAkLVCeDkPNXKdYHWGI7eRWkhRA57dRG1PxfaMdghIXYcstJFjl/82kVxDDix1hfAHJjD1yyIwoiFoIcp1DCDw4x1sn/GL34YeZlUuqSMIqzYFdWQUaAes9yWHl8SAwIXWgXOziY5DFqBBJfc7S36K1DCgkIlMJ2Z8wphw7MxjGCjrxDmKnUTsAevsrUSqggs8HvuOdpH/fJH1XbJjU7AJ7zmnSCmYyA5xBRqK8hxvLhU0nLYH/lqNoGf3+i39+cq4bSsPYBNv+QQP7u2+mdtoyFgnJ20ZRXBKlg6NS5ak4Sl5XAw8tN75zoZQ6OBSaqNYma/n0NNxJDBrNHCJbh/NYaRNIrwpYXcMt+Xsowg1BQukhRukdCgJdre6D/cQ3BEhzYGTVPNEh7COWLQH4YF5r7LQzlr88hjKH+4ikrolwx3g/RvjuSXqSNILeY44CQNewbaHB/y7Ley4cGNmTZMeqrVW0YdtHrpn87koQUBjjFacIY/Yi5K/6J+r3Zq3qU/a4TVbsCvFnCqSdh3Kg4b66KrYHCkmc0/r75b+axSEZoTTlwntP7PosTjadv6AYWwIKP1T5WRxqnFCleQe1rA9JYwK2ZHqZyYuuRGqzDf/MwHCh2uByhHNosKA94iVGgB3EKvlkEsS+qbQgLqzDxrN0RVua0Mq4XVDqpauCKqWh/BjdBvz/9l4dVgt9/b7X8/2Hhy2NwN9zpfdL7sbHf6ncedJ52vOyedlx26trf20xpfexs9j0w0iXztevtWs+bzzsIT/fEPyQSu8Q==</latexit><latexit sha1_base64="ZoT+FLqM4GOC1VdEvuyCKky7+Z8=">AAAKynicnVZbb9s2FHbbdOu0W7M97oVYkiItnMD2S4EAAZo0xS5osKxJL5jpZRRFyWx40UgqmcdpT/uHe9rLfssOJTm2ZWfoJiDW0TmHHw+/c2HiXHDrer2/bt2+s3b3vffvfRB9+NHHn3x6f/2zV1YXhrKXVAtt3sTEMsEVe+m4E+xNbhiRsWCv44unwf76khnLtTpzk5yNJMkUTzklDlTn63f+3EI7Ozvo20Lm1hHj0GkhJTETdEYAI9giHLOMK5/yrDDsURlhypRjhqtsanIkLgQxpRfwlBFqHux0bgrBrhXfKFfwsPGDTXzCNxF6gDaxpP5FibDlEh1tguKrqYwwnkFJnixAHTEIQXIFFHFawZ0naA8FtFNAkwSOo6vPgxJQDy6zXXR8dIIacfPV5hz6qdN0TGZIdgXSidmu0R6uhpuP9kBNUK4Fp5MaL57Dc1wy28ARN7bU+OPy4XLEh5BTljabzH3MbbO1RPDWO/F6be1W251dey1gtxnfemeSf8eUO+YNkQRoVYkhatDrPY7JhFlOVDnP2moCt/5/Gh6gmbwMeVMmktwy9x+y0E4JjrVzWlaEYaaS65aItjAleah5f4Bs01s6RRp0kgjuJohQo61FghGjoKkQROLgbRFQ15QRooJYy+wuwoLETIRm3DPM7rjQpWVZ7znt0PP7G73dXvWgZaHfCBud5jk5X7/7N040LSQ0drXTsN/L3cjDPOAU8KN6SJwQekEy4KyaCoVlea0YwqFgDOz3dgdcjXzGtGTOTMp5H88UwBviWBeFujc2tV0EoymGSSS7iEgb1JXgxrXCTmQMgsi04bVOZJCnAkLVCeDkPNXKdYHWGI7eRWkhRA57dRG1PxfaMdghIXYcstJFjl/82kVxDDix1hfAHJjD1yyIwoiFoIcp1DCDw4x1sn/GL34YeZlUuqSMIqzYFdWQUaAes9yWHl8SAwIXWgXOziY5DFqBBJfc7S36K1DCgkIlMJ2Z8wphw7MxjGCjrxDmKnUTsAevsrUSqggs8HvuOdpH/fJH1XbJjU7AJ7zmnSCmYyA5xBRqK8hxvLhU0nLYH/lqNoGf3+i39+cq4bSsPYBNv+QQP7u2+mdtoyFgnJ20ZRXBKlg6NS5ak4Sl5XAw8tN75zoZQ6OBSaqNYma/n0NNxJDBrNHCJbh/NYaRNIrwpYXcMt+Xsowg1BQukhRukdCgJdre6D/cQ3BEhzYGTVPNEh7COWLQH4YF5r7LQzlr88hjKH+4ikrolwx3g/RvjuSXqSNILeY44CQNewbaHB/y7Ley4cGNmTZMeqrVW0YdtHrpn87koQUBjjFacIY/Yi5K/6J+r3Zq3qU/a4TVbsCvFnCqSdh3Kg4b66KrYHCkmc0/r75b+axSEZoTTlwntP7PosTjadv6AYWwIKP1T5WRxqnFCleQe1rA9JYwK2ZHqZyYuuRGqzDf/MwHCh2uByhHNosKA94iVGgB3EKvlkEsS+qbQgLqzDxrN0RVua0Mq4XVDqpauCKqWh/BjdBvz/9l4dVgt9/b7X8/2Hhy2NwN9zpfdL7sbHf6ncedJ52vOyedlx26trf20xpfexs9j0w0iXztevtWs+bzzsIT/fEPyQSu8Q==</latexit>

(Deterministic)

(Stochastic)

(Any policy)

Probability the action is optimal:

Action Prior

⇡prior(a | s) := PrM⇠D

✓a = argmax

a0Q⇤

M (s, a0)

<latexit sha1_base64="D/2C8+IX3NYmWNrJVNYjdIDK9rM=">AAAFB3icfVPLbhMxFB1ogBJeLSzZWESItppWmQoJqVKkqhSJTUVBLa3IpJFn5s7Eih+D7WkJxh/AR/AN7BBbPoMNW/gM7iSBklTCi5nje861r+8jKTkztt3+funyQuPK1WuL15s3bt66fWdp+e5royqdwmGquNLHCTXAmYRDyyyH41IDFQmHo2T4tOaPTkEbpuSBHZXQE7SQLGcptWjqL72KS9Z3pWZK+xVKYsEyYlbJVofE+7rv9khsmCC7nsQccosKJKguBH3Xd/SRJy/7eydrKyakj1ZJrFkxsKv9pVZ7oz1e5CKIpqAVTNd+f3nhU5yptBIgbcqpMd2oXdqeo9qylINvPiTr6+tkn6ZDWoCpN824MlBODF2BATHZaW9sMtlzBSgBVo/8vxoHEo/X1EJIBLUDbXITEkxMgnkQIaHC1OYxsIOJwYxEgoAXSrOJjRelgQpDVRmeU7JcSRsSUyU5K0KSV5yXeFdIUvO2Uhbwhoyaga442iwbvg9JkuA5iVJDSxOk6915EJXmM0F3c00F4GMGKuscsOGbnhPZ2Jb5ZjOWcJYqIajMXAyl8S4+pRoB40rWOTsYlVhmTjgTzG7N6iUa0aGSGfYGWCenxaNaqzMSM5nbEfK1ys95mqr2xG/fMeyGyJ/IeUmpVYaa+vevCGPawyTXMaFujJNk1lWkvhv18D/RuVY0fz+TGUv9RIHZdBcEybO/rHs2T2qK5PlL51hes3WfT8lZNssg993NnmvGCWDDnRejqxVmMlVagu5EJfZEghUsplYcwc7ZgFnoNeNTg7UFFwnhmxhqTnYhZ5LVs+jJSita3SL4REtamxga4K3nBa/D2QWcDw115l6UdTsrvebG88ikx3kp4rBG/xPSd3+EiOYyx/CcbJo9DdzFO6z44D0OdDQ/vhfB682NqL0RvXzc2t6ZjvZicD94EKwEUfAk2A6eB/vBYZAGX4Mfwc/gV+Nj43PjS+PrRHr50tTnXjCzGt9+A6qqt3c=</latexit><latexit sha1_base64="D/2C8+IX3NYmWNrJVNYjdIDK9rM=">AAAFB3icfVPLbhMxFB1ogBJeLSzZWESItppWmQoJqVKkqhSJTUVBLa3IpJFn5s7Eih+D7WkJxh/AR/AN7BBbPoMNW/gM7iSBklTCi5nje861r+8jKTkztt3+funyQuPK1WuL15s3bt66fWdp+e5royqdwmGquNLHCTXAmYRDyyyH41IDFQmHo2T4tOaPTkEbpuSBHZXQE7SQLGcptWjqL72KS9Z3pWZK+xVKYsEyYlbJVofE+7rv9khsmCC7nsQccosKJKguBH3Xd/SRJy/7eydrKyakj1ZJrFkxsKv9pVZ7oz1e5CKIpqAVTNd+f3nhU5yptBIgbcqpMd2oXdqeo9qylINvPiTr6+tkn6ZDWoCpN824MlBODF2BATHZaW9sMtlzBSgBVo/8vxoHEo/X1EJIBLUDbXITEkxMgnkQIaHC1OYxsIOJwYxEgoAXSrOJjRelgQpDVRmeU7JcSRsSUyU5K0KSV5yXeFdIUvO2Uhbwhoyaga442iwbvg9JkuA5iVJDSxOk6915EJXmM0F3c00F4GMGKuscsOGbnhPZ2Jb5ZjOWcJYqIajMXAyl8S4+pRoB40rWOTsYlVhmTjgTzG7N6iUa0aGSGfYGWCenxaNaqzMSM5nbEfK1ys95mqr2xG/fMeyGyJ/IeUmpVYaa+vevCGPawyTXMaFujJNk1lWkvhv18D/RuVY0fz+TGUv9RIHZdBcEybO/rHs2T2qK5PlL51hes3WfT8lZNssg993NnmvGCWDDnRejqxVmMlVagu5EJfZEghUsplYcwc7ZgFnoNeNTg7UFFwnhmxhqTnYhZ5LVs+jJSita3SL4REtamxga4K3nBa/D2QWcDw115l6UdTsrvebG88ikx3kp4rBG/xPSd3+EiOYyx/CcbJo9DdzFO6z44D0OdDQ/vhfB682NqL0RvXzc2t6ZjvZicD94EKwEUfAk2A6eB/vBYZAGX4Mfwc/gV+Nj43PjS+PrRHr50tTnXjCzGt9+A6qqt3c=</latexit><latexit sha1_base64="D/2C8+IX3NYmWNrJVNYjdIDK9rM=">AAAFB3icfVPLbhMxFB1ogBJeLSzZWESItppWmQoJqVKkqhSJTUVBLa3IpJFn5s7Eih+D7WkJxh/AR/AN7BBbPoMNW/gM7iSBklTCi5nje861r+8jKTkztt3+funyQuPK1WuL15s3bt66fWdp+e5royqdwmGquNLHCTXAmYRDyyyH41IDFQmHo2T4tOaPTkEbpuSBHZXQE7SQLGcptWjqL72KS9Z3pWZK+xVKYsEyYlbJVofE+7rv9khsmCC7nsQccosKJKguBH3Xd/SRJy/7eydrKyakj1ZJrFkxsKv9pVZ7oz1e5CKIpqAVTNd+f3nhU5yptBIgbcqpMd2oXdqeo9qylINvPiTr6+tkn6ZDWoCpN824MlBODF2BATHZaW9sMtlzBSgBVo/8vxoHEo/X1EJIBLUDbXITEkxMgnkQIaHC1OYxsIOJwYxEgoAXSrOJjRelgQpDVRmeU7JcSRsSUyU5K0KSV5yXeFdIUvO2Uhbwhoyaga442iwbvg9JkuA5iVJDSxOk6915EJXmM0F3c00F4GMGKuscsOGbnhPZ2Jb5ZjOWcJYqIajMXAyl8S4+pRoB40rWOTsYlVhmTjgTzG7N6iUa0aGSGfYGWCenxaNaqzMSM5nbEfK1ys95mqr2xG/fMeyGyJ/IeUmpVYaa+vevCGPawyTXMaFujJNk1lWkvhv18D/RuVY0fz+TGUv9RIHZdBcEybO/rHs2T2qK5PlL51hes3WfT8lZNssg993NnmvGCWDDnRejqxVmMlVagu5EJfZEghUsplYcwc7ZgFnoNeNTg7UFFwnhmxhqTnYhZ5LVs+jJSita3SL4REtamxga4K3nBa/D2QWcDw115l6UdTsrvebG88ikx3kp4rBG/xPSd3+EiOYyx/CcbJo9DdzFO6z44D0OdDQ/vhfB682NqL0RvXzc2t6ZjvZicD94EKwEUfAk2A6eB/vBYZAGX4Mfwc/gV+Nj43PjS+PrRHr50tTnXjCzGt9+A6qqt3c=</latexit><latexit sha1_base64="D/2C8+IX3NYmWNrJVNYjdIDK9rM=">AAAFB3icfVPLbhMxFB1ogBJeLSzZWESItppWmQoJqVKkqhSJTUVBLa3IpJFn5s7Eih+D7WkJxh/AR/AN7BBbPoMNW/gM7iSBklTCi5nje861r+8jKTkztt3+funyQuPK1WuL15s3bt66fWdp+e5royqdwmGquNLHCTXAmYRDyyyH41IDFQmHo2T4tOaPTkEbpuSBHZXQE7SQLGcptWjqL72KS9Z3pWZK+xVKYsEyYlbJVofE+7rv9khsmCC7nsQccosKJKguBH3Xd/SRJy/7eydrKyakj1ZJrFkxsKv9pVZ7oz1e5CKIpqAVTNd+f3nhU5yptBIgbcqpMd2oXdqeo9qylINvPiTr6+tkn6ZDWoCpN824MlBODF2BATHZaW9sMtlzBSgBVo/8vxoHEo/X1EJIBLUDbXITEkxMgnkQIaHC1OYxsIOJwYxEgoAXSrOJjRelgQpDVRmeU7JcSRsSUyU5K0KSV5yXeFdIUvO2Uhbwhoyaga442iwbvg9JkuA5iVJDSxOk6915EJXmM0F3c00F4GMGKuscsOGbnhPZ2Jb5ZjOWcJYqIajMXAyl8S4+pRoB40rWOTsYlVhmTjgTzG7N6iUa0aGSGfYGWCenxaNaqzMSM5nbEfK1ys95mqr2xG/fMeyGyJ/IeUmpVYaa+vevCGPawyTXMaFujJNk1lWkvhv18D/RuVY0fz+TGUv9RIHZdBcEybO/rHs2T2qK5PlL51hes3WfT8lZNssg993NnmvGCWDDnRejqxVmMlVagu5EJfZEghUsplYcwc7ZgFnoNeNTg7UFFwnhmxhqTnYhZ5LVs+jJSita3SL4REtamxga4K3nBa/D2QWcDw115l6UdTsrvebG88ikx3kp4rBG/xPSd3+EiOYyx/CcbJo9DdzFO6z44D0OdDQ/vhfB682NqL0RvXzc2t6ZjvZicD94EKwEUfAk2A6eB/vBYZAGX4Mfwc/gV+Nj43PjS+PrRHr50tTnXjCzGt9+A6qqt3c=</latexit>

Average MDP

Pr(M1) = 0.5

Pr(M2) = 0.50.0 1.0

s0

⇡avg<latexit sha1_base64="GmId3aYON40WhY7H6UC86wih5bc=">AAAHg3icfVXbbttGEKWSukrZS5z0qejLIkJgO5ENUQhQwICAIBegDy3qBLITVFSEJTmkttoLu7uUo275Cf3AvvQn+gMdUpRl0U4JSDucOTN75rLLKOfM2MHg786du5/tfd6994X/5Vdff3N//8HDC6MKHcN5rLjS7yNqgDMJ55ZZDu9zDVREHN5Fi5eV/d0StGFKju0qh6mgmWQpi6lF1Wz/r8dhzmZ0mR2aIzIioaAfZ5S8mTlUlR+eHJo+PfIrSK3wH1+JFf70yiEUsXtbbizoQ56SMKNCoMkUYubMQVmDxtdAfXNwRC62Wx0czfZ7g5NB/ZCbQtAIPa95zmYP9v4JExUXAqSNOTVmEgxyO3VUWxZzQMLk+PiYnNF4QTMw1YsfFgbytWIiqM6YHA1OhkxOXQZKgNWr8jrGgcTwmlroE0HtXJvU9AnWNMISij6hwlTqWrDztcKsRIQCz5Rmax3PcgMFUlUJxslZqqTtE1NEKcv6JC04z3GvPonN74WygDsk1Mx1wVFn2eKPPokijBMptbA0QnP1tiVRaL5DepJqKgCTmatkNGaLX6dOJLUuKX0/lHAZK2yOTFwIuSlduKQaBcaVrGo2XuU4IZxwJpg93cVLVKJDIRMcK7BOklCzbG6p1uqShEymdoX2ClW2PHES0FLPA8NhC8oPsg3JtUoQUy3XQcjpZyxyxQlxtRxFu64iLifBFNc1zvWC9v5MJiwu1wisprsBiF5fWd3rtlFTNG4zbVl5ZeWQboy71iSBtJwMp84PI8CB2zZjohVWMlZagh4FOc5EhB3MGi2e3tHlnFmY+uHSYG/BBUKUPlJNyStImWTVMS7JYS84OiWYoiW9IVID3HXb8IrOK8DzoaGq3C95Nc5KP3Ehjr9gssTzkoX9Svo/IP24AaLUqhzDOElTPQ3chS9Y9mfZ1MHOQWkQLlbyN4htoaF0L7fyxKCAaUx3wPijelG6t+v1dlCzlm7cCLfDsL6KY1arat+NOGmsu1AOmNLW5n6q31v9rFtRHU7MeN3QGC8g0GU43xxbN4yRFnZ0/Vd3pAG1qsIk9j4ujFUC74ptKjUI5JJpJav7zW0xOOg+wXZr2LIKMd5uqOoIhK3otRtyuaH+FCUsnb5etU+wqmG30mrFapOqHW9htdb7+EUI2vf/TeFieBIMToI3z3rPXzTfhnve994j79ALvB+8596P3pl37sXev53vOo86ve5e92l32H22ht7pND7fejtPd/Qf20iwXw==</latexit><latexit sha1_base64="GmId3aYON40WhY7H6UC86wih5bc=">AAAHg3icfVXbbttGEKWSukrZS5z0qejLIkJgO5ENUQhQwICAIBegDy3qBLITVFSEJTmkttoLu7uUo275Cf3AvvQn+gMdUpRl0U4JSDucOTN75rLLKOfM2MHg786du5/tfd6994X/5Vdff3N//8HDC6MKHcN5rLjS7yNqgDMJ55ZZDu9zDVREHN5Fi5eV/d0StGFKju0qh6mgmWQpi6lF1Wz/r8dhzmZ0mR2aIzIioaAfZ5S8mTlUlR+eHJo+PfIrSK3wH1+JFf70yiEUsXtbbizoQ56SMKNCoMkUYubMQVmDxtdAfXNwRC62Wx0czfZ7g5NB/ZCbQtAIPa95zmYP9v4JExUXAqSNOTVmEgxyO3VUWxZzQMLk+PiYnNF4QTMw1YsfFgbytWIiqM6YHA1OhkxOXQZKgNWr8jrGgcTwmlroE0HtXJvU9AnWNMISij6hwlTqWrDztcKsRIQCz5Rmax3PcgMFUlUJxslZqqTtE1NEKcv6JC04z3GvPonN74WygDsk1Mx1wVFn2eKPPokijBMptbA0QnP1tiVRaL5DepJqKgCTmatkNGaLX6dOJLUuKX0/lHAZK2yOTFwIuSlduKQaBcaVrGo2XuU4IZxwJpg93cVLVKJDIRMcK7BOklCzbG6p1uqShEymdoX2ClW2PHES0FLPA8NhC8oPsg3JtUoQUy3XQcjpZyxyxQlxtRxFu64iLifBFNc1zvWC9v5MJiwu1wisprsBiF5fWd3rtlFTNG4zbVl5ZeWQboy71iSBtJwMp84PI8CB2zZjohVWMlZagh4FOc5EhB3MGi2e3tHlnFmY+uHSYG/BBUKUPlJNyStImWTVMS7JYS84OiWYoiW9IVID3HXb8IrOK8DzoaGq3C95Nc5KP3Ehjr9gssTzkoX9Svo/IP24AaLUqhzDOElTPQ3chS9Y9mfZ1MHOQWkQLlbyN4htoaF0L7fyxKCAaUx3wPijelG6t+v1dlCzlm7cCLfDsL6KY1arat+NOGmsu1AOmNLW5n6q31v9rFtRHU7MeN3QGC8g0GU43xxbN4yRFnZ0/Vd3pAG1qsIk9j4ujFUC74ptKjUI5JJpJav7zW0xOOg+wXZr2LIKMd5uqOoIhK3otRtyuaH+FCUsnb5etU+wqmG30mrFapOqHW9htdb7+EUI2vf/TeFieBIMToI3z3rPXzTfhnve994j79ALvB+8596P3pl37sXev53vOo86ve5e92l32H22ht7pND7fejtPd/Qf20iwXw==</latexit><latexit sha1_base64="GmId3aYON40WhY7H6UC86wih5bc=">AAAHg3icfVXbbttGEKWSukrZS5z0qejLIkJgO5ENUQhQwICAIBegDy3qBLITVFSEJTmkttoLu7uUo275Cf3AvvQn+gMdUpRl0U4JSDucOTN75rLLKOfM2MHg786du5/tfd6994X/5Vdff3N//8HDC6MKHcN5rLjS7yNqgDMJ55ZZDu9zDVREHN5Fi5eV/d0StGFKju0qh6mgmWQpi6lF1Wz/r8dhzmZ0mR2aIzIioaAfZ5S8mTlUlR+eHJo+PfIrSK3wH1+JFf70yiEUsXtbbizoQ56SMKNCoMkUYubMQVmDxtdAfXNwRC62Wx0czfZ7g5NB/ZCbQtAIPa95zmYP9v4JExUXAqSNOTVmEgxyO3VUWxZzQMLk+PiYnNF4QTMw1YsfFgbytWIiqM6YHA1OhkxOXQZKgNWr8jrGgcTwmlroE0HtXJvU9AnWNMISij6hwlTqWrDztcKsRIQCz5Rmax3PcgMFUlUJxslZqqTtE1NEKcv6JC04z3GvPonN74WygDsk1Mx1wVFn2eKPPokijBMptbA0QnP1tiVRaL5DepJqKgCTmatkNGaLX6dOJLUuKX0/lHAZK2yOTFwIuSlduKQaBcaVrGo2XuU4IZxwJpg93cVLVKJDIRMcK7BOklCzbG6p1uqShEymdoX2ClW2PHES0FLPA8NhC8oPsg3JtUoQUy3XQcjpZyxyxQlxtRxFu64iLifBFNc1zvWC9v5MJiwu1wisprsBiF5fWd3rtlFTNG4zbVl5ZeWQboy71iSBtJwMp84PI8CB2zZjohVWMlZagh4FOc5EhB3MGi2e3tHlnFmY+uHSYG/BBUKUPlJNyStImWTVMS7JYS84OiWYoiW9IVID3HXb8IrOK8DzoaGq3C95Nc5KP3Ehjr9gssTzkoX9Svo/IP24AaLUqhzDOElTPQ3chS9Y9mfZ1MHOQWkQLlbyN4htoaF0L7fyxKCAaUx3wPijelG6t+v1dlCzlm7cCLfDsL6KY1arat+NOGmsu1AOmNLW5n6q31v9rFtRHU7MeN3QGC8g0GU43xxbN4yRFnZ0/Vd3pAG1qsIk9j4ujFUC74ptKjUI5JJpJav7zW0xOOg+wXZr2LIKMd5uqOoIhK3otRtyuaH+FCUsnb5etU+wqmG30mrFapOqHW9htdb7+EUI2vf/TeFieBIMToI3z3rPXzTfhnve994j79ALvB+8596P3pl37sXev53vOo86ve5e92l32H22ht7pND7fejtPd/Qf20iwXw==</latexit><latexit sha1_base64="GmId3aYON40WhY7H6UC86wih5bc=">AAAHg3icfVXbbttGEKWSukrZS5z0qejLIkJgO5ENUQhQwICAIBegDy3qBLITVFSEJTmkttoLu7uUo275Cf3AvvQn+gMdUpRl0U4JSDucOTN75rLLKOfM2MHg786du5/tfd6994X/5Vdff3N//8HDC6MKHcN5rLjS7yNqgDMJ55ZZDu9zDVREHN5Fi5eV/d0StGFKju0qh6mgmWQpi6lF1Wz/r8dhzmZ0mR2aIzIioaAfZ5S8mTlUlR+eHJo+PfIrSK3wH1+JFf70yiEUsXtbbizoQ56SMKNCoMkUYubMQVmDxtdAfXNwRC62Wx0czfZ7g5NB/ZCbQtAIPa95zmYP9v4JExUXAqSNOTVmEgxyO3VUWxZzQMLk+PiYnNF4QTMw1YsfFgbytWIiqM6YHA1OhkxOXQZKgNWr8jrGgcTwmlroE0HtXJvU9AnWNMISij6hwlTqWrDztcKsRIQCz5Rmax3PcgMFUlUJxslZqqTtE1NEKcv6JC04z3GvPonN74WygDsk1Mx1wVFn2eKPPokijBMptbA0QnP1tiVRaL5DepJqKgCTmatkNGaLX6dOJLUuKX0/lHAZK2yOTFwIuSlduKQaBcaVrGo2XuU4IZxwJpg93cVLVKJDIRMcK7BOklCzbG6p1uqShEymdoX2ClW2PHES0FLPA8NhC8oPsg3JtUoQUy3XQcjpZyxyxQlxtRxFu64iLifBFNc1zvWC9v5MJiwu1wisprsBiF5fWd3rtlFTNG4zbVl5ZeWQboy71iSBtJwMp84PI8CB2zZjohVWMlZagh4FOc5EhB3MGi2e3tHlnFmY+uHSYG/BBUKUPlJNyStImWTVMS7JYS84OiWYoiW9IVID3HXb8IrOK8DzoaGq3C95Nc5KP3Ehjr9gssTzkoX9Svo/IP24AaLUqhzDOElTPQ3chS9Y9mfZ1MHOQWkQLlbyN4htoaF0L7fyxKCAaUx3wPijelG6t+v1dlCzlm7cCLfDsL6KY1arat+NOGmsu1AOmNLW5n6q31v9rFtRHU7MeN3QGC8g0GU43xxbN4yRFnZ0/Vd3pAG1qsIk9j4ujFUC74ptKjUI5JJpJav7zW0xOOg+wXZr2LIKMd5uqOoIhK3otRtyuaH+FCUsnb5etU+wqmG30mrFapOqHW9htdb7+EUI2vf/TeFieBIMToI3z3rPXzTfhnve994j79ALvB+8596P3pl37sXev53vOo86ve5e92l32H22ht7pND7fejtPd/Qf20iwXw==</latexit>

0.5 0.5s0

⇡prior<latexit sha1_base64="OOelhi+Dv11Cvs2UvL55MKpQXT4=">AAAHhXicfVXbbttGEKXSpkrZm9M85mVRNbCVyoYoBC1gQGiQC9CHFnUC2QkqKsKSHFJb7YXZXcpRt/yGfl9f+hd975CiIot2SkDa4cyZ2TOXXUY5Z8YOh393bn308e1Punc+9T/7/Isvvzq4+/WFUYWO4TxWXOnXETXAmYRzyyyH17kGKiIOr6Ll08r+agXaMCUndp3DTNBMspTF1KJqfvDXgzBnc7rKjkyfjEko6Ls5JS/mDlXlm4dHZkD7fgVxuWZKl36Nr62Vx+l7l1DE7mW5taAX+Y6EGRUCTaYQc2cOyxo0uQIamMM+udhtdtifH/SGJ8P6IdeFoBF6XvOcze/e/idMVFwIkDbm1JhpMMztzFFtWcwBCZPj42NyRuMlzcBUL35YGMg3iqmgOmNyPDwZMTlzGSgBVq/LqxgHEsNramFABLULbVIzIFjVCIsoBoQKU6lrwS42CrMWEQo8U5ptdDzLDRRIVSUYJ2epknZATBGlLBuQtOA8x70GJDZvC2UBd0ioWeiCo86y5R8DEkUYJ1JqaWmE5uptR6LQfI/0NNVUACazUMl4wpa/zZxIal1S+n4o4TJW2ByZuBByU7pwRTUKjCtZ1WyyznFGOOFMMHu6j5eoRIdCJjhYYJ0koWbZwlKt1SUJmUztGu0Vqmx54iSgpZ4HhuMWlG9kG5JrlSCmWq6CkNMvWOSKE+JqOYr2XUVcToMZrhuc6wXt/ZlMWFxuEFhNdw0QPX9vdc/bRk3RuMu0ZeWVlUO6Ne5bkwTScjqaOT+MAAdu14ypVljJWGkJehzkOBMRdjBrtHh+x5cLZmHmhyuDvQUXCFH6SDUlzyBlklUHuSRHvaB/SjBFS3ojpAa4667hFZ1ngOdDQ1W5X/NqnJV+6EIcf8FkieclCweV9H9A+m4LRKlVOYZxkqZ6GrgLn7Dsz7Kpg12A0iBcrOTvENtCQ+me7uSpQQHTmO2B8Uf1snQvN+vNoGYt3aQRboZhfRXHrNbVvltx2lj3oRwwpZ3N/Vy/t/pZt6I6nJjxpqExXkCgy3CxPbZuFCMt7Ojmr+5IA2pVhUnsfVwYqwTeFbtUahDIFdNKVveb22Fw0H2C7dawYxVivP1Q1REIW9FrN+RyTf0hSlg6fbVqH2BVw26k1YrVJlU73sBqo/fxixC07//rwsXoJBieBC8e9R4/ab4Nd7z73jfekRd4P3iPvZ+8M+/ci71/O/c733YedLvd4+6j7vcb6K1O43PP23u6P/4HxUKxYQ==</latexit><latexit sha1_base64="OOelhi+Dv11Cvs2UvL55MKpQXT4=">AAAHhXicfVXbbttGEKXSpkrZm9M85mVRNbCVyoYoBC1gQGiQC9CHFnUC2QkqKsKSHFJb7YXZXcpRt/yGfl9f+hd975CiIot2SkDa4cyZ2TOXXUY5Z8YOh393bn308e1Punc+9T/7/Isvvzq4+/WFUYWO4TxWXOnXETXAmYRzyyyH17kGKiIOr6Ll08r+agXaMCUndp3DTNBMspTF1KJqfvDXgzBnc7rKjkyfjEko6Ls5JS/mDlXlm4dHZkD7fgVxuWZKl36Nr62Vx+l7l1DE7mW5taAX+Y6EGRUCTaYQc2cOyxo0uQIamMM+udhtdtifH/SGJ8P6IdeFoBF6XvOcze/e/idMVFwIkDbm1JhpMMztzFFtWcwBCZPj42NyRuMlzcBUL35YGMg3iqmgOmNyPDwZMTlzGSgBVq/LqxgHEsNramFABLULbVIzIFjVCIsoBoQKU6lrwS42CrMWEQo8U5ptdDzLDRRIVSUYJ2epknZATBGlLBuQtOA8x70GJDZvC2UBd0ioWeiCo86y5R8DEkUYJ1JqaWmE5uptR6LQfI/0NNVUACazUMl4wpa/zZxIal1S+n4o4TJW2ByZuBByU7pwRTUKjCtZ1WyyznFGOOFMMHu6j5eoRIdCJjhYYJ0koWbZwlKt1SUJmUztGu0Vqmx54iSgpZ4HhuMWlG9kG5JrlSCmWq6CkNMvWOSKE+JqOYr2XUVcToMZrhuc6wXt/ZlMWFxuEFhNdw0QPX9vdc/bRk3RuMu0ZeWVlUO6Ne5bkwTScjqaOT+MAAdu14ypVljJWGkJehzkOBMRdjBrtHh+x5cLZmHmhyuDvQUXCFH6SDUlzyBlklUHuSRHvaB/SjBFS3ojpAa4667hFZ1ngOdDQ1W5X/NqnJV+6EIcf8FkieclCweV9H9A+m4LRKlVOYZxkqZ6GrgLn7Dsz7Kpg12A0iBcrOTvENtCQ+me7uSpQQHTmO2B8Uf1snQvN+vNoGYt3aQRboZhfRXHrNbVvltx2lj3oRwwpZ3N/Vy/t/pZt6I6nJjxpqExXkCgy3CxPbZuFCMt7Ojmr+5IA2pVhUnsfVwYqwTeFbtUahDIFdNKVveb22Fw0H2C7dawYxVivP1Q1REIW9FrN+RyTf0hSlg6fbVqH2BVw26k1YrVJlU73sBqo/fxixC07//rwsXoJBieBC8e9R4/ab4Nd7z73jfekRd4P3iPvZ+8M+/ci71/O/c733YedLvd4+6j7vcb6K1O43PP23u6P/4HxUKxYQ==</latexit><latexit sha1_base64="OOelhi+Dv11Cvs2UvL55MKpQXT4=">AAAHhXicfVXbbttGEKXSpkrZm9M85mVRNbCVyoYoBC1gQGiQC9CHFnUC2QkqKsKSHFJb7YXZXcpRt/yGfl9f+hd975CiIot2SkDa4cyZ2TOXXUY5Z8YOh393bn308e1Punc+9T/7/Isvvzq4+/WFUYWO4TxWXOnXETXAmYRzyyyH17kGKiIOr6Ll08r+agXaMCUndp3DTNBMspTF1KJqfvDXgzBnc7rKjkyfjEko6Ls5JS/mDlXlm4dHZkD7fgVxuWZKl36Nr62Vx+l7l1DE7mW5taAX+Y6EGRUCTaYQc2cOyxo0uQIamMM+udhtdtifH/SGJ8P6IdeFoBF6XvOcze/e/idMVFwIkDbm1JhpMMztzFFtWcwBCZPj42NyRuMlzcBUL35YGMg3iqmgOmNyPDwZMTlzGSgBVq/LqxgHEsNramFABLULbVIzIFjVCIsoBoQKU6lrwS42CrMWEQo8U5ptdDzLDRRIVSUYJ2epknZATBGlLBuQtOA8x70GJDZvC2UBd0ioWeiCo86y5R8DEkUYJ1JqaWmE5uptR6LQfI/0NNVUACazUMl4wpa/zZxIal1S+n4o4TJW2ByZuBByU7pwRTUKjCtZ1WyyznFGOOFMMHu6j5eoRIdCJjhYYJ0koWbZwlKt1SUJmUztGu0Vqmx54iSgpZ4HhuMWlG9kG5JrlSCmWq6CkNMvWOSKE+JqOYr2XUVcToMZrhuc6wXt/ZlMWFxuEFhNdw0QPX9vdc/bRk3RuMu0ZeWVlUO6Ne5bkwTScjqaOT+MAAdu14ypVljJWGkJehzkOBMRdjBrtHh+x5cLZmHmhyuDvQUXCFH6SDUlzyBlklUHuSRHvaB/SjBFS3ojpAa4667hFZ1ngOdDQ1W5X/NqnJV+6EIcf8FkieclCweV9H9A+m4LRKlVOYZxkqZ6GrgLn7Dsz7Kpg12A0iBcrOTvENtCQ+me7uSpQQHTmO2B8Uf1snQvN+vNoGYt3aQRboZhfRXHrNbVvltx2lj3oRwwpZ3N/Vy/t/pZt6I6nJjxpqExXkCgy3CxPbZuFCMt7Ojmr+5IA2pVhUnsfVwYqwTeFbtUahDIFdNKVveb22Fw0H2C7dawYxVivP1Q1REIW9FrN+RyTf0hSlg6fbVqH2BVw26k1YrVJlU73sBqo/fxixC07//rwsXoJBieBC8e9R4/ab4Nd7z73jfekRd4P3iPvZ+8M+/ci71/O/c733YedLvd4+6j7vcb6K1O43PP23u6P/4HxUKxYQ==</latexit><latexit sha1_base64="OOelhi+Dv11Cvs2UvL55MKpQXT4=">AAAHhXicfVXbbttGEKXSpkrZm9M85mVRNbCVyoYoBC1gQGiQC9CHFnUC2QkqKsKSHFJb7YXZXcpRt/yGfl9f+hd975CiIot2SkDa4cyZ2TOXXUY5Z8YOh393bn308e1Punc+9T/7/Isvvzq4+/WFUYWO4TxWXOnXETXAmYRzyyyH17kGKiIOr6Ll08r+agXaMCUndp3DTNBMspTF1KJqfvDXgzBnc7rKjkyfjEko6Ls5JS/mDlXlm4dHZkD7fgVxuWZKl36Nr62Vx+l7l1DE7mW5taAX+Y6EGRUCTaYQc2cOyxo0uQIamMM+udhtdtifH/SGJ8P6IdeFoBF6XvOcze/e/idMVFwIkDbm1JhpMMztzFFtWcwBCZPj42NyRuMlzcBUL35YGMg3iqmgOmNyPDwZMTlzGSgBVq/LqxgHEsNramFABLULbVIzIFjVCIsoBoQKU6lrwS42CrMWEQo8U5ptdDzLDRRIVSUYJ2epknZATBGlLBuQtOA8x70GJDZvC2UBd0ioWeiCo86y5R8DEkUYJ1JqaWmE5uptR6LQfI/0NNVUACazUMl4wpa/zZxIal1S+n4o4TJW2ByZuBByU7pwRTUKjCtZ1WyyznFGOOFMMHu6j5eoRIdCJjhYYJ0koWbZwlKt1SUJmUztGu0Vqmx54iSgpZ4HhuMWlG9kG5JrlSCmWq6CkNMvWOSKE+JqOYr2XUVcToMZrhuc6wXt/ZlMWFxuEFhNdw0QPX9vdc/bRk3RuMu0ZeWVlUO6Ne5bkwTScjqaOT+MAAdu14ypVljJWGkJehzkOBMRdjBrtHh+x5cLZmHmhyuDvQUXCFH6SDUlzyBlklUHuSRHvaB/SjBFS3ojpAa4667hFZ1ngOdDQ1W5X/NqnJV+6EIcf8FkieclCweV9H9A+m4LRKlVOYZxkqZ6GrgLn7Dsz7Kpg12A0iBcrOTvENtCQ+me7uSpQQHTmO2B8Uf1snQvN+vNoGYt3aQRboZhfRXHrNbVvltx2lj3oRwwpZ3N/Vy/t/pZt6I6nJjxpqExXkCgy3CxPbZuFCMt7Ojmr+5IA2pVhUnsfVwYqwTeFbtUahDIFdNKVveb22Fw0H2C7dawYxVivP1Q1REIW9FrN+RyTf0hSlg6fbVqH2BVw26k1YrVJlU73sBqo/fxixC07//rwsXoJBieBC8e9R4/ab4Nd7z73jfekRd4P3iPvZ+8M+/ci71/O/c733YedLvd4+6j7vcb6K1O43PP23u6P/4HxUKxYQ==</latexit>

Choose optimal action in averaged MDP:

⇡prior<latexit sha1_base64="OOelhi+Dv11Cvs2UvL55MKpQXT4=">AAAHhXicfVXbbttGEKXSpkrZm9M85mVRNbCVyoYoBC1gQGiQC9CHFnUC2QkqKsKSHFJb7YXZXcpRt/yGfl9f+hd975CiIot2SkDa4cyZ2TOXXUY5Z8YOh393bn308e1Punc+9T/7/Isvvzq4+/WFUYWO4TxWXOnXETXAmYRzyyyH17kGKiIOr6Ll08r+agXaMCUndp3DTNBMspTF1KJqfvDXgzBnc7rKjkyfjEko6Ls5JS/mDlXlm4dHZkD7fgVxuWZKl36Nr62Vx+l7l1DE7mW5taAX+Y6EGRUCTaYQc2cOyxo0uQIamMM+udhtdtifH/SGJ8P6IdeFoBF6XvOcze/e/idMVFwIkDbm1JhpMMztzFFtWcwBCZPj42NyRuMlzcBUL35YGMg3iqmgOmNyPDwZMTlzGSgBVq/LqxgHEsNramFABLULbVIzIFjVCIsoBoQKU6lrwS42CrMWEQo8U5ptdDzLDRRIVSUYJ2epknZATBGlLBuQtOA8x70GJDZvC2UBd0ioWeiCo86y5R8DEkUYJ1JqaWmE5uptR6LQfI/0NNVUACazUMl4wpa/zZxIal1S+n4o4TJW2ByZuBByU7pwRTUKjCtZ1WyyznFGOOFMMHu6j5eoRIdCJjhYYJ0koWbZwlKt1SUJmUztGu0Vqmx54iSgpZ4HhuMWlG9kG5JrlSCmWq6CkNMvWOSKE+JqOYr2XUVcToMZrhuc6wXt/ZlMWFxuEFhNdw0QPX9vdc/bRk3RuMu0ZeWVlUO6Ne5bkwTScjqaOT+MAAdu14ypVljJWGkJehzkOBMRdjBrtHh+x5cLZmHmhyuDvQUXCFH6SDUlzyBlklUHuSRHvaB/SjBFS3ojpAa4667hFZ1ngOdDQ1W5X/NqnJV+6EIcf8FkieclCweV9H9A+m4LRKlVOYZxkqZ6GrgLn7Dsz7Kpg12A0iBcrOTvENtCQ+me7uSpQQHTmO2B8Uf1snQvN+vNoGYt3aQRboZhfRXHrNbVvltx2lj3oRwwpZ3N/Vy/t/pZt6I6nJjxpqExXkCgy3CxPbZuFCMt7Ojmr+5IA2pVhUnsfVwYqwTeFbtUahDIFdNKVveb22Fw0H2C7dawYxVivP1Q1REIW9FrN+RyTf0hSlg6fbVqH2BVw26k1YrVJlU73sBqo/fxixC07//rwsXoJBieBC8e9R4/ab4Nd7z73jfekRd4P3iPvZ+8M+/ci71/O/c733YedLvd4+6j7vcb6K1O43PP23u6P/4HxUKxYQ==</latexit><latexit sha1_base64="OOelhi+Dv11Cvs2UvL55MKpQXT4=">AAAHhXicfVXbbttGEKXSpkrZm9M85mVRNbCVyoYoBC1gQGiQC9CHFnUC2QkqKsKSHFJb7YXZXcpRt/yGfl9f+hd975CiIot2SkDa4cyZ2TOXXUY5Z8YOh393bn308e1Punc+9T/7/Isvvzq4+/WFUYWO4TxWXOnXETXAmYRzyyyH17kGKiIOr6Ll08r+agXaMCUndp3DTNBMspTF1KJqfvDXgzBnc7rKjkyfjEko6Ls5JS/mDlXlm4dHZkD7fgVxuWZKl36Nr62Vx+l7l1DE7mW5taAX+Y6EGRUCTaYQc2cOyxo0uQIamMM+udhtdtifH/SGJ8P6IdeFoBF6XvOcze/e/idMVFwIkDbm1JhpMMztzFFtWcwBCZPj42NyRuMlzcBUL35YGMg3iqmgOmNyPDwZMTlzGSgBVq/LqxgHEsNramFABLULbVIzIFjVCIsoBoQKU6lrwS42CrMWEQo8U5ptdDzLDRRIVSUYJ2epknZATBGlLBuQtOA8x70GJDZvC2UBd0ioWeiCo86y5R8DEkUYJ1JqaWmE5uptR6LQfI/0NNVUACazUMl4wpa/zZxIal1S+n4o4TJW2ByZuBByU7pwRTUKjCtZ1WyyznFGOOFMMHu6j5eoRIdCJjhYYJ0koWbZwlKt1SUJmUztGu0Vqmx54iSgpZ4HhuMWlG9kG5JrlSCmWq6CkNMvWOSKE+JqOYr2XUVcToMZrhuc6wXt/ZlMWFxuEFhNdw0QPX9vdc/bRk3RuMu0ZeWVlUO6Ne5bkwTScjqaOT+MAAdu14ypVljJWGkJehzkOBMRdjBrtHh+x5cLZmHmhyuDvQUXCFH6SDUlzyBlklUHuSRHvaB/SjBFS3ojpAa4667hFZ1ngOdDQ1W5X/NqnJV+6EIcf8FkieclCweV9H9A+m4LRKlVOYZxkqZ6GrgLn7Dsz7Kpg12A0iBcrOTvENtCQ+me7uSpQQHTmO2B8Uf1snQvN+vNoGYt3aQRboZhfRXHrNbVvltx2lj3oRwwpZ3N/Vy/t/pZt6I6nJjxpqExXkCgy3CxPbZuFCMt7Ojmr+5IA2pVhUnsfVwYqwTeFbtUahDIFdNKVveb22Fw0H2C7dawYxVivP1Q1REIW9FrN+RyTf0hSlg6fbVqH2BVw26k1YrVJlU73sBqo/fxixC07//rwsXoJBieBC8e9R4/ab4Nd7z73jfekRd4P3iPvZ+8M+/ci71/O/c733YedLvd4+6j7vcb6K1O43PP23u6P/4HxUKxYQ==</latexit><latexit sha1_base64="OOelhi+Dv11Cvs2UvL55MKpQXT4=">AAAHhXicfVXbbttGEKXSpkrZm9M85mVRNbCVyoYoBC1gQGiQC9CHFnUC2QkqKsKSHFJb7YXZXcpRt/yGfl9f+hd975CiIot2SkDa4cyZ2TOXXUY5Z8YOh393bn308e1Punc+9T/7/Isvvzq4+/WFUYWO4TxWXOnXETXAmYRzyyyH17kGKiIOr6Ll08r+agXaMCUndp3DTNBMspTF1KJqfvDXgzBnc7rKjkyfjEko6Ls5JS/mDlXlm4dHZkD7fgVxuWZKl36Nr62Vx+l7l1DE7mW5taAX+Y6EGRUCTaYQc2cOyxo0uQIamMM+udhtdtifH/SGJ8P6IdeFoBF6XvOcze/e/idMVFwIkDbm1JhpMMztzFFtWcwBCZPj42NyRuMlzcBUL35YGMg3iqmgOmNyPDwZMTlzGSgBVq/LqxgHEsNramFABLULbVIzIFjVCIsoBoQKU6lrwS42CrMWEQo8U5ptdDzLDRRIVSUYJ2epknZATBGlLBuQtOA8x70GJDZvC2UBd0ioWeiCo86y5R8DEkUYJ1JqaWmE5uptR6LQfI/0NNVUACazUMl4wpa/zZxIal1S+n4o4TJW2ByZuBByU7pwRTUKjCtZ1WyyznFGOOFMMHu6j5eoRIdCJjhYYJ0koWbZwlKt1SUJmUztGu0Vqmx54iSgpZ4HhuMWlG9kG5JrlSCmWq6CkNMvWOSKE+JqOYr2XUVcToMZrhuc6wXt/ZlMWFxuEFhNdw0QPX9vdc/bRk3RuMu0ZeWVlUO6Ne5bkwTScjqaOT+MAAdu14ypVljJWGkJehzkOBMRdjBrtHh+x5cLZmHmhyuDvQUXCFH6SDUlzyBlklUHuSRHvaB/SjBFS3ojpAa4667hFZ1ngOdDQ1W5X/NqnJV+6EIcf8FkieclCweV9H9A+m4LRKlVOYZxkqZ6GrgLn7Dsz7Kpg12A0iBcrOTvENtCQ+me7uSpQQHTmO2B8Uf1snQvN+vNoGYt3aQRboZhfRXHrNbVvltx2lj3oRwwpZ3N/Vy/t/pZt6I6nJjxpqExXkCgy3CxPbZuFCMt7Ojmr+5IA2pVhUnsfVwYqwTeFbtUahDIFdNKVveb22Fw0H2C7dawYxVivP1Q1REIW9FrN+RyTf0hSlg6fbVqH2BVw26k1YrVJlU73sBqo/fxixC07//rwsXoJBieBC8e9R4/ab4Nd7z73jfekRd4P3iPvZ+8M+/ci71/O/c733YedLvd4+6j7vcb6K1O43PP23u6P/4HxUKxYQ==</latexit><latexit sha1_base64="OOelhi+Dv11Cvs2UvL55MKpQXT4=">AAAHhXicfVXbbttGEKXSpkrZm9M85mVRNbCVyoYoBC1gQGiQC9CHFnUC2QkqKsKSHFJb7YXZXcpRt/yGfl9f+hd975CiIot2SkDa4cyZ2TOXXUY5Z8YOh393bn308e1Punc+9T/7/Isvvzq4+/WFUYWO4TxWXOnXETXAmYRzyyyH17kGKiIOr6Ll08r+agXaMCUndp3DTNBMspTF1KJqfvDXgzBnc7rKjkyfjEko6Ls5JS/mDlXlm4dHZkD7fgVxuWZKl36Nr62Vx+l7l1DE7mW5taAX+Y6EGRUCTaYQc2cOyxo0uQIamMM+udhtdtifH/SGJ8P6IdeFoBF6XvOcze/e/idMVFwIkDbm1JhpMMztzFFtWcwBCZPj42NyRuMlzcBUL35YGMg3iqmgOmNyPDwZMTlzGSgBVq/LqxgHEsNramFABLULbVIzIFjVCIsoBoQKU6lrwS42CrMWEQo8U5ptdDzLDRRIVSUYJ2epknZATBGlLBuQtOA8x70GJDZvC2UBd0ioWeiCo86y5R8DEkUYJ1JqaWmE5uptR6LQfI/0NNVUACazUMl4wpa/zZxIal1S+n4o4TJW2ByZuBByU7pwRTUKjCtZ1WyyznFGOOFMMHu6j5eoRIdCJjhYYJ0koWbZwlKt1SUJmUztGu0Vqmx54iSgpZ4HhuMWlG9kG5JrlSCmWq6CkNMvWOSKE+JqOYr2XUVcToMZrhuc6wXt/ZlMWFxuEFhNdw0QPX9vdc/bRk3RuMu0ZeWVlUO6Ne5bkwTScjqaOT+MAAdu14ypVljJWGkJehzkOBMRdjBrtHh+x5cLZmHmhyuDvQUXCFH6SDUlzyBlklUHuSRHvaB/SjBFS3ojpAa4667hFZ1ngOdDQ1W5X/NqnJV+6EIcf8FkieclCweV9H9A+m4LRKlVOYZxkqZ6GrgLn7Dsz7Kpg12A0iBcrOTvENtCQ+me7uSpQQHTmO2B8Uf1snQvN+vNoGYt3aQRboZhfRXHrNbVvltx2lj3oRwwpZ3N/Vy/t/pZt6I6nJjxpqExXkCgy3CxPbZuFCMt7Ojmr+5IA2pVhUnsfVwYqwTeFbtUahDIFdNKVveb22Fw0H2C7dawYxVivP1Q1REIW9FrN+RyTf0hSlg6fbVqH2BVw26k1YrVJlU73sBqo/fxixC07//rwsXoJBieBC8e9R4/ab4Nd7z73jfekRd4P3iPvZ+8M+/ci71/O/c733YedLvd4+6j7vcb6K1O43PP23u6P/4HxUKxYQ==</latexit>

⇡avg<latexit sha1_base64="GmId3aYON40WhY7H6UC86wih5bc=">AAAHg3icfVXbbttGEKWSukrZS5z0qejLIkJgO5ENUQhQwICAIBegDy3qBLITVFSEJTmkttoLu7uUo275Cf3AvvQn+gMdUpRl0U4JSDucOTN75rLLKOfM2MHg786du5/tfd6994X/5Vdff3N//8HDC6MKHcN5rLjS7yNqgDMJ55ZZDu9zDVREHN5Fi5eV/d0StGFKju0qh6mgmWQpi6lF1Wz/r8dhzmZ0mR2aIzIioaAfZ5S8mTlUlR+eHJo+PfIrSK3wH1+JFf70yiEUsXtbbizoQ56SMKNCoMkUYubMQVmDxtdAfXNwRC62Wx0czfZ7g5NB/ZCbQtAIPa95zmYP9v4JExUXAqSNOTVmEgxyO3VUWxZzQMLk+PiYnNF4QTMw1YsfFgbytWIiqM6YHA1OhkxOXQZKgNWr8jrGgcTwmlroE0HtXJvU9AnWNMISij6hwlTqWrDztcKsRIQCz5Rmax3PcgMFUlUJxslZqqTtE1NEKcv6JC04z3GvPonN74WygDsk1Mx1wVFn2eKPPokijBMptbA0QnP1tiVRaL5DepJqKgCTmatkNGaLX6dOJLUuKX0/lHAZK2yOTFwIuSlduKQaBcaVrGo2XuU4IZxwJpg93cVLVKJDIRMcK7BOklCzbG6p1uqShEymdoX2ClW2PHES0FLPA8NhC8oPsg3JtUoQUy3XQcjpZyxyxQlxtRxFu64iLifBFNc1zvWC9v5MJiwu1wisprsBiF5fWd3rtlFTNG4zbVl5ZeWQboy71iSBtJwMp84PI8CB2zZjohVWMlZagh4FOc5EhB3MGi2e3tHlnFmY+uHSYG/BBUKUPlJNyStImWTVMS7JYS84OiWYoiW9IVID3HXb8IrOK8DzoaGq3C95Nc5KP3Ehjr9gssTzkoX9Svo/IP24AaLUqhzDOElTPQ3chS9Y9mfZ1MHOQWkQLlbyN4htoaF0L7fyxKCAaUx3wPijelG6t+v1dlCzlm7cCLfDsL6KY1arat+NOGmsu1AOmNLW5n6q31v9rFtRHU7MeN3QGC8g0GU43xxbN4yRFnZ0/Vd3pAG1qsIk9j4ujFUC74ptKjUI5JJpJav7zW0xOOg+wXZr2LIKMd5uqOoIhK3otRtyuaH+FCUsnb5etU+wqmG30mrFapOqHW9htdb7+EUI2vf/TeFieBIMToI3z3rPXzTfhnve994j79ALvB+8596P3pl37sXev53vOo86ve5e92l32H22ht7pND7fejtPd/Qf20iwXw==</latexit><latexit sha1_base64="GmId3aYON40WhY7H6UC86wih5bc=">AAAHg3icfVXbbttGEKWSukrZS5z0qejLIkJgO5ENUQhQwICAIBegDy3qBLITVFSEJTmkttoLu7uUo275Cf3AvvQn+gMdUpRl0U4JSDucOTN75rLLKOfM2MHg786du5/tfd6994X/5Vdff3N//8HDC6MKHcN5rLjS7yNqgDMJ55ZZDu9zDVREHN5Fi5eV/d0StGFKju0qh6mgmWQpi6lF1Wz/r8dhzmZ0mR2aIzIioaAfZ5S8mTlUlR+eHJo+PfIrSK3wH1+JFf70yiEUsXtbbizoQ56SMKNCoMkUYubMQVmDxtdAfXNwRC62Wx0czfZ7g5NB/ZCbQtAIPa95zmYP9v4JExUXAqSNOTVmEgxyO3VUWxZzQMLk+PiYnNF4QTMw1YsfFgbytWIiqM6YHA1OhkxOXQZKgNWr8jrGgcTwmlroE0HtXJvU9AnWNMISij6hwlTqWrDztcKsRIQCz5Rmax3PcgMFUlUJxslZqqTtE1NEKcv6JC04z3GvPonN74WygDsk1Mx1wVFn2eKPPokijBMptbA0QnP1tiVRaL5DepJqKgCTmatkNGaLX6dOJLUuKX0/lHAZK2yOTFwIuSlduKQaBcaVrGo2XuU4IZxwJpg93cVLVKJDIRMcK7BOklCzbG6p1uqShEymdoX2ClW2PHES0FLPA8NhC8oPsg3JtUoQUy3XQcjpZyxyxQlxtRxFu64iLifBFNc1zvWC9v5MJiwu1wisprsBiF5fWd3rtlFTNG4zbVl5ZeWQboy71iSBtJwMp84PI8CB2zZjohVWMlZagh4FOc5EhB3MGi2e3tHlnFmY+uHSYG/BBUKUPlJNyStImWTVMS7JYS84OiWYoiW9IVID3HXb8IrOK8DzoaGq3C95Nc5KP3Ehjr9gssTzkoX9Svo/IP24AaLUqhzDOElTPQ3chS9Y9mfZ1MHOQWkQLlbyN4htoaF0L7fyxKCAaUx3wPijelG6t+v1dlCzlm7cCLfDsL6KY1arat+NOGmsu1AOmNLW5n6q31v9rFtRHU7MeN3QGC8g0GU43xxbN4yRFnZ0/Vd3pAG1qsIk9j4ujFUC74ptKjUI5JJpJav7zW0xOOg+wXZr2LIKMd5uqOoIhK3otRtyuaH+FCUsnb5etU+wqmG30mrFapOqHW9htdb7+EUI2vf/TeFieBIMToI3z3rPXzTfhnve994j79ALvB+8596P3pl37sXev53vOo86ve5e92l32H22ht7pND7fejtPd/Qf20iwXw==</latexit><latexit sha1_base64="GmId3aYON40WhY7H6UC86wih5bc=">AAAHg3icfVXbbttGEKWSukrZS5z0qejLIkJgO5ENUQhQwICAIBegDy3qBLITVFSEJTmkttoLu7uUo275Cf3AvvQn+gMdUpRl0U4JSDucOTN75rLLKOfM2MHg786du5/tfd6994X/5Vdff3N//8HDC6MKHcN5rLjS7yNqgDMJ55ZZDu9zDVREHN5Fi5eV/d0StGFKju0qh6mgmWQpi6lF1Wz/r8dhzmZ0mR2aIzIioaAfZ5S8mTlUlR+eHJo+PfIrSK3wH1+JFf70yiEUsXtbbizoQ56SMKNCoMkUYubMQVmDxtdAfXNwRC62Wx0czfZ7g5NB/ZCbQtAIPa95zmYP9v4JExUXAqSNOTVmEgxyO3VUWxZzQMLk+PiYnNF4QTMw1YsfFgbytWIiqM6YHA1OhkxOXQZKgNWr8jrGgcTwmlroE0HtXJvU9AnWNMISij6hwlTqWrDztcKsRIQCz5Rmax3PcgMFUlUJxslZqqTtE1NEKcv6JC04z3GvPonN74WygDsk1Mx1wVFn2eKPPokijBMptbA0QnP1tiVRaL5DepJqKgCTmatkNGaLX6dOJLUuKX0/lHAZK2yOTFwIuSlduKQaBcaVrGo2XuU4IZxwJpg93cVLVKJDIRMcK7BOklCzbG6p1uqShEymdoX2ClW2PHES0FLPA8NhC8oPsg3JtUoQUy3XQcjpZyxyxQlxtRxFu64iLifBFNc1zvWC9v5MJiwu1wisprsBiF5fWd3rtlFTNG4zbVl5ZeWQboy71iSBtJwMp84PI8CB2zZjohVWMlZagh4FOc5EhB3MGi2e3tHlnFmY+uHSYG/BBUKUPlJNyStImWTVMS7JYS84OiWYoiW9IVID3HXb8IrOK8DzoaGq3C95Nc5KP3Ehjr9gssTzkoX9Svo/IP24AaLUqhzDOElTPQ3chS9Y9mfZ1MHOQWkQLlbyN4htoaF0L7fyxKCAaUx3wPijelG6t+v1dlCzlm7cCLfDsL6KY1arat+NOGmsu1AOmNLW5n6q31v9rFtRHU7MeN3QGC8g0GU43xxbN4yRFnZ0/Vd3pAG1qsIk9j4ujFUC74ptKjUI5JJpJav7zW0xOOg+wXZr2LIKMd5uqOoIhK3otRtyuaH+FCUsnb5etU+wqmG30mrFapOqHW9htdb7+EUI2vf/TeFieBIMToI3z3rPXzTfhnve994j79ALvB+8596P3pl37sXev53vOo86ve5e92l32H22ht7pND7fejtPd/Qf20iwXw==</latexit><latexit sha1_base64="GmId3aYON40WhY7H6UC86wih5bc=">AAAHg3icfVXbbttGEKWSukrZS5z0qejLIkJgO5ENUQhQwICAIBegDy3qBLITVFSEJTmkttoLu7uUo275Cf3AvvQn+gMdUpRl0U4JSDucOTN75rLLKOfM2MHg786du5/tfd6994X/5Vdff3N//8HDC6MKHcN5rLjS7yNqgDMJ55ZZDu9zDVREHN5Fi5eV/d0StGFKju0qh6mgmWQpi6lF1Wz/r8dhzmZ0mR2aIzIioaAfZ5S8mTlUlR+eHJo+PfIrSK3wH1+JFf70yiEUsXtbbizoQ56SMKNCoMkUYubMQVmDxtdAfXNwRC62Wx0czfZ7g5NB/ZCbQtAIPa95zmYP9v4JExUXAqSNOTVmEgxyO3VUWxZzQMLk+PiYnNF4QTMw1YsfFgbytWIiqM6YHA1OhkxOXQZKgNWr8jrGgcTwmlroE0HtXJvU9AnWNMISij6hwlTqWrDztcKsRIQCz5Rmax3PcgMFUlUJxslZqqTtE1NEKcv6JC04z3GvPonN74WygDsk1Mx1wVFn2eKPPokijBMptbA0QnP1tiVRaL5DepJqKgCTmatkNGaLX6dOJLUuKX0/lHAZK2yOTFwIuSlduKQaBcaVrGo2XuU4IZxwJpg93cVLVKJDIRMcK7BOklCzbG6p1uqShEymdoX2ClW2PHES0FLPA8NhC8oPsg3JtUoQUy3XQcjpZyxyxQlxtRxFu64iLifBFNc1zvWC9v5MJiwu1wisprsBiF5fWd3rtlFTNG4zbVl5ZeWQboy71iSBtJwMp84PI8CB2zZjohVWMlZagh4FOc5EhB3MGi2e3tHlnFmY+uHSYG/BBUKUPlJNyStImWTVMS7JYS84OiWYoiW9IVID3HXb8IrOK8DzoaGq3C95Nc5KP3Ehjr9gssTzkoX9Svo/IP24AaLUqhzDOElTPQ3chS9Y9mfZ1MHOQWkQLlbyN4htoaF0L7fyxKCAaUx3wPijelG6t+v1dlCzlm7cCLfDsL6KY1arat+NOGmsu1AOmNLW5n6q31v9rFtRHU7MeN3QGC8g0GU43xxbN4yRFnZ0/Vd3pAG1qsIk9j4ujFUC74ptKjUI5JJpJav7zW0xOOg+wXZr2LIKMd5uqOoIhK3otRtyuaH+FCUsnb5etU+wqmG30mrFapOqHW9htdb7+EUI2vf/TeFieBIMToI3z3rPXzTfhnve994j79ALvB+8596P3pl37sXev53vOo86ve5e92l32H22ht7pND7fejtPd/Qf20iwXw==</latexit>

⇡u<latexit sha1_base64="9mgtGTWKtB9Hoo9/Bo5ulMC7Olo=">AAAHf3icfVXrjttEFHYKJcXctiB+IcGIqNrdNhvFERJipUhVLxI/QGyr7LYizkZj+9gZMhczM842DH4AHpE/vAKvwLHjNBvvFkvJHJ/zzZnv3MZRzpmxw+HfnTvvvX/3g+69D/2PPv7k088O7n9+YVShYziPFVf6dUQNcCbh3DLL4XWugYqIw6to+bSyv1qBNkzJiV3nMBM0kyxlMbWomh/89SDM2ZyusiNzTMYkFPTNnJIXc4eq8vLhkenTYx8hl4VfI2t9hT19Cw5F7F6WWwviySMSZlQINJlCzJ05LGvQ5Bqobw6PycXumMPj+UFvOBjWD7kpBI3Q85rnbH7/7j9houJCgLQxp8ZMg2FuZ45qy2IOpf+AnJyckDMaL2kGpnrxw8JAvlFMBdUZk+PhYMTkzGWgBFi9Lq9jHEh0r6mFPhHULrRJTZ9gPiNMn+gTKkylrgW72CjMWkQo8ExpttHxLDdQIFWVoJ+cpUraPjFFlLKsT9KC8xzP6pPY/F4oC3hCQs1CFxx1li3/6JMoQj+RUktLIzRXbzsSheZ7pKeppgIwmIVKxhO2/HXmRFLrktL3QwlXscLiyMSFkJvShSuqUWBcySpnk3WO3cEJZ4LZ0328RCVuKGSCLQXWSRJqli0s1VpdkZDJ1K7RXqHK1k7sBLTU/cCw0YLyUrYhuVYJYqrlOgg5/YxJrjghrpajaH+riMtpMMN1g3O9oH0+kwmLyw0Cs+luAKLnb63ueduoKRp3kbasvLJySLfGfWuSQFpORzPnhxFgw+2KMdUKMxkrLUGPgxx7IsIKZo0WJ3d8tWAWZn64MlhbcIEQpY9UU/IMUiZZNcIlOeoFx6cEQ7SkN0JqgKfuCl7ReQY4HxqqzP2SV+2s9EMXYvsLJkuclyzsV9L/AembLRClVuYY+kma7GngLnzCsj/LJg92AUqDcLGSv0FsCw2le7qTpwYFDGO2B8Yf1cvSvdyst4OatXSTRrgdhvlVHKNaV+duxWlj3YdywJB2NvdT/d6qZ12Kajgx4k1BY7yAQJfhYju2bhQjLazo5q+uSANqZYVJrH1cGKsE3hW7UGoQyBXTSlb3m9thsNF9guXWsGMVor99V9UIhC3v9TbkckP9LkqYOn09a+9gVcNupdXy1SZVb7yF1Ubv4xchaN//N4WL0SAYDoIX3/UeP2m+Dfe8r7xvvSMv8L73Hns/emfeuRd7/3a+7Hzd+abb6R52B93hBnqn0+z5wtt7uj/8B8GYrnU=</latexit><latexit sha1_base64="9mgtGTWKtB9Hoo9/Bo5ulMC7Olo=">AAAHf3icfVXrjttEFHYKJcXctiB+IcGIqNrdNhvFERJipUhVLxI/QGyr7LYizkZj+9gZMhczM842DH4AHpE/vAKvwLHjNBvvFkvJHJ/zzZnv3MZRzpmxw+HfnTvvvX/3g+69D/2PPv7k088O7n9+YVShYziPFVf6dUQNcCbh3DLL4XWugYqIw6to+bSyv1qBNkzJiV3nMBM0kyxlMbWomh/89SDM2ZyusiNzTMYkFPTNnJIXc4eq8vLhkenTYx8hl4VfI2t9hT19Cw5F7F6WWwviySMSZlQINJlCzJ05LGvQ5Bqobw6PycXumMPj+UFvOBjWD7kpBI3Q85rnbH7/7j9houJCgLQxp8ZMg2FuZ45qy2IOpf+AnJyckDMaL2kGpnrxw8JAvlFMBdUZk+PhYMTkzGWgBFi9Lq9jHEh0r6mFPhHULrRJTZ9gPiNMn+gTKkylrgW72CjMWkQo8ExpttHxLDdQIFWVoJ+cpUraPjFFlLKsT9KC8xzP6pPY/F4oC3hCQs1CFxx1li3/6JMoQj+RUktLIzRXbzsSheZ7pKeppgIwmIVKxhO2/HXmRFLrktL3QwlXscLiyMSFkJvShSuqUWBcySpnk3WO3cEJZ4LZ0328RCVuKGSCLQXWSRJqli0s1VpdkZDJ1K7RXqHK1k7sBLTU/cCw0YLyUrYhuVYJYqrlOgg5/YxJrjghrpajaH+riMtpMMN1g3O9oH0+kwmLyw0Cs+luAKLnb63ueduoKRp3kbasvLJySLfGfWuSQFpORzPnhxFgw+2KMdUKMxkrLUGPgxx7IsIKZo0WJ3d8tWAWZn64MlhbcIEQpY9UU/IMUiZZNcIlOeoFx6cEQ7SkN0JqgKfuCl7ReQY4HxqqzP2SV+2s9EMXYvsLJkuclyzsV9L/AembLRClVuYY+kma7GngLnzCsj/LJg92AUqDcLGSv0FsCw2le7qTpwYFDGO2B8Yf1cvSvdyst4OatXSTRrgdhvlVHKNaV+duxWlj3YdywJB2NvdT/d6qZ12Kajgx4k1BY7yAQJfhYju2bhQjLazo5q+uSANqZYVJrH1cGKsE3hW7UGoQyBXTSlb3m9thsNF9guXWsGMVor99V9UIhC3v9TbkckP9LkqYOn09a+9gVcNupdXy1SZVb7yF1Ubv4xchaN//N4WL0SAYDoIX3/UeP2m+Dfe8r7xvvSMv8L73Hns/emfeuRd7/3a+7Hzd+abb6R52B93hBnqn0+z5wtt7uj/8B8GYrnU=</latexit><latexit sha1_base64="9mgtGTWKtB9Hoo9/Bo5ulMC7Olo=">AAAHf3icfVXrjttEFHYKJcXctiB+IcGIqNrdNhvFERJipUhVLxI/QGyr7LYizkZj+9gZMhczM842DH4AHpE/vAKvwLHjNBvvFkvJHJ/zzZnv3MZRzpmxw+HfnTvvvX/3g+69D/2PPv7k088O7n9+YVShYziPFVf6dUQNcCbh3DLL4XWugYqIw6to+bSyv1qBNkzJiV3nMBM0kyxlMbWomh/89SDM2ZyusiNzTMYkFPTNnJIXc4eq8vLhkenTYx8hl4VfI2t9hT19Cw5F7F6WWwviySMSZlQINJlCzJ05LGvQ5Bqobw6PycXumMPj+UFvOBjWD7kpBI3Q85rnbH7/7j9houJCgLQxp8ZMg2FuZ45qy2IOpf+AnJyckDMaL2kGpnrxw8JAvlFMBdUZk+PhYMTkzGWgBFi9Lq9jHEh0r6mFPhHULrRJTZ9gPiNMn+gTKkylrgW72CjMWkQo8ExpttHxLDdQIFWVoJ+cpUraPjFFlLKsT9KC8xzP6pPY/F4oC3hCQs1CFxx1li3/6JMoQj+RUktLIzRXbzsSheZ7pKeppgIwmIVKxhO2/HXmRFLrktL3QwlXscLiyMSFkJvShSuqUWBcySpnk3WO3cEJZ4LZ0328RCVuKGSCLQXWSRJqli0s1VpdkZDJ1K7RXqHK1k7sBLTU/cCw0YLyUrYhuVYJYqrlOgg5/YxJrjghrpajaH+riMtpMMN1g3O9oH0+kwmLyw0Cs+luAKLnb63ueduoKRp3kbasvLJySLfGfWuSQFpORzPnhxFgw+2KMdUKMxkrLUGPgxx7IsIKZo0WJ3d8tWAWZn64MlhbcIEQpY9UU/IMUiZZNcIlOeoFx6cEQ7SkN0JqgKfuCl7ReQY4HxqqzP2SV+2s9EMXYvsLJkuclyzsV9L/AembLRClVuYY+kma7GngLnzCsj/LJg92AUqDcLGSv0FsCw2le7qTpwYFDGO2B8Yf1cvSvdyst4OatXSTRrgdhvlVHKNaV+duxWlj3YdywJB2NvdT/d6qZ12Kajgx4k1BY7yAQJfhYju2bhQjLazo5q+uSANqZYVJrH1cGKsE3hW7UGoQyBXTSlb3m9thsNF9guXWsGMVor99V9UIhC3v9TbkckP9LkqYOn09a+9gVcNupdXy1SZVb7yF1Ubv4xchaN//N4WL0SAYDoIX3/UeP2m+Dfe8r7xvvSMv8L73Hns/emfeuRd7/3a+7Hzd+abb6R52B93hBnqn0+z5wtt7uj/8B8GYrnU=</latexit><latexit sha1_base64="9mgtGTWKtB9Hoo9/Bo5ulMC7Olo=">AAAHf3icfVXrjttEFHYKJcXctiB+IcGIqNrdNhvFERJipUhVLxI/QGyr7LYizkZj+9gZMhczM842DH4AHpE/vAKvwLHjNBvvFkvJHJ/zzZnv3MZRzpmxw+HfnTvvvX/3g+69D/2PPv7k088O7n9+YVShYziPFVf6dUQNcCbh3DLL4XWugYqIw6to+bSyv1qBNkzJiV3nMBM0kyxlMbWomh/89SDM2ZyusiNzTMYkFPTNnJIXc4eq8vLhkenTYx8hl4VfI2t9hT19Cw5F7F6WWwviySMSZlQINJlCzJ05LGvQ5Bqobw6PycXumMPj+UFvOBjWD7kpBI3Q85rnbH7/7j9houJCgLQxp8ZMg2FuZ45qy2IOpf+AnJyckDMaL2kGpnrxw8JAvlFMBdUZk+PhYMTkzGWgBFi9Lq9jHEh0r6mFPhHULrRJTZ9gPiNMn+gTKkylrgW72CjMWkQo8ExpttHxLDdQIFWVoJ+cpUraPjFFlLKsT9KC8xzP6pPY/F4oC3hCQs1CFxx1li3/6JMoQj+RUktLIzRXbzsSheZ7pKeppgIwmIVKxhO2/HXmRFLrktL3QwlXscLiyMSFkJvShSuqUWBcySpnk3WO3cEJZ4LZ0328RCVuKGSCLQXWSRJqli0s1VpdkZDJ1K7RXqHK1k7sBLTU/cCw0YLyUrYhuVYJYqrlOgg5/YxJrjghrpajaH+riMtpMMN1g3O9oH0+kwmLyw0Cs+luAKLnb63ueduoKRp3kbasvLJySLfGfWuSQFpORzPnhxFgw+2KMdUKMxkrLUGPgxx7IsIKZo0WJ3d8tWAWZn64MlhbcIEQpY9UU/IMUiZZNcIlOeoFx6cEQ7SkN0JqgKfuCl7ReQY4HxqqzP2SV+2s9EMXYvsLJkuclyzsV9L/AembLRClVuYY+kma7GngLnzCsj/LJg92AUqDcLGSv0FsCw2le7qTpwYFDGO2B8Yf1cvSvdyst4OatXSTRrgdhvlVHKNaV+duxWlj3YdywJB2NvdT/d6qZ12Kajgx4k1BY7yAQJfhYju2bhQjLazo5q+uSANqZYVJrH1cGKsE3hW7UGoQyBXTSlb3m9thsNF9guXWsGMVor99V9UIhC3v9TbkckP9LkqYOn09a+9gVcNupdXy1SZVb7yF1Ubv4xchaN//N4WL0SAYDoIX3/UeP2m+Dfe8r7xvvSMv8L73Hns/emfeuRd7/3a+7Hzd+abb6R52B93hBnqn0+z5wtt7uj/8B8GYrnU=</latexit>

⇡⇤b

<latexit sha1_base64="FgGScYc2Kms4DnQpOX6khpEcFUE=">AAAHgXicfVXbbttGEKWSpkrZm5M+9KEv2wqBLUc2RCFAARcCglyAPrSoE9hOUFEWluSQ2mov7O5SjrrlF/QL+9J/6B90SFGRRTslIO1wztnZszOzyyjnzNjh8O/Onbsf3fu4e/8T/9PPPv/iy70HDy+MKnQM57HiSr+NqAHOJJxbZjm8zTVQEXF4Ey2eV/ibJWjDlDyzqxymgmaSpSymFl2zvb8ehTmb0WV2YPpkTEJB380oeTVz6CovDw/MgPb9ihJdHvo1t0Yq9sl7eihi97rcIDiDPCZhRoVAyBRi5sx+WZPOrpEGZr9PLrYL7fdne73h8bB+yE0jaIye1zynswf3/gkTFRcCpI05NWYSDHM7dVRbFnMo/Ufk6OiInNJ4QTMw1YsfFgbytWMiqM6YHA+PR0xOXQZKgNWr8jrHgcTwmloYEEHtXJvUDAhmNMIEigGhwlTu2rDztcOsRIQGz5Rmax/PcgMFSlUJxslZqqQdEFNEKcsGJC04z3GtAYnN74WygCsk1Mx1wdFn2eKPAYkijBMptbA0Qrh624ooNN8RPUk1FYCbmatkfMYWv06dSGpfUvp+KOEqVlgcmbgQclO6cEk1GowrWeXsbJVjf3DCmWD2ZJcv0YkTCplgU4F1koSaZXNLtVZXJGQytSvEK1bZmomdgEjdDwxbLSgvZZuSa5Ugpxquk1DTz5jkShPyajuKdqeKuJwEUxzXPNcL2uszmbC4XDMwm+4GIXr5HnUv26CmCG532kJ5hXJIN+AumiSQlpPR1PlhBNhw22JMtMJMxkpL0OMgx56IsIJZ48WzO76aMwtTP1warC24QIjSR6kpeQEpk6w6xCU56AX9E4JbtKQ3QmmAq24LXsl5AXg+NFSZ+yWv2lnpQxdi+wsmSzwvWTiorP8j0ncbIlqtzDGMkzTZ08Bd+Ixlf5ZNHuwclAbhYiV/g9gWGkr3fGtPDBq4jekOGX9UL0r3ej3eTmrG0p01xu00zK/iuKtVte7GnDToLpUDbmmLuZ/q91Y961JUhxN3vC5ojBcQ6DKcb46tG8UoCyu6/qsr0pBaWWESax8XxiqBd8V2KzUJ5JJpJav7zW052Og+wXJr2KoKMd5uqOoIhK3o9TTUcsP9IUmYOn09ax9QVdNuldWK1RZVT7xF1drv4xchaN//N42L0XEwPA5ePek9fdZ8G+5733jfeQde4H3vPfV+9E69cy/2/u183SGdb7t3u/3usDtaU+90mjlfeTtP94f/AGZ9rv8=</latexit><latexit sha1_base64="FgGScYc2Kms4DnQpOX6khpEcFUE=">AAAHgXicfVXbbttGEKWSpkrZm5M+9KEv2wqBLUc2RCFAARcCglyAPrSoE9hOUFEWluSQ2mov7O5SjrrlF/QL+9J/6B90SFGRRTslIO1wztnZszOzyyjnzNjh8O/Onbsf3fu4e/8T/9PPPv/iy70HDy+MKnQM57HiSr+NqAHOJJxbZjm8zTVQEXF4Ey2eV/ibJWjDlDyzqxymgmaSpSymFl2zvb8ehTmb0WV2YPpkTEJB380oeTVz6CovDw/MgPb9ihJdHvo1t0Yq9sl7eihi97rcIDiDPCZhRoVAyBRi5sx+WZPOrpEGZr9PLrYL7fdne73h8bB+yE0jaIye1zynswf3/gkTFRcCpI05NWYSDHM7dVRbFnMo/Ufk6OiInNJ4QTMw1YsfFgbytWMiqM6YHA+PR0xOXQZKgNWr8jrHgcTwmloYEEHtXJvUDAhmNMIEigGhwlTu2rDztcOsRIQGz5Rmax/PcgMFSlUJxslZqqQdEFNEKcsGJC04z3GtAYnN74WygCsk1Mx1wdFn2eKPAYkijBMptbA0Qrh624ooNN8RPUk1FYCbmatkfMYWv06dSGpfUvp+KOEqVlgcmbgQclO6cEk1GowrWeXsbJVjf3DCmWD2ZJcv0YkTCplgU4F1koSaZXNLtVZXJGQytSvEK1bZmomdgEjdDwxbLSgvZZuSa5Ugpxquk1DTz5jkShPyajuKdqeKuJwEUxzXPNcL2uszmbC4XDMwm+4GIXr5HnUv26CmCG532kJ5hXJIN+AumiSQlpPR1PlhBNhw22JMtMJMxkpL0OMgx56IsIJZ48WzO76aMwtTP1warC24QIjSR6kpeQEpk6w6xCU56AX9E4JbtKQ3QmmAq24LXsl5AXg+NFSZ+yWv2lnpQxdi+wsmSzwvWTiorP8j0ncbIlqtzDGMkzTZ08Bd+Ixlf5ZNHuwclAbhYiV/g9gWGkr3fGtPDBq4jekOGX9UL0r3ej3eTmrG0p01xu00zK/iuKtVte7GnDToLpUDbmmLuZ/q91Y961JUhxN3vC5ojBcQ6DKcb46tG8UoCyu6/qsr0pBaWWESax8XxiqBd8V2KzUJ5JJpJav7zW052Og+wXJr2KoKMd5uqOoIhK3o9TTUcsP9IUmYOn09ax9QVdNuldWK1RZVT7xF1drv4xchaN//N42L0XEwPA5ePek9fdZ8G+5733jfeQde4H3vPfV+9E69cy/2/u183SGdb7t3u/3usDtaU+90mjlfeTtP94f/AGZ9rv8=</latexit><latexit sha1_base64="FgGScYc2Kms4DnQpOX6khpEcFUE=">AAAHgXicfVXbbttGEKWSpkrZm5M+9KEv2wqBLUc2RCFAARcCglyAPrSoE9hOUFEWluSQ2mov7O5SjrrlF/QL+9J/6B90SFGRRTslIO1wztnZszOzyyjnzNjh8O/Onbsf3fu4e/8T/9PPPv/iy70HDy+MKnQM57HiSr+NqAHOJJxbZjm8zTVQEXF4Ey2eV/ibJWjDlDyzqxymgmaSpSymFl2zvb8ehTmb0WV2YPpkTEJB380oeTVz6CovDw/MgPb9ihJdHvo1t0Yq9sl7eihi97rcIDiDPCZhRoVAyBRi5sx+WZPOrpEGZr9PLrYL7fdne73h8bB+yE0jaIye1zynswf3/gkTFRcCpI05NWYSDHM7dVRbFnMo/Ufk6OiInNJ4QTMw1YsfFgbytWMiqM6YHA+PR0xOXQZKgNWr8jrHgcTwmloYEEHtXJvUDAhmNMIEigGhwlTu2rDztcOsRIQGz5Rmax/PcgMFSlUJxslZqqQdEFNEKcsGJC04z3GtAYnN74WygCsk1Mx1wdFn2eKPAYkijBMptbA0Qrh624ooNN8RPUk1FYCbmatkfMYWv06dSGpfUvp+KOEqVlgcmbgQclO6cEk1GowrWeXsbJVjf3DCmWD2ZJcv0YkTCplgU4F1koSaZXNLtVZXJGQytSvEK1bZmomdgEjdDwxbLSgvZZuSa5Ugpxquk1DTz5jkShPyajuKdqeKuJwEUxzXPNcL2uszmbC4XDMwm+4GIXr5HnUv26CmCG532kJ5hXJIN+AumiSQlpPR1PlhBNhw22JMtMJMxkpL0OMgx56IsIJZ48WzO76aMwtTP1warC24QIjSR6kpeQEpk6w6xCU56AX9E4JbtKQ3QmmAq24LXsl5AXg+NFSZ+yWv2lnpQxdi+wsmSzwvWTiorP8j0ncbIlqtzDGMkzTZ08Bd+Ixlf5ZNHuwclAbhYiV/g9gWGkr3fGtPDBq4jekOGX9UL0r3ej3eTmrG0p01xu00zK/iuKtVte7GnDToLpUDbmmLuZ/q91Y961JUhxN3vC5ojBcQ6DKcb46tG8UoCyu6/qsr0pBaWWESax8XxiqBd8V2KzUJ5JJpJav7zW052Og+wXJr2KoKMd5uqOoIhK3o9TTUcsP9IUmYOn09ax9QVdNuldWK1RZVT7xF1drv4xchaN//N42L0XEwPA5ePek9fdZ8G+5733jfeQde4H3vPfV+9E69cy/2/u183SGdb7t3u/3usDtaU+90mjlfeTtP94f/AGZ9rv8=</latexit><latexit sha1_base64="FgGScYc2Kms4DnQpOX6khpEcFUE=">AAAHgXicfVXbbttGEKWSpkrZm5M+9KEv2wqBLUc2RCFAARcCglyAPrSoE9hOUFEWluSQ2mov7O5SjrrlF/QL+9J/6B90SFGRRTslIO1wztnZszOzyyjnzNjh8O/Onbsf3fu4e/8T/9PPPv/iy70HDy+MKnQM57HiSr+NqAHOJJxbZjm8zTVQEXF4Ey2eV/ibJWjDlDyzqxymgmaSpSymFl2zvb8ehTmb0WV2YPpkTEJB380oeTVz6CovDw/MgPb9ihJdHvo1t0Yq9sl7eihi97rcIDiDPCZhRoVAyBRi5sx+WZPOrpEGZr9PLrYL7fdne73h8bB+yE0jaIye1zynswf3/gkTFRcCpI05NWYSDHM7dVRbFnMo/Ufk6OiInNJ4QTMw1YsfFgbytWMiqM6YHA+PR0xOXQZKgNWr8jrHgcTwmloYEEHtXJvUDAhmNMIEigGhwlTu2rDztcOsRIQGz5Rmax/PcgMFSlUJxslZqqQdEFNEKcsGJC04z3GtAYnN74WygCsk1Mx1wdFn2eKPAYkijBMptbA0Qrh624ooNN8RPUk1FYCbmatkfMYWv06dSGpfUvp+KOEqVlgcmbgQclO6cEk1GowrWeXsbJVjf3DCmWD2ZJcv0YkTCplgU4F1koSaZXNLtVZXJGQytSvEK1bZmomdgEjdDwxbLSgvZZuSa5Ugpxquk1DTz5jkShPyajuKdqeKuJwEUxzXPNcL2uszmbC4XDMwm+4GIXr5HnUv26CmCG532kJ5hXJIN+AumiSQlpPR1PlhBNhw22JMtMJMxkpL0OMgx56IsIJZ48WzO76aMwtTP1warC24QIjSR6kpeQEpk6w6xCU56AX9E4JbtKQ3QmmAq24LXsl5AXg+NFSZ+yWv2lnpQxdi+wsmSzwvWTiorP8j0ncbIlqtzDGMkzTZ08Bd+Ixlf5ZNHuwclAbhYiV/g9gWGkr3fGtPDBq4jekOGX9UL0r3ej3eTmrG0p01xu00zK/iuKtVte7GnDToLpUDbmmLuZ/q91Y961JUhxN3vC5ojBcQ6DKcb46tG8UoCyu6/qsr0pBaWWESax8XxiqBd8V2KzUJ5JJpJav7zW052Og+wXJr2KoKMd5uqOoIhK3o9TTUcsP9IUmYOn09ax9QVdNuldWK1RZVT7xF1drv4xchaN//N42L0XEwPA5ePek9fdZ8G+5733jfeQde4H3vPfV+9E69cy/2/u183SGdb7t3u/3usDtaU+90mjlfeTtP94f/AGZ9rv8=</latexit>

Qinit(s, a) � Q⇤(s, a)<latexit sha1_base64="g0+n+Rxi15UBWwZ810V1fi+xr9Q=">AAAHyHicfVXdb9s2EJe7de60r3R73AsxI0jSOYEVDNgQIEDRD2AYFiwp0rSY5RiUdJI5i6RKUsk8ji/7E/e2l/0tO8pyHCvpBNg63v3u7se7I5VUJdNmNPqn9+CDDx9+1H/0cfjJp599/sXW4y8vtKxVCq9TWUr1NqEaSibgtWGmhLeVAsqTEt4k8+fe/uYKlGZSnJtFBRNOC8FyllKDqunW39txxab0qtjVe+SYxJz+PqXkbGpR5S6f7Ooh3Qu3cY0G16xWKHtCYia8bGY6VfbEubPpycoFPZhgpnWJC3hHzm7C+ZRNAp/06CZrzFP7yq0s3u9b9KSco0nXfGr1jmtA57dAQ72zRy7WfHf2pluD0cGoechdIWqFQdA+p9PHD/+NM5nWHIRJS6r1OBpVZmKpMiwtwYXbZH9/n5zSdE4L0H4RxrWGaqkYc6oKJo5HB4dMTGwBkoNRC3cbY0FgeEUNDIkvmNK5HhJsTIJ94ENCufbqRjCzpUIveIJCWUjFlrqyqDTUSFVmGKdiuRRmSHSd5KwYkrwuywpzDUmq39XSAGbIqJ6pukSdYfM/hiRJME4i5dzQBM1+tSZRq3KD9DhXlANuZiaz43M2/3ViedboMheGsYDrVGJzRGZjqLSz8RVVKLBSCl+z80WFY1aSknFmjjbxApXoUIsMZxOMxTlSrJgZqpS89nOVmwXaPcp1PHES0NLMA8NZjNyl6EIqJTPE+NdtEHI6wSJ7Tohr5CTZdOWpG0cT2ww14uwg6uZnImOpWyKwmvYOIHl5Y7Uvu0ZF0bjeacdaemsJ+cq4ac0yyN34cGLDOAEcuHUzxkpiJVOpBKjjqMKZSLCDRavFK+D4esYMTML4SmNvwUacuxCp5uQF5P6c4l3gyO4g2jsiuEVDBodIDTDruuGezgvA86HAV+6Xyo+zVE9sjOPPmXB4Xop46KX/A+I10gJR6lSOYZysrZ6C0sbPWPGna+tgZiAVcJtK8Rukplbg7PO1PNYo4DYmG2D8UTV39tXyfT+ofTt73gr3w7C+ssRdLXzelThurZvQEnBLa5v9uVl3+tm0wh9O3PGyoSleQKBcPFsdW3uYIi3s6PKv6UgL6lSFCex9WmsjOd4V6600IBBXTEnh7ze7xuCghwTbrWDNKsZ4m6H8EYg70Rs35HJH/T5KWDp1u2rvYdXA7qXVidUl1Tjew2qpD/GLEHXv/7vCxeFBNDqIzr4bPH3WfhseBV8H3wS7QRR8HzwNfgxOg9dB2vuhd9krerP+T/2qf91fLKEPeq3PV8HG0//rP8sRyAU=</latexit><latexit sha1_base64="g0+n+Rxi15UBWwZ810V1fi+xr9Q=">AAAHyHicfVXdb9s2EJe7de60r3R73AsxI0jSOYEVDNgQIEDRD2AYFiwp0rSY5RiUdJI5i6RKUsk8ji/7E/e2l/0tO8pyHCvpBNg63v3u7se7I5VUJdNmNPqn9+CDDx9+1H/0cfjJp599/sXW4y8vtKxVCq9TWUr1NqEaSibgtWGmhLeVAsqTEt4k8+fe/uYKlGZSnJtFBRNOC8FyllKDqunW39txxab0qtjVe+SYxJz+PqXkbGpR5S6f7Ooh3Qu3cY0G16xWKHtCYia8bGY6VfbEubPpycoFPZhgpnWJC3hHzm7C+ZRNAp/06CZrzFP7yq0s3u9b9KSco0nXfGr1jmtA57dAQ72zRy7WfHf2pluD0cGoechdIWqFQdA+p9PHD/+NM5nWHIRJS6r1OBpVZmKpMiwtwYXbZH9/n5zSdE4L0H4RxrWGaqkYc6oKJo5HB4dMTGwBkoNRC3cbY0FgeEUNDIkvmNK5HhJsTIJ94ENCufbqRjCzpUIveIJCWUjFlrqyqDTUSFVmGKdiuRRmSHSd5KwYkrwuywpzDUmq39XSAGbIqJ6pukSdYfM/hiRJME4i5dzQBM1+tSZRq3KD9DhXlANuZiaz43M2/3ViedboMheGsYDrVGJzRGZjqLSz8RVVKLBSCl+z80WFY1aSknFmjjbxApXoUIsMZxOMxTlSrJgZqpS89nOVmwXaPcp1PHES0NLMA8NZjNyl6EIqJTPE+NdtEHI6wSJ7Tohr5CTZdOWpG0cT2ww14uwg6uZnImOpWyKwmvYOIHl5Y7Uvu0ZF0bjeacdaemsJ+cq4ac0yyN34cGLDOAEcuHUzxkpiJVOpBKjjqMKZSLCDRavFK+D4esYMTML4SmNvwUacuxCp5uQF5P6c4l3gyO4g2jsiuEVDBodIDTDruuGezgvA86HAV+6Xyo+zVE9sjOPPmXB4Xop46KX/A+I10gJR6lSOYZysrZ6C0sbPWPGna+tgZiAVcJtK8Rukplbg7PO1PNYo4DYmG2D8UTV39tXyfT+ofTt73gr3w7C+ssRdLXzelThurZvQEnBLa5v9uVl3+tm0wh9O3PGyoSleQKBcPFsdW3uYIi3s6PKv6UgL6lSFCex9WmsjOd4V6600IBBXTEnh7ze7xuCghwTbrWDNKsZ4m6H8EYg70Rs35HJH/T5KWDp1u2rvYdXA7qXVidUl1Tjew2qpD/GLEHXv/7vCxeFBNDqIzr4bPH3WfhseBV8H3wS7QRR8HzwNfgxOg9dB2vuhd9krerP+T/2qf91fLKEPeq3PV8HG0//rP8sRyAU=</latexit><latexit sha1_base64="g0+n+Rxi15UBWwZ810V1fi+xr9Q=">AAAHyHicfVXdb9s2EJe7de60r3R73AsxI0jSOYEVDNgQIEDRD2AYFiwp0rSY5RiUdJI5i6RKUsk8ji/7E/e2l/0tO8pyHCvpBNg63v3u7se7I5VUJdNmNPqn9+CDDx9+1H/0cfjJp599/sXW4y8vtKxVCq9TWUr1NqEaSibgtWGmhLeVAsqTEt4k8+fe/uYKlGZSnJtFBRNOC8FyllKDqunW39txxab0qtjVe+SYxJz+PqXkbGpR5S6f7Ooh3Qu3cY0G16xWKHtCYia8bGY6VfbEubPpycoFPZhgpnWJC3hHzm7C+ZRNAp/06CZrzFP7yq0s3u9b9KSco0nXfGr1jmtA57dAQ72zRy7WfHf2pluD0cGoechdIWqFQdA+p9PHD/+NM5nWHIRJS6r1OBpVZmKpMiwtwYXbZH9/n5zSdE4L0H4RxrWGaqkYc6oKJo5HB4dMTGwBkoNRC3cbY0FgeEUNDIkvmNK5HhJsTIJ94ENCufbqRjCzpUIveIJCWUjFlrqyqDTUSFVmGKdiuRRmSHSd5KwYkrwuywpzDUmq39XSAGbIqJ6pukSdYfM/hiRJME4i5dzQBM1+tSZRq3KD9DhXlANuZiaz43M2/3ViedboMheGsYDrVGJzRGZjqLSz8RVVKLBSCl+z80WFY1aSknFmjjbxApXoUIsMZxOMxTlSrJgZqpS89nOVmwXaPcp1PHES0NLMA8NZjNyl6EIqJTPE+NdtEHI6wSJ7Tohr5CTZdOWpG0cT2ww14uwg6uZnImOpWyKwmvYOIHl5Y7Uvu0ZF0bjeacdaemsJ+cq4ac0yyN34cGLDOAEcuHUzxkpiJVOpBKjjqMKZSLCDRavFK+D4esYMTML4SmNvwUacuxCp5uQF5P6c4l3gyO4g2jsiuEVDBodIDTDruuGezgvA86HAV+6Xyo+zVE9sjOPPmXB4Xop46KX/A+I10gJR6lSOYZysrZ6C0sbPWPGna+tgZiAVcJtK8Rukplbg7PO1PNYo4DYmG2D8UTV39tXyfT+ofTt73gr3w7C+ssRdLXzelThurZvQEnBLa5v9uVl3+tm0wh9O3PGyoSleQKBcPFsdW3uYIi3s6PKv6UgL6lSFCex9WmsjOd4V6600IBBXTEnh7ze7xuCghwTbrWDNKsZ4m6H8EYg70Rs35HJH/T5KWDp1u2rvYdXA7qXVidUl1Tjew2qpD/GLEHXv/7vCxeFBNDqIzr4bPH3WfhseBV8H3wS7QRR8HzwNfgxOg9dB2vuhd9krerP+T/2qf91fLKEPeq3PV8HG0//rP8sRyAU=</latexit><latexit sha1_base64="g0+n+Rxi15UBWwZ810V1fi+xr9Q=">AAAHyHicfVXdb9s2EJe7de60r3R73AsxI0jSOYEVDNgQIEDRD2AYFiwp0rSY5RiUdJI5i6RKUsk8ji/7E/e2l/0tO8pyHCvpBNg63v3u7se7I5VUJdNmNPqn9+CDDx9+1H/0cfjJp599/sXW4y8vtKxVCq9TWUr1NqEaSibgtWGmhLeVAsqTEt4k8+fe/uYKlGZSnJtFBRNOC8FyllKDqunW39txxab0qtjVe+SYxJz+PqXkbGpR5S6f7Ooh3Qu3cY0G16xWKHtCYia8bGY6VfbEubPpycoFPZhgpnWJC3hHzm7C+ZRNAp/06CZrzFP7yq0s3u9b9KSco0nXfGr1jmtA57dAQ72zRy7WfHf2pluD0cGoechdIWqFQdA+p9PHD/+NM5nWHIRJS6r1OBpVZmKpMiwtwYXbZH9/n5zSdE4L0H4RxrWGaqkYc6oKJo5HB4dMTGwBkoNRC3cbY0FgeEUNDIkvmNK5HhJsTIJ94ENCufbqRjCzpUIveIJCWUjFlrqyqDTUSFVmGKdiuRRmSHSd5KwYkrwuywpzDUmq39XSAGbIqJ6pukSdYfM/hiRJME4i5dzQBM1+tSZRq3KD9DhXlANuZiaz43M2/3ViedboMheGsYDrVGJzRGZjqLSz8RVVKLBSCl+z80WFY1aSknFmjjbxApXoUIsMZxOMxTlSrJgZqpS89nOVmwXaPcp1PHES0NLMA8NZjNyl6EIqJTPE+NdtEHI6wSJ7Tohr5CTZdOWpG0cT2ww14uwg6uZnImOpWyKwmvYOIHl5Y7Uvu0ZF0bjeacdaemsJ+cq4ac0yyN34cGLDOAEcuHUzxkpiJVOpBKjjqMKZSLCDRavFK+D4esYMTML4SmNvwUacuxCp5uQF5P6c4l3gyO4g2jsiuEVDBodIDTDruuGezgvA86HAV+6Xyo+zVE9sjOPPmXB4Xop46KX/A+I10gJR6lSOYZysrZ6C0sbPWPGna+tgZiAVcJtK8Rukplbg7PO1PNYo4DYmG2D8UTV39tXyfT+ofTt73gr3w7C+ssRdLXzelThurZvQEnBLa5v9uVl3+tm0wh9O3PGyoSleQKBcPFsdW3uYIi3s6PKv6UgL6lSFCex9WmsjOd4V6600IBBXTEnh7ze7xuCghwTbrWDNKsZ4m6H8EYg70Rs35HJH/T5KWDp1u2rvYdXA7qXVidUl1Tjew2qpD/GLEHXv/7vCxeFBNDqIzr4bPH3WfhseBV8H3wS7QRR8HzwNfgxOg9dB2vuhd9krerP+T/2qf91fLKEPeq3PV8HG0//rP8sRyAU=</latexit>

Qmax(s, a) := maxM2M

Q⇤M (s, a)

<latexit sha1_base64="WdhKjECfwq6/Kr0IGQExREbf7FA=">AAAHyXicfVXdb9s2EJe7de60r3R73AsxI0jSOYEVDBgWIEDRD2DAFiwpkrSY5RqUdJI5i6RCUkldjk/7E/e0l/0tO8pKHCvpBNg63v3u7se7I5VUJdNmNPqn9+Cjjx9+0n/0afjZ5198+dXG46/PtaxVCmepLKV6k1ANJRNwZpgp4U2lgPKkhNfJ/Lm3v74EpZkUp2ZRwYTTQrCcpdSgarrx92ZcsSm9LLb1DjkkMafvppScTC2q3Nsn23pId0Jcot41C3LQouwRiZnwspnpVNkj506mR9cum+jDBDOtU1zABTm5MfqcTQaf9eAmbcxT+8pdW7zf9+hJOUeTrvnU6i3XgE5vgYZ6a4ecrwhv7Uw3BqO9UfOQu0LUCoOgfY6njx/+G2cyrTkIk5ZU63E0qszEUmVYWoILN8nu7i45pumcFqD9IoxrDdVSMeZUFUwcjvb2mZjYAiQHoxbuNsaCwPCKGhgSXzGlcz0k2JkEG8GHhHLt1Y1gZkuFXvAEhbKQii11ZVFpqJGqzDBOxXIpzJDoOslZMSR5XZYV5hqSVF/U0gBmyKieqbpEnWHz90OSJBgnkXJuaIJmv1qRqFW5RnqcK8oBNzOT2eEpm/8+sTxrdJkLw1jAVSqxOSKzMVTa2fiSKhRYKYWv2emiwjkrSck4MwfreIFKdKhFhsMJxuIgKVbMDFVKXvnBys0C7R7lOp44CWhp5oHhyEburehCKiUzxPjXbRByOsIie06Ia+QkWXflqRtHE9tMNeLsIOrmZyJjqVsisJr2DiB5eWO1L7tGRdG42mnHWnprCfm1cd2aZZC78f7EhnECOHCrZoyVxEqmUglQh1GFM5FgB4tWi3fA4dWMGZiE8aXG3oKNOHchUs3JC8j9OcXLwJHtQbRzQHCLhgz2kRpg1lXDPZ0XgOdDga/cb5UfZ6me2BjHnzPh8LwU8dBL/wfEi6QFotSpHMM4WVs9BaWNn7HiT9fWwcxAKuA2leIPSE2twNnnK3msUcBtTNbA+KNq7uyr5ft+UPt29rQV7odhfWWJu1r4vNfiuLWuQ0vALa1s9tdm3eln0wp/OHHHy4ameAGBcvHs+tja/RRpYUeXf01HWlCnKkxg79NaG8nxrlhtpQGBuGRKCn+/2RUGBz0k2G4FK1YxxlsP5Y9A3IneuCGXO+oPUcLSqdtV+wCrBnYvrU6sLqnG8R5WS32IX4Soe//fFc7396LRXnTyw+Dps/bb8Cj4Nvgu2A6i4MfgafBzcBycBWnvp960N+ux/i/9i/67/vsl9EGv9fkmWHv6f/0HscXISQ==</latexit><latexit sha1_base64="WdhKjECfwq6/Kr0IGQExREbf7FA=">AAAHyXicfVXdb9s2EJe7de60r3R73AsxI0jSOYEVDBgWIEDRD2DAFiwpkrSY5RqUdJI5i6RCUkldjk/7E/e0l/0tO8pKHCvpBNg63v3u7se7I5VUJdNmNPqn9+Cjjx9+0n/0afjZ5198+dXG46/PtaxVCmepLKV6k1ANJRNwZpgp4U2lgPKkhNfJ/Lm3v74EpZkUp2ZRwYTTQrCcpdSgarrx92ZcsSm9LLb1DjkkMafvppScTC2q3Nsn23pId0Jcot41C3LQouwRiZnwspnpVNkj506mR9cum+jDBDOtU1zABTm5MfqcTQaf9eAmbcxT+8pdW7zf9+hJOUeTrvnU6i3XgE5vgYZ6a4ecrwhv7Uw3BqO9UfOQu0LUCoOgfY6njx/+G2cyrTkIk5ZU63E0qszEUmVYWoILN8nu7i45pumcFqD9IoxrDdVSMeZUFUwcjvb2mZjYAiQHoxbuNsaCwPCKGhgSXzGlcz0k2JkEG8GHhHLt1Y1gZkuFXvAEhbKQii11ZVFpqJGqzDBOxXIpzJDoOslZMSR5XZYV5hqSVF/U0gBmyKieqbpEnWHz90OSJBgnkXJuaIJmv1qRqFW5RnqcK8oBNzOT2eEpm/8+sTxrdJkLw1jAVSqxOSKzMVTa2fiSKhRYKYWv2emiwjkrSck4MwfreIFKdKhFhsMJxuIgKVbMDFVKXvnBys0C7R7lOp44CWhp5oHhyEburehCKiUzxPjXbRByOsIie06Ia+QkWXflqRtHE9tMNeLsIOrmZyJjqVsisJr2DiB5eWO1L7tGRdG42mnHWnprCfm1cd2aZZC78f7EhnECOHCrZoyVxEqmUglQh1GFM5FgB4tWi3fA4dWMGZiE8aXG3oKNOHchUs3JC8j9OcXLwJHtQbRzQHCLhgz2kRpg1lXDPZ0XgOdDga/cb5UfZ6me2BjHnzPh8LwU8dBL/wfEi6QFotSpHMM4WVs9BaWNn7HiT9fWwcxAKuA2leIPSE2twNnnK3msUcBtTNbA+KNq7uyr5ft+UPt29rQV7odhfWWJu1r4vNfiuLWuQ0vALa1s9tdm3eln0wp/OHHHy4ameAGBcvHs+tja/RRpYUeXf01HWlCnKkxg79NaG8nxrlhtpQGBuGRKCn+/2RUGBz0k2G4FK1YxxlsP5Y9A3IneuCGXO+oPUcLSqdtV+wCrBnYvrU6sLqnG8R5WS32IX4Soe//fFc7396LRXnTyw+Dps/bb8Cj4Nvgu2A6i4MfgafBzcBycBWnvp960N+ux/i/9i/67/vsl9EGv9fkmWHv6f/0HscXISQ==</latexit><latexit sha1_base64="WdhKjECfwq6/Kr0IGQExREbf7FA=">AAAHyXicfVXdb9s2EJe7de60r3R73AsxI0jSOYEVDBgWIEDRD2DAFiwpkrSY5RqUdJI5i6RCUkldjk/7E/e0l/0tO8pKHCvpBNg63v3u7se7I5VUJdNmNPqn9+Cjjx9+0n/0afjZ5198+dXG46/PtaxVCmepLKV6k1ANJRNwZpgp4U2lgPKkhNfJ/Lm3v74EpZkUp2ZRwYTTQrCcpdSgarrx92ZcsSm9LLb1DjkkMafvppScTC2q3Nsn23pId0Jcot41C3LQouwRiZnwspnpVNkj506mR9cum+jDBDOtU1zABTm5MfqcTQaf9eAmbcxT+8pdW7zf9+hJOUeTrvnU6i3XgE5vgYZ6a4ecrwhv7Uw3BqO9UfOQu0LUCoOgfY6njx/+G2cyrTkIk5ZU63E0qszEUmVYWoILN8nu7i45pumcFqD9IoxrDdVSMeZUFUwcjvb2mZjYAiQHoxbuNsaCwPCKGhgSXzGlcz0k2JkEG8GHhHLt1Y1gZkuFXvAEhbKQii11ZVFpqJGqzDBOxXIpzJDoOslZMSR5XZYV5hqSVF/U0gBmyKieqbpEnWHz90OSJBgnkXJuaIJmv1qRqFW5RnqcK8oBNzOT2eEpm/8+sTxrdJkLw1jAVSqxOSKzMVTa2fiSKhRYKYWv2emiwjkrSck4MwfreIFKdKhFhsMJxuIgKVbMDFVKXvnBys0C7R7lOp44CWhp5oHhyEburehCKiUzxPjXbRByOsIie06Ia+QkWXflqRtHE9tMNeLsIOrmZyJjqVsisJr2DiB5eWO1L7tGRdG42mnHWnprCfm1cd2aZZC78f7EhnECOHCrZoyVxEqmUglQh1GFM5FgB4tWi3fA4dWMGZiE8aXG3oKNOHchUs3JC8j9OcXLwJHtQbRzQHCLhgz2kRpg1lXDPZ0XgOdDga/cb5UfZ6me2BjHnzPh8LwU8dBL/wfEi6QFotSpHMM4WVs9BaWNn7HiT9fWwcxAKuA2leIPSE2twNnnK3msUcBtTNbA+KNq7uyr5ft+UPt29rQV7odhfWWJu1r4vNfiuLWuQ0vALa1s9tdm3eln0wp/OHHHy4ameAGBcvHs+tja/RRpYUeXf01HWlCnKkxg79NaG8nxrlhtpQGBuGRKCn+/2RUGBz0k2G4FK1YxxlsP5Y9A3IneuCGXO+oPUcLSqdtV+wCrBnYvrU6sLqnG8R5WS32IX4Soe//fFc7396LRXnTyw+Dps/bb8Cj4Nvgu2A6i4MfgafBzcBycBWnvp960N+ux/i/9i/67/vsl9EGv9fkmWHv6f/0HscXISQ==</latexit><latexit sha1_base64="WdhKjECfwq6/Kr0IGQExREbf7FA=">AAAHyXicfVXdb9s2EJe7de60r3R73AsxI0jSOYEVDBgWIEDRD2DAFiwpkrSY5RqUdJI5i6RCUkldjk/7E/e0l/0tO8pKHCvpBNg63v3u7se7I5VUJdNmNPqn9+Cjjx9+0n/0afjZ5198+dXG46/PtaxVCmepLKV6k1ANJRNwZpgp4U2lgPKkhNfJ/Lm3v74EpZkUp2ZRwYTTQrCcpdSgarrx92ZcsSm9LLb1DjkkMafvppScTC2q3Nsn23pId0Jcot41C3LQouwRiZnwspnpVNkj506mR9cum+jDBDOtU1zABTm5MfqcTQaf9eAmbcxT+8pdW7zf9+hJOUeTrvnU6i3XgE5vgYZ6a4ecrwhv7Uw3BqO9UfOQu0LUCoOgfY6njx/+G2cyrTkIk5ZU63E0qszEUmVYWoILN8nu7i45pumcFqD9IoxrDdVSMeZUFUwcjvb2mZjYAiQHoxbuNsaCwPCKGhgSXzGlcz0k2JkEG8GHhHLt1Y1gZkuFXvAEhbKQii11ZVFpqJGqzDBOxXIpzJDoOslZMSR5XZYV5hqSVF/U0gBmyKieqbpEnWHz90OSJBgnkXJuaIJmv1qRqFW5RnqcK8oBNzOT2eEpm/8+sTxrdJkLw1jAVSqxOSKzMVTa2fiSKhRYKYWv2emiwjkrSck4MwfreIFKdKhFhsMJxuIgKVbMDFVKXvnBys0C7R7lOp44CWhp5oHhyEburehCKiUzxPjXbRByOsIie06Ia+QkWXflqRtHE9tMNeLsIOrmZyJjqVsisJr2DiB5eWO1L7tGRdG42mnHWnprCfm1cd2aZZC78f7EhnECOHCrZoyVxEqmUglQh1GFM5FgB4tWi3fA4dWMGZiE8aXG3oKNOHchUs3JC8j9OcXLwJHtQbRzQHCLhgz2kRpg1lXDPZ0XgOdDga/cb5UfZ6me2BjHnzPh8LwU8dBL/wfEi6QFotSpHMM4WVs9BaWNn7HiT9fWwcxAKuA2leIPSE2twNnnK3msUcBtTNbA+KNq7uyr5ft+UPt29rQV7odhfWWJu1r4vNfiuLWuQ0vALa1s9tdm3eln0wp/OHHHy4ameAGBcvHs+tja/RRpYUeXf01HWlCnKkxg79NaG8nxrlhtpQGBuGRKCn+/2RUGBz0k2G4FK1YxxlsP5Y9A3IneuCGXO+oPUcLSqdtV+wCrBnYvrU6sLqnG8R5WS32IX4Soe//fFc7396LRXnTyw+Dps/bb8Cj4Nvgu2A6i4MfgafBzcBycBWnvp960N+ux/i/9i/67/vsl9EGv9fkmWHv6f/0HscXISQ==</latexit>

(Constraint) Do not underestimate: 

Qmax(s, a) := maxM2M

Q⇤M (s, a)

<latexit sha1_base64="WdhKjECfwq6/Kr0IGQExREbf7FA=">AAAHyXicfVXdb9s2EJe7de60r3R73AsxI0jSOYEVDBgWIEDRD2DAFiwpkrSY5RqUdJI5i6RCUkldjk/7E/e0l/0tO8pKHCvpBNg63v3u7se7I5VUJdNmNPqn9+Cjjx9+0n/0afjZ5198+dXG46/PtaxVCmepLKV6k1ANJRNwZpgp4U2lgPKkhNfJ/Lm3v74EpZkUp2ZRwYTTQrCcpdSgarrx92ZcsSm9LLb1DjkkMafvppScTC2q3Nsn23pId0Jcot41C3LQouwRiZnwspnpVNkj506mR9cum+jDBDOtU1zABTm5MfqcTQaf9eAmbcxT+8pdW7zf9+hJOUeTrvnU6i3XgE5vgYZ6a4ecrwhv7Uw3BqO9UfOQu0LUCoOgfY6njx/+G2cyrTkIk5ZU63E0qszEUmVYWoILN8nu7i45pumcFqD9IoxrDdVSMeZUFUwcjvb2mZjYAiQHoxbuNsaCwPCKGhgSXzGlcz0k2JkEG8GHhHLt1Y1gZkuFXvAEhbKQii11ZVFpqJGqzDBOxXIpzJDoOslZMSR5XZYV5hqSVF/U0gBmyKieqbpEnWHz90OSJBgnkXJuaIJmv1qRqFW5RnqcK8oBNzOT2eEpm/8+sTxrdJkLw1jAVSqxOSKzMVTa2fiSKhRYKYWv2emiwjkrSck4MwfreIFKdKhFhsMJxuIgKVbMDFVKXvnBys0C7R7lOp44CWhp5oHhyEburehCKiUzxPjXbRByOsIie06Ia+QkWXflqRtHE9tMNeLsIOrmZyJjqVsisJr2DiB5eWO1L7tGRdG42mnHWnprCfm1cd2aZZC78f7EhnECOHCrZoyVxEqmUglQh1GFM5FgB4tWi3fA4dWMGZiE8aXG3oKNOHchUs3JC8j9OcXLwJHtQbRzQHCLhgz2kRpg1lXDPZ0XgOdDga/cb5UfZ6me2BjHnzPh8LwU8dBL/wfEi6QFotSpHMM4WVs9BaWNn7HiT9fWwcxAKuA2leIPSE2twNnnK3msUcBtTNbA+KNq7uyr5ft+UPt29rQV7odhfWWJu1r4vNfiuLWuQ0vALa1s9tdm3eln0wp/OHHHy4ameAGBcvHs+tja/RRpYUeXf01HWlCnKkxg79NaG8nxrlhtpQGBuGRKCn+/2RUGBz0k2G4FK1YxxlsP5Y9A3IneuCGXO+oPUcLSqdtV+wCrBnYvrU6sLqnG8R5WS32IX4Soe//fFc7396LRXnTyw+Dps/bb8Cj4Nvgu2A6i4MfgafBzcBycBWnvp960N+ux/i/9i/67/vsl9EGv9fkmWHv6f/0HscXISQ==</latexit><latexit sha1_base64="WdhKjECfwq6/Kr0IGQExREbf7FA=">AAAHyXicfVXdb9s2EJe7de60r3R73AsxI0jSOYEVDBgWIEDRD2DAFiwpkrSY5RqUdJI5i6RCUkldjk/7E/e0l/0tO8pKHCvpBNg63v3u7se7I5VUJdNmNPqn9+Cjjx9+0n/0afjZ5198+dXG46/PtaxVCmepLKV6k1ANJRNwZpgp4U2lgPKkhNfJ/Lm3v74EpZkUp2ZRwYTTQrCcpdSgarrx92ZcsSm9LLb1DjkkMafvppScTC2q3Nsn23pId0Jcot41C3LQouwRiZnwspnpVNkj506mR9cum+jDBDOtU1zABTm5MfqcTQaf9eAmbcxT+8pdW7zf9+hJOUeTrvnU6i3XgE5vgYZ6a4ecrwhv7Uw3BqO9UfOQu0LUCoOgfY6njx/+G2cyrTkIk5ZU63E0qszEUmVYWoILN8nu7i45pumcFqD9IoxrDdVSMeZUFUwcjvb2mZjYAiQHoxbuNsaCwPCKGhgSXzGlcz0k2JkEG8GHhHLt1Y1gZkuFXvAEhbKQii11ZVFpqJGqzDBOxXIpzJDoOslZMSR5XZYV5hqSVF/U0gBmyKieqbpEnWHz90OSJBgnkXJuaIJmv1qRqFW5RnqcK8oBNzOT2eEpm/8+sTxrdJkLw1jAVSqxOSKzMVTa2fiSKhRYKYWv2emiwjkrSck4MwfreIFKdKhFhsMJxuIgKVbMDFVKXvnBys0C7R7lOp44CWhp5oHhyEburehCKiUzxPjXbRByOsIie06Ia+QkWXflqRtHE9tMNeLsIOrmZyJjqVsisJr2DiB5eWO1L7tGRdG42mnHWnprCfm1cd2aZZC78f7EhnECOHCrZoyVxEqmUglQh1GFM5FgB4tWi3fA4dWMGZiE8aXG3oKNOHchUs3JC8j9OcXLwJHtQbRzQHCLhgz2kRpg1lXDPZ0XgOdDga/cb5UfZ6me2BjHnzPh8LwU8dBL/wfEi6QFotSpHMM4WVs9BaWNn7HiT9fWwcxAKuA2leIPSE2twNnnK3msUcBtTNbA+KNq7uyr5ft+UPt29rQV7odhfWWJu1r4vNfiuLWuQ0vALa1s9tdm3eln0wp/OHHHy4ameAGBcvHs+tja/RRpYUeXf01HWlCnKkxg79NaG8nxrlhtpQGBuGRKCn+/2RUGBz0k2G4FK1YxxlsP5Y9A3IneuCGXO+oPUcLSqdtV+wCrBnYvrU6sLqnG8R5WS32IX4Soe//fFc7396LRXnTyw+Dps/bb8Cj4Nvgu2A6i4MfgafBzcBycBWnvp960N+ux/i/9i/67/vsl9EGv9fkmWHv6f/0HscXISQ==</latexit><latexit sha1_base64="WdhKjECfwq6/Kr0IGQExREbf7FA=">AAAHyXicfVXdb9s2EJe7de60r3R73AsxI0jSOYEVDBgWIEDRD2DAFiwpkrSY5RqUdJI5i6RCUkldjk/7E/e0l/0tO8pKHCvpBNg63v3u7se7I5VUJdNmNPqn9+Cjjx9+0n/0afjZ5198+dXG46/PtaxVCmepLKV6k1ANJRNwZpgp4U2lgPKkhNfJ/Lm3v74EpZkUp2ZRwYTTQrCcpdSgarrx92ZcsSm9LLb1DjkkMafvppScTC2q3Nsn23pId0Jcot41C3LQouwRiZnwspnpVNkj506mR9cum+jDBDOtU1zABTm5MfqcTQaf9eAmbcxT+8pdW7zf9+hJOUeTrvnU6i3XgE5vgYZ6a4ecrwhv7Uw3BqO9UfOQu0LUCoOgfY6njx/+G2cyrTkIk5ZU63E0qszEUmVYWoILN8nu7i45pumcFqD9IoxrDdVSMeZUFUwcjvb2mZjYAiQHoxbuNsaCwPCKGhgSXzGlcz0k2JkEG8GHhHLt1Y1gZkuFXvAEhbKQii11ZVFpqJGqzDBOxXIpzJDoOslZMSR5XZYV5hqSVF/U0gBmyKieqbpEnWHz90OSJBgnkXJuaIJmv1qRqFW5RnqcK8oBNzOT2eEpm/8+sTxrdJkLw1jAVSqxOSKzMVTa2fiSKhRYKYWv2emiwjkrSck4MwfreIFKdKhFhsMJxuIgKVbMDFVKXvnBys0C7R7lOp44CWhp5oHhyEburehCKiUzxPjXbRByOsIie06Ia+QkWXflqRtHE9tMNeLsIOrmZyJjqVsisJr2DiB5eWO1L7tGRdG42mnHWnprCfm1cd2aZZC78f7EhnECOHCrZoyVxEqmUglQh1GFM5FgB4tWi3fA4dWMGZiE8aXG3oKNOHchUs3JC8j9OcXLwJHtQbRzQHCLhgz2kRpg1lXDPZ0XgOdDga/cb5UfZ6me2BjHnzPh8LwU8dBL/wfEi6QFotSpHMM4WVs9BaWNn7HiT9fWwcxAKuA2leIPSE2twNnnK3msUcBtTNbA+KNq7uyr5ft+UPt29rQV7odhfWWJu1r4vNfiuLWuQ0vALa1s9tdm3eln0wp/OHHHy4ameAGBcvHs+tja/RRpYUeXf01HWlCnKkxg79NaG8nxrlhtpQGBuGRKCn+/2RUGBz0k2G4FK1YxxlsP5Y9A3IneuCGXO+oPUcLSqdtV+wCrBnYvrU6sLqnG8R5WS32IX4Soe//fFc7396LRXnTyw+Dps/bb8Cj4Nvgu2A6i4MfgafBzcBycBWnvp960N+ux/i/9i/67/vsl9EGv9fkmWHv6f/0HscXISQ==</latexit><latexit sha1_base64="WdhKjECfwq6/Kr0IGQExREbf7FA=">AAAHyXicfVXdb9s2EJe7de60r3R73AsxI0jSOYEVDBgWIEDRD2DAFiwpkrSY5RqUdJI5i6RCUkldjk/7E/e0l/0tO8pKHCvpBNg63v3u7se7I5VUJdNmNPqn9+Cjjx9+0n/0afjZ5198+dXG46/PtaxVCmepLKV6k1ANJRNwZpgp4U2lgPKkhNfJ/Lm3v74EpZkUp2ZRwYTTQrCcpdSgarrx92ZcsSm9LLb1DjkkMafvppScTC2q3Nsn23pId0Jcot41C3LQouwRiZnwspnpVNkj506mR9cum+jDBDOtU1zABTm5MfqcTQaf9eAmbcxT+8pdW7zf9+hJOUeTrvnU6i3XgE5vgYZ6a4ecrwhv7Uw3BqO9UfOQu0LUCoOgfY6njx/+G2cyrTkIk5ZU63E0qszEUmVYWoILN8nu7i45pumcFqD9IoxrDdVSMeZUFUwcjvb2mZjYAiQHoxbuNsaCwPCKGhgSXzGlcz0k2JkEG8GHhHLt1Y1gZkuFXvAEhbKQii11ZVFpqJGqzDBOxXIpzJDoOslZMSR5XZYV5hqSVF/U0gBmyKieqbpEnWHz90OSJBgnkXJuaIJmv1qRqFW5RnqcK8oBNzOT2eEpm/8+sTxrdJkLw1jAVSqxOSKzMVTa2fiSKhRYKYWv2emiwjkrSck4MwfreIFKdKhFhsMJxuIgKVbMDFVKXvnBys0C7R7lOp44CWhp5oHhyEburehCKiUzxPjXbRByOsIie06Ia+QkWXflqRtHE9tMNeLsIOrmZyJjqVsisJr2DiB5eWO1L7tGRdG42mnHWnprCfm1cd2aZZC78f7EhnECOHCrZoyVxEqmUglQh1GFM5FgB4tWi3fA4dWMGZiE8aXG3oKNOHchUs3JC8j9OcXLwJHtQbRzQHCLhgz2kRpg1lXDPZ0XgOdDga/cb5UfZ6me2BjHnzPh8LwU8dBL/wfEi6QFotSpHMM4WVs9BaWNn7HiT9fWwcxAKuA2leIPSE2twNnnK3msUcBtTNbA+KNq7uyr5ft+UPt29rQV7odhfWWJu1r4vNfiuLWuQ0vALa1s9tdm3eln0wp/OHHHy4ameAGBcvHs+tja/RRpYUeXf01HWlCnKkxg79NaG8nxrlhtpQGBuGRKCn+/2RUGBz0k2G4FK1YxxlsP5Y9A3IneuCGXO+oPUcLSqdtV+wCrBnYvrU6sLqnG8R5WS32IX4Soe//fFc7396LRXnTyw+Dps/bb8Cj4Nvgu2A6i4MfgafBzcBycBWnvp960N+ux/i/9i/67/vsl9EGv9fkmWHv6f/0HscXISQ==</latexit>

8s,a : MaxQInit(s, a) � maxM2M

Q⇤M (s, a)

<latexit sha1_base64="zoT4O9geNsLf661v0eYUZ/wcbzU=">AAAIG3icfVVbb+NEFHYWlizmtoVHXkaEqu2SVnGFhBSp0movEkhUNKu2uyLORmN77AyZi3dm3DYM8wv4DfwIXuGRN8QrD7zwWzjjuE3jtlhKfOac75z5zmXGScmoNoPBP517b719/53ug3fD997/4MOPHm58fKplpVJykkom1asEa8KoICeGGkZelYpgnjDyMpk/9faXZ0RpKsWxWZRkwnEhaE5TbEA13eh8vhmXdIrPim29gw5QzPHFFKPR1ILKvX60rft4J9yENRhcvULDBmYPUUyFl81Mp8oeOjeaHl76xLlUmLGphZVDQxQbcmF0ag/xxegbQU0TLC7ImzvDoWvxPAfa8htdGX0SNWWfxvAqj5in9oW7tHi/L8ATcw4mXXEgt+Vq0PE1UF9v7aDTVQW2dqYPe4O9Qf2gm0LUCL2geY6mG/f/jTOZVpwIkzKs9TgalGZisTI0ZcSFm2h3dxcd4XSOC6L9IowrTcqlYsyxKqg4GOztUzGxBZGcGLVw1zGWCAivsCF95EumdK77CHqdQGt5H2GuvboWzGyp0AuegMAKqehSx4pSkwqoygzilDSXwvSRrpKcFn2UV4yVsFcfpfpNJQ2BHTKsZ6pioDN0/mMfJQnESaScG5yA2a9WJCrF1kiPc4U5gWRmMjs4pvPvJ5ZntS5zYRgLcp5KaI7IbExK7Wx8hhUIlEnha3a8KGFyGWKUUzNcxwtQgkMlMhh3YixMkqLFzGCl5LmfrNwswO5RruUJkwCWeh4onIHIvRZtSKlkBhj/ug4CTodQZM8JcLWcJOuuPHXjaGLrsQac7UXt/anIaOqWCKimvQFInl9Z7fO2UWEwrjJtWZm3MpJfGtetWUZyN96f2DBOCAzcqhljJaGSqVSCqIOohJlIoINFo4Vb5eB8Rg2ZhPGZht4SG3HuQqCao2ck9+cUrheHtnvRzhBBigb19oEagV1XDfd0nhE4H4r4yn1X+nGW6pGNYfw5FQ7OSxH3vfR/QLiYGiBIrcpRiJM11VOE2fgJLX5yTR3MjEhFuE2l+IGkplLE2acreaxBgDQma2D4YTV39sXyfTuoeTt73Ai3w6C+kkFWC7/vpThurOtQRiCllc1+W69b/axb4Q8nZLxsaAoXEFEunl0eW7ufAi3o6PKv7kgDalWFCuh9WmkjOdwVq1RqEBFnVEnh7ze7wsCghwjarciKVQzx1kP5IxC3otduwOWG+i5KUDp1vWp3sKpht9JqxWqTqh1vYbXUh/BFiNr3/03hdH8vGuxFoy97j58034YHwafBZ8F2EAVfBY+Dr4Oj4CRIOz93fu381vm9+0v3j+6f3b+W0HudxueTYO3p/v0f/RjmwA==</latexit><latexit sha1_base64="zoT4O9geNsLf661v0eYUZ/wcbzU=">AAAIG3icfVVbb+NEFHYWlizmtoVHXkaEqu2SVnGFhBSp0movEkhUNKu2uyLORmN77AyZi3dm3DYM8wv4DfwIXuGRN8QrD7zwWzjjuE3jtlhKfOac75z5zmXGScmoNoPBP517b719/53ug3fD997/4MOPHm58fKplpVJykkom1asEa8KoICeGGkZelYpgnjDyMpk/9faXZ0RpKsWxWZRkwnEhaE5TbEA13eh8vhmXdIrPim29gw5QzPHFFKPR1ILKvX60rft4J9yENRhcvULDBmYPUUyFl81Mp8oeOjeaHl76xLlUmLGphZVDQxQbcmF0ag/xxegbQU0TLC7ImzvDoWvxPAfa8htdGX0SNWWfxvAqj5in9oW7tHi/L8ATcw4mXXEgt+Vq0PE1UF9v7aDTVQW2dqYPe4O9Qf2gm0LUCL2geY6mG/f/jTOZVpwIkzKs9TgalGZisTI0ZcSFm2h3dxcd4XSOC6L9IowrTcqlYsyxKqg4GOztUzGxBZGcGLVw1zGWCAivsCF95EumdK77CHqdQGt5H2GuvboWzGyp0AuegMAKqehSx4pSkwqoygzilDSXwvSRrpKcFn2UV4yVsFcfpfpNJQ2BHTKsZ6pioDN0/mMfJQnESaScG5yA2a9WJCrF1kiPc4U5gWRmMjs4pvPvJ5ZntS5zYRgLcp5KaI7IbExK7Wx8hhUIlEnha3a8KGFyGWKUUzNcxwtQgkMlMhh3YixMkqLFzGCl5LmfrNwswO5RruUJkwCWeh4onIHIvRZtSKlkBhj/ug4CTodQZM8JcLWcJOuuPHXjaGLrsQac7UXt/anIaOqWCKimvQFInl9Z7fO2UWEwrjJtWZm3MpJfGtetWUZyN96f2DBOCAzcqhljJaGSqVSCqIOohJlIoINFo4Vb5eB8Rg2ZhPGZht4SG3HuQqCao2ck9+cUrheHtnvRzhBBigb19oEagV1XDfd0nhE4H4r4yn1X+nGW6pGNYfw5FQ7OSxH3vfR/QLiYGiBIrcpRiJM11VOE2fgJLX5yTR3MjEhFuE2l+IGkplLE2acreaxBgDQma2D4YTV39sXyfTuoeTt73Ai3w6C+kkFWC7/vpThurOtQRiCllc1+W69b/axb4Q8nZLxsaAoXEFEunl0eW7ufAi3o6PKv7kgDalWFCuh9WmkjOdwVq1RqEBFnVEnh7ze7wsCghwjarciKVQzx1kP5IxC3otduwOWG+i5KUDp1vWp3sKpht9JqxWqTqh1vYbXUh/BFiNr3/03hdH8vGuxFoy97j58034YHwafBZ8F2EAVfBY+Dr4Oj4CRIOz93fu381vm9+0v3j+6f3b+W0HudxueTYO3p/v0f/RjmwA==</latexit><latexit sha1_base64="zoT4O9geNsLf661v0eYUZ/wcbzU=">AAAIG3icfVVbb+NEFHYWlizmtoVHXkaEqu2SVnGFhBSp0movEkhUNKu2uyLORmN77AyZi3dm3DYM8wv4DfwIXuGRN8QrD7zwWzjjuE3jtlhKfOac75z5zmXGScmoNoPBP517b719/53ug3fD997/4MOPHm58fKplpVJykkom1asEa8KoICeGGkZelYpgnjDyMpk/9faXZ0RpKsWxWZRkwnEhaE5TbEA13eh8vhmXdIrPim29gw5QzPHFFKPR1ILKvX60rft4J9yENRhcvULDBmYPUUyFl81Mp8oeOjeaHl76xLlUmLGphZVDQxQbcmF0ag/xxegbQU0TLC7ImzvDoWvxPAfa8htdGX0SNWWfxvAqj5in9oW7tHi/L8ATcw4mXXEgt+Vq0PE1UF9v7aDTVQW2dqYPe4O9Qf2gm0LUCL2geY6mG/f/jTOZVpwIkzKs9TgalGZisTI0ZcSFm2h3dxcd4XSOC6L9IowrTcqlYsyxKqg4GOztUzGxBZGcGLVw1zGWCAivsCF95EumdK77CHqdQGt5H2GuvboWzGyp0AuegMAKqehSx4pSkwqoygzilDSXwvSRrpKcFn2UV4yVsFcfpfpNJQ2BHTKsZ6pioDN0/mMfJQnESaScG5yA2a9WJCrF1kiPc4U5gWRmMjs4pvPvJ5ZntS5zYRgLcp5KaI7IbExK7Wx8hhUIlEnha3a8KGFyGWKUUzNcxwtQgkMlMhh3YixMkqLFzGCl5LmfrNwswO5RruUJkwCWeh4onIHIvRZtSKlkBhj/ug4CTodQZM8JcLWcJOuuPHXjaGLrsQac7UXt/anIaOqWCKimvQFInl9Z7fO2UWEwrjJtWZm3MpJfGtetWUZyN96f2DBOCAzcqhljJaGSqVSCqIOohJlIoINFo4Vb5eB8Rg2ZhPGZht4SG3HuQqCao2ck9+cUrheHtnvRzhBBigb19oEagV1XDfd0nhE4H4r4yn1X+nGW6pGNYfw5FQ7OSxH3vfR/QLiYGiBIrcpRiJM11VOE2fgJLX5yTR3MjEhFuE2l+IGkplLE2acreaxBgDQma2D4YTV39sXyfTuoeTt73Ai3w6C+kkFWC7/vpThurOtQRiCllc1+W69b/axb4Q8nZLxsaAoXEFEunl0eW7ufAi3o6PKv7kgDalWFCuh9WmkjOdwVq1RqEBFnVEnh7ze7wsCghwjarciKVQzx1kP5IxC3otduwOWG+i5KUDp1vWp3sKpht9JqxWqTqh1vYbXUh/BFiNr3/03hdH8vGuxFoy97j58034YHwafBZ8F2EAVfBY+Dr4Oj4CRIOz93fu381vm9+0v3j+6f3b+W0HudxueTYO3p/v0f/RjmwA==</latexit><latexit sha1_base64="zoT4O9geNsLf661v0eYUZ/wcbzU=">AAAIG3icfVVbb+NEFHYWlizmtoVHXkaEqu2SVnGFhBSp0movEkhUNKu2uyLORmN77AyZi3dm3DYM8wv4DfwIXuGRN8QrD7zwWzjjuE3jtlhKfOac75z5zmXGScmoNoPBP517b719/53ug3fD997/4MOPHm58fKplpVJykkom1asEa8KoICeGGkZelYpgnjDyMpk/9faXZ0RpKsWxWZRkwnEhaE5TbEA13eh8vhmXdIrPim29gw5QzPHFFKPR1ILKvX60rft4J9yENRhcvULDBmYPUUyFl81Mp8oeOjeaHl76xLlUmLGphZVDQxQbcmF0ag/xxegbQU0TLC7ImzvDoWvxPAfa8htdGX0SNWWfxvAqj5in9oW7tHi/L8ATcw4mXXEgt+Vq0PE1UF9v7aDTVQW2dqYPe4O9Qf2gm0LUCL2geY6mG/f/jTOZVpwIkzKs9TgalGZisTI0ZcSFm2h3dxcd4XSOC6L9IowrTcqlYsyxKqg4GOztUzGxBZGcGLVw1zGWCAivsCF95EumdK77CHqdQGt5H2GuvboWzGyp0AuegMAKqehSx4pSkwqoygzilDSXwvSRrpKcFn2UV4yVsFcfpfpNJQ2BHTKsZ6pioDN0/mMfJQnESaScG5yA2a9WJCrF1kiPc4U5gWRmMjs4pvPvJ5ZntS5zYRgLcp5KaI7IbExK7Wx8hhUIlEnha3a8KGFyGWKUUzNcxwtQgkMlMhh3YixMkqLFzGCl5LmfrNwswO5RruUJkwCWeh4onIHIvRZtSKlkBhj/ug4CTodQZM8JcLWcJOuuPHXjaGLrsQac7UXt/anIaOqWCKimvQFInl9Z7fO2UWEwrjJtWZm3MpJfGtetWUZyN96f2DBOCAzcqhljJaGSqVSCqIOohJlIoINFo4Vb5eB8Rg2ZhPGZht4SG3HuQqCao2ck9+cUrheHtnvRzhBBigb19oEagV1XDfd0nhE4H4r4yn1X+nGW6pGNYfw5FQ7OSxH3vfR/QLiYGiBIrcpRiJM11VOE2fgJLX5yTR3MjEhFuE2l+IGkplLE2acreaxBgDQma2D4YTV39sXyfTuoeTt73Ai3w6C+kkFWC7/vpThurOtQRiCllc1+W69b/axb4Q8nZLxsaAoXEFEunl0eW7ufAi3o6PKv7kgDalWFCuh9WmkjOdwVq1RqEBFnVEnh7ze7wsCghwjarciKVQzx1kP5IxC3otduwOWG+i5KUDp1vWp3sKpht9JqxWqTqh1vYbXUh/BFiNr3/03hdH8vGuxFoy97j58034YHwafBZ8F2EAVfBY+Dr4Oj4CRIOz93fu381vm9+0v3j+6f3b+W0HudxueTYO3p/v0f/RjmwA==</latexit>

VMax<latexit sha1_base64="ZWzecYwxwiAHPiIpMY0UkzMFt9w=">AAAIKnicfVVbb9s2FJbbde68W7M+7oWYUSTpnMDKBgwIYKDoBdiABUuKJC1muQYlUTJnXlSSSuJx/CP7DfsRe90e91YM2NN+yA5lJbKVZAJsHZ7zncPvXEjFBaPaDIfvOnfuvnfv/e79D3offvTxJ58+2PjsVMtSJeQkkUyq1zHWhFFBTgw1jLwuFME8ZuRVPH/m7a/OiNJUimOzKMiE41zQjCbYgGq60fnqUVTQKT7Lt/Q2GqGI44spRkdTCyr35vGWHuDt3iNYg8FVK7Rfw+wBiqjwspnpRNkD546mB1c+USYVZmxqYenQPooMuTA6sQf44ug7QU0dLcrJ21vjoZWAl/6nEMBVlGgrylGzN+RUZeCz2r9KK+KJfekuLd7vS/DEnINJlxyobroKdLwCGujNbXTaFGRze/qgP9wdVg+6LoS10A/q53C6ce+fKJVJyYkwCcNaj8NhYSYWK0MTRiAbtLOzgw5xMsc50X7Ri0pNiqVizLHKqRgNd/eomNicSE6MWrhVjCUCwitsyAD5Aiqd6QGC1sfQaT5AmGuvrgQzWyr0gscgsFwqutSxvNCkBKoyhTgFzaQwA6TLOKP5AGUlYwXsNUCJfltKQ2CHFOuZKhnoDJ3/PEBxDHFiKecGx2D2q4ZEqdga6XGmMCeQzEymo2M6/3FieVrpUtfrRYKcJxKaI1IbkUI7G51hBQJlUviaHS8KGGSGGOXU7K/jBSjBoRQpTD8xFuZK0XxmsFLy3M9ZZhZg9yjX8oRJAEs1DxSOROjeiDakUDIFjH+tgoDTARTZcwJcJcfxuitP3Dic2GrIAWf7YXt/KlKauCUCqmmvAeIXV1b7om1UGIxNpi0r81ZGskvjujVNSebGexPbi2ICA9c0Y6wkVDKRShA1CguYiRg6mNdauGRG5zNqyKQXnWnoLbEh564HVDP0nGT+nMJt49BWP9zeR5CiQf09oEZg16bhns5zAudDEV+5Hwo/zlI9thGMP6fCwXnJo4GX/g8It0MN9PfEeuUoxEnr6inCbPSU5r+4ug5mRqQi3CZS/EQSUyri7LNGHmsQII3JGhh+WM2dfbl83wyq384e18LNMKivZJDVwu97KY5r6zqUEUipsdnvq3Wrn1Ur/OGEjJcNTeACIspFs8tja/cSoAUdXf5VHalBrapQAb1PSm0kh7uiSaUCEXFGlRT+frMNBga9h6DdijSsIoi3HsofgagVvXIDLtfUt1GC0qnVqt3CqoLdSKsVq02qcryB1VLfgy9C2L7/rwune7vhcDc8+rr/5Gn9bbgffB58EWwFYfBN8CT4NjgMToKk82vn984fnT+7v3X/6r7r/r2E3unUPg+Dtaf77392cOzw</latexit><latexit sha1_base64="ZWzecYwxwiAHPiIpMY0UkzMFt9w=">AAAIKnicfVVbb9s2FJbbde68W7M+7oWYUSTpnMDKBgwIYKDoBdiABUuKJC1muQYlUTJnXlSSSuJx/CP7DfsRe90e91YM2NN+yA5lJbKVZAJsHZ7zncPvXEjFBaPaDIfvOnfuvnfv/e79D3offvTxJ58+2PjsVMtSJeQkkUyq1zHWhFFBTgw1jLwuFME8ZuRVPH/m7a/OiNJUimOzKMiE41zQjCbYgGq60fnqUVTQKT7Lt/Q2GqGI44spRkdTCyr35vGWHuDt3iNYg8FVK7Rfw+wBiqjwspnpRNkD546mB1c+USYVZmxqYenQPooMuTA6sQf44ug7QU0dLcrJ21vjoZWAl/6nEMBVlGgrylGzN+RUZeCz2r9KK+KJfekuLd7vS/DEnINJlxyobroKdLwCGujNbXTaFGRze/qgP9wdVg+6LoS10A/q53C6ce+fKJVJyYkwCcNaj8NhYSYWK0MTRiAbtLOzgw5xMsc50X7Ri0pNiqVizLHKqRgNd/eomNicSE6MWrhVjCUCwitsyAD5Aiqd6QGC1sfQaT5AmGuvrgQzWyr0gscgsFwqutSxvNCkBKoyhTgFzaQwA6TLOKP5AGUlYwXsNUCJfltKQ2CHFOuZKhnoDJ3/PEBxDHFiKecGx2D2q4ZEqdga6XGmMCeQzEymo2M6/3FieVrpUtfrRYKcJxKaI1IbkUI7G51hBQJlUviaHS8KGGSGGOXU7K/jBSjBoRQpTD8xFuZK0XxmsFLy3M9ZZhZg9yjX8oRJAEs1DxSOROjeiDakUDIFjH+tgoDTARTZcwJcJcfxuitP3Dic2GrIAWf7YXt/KlKauCUCqmmvAeIXV1b7om1UGIxNpi0r81ZGskvjujVNSebGexPbi2ICA9c0Y6wkVDKRShA1CguYiRg6mNdauGRG5zNqyKQXnWnoLbEh564HVDP0nGT+nMJt49BWP9zeR5CiQf09oEZg16bhns5zAudDEV+5Hwo/zlI9thGMP6fCwXnJo4GX/g8It0MN9PfEeuUoxEnr6inCbPSU5r+4ug5mRqQi3CZS/EQSUyri7LNGHmsQII3JGhh+WM2dfbl83wyq384e18LNMKivZJDVwu97KY5r6zqUEUipsdnvq3Wrn1Ur/OGEjJcNTeACIspFs8tja/cSoAUdXf5VHalBrapQAb1PSm0kh7uiSaUCEXFGlRT+frMNBga9h6DdijSsIoi3HsofgagVvXIDLtfUt1GC0qnVqt3CqoLdSKsVq02qcryB1VLfgy9C2L7/rwune7vhcDc8+rr/5Gn9bbgffB58EWwFYfBN8CT4NjgMToKk82vn984fnT+7v3X/6r7r/r2E3unUPg+Dtaf77392cOzw</latexit><latexit sha1_base64="ZWzecYwxwiAHPiIpMY0UkzMFt9w=">AAAIKnicfVVbb9s2FJbbde68W7M+7oWYUSTpnMDKBgwIYKDoBdiABUuKJC1muQYlUTJnXlSSSuJx/CP7DfsRe90e91YM2NN+yA5lJbKVZAJsHZ7zncPvXEjFBaPaDIfvOnfuvnfv/e79D3offvTxJ58+2PjsVMtSJeQkkUyq1zHWhFFBTgw1jLwuFME8ZuRVPH/m7a/OiNJUimOzKMiE41zQjCbYgGq60fnqUVTQKT7Lt/Q2GqGI44spRkdTCyr35vGWHuDt3iNYg8FVK7Rfw+wBiqjwspnpRNkD546mB1c+USYVZmxqYenQPooMuTA6sQf44ug7QU0dLcrJ21vjoZWAl/6nEMBVlGgrylGzN+RUZeCz2r9KK+KJfekuLd7vS/DEnINJlxyobroKdLwCGujNbXTaFGRze/qgP9wdVg+6LoS10A/q53C6ce+fKJVJyYkwCcNaj8NhYSYWK0MTRiAbtLOzgw5xMsc50X7Ri0pNiqVizLHKqRgNd/eomNicSE6MWrhVjCUCwitsyAD5Aiqd6QGC1sfQaT5AmGuvrgQzWyr0gscgsFwqutSxvNCkBKoyhTgFzaQwA6TLOKP5AGUlYwXsNUCJfltKQ2CHFOuZKhnoDJ3/PEBxDHFiKecGx2D2q4ZEqdga6XGmMCeQzEymo2M6/3FieVrpUtfrRYKcJxKaI1IbkUI7G51hBQJlUviaHS8KGGSGGOXU7K/jBSjBoRQpTD8xFuZK0XxmsFLy3M9ZZhZg9yjX8oRJAEs1DxSOROjeiDakUDIFjH+tgoDTARTZcwJcJcfxuitP3Dic2GrIAWf7YXt/KlKauCUCqmmvAeIXV1b7om1UGIxNpi0r81ZGskvjujVNSebGexPbi2ICA9c0Y6wkVDKRShA1CguYiRg6mNdauGRG5zNqyKQXnWnoLbEh564HVDP0nGT+nMJt49BWP9zeR5CiQf09oEZg16bhns5zAudDEV+5Hwo/zlI9thGMP6fCwXnJo4GX/g8It0MN9PfEeuUoxEnr6inCbPSU5r+4ug5mRqQi3CZS/EQSUyri7LNGHmsQII3JGhh+WM2dfbl83wyq384e18LNMKivZJDVwu97KY5r6zqUEUipsdnvq3Wrn1Ur/OGEjJcNTeACIspFs8tja/cSoAUdXf5VHalBrapQAb1PSm0kh7uiSaUCEXFGlRT+frMNBga9h6DdijSsIoi3HsofgagVvXIDLtfUt1GC0qnVqt3CqoLdSKsVq02qcryB1VLfgy9C2L7/rwune7vhcDc8+rr/5Gn9bbgffB58EWwFYfBN8CT4NjgMToKk82vn984fnT+7v3X/6r7r/r2E3unUPg+Dtaf77392cOzw</latexit><latexit sha1_base64="ZWzecYwxwiAHPiIpMY0UkzMFt9w=">AAAIKnicfVVbb9s2FJbbde68W7M+7oWYUSTpnMDKBgwIYKDoBdiABUuKJC1muQYlUTJnXlSSSuJx/CP7DfsRe90e91YM2NN+yA5lJbKVZAJsHZ7zncPvXEjFBaPaDIfvOnfuvnfv/e79D3offvTxJ58+2PjsVMtSJeQkkUyq1zHWhFFBTgw1jLwuFME8ZuRVPH/m7a/OiNJUimOzKMiE41zQjCbYgGq60fnqUVTQKT7Lt/Q2GqGI44spRkdTCyr35vGWHuDt3iNYg8FVK7Rfw+wBiqjwspnpRNkD546mB1c+USYVZmxqYenQPooMuTA6sQf44ug7QU0dLcrJ21vjoZWAl/6nEMBVlGgrylGzN+RUZeCz2r9KK+KJfekuLd7vS/DEnINJlxyobroKdLwCGujNbXTaFGRze/qgP9wdVg+6LoS10A/q53C6ce+fKJVJyYkwCcNaj8NhYSYWK0MTRiAbtLOzgw5xMsc50X7Ri0pNiqVizLHKqRgNd/eomNicSE6MWrhVjCUCwitsyAD5Aiqd6QGC1sfQaT5AmGuvrgQzWyr0gscgsFwqutSxvNCkBKoyhTgFzaQwA6TLOKP5AGUlYwXsNUCJfltKQ2CHFOuZKhnoDJ3/PEBxDHFiKecGx2D2q4ZEqdga6XGmMCeQzEymo2M6/3FieVrpUtfrRYKcJxKaI1IbkUI7G51hBQJlUviaHS8KGGSGGOXU7K/jBSjBoRQpTD8xFuZK0XxmsFLy3M9ZZhZg9yjX8oRJAEs1DxSOROjeiDakUDIFjH+tgoDTARTZcwJcJcfxuitP3Dic2GrIAWf7YXt/KlKauCUCqmmvAeIXV1b7om1UGIxNpi0r81ZGskvjujVNSebGexPbi2ICA9c0Y6wkVDKRShA1CguYiRg6mNdauGRG5zNqyKQXnWnoLbEh564HVDP0nGT+nMJt49BWP9zeR5CiQf09oEZg16bhns5zAudDEV+5Hwo/zlI9thGMP6fCwXnJo4GX/g8It0MN9PfEeuUoxEnr6inCbPSU5r+4ug5mRqQi3CZS/EQSUyri7LNGHmsQII3JGhh+WM2dfbl83wyq384e18LNMKivZJDVwu97KY5r6zqUEUipsdnvq3Wrn1Ur/OGEjJcNTeACIspFs8tja/cSoAUdXf5VHalBrapQAb1PSm0kh7uiSaUCEXFGlRT+frMNBga9h6DdijSsIoi3HsofgagVvXIDLtfUt1GC0qnVqt3CqoLdSKsVq02qcryB1VLfgy9C2L7/rwune7vhcDc8+rr/5Gn9bbgffB58EWwFYfBN8CT4NjgMToKk82vn984fnT+7v3X/6r7r/r2E3unUPg+Dtaf77392cOzw</latexit>

Qmax(s, a)<latexit sha1_base64="U5Fj5lkSey0v5AaOSDEh/8eBUJs=">AAAIK3icfVVbb9s2FJa7de60W9M97oWYESTpnMAKChQIEKDoBdiABUuKJC1muQYlUTJnXlSSSuJx/CX7DfsRe90e97Rh2Nv+xw5lpYqVZAJsHZ7zncPvXEglJaPajEZ/9u689/7dD/r3Pgw/+viTTz+7v/bgVMtKpeQklUyq1wnWhFFBTgw1jLwuFcE8YeRVMn/m7a/OiNJUimOzKMmE40LQnKbYgGq61nu0Hpd0is+KTb2F9lHM8cUUo6OpBZV783BTD/FWCEvQu3qxjvYamD1AMRVeNjOdKnvg3NH04NJnPc6lwoxNLSwd2kOxIRdGp/YAXxx9I6hZhkNxQd7eGg+tBGwCnEIEF64DKdoJc9RiIak6BZ/W3ru8Yp7al+7S4v2+Ak/MOZh0xYHrhqtBx1dAQ72xhU7bimxsTe8PRjuj+kHXhagRBkHzHE7X7v4TZzKtOBEmZVjrcTQqzcRiZWjKCGSDtre30SFO57gg2i/CuNKkXCrGHKuCiv3Rzi4VE1sQyYlRC3cVY4mA8AobMkS+gkrneoig9wm0mg8R5tqra8HMlgq94AkIrJCKLnWsKDWpgKrMIE5JcynMEOkqyWkxRHnFWAl7DVGq31bSENghw3qmKgY6Q+c/DlGSQJxEyrnBCZj9qiVRKbZCepwrzAkkM5PZ/jGdfz+xPKt1mQvDWJDzVEJzRGZjUmpn4zOsQKBMCl+z40UJk8wQo5yavVW8ACU4VCKD8SfGwmApWswMVkqe+0HLzQLsHuU6njAJYKnngcKZiNwb0YWUSmaA8a+rIOB0AEX2nABXy0my6spTN44mtp5ywNlB1N2fioymbomAatprgOTFO6t90TUqDMY2046VeSsj+aVx1ZplJHfj3YkN44TAwLXNGCsJlUylEkTtRyXMRAIdLBot3DL75zNqyCSMzzT0ltiIcxcC1Rw9J7k/p3DdOLQ5iLb2EKRo0GAXqBHYtW24p/OcwPlQxFfuu9KPs1QPbQzjz6lwcF6KeOil/wPC7dAA/T2xWjkKcbKmeoowGz+lxU+uqYOZEakIt6kUP5DUVIo4+6yVxxoESGOyAoYfVnNnXy7fN4Oat7PHjXAzDOorGWS18PteiuPGugplBFJqbfbbet3pZ90Kfzgh42VDU7iAiHLx7PLY2t0UaEFHl391RxpQpypUQO/TShvJ4a5oU6lBRJxRJYW/32yLgUEPEbRbkZZVDPFWQ/kjEHei127A5Zr6NkpQOnW1arewqmE30urE6pKqHW9gtdSH8EWIuvf/deF0dyca7URHjwZPnjbfhnvBF8GXwWYQBY+DJ8HXwWFwEqS9n3u/9n7r/d7/pf9H/6/+30vonV7j83mw8vT//Q/ka+0f</latexit><latexit sha1_base64="U5Fj5lkSey0v5AaOSDEh/8eBUJs=">AAAIK3icfVVbb9s2FJa7de60W9M97oWYESTpnMAKChQIEKDoBdiABUuKJC1muQYlUTJnXlSSSuJx/CX7DfsRe90e97Rh2Nv+xw5lpYqVZAJsHZ7zncPvXEglJaPajEZ/9u689/7dD/r3Pgw/+viTTz+7v/bgVMtKpeQklUyq1wnWhFFBTgw1jLwuFcE8YeRVMn/m7a/OiNJUimOzKMmE40LQnKbYgGq61nu0Hpd0is+KTb2F9lHM8cUUo6OpBZV783BTD/FWCEvQu3qxjvYamD1AMRVeNjOdKnvg3NH04NJnPc6lwoxNLSwd2kOxIRdGp/YAXxx9I6hZhkNxQd7eGg+tBGwCnEIEF64DKdoJc9RiIak6BZ/W3ru8Yp7al+7S4v2+Ak/MOZh0xYHrhqtBx1dAQ72xhU7bimxsTe8PRjuj+kHXhagRBkHzHE7X7v4TZzKtOBEmZVjrcTQqzcRiZWjKCGSDtre30SFO57gg2i/CuNKkXCrGHKuCiv3Rzi4VE1sQyYlRC3cVY4mA8AobMkS+gkrneoig9wm0mg8R5tqra8HMlgq94AkIrJCKLnWsKDWpgKrMIE5JcynMEOkqyWkxRHnFWAl7DVGq31bSENghw3qmKgY6Q+c/DlGSQJxEyrnBCZj9qiVRKbZCepwrzAkkM5PZ/jGdfz+xPKt1mQvDWJDzVEJzRGZjUmpn4zOsQKBMCl+z40UJk8wQo5yavVW8ACU4VCKD8SfGwmApWswMVkqe+0HLzQLsHuU6njAJYKnngcKZiNwb0YWUSmaA8a+rIOB0AEX2nABXy0my6spTN44mtp5ywNlB1N2fioymbomAatprgOTFO6t90TUqDMY2046VeSsj+aVx1ZplJHfj3YkN44TAwLXNGCsJlUylEkTtRyXMRAIdLBot3DL75zNqyCSMzzT0ltiIcxcC1Rw9J7k/p3DdOLQ5iLb2EKRo0GAXqBHYtW24p/OcwPlQxFfuu9KPs1QPbQzjz6lwcF6KeOil/wPC7dAA/T2xWjkKcbKmeoowGz+lxU+uqYOZEakIt6kUP5DUVIo4+6yVxxoESGOyAoYfVnNnXy7fN4Oat7PHjXAzDOorGWS18PteiuPGugplBFJqbfbbet3pZ90Kfzgh42VDU7iAiHLx7PLY2t0UaEFHl391RxpQpypUQO/TShvJ4a5oU6lBRJxRJYW/32yLgUEPEbRbkZZVDPFWQ/kjEHei127A5Zr6NkpQOnW1arewqmE30urE6pKqHW9gtdSH8EWIuvf/deF0dyca7URHjwZPnjbfhnvBF8GXwWYQBY+DJ8HXwWFwEqS9n3u/9n7r/d7/pf9H/6/+30vonV7j83mw8vT//Q/ka+0f</latexit><latexit sha1_base64="U5Fj5lkSey0v5AaOSDEh/8eBUJs=">AAAIK3icfVVbb9s2FJa7de60W9M97oWYESTpnMAKChQIEKDoBdiABUuKJC1muQYlUTJnXlSSSuJx/CX7DfsRe90e97Rh2Nv+xw5lpYqVZAJsHZ7zncPvXEglJaPajEZ/9u689/7dD/r3Pgw/+viTTz+7v/bgVMtKpeQklUyq1wnWhFFBTgw1jLwuFcE8YeRVMn/m7a/OiNJUimOzKMmE40LQnKbYgGq61nu0Hpd0is+KTb2F9lHM8cUUo6OpBZV783BTD/FWCEvQu3qxjvYamD1AMRVeNjOdKnvg3NH04NJnPc6lwoxNLSwd2kOxIRdGp/YAXxx9I6hZhkNxQd7eGg+tBGwCnEIEF64DKdoJc9RiIak6BZ/W3ru8Yp7al+7S4v2+Ak/MOZh0xYHrhqtBx1dAQ72xhU7bimxsTe8PRjuj+kHXhagRBkHzHE7X7v4TZzKtOBEmZVjrcTQqzcRiZWjKCGSDtre30SFO57gg2i/CuNKkXCrGHKuCiv3Rzi4VE1sQyYlRC3cVY4mA8AobMkS+gkrneoig9wm0mg8R5tqra8HMlgq94AkIrJCKLnWsKDWpgKrMIE5JcynMEOkqyWkxRHnFWAl7DVGq31bSENghw3qmKgY6Q+c/DlGSQJxEyrnBCZj9qiVRKbZCepwrzAkkM5PZ/jGdfz+xPKt1mQvDWJDzVEJzRGZjUmpn4zOsQKBMCl+z40UJk8wQo5yavVW8ACU4VCKD8SfGwmApWswMVkqe+0HLzQLsHuU6njAJYKnngcKZiNwb0YWUSmaA8a+rIOB0AEX2nABXy0my6spTN44mtp5ywNlB1N2fioymbomAatprgOTFO6t90TUqDMY2046VeSsj+aVx1ZplJHfj3YkN44TAwLXNGCsJlUylEkTtRyXMRAIdLBot3DL75zNqyCSMzzT0ltiIcxcC1Rw9J7k/p3DdOLQ5iLb2EKRo0GAXqBHYtW24p/OcwPlQxFfuu9KPs1QPbQzjz6lwcF6KeOil/wPC7dAA/T2xWjkKcbKmeoowGz+lxU+uqYOZEakIt6kUP5DUVIo4+6yVxxoESGOyAoYfVnNnXy7fN4Oat7PHjXAzDOorGWS18PteiuPGugplBFJqbfbbet3pZ90Kfzgh42VDU7iAiHLx7PLY2t0UaEFHl391RxpQpypUQO/TShvJ4a5oU6lBRJxRJYW/32yLgUEPEbRbkZZVDPFWQ/kjEHei127A5Zr6NkpQOnW1arewqmE30urE6pKqHW9gtdSH8EWIuvf/deF0dyca7URHjwZPnjbfhnvBF8GXwWYQBY+DJ8HXwWFwEqS9n3u/9n7r/d7/pf9H/6/+30vonV7j83mw8vT//Q/ka+0f</latexit><latexit sha1_base64="U5Fj5lkSey0v5AaOSDEh/8eBUJs=">AAAIK3icfVVbb9s2FJa7de60W9M97oWYESTpnMAKChQIEKDoBdiABUuKJC1muQYlUTJnXlSSSuJx/CX7DfsRe90e97Rh2Nv+xw5lpYqVZAJsHZ7zncPvXEglJaPajEZ/9u689/7dD/r3Pgw/+viTTz+7v/bgVMtKpeQklUyq1wnWhFFBTgw1jLwuFcE8YeRVMn/m7a/OiNJUimOzKMmE40LQnKbYgGq61nu0Hpd0is+KTb2F9lHM8cUUo6OpBZV783BTD/FWCEvQu3qxjvYamD1AMRVeNjOdKnvg3NH04NJnPc6lwoxNLSwd2kOxIRdGp/YAXxx9I6hZhkNxQd7eGg+tBGwCnEIEF64DKdoJc9RiIak6BZ/W3ru8Yp7al+7S4v2+Ak/MOZh0xYHrhqtBx1dAQ72xhU7bimxsTe8PRjuj+kHXhagRBkHzHE7X7v4TZzKtOBEmZVjrcTQqzcRiZWjKCGSDtre30SFO57gg2i/CuNKkXCrGHKuCiv3Rzi4VE1sQyYlRC3cVY4mA8AobMkS+gkrneoig9wm0mg8R5tqra8HMlgq94AkIrJCKLnWsKDWpgKrMIE5JcynMEOkqyWkxRHnFWAl7DVGq31bSENghw3qmKgY6Q+c/DlGSQJxEyrnBCZj9qiVRKbZCepwrzAkkM5PZ/jGdfz+xPKt1mQvDWJDzVEJzRGZjUmpn4zOsQKBMCl+z40UJk8wQo5yavVW8ACU4VCKD8SfGwmApWswMVkqe+0HLzQLsHuU6njAJYKnngcKZiNwb0YWUSmaA8a+rIOB0AEX2nABXy0my6spTN44mtp5ywNlB1N2fioymbomAatprgOTFO6t90TUqDMY2046VeSsj+aVx1ZplJHfj3YkN44TAwLXNGCsJlUylEkTtRyXMRAIdLBot3DL75zNqyCSMzzT0ltiIcxcC1Rw9J7k/p3DdOLQ5iLb2EKRo0GAXqBHYtW24p/OcwPlQxFfuu9KPs1QPbQzjz6lwcF6KeOil/wPC7dAA/T2xWjkKcbKmeoowGz+lxU+uqYOZEakIt6kUP5DUVIo4+6yVxxoESGOyAoYfVnNnXy7fN4Oat7PHjXAzDOorGWS18PteiuPGugplBFJqbfbbet3pZ90Kfzgh42VDU7iAiHLx7PLY2t0UaEFHl391RxpQpypUQO/TShvJ4a5oU6lBRJxRJYW/32yLgUEPEbRbkZZVDPFWQ/kjEHei127A5Zr6NkpQOnW1arewqmE30urE6pKqHW9gtdSH8EWIuvf/deF0dyca7URHjwZPnjbfhnvBF8GXwWYQBY+DJ8HXwWFwEqS9n3u/9n7r/d7/pf9H/6/+30vonV7j83mw8vT//Q/ka+0f</latexit>

References:

[1]

[1]: Brunskill & Li. Pac-inspired option discovery in lifelong reinforcement learning. 2014.

[3]

[4]

[2]

[2]

[2]: Ramachandran & Amir. Bayesian inverse reinforcement learning. 2007.[3]: Singh, Jakkola, Jordan. Learning without state-estimation in partially observable markovian decision processes 2003

[4]: Astrom. Optimal control of markov processes with incomplete state information 1965

[5]: Rossman & Ramamoorthy. What good are actions? 2012.

[6]

[5]

[6]

[6]: Strehl et al.. Reinforcement learning in finite MDPs: PAC analysis. 2009.

Department of Computer Science, Brown University{yuu_jinnai,david_abel,yue_guo,george_konidaris,michael_littman}@brown.edu