projectivities and continuous action spaces
TRANSCRIPT
Projectivities and continuous action spacesfeat. Lena
Reinaldo Uribe M
Sept 30, 2013
w − l space as projective transformation from policyvalue/cost space
1
D
−D
Policy V
alu
e
Episode Length
w − l space as projective transformation from policyvalue/cost space
1
D
−D
Policy V
alu
e
Episode LengthD
−D
w
l
w − l space as projective transformation from policyvalue/cost space
1
D
−D
Policy V
alu
e
Episode Length
w − l space as projective transformation from policyvalue/cost space
1
D
−D
Policy V
alu
e
Episode LengthD
−D
w
l
Extension to continuous spacesSample task: two states, continuous actions
s1a1 ∈ [0, 1]r1 = 1 + (a1 − 0.5)2
c1 = 1 + a1
s2
a2 ∈ [0, 1]r2 = 1 + a2c2 = 1 + (a2 − 0.5)2
Extension to continuous spacesSample task: two states, continuous actions
s1a1 ∈ [0, 1]r1 = 1 + (a1 − 0.5)2
c1 = 1 + a1
s2a2 ∈ [0, 1]r2 = 1 + a2c2 = 1 + (a2 − 0.5)2
Extension to continuous spacesSample task: two states, continuous actions
Policy Space (Actions)
0
a2
1
0 a1 1
Extension to continuous spacesSample task: two states, continuous actions
Policy Values and Costs
Policy v
alu
e
Policy cost
4
4
Extension to continuous spacesSample task: two states, continuous actions
Policy Manifold in w − l
l
w
D/2
D/2