quantum-enhanced deliberation of learning agents using...

78
Projective Simulation Ion Implementation Quantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi Workshop on Quantum Cybernetics and Control University of Nottingham, January 20 th , 2015 Nicolai Friis Institute for Theoretical Physics, University of Innsbruck work in collaboration with Vedran Dunjko and Hans J. Briegel Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Upload: others

Post on 04-Jun-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Quantum-enhanced deliberation of

learning agents using trapped ions

Quaint-EPSRC-FQXi Workshop on

Quantum Cybernetics and Control

University of Nottingham, January 20th, 2015

Nicolai Friis

Institute for Theoretical Physics, University of Innsbruck

work in collaboration with Vedran Dunjko and Hans J. Briegel

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 2: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Outline

Projective Simulation

• Projective simulation (PS) agents

• Standard PS vs. reflecting PS (RPS)

• The RPS deliberation process

Quantum-enhanced deliberation using trapped ions

• Flexible implementations — coherent controlization

• Trapped ion setups

• Coherent controlization using trapped ions

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 3: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Outline

Projective Simulation

• Projective simulation (PS) agents

• Standard PS vs. reflecting PS (RPS)

• The RPS deliberation process

Quantum-enhanced deliberation using trapped ions

• Flexible implementations — coherent controlization

• Trapped ion setups

• Coherent controlization using trapped ions

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 4: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Outline

Projective Simulation

• Projective simulation (PS) agents

• Standard PS vs. reflecting PS (RPS)

• The RPS deliberation process

Quantum-enhanced deliberation using trapped ions

• Flexible implementations — coherent controlization

• Trapped ion setups

• Coherent controlization using trapped ions

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 5: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Outline

Projective Simulation

• Projective simulation (PS) agents

• Standard PS vs. reflecting PS (RPS)

• The RPS deliberation process

Quantum-enhanced deliberation using trapped ions

• Flexible implementations — coherent controlization

• Trapped ion setups

• Coherent controlization using trapped ions

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 6: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Outline

Projective Simulation

• Projective simulation (PS) agents

• Standard PS vs. reflecting PS (RPS)

• The RPS deliberation process

Quantum-enhanced deliberation using trapped ions

• Flexible implementations — coherent controlization

• Trapped ion setups

• Coherent controlization using trapped ions

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 7: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Outline

Projective Simulation

• Projective simulation (PS) agents

• Standard PS vs. reflecting PS (RPS)

• The RPS deliberation process

Quantum-enhanced deliberation using trapped ions

• Flexible implementations — coherent controlization

• Trapped ion setups

• Coherent controlization using trapped ions

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 8: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Projective Simulation (PS) Agents1

Agent may be. . .

• embodied

• autonomous

• active/passive

Env

iron

men

tSensors

Actuators

EpisodicCompositional

Memory

Percepts

Actions

PS Agent

Decision-making

input (percept)

→ deliberation

→ output (action)

Episodic & compositional memory:experience stored as clips

PS agents: deliberation process basedon a random walk in the clip space

Represented by Markov chain withtransition matrix P = [pij ]

Different types of deliberation processes

e.g., standard PS & reflecting PS

1 Briegel, De las Cuevas, Sci. Rep. 2, 400 (2012) [arXiv:1104.3787].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 9: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Projective Simulation (PS) Agents1

Agent may be. . .

• embodied

• autonomous

• active/passive

Env

iron

men

tSensors

Actuators

EpisodicCompositional

Memory

Percepts

Actions

PS Agent

Decision-making

input (percept)

→ deliberation

→ output (action)

Episodic & compositional memory:experience stored as clips

PS agents: deliberation process basedon a random walk in the clip space

Represented by Markov chain withtransition matrix P = [pij ]

Different types of deliberation processes

e.g., standard PS & reflecting PS

1 Briegel, De las Cuevas, Sci. Rep. 2, 400 (2012) [arXiv:1104.3787].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 10: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Projective Simulation (PS) Agents1

Agent may be. . .

• embodied

• autonomous

• active/passive

Env

iron

men

tSensors

Actuators

EpisodicCompositional

Memory

Percepts

Actions

PS Agent

Decision-making

input (percept)

→ deliberation

→ output (action)

Episodic & compositional memory:experience stored as clips

PS agents: deliberation process basedon a random walk in the clip space

Represented by Markov chain withtransition matrix P = [pij ]

Different types of deliberation processes

e.g., standard PS & reflecting PS

1 Briegel, De las Cuevas, Sci. Rep. 2, 400 (2012) [arXiv:1104.3787].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 11: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Projective Simulation (PS) Agents1

Agent may be. . .

• embodied

• autonomous

• active/passive

Env

iron

men

tSensors

Actuators

EpisodicCompositional

Memory

Percepts

Actions

PS Agent

Decision-making

input (percept)

→ deliberation

→ output (action)

Episodic & compositional memory:experience stored as clips

PS agents: deliberation process basedon a random walk in the clip space

Represented by Markov chain withtransition matrix P = [pij ]

Different types of deliberation processes

e.g., standard PS & reflecting PS

1 Briegel, De las Cuevas, Sci. Rep. 2, 400 (2012) [arXiv:1104.3787].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 12: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Projective Simulation (PS) Agents1

Agent may be. . .

• embodied

• autonomous

• active/passive

Env

iron

men

tSensors

Actuators

EpisodicCompositional

Memory

Percepts

Actions

PS Agent

Decision-making

input (percept)

→ deliberation

→ output (action)

Episodic & compositional memory:experience stored as clips

PS agents: deliberation process basedon a random walk in the clip space

Represented by Markov chain withtransition matrix P = [pij ]

Different types of deliberation processes

e.g., standard PS & reflecting PS

1 Briegel, De las Cuevas, Sci. Rep. 2, 400 (2012) [arXiv:1104.3787].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 13: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Projective Simulation (PS) Agents1

Agent may be. . .

• embodied

• autonomous

• active/passive

Env

iron

men

tSensors

Actuators

EpisodicCompositional

Memory

Percepts

Actions

PS Agent

Decision-making

input (percept)

→ deliberation

→ output (action)

Episodic & compositional memory:experience stored as clips

PS agents: deliberation process basedon a random walk in the clip space

Represented by Markov chain withtransition matrix P = [pij ]

Different types of deliberation processes

e.g., standard PS & reflecting PS

1 Briegel, De las Cuevas, Sci. Rep. 2, 400 (2012) [arXiv:1104.3787].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 14: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Projective Simulation (PS) Agents1

Agent may be. . .

• embodied

• autonomous

• active/passive

Env

iron

men

tSensors

Actuators

EpisodicCompositional

Memory

Percepts

Actions

PS Agent

Decision-making

input (percept)

→ deliberation

→ output (action)

Episodic & compositional memory:experience stored as clips

PS agents: deliberation process basedon a random walk in the clip space

Represented by Markov chain withtransition matrix P = [pij ]

Different types of deliberation processes

e.g., standard PS & reflecting PS

1 Briegel, De las Cuevas, Sci. Rep. 2, 400 (2012) [arXiv:1104.3787].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 15: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Projective Simulation (PS) Agents1

Agent may be. . .

• embodied

• autonomous

• active/passive

Env

iron

men

tSensors

Actuators

EpisodicCompositional

Memory

Percepts

Actions

PS Agent

Decision-making

input (percept)

→ deliberation

→ output (action)

Episodic & compositional memory:experience stored as clips

PS agents: deliberation process basedon a random walk in the clip space

Represented by Markov chain withtransition matrix P = [pij ]

Different types of deliberation processes

e.g., standard PS & reflecting PS

1 Briegel, De las Cuevas, Sci. Rep. 2, 400 (2012) [arXiv:1104.3787].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 16: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Projective Simulation (PS) Agents1

Agent may be. . .

• embodied

• autonomous

• active/passive

Env

iron

men

tSensors

Actuators

EpisodicCompositional

Memory

Percepts

Actions

PS Agent

Decision-making

input (percept)

→ deliberation

→ output (action)

Episodic & compositional memory:experience stored as clips

PS agents: deliberation process basedon a random walk in the clip space

Represented by Markov chain withtransition matrix P = [pij ]

Different types of deliberation processes

e.g., standard PS & reflecting PS

1 Briegel, De las Cuevas, Sci. Rep. 2, 400 (2012) [arXiv:1104.3787].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 17: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Projective Simulation (PS) Agents1

Agent may be. . .

• embodied

• autonomous

• active/passive

Env

iron

men

tSensors

Actuators

EpisodicCompositional

Memory

Percepts

Actions

PS Agent

Decision-making

input (percept)

→ deliberation

→ output (action)

Episodic & compositional memory:experience stored as clips

PS agents: deliberation process basedon a random walk in the clip space

C1 C2

C3 C4

p21

p12

p13 p31

p34

p43

p42p24

p11 p22

p33 p44

p14

p41

p23

p32

C1

Represented by Markov chain withtransition matrix P = [pij ]

Different types of deliberation processes

e.g., standard PS & reflecting PS

1 Briegel, De las Cuevas, Sci. Rep. 2, 400 (2012) [arXiv:1104.3787].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 18: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Projective Simulation (PS) Agents1

Agent may be. . .

• embodied

• autonomous

• active/passive

Env

iron

men

tSensors

Actuators

EpisodicCompositional

Memory

Percepts

Actions

PS Agent

Decision-making

input (percept)

→ deliberation

→ output (action)

Episodic & compositional memory:experience stored as clips

PS agents: deliberation process basedon a random walk in the clip space

C1 C2

C3 C4

p21

p12

p13 p31

p34

p43

p42p24

p11 p22

p33 p44

p14

p41

p23

p32

C1

Represented by Markov chain withtransition matrix P = [pij ]

Different types of deliberation processes

e.g., standard PS & reflecting PS

1 Briegel, De las Cuevas, Sci. Rep. 2, 400 (2012) [arXiv:1104.3787].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 19: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Variants of PS: Standard & Reflecting PS (RPS)

Standard PS

One clip network ⇔ Markov chain

Percept associated to clip i registered:

random walk starting from clip i

until action is obtained → output

Quantization

clips: state vectors |0 〉, |1 〉, |2 〉, . . .

associate unitary Ui to each clip, s.t.,

Ui |0 〉 =∑j

√pji | j 〉

measure in clip-basis, result: |k 〉if k is action → output

if not, apply Uk to |0 〉, repeat

Reflecting PS2

Desire output according to distribution πspecific to each percept

Separate clip network for every percept

Focus on one percept with transition

matrix P , s.t., Pπ = π

Quantization

two registers I/II: {| i 〉}I/IIuse Ui to construct operators UP , VP :

UP | i 〉I |0 〉II = | i 〉I Ui |0 〉IIVP |0 〉I | i 〉II = Ui |0 〉I | i 〉II

Encode π: |π′ 〉=∑i

√πi | i 〉I Ui |0 〉II

Szegedy-walk operator3 W (P ):

W (P ) |π′ 〉 = |π′ 〉2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].3 M. Szegedy, in Foundations of Computer Science, Proc. 45th IEEE Symp. Found. Comp. Sc., 32–41 (2004).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 20: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Variants of PS: Standard & Reflecting PS (RPS)

Standard PS

One clip network ⇔ Markov chain

Percept associated to clip i registered:

random walk starting from clip i

until action is obtained → output

Quantization

clips: state vectors |0 〉, |1 〉, |2 〉, . . .

associate unitary Ui to each clip, s.t.,

Ui |0 〉 =∑j

√pji | j 〉

measure in clip-basis, result: |k 〉if k is action → output

if not, apply Uk to |0 〉, repeat

Reflecting PS2

Desire output according to distribution πspecific to each percept

Separate clip network for every percept

Focus on one percept with transition

matrix P , s.t., Pπ = π

Quantization

two registers I/II: {| i 〉}I/IIuse Ui to construct operators UP , VP :

UP | i 〉I |0 〉II = | i 〉I Ui |0 〉IIVP |0 〉I | i 〉II = Ui |0 〉I | i 〉II

Encode π: |π′ 〉=∑i

√πi | i 〉I Ui |0 〉II

Szegedy-walk operator3 W (P ):

W (P ) |π′ 〉 = |π′ 〉2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].3 M. Szegedy, in Foundations of Computer Science, Proc. 45th IEEE Symp. Found. Comp. Sc., 32–41 (2004).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 21: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Variants of PS: Standard & Reflecting PS (RPS)

Standard PS

One clip network ⇔ Markov chain

Percept associated to clip i registered:

random walk starting from clip i

until action is obtained → output

Quantization

clips: state vectors |0 〉, |1 〉, |2 〉, . . .

associate unitary Ui to each clip, s.t.,

Ui |0 〉 =∑j

√pji | j 〉

measure in clip-basis, result: |k 〉if k is action → output

if not, apply Uk to |0 〉, repeat

Reflecting PS2

Desire output according to distribution πspecific to each percept

Separate clip network for every percept

Focus on one percept with transition

matrix P , s.t., Pπ = π

Quantization

two registers I/II: {| i 〉}I/IIuse Ui to construct operators UP , VP :

UP | i 〉I |0 〉II = | i 〉I Ui |0 〉IIVP |0 〉I | i 〉II = Ui |0 〉I | i 〉II

Encode π: |π′ 〉=∑i

√πi | i 〉I Ui |0 〉II

Szegedy-walk operator3 W (P ):

W (P ) |π′ 〉 = |π′ 〉2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].3 M. Szegedy, in Foundations of Computer Science, Proc. 45th IEEE Symp. Found. Comp. Sc., 32–41 (2004).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 22: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Variants of PS: Standard & Reflecting PS (RPS)

Standard PS

One clip network ⇔ Markov chain

Percept associated to clip i registered:

random walk starting from clip i

until action is obtained → output

Quantization

clips: state vectors |0 〉, |1 〉, |2 〉, . . .

associate unitary Ui to each clip, s.t.,

Ui |0 〉 =∑j

√pji | j 〉

measure in clip-basis, result: |k 〉if k is action → output

if not, apply Uk to |0 〉, repeat

Reflecting PS2

Desire output according to distribution πspecific to each percept

Separate clip network for every percept

Focus on one percept with transition

matrix P , s.t., Pπ = π

Quantization

two registers I/II: {| i 〉}I/IIuse Ui to construct operators UP , VP :

UP | i 〉I |0 〉II = | i 〉I Ui |0 〉IIVP |0 〉I | i 〉II = Ui |0 〉I | i 〉II

Encode π: |π′ 〉=∑i

√πi | i 〉I Ui |0 〉II

Szegedy-walk operator3 W (P ):

W (P ) |π′ 〉 = |π′ 〉2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].3 M. Szegedy, in Foundations of Computer Science, Proc. 45th IEEE Symp. Found. Comp. Sc., 32–41 (2004).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 23: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Variants of PS: Standard & Reflecting PS (RPS)

Standard PS

One clip network ⇔ Markov chain

Percept associated to clip i registered:

random walk starting from clip i

until action is obtained → output

Quantization

clips: state vectors |0 〉, |1 〉, |2 〉, . . .

associate unitary Ui to each clip, s.t.,

Ui |0 〉 =∑j

√pji | j 〉

measure in clip-basis, result: |k 〉if k is action → output

if not, apply Uk to |0 〉, repeat

Reflecting PS2

Desire output according to distribution πspecific to each percept

Separate clip network for every percept

Focus on one percept with transition

matrix P , s.t., Pπ = π

Quantization

two registers I/II: {| i 〉}I/IIuse Ui to construct operators UP , VP :

UP | i 〉I |0 〉II = | i 〉I Ui |0 〉IIVP |0 〉I | i 〉II = Ui |0 〉I | i 〉II

Encode π: |π′ 〉=∑i

√πi | i 〉I Ui |0 〉II

Szegedy-walk operator3 W (P ):

W (P ) |π′ 〉 = |π′ 〉2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].3 M. Szegedy, in Foundations of Computer Science, Proc. 45th IEEE Symp. Found. Comp. Sc., 32–41 (2004).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 24: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Variants of PS: Standard & Reflecting PS (RPS)

Standard PS

One clip network ⇔ Markov chain

Percept associated to clip i registered:

random walk starting from clip i

until action is obtained → output

Quantization

clips: state vectors |0 〉, |1 〉, |2 〉, . . .

associate unitary Ui to each clip, s.t.,

Ui |0 〉 =∑j

√pji | j 〉

measure in clip-basis, result: |k 〉if k is action → output

if not, apply Uk to |0 〉, repeat

Reflecting PS2

Desire output according to distribution πspecific to each percept

Separate clip network for every percept

Focus on one percept with transition

matrix P , s.t., Pπ = π

Quantization

two registers I/II: {| i 〉}I/IIuse Ui to construct operators UP , VP :

UP | i 〉I |0 〉II = | i 〉I Ui |0 〉IIVP |0 〉I | i 〉II = Ui |0 〉I | i 〉II

Encode π: |π′ 〉=∑i

√πi | i 〉I Ui |0 〉II

Szegedy-walk operator3 W (P ):

W (P ) |π′ 〉 = |π′ 〉2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].3 M. Szegedy, in Foundations of Computer Science, Proc. 45th IEEE Symp. Found. Comp. Sc., 32–41 (2004).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 25: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Variants of PS: Standard & Reflecting PS (RPS)

Standard PS

One clip network ⇔ Markov chain

Percept associated to clip i registered:

random walk starting from clip i

until action is obtained → output

Quantization

clips: state vectors |0 〉, |1 〉, |2 〉, . . .

associate unitary Ui to each clip, s.t.,

Ui |0 〉 =∑j

√pji | j 〉

measure in clip-basis, result: |k 〉if k is action → output

if not, apply Uk to |0 〉, repeat

Reflecting PS2

Desire output according to distribution πspecific to each percept

Separate clip network for every percept

Focus on one percept with transition

matrix P , s.t., Pπ = π

Quantization

two registers I/II: {| i 〉}I/IIuse Ui to construct operators UP , VP :

UP | i 〉I |0 〉II = | i 〉I Ui |0 〉IIVP |0 〉I | i 〉II = Ui |0 〉I | i 〉II

Encode π: |π′ 〉=∑i

√πi | i 〉I Ui |0 〉II

Szegedy-walk operator3 W (P ):

W (P ) |π′ 〉 = |π′ 〉2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].3 M. Szegedy, in Foundations of Computer Science, Proc. 45th IEEE Symp. Found. Comp. Sc., 32–41 (2004).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 26: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Variants of PS: Standard & Reflecting PS (RPS)

Standard PS

One clip network ⇔ Markov chain

Percept associated to clip i registered:

random walk starting from clip i

until action is obtained → output

Quantization

clips: state vectors |0 〉, |1 〉, |2 〉, . . .

associate unitary Ui to each clip, s.t.,

Ui |0 〉 =∑j

√pji | j 〉

measure in clip-basis, result: |k 〉if k is action → output

if not, apply Uk to |0 〉, repeat

Reflecting PS2

Desire output according to distribution πspecific to each percept

Separate clip network for every percept

Focus on one percept with transition

matrix P , s.t., Pπ = π

Quantization

two registers I/II: {| i 〉}I/IIuse Ui to construct operators UP , VP :

UP | i 〉I |0 〉II = | i 〉I Ui |0 〉IIVP |0 〉I | i 〉II = Ui |0 〉I | i 〉II

Encode π: |π′ 〉=∑i

√πi | i 〉I Ui |0 〉II

Szegedy-walk operator3 W (P ):

W (P ) |π′ 〉 = |π′ 〉2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].3 M. Szegedy, in Foundations of Computer Science, Proc. 45th IEEE Symp. Found. Comp. Sc., 32–41 (2004).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 27: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Variants of PS: Standard & Reflecting PS (RPS)

Standard PS

One clip network ⇔ Markov chain

Percept associated to clip i registered:

random walk starting from clip i

until action is obtained → output

Quantization

clips: state vectors |0 〉, |1 〉, |2 〉, . . .

associate unitary Ui to each clip, s.t.,

Ui |0 〉 =∑j

√pji | j 〉

measure in clip-basis, result: |k 〉if k is action → output

if not, apply Uk to |0 〉, repeat

Reflecting PS2

Desire output according to distribution πspecific to each percept

Separate clip network for every percept

Focus on one percept with transition

matrix P , s.t., Pπ = π

Quantization

two registers I/II: {| i 〉}I/IIuse Ui to construct operators UP , VP :

UP | i 〉I |0 〉II = | i 〉I Ui |0 〉IIVP |0 〉I | i 〉II = Ui |0 〉I | i 〉II

Encode π: |π′ 〉=∑i

√πi | i 〉I Ui |0 〉II

Szegedy-walk operator3 W (P ):

W (P ) |π′ 〉 = |π′ 〉2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].3 M. Szegedy, in Foundations of Computer Science, Proc. 45th IEEE Symp. Found. Comp. Sc., 32–41 (2004).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 28: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Variants of PS: Standard & Reflecting PS (RPS)

Standard PS

One clip network ⇔ Markov chain

Percept associated to clip i registered:

random walk starting from clip i

until action is obtained → output

Quantization

clips: state vectors |0 〉, |1 〉, |2 〉, . . .

associate unitary Ui to each clip, s.t.,

Ui |0 〉 =∑j

√pji | j 〉

measure in clip-basis, result: |k 〉if k is action → output

if not, apply Uk to |0 〉, repeat

Reflecting PS2

Desire output according to distribution πspecific to each percept

Separate clip network for every percept

Focus on one percept with transition

matrix P , s.t., Pπ = π

Quantization

two registers I/II: {| i 〉}I/IIuse Ui to construct operators UP , VP :

UP | i 〉I |0 〉II = | i 〉I Ui |0 〉IIVP |0 〉I | i 〉II = Ui |0 〉I | i 〉II

Encode π: |π′ 〉=∑i

√πi | i 〉I Ui |0 〉II

Szegedy-walk operator3 W (P ):

W (P ) |π′ 〉 = |π′ 〉2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].3 M. Szegedy, in Foundations of Computer Science, Proc. 45th IEEE Symp. Found. Comp. Sc., 32–41 (2004).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 29: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Variants of PS: Standard & Reflecting PS (RPS)

Standard PS

One clip network ⇔ Markov chain

Percept associated to clip i registered:

random walk starting from clip i

until action is obtained → output

Quantization

clips: state vectors |0 〉, |1 〉, |2 〉, . . .

associate unitary Ui to each clip, s.t.,

Ui |0 〉 =∑j

√pji | j 〉

measure in clip-basis, result: |k 〉if k is action → output

if not, apply Uk to |0 〉, repeat

Reflecting PS2

Desire output according to distribution πspecific to each percept

Separate clip network for every percept

Focus on one percept with transition

matrix P , s.t., Pπ = π

Quantization

two registers I/II: {| i 〉}I/IIuse Ui to construct operators UP , VP :

UP | i 〉I |0 〉II = | i 〉I Ui |0 〉IIVP |0 〉I | i 〉II = Ui |0 〉I | i 〉II

Encode π: |π′ 〉=∑i

√πi | i 〉I Ui |0 〉II

Szegedy-walk operator3 W (P ):

W (P ) |π′ 〉 = |π′ 〉2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].3 M. Szegedy, in Foundations of Computer Science, Proc. 45th IEEE Symp. Found. Comp. Sc., 32–41 (2004).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 30: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Variants of PS: Standard & Reflecting PS (RPS)

Standard PS

One clip network ⇔ Markov chain

Percept associated to clip i registered:

random walk starting from clip i

until action is obtained → output

Quantization

clips: state vectors |0 〉, |1 〉, |2 〉, . . .

associate unitary Ui to each clip, s.t.,

Ui |0 〉 =∑j

√pji | j 〉

measure in clip-basis, result: |k 〉if k is action → output

if not, apply Uk to |0 〉, repeat

Reflecting PS2

Desire output according to distribution πspecific to each percept

Separate clip network for every percept

Focus on one percept with transition

matrix P , s.t., Pπ = π

Quantization

two registers I/II: {| i 〉}I/IIuse Ui to construct operators UP , VP :

UP | i 〉I |0 〉II = | i 〉I Ui |0 〉IIVP |0 〉I | i 〉II = Ui |0 〉I | i 〉II

Encode π: |π′ 〉=∑i

√πi | i 〉I Ui |0 〉II

Szegedy-walk operator3 W (P ):

W (P ) |π′ 〉 = |π′ 〉2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].3 M. Szegedy, in Foundations of Computer Science, Proc. 45th IEEE Symp. Found. Comp. Sc., 32–41 (2004).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 31: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Variants of PS: Standard & Reflecting PS (RPS)

Standard PS

One clip network ⇔ Markov chain

Percept associated to clip i registered:

random walk starting from clip i

until action is obtained → output

Quantization

clips: state vectors |0 〉, |1 〉, |2 〉, . . .

associate unitary Ui to each clip, s.t.,

Ui |0 〉 =∑j

√pji | j 〉

measure in clip-basis, result: |k 〉if k is action → output

if not, apply Uk to |0 〉, repeat

Reflecting PS2

Desire output according to distribution πspecific to each percept

Separate clip network for every percept

Focus on one percept with transition

matrix P , s.t., Pπ = π

Quantization

two registers I/II: {| i 〉}I/IIuse Ui to construct operators UP , VP :

UP | i 〉I |0 〉II = | i 〉I Ui |0 〉IIVP |0 〉I | i 〉II = Ui |0 〉I | i 〉II

Encode π: |π′ 〉=∑i

√πi | i 〉I Ui |0 〉II

Szegedy-walk operator3 W (P ):

W (P ) |π′ 〉 = |π′ 〉2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].3 M. Szegedy, in Foundations of Computer Science, Proc. 45th IEEE Symp. Found. Comp. Sc., 32–41 (2004).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 32: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Variants of PS: Standard & Reflecting PS (RPS)

Standard PS

One clip network ⇔ Markov chain

Percept associated to clip i registered:

random walk starting from clip i

until action is obtained → output

Quantization

clips: state vectors |0 〉, |1 〉, |2 〉, . . .

associate unitary Ui to each clip, s.t.,

Ui |0 〉 =∑j

√pji | j 〉

measure in clip-basis, result: |k 〉if k is action → output

if not, apply Uk to |0 〉, repeat

Reflecting PS2

Desire output according to distribution πspecific to each percept

Separate clip network for every percept

Focus on one percept with transition

matrix P , s.t., Pπ = π

Quantization

two registers I/II: {| i 〉}I/IIuse Ui to construct operators UP , VP :

UP | i 〉I |0 〉II = | i 〉I Ui |0 〉IIVP |0 〉I | i 〉II = Ui |0 〉I | i 〉II

Encode π: |π′ 〉=∑i

√πi | i 〉I Ui |0 〉II

Szegedy-walk operator3 W (P ):

W (P ) |π′ 〉 = |π′ 〉2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].3 M. Szegedy, in Foundations of Computer Science, Proc. 45th IEEE Symp. Found. Comp. Sc., 32–41 (2004).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 33: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

Variants of PS: Standard & Reflecting PS (RPS)

Standard PS

One clip network ⇔ Markov chain

Percept associated to clip i registered:

random walk starting from clip i

until action is obtained → output

Quantization

clips: state vectors |0 〉, |1 〉, |2 〉, . . .

associate unitary Ui to each clip, s.t.,

Ui |0 〉 =∑j

√pji | j 〉

measure in clip-basis, result: |k 〉if k is action → output

if not, apply Uk to |0 〉, repeat

Reflecting PS2

Desire output according to distribution πspecific to each percept

Separate clip network for every percept

Focus on one percept with transition

matrix P , s.t., Pπ = π

Quantization

two registers I/II: {| i 〉}I/IIuse Ui to construct operators UP , VP :

UP | i 〉I |0 〉II = | i 〉I Ui |0 〉IIVP |0 〉I | i 〉II = Ui |0 〉I | i 〉II

Encode π: |π′ 〉=∑i

√πi | i 〉I Ui |0 〉II

Szegedy-walk operator3 W (P ):

W (P ) |π′ 〉 = |π′ 〉2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].3 M. Szegedy, in Foundations of Computer Science, Proc. 45th IEEE Symp. Found. Comp. Sc., 32–41 (2004).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 34: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

Achieved by Grover-like procedure!2

Recall Grover’s algorithm4

Search a set S for marked itemsfrom a subset M⊂ S

(0) Prepare |ψ 〉: uniformsuperposition of basis statesrepresenting all items

(1) Reflection over marked items |φ 〉

(2) Reflection over |ψ 〉

Repeat steps (1) and (2)

Note: every item found equally likely2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].4 Grover, Proc. 28th ACM Symp. Theory Comput. (STOC), 212–219 (1996) [arXiv:quant-ph/9605043].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 35: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

Achieved by Grover-like procedure!2

Recall Grover’s algorithm4

Search a set S for marked itemsfrom a subset M⊂ S

(0) Prepare |ψ 〉: uniformsuperposition of basis statesrepresenting all items

(1) Reflection over marked items |φ 〉

(2) Reflection over |ψ 〉

Repeat steps (1) and (2)

Note: every item found equally likely2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].4 Grover, Proc. 28th ACM Symp. Theory Comput. (STOC), 212–219 (1996) [arXiv:quant-ph/9605043].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 36: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

Achieved by Grover-like procedure!2

Recall Grover’s algorithm4

Search a set S for marked itemsfrom a subset M⊂ S

(0) Prepare |ψ 〉: uniformsuperposition of basis statesrepresenting all items

(1) Reflection over marked items |φ 〉

(2) Reflection over |ψ 〉

Repeat steps (1) and (2)

Note: every item found equally likely2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].4 Grover, Proc. 28th ACM Symp. Theory Comput. (STOC), 212–219 (1996) [arXiv:quant-ph/9605043].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 37: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

Achieved by Grover-like procedure!2

Recall Grover’s algorithm4

Search a set S for marked itemsfrom a subset M⊂ S

(0) Prepare |ψ 〉: uniformsuperposition of basis statesrepresenting all items

(1) Reflection over marked items |φ 〉

(2) Reflection over |ψ 〉

Repeat steps (1) and (2)

|ϕ ⟩

|ψ ⟩

Note: every item found equally likely2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].4 Grover, Proc. 28th ACM Symp. Theory Comput. (STOC), 212–219 (1996) [arXiv:quant-ph/9605043].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 38: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

Achieved by Grover-like procedure!2

Recall Grover’s algorithm4

Search a set S for marked itemsfrom a subset M⊂ S

(0) Prepare |ψ 〉: uniformsuperposition of basis statesrepresenting all items

(1) Reflection over marked items |φ 〉

(2) Reflection over |ψ 〉

Repeat steps (1) and (2)

|ϕ ⟩

|ψ ⟩

Note: every item found equally likely2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].4 Grover, Proc. 28th ACM Symp. Theory Comput. (STOC), 212–219 (1996) [arXiv:quant-ph/9605043].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 39: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

Achieved by Grover-like procedure!2

Recall Grover’s algorithm4

Search a set S for marked itemsfrom a subset M⊂ S

(0) Prepare |ψ 〉: uniformsuperposition of basis statesrepresenting all items

(1) Reflection over marked items |φ 〉

(2) Reflection over |ψ 〉

Repeat steps (1) and (2)

|ϕ ⟩

|ψ ⟩

Note: every item found equally likely2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].4 Grover, Proc. 28th ACM Symp. Theory Comput. (STOC), 212–219 (1996) [arXiv:quant-ph/9605043].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 40: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

Achieved by Grover-like procedure!2

Recall Grover’s algorithm4

Search a set S for marked itemsfrom a subset M⊂ S

(0) Prepare |ψ 〉: uniformsuperposition of basis statesrepresenting all items

(1) Reflection over marked items |φ 〉

(2) Reflection over |ψ 〉

Repeat steps (1) and (2)

|ϕ ⟩

|ψ ⟩

Note: every item found equally likely2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].4 Grover, Proc. 28th ACM Symp. Theory Comput. (STOC), 212–219 (1996) [arXiv:quant-ph/9605043].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 41: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

Achieved by Grover-like procedure!2

Recall Grover’s algorithm4

Search a set S for marked itemsfrom a subset M⊂ S

(0) Prepare |ψ 〉: uniformsuperposition of basis statesrepresenting all items

(1) Reflection over marked items |φ 〉

(2) Reflection over |ψ 〉

Repeat steps (1) and (2)

|ϕ ⟩

|ψ ⟩

Note: every item found equally likely2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].4 Grover, Proc. 28th ACM Symp. Theory Comput. (STOC), 212–219 (1996) [arXiv:quant-ph/9605043].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 42: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

Achieved by Grover-like procedure!2

Recall Grover’s algorithm4

Search a set S for marked itemsfrom a subset M⊂ S

(0) Prepare |ψ 〉: uniformsuperposition of basis statesrepresenting all items

(1) Reflection over marked items |φ 〉

(2) Reflection over |ψ 〉

Repeat steps (1) and (2)

|ϕ ⟩

|ψ ⟩

Note: every item found equally likely2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].4 Grover, Proc. 28th ACM Symp. Theory Comput. (STOC), 212–219 (1996) [arXiv:quant-ph/9605043].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 43: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

RPS: Grover-like procedure

(0) Prepare |π′ 〉: reprentation ofstationary distribution

(1) Reflection over actions

(2) Approximate5 reflection over |π′ 〉

Repeat steps (1) and (2)

Approximate reflection:Ui → UP , VP→W (P )→ PD(W )→ ARO

Sample, and repeat if no action is found.

Quantum RPS quadratically faster than classical RPS! Speed-up!2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].5 Magniez, Nayak, Roland, Santha, SIAM J. Comput. 40, 142 (2011) [arXiv:quant-ph/0608026].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 44: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

RPS: Grover-like procedure

(0) Prepare |π′ 〉: reprentation ofstationary distribution

(1) Reflection over actions

(2) Approximate5 reflection over |π′ 〉

Repeat steps (1) and (2)

Approximate reflection:Ui → UP , VP→W (P )→ PD(W )→ ARO

Sample, and repeat if no action is found.

Quantum RPS quadratically faster than classical RPS! Speed-up!2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].5 Magniez, Nayak, Roland, Santha, SIAM J. Comput. 40, 142 (2011) [arXiv:quant-ph/0608026].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 45: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

RPS: Grover-like procedure

(0) Prepare |π′ 〉: reprentation ofstationary distribution

(1) Reflection over actions

(2) Approximate5 reflection over |π′ 〉

Repeat steps (1) and (2)

Approximate reflection:Ui → UP , VP→W (P )→ PD(W )→ ARO

Sample, and repeat if no action is found.

Quantum RPS quadratically faster than classical RPS! Speed-up!2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].5 Magniez, Nayak, Roland, Santha, SIAM J. Comput. 40, 142 (2011) [arXiv:quant-ph/0608026].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 46: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

RPS: Grover-like procedure

(0) Prepare |π′ 〉: reprentation ofstationary distribution

(1) Reflection over actions

(2) Approximate5 reflection over |π′ 〉

Repeat steps (1) and (2)

Approximate reflection:Ui → UP , VP→W (P )→ PD(W )→ ARO

Sample, and repeat if no action is found.

Quantum RPS quadratically faster than classical RPS! Speed-up!2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].5 Magniez, Nayak, Roland, Santha, SIAM J. Comput. 40, 142 (2011) [arXiv:quant-ph/0608026].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 47: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

RPS: Grover-like procedure

(0) Prepare |π′ 〉: reprentation ofstationary distribution

(1) Reflection over actions

(2) Approximate5 reflection over |π′ 〉

Repeat steps (1) and (2)

Approximate reflection:Ui → UP , VP→W (P )→ PD(W )→ ARO

Sample, and repeat if no action is found.

Quantum RPS quadratically faster than classical RPS! Speed-up!2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].5 Magniez, Nayak, Roland, Santha, SIAM J. Comput. 40, 142 (2011) [arXiv:quant-ph/0608026].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 48: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

RPS: Grover-like procedure

(0) Prepare |π′ 〉: reprentation ofstationary distribution

(1) Reflection over actions

(2) Approximate5 reflection over |π′ 〉

Repeat steps (1) and (2)

Approximate reflection:Ui → UP , VP→W (P )→ PD(W )→ ARO

Sample, and repeat if no action is found.

Quantum RPS quadratically faster than classical RPS! Speed-up!2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].5 Magniez, Nayak, Roland, Santha, SIAM J. Comput. 40, 142 (2011) [arXiv:quant-ph/0608026].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 49: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

RPS: Grover-like procedure

(0) Prepare |π′ 〉: reprentation ofstationary distribution

(1) Reflection over actions

(2) Approximate5 reflection over |π′ 〉

Repeat steps (1) and (2)

Approximate reflection:Ui → UP , VP→W (P )→ PD(W )→ ARO

I

Iref HA L

Iref HBL

I

I

UP†

D0,III

UP

I

I

VP†

D0,I

IVP

Sample, and repeat if no action is found.

Quantum RPS quadratically faster than classical RPS! Speed-up!2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].5 Magniez, Nayak, Roland, Santha, SIAM J. Comput. 40, 142 (2011) [arXiv:quant-ph/0608026].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 50: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

RPS: Grover-like procedure

(0) Prepare |π′ 〉: reprentation ofstationary distribution

(1) Reflection over actions

(2) Approximate5 reflection over |π′ 〉

Repeat steps (1) and (2)

Approximate reflection:Ui → UP , VP→W (P )→ PD(W )→ ARO

H

H

H

W W2n

H

H

H

I

II

Aux n

Aux 1

Aux 0

W2

Sample, and repeat if no action is found.

Quantum RPS quadratically faster than classical RPS! Speed-up!2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].5 Magniez, Nayak, Roland, Santha, SIAM J. Comput. 40, 142 (2011) [arXiv:quant-ph/0608026].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 51: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

RPS: Grover-like procedure

(0) Prepare |π′ 〉: reprentation ofstationary distribution

(1) Reflection over actions

(2) Approximate5 reflection over |π′ 〉

Repeat steps (1) and (2)

Approximate reflection:Ui → UP , VP→W (P )→ PD(W )→ ARO

H

H

H

W W2n

H

H

H

I

II

Aux n

Aux 1

Aux 0

W2

PDHWL

D0,Aux

PDHWL†

Aux

I

II

Sample, and repeat if no action is found.

Quantum RPS quadratically faster than classical RPS! Speed-up!2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].5 Magniez, Nayak, Roland, Santha, SIAM J. Comput. 40, 142 (2011) [arXiv:quant-ph/0608026].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 52: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

RPS: Grover-like procedure

(0) Prepare |π′ 〉: reprentation ofstationary distribution

(1) Reflection over actions

(2) Approximate5 reflection over |π′ 〉

Repeat steps (1) and (2)

Approximate reflection:Ui → UP , VP→W (P )→ PD(W )→ ARO

H

H

H

W W2n

H

H

H

I

II

Aux n

Aux 1

Aux 0

W2

PDHWL

D0,Aux

PDHWL†

Aux

I

II

Sample, and repeat if no action is found.

Quantum RPS quadratically faster than classical RPS! Speed-up!2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].5 Magniez, Nayak, Roland, Santha, SIAM J. Comput. 40, 142 (2011) [arXiv:quant-ph/0608026].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 53: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

RPS: Grover-like procedure

(0) Prepare |π′ 〉: reprentation ofstationary distribution

(1) Reflection over actions

(2) Approximate5 reflection over |π′ 〉

Repeat steps (1) and (2)

Approximate reflection:Ui → UP , VP→W (P )→ PD(W )→ ARO

H

H

H

W W2n

H

H

H

I

II

Aux n

Aux 1

Aux 0

W2

PDHWL

D0,Aux

PDHWL†

Aux

I

II

Sample, and repeat if no action is found.

Quantum RPS quadratically faster than classical RPS! Speed-up!2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].5 Magniez, Nayak, Roland, Santha, SIAM J. Comput. 40, 142 (2011) [arXiv:quant-ph/0608026].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 54: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Projective Simulation and its variantsQuantum-enhanced deliberation

The RPS Decision-Making Process

RPS agent: output actions according to tail of stationary distribution π

RPS: Grover-like procedure

(0) Prepare |π′ 〉: reprentation ofstationary distribution

(1) Reflection over actions

(2) Approximate5 reflection over |π′ 〉

Repeat steps (1) and (2)

Approximate reflection:Ui → UP , VP→W (P )→ PD(W )→ ARO

H

H

H

W W2n

H

H

H

I

II

Aux n

Aux 1

Aux 0

W2

PDHWL

D0,Aux

PDHWL†

Aux

I

II

Sample, and repeat if no action is found.

Quantum RPS quadratically faster than classical RPS! Speed-up!2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].5 Magniez, Nayak, Roland, Santha, SIAM J. Comput. 40, 142 (2011) [arXiv:quant-ph/0608026].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 55: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Flexible Implementation — Coherent Controlization

Key element: flexible (updateable) implementation without prior knowledge of π

First step: construction of unitaries Ui: first column: real, positive entries√pji

For two clips: U(θ) = exp(−i θ

2Y)=

(cos θ

2− sin θ

2

sin θ2

cos θ2

)Extend by nested scheme of adding control7 (see also [8, 9, 10]), e.g., for (up to) 8 clips:

Remaining algorithm: some fixed operations + adding control: non-trivial, but possible10

7 Dunjko, Friis, Briegel, New J. Phys. (accepted, 2015) [arXiv:1407.2830].8 Araujo, Feix, Costa, Brukner, New J. Phys. 16, 093026 (2014) [arXiv:1309.7976].9 Thompson, Gu, Modi, Vedral, arXiv:1310.2927 [quant-ph] (2013).10 Friis, Dunjko, Dur, Briegel, Phys. Rev. A 89, 030303(R) (2014) [arXiv:1401.8128].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 56: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Flexible Implementation — Coherent Controlization

Key element: flexible (updateable) implementation without prior knowledge of π

First step: construction of unitaries Ui: first column: real, positive entries√pji

For two clips: U(θ) = exp(−i θ

2Y)=

(cos θ

2− sin θ

2

sin θ2

cos θ2

)Extend by nested scheme of adding control7 (see also [8, 9, 10]), e.g., for (up to) 8 clips:

Remaining algorithm: some fixed operations + adding control: non-trivial, but possible10

7 Dunjko, Friis, Briegel, New J. Phys. (accepted, 2015) [arXiv:1407.2830].8 Araujo, Feix, Costa, Brukner, New J. Phys. 16, 093026 (2014) [arXiv:1309.7976].9 Thompson, Gu, Modi, Vedral, arXiv:1310.2927 [quant-ph] (2013).10 Friis, Dunjko, Dur, Briegel, Phys. Rev. A 89, 030303(R) (2014) [arXiv:1401.8128].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 57: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Flexible Implementation — Coherent Controlization

Key element: flexible (updateable) implementation without prior knowledge of π

First step: construction of unitaries Ui: first column: real, positive entries√pji

For two clips: U(θ) = exp(−i θ

2Y)=

(cos θ

2− sin θ

2

sin θ2

cos θ2

)Extend by nested scheme of adding control7 (see also [8, 9, 10]), e.g., for (up to) 8 clips:

Remaining algorithm: some fixed operations + adding control: non-trivial, but possible10

7 Dunjko, Friis, Briegel, New J. Phys. (accepted, 2015) [arXiv:1407.2830].8 Araujo, Feix, Costa, Brukner, New J. Phys. 16, 093026 (2014) [arXiv:1309.7976].9 Thompson, Gu, Modi, Vedral, arXiv:1310.2927 [quant-ph] (2013).10 Friis, Dunjko, Dur, Briegel, Phys. Rev. A 89, 030303(R) (2014) [arXiv:1401.8128].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 58: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Flexible Implementation — Coherent Controlization

Key element: flexible (updateable) implementation without prior knowledge of π

First step: construction of unitaries Ui: first column: real, positive entries√pji

For two clips: U(θ) = exp(−i θ

2Y)=

(cos θ

2− sin θ

2

sin θ2

cos θ2

)Extend by nested scheme of adding control7 (see also [8, 9, 10]), e.g., for (up to) 8 clips:

Remaining algorithm: some fixed operations + adding control: non-trivial, but possible10

7 Dunjko, Friis, Briegel, New J. Phys. (accepted, 2015) [arXiv:1407.2830].8 Araujo, Feix, Costa, Brukner, New J. Phys. 16, 093026 (2014) [arXiv:1309.7976].9 Thompson, Gu, Modi, Vedral, arXiv:1310.2927 [quant-ph] (2013).10 Friis, Dunjko, Dur, Briegel, Phys. Rev. A 89, 030303(R) (2014) [arXiv:1401.8128].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 59: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Flexible Implementation — Coherent Controlization

Key element: flexible (updateable) implementation without prior knowledge of π

First step: construction of unitaries Ui: first column: real, positive entries√pji

For two clips: U(θ) = exp(−i θ

2Y)=

(cos θ

2− sin θ

2

sin θ2

cos θ2

)Extend by nested scheme of adding control7 (see also [8, 9, 10]), e.g., for (up to) 8 clips:

I

II

III

II II

III III

UHΘ1L

U HΘ2,Θ4,Θ5L U HΘ3,Θ6,Θ7L

UHΘ2L

UHΘ4L UHΘ5L

UHΘ3L

UHΘ6L UHΘ7L

Remaining algorithm: some fixed operations + adding control: non-trivial, but possible10

7 Dunjko, Friis, Briegel, New J. Phys. (accepted, 2015) [arXiv:1407.2830].8 Araujo, Feix, Costa, Brukner, New J. Phys. 16, 093026 (2014) [arXiv:1309.7976].9 Thompson, Gu, Modi, Vedral, arXiv:1310.2927 [quant-ph] (2013).10 Friis, Dunjko, Dur, Briegel, Phys. Rev. A 89, 030303(R) (2014) [arXiv:1401.8128].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 60: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Flexible Implementation — Coherent Controlization

Key element: flexible (updateable) implementation without prior knowledge of π

First step: construction of unitaries Ui: first column: real, positive entries√pji

For two clips: U(θ) = exp(−i θ

2Y)=

(cos θ

2− sin θ

2

sin θ2

cos θ2

)Extend by nested scheme of adding control7 (see also [8, 9, 10]), e.g., for (up to) 8 clips:

I

II

III

II II

III III

UHΘ1L

U HΘ2,Θ4,Θ5L U HΘ3,Θ6,Θ7L

UHΘ2L

UHΘ4L UHΘ5L

UHΘ3L

UHΘ6L UHΘ7L

Remaining algorithm: some fixed operations + adding control: non-trivial, but possible10

7 Dunjko, Friis, Briegel, New J. Phys. (accepted, 2015) [arXiv:1407.2830].8 Araujo, Feix, Costa, Brukner, New J. Phys. 16, 093026 (2014) [arXiv:1309.7976].9 Thompson, Gu, Modi, Vedral, arXiv:1310.2927 [quant-ph] (2013).10 Friis, Dunjko, Dur, Briegel, Phys. Rev. A 89, 030303(R) (2014) [arXiv:1401.8128].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 61: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Ion Trap Setups

(a) (b) (c)

String of, e.g., 40Ca+ ions in quadrupole trap — Paul trap

• Electric fields applied by blades (rapidly oscillating) & tips (static)

⇒ nearly harmonic 3−d trapping potential

• Ions individually addressable via position/frequency (dep. on setup)

• Coulomb coupling ⇒ bus (e.g., via vibrational modes)

(a) Figure from: Blatt & Wineland, Entangled states of trapped atomic ions, Nature 453, 1008–1015 (2008).

(b) Paul trap currently used by Blatt group (Innsbruck), image courtesy of B. Lanyon.

(c) Paul trap used in Wunderlich group (Siegen).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 62: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Ion Trap Setups

(a) (b) (c)

String of, e.g., 40Ca+ ions in quadrupole trap — Paul trap

• Electric fields applied by blades (rapidly oscillating) & tips (static)

⇒ nearly harmonic 3−d trapping potential

• Ions individually addressable via position/frequency (dep. on setup)

• Coulomb coupling ⇒ bus (e.g., via vibrational modes)

(a) Figure from: Blatt & Wineland, Entangled states of trapped atomic ions, Nature 453, 1008–1015 (2008).

(b) Paul trap currently used by Blatt group (Innsbruck), image courtesy of B. Lanyon.

(c) Paul trap used in Wunderlich group (Siegen).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 63: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Ion Trap Setups

(a) (b) (c)

String of, e.g., 40Ca+ ions in quadrupole trap — Paul trap

• Electric fields applied by blades (rapidly oscillating) & tips (static)

⇒ nearly harmonic 3−d trapping potential

• Ions individually addressable via position/frequency (dep. on setup)

• Coulomb coupling ⇒ bus (e.g., via vibrational modes)

(a) Figure from: Blatt & Wineland, Entangled states of trapped atomic ions, Nature 453, 1008–1015 (2008).

(b) Paul trap currently used by Blatt group (Innsbruck), image courtesy of B. Lanyon.

(c) Paul trap used in Wunderlich group (Siegen).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 64: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Ion Trap Setups

(a) (b) (c)

String of, e.g., 40Ca+ ions in quadrupole trap — Paul trap

• Electric fields applied by blades (rapidly oscillating) & tips (static)

⇒ nearly harmonic 3−d trapping potential

• Ions individually addressable via position/frequency (dep. on setup)

• Coulomb coupling ⇒ bus (e.g., via vibrational modes)

(a) Figure from: Blatt & Wineland, Entangled states of trapped atomic ions, Nature 453, 1008–1015 (2008).

(b) Paul trap currently used by Blatt group (Innsbruck), image courtesy of B. Lanyon.

(c) Paul trap used in Wunderlich group (Siegen).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 65: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Ion Trap Setups

(a) (b) (c)

String of, e.g., 40Ca+ ions in quadrupole trap — Paul trap

• Electric fields applied by blades (rapidly oscillating) & tips (static)

⇒ nearly harmonic 3−d trapping potential

• Ions individually addressable via position/frequency (dep. on setup)

• Coulomb coupling ⇒ bus (e.g., via vibrational modes)

(a) Figure from: Blatt & Wineland, Entangled states of trapped atomic ions, Nature 453, 1008–1015 (2008).

(b) Paul trap currently used by Blatt group (Innsbruck), image courtesy of B. Lanyon.

(c) Paul trap used in Wunderlich group (Siegen).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 66: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Scheme to Add Control to Unknown Unitaries7

Initial state:(α |g 〉1 + β |e 〉1

)|ψ 〉2 vibrational mode in |0 〉

(i) Cirac-Zoller method11: laser pulse on ion 1: |g 〉1 |0 〉 → | e 〉1 |1 〉

Ion 1

Èg\1È0\

Èe\1È0\

Èg\1È1\

Èe\1È1\Èg'\È0\Èe'\È0\

U

H1

H2

Sg

Se

Ion 2

Èg\2È0\

Èe\2È0\

Èg\2È1\

Èe\2È1\

10 Friis, Dunjko, Dur, Briegel, Phys. Rev. A 89, 030303(R) (2014) [arXiv:1401.8128].11 Cirac, Zoller, Phys. Rev. Lett. 74, 4091 (1995).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 67: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Scheme to Add Control to Unknown Unitaries7

(α |g 〉1 + β |e 〉1

)|ψ 〉2 |0 〉 → |e 〉1 |ψ 〉2

(α |1 〉+ β |0 〉

)(i) Cirac-Zoller method11: laser pulse on ion 1: |g 〉1 |0 〉 → | e 〉1 |1 〉

Ion 1

Èg\1È0\

Èe\1È0\

Èg\1È1\

Èe\1È1\

Ion 2

Èg\2È0\

Èe\2È0\

Èg\2È1\

Èe\2È1\

10 Friis, Dunjko, Dur, Briegel, Phys. Rev. A 89, 030303(R) (2014) [arXiv:1401.8128].11 Cirac, Zoller, Phys. Rev. Lett. 74, 4091 (1995).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 68: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Scheme to Add Control to Unknown Unitaries7

|e 〉1 |ψ 〉2(α |1 〉+ β |0 〉

)→ |e 〉1

(α |ψ′ 〉2 + β |ψ 〉2

)|0 〉

(ii) Hiding pulses: laser pulses on ion 2: |g 〉2 |1 〉 → |g′ 〉2 |0 〉

Ion 1

Èg\1È0\

Èe\1È0\

Èg\1È1\

Èe\1È1\

Ion 2

Èg\2È0\

Èe\2È0\

Èg\2È1\

Èe\2È1\ Èg'\2È0\Èe'\2È0\

10 Friis, Dunjko, Dur, Briegel, Phys. Rev. A 89, 030303(R) (2014) [arXiv:1401.8128].11 Cirac, Zoller, Phys. Rev. Lett. 74, 4091 (1995).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 69: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Scheme to Add Control to Unknown Unitaries7

|e 〉1(α |ψ′ 〉2 + β |ψ 〉2

)|0 〉 → |e 〉1

(α|ψ′ 〉2 +β U |ψ 〉2

)|0 〉

(iii) Unknown unitary U : laser pulse on ion 2: |ψ 〉2 → U |ψ 〉2

Ion 1

Èg\1È0\

Èe\1È0\

Èg\1È1\

Èe\1È1\

Ion 2

Èg\2È0\

Èe\2È0\

Èg\2È1\

Èe\2È1\ Èg'\2È0\Èe'\2È0\

U

10 Friis, Dunjko, Dur, Briegel, Phys. Rev. A 89, 030303(R) (2014) [arXiv:1401.8128].11 Cirac, Zoller, Phys. Rev. Lett. 74, 4091 (1995).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 70: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Scheme to Add Control to Unknown Unitaries7

|e 〉1(α|ψ′ 〉2 +β U |ψ 〉2

)|0 〉 → |e 〉1

(α|ψ 〉2 |1 〉+β U |ψ 〉2 |0 〉

)(iv) Undo hiding pulses: laser pulses on ion 2: |g′ 〉2 |0 〉 → |g 〉2 |1 〉

Ion 1

Èg\1È0\

Èe\1È0\

Èg\1È1\

Èe\1È1\

Ion 2

Èg\2È0\

Èe\2È0\

Èg\2È1\

Èe\2È1\ Èg'\2È0\Èe'\2È0\

10 Friis, Dunjko, Dur, Briegel, Phys. Rev. A 89, 030303(R) (2014) [arXiv:1401.8128].11 Cirac, Zoller, Phys. Rev. Lett. 74, 4091 (1995).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 71: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Scheme to Add Control to Unknown Unitaries7

|e 〉1(α |ψ 〉2 |1 〉+β U |ψ 〉2 |0 〉

)→

(α |g 〉1 |ψ 〉2+β |e 〉1 U |ψ 〉2

)|0 〉

(v) Switch back control: Cirac-Zoller pulse11 on ion 1: | e 〉1 |1 〉 → |g 〉1 |0 〉

Ion 1

Èg\1È0\

Èe\1È0\

Èg\1È1\

Èe\1È1\

Ion 2

Èg\2È0\

Èe\2È0\

Èg\2È1\

Èe\2È1\

10 Friis, Dunjko, Dur, Briegel, Phys. Rev. A 89, 030303(R) (2014) [arXiv:1401.8128].11 Cirac, Zoller, Phys. Rev. Lett. 74, 4091 (1995).

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 72: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Summary & Conclusion

• (R)PS: decision-making processes for autonmous learning agents

• Speed-up: Quantum RPS decides quadratically faster than classical RPS2

• Flexible (learning! growth!) implementation using coherent controlization7

• All elements of algorithm implementable in ion traps7

• Feasible implementation: detailed simulations including errors for simplescenario7

Π

8

Π

4

3 Π

8

Π

2

Σ0.0

0.1

0.2

0.3

0.4

DH�,N�

L

2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].7 Dunjko, Friis, Briegel, New J. Phys. (accepted, 2015) [arXiv:1407.2830].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 73: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Summary & Conclusion

• (R)PS: decision-making processes for autonmous learning agents

• Speed-up: Quantum RPS decides quadratically faster than classical RPS2

• Flexible (learning! growth!) implementation using coherent controlization7

• All elements of algorithm implementable in ion traps7

• Feasible implementation: detailed simulations including errors for simplescenario7

Π

8

Π

4

3 Π

8

Π

2

Σ0.0

0.1

0.2

0.3

0.4

DH�,N�

L

2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].7 Dunjko, Friis, Briegel, New J. Phys. (accepted, 2015) [arXiv:1407.2830].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 74: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Summary & Conclusion

• (R)PS: decision-making processes for autonmous learning agents

• Speed-up: Quantum RPS decides quadratically faster than classical RPS2

• Flexible (learning! growth!) implementation using coherent controlization7

• All elements of algorithm implementable in ion traps7

• Feasible implementation: detailed simulations including errors for simplescenario7

Π

8

Π

4

3 Π

8

Π

2

Σ0.0

0.1

0.2

0.3

0.4

DH�,N�

L

2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].7 Dunjko, Friis, Briegel, New J. Phys. (accepted, 2015) [arXiv:1407.2830].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 75: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Summary & Conclusion

• (R)PS: decision-making processes for autonmous learning agents

• Speed-up: Quantum RPS decides quadratically faster than classical RPS2

• Flexible (learning! growth!) implementation using coherent controlization7

• All elements of algorithm implementable in ion traps7

• Feasible implementation: detailed simulations including errors for simplescenario7

Π

8

Π

4

3 Π

8

Π

2

Σ0.0

0.1

0.2

0.3

0.4

DH�,N�

L

2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].7 Dunjko, Friis, Briegel, New J. Phys. (accepted, 2015) [arXiv:1407.2830].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 76: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Summary & Conclusion

• (R)PS: decision-making processes for autonmous learning agents

• Speed-up: Quantum RPS decides quadratically faster than classical RPS2

• Flexible (learning! growth!) implementation using coherent controlization7

• All elements of algorithm implementable in ion traps7

• Feasible implementation: detailed simulations including errors for simplescenario7

Π

8

Π

4

3 Π

8

Π

2

Σ0.0

0.1

0.2

0.3

0.4

DH�,N�

L

2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].7 Dunjko, Friis, Briegel, New J. Phys. (accepted, 2015) [arXiv:1407.2830].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 77: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Summary & Conclusion

• (R)PS: decision-making processes for autonmous learning agents

• Speed-up: Quantum RPS decides quadratically faster than classical RPS2

• Flexible (learning! growth!) implementation using coherent controlization7

• All elements of algorithm implementable in ion traps7

• Feasible implementation: detailed simulations including errors for simplescenario7

Π

8

Π

4

3 Π

8

Π

2

Σ0.0

0.1

0.2

0.3

0.4

DH�,N�

L

2 Paparo, Dunjko, Makmal, Martın-Delgado, Briegel, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].7 Dunjko, Friis, Briegel, New J. Phys. (accepted, 2015) [arXiv:1407.2830].

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents

Page 78: Quantum-enhanced deliberation of learning agents using ...qcc2015.weebly.com/uploads/6/6/5/5/6655648/friis.pdfQuantum-enhanced deliberation of learning agents using trapped ions Quaint-EPSRC-FQXi

Projective SimulationIon Implementation

Coherent Controlization using Trapped Ions

Thank you for your attention.

Nicolai Friis (ITP Innsbruck) Quantum-enhanced deliberation of learning agents