luca biferale dept. physics, infn & cast universityof rome ...cnls.lanl.gov/external/piml/luca...
TRANSCRIPT
FlownavigationbysmartparticlesviaReinforcementLearning
Luca BiferaleDept. Physics, INFN & CAST
University of Rome ‘Tor Vergata’[email protected] SANTA FE
CREDITS:SIMONACOLABRESE(TORVERGATAUNIV.ROME-IT);ANTONIOCELANI (ICTPTRIESTE-IT);KRISTIANGUSTAVSSON(GOTHEBORGUNIV.SWEDEN)
- PARTICLESINCOMPLEXFLOWSI:SMARTINERTIALPARTICLES
- PARTICLESINCOMPLEXFLOWSII:SMARTMICROSWIMMERS
- Flownavigationbysmartmicroswimmers viareinforcementlearning
SColabrese,KGustavsson,ACelani,LBiferalePhysicalReviewLetters118(15),158004,2017
-SmartInertialParticles
SColabrese,KGustavsson,ACelani,LBiferalearXiv preprintarXiv:1711.05853,2017
- Findingefficientswimmingstrategiesinathree-dimensionalchaoticflowbyreinforcementlearning
KGustavsson,LBiferale,ACelani,SColabreseTheEuropeanPhysicalJournalE40(12),110,2017
Drag: Stokes Timeβ<1 heavy particlesβ>1 light particles
PARTICLES INCOMPLEXFLOWS I:INERTIALPARTICLES
Preferential concentration!
Light(heavy)particles accumulateinside(outside)highly vortical regions
Maxey, J. Fluid Mech. 174, 441 (1987); Falkovich et al, Phys. Rev. Lett. 86, 2790 (2001)
heavy
light
tracerbubble heavy
Particle trapping in three-dimensional fully developed turbulenceL.B., G Boffetta, A Celani, A Lanotte, F ToschiPhysics of Fluids 17 (2), 021701
DESERT STORMS
RAININITIATION
PESTICIDE SPREADING DIESEL ENGINE INJECTIONS
TURBOMACHINES
Coherent structures and extreme events in rotating multiphase turbulent flows L.L.B., F Bonaccorso, IM Mazzitelli, MAT van Hinsberg, AS Lanotte, ...Physical Review X 6 (4), 041036 (2016)
Lagrangian properties of particles in turbulenceF Toschi, E BodenschatzAnnual Review of Fluid Mechanics 41, 375-404 (2009)
Na actions(densities)R = ⌦3
OBSERVATION:
DISCRETIZED VORTICYLEVELS
⇡n : si ! aj
⇡n
! ⇡n+1 ! · · ·⇡
opt
TRGETTARGET
Na actions(densities)R = ⌦3
OBSERVATION:
DISCRETIZED VORTICYLEVELS
⇡n : si ! aj
⇡n
! ⇡n+1 ! · · ·⇡
opt
TRGETTARGET
Qn(aj , si)
TRAINING:Q-LEARNINGALGORITHM
QUALITYMATRIXATSTEPnà
GREEDYPOLICYATSTEPn:
EXPECTEDDISCOUNTEDFUTURERETURNIFACTIONa_j istaken afterobservationofstates_i
s1
s2
s3
Qn(si, aj) = Rn + �Rn+1 + �2Rn+2 + �3Rn+3 + · · · =1X
t=n
�tRt
⇡n : a = arg max
a0Qn(a
0, s)
2.2 4.3 10.12.0 8.1 2.0
MYOPICàFAR-SIGTHEDà
� = 0� = 1
1.2 0.3 0.1
a1 a2 a3
s2 ! a3s3 ! a2
s1 ! a1⇡n
S
R0Q(a,s)
Q(s, a) Q(s, a) + ↵[R0+ �max
a0Q(s0, a0)�Q(s, a)]
OLDOBSERVATIONNEWOBSERVATION
⇡n ! ⇡n+1 ?
CHANGING RADIUSbn
TRAINING
NOEXPLORATION
TRAINING+EXPLORATION (ε-greedy)
OPTIMALACTIONS
EXAM
TIMEDEPENDENTFLOW
Clustering and turbophoresis in a shear flow without wallsF De Lillo, M Cencini, S Musacchio, G BoffettaPhysics of Fluids 28 (3), 035104 (2016)
ka = z "
OPTIMALSTRATEGY
Reddy, G., Celani, A., Sejnowski, T. J., & Vergassola, M. (2016). Learning to soar in turbulent environments. Proceedings of the National Academy of Sciences, 201606075.
- Flownavigationbysmartmicroswimmers viareinforcement learning
SColabrese,KGustavsson,ACelani,LBiferalePhysicalReviewLetters118(15),158004
-SmartInertialParticles
SColabrese,KGustavsson,ACelani,LBiferalearXiv preprintarXiv:1711.05853
- Findingefficientswimmingstrategiesinathree-dimensionalchaotic flowbyreinforcement learning
KGustavsson,LBiferale,ACelani,SColabreseTheEuropeanPhysicalJournalE40(12),110
CREDITS:SIMONACOLABRESE(TORVERGATAUNIV.ROME-IT);ANTONIOCELANI (ICTPTRIESTE-IT);KRISTIANGUSTAVSSON(GOTHEBORGUNIV.SWEDEN)