dynamic problems and learning systems to solve them
Post on 04-Feb-2022
1 Views
Preview:
TRANSCRIPT
Dynamic Problems and Learning Systems toSolve Them
Pablo Zegerspzegers@uandes.cl
College of Engineering, University of the Andes, Chile
2008
Zegers Dynamic Problems and Learning Systems to Solve Them
IntroductionDefinition
I Many dynamical problems can be represented by the followingsystem of linear differential equations
z = Az + Bx
y = Cz + Bx
I Many more can be expressed by the nonlinear system
z = f (z, x)
y = g (z, x)
Zegers Dynamic Problems and Learning Systems to Solve Them
IntroductionThe Problem
I Given the richness of the nonlinear differential equations family, wewould like to be able to train neural networks capable of displayingthe same amount of richness.
I The differential equation model is interesting, among other reasons,because it has the states z that summarize the past (triggerproblem).
Zegers Dynamic Problems and Learning Systems to Solve Them
IntroductionSubproblems
I Emphasis on the input
I Memorize xI Compress xI Reconstruct x
I Emphasis on the output
I Forecast next future values of x. In this case y should resemblefuture of the input.
I Classification x into one of many classes. in this case yconverges to one of many attractor basins.
I Inverse problems such as inverse kinematics.I Map x into y or as it is normally called: system identification.
I Emphasis on the objective function
I Produce y that minimizes a functional such as in controlproblems.
Zegers Dynamic Problems and Learning Systems to Solve Them
IntroductionSubproblems (cont.)
I Unlimited trajectories.
I Limited Trajectories.
Zegers Dynamic Problems and Learning Systems to Solve Them
IntroductionSolution Approaches
I First we will try evolutionary approaches that do not require anyprevious knowledge.
I Then we will use everything we know about static neural networks,i.e. neural networks that do not have feedback, because they areeasier to train.
I We will finally use fully connected recurrent neural networks whosebasic unit is a neuron capable of implementing a differentialequation.
1
τixi (t) = −xi (t) +
n∑j=1
wij fj (xj (t))
with i ∈ [1, . . . , n]
Zegers Dynamic Problems and Learning Systems to Solve Them
IntroductionSolution Approaches (cont.)
RNNs are interesting because they produce extremely compact solution.However, they are difficult to train.
Taken from Siegelmann, 1997.
Zegers Dynamic Problems and Learning Systems to Solve Them
Evolutionary ApproachesRandom Recurrent Neural Networks Produce Limit Cycles
I Randomly generated recurrent neural networks spontaneouslyproduce limit cycles.
I It may be possible that to procude interesting dynamicbehavior is not that difficult!
Dayhoff, Palmadesso, Richards, and Lin, “Patterns of Dynamic Activity and Timing in Neural NetworkProcessing”, Neural Networks and Pattern Recognition, edited by Omidvar and Dayhoff, Academic Press, 1998.
Zegers Dynamic Problems and Learning Systems to Solve Them
Evolutionary ApproachesNEAT
I Start small,
I Optimize in this manageable space,
I Use different exisiting solutions to generate new ones,
I Systematically elaborate on existing structures by adding newdimensions as necessary, and
I Make them compete.Stanley and Miikkulainen, “Competitive Coevolution through Evolutionary Complexification”, Journal of ArtificialIntelligence Research, vol. 21, pp. 63-100, 2004.
Zegers Dynamic Problems and Learning Systems to Solve Them
Evolutionary ApproachesNEAT (cont.)
Taken from Stanley and Miikkulainen, 2004.
Zegers Dynamic Problems and Learning Systems to Solve Them
Evolutionary ApproachesNEAT (cont.)
Taken from Stanley and Miikkulainen, 2004.
Zegers Dynamic Problems and Learning Systems to Solve Them
Evolutionary ApproachesNEAT (cont.)
I Innovation numbers tell wichgenes match up withouttopological analysis. Thismechanism allows to preserveexisting functionality.
I Only individuals within aniche (species) compete.This allows new architecturessurvive even if they had notexpressed their potential.
Taken from Stanley and Miikkulainen, 2004.
Zegers Dynamic Problems and Learning Systems to Solve Them
Evolutionary ApproachesNEAT (cont.)
The robot Duel:
I Robots start facing away.
I Each robots has sensors to detectfood and the opponent position.
I They also have sensors that tellenergy difference and where the wallsare.
I Sandwiches represent energy.
I If the robots collide, the one withmore energy wins.
I Complex interaction betweenforaging, pursuit, and evasionbehaviors.
Taken from Stanley and Miikkulainen, 2004.
Zegers Dynamic Problems and Learning Systems to Solve Them
Evolutionary ApproachesNEAT (cont.)
Taken from Stanley and Miikkulainen, 2004.
Zegers Dynamic Problems and Learning Systems to Solve Them
Evolutionary ApproachesNEAT (cont.)
Taken from Stanley and Miikkulainen, 2004.
Zegers Dynamic Problems and Learning Systems to Solve Them
Evolutionary ApproachesNEAT (cont.)
Best complexifying network:
I 11 hidden units, 202connections.
I Lateral and recurrentconnections.
I Very complicated to findthrough fixed-topologyevolution or simplification.
Taken from Stanley and Miikkulainen, 2004.
Zegers Dynamic Problems and Learning Systems to Solve Them
Static Neural Networks for Dynamical ProblemsDelays (Sejnowski and Rosenberg, 1987)
Zegers Dynamic Problems and Learning Systems to Solve Them
Static Neural Networks for Dynamical ProblemsTime Delay Neural Network (Lang and Hinton, 1988)
Zegers Dynamic Problems and Learning Systems to Solve Them
Static Neural Networks for Dynamical ProblemsTime Delay Neural Network (Lang and Hinton, 1988) (cont.)
Zegers Dynamic Problems and Learning Systems to Solve Them
Static Neural Networks for Dynamical ProblemsNonlinear Autoregressive with Exogenous Variables (Siegelmann et al, 1987)
I The underlying process is model according
yt = f (yt−1, yt−2, . . . , xt , xt−1, xt2 , . . .) + ε
where f is implemented by a static neural network.
I NARX networks are as capable as a Turing Universal Machine!(Siegelmann, 1997)
Zegers Dynamic Problems and Learning Systems to Solve Them
Static Neural Networks for Dynamical ProblemsNonlinear Autoregressive with Exogenous Variables (Siegelmann et al, 1997) (cont.)
Taken from Siegelmann et al, 1997.
Zegers Dynamic Problems and Learning Systems to Solve Them
Arbitrary Recurrent Neural NetworksDirect Design
The nonlinear system
1
τixi (t) = −xi (t) +
n∑j=1
wij fj (xj (t))
with i ∈ [1, . . . , n] can be approximated to
1
τixi (t) = −xi (t) +
n∑j=1
wijxj (t)
if the neurons operate in the linear region.
0
20
40
60
80
100 1
0. 5
0
0.5
11
0. 5
0
0.5
1
F irst DimensionS econds
Se
con
d D
ime
nsi
on
Taken from Zegers and Sundareshan, 2003.Zegers Dynamic Problems and Learning Systems to Solve Them
Partially Recurrent SystemsDynamic Neural Network
I A specific type of partially recurrent neural network (page 180 ofHertz et al).
I In these cases the system is mostly feedforward and the recurrentconnections are used only in some specific places.
I Zegers and Sundareshan (2001, 2003).
I Zegers (USP 6751601).
Desired TrajectorySimple Trajectory
Zegers Dynamic Problems and Learning Systems to Solve Them
Partially Recurrent SystemsDynamic Neural Network (cont.)
DesiredTrajectory
.( )h .( )
yN
(t)o (t)
gRNN NRNN
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
First Dimension
Sec
ond
Dim
ensi
on
Zegers Dynamic Problems and Learning Systems to Solve Them
Partially Recurrent SystemsDynamic Neural Network (cont.)
DesiredTrajectory
yN
(t)
.( )h .( )g
o (t)
e (t)
Input
RNN NRNN
Modulating
−1 0 1−1
−0.5
0
0.5
1−0.5000
−1 0 1−1
−0.5
0
0.5
1−0.3333
−1 0 1−1
−0.5
0
0.5
1−0.1667
−1 0 1−1
−0.5
0
0.5
1+0.0000
−1 0 1−1
−0.5
0
0.5
1+0.1667
−1 0 1−1
−0.5
0
0.5
1+0.3333
−1 0 1−1
−0.5
0
0.5
1+0.5000
−1 0 1−1
−0.5
0
0.5
1−0.4167
−1 0 1−1
−0.5
0
0.5
1−0.2500
−1 0 1−1
−0.5
0
0.5
1−0.0833
−1 0 1−1
−0.5
0
0.5
1+0.0833
−1 0 1−1
−0.5
0
0.5
1+0.2500
−1 0 1−1
−0.5
0
0.5
1+0.4167
−1 0 1−1
−0.5
0
0.5
1+0.5833
Zegers Dynamic Problems and Learning Systems to Solve Them
Partially Recurrent SystemsDynamic Neural Network (cont.)
Zegers Dynamic Problems and Learning Systems to Solve Them
Partially Recurrent SystemsDynamic Neural Network (cont.)
Zegers Dynamic Problems and Learning Systems to Solve Them
Partially Recurrent SystemsDynamic Neural Network (cont.)
Zegers Dynamic Problems and Learning Systems to Solve Them
Fully Connected Recurrent Neural NetworksVariational Approaches
I Barak Pearlmutter (1990, 1995).
Zegers Dynamic Problems and Learning Systems to Solve Them
Fully Connected Recurrent Neural NetworksReinforcement Learning
I Tedrake, 2005. Tomada por Andrew Hetherington.
Zegers Dynamic Problems and Learning Systems to Solve Them
Fully Connected Recurrent Neural NetworksReinforcement Learning
I Check Andrew Ng’s research (see Ng’s helicopter video, 2007).
Zegers Dynamic Problems and Learning Systems to Solve Them
Universal Approximation Capacity of RNNsFunahashi and Nakamura’s Theorem
I In 1993 Funahashi and Nakamura proved that fully connectedrecurrent neural networks of the type
1
τixi (t) = −xi (t) +
n∑j=1
wij fj (xj (t))
with i ∈ [1, . . . , n] are universal approximators.
Zegers Dynamic Problems and Learning Systems to Solve Them
Training OptimizationNice Tricks!
I Teacher forcing.
I Complexifying objective function.
Zegers Dynamic Problems and Learning Systems to Solve Them
Other Dynamic ProblemsIt Can Be Very Complicated!
I Other neuron models.
I Connection delays.
I Complex spaces.
I Noisy trajectories.
I Stochastic processes.
I High dimensionalities.
Zegers Dynamic Problems and Learning Systems to Solve Them
Analog ProblemsThere Are Many Other Dynamic Problems
I Compressor systems.
I Finding density functions.
I Training problems.
I Solving partial differential equations.
I Mapping surfaces and volumes.
Zegers Dynamic Problems and Learning Systems to Solve Them
top related