physics and machine learning
DESCRIPTION
Physics and Machine Learning. “All the tricks that physicists’ use eventually end up in machine learning”. Energy – Physics definitions. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/1.jpg)
Physics and Machine Learning
“All the tricks that physicists’ use eventually end up in machine
learning”
![Page 2: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/2.jpg)
Energy – Physics definitionsEnergy - A measure of being able to do work. There are many forms of energy, such as heat, mechanical, electrical, radiant, chemical, and nuclear energies. Energy is measured in such units as the joule (J), erg, kilowatt-hour (kW-hr), kilocalorie (kcal), foot-pound (ft-lb.), electron-volt (ev), and British thermal unit (BTU). –NASA.gov
"It is important to realize that in physics today, we have no knowledge of what energy is. We do not have a picture that energy comes in little blobs of a definite amount." -Richard Feynman "Lectures on Physics"
![Page 3: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/3.jpg)
Energy - Physics II
Images: http://meso.phys.northwestern.edu/research/magneticarrays.html
• Magnetic Arrays and Spin
![Page 4: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/4.jpg)
Energy - Machine Learning
• H=-½iJwiJSiSJ Si SJWiJ
Vs.
Vs.
Vs. Vs.
![Page 5: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/5.jpg)
Energy - Machine Learning II
• Energy is the difference in weight between all nodes that agree and all nodes that disagree.
• The more weights, the greater energy. • The “closer” the call, the lower |H|.
![Page 6: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/6.jpg)
Energy Minima
• Retrieval States – attractors
• Mixture States – linear combinations of odd numbered attractors
• Spin Glass States – uncorrelated to attractors.
![Page 7: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/7.jpg)
Ferromagnetics
![Page 8: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/8.jpg)
Energy MetaphorImagine the atomic magnets as movable objects with the freedom to flip, but you control their position. Each iteration of learning is like forcing all magnets to be closer together, as such the network energy is potential energy and the flipping of spins is the expression of kinetic energy.
![Page 9: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/9.jpg)
Temperature – Physics I
• Extending the ferromagnetic example• As temperature increases, the impact of
other atomic magnets’ spins is decreased.• At absolute zero, temperature has no
impact.• At the critical temperature(Tc), spin has no
impact.
![Page 10: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/10.jpg)
Temperature – Ferromagnetics
• Si = +1 w/ probability g(hi); else -1• g(h) = 1/(1+exp(-2βh))• β = 1/(kBT)
• kB = Boltzman’s constant• T = temperature• Fβ(+/-hi)=1/(1+exp(-/+ 2βhi) Fβ(+/-hi)• Fβ(+/-hi) is a logistic function
![Page 11: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/11.jpg)
Temperature – Machine Learning
• Logistic function• Noise• Used in the elimination of spurious
local minima
![Page 12: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/12.jpg)
Mean Field Theory – Physics• The individual measurement and
summation of each member of a magnetic array is too expensive
• Physicists look to average values as an inexpensive way to extract further truth from a complex combinatronics problem.
![Page 13: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/13.jpg)
Mean Field Theory – Physics• hi=JwiJSJ+hext
• <hi>=JwiJ<SJ>+hext
• <Si>=tan(β<hi>) = tanh(βJwiJ<SJ>+hext)• <S>=tanh(βJ(S))
<S>
TcT
1
![Page 14: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/14.jpg)
Mean Field Theory – stochastic model
• <Si>=tanh(β/NJuζuiζu
J<SJ>)• We allow an assumption, that <Si> is
proportional to one of the stored patterns ζv
i • <Si>=m ζv
i
• <Ncorrect>=½N(1+m)
TcT
![Page 15: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/15.jpg)
Mean Field Theory • <Ncorrect>=½N(1+m)• There is a point at which noise
overcomes the ability of a network to make an informed decision.
![Page 16: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/16.jpg)
Conclusions• All these metaphors that pull from
physics are very tightly linked to energy.
• All metaphors concentrate on atomic events. (exception that proves the rule: mean field theory).
![Page 17: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/17.jpg)
Extra Time? Extra Topics!
![Page 18: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/18.jpg)
Entropy – Physics• The inevitable progression toward
chaos• The motion of energy and matter
away from an organized state.
![Page 19: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/19.jpg)
Entropy – Machine Learning• S = -PlogP
• For S = -Plog2P (the binary case) this is the average amount of additional information required to specify one of the staties.
![Page 20: Physics and Machine Learning](https://reader035.vdocument.in/reader035/viewer/2022081418/56815e00550346895dcc42cb/html5/thumbnails/20.jpg)
Quantum MechanicsFits within the context of our
expectation for where to look for Physics crossover
• Atomic – discrete and binary• Energy specificLast Class Lecture – use of Dyads.