trivadis techevent 2016 backpropagation - das e = mc2 des machine learning - in neuronalen netzen by...

BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENF HAMBURG KOPENHAGEN LAUSANNE MÜNCHEN STUTTGART WIEN ZÜRICH

Backpropagation: E=mc2 TE Herbst 2016

Ralf Moosandl

INFOBOX – Lesen und Löschen •  Auf der ersten Folie kann eine

Überschrift und ein optionaler Untertitel platziert werden

•  Der/die Titel wird direkt unter den Namen geschrieben (Shift+Return)

•  Werden mehrere Referenten genannt bitte nur die Namen untereinander schreiben (meist ist dann kein Platz für Titel, etc.)

Agenda

2 04/10/16

1.  Neuron and Brain 2.  Artificial neural networks 3.  Repeating and Learning 4.  Backpropagation

INFOBOX – Lesen und Löschen •  Wenn die Agenda als Zwischenseite

verwendet wird, bitte das betreffende Kapitel mit roter Schriftfarbe hervorheben

•  Zum optimalen Ausrichten der Objekte

die Führungslinien (mit einem Rechtsklick auf die Seite) anzeigen

3 04/10/16

Neuron and Brain

INFOBOX – Lesen und Löschen •  Im Kapiteltrenner wird der Text des

Kapitels zentriert in das Textfeld geschrieben

•  Achte bitte darauf die Kapitel möglichst kurz zu nennen, weniger Text und knackige Namen sind mehr !

Neuron and Brain

4 04/10/16

INFOBOX – Lesen und Löschen •  Aufzählungszeichen auf Textseiten

fügen sie hinzu/entfernen sie durch Listenebene erhöhen /Listenebene verringern

•  Besonders für die erste Ebene gilt, dass sie NICHT auf Aufzählungs-zeichen klicken, sondern wie oben beschrieben arbeiten

Neuron and Brain

5 04/10/16




Neuron and Brain

6 04/10/16




Neuron and Brain

7 04/10/16




8 04/10/16

Artificial Neural Networks





9 04/10/16





10 04/10/16




  Origins: Algorithms that try to mimic the brain

  Widely used in 80s and early 90s

  Popularity diminished in late 90s

  Recent Resurgence: State-of-the-art technique for many applications, as computational power raised

  Brain learns to see, hear, sensitive stuff, do math

  In old days you would have to write different kinds of software algorithms, but the brain does it the way, it does it

11 04/10/16

Repetition vs. Understanding





12 04/10/16

  2 Groups, closed to the door: try to remember, others: try to understand




1

1 2

2

3

3


13 04/10/16

  Question: Is there a dot missing and if so which coordinates does it have?

Answere of Group R:

Answere of Group U:




1

1 2

2

3

4

3 4


14 04/10/16

  E= m c c ?




1

15 04/10/16

Evolution




Evolution

16 04/10/16

  Starting Point

  Draw Example




Evolution

17 04/10/16

  Modify number of neurons




Evolution

18 04/10/16

  2 hidden layers




Evolution

19 04/10/16

Example




Evolution

20 04/10/16

Activation function




Evolution

21 04/10/16

Advanced NN




Evolution

22 04/10/16

Perturbing all Weights

Randomly perturbe one weight and check whether performance improves

Very inefficient. We need to do multiple forward passes on a representativ set of training cases just to change one weight.

Backprop is much better. N weights => Backprop is N times more efficient

  Large weight perturbations at the end of learning make things worse because weights need to have the right relative values

Changing many weights in parallel does not improve the situation through the noise created by all the changes to other weights

Changing the activation function make things better, as there are fewer activities then weights. Backprop wins by a factor of the number of neurons




23 04/10/16

Backpropagation




Backpropagation

24 04/10/16

  Change in Error caused by change of weights




Backpropagation

25 04/10/16

  Example




Backpropagation

26 04/10/16

  The idea behind backpropagation

We do not know what the hidden units ought to do, but we can compute how (fast) the error changes as we change a hidden activity

We can calculate how changing weights of the last layer changes the error

  After that:

We can calculate how changing weights of the layer before the last layer changes the error

It is an algorithm to take one training case and efficiently calc the change of the error w.r.t. the change of a weight




Ralf Moosandl Senior Consultant

Tel. +41 58 459 52 42 [email protected]

04/10/16 Name Presentation 27

INFOBOX – Read and delete •  There are two versions of the last slide

available, one for the contact details of a speaker, and one for two or more speakers.

•  Name, title and location always underneath one another in one row (Shift+Return)

•  This idea is that this is the last slide (also for questions and answers) and is on the screen for a long time at the end of the presentation, so the viewers have the chance to write down the contact data J

Session Feedback – now

TechEvent September 2016 28 09.09.2016

Please use the Trivadis Events Mobile App to give session feedback

Use "My schedule" if you registered for this session

Otherwise use "Agenda" and the search function

If the mobile App does not work (or if you have a Windows Phone) use your Mobile Browser

–  URL: http://trivadis.quickmobileplatform.eu/

–  Username: <your_loginname> (like svv)

–  Password: sent by mail...

trivadis techevent 2016 backpropagation - das e = mc2 des machine learning - in neuronalen netzen by...

Technology