machine learning and deep learning with the wolfram language … · 2018-10-19 · viktor khryapa...

Machine Learning and Deep Learning

with the Wolfram LanguageJérôme Louradour - Wolfram Research

[email protected]

Wolfram Language

http://reference.wolfram.com/language/

In[]:=

◼ 5000+ functions

◼ High-level and Coherent

◼ Interactive notebook

◼ Polished documentation

◼ Knowledgebase access

2 2018-10-13_AIUkraine.nb

http://reference.wolfram.com/language/

In[]:= blurSingleFaceimage_, face_ := ImageComposeimage, Blurface["Image"], 20, face"Position";

BlurFacesimage_ := FoldblurSingleFace, image, FindFacesimage, "Position", "Image";

In[]:= BlurFaces

Out[]=

2018-10-13_AIUkraine.nb 3

In[]:=

In[]:= What are the notable people from Kiev?

Kyiv CITY notable people born in city

Out[]= Mila Kunis → Day: Sun 14 Aug 1983 , Milla Jovovich → Day: Wed 17 Dec 1975 , Andriy Shevchenko → Day: Wed 29 Sep 1976 , Golda Meir → Day: Sun 15 May 1898 , Kazimir Malevich → Day: Sun 23 Feb 1879 ,

Vladimir Horowitz → Day: Thu 1 Oct 1903 , John Demjanjuk → Day: Sat 3 Apr 1920 , Mikhail Bulgakov → Day: Fri 15 May 1891 , Vaslav Nijinsky → Day: Wed 12 Mar 1890 , Max Levchin → Year: 1975 ,

Alexandr Dolgopolov → Day: Mon 7 Nov 1988 , Elena Baltacha → Day: Sun 14 Aug 1983 , Louise Nevelson → Day: Sun 23 Sep 1900 , Yevgeny Primakov → Day: Tue 29 Oct 1929 ,

Irène Némirovsky → Day: Tue 24 Feb 1903 , Sergiy Stakhovsky → Day: Mon 6 Jan 1986 , Victor Pinchuk → Day: Wed 14 Dec 1960 , German Khan → Day: Fri 26 Oct 1962 ,

Viktor Khryapa → Day: Tue 3 Aug 1982 , Vitaly Potapenko → Day: Fri 21 Mar 1975 , Anatole Litvak → Day: Fri 23 May 1902 , Denis Kudla → Day: Mon 17 Aug 1992 , Ephraim Katzir → Day: Mon 29 May 1916 ,

Dimitrij Ovtcharov → Day: Fri 2 Sep 1988 , Dema Kovalenko → Day: Sun 28 Aug 1977 , Anna Sten → Day: Wed 16 Dec 1908 , Leonid Fedun → Day: Tue 5 Apr 1955 , Alex Kuznetsov → Day: Thu 5 Feb 1987 ,

Zhan Beleniuk → Day: Thu 24 Jan 1991 , Mariya Koryttseva → Day: Sat 25 May 1985 , Nikolai Kuksenkov → Day: Fri 2 Jun 1989 , Anastasia Grymalska → Day: Thu 12 Jul 1990 ,

Gleb Lozino-Lozinskiy → Day: Sat 25 Dec 1909 , Daryna Zevina → Day: Thu 1 Sep 1994 , Józef Bohdan Zaleski → Day: Sun 14 Feb 1802 , Tetiana Luzhanska → Day: Tue 4 Sep 1984 ,

Tetyana Arefyeva → Day: Tue 3 Sep 1991 , Mykola Suk → Day: Fri 21 Dec 1945 , Mikhail Morgulis → Day: Wed 1 Oct 1941 , Anatoly Bannik → Month: Dec 1921 , Vladimir Novosiad → Day: Fri 12 Apr 1968 ,

Yonnie Starr → Day: Fri 11 Aug 1905 , Nina Svetlanova → Day: Sat 23 Jan 1932 , Margaryta Pesotska → Day: Fri 9 Aug 1991 , Angelina Kysla → Day: Fri 15 Feb 1991 ,

Anissa Khelfaoui → Day: Thu 29 Aug 1991 , Alexander Peli → Year: 1915 , Pavlo Tymoshchenko → Day: Mon 13 Oct 1986 , Jerzy Zagórski → Day: Tue 3 Dec 1907

In[]:= Hold @ Entity"City", "Kiev", "Kiev", "Ukraine"EntityProperty"City","PeopleBornInCity"

Out[]= Hold Kyiv notable people born in city


http://www.wolframalpha.com/

Machine Learning in the Wolfram Language

Tools to Train, Evaluate and Deploy models

◼ Supervised

◼ Classification → Classify

◼ Regression → Predict

◼ Unsupervised

◼ Clustering → FindClusters

◼ Dimensionality reduction → DimensionReduce

◼ Density estimation → LearnDistribution, AnomalyDetection

Model Zoo

◼ Big Neural Network Repository

High-level Applications

◼ Computer Vision

◼ Natural Language Processing

◼ Audio Signal Processing


Applications: Computer Vision

Object Recognition

In[]:= ImageIdentify

Out[]= Easter egg


Semantic Feature Extraction

In[]:= FacialFeatures

Out[]= Image → , Age → 26, Gender → Male, Emotion → happiness , Image → , Age → 37, Gender → Male, Emotion → anger ,

Image → , Age → 44, Gender → Male, Emotion → happiness , Image → , Age → 30, Gender → Male, Emotion → neutral ,

Image → , Age → 25, Gender → Male, Emotion → neutral , Image → , Age → 32, Gender → Male, Emotion → anger , Image → , Age → 26, Gender → Male, Emotion → neutral ,

Image → , Age → 43, Gender → Male, Emotion → happiness , Image → , Age → 28, Gender → Male, Emotion → happiness ,

Image → , Age → 34, Gender → Male, Emotion → happiness , Image → , Age → 30, Gender → Male, Emotion → neutral


Art

In[]:= ImageRestyle ,

Out[]=


Applications: Natural Language Processing

Question Answering

In[]:= StringTake[WikipediaData["Sergei Polunin"], 1000]

Out[]= Sergei Vladimirovich Polunin (Ukrainian: Сергій Володим́ирович Полун́ін, Serhiy Volodymyrovych Polunin;

Russian: Сергей́ Владим́ирович Полун́ин, Sergey Vladimirovich Polunin; born 20 November 1989) is a Ukrainian ballet dancer, actor and model.

As a freelance principal dancer, Polunin is guest artist at various theaters worldwide such as Royal Ballet, Sadler's Wells Theatre, Bolshoi Theatre, Stanislavski

and Nemirovich-Danchenko Moscow Academic Music Theatre, La Scala Theatre, Teatro San Carlo and is currently permanent guest artist for the Bayerisches Staatsballet.

== Life and career ==

Sergei Polunin was born in Kherson, Ukrainian SSR. From the age of four to eight, he trained at a gymnastics academy,

and then spent another four years at the Kiev State Choreographic Institute. His mother, Galina, moved with him to Kiev, while his father,

Vladimir Polunin, worked in Portugal to support them.After Polunin graduated from the Kyiv Choreographic Academy (КДХУ) he joined the British Roy

In[]:= FindTextualAnswer[

WikipediaData["Sergei Polunin"],

"What is the nationality of Sergei Polunin?",

3, "HighlightedSentence"] // Column

Out[]=

Sergei Polunin was born in Kherson, Ukrainian SSR.

Polunin also holds Serbian citizenship.

Sergei Vladimirovich Polunin ( Ukrainian : Сергій Володим́ирович Полун́ін, Serhiy Volodymyrovych Polunin;


Entity Recognition (and more...)

In[]:= TextContents"The flag of Ukraine is blue and yellow. In 1934 Kiev became the capital of Soviet Ukraine.

The city has a density of 3,299 people/km², with a population of 2,887,974 people in July 2015 and an area of 839 km²(324 sq mi)."

Out[]=

String Type Position Probability Interpretation HighlightedSnippet

Ukraine Country {13, 19} 0.926602 Ukraine The flag of Ukraine is blue and yellow. In 1934 Kiev became

blue Color {24, 27} 0.97199 The flag of Ukraine is blue and yellow. In 1934 Kiev

yellow Color {33, 38} 0.989897 of Ukraine is blue and yellow . In 1934 Kiev became the

1934 Date {44, 47} 0.860395 1934 is blue and yellow. In 1934 Kiev became the capital

Kiev AdministrativeDivision {49, 52} 0.956012 Kiev, Ukraine blue and yellow. In 1934 Kiev became the capital of

Kiev City {49, 52} 0.94606 Kiev, Kiev, Ukraine blue and yellow. In 1934 Kiev became the capital of

Ukraine Country {83, 89} 0.785337 Ukraine yellow. In 1934 Kiev became the capital of Soviet Ukraine .

3,299 people/km² Quantity {118, 133} 0.8 3299 people/km2 The city has a density of 3,299 people/km² , with a population of

2,887,974 people Quantity {157, 172} 0.9 2 887974 people with a population of 2,887,974 people in July 2015 and an area

July 2015 Date {177, 185} 0.934119 Jul 2015 of 2,887,974 people in July 2015 and an area of 839

839 km² Quantity {202, 208} 0.8 839 km2 people in July 2015 and an area of 839 km² (324 sq mi).

324 sq mi Quantity {210, 218} 0.8 324 mi2 people in July 2015 and an area of 839 km²( 324 sq mi ).

In[]:= TextContents["I have a dog. I eat an hot dog."]

Out[]=

String Type Position Probability Interpretation HighlightedSnippet

dog Species {10, 12} 0.541379 Infraspecies:Canis Lupus Familiaris I have a dog . I eat an hot dog

hot dog Food {24, 30} 0.8 Entity["Food", {EntityProperty["Food", "FoodType"] -> ContainsExactly[{Entity["FoodType", "Frankfurter"]}], EntityProperty["Food", "AddedFoodTypes"] -> ContainsExactly[{}]}] I have a dog. I eat an hot dog

In[]:= notablePeople = TextCases[

WikipediaData["Kiev"],

"Person" → "Interpretation"]

Out[]= Abraham Ortelius , Joseph M. Marshall III , Aung San Suu Kyi , Paul Sefchek , Natalia Khoreva , Aung San Suu Kyi , Ptolemy , Andrew , Aung San Suu Kyi ,

Paul Sefchek , Natalia Khoreva , Batu Khan , Taras Shevchenko , Cyril of Alexandria , Josef Stalin , Vitali Klitschko , Vitali Klitschko , Tsar Nicholas I , Lenin ,

Josef Stalin , Vladimir the Great , Aung San Suu Kyi , Natalia Khoreva , Mikhail Bulgakov , Viktor Yanukovych , Shakira , Mikhail Bulgakov , Valentin Boreyko ,

Martin Luther King , Vladimir Horowitz , Milla Jovovich , Kazimir Malevich , Golda Meir , Alexander Markowich Ostrowski , Nicholas Pritzker II , Andriy Shevchenko , Igor Sikorsky


In[]:= notablePeopleBornInKiev = DeleteDuplicates@SelectnotablePeople, # place of birth === Kyiv CITY &

Out[]= Mikhail Bulgakov , Vladimir Horowitz , Milla Jovovich , Kazimir Malevich , Golda Meir , Andriy Shevchenko

In[]:= AssociationThreadnotablePeopleBornInKiev → EntityValuenotablePeopleBornInKiev, occupation

Out[]= Mikhail Bulgakov → {author}, Vladimir Horowitz → {pianist}, Milla Jovovich → {actor}, Kazimir Malevich → {painter}, Golda Meir → {politician}, Andriy Shevchenko → {soccer player}

In[]:= TextCasesWikipediaData["Kiev"],

Mikhail Bulgakov PERSON , Vladimir Horowitz PERSON , Milla Jovovich PERSON , Kazimir Malevich PERSON , Golda Meir PERSON , Andriy Shevchenko PERSON → "HighlightedSnippet"

Out[]= Mikhail Bulgakov → Andrew's Church; the home of Kiev born writer, Mikhail Bulgakov ; the monument to Yaroslav the Wise, the Grand, Mikhail Bulgakov , Russian writer,

Vladimir Horowitz → Vladimir Horowitz , classical pianist, Milla Jovovich → Milla Jovovich , American actress,

Kazimir Malevich → Kazimir Malevich , pioneer of geometric abstract art and the originator of the avant-garde Suprematist movement,

Golda Meir → Golda Meir , Israeli politician, the fourth Prime Minister of Israel, Andriy Shevchenko → Andriy Shevchenko , Ukrainian footballer

locations = TextCasesWikipediaData["Sergei Polunin"], "Location" → #String → #Interpretation &;

In[]:= locationsStats = Map[Counts, GroupBy[locations, Last → First]]

Out[]= GeoPosition[{49., 32.}] → Ukrainian → 3, GeoPosition[{60., 100.}] → Russian → 5, Russia → 1, GeoPosition[{55.7603, 37.6186}] → Bolshoi Theatre → 1,

GeoPosition[{55.75, 37.62}] → Moscow → 1, GeoPosition[{45.4678, 9.18861}] → La Scala Theatre → 1, GeoPosition[{46.63, 32.6}] → Kherson → 1, GeoPosition[{50.4499, 30.5507}] → Kiev → 3,

GeoPosition[{39.5, -8.}] → Portugal → 1, GeoPosition[{50.43, 30.52}] → Kyiv → 1, GeoPosition[{43.0442, -88.2578}] → Academy → 1, GeoPosition[{51.5009, -0.177436}] → Royal Albert Hall → 2,

GeoPosition[{38., -97.}] → American → 2, GeoPosition[{44.8167, 20.4594}] → National Museum of Serbia → 1, GeoPosition[{44., 21.}] → Serbian → 1, GeoPosition[{55.04, 82.93}] → Novosibirsk → 2,

GeoPosition[{55.6432, 37.6662}] → Moscow → 2, GeoPosition[{49.9563, 14.5891}] → Bohemian → 1, GeoPosition[{46.52, 6.62}] → Lausanne → 1, GeoPosition[{54., -2.}] → British → 1, United Kingdom → 1

In[2]:= GeoBubbleChart[Map[Total, locationsStats], ChartLabels → Values@Map[First@*Keys, locationsStats], GeoProjection → "Equirectangular", ImageSize → Scaled[0.7]]

Out[2]=


Model Zoo: Built-in Classifiers

In[]:= Classify["NotablePerson"]

Out[]= Milla Jovovich

In[]:= Classify["Spam"][{

"Hi Bob, I'll travel to Kiev!",

"You won a free travel to Kiev!"

}]

Out[]= {False, True}

In[]:= Classify["Sentiment"][{

"I have a new computer",

"I had to reinstall my new computer"

}]

Out[]= {Positive, Negative}


Model Zoo: Neural Net Repository◼ https://resources.wolframcloud.com/NeuralNetRepository

◼ 75+ networks, growing

◼ Demo of typical use

Images

In[]:= NetModel["ResNet-101 Trained on YFCC100m Geotagged Data"]

Out[]= NetChain Input port: imageOutput port: classNumber of layers: 43

In[]:= position = NetModel["ResNet-101 Trained on YFCC100m Geotagged Data"]

Out[]= GeoPosition[{50.4537, 30.5197}]

In[]:= GeoGraphics[position]

Out[]=


https://resources.wolframcloud.com/NeuralNetRepository

GeoBubbleChart

NetModel["ResNet-101 Trained on YFCC100m Geotagged Data"] , {"TopProbabilities", 30}

Out[]=


In[]:= colorizeimg_Image :=

ImagePrepend

ArrayResampleNetModel"Colorful Image Colorization Trained on ImageNet Competition Data"img,PrependReverse@ImageDimensions@img,2

, ImageDataColorSeparateimg,"L", Interleaving → False, ColorSpace → "LAB"

In[]:= colorize /@ ,

Out[]= ,


Text: Word Embeddings

In[]:= animals =

{"Alligator", "Ant", "Bear", "Bee", "Bird", "Camel", "Cat", "Cheetah", "Chicken", "Chimpanzee", "Cow", "Crocodile", "Deer", "Dog", "Dolphin", "Duck", "Eagle", "Elephant", "Fish", "Fly"};

fruits = {"Apple", "Apricot", "Avocado", "Banana", "Blackberry", "Blueberry", "Cherry", "Coconut", "Cranberry", "Grape",

"Turnip", "Mango", "Melon", "Papaya", "Peach", "Pineapple", "Raspberry", "Strawberry", "Ribes", "Fig"};

In[]:= FeatureSpacePlot[

Join[animals, fruits],

FeatureExtractor → NetModel["GloVe 100-Dimensional Word Vectors Trained on Wikipedia and Gigaword 5 Data"]]

Out[]=

Alligator

Ant

Bear

Bee

Bird

Camel

Cat

Cheetah

Chicken

Chimpanzee

Cow

Crocodile

Deer

Dog

Dolphin

Duck

EagleElephant

Fish

Fly

Apple

Apricot

AvocadoBanana

Blackberry

Blueberry

Cherry

Coconut

Cranberry

Grape Turnip

Mango

Melon

Papaya

Peach

Pineapple

Raspberry

Strawberry

Ribes

Fig


Text: Contextual Word Embeddings

In[]:= NetModel["ELMo Contextual Word Representations Trained on 1B Word Benchmark"]

Out[]= NetGraph

E C M #

M

M #

M

M

SR

SR

C

+

SR

+

SR SM

CSR SR SM

SMM

Inputs OutputsInput: expression ContextualEmbedding/1: matrix (size: n4 ×1024)

ContextualEmbedding/2: matrix (size: n6 ×1024)Embedding: matrix (size: n8 ×1024)

In[]:= MatrixPlot /@ NetModel["ELMo Contextual Word Representations Trained on 1B Word Benchmark"]["Hello world!"]

Out[]= ContextualEmbedding/1 →1 500 1024

123

1 500 1024

123, ContextualEmbedding/2 →

1 500 1024

123

1 500 1024

123, Embedding →1 500 1024

123

1 500 1024

123

In[]:= averagedElmo = Withelmo = NetModel"ELMo Contextual Word Representations Trained on 1B Word Benchmark",NetFlatten @ NetGraphelmo, ThreadingLayer[(#1+#2+#3)/3&],MapNetPort[{1,#}]&, NetInformationelmo,"OutputPortNames"→2

Out[]= NetGraph

E C

M

M #

M

M # M SR

SR

C

+

SR

+

SR SM

#CSR SR SM

SMM

Inputs OutputsInput: expression Output: matrix (size: n2 ×1024)


sentences = {

"Apple makes laptops", "Apple pie is delicious", "Apple juice is full of sugar",

"Apple baked with cinnamon is scrumptious", "Apple reported large quarterly profits", "Apple is a large company"};

In[]:= FeatureSpacePlotsentences, FeatureExtractor → First@averagedElmo[#] &, LabelingFunction → Callout

Out[]=

Apple makes laptops

Apple pie is delicious

Apple juice is full of sugar

Apple baked with cinnamon is scrumptious

Apple reported large quarterly profits

Apple is a large company


Automated Machine Learning

Example: Training a Classifier

In[]:= scrapeImages[string_] := Thread[WebImageSearch[string, "Thumbnails", MaxItems → 40] → string]

In[]:= classes = {"Bortsch", "Kapusniak", "Solianka"};

In[]:= images = Union @@ Map[scrapeImages, classes];

In[]:= {training, test} = TakeList[RandomSample[images], {80, 40}];

In[]:= RandomSample[training, 5]

Out[]= → Solianka, → Bortsch, → Bortsch, → Kapusniak, → Bortsch

In[]:= classifier = Classify[training, TimeGoal → Quantity[20, "Seconds"]]

Out[]= ClassifierFunction Input type: ImageClasses: Bortsch, Kapusniak, Solianka

Data not in notebook; Store now »

In[]:= cm = ClassifierMeasurements[classifier, test]

Out[]= ClassifierMeasurementsObject Classifier: LogisticRegressionNumber of test examples: 40

Data not in notebook; Store now »

In[]:= cm["ConfusionMatrixPlot"]

Out[]=

14

13

13

Bortsch

Kapusniak

Solianka

Bort

sch

Kapusnia

k

Solia

nka

13

15

12

predicted class

actu

alcla

ss

11

0

3

0

13

0

2

2

9


In[]:= cm["WorstClassifiedExamples" → 5]

Out[]= → Bortsch, → Solianka, → Solianka, → Solianka, → Kapusniak

In[]:= form = FormFunction[{"image" → "Image"}, classifier[#image, "TopProbabilities"] &]

Out[]= FormFunctionimage Browse…

Submit

In[]:= url = CloudDeploy[form, Permissions → "Public"]

Out[]= CloudObjecthttps://www.wolframcloud.com/objects/560b9dbb-96fb-44b2-8f5d-f982fe9406e8

In[]:= URLShorten[url]

Out[]= https://wolfr.am/yjJdU3Wl


https://www.wolframcloud.com/objects/560b9dbb-96fb-44b2-8f5d-f982fe9406e8

Automated Machine Learning

Feature Extraction

In[]:=

0.51

1

111

1

0.51

1

21

1

1

1

1

1

1

1

1

22

11

1

2.0.5

1

1.5

1

22.

1

1

1

1

1

1

1

1

1

121

21 11 1 1 11

1

1

1

1

1

1

1

1

1

2

1

1

11

21

2

1

1

0.5

1

1.5

1

2

Audio

NumericalVectorSequence

NumericalSequence

BooleanVectorNominalVector

NumericalVector

Text

NominalSequence

Color

ComplexVector

Image3D

Image

Location

NominalBag

NumericalBag

NumericalTensorSequence

Hyperparameters tuning

◼ Initial set of configurations (models + hyperparameters)

◼ Experiments on small datasets

◼ Most promising configurations trained on larger datasets

In[]:=


In[]:= mnist = RandomSample[ResourceData["MNIST"], 30 000];

In[]:= digitClassifier = Classify[mnist, TimeGoal → 45]

Out[]= ClassifierFunction Input type: ImageNumber of classes: 10

In[]:= ClassifierInformation[digitClassifier]

Out[]=

Classifier information

Data type Image

Number of classes 10

Accuracy 90.9% ± 0.53%

Method LogisticRegression

Single evaluation time 1.77ms/example

Batch evaluation speed 38.1 examples/ms

Loss 0.339 ± 0.019

Model memory 373. kB

Training examples used 30000 examples

Training time 1min 27s

●

●

●

●

●

50 100 500 1000 5000 104

0.4

0.6

0.8

1.0

1.2

training examples used

Learning curve

Interactivity and user-friendliness

◼ Progress bar

◼ Interruptibility

◼ Training time specification

◼ Measurements & Learning curves


Neural Networks framework Polished High-level framework without performance sacrifices

◼ User-friendly

◼ Interactive

◼ Automatic support of variable-length sequences

◼ Repository of pretrained network

◼ Easy to do "Network surgery"

◼ Pre and Post-processing in the network

◼ Check of constraints, human-readable error messages

In[3]:= LongShortTermMemoryLayer[5, "Input" → 10]

LongShortTermMemoryLayer : Specification 10 is not compatible with port "Input", which must be a n× matrix.

Out[3]= $Failed

◼ Performance

◼ MXNet back-end

◼ Multi-GPU and TensorCore support (Mixed-precision)

◼ Documentation

◼ https://reference.wolfram.com/language/tutorial/NeuralNetworksOverview.html

◼ Wolfram Support

◼ CloudDeploy


https://reference.wolfram.com/language/tutorial/NeuralNetworksOverview.html

Network Graph Visualisation

In[4]:= NetModel["ELMo Contextual Word Representations Trained on 1B Word Benchmark"]

Out[4]= NetGraph

E C M #

M

M #

M

M

SR

SR

C

+

SR

+

SR SM

CSR SR SM

SMM

cnn 1: NetChainInput 3-tensor (size: n1 ×50×16)

1 ConvolutionLayer 3-tensor (size: n2 ×50×32)2 AggregationLayer matrix (size: n2 ×32)

Output matrix (size: n2 ×32)

2: AggregationLayerParametersFunction: Max

Levels: 2

PortsInput: 3-tensor (size: n×50×32)Output: matrix (size: n×32)


In[]:= NetModel["Wolfram FindTextualAnswer Net for WL 11.3 (Raw Model)"]

Out[]= NetGraph

GR

C

+

GR SR

MX

GR

+GR SR

D

LSTM

SR

C

LSTM SR D

+

LSTM

SR

C

LSTM SR

D

M

M

M

M S

S

Inputs OutputsWordMatch: matrix (size: n1 ×3) End: matrix (size: n1 ×1)Question: string StartActivation: matrix (size: n1 ×2)Context: string EndActivation: matrix (size: n1 ×2)

Start: matrix (size: n1 ×1)


Example: Transfer Learning

In[]:= inception = NetModel["Inception V3 Trained on ImageNet Competition Data"]


In[]:= extractor = NetTake[inception, 30]

Out[]= NetChain Inputimage3-tensor (size: 3×299×299)

conv_conv2d ConvolutionLayer 3-tensor (size: 32×149×149)conv_batchnorm BatchNormalizationLayer 3-tensor (size: 32×149×149)conv_relu Ramp 3-tensor (size: 32×149×149)conv_1_conv2d ConvolutionLayer 3-tensor (size: 32×147×147)conv_1_batchnorm BatchNormalizationLayer 3-tensor (size: 32×147×147)conv_1_relu Ramp 3-tensor (size: 32×147×147)conv_2_conv2d ConvolutionLayer 3-tensor (size: 64×147×147)conv_2_batchnorm BatchNormalizationLayer 3-tensor (size: 64×147×147)conv_2_relu Ramp 3-tensor (size: 64×147×147)pool PoolingLayer 3-tensor (size: 64×73×73)conv_3_conv2d ConvolutionLayer 3-tensor (size: 80×73×73)conv_3_batchnorm BatchNormalizationLayer 3-tensor (size: 80×73×73)conv_3_relu Ramp 3-tensor (size: 80×73×73)conv_4_conv2d ConvolutionLayer 3-tensor (size: 192×71×71)conv_4_batchnorm BatchNormalizationLayer 3-tensor (size: 192×71×71)conv_4_relu Ramp 3-tensor (size: 192×71×71)pool1 PoolingLayer 3-tensor (size: 192×35×35)Inception1 NetGraph (23 nodes) 3-tensor (size: 256×35×35)Inception2 NetGraph (23 nodes) 3-tensor (size: 288×35×35)Inception3 NetGraph (23 nodes) 3-tensor (size: 288×35×35)Inception4 NetGraph (14 nodes) 3-tensor (size: 768×17×17)Inception5 NetGraph (32 nodes) 3-tensor (size: 768×17×17)Inception6 NetGraph (32 nodes) 3-tensor (size: 768×17×17)Inception7 NetGraph (32 nodes) 3-tensor (size: 768×17×17)Inception8 NetGraph (32 nodes) 3-tensor (size: 768×17×17)Inception9 NetGraph (20 nodes) 3-tensor (size: 1280×8×8)Inception10 NetGraph (29 nodes) 3-tensor (size: 2048×8×8)Inception11 NetGraph (29 nodes) 3-tensor (size: 2048×8×8)global_pool PoolingLayer 3-tensor (size: 2048×1×1)flatten FlattenLayer vector (size: 2048)

Output vector (size: 2048)

In[]:= trainingPreprocessed = extractor[training[[All, 1]], TargetDevice → "GPU"] → training[[All, 2]];

In[]:= head = NetChain[<|

"dropout" → DropoutLayer[],

"lin" → LinearLayer[],

"softmax" → SoftmaxLayer[]

|>, "Output" → NetDecoder[{"Class", classes}]

]

Out[]= NetChain uninitialized

Input tensordropout DropoutLayer tensorlin LinearLayer vector (size: 3)softmax SoftmaxLayer vector (size: 3)

Output class

In[]:= trained = NetTrain[head, trainingPreprocessed, MaxTrainingRounds → Quantity[20, "Seconds"], TargetDevice → "GPU"]

Out[]= NetChain Input vector (size: 2048)dropout DropoutLayer vector (size: 2048)lin LinearLayer vector (size: 3)softmax SoftmaxLayer vector (size: 3)

Output class


In[]:= netClassifier = NetJoin[extractor, trained]

Out[]= NetChain Inputimage3-tensor (size: 3×299×299)

conv_conv2d ConvolutionLayer 3-tensor (size: 32×149×149)conv_batchnorm BatchNormalizationLayer 3-tensor (size: 32×149×149)conv_relu Ramp 3-tensor (size: 32×149×149)conv_1_conv2d ConvolutionLayer 3-tensor (size: 32×147×147)conv_1_batchnorm BatchNormalizationLayer 3-tensor (size: 32×147×147)conv_1_relu Ramp 3-tensor (size: 32×147×147)conv_2_conv2d ConvolutionLayer 3-tensor (size: 64×147×147)conv_2_batchnorm BatchNormalizationLayer 3-tensor (size: 64×147×147)conv_2_relu Ramp 3-tensor (size: 64×147×147)pool PoolingLayer 3-tensor (size: 64×73×73)conv_3_conv2d ConvolutionLayer 3-tensor (size: 80×73×73)conv_3_batchnorm BatchNormalizationLayer 3-tensor (size: 80×73×73)conv_3_relu Ramp 3-tensor (size: 80×73×73)conv_4_conv2d ConvolutionLayer 3-tensor (size: 192×71×71)conv_4_batchnorm BatchNormalizationLayer 3-tensor (size: 192×71×71)conv_4_relu Ramp 3-tensor (size: 192×71×71)pool1 PoolingLayer 3-tensor (size: 192×35×35)Inception1 NetGraph (23 nodes) 3-tensor (size: 256×35×35)Inception2 NetGraph (23 nodes) 3-tensor (size: 288×35×35)Inception3 NetGraph (23 nodes) 3-tensor (size: 288×35×35)Inception4 NetGraph (14 nodes) 3-tensor (size: 768×17×17)Inception5 NetGraph (32 nodes) 3-tensor (size: 768×17×17)Inception6 NetGraph (32 nodes) 3-tensor (size: 768×17×17)Inception7 NetGraph (32 nodes) 3-tensor (size: 768×17×17)Inception8 NetGraph (32 nodes) 3-tensor (size: 768×17×17)Inception9 NetGraph (20 nodes) 3-tensor (size: 1280×8×8)Inception10 NetGraph (29 nodes) 3-tensor (size: 2048×8×8)Inception11 NetGraph (29 nodes) 3-tensor (size: 2048×8×8)global_pool PoolingLayer 3-tensor (size: 2048×1×1)flatten FlattenLayer vector (size: 2048)dropout DropoutLayer vector (size: 2048)lin LinearLayer vector (size: 3)softmax SoftmaxLayer vector (size: 3)

Output class

In[]:= netClassifier , "TopProbabilities"

Out[]= {Solianka → 0.462191, Kapusniak → 0.41293, Bortsch → 0.124879}

In[]:= cmNet = ClassifierMeasurements[netClassifier, test];

cmNet["ConfusionMatrixPlot"]

Out[]=

13

16

11

Bortsch

Kapusniak

Solianka

Bort

sch

Kapusnia

k

Solia

nka

13

15

12

predicted class

actu

alcla

ss

12

0

1

0

15

1

1

0

10


Neural Networks Surgery and inspection

In[]:= net = NetModel["Wolfram ImageIdentify Net for WL 11.1"]


In[]:= visualizeFeaturesimg_, level_ := Image /@ NetTakeNetModel"Wolfram ImageIdentify Net for WL 11.1", levelimg;

In[]:= visualizeFeatures , 5

Out[]= , , , , , , , , , , , , , , ,

, , , , , , , , , , , , , , , ,

, , , , , , , , , , , , , , , ,

, , , , , , , , , , , , , , , ,


In[]:= AnimatevisualizeFeatures , level, {level, Range[22]}


In[]:= filterDisplay= Image3DMapThreadImageMultiply, ColorSeparateImage#, Interleaving→False, Red,Green,Blue, ImageSize→Tiny&;

In[]:= filterDisplay /@ NetExtract[NetModel["Wolfram ImageIdentify Net for WL 11.1"], {"conv_1", "Weights"}]

Out[]= , , , , , , , , , ,

, , , , , , , , , ,

, , , , , , , , , , ,

, , , , , , , , , , ,

, , , , , , , , , , ,

, , , , , , , , , ,


What’s next

Automatic Machine Learning

◼ Reversible Generative Models

◼ Few-Shot learning

Neural Networks

Take-away messages◼ The power of Transfer Learning

or

Why you should not need to design your network from scratch

◼ Best solutions to build application: smart combination of Machine Learning and knowledge

◼ The Grail: Mapping visual/textual/audio entities into a unique semantic space


Дякую

Out[]= Thank YouДякую谢谢고맙습니다

شكرا

متشکرم

ありがとうございました

אדאנק

ขอขอบคุณآپ کا شکریہ

ĎakujemTak

நன்றி

Twdh

'D'Nq

Dakujem

Mulţumesc

Спасибо

Danke

Շնորհակալություն

SpasiboΕυχαριστώ

Děkuji

DekujiCảm Ơn Bạn

Благодаря

Dakuu

Obrigado

Ap Ka Shkryہ

Aciu Ačiū

Shkra

Blagodara

Mtshkrm

Salamat Teşekkür Ederim

Cam On BanAsante Dziękuję Ci

Dziekuje Ci

תודה

DankieTack Så Mycket

Tack Sa MycketMerci

Go Raibh Maith Agat

Nanri

Aitäh

Aitah

GraciasPaldies

Grazie

Grazzi

Xie Xie

Хвала Вам

Diolch

Gomabseubnida

MultumescHvala Vam

Khx Khxbkhun

Tesekkur Ederim

Takk Skal Du Ha

Þakka Þér

Eucharistoধ�যবাদ

Snorhakalut'Yun

Dhan'Yabada

Terima Kasih

Dhan'Yavada

Kiitos

Arigatougozaimashita

Thakka Ther

Dank Je Wel

Faleminderit

Hvala Tiध�यवाद

Gratias Tibi

Related links◼ Blog posts

◼ http://blog.wolfram.com/2017/10/10/building-the-automated-data-scientist-the-new-classify-and-predict/

◼ http://blog.wolfram.com/2018/02/15/new-in-the-wolfram-language-findtextualanswer/

◼ http://blog.wolfram.com/2018/06/14/launching-the-wolfram-neural-net-repository/

◼ Design of the Wolfram Language on Twitch

◼ https://www.twitch.tv/stephen_wolfram/videos/all


http://blog.wolfram.com/2017/10/10/building-the-automated-data-scientist-the-new-classify-and-predict/

http://blog.wolfram.com/2018/02/15/new-in-the-wolfram-language-findtextualanswer/

http://blog.wolfram.com/2018/06/14/launching-the-wolfram-neural-net-repository/

https://www.twitch.tv/stephen_wolfram/videos/all

machine learning and deep learning with the wolfram language … · 2018-10-19 · viktor khryapa...

Documents