tensorflow input pipelineassembling identical subgraphs 3. use pre-trained weights (from vgg-19) a....
TRANSCRIPT
![Page 1: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/1.jpg)
TensorFlow Input PipelineCS 20SI:
TensorFlow for Deep Learning ResearchLecture 9
2/10/2017
1
![Page 2: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/2.jpg)
2
![Page 3: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/3.jpg)
Announcements
3
Assignment 2 is out. It’s fun, but tricky. Start early!
No class next Wednesday (Feb 15) because of TensorFlow Dev Summit(sign up for live streaming party on campus)
Email me if you’d like to do IG for assignment 1
![Page 4: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/4.jpg)
Agenda
Data Readers Revisited
TFRecord
Variable Initializer
Graph Collection
Style Transfer
4
![Page 5: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/5.jpg)
Queues
5
tf.Session objects are designed to multithreaded → can run ops in parallel
![Page 6: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/6.jpg)
Queues
6
Important TensorFlow objects for computing tensors asynchronously in a graph.
● Multiple threads prepare training examples and push them in the queue ● A training thread executes a training op that dequeues mini-batches from the
queue
![Page 7: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/7.jpg)
Queues
7
Important TensorFlow objects for computing tensors asynchronously in a graph.
● All threads must be able to stop together● Exceptions must be caught and reported● Queues must be properly closed when stopping.
![Page 8: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/8.jpg)
Queues
8
TensorFlow queues can’t run without proper threading, but threading isn’t exactly pleasant in Python
![Page 9: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/9.jpg)
tf.Coordinator and tf.train.QueueRunner
9
● QueueRunnercreate a number of threads cooperating to enqueue tensors in the same queue
![Page 10: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/10.jpg)
tf.Coordinator and tf.train.QueueRunner
10
● QueueRunnercreate a number of threads cooperating to enqueue tensors in the same queue
● Coordinatorhelp multiple threads stop together and report exceptions to a program that waits for them to stop
Very similar to threadpool in CS110Don’t worry if this sounds confusing.Example in a bit
![Page 11: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/11.jpg)
Queues
11
Queue What’s it? Ops supported
tf.FIFOQueue Dequeues elements in first in first out order
enqueueenqueue_manydequeue
tf.RandomShuffleQueue Dequeues elements in a random order
enqueueenqueue_manydequeue
tf.PaddingFIFOQueue FIFOQueue with padding to supports batching variable_size tensors
enqueueenqueue_manydequeuedequeue_many
tf.PriorityQueue FIFOQueue whose enqueue and queue have another argument: priority
enqueueenqueue_manydequeue
![Page 12: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/12.jpg)
12
Queues
![Page 13: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/13.jpg)
Create a queue
13
tf.FIFOQueue(capacity, min_after_dequeue, dtypes, shapes=None, names=None ...)
Same for other queues
![Page 14: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/14.jpg)
Queue example
14
all_data = 10 * np.random.randn(N_SAMPLES, 4) + 1 all_target = np.random.randint(0, 2, size=N_SAMPLES)
queue = tf.FIFOQueue(capacity=50, dtypes=[tf.float32, tf.int32], shapes=[[4], []])
enqueue_op = queue.enqueue_many([all_data, all_target])data_sample, label_sample = queue.dequeue()
qr = tf.train.QueueRunner(queue, [enqueue_op] * NUM_THREADS)with tf.Session() as sess:
# create a coordinator, launch the queue runner threads.coord = tf.train.Coordinator()enqueue_threads = qr.create_threads(sess, coord=coord, start=True)for step in xrange(100): # do to 100 iterations
if coord.should_stop():break
one_data, one_label = sess.run([data_sample, label_sample])coord.request_stop()coord.join(enqueue_threads)
09_queue_example.py
![Page 15: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/15.jpg)
Queue example
15
# dummy dataall_data = 10 * np.random.randn(N_SAMPLES, 4) + 1 all_target = np.random.randint(0, 2, size=N_SAMPLES)
In practice, you can use any op to read in your data,even placeholder!
![Page 16: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/16.jpg)
Queue example
16
...
queue = tf.FIFOQueue(capacity=50, dtypes=[tf.float32, tf.int32], shapes=[[4], []])# create queue. # dtypes specifies types of data and label# shapes specifies shape of data and label
...
![Page 17: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/17.jpg)
Queue example
17
...
enqueue_op = queue.enqueue_many([all_data, all_target])data_sample, label_sample = queue.dequeue()
# a common practice is to enqueue all data at once, but dequeue one by one...
![Page 18: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/18.jpg)
Queue example
18
...
qr = tf.train.QueueRunner(queue, [enqueue_op] * NUM_THREADS)with tf.Session() as sess:
# create a coordinator, launch the queue runner threads.coord = tf.train.Coordinator()enqueue_threads = qr.create_threads(sess, coord=coord, start=True)for step in xrange(100): # do to 100 iterations
if coord.should_stop():break
one_data, one_label = sess.run([data_sample, label_sample])coord.request_stop()coord.join(enqueue_threads)
.... You can use data_sample and label_sample to do all the training ops as if with placeholders
![Page 19: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/19.jpg)
Dequeue multiple elements?
19
![Page 20: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/20.jpg)
tf.train.batch or tf.train.shuffle_batch if you want to your batch to be shuffled
20
I have never been able to get these to work with independent queues
Re: dequeue_many is tricky with queues
![Page 21: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/21.jpg)
tf.Coordinator
21
Can be used to manage the threads you created without queues
![Page 22: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/22.jpg)
tf.Coordinator
22
import threading
# thread body: loop until the coordinator indicates a stop was requested.# if some condition becomes true, ask the coordinator to stop.
def my_loop(coord):while not coord.should_stop():
...do something... if ...some condition...: coord.request_stop()
# main code: create a coordinator.coord = tf.Coordinator()
# create 10 threads that run 'my_loop()'# you can also create threads using QueueRunner as the example abovethreads = [threading.Thread(target=my_loop, args=(coord,)) for _ in xrange(10)]
# start the threads and wait for all of them to stop.for t in threads: t.start()coord.join(threads)
Just like threadpool
Take CS110 for more threading fun!
![Page 23: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/23.jpg)
Data Readers
23
![Page 24: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/24.jpg)
Three ways to read in data
24
1. Through tf.constant (make everything a constant)
It’ll seriously bloat your graph (you’ll see in assignment 2)
![Page 25: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/25.jpg)
Three ways to read in data
25
1. Through tf.constant (make everything a constant)NO
2. Feed dict
Storage Client Workers
Slow when client and workers are on different machines
![Page 26: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/26.jpg)
Three ways to read in data
26
1. Through tf.constant (make everything a constant)NO
2. Feed dictMAYBE …
3. Data readers
![Page 27: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/27.jpg)
27
Data Readers
Storage Worker
Readers allow us to load data directly into the worker process.
![Page 28: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/28.jpg)
tf.TextLineReaderOutputs the lines of a file delimited by newlinesE.g. text files, CSV files
tf.FixedLengthRecordReaderOutputs the entire file when all files have same fixed lengthsE.g. each MNIST file has 28 x 28 pixels, CIFAR-10 32 x 32 x 3 tf.WholeFileReaderOutputs the entire file content
tf.TFRecordReaderReads samples from TensorFlow’s own binary format (TFRecord)
tf.ReaderBaseTo allow you to create your own readers
28
Different Readers for different file types
![Page 29: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/29.jpg)
filename_queue = tf.train.string_input_producer(["file0.csv", "file1.csv"])
reader = tf.TextLineReader()key, value = reader.read(filename_queue)
29
Read in files from queues
![Page 30: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/30.jpg)
filename_queue = tf.train.string_input_producer(["heart.csv"])
reader = tf.TextLineReader(skip_header_lines=1)key, value = reader.read(filename_queue)
30
Read in files from queues
string_input_producer is really a queue
![Page 31: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/31.jpg)
filename_queue = tf.train.string_input_producer(["heart.csv"])
reader = tf.TextLineReader(skip_header_lines=1)key, value = reader.read(filename_queue)
with tf.Session() as sess: coord = tf.train.Coordinator() threads = tf.train.start_queue_runners(coord=coord) for _ in range(1): # generate 1 example features, labels = sess.run([data_batch, label_batch]) coord.request_stop() coord.join(threads)
31
Read in files from queues
Need Coordinator and QueueRunner
![Page 32: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/32.jpg)
filename_queue = tf.train.string_input_producer(["heart.csv"])
reader = tf.TextLineReader(skip_header_lines=1)key, value = reader.read(filename_queue)
with tf.Session() as sess: coord = tf.train.Coordinator() threads = tf.train.start_queue_runners(coord=coord) for _ in range(1): # generate 1 example key, value = sess.run([key, value]) print valuee # 144,0.01,4.41,28.61,Absent,55,28.87,2.06,63,1 print key # data/heart.csv:2 coord.request_stop() coord.join(threads)
32
Read in files from queues
Value is just text. Need to convert to 2 tensors: + Features tensor+ Label tensor
![Page 33: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/33.jpg)
Live example(05_csv_reader.py)
33
![Page 34: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/34.jpg)
TFRecord
34
TensorFlow’s binary file format
a serialized tf.train.Example protobuf object
![Page 35: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/35.jpg)
Why binary?
35
● make better use of disk cache● faster to move around ● can store data of different types (so you can put both
images and labels in one place)
![Page 36: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/36.jpg)
Convert normal files to TFRecord
36
● Super easy● Live example
![Page 37: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/37.jpg)
Read in TFRecord
37
● Using TFRecordReader, duh● Live example
![Page 38: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/38.jpg)
Assignment 2:Style Transfer
38
![Page 39: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/39.jpg)
Deadpool and Guernica
![Page 40: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/40.jpg)
Deadpool and Guernica
![Page 41: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/41.jpg)
Deadpool and Guernica
![Page 42: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/42.jpg)
![Page 43: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/43.jpg)
![Page 44: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/44.jpg)
Style Transfer
44
Not too much math, but implementation is tricky
![Page 45: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/45.jpg)
Mathy stuff
45
Find a new image:● whose content is closest to the content image and● whose style is closest to the style image
![Page 46: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/46.jpg)
It’s all about the loss functions
46
● Content loss To measure the content loss between the content of the generated image and the content of the content image
● Style lossTo measure the style loss between the style of the generated image and the style of the style image
![Page 47: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/47.jpg)
47
What is the content/style of an image?
![Page 48: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/48.jpg)
Content/style of an image
48
Feature visualization have shown that:
● lower layers extract features related to content● higher layers extract features related to style
![Page 49: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/49.jpg)
Loss functions revisited
49
● Content loss To measure the content loss between the feature map in the content layer of the generated image and the content image
● Style lossTo measure the style loss between the feature maps in the style layers of the generated image and the style image
![Page 50: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/50.jpg)
Loss functions revisited
50
● Content loss To measure the content loss between the feature map in the content layer of the generated image and the content image
Paper: ‘conv4_4’
● Style lossTo measure the style loss between the gram matrices of feature maps in the style layers of the generated image and the style image
Paper: [‘conv1_1’, ‘conv2_1’, ‘conv3_1’, ‘conv4_1’ and ‘conv5_1’]
![Page 51: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/51.jpg)
Loss functions revisited
51
● Content loss To measure the content loss between the feature map in the content layer of the generated image and the content image
Paper: ‘conv4_4’
● Style lossTo measure the style loss between the gram matrices of feature maps in the style layers of the generated image and the style image
Paper: [‘conv1_1’, ‘conv2_1’, ‘conv3_1’, ‘conv4_1’ and ‘conv5_1’]
Give more weight to deeper layersE.g. 1.o for ‘conv1_1’, 2.0 for ‘conv2_1’, ...
![Page 52: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/52.jpg)
Loss functions revisited
52
● Content loss
● Style loss
![Page 53: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/53.jpg)
Optimizer
53
Optimizes the initial image to minimize the combination of the two losses
Do not optimize the weights!
![Page 54: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/54.jpg)
Tricky implementation details
54
1. Train input instead of weights
![Page 55: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/55.jpg)
Tricky implementation details
55
1. Train input instead of weights2. Multiple tensors share the same variable to avoid
assembling identical subgraphsa. Content imageb. Style imagec. Initial image
![Page 56: TensorFlow Input Pipelineassembling identical subgraphs 3. Use pre-trained weights (from VGG-19) a. Weights and biases already loaded for you b. They are numpy, so need to be converted](https://reader033.vdocument.in/reader033/viewer/2022060711/607783e336336f13a100ce74/html5/thumbnails/56.jpg)
Tricky implementation details
56
1. Train input instead of weights2. Multiple tensors share the same variable to avoid
assembling identical subgraphs3. Use pre-trained weights (from VGG-19)
a. Weights and biases already loaded for youb. They are numpy, so need to be converted to
tensorsc. Must not be trainable!!