introduction to tf - texas a&m...
Post on 20-May-2020
3 Views
Preview:
TRANSCRIPT
Introduction toPresented by Xie Yaochen
2017.05.13
A brief introduction to Deep Learning
Get started with a Neural Networks
Advanced Models
Convolutional Neural Networks (CNN)
Advanced Models
Convolutional Neural Networks (CNN)
Convolutional Layers
Advanced Models
Convolutional Neural Networks (CNN)
Pooling Layers
Advanced Models
Recurrent Neural Networks (RNN)
Advanced Models
Recurrent Neural Networks (RNN)
Variants of RNN:
LSTM
GRU
Advanced Models
Methods/Tricks to deal with Overfitting
Regularization
Activation (Relu…)
Dropout
Batch & Batch normalization
….
Advanced Models
Optimizer
Stochastic gradient descent (SGD)
Momentum
Adagrad
Adam
….
What is
TensorFlow™ is an open source software library for numerical computation using data flow graphs open-sourced by Google.
But what does it actually do? TensorFlow provides primitives for defining functions on tensors and
automatically computing their derivatives.
Okay, I know. So what is a Tensor?
Tensor: N-dimensional array
A scalar is a tensor
A vector is a tensor
A matrix is a tensor
e.g. Image represented as 3-d tensor rows, cols, channels(RGB)
3 # a rank 0 tensor; this is a scalar with shape []
[1. ,2., 3.] # a rank 1 tensor; this is a vector with shape [3]
[[1., 2., 3.], [4., 5., 6.]] # a rank 2 tensor; a matrix with shape [2, 3]
[[[1., 2., 3.]], [[7., 8., 9.]]] # a rank 3 tensor with shape [2, 1, 3]
When and Why should we use TensorFlow?
Frame work
Developing Language
Supported API 安装难度 灵活性 上⼿手难度
Caffe C++/CUDA C++/python/ matlab
*** ** **
mxNet C++/CUDA Matlab/JS/ C++/Scala
** ** *
Tensorflow C++/CUDA/ python
C++/python * *** ***
Other: Theano, Torch… (CPU only)
How to use
One command to install TensorFlow
$ pip install tensorflow
( for Linux / Mac OS )
or install by Anaconda
( for Windows)
Two steps to run your TensorFlow
Building the computational graph
Running the computational graph
Re : Zero 从零开始的Tensorflow
(假设⼤大家都会python以及常⽤用库,如numpy,的基本使⽤用⽅方法)
Define a constant:
node1 = tf.constant(3.0, tf.float32)
node2 = tf.constant(4.0) # also tf.float32 implicitly
print(node1, node2)
Output
Tensor("Const:0", shape=(), dtype=float32) Tensor("Const_1:0", shape=(), dtype=float32)
Re : Zero 从零开始的Tensorflow
Get value of a tensor
sess = tf.Session()
print(sess.run([node1, node2]))
Output
[3.0, 4.0]
Re : Zero 从零开始的Tensorflow
Operations
node3 = tf.add(node1, node2)
print("node3: ", node3)
print("sess.run(node3): ",sess.run(node3))
Output
node3: Tensor("Add_2:0", shape=(), dtype=float32)
sess.run(node3): 7.0
Re : Zero 从零开始的Tensorflow
Placeholder
a = tf.placeholder(tf.float32)
b = tf.placeholder(tf.float32)
adder_node = a + b # + provides a shortcut for tf.add(a, b)
print(sess.run(adder_node, {a: 3, b:4.5}))
print(sess.run(adder_node, {a: [1,3], b: [2, 4]}))
Output
7.5
[ 3. 7.]
Re : Zero 从零开始的Tensorflow
Variable
W = tf.Variable([.3], tf.float32)
b = tf.Variable([-.3], tf.float32)
x = tf.placeholder(tf.float32)
linear_model = W * x + b
Initialize and run session
init = tf.global_variables_initializer()
sess.run(init)
Re : Zero 从零开始的Tensorflow
Variable
print(sess.run(linear_model, {x:[1,2,3,4]}))
Output
[ 0. 0.30000001 0.60000002 0.90000004]
Re : Zero 从零开始的Tensorflow
To evaluate the model
y = tf.placeholder(tf.float32)
squared_deltas = tf.square(linear_model - y)
loss = tf.reduce_sum(squared_deltas)
print(sess.run(loss, {x:[1,2,3,4], y:[0,-1,-2,-3]}))
Output
23.66
Re : Zero 从零开始的Tensorflow
Optimizing
optimizer = tf.train.GradientDescentOptimizer(0.01)
train = optimizer.minimize(loss)
sess.run(init) # reset values to incorrect defaults.
for i in range(1000):
sess.run(train, {x:[1,2,3,4], y:[0,-1,-2,-3]})
print(sess.run([W, b]))
MNIST - To Say “Hello World!”
MNIST is a simple computer vision dataset. It consists of images of handwritten digits like these:
Each image is 28 pixels by 28 pixels. We can interpret this as a big array of numbers:
MNIST - To Say “Hello World!”
Step 1: Pretreat the images and labels (total size = 55000)
Flatten this array into a vector of 28x28 = 784 numbers
Convert the labels into one-hot vectors
(For example, 3 would be [0,0,0,1,0,0,0,0,0,0] )
MNIST - To Say “Hello World!”
Step 2: Softmax Regressions
MNIST - To Say “Hello World!”
Step 3: To train and evaluate the model
Loss : Cross-entropy
Where y is our predicted probability distribution, and y′ is the true distribution (the one-hot vector with the digit labels)
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)
MNIST - A Further Example (CNN)
Convolution and Pooling (see example3.py)
References:
1. The MNIST Database: http://yann.lecun.com/exdb/mnist/
2. TensorFlow Official Document: https://www.tensorflow.org/
3. Colah’s Blog: http://colah.github.io/
4. Stanford Course CS224d: https://cs224d.stanford.edu/lectures/
5. Prof. Jordi Torres’ Home Page: http://www.jorditorres.org/
6. Fabien Baradel’s Blog: https://fabienbaradel.github.io/
Also where you could learn more about Deep Learning and TensorFlow
top related