tensorflow vs mxnet
TRANSCRIPT
DEEP LEARNING IN THE CLOUD: TENSORFLOW VS MXNET
ASHISH BANSAL, SNR DIR DATA SCIENCE @ CAPITAL ONE
@ASH_BANS | LINKEDIN.COM/IN/BANSALASHISH
AGENDA FOR TODAY
• What is deep learning
• Pre-requisites for deep learning
• Libraries landscape
• TensorFlow vs MxNet
• Programming model
• Popularity, developer support, cloud, deployment
• Summary
Deep Learning in the cloud Ashish Bansal 1
WHAT IS DEEP LEARNING?• Variant of neural networks
• Inspired by human brain, but only very loosely mimics it (very very simple compared to the brain)
• Saves a ton of data scientist time in feature engineering–spend more time in tuning
• GPUs make it very efficient. Thank you Gamers!
• Requires large amounts of data
• Given enough data, will generally outperform other algorithms
• All popular references to AI that you read refer to deep learning advances
Deep Learning in the cloud Ashish Bansal 2
WHAT MAKES DEEP LEARNING DEEP: FEATURE HIERARCHY OF TRAINED REPRESENTATIONS
Deep Learning in the cloud Ashish Bansal 3
• 2 layer models are not deep because there is no feature
hierarchy
• Neural networks with 1 hidden layer are not considered
deep for the same reason
• Classification trees are not deep as all decisions are
made in the input space
Source: Unreasonable Effectiveness of Deep Learning by Yann LeCun (http://www4.ncsu.edu/~pcombet/data2015/lecun2015.pdf)
PRE-REQUISITES FOR DEEP LEARNING
• Data, and lots of it
• A way to express or code neural networks
• A fast processor adept at matrix computations
• High throughput between storage and compute for several passes during
training
Note: Training, by far, is the most time taking activity. Once trained, networks can classify fairly quickly
Deep Learning in the cloud Ashish Bansal 4
PRE-REQUISITES FOR DEEP LEARNING
• Data, and lots of it: Is it on the cloud? Key challenge here is getting data to the cloud
• A way to express or code neural networks: Can be solved with TensorFlow or MxNet
• A fast processor adept at matrix computations: GPU based cloud units like AWS P2
instances
• High throughput between storage and compute for several passes during training:
Large memory on the cloud instances, like p2.8xlarge, and parallelization
Deep Learning in the cloud Ashish Bansal 5
LIBRARIES LANDSCAPE
Deep Learning in the cloud Ashish Bansal 6
TENSORFLOW & MXNET OVERVIEW
Deep Learning in the cloud Ashish Bansal Source: https://svds.com/getting-started-deep-learning/ 7
TENSORFLOW VS MXNET: GITHUB VIEW
Deep Learning in the cloud Ashish Bansal Source: Github. Data as of Apr 18, 2017
8
0
10000
20000
30000
40000
50000
60000
Stars Forks Commits Contributors
Deep Learning Libraries: Github Metrics
TensorFlow Keras MxNet Qix Torch Caffe Theano DL4J CNTK DSSTNE
TENSORFLOW VS MXNET: STACK OVERFLOW VIEW
Deep Learning in the cloud Ashish Bansal Source: Stack Overflow, Data as of Apr 18, 2017 9
0
5000
10000
15000
20000
25000
TensorFlow Keras MxNet Qix Torch Caffe Theano DL4J CNTK DSSTNE
Stack Oveflow Activity
TENSORFLOW VS MXNET: PERFORMANCE VIEW
Deep Learning in the cloud Ashish Bansal Source: https://arxiv.org/pdf/1608.07249.pdf 10
AlexNet Performance (5 layers, 81K params) ResNet-56 Performance (56 layers, 850K params)
TENSORFLOW VS MXNET: CLOUD SUPPORT VIEW
• TensorFlow is fully supported on AWS EC2, Google Cloud & Microsoft Azure
• MxNet is fully supported on AWS EC2 (preferred) and Microsoft Azure
• Your mileage may vary for MxNet on Google Cloud
Deep Learning in the cloud Ashish Bansal 11
TENSORFLOW VS MXNET: DEPLOYMENT VIEW
TensorFlow MxNet
Android Yes OOB Yes – needs additional libs & compilation
iOS Yes OOB Yes – needs additional libs & compilation
Raspberry Pi Yes OOB Yes OOB
Windows YMMV Yes
Unix/Linux Yes OOB Yes OOB
Deep Learning in the cloud Ashish Bansal 12
SUMMARY
• Lots of momentum and support behind TensorFlow
• TensorFlow has better RNN capabilities
• TensorFlow has better tutorials and online guides. It also has more
supporting material like Stack Overflow questions etc.
• TensorFlow enjoys greater support on the cloud, and has more
deployment options
• MxNet has more language bindings, and is usually faster
Deep Learning in the cloud Ashish Bansal 13
QUESTIONS@ASH_BANS | LINKEDIN.COM/IN/BANSALASHISH
Deep Learning in the cloud Ashish Bansal 14