Download - Affordable AI Connects To A Better Life
![Page 1: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/1.jpg)
Affordable AIConnects To A Better LifeBofu Chen, Sep 21, 2016
![Page 2: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/2.jpg)
Intelligent GatewayAffordable AI TechniquesImplementationExample: Pepper RobotExample: Campus Security System
AGENDA
![Page 3: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/3.jpg)
Intelligent Gateway
![Page 4: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/4.jpg)
Photo: Robert Bond
![Page 5: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/5.jpg)
Photo: Robert Bond
![Page 6: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/6.jpg)
Cat Recognition
Photo: Robert Bond
![Page 7: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/7.jpg)
Photo: Robert Bond
![Page 8: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/8.jpg)
Deep Learning Inference
Cat!
No Backpropagation
![Page 9: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/9.jpg)
Inference Essentials
MBComputing Time Memory Usage
Shorten the prediction time is always welcome
Device memory is limited, but deep learning model can
be huge
![Page 10: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/10.jpg)
Techniques To Make AI Affordable
![Page 11: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/11.jpg)
Inference Researches
Weight Storage Hardware Usage
Reduce weight storage size without sacrificing accuracy
Utilize computing components (CPU, GPU, etc.) as many as possible
simultaneously
Binarized Neural Networks, http://arxiv.org/abs/1602.02830 | XNOR-Net, http://arxiv.org/abs/1603.05279 | DoReFa-Net, https://arxiv.org/abs/1606.06160 | DeepX, http://niclane.org/pubs/deepx_ipsn.pdf
![Page 12: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/12.jpg)
Approaches
CompressionNvidia TensorRT Optimization
ThroughputPower efficiency
Memory usageKeep accuracy
Speed up
Low-level speed up
![Page 13: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/13.jpg)
Nvidia TensorRTLike a model compiler
Production Deep Learning with NVIDIA GPU Inference Engine, https://devblogs.nvidia.com/parallelforall/production-deep-learning-nvidia-gpu-inference-engine/
![Page 14: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/14.jpg)
Pruning
Learning both Weights and Connections for Efficient Neural Networks, https://arxiv.org/abs/1506.02626
![Page 15: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/15.jpg)
Quantization
How to Quantize Neural Networks with TensorFlow, https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/how_tos/quantization/index.md
DNN is noise tolerable
FP16 to INT8
Hardware speedup
FPU to ALU
![Page 16: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/16.jpg)
Inference Without DL Frameworks
Likely A compiler intermediate representation for image recognition and heterogeneous computing, http://liblikely.org/
![Page 17: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/17.jpg)
Implementation
![Page 18: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/18.jpg)
Deep Learning Computer Vision
NVIDIA TX1Pre-trainedOn Server
![Page 19: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/19.jpg)
End Devices (Sender)
Architecture
End Devices (Receiver)
Intelligent Gateway
NVIDIA TX1
Ubuntu
Tensorflow
REST
TensorRT
gRPC
![Page 20: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/20.jpg)
Inference Choices
TensorRTFast Object
Slow Motion
![Page 21: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/21.jpg)
TensorFlow on TX1
DONEModel Server
Maximize Performance
NEXTInference Optimization
on Ubuntu
![Page 22: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/22.jpg)
Other Attempts
Raspberry Pi 3 Qualcomm Snapdragon 801
0.9s/img
GoogLeNet
Real-TimeInception v3
![Page 23: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/23.jpg)
Pepper, The Emotional Robot
![Page 24: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/24.jpg)
![Page 25: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/25.jpg)
HW Specification
4-core1.9 GHz 4 GB 790 MHz
Pepper motherboard specification, http://doc.aldebaran.com/2-4/family/pepper_technical/motherboard_pep.html
![Page 26: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/26.jpg)
Vision and Speech Limitations
Instead offace identification Keywords instead of NLP
FaceRecognition SpeechRecognition
![Page 27: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/27.jpg)
Cloud Solution Drawbacks
CostConnectivity Privacy
Need to ensure bandwidth, stability and latency are
good enough
Huge amount ofimage transmission
You might want to keep family information locally
![Page 28: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/28.jpg)
Architecture
Pepper Gateway
NVIDIA TX1
Ubuntu
Tensorflow
REST
TensorRT
gRPC
![Page 29: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/29.jpg)
![Page 30: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/30.jpg)
Real World Gesture Recognition Algorithm
![Page 31: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/31.jpg)
![Page 32: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/32.jpg)
![Page 33: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/33.jpg)
![Page 34: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/34.jpg)
![Page 35: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/35.jpg)
![Page 36: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/36.jpg)
![Page 37: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/37.jpg)
![Page 38: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/38.jpg)
![Page 39: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/39.jpg)
![Page 40: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/40.jpg)
![Page 41: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/41.jpg)
![Page 42: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/42.jpg)
![Page 43: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/43.jpg)
![Page 44: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/44.jpg)
Campus Security System
![Page 45: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/45.jpg)
Current Solution
CloudEnd Device
![Page 46: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/46.jpg)
Current Solution
Cloud
NOTINTELLIGENT
![Page 47: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/47.jpg)
Current Solution
Cloud
NOTINTELLIGENT
NOTREAL-TIME
![Page 48: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/48.jpg)
Architecture
Security Gateway
NVIDIA TX1
Ubuntu
Tensorflow
REST
TensorRT
gRPC
![Page 49: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/49.jpg)
StudentStudent
SuspectsStudent StudentStudent StudentStudent
DT42
![Page 50: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/50.jpg)
![Page 51: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/51.jpg)
![Page 52: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/52.jpg)
![Page 53: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/53.jpg)
![Page 54: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/54.jpg)
![Page 55: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/55.jpg)
Violent Event
![Page 56: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/56.jpg)
Kinect v2
UpdateUSB Firmware
Open Source Libraries
Fix data transmission issue libfreenect2 and pylibfreenect2 make enablement easier
MS Kinect v2 on Nvidia Jetson TX1, http://jetsonhacks.com/2016/07/11/ms-kinect-v2-nvidia-jetson-tx1/
![Page 57: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/57.jpg)
We Are DT42
![Page 58: Affordable AI Connects To A Better Life](https://reader034.vdocument.in/reader034/viewer/2022042723/587319291a28ab673e8b5d07/html5/thumbnails/58.jpg)