defect inspection from scratch to productionon-demand.gputechconf.com › gtc › 2018 ›...
TRANSCRIPT
![Page 1: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/1.jpg)
Andrew Liu, Ryan Shen
Deep Learning Solution Architect
DEFECT INSPECTION FROM SCRATCH TO PRODUCTION
![Page 2: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/2.jpg)
2
AGENDA
Defect Inspection and its challenges
NGC Docker images
Model set up - Unet
Data preparation - DAGM
Production - Speed up with TensorRT
![Page 3: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/3.jpg)
3
INDUSTRIAL DEFECT INSPECTION
![Page 4: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/4.jpg)
4
INDUSTRIAL DEFECT INSPECTION
![Page 5: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/5.jpg)
5
NGC DOCKER IMAGES
![Page 6: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/6.jpg)
Benefits for Deep Learning WorkflowHigh Level Benefits and Feature Set
Single software stack
Develop once, deploy anywhere
Scale across teams of practitioners
Developer, DevOp, QC
![Page 7: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/7.jpg)
7
Defect inspection Workflowfrom scratch to production within container
Training Inference
NGC optimized docker image TENSORRT GRE
docker pull
nvcr.io/nvidia/tensorflow
:18.02-py3
docker pull
nvcr.io/nvidia/tensorrt:
18.02-py2
![Page 8: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/8.jpg)
8
MODEL SET UP
![Page 9: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/9.jpg)
9
WAYS FOR DEFECT INSPECTION
Depends on domain adaptation
Image SegmentationObject Detection
Image Classification
Label: 0 / 1 (Defect / Non Defect)
Label: Bounding-Box Label: Polygons Mask
![Page 10: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/10.jpg)
10
CNN STRUCTURELeNet
Source: Design of Deep Convolutional Neural Network Architectures for Automated Feature Extraction in Industrial Inspection, D. Weimer et al, 2016
![Page 11: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/11.jpg)
11
“defect
class1”
1000-dim vector
< 1 millisecond
convnets perform classification
end-to-end learning
https://arxiv.org/abs/1605.06211
![Page 12: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/12.jpg)
12
~1/10 second
end-to-end learning
???
lots of pixels, little time?
https://arxiv.org/abs/1605.06211
![Page 13: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/13.jpg)
51
22
51
22
51
22
1 16 16
16 32 32
32 64 64
64 128 128 256
128 256 256
25
62
25
62
25
62
12
82
12
82
12
82
64
2
64
2
64
2
32
2
32
2
32
2
12
82
12
82
12
82
128 64 64
128 128
64
2
64
2
64
2
64 32 32
25
62
25
62
25
62
51
22
51
22
51
22
32 16 16
3X3 Conv2d+ReLU
2X2 MaxPool
2X2 Conv2dTranspose
copy and concatenate
U-Net structure
![Page 14: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/14.jpg)
14
KERAS IMPLEMENTATIONConvolution
![Page 15: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/15.jpg)
15
DECODINGdeconvolution
![Page 16: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/16.jpg)
16
Image segmentation on medical imagesSame process among various use cases
MRI image
Left ventricle
heart disease
Data Science BOWL2016
Data Science BOWL 2017
CT image
Nodule
Lung cancer
Image
Nuclei
Drug discovery
Data Science BOWL2018
![Page 17: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/17.jpg)
17
DATA PREPARATION
![Page 18: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/18.jpg)
18
INDUSTRIAL OPTICAL INSPECTIONGerman Association for Pattern Recognition
• http://resources.mpi-inf.mpg.de/conferences/dagm/2007/prizes.html
![Page 19: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/19.jpg)
19
INDUSTRIAL OPTICAL INSPECTIONGerman Association for Pattern Recognition
Pass NG Pass
NGNGPass Pass
NG
![Page 20: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/20.jpg)
20
DATA DETAILS
• Original images are 512 x 512 grayscale format
• Output is a tensor of size 512 x 512 x 1
• Each pixel belongs to one of two classes
• Training set consist of 100 images
• Validation set consist of 50 images
![Page 21: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/21.jpg)
21
MORE EXAMPLES
![Page 22: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/22.jpg)
22
IMBALANCE DATA
![Page 23: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/23.jpg)
23
Dice Metric (IOU)• Metric to compare the similarity of two samples:
2𝐴𝑛𝑙________________________________
𝐴𝑛 + 𝐴𝑙
• Where: • An is the area of the contour predicted by the network • Al is the area of the contour from the label• Anl is the intersection of the two
• The area of the contour that is predicted correctly by the network
• 1.0 means perfect score.
• More accurately compute how well we’re predicting the contour against the label
• We can just count pixels to give us the respective areas
![Page 24: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/24.jpg)
24
LOSS FUNCTION WITH KERAS
24
![Page 25: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/25.jpg)
25
LEARNING CURVES
25
LOSS
![Page 26: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/26.jpg)
26
APPLICATION: INDUSTRIAL INSPECTIONNVIDIA
• 1000 defect-free, 150 defect images
• Challenges: Not all deviations from the
texture are necessarily defects.
![Page 27: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/27.jpg)
27
PRODUCTION
![Page 28: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/28.jpg)
28
FINAL DECISIONPlus Human Logic
Size, Position, … etc
![Page 29: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/29.jpg)
29
INFERENCE PIPELINE
Post-processing and decision makingInference
Camera
Machine
P40
DGX Server
TensorRT+GRE
Fetch image
TensorRT+GRE
Detectors/Classifiers/Segment Composite Result Metadata
DataCenter
Edge
Defect Pattern Ratio Defect Level Defect region size Defect counts
…
Domain Criteria Determine threshold
Precision/ Recall
![Page 30: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/30.jpg)
30
Development flow
Called UFF, Universal Framework FormatONNX
![Page 31: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/31.jpg)
31
Defect inspection Workflow
from scratch to production within container
Training Inference
NGC optimized docker image TENSORRT GRE
docker pull
nvcr.io/nvidia/tensorflow
:18.02-py3
docker pull
nvcr.io/nvidia/tensorrt:
18.02-py3
![Page 32: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/32.jpg)
32
SUMMARY
Challenges Delivers
Training , inference environment is hard to build, maintain, share. Using NGC Docker images.
Model optimizations and speed up throughput. TensorRT + GRE SDK
So many deep learning model out there, how to choose the right
model?
If your dataset, demand requirement fit the scenario
like we do. U-Net model is great choice for
segmentation task.
![Page 33: DEFECT INSPECTION FROM SCRATCH TO PRODUCTIONon-demand.gputechconf.com › gtc › 2018 › presentation › s...DEFECT INSPECTION FROM SCRATCH TO PRODUCTION. 2 AGENDA Defect Inspection](https://reader035.vdocument.in/reader035/viewer/2022070817/5f139d9e8594495abc07ae0c/html5/thumbnails/33.jpg)
33
Thank You