Introducing IBM PowerAI VisionAccelerate AI Vision deployments and increase user productivity

Srini Chitiveli (svchitiv@us.ibm.com)Product Manager

Teaching a computer to recognize a bicycle

ArtificialIntelligence

Mimic Humans

MachineLearning

Learn withExperience

Deep Learning(Neural Networks)

Self-Learn with More Data

Deep Learning Has Revolutionized Machine Learning

4

0

20

40

60

80

100

Source: Google Trends. Search term “Deep Learning”

# of Searches for Deep Learning from 2011 to 2017

TraditionalMachine Learning

Deep Learning

Accuracy

Data

5

Machine Learning

Deep Learning

Input

Deep Neural Network

OutputFeature Extraction & Classification

Input Feature Extraction

Features Classification Output

Machine Learning Algorithms

66

26% Errors

Machine Learning Based

3% Errors

Deep Learning Based

20162011

Humans

5% Error

DEPLOY & INFER

MAINTAIN

ACCURACY

Data Changes, Constant Iteration Required

Model Tuning/ Pruning, Scale &

Performance, Resiliency, Application

Access

DATA

PREPARATION

Complexity /

Technology Rapidly

Changing

Volume, Multi-

SourceLabeling &

Tagging,Ingestion

BUILD, TRAIN,

OPTIMIZE

Hyper-parameter Complexity,

Massive Compute Intensive

Iterations, Long Training Times,

Limited Resources

UP & RUNNING

Pain Points – Deep Learning Pipeline

Sharing Valuable Resources Across Multiple Users, Multiple Lines of Business, Multiple ApplicationsWith Security, Resiliency, and at Scale

IBM PowerAI Vision: ”Point-and-Click” AI for Images & Video

8

Label Image orVideo Data

Auto-Train AI Model

Package & Deploy AI Model

Core Capabilities

9

• RHEL and Ubuntu• Classification of Images• Object Detection• Auto labeling of images and Videos• Prebuilt models for classification• RESTFul interface to integrate into solutions • Bring Your Own Model - Import Custom Models to train and host for inference• Data Augmentation

• Advance metrics on trained models

• Multiple classification of results• Inference on remote x86 and Power remote servers• Compress and accelerate models for FPGA cards on Embedded devices• Optimize and deploy models on FPGAs in Data center (Alveo U200)

• IBM Intelligent Video Analytics for end user experience

Platform support for Training & Inference10

Platform Operating System CUDA

Power8 • RHEL 7.5• Ubuntu 18.04

CUDA 10

Power9 • RHEL 7.5• Ubuntu 18.04

CUDA 10

• Introduced support for Ubuntu 18.04

• Customers can now use this operating system to train and deploy models with PowerAI Vision

Image Classification

1. Get your images2. Create categories3. Assign images to

individual categories4. Train5. Deploy & Infer

LABEL

INFER

TRAINING DATASET

DEMO LINK

Object detection on static images

1. Get your images2. Draw boundaries

around objects of interest

3. Train4. Deploy & Infer

DEMO LINK

LABEL

INFER

TRAINING DATASET

Image Segmentation

• Ability to label objects with multiple point polygons.

• Segmentation helps compute relative size and position of objects compared to rest of the image

I need to count inventory on the shelves.

-- Retail supervisor

LABEL

INFER

Demo Link

Semi-Automatic Labeling using PowerAI Vision

14

Train DL Model

Define LabelsManually Label Some

Images / Video Frames

Manually LabelUse Trained DL

Model

Run Trained DL Model on Entire Input Data to Generate Labels

Correct Labels on Some Data

Manually Correct Labels on Some Data

Repeat Till Labels Achieve Desired Accuracy

Auto labeling on Videos & Images

1. Identify small video2. Manually label

dataset3. Train model4. Use model to auto

label video5. Audit dataset for

accuracy6. Re-train model for

higher accuracies

VIDEO

AUTO LABEL

APPLICATION COUNTING CARS

DEMO LINK

Installed base models▪ Prebuilt base-models for known objects around us

▪ Prebuilt base-models transfer learn faster on the defined topics

▪ Import custom base-models for transfer learning

Rich set of RESTFul APIs

APIs to programmatically:

• Create datasets for Classification and Object detection

• Export and Import datasets

• Trigger and monitor training process

• Deploy models for inference

• Build solutions by detecting objects in segmented areas

Augment limited datasets for higher accuracywork with limited data

• Generate variety of images for initial datasets

• Software can apply filters to augment data and increase images for training

• Augmented data reduces overfitting for small datasets and increases accuracy

Limited dataset Inbuilt augmentation algorithms Augmented datasets

Advanced metrics on model training19

• Introducing metrics to show accuracy of trained model

• Total Recall• Total Precision• PR Curve• Confusion

Matrix• Hyper Params• Loss Vs

Iteration• Total Accuracy

• Insights for Data scientists to identify issues with dataset

Multiple classification of Images

Group Name / DOC ID / Month XX, 2018 / © 2018 IBM Corporation

20

• Inference now result multiple categories for an image sorted by confidence

• Useful for users to analyze issues with datasets and balance out images per category to increase accuracies

• REST APIs for inference now accept a threshold for results

Bring your Own Model for Training and Inference21

• Data scientists can now out-source training and deployment jobs on their custom models

• Data scientists can focus on innovating models for futures

• Limited to TensorFlow only

22

Train on central server but deploy on several remote

servers

1 - N

Datacenter

PAI Vision for Training and

Inference

AC922

.

Trained models

1. Manually export trained models from Central server to remote locations

2. Import and deploy models with Inference-only license of PowerAI Vision

3. Once the models are deployed, each plant can work stand alone for inference.

4. Supports Power servers with minimum of one GPU

Provided in MVP

New feature (Inference-only)

Models trained on PAI Vision; Password protected

Remote Plant

PAI Vision for Inference

only

AC922/LC922/x86

Trained models

Remote Plant

Remote Plant

Remote Plant

PAI Vision for Inference

only

AC922/LC922/x86

Trained models

Inference on Remote servers

Group Name / DOC ID / Month XX, 2018 / © 2018 IBM Corporation

23

• Train on an AC944 with 4 GPUs

• Deploy models on remote servers with • no accelerator (aka CPU

only)• With GPU• With FPGA (Power only)

• Ability to use existing x86 servers for inference

• Run inference closer to the source of data (at the edge)

Version 1.1.1

Version 1.1.2

Deep Learning Models

COD:FRCNN

CIC: google-net

Base: CAFFE

COD: FRCNN, Tiny YOLOand Custom model

CIC: google-net and Custom model

Base: CAFFE and Tensorflow

Processors POWER POWER and x86

Acceleratoroptions

GPU GPU/CPU only/FPGA (Power Only)

OS RHEL 7.5 RHEL 7.5 and Ubuntu18.04

Train, Optimize and Infer on FPGA (Embedded and data center)

CPU +

AcceleratorNeural network processor

Embedded GPU

Embedded FPGA

FPGAs, CPUs, GPUs

Trained

DNN model

Model Parser & Compiler

Model Optimization(Layer Merge + Quantization)

Output model + weights

PowerAI VisionMap to Different

Platforms

Data Center: Train model & Compile to EdgeCloud or Edge

24

Xilink Alveo U200

Accelerateusingappropriatetechnology

• ML-Suite• TensorRT• PIE

Sim

ilar R

esu

lts

Why FPGA ?25

• Built only for inference, NOT for training

• Inexpensive• Embed into Servers• Embed into devices

like cameras• Use low power• Perform better or at

par with GPUs• Fast adoption in the

market place

* Source: Accelerating DNNs with Xilinx

Alveo Accelerator Cards White Paper

Introducing acceleration of models for FPGA

• Ability to compress optimized models for a reference FPGA card (Xilink ZC706 Evaluation kit)

• Provide support (L1, L2, L3) only for generating the model

• Instructions to compress models for other Xilink chips

• Building parts specialized in FPGA cards

• V3Red technologies

• KRTKL ( snickerdoodle board)

• PointRData Systems

Xilinx Zynq-7000 SoC ZC706 Evaluation Kit

Datacenter

PAI Vision for Training and

Inference

AC922

Trained models

Train to deploy on server with FPGA cards (Alveo U200)

Group Name / DOC ID / Month XX, 2018 / © 2018 IBM Corporation27

TRAIN

DEPLOY

• Model quantization now part of training

• Modeling optimization for FPGA is now a clickable job

• Support limited to Xilink Alveo U200 on Power system only

• Need server with FPGA in ICP cluster

Technology Preview

IBM Intelligent Video Analytics

Detect Changes to Patterns

Facial Recognition & People Search

Redaction of Faces

Video Analytics Software with Pre-Trained AI Models

Complex Event Monitoring with GUI-based Configuration

Targeted at Public Safety, Remote Monitoring, etc

28

demo

Customize intelligence video analytics

29

New AI Model

PowerAI Vision

Label

Train

Deploy

IBM IVA

GUI

AI Models

Event Detection

Live Video

New Video Training Data

GPU-Accelerated Power Servers for

Model Training

GPU-Accelerated Power Servers for

Inference

demo

30

Customizing Video Analytics with PowerAI Vision

DEMO PAIV

PowerAI VisionIntelligent Video

Analytics

Select library of training images/videos

Auto-label / classify objects of interest

Train Deploy & Infer

Run & Manage

Re-train

Power Play

DEMO INTEGRATION

Hardware Advantage: 5x Faster Data Communication with Unique

CPU-GPU NVLink High-Speed Connection

1 TB

Memory

Power 9

CPU

V100

GPU

V100

GPU

170GB/s

NVLink150 GB/s

1 TB

Memory

Power 9

CPU

V100

GPU

V100

GPU

170GB/s

NVLink150 GB/s

IBM AC922 Power SystemDeep Learning Server (4-GPU Config)

Store Large Models in System Memory

Operate on One Layer at a Time

Fast Transfer via NVLink

31

Inference API deployment for

Cloud

Video and Image Labeling

Data Preprocessing

Self-defined Training

with visualized monitoring

Custom Learning

Complete solution built from Ground Up

Data Lake & Data Stores

Distributed Computing

ML & DL Libraries & Frameworks

Accelerated Servers Storage

TrainingData

Inference accelerator

generation for edge

Inference

AI DEVELOPMENT WORKFLOW

Testing & Measurement

33

Track how customers navigate store, identify

fraudulent actions, detect low inventory for items

Generates track summary, flags missing objects, alerts on suspicious

behavior

Utilize surveillance cameras to ensure worker

safety compliance

Enable zone monitoring, heat maps, detection of

loitering

Identify faulty or worn out equipment in remote & hard to reach locations

Alert to schedule maintenance job, along

with critical infrastructure security

34

Identify, track, and apprehend suspects to improve public safety.

Facial recognition, Object missing / left behind,

Track Summary

Remotely assess potential threats and current status

of critical assets.

object recognition, asset condition analysis, theft

and vandalize, critical infrastructure security

Public and private monitoring of perimeters

and restricted areas.

Zone violation, Intrusion detection, Heat map,

Loitering

Possibilties

35

SMARTER CITIES

DRONE SURVEILLANCE SPORTS AND ENTERTAINMENT

Possibilties

36

RETAIL FRAUD EDGE INFERENCE

37

Value proposition

➢AI Made Simpleo Clicker tools to train models with no coding or expertise in technologies

➢End to End ecosystem o Disjoint activities streamlined into simple sequential taskso Life cycle management for models and datao Train on server but deploy on cloud or edge for inference

➢Enterprise grade offeringo Collaborative platform between several personaso Open architecture extensible with existing enterprise assetso Backed with support and services from IBM and business partners

Getting started with PowerAI Vision

Offering detailsIntroduction to IBM PowerAI Vision

IBM MarketPlace

IBM Developer Works

Access to SoftwareDownload Trial version of PowerAI Vision

Access Technology preview as a service

38

Developer JourneysTrain models for Advanced Driving Assistant Systems

Train models to detect flavors of Coke

Classify Photo resist wafers

Count cars and objects

VideosTrain models for Classification and Object detection

Train models for Advanced Driving Assistant Systems

Continuous learning for data labeling

Demos on YouTube

Srini Chitiveli (svchitiv@us.ibm.com)Product Manager

THANK YOU

Introducing IBM PowerAI VisionAccelerate AI Vision deployments and increase user productivity