Small is the New Big: Data Analytics on the EdgeAn overview of processors and algorithms for deep learning techniques on the edge

Dr. Abhay Samant

VP Engineering, Hiller Measurements

Adjunct Faculty, University of Texas

Agenda

• Edge Computing

• Applications: Self-Driving Cars, Security in IMD, RF Machine Learning

• Processor Landscape: GPUs, FPGA

• Algorithms

Edge Computing

• Cloud Computing is going through a fundamental shift

• Centralized vs De-centralized architecture

• Edge Computing brings core building blocks• Compute

• Storage

• Network

Cloud Edge

• Extension of public cloud• CDNs are example of this

topology

• Cloud Edge HW is maintained by cloud provider

• Think of it as an extension to the public code

• Think from a business angle

Cloud Edge

5G for IoT Applications

Courtesy: NI

Device Edge

• Device Edge:• Specialized device acting as node

gateway that mimics public cloud capabilities

• Customers own the hardware that runs the edge software stack

• AWS Green Grass and Microsoft Azure

• Bring device registry, device twins, communication, local storage and sync capability

Device Edge

Moore’s Law & Commercial Technology Impact

ADCs / DACs CPUs / FPGAs RF Components

Courtesy of ADMS Design AB Courtesy of Steve Cherry, IEEE Spectrum, July 2004Courtesy of Steven Pemberton

Courtesy: NI

Machine Learning Application Landscape

The Connected Car

V2V V2V

Sensor Fusion

ULTRASONIC

CAMERA

LIDAR

SHORT RANGE RADARLONG RANGE RADAR

ADAS Architectures Continue to Evolve

Sensor

Electronic Control Module (ECM)

Source: electronics-eetimes

SMART SENSORS/DECENTRALIZED PROCESSING RAW SENSOR DATA/CENTRALIZED PROCESSING

HYBRID SENSOR/PROCESSING

ADAS Sensor Fusion Example

RACAM(RADAR + CAMERA)

INTELLIGENT FORWARDVIEW CAMERA (IFV-100)

COLLISIONMITIGATION SYSTEM (CMS)

Deep Learning For Self-Driving Cars

• Environmental perception is key to autonomous driving, e.g. lane position

• Traditional feature recognition and image processing techniques don’t scale to needed complexity

• Deep neural networks learn efficient feature representation

• Inductive learning leads to evolving software operation that is challenging to test

DEEP NEURAL NETWORK

Machine Learning in RF Systems

• Some unique characteristics of RF ML• Data Rate is much higher

• RF signals are represented as complex numbers

• MIMO Systems

• Mixed signals (bits, complex-valued, RF)

• Protocol-based signals

Feature Learning

• Existing expertise used to best describe RF signals pertinent to a specific RF task.

• Deep Learning has achieved excellent performance in vision and speech applications by learning features similar to those learned by the brain from sensory data.

• Can machine learning of RF features help with many of the spectrum challenges?

Attention and Saliency

• Next-generation RF systems moving from MHz to GHz of spectrum.

• Requires focus on the right signals, ability to ignore others. Humans are exquisite at consuming, prioritizing, and processing visual and auditory information.

• Top-down attention is a goal-driven mechanism, which causes us to focus our cognitive processing on visual information most pertinent to a task at hand.

• Can be myopic, attention is complemented by a bottom-up (e.g. data-driven) mechanism called saliency

• For understanding an RF scene, stored RF concepts such as signals and transmitter types can be used to identify RF objects and model behavior.

Autonomous RF Sensor Configuration and Waveform Synthesis

• Sensor Configuration• Ability to adapt RF front end configuration

• Analog front end• Beam steering patterns• Bandwidth, frequency, power

• Mimics visual processing in human beings• Increases application such as self-driving cars

• Waveform Synthesis• Present systems allow to pick between two defined waveforms• ML techniques allow RF systems to synthesis a new waveform• How to share key parameters with receivers?

Security in Implantable Medical Devices

Data Storage SecurityNetwork and

Transmission SecurityApplication Layer

Security

Availability Efficiency

QualityReliability

Robustness Access

AuthorizationAuthentication

Machine Learning Compute Platforms

nVIDIA Jetson TX2

• Integrated SoC• 256-core NVIDIA Pascal GPU• Hex-core ARMv8 64-bit CPU complex• 8GB of LPDDR4 memory with a 128-bit

interface.

• The CPU complex combines a dual-core NVIDIA Denver 2 alongside a quad-core ARM Cortex-A57.

• Fits a small Size Weight, and Power (SWaP) footprint• 50 x 87 mm• 85 grams• 7.5 watts of typical energy usage.

• Jetson TX1 available for lower resources The Jetson TX2 module

nVIDIA Platform Architecture

16nm nVIDIATegra Parker

cuDNN and TensorFlow RT

libraries

Recurrent Neural NetLong Short Term

MemoryOnline reinforcement

Multimedia streaming network

2 Pascal Streaming multi-core processors

128 CUDA Cores

FPGA as viable platform for ML

• Currently, GPUs are considered good for ML algorithms such as DNN• Regular parallelism

• Optimized for TFLOPS

• New advances in FPGA and ML algorithms could change this trend

• Intel 14nm Stratix10 FPGA is one example

• Increased floating point DSPs

• On-chip RAMs• Improved

Frequenices• High BW Memories

• Exploiting sparsity in datasets

• Lower bit resolution

HW Trends

Algorithm Trends

Understanding ML Algorithms

Rosenblatt, Physiological Review, 1958, posed three questions1. How is information about physical world sensed or detected by biological system?2. In what form is information stored or remembered?3. How does information influence recognition and behavior?

Understanding ML Algorithms

• Basic perceptron operations used

• Across multiple ensembles and layers

• Same input applied for all weights

• Activation Function

• Bias setting at each level sets initial conditions

• Convolutional Neural Networks• Same set of weights used across

inputs

Testbed System Level Architecture• System View

• Resource Utilization

• Node Management

• Nodes• Simple filters

• Algorithms

• Neural nets

• Custom

• Network Graph• Topology of neural network

• Neural Network Sensitivity• Manages sensitivity of neural nets

• Neuron View• View of what the neuron sees (image, signal, ….)

System

Network GraphNeural Network

Sensitivity10-Best

Nodes

System Level Architecture

• Three key blocks• Training – Inferencing -- Analysis

Summary

• Edge Computing

• Applications: Self-Driving Cars, Security in IMD, RF Machine Learning

• Processor Landscape: GPUs, FPGA

• Algorithms