TI Information – Selective Disclosure TI Information – Selective Disclosure

TI mmWave in Building Automation

For 2019/11/13 Seminar

End Equipment Overview IP Network Camera

Motion Detectors

Automated Doors & Gates

People Counting

1

TI Information – Selective Disclosure

mmWave Sensors – Technology Overview

2

• Motion Detection – Indoor detection of breathing, talking, typing for lighting control and security. Outdoor detection at extended distances (50m+) for security. Motion classification of people vs animals vs background.

• People Counting – Counting and location tracking of multiple people for security, retail, elevators, factories.

• Automated Doors – Tracking of objects approaching automatic door. Classification of object direction, size.

• IPNC Surveillance – Augmenting security cameras for enhanced motion detection or point-tilt-zoom (PTZ) control

When to use

mmWave

sensing?

• mmWave technology is robust against environmental influences such as dazzling sunlight, no light, weather conditions like rain and fog and changes in temperature

• Detect very fine motions, RF technology for penetration through materials like plastic, fabric, and drywall. Use in privacy conscious applications

• RFCMOS technology enables analog/digital integration in a single low-power, small, single-chip solution

Why Now?

• mmWave is the band of spectrum between 30GHz and 300GHz

• Electromagnetic waves used for sensing, imaging and communications

• mmWave sensors measure with high accuracy range, velocity and angle of remote objects

What is

mmWave

sensing

TI Information – Selective Disclosure

1. A synthesizer (synth) generates a “chirp"

2. The chirp is transmitted by the TX antenna

3. The chirp is reflected off an object and the reflected chirp is received

at the RX antenna.

4. The RX signal and TX signal are ‘mixed’ and the resulting signal is called an

‘IF signal’.

1 2

3

4

IF signal

Synth

TX ant.

RX ant. mixer

Tx = sin 𝑤1𝑡 + ϕ1) IF = sin 𝑤1 − 𝑤2 𝑡 + (ϕ1−ϕ2)

Rx = sin 𝑤2𝑡 + ϕ2)

A

t fc=77GHz

t Tc=40us

f 81GHz

B=4GHz

S

Basics of FMCW (Frequency Modulation Continue Wave)

The fundamentals of millimeter wave sensors, http://www.ti.com/lit/wp/spyy005/spyy005.pdf

3

TI Information – Selective Disclosure

Basics of FMCW (Range Measurement)

By working with FFT on these IF signals to get Range, Velocity, Angle

information of detecting object

IF frequency = Tx frequency – Rx frequency

IF frequency

4

TI Information – Selective Disclosure

Basics of FMCW (Velocity and Angle Measurement)

Velocity and Angle of object reflects in phase difference of IF signal.

IF frequency = Tx frequency – Rx frequency

Multiple

Transmission chirps

separated in time

Multiple chirps for velocity detection

Reception of chirps over different

antennas separated in space

(distance)

Multiple received chirps.

Reflected Signal from

moving object has

different phase for two

reflected chirps.

(Intermediate frequency)

Multiple antennas for angle detection

5

TI Information – Selective Disclosure

TI Innovation – Single-Chip CMOS

6

Discrete Multi-Chip mmWave Sensor • Discrete solution – expensive

• Complex and critical signal routes

• Unconventional packaging

• Prone to noise

• Lack of system level monitoring

• Crude implementation of RF and Baseband safety

TI Single-Chip mmWave Sensor • Smaller in size

• Simpler design

• Built in monitoring and calibration (SIL)

• High Resolution, less false positives

• Programmable core

• Lower Power

PCB1 PCB2 PCB

mmWave

RX+TX+ADC

MCU + DSP

TI Information – Selective Disclosure

IWR1xxx mmWave Signal Processing

7

RF Front-End ADC ADC Data

IWR1443

IWR6843

Pre-

Processing (Interference Mitigation)

1st Dim FFT

(Range)

2nd Dim FFT

(Velocity)

3rd Dim FFT

(Angle Arrival) Detection

Point Cloud [Range, Velocity, Angle]

Clustering Tracking Object Classification Objects

TI Information – Selective Disclosure

Industrial Radar – Path of Development

8

Discover

“I am looking for a new sensor for my

project”

Evaluate

“I have purchased an EVM to evaluate

mmWave sensors”

Design

“I have selected mmWave sensors for my

project”

• Tools & SW that provide building

blocks to develop unique mmWave

sensor applications

• Enable rapid development, test and

deployment

mmWave SDK & LIB

IWR1xxxBOOST EVM

App Notes, TID & Training

• Tools & SW focused on shortening

evaluation time by simplifying getting

started and providing reference

software to start with

mmWave Labs

Chirp Database

OOB Demo + Visualizer

mmWave Studio

• Tools focused on showcasing

mmWave sensor capabilities and

potential

• Enable Proof of Concept validation of

use cases

Sensing Estimator

TI.com Portal

White Papers & Videos

mmWave Experiments

mmWave Sensors Forum (e2e)

TI Information – Selective Disclosure

Object Range Detection

9

TI Confidential – NDA Restrictions Traffic monitoring Intelligent street lighting

Level sensing

Forklifts Robotics

Drones

People counting

Perimeter

security

IWR mmWave Sensors TI's single chip mmWave sensors integrate

a DSP, MCU and RF front-end to detect

range, velocity and angle

Measure tank fluid level

with unprecedented

accuracy for inventory

control and early leak

detection

Sensing performance that improves pedestrian safety and provides

power/cost savings through intelligent triggering of lighting

Detect objects in

obstructed views for

intelligent safety

Unprecedented

accuracy at the

micrometer level

Enable autonomous flight for

building, land surveying and

delivering packages

Detect people in a zone of

interest and trigger actions

Enabling security

systems with motion

sensitive detection

and tracking

Detect traffic location and

volume more accurately

Doors & Gates Intelligent perimeter

detection

TI Information – Selective Disclosure

Building Automation – Problems to Solve using mmWave

11

False Detection – Fine Motion

• Sensing systems to detect occupancy today are prone to false detection

• In security, all detection events must be responded to by a person (camera, guard, police)

• Costs $$$, false detection desensitize responders to real threats

• Need sensing systems that can ignore common false detection sources (environment, objects, movement outside ROI)

People Localization and Counting

• Locating and counting people can be done today, but requires complex and expensive technologies like stereo vision and 3D ToF

• Doing this accurately requires expensive processing and complex SW.

• Need for less expensive, higher accuracy solutions

• TI mmWave on-board processing enables counting and tracking of multiple people in single-chip

Motion Classification

• Classification of objects and motion such as determining if John, a forklift, or a dog has just moved into a ROI

• Can be done today, but requires complex and expensive technologies like vision processors.

• Need for less expensive solutions that require less complex SW implementations

• TI mmWave range, velocity, angle, data along with point cloud size can be used for classification

TI Information – Selective Disclosure

mmWave Sensors for Building Automation – Offering Summary

12

Application Occupancy Detection and

Automated Doors and Gates

Motion Detectors and IP Network

Cameras

Benefits Track and separate multiple people

simultaneously with a single chip sensor. Simplify

solution by ignoring non-moving (static) objects

like chairs and tables and filtering moving objects

such as fans, curtains, or blinds.

Robust detection and localization of slow

moving people even in cluttered outdoor

areas with lots of movement from clutter like

trees and shrubs

Hardware

(field of view)

IWR6843 ISK EVM

(120° horizontal, 30°

vertical)

IWR6843 ODS EVM

(120° horizontal,

120° vertical)

IWR6843 ISK EVM

(120° horizontal, 30° vertical)

Example

Environments

Conf

Room

Open

Office

Open Office

(ceiling mount)

Outdoors

Example Coverage

Area

6m x 6m 14m x

14m

16m x 16m 25m x 50m

Example Range

(resolution)

6m

(0.048m)

14m

(0.12m)

8m

(0.12m)

55m

(0.5m)

Example Velocity

(resolution)

5.17m/s

(0.082m/s)

5.25m/s

(0.082m/s

5.25m/s

(0.082m/s)

6.2m/s

(0.1m/s)

Applicable

Onboard Algorithms

Static Clutter Removal,

Group Tracking/Counting

Static Clutter Removal,

Group Tracking for Object filtering

Provided Example

Demonstrations

• People Counting

Lab

• Indoor False

Detection Mitigation

Lab

• Overhead People

Counting Lab

50m Outdoor False Detection Filtering Lab

Discover

ti.com and

videos

1. Review mmWave Building Automation

Applications page

2. Watch the People Counting Video

3. Watch the People Counting Applications Video

4. Watch the Intelligence at the Edge Video

Evaluate

Evaluation

Kits

1. Order IWR6843ISK + MMWAVEICBOOST EVM

2. Order IWR6843 ODS Antenna Board

3. Order mmWave POE Board

Experiments

1. Fine Motion Detection vs PIR Experiment

2. People Counting through drywall, glass, wood

Experiment

3. Response of radar to rain Experiment

4. *NEW* Fall Detection with IWR6843

Labs

1. People Counting Lab

2. Indoor False Detection Mitigation Lab

3. 50m Outdoor False Detection Filtering Lab

4. Overhead People Counting Lab

Reference

Designs

1. People Counting and Tracking Reference Design

using IWR6843

Design Explore

Device

1. Product Folder IWR6843

2. Reference IWR6843 datasheet, errata and TRM

3. Review IWR6843 EVM schematics and layout

Follow the links below for where to go on

ti.com

TI Information – Selective Disclosure

L: Conference Room with Static Clutter Removal for chairs and table

R: Hallway Scene person in GREEN tracked at 14m with Medium Range

Configuration and Group Tracking

13

Base configurations of people counting TI Design support 6m and

14m operation.

Tuning of parameters in TI Design enables variety of applications

and environments

1. Discover mmWave offering for people tracking and counting page here

1. Watch Video: People Counting Applications & Benefits

2. Watch Video: Intelligence at the Edge

2. Evaluate the performance

1. Order IWR6843 EVM here

2. Download People Counting Lab

3. Download Indoor False Detection Mitigation Lab

3. Design custom boards with IWR6843 silicon

1. Reference IWR6843 datasheet, errata and TRM

2. Review IWR6843 EVM schematics and layout

4. Leverage turn-key or design custom solutions using 3rd Parties such as:

1. Turn-Key and Custom: Ainstein

2. Turn-Key and Custom: D3 Engineering

3. Turn-Key and Custom: Smart Radar Systems (SRS)

Mounting assumes 1.5-2.5m elevation, with 10 degree downtilt

Wall Mounted People Tracking and Counting Reference Design using mmWave Radar Sensor TIDEP-01000, Design Status: On ti.com

Short Range

Configuration

Medium Range

Configuration

HW / EVM IWR6483 ISK EVM

Field of View 120° Horizontal, 30° Vertical

Max Range 6m 14m

Example Area 6m x 6m 6m x 14m

14m x 14m

Range Resolution 4.8cm 12cm

Max Velocity 5.17 m/s 5.25 m/s

Velocity Resolution 0.082 m/s 0.082 m/s

Algorithms Used Static Clutter Removal, Group

Tracking, False Detection

Mitigation

Static Clutter Removal, Group

Tracking, False Detection

Mitigation

System Power ~1.5W

Location accuracy Person location within <16cm

Counting density 3 persons per square meter

Demonstrated accuracy +/- 0 persons +/- 1 persons +/- 2 persons

3 people in scene >95% of frames 100% of frames 100% of frames

5 people in scene >51% of frames >85% of frames 100% of frames

7 people in scene >59% of frames >85% of frames >98% of frames

9 people in scene >14% of frames >43% of frames >84% of frames

TI Information – Selective Disclosure 14

Ability to detect height of people and classify as

standing/sitting/laying down (YELLOW – standing, BLUE – sitting)

Mounting and sensing distance assumes 3m elevation

POE enables simplified integration with existing infrastructure

March 2019

(Release on TI REX)

HW / EVM IWR6843 ODS EVM

IWR6843 Power Over Ethernet Adaptor

Field of View 120° Horizontal, 120° Vertical

Max Range 4m – radial

Example Area 6m x 6m

Range Resolution 12cm

Max Velocity 5.25m/s

Velocity Resolution 0.082m/s

Algorithms Used Static Clutter Removal, Group Tracking, Stance Detection

System Power ~1.5W

Performance Details Max count: 3 people

Stances detected: standing / sitting

Base configurations of ceiling mounted people counting TI Design

support 360° radial operation.

Tuning of parameters in TI Design enables variety of applications

and environments

Ceiling Mounted People Tracking and Counting Reference Design using mmWave Radar Sensor and POE Design Status: Available on TI REX

Electrical Power

Data + Configuration

IEEE

802.3a

f Class

1 POE

1. Discover mmWave offering for people tracking and counting page here

1. CES 2019 : Video Standing / Sitting detection

2. CES 2019 : Video Conference Room Implementation with 3P Ainstein

2. Evaluate the performance

1. Order IWR6843 ODS EVM + MMWAVEICBOOST

2. Order mmWave POE Board

3. Download Overhead People Counting Lab

3. Design custom boards with IWR6843 silicon

1. Reference IWR6843 datasheet, errata and TRM

2. Review IWR6843 EVM schematics and layout

4. Leverage turn-key or design custom solutions using 3rd Parties such as:

1. Custom: Ainstein

2. Turn-Key and Custom: RF Beam

TI Information – Selective Disclosure

Intelligent Motion Detection

• Only turn on camera if radar detects and verifies motion

• Reduce false detection, less false alarms

• Result is system resource conservation:

– Reduce Power Consumption

– Reduce Network Bandwidth – more cameras in system

– Reduce Video Storage – less server storage required

15

Radar ON: Low Power

State

Motion Detected

and classified by Radar

Video Stream

Turned ON

Vision Fusion / PTZ Control

• Use of both camera vision and radar combined to determine position and

velocity of people

• Use radar to identify targets even in rain, fog, dust, and other extreme

conditions

• Locate and track targets for PTZ and focus control

Outdoor 50m People Tracking and False Detection Mitigation Application Usage

TI Information – Selective Disclosure

Windy day testing showing false detection filtering with Group

Tracking algorithm

16

Example Configuration

HW / EVM IWR6843 ISK EVM (ES2.0)

Max Field of View /

Coverage Area

120° Horizontal, 30° Vertical

Max Range 56m

Range Resolution 50cm

Max Velocity 6.2m/s

Velocity Resolution 0.10m/s

Algorithms Used Static Clutter Removal,

Group Tracking (configured for outdoor, false detection filtering)

System Power ~1.5W (not duty cycled)

100mW-500mW (duty cycled, 1 frame to 5 frames)

Performance Metrics • Max range: 56 m departing, 56 m approaching

• Sneaking person (Crouched) detected @56m departing and

@42m approaching

Field of View (horizontal)

at measured distance

55m +/- 45°

40m +/- 60°

30m +/- 60°

20m +/- 60°

10m +/- 60°

Detects slow moving people even in cluttered outdoor areas with

lots of motion

• Robust sensing regardless of environment, uses integrated DSP on

IWR6843 and algorithms to ignore point cloud from dynamic clutter

like trees and shrubs

• Use with IP Network Cameras to turn cameras on or send alerts

when a moving person is detected only Moving Trees/Shrubs

seen but filtered

X-Y Scatterplot View

Cumulative

Range vs Doppler View

See person starting

at ~55m

Person Tracked

@ 50m

Outdoor 50m People Tracking and False Detection Mitigation

1. Discover mmWave offering for people tracking and counting page here

1. Watch Video: Intelligence at the Edge

2. Evaluate the performance

1. Order IWR6843 ISK EVM here

2. Download 50m People Tracking and False Detection Mitigation Lab

3. Design custom boards with IWR6843 silicon

1. Reference IWR6843 datasheet, errata and TRM

2. Review IWR6843 EVM schematics and layout

4. Leverage turn-key or design custom solutions using 3rd Parties such as

1. ODM: Alpha Networks

2. ODM: Primax

3. Custom: Smart Radar Systems (SRS)

4. Custom: Colorado Engineering (CEI)

TI Information – Selective Disclosure

Evaluate today!

Detecting Human Falls and Stance with IWR6843

17

Shows the change in shape and height of mmWave 3D point cloud when person

stands, sits, or lays down

• Uses wide-angle antenna on IWR6843 ODS EVM to visualize the point cloud of

person standing, sitting, or laying down

• Point cloud and velocity information could be used to create algorithm to determine if

person has fallen down for applications such as elderly monitoring

• Visualization of stance was demonstrated at ranges of 2m and 5m

More information:

including

Documentation

and Software

Available on TI

Resource

Explorer

Hardware IWR6843 ODS

EVM

Base Truth

Color

indicates

height

relative to

EVM

Standing Sitting Laying Down

EVM

(backside)

3D point

cloud

TI Confidential – NDA Restrictions

Building Automation – Technology Comparison

18

Passive Infrared Cameras

Pros:

• Simple, low power consumption (uA-mA)

Cons:

• Low sensitivity to motion

• False detection outdoors from sunlight, temperature

• Limited range (5-10m), no position/range information

TI mmWave Radar

Pros:

• Algorithms applied for variety of applications

• Video for recording and monitoring

Cons:

• False detection from shadows, occlusion, day/night cycles,

changing environment.

• No position/range information

• Privacy considerations

Measures change in infrared light to detect

motion

Video image processor analyzes imagery to

determine people movement and behavior

Active Infrared (3D ToF, LIDAR)

Pros:

• High sensitivity to motion (breathing, typing)

• Simple static and dynamic object separation

• Onboard DSP processing for single-chip tracking,

classification of objects

• Extended range for person detection (50m+)

• Insensitive to weather, changing environments

Cons:

• Lower angular resolution than camera or active infrared

TI’s fully-integrated, single-chip 77GHz and

60GHz mmWave radar

Pros:

• High angular resolution provides rich dataset similar to

camera

Cons:

• Limited range in presence of sunlight (5-10m)

• Requires substantial processing to separate and classify

relevant objects

• System complexity (optics, illumination, processing)

Measurement of infrared light time of flight

TI Information – Selective Disclosure

IWR6843 Industrial Starter Kit

19

Antenna

Board

Carrier

Board

Antenna

Parameters

IWR6843 ISK IWR6843 ODS IWR6843 AOP

Azimuth FOV +/- 60 degree +/- 60 degree +/- 65 degree

Elevation FOV +/- 20 degree +/- 60 degree +/- 65 degree

Azimuth Angle

Resolution 15 degree 29 degree 29 degree

Elevation Angle

Resolution 58 degree 29 degree 29 degree

Max Expected

Distance

(Human)

50m 20m 25m

Bandwidth 3.5GHz

(60.5 – 64GHz)

1.7GHz

(61 – 62.7GHz)

4GHz

(60 – 64GHz)

• Carrier board with swappable antenna

boards

• Swappable antenna boards support

different antenna designs

TI Information – Selective Disclosure

DCA1000 Raw Data Capture Solution

20

• Enables RF performance evaluation with PC environment

• mmWave ADC data can be captured and brought onto PC for advanced algorithm

development and prototyping

• Data capture (recording) via Ethernet interface at high speed ( ADC, User data,

debug data)

• mmWave Studio software tool used to visualize object range/velocity/angle

x W R 1 x + D C A 1 0 0 0

or