Interactive Radars for Smart Cars Introducing industry-leading product system solutions for todays and future cabin
applications
Neha Baheti, Product Marketing mmWave MMICs
1 2018-01-31 Copyright © Infineon Technologies AG 2018. All rights reserved.
Copyright © Infineon Technologies AG 2018. All rights reserved. 2
Agenda
Introduction – Future of Car Cabins
Application Scenarios
Gesture Sensing for Switchless Controls
Vital Sense Monitoring & Use-Cases
Conclusion
Demonstration
1
2
3
4
2018-01-31
No longer a science fiction: Reshaping the ways we drive
3 major benefits drive autonomous and
assisted driving:
• Convenience: Spend time on
alternative activities while driving
• Safety: Reduction of accidents
that are attributed to human
errors
• Economy: New business
opportunities through connected
mobility
Different Levels of Autonomous Driving Ahead
3 2018-01-31 Copyright © Infineon Technologies AG 2018. All rights reserved.
Market penetration per level of automation
2015 up to 20m vehicles p.a.
2020 up to 20m vehicles p.a.
2025 up to 10m vehicles p.a.
2030 up to 10m vehicles p.a.
2035 up to 10m vehicles p.a.
Our believe: The buying decision for a car will be increasingly impacted by the cabin concept safety & convenience features and its implicated leisure and work place
offerings.
AD
AS
L1 L2
Auto
mate
d D
rivin
g
L3 L4 L
5
Driverless
Fully automated
Highly automated
Partially automated
Assisted
Source: IHS, McKinsey, Infineon
Caution Comfort Convenience
2015
Automated & assisted driving will reshape cabin concepts – from driving cockpit to leisure and work place
Automotive Mega Trend
Impact on Driver
Partially Autonomous
Driving
Full-time driver turns into an
occasional driver
4 2018-01-31 Copyright © Infineon Technologies AG 2018. All rights reserved.
Fully automated drive
Driver turns into a user
Control of steering in emergencies Presence of Driver is mandate
Alternative activities (entertainment, workstation, meetings, wellness & healthcare, etc.)
Driver = Passenger
Automotive
Mega Trend
Impact on
Driver
Autonomous Driving
The driver becomes a user
ADVANCED CABIN COMFORT & SAFETY becomes a hot spot
Impact on
Cabin New forms of user
behavior will lead to new application spectrums
Application 1: Gesture Sensing New Input Modality for Car Cabins
5 2018-01-31 Copyright © Infineon Technologies AG 2018. All rights reserved.
Copyright © Infineon Technologies AG 2018. All rights reserved. 6
Evolution of Car Dashboards – Yesterday, Today & Tomorrow
2018-01-31
Copyright © Infineon Technologies AG 2018. All rights reserved. 7
New forms of user behavior will create new applications
2018-01-31
1. Button-free cabin dashboard/ HUDs.
2. Gestures act as additional input modality.
3. Controlling dashboard from distance adds to driver’s convenience.
4. Customize signatures for multi-functional operations.
Gesture Sensing Use Cases in Car
Benefits of Gesture Control in Smart Car
Source: Continental
Dashboard Control
Sunroof Opening
Source: FIAT
Volume Control for Infotainment
Accept/Reject Calls
Source: CES
Copyright © Infineon Technologies AG 2018. All rights reserved. 8
Technology Comparison for Gesture Sensing Traditionally used….
2018-01-31
Capacitive Sensing
Ultrasonic Transducers
Hand-gestures disturbs the E-Field generated by Electrodes
1
Signal Conditioning
Signal Conditioning
Exte
rnal
Ele
ctr
odes ADC
ADC
Signal Processing
2 Capacitive differences are converted to analog signal
Host Controller
Gesture Modelling
Position Tracking
Gesture Recognizer
Apps P
rocessor
1
Piezo-Electric MEMS
ultrasonic transducers/ transceivers array at 36-
44kHz
Signal Conditioning
DAC Pulse
Stream Generation
Signal Conditioning
Beam forming
ADC Signal Processing
Gesture
Modelling
Gesture Recognizer
Apps P
rocessor
Pattern Recognition through ML, DL Algorithms like HMM, CNN, LSTM
Receive
Transmit
Position Tracking
Copyright © Infineon Technologies AG 2018. All rights reserved. 9
Technology Comparison for Gesture Sensing Towards Robustness….
2018-01-31
Radar
Time-of-Flight
Active illumination by near IR 850-900nm
1
2 Phase-shift per pixel calculates distance
3
High-Quality depth map
mmWave Sensor
IF Frequency
Signal Conditioning ADC
Signal Transformation
Feature Extraction
Gesture Recognition
Apps P
rocessor
Signal Conditioning
Radar Signal Generation
Further technological comparison snapshot (Relevant for Gestures Only)
Radar Capacitive Ultrasonic Camera ToF
Distance 5cm – 10m * 0 – 50cm 10 cm – 5m 0-5 m
Object Detection Yes Yes Yes Yes
Object Classification Yes ** No No Yes
Stationary target detection
Yes No Yes Yes
Immunity to Interferences (EMI)
High* Low -NA- -NA-
Environmental influences
Heavy rain and snow only
Rain, noise, temperature smoke, dirt
Wind, rain, noise, temperature
Rain, snow, light source
Design aesthetics Can be concealed (eg. plastic cover)
Requires opening window
Requires opening window
Requires opening window
Computing complexity
Medium Low Low High
Power consumption Medium/low Low Low High
BOM cost Medium Low Low High
False Alarms Low High High Low
10 2018-01-31 Copyright © Infineon Technologies AG 2018. All rights reserved.
• * Frequency and Conducted Power Dependent
• ** With Advanced Alogrithms & Signal Processing
Copyright © Infineon Technologies AG 2018. All rights reserved. 11
3D sensing selection tree from Infineon Solutions: Choosing radar or TOF , Depends on Use-Case
2018-01-31
Solution
Capabilities /Features
Sub use cases/ Functions
Master use cases
Finger tracking Hand tracking
Low range-resolution (70 cm)
High accuracy (1% of range)
High range-resolution
(2 cm)
Human Machine Interface (HMI)
Space & Motion
Macro gesture (Leg-kick, hand waving, Body-moves)
Distinguish each fingers or
body parts
Finger tip mouse
Simple presence detection
Objects Counting/Tracking/Density map
Environmental scanning
60 GHz ToF 24 GHz 60 GHz ToF
Hidden in case No privacy concerns
Face recognition
Indoor/Outdoor Range up to
100 m
Recognizes body contour
Indoor/outdoor up to 7 m
Presence detection / Distance measurement / Position information
Vital sensing
Counting fingers
Drawing
Picture of shape
System-Cost-Benefit
Smallest form factor
Indoor ~10m Outdoor ~4 m
Collision Avoidance
Body tracking
Biometrics
Radar ToF
To get picture of target shape
Authentication Power optimization by
duty cycling
Tiniest finger movement in sub-millimeter range (micro-
gesture)
Measurement independent to environmental conditions
Patient monitoring
Sleep monitoring
Distinguish people from static object
Copyright © Infineon Technologies AG 2018. All rights reserved. 12
Macro, Micro and Vital : Radar as a multi-functional technology
2018-01-31
Macro Doppler Frame
Micro Doppler Frame
Vital Doppler Frame
Scenario Radar Multi-Modes
32 chirps, 256 samples/chirp
32 chirps, 32 samples/chirp, Faster chirp time
10s, 1ms chirp time, Min. 64 samples, 5 ms off time
16% duty cycling
Outputs
Moving humans and objects are detected
Small bodily motions are detected
Periodic heart beat and respiratory signal detected
13 2018-01-31 Copyright © Infineon Technologies AG 2018. All rights reserved.
Application 2: Vital Sensing For in-car passenger safety
Copyright © Infineon Technologies AG 2018. All rights reserved. 14
Scenario from L3 : What if the driver is ill-conditioned to take back the control?
2018-01-31
Driver can let go of the steering wheel and gas pedal
Driver presence & alertness needs to be continuously
monitored Traffic Jam Assist Deactivated
Traffic Jam Assist Activated Driver presence & wellness necessary to take control
E-Call
Copyright © Infineon Technologies AG 2018. All rights reserved. 15
Child Monitoring Using Vital Sensing
2018-01-31
39 32 35 39
31
43 39
47
29 36
43 33
49
33 35 44
32 25
39 42
Heatstroke deaths of children in car
(US) Heatstroke deaths of children in car (…
Source : kids&cars.org ; noheatstroke.org
742 deaths
2019: Start 2021: Protocol Release 2022: Implementation * Roadmap NCAP 2025
Robust Decision Making
Compatible with thick winter clothing and sun shades
Independent of interior and ambient temperatures
Distinguish breathing motions from background noise
Low latency based classification
Robust against false positives
Direct vehicle integration (e.g. LIN)
Copyright © Infineon Technologies AG 2018. All rights reserved. 16
Vital Sensing Use-Cases In-Car
2018-01-31
Driver /Passenger Vital Sense Monitoring
Child Vital Sense Monitoring
Static Objects in car
Macro Doppler
Micro Doppler
N Y
Macro Doppler
Micro Doppler
N N
Moving lifeform in car
Macro Doppler
Micro Doppler
Y Hidden
Challenging Scenario
Copyright © Infineon Technologies AG 2018. All rights reserved. 17
Proof-Of Concept : VSM using FMCW based Interferometry Architecture
2018-01-31
Experiments performed with mmwave 60 GHz sensor
60GHz Sensor
30 cm
Short distance measurement Measurement Close to Body
Heart Rate =93.72/min
Breath Rate =20.2/min
Can be implemented on 24GHz ISM Band MMIC Solutions
Copyright © Infineon Technologies AG 2018. All rights reserved. 18
Infineon Sensor Offerings to Make Cabins Interactive
2018-01-31
Power Supply
Network
Battery
Microcontroller
Sensors
Component
24GHz ISM Band Radar MMICs
BGT24ATR11 BGT24ATR12
BGT24AR2 BGT24AR4
Component
60GHz Radar MMICs
BGT60TR24
Please contact local sales for more information on variants & demo platforms.
BGT24AT2
Component
ToF
Demo Platforms
KIT_ATV_24GHz_EVB Distance2Go Sense2GoL Position2Go
Demo Platforms
DEMO BGT60TR24 (v8 & v9)
3D Image Sensor REAL3™ IRS1125C (Consumer)
IRS1645C (Consumer)
IRS1125A (Automotive)
Demo Platforms
Please contact local sales for more information on variants & demo
platforms