autonomous driving with the mipi camera and … driving with the mipi camera and sensor interfaces...
TRANSCRIPT
Hezi Saar
April 6, 2017
Linley Autonomous Hardware Conference
Autonomous Driving with the MIPI Camera and Sensor Interfaces
© 2016 Synopsys, Inc. 2
Abstract
• Advanced Driver Assistance Systems (ADAS) SoCs for self-driving cars incorporate numerous interfaces for functions such as camera, radar, lidar and sensors. It is vital for such interfaces to meet the new stringent automotive standards, and offer the low power and high performance requirements that designers are looking for. The MIPI interfaces for camera (CSI-2) and sensors (I3C) are playing an essential role in enabling ADAS SoCs for autonomous driving. This presentation describes how these MIPI specifications are implemented in automotive SoCs and why.
© 2016 Synopsys, Inc. 3
Agenda
Implementation of MIPI interfaces in applications beyond mobile
MIPI camera and sensor specifications for automotive applications
Meeting automotive reliability requirements
Summary
© 2016 Synopsys, Inc. 4
MIPI Specifications in New ApplicationsAutomotive, IoT / Wearables, Virtual / Augmented Reality
© 2016 Synopsys, Inc. 5
Safety-Critical ADAS ApplicationsRequire ISO 26262 Functional Safety Compliance and ASIL Certification
Electronics Failures Can Have Hazardous Impact
≠
© 2016 Synopsys, Inc. 6
Advanced Driver Assistance Systems (ADAS)
• Passive Driver Assistance Systems– Back-up, surround view camera
– Distance alert system
A Rapidly Evolving Technology: Car Mirrors, Multiview Systems
• Active Driver Assistance Systems– Back-up camera with identification & braking
– Collision avoidance
© 2016 Synopsys, Inc. 7
Example: Automotive Surround ViewUsing MIPI CSI-2 Image Sensors & DSI Display
Rear Camera
Display
CAN Interface
MPU Proprietary, LVDS or Ethernet Switch
DRAM Flash Memory
Power SupplyFront Camera
Module
Left Camera Module
Right Camera Module
Rear Camera Module
Other Camera Module
Vbat
LVDS or Ethernet Link
MIPI CSI-2 Image Sensors
MIPI DSI Display
Front Camera
Left Camera
Right Camera
Other Camera Module
© 2016 Synopsys, Inc. 8
Automotive Applications Require Different SoC Architectures
High-End ADAS Infotainment MCU
• LPDDR4, Ethernet AVB, MIPI, HDMI, PCIe, SATA, ADC
• Embedded Vision• Security• Sensor Fusion• Requires Functional Safety
• USB, LPDDR4, Ethernet AVB, MIPI, HDMI, PCIe, SATA, ADC, UFS, eMMC
• Real-time Multimedia• Security• Sensor Fusion
• IP: Ethernet 10/100/1000, ADC, I/F peripherals• Medium Density NVM
ARC HS
ARC HSARC HS
© 2016 Synopsys, Inc. 9
MIPI CSI-2 and D-PHY Specifications
© 2016 Synopsys, Inc. 10
MIPI CSI-2 Over D-PHY Overview
CSI-2 Device
DPH Y
CSI-2 Host
DPHY
Clk+
Clk-
L0+
L0-
Clk+
Clk-
L0+
L0-L1+
L1-
L1+
L1-
Frame Buffer
CSI-2 Transmitter
PacketBuilder
LaneDistribution
CCI SlaveSCL
SDA
SCL
SDA
CCI Master
CSI-2 Packet
CSI-2 Packet
D-PHYHS Burst
D-PHYHS Burst
CSI-2 Receiver
PacketDecoder
LaneMerger
CSI-2 Packet
CSI-2 Packet
Frame Buffer
VC CRCD
T WC ECC
Payload byte size Data CRC processing
ECC protecting the header
Data Format Definition
Virtual Channel Identification
Packet Builder
© 2016 Synopsys, Inc. 11
MIPI CSI-2 Packets
Data PFPHFS FEData PFPH
KEY:SoT – Start of Transmission EoT – End of Transmission LPS – Low Power StatePH – Packet Header PF – Packet Footer + Filler (if applicable)FS – Frame Start FE – Frame EndLS – Line Start LE – Line End
Frame Start Packet
Frame End Packet
First Packet of Data
Last Packet of Data
VVALID
HVALID
DVALID
SoT LPSEoT SoT EoTLPSEoT SoTEoT SoT
Short packets used for frame synchronization
Image data
Low power states between
image lines
Short packets used for frame synchronization
© 2016 Synopsys, Inc. 12
MIPI CSI-2 v2.0 Feature Enhancements for Automotive Applications • RAW-16 and RAW-20 color depth
– improves intra-scene High Dynamic Range (HDR) and Signal to Noise Ratio (SNR) to bring “advanced vision” capabilities to autonomous vehicles and systems
• Latency Reduction and Transport Efficiency (LRTE) feature – supports increased image sensor aggregation without adding to system cost;
facilitate real-time perception, processing and decision-making; and optimized transport to reduce the number of wires, toggle rate and power consumption
• Differential Pulse Code Modulation (DPCM) 12-10-12 compression– reduces bandwidth while delivering superior SNR images devoid of compression
artifacts for mission-critical vision applications
• Scrambling to reduce Power Spectral Density (PSD) emissions– minimize radio interference and allow further reach for longer channels
• Expanded number of virtual channels from 4 to 32 – accommodate the proliferation of image sensors with multiple data types and
support multi-exposure and multi-range sensor fusion for Advanced Driver Assistance Systems (ADAS) applications such as enhanced collision avoidance Image courtesy of the MIPI Alliance
© 2016 Synopsys, Inc. 13
Virtual Channels
Virtual Channel 0 – Line 0Virtual Channel 0 – Line 1Virtual Channel 0 – Line 2Virtual Channel 0 – Line 3Virtual Channel 0 – Line 4
Virtual Channel 0 – Line N
Virtual Channel 1 – Line 0Virtual Channel 1 – Line 1Virtual Channel 1 – Line 2Virtual Channel 1 – Line 3Virtual Channel 1 – Line 4
Virtual Channel 1 – Line M
Line 0Line 1Line 2Line 3Line 4
Line N
Line 0Line 1Line 2Line 3Line 4
Line M
© 2016 Synopsys, Inc. 14
D-PHY Architecture
• Synchronous forwarded DDR clock link architecture
• One clock and multiple data lanes configuration
• Static/dynamic de-skew supported through calibration
• No encoding overhead• Low-power and high-speed modes• Primarily targeting camera and display• Spread spectrum clocking supported for
EMI/EMC considerations• Large ecosystem, proven in millions of
phones and cars
The Popular Physical Layer Used for CSI-2 and DSI Specifications
Two Data Lane Configuration
© 2016 Synopsys, Inc. 15
MIPI I3C Specification Standardizing Sensor Interface
© 2016 Synopsys, Inc. 16
Why I3C?
• Today Smartphones typically have 10-15+ sensors– Which require 12-18+ pins
• Different Sensors have different requirements – Fingerprint vs Compass
• Typical approach is to connect sensors using a mix of I2C and SPI – I2C for lower data rates and SPI for higher
data ratesMultiple side band signals– For interrupts, chip selects, power
management
The Challenge of Integrating Multiple Sensors with Different Requirements
This will increase the package size and add complexity which translates into additional costs
So many I/Os
required!!
Image courtesy of the MIPI Alliance
No Standard driver for these fragmented interfaces
© 2016 Synopsys, Inc. 17
Why I3C?
It takes the goodness of I2C – Two-wire, Simple
It takes the goodness of SPI – Low Power and Speed
Adds features such as– In-band interrupt/command support– Dynamic addressing– Advanced power management– High data rates
While maintaining support for legacy I2C sensors
– Evolutionary, not revolutionary
You Can do Everything with Two IOs!
I/Osreduced to just two!!
Image courtesy of the MIPI Alliance
© 2016 Synopsys, Inc. 18
I3C Enables Efficient System ArchitecturesExample: Sensor Hub
Power Reduction, More Efficient System, Faster Data Transfer
© 2016 Synopsys, Inc. 19
MIPI CSI-2, D-PHY & MIPI I3C
• Supports advancements in imaging for new applications: Health, Convenience, Security, Lifestyle, Efficiency
• Camera Controller Interface (CCI) and Always-ON advancement considerations using I2C and future MIPI I3C
Sensor
Sensor
Sensor
Primary Camera Module
Image Sensor
Lens Actuators/Controllers
SDASCL
Secondary Camera Module
Image Sensor/SoC
Pixels
Pixels
Host Processor/ISP
© 2016 Synopsys, Inc. 20
Summary
© 2016 Synopsys, Inc. 21
MIPI Specs for Multimedia, Storage, Sensor & Wireless Connectivity in Automotive Applications
• Real time video & data network
• Gateways• Telematics• V2V• V2I• Security
Vehicle Networks & V2X
• Navigation• Audio/video• Entertainment
Infotainment
• Instrument clusters• Voice recognition• Hi-def displays• Surround view
Driver Information
• Parking assist• Lane departure warning
& Lane keep aid• Pedestrian detection &
correction• Automatic emergency
braking
Driver Assistance
Image courtesy of the MIPI Alliance
© 2016 Synopsys, Inc. 22
Meet quality levels required for automotive applications
Accelerate ISO 26262 functional safety assessments to help ensure designers reach target ASIL levels
Reduce risk & development time for AEC-Q100 qualification of SoCs
Key Requirements of Automotive-Grade IPReduce Risk and Accelerate Qualification for Automotive SoCs
Reliability
Functional Safety
Quality
© 2016 Synopsys, Inc. 23
ASIL Ready ISO 26262 Certified MIPI IP
• MIPI CSI-2, D-PHY and I3C protocols
– Leveraged for mobile and beyond – IoT, automotive, AR/VR
• Enables new set of applications in automotive, AR/VR, IoT markets
– Lowers integration risk for application processors, bridge ICs and multimedia co-processors
• Future proof IP supporting variety of speeds, proven in silicon
– Reduces cost and power for multiple instantiations
– Testability features enable low cost manufacturing
Single-Vendor Solution, Production-Proven, Interoperable
Industry’s first MIPI I3C Demo
High-End Processor
Image Signal
Processing
CSI-2 Host Controller
D-PHY
CSI-2 Host Controller
D-PHY
DSI Device Controller
D-PHYCSI-2 Device
Controller
D-PHY
DSI Host Controller
D-PHY
I3C
Synopsys Provides Complete Camera, Display & Sensor Interface IP Solutions
Thank You