case study: vision and radar-based sensor fusion · 3 automated driving system toolbox case study:...
TRANSCRIPT
![Page 1: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/1.jpg)
1© 2017 The MathWorks, Inc.
Automated Driving System Toolbox
Case study:
Vision and radar-based sensor fusion
Seo-Wook Park
Principal Application Engineer
The MathWorks, Inc.
![Page 2: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/2.jpg)
2
Automated Driving System Toolbox
Examples for vision and radar-based sensor fusion
![Page 3: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/3.jpg)
3
Automated Driving System Toolbox
Case study: Vision and radar-based sensor fusion
Synthesize data to
further test the algorithm
▪ Create driving scenario
▪ Add sensor detections
▪ Generate synthetic data
▪ Test the algorithm
Explore a baseline
sensor fusion algorithm
▪ Read logged vehicle data
▪ Visualize vehicle data
▪ Create multi-object tracker
▪ Implement forward
collision warning
Test the algorithm
with new data set
▪ Test the baseline algorithm
with new data set
▪ Tune algorithm parameters
▪ Customize the algorithm
![Page 4: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/4.jpg)
4
Explore a baseline sensor fusion algorithm
birdsEyePlot
▪ coverageAreaPlotter
▪ plotCoverageArea
▪ detectionPlotter
▪ plotDetection
▪ laneBoundaryPlotter
▪ plotLaneBoundary
▪ trackPlotter
▪ plotTrack
Video Display
▪ VideoReader
▪ readFrame
▪ imshow
multiObjectTracker
▪ objectDetection
▪ trackingEKF
▪ trackingUKF
▪ trackingKF
▪ updateTracks
'01_city_c2s_fcw_10s.mp4'
t1_ForwardCollisionWarningTrackingExample.m
Video Annotation
▪ parabolicLaneBoundary
▪ insertLaneBoundary
▪ vehicleToImage
▪ insertObjectAnnotation
![Page 5: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/5.jpg)
5
ScenarioPlayer
▪ play/pause
▪ stop
▪ pause at
▪ slider
Pause at FCW
Lidar Point Cloud
▪ pcplayer
▪ pointCloud
▪ findPointsInROI
▪ findNeighborsInRadius
▪ select
▪ pcdenoise/pcdownsample
▪ pcfitplane
Display LaneBoundaryType
t2_fcwSensorFusion.m
+ ScenarioPlayer + PointCloudExplore a baseline sensor fusion algorithm
![Page 6: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/6.jpg)
6
Baseline sensor fusion algorithm
imu
radarSensor
visionSensor
lane
findNonClutterRadar
updateTracks
multiObjectTracker
assessThreat
findMostImportantObject
FCW
objectDetection
findNonClutterRadar
![Page 7: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/7.jpg)
7
FCW
Warn
Safe
egoLane
findNonClutterRadar
imu
radarSensor
lane
CalculateGroundSpeed
Ego vehicle speed
Ground speed MotionType
Moving or
Stationary
objectZonePosition
Safe / Warn / FCW
findNonClutterRadar
Relative
speed
enumeration
Safe (0)
Warn (1)
FCW (2)
end
if Zone == zoneEnum.FCW || (Zone ~= zoneEnum.Safe
&& MotionType == objectMotionEnum.Moving)
clutter
non-clutter
![Page 8: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/8.jpg)
8
Forward Collision Warning using Sensor Fusion
imu
radarSensor
visionSensor
lane
findNonClutterRadar
updateTracks
multiObjectTracker
assessThreat
findMostImportantObject
FCW
objectDetection updateTracks
multiObjectTracker
objectDetection
![Page 9: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/9.jpg)
9
Multi-Object Tracker
updateTracks
multiObjectTracker
objectDetection
Parameters
AssignmentThresholdConfirmationParameters
NumCoastingUpdates...
FilterInitializationFcn
trackingEKF
State-transition functionMeasurement function
StateCovarianceMeasurementNoise
ProcessNoise...
• Linear Kalman filter• Extended Kalman filter• Unscented Kalman filter
•Update tracker with
new detections
• Creates a multi-object tracker
object using a global nearest
neighbor (GNN) criterion
Tracks
![Page 10: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/10.jpg)
10
Global Nearest Neighbor (GNN)
track
▪ Evaluate each detection in predicted track region and find “best” one to
associate with the track based on Mahalanobis distance and
assignDetectionsToTracks 1)
𝑡1
1) James Munkres's variant of the Hungarian assignment algorithm
𝑡2 𝑡3𝑡4
𝑡5
𝑑1
𝑑3 𝑑4
𝑑2
![Page 11: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/11.jpg)
11
Multi-Object Tracker
Track Manager
Multi-Object Tracker
Kalman filter
New
Detections
Updated
Tracks
• Assigns detections to tracks
• Creates new tracks
• Updates existing tracks
• Removes old tracks
• Predicts and updates state of track
• Supports linear, extended, and
unscented Kalman filters
Time
Measurement
MeasurementNoise
SensorIndex
MeasurementParameters
ObjectClassID
ObjectAttributes
TrackID
Time
Age
State
StateCovariance
IsConfirmed
IsCoasted
ObjectClassID
ObjectAttributes
![Page 12: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/12.jpg)
12
Multi-Object Tracker
Track Manager
Multi-Object Tracker
Kalman filter
Assign
detections
To
existing tracks
Initiate tracks
Unassigned
detectionsInitialize filter
Update & confirm
tracks
Assigned
detections Correct filter with
new measurement
Calculate distance
Predict &
Delete tracks
Unassigned
tracks Predict filter
to new time
New
Detections
Updated
Tracks
Tracks list
function filter = initcaekf(detection)
...
filter = trackingEKF(@constacc, ... % State-transition function (const-accel model)
MF, ... % Measurement function
state, ... % Initializing state vector
'StateCovariance', stateCov, ... % State estimation error covar
'MeasurementNoise', detection.MeasurementNoise(1:4,1:4), ... % Measurement noise covar
'StateTransitionJacobianFcn', @constaccjac, ... % State-transition function Jacobian
'MeasurementJacobianFcn', MJF, ... % Measurement Jacobian function
'ProcessNoise', Q); % Process noise covariance
end
tracker = multiObjectTracker(...'FilterInitializationFcn', @initcaekf, ... % Handle to tracking Kalman filter
'AssignmentThreshold', 35, ... % Normalized distance from track for assignment
'ConfirmationParameters', [2 3], ... % Confirmation parameters for track creation
'NumCoastingUpdates', 5); % Coasting threshold for track deletion
trackingKF, trackingUKF
Time
Measurement
MeasurementNoise
SensorIndex
MeasurementParameters
ObjectClassID
ObjectAttributes
TrackID
Time
Age
State
StateCovariance
IsConfirmed
IsCoasted
ObjectClassID
ObjectAttributes
![Page 13: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/13.jpg)
13
NumCoastingUpdates
'NumCoastingUpdates', 5 'NumCoastingUpdates', 20
Coasting threshold for track deletion.• A track coasts when no detections are assigned to the
track after one or more predict steps.
• If the number of coasting steps exceeds this threshold,
the track is deleted.
![Page 14: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/14.jpg)
14
Automated Driving System Toolbox
Case study: Vision and radar-based sensor fusion
Explore a baseline
sensor fusion algorithmTest the algorithm
with new data set
Synthesize data to
further test the algorithm
▪ Read logged vehicle data
▪ Visualize vehicle data
▪ Create multi-object tracker
▪ Implement forward
collision warning
▪ Test the baseline algorithm
with new data set
▪ Tune algorithm parameters
▪ Customize the algorithm
▪ Create driving scenario
▪ Add sensor detections
▪ Generate synthetic data
▪ Test the algorithm
▪ Generate C code
![Page 15: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/15.jpg)
15
Test the baseline algorithm for new data set
'05_highway_lanechange_25s.mp4'‘fcw_0_Baseline'
'01_city_c2s_fcw_10s.mp4'
t2_fcwSensorFusion.m
![Page 16: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/16.jpg)
16
What causes the false positive?
False
Positive
Pop Quiz
A: radar ghost
detection
B: radar clutter
C: inaccurate
tracking
D: too sensitive
warning
E: program bug
B: radar clutter
C: inaccurate
tracking
D: too sensitive
warning
![Page 17: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/17.jpg)
17
Baseline sensor fusion algorithm (fcw_0_Baseline)
imu
radarSensor
visionSensor
lane
findNonClutterRadar
updateTracks
multiObjectTracker
assessThreat
findMostImportantObject
FCW
objectDetection
%%%%%% FCW Triggering Condition
if ttc<0 % Triggering if ttc<0 (target is approaching to host)
...
if ((rangeMagnitude < brakingDistance) &&...
(zone==zoneEnum.FCW))
activateFCW = true; % FCW activated
D: too sensitive
warning
![Page 18: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/18.jpg)
18
%%%%%% UPDATED ALGORITHM %%%%%%%%%%%%%%%%%%%%%%%%%%%%%
if ttc<0 && trackCounter>=minTrackCnt
...
if ((rangeMagnitude < brakingDistance) &&...
(zone==zoneEnum.FCW))
activateFCW = true; % FCW activated
Modified algorithm (fcw_1_PostProcessing)
imu
radarSensor
visionSensor
lane
findNonClutterRadar
updateTracks
multiObjectTracker
assessThreat
findMostImportantObject_v1
FCW
objectDetection
minTrackCnt = 4
![Page 19: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/19.jpg)
19
fcw_0_Baseline vs. fcw_1_PostProcessing
fcw_1_PostProcessing
fcw_0_Baseline
![Page 20: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/20.jpg)
20
C: inaccurate
tracking
Baseline sensor fusion algorithm (fcw_0_Baseline)
imu
radarSensor
visionSensor
lane
findNonClutterRadar
updateTracks
multiObjectTracker
assessThreat
findMostImportantObject
FCW
objectDetection
if Zone == zoneEnum.FCW || (Zone ~= zoneEnum.Safe
&& MotionType == objectMotionEnum.Moving)
B: radar clutter
![Page 21: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/21.jpg)
21
Modified algorithm (fcw_2_PreProcessing)
imu
radarSensor
visionSensor
lane
findNonClutterRadar_v1
updateTracks
multiObjectTracker
assessThreat
findMostImportantObject
FCW
objectDetection
if Zone ~= zoneEnum.Safe
![Page 22: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/22.jpg)
22
fcw_0_Baseline vs. fcw_2_PreProcessing
fcw_2_PreProcessing
fcw_0_Baseline
![Page 23: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/23.jpg)
23
Automated Driving System Toolbox
Case study: Vision and radar-based sensor fusion
Explore a baseline
sensor fusion algorithmTest the algorithm
with new data set
Synthesize data to
further test the algorithm
▪ Read logged vehicle data
▪ Visualize vehicle data
▪ Create multi-object tracker
▪ Implement forward
collision warning
▪ Test the baseline algorithm
with new data set
▪ Tune algorithm parameters
▪ Customize the algorithm
▪ Create driving scenario
▪ Add sensor detections
▪ Generate synthetic data
▪ Test the algorithm
![Page 24: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/24.jpg)
24
Euro NCAP TEST PROTOCOL – AEB VRU systems
Car-to-VRU Nearside Child (CVNC)
▪ a collision in which a vehicle travels
forwards towards a child pedestrian
crossing it's path running from
behind and obstruction from the
nearside and the frontal structure
of the vehicle strikes the pedestrian
at 50% of the vehicle's width when
no braking action is applied.
5 km/h
20-60 km/h
![Page 25: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/25.jpg)
25
Create Driving Scenario
birdsEyePlot
▪ coverageAreaPlotter
▪ plotCoverageArea
▪ detectionPlotter
▪ plotDetection
▪ laneBoundaryPlotter
▪ plotLaneBoundary
▪ trackPlotter
▪ plotTrack
▪ road
▪ vehicle
▪ actor
▪ path
▪ radarDetectionGenerator
▪ visionDetectionGenerator
▪ chasePlotdrivingScenario ▪ plot
Egocentric projective
perspective plot
driving
scenario plot
Generate radar or vision detections
for driving scenario
t3_1_playDrivingScenario.m
![Page 26: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/26.jpg)
26
Workflow for driving scenario
Create a driving scenario object
Add a road to driving scenario
Create a vehicle or actor within
driving scenario
Create actor or vehicle path in
driving scenario
Create driving scenario plot and
egocentric projective perspective plot
Generate radar or vision detections
for driving scenario
Advance driving scenario
simulation by one time step
Or move actor to time
s = drivingScenario
road(s,RoadCenters,Width)
v = vehiclea = actor
path(v,waypoints,speed)path(a,waypoints,speed)
plot(s)chasePlot(egoCar)
radarDetectionGeneratorvisionDetectionGenerator
advance(s)Or, move(v,time)
generateDetections
updatePlots(s)
updateBirdsEyePlot
Update scenario and
chase plots
Update Birds-eye plot
Generate sensor
detections
A
A
![Page 27: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/27.jpg)
27
Sensor model for vision detection
visionSensor = visionDetectionGenerator(...'SensorIndex', 3, ... % Unique sensor identifier
'UpdateInterval', 0.1, ... % Required interval between sensor updates (sec)
'SensorLocation', [2.1,0], ... % Location of sensor center in vehicle coord. [x,y](meters)
'Height', 1.1, ... % Sensor's mounting height from the ground (meters)
'Yaw', 0, ... % Yaw angle of sensor mounted on ego vehicle (deg)
'Pitch', 0, ... % Pitch angle of sensor mounted on ego vehicle (deg)
'Roll', 0, ... % Roll angle of sensor mounted on ego vehicle (deg)
'Intrinsics', cameraIntrinsics( ... % Sensor's intrinsic camera parameters
[800,800], ... % [fx, fy] in pixels
[320,240], ... % [cx, cy] in pixels
[480,640]), ... % [numRows, numColumns] in pixels
'MaxRange', 150, ... % Maximum detection range (meters)
'MaxSpeed', 50, ... % Maximum detectable object speed (m/s)
'MaxAllowedOcclusion', 0.5, ... % Maximum allowed occlusion [0 1]
'DetectionProbability', 0.9, ... % Probability of detecting a target (fraction)
'FalsePositivesPerImage',0.1); % Number of false positives per image (fraction)
![Page 28: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/28.jpg)
28
Sensor model for radar detection
radarSensor = radarDetectionGenerator(...'SensorIndex', 1, ... % Unique sensor identifier
'UpdateInterval', 0.05, ... % Required interval between sensor updates (sec)
'SensorLocation', [3.7,0], ... % Location of sensor center in vehicle coord. [x,y] in meters
'Height', 0.2, ... % Sensor's mounting height from the ground (meters)
'Yaw', 0, ... % Yaw angle of sensor mounted on ego vehicle (deg)
'Pitch', 0, ... % Pitch angle of sensor mounted on ego vehicle (deg)
'Roll', 0, ... % Roll angle of sensor mounted on ego vehicle (deg)
'FieldOfView', [90,4.5], ... % Total angular field of view for radar [az el] in deg
'MaxRange', 60, ... % Maximum detection range (meters)
'RangeRateLimits', [-100,40], ... % Minimum and maximum range rates [min,max] in m/s
'DetectionProbability',0.9, ... % Detection probability
'FalseAlarmRate', 1e-6, ... % Rate at which false alarms are reported
'AzimuthResolution', 12, ... % Azimuthal resolution of radar (deg)
'RangeResolution', 1.25, ... % Range resolution of radar (meters)
'RangeRateResolution', 0.5, ... % Range rate resolution of radar (m/s)
'RangeBiasFraction', 0.05); % Fractional range bias component of radar
![Page 29: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/29.jpg)
29
Generate Synthetic Data from Driving Scenario
birdsEyePlot
▪ coverageAreaPlotter
▪ plotCoverageArea
▪ detectionPlotter
▪ plotDetection
▪ laneBoundaryPlotter
▪ plotLaneBoundary
▪ trackPlotter
▪ plotTrack
generateSyntheic
▪ radarClusterDetection
▪ trackRadarDetections
▪ dataPacking
drivingScenario
▪ road
▪ vehicle
▪ actor
▪ actorProfiles
▪ path
▪ plot
▪ chasePlot
▪ radarDetectionGenerator
▪ visionDetectionGenerator
![Page 30: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/30.jpg)
30
Generate Synthetic Data from Driving Scenario
advance(s)Or, move(v,time)
generateDetections
updatePlots(s)
updateBirdsEyePlot
rDets = radarSensor(targets,time)vDets = visionSensor(targets,time)
Radar detection (MRR) Radar detection (LRR) Vision detection
trackRadarDetections Radar tracks
by multi-object tracker
targets = targetPoses(egoCar)Get target poses (positions &
orientations) w.r.t. ego vehicle
dataPacking Pack the detections into
a desired data format
radarClusterDetections(rDets) Radar cluster
Cluster
size
radarSensor = radarDetectionGeneratorvisionSensor = visionDetectionGenerator
![Page 31: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/31.jpg)
31
Sensor Fusion with Synthetic Data (baseline algorithm)
t3_3_fcwSensorFusionWithSyntheticData.m
![Page 32: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/32.jpg)
32
What causes the false positive?Pop Quiz
A: no fusion
(radar only track)
B: radar clutter
E: program bug
False Alarm
by radar track
C: inaccurate
tracking
D: too sensitive
warning
A: no fusion
(radar only track)
![Page 33: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/33.jpg)
33
Baseline sensor fusion algorithm (fcw_0_Baseline)
imu
radarSensor
visionSensor
lane
findNonClutterRadar
updateTracks
multiObjectTracker
assessThreat
findMostImportantObject
FCW
objectDetection
%%%%%% FCW Triggering Condition
if ttc<0 % Triggering if ttc<0 (target is approaching to host)
...
if ((rangeMagnitude < brakingDistance) &&...
(zone==zoneEnum.FCW))
activateFCW = true; % FCW activated
A: no fusion
(radar only track)
![Page 34: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/34.jpg)
34
%%%%%% UPDATED ALGORITHM %%%%%%%%%%%%%%%%%%%%%%%%%%%%%
if ttc<0 ...
&& (egoVelocity >= minThreatSpd) ...
&& (trackCounter>=minTrackCnt) ...
&& (classification~=classificationEnum.Unknown)
...if ((rangeMagnitude < brakingDistance) &&...
(zone==zoneEnum.FCW))
activateFCW = true; % FCW activated
Modified algorithm (fcw_3_PostProcessing)
imu
radarSensor
visionSensor
lane
findNonClutterRadar
updateTracks
multiObjectTracker
assessThreat
findMostImportantObject_v2
FCW
objectDetection
![Page 35: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/35.jpg)
35
Sensor Fusion with Synthetic Data (fcw_3_PostProcessing)
track with unknown
classification is
disqualified for “mio”
![Page 36: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/36.jpg)
36
Sensor Fusion with Synthetic Data (fcw_3_PostProcessing)
FCW detected
by pedestrian
detection
![Page 37: Case study: Vision and radar-based sensor fusion · 3 Automated Driving System Toolbox Case study: Vision and radar-based sensor fusion Synthesize data to further test the algorithm](https://reader036.vdocument.in/reader036/viewer/2022062606/5fe5ac40637a6708a2005e34/html5/thumbnails/37.jpg)
37
Automated Driving System Toolbox
Case study: Vision and radar-based sensor fusion
Synthesize data to
further test the algorithm
▪ Create driving scenario
▪ Add sensor detections
▪ Generate synthetic data
▪ Test the algorithm
Explore a baseline
sensor fusion algorithm
▪ Read logged vehicle data
▪ Visualize vehicle data
▪ Create multi-object tracker
▪ Implement forward
collision warning
Test the algorithm
with new data set
▪ Test the baseline algorithm
with new data set
▪ Tune algorithm parameters
▪ Customize the algorithm