![Page 1: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/1.jpg)
End-to-end Lane Detection through Differentiable Least-Squares Fitting
Wouter Van Gansbeke, Bert De Brabandere, Davy Neven, Marc Proesmans, Luc Van Gool
arXiv:1902.00293v3 [cs.CV] 5 Sep 2019
ECE 285 – Autonomous Driving Systems
Presented by – Anirudh Swaminathan – April 23, 2020
![Page 2: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/2.jpg)
Why Lane Detection?
Detecting lanes is important because:-
Position vehicle within the lane
Plan future trajectory, lane departures
2
![Page 3: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/3.jpg)
Lane Detection Background
Previous methods before this paper:-
Two step pipelines
First step -> segment lane line markings
Second step -> fit a lane line model to post-processed mask
3
![Page 4: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/4.jpg)
2-stage examples
Classical SIFT[20] / SURF[2] for feature extraction
RANSAC / spline / polynomial for parameters of best fitting model
Deep Learning Based Instance Segmentation – LaneNet [24]
Curve fitting mostly same
[24] - Towards End-to-End Lane Detection: an Instance Segmentation ApproachDavy Neven, Bert De Brabandere, Stamatios Georgoulis, Marc Proesmans, Luc Van Gool ESAT-PSI, KU LeuvenarXiv:1802.05591v1 [cs.CV] 15 Feb 2018[2] H. Bay, T. Tuytelaars, and L. Van Gool. Surf: Speeded up robust features. Proceedings of the European Conference on Computer Vision, 2006[20] D. G. Lowe. Object recognition from local scale-invariant features. In Proceedings of the IEEE International Conference on Computer Vision, 1999.
4
![Page 5: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/5.jpg)
Objective of the Paper
End-to-end manner
Directly regress lane parameters
5
![Page 6: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/6.jpg)
Motivation
Why single step?
Parameters not optimized for true task True task – estimating lane curvature parameters
Proxy task – Segmenting lane markings
Prevents instabilities in curve fitting 2 step –> outliers
End-to-end -> implicitly learn features to prevent instabilities
6
![Page 7: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/7.jpg)
Methodology
Key Idea -> Integrate curve fitting step as a differentiable in-network optimization step
Deep Network for the feature extraction step
Key Idea -> A geometric loss function for the network
7
![Page 8: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/8.jpg)
Framework
The framework consists of 3 main modules:-
Deep network to generate weighted pixel coordinates
Differential weighted least squares fitting module
Geometric Loss Function
8
![Page 9: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/9.jpg)
Example Architecture – Figure 1 from the paper 9
![Page 10: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/10.jpg)
Generating Weighted Pixel Coordinates
First Module of network
Normalized Coordinates -> x map and y map
Each coordinate -> weight w
Feature map -> same spatial dimensions as that of input image
10
![Page 11: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/11.jpg)
Feature Maps
Non-negative weights
Width – w, height – h; m = w * h
M triplets generated – (x, y, w)
One feature map for each lane
11
![Page 12: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/12.jpg)
Example Architecture – Figure 1 from the paper 12
![Page 13: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/13.jpg)
Weighted Least Squares Layer
M triplets (x, y, w) -> weighted points in 2D space
Fit curve
Module output -> n parameters of best-fitting curve
13
![Page 14: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/14.jpg)
Background - Least Squares Fitting
𝑋𝑋𝑋𝑋 = 𝑌𝑌;𝑋𝑋 ∈ 𝑅𝑅𝑚𝑚𝑚𝑚𝑚𝑚 ;𝑋𝑋 ∈ 𝑅𝑅𝑚𝑚×1;𝑌𝑌 ∈ 𝑅𝑅𝑚𝑚×1
X is input, 𝑋𝑋 are parameters, and Y is output
Least Squares -> 𝑋𝑋 = 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎 ||𝑋𝑋𝑋𝑋 − 𝑌𝑌||2
Normal Equation -> 𝑋𝑋 = 𝑋𝑋𝑇𝑇𝑋𝑋 −1𝑋𝑋𝑇𝑇𝑌𝑌
14
![Page 15: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/15.jpg)
Background – Weighted Least Squares
Least squares extended
𝑊𝑊 ∈ 𝑅𝑅𝑚𝑚×m ; Diagonal matrix -> weights for each pair of observations
𝑋𝑋 = 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎 ||𝑊𝑊12(𝑋𝑋𝑋𝑋 − 𝑌𝑌)||2
𝑋𝑋𝑇𝑇𝑊𝑊𝑋𝑋𝑋𝑋 = 𝑋𝑋𝑇𝑇𝑊𝑊𝑌𝑌
𝑋𝑋 = 𝑋𝑋𝑇𝑇𝑊𝑊𝑋𝑋 −1𝑋𝑋𝑇𝑇𝑊𝑊𝑌𝑌
15
![Page 16: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/16.jpg)
Backprop through the layer
Equations involve differentiable matrix operations
Calculate the derivative of 𝑋𝑋 wrt W
Refer to [10] to derive backprop
M. B. Giles. An extended collection of matrix derivative results for forward and reverse mode automatic differentiation.Technical report, University of Oxford, 2008.
16
![Page 17: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/17.jpg)
Example Architecture – Figure 1 from the paper 17
![Page 18: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/18.jpg)
Geometric Loss Function - precursor
Usually, L2 loss used for curve fitting
Here, 𝑋𝑋𝑖𝑖 and �𝑋𝑋𝑖𝑖 -> generated and groundtruth curve parameters
18
![Page 19: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/19.jpg)
Geometric Loss Function
Lane Detection -> geometric interpretation
Minimize squared area between predicted curve and ground truth curve
19
![Page 20: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/20.jpg)
Geometric Meaning
20
![Page 21: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/21.jpg)
Geometric Loss Function – Parabola
This paper -> lane curves parabolic
𝑦𝑦 = 𝑋𝑋0 + 𝑋𝑋1𝑥𝑥 + 𝑋𝑋2𝑥𝑥2; Δ𝑋𝑋𝑖𝑖 = 𝑋𝑋𝑖𝑖 − �𝑋𝑋𝑖𝑖
21
![Page 22: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/22.jpg)
Optional Transformations
Weighted coordinates -> another reference frame
Use fixed transformation matrix H
Lane line -> better as parabola from top-down/ortho view(BEV)
22
![Page 23: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/23.jpg)
Experiment – Ego Lane Detection
Ego lane -> the current lane of the vehicle
Two lane marking -> one left and one right
Parabola -> upto fixed distance t from car
Overall error = average over 2 lanes, and average over images
23
![Page 24: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/24.jpg)
Dataset
TuSimple Dataset [29]
Manually select and clean up the annotations of 2535 images
Filter out images where ego-lane cannot be detected unambiguously
20% images -> validation set
Not include images of single temporal sequence in both train and val sets
[29] TuSimple. Tusimple benchmark, 2017. 24
![Page 25: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/25.jpg)
Annotation
Ground truth curve parameters -> parabola
Draw curve of fixed thickness as dense label
25
![Page 26: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/26.jpg)
Baseline – Cross-entropy training
Training Segmentation
Per-pixel binary cross-entropy loss
Testing Segmentation mask generated
Parabola fitted in least squares sense
26
![Page 27: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/27.jpg)
End-to-end training
ERFNet [28] -> network architecture
350 epochs; 1 GPU; 256*512 resolution; batch size 8
Adam[19] with LR 10−4
PyTorch [26][28] E. Romera, J. M. Alvarez, L. M. Bergasa, and R. Arroyo. Efficient convnet for real-time semantic segmentation. In IEEE Intelligent Vehicles Symposium, pages 1789–1794, 2017.
[19] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations, 2015.
[26] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. De- Vito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer. Automatic differentiation in pytorch. In NIPS-W, 2017. 27
![Page 28: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/28.jpg)
Example Architecture – Figure 1 from the paper 28
![Page 29: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/29.jpg)
Detour -ERFNet
29
![Page 30: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/30.jpg)
ERFNet
Semantic Segmentation
Typical Encoder – Decoder architecture
Last layer -> adapted to output 2 feature maps
One for each ego lane
Transform weighted coordinates using fixed H to top-down view
[28] E. Romera, J. M. Alvarez, L. M. Bergasa, and R. Arroyo. Efficient convnet for real-time semantic segmentation. In IEEE Intelligent Vehicles Symposium, pages 1789–1794, 2017 30
![Page 31: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/31.jpg)
Results - Quantitative
31
![Page 32: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/32.jpg)
Result – Training curves
32
![Page 33: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/33.jpg)
Qualitative Results 33
![Page 34: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/34.jpg)
Analysis
Lower error than cross-entropy method
Convergence slower -> supervision signal weaker
Generated weight maps look like segmentation maps the network eventually discovers that the most consistent way to satisfy the loss function is to focus
on the visible lane markings in the image, and to map them to a segmentation-like representation in the weight maps.
34
![Page 35: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/35.jpg)
Further Experiments – Multi-lane detection
4 lanes total -> ortho-view
Line prediction branch; horizon prediction branch
Horizon prediction branch -> regression -> estimate horizon
Line prediction branch -> whether lane is present or not
35
![Page 36: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/36.jpg)
Architecture
36
![Page 37: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/37.jpg)
Architecture details
Side branches -> 4 conv layers -> each 3x3
Then max pool -> FC layer
Losses for 3 tasks -> combined linearly
37
![Page 38: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/38.jpg)
Dataset
3626 images
20% validation set
2782 test set images
38
![Page 39: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/39.jpg)
Comparison
ERFNet without backprop through least squares layer -> baseline
[25] Spatial CNN
[24] -> Instance Segmentation approach
[24] D. Neven, B. De Brabandere, S. Georgoulis, M. Proesmans, and L. Van Gool. Towards end-to-end lane detection: an instance segmentation approach. arXiv:1802.05591, 2018.
[25] X. Pan, J. Shi, P. Luo, X. Wang, and X. Tang. Spatial as deep: Spatial cnn for traffic scene understanding. In AAAI, 2018. 39
![Page 40: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/40.jpg)
Results Quantitative
40
![Page 41: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/41.jpg)
Results -Qualitative
41
![Page 42: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/42.jpg)
Analysis
Improve upon baseline by 0.7%
Faster than benchmarks in test time -> no post-processing required
71 fps on NVIDIA 1080Ti
42
![Page 43: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/43.jpg)
ADVANTAGES
Optimized for true task -> prevents instabilities in curve fitting
Offers degree of interpretability Generated weight maps -> segmentation-like
Can be inspected and visualized
Geometry aware criterion is loss function
Handle large variance, faded lane markings
Moves complexity from post-processing to network -> one-shot fitting
43
![Page 44: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/44.jpg)
DISADVANTAGES
Loss function -> more complicated for higher order curves
Fixed transformation H to ortho-view If ground plane is different (ex. Sloping uphill), then bad lane parameters in test time
Local minimum possible – author Vanishing point in horizon/left corner of image features -> good curve -> no improvement
Multi-lane -> fixed number of maps -> pre-defined order Lane changes hard; Order ambiguous
Instance segmentation -> not subject to specific order
Quantitative results -> comparatively worse from slide 40
44
![Page 45: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/45.jpg)
KEY TAKEAWAYS
Including differentiable in-network optimization step.
Geometric Loss function relevant to the task
45
![Page 46: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/46.jpg)
Question to the class
Why do you think that the loss in the Least Squares layer is only back-propagated to the coordinate weights only, and not to the coordinates themselves?
46
![Page 47: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/47.jpg)
THANK YOU!
![Page 48: End-to-end Lane Detection through Differentiable …cvrr.ucsd.edu/ece285sp20/files/diff-ls-lane.pdfEnd-to-end Lane Detection through Differentiable Least-Squares Fitting Wouter Van](https://reader035.vdocument.in/reader035/viewer/2022071014/5fcc835186e93d5ec6560930/html5/thumbnails/48.jpg)
Questions?