towards the measurement of a 3d dynamic urban environment using an intelligent...
TRANSCRIPT
![Page 1: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/1.jpg)
Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicle
Huijing Zhao, [email protected] Lab of Machine Perception (MOE)Peking University
![Page 2: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/2.jpg)
Our final goalis to enhance driver safety, traffic data collection and
generating a digital copy of a dynamic urban environment.
We want to detect the moving objects in the surroundings, and track their states, such as speed, direction, and size, so that dangerous situations can be predicted.
We also want to generate a 3D copy of the dynamic urban scenery that contains both stationary objects, e.g. buildings, trees, road etc., and mobile objects, e.g. people, bicycles and cars.
We also want to locate our-self into the virtual space.
![Page 3: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/3.jpg)
![Page 4: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/4.jpg)
Three ProblemsThree Problems–– LocalizationLocalization–– MappingMapping–– Mobile objectsMobile objects’’
detection, tracking detection, tracking and classificationand classification
ObjectiveObjective–– Sensing a large dynamic environment in order for Sensing a large dynamic environment in order for
driving safety, 3D mapping and traffic data collection.driving safety, 3D mapping and traffic data collection.
Our SolutionOur Solution : : An Intelligent VehicleAn Intelligent Vehicle–– localization errors in urban arealocalization errors in urban area–– a mixture of data from both mobile and static objects a mixture of data from both mobile and static objects
![Page 5: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/5.jpg)
Laser ScannerL2,L3,L4
L1 L5
GPS IMU
Video Camera
![Page 6: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/6.jpg)
Approach 1Approach 1
Localization
EnvironmentVehicle pose
Map(Static)
MovingObject
Advantage:vehicle’s pose estimation and environmental perception are conducted by individual modules. Algorithms are straight forward and mature.Disadvantage: the accuracy of vehicle’s pose estimation is heavily dependant on positioning sensors. Difficult to detect slow motion objects, such as people, due to pose error.
Laser Scanner, Camera
GPS, IMU, VMS
![Page 7: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/7.jpg)
Approach 2Approach 2
Advantages:using map to improve the accuracy and efficiency of both localization and moving object detection. Algorithms are simple.Disadvantage:need a reliable map, which is a strong limit to applications.
Localization
EnvironmentVehicle pose
Map
Map
GPS, IMU, VMS
Laser Scanner, Camera
EnhancedMap
MovingObject
![Page 8: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/8.jpg)
Our Try !Our Try !
SLAM in Dynamic Environment
Environment
3D Map(Static & Mobile)
Laser Scanner, Camera
GPS, IMU, VMS
Vehicle Pose
2D Map(static)
2D Mobile Object
Advantage:vehicle pose estimation, mapping, moving object detection and tracking are treated as a simultaneous optimization problem.
Many possibilities !Disadvantage:rely on software processing. Algorithm is complicated. Computation cost is high.
We try to make a framework !
![Page 9: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/9.jpg)
GPS+IMU
SLAM with Object Tracking and Object Tracking and ClassificationClassification
Horizontal Laser Scanner
![Page 10: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/10.jpg)
![Page 11: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/11.jpg)
people
group
car
bicycle
moving
groupseed
Objects
Laser Points
Classification Results
![Page 12: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/12.jpg)
ExperimentalCourse
Length:4.5kmTime:15min
StartEnd
西門
Peking Univ.
Final Final MapMap
![Page 13: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/13.jpg)
A map of static & mobile objectsA map of static & mobile objects
peoplegroupbicyclecarHoststatic
![Page 14: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/14.jpg)
peoplegroupbicyclecarHoststatic
A map of static & mobile objectsA map of static & mobile objects
![Page 15: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/15.jpg)
Our Try !Our Try !
SLAMin Dynamic Environment
Environment
3D Map(Static & Mobile)
Laser Scanner, Camera
GPS, IMU, VMS
Vehicle Pose
2D Map(static)
2D Mobile Object
Advantage:vehicle pose estimation, mapping, moving object detection and tracking are treated as a simultaneous optimization problem.
Many possibilities !Disadvantage:rely on software processing. Algorithm is complicated. Computation cost is high.
We try to make a framework !
![Page 16: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/16.jpg)
Vertical Laser Scanning
3D Mapping
![Page 17: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/17.jpg)
time
lase
r sca
n
A Piece of Range ImageA Piece of Range Image
![Page 18: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/18.jpg)
![Page 19: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/19.jpg)
![Page 20: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/20.jpg)
![Page 21: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/21.jpg)
![Page 22: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/22.jpg)
![Page 23: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/23.jpg)
![Page 24: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/24.jpg)
![Page 25: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/25.jpg)
![Page 26: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/26.jpg)
Colored by Surface Segments
pedestrians
![Page 27: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/27.jpg)
pedestrians
![Page 28: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/28.jpg)
Slant Laser Scanning
3D Mapping
![Page 29: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/29.jpg)
Video Camera
Horizontal Laser Scanning
Laser + Video
![Page 30: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/30.jpg)
Previous IVs
- Univ. of Tokyo
- Asia Aerial Survey Co.
- Mazda Motor Co.
![Page 31: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/31.jpg)
GPS(G12)
Ladybug2
Laser Scanner
Laser Scanenr
Intersense IMU
GPS(Ublox)
PC
Intelligent Cart for 3D Acquisition
Battery
![Page 32: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/32.jpg)
L1+L2+L3→C1
L1+L2+L3→C3
L1+L2+L3→C2
L1+L2+L3
![Page 33: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/33.jpg)
Automated 3D Modeling Semi-Automated Extraction
Data Fusion with Geo-databaseAutomated 3D Modeling
![Page 34: Towards the Measurement of a 3D Dynamic Urban Environment using an Intelligent Vehicleposs.pku.edu.cn/Data/SIAT08.pdf · 2008-12-15 · a 3D Dynamic Urban Environment using an Intelligent](https://reader031.vdocument.in/reader031/viewer/2022041610/5e373627ca21b1128e33db09/html5/thumbnails/34.jpg)