thu huong nguyen - on road defects detection and classification
TRANSCRIPT
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
On road defects detection and classification
Thu Huong Nguyen, Aleksei Zhukov, The Long Nguyen
Irkutsk State Technical University
The 7th to 9th April 2016
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 1 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Outline
Introduction
Data Collection
Construction of Map of Defects
Defects Detection and Classification Method on Road Pavement
Conclusion
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 2 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Introduction
• In this talk proposes an automatic defect pavement detectionand classification system capable of identifying and retrievingpavement surface images containing block cracks, longitudinalcracks, potholes from a road pavement survey image database.
• The experimental results, achieved using images from Irkutstroads, are encouraging for the development of automaticpavement defects detection systems.
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 3 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Motivation
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 4 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Objective
• We proposes algorithms: Graph cuts method for imagessegmentation, Random forest for classification and imageprocessing algorithms for features extraction.
• Automatic learning methods are used, capable of learning theimage statistical features from texture variations of the roadbackground and of the defects areas.
• The features studied are Histogram, Histogram chain code,Moments-hull, shape of features.
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 5 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Main steps
1.To detect defect position (ROI).2.Defect is described by its features.3.To classify defect each using these different defect features suchas Chain Code Histogram, Hu-Moments, size of defectregion(width and length, area) and histogram of image.
Our approach
The following algorithms have been used: Graph cuts method forimage segmentation, Random Forests algorithm for dataclassification.
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 6 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Data Collection
We used two datasets:
1 Our own dataset include:• 500 images are collected by camera (Canon D100 16 mega
pixel).• Images are captured in conventional daylight condition.• Distance from camera to surface of road is 1m-1.2m.
2 SARA• More 700 images.• Collection by Center for Telecommunications and Multimedia,
INESC TEC, Portugal.
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 7 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Segmentation by Graph cuts method
• Graph cuts uses power optimization algorithm, which isapplied specifically to those models which employ amax-flow/min-cut optimization (other graph cuttingalgorithms may be considered as graph partitioningalgorithms).
• Finds strong local minima of our np-complete energy function.
• Graph-cuts have been around in computer vision for quitesome time (e.g. [Roy,ICCV98]).
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 8 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Segmentation by Graph cuts method
Figure: Example Graph cuts segmentation method
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 9 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Max-Flow Problem
Task : Maximize the flow from the sink to the source such that:
• The flow it conserved for each node
• The flow for each pipe does not exceed the capacity
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 10 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Ford Fulkerson algorithm (1956)
Input: Given a network G = (V ,E ) with flow capacity c , a sourcenode s, and a sink node t
Output: Compute a flow f from s to t of maximum value.f (u, v)← 0 for all edges (u, v)while there exists a path p from s to t in the residual network Gj
doFind cf (p) = min {cf (u, v) | (u, v) ∈ p} for eachedges(u, v) ∈ p do
f [u, v ] = f [u, v ] + cf (p);f [v , u] = f [v , u]− cf (p);
end
end
Algorithm 1: Ford Fulkerson algorithm (1956)
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 11 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Result of max flow
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 12 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Min-Cut Problem
Task : Minimize the cost of the cut
• Each node is either assigned to the source S or sink T
• The cost of the edge (i , j) is taken if (i ∈ S) and (j ∈ T )
Finding min-cut |C | =∑
e∈C we
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 13 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Result of Graph cut segmentation
Figure: Fig(a) Result of Graph cut segmentation method. Fig(b) Resultof Random forest algorithm
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 14 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Flowchart
End
Load model classification of
machine learning
Load road pavement image
database
Classification based on
RandomForest algorithm
Return type of defect road
pavement
Create features vector
Features extraction
Preprocessing image
Begin
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 15 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Random Forest algorithm
• Random forest (or random forests) is an ensemble classifierthat consists of many decision trees and outputs the class thatis the mode of the class’s output by individual trees.
• The method combines Breiman’s ”bagging” idea and therandom selection of features.
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 16 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Flowchart
Figure: Flow chart of Random Forest algorithm [Girish (2015)].
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 17 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Random Forest practical consideration
• Splits are chosen according to a purity measure:E.g. squared error (regression), Gini index or devinace(classification)
• How to select N?Build trees until the error no longer decreases
• How to select M?Try to recommend defaults, half of them and twice of themand pick the best.
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 18 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Training time, Correct rate and Error test of RandomForest classification algorithm
Random Forest 100 trees 50 trees 100 trees 100 treesdepth:2 depth:2 depth:5 depth:10
Training time(sec) 250 150 50 140
Correct rate (%) 91.45 80.5 93.29 96.66
MSE 0.393 0.516 0.366 0.3
Table: Training time, Correct rate and Error test of Random Forestclassification algorithm
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 19 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
Conclusion
• In this talk we suggested the novel approach for roadpavements defects automatic detection and classification.This method is based on the construction of an irregularlattice derived from the original image. The lattice iscomposed only by straight line segments.
• We also propose to use to Graph cut method, which improvequality of image segmentation. From this we can detectionpart of pavement defect - non defect.
• The classification algorithm - Random Forest was able tocorrectly classify all the images contained in the two first sets.In the test set simulating the real environment the achievedclassification results were 95,5%.
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 20 / 21
IntroductionData Collection
Construction of Map of DefectsDefects Detection and Classification Method on Road Pavement
ConclusionIrkutsk State Technical University
THANK YOU SO MUCH !!!
Nguyen Thu Huong, Aleksei Zhukov, The Long NguyenAIST 2016 On road defects detection and classification 21 / 21