Principal Component Analysis for Speed LimitTraffic Sign Recognition

Sergio Eduardo Perez-Perez, Sheila Esmeralda Gonzalez-Reyna*,Sergio Eduardo Ledesma-Orozco, Juan Gabriel Avina-Cervantes

Universidad de Guanajuato, Division de Ingenierias Campus Irapuato-Salamanca.Carretera Salamanca-Valle de Santiago km 3.5 + 1.8 km. Comunidad de Palo Blanco,

C.P. 36885. Salamanca, Gto., Mexico.email: {se.perezperez, se.gonzalezreyna, selo, avina}@ugto.mx

Abstract—Traffic Sign Recognition has recently becomea popular research field. The main applications includeAutonomous Driving Systems, Driver Assistance Systems,road sign inventory, to name a few. In this paper, a speedlimit traffic sign recognition system is proposed based onPrincipal Component Analysis (PCA) with preprocessingsteps, that help on perspective and extreme luminancevariation correction. The classification is performed by afeed-forward neural network, or Multi-Layer Perceptron(MLP). Experimental results present a classification accu-racy similar to some state of the art systems, but with amore compact scheme.

Keywords—Driver Assistance Systems, Traffic SignRecognition, Feature Extraction, Principal Component A-nalysis, Multi-Layer Perceptron.

I. INTRODUCTION

When driving a car, a driver needs to be informedabout what is happening in his environment, therefore,traffic signs play an important role. There are fourtypes of traffic signs: information, warning, prohibitionand obligation. Depending on their type, all signs havespecific shapes and colors so that they can be easilydistinguished from the background.

Traffic Sign Recognition (TSR) is a challengingproblem because road signs can be confused with thebackground of the scene, when objects have similar coloror shape. Traffic Sign Recognition (TSR) is commonlytreated as a two stage problem: detection and classifi-cation. The detection stage attempts to locate regionsof interest (RoI), i.e. the traffic signs, within complexscenes. Sometimes color information is used to segmentsigns, in color spaces like HSV [1], [2], HSI [3] andCIELab [4]. However color information is sensitive toweather and lighting conditions, making traffic signsegmentation unreliable when color is used.

The recognition stage aims to identify the traffic signitself. In recognition, different approaches have beenproposed. Thus, traffic signs have been characterized bydifferent features like Histograms of Oriented Gradients(HOG) [5], contrast normalized images [6], segmentedimages [2] and Local Contour Patterns (LCP) [7], toname a few. Classification tools for recognition includeSupport Vector Machines (SVM) [2], [8] and Convolu-tional Neural Networks (CNN) [9].

There are several important factors raising TSR chal-lenge: weather conditions and varying illumination affectvisibility, sunlight leads to lackluster colors, perspec-tive and disorientation modify shape information, partialocclusion due to pedestrians, trees and other vehiclesdiminish image information, detrimented signs look sig-nificatively different from good signs, among others.Figure 1 shows some of these factors.

In this paper, a Traffic Sign Recognition approachis proposed, based on Principal Component Analysis(PCA) for feature simplification and reduction and aMulti-Layer Perceptron (MLP) for classification. Thepreprocessing stage corrects luminance variations byhistogram adjusting and image perspective with a bilinearinterpolation scalling.

This document is organized as follows. In Section II,PCA methodology is explained. Experiments and resultsare deeply described in Section III and then discussed inSection IV.

II. METHODOLOGY

A. Luminance Correction

As earlier mentioned, real images are taken underdifferent lighting conditions, modifying therefore thecolor perception. This problem can be corrected by

978-1-4799-2370-0/13/$31.00 ©2013 IEEE

a) b) c)

d) e) f)

Figure 1. Factors that affect TSR systems: a) lackluster color, b)bright images, c) blurring, d) dark images, e) shadows, f) perspective.

a histogram adjusting for color enhancement. In thisexperiment, luminance mean µ and standard deviationσ are used.

The original equation for histogram adjusting to[0, 255] is

gi =fi −min(f)

max(f)−min(f)255, (1)

were f and g contain values in the range [0, 255].

Considering that an image can have both 0 and 255values and still look dark, histogram adjusting can beperformed using the mean luminance value µ and itsstrongest variations σ. The original equation must bechanged to (2), where min(f) will be replaced by µ−σ,and max(f) by µ+ σ.

g(p) =f(p)− (µ− σ)

2 ∗ σ255 (2)

B. Principal Component Analysis

Principal Component Analysis (PCA) decorrelates aset of data composed of many related variables. The aimof this method is to locate and extract the most importantinformation and express it as a set of new decorrelatedinformation.

Sirovich and Kirby [10] developed in 1990 a methodfor faces approximation and Turk and Pentland used itfor classification based on PCA [11].

Let P be a set of M images size N × N pix-els, each one represented as a N2-dimensional vector

a) b) c)

d) e) f)

Figure 2. Luminance correction. Original a) dark, b) bright and c)ideal images. Corrected d) dark, e) bright and f) ideal (non corrected).

~Xi, i = 1, 2, . . . ,M (i.e. N2 features). The training setP is therefore an input matrix of size N2 ×M , with amean vector ~µ defined as

~µ =1M

M∑i=1

~Xi. (3)

In order to center the data, the mean vector willbe subtracted from every vector ~Xi. The result is amatrix A with zero mean in every feature. Following,the covariance matrix is defined in (4), as

C =1

M − 1AA

T, (4)

where C has N2 × N2 elements and N2 < M . Theconvergence of the method depends on the images size.Therefore images should be small for this algorithm tobe feasible. There is another variant that works fine onlarge images [11], but it is out of the scope of this study.

The next step in the process, is to compute theeigenvalues λ and the eigenvectors ~V of the covariancematrix C. Finally, the new data projection is given by

~Y = QT ~X, (5)

where Q = {~V1, ~V2, . . . , ~Vk}. When dimensionality re-duction is intended, k < M .

Figure 3. Frequency of observations per class.

III. EXPERIMENTAL RESULTS

A. Traffic Signs Dataset

For the experiments presented here, the German Traf-fic Sign Recognition Benchmark (GTSRB) [12] wasused. This is a free database consisting in over 50, 000images for both training and testing processes, dividedin 43 classes. All samples are already split into trainingset and validation set. The traffic signs present strongvariations in illumination and color, and are taken underdifferent weather, rotation and perspective conditions. Inaddition, images have different sizes, varying from 15 to200 width and height pixels.

In this study, only speed limit traffic signs wererecognized. There are eight different classes for thesesigns, corresponding to 12, 780 samples for training and4, 170 signs for validation. Every image was scaled to afixed size of 28 × 28 pixels by a bilinear interpolation,and then converted to grayscale. In order to minimizevarying illumination and color conditions, the histogramwas adjusted with (2) for either extreme bright (meanabove 180 in a [0, 255] grayscale range) or dark images(mean below 65). Figure 2 shows corrected dark, andbright images.

Figure 3 shows the unbalanced dataset frequencyof observations per class. Both, the training and thevalidation sets have similar probability distributions persign.

B. Dimensionality Reduction

Once the datasets were properly adjusted and prepro-cessed, the algorithm of PCA was applied with the main

Figure 4. Amount of information per number of eigenvectors fordata reduction.

purposes of feature extraction and data simplification.

Figure 4 demonstrates the possibility of dimensional-ity reduction when PCA was applied. The initial amountof attributes per image is 784. However, the use of lesseigenvectors for data projection leads to a reduced datasetwith a minimal loss of information. Considering the useof 13 eigenvectors produce a training set with 90.07% ofremaining information. Similarly, with 33 eigenvectors,95.11% of knowledge is conserved, while 98.47% ofinformation is contained in 100 eigenvectors.

Several tests were performed for different amountof eigenvectors for dimensionality reduction. However,when 13 eigenvectors were used, the final classificationresults were not accurate (classification rate below 90%).Therefore, another test for classification varying thenumber of attributes was performed. Figure 5 shows thatincrementing the number of attributes, also incrementsthe classification performance, raising the stability forboth training and validation after 80 characteristics.

C. MultiLayer Perceptron Optimal Configuration

The MLP accuracy is proportional to the number ofhidden neurons. During this research, the optimal numberof hidden units was found by running a performancetest, where a new MLP was created, trained and testedusing a varying number of neurons in the hidden layer.The results for both training and validation datasets aredepicted in Fig. 6. When the number of neurons ofthe hidden layer was incremented beyond 60 units, weobserved no significative improvement.

Figure 5. Training and testing errors for varying number of attributesand fixed MLP hidden layers.

Figure 6. Performance for both training and validation sets.

The best results obtained for the final configurationare shown in Table I. The best classification accuracieswere obtained for those classes where there existed alarger number of samples. Recognition overall perfor-mance can be improved with the use of a different clas-sification algorithm were the frequency of observationsper class is considered, like k-NN or Bayes rule basedclassifiers.

There exists a similar scheme in the literature, pro-posed by Fleyeh and Davami et al [2]. Both systems usePCA for feature extraction, but they apply it on HSVcolor space segmented images. Our system outperformsFleyeh and Davami’s accuracy (97.9% reported), andno segmentation step is needed. This is an advantagesince even with HSV color space, colors can not be

Table I. PROPORTIONS OF TRUE CLASS ACCURACIES.

Class number Training Set (%) Test Set (%)1 91.9 86.72 99.0 96.13 99.3 97.14 98.2 94.05 99.7 92.76 98.9 91.97 98.5 87.38 99.1 90.0

average 98.9 93.1

accurately segmented in special cases, e.g. with shad-ows or when the sunlight has seriously affected colorperception (original red signals are now pink or evenyellow). Furthermore, in this paper a public larger dataset has been used, with 12,780 images for training and4,170 for testing purposes, while [2] used only 648proprietary images for both stages. In this experiment 80eigenvectors were used for accurate recognition whilein [2] only 20 eigenvectors could describe the trafficsigns, since they used binary images resulting fromsegmentation stage. Classification tools are similar forboth cases, since in our work a MLP was used and in[2] a SVM performed this task.

IV. CONCLUSIONS

In this study, traffic signs were accurately classified bythe use of the popular “eigenfaces” approach [11]. Takinginto account that traffic sign recognition works in realstructured and unstructured environments, the probleminvolves factors that seriously affect intelligent systemsperformance. In this paper, three main issues are consid-ered: extreme varying illumination, lackluster color andperspective. The first two conditions are corrected by theimage histogram adjusting, while the latter is accuratelyrectified by the image scaling to a fixed squared size.

Experimental results demonstrate the validity of theproposed approach, working with only 80 out of theoriginal 784 attributes, which represent the 10% of theoriginal amount of characteristics. The MLP used forclassification is conformed with 60 hidden units, achiev-ing a 98.9% of recognition accuracy.

The proposed approach achieves accurate results com-parable to those presented in [12] for speed limits, witha more compact scheme. The final performance can beimproved by the use of a classification tool that worksbetter with unbalanced datasets, like k-NN or Bayes rulebased classifiers.

The PCA with preprocessing system presented herepresents efficient results for multiclass problems wherethe number of classes is not so high. In the future,classification schemes for large, unbalanced, multiclassdatasets shall be implemented, in order to create a systemcapable to identify all 43 classes in the original GTSRBdatabase.

ACKNOWLEDGEMENTS

This work has been supported by the National Councilof Science and Technology of Mexico (CONACYT)under Grant number 329483/229696, and for Universidadde Guanajuato through PIFI-2012.

REFERENCES

[1] C. Paulo and P. Correia, “Automatic detection and classificationof traffic signs,” in Image Analysis for Multimedia InteractiveServices, 2007. WIAMIS ’07. Eighth International Workshopon, 2007, pp. 11–11.

[2] H. Fleyeh and E. Davami, “Eigen-based traffic sign recogni-tion,” IET Intelligent Transport Systems, vol. 5, no. 3, pp. 190–196, 2011.

[3] S. M. Bascon, J. A. Rodriguez, S. L. Arroyo, A. F. Ca-ballero, and F. Lopez-Ferreras, “An optimization on pictogramidentification for the road-sign recognition task using SVMs,”Computer Vision and Image Understanding, vol. 114, no. 3,pp. 373 – 383, 2010.

[4] G. Siogkas and E. Dermatas, “Detection, tracking and classifi-cation of road signs in adverse conditions,” in ElectrotechnicalConference, 2006. MELECON 2006. IEEE Mediterranean,2006, pp. 537–540.

[5] F. Zaklouta, B. Stanciulescu, and O. Hamdoun, “Traffic signclassification using K-d trees and random forests,” in NeuralNetworks (IJCNN), The 2011 International Joint Conferenceon, 2011, pp. 2151–2155.

[6] D. Ciresan, U. Meier, J. Masci, and J. Schmidhuber, “Multi-column deep neural network for traffic sign classification,”Neural Networks, vol. 32, no. 0, pp. 333 – 338, 2012.

[7] F. Parada-Loira and J. Alba-Castro, “Local contour patterns forfast traffic sign detection,” in Intelligent Vehicles Symposium(IV), 2010 IEEE, 2010, pp. 1–6.

[8] S. Lafuente-Arroyo, P. Gil-Jimenez, R. Maldonado-Bascon,F. Lopez-Ferreras, and S. Maldonado-Bascon, “Traffic signshape classification evaluation I: SVM using distance to bor-ders,” in Intelligent Vehicles Symposium, 2005. Proceedings.IEEE, 2005, pp. 557–562.

[9] P. Sermanet and Y. LeCun, “Traffic sign recognition with multi-scale convolutional networks,” in Neural Networks (IJCNN),The 2011 International Joint Conference on, 2011, pp. 2809–2813.

[10] M. Kirby and L. Sirovich, “Application of the Karhunen-Loeveprocedure for the characterization of human faces,” PatternAnalysis and Machine Intelligence, IEEE Transactions on,vol. 12, no. 1, pp. 103–108, 1990.

[11] M. Turk and A. Pentland, “Eigenfaces for recognition,” J.Cognitive Neuroscience, vol. 3, no. 1, pp. 71–86, Jan. 1991.[Online]. Available: http://dx.doi.org/10.1162/jocn.1991.3.1.71

[12] J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel, “Thegerman traffic sign recognition benchmark: A multi-class clas-sification competition,” in Neural Networks (IJCNN), The 2011International Joint Conference on, 2011, pp. 1453–1460.