Automation in Construction 44 (2014) 84–91

Contents lists available at ScienceDirect

Automation in Construction

j ourna l homepage: www.e lsev ie r .com/ locate /autcon

Evaluation of artificial intelligence tool performance and uncertainty forpredicting sewer structural condition

Vitor Sousa ⁎, José P. Matos, Natércia MatiasDECivil, Instituto Superior Técnico, Universidade de Lisboa, Av. Rovisco Pais, 1049-001 Lisbon, Portugal

⁎ Corresponding author. Tel.: +351 218418381.E-mail address: vitor.sousa@tecnico.ulisboa.pt (V. Sou

http://dx.doi.org/10.1016/j.autcon.2014.04.0040926-5805/© 2014 Elsevier B.V. All rights reserved.

a b s t r a c t

a r t i c l e i n f o

Article history:Received 26 November 2013Received and Revised form 26 March 2014Accepted 5 April 2014Available online xxxx

Keywords:Artificial neural networksOptimizationSewer structural conditionSupport vector machines

The implementation of a risk-informed assetmanagement systemby awastewater infrastructure utility requiresinformation regarding the probability and the consequences of component failures. This paper focuses on theformer, evaluating the performance of artificial intelligence tools, namely artificial neural networks (ANNs)and support vector machines (SVMs), in predicting the structural condition of sewers. The performance ofthese tools is compared with that of logistic regression on the case study of the wastewater infrastructures ofSANEST— Sistema de Saneamento da Costa do Estoril (Costa do Estoril Wastewater System). The uncertainty asso-ciated to ANNs and SVMs is quantified and the results of a trial and error approach and the use of optimizationalgorithms to develop SVMs are compared. The results highlight the need to account for both the performanceand the uncertainty in the process of choosing the best model to estimate the sewer condition, since the ANNspresent the highest average performance (78.5% correct predictions in the test sample) but also the highestdispersion of performance results (73% to 81% correct predictions in the test sample), whereas the SVMs havelower average performance (71.1% without optimization and 72.6% with the parameters optimized using theCovariance Matrix Adaptation Evolution Strategy) but little variability.

© 2014 Elsevier B.V. All rights reserved.

1. Introduction

During the last decades there has been a trend to develop and imple-ment formal assetmanagement systems forwastewater infrastructures.These asset management systems have been gradually evolving fromreactive to proactive stances and their scope has broadened significantlyto the point of being considered the central element in the technicalmanagement of water and wastewater infrastructures [1–6].

One of the first proactive-based asset management systemswas de-veloped by the Water Research Centre, in which the defects observedduring Closed-Circuit Television (CCTV) inspections were rated inorder to obtain a classification for the sewer condition. Originally, theapproachwas used only tomanage the critical sewers, that is, managingproactively the sewers that entail very high economic consequences incase of failure, and reactively the remaining [7]. However, due to thegrowing awareness of the non-economic dimension of sewer failures,the application of this approachwas expanded to the non-critical assets[8]. This approach has been implemented worldwide, with adjustmentsintroduced by national institutions and local municipalities [9,10].

More complex and comprehensivemodels were also developedwiththe purpose of optimizing decisions and prioritizing interventions, bytaking into account hydraulic, environmental, social and economic con-strains (MARESS— [11]; RERAUVIS— [12]; CARE-S— [13]). Additionally,

sa).

there is a growing demand for conducting periodical sewer inspectionsin order to complywith legal requirements (e.g., inGermany,most Statesrequire the inspection of the total sewer network once in ten years). Thishas led to the development of models for assisting decisions regardingwhich sewers are to be inspected (AQUA-WertMin — [14]; SCRAPS —

[15–17]).If there is the need to support rehabilitation or inspection decisions,

the models developed to predict interventions in sewer systems shouldinclude amodule for estimating the evolution of the sewer condition [18].Most of such models either require information that is not always avail-able and is usually not easily obtained (e.g., soil aggressiveness) or arebased on statistical analysis. The traditional statistical models requirethe previous knowledge of the function/structure that better representsthe effect of the different sewer characteristics (e.g., material, diameter,age) on its performance. This is a major drawback because the effect ofthe interactions between different sewer characteristics and how theyrelate between them and with the sewer performance is not knownneither easy to determine (e.g. diameter and age interact as sum, a prod-uct, a power, a logarithmor any othermathematical formulation). Conse-quently, in most cases, the statistical models consider only one (usuallythe age) or two of these characteristics (usually the age in combinationwith one of the others). Artificial intelligence tools are an alternativethat can be used in classification and pattern identification problemssuch as this (e.g. [19]). The present paper discusses the use of artificialneural networks (ANNs) and support vector machines (SVMs) to esti-mate the condition of sewers, being the results compared with those of

85V. Sousa et al. / Automation in Construction 44 (2014) 84–91

logistic regression. The models were fitted using data from the periodicCCTV inspection program that has been implemented in the SANEST —

Sistema de Saneamento da Costa do Estoril (Costa do Estoril WastewaterSystem) since 2005.

2. Sewer condition modeling

2.1. Approaches

Depending on the type of output provided, themodels for determin-ing sewer conditions can be classified as deterministic or stochastic[20,21]. The former provide estimates in an absolute and exact formatwhile the latter comprise some form of uncertainty/variability quantifi-cation associated with the estimate. The deterministic models are themost common, but in a risk-informed context it is important to evaluatethe uncertainty of the estimates in order to determine the best options.Themodels for predicting sewer conditions can be further classified intoempirical or mechanistic [20,21]. The empirical models use statisticaltools and methods to obtain relations between known variables andthe sewer condition based on historical records. These models entailan implicit assumption that the pattern of deterioration will remainthe same in the future. The mechanistic models seek to represent thephysical, chemical and/or biological phenomena that take place withinthe sewers and are relevant to explain their condition. These models re-quire information and data that is not generally available or cannot beeasily obtained. There is another class of models (expert-basedmodels)that rely on expert opinion to define the relation between the inputsand the outputs. Historically, these models (e.g. MOSIMO — [22,23];SCRAPS — [15–17]) have been less explored and are seldom put intopractice, although for situations of information scarcity they may bethe only viable option for estimating the sewer condition.

Empirical models are the most widely studied. These models, alsocalled statistical models, are built from statistical analyses of operationand maintenance failure records (e.g., clogging; collapse) or conditionclassification based on inspection data (e.g., operational or structuralcondition classification using a rating protocol). Two main categoriesof empirical models can be identified [19,24]: i) function-basedmodels;and ii) data-based models. Both model categories rely on fittingobserved data. However, for function-based models the mathematicalexpressions relating the inputs with the outputs are pre-defined at theoutset. In this case, the fitting operation seeks to determine the coeffi-cients of the functions that minimize the error between the observedand the estimated outputs. In the data-based models there is no pre-defined expression relating the inputs with the outputs such that thefitting operation simultaneously adjusts the relation between the inputsand the outputs and the relative weight of each input. Table 1 resumesthe main classes and types of function-based and data-based modelsthat have been used for estimating sewer condition.

The references presented in Table 1 are specific to sewer systems.There are also similar studies on water supply networks that use

Table 1Empirical models used for estimating the condition of sewers.

Category Class

Function-based Deterministic

Stochastic

Data-based Artificial intelligence

Genetic programing

alternative empirical models which could be adapted for sewer systems(e.g., [55–58]).

The present paper focuses on the application of ANNs and SVMs,which are machine learning techniques. These techniques have theability to learn the patterns of the underlying process from past dataand generalize the relationships between input and output data, beingable to predict or estimate an output given a new set of input variablesfrom the vicinity of the training domain. Some brief details on thesetechniques are providednext, alongwith a reviewon logistic regression.

2.2. Logistic regression (LR)

The logistic regression (LR) is a type of generalized linearmodel thatextends the linear regression by linking the range of real numbers to the0–1 range, allowing approaching situationswhere the response variableis qualitative and takes on only two possible values [59]. LR assumes theresponse to be a Bernoulli random variable and provides a predictionof the chance that the response will assume one of the categoricalresponse levels [60]. Considering that the value 1 represents theevent of interest, the relation between the probability of it happening(P[yk = 1]) and the predictors (xi) is given by a logistic model:

P yk ¼ 1½ � ¼ pk ¼1

1−e− βokþ

Xni¼1

βikxi

! ð1Þ

where pk is the probability that the kth case experiences the event of in-terest;βik is the value of the ith regression coefficient of the kth case; xi isthe ith predictor; and n is the number of predictors. If pk ≥ 0.5 the casefalls into class 1, otherwise it falls into class 0. The model assumes thatthe predictors are not highly correlated since, as in the linear regression,this can cause problems with the estimation of the coefficients [61].Nonetheless, LR is regarded to be robust even when the assumptionsare not fully met [62]. Usually, the coefficients βik are obtained usingthe maximum likelihood estimates [63].

The logistic function provides a mean for mapping from the predic-tor domain onto the [0, 1] interval [64]. Other commonly used functionis the normal probability distribution, resulting in the so called probitregression model. Another alternative, the multinomial logistic regres-sion, expands the LR for response variables with m N 2 classes. In thiscase, there will bem − 1 complementary link functions.

2.3. Artificial neural networks (ANNs)

Since Warren S. McCulloch and Walter H. Pitts proposed the firstartificial model for a biological neuron of human brain in 1943, numer-ous methods for building neuro-inspired computational models havebeen proposed and investigated [65]. ANNs are defined not only bytheir use of artificial neurons, but also by the network structureconnecting them. In addition, there are other important features to be

Type References

Linear regression [25–27]Non-linear regression [28,29]Survival function [14,24,30,31]Ordinal regression [24,32–35]Markov chains [19,28,29,36–40]Semi-Markov chains [24,41,42]Discriminant analysis [19,24]Artificial neural networks — ANNs [19,24,43–45]Fuzzy set [46–49]Case based reasoning — CBR [50]Support vector machines — SVMs [51]Evolutionary polynomial regression — EPR [52–54]

86 V. Sousa et al. / Automation in Construction 44 (2014) 84–91

accounted for such as bias, the fitness functions and the training algo-rithms. ANNs are defined as a type of information processing systemthat resembles the human brain [66] and their types vary from thosewith only one or two layers of single direction logic, to complex multi-input with many directional feedback loops and layers. Among thevarious types of ANNs, the multilayer perceptron networks (MLPnetworks) and the radial basis function networks (RBF networks) aretwo of the most commonly used as universal approximators. Both arefeed-forward ANNs, with the MLP networks having a seemingly non-parametric architecture while the RBF networks being non-linearparametric models based on the combinations of the Gaussian functions.

MLP networks are fully connected feed-forward networks com-prised of three ormore layers. The input layer receives input data exter-nal to the neural network and includes a neuron for each real-valueinput and a neuron for each nominal-class or ordinal-class input. Theoutput layer holds the “response” of the MLP, having one neuron forreal-value outputs and a neuron for each of nominal-class or ordinal-class outputs. In between there can be any number of hidden layerswith an arbitrary number of neurons, but usually most MLP networkscomprise only one or two hidden layers. Usually each neuron addsand transforms the contributions of the neurons from the previouslayerwhich are connected to it. In this case, the total input to the neuronk in layer l (sk(l)) is simply the weighted sum of the separate outputsfrom each of the connected neurons in the previous layer (yj(l − 1))plus a bias or offset term θk(l):

sk lð Þ ¼Xnj

wjk lð Þyj l−1ð Þ þ θk lð Þ ð2Þ

where wjk(l) is the weight of the connection from neuron j in thelayer l − 1 to neuron k in the layer l; and n is the number of neurons in

Table 2Dataset statistics.

Material/diameter Sewers [no.] Total length [m] Average age [yea

VC (1) 134 4370.50 54.55200 7 186.13 45.00250 15 389.41 58.13300 38 1232.85 49.74350 69 2484.68 58.17400 1 42.23 39.00

PC (2) 53 1408.70 29.85315 1 51.26 30.00500 52 1357.44 29.85

PVC (3) 348 12682.20 11.53200 3 80.44 8.00250 59 2291.46 10.37315 38 957.03 12.39400 112 4347.90 11.59500 73 2868.81 12.26630 27 1132.64 10.37700 30 915.38 12.00800 6 88.54 12.00

HDPE (4) 122 4102.04 9.84360 38 1206.47 10.00400 4 111.03 9.75450 4 217.33 9.00500 66 2154.48 9.92600 10 412.73 9.00

C-PP (5) 60 1771.99 9.65315 26 908.06 9.96400 4 122.89 12.00500 29 713.70 9.03630 1 27.34 10.00

C-PVC (6) 28 1033.74 4.42350 7 165.00 6.20400 21 868.74 4.00

Total 745 25369.17 19.92

the layer l − 1. The contribution for positive wjk(l) is considered as anexcitation and for negative wjk(l) as an inhibition. The input (sk(l))is then translated into an output by an activation function that is,usually, defined for a layer and not for each neuron (Fl). Often, theactivation function is a non-decreasing function of the total input of theunit [67]:

yk lð Þ ¼ Fl sk lð Þ½ �: ð3Þ

In most cases, some sort of threshold function is used. This can be ahard limiting threshold function (a sign function), a linear or semi-linear function (identity, exponential), or a smoothly limiting threshold(an s-shaped function such as sigmoid or hyperbolic tangent) [68]. If theactivation function is the identity function, this model resembles a mul-tiple linear regression. However, while the multiple linear regressionhas a closed form solution for its regression coefficients, by virtue ofits multiple layers and potentially non-linear activation functions theneural network must resort to an iterative process [69]. The numberof hidden layers, the number of neurons in each hidden layer and thechosen activation functions at each layer are some of the parametersthat need to be optimized when creating a MLP network.

RBF networks have three layers [70]: i) an input layer; ii) a hiddenlayer; and iii) a summation layer. The input layer is similar to the MLPnetworks. The hidden layer has a variable number of neurons, eachconsisting of a radial basis function centered on a point with as manydimensions as there are predictor variables. A hidden neuron computesthe Euclidean distance of the test case from the neuron's center pointand then applies the RBF kernel function to this distance using thespread values. In the summation layer the value coming out of a neuronin the hidden layer (ϕj(x)) is multiplied by a weight (wjk) associated

rs] Average depth [m] Average slope [%] Average length [m]

2.52 2.14 32.622.68 1.32 26.592.41 1.09 25.961.98 2.95 32.442.82 1.83 36.012.31 1.11 42.232.47 2.08 26.582.73 2.09 51.262.47 2.08 26.102.88 1.72 36.442.19 7.22 26.812.34 4.14 38.842.46 0.90 25.192.98 1.75 38.823.03 0.87 39.303.12 0.81 41.953.47 0.53 30.513.47 0.34 14.763.53 1.23 33.623.70 0.96 31.753.31 1.68 27.762.07 1.26 54.333.76 1.50 32.642.08 0.27 41.273.02 1.51 29.534.42 2.83 34.933.23 0.26 30.721.72 0.46 24.613.40 2.71 27.343.87 1.24 39.762.83 2.71 33.004.12 0.89 41.372.94 1.71 34.14

Table 3LR models detailed statistics.

Predictors Regression coefficients(βi)

Wald statistics Significance(p-value)

Odds ratio(OR)

Complete modelMaterial 26.81 0.001 −0.83 0.21 0.65 0.442 −1.44 1.50 0.22 0.243 1.13 2.58 0.11 3.094 0.31 0.19 0.66 1.365 0.66 0.78 0.38 1.93

Diameter 0.00 0.33 0.57 1.00Length 0.02 8.84 0.00 1.02Depth 0.41 16.68 0.00 1.50Slope −0.09 0.65 0.42 0.91Age 0.07 5.20 0.02 1.08Velocity 0.19 0.75 0.39 1.21Constant −4.26 23.88 0.00 0.01

Reduced modelMaterial 26.51 0.001 −0.77 0.19 0.67 0.462 −1.35 1.38 0.24 0.263 1.11 2.51 0.11 3.034 0.32 0.21 0.65 1.385 0.62 0.69 0.40 1.86

Length 0.02 10.17 0.00 1.02Depth 0.41 17.66 0.00 1.51Age 0.07 5.25 0.02 1.08Constant −4.31 30.24 0.00 0.01

87V. Sousa et al. / Automation in Construction 44 (2014) 84–91

with the neuron and passed to the summation which adds up theweighted values and presents this sum as the output of the network:

yk xð Þ ¼XMj¼0

wjkϕ j xð Þ ð4Þ

with

ϕ0 xð Þ ¼ 1 and ϕ j xð Þ ¼ exp −x−μ j

��� ���22σ2

j

264

375for j N 0 ð5Þ

where μj are the coordinates of the center of each hidden-layer RBFfunction; σj are the radius (spread) of each RBF function in eachdimension; andM is the number of neurons in the hidden layer.

RBF networks have a number of advantages overMLPs. Namely, theycan model any nonlinear function using a single hidden layer (whichremoves some design-decisions about numbers of layers) and the sim-ple linear transformation in the output layer can be optimized fullyusing traditional linear modeling techniques (which are fast and donot suffer from problems such as local minima which affect MLP train-ing techniques). On the other hand, before linear optimization can beapplied, the number of radial units must be decided and their centersand radius must be set. Although faster than MLP training, the algo-rithms to do this are equally prone to discover sub-optimal combina-tions. Also, RBF networks are not adequate for extrapolation beyondknown data and experience indicates that they require substantiallymore neurons in the hidden layer to adequately model most functions[70,71].

Table 4LR model performance predicting the structural condition of the test sample.

Observed Predicted (complete) Correct/inc

Categories 1 2

1 68 13 83.9%/16.12 33 21 38.9%/61.1Correct/incorrect 67.3%/32.7% 61.8%/38.3% 65.9%/34.1

2.4. Support vector machines (SVMs)

Support vector machines (SVMs) are supervised learning modelsused for classification tasks. Developed by Cortes and Vapnik [72], theSVMs are derived from Vapnik and Chervonenkis' statistical learningtheory [73]. SVM classification methods are based on the principle ofoptimal separation of classes. Originally, the SVMs were developed forclassifying linearly separable classes by selecting the linear classifiersminimizing the generalization error, or at least an upper bound onthis error, derived from structural risk minimization. In this conception,the decision hyperplane that characterizes the SVM leaves a maximummargin between the two classes, where the margin is defined asthe sum of the distances of the hyperplane from the closest point ofthe two classes [74]. The efficient application to non-linear and non-separable classification problemswasmade possible by using the kerneltrick and including a soft margin, respectively. The kernel trick enablesthe linearization of a non-linear hyperplane, while guarantying thatthe dot product with the vector normal to the hyperplane is easilysolved. Common kernels include polynomial, radial basis or hyperbolictangent functions. The soft margin allows for mislabeled points to beaccounted for by including a slack variable measuring the degree ofmisclassification. For non-separable classes, the SVMs try to find thehyperplane that maximizes the margin and, at the same time, mini-mizes a quantity proportional to the number of misclassification errors.The tradeoff betweenmargin andmisclassification error is controlled bya positive constant included in the SVM objective function.

In the present paper, C-SVMs were used. The capacity, C, representsthe weight of the misclassification error versus the complexity of themodel in the following objective function:

minw;b

12wTw þ C

Xni¼1

ξi

" #ð6Þ

subject to the constraints:

yi wTϕ xið Þ þ bh i

≥1−ξi and ξi ≥ 0; i ¼ 1;…;n ð7Þ

where w is the vector normal to the hyperplane; b is the hyperplaneoffset parameter; ξi are the slack variables measuring the degree ofmisclassification; ϕ is the kernel used to transform the data from theinput (independent) to the feature space; and x, and y are the datapoints of the dataset with n points.

3. Case study

3.1. The SANEST sewer system

SANEST — Sistema de Saneamento da Costa do Estoril is an infra-structure system close to Lisbon, covering the interception, conveyanceand treatment of wastewater generated in the municipalities of Cascaisand Oeiras and part of the municipalities of Amadora and Sintra.SANEST's system covers over 22,000 ha, serving at present approxi-mately 800,000 equivalent inhabitants.

The system is composed by 20 trunk sewers (representing a net-work of 120 km) that collect thewastewater from themunicipal sewersand convey it to a large interceptor. The interceptor, with a length of

orrect Predicted (reduced) Correct/incorrect

1 2

% 68 13 83.9%/16.1%% 33 21 38.9%/61.1%% 67.3%/32.7% 61.8%/38.3% 65.9%/34.1%

Table 5Best ANNs for predicting the structural condition considering the complete and reduced models.

Classification case Train algorithm Error function Correlation Number of neurons Activation function

Train Test Hidden layer Output layer Hidden layer Output layer

Structural— complete BFGS SOS 76.72 78.52 17 2 Hyperbolic tangent ExponentialStructural— reduced BFGS CE 69.02 74.81 32 2 Exponential Softmax

88 V. Sousa et al. / Automation in Construction 44 (2014) 84–91

24.7 km stretching along the coast line from Linda-a-Velha to Cascais,discharges at the Guia Wastewater Treatment Plant (WWTP), wherethe sewage is treated before being discharged into the Atlantic Oceanthrough a long sea outfall. The Guia WWTP is completely buried 30 munderground and operates under a non-conventional treatmentprocess with different modes for summer and winter seasons. It treats59.1 million m3 of wastewater annually.

3.2. Data collection

To support the pro-active management of the sewer system,inspection programs have been carried out periodically since 2005(2005–2006; 2009–2010). Presently, the sewers are being inspectedfor the third time with the support of a specialized GeographicalInformation System containing detailed data of each sewer (diameter,depth, length, slope, material and age). Data was collected from thereports of the first (2005–2006) and second (2009–2010) inspectionsof the Caparide, Castelhana, Marianas and Sassoeiros trunk sewers.These trunk sewers were selected for analysis by the team responsiblefor managing the SANEST sewer system because they are amongthe group with the highest defect rate. The data was screened bydisregarding all sewer reaches with incomplete information. This pro-cess reduced the data sample to 25.4 km of fully characterized sewers.The majority of the sewer pipes use materials such as PVC — PolyvinylChloride (12,682 m), VC — Vitrified Clay (4370 m) and HDPE — High-Density Polyethylene (4102 m). A smaller amount of pipes use PC —

Portland Concrete, C-PP — Corrugated Polypropylene and C-PVC —

Corrugated Polyvinyl Chloride. The pipe material was coded from1 to 6 in an inverse relation with the average age for the logisticregression. Table 2 resumes the available information regarding thecharacteristics of the sewers. The design velocity of the sewers (halfsection flow) was also computed and entered as a variable.

The sewer structural condition was determined using the WRc [9]rating protocol and adopting the peak defect criteria. The classificationscale used ranged from class 1 (best condition) to class 5 (worse condi-tion). Despite the use of a 5 level scale, for rehabilitation decisionmaking purposes the sewers are usually divided in two groups depend-ing on the results of the CCTV inspection. The sewers in classes 4 and 5are considered as high priority for rehabilitation in the short term,whereas the sewers in conditions 1, 2 and 3 are scheduled for inspec-tion. In the present paper two categories of sewers were consideredbased on their structural condition: i) Category 1 — the sewers that donot require immediate intervention (sewers in conditions 1, 2 or 3);and ii) Category 2 — the sewers that require immediate intervention(sewers in conditions 4 or 5). The dataset was randomly split into610 sewers for training and 135 sewers for testing in order to ensurethat the same cases were used for training and testing the differentmethods.

Table 6ANN model performance predicting the structural condition of the test sample.

Observed Predicted (complete) Correct/inc

Category 1 2

1 74 16 82.2%/17.82 13 32 71.1%/28.9Correct/incorrect 85.1%/14.9% 66.7%/33.3% 78.5%/21.5

3.3. Logistic regression (LR)

Two models were developed using LR: i) a complete model includ-ing all predictors available; and ii) a reduced model including only thesignificant predictors. For the latter, the backward method based onthe Wald statistics was used to select the predictors to include in themodel. However, for the studied sample, the material, length, depthand age have consistently emerged as the significant predictors inde-pendently of method (forward or backward) and criteria (likelihoodratio statistics, conditional statistics or Wald statistics) used in theselection. The LR model details are presented in Table 3. The ratio ofthe odds of an event occurring in one group (pa) to the odds of itoccurring in another group (pb) is given by:

OR ¼ papb

¼ eβ ð8Þ

were β is the regression coefficient of the predictor. Analyzing theresults, the sewer depth and some pipe materials (PVC, HPDE, C-PP)are the characteristics that most increase the odds of falling in category2. Table 4 presents the results of the LR models for predicting the struc-tural category of the sewers in the test sample.

3.4. Artificial neural networks (ANNs)

An automated heuristic approach was adopted to choose the bestANN configuration considering only one hidden layer with up to 50neurons, for multilayer perceptron (MLP) ANNs, and between 50 and100 neurons, for radial basis function (RBF) ANNs. Several activationfunctions were tested (exponential; sigmoid logistic; softmax; identity;hyperbolic tangent), both in the hidden and the output neurons of theMLPANNs, using different training algorithms (gradient descent; conju-gate gradient descent; BFGS— Broyden, Fletcher, Goldfarb and Shanno)and error functions (CE — cross entropy; SOS — sum of squares). Foreach combination 100 ANNs were developed and the 10 best solutionsbased in the overall accuracy were compared. The best results wereobtained with MLP networks. The BFGS was always the best trainingalgorithm but the error function and activation functions varied widelywithout significant performance differences among the 10 best ANNs(5% for the training and 3% for the test). Table 5 resumes the bestANNs obtained for predicting the structural condition consideringthe complete and reduced models and the corresponding confusionmatrixes for the test sample are presented in Table 6.

The ANN corresponding to the completemodel was used to evaluatethe uncertainty affecting themodel due to the initial weights of the neu-ron connections. Randomly varying the initial weights of the neuronconnections on 100 ANNs resulted in correlations ranging from 67% to78%, for the train data (69% on average), and from 73% to 81%, for thetest data (74% on average).

orrect Predicted (reduced) Correct/incorrect

1 2

% 80 7 92.0%/8.0%% 27 21 43.8%/56.2%% 74.8%/25.2% 75.0%/25.0% 74.8%/25.2%

Table 7SVM model performance predicting the structural condition of the test sample.

Observed Predicted (complete) Correct/incorrect Predicted (reduced) Correct/incorrect

Category 1 2 1 2

1 80 7 92.0%/8.0% 80 7 92.0%/8.0%2 32 16 33.3%/66.7% 32 16 33.3%/66.7%Correct/incorrect 71.4%/28.6% 69.6%/30.4% 71.1%/28.9% 71.4%/28.6% 69.6%/30.4% 71.1%/28.9%

89V. Sousa et al. / Automation in Construction 44 (2014) 84–91

However, the ANNs are also sensible to the specific cases used fortraining and testing. The ANN corresponding to the complete modelwas also used to evaluate the combined effect of the influence of thecases selected for training and testing and the neuron's initial weights.Randomly varying the initial weights of the neuron connections andrandomly sampling the dataset (80% for train and 20% for test) of2000 ANNs resulted in correlations ranging from 67% to 70%, for thetrain data (68% on average), and from 71% to 76%, for the test data(75% on average).

3.5. Support vector machines (SVMs)

Contrarily to the ANNs, the SVMs provide a means of comparing thegeneralization of different models. For the type of SVMs used in thepresent research, this can be achieved by comparing the capacity. Thecapacity (C) is a coefficient that regulates the trade-off between trainingerrors and prediction risk minimization. Higher C values lead to higherweights given to in-sample misclassifications and lower generalizationof the machine.

An automated trial and error approach was adopted for testing line-ar, polynomial, and radial basis function and sigmoid kernels varyingthe respective parameter values. A five-fold cross validationwas carriedout varying the capacity from 0.1 to 1 with 0.01 increments. Linear,polynomial (degree = 2 to 4; gamma = 0.1 to 0.2; coefficient = 0 to2), Gaussian (gamma = 0.1 to 0.2) and sigmoid (gamma = 0.1 to 0.2;coefficient = 0 to 2) kernels were tested.

Due to the transformation introduced by the kernel, it is not possibleto make a direct comparison of the value of C. Nevertheless, since theoverall classification accuracy was the same for all kernels tested, theradial basis function kernel was selected because, along with the linearkernel, resulted in the lowest C (0.110). Using the radial basis functionkernel with a C = 0.140, SVMs were developed for the complete andreduced set of predictors (Table 7). The accuracy on the training was68.4 in both cases.

The SVMcorresponding to the completemodelwas used to evaluatethe influence of the cases used for testing and training on the accuracyand the capacity. Randomly sampling each trial (80% for train and 20%for test) resulted in an improvement of the correlation in the test(74.5%) and a slight decrease in the train (67.4%). Approximately thesame results (67.5% for the train and 74.8% for the test) were obtainedby randomly sampling each trial (80% for train and 20% for test) andcarrying out a five-fold cross validation varying the capacity from0.1 to 1 with 0.01 increments.

To optimize the SVM parameters the Covariance Matrix AdaptationEvolution Strategy (CMA-ES) [75] was used. The CMA-ES is a generalpurpose optimization tool based on a random search evolutionaryalgorithm adequate to real-parameter optimization of non-linear andnon-convex functions, in which the candidate solutions are sampled

Table 8CMA-ES optimized SVM model performance predicting the structural condition of the test sam

Observed Predicted (complete) Correct/inc

Category 1 2

1 65 15 81.2%/18.82 22 33 60.0%/40.0Correct/incorrect 74.7%/25.3% 68.8%/31.3% 72.6%/27.4

according to a multivariate normal distribution. The main domain ofapplication of the technique is non-separable functions that areill-conditioned or rugged and an interesting feature of the CMA-ESis its quasi parameter-free nature. Also, the CMA-ES is reported toovercome several problems often associated with evolutionary algo-rithms [76] and the algorithm's performance is well documented inthe comparative analyses by Hansen [77] and Garcia et al. [78]. In thepresent research it was used as a hyperparameter optimization tooland the results are present on Table 8.

4. Discussion and conclusions

The different methods yielded similar overall result, with the logisticregression providing the lowest correlations and the ANNs the highest.Regarding the SVMs, the parameter optimization improved the resultsup to a level closer to the ANNs. However, since the main goal of model-ing the condition of sewers is to identify the sewer reaches thatmay needintervention in order to assist in implementing selective inspectionprograms, the ANNs' results have been slightly superior given theadopted approaches. For wastewater utilities such as SANEST they aremore concerned with preventing failures than inspecting some sewersthat are in good condition, the best model should be evaluated in termsof the number of sewers in category 2 that aren't identified (false nega-tives. The false positives (sewers in category 1 predicted as category 2)are less relevant because there is no increased threat of failure.

The performance of the artificial intelligence tools analyzed mustnot be considered deterministic because the results are susceptibleto several factors. Besides the type (e.g., MLP; RBF), the structure(e.g., number of layers; number of neurons in each layer; the activationfunctions of the neurons) and the numerical options (e.g., trainingalgorithm; error function), the initial weight of the neuron connectionsintroduces a significant variability into the results. SVMs, on the otherhand, are sensible to the values of the parameters of the kernel, thetype of kernel used and the capacity. Both artificial intelligence toolsare also dependent on the cases in the training and testing samples, asshowed by the analysis carried out.

Considering the uncertainty in theANN and SVMresults, it is evidentthat the solution space of both tools overlaps and a decision on the bestmodel is not clear. On one hand, the ANNs have the potential to achievebest results when compared to the SVMs. On the other hand, the ANNsare more sensible to the approach used in the development and theirperformance may vary in a wider range of results when comparedto the SVMs. Furthermore, the SVMs are also less sensible to the inde-pendent variables used in themodel (complete and reducedmodel per-formances are the same). Therefore, an approach is required to comparethese artificial intelligence tools taking into consideration their stochas-tic nature. In a practical perspective, amore conservative attitudewouldrecommend the support vector machines (lower performance and less

ple.

orrect Predicted (reduced) Correct/incorrect

1 2

% 65 15 81.2%/18.8%% 22 33 60.0%/40.0%% 74.7%/25.3% 68.8%/31.3% 72.6%/27.4%

90 V. Sousa et al. / Automation in Construction 44 (2014) 84–91

variability)while a less conservative attitudewould recommend the ar-tificial networks (higher performance and more variability).

Acknowledgments

The authors acknowledge the SANEST for providing the data used inthe study. The ICIST-IST Research Institute support, the Fulbright/FLADgrant supporting research at UCDavis and the grant SFRH/BD/35925/2007 from the Portuguese National Science Foundation are alsoacknowledged.

References

[1] H. Alegre, M.C. Almeida (Eds.), Strategic Asset Management of Water Supply andWastewater Infrastructures, Invited Papers from the IWA Leading Edge Conferenceon Strategic Asset Management (LESAM), Lisbon, Portugal, 2007.

[2] ISO 24510:2007, Activities Relating to Drinking Water and Wastewater Services:Guidelines for the Assessment and for the Improvement of the Service to Users,International Organization for Standardization (ISO), 2007.

[3] ISO 24511:2007, Activities Relating to Drinking Water and Wastewater Services:Guidelines for the Management of Wastewater Utilities and for the Assessment ofWastewater Services, International Organization for Standardization (ISO), 2007.

[4] ISO 24512:2007, Activities Relating to Drinking Water and Wastewater Services:Guidelines for the Management of Water Utilities and for the Assessment ofWater Services, International Organization for Standardization (ISO), 2007.

[5] H. Alegre, Strategic infrastructure asset management: concepts, ‘schools’ andresearch needs, Water Asset Manage. Int. 4 (1) (2008) 7–14.

[6] H. Alegre, Is strategic asset management applicable to small and medium utilities?Water Sci. Technol. 62 (9) (2010) 2051–2058.

[7] WEF/ASCE, Existing sewer evaluation and rehabilitation, Water EnvironmentFederation (WEF), Manual of Practice FD-6/American Society of Civil Engineers(ASCE), Manuals and Reports on Engineering Practice No. 62, USAsecond ed., 1994.

[8] R. Fenner, L. Sweeting, A decision support model for the rehabilitation of “non-critical”sewers, Water Sci. Technol. 39 (9) (1999) 193–200.

[9] WRc, Sewerage Rehabilitation Manual, fourth ed. Water Research Center (WRc),England, 2001.

[10] NRC-CNRC, Assessment and Evaluation of Storm and Wastewater Collection Systems,National Guide to Sustainable Municipal Infrastructure (InfraGuide), Canada, 2004.

[11] S. Renya, Optimal Planning of Sewer Systems Rehabilitation, PhD thesis PurdueUniversity, USA, 1993.

[12] RERAU, Méthodologie de programmation de réhabilitation des collecteurs visitables,Projet National RERAU (Réhabilitation des Réseaux d'Assainissement Urbains),France, 1998. (in French).

[13] CARE-S — Computer Aided REhabilitation of Sewer Networks, Project NumberEVK1-CT-2002-00106, http://care-s.unife.it/ .

[14] R. Baur, R. Herz, Selective inspection planning with ageing forecast for sewer types,Water Sci. Technol. 46 (6–7) (2002) 379–387.

[15] M. Hahn, R. Palmer, M. Merrill, B. Andrew, A. Lukas, Prioritizing sewer line inspec-tion with an expert system, Proceedings of 29th Annual Water Resources Planningand Management Conference, USA, 1999.

[16] M. Hahn, R. Palmer, M. Merrill, B. Andrew, A. Lukas, Knowledge acquisition andvalidation of an expert system for prioritizing the inspection of sewers, Proceedingsof Joint Conference on Water Resource Engineering and Water Resources Planningand Management, USA, 2000.

[17] M. Hahn, R. Palmer, M. Merrill, B. Andrew, A. Lukas, Expert system for prioritizingthe inspection of sewers: knowledge base formulation and evaluation, J. WaterRes. Plan. Manag. 128 (2) (2002) 121–129.

[18] H. Korving, Probabilistic Assessment of Combined Sewer Systems, PhD Thesis DelftUniversity of Technology, Delft, The Netherlands, 2004.

[19] H. Tran, Investigation of Deterioration Models for Stormwater Pipe Systems, PhDThesis Victoria University, School of Architectural, Civil and Mechanical EngineeringFaculty of Health, Engineering and Science, Victoria, Australia, 2007.

[20] D.M. Abraham, R. Wirahadikusumah, Development of prediction models for sewerdeterioration, Proceeding of the International Conference on Durability of BuildingMaterial and Components, Vancouver, Canada, 1999, pp. 1257–1267.

[21] J. Mehle, S. O'Keefe, P. Wrase, An Examination of Methods for Condition Rating ofSewer Pipelines, Center for Development of Technical Leadership, University ofMinnesota, USA, 2001.

[22] V. Sousa, M. Silva, T. Veigas, J. Matos, J. Martins, A. Teixeira, Technical managementof sewer networks: a simplified decision tool, LESAM 2007 — 2nd Leading EdgeConference on Strategic Asset Management, 17–19 October, Lisbon, Portugal, 2007.

[23] V. Sousa, F. Ferreira, N. Almeida, J. Matos, J. Martins, A. Teixeira, A simplified techni-cal decision support tool for the asset management of sewer networks, Water AssetManage. Int. 5 (1) (2009) 3–9.

[24] E.V. Ana, Sewer Asset Management — Sewer Structural Deterioration Modeling andMulticriteria Decision Making in Sewer Rehabilitation Projects Prioritization, PhDThesis Faculty of Engineering, Vrije Universiteit Brussel, Brussels, Belgium, 2009.

[25] F. Chughtai, T. Zayed, Structural condition models for sewer pipeline, Pipelines2007: Advances and Experiences with Trenchless Pipeline Projects, 8–11 July,Boston, USA, 2007.

[26] F. Chughtai, T. Zayed, Sewer pipeline operational condition prediction using multi-ple regression, Pipelines 2007: Advances and Experiences with Trenchless PipelineProjects, 8–11 July, Boston, USA, 2007.

[27] F. Chughtai, T. Zayed, Infrastructure condition prediction models for sustainablesewer pipelines, J. Perform. Constr. Facil. 22 (5) (2008) 333–341.

[28] R. Wirahadikusumah, D. Abraham, T. Iseley, Challenging issues in modeling deterio-ration of combined sewers, J. Infrastruct. Syst. 7 (2) (2001) 77–84.

[29] L.A. Newton, D.J. Vanier, MIIP report: the state of Canadian sewers — analysis ofasset inventory and condition, B-5123.11, Institute for Research in Construction(IRC), National Research Council Canada (NRC-CNRC), Canada, 2006.

[30] S. Hörold, R. Baur, Modeling sewer deterioration for selective inspection planning—case study Dresden, Proceedings 13th EJSW on Service Life Management Strategiesof Water Mains and Sewers, 8–12 September, Switzerland, 1999.

[31] R. Baur, W. Zielichowski-Haber, I. Kropp, Statistical analysis of inspection data forthe asset management of sewer networks, Proceedings 19th EJSW on ProcessData and Integrated Urban Water Modeling, Lyon, France, 2004.

[32] Y. Yang, Statistical Models for Assessing Sewer Infrastructure Inspection Require-ments, MSc. Thesis Department of Civil and Environmental Engineering, Universityof Alberta, Edmonton, Alberta, Canada, 1999.

[33] T.S. Ariaratnam, E.A. Assaly, Y. Yuqing, Assessment of infrastructure inspectionneeds using logistic models, J. Infrastruct. Syst. 7 (4) (2001) 66–72.

[34] J. Davies, B. Clarke, J. Whiter, R. Cunningham, The structural condition of rigid sewerpipes: a statistical investigation, Urban Water J. 3 (2001) 277–286.

[35] O. Pohls, The Analysis of Tree Root Blockages in Sewer Lines & Their PreventionMethods, MSc. Thesis Institute of Land and Food Resources, University ofMelbourne, Melbourne, Australia, 2001.

[36] P.J. Coombes, T. Micevski, G. Kuczera, Deterioration, depreciation and serviceabilityof stormwater pipes, Stormwater Industry Association: 2002 Conference on UrbanStormwater Management, 23–24 April, Orange, NSW, Australia, 2002.

[37] T. Micevski, G. Kuczera, P. Coombes, Markov model for storm water pipe deteriora-tion, J. Infrastruct. Syst. 8 (2) (2002) 49–56.

[38] S. Baik, H.S. Jeong, D.M. Abraham, Estimating transition probabilities in Markovchain-based deterioration models for management of wastewater systems, J. WaterRes. Plan. Manag. 132 (1) (2006) 15–24.

[39] D.-H. Koo, S.T. Ariaratnam, Innovativemethod for assessment of underground sewerpipe condition, Autom. Constr. 15 (2006) 479–488.

[40] Y. Le Gat, Modelling the deterioration process of drainage pipelines, UrbanWater J. 5(2) (2008) 97–106.

[41] Y. Kleiner, Scheduling inspection and renewal of large infrastructure assets,J. Infrastruct. Syst. 7 (4) (2001) 136–143.

[42] J. Dirksen, F.H.L.R. Clemens, Probabilistic modeling of sewer deterioration usinginspection data, Water Sci. Technol. 57 (10) (2008) 1635–1641.

[43] M. Najafi, G. Kulandaivel, Pipeline condition prediction using neural networkmodels, Pipelines 2005: Optimizing Pipeline Design, Operations, and Maintenancein Today's Economy, ASCE, Reston, VA, USA, 2005, pp. 767–775.

[44] D.H. Tran, A.W.M. Ng, B.J.C. Perera, P. Davis, Application of probabilistic neuralnetworks in modeling structural deterioration of stormwater pipes, Urban WaterJ. 3 (3) (2006) 175–184.

[45] Z. Khan, T. Zayed, O. Moselhi, Structural condition assessment of sewerpipelines, J. Infrastruct. Syst. 24 (2) (2010) 170–179.

[46] J. Yan, K. Vairavamoorthy, Fuzzy approach for pipe condition assessment, Proc.,New Pipeline Technologies, Security, and Safety, ASCE, Reston, VA, 2003,pp. 466–476.

[47] Y. Kleiner, B. Rajani, R. Sadiq, Modeling failure risk in buried pipes using fuzzyMarkov deterioration process, 4th International Conference on Decision Making inUrban and Civil Engineering, 28–30 October, Porto, Portugal, 2004, pp. 1–11.

[48] Y. Kleiner, R. Sadiq, B. Rajani, Modeling failure risk in buried pipes using fuzzyMarkov deterioration process, Pipeline Engineering and Construction: What's onthe Horizon?, ASCE, San Diego, California, USA, 2004. 7–16.

[49] Y. Kleiner, B. Rajani, R. Sadiq, Modelling the deterioration of buried infrastructure asa fuzzy Markov process, J. Water Supply Res. Technol. 55 (2) (2006) 67–80.

[50] R.A. Fenner, G. McFarland, O. Thorne, Case-based reasoning approach for managingsewerage assets, Proceedings of the Institution of Civil Engineers: Water Manage-ment 160 (WM1), 2007, pp. 15–24.

[51] J. Mashford, D. Marlow, T. Tran, R. May, Prediction of sewer condition grade usingsupport vector machines, J. Comput. Civ. Eng. 25 (4) (2011) 283–290.

[52] D. Savic, O. Giustolisi, L. Berardi,W. Shepherd, S. Djordjevic, A. Saul, Modelling sewerfailure by evolutionary computing, Proceedings of the Institution of Civil Engineers:Water Management 159 (WM2), 2006, pp. 111–118.

[53] R. Ugarelli, S.M. Kristensin, J. Røstum, S. Sægrov, V. Di Frederico, Statistical analysisand definition of blockages-prediction formulae for the wastewater network ofOslo by evolutionary computing, 11th International Conference in Urban Drainage,Edinburgh, Scotland, UK, 2008.

[54] D.A. Savic, O. Giustolisi, D. Laucelli, Asset deterioration analysis using multi-utilitydata and multi-objective data mining, J. Hydroinformatics 11 (3–4) (2009)211–224.

[55] J. Røstum, Statistical Modelling of Pipe Failures in Water Networks, PhD. ThesisNorwegian University of Science and Technology, Trondheim, Norway, 2000.

[56] A. Vanrenterghem-Raven, Risk factors of structural degradation of an urban waterdistribution system, J. Infrastruct. Syst. 13 (1) (2007) 55–64.

[57] P.D. Rogers, N.S. Grigg, Failure assessment modeling to prioritize water piperenewal: two case studies, J. Infrastruct. Syst. 15 (3) (2009) 162–171.

[58] S. Yamijala, S.D. Guikema, K. Brumbelow, Statistical models for the analysis of waterdistribution system pipe break data, Reliab. Eng. Syst. Saf. 94 (2009) 282–293.

[59] D.C. Montgomery, G.C. Runger, Applied Statistics and Probability for Engineers, thirded. John Wiley & Sons, New York, 2003.

91V. Sousa et al. / Automation in Construction 44 (2014) 84–91

[60] T.T. Allen, Introduction to Engineering Statistics and Six Sigma: Statistical QualityControl and Design of Experiments and Systems, Springer-Verlag, London, 2006.

[61] V. Bewick, L. Cheek, J. Ball, Statistics review 14: logistic regression, Crit. Care 9 (1)(2005) 112–118.

[62] J.F. Hair, R. Tatham, R. Anderson, W. Black, Multivariate Data Analysis, Prentice Hall,Upper Saddle River, NJ, 1998.

[63] J.O. Rawlings, G. Sastry, S.G. Pentula, D.A. Dickey, Applied Regression Analysis: AResearch Tool, second ed. Springer, New York, Berlin, Heidelberg, Barcelona,Hong Kong, London, Milan, Paris, Singapore, Tokyo, 1998.

[64] J.P. Marques de Sá, Applied Statistics: Using SPSS, STATISTICA, MATLAB and R,Springer, Berlin, Heidelberg, New York, 2007.

[65] N.K. Kasabov, Foundations of Neural Networks, Fuzzy Systems, and Knowledge-Engineering, MIT Press, Cambridge, 1996.

[66] M.H. Hassoun, Fundamentals of Artificial Neural Networks, MIT Press, Cambridge,1995.

[67] B. Kröse, P. van der Smagt, An Introduction to Neural Networks, The University ofAmsterdam, 1996.

[68] Y.H. Hu, J.-N. Hwang (Eds.), Handbook of Neural Network Signal Processing, CRCPRESS, Boca Raton, London, New York, Washington, D.C., 2002

[69] B.Warner, M. Misra, Understanding neural networks as statistical tools, Am. Stat. 50(4) (1996) 284–293.

[70] S. Haykin, Neural Networks: A Comprehensive Foundation, second ed. Prentice Hall,Upper Saddle River, NJ, 1999.

[71] C. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, Oxford,1995.

[72] C. Cortes, V. Vapnik, Support-vector networks, Mach. Learn. 20 (3) (1995) 273–297.[73] V.N. Vapnik, A.Y. Chervonenkis, On the uniform convergence of relative frequencies

of events to their probabilities, Theory Probab. Appl. 16 (2) (1971) 264–280.[74] V. Vapnik, The Nature of Statistical Learning Theory, Springer-Verlag, New York,

1995.[75] A. Auger, N. Hansen, A restart CMA Evolution Strategy with Increasing Population

Size, IEEE Congress on Evolutionary Computation, IEEE, vol. 2, 2005, pp. 1769–1776.[76] N. Hansen, The CMA evolution strategy: a tutorial, http://www.lri.fr/~hansen/

cmatutorial.pdf .[77] N. Hansen, The CMA evolution strategy: a comparing review, in: J. Lozano, P.

Larrañaga, I. Inza, E. Bengoetxea (Eds.), Towards a New Evolutionary Computation,Studies in Fuzziness and Soft ComputingSpringer Berlin/Heidelberg, 2006,pp. 75–102.

[78] S. García, D. Molina, M. Lozano, F. Herrera, A study on the use of non-parametrictests for analyzing the evolutionary algorithms' behaviour: a case study on theCEC'2005 special session on real parameter optimization, J. Heuristics 15 (6)(2009) 617–644.