Development and Testing of a Portable Vision Based SHM System for Rapid Deployment on Rural Bridge Sites

D. Lydon1, S.E. Taylor1 D.Robinson1, M.Lydon1 D. Hester1

1School of Natural and Built EnvironmentQueens University Belfast

BT9 5AG

ABSTRACTStructural Health Monitoring (SHM) techniques can provide vital information on the performance and capacity of new and existing structures. Generally sensors are embedded or surface mounted to assess the integrity of the structure and provide a means of damage detection or failure prediction. However, existing systems are limited by the need for contact with the structure and demand onerous set up procedures. The lack of a permanent power supply on rural site adds an additional limitation to many existing systems. This paper details the development and testing of a fully portable vision based SHM system. Standard DSLR cameras along with adapted digital image correlation methods are utilized to measure deflections and associate the response to live loading. A series of laboratory investigations have been carried out at Queens University Belfast to develop and validate this system as a viable means of displacement measurement. The system was subsequently trialed at two bridge sites in Northern Ireland, a 20m span truss bridge and a 7m span masonry arch bridge. In each case the structures were identified as showing significant signs of deterioration and the outcome of visual inspections highlighted concerns for the asset owners. Existing SHM systems were not feasible due to access, power and cost limitations, the vision based system allowed for rapid site deployment and real time monitoring of the structures under live loading. The results obtained on site were then compared with theoretical analysis methods to determine the true structural condition.

Keywords: Computer Vision, Structural Health Monitoring

INTRODUCTIONExisting Civil infrastructure is under an increasing level of stress from loading/environmental effects. These effects can be detrimental to the integrity of the bridges, and must be monitored in order to avoid dangerous incidents and insure public safety. Visual inspections remain to be the most common method of bridge inspection worldwide. This method is used as a means of detecting obvious damage to structures such as cracks/shifting of components and is carried out by following a set of established guidelines according to bridge type. This method has many limitations which affect its reliability and is extremely sensitive to human error, particularly since a visual inspection is rarely carried out by a senior engineer. A survey of the reliability of visual inspections has detailed the high level of variability in this assessment method. [1]

In recent years Structural Health Monitoring (SHM) systems have been developed to try to overcome these limitations. SHM can provide an unbiased means of determining the true state of our aging infrastructure. Sensor systems are used to monitor bridge deterioration and provide real information on the capacity of individual structures, hence extending bridge life and improving safety. Changes in stiffness is usually measured using strain sensors, but recent research has indicated that measuring displacement changes from calibrating vehicles can be used as a method of detecting bridge condition. [2]

The deflection readings are gathered with respect to vehicles of known weight passing over the structure. If pre-weighed calibration trucks are not available for testing, vehicle weights can be either gathered from a database in order to gain approximate readings, or by using a weigh-in-motion system to gain precise information on vehicle weights. Traditionally displacement is measured using transducers or accelerometers which are attached to fixed points on the structure. The transducers, such as linear variable differential transducers (LVDT), give a direct reading of displacement but generally require an independent frame for mounting, which for most bridges make them impractical for use in the field. While these readings can, under certain circumstances, provide a reasonably accurate estimate of deflection, there are several related problems with the method, such as:

Durability as the sensors can be damaged easily. Equipment is expensive to purchase and time consuming to set up on site

In recent years, new methods have been investigates to address these issues, such as GPS readers, Laser Vibrometers and Computer vision techniques. GPS readers have been proved to be a reliable method of detecting deflection but require satellite connectivity to function, which may not be feasible at all sites, and they are susceptible to high electro-magnetic noise [3]they are also not ideal for short span bridges where movement ranges are modest. Laser vibrometers are comparable in effectiveness with transducers[4] , but they are costly and difficult to set up/operate. On the contrary, video cameras are cheap, reasonably easy to use and can inspect a structure from a distance. They also mimic human visual inspection. The main drawback in their use as part of an SHM system is that the captured images require complex and intelligent data processing and analysis through computer vision algorithms. This paper details the development and testing of an algorithm for capturing deflection of bridge sites in rural areas.

Algorithm DevelopmentThe program structure for object tracking in video files is generally broken down into the following steps:

Image Registration/Key point Generation Tracking of registered points through consecutive frames. Results Display

The algorithm in this paper uses the BRISK [5] method for image registration and key point generation inside the area of the bridge that was selected for tracking by the user. This method was chosen due to the high reliability of the key points produced and enhanced processing time compared to alternative methods. The key points, or features, that are outputted from the BRISK method are then used as initialization values for a Kanade-Lucas-Tomasi(KLT) object tracking method[6] which tracks the movement of these points throughout the video. The algorithm works by generating an image pyramid, where each levels is down-sampled by a factor of two in width and height. The point tracker begins tracking each point in the lowest resolution level, and continues tracking until all levels have been checked. This method is applicable to the recordings gathered because there will not be a large displacement between frames as the videos were captured at a rate of 60fps. The method uses a threshold value for the amount a feature can be expected to move between frames to eliminate points that cannot be reliably tracked, reducing the error in the display of results. The positions of the points in each frames are stored in a matrix in Matlab, and the Euclidean distance between the points in subsequent frames is calculated and plotted with respect to time in order to obtain a plot of displacement of points in a video.

LABORATORY WORK

A steel beam was simply supported under an accurately calibrated 600kN capacity hydraulic actuator in the laboratory as shown in Figure 1 and 3mm of deflection was applied to the beam. A pattern was applied to the beam under the point that load would be applied at in order to increase the accuracy of the feature extraction used in the deflection calculation. An LVDT was placed under the steel beam and set to read continuously. A Nikon d5300

camera was set up at a monitoring distance of 2m. This was used to capture the videos required for deflection calculation.

Figure 1 Test Rig in the LaboratoryExperimental Programme Details of Beam used for TestingThe details of the beam used for the testing are as follows:

Breadth: 48mm Depth : 24mm Modulus of Elasticity:2.05 x 10⁵ n/mm² Area: 1152 mm Second Moment of Area: 55.296 x 10³mm⁴

Test Plan & Results for Test SeriesA series of loading cycles were carried out on the beam. The details are presented in Table 1. A deflection of 3mm was applied using stroke control at varying rates by the test rig and displacement was monitored by LVDT. In this test series a Spectra data logger was used for data acquisition from the LVDT. Load monitoring was not implemented in this test series, so all figures only contain LVDT and camera findings. Further verification was provided by the use of a dial gauge placed under the load application point which was monitored by extracting key-frames from the captured video at a rate of 4Hz.

Table 1 Test Plan for Algorithm Validation Test SeriesTest Number Deflection Rate (mm/s) Scanning Rate(Hz)

1 0.5 12 1 13 0.5 24 1 25 0.5 4

6 1 4

Figure 2 Results from Laboratory Trials of Computer Vision Algorithm

The results of Test 6 are shown in Figure 2. The maximum deflection as determined by the algorithm was compared to that of the LVDT and dial gauge and an error percentage generated for each test series.

INSERT TABLE

This test series detailed an experimental program carried out to assess the accuracy of camera based deflection measurements. The findings indicate that a computer vision algorithm can provide deflection calculations which correspond well with verified measurements from an LVDT. The next step in the development of the algorithm was to perform field trials at suitable location, as will be described in the following section.

FIELD WORKVerners Bridge A bridge that was marked for repair for Transport NI was made available for testing in order to validate the performance of the algorithm in field conditions.. A series of passes by a calibrated weight truck over this bridge were recorded and displacement was calculated – the results from a series of passes are shown in Figure 3.

Figure 3 Results from Midspan Monitoring on Verners Bridge Due to an error with calibration, the results from the accelerometers used for ground truth verification were not available; nevertheless there is a close correlation between readings of the camera solution for subsequent passes on the same location. The difficulty with accelerometer setup provides a further validation of the validity of vision systems as a method, with no issues occurring with setup or monitoring with the camera equipment.

Majors BridgeMajors Bridge is a masonry arch bridge which carries two lanes of traffic on the A34 Lisnagole road north of Lisnaskea. The bridge has a cumulative span of 7.41m, Cover to arch ring 0.11m  (approx. 0.2m measured on site recently) and a deck width 9.0m.A few approximate measurements are presented below and the east elevation can be seen in Figure 4.

Voussoir depth at keystone 0.77m. Soffit to top of parapet wall 1.87m. Parapet wall to road surface 0.9m. Parapet wall thickness 0.48m.

Figure 4 Site layout elevation at Majors Bridge

Concerns about the bridge condition were raised subsequent to a traditional bridge inspection carried out by Transport NI. Excessive movement was reported under normal traffic loading, in order to assess this, Queens University have proposed a Structural Health Monitoring (SHM) assessment using camera based monitoring system. An initial site investigation was carried out to determine the suitability of the proposed system. Access was gained to the through a housing development on the south east of structure, this area provided a safe location of parking and setting up of equipment. As shown in the cover photo the area surrounding the bridge is overgrown but the water level under was low enough to allow safe easy access to the underside of the bridge.During this initial site visit some short term monitoring was carried out using the camera based system. It was also intended to use a dial gauge system to provide validation of the results but this was not possible during this visit due to the very uneven ground conditions.

Monitoring SetupThe camera was set up at a distance of 2.9m from the spandrel wall on the south east side of the structure as shown in Figure 5. Monitoring was carried out under live traffic to determine if the system could provide information on the bridge behavior.

Figure 5 Camera monitoring location

Targets were attached to the underside of the bridge using a non-destructive heavy duty double sided tape. The target locations are shown in Figure 6 and this allowed for accurate calculation of the movement based on the known geometry of the target.

Figure 6 Targets attached to bridge soffit

Trial Data Collection

The results shown in Figure 7 show the measured response of the bridge structure under traffic loading. The actual weight of the vehicles is unknown but images of the vehicles have been embedded on the graph to give an indication of vehicle type. In this case a maximum deflection of ~1.2mm was measured at the mid span.

0 2 4 6 8 10 12 14 16 18 20

-1.4

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

Test scan: Majors Bridge

time (seconds)

Dis

plac

emen

t in

vert

ical

dir

ecti

on (m

m)

Figure 7 Midspan displacement in vertical direction due to live traffic as shown in embedded picture.

The purpose of this initial visit was to determine the suitability of the site layout for camera based monitoring. The trial monitoring carried out indicates this method of assessment is suitable for this site. However, to provide a more critical analysis of this bridge a more exhaustive monitoring program would be required along with knowledge of the specific traffic loading.

CONCLUSIONSThe results detailed in this paper have shown the efficacy and ease of setup of a camera based monitoring setup for use at rural sites. In each case the camera apparatus was set up and readings were gathered in a short space of time with inexpensive equipment. The results shown correlate with those from a model that was analyzed prior to site deployment. Further field work and algorithm enhancement is required in order to make this process a completely viable solution for SHM, but the results shown here indicate that it is a promising area for development.

REFERENCES[1] Baystate Roads Program, Tech Notes Reliability of Visual Inspection for Highway Bridges , 2003.[2] T. Ojio, C. Carey, E. OBrien, C. Doherty, S. Taylor, Contactless Bridge Weigh-in-Motion, J. Bridg. Eng. (2016) 4016032. doi:10.1061/(ASCE)BE.1943-5592.0000776.

[3] M. Meo, G. Zumpano, X. Meng, E. Cosser, G. Roberts, A. Dodson, Measurements of dynamic properties of a medium span suspension bridge by using the wavelet transforms, Mech. Syst. Signal Process. 20 (2004) 1112–1133. doi:10.1016/j.ymssp.2004.09.008.[4] H.H. Nassif, M. Gindy, J. Davis, Comparison of laser Doppler vibrometer with contact sensors for monitoring bridge deflection and vibration, NDT E Int. 38 (2005) 213–218. doi:10.1016/j.ndteint.2004.06.012.[5] S. Leutenegger, M. Chli, R.Y. Siegwart, BRISK: Binary Robust invariant scalable keypoints, in: 2011 Int. Conf. Comput. Vis., IEEE, 2011: pp. 2548–2555. doi:10.1109/ICCV.2011.6126542.[6] C. Tomasi, T. Kanade, Detection and Tracking of Point Features, Carnegie Mellon Univ. Tech. Rep. (1991) 91–132. http://www.lira.dist.unige.it/teaching/SINA/slides-current/tomasi-kanade-techreport-1991.pdf.