A New Methodology for Bridge Inspections in Linear Infrastructures from Optical Images and HD Videos Obtained by UAV

Abstract

Many bridges and other structures worldwide present a lack of maintenance or a need for rehabilitation. The first step in the rehabilitation process is to perform a bridge inspection to know the bridge′s current state. Routine bridge inspections are usually based only on visual recognition. In this paper, a methodology for bridge inspections in communication routes using images acquired by unmanned aerial vehicle (UAV) flights is proposed. This provides access to the upper parts of the structure safely and without traffic disruptions. Then, a standardized and systematized novel image acquisition protocol is applied for data acquisition. Afterwards, the images are studied by civil engineers for damage identification and description. Then, specific structural inspection forms are completed using the acquired information. Recommendations about the need of new and more detailed inspections should be included at this stage when needed. The suggested methodology was tested on two railway bridges in France. Image acquisition of these structures was performed using an UAV for its ability to provide an expert assessment of the damage level. The main advantage of this method is that it makes it possible to safely accurately identify diverse damages in structures without the need for a specialised engineer to go to the site. Moreover, the videos can be watched by as many engineers as needed with no personal movement. The main objective of this work is to describe the systematized methodology for the development of bridge inspection tasks using a UAV system. According to this proposal, the in situ inspection by a specialised engineer is replaced by images and videos obtained from an UAV flight by a trained flight operator. To this aim, a systematized image/videos acquisition method is defined for the study of the morphology and typology of the structural elements of the inspected bridges. Additionally, specific inspection forms are proposed for every type of structural element. The recorded information will allow structural engineers to perform a postanalysis of the damage affecting the bridges and to evaluate the subsequent recommendations.

1. Introduction

Many bridges and other structures worldwide present a lack of maintenance or a need for rehabilitation. For example, according to the American Road and Transportation Builder Association, 220,000 U.S. bridges, 36% of the total, need repair, replacement or major rehabilitation [1]. The first step in the rehabilitation process is to perform a bridge inspection to know the current state of the bridge. Routine bridge inspections are usually based only on a visual check. If any important damage is found during the check, the inspection is extended and complemented by ancillary information, such as monitoring systems, to perform a complete diagnostic of the state of the structure. The bridge inspection must be performed by a structural engineer who must be able to assess the state of the bridge and the extent of the damage, if any. The access to all parts of the bridge is undoubtedly one of the principal limiting factors during a bridge inspection. Moreover, some bridges are built in remote areas or poorly accessible areas such as rivers, deep valleys or even over the sea. Due to all these factors, recently, many efforts to use modern technology such as unmanned aerial vehicles (UAV) or other remote sensing technologies to assist in bridge inspections by have been made. Valença et al. [2] developed a new method called MCrack-TLS to automatically assess cracks in concrete bridges. This method combines image processing and terrestrial laser scanning to increase productivity and record all data processed. Results of MCrack-TLS analysis confirm those obtained for structures that used other methods to assessing cracks [3,4,5,6]. The use of UAV for bridge inspection can help obtain data from several access areas of the bridge due to the versatility of these vehicles compared to other remote sensing technologies such as terrestrial laser scanning. In this sense, a comparison between conventional inspection practices and a drone-based inspection was made by Chan et al. [7]. This research concluded that inspections assisted by drones have some important advantages, such as cost, time, and reduction of risk for inspectors. Similar conclusions were drawn by Koch et al. [8] after analysing different large concrete structures (Figure 1).

Many efforts are being successfully made to improve the quality and autonomy of UAS-enabled bridge inspections [9,10,11,12,13,14]. Nevertheless, a systematic methodology for inspections is needed to improve the overall quality of bridge inspections. One of the main aspects for improving the consistency of this inspection is to follow a clear and simple methodology. Barrile et al. [15] stated that the use of UAV technology in infrastructure surveying spread recently in different applications. Different authors [16,17,18,19,20] have studied the need for a standard methodology and workflow for data acquisition and analysis. For example, the research completed for evaluating the bridge inspection quality in New York State in 2021 [21] concluded that some action should be taken to clarify the desired content of inspection reports. Similar conclusions were drawn by Dorafshan and Maguire [22] after reviewing current bridge inspection practices, including those assisted by unmanned aerial vehicles, concluding that there is a need for continuous improvement of bridge inspection procedures. Plotnikov and Collura [23] recommended the development of a simple and comprehensive guidebook to assist stakeholders with integrating UAV into bridge inspections. To avoid the lack of information and to make the available information easier to understand, it is also important to systematize data collection when viewing the images. A clear and systematized protocol using UAV for bridge inspection is also needed. Riveiro et al. [24] developed a methodology for the inventory of historical infrastructures which included alphanumeric data collection and digital image capturing and featured measuring by low cost equipment. Rolander et al. [25] identified visual inspection as the primary tool used to perform highway bridge inspections. Valenzuela et al. [26] developed an integrated bridge index (IBI) as a help for prioritizing bridge rehabilitation. This index weighted not only visual inspections of bridges but also factors such as seismic risk, hydraulic vulnerability and strategic importance. Some other authors have also examined the need for a framework for automated UAS-based structural condition assessment of bridges [27]. This general framework included the following steps: path definition, assessment criteria (preparation, flight path generation, UAS-based data acquisition, photogrammetric reconstruction, modelling and visualisation) and structural condition assessment. A similar workflow was presented by Barrile et al. [15] for UAV surveys of bridges and viaducts. This workflow classified the tasks to be undertaken in three groups: data acquisition, data elaboration and feature extraction presentation, including a structural analysis of bridges in the last step. Sacks et al. [28] proposed an integrated bridge inspection system for rapid survey and assessment in infrastructure maintenance. This method included a process map defining the proposed inspection process, a list of typical bridge elements, possible connections between elements, a defect table and a definition of the required information contents.

When using UAVs for bridge inspection, it is unnecessary for the structural engineer to conduct an in situ inspection, since the videos and images can be remotely collected by drones and analyzed later by engineers. Moreover, the pandemic caused by SARS-CoV-2 has shown that alternative methods that reduce the number of people needed to inspect a bridge should be examined. UAV-assisted inspections require fewer workers as evidenced during the COVID-19 pandemic. Drone technology can assist structural inspections at a low cost and with less injury risk compared to conventional methods [4,29].

Most of the existing scientific literature about using drones in structural inspections is focused on the improvement of the devices, the automatic detection and measurement of length and cracks aperture, etc. However, there is neither a consensus nor a detailed methodology about the procedure to obtain information for preliminary bridge inspections and for the decision-making process. Therefore, this work fills the gap between the data collection and the structural condition assessment of bridges.

The main purpose of this work is to present a new protocol for systematizing bridge inspection. Although this methodology can be used for traditional inspections, it is designed for bridge inspections assisted by UAV. Then, the conventional inspection performed in situ by structural engineers is replaced by collecting images and recording videos from UAV flights. To do so, ad hoc inspection forms have been developed covering all bridges usually used in linear infrastructures. Hence, the proposed methodology is a standardized, cost-effective, safe, and reliable procedure for inspecting bridges that can be easily adapted to other type of structures such as buildings, communication towers, dams, etc.

2. Materials and Methods

The workflow of the proposed methodology begins with the acquisition of images and videos by an unmanned aerial vehicle (UAV), which allows to safe access to all parts of the structure without traffic disruptions. To avoid erroneous information and/or loss of information, the image acquisition cannot be randomly performed and must follow a standardized protocol, such as the one proposed in this work. Therefore, in this paper, a standardized and systematized proposed image acquisition protocol is applied for data acquisition. The first document to be completed by engineers using the information provided by an UAV is the general information form. After data acquisition, the images and videos are studied by specialized structural engineers for description and damage identification. Also, the structural elements to be reviewed and the list of potential damage must be systematized. Specific standardized forms designed for structural inspection are then filled in using the acquired information. Finally, if necessary, recommendations about the need for new and more detailed inspections, either by specialized engineers or by UAVs should be stated (Figure 2).

The suggested methodology was evaluated by studying two railway bridges in France. Image acquisition of these structures was performed using an UAV for its subsequent expert ability to compile a preliminary evaluation of the damage level.

2.1. Common Damages of Structures

Bridges are composed mainly of abutments, columns, deck (or arch), cables (if any), parapet/handrail and pavement. These elements are built of different materials such as masonry, reinforced concrete, metal, reinforced earth walls and cables. All these elements and materials are susceptible to damage. The following is a descriptive list of the most common damages affecting bridges. We include this list to keep the methodology comprehensible to scientists unfamiliar with this field of research:

• Leaching/efflorescence is a phenomenon that comprises the migration of a salt to the surface of concrete, leaving a coat of salt (Figure 3a,b). Efflorescence can present a white, brown, green or yellow color, depending on the type of salts.
• Lichen growth occurs when a stone-like material (masonry, concrete, bricks, etc.) support exhibits an excessive humidity (Figure 3c,e).
• Rust stains are due to lack of covering in structural reinforcement or to lack of covering in the formwork connectors (Figure 3f).
• Cracking is likely to occur when a structure cannot accommodate a movement (Figure 3b,d,h). Alternatively, cracks can appear because of other nonstructural issues (e.g., thermal effects). They can affect columns, beams, shear-keys, etc. and can present different configurations and distribution patterns. The location, length, orientation, and the opening of cracks are of paramount interest.
• Spalling is a term used to describe areas of concrete that have cracked and delaminated from the substrate. Steel reinforcement corrosion comprises the gradual destruction of the steel reinforcement of reinforced concrete structures by chemical and/or electrochemical reaction with their environment. It can affect any part of the reinforced concrete structure (Figure 3g).
• Structural steel corrosion comprises the gradual destruction of steel structural elements by chemical and/or electrochemical reaction with their environment (Figure 3c–e). It can affect any part of the steel structure. Special attention must be paid in suspension and cable-stayed bridges to the cables and anchorages. This Figure shows some examples of steel structures corrosion.
• Scaling of concrete surface comprises local peeling or flaking of a finished surface or hardening concrete because of abrasion, salt expansion, erosion or freeze/thaw action.
• Ground cracks in the ground-structure interface are developed by the relative movement of the structure and the surrounding ground.
• Constructive and thermal joints can suffer relative movements (opening or closing) because of deformations and displacements.
• Some structures can present anomalous tilts and displacements. Excessive displacements can put structural stability at risk.

2.2. Image Acquisition Protocol

Some recommendations are made for data gathering systematization for routine visual inspections based on the drones’ observations during the inspection flights. The systematization of this task is always important, even when the visual inspection is performed in situ by a structural engineer. It is even more important when the videos are being recorded by nonspecialist technicians or engineers. The following considerations should be accounted for in image acquisition:

• Before the inspection flight, it is necessary to establish a safety and security check/protocol. In this work it is followed the using steps defined by the European Union Safety Agency (EASA) https://www.easa.europa.eu/domains/civil-drones (accessed on 7 February 2022) and Specific country laws for drone flights must also be considered if applicable.
• It is recommended to carefully plan the time of the day to perform the image acquisition to get high-quality images, avoiding back-lit images, shadows, etc.
• Some parts of a bridge could be inaccessible for certain UAVs, such as the elastomeric bearing pads or the space between beams. To avoid this issue, it is recommended to combine small and large UAVs that can ensure a complete bridge inspection.
• Both laterals, top and bottom parts of the bridge should be inspected.
• It is recommended to systematize the order of image recording on a bridge inspection: from the first abutment to the last one, and all the intermediate columns. For example, the first abutment (A1) should be the one on the northwest. Its coordinates will determine the bridge location and the flight sequence A1-P1-P2-P3-…-Pn−1, Pn, A2 (Figure 4).

According to the above considerations, the proposed inspection sequence begins with a complete zenithal flight over the bridge deck in strips equal to or less than a specific width. This depends on the UAV model, its camera, and the size of the bridge. In addition, it should be established specifically for each case study. The UAV should fly from the first abutment (A1) to the last abutment (A2). If the width of the deck exhibits more than the specific width for the case study, a new flight should be completed to cover the whole deck in the opposite direction (i.e., from A2 to A1 and so on; Figure 4a). Flights can be segmented on long bridges.

Next, the bottom part of the bridge is inspected, namely, the abutments, the piers and the basal part of the deck (ceiling of the bridge, from below the bridge). This is the reason why the selected camera needs to possess a tight angle looking up (Figure 4). First, abutment A1 is inspected, followed by the bottom part of the first deck span, with a sequence of flights like that used for the top part of the deck. This procedure is repeated for each span of the deck until reaching abutment A2 (Figure 4c).

Several targets (markers) were printed and placed in an easily accessible place on the scene, forming a right angle, before photos were taken (Figure 4). These targets are used in photogrammetry software as reference points for coordinate system and scale definition or as a valid match between images to help camera alignment procedures. However, there is no good GPS signal under bridges. Because of this, the mission is done by a human operator following the steps defined in the proposed systematization of the image acquisition sequence. In this case, the drone has an anticollision system to avoid crashing into the structure while getting as close as possible.

Abutment inspection will be performed on all three visible sides, when possible in a counterclockwise direction and in a bottom-up approach on each abutment (Figure 5a).

Column inspection will be performed on all the four sides in an counterclockwise direction and in a bottom-up approach. The first side to be inspected will be the closest one to abutment 1 (Figure 5b). For circular columns, four positions, diametrically opposed, will be defined, the first being the one closest to abutment 1. It will be inspected in the same way as for rectangular section piles, counterclockwise and from bottom to top (Figure 5c).

2.3. Inspections Form

The systematization of data collection is a key task when the images are analyzed to avoid the loss of information and to make the information easier to understand. To do so, specific inspection forms have been developed, covering all types of bridges that are usually used in linear infrastructures.

First, a general form must be filled with the general information of the bridge. This form contains information regarding the type of bridge, main geometrical parameters, history, location, function, materials of the different parts of the bridge, photographs, etc. (Table S1 in Supplementary Materials (SM)).

Later, the inspection results of each bridge element (i.e., abutment, column, deck, cables, parapet/handrail or pavement) must be described in some additional forms selected according to the material of the specific element. In this work, to simplify the procedure as much as possible, we have defined the following types of materials: (M) masonry, (C) concrete, (S) metal, (REW) reinforced earth walls and (CB) Cables.

Thus, for each material, the elements, and the type of potential damage to be inspected is defined. To this end, a list of the most common damages that could affect every type of construction material and element and to which the UAV operator must pay attention during the inspection has been defined.

Once a bridge has been divided into different structural elements, the state of each must be evaluated according to its material.

As an example,

Table 1. Potential damages that can affect the different parts of a masonry bridge (M).

Logo (If Any)	Structural Element
Damage	Abutment	Column	Deck (or Arch)	Cables	Parapet/Handrail	Pavement
M1. Mortar joint degradation	X	X	X	-	X	X
M2. Biological growth in joints	X	X	X	-	X	X
M3. Stone degradation	X	X	X	-	X	X
M4. Efflorescence in joints	X	X	X	-	X	-
M5. Efflorescence in masonry stones	X	X	X	-	X	-
M6. Mineral precipitation in joints (speleothems)	X	X	X	-	-	-
M7. Lichen growth	X	X	X	-	X	X
M8. Loss and/or disengage of masonry stones	X	X	X	-	X	-
M9. Erosion or scour of columns and/or abutments	X	X	-	-	-	-
M10. Damp and seepage through abutments	X	-	-	-	-	-
M11. Displacement of the beam or the elastomeric bearing pad	X	X	-	-	-	-
M12. Elastomeric bearing pad degradation	X	X	-	-	-	-

summarizes the potential damage that can affect the different parts of a masonry (M) bridge and that must be inspected and evaluated from a damage perspective. The rest of the tables for each type of material (i.e., concrete, metal, reinforced earth walls and cables) and information about the general state of damage are included as Supplementary Material to this paper (SM: Tables S2–S6).

Finally, once the different elements of the bridges have been defined according to type of material, the inspection by the UAV and the subsequent analysis of the images/videos by structural engineers will allow for the definition of general damage according to the previously established criteria.

The criteria established in this methodology seek to simplify the decision-making process as much as possible, which enable engineers to systematize and automate the process in the future.

The final decision of the inspection is established as recommendations or intervention criteria. In this final decision, the need of taking actions for the inspected element should be stated based on three levels of risk (R1, none/low; R2, moderate; R3, high/very high).

The proposed intervention criteria are: (A) IC1, associated to level of risk R1 that recommends new regular inspections (with a frequency according to the age of the bridge); (B) IC2, associated to level of risk R2 that recommends a detailed inspection by a specialized engineer, with auxiliary means, including UAVs, if necessary; and (C) IC3, associated to level of risk R3 that recommends an urgent detailed inspection by a specialized engineer, with auxiliary means, including UAVs, if necessary.

The three levels of risk are obtained from the amount of damage spreading (A) and the level of damage (B) (Figure 2). It is variable for each type of damage, according to the relative importance for the safety of the bridge.

Following the same simplified guidelines proposed for a better systematization of the methodology, the damage spreading, defined as the extension of the observed damage of an element, is also defined in three steps: (A1) none/limited, (A2) moderate, and (A3) widespread. Finally, the also three levels of intensity, or level, of damage affecting the considered element of the bridge are classified as: (B1) none/low, (B2) moderate, and (B3) high/very high.

Succeeding the above example, for each type of damage of the elements (M1 to M12 of the masonry elements and shown in Table 1), the general damage must be stated according to the abovementioned criteria.

Table 2. Example of table for the evaluation of the state of damage of mortar joint degradation on masonry bridges (M1). See the remaining tables for all types of damage in Supplementary Materials.

Damage	A. Damage Spreading:	B. Level of Damage	R. Risk	IC. Intervention Criteria (Recommendation)
M1. Mortar joint degradation	1. None/limited: none or some joints (<10% of the area)	1. None/low: none or only discolouration	R1. None/low: A1/B1, A1/B2, A2/B1, A3/B1	IC1. New regular inspection (frequency according to the age of the bridge): R1
	2. Moderate: Some areas (10–50%)	2. Moderate: weathering (the mortar looks like a sandy soil)	R2. Moderate: A1/B3, A2/B2, A3/B2	IC2. A detailed inspection by specialized engineer is needed, with auxiliary means if necessary: R2
	3. Widespread (>50%)	3. High/very high: disintegration of joints	R3. High/very high: A2/B3, A3/B3	IC3. An URGENT detailed inspection by a specialised engineer is needed, with auxiliary means if necessary: R3

shows an example of damage M1 (mortar joint degradation). The first task is to observe the damage spreading, so it will be catalogued as A1 “none or limited” if the spreading is less than 10%, A2 “moderate” if it is between 10 and 50% and A3 “widespread” if it is over 50%. Then, the level of damage must be classified as B1 “none or low” if the level of damage is none or only discolouration, B2 “moderate” if weathering is observed and the mortar looks like a sandy soil or B3 “high or very high” if disintegration of joints is observed. Next, the risk is classified according to the combination of damage spreading and the level of damage. Risk is classified as R1 “none or low” for the following combinations: A1/B1, A1/B2, A2/B1 and A3/B1. It is classified as R2 “moderate” for A1/B3, A2/B2 and A3/B2 and as R3 “high or very high” for A2/B3 and A3/B3. Finally, the intervention criteria recommendation is defined depending on the risk classification. A new regular inspection is recommended for risk R1 (IC1), a detailed inspection for R2 (IC2) and an urgent inspection for R3 (IC3). The other states of damage for each masonry element are shown in the Supplementary Materials.

As mentioned above, for each type of damage and structural element affected, a different criterion is established for each of the levels of extension and intensity. This will be a function of the relative importance that these aspects present for the safety of the bridge. Likewise, combinations of these parameters (damage spreading, A, and level of damage, B) establish the level of risk, presenting different criterion depending on the element and the construction material in question, for the same reason stated above.

As a summary of the methodology defined above (Figure 2), we must: (1) complete the form of general information of the bridge; (2) divide the structure into different elements (i.e., abutment, column, deck, cables, parapet/handrail or pavement); (3a) for every element identified in step 2, identify the material; (3b) inspect the element paying attention to the potential type of damage (Table 1); and (3c) evaluate the state of damage of every element according to the information given in all sections (Table 2).

It should be noted that the proposed methodology only replaces the visual inspection. After the evaluation process, if it is concluded that there is a moderate or high/very high risk, a detailed inspection by a specialized engineer is needed, with auxiliary means if necessary.

2.4. Evaluation of Suggested Methodology: Case Study

The proposed methodology for the bridge inspection was applied to two bridges over railway infrastructure in the French region of Lavance-Épercy, close to the Swiss border. The first bridge is a masonry structure with a single arch, and the second one is a concrete bridge with reinforced concrete supports with a rectangular shape and a deck with a composite structure (concrete/steel).

The images used for the inspection were acquired by an unmanned aerial vehicle (UAV). This comprised a six-rotor multicopter model DJI Matrice 600 Pro (Figure 6). The maximum takeoff weight of this equipment is 15.5 kg with a payload of up to 6 kg. Stationary flight time is approximately 38 min without payload and 18 min with 6 kg payload. The maximum speed is 65 km/h (without wind) with a transmission range of up to 5 km. In addition, the system has an integrated anticollision system.

For the inspection of the first bridge, two inspection videos were used. The first video duration is 4 min and 32 s and the second one is 4 min and 25 s. For the inspection of the second bridge, three inspection videos were used. The three videos have a duration of 5:12, 5:07 and 4:20.

The geolocation of the drone can be also seen in the videos in the upper right-hand corner. In addition to the georeferenced videos, the raw videos without geolocation are also available for inspection if needed.

3. Results

The proposed methodology was tested on both bridges previously described. The results of the first and second evaluated bridges (overpasses) are shown below.

As previously described, the first document to be completed using the information provided by the UAV is the general information form (

Table 3. Form of general information from bridge 1.

Code: 2019/001
Reference: Demonstrator bridge 1: Masonry bridge
Bridge abutment 1 location (UTM coordinates)	X: 46.336143096035585
	Y: 5.6907070261430475
Slide 1 photograph	Slide 2 photograph
Top view photogram	Bottom view photogram
Construction date: Not available	Length of bridge: 10.8 m
Number of spans: 1	Maximum height of the columns: unknown
Traffic on the bridge: Railway Road vehicles Mixed Pedestrian	Relevance of the upper track: Moderate Medium High
Lower circulation: Railway Road Natural feature: river, gorge…	Relevance of the lower track (if any): Moderate Medium High
Bridge abutment type Masonry Concrete Reinforced earth wall	Bridge columns (piers) type Masonry Concrete Metal Composite None
Bridge superstructure (deck/arch) Masonry Concrete Metal Composite	Steel cables elements Cable-stayed bridge Suspension bridges None
Bridge road pavement Masonry Concrete Asphalt pavement Without pavement Not available	Parapet/handrail Masonry Concrete Metal Composite None
Bearing Composite neoprene/steel Lead None Not available	Expansion joint: Yes None Not available
Wing walls Concrete Metal None	Number of bridge columns (piers): 0

). Once this form has been completed, the different structural elements of the bridge must be inspected and evaluated according to the material type, as shown in the methodological section (see Figure 1).

The studied bridge has construction elements made of masonry and of metal. Damage catalogued as M6 (mineral precipitation in joints) and M7 (lichen growth) were identified in the masonry elements, both detected in the abutments and the arch.

Damage M6, corresponding to mineral precipitation in joints (speleothems), showed a moderate damage spreading (some areas about 10–50%) and the level of damage was moderate (light to medium mineral precipitation). Based on this, risk A2/B2 was assigned, which shows that the recommendation regarding the intervention criterion corresponds to new regular inspections whose frequency will be based on the age of the bridge: IC1 (Figure 7).

Damage M7 (Lichen growth) detected on the abutments and the arch of bridge 1 was catalogued as a moderate damage spreading (some areas about 10–50%) and the level of damage as moderate (medium-density colonization). Based on this, a risk A2/B2 was assigned, which shows that the recommendation regarding the intervention criterion corresponds to new regular inspections whose frequency will be assessed according to the age of the bridge: IC1.

Concerning the handrail elements made of metal, damage S1 (Corrosion of metal profiles, specifically metal gussets) was detected. This damage was catalogued as moderate damage spreading (some areas about 10–50%) and the level of damage as none/low (none or change in colour). Based on this, a risk A2/B1 was assigned, which shows that the recommendation regarding the intervention criterion corresponds to new regular inspections whose frequency will be assessed according to the age of the bridge: IC1.

As for the first bridge, the first document to be completed using the videos and images provided by the UAV is the general information form of bridge 2 (

Table 4. Form of general information about bridge 2.

Code: 2019/002
Reference: Demonstrator bridge 2: Concrete bridge
Bridge abutment 1 location (UTM coordinates)	X: 46.34248069440977
	Y: 5.692425433712043
Slide 1 photograph	Slide 2 photograph
Top view photogram	Bottom view photogram
Construction date: Not available	Length of bridge: 11.6 m
Number of spans: 3	Maximum height of the columns: Unknown
Traffic on the bridge: Railway Road vehicles Mixed Pedestrian	Relevance of the upper track: Moderate Medium High
Lower circulation: Railway Road Natural feature: river, gorge…	Relevance of the lower track (if any): Moderate Medium High
Bridge abutment type Masonry Concrete Reinforced earth wall	Bridge columns (piers) type Masonry Concrete Metal Composite None
Bridge superstructure (deck/arch) Masonry Concrete Metal Composite	Steel cables elements Cable-stayed bridge Suspension bridges None
Bridge road pavement Masonry Concrete Asphalt pavement Without pavement Not available	Parapet/handrail Masonry Concrete Metal Composite None
Bearing Composite neoprene/steel Lead None Not available	Expansion joint: Yes None Not available
Wing walls Concrete Metal None	Number of bridge columns (piers): 2

).

On bridge 2, construction elements of concrete, composite elements (metal plus concrete) and construction elements of metal were identified. The damage was catalogued as C2 (Concrete spalling because of reinforcement corrosion) in the concrete elements (abutment A1 and column P1, Figure 8). C3 damage (failures caused by impacts) affected only the column P1 (Figure 8a). Rust stains due to the lack of covering in structural reinforcement, named C4, involved the deck (Figure 9). Finally, damage C7 (Lichens growth) was identified on the abutments, piers, and deck (Figure 9b).

Damage C2, affecting both columns of concrete, presented a moderate spreading (some areas about 10–50%), and the level of damage was moderate (i.e., spalling revealed reinforcements in a length lower than 10 cm). Based on this, a risk A2/B2 was assigned, which indicates that a detailed inspection by a specialized engineer is needed, with auxiliary means if necessary: IC2 (Figure 8a).

However, the same damage C2 that involved abutments of concrete was catalogued as none/limited (i.e., none or some small spalling affecting less than 10% of the area). The level of damage of abutments was moderate and the risk was A1/B2. Therefore, it is recommended to perform new regular inspections (frequency according to the age of the bridge): IC1 (Figure 8b).

Damage C3, failures caused by impacts on columns, presented a low spreading, a moderate level of damage (i.e., spalling revealed reinforcements but reinforcements were not corroded) that corresponds to A1/B2 risk, requiring only new regular inspection (frequency according to the age of the bridge): IC1 (Figure 9a).

Lichen growth also affected the three elements of the bridge (i.e., abutment, column and deck). Damage spreading was moderate since only some areas were affected (10–50%) and the colonization exhibited a medium density that indicated a moderate level of damage. Therefore, a low risk exists, and the intervention criteria consists in a new regular inspection according to the age of the bridge (Figure 9b).

Damage C4 (i.e., rust stains due to lack of covering in structural reinforcement) was detected on the deck and exhibited moderate damage spreading, because only some areas were affected (10–50%). The level of damage was high/very high, since rust linear stains presented a width higher than 5 mm. Under these conditions the risk is moderate (A2/B3) and the recommended intervention criteria consists in a detailed inspection by a specialized engineer with auxiliary means if necessary (IC2, Figure 10a).

The last element affected was the handrail that presented corrosion of some metal profiles (S1). The damage consisted in changes in color. This implies that the risk is low; consequently, a new regular inspection is recommended (Figure 10b).

4. Discussion

The proposed methodology based on a systematization of the bridge inspection process by an UAV allows the elimination of conventional in situ inspections carried out by structural engineers in the field. This methodology was tested on two bridges in France. The proposed procedure is based on the analysis of images and videos acquired by an UAV. The captured information is then analyzed by structural engineers following a simple decision-making process. On the one hand, two structural materials were observed in the analysis of bridge 1: masonry and steel. Different damage was detected in these materials: corrosion of metal profiles (specifically metal gussets), mineral precipitation in joints (M6) and lichen growth (M7). Construction elements of concrete, composite elements (metal plus concrete) and construction elements of metal were observed on bridge 2. On this bridge concrete spalling because of reinforcement corrosion (C2) was identified in the reinforced concrete elements (i.e., abutment and column). Moreover, failures caused by impacts were detected affecting only column (C3), and rust stains due to lack of covering of structural reinforcement were identified on the deck (C4). Finally, lichen growth was identified on the abutment, the column, and the deck (C7). After the performed analysis, it can be stated that the proposed technique can detect damage on bridges as an alternative to a traditional in situ procedure.

It is important to point out that light conditions are of paramount importance to obtain high-quality images that enable the identification of the extent and severity of the damage. In this sense, it is recommended to handpick the time of the day for the inspection to avoid back-lit conditions, considering the season and the latitude of the site. Moreover, the quality of the camera is of great importance, not only its resolution but also lens, image sensor, exposure, contrast detection system, etc.

It is also strongly recommended to follow the proposed protocol for data acquisition to obtain a consistent collection of inspection forms.

Another important aspect of the performed inspections for testing the methodology is that there are some parts of the bridge without images, such as the elastomeric bearing pads. To collect images of this part was not possible during the inspection performed because of the size of UAV used for the inspections. To avoid this issue, it is recommended to fly with smaller and versatile drones that can access these parts.

The proposed methodology is simple and systematized to make both the workflow and the decision-making process easier. To make this possible, the construction elements and the type of damage were grouped. Moreover, a level scale of only three levels was used for defining the intervention criterion (i.e., damage spreading, level of damage and risk). These intervention criteria or final recommendations were reduced to three options. The adoption of this criterion has yielded a process that is both understandable and easily reproducible.

In addition, adopting this criterion can make a future process automation easier for inspection. In this sense, a machine learning system can be implemented for recognition of the structural elements, material textures, type of damages and their extension and levels of intensity, determining structural risk and providing an automatic response. Although the system must be supervised by expert engineers, bridge inspection could be more cost-effective, less time-consuming, and more generalised by using the proposed methodology. Currently, new research is working to apply these algorithms for structural damage detection, although they are usually referred to a single type of damage such concrete cracking of [30]. Implementing these systems needs a general methodology, as proposed here to provide a general vision of the inspection.

Moreover, images and videos from inspections could be easily stored and shared and can be checked by as many engineers and times as needed with no displacement of personnel required.

It is worth noting that although in some papers (e.g., [2,5]) the authors derive some quantitative parameters (e.g., cracks aperture, dip and length, etc.) from the UAV inspection, the proposed methodology is basically designed to perform a systematic preliminary qualitative description of the damage affecting a bridge and assessing the importance of the damage using a decision-making process based on expert image and video analyses and therefore, no quantitative validation has been performed.

The proposed methodology provides a cost-effective, safe, and reliable tool available for public administrations and infrastructure managers. Additionally, this methodology will allow for checking more structures in less time, increasing the structural safety of these elements in the future.

5. Conclusions

A novel proposed methodology for bridge inspection from UAV images was applied to two railway bridges in the French region of Lavancia-Épercy. The presented workflow began with the image acquisition by UAV flights, which allowed access to the highest parts of the bridges with security and detail. Afterwards, the images were studied by structural engineers, experts in forensic engineering, for damage detection. While watching the videos, some inspection forms designed ad hoc for the bridge inspection were filled in. Finally, a recommendation about the need of new and more in detailed inspections was stated.

This method has the major advantage of making it possible to safely, accurately and without the need for a specialised engineer to go to the site inspect a variety of infrastructure. Furthermore, the inspection rarely disrupts the normal operation of the infrastructure. Moreover, the videos can be watched as many times and by many engineers as needed, with no personal displacement and can be shared with other experts. In addition, the drone operator can record the images as he does any other railway inspection, such as the catenary wire, insulators, etc.

We have made some recommendations for data gathering systematization for routine visual inspections based on UAV observations during the inspection flights. The standardization of this task is always important, even when the visual inspection is done in situ by a structural engineer. It is even more important when the videos are being recorded by nonspecialist technicians or engineers.

It is recommended to perform inspections at a certain time of the day to obtain better images, avoiding back-lit images, shadows, etc.

The intervention criteria or recommendation is given in every inspection form completed. This recommendation is classified into three categories: (1) new regular inspection (frequency according to the age of the bridge); (2) a detailed inspection by a specialized engineer is needed, with auxiliary means if necessary; and (3) an urgent detailed inspection by a specialised engineer is needed, with auxiliary means if necessary.

This information is of paramount importance for the structural engineers and linear infrastructures managers since it allows to take decisions and to identify risky situations on bridges.