In-Motion, Non-Contact Detection of Ties and Ballasts on Railroad Tracks

Abstract

This study aims to develop a robust and efficient system to identify ties and ballasts in motion using a variety of non-contact sensors mounted on a robotic rail cart. The sensors include distance LiDAR sensors and inductive proximity sensors for ferrous materials to collect data while traversing railroad tracks. Many existing tie/ballast health monitoring devices cannot be mounted on Hyrail vehicles for in-motion inspection due to their inability to filter out unwanted targets (i.e., ties or ballasts). The system studied here addresses that limitation by exploring several approaches based on distance LiDAR sensors. The first approach is based on calculating the running standard deviation of the measured distance from LiDAR sensors to tie or ballast surfaces. The second approach uses machine learning (ML) methods that combine two primary algorithms (Logistic Regression and Decision Tree) and three preprocessing methods (six models in total). The results indicate that the optimal configuration for non-contact, in-motion differentiation of ties and ballasts is integrating two distance LiDAR sensors with a Decision Tree model. This configuration provides rapid, accurate, and robust tie/ballast differentiation. The study also facilitates further sensor and inspection research and development in railroad track maintenance.

1. Introduction

Various studies have aimed at identifying, detecting, and assessing the condition of railroad ties and ballasts, mainly using vision systems or ground-penetrating radars [1,2,3,4,5]. The emphasis is often placed on how ballast and tie degradation may increase the likelihood of a derailment [6,7,8,9,10]. Other studies have used alternative technologies for making such assessments, such as the research by Zhao et al. that has developed and fabricated a laser-speckle strain sensor to measure the transfer length of concrete ties [11].

The mentioned techniques are non-contact and have the potential to be mounted onboard Hyrail vehicles or manned and unmanned track geometry cars for autonomous, in-motion assessment of tracks. Their goal is to provide an early warning to track engineers for intervention before any track fault progresses to costly maintenance issues. These studies can be collectively classified as track health condition monitoring research. However, these methods lack the ability to filter out unwanted targets—for example, tie inspection systems often struggle to ignore ballasts, reducing their overall effectiveness.

The primary purpose of our study was to develop an automated method for differentiating between ties and ballasts, allowing health monitoring systems to focus more accurately on their respective targets during in-motion operations. This study was mainly aimed at distinguishing between ties and ballasts as part of a more extensive study to assess the early stages of track instability from tie vibrations. Obviously, to determine tie vibrations, one needs first to differentiate them from ballasts; hence, this study serves as a preliminary step in developing non-contact, in-motion devices for inspecting ties or even ballasts. The purpose of this system is to serve as an auxiliary unit that enhances the performance of other track stability inspection devices. By providing real-time, high-accuracy differentiation between ties and ballasts, this system helps to narrow the focus of other inspection tools, enabling them to more effectively target specific track components.

The existing methods used to distinguish between ties and ballasts are mainly based on ultrasonic or vision methods. For instance, Datta et al. [12] developed an in-motion method for reconstructing the deflection profile of railroad ties by means of non-contact ultrasonic testing. To demarcate the ties, they use a vision-based image classification approach. Sabato and Niezrecki [13] also used Digital Image Correlation (DIC) for a similar goal. Bojarczak et al. [5] developed an algorithm based on a deep neural network for semantic segmentation of ties and ballasts and eventually detecting ballast unevenness. Despite these methods’ effectiveness, a few issues exist, preventing them from being installed on standard track geometry cars. First, their effectiveness for onboard applications is limited because they are mainly intended for stationary or quasi-static implementation. Second, they require significant post-processing, making them unsuitable for real-time or near-real-time measurements.

To further advance tie and ballast detection methods, this study evaluated the application of LiDAR-based methods that can be used at speeds suitable for Hyrail vehicle implementation. The following sections will describe the system, its setup on a robotic track cart for technological feasibility evaluation testing, the assessment of the test results through developing new data analysis methods, and the application of the final system.

2. Setup

The experimental evaluation evaluated various methods and hardware implementations to achieve the study’s goal of differentiation between ties and ballasts in motion. A remote-controlled track cart, shown in Figure 1, was constructed and used to house the required sensors and electronics [14]. A rail cart was selected due to its significant advantage over other potential solutions, such as aerial vehicles and legged robots, because of their poor performance in railroad environments and lack of ability to be tested over extensive distances. This rail cart can successfully replicate Hyrail vehicles and track geometry cars that are mainly used for track inspections. It is a stable platform that allows for continuous monitoring of track conditions as the cart moves.

The cart is equipped with a traction-braking system and can run on various railroad tracks. The cart’s frame is made of 80/20 extruded aluminum, offering flexibility in mounting multiple sensors and data acquisition units. Its flexibility allows efficient testing across diverse railroad environments, including branch lines and mainlines. This flexibility is crucial for the research objectives, as the cart can easily navigate various track conditions while ensuring that sensors maintain a stable orientation towards their targets for accurate data collection. Additionally, it features a “kill switch” to enable emergency braking if needed. Its tapered and flanged wheels provide a stable run up to 10 mph (~16 km/h) in both forward and reverse directions. Furthermore, the cart is designed to fit inside the bed of a pickup truck, facilitating easy transportation to various track locations.

3. Methodology

Various sensors, configurations, and data analysis methods were developed and explored to achieve optimal accuracy, efficiency, and robustness, ensuring that the system complies with railroad standards. A set of sensors that exhibit promising potential for this study will be discussed, followed by a series of tests for selecting the most suitable sensors and the required data analysis for tie/ballast differentiation.

3.1. Sensors

The sensors selected for the study are shown in Figure 2. They include an inductive proximity sensor for ferrous materials, distance LiDAR sensors, temperature and humidity sensors, and accelerometers. The inductive proximity sensor is on the far-left side of the cart, right on top of the tie plates. Two distance LiDAR sensors point downward on the right and left sides of the cart. Humidity, temperature, and acceleration are ancillary sensors for possible application of the track cart in other railroad-related system developments, such as evaluating the migration of flange grease on rails and assessing track stability through Doppler LiDAR sensors. Because these studies are outside this paper’s scope, these sensors will not be discussed here.

The sensor selection was based on an early evaluation of various sensors, in which some were rejected and others selected because of their promise of yielding satisfactory results. The sensor selection was based on availability, cost-effectiveness, ease of integration, and effectiveness in identifying ties or ballasts. The sensors that were not selected were infrared temperature sensors (or cameras) and time-of-flight LiDAR sensors. They were found to be too slow for in-motion measurements, even at low speeds. Others, such as time-of-flight acoustic sensors, radars, and surface reflectivity/intensity sensors, were proven to provide insufficient functional resolution to capture tie and ballast patterns definitively.

The sensors selected for further integration into the system and track evaluation were an inductive proximity sensor for ferrous materials and distance LiDAR sensors, as will be discussed further next.

3.1.1. Inductive Proximity Sensor for Ferrous Materials

Ties are typically connected to rails with metal fasteners like spikes and tie plates. One way to detect ties and identify their locations is to detect tie plates and possibly spikes using inductive proximity sensors for ferrous materials. Typically, an “inductive proximity sensor” is equipped with a Hall effect probe, which triggers a pulse when it detects a ferrous object in its proximity (Hall effect sensors, named for the physicist Edwin Hall, incorporate one or more Hall elements, generating a voltage proportional to an axial component in a magnetic field. They are commonly used in proximity sensing, positioning, speed detection, and current sensing applications in many industrial devices). For rail applications, the sensor can be positioned above the tie plate and spikes, and the generated pulses can be used to identify tie locations.

3.1.2. Distance LiDAR Sensors

LiDAR, Light Detection and Ranging [15], is a remote sensing method that uses a pulsed laser which can be triangulated to measure the distance from the target point to the lens itself [16]. Ties and ballasts have different surface characteristics to LiDAR, with ties featuring a smoother and ballasts having a rougher pattern. This contrast can be detected in motion using downward-facing vertical LiDAR sensors to identify alternating tie and ballast sections.

Two Keyence IL-600 sensors from Keyence corporation of America (Itasca, IL, USA) were adopted for this study due to their off-the-shelf availability and cost-effectiveness. The Keyence IL-600 is a precise distance sensor with a nominal accuracy of 50 μm and a response time of 0.33, 1.0, 2.0, or 5.0 ms. The fastest response time ensures enough spatial resolution (up to 1.3 cm) for the sensor to record sufficient samples for each tie (about 16 samples) at speeds of up to 40 mph (~65 km/h). A higher number of samples per tie is needed to raise the confidence in the measured data on each tie, with 3–4 samples potentially being the minimum. The required 200–1000 mm installation range provides sufficient standoff distance for most railroad applications.

As shown in Figure 3, two sensors, designated as “right” and “left” sensors, were implemented in line with each other in the lateral direction to enable the collection of data on two parts of the track. Using two sensors allows data to be gathered at two locations on ties and ballasts, reducing the likelihood of false positives and negatives. In addition, having two LiDAR sensors would enable us to develop more sophisticated data analysis methods based on measurements from both sensors. This will be elaborated further in the Analysis section. Although not part of the objectives of this study, the sensors could possibly be used to detect track anomalies, such as sunk, tilted, or bowed ties, by comparing their outputs with each other. For instance, a tilted or bowed tie can be identified from the data by comparing the left and right sensor measurements. If they return different values over a tie, this means that the tie is tilted. The same procedure can be followed for different anomalies.

3.1.3. Tachometer

A tachometer is also employed to monitor the location and speed of the rail cart along the track. This device facilitates the transformation from the time domain to the spatial domain. The tachometer comprises an 18-tooth gear and a Hall effect probe. Along with the cart’s wheel diameter of 7.5 inches, a linear resolution of 1.66 cm can be achieved, as derived from Equation (1). The tachometer is attached to the front shaft of the cart, which is a passive (idler) shaft. This configuration was chosen because the active (driver) shaft is directly connected to the DC motor, causing it to experience slippage (rotation without corresponding linear movement) more often, which could result in false information. Thus, using the passive shaft minimizes the occurrence of erroneous readings.

$$Linearresolution={\displaystyle \frac{1}{numberoftooth}}\times \pi \times {\displaystyle \frac{wheeldiameter}{2}}$$

(1)

3.2. Data Acquisition Unit

A single data acquisition unit with 16 A/D channels was selected to record the necessary data throughout track tests. The data were sampled at 1000 Hz. The 1000 Hz sampling frequency was selected based on conducting a series of tests in the early stages of the study to assess the most beneficial sampling rate. The sampling frequency provides approximately 11 samples per tie, assuming an 8-inch-wide tie (~20 cm) and a forward speed of 40 mph (~65 km/h), representing the higher range of the intended speed for measurements.

4. Experiments

Three groups of tests were conducted: in the laboratory, on a revenue-service track at walking speeds, and in a simulated track environment at high speeds. These tests aimed to thoroughly investigate the applicability and effectiveness of the proposed system. The laboratory testing focused on evaluating the sensors’ ability to capture tie and ballast surface figures in a controlled setting. The revenue-service track testing assessed the system’s performance in an uncontrolled environment. Finally, high-speed testing on the simulated track evaluated the system’s performance under high-speed conditions resembling those of in-motion inspections. The data collected from these experiments were used to develop robust data analysis methods for distinguishing ties and ballasts based on sensor measurements.

4.1. Laboratory Testing

The system was tested on a 40 ft track panel (~12 m) at the Railway Technologies Laboratory (RTL). The robotic rail cart, equipped with all the mentioned sensors, ran over a small section of the track with ties and ballasts to collect data. As shown in Figure 4, the left distance LiDAR sensor was put on an unballasted section with only ties to establish reference measurements for the ties. These measurements were used as the “ground truth” for another sensor on the right with both ties and ballasts. In parallel with the LiDAR sensors, an inductive proximity sensor was used to detect the ties. This sensor was used for two purposes: first, to evaluate its applicability for differentiating between ties and ballasts; second, to verify the accuracy of the LiDAR measurements in distinguishing between the ties and ballasts.

The tests were intended to evaluate the system’s general capability and the applicability of the sensors in tie/ballast detection at low speeds in a controlled environment. Unless the LiDAR’s measurements are sufficiently consistent and repeatable, its success for field measurements will be doubtful because of the uncontrolled environment common in track measurements.

Figure 5a shows the measured distances from LiDAR sensors to their target surface (tie, ballast, or ground) and an inductive proximity sensor. Note that the left distance LiDAR measurement is used as the ground truth. The terms “activated” and “null” for the inductive proximity sensor are used to indicate when the sensor is detecting a tie plate (hence, a magnetic field is generated) and when no tie plate is detected and a magnetic field is not generated. As shown in Figure 5a, the system accurately captured the surface profile of the traversed track, down to its finest details. The recorded variations are highly tangible and clear, allowing one to easily visualize the track layout. This level of precision stands in contrast to other similar instruments mentioned before, which often suffer from low resolution or slow sampling rates, whereas the LiDAR sensors used in this system demonstrate superior accuracy and agility. The data in Figure 5a show irregular and noisy measurements over ballast sections (unshaded areas) with more significant and frequent variations. In contrast, the tie sections (orange-shaded areas) exhibit smoother and relatively flat measurements compared to the ballast sections. The measurements also show the surface height differences between ties and ballasts, which agree with a visual inspection of the track. The variations observed in the inductive proximity sensor measurements are attributed to the surface irregulates of the tie plates, as verified by visual observations.

Figure 5a shows that the inductive proximity sensor does not detect the first half of the second tie. The reason for this is illustrated in Figure 5b, which shows that the first half of the left tie plate is sunk into the tie, making it out of the inductive proximity sensor’s range. In contrast, the right tie plate is not sunk and remains in the sensor’s measurement range; therefore, it is detected. Inductive proximity sensors tend to have a small working range that requires putting the sensor as close as possible to the tie plate (maximum: 15 mm), increasing the likelihood of interference with objects that may be present on a track. Although there are inductive proximity sensors with more extensive working ranges, their magnetic fields could get diverted to the rail, significantly increasing the rate of false positives. At the conclusion of the laboratory tests, we deemed the inductive proximity sensors unsuitable for revenue-service testing despite their success in the controlled laboratory tests.

4.2. Track Testing

The track tests were performed in two different settings. First, the rail cart ran and collected data on a branch line with a 115RE rail that is used infrequently and has somewhat degraded tie and ballast conditions. Second, it was set on a main line with a 136RE rail with far better tie and ballast conditions. These tests were conducted to assess the system’s performance in recording the necessary data at higher speeds, over more extensive distances, and in more realistic environments, including challenging conditions, such as those encountered on branch lines, which could represent worst-case scenarios.

Figure 6 shows a sample section of the track testing for each setting. The raised and smooth surfaces represent the ties, while the lower and more irregular surfaces are the ballast. The highlighted sections mark the locations of the ties. As the plots indicate, the surface tracks are well-captured by the sensors. Note that the left and right measurements are slightly shifted to enable easier differentiation between the two. Plotting them without the shift would place them nearly on top of each other, making it more difficult to identify any differences. Additionally, some data drop-ins observed during branch-line track testing (Figure 6a) were caused by sunlight interference with the lasers. This issue was addressed in future tests by adding shade covers to block the sunlight. As shown in Figure 6b, there were no drop-ins during the mainline testing because the covers were installed for this test. Additionally, the mainline testing results in Figure 6b display smoother tie surfaces compared to the branch-line testing. For example, some cracks are visible in Figure 6a but not in Figure 6b.

It must be noted that the track tests in both settings were limited to walking speeds, in the range of approximately 1.5 to 4.0 mph (2.4 to 6.4 km/h). The track cart was remotely controlled by an operator who was following it. Although the cart can move at speeds of up to 10 mph (16 km/h), the test speeds were limited to speeds the operator could walk while safely negotiating the ties and ballasts.

4.3. High-Speed Testing

Although the earlier branch-line track testing indicates that the LiDAR system can successfully differentiate between ties and ballasts, these runs were performed at low speeds, usually less than 4 mph (6.4 km/h). The low speeds were due to the speed limitation of the track cart used for the tests and the need for safe operation while following the cart. Due to lack of access to track time and standard Hyrail vehicles, it was decided to answer the question about how well the system can perform at higher speeds by conducting a series of tests with a road vehicle over a simulated track consisting of alternating wood planks and ballasts. The main concern was the adequacy of the selected sampling rate, specifically, having sufficient data points on the surface of the ties and ballasts to adequately identify them with low numbers of false positives and false negatives.

The simulated track on an asphalt surface outside the RTL facility and the system installation on the rear of a Chevy Silverado pickup truck are shown in Figure 7. Figure 7a displays the simulated track that consisted of 13 planks with the same width as an 8-inch tie (~20 cm) and a length of 2 ft (~0.66 m). The planks were painted to resemble the tie surface color. The space between the planks was filled with ballast nearly 3 to 4 cm lower than the top surface of the planks to provide a clear distinction between ties and ballasts in the data. Figure 7b shows the left and right LiDAR sensors mounted to an aluminum structure attached to the hitch of a Chevy Silverado pickup truck. This installation proved to be accessible and adequate for the intended tests. This setup was chosen to ensure safe operation while closely replicating real-world conditions during high-speed testing with Hyrail vehicles. A road vehicle, due to its nature, introduces more stochastic vibrations compared to standard Hyrail vehicles, as they have a more flexible suspension system [17,18], presenting a worst-case scenario. However, after initial testing, it was determined that the system effectively mitigated the impact of these unwanted vibrations, still capturing satisfactory data. As shown in Figure 8, despite the vehicle’s vibrations, the sensors successfully recorded the surface figure of the simulated track.

Figure 8 shows a segment of the measured distances from the LiDAR sensors to the surface of the ties and ballasts at 19 and 37 mph (30 and 59 km/h). Comparing the measurements with the track’s setup indicates that the sensors correctly identified the width of the ties and the gaps between them, which were filled with ballast. Beyond the sample measurements shown in Figure 8, we performed tests at speeds ranging from 4 to 37 mph (6.4 to 59 km/h), specifically at 19, 31, and 34 mph (30, 50, and 55 km/h). Although not included here, for brevity, the results for other speeds are similar to those shown in Figure 8, and they similarly indicate the success of the LiDAR system in sampling the ties and ballast sections.

Interestingly, the results in Figure 8 and other measurements indicate that the LiDAR sensors successfully capture irregularities associated with tie and ballast surfaces, with the ballast exhibiting more significant surface variations than the ties, as expected. Comparing Figure 8a,b shows a notable consistency in measurements. The tie surface features in Figure 8b closely resemble those in Figure 8a, proving the system’s repeatability. For example, the irregular surface of the first tie, caused by a piece of tarp on top of it, was detected in both tests. It is worth mentioning that the ballast surface has many associated uncertainties, so data from those sections were not expected to be repeatable. For safety reasons, shade covers were not used to block sunlight during this series of tests, resulting in some data drop-ins observed during high-speed testing (Figure 8). However, these drop-ins can be neglected, as they comprise less than 1 percent of the data.

5. Analysis

Although the unprocessed (raw) LiDAR measurements are distinct enough to the human eye for tie and ballast classification, an automatic differentiation method is needed to process large datasets resulting from track measurements over extended distances. The purpose of this section is to explore various data analytic methods that can process the LiDAR sensor measurements, enabling accurate and efficient differentiation of ties from ballasts in an automated manner.

First, a statistical data analysis method (i.e., moving standard deviation) is evaluated because of its effectiveness in extracting surface variation characteristics. Next, two machine learning (ML) methods, namely Decision Tree and Logistic Regression, are considered to better deal with data uncertainty and ambiguity. Additionally, several preprocessing techniques are investigated to transform the LiDAR measurements into more suitable parameters for training the ML models. The downsides and upsides of each model are elaborated, and the model shown finally to be the best in terms of accuracy and robustness will be selected for analyzing the LiDAR data for tie and ballast differentiation.

5.1. Moving Standard Deviation

As noted earlier, based on visual observation of the track and the measurements, tie surfaces are expected to have fewer variations than ballast surfaces because of their different surface characteristics, irrespective of their height level. This approach uses surface variance to distinguish between ties and ballasts. A moving window over discrete data sections is used to calculate the standard deviation within each window, denoted as “moving standard deviation” here. Equation (2) represents the standard deviation formula. Figure 9 illustrates the process of moving standard deviation. The red bracket with the size of $ws$ moves along the data and calculates the standard deviation of the data within the “i” data points within the window (i.e., $x_i$). $\overline{x}$ denotes the average of the data points, $x_i$.

$$Standarddeviation=\sqrt{{\displaystyle \frac{\sum _{i=1}^{ws}{\left(x_i-\overline{x}\right)}^2}{ws-1}}}$$

(2)

It is expected that the resulting moving standard deviation would enforce the surface characteristics in the ties and ballasts, leading to more confidence in differentiating between them than would be possible from the individual data points, $x_i$. The standard deviation over the ties is expected to be lower than over the ballasts, enabling us to identify each.

Figure 10 shows a sample plot for the mainline track testing mentioned earlier. The left axis shows the measured distance from the LiDAR sensor to the tie and ballast surfaces. The right axis shows the moving standard deviation for a spatial window of 4 inches (~10 cm). A spatial window is used because the measurements are made at various speeds that would yield differing numbers of samples on tie and ballast surfaces. Using a spatial window, such as one over 4 inches, would enable a more direct comparison between measurements at different speeds by relating them to a physical reference, such as one-half of the nominal width of a tie. The moving standard deviation (the purple line) exhibits an interesting pattern: it reaches a trough toward the center of the tie and a peak near the boundary between the ties and ballasts. This is because the data near the center of the tie are more consistent and have fewer variations, leading to lower standard deviations. In contrast, the measurements at the interface between the ties and ballasts are highly varied, leading to larger standard deviations.

The newly introduced parameter, moving standard deviation, is more interpretable for computers in distinguishing ties and ballasts compared to using measured distances. This method captures surface variations more effectively and enforces new distinct patterns associated with ties and ballasts (local minimums and maximums), enabling easier differentiation between ties and ballasts than before.

Even though this method proved to be successful in differentiating between ties and ballast on the mainline track, it may prove inadequate for branch lines where the variation in data is larger than in the sample data used here. Specifically, large variations in tie surfaces, such as cracks and cuts or displaced and titled ties, can adversely affect the accuracy of the results. Additionally, the ambiguities in correctly identifying the troughs and peaks could lead to errors in autonomously identifying ties and ballasts. A more capable data analysis approach is desired to address these uncertainties.

5.2. Machine Learning Approaches

Another approach for detecting and learning data patterns and better differentiating between ties and ballasts is machine learning (ML). ML models can generally handle highly uncertain tasks, making them well-suited for this task. Due to the high capability of ML models in handling large datasets and automating decision-making processes, the newly developed models are expected to address the challenges previously associated with using the moving standard deviation. With the extensive amount of data collected through our experiments, these models can learn complex patterns and adapt to various conditions, improving the accuracy and reliability of tie and ballast differentiation. The desired ML model will receive the LiDAR data as input and return a classification of tie or ballast as output (prediction).

Like the moving standard deviation method, a moving window is used to process the data at each location. However, unlike the moving standard deviation, which uses a spatial window, this method defines the window size based on the number of surrounding data points required to classify an individual point as tie or ballast. The window size is a tunable parameter that needs to be optimized to achieve the best results. To select the required window size for our model, we evaluated the accuracy that can be achieved for each method for various window sizes, ranging from small to large. This task considers two standard classification “algorithms”: Logistic Regression and Decision Tree. Through a series of initial tests, these two algorithms were selected from a larger pool of classification algorithms, including Support Vector Machines (SVMs), k-Nearest Neighbors (k-NN), and neural networks. Logistic Regression and Decision Tree were chosen because they belong to two distinct families—linear and non-linear, respectively—and demonstrated higher accuracy [19].

Logistic Regression uses a linear function to model the relationship between features (i.e., inputs) and a binary outcome. It then applies a sigmoid function to convert this linear output into a 0 or 1 probability, indicating the likelihood of belonging to a specific class [20]. A “0” probability suggests a high likelihood of ballast, and “1” indicates a tie. Equation (3) shows the formula associated with Logistic Regression, where $b_0+b_{1}x$ is the linear function for separating the classes (i.e., ties and ballasts). $b_0$ and $b_1$ are the parameters that will be learned through the training process.

$$p={\displaystyle \frac{1}{1+e^{-(b_0+b_{1}x)}}}$$

(3)

In contrast, as Breiman et al. (1984) described, the Decision Tree creates a tree-like structure, where each internal node splits the data based on a chosen feature value. This process continues recursively until the data at each leaf node belong predominantly to a single class [21].

These algorithms differ in their interpretability. Logistic Regression provides a single equation representing the entire model, making it relatively straightforward to understand the pattern of each input. On the other hand, a Decision Tree offers a series of branching rules that often provide a more intuitive understanding of how decisions are made, as the tree visually represents the decision process.

Both Logistic Regression and Decision Tree are supervised learning algorithms, meaning they require training with input data and their respective output labels. In other words, the models need ground-truth labeled data to learn the patterns associated with each label (tie or ballast) and effectively differentiate ties from ballasts. Once trained, these models can predict future unseen data. A small portion of each dataset (approximately 20 percent) is manually labeled as either tie or ballast (i.e., 1 or 0) to construct a ground-truth “labeled dataset” to facilitate this. A portion of the labeled dataset (approximately 60 percent), called here the “training dataset”, is used to train the models. The performance of these trained models is then evaluated using the remaining 40 percent of the labeled dataset, referred to as the “test dataset”. Various train–test splits, ranging from 50–50 to 90–10, were tested, and the 60–40 split was found to maximize accuracy while minimizing overfitting. Finally, the final trained model was applied to the unlabeled dataset to identify ties and ballasts.

Initially, the measured distances from one of the LiDAR sensors (e.g., the left sensor) to tie or ballast surfaces were used as “input” to train the ML models using both Decision Tree and Logistic Regression algorithms. The difference between the left and right LiDAR sensor measurements was explored as an input for training the models. Finally, to improve the models further, the standard deviation of the difference between the left and right measurements was also evaluated to train the models.

In summary, the measured distance from the LiDAR sensors to the tie or ballast surfaces was considered to be the “input” for training the ML model in the first place and then gradually upgraded to the difference between the left and right LiDAR sensors’ measurements, and then the standard deviation of the difference between the left and right measurements was used to achieve higher accuracy and robustness. In Section 5.3, the logic behind these models will be described further, and the most efficient and robust model will be identified. Figure 11 illustrates how six different models were developed based on three different inputs (i.e., preprocessing methods) and two different algorithms.

5.3. Application of Machine Learning to LiDAR Data

The primary objective of this section is to explore various parameter configurations to develop the most optimal and robust model for analyzing the LiDAR sensors’ measurements for tie and ballast differentiation. Three essential parameters have been identified that significantly impact the model’s performance: the algorithm, the input for training the ML models, and the window size. As previously mentioned, the two algorithms under consideration are Logistic Regression and Decision Tree, and the three inputs are illustrated in Figure 11. These six models will be evaluated across different window sizes based on their accuracy. Here, “accuracy” is assessed by Equation (4).

$$Accuracy={\displaystyle \frac{Numberofdatapointsthatarecorrectlyclassifiedbythemodel}{Numberofdatapointsinthetestdataset}}$$

(4)

This section is divided into three subsections, each dedicated to analyzing one input. For each input, both algorithms (Logistic Regression and Decision Tree) are considered across a wide range of window sizes.

5.3.1. Measured Distance from LiDAR Sensors to Tie/Ballast Surfaces

First, for simplicity, we used the measured distances from LiDAR sensors to tie/ballast surfaces for training the machine learning models for both Logistic Regression and Decision Tree algorithms. Figure 12 shows a sample of distance LiDAR measurements from the mainline track testing. Distinct patterns for ties (orange-shaded) and ballasts (unshaded) are evident throughout the data, which were explored by the models.

The models were assessed across various window sizes, ranging from 5 to 55 data points. This range was determined to be sufficient after several trial-and-error iterations. Larger window sizes would lead to higher accuracies but require more data points. Therefore, selecting the smallest window size to achieve the necessary accuracy is advantageous. Figure 13 shows the performance of ML models using Decision Tree/Logistic Regression over different window sizes when trained on the measured distances from LiDAR sensors to tie/ballast surfaces. As shown in Figure 13, the Decision Tree algorithm exhibits higher accuracies than Logistic Regression. Additionally, the accuracy for both algorithms increases with the increase in window size. Both algorithms, however, reach a plateau at higher window sizes.

The analysis shows that the tie/ballast differentiation achieved by both algorithms can be negatively influenced by the condition of ties (e.g., elevated, sunk, tilted, or cracked) or factors such as their position, elevation, and alignment. To remedy this issue, we evaluated other approaches, such as using the difference between the left and right LiDAR measurements to gauge the two measurements against each other and possibly make the ML models less sensitive to tie conditions.

A model using the Decision Tree algorithm, trained on the measured distances from the LiDAR sensors to tie/ballast surfaces with a window size of 40, was applied to an unseen (unlabeled) section of the data. In Figure 14, the blue line represents the model’s predictions, with 1 indicating a tie and 0 indicating ballast. At a specific location in this dataset, two ties with cracks are present, as shown in Figure 14. The model struggles to distinguish between ties and ballasts in these cracked sections.

5.3.2. The Difference between Left and Right LiDAR Sensor Measurements

The difference between the left and right LiDAR sensor measurements could be used as input to train the model. This approach is expected to enhance the signal-to-noise ratio, as ballasts typically exhibit more significant surface irregularities in lateral, longitudinal, and vertical directions. While significant differences in surface figures between a tie’s left and right sides (i.e., where the sensors are pointed) are not expected, the ballast sections’ left and right sides can vary considerably. Hence, by comparing—or differencing—the left and right LiDAR measurements, more regularities in the tie sections and irregularities in ballast sections can be enforced in the data, facilitating a more accurate distinction between the two.

Figure 15 shows a sample section of the difference between the left and right LiDAR sensor measurements. Comparing Figure 15 with Figure 12, it is evident that the surface figure variation between ties (orange-shaded) and ballasts (unshaded) is now much more pronounced here.

Figure 16 shows the performance of ML models using Decision Tree/Logistic Regression over different window sizes when trained on the difference between the left and right LiDAR sensor measurements. Figure 16 proves that ML models perform better when trained on the difference between the left and right LiDAR sensor measurements than before by showing a slight increase in accuracy based on the Decision Tree algorithm across all window sizes. The improvement is attributed to the improved classification of abnormal ties as ties. However, Logistic Regression could not learn the pattern from these data, since it employs a simple linear classifier; hence, its results are not shown. As expected, using the difference between the left and right measurements as input to the models could result in a more accurate and reliable model. A similar trend to that observed in Figure 13 can be seen in Figure 16, where accuracy increases with larger window sizes and reaches a plateau at higher values.

5.3.3. Machine Learning on the Standard Deviation of the Difference between the Left and Right Sensor Measurements

Although the previous method demonstrated good performance, there is potential to achieve even higher accuracy. Another approach could be combining standard deviation with machine learning to combine the strengths of both methods. Machine learning excels in classification tasks by learning patterns from different classes to distinguish them. Meanwhile, the standard deviation method reveals more precise and more distinct patterns in a dataset. Integrating these two approaches may achieve better classification performance. The idea is to apply the “moving standard deviation” mentioned earlier to the difference between the left and right sensor measurements.

Figure 17 shows the standard deviation of the difference between the left and right sensor measurements from the previous subsection. The data have clear and distinct patterns over ties (orange-shaded) and ballasts (unshaded).

Figure 18 shows the performance of ML models using Decision Tree/Logistic Regression over different window sizes when trained on the standard deviation of the difference between the left and right sensor measurements. Figure 18 reveals that the Decision Tree algorithm achieves accuracies comparable to those obtained using the difference between the left and right sensors. Additionally, it performs more consistently across various window sizes, demonstrating robust performance at different speeds. This is because the Decision Tree algorithm does not rely on the number of data points, which can vary at higher speeds due to a constant data acquisition rate, resulting in fewer recorded data points. Logistic Regression, which previously performed poorly, now exhibits behavior similar to the Decision Tree algorithm, demonstrating improved and consistent accuracy across different window sizes.

Training the machine learning model using the Decision Tree algorithm on the standard deviation of the difference between the left and right LiDAR sensor measurements yields the highest possible accuracy, regardless of the window size. The final model, identified as Model 6 in Figure 11, with a window size of 35, was selected for its excellent performance in tie/ballast differentiation compared to the others. Figure 19a shows the model’s predictions over an unseen section of the data in a blue line, with 1 indicating a tie and 0 indicating ballast. The high accuracy and precision of the model in distinguishing ties and ballasts are visually evident in this figure. In the entire sampled section, all the ties and ballasts are successfully identified, and the boundaries of each are marked with high precision. The two ties marked with black stars are the same ties shown in Figure 14. The model now performs significantly better than the previous version.

Figure 19b presents the confusion matrix for the model, where 95.2% of the ballast data points are classified as ballast, and 92.4% of the ties are classified as ties. The high and closely matched percentages demonstrate the model’s effectiveness in accurately classifying both ties and ballasts. The total accuracy of this model is approximately 93.8%. Note that all the accuracies and percentages are calculated for the test dataset.

6. Application

The final developed system is capable of accurately distinguishing ties and ballasts in real time while mounted on moving platforms such as Hyrail vehicles and track geometry cars. It operates at speeds of up to 40 mph (65 km/h), which aligns with the operating speed of Class I railroads and performs successfully across various track conditions. This system can be integrated with other track inspection technologies, including ground-penetrating radar, infrared sensors, ultrasonic sensors, and Doppler LiDARs, to focus on their respective targets more effectively. Many of these devices, currently limited to stationary or quasi-stationary applications due to their inability to differentiate between targets, can now perform in-motion analysis over extended distances with the help of this system, facilitating railroad maintenance, reducing costs, and enhancing safety.

7. Conclusions

The goal of this research was to develop a robust, accurate, and effective method for identifying ties and ballasts—a crucial step before implementing inspection approaches that rely on differentiating between ties and ballasts in motion. The system studied here uses two downward-facing distance LiDAR sensors on a moving rail cart. The LiDAR sensors measure the differing Doppler effects caused by ties and ballasts.

Seven techniques were proposed to process the distance of the LiDAR sensors’ measurements, starting with the moving standard deviation method aimed at extracting surface figure variations of the track. Despite its simplicity, this method proved unreliable and unsuitable for automated detection. Subsequently, two machine learning algorithms—Logistic Regression and Decision Tree—and three preprocessing methods (inputs) were explored to automatically differentiate between ties and ballasts. Among these six ML models, the one with the Decision Tree algorithm exhibited exceptional accuracy, robustness, and efficiency, even at high speeds, mainly when applied to the standard deviation of the difference between the left and right LiDAR sensors’ measurements; furthermore, unlike computer vision systems designed for similar purposes, this system requires significantly fewer computational resources, making it more feasible for real-time implementation.

In conclusion, the developed system integrates two distance LiDAR sensors (left and right) with Decision Tree algorithms applied to the standard deviation of the difference between the sensors’ measurements, providing an effective and practical solution for non-contact, in-motion differentiation of ties and ballasts. This system holds the potential to be used onboard Hyrail vehicles, offering a valuable tool for differentiating between ties and ballasts during track inspection or health monitoring, facilitating more targeted and efficient maintenance interventions.