Developing Forest Road Recognition Technology Using Deep Learning-Based Image Processing

Abstract

This study develops forest road recognition technology using deep learning-based image processing to support the advancement of autonomous driving technology for forestry machinery. Images were collected while driving a tracked forwarder along approximately 1.2 km of forest roads. A total of 633 images were acquired, with 533 used for the training and validation sets, and the remaining 100 for the test set. The YOLOv8 segmentation technique was employed as the deep learning model, leveraging transfer learning to reduce training time and improve model performance. The evaluation demonstrates strong model performance with a precision of 0.966, a recall of 0.917, an F1 score of 0.941, and a mean average precision (mAP) of 0.963. Additionally, an image-based algorithm is developed to extract the center from the forest road areas detected by YOLOv8 segmentation. This algorithm detects the coordinates of the road edges through RGB filtering, grayscale conversion, binarization, and histogram analysis, subsequently calculating the center of the road from these coordinates. This study demonstrates the feasibility of autonomous forestry machines and emphasizes the critical need to develop forest road recognition technology that functions in diverse environments. The results can serve as important foundational data for the future development of image processing-based autonomous forestry machines.

1. Introduction

The efficient and sustainable management of forest resources is essential for wood production, a major economic activity worldwide [1,2,3,4,5,6]. The forestry industry is experiencing rapid technological advancements, including the introduction of autonomous machinery [7,8]. While these autonomous machines can reduce workforce requirements and improve efficiency, the complex road environments in forest areas pose significant challenges to their operation [9]. Forest road detection technology has emerged as a key solution to address this issue [10,11].

Forest road detection technology supports the safe and efficient operation of autonomous forestry machines under various environmental conditions [12]. Forest roads are characterized by a variety of obstacles, such as trees, leaves, rocks, and animals, as well as sudden terrain changes [13]. Consequently, the accurate detection and analysis of road conditions are crucial [14,15]. These capabilities play an important role in planning optimal routes, ensuring safe movement, and performing tasks effectively. Furthermore, forest road detection technology can greatly improve the efficiency of wood production. For example, it can reduce work time and fuel consumption by performing the real-time monitoring of road conditions and enabling logging trucks and other forestry machines to move along optimized routes within the forest area [16]. Not only does it help lower overall production costs, but real-time road condition monitoring also contributes to enhancing work speed and accuracy, preventing mechanical damage to vehicles, and reducing maintenance costs [17].

Environmental protection and sustainable forest management are paramount considerations in modern forestry [18]. In this context, forest road detection technology emerges as a pivotal element that can revolutionize wood production and the operation of autonomous forestry machines. This technology holds immense potential to enhance work efficiency, reduce transportation costs, safeguard worker safety, and contribute to the preservation of forest ecosystems [19].

Reviewing previous studies on the development of autonomous forestry machinery and forest road recognition technology, Ringdahl et al. [20] conducted a study on a forwarder capable of autonomous driving along pre-programmed routes using high-precision GPS and gyroscopes. Leidenfrost et al. [21] developed an autonomous driving system for forest roads based on a compass, ultrasonic sensors, and stereoscopic cameras. Fleischmann et al. [22] presented a new approach to forest road detection using stereo vision. Nakagomi et al. [12] developed technology to recognize forest road surfaces using LiDAR-SLAM and U-Net. Lei et al. [11] developed technology combining image processing and 2D LiDAR detection to detect and recognize irregular roads in forest environments, using the improved SEEDS SVM method for rapid image classification and recognition.

As demonstrated by these studies, active research is underway on autonomous driving-related technologies to enable self-driving on forest roads and in forest environments. In most of these previous studies, sensors such as LiDAR and GPS were used to develop autonomous driving systems. Sensors exposed to external environments are highly affected by environmental and meteorological conditions in terms of accuracy and reliability [23,24]. To achieve autonomous driving on forest roads, the system must automatically recognize the road irrespective of changes in the surrounding environment. However, detecting forest roads is quite challenging due to the diverse color changes based on environmental conditions (light, season, soil), and the typically irregular shape of the roads [25]. To overcome these challenges, researchers increasingly resort to deep learning-based image processing techniques, leveraging the advantages of deep learning-based object recognition methods in extracting irregular objects such as vehicles and people. In this study, we developed technology to automatically recognize forest roads using deep learning.

2. Materials and Methods

2.1. Image Collection

The study area is located in a forest road section near Jikdong-ri, Soheul-eup, Pocheon-si, Gyeonggi-do, South Korea (37°45′45.975″ N, 127°10′31.687″ E). Videos were collected while driving a tracked forwarder back and forth along a 1.2 km road section (Figure 1). Three cameras (DFK 36CR0234-I67, The Imaging Source, Bremen, Germany) were installed to obtain simultaneous video recordings from the left, right, and center viewpoints (Figure 2). These recordings captured the same driving route under various weather conditions. Each drive lasted approximately 20 min, resulting in a total of 120 min of video footage across six drives. Since the shape and brightness of the forest road seen in the captured image could be changed by the amount and the incidence angle of the light source, we tried to take photos on various ambient light conditions such as in the morning, and in the afternoon on three different levels of cloudiness as shown in

Table 1. Meteorological conditions on image collection dates.

Image Collection Dates	Meteorological Conditions
	Temperature (°C)	Humidity (%)	Average Cloud Cover (1/10)
11 September 2023	26.2	76	7
13 September 2023	25.9	91	3
16 October 2023	25.9	62	0

, considering the average amount of cloud as a meteorological variable. In addition, since wood harvesting work is not performed in rainy or snowy weather conditions because it is dangerous, experiments were conducted by varying only the light source and the average amount of cloud that can affect the angle of incidence. Frames were extracted from the videos at regular intervals, ensuring that images did not overlap during training. A total of 633 images were extracted from the videos recorded from the three directions. The extracted images were saved in JPEG format with a resolution of 1920 × 1080.

2.2. Image Dataset Composition and Annotation

The image size was adjusted to 640 × 480 pixels to facilitate the training and testing of the model. From the total 633 images captured under natural conditions, 533 were randomly selected and adjusted to meet the input requirements of the YOLOv8 model. The dataset was split into training (80%), validation (10%), and test (10%) sets. Once deep learning training was completed, the test set was used to evaluate the model’s performance with images captured under different conditions. To create mask images following the shape of forest roads, polygonal regions were manually annotated using the visual geometry group (VGG) Image Annotator program [26]. Figure 3a,b shows examples of manual masking along the contour of the forest road in the image.

2.3. YOLOv8 Segmentation

In general, deep learning-based object detection techniques integrate a classification step (to determine meaningful objects within the image) and a localization step (to estimate the location of classified objects). Widely used techniques include You Only Look Once (YOLO) and regions with convolutional neural network (R-CNN) features. Since Redmon et al. [27] first proposed “You Only Look Once: Unified, Real-Time Object Detection”, various algorithms based on YOLO have been developed, including YOLOv4, YOLT, and the latest version, YOLOv8. YOLOv8 shares the same architecture as YOLOv6 but incorporates significant improvements, such as a new neural network architecture that utilizes both feature pyramid network (FPN) and path aggregation network (PAN), along with new labeling, offering a variety of enhancements over previous versions [28]. YOLOv8 uses the modified CSPDarknet53 backbone as its feature extractor (Figure 4). The CSPLayer used in YOLOv5 has been replaced with the C2f module, and the spatial pyramid pooling fast (SPPF) layer accelerates computation by pooling image features into a fixed-sized map. Each convolution applies batch normalization (BN) and SiLU activation. The head module of YOLOv8 is divided into three branches, each independently responsible for a specific task: processing objectness, classification, and regression [29].

2.4. Deep Learning Training

In this study, the modified CSPDarknet53 was employed as the base network for feature extraction [30]. Deep learning architectures with 255 layers pose significant challenges, including the number of layers, activation functions, and hyperparameters to consider for each new problem (model), leading to extended training times. To address these limitations and reduce the number of forest road images required to train the entire network, transfer learning was applied using the Microsoft Common Objects in Context (MS-COCO) dataset, which comprises 80 object categories and 200,000 images.

To fine tune the weights for forest road detection, only the weights of the RPN, classifier, and mask generation parts of the network were trained. The training model received color images resized to 640 × 480 pixels, with bounding boxes and masks annotated around each forest road. The learning rate, weight decay, momentum, batch size, and number of epochs were set to 0.001, 0.0005, 0.937, 64, and 300, respectively. These hyperparameters were chosen to achieve optimal performance based on the model’s accuracy and the maximum resources of the hardware used for deep learning training.

Table 2. Hardware specifications used for deep learning training.

Hardware	Type
CPU	AMD Ryzen 7 3700X @ 3.6 GHz × 16
Memory	64 GB
GPU	NVIDIA GeForce RTX2080 Ti × 2

presents the specifications of the hardware used for deep learning training.

2.5. Deep Learning Evaluation Methods

To evaluate the forest road detection performance, we used precision, recall, F1 score, and mean average precision (mAP) as evaluation metrics. Precision is the ratio of true positives to the total number of instances predicted as positive by the model. Recall is the ratio of true positives to the total number of actual positive instances. In forest road detection, higher precision indicates higher detection accuracy, while higher recall indicates higher detection efficiency. The F1 score, which is the harmonic mean of precision and recall, provides an overall measure of detection performance. For object recognition tasks, mAP is commonly used as an evaluation metric, calculated by averaging the precision values for each class. For the evaluation, we randomly selected 60 images. In this study, an object is considered detected when the intersection over union (IoU) between the predicted bounding boxes and the ground truth bounding boxes is greater than or equal to 0.5. An IoU threshold of 0.5 (IoU ≥ 0.5) means that the two bounding boxes overlap by at least 50%, indicating a significant match. The precision, recall, and F1 score were calculated using Equations (1) to (4) based on IoU ≥ 0.5. Similarly, the mAP was calculated based on an IoU ≥ 0.5, meaning that only objects with an IoU of 0.5 or higher were considered detected.

$$\mathrm{Precision}= {\displaystyle \frac{{\mathrm{T}}_{\mathrm{p}}}{{\mathrm{T}}_{\mathrm{p}}+{\mathrm{F}}_{\mathrm{p}}}}$$

(1)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}={\displaystyle \frac{{\mathrm{T}}_{\mathrm{p}}}{{\mathrm{T}}_{\mathrm{p}}+{\mathrm{T}}_{\mathrm{n}}}}$$

(2)

$$\mathrm{F} 1 ={\displaystyle \frac{2\times \mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\times \mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}+\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}}$$

(3)

$$\mathrm{m}\mathrm{A}\mathrm{P} = {\displaystyle \frac{1}{\mathrm{N}}}\sum _{\mathrm{i}=1}^{\mathrm{N}}{\mathrm{A}\mathrm{P}}_{\mathrm{i}}$$

(4)

where

• ${\mathrm{T}}_{\mathrm{p}}$ = the number of true positives (correct detections);
• ${\mathrm{T}}_{\mathrm{n}}$ = the number of true negatives (correct rejection);
• ${\mathrm{F}}_{\mathrm{p}}$ = the number of false positives (false detection);
• ${\mathrm{F}}_{\mathrm{n}}$ = the number of false negatives (miss);
• $\mathrm{N}$ = the number of classes or categories, and;
• ${\mathrm{A}\mathrm{P}}_{\mathrm{i}}$ = APi is the average precision (AP) for class i.

2.6. Image-Based Forest Road Center Extraction Algorithm

YOLOv8 segmentation was used to separately extract regions containing masked forest roads. An image post-processing algorithm was then added to compute the center of the extracted forest roads. This center extraction algorithm employs functions provided by Python-based OpenCV, as detailed in Figure 5.

The regions detected as forest roads by YOLOv8 segmentation were color masked and then filtered through an RGB color filter. Subsequently, grayscale conversion was applied to create a binary image. The binarized image, as shown in Figure 6, was used to detect the start and end points of the left and right edges of the forest road through a histogram. To visually represent the endpoints of the extracted forest road, a green line was drawn, and the central values of the left and right coordinates were marked in yellow. The red line indicates the center of the camera’s horizontal axis, which corresponds to the center of the forestry machine.

Using this method, the center of a forest road can be extracted by identifying the edges in an image recognized as a forest road using the deep learning-based image processing. This center is necessary for calculating the lateral offset required for autonomous driving. The developed image-based forest road center extraction algorithm was validated using real videos of forest roads filmed while driving on straight sections and around corners curving left and right. The field applicability of this algorithm was assessed using real videos of driving on forest roads filmed in environments not used in the deep learning training.

3. Results

3.1. Results of the Trained YOLOv8-Based Model

The training loss values of the YOLOv8 model across different epochs are plotted in Figure 7a,b. Overall, the loss values show a consistent downward trend, indicating that the model’s performance progressively improved during training. Convergence to low and stable loss values across all loss types further demonstrates the model’s effective learning from the datasets.

Figure 8 shows an example image in which YOLOv8 segmentation successfully detects and segments a forest road. The blue-colored area depicts the identified forest road segment. Only the segmented results of the objects that meet the confidence threshold of IoU ≥ 0.5 are outputted.

Table 3. Performance evaluation of YOLOv8 approaches on forest road.

Approach	Precision	Recall	F1-Score	mAP
YOLOv8	0.966	0.917	0.941	0.963

presents the classification performance metrics for the trained model, showing excellent results with a precision of 0.966, a recall of 0.917, an F1-score of 0.941, and an mAP of 0.963.

3.2. Results of the Image-Based Forest Road Center Extraction Algorithm

The areas recognized as forest roads were separately extracted using the image-based forest road center extraction algorithm. The road edge lines were detected using a histogram. Figure 9b shows the image with the road area blue-masked on the original image in Figure 9a. The masked image was then processed with an RGB filter, changing only the blue areas to white and the rest to black (Figure 9c). This binarized image is used for generating a histogram, representing the sum of the pixels along the X-axis. Due to perspective, a road in the histogram typically appears in a trapezoidal shape, and the coordinates of the left and right edges of the trapezoid base are calculated and extracted.

The extracted center points are used as the extracted center of the forest road. The difference between an unknown center, such as the center of a vehicle, and the extracted forest road center, converted into the lateral offset of the vehicle, can be used as the autonomous driving–steering ratio applicable as a reference for autonomous driving and steering. Figure 9e illustrates the final image output through the image-based forest road center extraction algorithm.

3.3. Verification of Image-Based Forest Road Center-Extraction Algorithm

Figure 10 illustrates the results of applying the image-based forest road center-extraction algorithm to images captured while driving on forest roads, including straight sections and corners curving left and right. Despite using images captured under different conditions to those used for deep learning model training, the algorithm successfully recognized these road features and correctly extracted the road center (Figure 10a–c).

However, the algorithm encountered difficulties when the forest road had forks or multiple paths. In these situations, the algorithm recognized all visible paths in the image, leading to challenges in extracting a single center (Figure 10d). Additionally, roads with ambiguous shapes or objects resembling roads can cause misidentification, as seen in Figure 10e,f.

To reduce such recognition errors, the deep learning model’s performance can be improved by providing additional training data that includes images of various forest roads and environments (e.g., different weather conditions and lighting variations). Alternatively, sensor fusion with other sensors, such as GPS and IMU, can be employed to overcome the limitations of image processing alone and enhance the reliability of autonomous driving systems.

4. Discussion

This paper developed a road recognition technology using deep learning-based image processing to identify forest roads and extract their boundaries to introduce autonomous forestry machinery. The trained deep learning model demonstrated high accuracy and reliability, with a precision of 0.966, a recall of 0.917, an F1-score of 0.941, and an mAP of 0.963. An algorithm was additionally developed to extract the center of the forest road using the forest road detected by deep learning. This algorithm detects the coordinates of both ends of the road and extracts the center based on these coordinates. Using forestry machinery equipped with autonomous driving systems offers numerous advantages, such as reducing labor and preventing accidents, which leads to more active research and the development of such systems. Ringdahl et al. [20] developed an autonomous driving system for forwarders using Gyro and DGPS. Leidenfrost et al. [21] proposed a method for autonomous driving on forest paths without GPS using a compass, ultrasonic sensors, and stereoscopic cameras. Nakagomi et al. [12] researched and developed technology to recognize the surface of forest roads by integrating Li-DAR-SLAM and U-Net. Giusti et al. [31] showed the possibility of autonomous UAV driving along a forest trail using deep learning-based image classification using DNN. Çalışkan and Sevim [13] used deep learning (CNN) to extract only forest paths from high-resolution orthoimages, and as a result of comparing four models, the model using ResNet-50 showed the best performance with an accuracy of 98.49%. Since forestry machinery typically travels along designated forest roads, most autonomous driving technologies utilize the coordinates of the forest road or detect the road itself. Although deep learning-based image processing is generally more robust than conventional methods, it can still misrecognize shapes such as wheel tracks and vines (as shown in Figure 10e,f). It is necessary to improve the performance of the deep learning model by capturing additional forest roads to overcome these limitations. Additionally, there were instances, as shown in Figure 10d, in which two forest roads were both recognized at a fork. It needs an algorithm to decide which way to go based on the path planning assisted by additional devices such as a GPS or a predefined map.

Further research is needed to enhance the stability of the autonomous driving system by integrating sensors such as GPS and LiDAR. This integration can help overcome the limitations of image processing technology and achieve more reliable autonomous driving in the forestry sector.

5. Conclusions

This study focused on developing forest road recognition technology for autonomous forestry machines using deep learning-based image processing techniques. This was achieved by developing an algorithm that applies the YOLOv8 segmentation model to recognize roads and extract their center. This model demonstrated high accuracy and reliability, with a precision of 0.966, a recall of 0.917, an F1 score of 0.941, and an mAP of 0.963. Additionally, an image-based algorithm was developed to extract the center from the forest road areas detected by the YOLOv8 segmentation model. This algorithm involves detecting the coordinates of both road edge lines through RGB filtering, grayscale conversion, binarization, and histogram analysis. The road center is then calculated based on these coordinates.

The results of this study demonstrate the feasibility of autonomous forestry machines and highlight the importance of developing forest road recognition technology for various forest environments. The technology presented here serves as a crucial foundation for improving the autonomous driving capabilities of forestry machines in the future. Future research directions include collecting additional data to improve the deep learning model’s performance in recognizing forest roads under diverse environmental conditions. Additionally, research is needed to explore sensor fusion with GPS and LiDAR to address the limitations of image processing techniques.

This study has laid an important technological foundation for the development of autonomous forestry machines. This technology is expected to significantly contribute to sustainable forest management and enhance the efficiency of forestry machinery.