Tree Stem Detection and Crown Delineation in a Structurally Diverse Deciduous Forest Combining Leaf-On and Leaf-Off UAV-SfM Data

Abstract

Accurate detection and delineation of individual trees and their crowns in dense forest environments are essential for forest management and ecological applications. This study explores the potential of combining leaf-off and leaf-on structure from motion (SfM) data products from unoccupied aerial vehicles (UAVs) equipped with RGB cameras. The main objective was to develop a reliable method for precise tree stem detection and crown delineation in dense deciduous forests, demonstrated at a structurally diverse old-growth forest in the Hainich National Park, Germany. Stem positions were extracted from the leaf-off point cloud by a clustering algorithm. The accuracy of the derived stem co-ordinates and the overall UAV-SfM point cloud were assessed separately, considering different tree types. Extracted tree stems were used as markers for individual tree crown delineation (ITCD) through a region growing algorithm on the leaf-on data. Stem positioning showed high precision values (0.867). Including leaf-off stem positions enhanced the crown delineation, but crown delineations in dense forest canopies remain challenging. Both the number of stems and crowns were underestimated, suggesting that the number of overstory trees in dense forests tends to be higher than commonly estimated in remote sensing approaches. In general, UAV-SfM point clouds prove to be a cost-effective and accurate alternative to LiDAR data for tree stem detection. The combined datasets provide valuable insights into forest structure, enabling a more comprehensive understanding of the canopy, stems, and forest floor, thus facilitating more reliable forest parameter extraction.

1. Introduction

Precise forest parameters play a pivotal role in facilitating efficient forest management and conservation [1]. Understanding the structure, health, and composition of a forest stand not only improves its management but also enables effective forecasting of future trends. This becomes especially crucial due to the increasing stressors on forest ecosystems induced by the effects of climate change and human-driven land use conversions, which lead to a decline in forest health and coverage in several parts of the world [2,3]. In particular, managing at the individual tree level is gaining increasing importance for conducting targeted interventions, guiding forest development, and ensuring sustainable forest management under the intensifying impacts of climate change and usage pressure [4].

With the growing availability of earth observation data and techniques, forest monitoring from local to global scales is becoming feasible. Satellite remote sensing data, with their respective spectral and temporal resolution, can be utilized for a wide range of forest applications [5]. However, their coarser pixel size limits their use for forest monitoring analysis on an individual tree level. Light Detection and Ranging (LiDAR) techniques, such as mobile laser scanning (MLS), have been shown to provide high-resolution data with detailed information on the structure of individual trees (please refer to the Abbreviations) [6,7]. Nonetheless, the use of MLS is impractical for large areas, and its cost-intensive equipment makes it inaccessible to many users. Although LiDAR tools mounted on unoccupied aerial vehicles (UAV) can overcome some of the spatial limitations of MLS and generate high-density forest point clouds [8,9], their accessibility within the forestry sector is still limited due to cost constraints.

For local forest monitoring, UAVs equipped with continuously improving optical camera systems have demonstrated their potential as a cost-effective and user-friendly tool for deriving various forest parameters [10,11]. Operating at low flight altitudes, they provide images with very high geometrical resolution of a few centimeters, enabling analysis on individual tree level and even capturing within-variability of single tree crowns [12]. UAV data have, therefore, been used to derive a range of forest parameters quantifying the vertical and horizontal forest structure—such as tree height, breast height diameter, crown shapes, and canopy gaps—categorizing tree species, estimating tree health and above-ground biomass, or detecting diseases [13,14,15].

1.1. UAV Imagery Processing Using Structure from Motion

One of the great advantages of UAV data is their ability to generate 3D information, such as 3D point clouds, from 2D drone imagery applying photogrammetric processing steps, commonly known as structure from motion (SfM) [16]. During data acquisition, highly overlapping images are captured, providing different perspectives on the same ground spots. Prominent feature points are extracted from each image, and features corresponding to the same 3D point are matched in the overlapping regions of different images. Aerial triangulation, such as bundle adjustment, is then applied to define camera positions and orientations, as well as to obtain 3D geometry, creating tracks from a set of matched features. Based on these estimated camera and image positions, densification algorithms (dense stereo matching) can be used to generate dense 3D point clouds. From the point cloud, a digital surface model (DSM) can be derived and, by projecting the single images using the DSM, an orthomosaic can be generated [16,17]. These data products can be further used to extract specific forest parameters. The UAV-SfM approach has the potential to provide both geometric and spectral datasets, serving as input data for various forest parameter extraction algorithms. While methods integrating geometrical point and spectral image data are increasingly used in the field of forestry, most studies rely on LiDAR data rather than on UAV-SfM point clouds [18,19].

1.2. UAV-Data-Based Products for Tree Crown Delineation

In order to analyze tree parameters on an individual tree level within a forest stand, it is necessary to first segment single trees. This involves delineating the projected tree crown, which can be identified in the orthomosaic or height models, as separate objects. This individual tree crown delineation (ITCD) serves as the foundation for various subsequent analysis steps, including tree species classification, environmental and forest monitoring at the tree level, and the extraction of individual tree parameters directly from remote sensing data [20,21,22,23]. Over the past few decades, numerous ITCD methods have developed, based on generalized characteristics of trees or forests. Most of these methods can be applied to either 2D raster such as orthomosaic, 2.5D raster such as canopy height models (CHMs), or 3D data products such as point clouds. However, there are certain methods that specifically rely on 3D data and cannot be used when only 2D data are available. These methods often originate from the field of laser scanning and are increasingly being tested for point clouds derived from SfM as well [24].

Most methods typically assume a similar—hemispherical to conical—tree shape, with one tree top located at the center of the crown. Due to its exposed position, the top of the tree receives the highest solar radiation, resulting in the highest intensity and brightness values [20,22]. Algorithms are expected to yield better results for forests with a sparse canopy, lower species diversity, and similar age structure. These characteristics are more commonly found in managed forests and in coniferous, savannas, or tundra forest systems [22].

The analysis is often divided into two parts: tree detection, which involves identifying the position of a tree trunk, and delineating the (entire) tree crown [20]. In the following, some of the more commonly used methods for tree detection and delineation will be presented.

Local maxima and region growing: This method builds upon the previously mentioned canopy characteristics. Initially, individual trees are detected by identifying local maxima, which ideally represent tree midpoints. These maxima can be based on both CHM values and brightness values [20,22]. Difficulties may arise when defining the search radius for local maxima, which is derived from pixel size and average crown diameter and due to the fact that, in reality, tree crowns are not symmetrically aligned around a central point. Smoothing filters can reduce unwanted noise within the maxima [23,25,26,27]. Wulder et al. [28] propose using varying local maxima search radii, each based on the semivariance of the pixel.

Starting from these initial seed points, neighboring pixels or objects that exhibit similarity are added to the crown objects until a termination criterion is met, indicating that the crown has been delineated. This process is known as region growing [20].

Valley-following approach: The valley-following approach, initially introduced by Gougeon [29], consists of two parts. The first part involves classifying the areas between the individual tree canopies, while the second part utilizes a rule-based method to refine these classified areas [29]. These intermediate areas are referred to as valleys. They are characterized by higher shading, resulting in lower intensity and brightness values compared to the surrounding areas or represent local minima values [25,30]. According to Ke and Quackenbush [20] and Workie [23], relying solely on this shade-following approach often leads to incomplete separation of individual trees because, depending on tree density, not all crowns are adequately separated from each other by “valleys”.

Watershed segmentation: The watershed segmentation, first described by Beucher and Lantuéjoul [31], is a form of image processing segmentation. It also draws upon the topological analogy of the canopy described earlier. In this method, the values of a gray-scale layer are inverted and “flooded”, starting from local minima (tree tops). The resulting individual watersheds are separated from each other by dams, creating distinct segments [21,22,32]. Markers can be used in this process, representing the local minima from which the “flooding” originates, ideally representing tree tops. The resulting segments are supposed to delimit the individual tree tops [23]. According to Derivaux et al. [32], a common challenge in this method is over-segmentation, which can be mitigated through targeted marker placement, application of smoothing filters, or through the combined use with region-merging techniques and other methods.

Template matching: Template matching is a method that can be employed to detect individual tree crowns when the tree crowns exhibit similar shapes and comparable spectral values. Templates are created based on a gray value layer, representing patterns of typical tree shapes and values, mostly averaged. Both radiometric and geometric properties of the crown are utilized, and different viewing angles can be considered. The template is compared to all possible tree points, with high correlation values above a defined threshold representing individual trees [21,22,25,33].

Deep learning methods: Deep learning methods, such as instance segmentation, are increasingly being utilized for the detection and delineation of single tree crowns [34]. Commonly used model architectures are Mask R-CNN [35], artificial neural networks [15], or U-nets [36]. These methods offer advantages such as the ability to use multi-band images as input, instead of relying solely on a single band, and a focus on textural features, which is beneficial as adjacent trees might have very similar spectral properties. However, a disadvantage is the need for training data, which are typically obtained through time-consuming manual crown delineation and/or field work.

Point cloud-based methods: Terrestrial/airborne laser scanning and UAV-SfM provide 3D point cloud data that can be directly used for the single crown delineation. Most studies focusing on 3D methods utilize LiDAR data, as UAV-SfM does not penetrate dense crown structures well, especially during leaf-on season. As a result, UAV-SfM leaf-on point clouds tend to be less dense for lower canopy and forest floor areas. Various point-cloud-based approaches have been applied for ITCD, including K-means clustering [37], template matching [38], voxel space approaches [39], and mean-shift segmentation [40]. Deep learning models are trained using reference point clouds and have been used to segment single trees [39], employing frameworks such as PointNet [41].

Several recent studies applied one or multiple of the above-mentioned methods to detect tree positions and perform crown delineation using UAV-based imagery data products. A selection of relevant studies is presented in

Table 1. Selection of recent studies on tree detection and crown delineation in forest ecosystems using UAV optical data.

Paper	Forest Characteristics	UAV Imagery Type	Camera Angle	Tree Detection Algorithm	Crown Delineation Algorithm	Description
[42]	Dry conifer forest, low tree density	DJI Phantom 4 Pro (RGB)	Nadir & oblique	Local maxima on CHM	-	Various flight altitudes, patterns, and camera angles were tested, and better accuracy values were achieved by combining crosshatch flight patterns with nadir camera angles. The maximum F-score values ranged between 0.429 and 0.771.
[43]	Broadleaf forest, test sites with different stand densities	DJI Phantom 4 Pro (RGB)	Nadir	Local maxima on CHM	Region growing + inverse watershed segmentation	The highest overall accuracy (F-score = 0.79) was obtained for the low-density stand by applying a region growing algorithm on the CHM. Accuracy also varied among different tree species, with the best results obtained for Caspian poplar and the lowest for Persian ironwood. In high-density stands, the crown delineation results could be improved by applying weak gaussian filtering to the CHM.
[44]	Mixed conifer forest, open canopy	DJI Phantom 3 (RGB)	Nadir	Local maxima on CHM	-	Fixed window sizes were used in the local maxima filter, and it was observed that accuracy decreased as the filter sizes exceeded 1 × 1 m. Challenges in tree detection were specifically noted in steep areas and regions with high canopy closure. DTMs obtained through SfM tended to overestimate height in dense vegetation in comparison to DTMs derived from airborne laser scanning.
[45]	Mixed-conifer forest, moderate density	DJI Phantom 4 (RGB)	Nadir, oblique & composite	Variable window filter (VWF), 3D point cloud-based algorithms	-	Different flight parameters (altitude, camera angle, and image overlap), SfM processing settings (depth filtering, alignment, and dense cloud quality), and tree detection algorithms (CHM smoothing, VWF parameters, and point-cloud-based methods) were investigated. Higher accuracies were achieved at high flight altitudes (120 m) and with high image overlap (90%). The combination of nadir and oblique imagery resulted in detection rates worse than using only nadir data. CHM-based VWF methods produced the most accurate results, with F-scores up to 0.664 (trees > 10 m) and 0.826 (trees > 20 m).
[46]	Pine tree plantations	DJI Mavic Pro (RGB)	Oblique	Local maxima on CHM	-	Prior to tree detection using local maxima, the CHM was mean-filtered with a user-defined filter size. Accuracy of up to 0.78 (F-score) was achieved.
[47]	Spruce-pine forest	DJI Phantom 4 Pro, Parrot Disco-Pro Ag & DJI Matrice 210 (RGB & multispectral)	Nadir	Local maxima on CHM	Watershed segmentation	Using consumer-grade cameras yielded higher tree detection rates and more accurate crown diameters compared to multispectral cameras. Cameras with higher spatial resolution performed better at higher flight altitudes, whereas the opposite was observed for cameras with lower resolution. The best results were achieved with the DJI Phantom 4 RGB drone, detecting 84% of the trees correctly. The mean absolute error of crown diameters derived was 0.79–0.99 m (Phantom 4, RGB) and 0.88–1.16 m (Zenmuse X5S).
[48]	Mixed conifer forest, open canopy	DJI Phantom 3 (RGB)	Nadir	Local maxima on CHM	-	Different window sizes for local maxima detection were tested, and the performance of smoothed and non-smoothed CHM was compared. Lower window sizes for local maxima and smoothing proved to be more successful in detecting trees. The overall F-score value was 0.86.
[49]	Forest plantations, high canopy density	DJI M600 Pro (5-lens oblique)	Oblique	Adaptive-/fixed-kernel bandwidth mean-shift (AMS/FMS), region growing on CHM	AMS, FMS, region growing on CHM	Kernel bandwidth was defined based on canopy properties and applied to the mean-shift tree detection and delineation algorithm. The AMS method outperformed FMS and seed-based region growing methods, achieving an overall accuracy of ≥0.72 for tree detection and a relative RMSE of ≤0.13 for crown width.
[50]	Orchard yard/naturally wooded pasture/urban trees, low tree density	DJI Phantom 4 (RGB)		Single shot detector (SSD) deep learning model on raster data (returning bounding boxes around tree position)		Several additional datasets were derived from the RGB and DSM and were used for training purpose. SSD was used for tree detection and species classification. Ensembled models with different input datasets generally demonstrated higher performance compared to models based on only one type of input data.
[51]	Conifer and mixed regenerating forest stands, low tree density	DJI Phantom 4 Pro (RGB)	Nadir	-	Mask R-CNN on raster data	Mask R-CNN were trained using manually delineated crowns; pretrained networks were also incorporated. An average F1-score of 0.91 was achieved. ITCD remained more challenging for heterogeneous and denser forest stands, as well as for smaller crowns.
[35]	Conifer plantation, moderate tree densities	DJI Phantom 4 (multispectral)		Local maxima on RGB and CHM	Marker-controlled watershed segmentation, Mask R-CNN on raster data	The performance of local maxima, marker-controlled watershed segmentation and Mask R-CNN were compared. Local maxima and watershed algorithms scored the best results when applied to the CHM. Overall, Mask R-CNN outperformed the classic algorithms.
[52]	Subtropical broadleaf forest, moderate tree density	DJI Matrice 600 (RGB & hyperspectral)		-	Watershed-spectral-texture-controlled normalized cut	An extension was made to the CHM-based watershed segmentation to reduce over-segmentation. Objects segmented by the watershed algorithm were clustered based on the normalized cut criterion, considering both spectral and textural information.

.

As emphasized by several of these studies, detection and delineation of tree crowns in complex forest structures with dense canopy closure and overlapping tree crowns remains challenging [35,51]. Some studies focus on ITCD in forest with low to moderate canopy closure or open forest stands [42,48,53], as well as on forest plantations characterized by a more regular tree spacing and structure [44,46]. These types of forest tend to facilitate the detection of crown boundaries and the number of trees, as most algorithms perform better in homogeneous forest stands with lower canopy closure [19].

The objective of this study was to develop a method that focuses on the utilization and combination of leaf-on and leaf-off UAV-SfM data to derive stand parameters such as stem positions and crown delineation of individual trees within dense, structurally diverse deciduous forest environments. An approach was devised to extract stem positions using the leaf-off point cloud. These stem co-ordinates were subsequently combined with leaf-on datasets to derive crown delineations. Furthermore, a systematic accuracy assessment has been implemented to distinguish error sources in stem detection. This assessment involves comparing the extracted stem positions, a manually digitized UAV stem position dataset, and an independent MLS-based reference dataset.

2. Materials and Methods

Figure 1 illustrates the overall workflow that was developed and applied in this study. UAV RGB imagery of the forest was acquired twice: once on 4 August 2021 during leaf-on conditions (May–October) and once on 29 March 2022 during leaf-off conditions (November–April). Using an SfM approach, point clouds, orthomosaic, and a DSM were computed. A ground classification from the leaf-off data was used to normalize the point cloud and generate a digital terrain model (DTM). From the DTM and DSM, a CHM was extracted. A slice of the normalized leaf-off point cloud is used as input data for the tree stem detection method. Different reference datasets have been acquired for accuracy assessment. The ability of the UAV data and the proposed algorithm to detect stems has also been analyzed based on the respective tree type, such as overstory, understory, or tilted trees. In a second step, the extracted stem co-ordinates, leaf-on orthomosaic, and CHM are used as markers to derive individual crown delineation through a region growing algorithm.

In the following paragraphs, the workflow will be described in detail, outlining the steps taken to acquire and process the UAV imagery, collect reference data, derive stem positions and ITCD, and validate the obtained results.

2.1. Study Site Hainich National Park

The present analysis was conducted within the Hainich National Park in Thuringia, central Germany (Figure 2). The study area, known as “Huss site”, is located within the forest stand of the national park and has an area of 28.2 ha with an average tree cover density of 91% [54]. The center of the area is located at 10°26′5″E and 51°4′47″N. Within the test side Huss, a wide range of ecological investigations are conducted and several instruments for long-term forest monitoring have been established [55]. Surrounding the national park is the Hainich forest hill chain, which spans an area of approximately 20,000 ha. To the south, the mountain range borders the Thuringian Forest. To the east, it slopes towards the Thuringian Basin, while, to the west, it slopes relatively steeply to the valley of the Werra River. The elevations in the area range between 225 and 493 m a.s.l. [55,56,57].

The Hainich National Park covers an area of 7520 ha and includes the southern part of the natural area. It was established in December 1997 [58] and has been a UNESCO World Heritage Site of ancient and primeval beech forests of the Carpathians and other regions of Europe since June 2011. The national park area has been used for military purposes since 1935, leading to large-scale clearing in some areas. Due to military use, there were almost no other human factors of influence. Following the withdrawal of the Soviet army in 1991, the area became a succession area where reforestation is taking place without human regulation [59]. The current objective of the national park is to allow the natural dynamics to shape these forest areas and to study them. As a result, 90% of the national park is free from anthropogenic use and subject to natural processes [56,59]. The majority of the Huss site is covered by deciduous forests; based on an inventory by the Hainich National Park administration from 2013, it is primarily composed of beech trees (Fagus sylvatica) accounting for 78% of the trees. Other tree species interspersed in the forest include sycamore (Acer pseudoplatanus) with 12% share, ash (Fraxinus excelsior) with 5% share, hornbeam (Carpinus betulus) with 2% share, Norway maple (Acer platanoides) with 2% share, wych elm (Ulmus glabra), and oak (Quercus spec.). As per the inventory, the mean diameter at breast height (DBH) of the predominant and dominant stock is 56.77 cm, while the mean diameter of all trees (with a DBH greater than 7 cm) is 32.20 cm. Since 2018, the area has experienced severe drought events, particularly during the vegetation periods, that have caused water stress and widespread tree damage events on localized scales [60]. The impact extends not only to the forest structure but also to the diverse fauna, which includes approximately 10,000 species dependent on the forest habitat. Notable examples include the European wildcat, roe deer, wild boar, and various bat species [56].

2.2. Acquisition and Preprocessing of UAV Imagery

UAV images were acquired for the study area at two time points: 4 August 2021 (leaf-on) and 29 March 2022 (leaf-off). Both datasets were captured using a DJI (Da-Jiang Innovations Science and Technology Co., Ltd., Shenzhen, China) Phantom 4 Pro equipped with a real-time kinematic (RTK) receiver and an RGB camera. The camera features a 1-inch CMOS sensor with 20 MP. It utilizes a wide-angle lens with a focal length of 24 mm (equivalent to 35 mm) and a field of view spanning 84°. The mechanical shutter speed ranges from 8 to 1/2000 s, and the supported data format is JPEG [61]. Throughout the entire flight period, a correction signal from a reference station was received via mobile internet connection using the German satellite positioning service (SAPOS). This enabled highly accurate real-time positioning with errors in the range of a few centimeters, eliminating the need for ground control points [11].

The acquisition and camera parameters for the two UAV missions are summarized in

Table 2. UAV mission and acquisition parameters (in bold). Wind speed was measured at the weather station Weberstedt/Hainich located 5 km to the NE of the test site. The mission footprint (i.e., the entire area covered by the UAV campaign) is noticeably larger than the Huss site because the flight planning accounted for a buffer surrounding the site. Abbreviation: ISO = International Organization for Standardization.

Parameter	4 August 2021	29 March 2022
Time (UTC + 1) Of First Shot	12:53 a.m.	12:34 a.m.
Wind Speed	5.0–6.1 km/h	3.2–6.5 km/h
Clouds	overcast (8/8)	overcast (8/8)
Mission Duration	40 min	120 min
No. Images	573	1842
Image Overlap (Front/Side)	85%/80%	85%/80%
Flight Speed	5.9 m/s	5.0–6.0 m/s
Shutter Speed	1/320 s (shutter speed priority)	1/320 s (shutter speed priority)
Distortion Correction	yes	yes
Gimbal Angle	−90° (nadir)	1 × −90° (nadir) 2 × −70° (oblique)
Flight Altitude Over Tower	110 m	110 m
ISO Sensitivity	ISO200	ISO400
Aperture	F/5.0 (exposure value −0.66)	F6.3–F/7.1 (exposure value 0)
Geometric Resolution (Ground)	3.56 cm	4.29 cm
Area Covered	0.518 km2	0.533 km2

. The leaf-on data consist of a single overflight of the area, capturing images from a nadir perspective. The leaf-off dataset comprises three overflights of the area: one from a nadir perspective and two using an oblique camera perspective (70° or 20° off-nadir), each taken from an opposite flight direction. Oblique flights were included as they provide additional viewing angles on tree stems and forest floor, and ensure a more complete point cloud. For all flights, a standard parallel-flight line pattern was chosen. These flight parameters were selected with the aim of obtaining comprehensive information about the forest stand and its components (canopy, stems, and forest floor) during the respective acquisition date. The flux tower within the Hainich National Park [62] was selected as take-off and landing spot to ensure a direct line of sight throughout the flights.

The UAV images were preprocessed using an SfM workflow in the software Agisoft Metashape (version 1.8.0). An overview of the applied parameters can be found in

Table 3. Parameters of the Structure from Motion (SfM) processing steps (Agisoft Metashape) (in bold) using the UAV RGB imagery as input.

Parameter	4 August 2021	29 March 2022
Photo alignment accuracy	High (original image resolution)	High (original image resolution)
Image preselection	Generic/Reference	Generic/Reference
Key/tie point limit	40,000/10,000	40,000/4000
Detected tie points	1,261,536	415,144
Aligned cameras	573/573	1839/1842
Adaptive camera model fitting	Off	Off
Optimize camera alignment	No	No
Average error of camera position (x, y, z)	0.98, 1.10, 2.37 cm	0.16, 0.06, 1.62 cm
Effective reprojection error	0.893 pix	1.25 pix
Dense cloud quality	High	Ultra-high (original image resolution)
Depth filtering	Mild	Mild
Number of points (dense cloud)	approx. 158 million	approx. 174 million

. The RGB images from both flight campaigns were processed without prior modification, resulting in 3D point clouds and orthomosaics (raster) for both the leaf-off and the leaf-on season. All images of the leaf-off flights (nadir and oblique) were processed together within a single SfM workflow. The orthomosaics were resampled to a geometric resolution of 0.05 m. The ETRS89/UTM 32N co-ordinate system (EPSG: 25832) was employed for the entire study and all associated data. The geometric accuracy of the processing data products was assessed through visual inspection to ensure they do not contain any unwanted artifacts. Additionally, calculated errors, such as the average error of camera positions and the reprojection error, were examined. The results indicate good geometric accuracy and a very high alignment of the bitemporal data products (Table 3).

The leaf-off point cloud was used to perform a ground classification, detecting points belonging to the forest floor. Out of the 174 million points of the leaf-off point cloud, 28 million were classified as ground points. Using these ground points, the leaf-off point cloud was terrain normalized. For accuracy assessment, the ground classification was compared to the elevation of the airborne lidar dataset from the Thuringian State Office of Land Management and Geological Information, which was acquired in 2017 and has a resolution of 1 m. The ground points from both datasets were converted to raster layers with a resolution of 5 m. Pixel-wise differences were calculated, resulting in an overall RMSE of 0.450 m.

The elevation of the classified leaf-off ground points (DTM) and the leaf-on canopy points (DSM) were used to generate a CHM with a geometric resolution of 0.1 m. All the processing steps were carried out using the software LASTools (version 211112). Figure 3 illustrates a profile cross-section of the two point clouds and their combination. UAV leaf-off data alone from the same study site two years earlier (2019) have already been used by Thiel et al. [63] to delineate dead wood debris and have been proven to be suitable for deriving information from the forest floor.

2.3. Reference Data

(a)

Inventory by the Hainich National Park administration

The Huss site was manually inventoried by the Hainich National Park Administration in 2013 during a field campaign. The data include the number of all trees for the entire study site, along with various tree parameters. A subset of the inventory was used, containing a total of 3251 living trees with pronounced and well-developed crowns, which form the predominant and dominant stock of the forest stand. This corresponds to classes 1 and 2 according to tree classes proposed by Kraft [64]. The data were used to compare the total numbers of overstory tree stems.

(b)

Stem co-ordinates

Two datasets were generated to validate the detected stem positions and analyze error sources. For both, a subset of the study site located in the north-eastern part was used, referred to as the validation area (Figure 4). The first one is based on an MLS point cloud acquired in August 2022 and is used as an independent (MLS-based) reference dataset. The second one was obtained from the UAV leaf-off point cloud and represents all detectable tree stems within the used UAV data; hereafter, it is referred to as “UAV detectable” (Figure 4).

For both point clouds, trees with visible trunks and bases were carefully digitized manually. To do so, the whole point cloud as well as point cloud slices from different height levels were inspected visually to detect stems. Each selected stem point had to correspond to an overstory tree, in order to exclude small understory trees in the reference dataset. A marker was placed at the base of the stem and the stem co-ordinates were derived from it. These stem base co-ordinates were then used as stem co-ordinates, even if the stem did not grow vertically. This means that the trunk co-ordinate is not the midpoint between the base of the trunk and the crown. The accuracy of the individual digitized stem base co-ordinates from the MLS datasets was further verified in the field by manually georeferencing and conducting an inventory of a sample of 145 overstory trees within a subset of the validation area. Within the validation area, a total of 802 overstory trees were selected from the MLS dataset, while a total of 578 overstory trees were selected from the UAV dataset.

In order to use the MLS dataset, certain preprocessing steps were performed. After data acquisition in the field by a ZEB Horizon laser scanner (GeoSLAM, Ltd., Nottingham, UK), the scan was georeferenced using three ground control points located within the MLS scan area. The exact locations of these control points, measured with a carrier phase GNSS rover, were available from previous studies. This allowed the MLS data to be georeferenced with an RMSE of 0.088 m. Further processing included the normalization of the dataset using previously classified ground points within the same MLS dataset.

Since this study focuses only on overstory trees, all selected trees in both datasets were checked to see if they form an upper canopy crown. This step is necessary as the Hainich National Park is an unmanaged forest and contains a high number of small understory trees, oftentimes just of a few meters’ height. For the UAV-detectable dataset, trees that were not selected as overstory trees were categorized additionally into one of the following groups: small trees (understory), tree stumps, and tilted stems. Point clusters that were not clearly identifiable as a tree trunk in the UAV point cloud were labeled as unclassified trees.

(c)

Crown shapes

Individual crown shapes were carefully digitized by hand using the RGB leaf-on orthomosaic, CHM, and leaf-on and leaf-off point clouds. This derivation process encompassed the entire Huss study site. Figure 5 shows the RGB and CHM data, along with the resulting reference tree canopy delineations.

It is important to acknowledge that, despite expertise, challenges arose during the derivation process. At times, it was difficult to determine whether a given area contained a single tree crown with branches or multiple tree crowns.

2.4. Method for Automatic Detection of Stem Coordinates

For the derivation of trunk co-ordinates, the leaf-off point cloud is used. The approach involves grouping points that belong to a trunk into one cluster and representing the center of each cluster as a single point with co-ordinates. From these clusters, various metrics are computed to subsequently identify tree trunks whose crown also appear in the leaf-on point cloud. To focus on relevant points, the normalized leaf-off point cloud is trimmed to include only points within the height range of 3 to 15 m (LASTools command las2las) (Figure 6). The lower threshold of 3 m was chosen to prevent ground points and downed dead wood debris from being included, while the upper threshold of 15 m was selected to avoid including lower branches of the canopy and to facilitate the removal of smaller trees measuring just a few meters in heights. These thresholds may need to be adjusted based on the specific characteristics of the study site and were customized for the forest of the Huss site.

To reduce noise in the leaf-off point cloud, the Statistical Outlier Filter (SOR) operation was applied using CloudCompare (version 2.12.0). The filtered point cloud exhibits a 1.67% reduction in points, equivalent to approximately 400,000 points. The remaining points were then grouped into clusters using the Label Connected Components (LCC) function (Figure 7). The parameter settings for the SOR algorithms are as follows: number of points = 250, standard deviation = 3. For LLC function, the settings are number of points = 100, octree level = 12. This process yields a total of 10,110 potential tree trunks for the entire study area.

A Python code is employed to calculate the following parameters that describe each cluster: minimum, maximum, and mean X, Y, and Z co-ordinates, center co-ordinates (co-ordinate half way between the minimum and maximum X and Y co-ordinate), number of points, difference between the minimum and maximum Z height, and orientation angle (orientation of the points within the cluster using the 3-dimensional distance between the lowest point and the highest point).

With the calculation of individual co-ordinates for each cluster, the clusters can now be described by their center co-ordinate representing potential stem co-ordinates and can be further filtered for actual tree trunks according to their parameter values. Only points associated with a cluster containing more than 750 points, a difference between the maximum z-height and the minimum z-height exceeding three meters, and an orientation angle less than 15 degrees are considered. The angle threshold was chosen not only to exclude clusters that do not represent stems due to their orientation but also to exclude standing deadwood, which often is tilted. To avoid overlapping stem candidates belonging to the same tree, each stem candidate co-ordinate was buffered by the distance of 0.375 m. In cases where two or more co-ordinates overlap, the centroid of the buffered area is calculated and used as the co-ordinate of the tree trunk. This process results in 3739 trunk co-ordinates for the entire study area, which are subsequently used to derive the tree crowns.

2.5. Method for Automatic Delineation of Tree Crown Shapes

The objective of this step is the delimitation of the individual tree crowns within the study area by segmenting the input data, aiming to represent each tree crown as a separate object. Initially, a tree mask was generated to exclude areas such as understory and non-forest areas (e.g., open clearings). This was accomplished by applying a segmentation to the study area and removing objects with a mean height lower than 20 m. Various algorithms were tested using the extracted tree stem positions as input data for crown delineation within the generated tree mask. Ultimately, a region growing method implemented in the software Trimble eCognition (version 10.2) was chosen (Figure 8).

The selected method utilizes the previously derived tree positions as starting points, which ideally also correspond to the highest and brightest point within each crown. Employing specific growing criteria, each starting point is iteratively expanded by including surrounding pixels until it forms an object that represents a complete crown. To prevent starting points from falling within a small local minimum inside the crown, they were expanded first to form a circle with radius of 50 cm. Two criteria were employed for the growing process. Initially, a median-filtered brightness layer was used, calculated by averaging the intensities of all three RGB layers. Adjacent pixels above a certain brightness threshold were added to the growing crown. Secondly, the CHM was utilized, enlarging the crowns by including pixels above a certain tree height threshold. The same thresholds were applied to all objects. The height threshold started with the highest CHM values of the study site (40 m) and then iteratively lowered by 1 m until every pixel was added to one crown object. To refine the results, very small objects with an area below 7.5 m² were merged with their neighboring crowns, resulting in a total number of objects that is less than the number of detected tree stem positions (Figure 8).

The results of the crown delineation were compared to ITCD methods that do not integrate markers. Among these methods were a CHM-based watershed segmentation without markers (using Trimble eCognition software, version 10.2) and a pretrained convolutional neural network on the leaf-on RGB orthomosaic (using the Python package DeepForest, version 1.2.1).

2.6. Accuracy Assessment

(a)

Stem detection

The extracted tree stem positions were validated against the UAV-detectable stem points and the MLS reference stem points within the validation area (Figure 4). For each extracted tree position, the nearest tree position from the respective validation dataset was detected, and each reference point was linked to the closest extracted tree position. A distance threshold of 1.5 m was used between the nearest extracted and reference stem point to determine correctly extracted stem positions. This threshold was chosen considering the average stem diameter at breast height of overstory trees within the study site, which is 49.2 cm. This value was obtained from a subset of 159 overstory trees where the diameter was manually measured. Due to the manner in which the reference points were generated through manual digitization, they are positioned on one site of the trunk rather than in the middle. As a result, minor shifts between stem positions of different datasets can occur. Various additional threshold values between 1.0 and 2.0 m were tested, but they only resulted in slight changes in the accuracy values. Therefore, 1.5 m was chosen as a stable threshold that captures correctly extracted stem points without mismatching not corresponding stem positions. Thus, any extracted stem point with a reference point within a radius of 1.5 m is considered as true positive (TP). False positives (FP) are extracted stem points outside the respective radius, and false negatives (FN) are reference stem points with no corresponding extracted stem point within the distance thresholds. Recall, Precision, and F1-score were calculated for all combinations of datasets, whereby Recall, Precision, and F1-score are defined as following [65]:

$$Recall=TP/(TP+FN)$$

(1)

$$Precision=TP/(TP+FP)$$

(2)

$$F1\text{-}score=2\times \frac{\left(Recall\times Precision\right)}{(Recall+Precision)}$$

(3)

(b)

Crown delineation

To assess the accuracy of the extracted tree crown delineation compared to the reference crowns, the intersection over union (IoU) metric was used. The IoU is calculated by comparing the overlapping area between matching extracted crown polygons and reference crown polygons (intersection) with the total area of both polygons combined (union). For each extracted polygon, the reference polygon with the highest overlap was considered the matching polygon. An IoU score of 1 indicates an identical reference and extracted crown; 0 means there is no overlapping area between them. The mean value of all IoU polygon scores is calculated, as well as the percentage of crowns with an IoU ≥ 0.5. This threshold is commonly used in various studies to define true positive tree crowns [36,51,66,67].

In addition, precision, recall, and F1-score (Equations (1)–(3)) were calculated for each crown and as mean values. These values were computed by comparing the correctly delineated crown area to the area of extracted and reference crown, respectively. The correctly delineated crown area (TP) is defined as the overlapping area (intersection) of reference and extracted areas. The area of the extracted crown equals the sum of TP and FP, while the area of the reference crown equals the sum of TP and FN.

3. Results

3.1. Stem Detection

A total of 3739 tree trunks were detected by the algorithm for the whole study site (Figure 9), which aligns well with the inventory conducted by the National Park Administration, counting 3251 trees for the Huss site. However, it is important to consider the time difference of 9 years. The number of extracted trees also closely matches the number of reference crowns, with 3700 crowns delineated within the study site.

As mentioned in Section 2.3, precision, recall, and F1-score have been calculated for the extracted stem positions compared to both the MLS reference dataset and the UAV-detectable dataset, as well as for the two validation datasets compared to each other. The comparison between extracted trees and UAV-detectable trees, both based on the same UAV-SfM dataset, evaluates the algorithm’s performance in detecting tree stems present in the UAV data (

Table 4. Results of the accuracy assessment for stem detection, comparing the three available datasets with each other (detected stem positions, UAV-detectable stem positions, and MLS reference stem positions). TP = true positive, FN = false negative, FP = false positive. The color coding (dark green–green–yellow–orange–red) indicates the value range within which each respective number lies between the highest (green) and lowest (red) accuracy values.

	Validation	Extracted	No. Validation Trees	No. Extracted Trees	TP	FN	FP	Precision	Recall	F1-Score
(a)	UAV detectable	Prop. Method	578	626	482	96	144	0.770	0.834	0.801
(b)	Reference (MLS)	UAV detectable	802	578	552	232	26	0.955	0.704	0.811
(c)	Reference (MLS)	Prop. Method	802	626	543	236	83	0.867	0.697	0.773

, row (a)). Validating the extracted positions against the MLS reference data demonstrates how many correct trees can be detected out of all present trees (Table 4, row (c)). The comparison between the UAV-detectable and MLS reference datasets reflects the ability of drone data in generating a correct and complete digital copy of tree positions (Table 4, row (b)). In total, 626 of the extracted tree positions lie within the validation area, compared to a total of 578 UAV-detectable trees and a total of 802 MLS reference trees within the same subset. The F1-scores for the extracted tree stems are 0.773 when validating with the MLS reference dataset and 0.801 when validating with the UAV-detectable stems.

In addition to the 578 labeled UAV overstory trees, tree stems present in the UAV datasets classified into small trees (understory), tree stumps, tilted stems, and point clusters that are not clearly recognizable as a tree trunk (unclassified trees) were gradually added to the dataset and used for the accuracy assessment (

Table 5. Results of the accuracy assessment for the stem detection comparing the detected stem positions using the proposed method with the UAV-detectable dataset, considering different types of trees labeled within the UAV point cloud (overstory trees, small/understory trees, tree stumps, tilted trees, and unclassified trees). TP = true positive, FN = false negative, FP = false positive. The color coding (dark green–green–yellow–orange–red) indicates the value range within which each respective number lies between the highest (green) and lowest (red) accuracy values.

	Type of Validation Tree (UAV-Detectable Dataset)	No. Validation Trees	No. Extracted Trees	TP	FN	FP	Precision	Recall	F1-Score
(a)	Only overstory trees	578	626	482	96	144	0.770	0.834	0.801
(b)	Overstory + understory trees	671	626	543	120	83	0.867	0.819	0.843
(c)	(b) + unclassified trees	731	626	583	140	43	0.931	0.806	0.864
(d)	(c) + tree stump and tilted trees	804	626	611	183	15	0.976	0.770	0.861

).

3.2. Crown Delineation

Individual tree crowns were generated for the entire study site. Two subsets comparing the extracted crowns to the reference crowns are shown in Figure 10. A total of 3414 crowns were delineated for the entire Huss study site, compared to 3700 crowns in the crown delineation reference dataset. The deviation to the detected stem positions (3739 tree positions were detected) can be explained by the fact that 325 tree positions did not have a corresponding delineated crown. The IoU score was calculated for all extracted crowns, resulting in a mean score of 0.442. A proportion of 42.9% of all crowns had a score ≥ 0.5.

The mean precision value for all crowns was 0.680, and the mean recall value was 0.614, resulting in an F1-score of 0.645. As a high number of false positives (area) indicates over-segmentation, while a high number of false negatives (area) indicates under-segmentation, low precision values indicate a tendency of over-segmented crowns and low recall values indicate a tendency of under-segmented crowns. Therefore, no strong trend of either under- or over-segmentation can be observed in the applied method, as precision and recall have similar value ranges. However, slightly lower recall values may indicate partial under-segmentation of crowns, which aligns well with the slightly lower number of extracted crowns compared to the reference crowns.

The results of the approaches, which did not include tree stem markers, resulted in a mean IoU score of 0.393 for the watershed segmentation and 0.238 for the DeepForest segmentation.

Moreover, tree height was calculated for each of the extracted crowns utilizing the CHM. This analysis exemplifies the use of individual tree crowns for deriving further forest parameters. It resulted in a mean tree crown height per object of 30.45 m (using the mean height value per crown) and a mean tree height of 33.72 m (using the maximum height value per crown).

4. Discussion

4.1. Stem Detection

The total number of trees detected by the algorithm aligns well with the count of trees recorded by the National Park Administration. The minor discrepancies in the numbers could stem from differing definitions: the inventory data classify trees in classes based on dominance [64], while our categorization distinguishes between overstory and understory. For instance, if co-dominant trees are also included in the inventory data, the count increases to 4876. Comparing the extracted trees to the MLS reference dataset, the results showed a slight underestimation of the total number of trees, approximately by 22%. The precision value of 0.867 indicates a high reliability of the extracted trees. However, the recall value of 0.697 underscores that some of the trees were missed.

Interestingly, when comparing the extracted tree stems with the UAV-detectable data, the situation is reversed: recall is slightly higher than precision. The high recall value of 0.834 suggest that the developed algorithm performs well and is able to detect most of the trees present in the used UAV dataset. A considerable percentage of trees not detected by the algorithm can, therefore, be attributed to an incomplete UAV dataset, where the drone may not be able to detect all relevant tree stems. The slightly lower precision values are mainly due to the algorithm detecting point clusters as trees that are outside the defined scope (i.e., trees that are not overstory trees). The Hainich National Park represents an unmanaged and heterogeneous forest environment, where various tree types of different ages and growth forms, which together form a complex structure, are present. Apart from dominant overstory trees, the area is characterized by an understory layer of smaller trees, remaining tree stumps from broken or fallen trees, and standing dead wood. When using a remote-sensing-based approach in structurally diverse forest environments, it is important to consider these distinct tree types. Defining the tree type to be detected is crucial, as the number of trees can vary significantly depending on factors such as the inclusion of understory trees. As this study employs stem positions for crown delineations, the goal of the tree detection was to detect overstory trees. Since, so far, it is challenging to derive reliable stem parameters such as DBH directly from the UAV-SfM point cloud, detected stem positions cannot be filtered by DBH class. However, by analyzing the 3D extend of the point clusters, we were able to identify tilted stems based on their growing angle and excluded them. Although tilted stems are not of interest in this case, the same approach could be further explored to detect snags, standing dead trees, which are often tilted. However, as mentioned before, identifying only overstory trees was not met in every case. The results in Table 5 illustrate in more detail that some of the extracted trees are likely not part of the canopy. The precision values increase gradually when understory trees, tree stumps, tilted stems, and unclassified stems are included. When all types of tree stems are considered, nearly all extracted tree stems are correct (precision of 0.976). This indicates that the extracted tree positions are very reliable but do not in every case represent overstory trees. When matched with their closest UAV-detectable tree (max. distance 1.5 m) among the 626 extracted trees within the validation area, 129 of them do not correspond to a classified overstory tree but to one of the other categorized tree types, most of them to understory trees. To improve the detection of only overstory trees, it is possible to use a point cloud slice from a higher forest layer, which could exclude smaller understory trees and tree stumps. However, this approach may increase the number of missed overstory trees as the point cloud becomes sparser with elevated tree height. To be able to filter trees not only by the above tree categories but also by tree size, it would be of great interest to derive parameters such as DBH directly from the point cloud. To improve the UAV-SfM point cloud, different flight parameters were tested during this study, but further investigation regarding the acquisition parameters could be conducted, e.g., by adding more oblique flights or changing the camera settings [68].

While most studies using optical UAV data focus on raster-based tree detection methods (Table 1), such as methods based on local maxima filtering from CHMs [27,69], it has been demonstrated that point cloud approaches are also suitable for producing accurate stem positions using UAV-SfM point clouds. These methods often stem from the laser scanning sector; their potential for UAV-SfM point clouds, especially under leaf-off conditions, could be further investigated.

To evaluate the suitability of UAV-SfM point clouds in general for detecting single tree positions, independent of the chosen stem detection approach, we compared the UAV-detectable stem positions with the MLS reference stem positions. This comparison was performed visually and by validating the UAV-detectable dataset with the MLS reference dataset (Table 4, row (b)). The accuracy assessment between the two datasets demonstrated very high precision values, indicating the high reliability of trees visible in the SfM point cloud. However, recall values are considerably lower, highlighting the already mentioned fact that tree stems are missing in the SfM dataset. Nevertheless, it could be demonstrated that point clouds based on UAV-SfM approaches are a cost-effective and accurate substitute to LiDAR in tree stem detection. This goes along with several studies that compared UAV-SfM and airborne LiDAR point clouds of forests and stated that UAV-SfM can be an adequate substitute to airborne LiDAR point clouds for forest applications [70,71,72].

4.2. Crown Delineation

By applying the proposed method using detected stems for crown delineation, a moderate F1-score could be achieved. The resulting IoU score remains relatively low. As mentioned earlier, the model does tend to slightly under-segment the crowns.

The moderate accuracy values reflect the challenging nature of defining crown boundaries in a dense, structurally diverse, and complex forest such as the Hainich National Park. Even with careful observation, it is often difficult to distinguish visually individual crowns within the orthomosaic or point cloud. For example, neighboring trees of the same tree species can appear as either two individual trees or two branches belonging to the same tree. These ambiguities are hard to resolve using only leaf-on data, whether through visual interpretation or ITCD algorithms based on canopy information. To address these limitations, we analyzed the combination of leaf-off and leaf-on data for crown delineation, reasoning that every individual tree should have a detectable tree stem, while crown parts without a detected tree stem would correspond to branches. We compared the crown delineation with several commonly used ITCD methods that do not employ markers. The results of the marker-less approaches yielded lower accuracy values compared to the proposed method, which involves including tree stem positions from leaf-off data as markers. This indicates that the integration of stem positions as markers improved the results. Regarding the comparison of both tree stems and crowns with the reference data within the corresponding subset, the numbers of reference crowns (606), extracted crowns (617), and extracted tree stems (626) match very well. However, the number of MLS reference tree stems is higher (802). Considering this, it can be assumed that the visual delineation of crowns in the UAV data, as well as extracted tree stems and crowns, underestimate the number of overstory trees/crowns by approximately 22–24% in dense forest stands. As most ITCD studies validate their crowns by comparing them to crowns generated by visual interpretation of the data [19,20], it is likely that many remote-sensing-based studies underestimate the number of overstory trees/crowns in dense forest environments.

To improve the crown delineation results, the use of other algorithms could be further explored. Several deep learning approaches have shown promising results [34,36,51,66,73] and new approaches are constantly developed. Additionally, it would be worth investigating how prior-derived stem positions from leaf-off data could be used as additional input for deep learning crown delineation.

4.3. Combination of Leaf-off and Leaf-on UAV Data to Model Forest Structure

Building on our findings, we recommend combining leaf-off and leaf-on SfM point clouds, especially for dense deciduous forest stands. Our study has demonstrated that these datasets contain highly complementary information, as shown in Figure 3. Leaf-on data provide limited under-canopy information due to the density of the canopy, while relying solely on leaf-off data makes it challenging to accurately estimate crown parameters and tree heights [74]. However, by leveraging both datasets, a more comprehensive picture of the forest canopy, tree stems, and forest floor can be obtained, enabling more accurate derivation of forest parameters. This option, of course, is only possible for areas with deciduous forests experiencing a leaf-off season. This finding aligns with the results of other studies that have combined leaf-off and leaf-on SfM data primarily to generate accurate height models [74,75,76]. For instance, Nasiri et al. [75] generated a CHM by utilizing both leaf-off and leaf-on data from UAV imagery to derive tree height and crown diameter. Moudrý et al. [76] reported comparable accuracy between DTMs derived from leaf-off SfM data and those from airborne LiDAR. In our study, we not only used leaf-off and leaf-on data to generate height models but also used the tree detection results from the leaf-off point cloud to enhance the accuracy of other forest-related parameters that rely on accurate tree detection and identification, such as crown delineation in our case.

Several ITCD methods, such as region growing approaches, require markers as starting points for crown delineation. Ideally, these markers should represent the highest points of the crowns. Although watershed segmentation can be applied without markers, it has been reported to work better when markers are included [32]. In dense deciduous forest environments, obtaining these markers solely from CHMs is challenging. However, our method, which combines leaf-off and leaf-on data, provides accurate tree stem positions that can serve as marker points. Although, it must be acknowledged that, even with highly accurate stem positions, crown delineation is limited if the leaf-on data do not allow the distinction between neighboring crowns. Nevertheless, to accurately determine the number and location of trees in a dense forest site, it is essential to look beneath the canopy. Some studies are exploring the possibility of under-canopy UAV flights for the data retrieval of stem and forest floor [77,78]. However, currently, automated flight missions under dense forest canopies cannot be realized.

5. Conclusions

This study emphasizes the significance of integrating leaf-off and leaf-on SfM point clouds for single-tree-based monitoring and management using UAVs equipped with optical RGB camera systems. Our focus was on deriving overstory tree stem positions and crown delineations using UAV-SfM data products.

Tree stem detection was carried out by applying a clustering and rule-based approach on the leaf-off point cloud. Accuracy assessment involved comparing three datasets: the extracted stems, manually detected UAV stems, and MLS reference stems. We found that the proposed method delivers highly reliable stem positions (precision of 0.867). Recall values were slightly lower, indicating that approximately 22% of the present trees were missed. Categorizing different tree types allowed us to analyze the type of tree stem being detected by the algorithm. This analysis revealed that the distinction between overstory and understory tree stems is not always met. When working in dense structurally diverse forest environments, the high number of small understory trees has to be considered. Further investigations could explore the potential of UAV-SfM leaf-off data for detecting understory trees. Validating the UAV detectable stems against the MLS reference stems demonstrated high accuracy values (F1-score of 0.811), confirming the potential of UAV-SfM point clouds in general as a cost-effective alternative to LiDAR data. Using the extracted stem positions as markers for ITCD improved the results compared to methods without markers. However, the moderate accuracy values (F1-score of 0.645, mean IoU of 0.442) indicate the challenges of delineating crowns in dense forests. Promising improvements are expected with the development of new deep learning approaches. The results highlighted that both the number of overstory tree stems and crowns tends to be underestimated, probably even within the crown reference data, indicating potential underestimations in many studies focusing on dense deciduous forests. This underscores the challenges in collecting reliable reference data, as neither the view of the canopy from above via remote sensing data nor from below via field surveys seem to provide the complete picture.

Overall, we advocate for the use of a combined leaf-off and leaf-on SfM approach in future forest monitoring and management tasks. While our focus was on tree stem detection and ITCD, the potential of a combined leaf-off–leaf-on point cloud could be further explored for the derivation of other forest parameters and methods.