1. Introduction
Interpreting three-dimensional scenes from raw point-cloud data has become a focal research topic spanning the computer vision and geospatial intelligence communities. Although point-cloud analysis already underpins diverse applications—from intraoperative surgical guidance and immersive virtual environments to autonomous mobile platforms [
1,
2,
3,
4]—its deployment in metropolitan settings imposes substantially more stringent demands. High-fidelity semantic segmentation of city-wide LiDAR acquisitions is a prerequisite for geographic information retrieval, self-driving perception stacks, and large-area digital twin reconstruction [
5,
6,
7,
8], yet assigning a correct class label to every measured point within such expansive environments persists as an unsolved and formidable challenge.
In contrast to regularly gridded imagery that can be processed by standard convolutional operators, raw point clouds are inherently orderless and exhibit spatially varying density, precluding naive adoption of two-dimensional deep learning pipelines [
9]. The seminal PointNet architecture [
10] enables end-to-end learning on unstructured geometry via shared MLPs and symmetric global aggregation. Outdoor urban scenes exacerbate these challenges: sparse sampling blurs object contours, while large-scale inference demands lean architectures. The highly influential RandLA-Net [
11] addressed this by coupling stochastic point decimation with a lightweight local feature aggregation block, delivering competitive accuracy at minimal computational cost. Numerous subsequent lightweight designs have built upon this paradigm [
12,
13,
14,
15,
16,
17]. Notwithstanding these advances, label ambiguity at semantic interfaces and density-induced aggregation artifacts continue to constrain the performance ceiling of current approaches in geometrically complex urban environments.
Conventional encoder–decoder frameworks typically capture neighborhood context by computing feature differences between each query point and its K-nearest neighbors (KNN) [
18], followed by permutation-invariant pooling operations such as attention-weighted summation. While KNN is computationally attractive, it is fundamentally mismatched to extreme density gradients: in under-sampled sub-regions, the neighborhood may be populated entirely by semantically irrelevant points, thereby injecting cross-class noise into the feature representation. Equally limiting is the indiscriminate weighting of all neighbors irrespective of their structural affinity—outlier points contribute on an equal basis with morphologically consistent ones [
19], eroding boundary sharpness and diminishing classification confidence. An additional weakness stems from the exclusive reliance on vertical displacement as the principal shape discriminator; classes that share comparable elevation characteristics cannot be differentiated by height cues alone, imposing an intrinsic upper bound on parsing granularity.
MLP- and CNN-based approaches project per-point attributes into high-dimensional latent spaces via shared-parameter networks and aggregate local neighborhood statistics. PointNet [
10] inaugurated this paradigm by processing raw coordinates through point-wise MLPs coupled with a symmetric global descriptor. RandLA-Net [
11] couples random subsampling with attention-based local aggregation to enable efficient large-scale inference. LACV-Net [
15] addresses local perceptual ambiguity through a local adaptive feature augmentation module, a constraint loss, and a C-VLAD descriptor layer. PointNet++ [
20] extended this foundation with hierarchical set abstraction layers, progressively compressing the point set while retaining local geometric detail. KPConv [
21] eliminated KNN grouping by positioning learnable kernel points in Euclidean space and applying deformable convolutions directly on the point manifold. PointNeXt [
22] refines PointNet++ with inverted residuals and separable MLPs, yielding notable accuracy gains without architectural overhaul. NeiEA-Net [
23] refines neighborhood encoding through selective neighbor filtering combined with weighted-fusion pooling. GreenSegNet [
24] concentrates on vegetation discrimination in mobile-mapped settings using task-specific augmentation pipelines.
Graph-based approaches model point clouds as vertex–edge structures and propagate information along learned topologies using graph convolutions or spectral filters. ECC-Net [
25] first deployed edge-conditioned convolutions for segmentation, generating dynamic filter weights conditioned on edge attribute vectors. DGCNN [
26] introduced the EdgeConv operator, constructing a dynamic graph in feature space and recalculating edge connectivity at each layer to track evolving semantic neighborhoods. PointNGCNN [
27] augmented graph neighborhood filters with spatial distribution embeddings to enrich structural descriptors. AGConv [
28] further generalized graph convolution by producing adaptive kernels from data-driven feature similarities, increasing convolution flexibility. Despite effective geometric relational modeling, graph-based methods incur substantial memory and computation costs for city-scale inputs and do not explicitly address KNN density bias, elevation ambiguity in neighbor weighting, or boundary label inconsistency—limitations that motivate the geometry-aware point-based approach proposed herein.
Attention- and Transformer-based [
29] methods have achieved notable results by modeling long-range dependencies through self-attention. Point Transformer v2 [
30] employs grouped vector self-attention to capture nuanced inter-point correlations, establishing strong classification baselines. PTv3 [
31] tailors this scheme to resource-limited scenarios through hierarchical vector attention, preserving robust performance with leaner overhead. Swin3D [
32] extends shifted-window attention to 3D, achieving scalable non-local aggregation at the cost of increased complexity. PATs [
33] replace standard multi-head attention with group shuffle attention to reduce quadratic complexity while preserving expressive context modeling. GAC [
34] directs attention selectively toward the most informative neighbors via graph-guided affinity computation. Point Transformer [
35] applies self-attention directly to unordered point sets, achieving state-of-the-art results on small-scale benchmarks. Point Cloud Transformer [
36] tackles permutation invariance through an offset-attention module. Point-BERT [
37] adapts masked language modeling to point clouds by tokenizing local patches and pretraining with masked patch prediction. While these methods excel at global context modeling, they frequently entail memory-intensive computation, lack explicit boundary constraints, and handle density-irregular regions suboptimally—limitations that GFE-Net specifically targets through structured hierarchical aggregation and principled boundary regularization.
Recent studies highlight the need for geometry-aware, multi-scale learning in remote-sensing point clouds. Geng et al. [
38] and Zhao et al. [
39] enhanced local–global consistency via multi-scale and sparse voxel attention, while Wang et al. [
40] and Liang et al. [
41] incorporated fusion attention and geometric cues for urban airborne LiDAR segmentation. Engelman et al. [
42] and Nurunnabi et al. [
43] explored PointNet-derived architectures for aerial point-cloud classification. At the urban scale, GraNet [
44] and the SensatUrban benchmark [
45] advance large-area parsing and establish RandLA-Net as a competitive baseline. Nevertheless, existing methods remain vulnerable to extreme density variations and boundary ambiguity induced by complex structures, with density-induced neighborhood contamination and label inconsistency still unaddressed in a unified geometry-aware framework—gaps that GFE-Net specifically targets.
Driven by the foregoing observations, this work aims to equip point-feature aggregation with theoretically grounded and computable affinity measures. Specifically, we introduce a morphology-driven contextual gating (MCG) unit that derives neighbor-wise calibration weights from hierarchical shape descriptors enriched with elevation difference statistics. Unlike scalar attention weights, the MCG unit generates neighbor-specific scalar gates through a compact nonlinear mapping, enabling selective reinforcement of structurally coherent neighbors and attenuation of geometrically anomalous ones. The MCG unit further incorporates skip connections to stabilize gradient propagation. MCG is embedded within the broader Structure-Guided Neighborhood Adaptation (SGNA) module—paired with positional context encoding (PCE) units that jointly encode absolute point coordinates and relative spatial offsets—thereby comprehensively counterbalancing the density bias inherent to KNN search.
To expand the representational capacity beyond what local morphology alone can provide, we present the Local–Global Interactive Enhancement (LGIE) module. The LGIE module routes point features through two concurrent branches: the first extracts spatially explicit local topology from Euclidean neighborhoods, while the second distills holistic co-activation statistics from the implicit feature manifold through low-rank bilinear decomposition. Both branches are merged and refined via a residual subtraction mechanism, producing highly discriminative embeddings that are particularly beneficial for categories sharing confusable shape profiles.
To mitigate boundary label inconsistency without resorting to post-processing heuristics, we propose the Neighborhood Prediction Consistency (NPC) loss. The NPC loss penalizes prediction disagreements between a query point and its neighbors with strengths modulated by a Gaussian distance-decay kernel, imposing heavier penalties on nearby mismatches and lighter ones on distant discrepancies. This formulation rests on the proximity principle—geometrically close points are statistically more likely to share a semantic label—offering a principled alternative to heuristic margin-based boundary losses.
In summary, the contributions of this paper are fourfold:
We propose the SGNA module, which incorporates an MCG unit to compute morphology-conditioned neighbor-wise affinity weights, improving local aggregation under non-uniform density. Ablation experiments further evaluate the contributions of contextual gating and residual connectivity.
We develop a plug-and-play LGIE module that simultaneously captures explicit local spatial topology and implicit scene-wide co-activation patterns through dual concurrent branches, enhancing inter-class feature separability, especially for elevation-similar categories.
We introduce the NPC loss employing Gaussian distance-decay weighting to enforce spatially consistent prediction assignment, improving boundary precision through a theoretically motivated penalty formulation—with comparative experiments validating the superiority of Gaussian decay over alternative distance-weighting strategies.
GFE-Net is systematically benchmarked across two large-scale datasets against both attention-centric and Transformer-based architectures, with ablation studies and efficiency profiling thoroughly substantiating each design decision.
2. Materials and Methods
The proposed GFE-Net adopts a hierarchical encoder–decoder backbone. To provide a top–down overview,
Section 2.1 first presents the complete network pipeline and the roles of subsampling, upsampling, skip connections, and the C-VLAD bridge.
Section 2.2,
Section 2.3 and
Section 2.4 then detail the three complementary components: the Structure-Guided Neighborhood Adaptation (SGNA) module for morphology-driven local aggregation, the Local–Global Interactive Enhancement (LGIE) module for dual-branch feature refinement, and the Neighborhood Prediction Consistency (NPC) loss for spatially coherent boundary supervision. Lastly,
Section 2.5 summarizes the complete training and inference procedure in algorithmic form.
2.1. Overall Network Architecture
As illustrated in
Figure 1, GFE-Net employs a five-stage hierarchical encoder–decoder. Each encoder stage contains the proposed SGNA block followed by LGIE, using updated designations that replace the previous HGFA and DRB labels to clarify that no external backbone is involved. Random subsampling reduces the point resolution from N to N/4, N/16, N/64, and N/128, while the feature dimension increases from 16 to 64, 128, 256, and 512. C-VLAD bridges the encoder and decoder, and nearest-neighbor interpolation with lateral skip fusion restores the original resolution prior to per-point classification.
2.2. Structure-Guided Neighborhood Adaptation Module
Conventional local feature aggregation modules encode neighborhood context by projecting feature differentials into high-dimensional embedding spaces. For outdoor point clouds acquired over urban terrain, points tend to be distributed relatively uniformly in the horizontal plane but exhibit pronounced vertical stratification—building facades and rooftop surfaces display smooth elevation gradients, whereas vegetation and ground regions present erratic vertical profiles. Leveraging these geometric regularities, the SGNA module introduces hierarchical shape descriptors alongside elevation difference statistics to guide adaptive neighborhood weighting, substantially improving aggregation fidelity in density-heterogeneous regions.
As depicted in
Figure 2, the SGNA module integrates PCE units for positional context encoding and MCG units for morphology-driven contextual gating, culminating in an adaptively aggregated feature representation.
2.2.1. Positional Context Encoding Unit
At each encoder stage, the PCE unit first subsamples the input point cloud and constructs KNN neighborhoods for each surviving point. Following standard KNN edge-feature construction [
18,
34], neighborhood context is encoded from the feature and coordinate residuals between each query point and its neighbors. Given the input feature matrix
P, each point
carries a d-dimensional attribute vector, and
denotes its Euclidean KNN set. The feature contrast is
, and the geometric offset is
, where
is the XYZ coordinate of
. Shared MLPs project both terms to
-dimensional vectors, and the combined neighborhood descriptor is
The local descriptor set is subsequently forwarded to the MCG unit for adaptive neighbor-wise gating.
2.2.2. Morphology-Driven Contextual Gating Unit
The MCG unit assigns a morphology-conditioned scalar gate to each of the K neighbors. The gate vector is therefore defined over the neighbor dimension rather than over feature channels. Each scalar is broadcast across the channels of the corresponding neighbor representation prior to pooling, so structurally consistent neighbors are emphasized and anomalous or cross-boundary neighbors are suppressed.
For query point
i, let
collect the relative and absolute elevation descriptors of its
K neighbors, and let
contain the associated feature contrasts. The shared mapping
is applied row-wise to
, and softmax normalization is performed across the
K neighbors:
Here,
, and
is a scalar gate for the
k-th neighbor. It is broadcast along the
feature channels before multiplication. Hence, MCG performs neighbor-wise morphological calibration rather than channel-wise gating. With a residual bypass, the gated feature matrix
is
The gated set
is then routed through a joint pooling block combining attentive pooling (for weighted selection of salient neighbors) with max-pooling (for extracting dominant activations), yielding the aggregated query-level feature
:
where
and
denote attentive and max-pooling, respectively. The MCG unit produces a
-dimensional feature embedding. Crucially, the morphology-conditioned neighbor gates, their channel-wise broadcasting, and the residual calibration in Equations (
2)–(
4) are formulations introduced in this work.
2.3. Local–Global Interactive Enhancement Module
For unstructured point-cloud data, a multi-branch processing strategy is adopted to enable the network to simultaneously capture fine-grained local topology and holistic scene-wide co-activation patterns. Its detailed architecture is shown in
Figure 3.
To extract local spatial structure in Euclidean space, we adopt the standard dynamic edge-feature construction used in graph-based point-cloud learning [
34]. KNN constructs directed edges from each query point
to its neighbors, forming
, which encodes absolute coordinates and relative displacements. Aggregating all points gives
, which is processed by a shared MLP and max-pooling over
K neighbors:
A parallel feature-space graph is constructed using the same edge-feature principle [
34]. Centering on
in the C-dimensional embedding gives
, and the full feature graph is
. Following the same pooling procedure,
A supplementary shared MLP encodes the feature graph, and the local geometric context
is concatenated with the local feature context
to form the fused local descriptor:
The local fusion block is resolution-agnostic and can be inserted into arbitrary stages of a CNN backbone without architectural modification.
Beyond local modeling, a scene-wide co-activation block uses holistic cues from the entire point set. Its outer-product interaction follows the general bilinear-pooling principle [
46], while the low-dimensional treatment is motivated by compact bilinear representations [
47]. Unlike quadratic self-attention, the proposed LGIE module derives compact channel and spatial descriptors before their bilinear interaction.
The global channel descriptor
is obtained by first applying a linear projection
to
, activating with ReLU to enforce non-negativity, and then averaging over the
N spatial positions:
A reduction factor compresses the channel dimension, yielding , where captures the global average activation of channel j.
Symmetrically, a spatial descriptor
is produced via a distinct weight matrix
and channel-dimension average pooling:
with
, where
reflects the mean activation of point
i across the compressed feature map. The outer product of
and
, square-rooted for low-rank approximation, yields the global bilinear response:
Using shared MLPs and dual residual connections, the network restores channel dimensionality and assembles the scene-wide perception map:
To further purify the global representation,
is subtracted from
in a residual-filtering operation, suppressing redundant global components. A nonlinear activation
is applied to improve representational capacity:
2.4. Neighborhood Prediction Consistency Loss
To sharpen label assignment at semantically ambiguous interfaces between urban objects, GFE-Net incorporates a Neighborhood Prediction Consistency (NPC) regularization term into the overall training objective. For each query point and its neighborhood , a binary disagreement indicator is defined as when the predicted semantic label of the k-th neighbor differs from that of , and otherwise.
To account explicitly for spatial proximity, we adopt the standard Gaussian edge-weighting form [
48] to define the distance decay. The disagreement indicator and its integration into the NPC regularizer are introduced in this work:
where
and
are the normalized coordinates of the query point and its
k-th neighbor. We use
and
for all experiments. The Gaussian kernel decreases monotonically with squared Euclidean distance, so closer disagreements receive larger weights while distant disagreements are down-weighted.
This design follows the spatial proximity principle that nearby points are statistically more likely to share the same semantic label. At the same time, the Gaussian decay limits the influence of distant neighbors and reduces excessive consistency regularization across genuine object boundaries.
The complete training objective is formulated as follows:
where
denotes the number of encoder stages and
40,960 is the number of full-resolution query points. The weighted cross-entropy and inter-layer constraint terms follow [
15], whereas the NPC term in Equations (
13) and (
14) is proposed in this work.
2.5. Algorithmic Summary of Training and Inference
For clarity and reproducibility, Algorithm 1 summarizes the complete forward pass and explicitly separates the operations used during training from those retained at inference.
Generative AI tools were used exclusively for language polishing. No AI tools were involved in the study design, experiments, or data interpretation.
| Algorithm 1 Forward Pass, Training, and Inference of GFE-Net. |
Input: point coordinates, point attributes, ground-truth labels (training only), neighborhood size , and five target resolutions . Output: point-wise class probabilities, predicted labels, and learned parameters after training. |
| Step | Operation |
| 1 | Input Pre-Processing: Concatenate coordinates and attributes; apply stochastic data augmentation only during training, and obtain the initial embedding:
|
| 2 | Hierarchical Encoding: For , randomly subsample to the target resolution, construct KNN neighborhoods, apply SGNA and LGIE, and retain the output as a skip feature. The first downsampling operation, , is the identity mapping.
|
| 3 | C-VLAD Bridge: Aggregate the multi-resolution encoder features into a compact scene descriptor and initialize the decoder bottleneck:
|
| 4 | Hierarchical Decoding: For , use nearest-neighbor interpolation and fuse the corresponding encoder skip feature:
|
| 5 | Per-Point Prediction: Apply three fully connected layers (with dropout during training), followed by softmax:
|
| 6 | Training Loss: Compute the weighted cross-entropy loss, the inter-layer constraint loss, and the NPC loss in Equation (13); combine them according to Equation (14). The fixed NPC settings are , , , and in normalized coordinate space.
|
| 7 | Parameter Update: During training, backpropagate the total loss and update all learnable parameters using Adam; repeat for 100 epochs with an initial learning rate of 0.01.
|
| 8 | Inference: Disable stochastic augmentation and dropout, execute Steps 1–5, omit loss computation and parameter updates, and return:
|
5. Conclusions
This paper presents GFE-Net, a Geometry-Enhanced Feature Extraction Network for semantic segmentation of large-scale urban LiDAR point clouds. The SGNA module, incorporating the MCG unit, delivers morphology-aware, density-robust neighborhood aggregation via structure-conditioned neighbor-wise gating. The LGIE module enriches feature representations by jointly processing local Euclidean structure and scene-wide bilinear co-activation patterns through two parallel branches. The NPC loss enforces spatially consistent prediction assignment at object interfaces via a theoretically motivated Gaussian distance-decay weighting kernel.
Experiments on S3DIS and SensatUrban yielded 89.6%/73.1% and 93.3%/61.1% OA/mIoU, respectively. Comparative evaluations demonstrated that GFE-Net achieves competitive performance across both indoor and large-scale urban scenarios, despite not attaining the highest mIoU under certain S3DIS evaluation protocols. Ablation studies quantified the individual contributions of each component, while efficiency profiling provided explicit trade-offs in parameters, GFLOPs, and runtime, offering a clear basis for practical deployment.
Collectively, the SGNA, LGIE, and NPC loss components synergistically advance the network’s capacity to model fine-grained local morphology and broader scene context, delivering semantically coherent and boundary-precise three-dimensional understanding. Future work will explore extending the proposed components to scene flow estimation, instance-level labeling, and multi-modal fusion with RGB imagery.