DualGLEAN: Dual Allocation for VLM-Guided Generalized Category Discovery in Remote Sensing Images

Highlights

What are the main findings?

• Integrating a vision-language model (VLM) as an external semantic oracle into generalized category discovery for remote sensing images significantly improves novel-class discovery accuracy, but only when the VLM-derived contrastive signal is routed to a dedicated projector space and the query budget is allocated to genuinely boundary-critical samples; under naive single-space integration, VLM guidance degrades performance below the baseline.
• The proposed DualGLEAN framework achieves state-of-the-art performance on the AID and RSSDIVCS datasets, generalizes as a plug-in module across four distinct GCD baselines with zero additional trainable parameters, and maintains consistent accuracy across seven VLM backbones from four providers, with a total API cost of approximately CNY 2.45 per full training run on AID.

What are the implications of the main findings?

• The dual resource allocation perspective—space allocation (where the VLM signal should operate) and budget allocation (which samples deserve costly VLM queries)—provides a principled framework for integrating external AI oracles into remote sensing recognition pipelines, moving beyond ad hoc adoption of foundation models in Earth observation.
• Remote sensing practitioners can deploy VLM-assisted category discovery at negligible financial cost, using off-the-shelf VLMs without remote-sensing-specific fine-tuning, across diverse base architectures and VLM providers, making foundation-model-guided open-world recognition practical for real-world applications such as disaster assessment and land-cover monitoring.

Abstract

Generalized category discovery (GCD) aims to classify known categories while discovering novel ones in unlabeled data, yet existing methods lack mechanisms to correct boundary-ambiguous samples that receive noisy pseudo-labels, as they primarily rely on visual feature learning without external semantic guidance. Vision-language models (VLMs) offer a natural source of cross-modal semantic correction. However, applying VLM-guided contrastive signals directly within the GCD training loop proves counterproductive because the locally-oriented InfoNCE loss conflicts geometrically with the globally oriented K-means objective in the shared backbone space. We identify the root cause as a dual resource allocation problem: the VLM-derived signal must be allocated to the correct feature subspace to avoid geometric conflict with K-means clustering (space allocation), and the limited VLM inference budget must be allocated to the correct samples to maximize discriminative return (budget allocation). These two decisions are coupled; failure on either renders the other ineffective. To resolve this, we propose DualGLEAN, a framework that addresses the dual allocation challenge through two coupled mechanisms: decoupled contrastive alignment (DCA), which routes the VLM-guided neighbor contrastive loss to a dedicated projector space while preserving the backbone space for global clustering, and compound uncertainty querying (CUQ), a three-stage filtering metric that jointly evaluates predictive entropy, boundary proximity, and local label inconsistency to direct VLM queries exclusively to truly boundary-critical samples. Extensive experiments on the AID and RSSDIVCS datasets demonstrate that DualGLEAN achieves strong performance, improves four diverse GCD baselines as a plug-in module, generalizes across seven VLM backbones, introduces zero additional trainable parameters to the base GCD network, and incurs a total VLM API cost of only CNY 2.45 per full training run on the AID dataset under the default search-scope configuration, with the cost scaling linearly with the query budget.

1. Introduction

Generalized category discovery (GCD) [1] aims to simultaneously classify known categories and discover novel ones in unlabeled data by leveraging limited labeled priors. In the remote sensing domain, GCD holds significant practical value for applications such as novel damage assessment in disaster response, unknown facility identification in urban expansion, and land-surface dynamic monitoring, where new land-cover types continuously emerge and exhaustive annotation is infeasible [2,3]. Current mainstream GCD methods [4,5,6] follow a common paradigm: a Vision Transformer (ViT) [7] backbone extracts feature representations, contrastive learning objectives shape the feature manifold, and K-means clustering performs final category assignment. This paradigm primarily relies on visual self-supervised and semi-supervised signals to learn discriminative features. While effective on object-centric natural images, it faces a fundamental limitation when applied to remote sensing scenes characterized by high inter-class visual similarity and severe intra-class variation.

The core limitation of purely vision-driven GCD lies in its inability to correct ambiguous samples that reside near class decision boundaries. In remote sensing, many categories serving structurally and functionally distinct purposes nonetheless share nearly identical low-level visual patterns. Visually confusable pairs such as school vs. church, or residential zones of varying density, form a pervasive “same spectrum for different objects” challenge. Samples at these boundaries receive noisy pseudo-labels that propagate errors throughout training. Recent work in natural language processing has demonstrated that large language models (LLMs) can serve as effective external knowledge sources for GCD: GLEAN [8] (renamed DeLFGCD in its published version) queries an LLM to select semantically coherent positive neighbors for ambiguous text samples, yielding substantial performance gains through contrastive alignment guided by LLM judgments. However, GLEAN is designed for the textual domain, and its single-space contrastive strategy does not anticipate the geometric conflict that arises when VLM-guided signals are applied in the visual GCD pipeline. As we confirm experimentally (Section 4.2), directly adopting this paradigm in remote sensing degrades performance below the baseline, motivating the need for a framework that specifically addresses the optimization conflicts inherent to VLM-in-the-loop training.

We trace this degradation to a dual resource allocation problem that is intrinsic to VLM-in-the-loop training yet remains unrecognized in prior work. Effectively integrating an external oracle into the GCD training loop requires simultaneously solving two coupled allocation decisions:

Space allocation: where should the VLM signal operate? The VLM-selected neighbor pair drives an InfoNCE-based contrastive loss [9] that optimizes local pairwise similarity. In the standard single-space design, this loss is applied directly to the backbone feature space, which is also the space used by the classification head and by K-means clustering at inference. These two objectives impose geometrically incompatible constraints: the neighbor contrastive loss pulls specific anchor–neighbor pairs together into tight local cliques, while K-means requires globally separable, approximately isotropic clusters. This is not merely a gradient-weighting issue solvable by tuning loss coefficients; the geometric requirements are fundamentally contradictory. The InfoNCE objective demands a manifold organized by local adjacency, whereas K-means assumes a manifold organized by global centroids. When forced to coexist in a single space, the locally oriented signal overwhelms the globally oriented one (due to its batch-wide denominator producing dominant gradients), fragmenting the cluster structure and collapsing novel-class recognition accuracy while even eroding the separability of known classes under certain conditions (Section 4.2).

Budget allocation: which samples deserve the limited VLM queries? VLM inference incurs both direct financial costs per API call and substantial training latency from network round-trips, severely constraining the number of queryable samples per training round. The default strategy of selecting candidates by predictive entropy alone treats all high-entropy samples as equally deserving of VLM attention. In remote sensing, however, high predictive entropy is a noisy signal: many samples exhibit elevated entropy simply due to early-stage under-convergence rather than genuine boundary ambiguity, while some truly boundary-critical samples may have moderate entropy if they are confidently but incorrectly assigned. A single scalar cannot distinguish these cases. Wasting VLM queries on under-converged samples while missing truly ambiguous ones constitutes a budget allocation failure in both the financial and temporal dimensions.

Critically, these two allocation decisions are coupled. Misallocating the signal space renders even correctly selected samples harmful to the clustering objective, while misallocating the query budget wastes correctly configured signal channels on non-critical samples. A principled VLM-in-the-loop framework must resolve both dimensions jointly.

To address this dual allocation challenge, we propose DualGLEAN, a framework that resolves the coupled space and budget allocation decisions through two interdependent mechanisms. The motivation and overall framework are illustrated in Figure 1.

DCA solves the space allocation problem: where should the VLM contrastive signal be placed to avoid disrupting the clustering structure? DualGLEAN introduces decoupled contrastive alignment (DCA), which routes the VLM-guided neighbor contrastive loss to a low-dimensional projector space (256-d) while keeping the classification loss, cluster-instance alignment loss, and K-means evaluation in the high-dimensional backbone space (768-d). This spatial separation ensures that the locally-oriented InfoNCE gradients shape only the projector manifold, where local pairwise alignment is the correct and sufficient objective, while the backbone space is free to maintain the globally separable structure required by K-means. Critically, the decoupling leverages the existing MLP projection head already present in standard GCD architectures, introducing zero trainable parameters to the local model; it is a routing-level architectural decision rather than a structural addition.

CUQ solves the budget allocation problem: which samples deserve the costly VLM queries? DualGLEAN employs compound uncertainty querying (CUQ), which filters unlabeled samples through three complementary criteria: predictive entropy, measuring global assignment uncertainty; boundary proximity, defined as the distance ratio to the two nearest cluster centers; and local label inconsistency, computed as the pseudo-label entropy within the k-NN neighborhood. Only samples that are simultaneously globally uncertain, geometrically positioned at a decision boundary, and surrounded by locally conflicting evidence survive this three-stage filtering and are issued as VLM queries. By precisely targeting boundary-critical samples, CUQ maximizes the discriminative return per VLM query, avoiding wasteful expenditure on non-critical samples.

The synergy between DCA and CUQ is essential. DCA ensures the VLM signal is applied in a compatible geometric subspace where it helps without harming the clustering objective, while CUQ ensures that only the most discriminative samples enter that channel. Together, they form the dual allocation foundation of DualGLEAN.

The main contributions of this paper are summarized as follows:

• We identify that integrating VLM guidance into the GCD training loop is fundamentally a dual resource allocation problem, comprising space allocation (where the VLM-derived signal should operate) and budget allocation (which samples deserve VLM queries), and show that these two dimensions are coupled: failure on either dimension renders the other ineffective. This perspective provides a principled framework for understanding why naive VLM adoption degrades performance.
• We propose decoupled contrastive alignment (DCA), a routing-level mechanism that separates the VLM-guided neighbor contrastive loss into a dedicated projector space while preserving the backbone space for global clustering objectives. This eliminates the geometric conflict between locally oriented contrastive alignment and globally oriented K-means evaluation without introducing new trainable parameters to the base network.
• We design compound uncertainty querying (CUQ), a remote-sensing-specific metric that jointly evaluates predictive entropy, boundary proximity, and local label inconsistency to precisely target VLM queries at samples requiring external semantic correction. This resolves the budget allocation dimension by filtering out false positive uncertainty samples.
• Extensive experiments on multiple remote sensing datasets demonstrate consistent improvements. The framework generalizes as a plug-in module across diverse GCD baselines (GCD, SimGCD, SelEx) and remains robust to the choice of VLM backbone.

2. Related Work

2.1. Generalized Category Discovery

The GCD task aims to jointly recognize known classes and discover novel ones in unlabeled data using limited labeled samples. Existing methods are primarily categorized into parametric and non-parametric approaches. Parametric methods train classifiers end-to-end: SimGCD [4] combines self-distillation with entropy regularization; SPTNet [10] and APTgcd [11] employ prompt tuning to enhance adaptability; PromptCCD [12] manages knowledge through a Gaussian mixture prompt pool. Non-parametric methods follow a “learn features first, then cluster” paradigm: CMS [5] introduces a contrastive mean shift objective for discovering arbitrarily shaped clusters; SelEx [6] designs a hierarchical semi-supervised K-means strategy for fine-grained discrimination; InfoSieve [13] devises an information-theoretic sieving mechanism to filter low-quality pseudo-labels. In the remote sensing domain, the broader open-world recognition landscape has evolved from out-of-distribution detection [14,15] to open-set recognition, with parallel advances in hyperspectral object detection [16,17]. Unsupervised learning has likewise been extensively explored for remote sensing image enhancement, such as hyperspectral super-resolution [18,19,20]; yet GCD remains largely unexplored: to our knowledge, only Xu et al. [21] and our prior work MVTopo [22] have directly addressed this task.

Despite their diversity, all the above methods share a common limitation: they primarily rely on visual self-supervised or semi-supervised signals to shape feature representations and lack any external correction mechanism for boundary-ambiguous samples, those residing between visually similar categories. The resulting noisy pseudo-labels propagate errors throughout training, particularly in remote sensing scenes where inter-class visual similarity is pervasive. This motivates us to explore external knowledge sources, specifically vision-language models, as a complementary supervision channel for GCD.

2.2. Foundation Models for Category Discovery

Recent work has begun to leverage the semantic reasoning capabilities of foundation models, both large language models (LLMs) and vision-language models (VLMs), to assist generalized category discovery beyond purely visual self-supervision.

In the textual domain, Loop [23] first proposes integrating LLMs into the GCD pipeline, using LLM feedback to both guide cluster assignment and generate accurate category names for discovered clusters. GLEAN [8] extends this idea by employing diverse LLM feedback to improve instance-level contrastive features, generate category descriptions, and align uncertain instances with LLM-selected categories, demonstrating substantial gains on text GCD benchmarks.

In the visual domain, several methods exploit the cross-modal alignment of CLIP [24] to introduce textual semantics. CLIP-GCD [25] leverages language guidance from CLIP’s text encoder to supplement visual representations for category discovery. TextGCD [26] constructs a visual lexicon and retrieves relevant text descriptions, employing cross-modality co-teaching between visual and textual branches. GET [27] unlocks CLIP’s multi-modal potential by jointly leveraging vision and text embeddings, achieving state-of-the-art results on GCD benchmarks. In remote sensing, RemoteCLIP [28] establishes a domain-specific vision-language foundation model that learns aligned visual–textual representations for remote sensing imagery. More broadly, recent surveys have examined the rapid convergence of foundation models and remote sensing, documenting the shift from task-specific architectures to general-purpose pretrained models for Earth observation [29,30].

However, existing approaches either operate in text-only domains (Loop, GLEAN) or use foundation models solely as fixed feature extractors (CLIP-GCD, TextGCD, GET) without actively querying them during training for sample-level guidance. When the VLM-guided contrastive signal is introduced as an additional training objective, as required for correcting boundary-ambiguous samples, a fundamental optimization conflict emerges between the locally oriented contrastive loss and the globally oriented clustering objective, a challenge that no prior work has addressed.

2.3. Uncertainty-Based Sample Selection

Selecting informative samples is a long-standing problem in active learning and semi-supervised learning. Classical uncertainty sampling selects the samples for which the model is most uncertain, measured by predictive entropy [31], margin between the top two predictions [32], or committee disagreement [33]. BADGE [34] advances this by jointly capturing uncertainty and diversity through gradient embeddings, demonstrating that hybrid criteria outperform single-metric selection.

In the GCD literature, sample selection strategies are implicit but critical. InfoSieve [13] employs an information-theoretic criterion to filter unreliable pseudo-labels before using them for training. SelEx [6] applies hierarchical K-means with selective expertise to focus learning on fine-grained distinctions. Active GCD [35] explicitly bridges active learning with GCD, proposing an adaptive sampling strategy that balances uncertainty and diversity when selecting samples for oracle annotation. In the VLM-guided setting, GLEAN [8] selects query candidates based solely on predictive entropy to determine which samples are sent to the LLM for neighbor selection.

A common limitation across these approaches is their reliance on a single uncertainty metric. In remote sensing scenes where many categories share high visual similarity, predictive entropy alone cannot distinguish genuinely boundary-critical samples from those that are merely under-converged. This motivates our compound uncertainty design that integrates multiple complementary signals: global assignment uncertainty, geometric boundary proximity, and local neighborhood disagreement, to precisely identify samples where VLM intervention provides maximum value.

3. Proposed Method

3.1. Problem Formulation and Overview

We consider the standard GCD setting. Let ${\mathcal{D}}_L={\left\{({\mathbf{x}}_i,y_i)\right\}}_{i=1}^{N_L}$ denote a labeled set where $y_i\in {\mathcal{C}}_L$, and ${\mathcal{D}}_U={\left\{{\mathbf{x}}_j\right\}}_{j=1}^{N_U}$ an unlabeled set whose samples may belong to either the known classes ${\mathcal{C}}_L$ or novel classes ${\mathcal{C}}_U$, with ${\mathcal{C}}_L\cap {\mathcal{C}}_U=\varnothing$. The goal is to learn a model that simultaneously classifies samples from ${\mathcal{C}}_L$ and discovers clusters corresponding to ${\mathcal{C}}_U$ in the unlabeled data, given only the total number of categories $|{\mathcal{C}}_N|=|{\mathcal{C}}_L|+|{\mathcal{C}}_U|$.

Our framework augments the standard GCD pipeline with a vision-language model (VLM) that serves as an external semantic oracle to correct boundary-ambiguous samples. The overall architecture is illustrated in Figure 2. A ViT backbone $f(·)$ extracts feature representations ${\mathbf{h}}_i=f\left({\mathbf{x}}_i\right)\in {\mathbb{R}}^{768}$, which are fed into a projection head $g(·)$ to produce projected features ${\tilde{\mathbf{z}}}_i=g\left({\mathbf{h}}_i\right)\in {\mathbb{R}}^{256}$. The backbone features ${\mathbf{h}}_i$ are used for classification (via learnable prototypes) and final K-means evaluation, while the projected features ${\tilde{\mathbf{z}}}_i$ serve as the optimization space for VLM-guided contrastive learning. CUQ identifies boundary-critical samples from the unlabeled set, which are then issued as VLM queries for semantic neighbor selection. The VLM-selected positive pairs drive a contrastive alignment loss in the projector space, while a cluster-instance alignment loss operates in the backbone space.

Figure 2 depicts the training-time pipeline. Importantly, the VLM and the DCA projector path are involved only during training. At inference, DualGLEAN reduces to the standard GCD pipeline: the ViT backbone extracts features ${\mathbf{h}}_i$, and K-means assigns clusters on these backbone features, without any VLM query or projector-space loss. The inference process is therefore identical to that of the base GCD model (e.g., MVTopo), introducing zero additional inference-time computation, parameters, or API cost.

The remainder of this section is organized as follows. Section 3.2 describes the VLM-guided neighbor selection module that provides external semantic supervision. Section 3.3 presents DCA, which resolves the space allocation conflict between contrastive alignment and clustering. Section 3.4 details CUQ, which resolves the budget allocation through compound uncertainty filtering. Section 3.5 summarizes the overall training objective.

3.2. VLM-Guided Neighbor Contrastive Learning

Samples residing near category decision boundaries receive unreliable pseudo-labels under standard visual self-supervision, as their visual features are equidistant from multiple cluster centers. To provide explicit semantic correction for such samples, DualGLEAN employs a VLM as an external oracle that judges visual–semantic similarity beyond what the backbone can resolve.

Periodic clustering and neighbor mining. At regular intervals during training (every 5 epochs), the model extracts backbone features for all training samples and performs K-means clustering to obtain cluster assignments ${\widehat{y}}_i$ and cluster centers ${\left\{{\mathsf{\mu }}_k\right\}}_{k=1}^K$. For each sample ${\mathbf{x}}_i$, we retrieve its k-nearest neighbors ${\mathcal{N}}_k\left({\mathbf{x}}_i\right)$ in the normalized feature space using FAISS [36]. CUQ (Section 3.4) then selects a budget of v high-uncertainty samples as the query set $\mathcal{Q}$.

VLM neighbor selection. For each query sample ${\mathbf{x}}_i\in \mathcal{Q}$, we present the VLM with the anchor image and a set of m candidate neighbor images drawn from ${\mathcal{N}}_k\left({\mathbf{x}}_i\right)$. The VLM is prompted to identify which candidate is most semantically similar to the anchor in terms of scene category, considering both visual content and high-level semantics. The VLM returns the index of the selected positive neighbor ${\mathbf{x}}_{i^{+}}$, along with an optional confidence score. This process leverages the VLM’s cross-modal reasoning capability to make semantic judgments that the purely visual backbone cannot resolve, for instance, distinguishing dense residential from commercial area based on building function rather than mere texture pattern. The complete prompt templates are provided in Appendix A.

Neighbor contrastive loss. Given the VLM-selected positive pair $({\mathbf{x}}_i,{\mathbf{x}}_{i^{+}})$, we define an InfoNCE-based contrastive loss that pulls the anchor toward its VLM-confirmed neighbor while pushing it away from other samples in the batch. Crucially, this loss operates in the projector space rather than the backbone space (the rationale is detailed in Section 3.3):

$${\mathcal{L}}_{\mathrm{ncl}}=-{\displaystyle \frac{1}{\left|\mathcal{Q}\right|}}\sum _{i\in \mathcal{Q}}\log {\displaystyle \frac{\exp ({\tilde{\mathbf{z}}}_i·{\tilde{\mathbf{z}}}_{i^{+}}/{\tau }_p)}{{\sum }_{j\ne i}\exp ({\tilde{\mathbf{z}}}_i·{\tilde{\mathbf{z}}}_j/{\tau }_p)}},$$

(1)

where ${\tilde{\mathbf{z}}}_i=g\left(f\left({\mathbf{x}}_i\right)\right)\in {\mathbb{R}}^{256}$ is the L2-normalized projector output, and ${\tau }_p$ is the temperature.

Cluster-instance alignment loss. To further leverage the VLM’s semantic judgment at the cluster level, we align each query sample with its VLM-confirmed cluster center while pushing it away from competing centers. This loss operates in the backbone space:

$${\mathcal{L}}_{\mathrm{ci}}=-{\displaystyle \frac{1}{\left|\mathcal{Q}\right|}}\sum _{i\in \mathcal{Q}}\log {\displaystyle \frac{\exp ({\mathbf{h}}_i·{\mathsf{\mu }}_{c^{+}}/{\tau }_b)}{{\sum }_{k=1}^K\exp ({\mathbf{h}}_i·{\mathsf{\mu }}_k/{\tau }_b)}},$$

(2)

where $c^{+}$ is the cluster index assigned to the VLM-selected neighbor ${\mathbf{x}}_{i^{+}}$, and ${\tau }_b$ is the backbone-space temperature.

Both losses are InfoNCE objectives computed with cosine similarity—dot products on L2-normalized embeddings—which is the standard choice in contrastive self-supervised learning [9]. This is complementary to the Euclidean geometry used elsewhere: K-means clustering and the CUQ boundary-proximity term (Section 3.4) operate on Euclidean distances, the natural metric for global cluster structure. Regarding the gradient of cosine similarity, the “dead zone” of directly maximizing the raw cosine (a gradient $\propto \sin \theta$ that vanishes for already aligned pairs) does not arise in the InfoNCE formulation: its gradient is proportional to $(1-P_{+})$, where $P_{+}$ is the softmax probability of the positive sample, and therefore saturates only once the positive is already clearly separated from all negatives—i.e., at the desired converged state.

3.3. Decoupled Contrastive Alignment (DCA)

Motivation: the space allocation problem. DCA is designed to answer a specific question: where should the VLM-derived contrastive signal be placed so that the neighbor alignment it provides does not disrupt the global clustering structure required by K-means?

The VLM-in-the-loop framework introduces a new loss signal into an existing multi-objective training pipeline. The fundamental question is not merely loss weighting but whether the geometric constraints imposed by the new signal are compatible with those of the existing objectives in a shared space. The neighbor contrastive loss ${\mathcal{L}}_{\mathrm{ncl}}$ (Equation (1)) imposes a local adjacency constraint: it pulls an anchor toward a specific VLM-selected neighbor, optimizing for a manifold organized by pairwise proximity. The classification and K-means objectives impose a global centroid constraint: they require samples to cluster around class-representative prototypes, optimizing for a manifold organized by inter-cluster separability. These two classes of geometric constraints are structurally incompatible.

Concretely, when the VLM identifies a park sample as semantically similar to a resort sample, two visually confusable remote sensing categories, ${\mathcal{L}}_{\mathrm{ncl}}$ pulls their backbone features together while the classification loss simultaneously pushes them toward their respective class prototypes. In a shared space, the stronger InfoNCE gradient dominates this tug-of-war, collapsing the separation between otherwise distinguishable categories. Standard GCD methods avoid this conflict because all their losses, including classification, self-distillation, and mean-shift, share the same global-clustering objective. The VLM-guided neighbor signal is the first loss in this pipeline that optimizes for local pairwise geometry, creating a qualitatively new and unresolved allocation problem.

When both types of losses share the backbone space, the InfoNCE gradient dominates due to its batch-wide denominator, a structural consequence of its formulation rather than a mere tuning issue, and triggers a geometric conflict that ultimately leads to manifold collapse, with the globally separable cluster structure required by K-means disintegrating into fragmented local neighborhoods. The core tension is thus: the contrastive alignment that VLM supervision provides is valuable, but the space in which it is applied determines whether it helps or harms. Decoupling these two decisions, preserving the alignment signal while isolating its geometric side effects, is the design principle behind DCA.

Design. We resolve this conflict by routing the two types of objectives to separate feature spaces according to their geometric requirements:

• Projector space (${\mathbb{R}}^{256}$): The neighbor contrastive loss ${\mathcal{L}}_{\mathrm{ncl}}$ operates on ${\tilde{\mathbf{z}}}_i=g\left({\mathbf{h}}_i\right)$, where $g(·)$ is the MLP projection head (768 → 2048 → 2048 → 256) that already exists in the DINO-based GCD architecture. The low-dimensional projector space has limited information capacity, making it well-suited for encoding local pairwise similarity relations without needing to maintain global structure.
• Backbone space (${\mathbb{R}}^{768}$): The cluster-instance loss ${\mathcal{L}}_{\mathrm{ci}}$, the classification loss, and the final K-means evaluation all operate on ${\mathbf{h}}_i$. The high-dimensional backbone space has sufficient capacity to represent global semantic category structure with well-separated clusters.

The decoupling is purely a routing-level modification. ${\mathcal{L}}_{\mathrm{ncl}}$ computes gradients with respect to ${\tilde{\mathbf{z}}}_i$, which backpropagates through g and then to f, while ${\mathcal{L}}_{\mathrm{ci}}$ computes gradients directly with respect to ${\mathbf{h}}_i$. No new parameters are introduced; the existing projection head $g(·)$ naturally serves as the separation boundary. Figure 3 illustrates this decoupling.

The practical consequence of this routing is straightforward. If ${\mathcal{L}}_{\mathrm{ncl}}$ operates on the backbone feature $\mathbf{h}$, the same space where K-means evaluates, InfoNCE pulls anchor–neighbor pairs together in the exact space that requires global separability, directly damaging the clustering structure. If ${\mathcal{L}}_{\mathrm{ncl}}$ operates on the projector feature $\tilde{\mathbf{z}}$ instead, InfoNCE can aggressively align VLM-selected pairs without consequence for K-means, because $\tilde{\mathbf{z}}$ is not the evaluation space. The projector’s MLP layers ($768\to 2048\to 2048\to 256$) further attenuate the gradient before it reaches the backbone. Through this gradient routing dynamic, $\mathbf{h}$ receives only a weak, indirect signal from ${\mathcal{L}}_{\mathrm{ncl}}$, which is sufficient to benefit from the VLM’s semantic judgment yet too weak to reshape the global cluster topology.

Technical advantage. DCA enables the VLM-guided contrastive alignment to operate at full strength in its dedicated space while keeping the backbone globally separable. Rather than balancing incompatible constraints in a single space through loss weighting, which cannot reconcile fundamentally contradictory geometric requirements, we route them to spaces whose dimensionalities naturally match their geometric objectives. The low-dimensional projector space (${\mathbb{R}}^{256}$) has sufficient capacity to encode whether two samples are semantically similar, a binary relation that aligns naturally with the contrastive alignment objective, but insufficient capacity to represent the full global category topology, naturally bounding its influence. This capacity argument follows the information-bottleneck principle [37]: compressing the backbone embedding to a 256-d code retains only the information necessary for the binary similarity judgment while discarding the global topology that K-means requires. It is also consistent with prior analyses showing that the InfoNCE objective reshapes the geometry of the space in which it operates, inducing dimensional collapse in contrastive self-supervised learning [38]. By routing ${\mathcal{L}}_{\mathrm{ncl}}$ to the disposable projector space, DCA confines this geometric reshaping away from the backbone, which remains intact for clustering. The backbone space (${\mathbb{R}}^{768}$) retains the high capacity needed for K-class global separability without interference.

In this design, the InfoNCE gradients from ${\mathcal{L}}_{\mathrm{ncl}}$ primarily shape the projector manifold, while their influence on the backbone is attenuated by passage through the multi-layer MLP, effectively decoupling the two geometric regimes. In essence, DCA answers where the VLM-derived contrastive signal should align features: not in the evaluation space but in a separate, dimensionally matched space where alignment helps without destabilizing the global cluster topology. As we demonstrate in Section 4.2, this allocation eliminates the known-class clustering collapse observed under single-space integration, restoring K-means accuracy to baseline levels while simultaneously enabling VLM-guided semantic correction to provide gains above the baseline.

3.4. Compound Uncertainty Querying (CUQ)

Motivation: the budget allocation problem. CUQ is designed to answer a specific question: given a fixed per-round budget of v VLM queries, which v samples deserve these costly queries to maximize discriminative return? VLM queries incur both per-call API fees and network round-trip latency, making each wasted query costly in two dimensions. The standard approach of ranking all samples by predictive entropy implicitly assumes that high entropy is synonymous with boundary ambiguity. However, in remote sensing scenes with pervasive inter-class visual similarity, entropy is an unreliable discriminator: (1) under-converged samples in early training stages can exhibit high entropy simply because their class-conditional distributions are not yet formed, and (2) incorrectly but confidently assigned boundary samples can exhibit moderate entropy despite being precisely the cases where VLM correction would be most valuable. These two failure modes are structural, not transient. They arise because entropy measures global assignment flatness, which conflates genuine category ambiguity with insufficient model convergence. Resolving this requires moving beyond single scalar uncertainty and jointly observing complementary dimensions of boundary criticality.

Design. CUQ implements the budget allocation decision through a three-stage filtering pipeline that progressively narrows the candidate pool before querying the VLM. The core idea is that no single scalar can reliably identify boundary-critical samples; instead, we compute the following compound uncertainty score by jointly evaluating three complementary dimensions of boundary criticality:

(1) Predictive entropy ${\mathcal{H}}_i$ measures global assignment uncertainty:

$${\mathcal{H}}_i=-{\displaystyle \sum _{k=1}^K}{p}_{ik}\log p_{ik},$$

(3)

where $p_{ik}$ is the soft cluster assignment probability of sample ${\mathbf{x}}_i$ to cluster k, computed via a Student’s t-distribution kernel over distances to cluster centers.

(2) Boundary proximity $B_i$ measures geometric positioning relative to the decision boundary between the two nearest clusters:

$$B_i=1-{\displaystyle \frac{d({\mathbf{h}}_i,{\mathsf{\mu }}_1)}{d({\mathbf{h}}_i,{\mathsf{\mu }}_2)}},$$

(4)

where ${\mathsf{\mu }}_1$ and ${\mathsf{\mu }}_2$ are the nearest and second-nearest cluster centers, respectively. $B_i$ approaches 1 when the sample is equidistant from both centers (geometrically on the decision boundary) and approaches 0 when it is firmly within one cluster.

(3) Local label inconsistency $L_i$ measures pseudo-label disagreement within the sample’s local neighborhood:   

$$L_i=-{\displaystyle \sum _{k=1}^K}{\widehat{p}}_{ik}\log {\widehat{p}}_{ik},$$

(5)

where ${\widehat{p}}_{ik}$ is the proportion of sample ${\mathbf{x}}_i$’s k-nearest neighbors that are assigned (by pseudo-label) to cluster k. High $L_i$ indicates that the local neighborhood contains samples from multiple categories, a hallmark of genuine decision boundary regions rather than mere model uncertainty.

The compound uncertainty score is defined as their weighted combination:

$$U_i={\mathcal{H}}_i+\alpha ·B_i+\beta ·L_i,$$

(6)

where $\alpha$ and $\beta$ control the relative contribution of each component (set to $\alpha = 0.5$, $\beta = 0.3$ in our experiments). These coefficients reflect the role and risk of each component: $B_i$ is informative but prone to false positives—the component ablation (Section 4.4.2) shows that adding B to entropy alone degrades All accuracy from 89.90% to 88.07%—so it receives a moderate weight $\alpha =0.5$; $L_i$ acts primarily as a verification gate that suppresses false positives, so it receives a lower weight $\beta =0.3$. The top-v samples ranked by $U_i$ form the query set $\mathcal{Q}$ sent to the VLM. Figure 4 summarizes the three-stage CUQ filtering pipeline.

Technical advantage. CUQ transforms VLM querying from a passive, single-criterion selection into an active, multi-dimensional filtering process. Each component targets a distinct failure mode. ${\mathcal{H}}_i$ captures globally uncertain samples. $B_i$ identifies samples geometrically positioned at decision boundaries, which is critical for distinguishing visually similar RS categories such as medium-density residential and dense residential. $L_i$ filters out false positives: samples that are globally uncertain or boundary-positioned but whose neighborhoods are actually homogeneous, indicating that the uncertainty stems from under-convergence rather than true category ambiguity. The three-stage filtering ensures that VLM queries are reserved exclusively for samples that are simultaneously globally uncertain, geometrically boundary-critical, and surrounded by conflicting local evidence. In essence, CUQ answers whom to query: not every uncertain sample but only those whose uncertainty is corroborated by geometric positioning and neighborhood evidence. These are precisely the samples where external semantic judgment provides maximum discriminative return per query.

This budget-allocation design directly addresses the semantic sparsity inherent to VLM-in-the-loop training. Because VLM inference is costly, only a small fraction of samples can receive explicit semantic feedback per round, so the semantic supervision is necessarily sparse in coverage. CUQ turns this sparsity from a limitation into a manageable resource by concentrating the budget on boundary-critical samples where semantic correction yields the highest return. Moreover, the sparse signal is not confined to the queried samples: the neighbor contrastive loss ${\mathcal{L}}_{\mathrm{ncl}}$ propagates gradients through its batch-wide denominator, and the cluster-instance alignment loss ${\mathcal{L}}_{\mathrm{ci}}$ broadcasts each VLM judgment to the entire cluster via its centroid, thereby amplifying the reach of the sparse semantic feedback.

3.5. Overall Training Objective

The base GCD training framework follows the parametric classification paradigm. A set of $K=|{\mathcal{C}}_N|$ learnable class prototypes serves as the classifier. A mean-teacher framework with self-distillation generates sharpened pseudo-labels. The base loss combines supervised classification on labeled samples, unsupervised clustering via self-distillation, a topology-aware contrastive mean shift objective for feature compactness, and a memory-based consistency regularization:

$${\mathcal{L}}_{\mathrm{base}}=(1-\gamma ){\mathcal{L}}_{\mathrm{cluster}}+\gamma {\mathcal{L}}_{\mathrm{cls}}+(1-\gamma ){\mathcal{L}}_{\mathrm{con}}+\gamma {\mathcal{L}}_{\mathrm{mem}},$$

(7)

where $\gamma$ is the supervised loss weight.

The VLM-guided losses are added as complementary objectives:

$$\mathcal{L}={\mathcal{L}}_{\mathrm{base}}+{\lambda }_1{\mathcal{L}}_{\mathrm{ncl}}+{\lambda }_2{\mathcal{L}}_{\mathrm{ci}},$$

(8)

where ${\lambda }_1$ and ${\lambda }_2$ control the contribution of the neighbor contrastive loss (projector space) and cluster-instance alignment loss (backbone space), respectively.

Training procedure. The training alternates between two phases: (1) a standard training phase where the model optimizes $\mathcal{L}$ using the current VLM-selected neighbor pairs and (2) a re-mining phase (every 5 epochs) where the model re-extracts features, re-clusters, recomputes compound uncertainty scores, and queries the VLM for fresh neighbor selections. This periodic refresh ensures that the VLM guidance adapts to the evolving feature space as training progresses, progressively targeting new boundary regions that emerge as clusters refine. The VLM-selected pairs and their associated losses are cached between re-mining phases to avoid redundant VLM calls during standard training iterations. The complete procedure is summarized in Algorithm 1.

Algorithm 1 DualGLEAN: dual allocation training with DCA and CUQ
Require:  Labeled set ${\mathcal{D}}_L$, unlabeled set ${\mathcal{D}}_U$, DINO-pretrained backbone $f(·)$, projector $g(·)$, VLM oracle, re-mining interval T, query budget v. Ensure:  Trained backbone $f(·)$ and final cluster assignments.
1: Freeze all ViT blocks except the last.   2: Run K-means on all backbone features to obtain initial centers $\left\{{\mathsf{\mu }}_k\right\}$.   3: for $epoch=1$ to E do
4:    if $epoch\mathrm{mod}T=0$ then	▹ Re-mining phase
5:    Re-cluster: extract $\left\{{\mathbf{h}}_i\right\}$, run K-means → centers $\left\{{\mathsf{\mu }}_k\right\}$, pseudo-labels $\widehat{y}$, soft assignments $p_{ik}$.
6:    Select query set $\mathcal{Q}$: top-v samples by $U_i={\mathcal{H}}_i+\alpha B_i+\beta L_i$ (Equation (6)).	▹ CUQ
7:    For each ${\mathbf{x}}_i\in \mathcal{Q}$, query the VLM over KNN candidates ${\mathcal{N}}_k\left({\mathbf{x}}_i\right)$→ neighbor ${\mathbf{x}}_{i^{+}}$ and cluster $c^{+}$.   8:    end if
9:    for each batch do	▹ Standard training phase
10:    Forward ${\mathbf{h}}_i=f\left({\mathbf{x}}_i\right)$, ${\tilde{\mathbf{z}}}_i=g\left({\mathbf{h}}_i\right)$.
11:    $\mathcal{L}={\mathcal{L}}_{\mathrm{base}}+{\lambda }_1{\mathcal{L}}_{\mathrm{ncl}}+{\lambda }_2{\mathcal{L}}_{\mathrm{ci}}$ (Equation (8)), with ${\mathcal{L}}_{\mathrm{ncl}}$ on ${\tilde{\mathbf{z}}}_i$ and ${\mathcal{L}}_{\mathrm{ci}}$ on ${\mathbf{h}}_i$.	▹ DCA routing
12:    Update $f,g$ by SGD on $\mathcal{L}$. 13:    end for 14: end for 15: Run K-means on $\left\{{\mathbf{h}}_i\right\}$ to obtain final cluster assignments.

4. Experiments

4.1. Experimental Setup

Datasets. We evaluate on two remote sensing scene classification datasets. AID [39] contains 10,000 images across 30 aerial scene categories. RSSDIVCS [40] contains 55,445 images across 70 scene categories. Following the standard GCD protocol [1], classes in each dataset are randomly split into 50% known (${\mathcal{C}}_L$) and 50% novel (${\mathcal{C}}_U$); 50% of images from known classes form the labeled set ${\mathcal{D}}_L$, while all remaining images constitute the unlabeled set ${\mathcal{D}}_U$.

Evaluation protocol. We follow the standard GCD evaluation [1]: after training, K-means clustering is applied to the backbone features of the unlabeled set, and the Hungarian algorithm [41] matches clusters to ground-truth labels. Clustering accuracy is reported separately for All classes, Known (Old) classes, and Novel (New) classes.

Implementation details. We adopt ViT-B/16 [7] with DINO [42] pretrained weights as the backbone. Input resolution is $224\times 224$. Only the last Transformer block and the projection head are fine-tuned. We use SGD optimizer (momentum 0.9, weight decay ${10}^{-4}$, initial learning rate 0.1) with cosine annealing over 40 epochs and batch size 512. The supervised loss weight $\gamma =0.35$. For the VLM, we use Qwen3-VL-Flash as the default oracle. The VLM query budget $v=128$, the KNN neighbor pool size $k=20$, and 5 answer options are presented to the VLM per query. Feature re-clustering and VLM re-querying occur every $T=5$ epochs. The compound uncertainty weights are set to $\alpha =0.5$, $\beta =0.3$. The neighbor contrastive loss weight ${\lambda }_1=0.3$ and cluster-instance loss weight ${\lambda }_2=0.05$. The projector-space temperature ${\tau }_p=0.2$ and backbone-space temperature ${\tau }_b=0.07$. All experiments are conducted on an NVIDIA RTX 5090 GPU (NVIDIA Corporation, Santa Clara, CA, USA).

Baselines. We compare against representative GCD methods spanning both parametric and non-parametric paradigms: GCD [1], SimGCD [4], CMS [5], InfoSieve [13], SelEx [6], and APTgcd [11]. We also include our prior work MVTopo [22] as a strong remote-sensing-specific baseline. For fair comparison, all methods use the same ViT-B/16 backbone with DINO pretrained weights, identical data splits, and the same evaluation protocol.

4.2. Naive VLM Transfer: Why Direct Integration Fails

Before presenting DualGLEAN, we first demonstrate that directly adopting the GLEAN paradigm, i.e., applying VLM-guided neighbor contrastive learning in the backbone space with entropy-based query selection, is counterproductive in remote sensing GCD. We integrate this naive VLM module into the MVTopo baseline and evaluate on AID and RSSDIVCS.

As shown in

Table 1. Naive VLM transfer vs. MVTopo baseline. VLM-guided contrastive learning applied directly in the backbone space degrades performance below the baseline on both datasets, confirming the dual allocation failure.

Method	AID			RSSDIVCS
	All	Old	New	All	Old	New
MVTopo [22] (baseline)	86.84	92.12	84.16	91.72	96.89	89.14
+Naive GLEAN	82.03	92.83	76.55	86.80	90.73	84.84

, the naive VLM integration degrades overall performance below the MVTopo baseline on both datasets. Notably, the degradation is asymmetric: Novel-class accuracy suffers the most severe and consistent drop across both datasets (AID: $-7.61\%$, RSSDIVCS: $-4.30\%$), while Old-class accuracy shows mixed behavior, a slight increase on AID ($+0.71\%$) but a substantial drop on RSSDIVCS ($-6.16\%$). This asymmetry is consistent with the dual allocation analysis: the InfoNCE gradient reshapes the backbone feature manifold toward local pairwise structures, which disproportionately disrupts novel-class clusters that lack supervised anchors, while known classes, partially stabilized by labeled samples, are less uniformly affected. The overall accuracy degradation confirms that VLM guidance, when naively integrated, is not merely ineffective but actively harmful, motivating the dual allocation design of DualGLEAN.

4.3. Comparison with Existing Methods

Table 2. Clustering accuracy comparison on AID and RSSDIVCS. Best results are bold, second-best are underlined.

Method	AID			RSSDIVCS
	All	Old	New	All	Old	New
GCD [1]	73.93	81.95	69.86	81.00	80.58	81.20
SimGCD [4]	79.52	78.12	80.23	87.85	89.66	86.94
CMS [5]	71.54	78.99	67.77	76.57	77.16	76.28
InfoSieve [13]	86.11	97.75	80.21	87.49	96.90	83.49
SelEx [6]	85.55	89.63	83.48	91.22	95.99	88.84
APTgcd [11]	82.61	93.54	77.07	87.96	96.90	83.49
MVTopo [22]	86.84	92.12	84.16	91.72	96.89	89.14
DualGLEAN	91.54	96.29	89.14	92.90	97.08	90.81

reports the main results on AID and RSSDIVCS. DualGLEAN, a plug-in module that augments GCD baselines through DCA and CUQ, is denoted as DualGLEAN (shown here with MVTopo as the base GCD pipeline, following our prior work).

Several observations emerge from Table 2. Overall improvement. DualGLEAN achieves the highest accuracy among all compared methods on both datasets. On AID, we attain 91.54% All accuracy, outperforming InfoSieve by 5.43% and our prior work MVTopo by 4.70%. The gain is particularly pronounced on Novel accuracy (89.14%, +4.98% over MVTopo), indicating that VLM-guided semantic correction primarily benefits the discovery of unseen categories where visual self-supervision alone is insufficient. On RSSDIVCS, DualGLEAN reaches 92.90% All and 90.81% Novel, outperforming SelEx by 1.68% and 1.97%, respectively. The larger improvement on AID reflects its higher inter-class visual similarity (30 fine-grained scene categories), which creates more boundary-ambiguous samples that benefit from VLM intervention. The comparatively smaller gain on RSSDIVCS stems from the already-strong MVTopo baseline (91.72% All), which leaves limited headroom; in this high-performance regime, a 1.18% All and 1.67% Novel improvement remains meaningful. The value of DualGLEAN is more clearly evidenced by its plug-in generality (Section 4.5.1), where it improves GCD, SimGCD, and SelEx by +5.60%, +6.32%, and +4.47% All, respectively, and it delivers these gains with zero additional trainable parameters (Section 4.7) and a total VLM API cost of only CNY 2.45 per full AID training run (Section 4.6).

Balance between known and novel performance. A key challenge in GCD is maintaining known-class accuracy while improving novel-class discovery; parametric methods often overfit known classes at the expense of novel ones. DualGLEAN achieves a favorable balance: on AID, Old accuracy reaches 96.29% (approaching InfoSieve’s 97.75%, yet InfoSieve attains only 80.21% on New), while simultaneously achieving the highest Novel accuracy among all compared methods (89.14%). On RSSDIVCS, we achieve 97.08% Old and 90.81% New, both the highest among all compared methods. This balanced improvement confirms that the dual allocation design does not trade off one objective against the other: space decoupling protects the backbone’s global clustering structure (preserving known-class separability), while compound uncertainty directs VLM queries toward novel-class boundary regions where external semantic judgment provides maximum discriminative return.

Consistency across datasets. The gains are consistent across both datasets despite their differing scales (30 vs. 70 classes) and compositions. On AID, where categories such as resort and park share highly similar visual patterns, VLM semantic judgment provides decisive discrimination that visual features alone cannot achieve. On RSSDIVCS, the larger category space creates a more challenging neighbor selection problem, since there is a higher chance of selecting visually similar but semantically different neighbors, yet CUQ effectively filters the candidate pool and maintains robust performance.

4.4. Ablation Studies

All ablation experiments in this section are conducted on the AID dataset.

4.4.1. Module-Level Ablation

To quantify the contribution of each proposed mechanism, we evaluate three variants: (1) MVTopo baseline, (2) MVTopo + DCA only (space decoupling, no compound querying), (3) MVTopo + CUQ only (compound querying, no space decoupling), and (4) MVTopo + DualGLEAN (the full plug-in).

Table 3. Module-level ablation on AID. DCA and CUQ each contribute independently; their combination (DualGLEAN) achieves the highest accuracy.

DCA	CUQ	All	Old	New
		86.84	92.12	84.16
✓		89.45	91.53	88.40
	✓	90.93	95.19	88.78
✓	✓	91.54	96.29	89.14

reveals the distinct roles and synergy of the two mechanisms. DCA alone (+2.61% All, +4.24% New) primarily benefits novel-class discovery. By routing ${\mathcal{L}}_{\mathrm{ncl}}$ to the projector space, DCA prevents the InfoNCE gradient from fragmenting the backbone feature manifold into local neighbor cliques. This clean separation allows novel-class samples, which lack any supervised signal, to form coherent, well-separated clusters in the backbone space without being pulled toward spurious local neighborhoods. The slight Old-accuracy drop ($-0.59\%$) under DCA-only indicates that without CUQ, VLM queries may occasionally target non-critical samples, introducing mild noise into the known-class region. CUQ alone (+4.09% All) benefits both Old (+3.07%) and New (+4.62%) accuracy by concentrating the VLM budget on genuinely boundary-critical samples. Importantly, even without space decoupling, the compound metric still yields gains because correctly identified query samples provide useful signals despite the suboptimal signal space; the budget allocation improvement partially compensates for the space allocation deficiency. Full model (+4.70% All, +4.17% Old, +4.98% New) achieves the highest accuracy, outperforming each individual mechanism. The synergy is clear: DCA ensures that VLM signals land in a compatible geometric subspace, while CUQ ensures that only the most discriminative samples enter that channel. Failure on either dimension, as demonstrated by DCA-only for budget failure and CUQ-only for space failure, leaves performance on the table; resolving both jointly unlocks the full gain.

4.4.2. Compound Uncertainty Component Ablation

To validate the contribution of each component in our compound uncertainty metric $U_i={\mathcal{H}}_i+\alpha B_i+\beta L_i$, we ablate the boundary proximity (B) and local label inconsistency (L) terms while keeping predictive entropy ($\mathcal{H}$) as the base signal.

Table 4. Compound uncertainty component ablation on AID. All components contribute positively; the full three-component metric achieves the highest accuracy. A checkmark (✓) indicates that the corresponding component is included in the compound uncertainty score.

$\mathcal{H}$	B	L	All	Old	New
✓			89.90	94.36	87.64
✓	✓		88.07	93.30	85.42
✓		✓	89.97	94.21	87.82
✓	✓	✓	91.54	96.29	89.14

reveals a nuanced interaction pattern that validates our compound design. Entropy only ($\mathcal{H}$, first row) serves as the single-metric baseline and achieves 89.90% All. Adding boundary proximity ($\mathcal{H}+B$, second row) surprisingly degrades performance to 88.07% All ($-1.83\%$). This counter-intuitive result exposes a critical failure mode: B alone identifies samples geometrically positioned at decision boundaries, but many such samples have homogeneous local neighborhoods; their high boundary score reflects early-stage under-convergence rather than genuine category ambiguity. Querying the VLM about these false-positive boundary samples wastes the budget and introduces misleading correction signals. Adding local label inconsistency ($\mathcal{H}+L$, third row) recovers to 89.97% All, marginally above entropy-only. L filters out samples whose neighborhoods are homogeneous, but without B, it cannot distinguish samples near cluster centers that happen to have heterogeneous neighborhoods from those truly at decision boundaries. Full metric ($\mathcal{H}+B+L$, fourth row) achieves 91.54% All, the highest accuracy among all variants, yielding a 1.64% improvement over entropy-only. The conjunction of all three signals is essential: $\mathcal{H}$ provides global uncertainty awareness, B identifies geometric boundary positioning, and L acts as a verification gate that suppresses false positives from B. Only samples passing all three criteria, being globally uncertain, geometrically boundary-positioned, and locally contested, reach the VLM. This three-way filtering effect explains both the failure of $\mathcal{H}+B$, where the verification gate is missing, and the marginal gain of $\mathcal{H}+L$, where boundary awareness is missing, and confirms that budget allocation in remote sensing GCD requires multi-dimensional evidence rather than any single scalar.

To further verify the sensitivity of DualGLEAN to the CUQ weighting coefficients, we independently vary $\alpha$ and $\beta$ around the default configuration ($\alpha =0.5$, $\beta =0.3$) and report the results in

Table 5. Weight sensitivity of the CUQ components on AID. The default configuration ($\alpha =0.5$, $\beta =0.3$) is listed first.

$\mathit{\alpha }$	$\mathit{\beta }$	All	Old	New
0.50	0.30	91.54	96.29	89.14
0.25	0.30	91.42	94.64	89.79
1.00	0.30	90.99	94.52	89.20
0.50	0.15	89.21	94.21	86.68
0.50	0.60	91.64	95.94	89.45

.

The default configuration ($\alpha =0.5$, $\beta =0.3$) achieves 91.54% All accuracy, which is essentially optimal—the best variant ($\alpha =0.5$, $\beta =0.60$) reaches only 0.10% higher. The framework is largely insensitive to the boundary-proximity weight: varying $\alpha$ over a $4\times$ range (0.25–1.00) changes All accuracy by at most 0.55%. The local-inconsistency weight $\beta$ is somewhat more influential: increasing it to 0.60 leaves accuracy nearly unchanged (91.64%), whereas reducing it to 0.15 degrades All to 89.21% ($-2.33\%$) and Novel to 86.68% ($-2.46\%$). This asymmetric sensitivity aligns with the role of L as a verification gate—when $\beta$ is too small, false positive boundary samples are no longer suppressed and the VLM budget is wasted on non-critical samples. Overall, these results confirm that the chosen static weights are well-justified and robust across a wide range of perturbations and that the CUQ metric does not require per-dataset coefficient tuning.

4.5. Generality Analysis

4.5.1. Plug-In Generality Across GCD Baselines

A key claim of this work is that the proposed VLM-guided module functions as a general-purpose enhancement rather than being tied to a specific base architecture. To verify this, we integrate DualGLEAN with three representative GCD baselines, namely, GCD [1], SimGCD [4], and SelEx [6], and evaluate on both AID and RSSDIVCS.

Table 6. Plug-in generality on AID and RSSDIVCS. Integrating our VLM-guided module consistently improves all three GCD baselines across both datasets.

Method	AID			RSSDIVCS
	All	Old	New	All	Old	New
GCD [1]	73.93	81.95	69.86	81.00	80.58	81.20
GCD + DualGLEAN	79.53	83.56	77.49	84.71	91.41	82.86
SimGCD [4]	79.52	78.12	80.23	87.85	89.66	86.94
SimGCD + DualGLEAN	85.84	87.39	85.06	89.09	92.85	88.21
SelEx [6]	85.55	89.63	83.48	91.22	95.99	88.84
SelEx + DualGLEAN	90.02	94.64	87.68	92.25	96.82	89.96

demonstrates that our VLM-guided module generalizes effectively across diverse GCD architectures. Integrating DualGLEAN yields consistent and substantial gains for all three baselines on AID: GCD improves by 5.60% All (73.93%→79.53%), SimGCD by 6.32% (79.52%→85.84%), and SelEx by 4.47% (85.55%→90.02%). Two patterns are noteworthy. First, the improvement magnitude inversely correlates with baseline strength: the weakest baseline (GCD, 73.93%) gains the most (+5.60%), while the strongest (SelEx, 85.55%) gains less but still substantially (+4.47%). This indicates that VLM guidance is especially beneficial when the base visual representation is least reliable, precisely the regime where external semantic judgment is most needed. Second, the weaker the baseline, the larger the Novel-class gain relative to Old: on GCD, Novel improves by $+7.63\%$ vs. Old $+1.61\%$ ($5.7\times$ larger), whereas on the stronger SelEx, the two gains are comparable ($+4.20\%$ vs. $+5.01\%$). This gradient suggests that VLM semantic correction is particularly impactful when the base visual representation is too weak to separate novel categories at all, whereas for stronger baselines that already form reasonable clusters, VLM guidance provides a more balanced improvement across both known and novel classes.

On RSSDIVCS, the module likewise improves all three baselines on every metric: GCD gains $+3.71\%$ All (81.00%→84.71%), SimGCD $+1.24\%$ (87.85%→89.09%), and SelEx $+1.03\%$ (91.22%→92.25%). The gains are smaller than on AID because the RSSDIVCS baselines are already strong—SelEx reaches 91.22% All—leaving limited headroom; nonetheless, the inverse-correlation pattern persists, with the weakest baseline (GCD) benefiting most. Notably, GCD’s Old-class accuracy improves markedly by $+10.83\%$ (80.58%→91.41%), indicating that VLM semantic correction substantially stabilizes known-class boundaries when the base representation is weak. These results establish that DualGLEAN is not tied to a specific base architecture but functions as a general-purpose enhancement module for GCD.

4.5.2. Robustness to VLM Backbone

To assess whether DualGLEAN depends on a specific VLM, we replace the default Qwen3-VL-Flash with six alternative VLMs spanning a wide range of architectures and scales: Qwen2.5-VL-7B, Qwen2.5-VL-72B, Step-3.7, DeepSeek-V4-Pro, Kimi-K2.5, and GLM-4V-9B, and re-evaluate on AID under identical settings.

Table 7. VLM backbone robustness on AID. The framework maintains consistent performance across different VLM architectures, confirming that the dual allocation design is independent of the specific VLM choice.

VLM	Params	All	Old	New
Qwen3-VL-Flash (default)	∼8B *	91.54	96.29	89.14
Qwen2.5-VL-7B	7B	91.20	95.90	88.82
Qwen2.5-VL-72B	72B	91.61	96.57	89.10
Step-3.7 Flash	∼198B (11B active) †	88.19	92.35	86.08
DeepSeek-V4-Pro	∼1.6T ‡	87.41	91.76	85.20
Kimi-K2.5	∼1T (32B active)	86.32	91.53	83.68
GLM-4V-9B	9B	87.63	92.87	84.98

* Approximate; Alibaba has not disclosed the exact parameter count for the Flash endpoint. See Qwen3-VL Technical Report [43]. ^† Step-3.7 Flash: ∼198B total (MoE), ∼11B active per token. ^‡ Approximate; DeepSeek has not yet released an official technical report.

reports performance across seven VLMs spanning diverse architectures and providers. Two performance tiers emerge naturally. The Qwen family (Qwen3-VL-Flash, Qwen2.5-VL-7B, Qwen2.5-VL-72B) consistently achieves All-accuracy in the range 91.20–91.61%, with a within-family variation of only 0.41%. The remaining four VLMs (Step-3.7, DeepSeek-V4-Pro, Kimi-K2.5, GLM-4V) form a second tier at 86.32–88.19%, roughly 3–5% below the Qwen group. This gap suggests that VLM visual reasoning quality does influence the precision of neighbor selection, and that VLMs with stronger fine-grained visual discrimination, as exhibited by the Qwen series, yield higher-quality positive pairs. Nevertheless, even the weakest VLM tested (Kimi-K2.5, 86.32% All) still provides meaningful semantic guidance, and the consistent ordering across All, Old, and New metrics confirms that the dual allocation mechanism generalizes across VLM families. The framework therefore remains deployable with a broad range of VLMs, with the Qwen series recommended when maximum accuracy is required.

4.6. VLM Query Efficiency

A practical concern with VLM-in-the-loop training is the computational overhead of API calls. To quantify this, we profile the token consumption and cost of DualGLEAN on the AID dataset (30 classes, 128 query samples per round, Qwen3-VL-Flash as the VLM). The VLM is invoked for three distinct purposes during each re-mining phase, each with a different query pattern.

Table 8. VLM query efficiency on AID (Qwen3-VL-Flash, 128 query samples, 40 epochs). Category characterization generates semantic descriptions for each of the 30 K-means clusters. Neighbor selection sends the anchor and 5 KNN candidates to the VLM for each query sample. Cluster-instance alignment uses a text-and-single-image query to match anchors to the most semantically appropriate cluster.

	Tokens	% of Cycle
Per-Query Type
Category Characterization (5 img)	2.7K	14%
Neighbor Selection (1 + 5 img)	3.2K	68%
Cluster-Instance Alignment (1 img)	0.8K	18%
Overall Training
Tokens per re-mining cycle	594K	—
Full training tokens	4.75M	—
Full training API cost	CNY 2.45	—

breaks down the token consumption by query type. Neighbor selection dominates the per-cycle budget at 68% (406K/594K), as each of the 128 query samples requires a 6-image VLM call. Category characterization accounts for 14% (82K/594K): 30 cluster descriptions at 2.7K tokens each, amortized over all query samples. Cluster-instance alignment contributes only 18% (106K/594K), a lightweight text-and-single-image query that is entirely optional (disabling it, i.e., setting ${\lambda }_2=0$, reduces per-cycle cost to 488K tokens).

The end-to-end training cost is inherently tied to the search-scope configuration. Under the default setting ($v=128$, $k=20$, 5 options, 5 reps), the full 40-epoch training on AID consumes approximately 4.75M tokens at a total API cost of CNY 2.45, a negligible expense compared to the GPU compute cost of training. The dominant cost driver is the query budget v (neighbor selection and cluster-instance alignment each scale linearly with v, together accounting for 86% of per-cycle tokens), while the category characterization overhead is proportional to the number of dataset classes rather than v. Doubling v adds approximately 512K tokens per cycle, allowing flexible trade-offs between accuracy and cost. These numbers are reported for Qwen3-VL-Flash on AID; costs for other VLMs or datasets can be estimated by multiplying the per-cycle token figures by the respective API pricing and the number of re-mining cycles. Local deployment or prompt compression would further reduce costs.

4.7. Model Complexity

DCA is designed to introduce zero additional trainable parameters to the base network by reusing the existing DINOHead MLP projection head.

Table 9. Model complexity with and without DualGLEAN. Params are unchanged; the FLOPs increase is marginal (<5%) and training only.

Method	Params (M)	FLOPs (G)
MVTopo [22]	94.40	17.13
MVTopo + DualGLEAN	94.40	18.04
GCD [1]	91.97	16.87
GCD + DualGLEAN	91.97	17.59
SimGCD [4]	92.01	16.90
SimGCD + DualGLEAN	92.01	17.57
SelEx [6]	108.79	16.89
SelEx + DualGLEAN	108.79	17.58

confirms this: across all four GCD baselines, integrating DualGLEAN leaves the parameter count unchanged while incurring only a marginal FLOPs increase of 0.69–0.90 G (within 5%). The small FLOPs overhead stems from the extra forward pass through the projector for the VLM-selected neighbor image during training and the CUQ scoring computations during re-mining. Crucially, these overheads are training only: at inference, the model is identical to the base architecture, with no additional computation.

4.8. Hyperparameter Sensitivity

We analyze sensitivity to four key search-scope hyperparameters: the number of KNN neighbors (k), the VLM query budget (v), the number of VLM answer options, and the number of re-mining repetitions per epoch.

Table 10. Hyperparameter sensitivity on AID. The default configuration ($k = 20$, $v = 128$, options = 5, reps = 5) is used in all other experiments.

k	Budget v	Options	Reps	All	Old	New
5	32	2	5	86.66	95.43	82.22
10	64	3	5	90.42	93.34	88.94
20	128	5	5	91.54	96.29	89.14
20	128	5	10	93.89	95.90	92.87

reports results on AID under four configurations spanning a $4\times$ range in budget.

Three trends emerge from Table 10. First, performance scales consistently with search scope: reducing k, v, and the number of answer options to one-quarter of the default ($k = 5$, $v = 32$, 2 options) drops All-accuracy to 86.66% (−4.88%), confirming that a sufficient candidate pool is necessary for CUQ to identify genuinely boundary-critical samples. Second, scaling up to half the default ($k = 10$, $v = 64$, 3 options, 90.42%) and then to the full default ($k = 20$, $v = 128$, 5 options, 91.54%) shows diminishing returns (+3.76% then +1.12%), indicating that the default configuration sits near the knee of the cost–accuracy curve and offers a practical trade-off. Third, doubling the re-mining repetitions to 10 yields a further +2.35% All gain (93.89%), with the Novel-class accuracy improving most (+3.73%), suggesting that more frequent VLM feedback accelerates convergence on hard novel categories. In practice, the default ($k = 20$, $v = 128$, 5 options, 5 reps) is recommended; doubling repetitions is suitable when computational budget permits.

We further probe sensitivity to the two VLM-guided loss weights: the neighbor contrastive weight ${\lambda }_1$ (i.e., DCA’s ${\mathcal{L}}_{\mathrm{ncl}}$ in the projector space) and the cluster-instance alignment weight ${\lambda }_2$ (i.e., ${\mathcal{L}}_{\mathrm{ci}}$ in the backbone space). We independently sweep each weight while holding the other at its default, using multiplicative spacing to capture sensitivity across a logarithmic range. Every sweep includes a zero-ablation point to isolate the contribution of the corresponding component. The default configuration (${\lambda }_1=0.3$, ${\lambda }_2=0.05$) serves as the shared anchor.

Table 11. Loss weight sensitivity on AID. The default (${\lambda }_1=0.3$, ${\lambda }_2=0.05$) is used in all other experiments.

${\mathit{\lambda }}_1$	${\mathit{\lambda }}_2$	All	Old	New
0.0	0.05	87.21	94.99	83.26
0.1	0.05	90.89	95.59	88.52
0.3	0.05	91.54	96.29	89.14
0.6	0.05	86.90	94.68	82.96
0.3	0.0	88.03	92.08	85.98
0.3	0.1	90.88	95.51	88.54

reveals several patterns. Sweeping ${\lambda }_1$ with ${\lambda }_2$ fixed at 0.05 shows that the neighbor contrastive signal is essential: disabling it entirely (${\lambda }_1=0.0$) costs 4.33% All-accuracy, confirming that DCA-guided VLM neighbor alignment carries substantial discriminative information. Performance rises sharply from 0.0 to 0.1 and plateaus at 0.3, with the default yielding the optimal accuracy. Over-weighting to ${\lambda }_1=0.6$, with ${\lambda }_2=0.5$ as a combined stress, degrades accuracy to 86.90%, below even the zero-ablation point for ${\lambda }_1$, indicating that over-emphasizing VLM signals drowns out the base GCD learning objectives.

Sweeping ${\lambda }_2$ with ${\lambda }_1$ fixed at 0.3 shows a different sensitivity profile. Disabling cluster-instance alignment entirely (${\lambda }_2=0.0$) costs 3.51% All-accuracy, but the Old-class accuracy drops most sharply (96.29%→92.08%, −4.21%). This confirms that ${\mathcal{L}}_{\mathrm{ci}}$ predominantly stabilizes known-class boundaries: without it, the VLM-selected positive cluster signal is absent, and known-class samples, whose supervised anchors are already sparse, lose their structural alignment with class prototypes. Doubling ${\lambda }_2$ to 0.1 yields a mild 0.66% degradation, indicating that ${\mathcal{L}}_{\mathrm{ci}}$ is relatively tolerant of moderate over-weighting. Taken together, the results confirm the robustness of the nominal configuration (${\lambda }_1=0.3$, ${\lambda }_2=0.05$): both components are necessary, neither dominates, and the framework tolerates moderate deviations from the default without catastrophic degradation.

4.9. Qualitative Analysis

To intuitively verify the representational capabilities and geometric integrity of our framework, we visualize the t-SNE embeddings of the backbone features on the RSSDIVCS dataset in Figure 5. As observed in Figure 5a, the baseline method MVTopo forms loosely separated clusters, where visually confusable classes exhibit blurred and overlapping boundaries. When the naive GLEAN strategy is applied directly within the backbone space (Figure 5b), the powerful yet locally oriented contrastive signals force pairwise alignments that severely fragment the global cluster structure, resulting in scattered sub-clusters and destroyed manifold topology. This visual evidence directly corroborates our geometric conflict hypothesis. In contrast, DualGLEAN (Figure 5c) achieves clean, compact, and well-separated clusters across both known and novel categories. The stark difference between (b) and (c) confirms that our decoupled contrastive alignment (DCA) successfully prevents manifold fragmentation in the backbone space, while CUQ delivers precise semantic corrections that sharpen the boundaries of ambiguous clusters.

To deeply understand the internal mechanism and the targeting precision of the compound uncertainty querying (CUQ) module, we visualize the dynamic evolution of the selected VLM query candidates on the AID dataset. As depicted in Figure 6, the red circles highlight the specific samples filtered by CUQ for external semantic querying at epochs 15, 25, and 35.

The visualization reveals a clear trajectory of how the framework allocates its VLM budget during the feature learning process. Early in the training (Figure 6a, epoch 15), the feature manifold exhibits high overlap among visually similar aerial scenes, and CUQ accurately concentrates the queries within these highly confused, interlocking regions. As the network is continuously optimized and the clusters become more distinctly separated (Figure 6b,c), the selected query samples do not scatter randomly; instead, they strictly trace the shrinking decision boundaries.

By epoch 35, the red circles are distributed exclusively along the extreme fringes of the refined clusters. They bypass both the globally confident samples (cluster centers) and the uninformative outliers. This dynamic tracking visually substantiates that the CUQ mechanism achieves its design objective effectively: it successfully filters out false positive uncertain samples and consistently allocates the limited VLM query budget to genuinely boundary-critical regions, thereby maximizing the semantic correction efficiency of the framework.

Collectively, these two visualizations provide direct evidence of the advantages of DualGLEAN. Figure 5 shows clean, compact, well-separated clusters for both known and novel categories, while Figure 6 reveals that this advantage originates from CUQ’s precise allocation of the VLM budget to boundary-critical samples.

5. Discussion

Our results collectively establish that VLM-in-the-loop training, when properly configured through the dual allocation framework, consistently improves GCD performance in remote sensing across all evaluated settings. The key finding is not merely that VLMs provide useful semantic guidance; rather, it is that the manner in which this guidance is integrated determines whether it helps or harms. The naive transfer experiment demonstrates this starkly: the same VLM judgments that are actively counterproductive under naive integration become strongly beneficial under DCA + CUQ.

The dual allocation perspective also explains the asymmetric degradation pattern observed under naive integration. Novel-class clusters, which lack supervised anchors, are disproportionately fragmented by the InfoNCE gradient because no countervailing supervised signal stabilizes their structure. Known classes are partially buffered by their limited labeled samples, but this protection is dataset-dependent, as seen in the Old-class drop on RSSDIVCS. This asymmetry validates the coupling argument: space allocation and budget allocation must be resolved jointly because their failures interact.

The VLM robustness results carry practical significance for the remote sensing community, where the adoption of foundation models is accelerating [30]. The consistent within-family stability of the Qwen series, and the moderate but acceptable performance of non-Qwen VLMs, confirms that the framework is deployable across diverse VLM providers. The modest total API cost (CNY 2.45 on AID under the default search-scope setting, scaling linearly with query budget) eliminates the concern that VLM-in-the-loop methods are prohibitively expensive, and the zero-parameter overhead ensures compatibility with existing model deployments.

A related practical question concerns the known limitations of VLMs—comparatively weak spatial reasoning, a tendency to hallucinate non-existent visual details, and the reduced resolution at which large remote sensing images are processed. Our design mitigates these artifacts in three ways. First, GCD is a scene-level classification task that does not require fine-grained spatial localization; the VLM’s comparative judgment relies on global scene semantics, and the $224\times 224$ input retains the macro-level cues that distinguish scene categories. Second, we query the VLM for a closed-set comparative choice among candidate neighbors rather than open-ended generation, which is substantially less prone to hallucination. Third, the influence of any imperfect VLM judgment is deliberately localized: CUQ restricts queries to a small set of boundary-critical samples, and DCA routes the VLM signal through the MLP projection head, attenuating harmful gradients before they reach the backbone; the robustness across seven VLMs of varying quality (Section 4.5.2) empirically confirms that the framework does not hinge on the absence of VLM errors.

A further consideration concerns the static weighting of the CUQ components. The compound metric $U_i={\mathcal{H}}_i+\alpha B_i+\beta L_i$ uses fixed coefficients ($\alpha =0.5$, $\beta =0.3$) throughout training, whereas the relative informativeness of predictive entropy, boundary proximity, and local label inconsistency may drift as the feature manifold reorganizes (Figure 6). Empirically, the fixed weights prove robust: the component ablation (Section 4.4.2) confirms that the full three-component metric outperforms every subset, and the loss-weight sensitivity analysis (Section 4.8) shows that the framework tolerates moderate deviations from the nominal configuration. Nonetheless, adapting the weights over time—for instance, annealing the entropy contribution as clusters tighten, in the spirit of curriculum learning [44]—is a promising direction for further improving the allocation of the VLM budget.

6. Conclusions

This paper addressed the challenge of integrating vision-language models as external knowledge sources for remote sensing generalized category discovery. We identified that effective VLM-in-the-loop training is fundamentally a dual resource allocation problem, comprising space allocation, which determines where the VLM signal should operate, and budget allocation, which determines which samples deserve costly VLM queries, and showed that these two dimensions are coupled. To resolve this, we proposed decoupled contrastive alignment (DCA), which separates the VLM-guided neighbor contrastive loss into a projector space via the existing MLP head without additional trainable local parameters, and compound uncertainty querying (CUQ), which applies three-stage filtering to concentrate VLM queries on truly boundary-critical samples.

Experiments on AID and RSSDIVCS demonstrate consistent accuracy improvements over the compared GCD methods, with All-accuracy gains of up to 5.43% over the evaluated baselines. DualGLEAN generalizes as a plug-in module: integrating it with four distinct GCD baselines yields gains of 4.47–6.32% All-accuracy on AID, with zero additional trainable parameters and negligible FLOPs overhead. Robustness experiments across seven VLM backbones spanning four providers confirm that the dual allocation mechanism, rather than the choice of any particular VLM, drives the performance gain. Under the default search-scope configuration on AID, the total VLM API cost per full training run is approximately CNY 2.45 (scaling linearly with the query budget and number of re-mining cycles), demonstrating that high-cost external oracles can be integrated into GCD training with minimal computational expense. The plug-in generality across both GCD baselines and VLM backbones further validates that this cost-efficiency transfers across diverse experimental settings.

A current limitation is that the VLM query pipeline relies on cloud-based API calls, introducing network latency that lengthens training wall-clock time. Local VLM deployment or asynchronous query scheduling would mitigate this. Our framework also assumes the total number of categories $|{\mathcal{C}}_N|$ is known a priori; extending the compound uncertainty metric to simultaneously estimate the category count and the query set is an important direction for fully unsupervised deployment. Finally, the current work uses a generic VLM without remote-sensing-specific fine-tuning; combining our allocation framework with a domain-adapted VLM such as RemoteCLIP may further narrow the gap between the Qwen and non-Qwen performance tiers observed in our VLM robustness study.