3.1. Motivation
Because of the continuous developments in the field of data stream mining, many algorithms have been proposed in recent years for learning from data streams. However, the issues of multiclass imbalance and concept drift within data streams have not been widely studied, especially when these two phenomena occur simultaneously. Additionally, most algorithms addressing such issues assume that labels for all instances in the data stream are immediately available (i.e., supervised learning algorithms), overlooking the problem of labels potentially not arriving in a timely manner. To address the additional challenge of limited labels, researchers have turned their attention to active learning [
28,
29,
30] and learning from data streams under limited label budgets. Existing active learning solutions involve selecting a finite number of instances for labeling within a fixed budget, which can be considered as uniform allocation of the budget across the data stream. However, in scenarios with concept drift, classifiers need to swiftly adjust during drift periods to adapt to new concepts. However, under a fixed budget, it is not possible to ensure that instances of new concepts are learned in a timely manner. Therefore, this paper proposes a dynamic label budget adjustment strategy for scenarios with concept drift, aiming to learn from data streams containing concept drift, thereby reducing the impact of concept drift on classifiers and accelerating recovery.
The key to active learning is to find the most important instances for labeling and to explore the instance space as much as possible. In the case of a limited label budget, developing a well-designed label request strategy is pivotal for enhancing the performance of active learning. Most active learning methods use a mixed strategy, where the core of the mixed strategy is the uncertainty strategy. The uncertainty strategy aims to select instances that are most uncertain or most representative in predictions, typically those near the decision boundary. Existing methods mostly determine whether to request labels by comparing the uncertainty measures of instance predictions with a threshold. However, most of them consider only one measure, namely, the difference between the probability of the most predicted class and the second most predicted class, which is not comprehensive for identifying representative instances. Additionally, they use original thresholds in comparisons, thus failing to emphasize the importance of new instances at concept drift points. Therefore, this paper proposes a hybrid label request strategy, including an uncertainty strategy based on a dual-margin dynamic threshold matrix and a random strategy. The uncertainty strategy combines two predictive uncertainty measures, complementing each other to better identify instances with high information content in the data stream. This increases the uncertainty threshold under a dynamic labeling budget, favoring a higher selection probability for instances during concept drift and for minority class instances.
Learning from multiclass imbalanced data streams is also a major challenge, as minority classes are difficult to classify correctly because of their insufficient representation. The most common approach to address such issues is to combine resampling with online ensembles, but previous methods did not consider label budgets during oversampling. Theoretically, with lower budgets, fewer instance labels are accessible in the data stream; hence, the oversampling factor should be higher. Additionally, existing methods consider only class proportions when performing oversampling, meaning higher levels of oversampling are applied only to minority class instances. They overlook the classifier’s own classification performance, even though the classifier may have poor performance on majority class instances. Therefore, this paper proposes an instance-adaptive sampling strategy based on label budgets, which comprehensively considers class proportions and classification performance to determine the sampling rate of instances. Moreover, since each base classifier in the ensemble exhibits a different classification performance for different classes in the data stream, this strategy also ensures that the ensemble has higher diversity.
3.2. Dynamic Label Budget Strategy in Concept Drift Scenarios
When concept drift will occur in a data stream is uncertain, but a concept drift detector can effectively help identify it. It monitors the performance of base classifiers to infer changes in the data distribution and issues an alert when the performance of a base classifier is significantly affected, indicating potential concept drift. There are various choices for drift detectors, such as the drift detection method (DDM) [
32] and adaptive windowing (ADWIN) [
8]. In this paper, ADWIN was used as the drift detector, with two different thresholds set for ADWIN: one serves as a drift alarm to warn of impending concept drift, and the other acts as a drift confirmer to verify that concept drift has occurred in the data stream. Additionally, ADWIN is used as a trigger for ensemble updates. When ADWIN detects a change, a new classifier is trained to replace the worst-performing classifier in the ensemble. The initialization of the new classifier is performed on a sliding window that holds the most recent instances.
This paper proposes a dynamic label budget strategy based on three stages (stable, warning, and confirmation). During the stable stage of the data stream, the label budget is consistently maintained at the default setting B. Subsequently, paired ADWINs running on each base classifier detect concept drift in the data stream. If at least one classifier reports a warning or confirms a drift, corresponding budget adjustments are made. The adjustment of the current label budget of the data stream is shown in Equation (1). When a warning is detected, the budget temporarily increases to 1.5B for
sizew instances following the warning flag
twarning. If no drift is detected afterward, the budget reverts to the default setting. When a drift is confirmed, the budget increases further to 2B and remains at this level for the subsequent
sized instances to learn more instances and adapt to the new concept. Finally, the budget reverts to the default setting.
This strategy allows the algorithm to temporarily increase the number of labeled instances during drift and then return to the original level, indirectly reducing the number of labeled instances during the stable phase. By flexibly allocating the labeling budget in the context of concept drift, the impact of concept drift on the classifier is reduced, and the classifier quickly adapts to new concepts. In this paper, the default settings are sizew = 50 and sized = 100.
3.3. Uncertainty Strategy Based on Dual-Margin Dynamic Threshold Matrix
This section introduces the proposed hybrid label request strategy, including uncertainty strategy and random strategy. The uncertainty strategy is driven by a dual-margin dynamic threshold matrix to select instances with high information content from the data stream. The random strategy marks instances from the data stream without selection, ensuring that this subset of instances reflects the class distribution of the data stream. It is noteworthy that most multiclass imbalanced active learning algorithms use a label sliding window to estimate class distribution [
13], with the imbalance ratio for each class calculated as shown in Equation (2). Here,
labelnumy represents the count of instances labeled as
y within the sliding window,
sizelab denotes the size of the window,
numNull denotes the number of instances selected by the uncertainty strategy (used as placeholders), and
C is the number of classes. Although this method can approximate the class distribution in the data stream, its accuracy heavily depends on the size of the label sliding window. A window that is too small may lack sufficient instances to represent the distribution, while a window that is too large may include outdated concepts, both of which can affect the accurate estimation of the current class distribution.
To more accurately estimate the class imbalance state of the data stream, this paper employs a time decay model to simulate class proportion changes. This model persists throughout the entire learning process of the data stream and is updated only by labeled instances selected by the random strategy, ensuring it reflects the true distribution of the data stream. As shown in Equation (3), the initial distribution of each class
cd[
i] is set to 1/
C,
. Here,
represents the time decay factor, used to control the rate at which old instances are forgotten. For each new instance selected by the random strategy, all class distributions are updated based on the requested label
y, increasing the corresponding
cd[
y], while simultaneously decaying the other classes. This strategy can effectively estimate the overall distribution of the data stream and reduce the impact of outdated concepts. Based on this, a new imbalance ratio calculation formula is introduced, as shown in Equation (4), and used in subsequent steps.
The proposed label request strategy begins with the random strategy. If the conditions for the random strategy are met, the true labels of instances are requested; otherwise, the uncertainty strategy is employed. In the uncertainty strategy, two measures of predictive uncertainty, namely, minimum marginal uncertainty (
marginmin) and maximum marginal uncertainty (
marginmax), are combined.
marginmin is calculated as the difference between the probability of the most likely class and the probability of the second most likely class, while
marginmax is calculated as the difference between the probability of the most likely class and the probability of the least likely class. These measures serve to indicate the certainty or uncertainty of the classifier’s prediction for an instance, with a significantly larger probability for the most likely class compared to the second most likely or least likely classes indicating certainty and vice versa. Therefore, active learning selects instances with smaller
marginmin or
marginmax for requesting and learning. Based on these two measures, dual-margin dynamic threshold matrices
and
are proposed, both of which are matrices with dimensions of
. Initially, each element of
is set to
, and each element of
is set to
. During the label request process, the marginal values
marginmin and
marginmax of instances are compared with the corresponding thresholds in
and
, respectively, to select instances with greater information content. Furthermore, to emphasize the learning rate of instances during concept drift, the comparison process considers not only the original elements of
and
but also the label budget. The dynamic thresholds used for comparison in
and
are calculated as shown in Equation (5). During the stable phase of the data stream (i.e.,
budget = B), the original thresholds are used. When a drift warning or confirmation occurs in the data stream, the thresholds are raised to increase the learning opportunities for new concept instances.
Algorithm 1 shows the pseudocode of the proposed uncertainty strategy. For a new incoming instance,
x, the ensemble,
E, is first used for prediction (line 1), where
yc1 represents the class with the highest probability in the ensemble prediction,
yc2 is the second highest probability class, and
ymin is the least probable class. Then, the minimum margin,
marginmin(
x), and maximum margin,
marginmax(
x), of
x are calculated, and the dynamic thresholds
dm1 and
dm2, used for comparison, are obtained through Equation (5) (lines 2–4). If
marginmin(
x) is less than
dm1 or
marginmax(
x) is less than
dm2, this indicates that the ensemble cannot predict the class of
x with certainty. Therefore, the true label of
x is requested, and the corresponding elements of the dual-margin threshold matrix are adjusted based on the prediction result of
x. The adjustment process is as follows (lines 8–19).
Algorithm 1 Uncertainty Strategy |
Input: ensemble classifier E, processed instance x, current class distribution cd[C], minimum margin threshold matrix M1, and maximum margin threshold matrix M2. |
Output: // request label? |
Process: |
1. Make predictions on x by E. |
2. Calculate . |
3. Calculate . |
4. Calculate dynamic thresholds dm1 and dm2 using Equation (5). |
5. if (marginmin(x)≤ dm1 || marginmax(x)≤ dm2) then //meeting the uncertainty strategy |
6. labeling = true. |
7. Get real label y of x and calculate imby using Equation (4). |
8. if (y==yc1) then |
9. M1[yc1][yc2] = M1[yc1][yc2] × (1 − β × marginmin(x)). |
10. M2[yc1][ymin] = M2[yc1][ymin] × (1 − β × marginmax(x)). |
11. if (imby > 1) then |
12. M1[yc1][yc2] = M1[yc1][ yc2]×(1 – β × marginmin(x)). |
13. M2[yc1][ymin] = M2[yc1][ ymin]×(1 – β × marginmax(x)). |
14. else if (y==yc2 && imby > 1) then |
15. M1[yc1][yc2] = M1[yc1][ yc2] × (1 – β × marginmin(x)). |
16. M2[yc1][ymin] = M2[yc1][ ymin] × (1 – β × marginmax(x)). |
17. else |
18. M1[yc1][yc2] = M1[yc1][ yc2]×(1 + β×marginmin(x)). |
19. M2[yc1][ymin] = M2[yc1][ ymin]×(1 + β×marginmax(x)). |
20. end if |
- (1)
If y = yc1, indicating that the ensemble prediction is correct, the classification difficulty of x is not high at this time. Therefore, the corresponding elements of the dual-margin threshold matrices M1 and M2 should be reduced to decrease the frequency of labeling requests for such instances, thereby leaving learning opportunities for other challenging instances. Additionally, if x belongs to the majority class (i.e., imby > 1), the threshold should be further decreased. This approach aims to reduce the learning opportunities for majority class instances, thereby enhancing the importance of minority classes.
- (2)
Otherwise, when y = yc2 and imby > 1, the threshold is only lowered once. This means that the ensemble’s prediction is close to accurate, and lowering the threshold is also to further reduce the learning opportunities for majority class instances. It should be noted that although the adjustment conditions for M1 and M2 are consistent, the magnitudes of the adjustment are different depending on the corresponding marginmin(x) or marginmax(x), where β is a tuning factor between 0 and 1.
- (3)
Finally, if y ≠ yc1 and y ≠ yc2, this indicates that the ensemble’s prediction result is very poor, and this class instance is difficult to classify by the current ensemble. Therefore, the corresponding threshold should be increased to ensure that this class instance has more learning opportunities.
In summary, the core of the active learning algorithm proposed in this paper is the uncertainty strategy leveraging the dual-margin dynamic threshold matrix, which is used to select instances with the maximum information and minority class instances for labeling from the data stream, and is used for ensemble learning. At the same time, combined with the random strategy, instances selected by the random strategy can well characterize the current class distribution of the data stream, which complements and improves the uncertainty strategy.
3.4. Instance-Adaptive Sampling and Weighted Ensemble Strategy
The performance of ensemble classifiers is closely related to the distribution of instances and the diversity among base classifiers, especially in multiclass imbalanced data streams with limited label budgets. To address the issue of insufficient labeled instances, it is necessary to oversample labeled instances and increase the learning rate of minority class instances. Additionally, as the classification performance of base classifiers varies within the ensemble, different levels of oversampling should be applied to the same instance, ensuring a higher diversity within the ensemble. The algorithm proposed in this paper learns from multiclass imbalanced data streams under limited label budgets, where the sampling frequency of instances is determined based on class proportions and classification performance. This paper defines classification performance as the real-time recall of base classifiers, which effectively reflects the performance of classifiers in imbalanced data streams. The formula for calculating the recall is shown in Equation (6).
Firstly, given a new instance (
x,
y), the adaptive sampling weights of each base classifier in the ensemble for this instance are as shown in Equation (7), where
budget represents the current label budget,
y denotes the class of the current instance,
cd is the previously calculated class distribution,
E is the current ensemble, and
j represents the index of the base classifier. By using the instance weight formula inversely proportional to the label budget, the problem of insufficient labeled instances can be effectively addressed. Under different cases, the lower the recall rate,
RecallE[
j], of the classifier, the smaller the class weight,
cd[
y], of the instance and the higher the corresponding instance sampling weight,
SWj. Therefore, during the training process, difficult classification instances and minority class instances can receive sufficient attention. During the sampling process, the algorithm follows the idea of online bagging and implements online resampling using the Poisson distribution, where
k~
Poisson (
SWj) yields the oversampling frequency of each base classifier for the current instance.
In the weighted ensemble stage, because of the varying performance of each base classifier in the ensemble, it is necessary to weight the predictions of the base classifiers. This paper uses the combined results of the base classifier recall rate and kappa coefficient for classifier weighting. The kappa coefficient is also a commonly used performance metric in imbalanced data streams, which evaluates classifier performance based on the consistency of predicted results and data statistical distributions. The weight calculation of base classifier
j in the weighted prediction stage is shown in Equation (8). It is noteworthy that the recall rate adopts the single-class recall rate on the current training instance, so the better the classification performance of the base classifier on the class to which the instance belongs, the higher its weight. By comprehensively considering these two metrics, weights can be flexibly allocated among the base classifiers in the ensemble to achieve good classification results.
3.5. Overall Framework and Process of the AdaAL-MID
This section presents the framework and workflow of the proposed adaptive active learning algorithm. The algorithm framework, as shown in
Figure 2, consists of an ensemble classifier, an ADWIN detector for drift detection, and an instance sliding window for ensemble updating. The working steps of the AdaAL-MID are as follows:
- (1)
For each new incoming instance, the ensemble classifier first predicts it and outputs the prediction result.
- (2)
Enter the mixed label request strategy; if the label request strategy is met and the label budget is sufficient, then request the true label. Otherwise, if the label request strategy is not met or the label budget is insufficient, proceed to the next instance.
- (3)
Combine the true label with the instance to form a new training instance, and store the new training instance into the corresponding sliding window according to the label. Update the ensemble classifier online through the adaptive sampling strategy and the weighted ensemble strategy.
- (4)
If the drift detector ADWIN detects a change, create a new base classifier, and sort the instances in the instance sliding window by time to form an instance sequence for initializing the new classifier. Finally, replace the worst classifier in the ensemble.
- (5)
Adjust the label budget to the corresponding value based on the detected drift flag.
Algorithm 2 presents the pseudocode for the learning process of the AdaAL-MID algorithm. In the initial learning phase of the data stream, the ensemble classifier is initialized by directly requesting the true labels of instances, with the initial phase consisting of
sizeW instances (a sliding window size). After initialization, the decision to request label for subsequent instances is determined by the mixed-label request strategy. First, a random value
is generated through the random strategy. If this value is less than the random selection rate R, the random strategy is satisfied, and the true label
y of the instance is requested. The current class distribution
cd is updated based on its class (lines 5–8). If the random strategy does not satisfy the condition, the uncertainty selection strategy activates, and algorithm 1 determines whether to request a label (lines 9–10). If the label request strategy is satisfied and there is a surplus in the current label budget (i.e., (
l/
p) <
b), the true label of the instance is requested, the number of marked instances
l is increased by one, and the training phase is entered (lines 11–13). First, according to the class to which the instance belongs,
y, the instance is stored in the corresponding class’s sliding window
Win[
y] (line 14). Next, the corresponding adaptive sampling strategies and weighted ensemble strategies are executed for each base classifier in the ensemble, considering the class proportions and classifier performance comprehensively to determine the instance sampling weight,
SWj, and obtaining the online resampling count through a Poisson distribution
. Then, the weight,
CWj, of the base classifier is obtained through a weighted ensemble strategy combining the recall rate and kappa coefficient, and the ensemble classifier is updated online based on this weight (lines 15–20). Finally, ADWIN is used to detect potential concept drifts in the data stream. If a drift flag is successfully triggered, a new classifier,
Cnew, is retrained on the most recent instances in the sliding window, replacing the worst classifier in the ensemble. The label budget is dynamically adjusted according to the strategy proposed in
Section 3.2 under this drift flag. This process is repeated until learning is complete (lines 21–25).
Algorithm 2 AdaAL-MID |
Input: data stream, S; number of classifiers, N; size of instance sliding window, sizeW; initial value of M1, θ1; initial value of M2, θ2; labeling budget, b. |
Symbols: ensemble classifiers, E; number of processed instances, p; number of labeled instances, l; minimum margin threshold matrix, M1[C][C]; maximum margin threshold matrix, M2[C][C]; random selection ratio, R; instance sliding window, Win[C][sizeW/C]; and current class distribution, cd[C]. |
Output: final ensemble classifier, E, and prediction result. |
Process: |
1. While (S.hasNext()) do |
2. x = S.nextInstance() |
3. p = p + 1. // processed instances plus one. |
4. labeling = false. |
5. // random strategy |
6. if (p < sizeW or δ < R) then 7. labeling = true. |
8. update the current class distribution cd based on Equation (3) |
9. else if (UncertaintyStrategy(E, x, cd, M1, M2)) // uncertainty strategy |
10. labeling = true. 11. if (labeling and (l/p) < b) // surplus in the label budget |
12. y = x.classvalue(). // request the real label y of instance x |
13. l = l + 1. //number of instances labeled plus one |
14. // store instances (x, y) in the corresponding instance sliding window Win[y] based on label y |
15. do // adaptive sampling strategy |
16. calculate the sampling weight SWj on instance (x, y) using Equation (7). 17. |
18. calculate the classifier weight CWj on instance (x, y) using Equation (8). |
19.
do |
20. update classifier j on instance (x, y) with the weights of the classifier CWj. |
21. if ADWIN detect changes |
22. create a new classifier Cnew and initialize it on Win[C][sizeW/C]. |
23. replace the worst classifier in ensemble E with Cnew |
24. change the current label budget b using Equation (1). // dynamically adjust label budget based on drift level |
25. end while |