Low-Resource Malware Family Detection by Cross-Family Knowledge Transfer

Abstract

Low-resource malware families are highly susceptible to being overlooked when using machine learning models or deep learning models for automated detection because of the small amount of data samples. When we target to train a classifier for a low-resource malware family, the training data using the family itself is not sufficient to train a good classifier. In this work, we study the relationship between different malware families and improve the performance of the malware detection model based on machine learning method in low-resource malware family detection. First, we propose an empirical supportive score to measure the transfer quality and find that transferring performance varies a lot between different malware families. Second, we propose a Sequential Family Selection (SFS) algorithm to select multiple families as the training data. With SFS, we only transfer knowledge from several supportive families to target low-resource families. We conduct experiments on 16 families and 4 malware detection models, the results show that our model could outperform best baselines by 2.29% on average and our algorithm achieves 14.16% improvement in accuracy at the highest. Third, we study the transferred knowledge and find that our algorithm could capture the common characteristics between different malware families by proposing a supportive score and achieve good detection performance in the low-resource malware family. Our algorithm could also be applicable to image detection and signal detection.

1. Introduction

Android is an open-source system framework. It has become one of the most popular mobile ecosystems. With the popularity of mobile devices in daily life increasing every year, more researchers are paying attention to the security of the Android ecosystem [1]. Various malware detection approaches have been raised in our community.

Malware attempts to control the user’s use system without authorization, steal personal information, encrypt important electronic files or cause other damages. A malware family is composed of malware samples with common characteristics. Common characteristics usually include the same code segment, pattern, application characteristics and similar behavior. The number of each malware family varies from just a few to tens of thousands. Low resource malware family refers to a family with less data, and its data are not enough to train the malware detection model alone. Although low-resource malware families have less data volume, these malware families can still bring great software security risks. When we target to train a classifier for a low-resource malware family, the training data using the family itself are not sufficient to train a good classifier. Androzoo [2] currently contains 17,927,200 different APKs with hundreds of malware families. However, some malware families have less than 100 or 500 apks, which is not enough to train a good malware classifier. For the unpopular malware, we may never find enough samples even if we label all the existing apks. At the same time, it is hard for us to have sufficient labels in time with the continuous evolution of malware.

There are some widely used malware data sets, in which a large number of malware families have only a few data samples. For example, MalGenome [3] dataset covers 49 families, and each family contains 1 to 309 malware samples. The top three families occupy roughly 70% of the overall dataset, while over 30 families have less than 10 samples. The distribution suggested that, as long as the detection approach can successfully detect the top families, the overall result will be good enough. The malware families with few samples are ignored. If we directly use all the training datasets from other malware families, most of the malware detection models may not be robust enough to transfer between different malware families. To detect low-resource malware families, Tran et al. first used prototype learning to create prototype representations for the target malware family, and used the twin network to classify malware [4]. Subsequent work further improved the generation of prototype representations [5], or improved the training of twin networks using meta-learning [6,7,8] and contrastive loss functions [9]. Kamaci et al. [10] established novel distance concepts to measure the relative difference between two objects. Alsboui [11] proposed a graph-based dynamic multi-mobile agent itinerary planning method to cover all nodes in the network. However, these methods use only a small number of samples related to the target low-resource malware family for model training, ignoring the relationship between the target malware family and a large number of existing malware families. In this paper, we seek to study the relatedness of malware families and leverage the relatedness to improve the malware detection performance of low-resource families. Our work focuses on three research questions:

First, does use different malware families as training datasets could help the detection of target low-resource malware? Intuitively, training with similar malware tends to achieve better transferring to the low-resource malware family and dissimilar malware may even harm the performance. We propose to measure the similarity with empirical experiments. Specifically, we train a malware detection model with one family $m_{train}$ and test on the target family $m_{test}$, and define the test performance as the supportive score from $m_{train}$ to $m_{test}$. Our work shows that the transferring performance varies a lot between different malware families, and we could achieve good performance by selecting the family with the biggest supportive score.

Second, we further study whether it is more helpful to use multiple malware families as the training set? We found that if we neglect the differences between distinct malware families and train the model with all families in the possible training data, the malware detection performance may even be worse than only selecting one most supportive malware family. We propose a Sequential Family Selection (SFS) algorithm to carefully select multiple families as the training set. Our algorithm could be easily adapted to any detection model. We conduct experiments to validate its performance and test on 16 malware families and four representative detection models csbd, drebin, mamadroid, and droidsieve. Our results show that SFS improves the performance of all the malware detection models. We also evaluate the performance on datasets from future time and SFS still achieves better performances.

Third, we try to understand why the supportive score between some malware families is higher which means having better transferring performance. We hypothesize that this is because of the similar characteristics between different malware families. We study two popular characteristics about whether malware steals user data and whether it displays advertisements. We found that malware with the same characteristics tends to have high supportive scores. Most supportive relations are the same for different malware detection algorithms, while some are varied for different detection models. Our work makes three contributions:

• We make the first systematic study of the relatedness between malware families. We propose to measure the malware family similarity with an empirical supportive score and find it is the key to good transfer performance.
• We propose a new Sequential Family Selection algorithm to target the low-resource families and validate it on 16 families and 4 different malware models. Our results show that the combination of our algorithm and the malware detection model based on machine learning and deep learning method can greatly improve the performance of malware detection.
• We study the relationship between the performance of knowledge transfer and the characteristics of malware. We found that the success of knowledge transfer is essentially due to the similar behavior characteristics between different malware families.

2. Related Work

In this section, we first overview the common Android malware detection methods. Then, we discuss low-resource malware detection.

2.1. Android Malware Detection Based on Machine Learning

Many researchers have studied various malware detection methods in our daily lives. Malware detection methods are emerging rapidly, such as methods based on static features [12,13,14,15,16], methods based on dynamic features, etc. [17,18,19,20,21].Static features of applications include API calls, permissions, opcode, etc. These are extracted by analyzing the structure of applications. Dynamic features such as system calls, behavior characteristics, network traffic, etc. [22].These features are extracted during the period when the application is running. Mudflow [23] uses the flows between APIs as the malware features to detect malware. Deep4maldroid [24] leverages the constructed graph to train malware detection models. DroidAPIMiner [25] provides a lightweight malware classifier by conducting a thorough analysis of apks at the API level. However, these Android malware detection methods are more concerned with the performance of the algorithm on the overall dataset and ignore the malware detection performance in the low-resource malware family.

2.2. Low-Resource Malware Detection Based on Machine Learning

To enhance the detection of low-resource malware families, several researchers have improved models to enhance the detection of low-resource malware families. In 2019, Tran et al. used prototype learning to create prototype representations for target malware families and used twin networks for malware classification [4]. The improvement of the subsequent work is divided into two main directions; first, to further improve the generation of prototype representations, Chai et al. used dynamic prototype networks to generate prototype representations [5] and Tang et al. used multilayer convolutional neural networks to generate prototype representations [6]. Second, to better train the network, Bai et al. used a contrast loss function to better train the twin network [9]. In addition, Tran et al. used meta-learning to train memory neural networks for malware family classification [26]. However, all these methods use only samples of the target malware family for model training and prediction, but ignore data samples of malware families related to the target malware family.

To address the problem of low-resource malware detection, some other researchers have increased the data samples of low-resource malware families by generating new data [27,28,29]. Zahra et al. used generative adversarial networks to increase the data samples of low-resource malware families by generating new sample signatures of malware  [30]. Chen et al. proposed a malware detection model called Adv4Mal, which generates new data based on specific signatures of malware to supplement the training data of the low-resource malware family [31]. These methods use artificially constructed data, while this paper uses real data related to the target malware family.

Table 1. Low-resource malware detection based on machine learning.

Type	Technique	Author (Published Year)	Disadvantage
Improvement of the generation of prototype representations.	They used dynamic prototype network to generate prototype representations.	Chai et al. (2022)  [5]	The data samples of malware families related to the target malware family are ignored.
	They used multilayer convolutional neural networks to generate prototype representations.	Tang et al. (2020)  [6]
Improvement of the training network.	They used a contrast loss function to better train the twin network.	Bai et al. (2020)  [9]	Samples of the target malware family were used only.
	They used meta-learning to train memory neural networks for malware family classification.	Tran et al. (2020)  [26]
Increase of data samples.	They used generative adversarial networks to generate new sample signatures of malware.	Zahra et al. (2021)  [30]	Artificially constructed data are used only.
	They proposed Adv4Mal to generate new data based on specific signatures of malware to supplement the training data of the low-resource malware family.	Chen et al. (2020)  [31]

shows the differences between these related work of low-resource malware detection based on machine learning method. This paper proposes sequential family selection algorithm that does not require the generation of any forged new data, but rather supplements the existing data with low-resource malware families. Meanwhile, the sequential family selection algorithm uses knowledge transferring among malware families to select relevant real data samples to improve the performance of low-resource malware family detection. Theoretically, the sequential family selection algorithm can be combined with the above malware detection methods and further improve its detection performance on low-resource malware families.

3. Methodology

In this section, we first introduce how to measure the similarity between different malware families. We could achieve good transferring performance on low-resource families by selecting a most similar malware family. Then we introduce a Sequential Family Selection (SFS) algorithm to select multiple families as training data and achieve better performance.

3.1. Malware Family Similarity

It is noticed that the malware detection performance differs significantly with different malware families in the training set when using the same malware detection algorithm. Moreover, the impact of different families differs while malware detection methods vary. It is interesting to explore the similarity between malware families. There are two general ways to measure the similarity between malware families, malware characteristics and empirical metrics.

Researchers obtain the characteristics of mobile applications through dynamic analysis and static analysis, including operation code, API calls, behavior characteristics et al. Malware families with similar characteristics are more likely to transfer knowledge to each other. The method of determining the similarity between malware families based on features is interpretable. However, it is hard to define all the characteristics of android applications, especially for the rarely studied low-resource malware families. Futhermore it is usually time-consuming to analyze the malware applications. We propose an empirical supportive score to measure the transfer quality. Specifically, we train a malware detection model with one family $m_{train}$ and test on the target family $m_{test}$, and define the test performance as the supportive score from $m_{train}$ to $m_{test}$. To achieve good performance on a given test set, we could select the family with the biggest supportive score as training data.

This quantitative relationship between malware families directly corresponds to the performance of malware detection performance and could help to improve the detection performance. We further explore the relationship between the characteristics of malware families and the supportive score. We find that the supportive score is highly correlated to the human summarized characteristics.

3.2. Sequential Family Selection (SFS) Algorithm

In this section, we first formalize the problem. Then we propose two baselines, the most supportive family only and the training with all the malware families. We further propose a new malware family selection algorithm to carefully select multiple families as the training set. Finally, we compare the performance of these four malware detection methods and validate our research questions. Formally, we target to test the performance of a target malware family $m_t$, which have a validation and test dataset. The training data include a set of malware families $S_{train}=\{m_1,m_2,\dots ,m_n\}$, and each malware family corresponds to a training dataset $D_i$, which contains malware from the corresponding family and randomly sampled benign. The benign samples of different groups of training data are not overlapping. We have two baseline settings for this problem. The first one is to train all the malware families in the training dataset and test on the target malware family. This is equivalent to neglecting the differences between different malware families. The second one is to find a training family which is the most supportive one to target the malware family. We empirically calculate the supportive malware family from the malware family $m_a$ to malware family $m_b$ by training on $m_a$ and test on $m_b$.

To achieve better performance, we propose a Sequential Family Selection algorithm (SFS). SFS’s target is to select from a subset of malware families in training data. SFS starts from an empty set and selects families one by one. In each step, we will try to combine each candidate with the selected families and evaluation on the target test set. We select the best family with the best performance, i.e., with the biggest supportive score, and add it to the selected set. Formally, we initialize the selected family dataset $S_{selected}$ as empty set before selection, and all the families in the training data are in the candidate set. In the first step, we train each malware family in the candidate set separately and evaluate the target malware family $m_t$. We add the malware family with the best performance into $S_{selected}$. We also will remove the selected family from the candidate set. Secondly, we combine $S_{selected}$ with the other malware families in the candidate set separately and evaluate on $m_t$. We select the malware family with the best combinations and add the malware into $S_{selected}$. In the following steps, we repeat iteratively try to combine the malware families in the candidate set with $S_{selected}$ and add the best family into $S_{selected}$. The algorithm terminates when we add all the families into $S_{selected}$. Finally, we choose the combination with the best performance in all $S_{selected}$ history and return it as our final selection. We further describe the whole algorithm in Algorithm 1. Our algorithm is independent of the malware detection model and could improve the performance of any model.

Algorithm 1 Sequential Family Selection algorithm (SFS).

Input: A malware classifier $C\left(\right)$. A target malware family $m_t$, includes a validation set $D_{valid}$ and a test set $D_{test}$. A set of training malware families $S_{train}=\{m_1,m_2,\dots ,m_n\}$, each $m_i$ corresponds to a set of training samples $D_i$. Output: A subset of $S_{train}$.     1: Suppose added training set $S_{selected}=\varphi$, best training set $S_{best}=\varphi$, best validation performance is $Scor{e}_{best}$=0.     2: while$S_{train}$ is not empty set do     3:     $best\_family\_in\_step=None$, $best\_score\_in\_step=0$     4:     for $m_i$ in $S_{train}$ do     5:         current training set $S_c$=$S_{added}$∪$\left\{m_i\right\}$     6:         train classifier with $S_c$, model $=C\left(S_c\right)$     7:         test on $D_{valid}$, $score=evaluation\left(model\left(D_{valid}\right)\right)$     8:         if $score>best\_score$ then     9:              $best\_score\_in\_step=score$     10:            $best\_family\_in\_step=m_i$     11:     $S_{selected}=S_{selected}\cup \{best\_family\_in\_step\}$     12:     $S_{train}=S_{train}-\{best\_family\_in\_step\}$     13:     if $best\_score>Scor{e}_{best}$ then     14:         $S_{best}=S_{selected}$     15:         $Scor{e}_{best}=best\_score$     16: return$S_{best}$

4. Experimental Setup

4.1. Malware Detection Approaches

Our method is designed to benefit malware detection by studying the relationship between the malware families. As for the malware detection approaches, we apply four popular malware detection methods csbd, drebin, mamadroid, and droidsieve.

Csbd [19] extracts the control flow graph as the features of malware detection. It was first proposed by Kevin from the University of Luxembourg. This method performs the static analysis on the application bytecode to extract the control flow graph, takes the basic blocks of the control flow graph as features of the application, and uses classification algorithms in machine learning to assign it. Drebin [20] uses multiple static analysis approaches to extract multiple features of the application from disassembled code and the information file as the features for Android malware detection. It is a lightweight detection method for Android malware. SVM algorithm is used to automate the classification of applications. Mamadroid [21] is a malware detection model based on application behavior. This method extracts and abstracts the sequence of calls between APIs in an application, constructs feature vectors based on Markov chain, and uses different classification algorithms in machine learning to assign applications. Droidsieve [22] utilizes the confusion invariant features and artifacts introduced by the confusion mechanism used in malware attacks to classify malware.

4.2. Malware Corpus

To prepare for our study on malware detection, we collect a set of Android applications from Androzoo, an open Android datasets collection project [2]. The Android apks from Androzoo were obtained from various app markets. As VirusTotal is broadly utilized for Android malware labeling, we use the scanned results of VirusTotal to resolve the labels of the collected Android apks. VirusTotal uses more than 70 anti-virus scanners and URL/domain block list services to check items [32]. The Android apk is labeled as benign software when no engines in VirusTotal marks it as positive. In order to ensure the reliability of the collected dataset, we label the apk as malware when at least five engines in VriusTotal label this apk as malicious. We use Euphony [33] to acquire more information about the malware family of each data in the collected apks. We collect a dataset containing over 20,000 malware with family-type information and 20,000 benign software, and the time span of the dataset is 2015 to 2016. Note that, we will construct different groups of training and test sets targeting diverse cases we have considered in this paper.

4.3. Testing Dataset

Many different malware families exist in our community. We pick 16 random families from the malware corpus for our study. To ensure the consistency of our experiments, the number of each malware family is 500. We then combine 500 benign software with each malware family to construct the test set. The collected malware families belong to different categories with different malicious characteristics.

Table 2. The detailed description of the selected malware family.

Malware Family	Description
Ginmaster	Ginmaster is a Trojanized and re-packaged application family distributed in Chinese thirty-party stores targeting Android mobile devices [1]. GinMaster steals confidential information from the device and sends it to a remote website [34].
Artemis	The Artemis Trojan is a Trojan infection that prevents computer users from using the infected computer effectively. If the Artemis Trojan is installed on a computer, the affected computer user will not be able to install or delete any new software [35].
Nandrobox	Nandrobox is a Trojan horse for Android devices that steals information from the compromised device [36]. NandroBox sends out SMS messages to a specific number, and then intercepts incoming messages from that number to cover its tracks [37].
Plankton	Plankton is a trojan that affects mobile devices running the Android operating system. It may arrive as part of repackaged Android apps and downloaded from third-party Android app markets. It changes the device’s settings and steals information stored in the device [38].
Droidkungfu	DroidKungFu is a malware that affects Android OS [38]. This malware is capable of rooting vulnerable Android phones and may successfully evade the detection from current mobile anti-virus software [39].
Gingermaster	GingerMaster is an Android malware that contains a root exploit packaged within an infected app [40]. GinegerMaster acts to be a normal application on the users’ phone, and once the application is launched on an Android device, it acquires root privileges through GingerBreak on the device and then accesses sensitive data [41].
Leadbolt	Leadbolt is a form of adware program that allows users to see numerous ads appearing on their computer screens. It is bundled with free or pirated software, and when users install such software, it enters their computer [42].
Umeng	Umeng is a known form of adware, a type of malicious software that runs on users’ computers to display unwanted ads without their consent. After installation, umeng starts displaying unwanted ads when you browse the Internet with your browser [43].
Mulad	Mulad is mainly used to generate income by injecting Adsense code into legitimate free apps that are then repacked and uploaded to third-party marketplaces. While they are not malicious in nature, these apps are hijacked and used by others to generate income [44].
Waps	Waps is classified as a file infector. It has the capability to propagate by attaching its code to other programs or files [45].
Domob	Domob is a type of malware that has the capability to propagate by attaching its code to other programs or files [46].
Mobwin	Mobwin is a type of potentially unwanted program and always pretends to be legitimate software that has the ability to help users remove PC threats and optimize system performance. Once installed on the device, Mobwin will scan users’ machines automatically and show you lots of system vulnerabilities, errors, and threats hidden in the laptop or computers [47].
Revmob	RevMob is a type of malicious software that runs on users’ computers to display unwanted ads without their consent. Revmob starts displaying unwanted ads when users browse the Internet with their browser [48].
Admogo	Admogo is an unwanted program module that is incorporated into Android applications and is responsible for advertising on mobile devices [49].
Wapsx	Wapsx includes apps that incorporate an advertising package. The incorporation of an advertising package that has the capability to display advertisements in the notification tray may be potentially unwanted [50].
Utchi	Utchi is a potentially unwanted malware family that comes bundled with genuine applications that are available free of cost. It automatically gets installed when the application it is bundled with is installed [51].

shows the name and the description of the selected malware family. Note that the malware family in the test set does not appear in the training dataset.

5. Results and Discussions

In this section, we illustrate the impact of different malware families in training set on the malware detection and obtain the supportive score of the 16 malware families.

5.1. Dataset Construction

We construct 16 malware benchmarks to be used as the 16 test sets. Each test set consists of 1000 apks with 500 malware and 500 benign. The dataset used in this section is from 2015. The test set of each experiment contains merely one malware family. To minimize the variability of the experimental results, we perform each set of experiments five times. We then use the average of the five results as the final outcome of each experiment.

5.2. Supportive Score of Malware Families

We further analyze the impact of malware detection performance when using distinct malware families for training.

Table 3. Malware detection performance with different malware families in the training set using csbd. The number in bolds show the best results.

	Test	ginmaster	artemis	nandrobox	plankton	droidkungfu	gingermaster	leadbolt	umeng	mulad	waps	domob	mobwin	revmob	admogo	wapsx	utchi
Train
ginmaster		–	69.1	95.4	57.9	92.9	82.1	55.8	68.6	92.2	85.9	80.2	90.3	51.7	96.0	90.0	66.1
artemis		63.1	–	95.0	52.5	84.4	69.4	51.7	73.9	84.3	72.9	73.6	81.7	59.9	96.5	73.2	63.0
nandrobox		51.2	56.3	–	50.1	50.2	53.8	50.4	50.4	52.2	54.3	50.9	52.7	53.4	50.2	52.2	49.9
plankton		50.2	52.3	49.9	–	49.7	51.7	50.6	50.1	50.4	53.2	50.0	52.2	53.3	49.9	51.1	49.9
droidkungfu		52.0	53.1	51.4	50.5	–	54.2	50.7	50.2	74.3	57.2	94.4	68.8	53.4	99.2	55.6	50.0
gingermaster		72.1	64.7	73.4	52.1	67.3	–	51.1	65.5	95.1	61.2	56.2	60.3	53.3	94.2	59.5	49.5
leadbolt		51.9	52.0	49.7	71.1	50.7	51.6	–	50.2	53.2	53.6	50.0	52.1	53.3	49.8	51.1	49.6
umeng		51.7	54.1	87.2	51.2	73.5	55.2	52.1	–	65.1	62.2	59.9	58.7	53.3	94.0	67.9	49.9
mulad		51.1	52.7	50.0	50.5	79.0	59.1	50.3	50.2	–	54.2	52.4	82.7	53.4	53.7	53.6	50.0
waps		57.2	66.9	63.3	51.2	59.7	55.1	52.1	65.5	82.9	–	55.2	55.3	53.4	59.0	97.8	68.0
domob		51.6	52.9	51.4	50.3	82.5	53.2	50.5	50.2	64.7	55.9	–	66.5	53.5	55.5	53.7	50.0
mobwin		50.7	52.7	49.8	49.9	56.8	52.0	50.0	49.9	56.4	53.3	53.2	–	53.1	53.9	52.2	49.8
revmob		49.9	52.9	49.6	51.6	49.6	51.5	50.8	49.7	50.1	53.0	49.7	51.9	–	49.7	50.8	49.5
admogo		50.5	52.5	50.0	50.8	66.4	52.1	50.3	50.2	53.9	53.4	52.3	60.8	53.4	–	51.6	50.0
wapsx		56.6	62.3	50.0	50.9	55.6	53.5	50.5	50.5	78.2	93.7	50.7	52.8	53.4	50.1	–	68.1
utchi		50.4	52.4	50.0	50.1	50.0	51.9	50.3	50.2	50.6	53.4	50.2	52.3	53.4	50.1	51.2	–

,

Table 4. Malware detection performance with different malware families in the training set using drebin. The number in bolds show the best results.

	Test	ginmaster	artemis	nandrobox	plankton	droidkungfu	gingermaster	leadbolt	umeng	mulad	waps	domob	mobwin	revmob	admogo	wapsx	utchi
Train
ginmaster		–	80.5	93.2	69.6	95.3	90.9	92.4	80.3	95.9	92.8	91.6	88.8	47.5	85.4	90.0	95.3
artemis		79.7	–	92.1	51.5	92.4	93.3	91.2	75.9	95.5	85.2	93.2	90.3	90.5	94.7	77.9	92.9
nandrobox		64.8	75.1	–	50.5	66.7	70.5	98.8	51.5	94.4	61.8	59.8	55.2	49.5	75.5	59.4	49.5
plankton		53.2	52.4	49.9	–	51.9	52.9	99.4	51.7	60.8	53.5	50.4	52.3	50.0	49.8	53.3	77.5
droidkungfu		73.0	64.4	83.3	57.3	–	73.6	97.3	77.7	97.3	82.7	95.0	94.7	48.9	98.1	85.3	94.3
gingermaster		85.2	80.2	89.0	58.5	81.2	–	95.1	72.0	96.6	92.2	84.8	78.6	48.4	75.8	87.1	96.2
leadbolt		76.2	57.0	71.2	92.3	73.0	78.9	–	73.5	93.0	80.3	83.3	74.0	48.9	49.5	81.2	95.6
umeng		74.2	75.8	92.7	58.4	93.9	79.9	94.1	–	93.8	84.8	94.8	92.5	47.7	95.6	85.7	80.6
mulad		53.8	57.8	59.2	50.3	83.0	64.2	100.0	50.3	–	55.0	61.6	57.5	50.0	51.0	57.7	50.0
waps		86.8	82.3	93.2	63.7	95.3	94.9	93.0	88.8	96.3	–	95.7	92.7	47.5	96.2	96.4	95.8
domob		74.1	70.8	89.3	53.5	96.4	79.8	95.3	85.0	95.0	85.5	–	94.0	48.2	97.3	84.1	89.5
mobwin		67.7	78.1	69.2	52.0	91.3	72.8	93.9	70.7	93.3	82.6	92.8	–	47.1	96.4	79.9	77.7
revmob		59.5	54.5	48.7	52.3	49.2	59.0	96.2	49.9	65.4	55.9	50.7	54.0	–	48.4	52.9	48.1
admogo		56.0	55.5	50.4	49.9	84.4	55.8	99.3	55.0	74.3	61.6	92.1	94.0	49.7	–	63.9	51.7
wapsx		84.0	81.4	93.3	59.4	96.6	84.5	96.3	87.5	97.8	95.0	94.1	93.8	48.8	97.6	–	98.1
utchi		57.4	51.5	49.4	50.5	50.7	54.6	99.2	52.6	49.6	67.1	54.2	52.0	49.4	49.5	62.4	–

,

Table 5. Malware detection performance with different malware families in the training set using mamadroid. The number in bolds show the best results.

	Test	ginmaster	artemis	nandrobox	plankton	droidkungfu	gingermaster	leadbolt	umeng	mulad	waps	domob	mobwin	revmob	admogo	wapsx	utchi
Train
ginmaster		–	74.4	45.7	47.8	81.7	77.8	70.3	75.7	79.9	79.4	74.6	63.2	39.9	84.7	81.8	88.6
artemis		60.1	–	88.4	53.7	85.8	70.3	59.2	55.2	87.9	81.0	88.8	72.4	90.0	50.6	75.6	46.1
nandrobox		51.6	55.9	–	49.8	51.8	54.9	49.6	52.1	50.7	52.2	57.6	53.3	49.5	50.0	51.0	49.5
plankton		50.9	51.4	49.9	–	52.2	52.4	52.2	50.1	52.5	50.3	56.0	53.5	50.6	49.9	50.2	49.8
droidkungfu		59.5	58.4	60.5	55.1	–	68.2	55.5	62.8	85.2	68.0	89.6	76.5	48.0	53.6	68.6	48.6
gingermaster		69.2	73.1	84.1	49.4	82.9	–	56.8	65.1	81.0	70.6	80.7	65.0	47.3	92.0	77.8	95.7
leadbolt		67.5	57.2	42.0	47.4	55.8	55.0	–	68.9	57.4	71.5	50.6	51.8	45.6	85.8	77.9	92.4
umeng		72.1	53.0	48.1	47.1	76.9	72.9	67.6	–	46.6	73.9	75.0	71.1	41.8	87.7	81.6	89.7
mulad		51.4	54.7	50.2	54.5	54.1	55.2	50.8	50.2	–	53.2	50.3	54.6	50.3	50.9	51.8	49.7
waps		62.0	66.8	51.0	48.6	68.4	65.9	56.2	55.9	84.6	–	58.5	55.6	48.4	92.4	93.4	47.6
domob		55.4	58.7	76.5	54.8	79.2	61.5	50.7	54.1	68.4	59.6	–	77.6	48.5	53.7	56.3	48.3
mobwin		59.2	57.3	54.1	47.4	81.2	66.4	53.4	64.4	83.9	58.9	88.0	–	47.4	52.9	60.7	69.1
revmob		49.8	51.1	49.9	49.8	49.9	51.6	49.9	49.8	49.8	50.1	49.8	52.2	–	49.9	49.9	49.8
admogo		50.3	50.7	50.1	49.8	66.5	51.7	49.8	50.4	50.2	50.8	65.1	62.9	49.8	–	50.0	49.8
wapsx		59.0	64.4	52.2	49.2	67.9	64.2	66.1	57.8	77.9	88.1	59.6	55.1	48.3	92.2	–	51.9
utchi		50.4	50.7	49.9	51.0	50.2	52.3	50.5	50.4	49.8	49.8	49.7	53.9	49.9	49.9	50.2	–

, and

Table 6. Malware detection performance with different malware families in the training set using droidsieve. The number in bolds show the best results.

	Test	ginmaster	artemis	nandrobox	plankton	droidkungfu	gingermaster	leadbolt	umeng	mulad	waps	domob	mobwin	revmob	admogo	wapsx	utchi
Train
ginmaster		–	80.4	97.0	87.3	94.1	94.1	64.4	78.9	99.3	94.6	92.6	94.1	50.2	98.0	98.0	97.4
artemis		86.5	–	83.7	77.0	95.8	98.2	56.8	73.8	99.5	93.3	72.2	91.6	94.1	99.2	92.4	96.9
nandrobox		50.3	55.4	–	50.0	50.8	52.5	50.1	50.0	67.9	50.2	50.4	52.6	50.0	50.1	51.2	50.0
plankton		50.9	50.1	50.0	–	51.0	51.7	53.8	50.0	51.2	51.6	50.2	52.4	50.1	50.1	51.3	55.6
droidkungfu		62.9	58.5	62.2	64.7	–	67.3	55.0	55.4	98.6	68.7	77.6	81.6	50.1	99.7	74.2	68.1
gingermaster		81.8	84.9	76.8	66.2	94.2	–	54.1	60.7	99.8	90.7	66.5	83.4	50.5	99.2	88.4	70.2
leadbolt		70.7	56.7	78.7	95.6	89.7	78.2	–	69.0	98.4	86.5	92.9	80.0	96.0	64.1	88.5	85.7
umeng		76.6	65.8	96.0	78.9	89.5	82.2	72.4	–	63.2	88.4	95.4	94.4	50.2	98.3	93.1	68.0
mulad		50.5	50.6	50.1	50.3	61.2	53.1	50.2	50.0	–	51.3	50.7	54.3	50.0	50.2	51.8	50.0
waps		81.9	82.4	85.5	86.9	97.3	93.1	61.3	70.6	99.7	–	77.6	86.7	50.6	99.6	99.2	97.6
domob		68.9	60.9	86.9	67.4	98.2	69.7	59.3	75.1	99.5	81.8	–	95.4	50.0	99.7	80.9	66.7
mobwin		60.9	67.1	59.0	56.2	95.8	63.8	51.6	59.6	98.6	64.1	75.0	–	50.0	99.8	66.3	62.6
revmob		49.8	50.7	49.9	50.2	49.8	51.6	53.9	49.8	49.9	49.9	49.8	52.0	–	49.8	49.8	49.8
admogo		50.8	50.5	50.2	50.2	89.3	53.2	50.0	50.0	68.0	50.4	54.2	68.4	50.0	–	51.9	50.0
wapsx		82.7	79.2	95.0	83.3	97.9	89.0	62.1	82.2	99.6	96.1	86.2	92.2	49.9	98.8	–	97.5
utchi		50.0	50.0	50.0	50.0	50.0	51.7	50.0	50.0	50.0	50.0	50.0	52.2	50.0	50.1	50.0	–

shows the accuracy of our experiment results for the different malware families in the training set. We bold the maximum values in Table 3, Table 4, Table 5, and Table 6 for the experimental groups that test the same malware family, which is the supportive score of the corresponding malware family.

It is interesting to find that accurate malware detection is possible when the malware families in the training and test sets are completely different. However, the malware detection performance differs between the four algorithms. The accuracy of malware detection between 37.7% pairs in csbd is higher than 55%, 79.1% pairs in drebin are higher than 55% and 51.6% pairs in mamadroid are higher than 55%. Moreover, 60.8% pairs in droidsieve are higher than 55%. It can be seen that only 25, 109, 36, and 72 (10.4%, 45.4%, 15%, and 30% in proportion) groups of experiments yield an accuracy of over 80%. The accuracy of malware detection is highly related to the malware family used in the training dataset. For example, gingermaster works the best and revmob performs the worst when testing the malware family ginmaster in csbd, with a difference of 22.2% in the terms of accuracy. The training set with leadbolt could achieve an accuracy of 92.3% when detecting plankton in drebin, but only 48.7% when using admogo as the training set.

5.3. Low-Resource Malware Family Detection

In this section, we investigate the performance of our algorithm in the case of low-resource malware detection. We apply SFS algorithm to four malware detection method, csbd, drebin, mamadroid and droidsieve. Each malware detection method is performed in 16 sets of experiments.

Low-resource malware families refer to the malware families that have a small amount of data but do not have enough data to train a malware detection model. The target malware family with only 10 samples in the training set, can be considered as a low-resource malware family. Other malware families of each group of experiments have 500 samples. We first conduct the experiment using data from 2015.

For a detailed discussion, we illustrate the SFS process of the detection of plankton using drebin. The leftmost part of Figure 1 shows the first step of the SFS algorithm. We train the other 15 families separately. It can be seen that training on leadbolt could perform better than any other malware families. We take leadbolt as the base set of our next iteration of SFS. The rest of the malware families are combined with the base set leadbolt, separately. The second part of Figure 1 shows that adding gingermaster to leadbolt could perform the best. The combination of leadbolt and gingermaster is taken as a new training dataset for the next round of the experiment. This is repeated until all the malware families are added to the training set. The best solution for this method is “leadbolt + gingermaster + ginmaster + utchi + wapsx + mulad + droidkungfu”.

To ensure the validity of the experiments, when a certain number of malware samples are added in each round of experiments, we then combine them with the same number of benign samples. We also perform each experiment five times to minimize the error of the experimental results. The results of each experiment are obtained by taking the average of the five experiments. We compare the best malware detection performance of training with only one family, the malware detection performance trained with all families, and the malware detection performance using our algorithm.

Table 7. The comparison of low-resource malware family detection. The number in bolds shows the best results.

	Csbd			Drebin			Mamadroid			Droidsieve
	One	All	SFS	One	All	SFS	One	All	SFS	One	All	SFS
admogo	93.27	99.2	99.51	89.79	96.83	98.92	85.63	89.55	95.26	97.47	99.67	100
artemis	69.68	71.76	80.66	76.29	86.66	88.53	66.54	68.55	75.33	85.76	88.96	92.16
domob	96.50	98.20	99.28	84.16	96.84	97.92	83.52	86.62	90.38	94.90	98.45	99.22
droidkungfu	83.49	77.31	97.65	88.14	96.55	98.29	79.39	84.81	88.98	95.14	99.31	99.55
gingermaster	75.51	87.27	90.93	79.81	94.95	96.26	73.40	75.90	82.03	92.07	96.24	99.01
ginmaster	72.33	78.03	81.99	77.53	88.71	89.65	60.23	66.85	73.82	84.55	89.55	92.67
leadbolt	85.35	67.61	85.97	78.22	81.43	87.07	63.89	58.91	67.73	84.90	76.69	84.90
mobwin	94.34	94.55	96.30	84.95	94.07	95.97	71.08	75.96	78.76	94.39	96.44	99.21
mulad	93.52	98.72	99.26	91.97	97.03	99.77	89.30	87.11	92.24	98.41	99.67	99.98
nandrobox	95.65	96.22	96.84	91.80	94.41	97.55	86.52	84.36	91.53	96.76	98.27	99.24
plankton	94.21	80.06	94.21	93.69	95.53	96.88	86.60	50.49	86.76	92.63	96.88	98.96
revmob	84.21	79.34	89.24	92.43	91.06	93.90	93.16	83.13	93.16	93.33	93.47	96.88
umeng	91.02	98.22	99.39	83.66	84.82	94.26	62.44	62.15	71.22	92.71	90.08	98.45
utchi	100	99.53	100	95.31	97.18	99.84	90.20	47.98	92.22	100	99.69	100
waps	88.32	94.5	94.87	84.05	94.81	95.87	78.15	84.25	88.60	92.22	97.63	99.04
wapsx	95.02	98.1	98.37	82.14	96.88	98.31	81.85	87.20	91.54	94.73	99.35	99.63
Average	88.28	88.66	94.03	85.87	92.99	95.56	78.24	74.61	84.97	93.12	95.02	97.43

shows the comparison of the malware detection performance between the three cases. It can be seen that SFS algorithm outperforms when applied to all the four malware detection methods. In particular, some low-resource malware family detection can be improved by over 10% in the terms of accuracy using our SFS algorithm, such as the detection of malware droidkunfu with Csbd and the detection of malware umeng with Drebin. The malware detection performance of different malware detection algorithms varies when malware families in the dataset are different. For example, the malware detection accuracy of Droidsieve is greater than 90% in 13 out of the 16 groups of experiments. With our SFS algorithm, the detection results using Droidsieve for artemis and ginmaster are improved by 6.4% and 8.12%, respectively. We also compare the average malware detection accuracy of the 16 malware families. The performance of the malware detection method using SFS algorithm has been improved. Mamadroid with SFS algorithm has the highest improvement in malware detection, which can reach 6.73%.

The Android mobile applications are constantly evolving with the rapid development of technology [52]. New malware is often constantly updated to evade malware detection. This leads to the fact that malware detection algorithms that can achieve very good results in one year may not be able to classify new malware produced in the next year.

To explore the sustainability of SFS algorithm, we use the model trained by the data collected in 2015 to test the data in 2016, that is, the malware detection classifier obtained by using outdated data is trained to detect future malware data samples by observing the performance of malware detection algorithm to further evaluate the sustainability of SFS algorithm.

It can be seen from

Table 8. The malwaredetection ability when using the outdated training data. The number in bolds shows the best results.

	Csbd			Drebin			Mamadroid			Droidsieve
	One	All	SFS	One	All	SFS	One	All	SFS	One	All	SFS
admogo	69.36	94.17	93.08	84.50	93.80	93.48	76.48	86.46	87.14	92.66	98.56	98.5
artemis	65.20	64.25	69.55	70.52	72.03	73.28	62.20	55.49	66.46	78.88	77.18	83.22
domob	88.70	84.42	96.76	81.22	93.32	93.30	80.66	82.8	85.39	92.22	95.48	97.68
droidkungfu	84.02	78.00	97.36	86.96	96.78	97.54	79.03	85.40	88.00	94.64	99.40	99.22
gingermaster	66.18	77.77	85.36	78.17	84.23	85.20	57.84	74.49	63.37	85.8	91.28	96.56
ginmaster	60.73	65.79	71.95	72.21	87.85	85.90	60.53	67.20	65.24	79.88	89.12	91.72
leadbolt	72.00	56.99	65.59	78.62	66.98	74.96	60.64	64.23	65.13	80.00	58.52	80.00
mobwin	94.87	96.37	98.06	85.72	94.54	97.22	72.02	73.86	73.96	93.72	97.94	98.92
mulad	67.33	73.92	71.56	69.71	72.86	69.11	70.78	63.17	68.10	70.88	77.54	77.00
nandrobox	91.75	92.67	93.03	88.22	92.82	95.04	84.84	72.36	84.24	91.92	95.94	96.22
plankton	67.51	60.60	67.51	75.38	85.08	86.44	76.00	56.72	72.90	72.36	76.58	88.96
revmob	69.16	64.76	72.4	75.42	73.91	73.09	71.96	67.03	71.96	83.98	78.64	86.70
umeng	70.19	76.34	70.31	74.54	70.33	69.77	57.04	60.95	52.69	79.5	82.66	89.26
utchi	99.90	99.46	99.88	95.58	97.32	99.62	88.33	50.99	91.43	99.90	99.74	99.80
waps	88.46	94.31	95.00	84.27	94.65	95.03	77.92	84.03	87.19	92.20	97.06	97.94
wapsx	84.93	94.50	92.70	82.48	93.71	93.39	76.61	82.44	83.58	89.44	96.92	99.08
Average	77.52	79.65	83.76	80.22	85.64	86.40	72.06	70.48	75.42	86.12	88.29	92.55

that SFS algorithm does not work well when some malware families, such as mulad and umeng, use the outdated data for training, but SFS algorithm still has a better detection performance for most low-resource malware family detection. Table 8 shows that SFS algorithm performs the best on average. It can be assumed that SFS algorithm can support the sustainability of malware detection algorithms. From the average value, the combination of SFS algorithm with csbd, drebin, mamadroid and droidsieve can improve their malware detection accuracy by 4.11%, 0.76%, 3.36% and 4.26%, respectively. For a specific malware family and detection algorithm, SFS algorithm can greatly improve its performance. For example, the malware detection algorithm csbd trained the malware detection classifier using data from 2015, and SFS algorithm still improve the detection accuracy of the malware family droidkungfu by 13.32% when detecting the malware family droidkungfu in 2016.

5.4. Zero-Resource Malware Family Detection

In this section, we further investigate the most extreme case of the low-resource malware family detection, which is the zero-resource malware family detection. Zero-resource is the most extreme case of low-resource malware family detection. Zero-resource detection means that the malware detection model has never seen this malware family, that is, the zero-resource malware family is not included in the training dataset.

Table 9. The comparison of zero-resource malware family detection. The number in bolds shows the best results.

	Csbd			Drebin			Mamadroid			Droidsieve
	One	All	SFS	One	All	SFS	One	All	SFS	One	All	SFS
admogo	99.16	98.78	99.44	98.14	96.60	98.90	92.37	91.09	95.80	99.78	99.74	99.98
artemis	69.12	75.97	78.24	82.35	85.03	86.60	74.42	68.25	76.71	84.92	87.32	90.18
domob	94.40	94.52	97.97	95.68	96.54	97.20	89.57	87.60	91.70	95.36	98.44	99.06
droidkungfu	92.94	95.96	97.82	96.60	96.34	97.52	85.84	86.09	89.19	98.18	99.28	99.36
gingermaster	82.07	84.75	87.63	94.85	94.62	96.09	77.78	75.16	84.37	98.22	96.28	99.01
ginmaster	72.08	74.48	77.38	86.76	88.02	88.48	72.12	68.40	75.20	86.46	89.80	92.12
leadbolt	55.80	54.51	58.41	99.95	94.36	99.95	70.31	61.50	70.31	72.42	73.54	82.62
mobwin	90.29	91.17	95.10	94.73	93.90	95.26	77.63	78.01	78.43	95.42	96.32	99.08
mulad	95.13	98.75	99.29	97.77	96.61	98.71	87.86	81.10	91.00	99.80	99.74	99.88
nandrobox	95.36	96.14	96.90	93.28	94.28	95.62	88.44	76.25	93.09	97.04	97.04	97.90
plankton	71.08	73.21	75.31	92.26	89.26	93.80	55.07	47.60	59.40	95.62	95.62	98.04
revmob	59.87	59.83	61.24	90.48	90.98	92.02	89.97	62.80	89.97	96.02	93.32	96.64
umeng	73.94	97.67	98.67	88.81	83.80	91.81	75.73	64.00	75.73	82.16	88.50	97.78
utchi	68.10	64.14	68.10	98.14	97.04	98.96	95.69	48.60	95.69	97.62	98.12	99.66
waps	93.65	75.24	94.72	94.99	94.57	96.01	88.06	84.50	90.20	96.12	97.78	99.00
wapsx	97.85	98.09	98.40	96.38	96.82	98.34	93.37	88.99	93.37	99.18	99.32	99.58
Average	81.93	83.33	86.54	93.82	93.05	95.33	82.14	73.12	84.39	93.40	94.39	96.87

shows the comparison of malware detection performance when the malware family to be detected does not exist in the training data set. It shows that SFS algorithm performs the best in all of the experiments when the malware family to be detected does not exist in the training dataset.

In 60.4% of the experiments, the malware detection performance training on all malware families is worse than the one training based on SFS. This shows that we cannot simply improve the malware detection performance on the target family by adding more but unrelated malware data. Using all malware families for training is equivalent to ignoring the differences between malware families in the hope of using one model to detect all malware. Our results also show this is a bad solution for malware detection.

The performance of SFS is higher than the two baselines in all of the experiments. This shows the effectiveness of SFS. This shows that carefully selecting multiple malware families is better than only selecting one most supportive family.

Figure 2 shows the performance of plankton in terms of accuracy. The size of the training set for the four malware detection methods in each experiment is the same. The horizontal coordinate indicates the name of the malware family for which the highest accuracy can be achieved by adding the target malware family. We could see that the malware detection accuracy of plankton reaches the highest in the middle of the process of SFS.

6. Relationship between Malware Families

Our results show that it could be supported by training with different malware families for the low-resource malware detection. However, the performance varies between different malware families. Therefore, we further dwell on understanding under what circumstances could support the low resource malware detection? We hypothesize that the knowledge transfer between different malware families is because they have similar characteristics. In this section, we study two popular characteristics: whether malware steals user data and whether malware displays advertisements to mobile users.

6.1. Malware Categories

Steal Data. Some of the malware families could steal user information from the device. The report [53] points out that 60.7% of the applications collected Android ID and other unique device identification information, 55.4% of the applications collected application list information, 13.7% of the applications collected clipboard information, such information can be used for character portraits, personalized push, and other business. The sensitivity of this kind of information is relatively high.

Display Advertisements. Some of these malware families are sorted as adware. It is a malicious application that puts unneeded ads on users’ screens, especially when accessing web services. Adware lures users to view ads that offer lucrative products and entices them to click on that ad. Once the user clicks on the ad, the developer of the unwanted application generates revenue. Some common examples of adware include weight loss programs that make money in a shorter period of time and on-screen warnings about fake viruses.

6.2. Analysis

Table 10. The characteristic of the malware families. The malware with the characteristics of stealing user data or displaying advertisements are labeled as ‘∘’ and those without these characteristics are labeled as ‘×’.

Family Name	Steal User Data	Display Ads
Ginmaster	∘	×
Artemis	×	×
Nandrobox	∘	×
Plankton	∘	×
Droidkungfu	∘	×
Gingermaster	∘	×
Leadbolt	∘	×
Umeng	×	∘
Mulad	×	∘
Waps	×	∘
Domob	×	×
Mobwin	×	×
Revmob	×	×
Admogo	×	∘
Wapsx	×	∘
Utchi	×	×

shows the special characteristics of the 16 malware families. We label the malware with the characteristics of stealing user data or displaying advertisements as ‘∘’ and those without these characteristics as ‘×’. To better show the relationship between the malware detection performance and the malware characteristics, we further leverage TSNE to map the malware detection performance in Table 8 into two-dimension vectors. The results are shown in Figure 3 for all the four malware detection models.

In Figure 3, we could find that (1) most of the malware families with the same characteristics are close to each other for all the four malware classification models. For example, the malware families that steal user data such as ginmaster, gingermaster, and nandrobox are close to each other, and plankton and leadbolt are also close to each other. The malware families that display ads such as waps, wapsx, and umeng are always close to each other. This shows that most of the similarities could be captured by all the malware detection models. (2) The malware relatedness has a slight difference between different malware detection algorithms. For example, for both csbd and mamadroid, mulad is far from the ads cluster. However, in drebin and droidsieve, mulad is close to the ads cluster, while admogo is the opposite. This may be because that drebin and droidsieve are not good at capturing the corresponding similar characteristics between admogo and other ads malware.

The results show that most of the supportive scores match our human knowledge. A higher supportive score means the characteristics of different malware are more similar. If the target malware family uses similar technology or has similar targets to the training family, our model could leverage this knowledge and achieve good results.

Although summarizing common characteristics could improve the human understanding of the knowledge transfer, using an empirical supportive score is a better way if we only target to improve the low-resource malware family performance. There are two reasons: (1) Specific malware detection models may not be good at capturing the common characteristics even if they exist. For example, csbd and mamadroid show that mulad is similar to ads cluster while drebin or droidsieve cannot. In this case, it is better to use a different family for drebin or droidsieve algorithm. (2) Summarizing all characteristics need a huge effort from experts, and experts may have less interest to study the low-resource malware families because they often have less impact. In this case, an empirical supportive score is much easy to get and only costs computer resources.

7. Conclusions and Future Work

Our work studies the cross-family knowledge transfer for low resource malware family detection. We quantify the knowledge transfer ability between malware families by supportive scores. We propose Sequential Family Selection algorithm to select multiple malware familes related to the target malware family to support low resource malware family detection based on the supportive scores of different malware families. The experiment shows that the Sequential Family Selection algorithm can better improve the performance of the malware detection model based on machine learning method in low-resource malware family detection. The research in this paper demonstrates that cross-family knowledge transfer can effectively improve the detection performance of low-resource malware. Furthermore, by analyzing the two behavioral characteristics of stealing user data and displaying advertisements, it could be found that the knowledge transfer between different malware families is due to their common characteristics.

In future work, we plan to set different weights on each malware family in the training dataset. The weights are based on the contribution to the target malware family detection. Each target malware family can select some specific families to improve its detection performance. New knowledge transfer methods can also be further explored to achieve better detection results for low-resource malware detection.