Classifying the Severity of Cyberbullying Incidents by Using a Hierarchical Squashing-Attention Network

Abstract

Cyberbullying has become more prevalent in online social media platforms. Natural language processing and machine learning techniques have been employed to develop automatic cyberbullying detection models, which are only designed for binary classification tasks that can only detect whether the text contains cyberbullying content. Cyberbullying severity is a critical factor that can provide organizations with valuable information for developing cyberbullying prevention strategies. This paper proposes a hierarchical squashing-attention network (HSAN) for classifying the severity of cyberbullying incidents. Therefore, the study aimed to (1) establish a Chinese-language cyberbullying severity dataset marked with three severity ratings (slight, medium, and serious) and (2) develop a new squashing-attention mechanism (SAM) of HSAN according to the squashing function, which uses vector length to estimate the weight of attention. Experiments indicated that the SAM could sufficiently analyze sentences to determine cyberbullying severity. The proposed HSAN model outperformed other machine-learning-based and deep-learning-based models in determining the severity of cyberbullying incidents.

1. Introduction

With the prevalence of the Internet, social media has become a convenient platform for people to communicate. However, although social media is convenient and beneficial for communication, it has created several problems. In 2017, the Royal Society for Public Health (Royal Society for Public Health (RSPH) is an independent campaigning and educational charity dedicated to improving and protecting the health of people. https://www.rsph.org.uk/, accessed on 31 July 2021) in the UK reported that social networking sites contributed to physical and mental health effects such as anxiety, loneliness, and lack of sleep among nearly 1500 young people. The results of the aforementioned organization also indicated that Instagram is the social media platform most likely to have a negative effect on the mental health of young users and that Snapchat is the most likely platform to make young people feel bullied or neglected. Social media causes psychological damage, and the associated problems are exacerbated by cyberbullying [1,2,3,4,5]. People who communicate using social media are more likely to be victims of abuse or disrespect than those who do not. Therefore, cyberbullying is a critical research issue that requires increased attention in today’s online social media climate.

People on social media platforms often communicate using text-based comments or messages, which have become the primary means of cyberbullying. To manage to cyberbully, studies have combined natural language processing (NLP) and machine learning (ML) to identify automatically whether the text contains cyberbullying content. Specifically, a considerable amount of content on Twitter contains cyberbullying content [6]; thus, a cyberbullying detection technology that covers multiple social media platforms and targets various topics was developed in a previous study [7]. Moreover, numerous studies have indicated that compared with ML, deep learning (DL) has higher effectiveness in automatic cyberbullying detection [8]. However, DL models have rarely been employed in real-world environments because cyberbullying is not well-defined. Various studies have reported inconsistent findings on cyberbullying, which makes it difficult to eradicate cyberbullying effectively.

To address the issue of cyberbullying, additional emphasis should be placed on identifying cyberbullying severity. Since researchers have focused on binary classification for cyberbullying, the grading results among various cyberbullying content types differ considerably. Most social media content may be easily tolerated or ignored; however, some content remains sufficiently serious to cause adverse outcomes. In addition, the direct analysis of social media’s content to detect cyberbullying needs some variables such as behavioral patterns, personal factors, and environmental events [9,10,11,12]. These variables are very difficult to clearly define, because different cases have different behaviors. Therefore, there are no effective keywords, sentences, or patterns used to match the content of social media. To prevent cyberbullying incidents, a proposed detection model should be able to take action immediately; however, this feature is difficult to apply in practice. Therefore, if a cyberbullying detection model can classify cyberbullying incidents among different severity levels, the spread and influence of cyberbullying can be effectively prevented. Cyberbullying detection tasks mainly focus on whether text on contains cyberbullying content. Determining the severity of cyberbullying content can assist in avoiding cyberbullying incidents and help victims feel safe. Figure 1 displays the concept of a multi-class cyberbullying classification task. The present study identified the cyberbullying level based on a full dialogue (i.e., title, content, and all comments), because the same sentences may appear within different contexts in different dialogue, which results in varying influences and meanings. Therefore, the cyberbullying level can be obtained more easily when detected using a full dialogue.

To address the problem of grading the cyberbullying severity for a full dialogue, the current study aimed to (1) establish a Chinese-language cyberbullying severity dataset marked with cyberbullying severity ratings (slight, medium, and serious) based on cyberbullying characteristics identified in full dialogues and (2) develop a cyberbullying severity classification model based on a hierarchical neural network (HAN) [13] that can effectively identify cyberbullying severity level. The model proposed in this paper is the hierarchical squashing-attention network (HSAN), which can automatically determine the severity level of various cyberbullying incidents.

2. Literature Review

This section addresses research issues involving automatic cyberbullying detection systems based on an extensive literature review. A considerable amount of literature on applications of the deep learning approach is also reviewed.

2.1. Cyberbullying

Cyberbullying is bullying conducted using tools or channels such as electronic equipment or communication software. With advances in technology, sources of cyberbullying have become increasingly diverse. Cyberbullying incidents have increased yearly and have attracted increasing attention [14,15]. Currently, the definition of cyberbullying is not clear, because similar contents from multiple sources may represent different opinions and meanings. Patchin and Hinduja [16] defined cyberbullying as the harm intentionally and repetitively caused through electronic media. The basic definition of bullying also includes the characteristics of intent, repetition, and harm. Thus, cyberbullying is the transmission of malicious text messages or images intended to harass others over the Internet [17]. Snakenborg et al. [18] defined cyberbullying as the use of electronic media by individuals or groups to disseminate information about others maliciously and repetitively. Cyberbullying messages are offensive or sexually suggestive and cause the victim to feel afraid or offended. Automatic cyberbullying detection techniques have attracted an increasing amount of attention, particularly in NLP and ML. Automatic cyberbullying detection tasks mainly focus on whether the text contains cyberbullying content [19]. Cyberbullying events [20] involving racial, sexual, and sexual orientation discrimination are the object of cyberbullying classification [21]. In summary, cyberbullying is a crucial topic that requires increased attention. Various types of cyberbullying incidents produce varying degrees of harm. Cyberbullying incident severity may be affected by factors such as incident background or the environment of the people involved [22]. Therefore, the real-time detection and classification of cyberbullying severity can assist the management of real-world cyberbullying problems. Moreover, such detection and classification can enable organizations to use appropriate mitigation techniques based on cyberbullying severity to reduce the number of cyberbullying incidents in real time.

2.2. Machine Learning

Text classification is common in NLP. Numerous studies have focused on sentiment analysis, spam classification, and topic classification. The traditional text classification method uses features generated using a vector space model, such as term frequency-inverse document frequency (TFIDF), to represent a text vector, which is typically a high-dimensional and sparse feature vector that often ignores contextual semantic and word order information [23]. ML algorithms can detect cyberbullying or malicious speech based on the aforementioned features. Most methods use labeled training data for supervised learning. Reynolds et al. [24] collected and labeled data from a website and trained a C4.5 decision tree ML model to achieve an accuracy of 78.5%. Dinakar et al. [25] employed four ML approaches to detect comments related to three sensitive categories (i.e., racial discrimination, sexual bullying, and personal attacks) on YouTube. Dadvar et al. [19] revealed that research on cyberbullying detection has mainly focused on the content of comments and has ignored user background information. Therefore, to mark the bullying type of a comment, a user’s age and comment history should be included. Additional features, such as the frequency of attack words, have been used to improve cyberbullying detection in YouTube comment data when using a support vector machine (SVM) model. Huang et al. [26] used keywords as features to detect cyberbullying content automatically in a Twitter dataset. They improved the detection accuracy by considering user relationships and conducting social network analysis. Zhao et al. [27] used the word embedding method to predefine attack vocabulary and assign weights to cyber features for detecting cyberbullying incidents by using an SVM classifier, which exhibited an excellent performance when using Twitter data. Chatzakou et al. [21] distinguished general, offensive, and cyberbullying users on the basis of cyberbullying repetitiveness and used a random forest classifier to detect cyberbullying incidents whose areas under the curve exceeded 90%. Gutiérrez-Esparza et al. [28] proposed the use of an OneR model to classify situations of cyber aggression on social networks among Spanish-language users in Mexico. Their results indicated that the OneR model classified such situations with greater than 90% accuracy. Cheng et al. [29] considered social networks to include multiple modes of information, such as images, self-introductions, times, and locations. Such information is relevant for obtaining effective cyberbullying detection models constructed using a heterogeneous network of multimodal information.

2.3. Deep Neural Networks

An increasing number of studies on cyberbullying detection have employed DL methods to eliminate the manual feature extraction stage required in traditional ML because no specific or clear characteristics exist for detecting cyberbullying on different platforms. Deep neural networks have been widely applied to address various problems, such as environmental sound classification [30,31], scene parsing [32], large-scale projects construction [33], and near-misses [34]. Al-Ajlan and Ykhlef [6] reviewed studies on cyberbullying detection techniques for Twitter content and determined that most research methods for improving detection efficiency focused on the feature extraction stage without considering the semantics of words; however, language changes over time, and offensive speech is constantly being innovated. Optimizing detection performance by using feature engineering approaches is difficult. Many researchers have adopted NLP approaches and DL methods to solve cyberbullying detection problems with a backpropagation algorithm for deep neural networks to obtain a set of word vectors that learn semantics and grammar by using a corpus.

Accordingly, DL methods have a superior detection performance to traditional ML methods. Another critical model for sequential data is a recurrent neural network (RNN), which has been used to solve numerous NLP tasks. An RNN is a neural network-based model that uses a memory mechanism to remember the last state to the next time step. However, RNNs may exhibit a vanishing gradient problem if the data contain long sequence features. Therefore, Hochreiter and Schmidhuber [35] proposed a new RNN model based on long short-term memory (LSTM), which contains three gates that are used as control mechanisms of the cell (i.e., the forget gate, input gate, and output gate). These gates can manage and predict critical hidden features with long intervals in the time series and overcome the drawbacks of the original RNN. Moreover, Chung et al. [36] proposed the gated recurrent unit (GRU) model to simplify the architecture of LSTM. In their model, the forget and input gates are replaced with the “update gate” to expedite execution and reduce memory consumption. Menini et al. [37] developed a cyberbullying detection system for educators that targeted students’ social network relationships on Instagram and used LSTM to classify offensive messages. Since hate speech is on the rise, identifying whether repeated attacks against the same student have occurred is critical. In summary, RNNs have been employed in numerous studies to extract hidden features from text data effectively. Thus, RNN-based models, such as the simple RNN, GRU, and LSTM models, apply to numerous sequential data classification and regression problems.

Convolutional neural networks (CNNs) are used to retain semantics, eliminate time-consuming feature extraction, and effectively improve classification accuracy. In general, CNN models are used to extract hidden features from an image. In recent years, CNN applied to extract hidden features from text data based on word embeddings, and it can improve performances. This method can also be applied to text for extracting hidden local features of text sequences. Kim [38] proposed that the CNN model can be used to extract hidden features from the text for sentence classification tasks and that word2vec pretrained word vectors can be used as input features for CNN models. CNN uses different numbers of convolution kernels to extract multilevel features from text according to the model weight sharing approach. Moreover, the max-pooling approach has been used to extract critical features, and the SoftMax function has been used to determine the probability that each feature belongs to a certain category. Therefore, the CNNs method has been widely applied in text classification studies. Agrawal and Awekar [7] revealed that current cyberbullying detection technology can only classify bullying speech for a single topic on a specific platform; therefore, these researchers used CNN, LSTM, bidirectional LSTM, and bidirectional LSTM with an attention mechanism to overcome this drawback. The results of the aforementioned authors indicated that DL models can achieve high performance on various topics across multiple social platforms, such as Formspring, Twitter, and Wikipedia. Rosa et al. [8] compared SVMs and logistic regression with three DL methods (i.e., CNN, C-LSTM, and CNN-LSTM) to address text classification issues. Their results indicated that C-LSTM outperformed the other models on the Formspring bullying dataset. However, the combination of a CNN and an RNN also exhibits favorable performance in text classification tasks. Lai et al. [39] proposed a recurrent CNN (RCNN) to obtain semantic word vectors and used max-pooling to extract crucial features for obtaining the entire text vector. The RCNN exhibited a high classification performance. In addition, the bidirectional encoder representations from Transformers (BERT) are a powerful representation learning model, with trainings on two tasks such as masked language modeling and next sentence prediction [40]. BERT belongs to a pretrained model to obtain text representations and to improve performances in the downstream task. Therefore, many researchers use the BERT to be a pretrained model and fine-tune the models in many tasks [41,42].

In summary, many studies have addressed automated cyberbullying detection; however, their definitions of cyberbullying differ. Some studies have detected cyberbullying from only a single sentence; some studies have used an attack vocabulary to match words, and some studies have used supplemental user background information for detecting cyberbullying. According to common cyberbullying characteristics, a pattern of behavior repeated over time is a critical defining factor of cyberbullying, and optimizing detection only by adding features is difficult. However, DL methods have obtained satisfactory performance in several studies and have outperformed various feature engineering approaches. Furthermore, if identification is based on the complete context of an article, high detection accuracy can be achieved.

2.4. Hierarchical Neural Networks

People generally understand a sentence based on its keywords. Moreover, they understand a full article according to its key sentences [31]. Yang et al. [13] combined the attention mechanism with hierarchical hidden feature extraction to classify documents into four layers according to their hierarchical structure. First, the bidirectional GRU obtains each sentence feature, and weighted words are provided by the attention mechanism. The same process is then used to extract hidden features from the document. The attention mechanism uses these features to weigh the sum of important sentences. Finally, the SoftMax function was used to obtain the probability of each class. The attention mechanism can intuitively explain the importance of each word in a sentence and the importance of each sentence in an entire document. The HAN method is superior to methods that do not use a hierarchical extraction process for text classification. Gao et al. [43] combined the HAN architecture with a CNN to develop the Hierarchical Convolutional Attention Networks (HCAN) text classification model. These researchers demonstrated that a CNN can efficiently extract features from proximal words during training processing. Moreover, the attention mechanism is used to capture long-distance semantic relationships in the same manner as an RNN does. To shorten the training time and obtain favorable results, Cheng et al. [44] used the HAN method for cyberbullying detection. In their study, the HAN method outperformed keyword database development and sentiment analysis in cyberbullying detection. RNN, CNN, and hierarchical models have been frequently used to determine semantic relationships in text accurately and obtain favorable results in NLP. The hierarchical text extraction method is similar to human text reading and has obtained superior performance in numerous text classification tasks. Therefore, by adding the “message interval” feature to the HAN architecture, cyberbullying can be accurately identified. However, real-world cyberbullying cases are generally unbalanced. Previously proposed methods are not designed to address imbalanced data and thus exhibit poor performance in real-world cyberbullying detection.

3. Methodology

The proposed HSAN technical architecture is described in the following text. The HSAN is an improved version of the HAN in which the attention weight computation by SoftMax function is modified using the squashing function, which is designed according to the squashing function of capsule networks [45]. The squashing-attention mechanism (SAM) uses the length of the vector to estimate the attention weight for each word and each sentence. Figure 2 presents the five processes conducted in the current study: (1) data collection, (2) data labeling and filtering, (3) data preprocessing, (4) HSAN model construction and cyberbullying severity classification, and (5) loss function and model optimization. The details of each process are described in the following text.

3.1. Data Collection

Cyberbullying often involves text that targets individuals on the Internet. For example, the main perpetrator produces cyberbullying content on a post, and other perpetrators supplement the cyberbullying content with additional comments targeting the victim. Determining whether cyberbullying has occurred by simply viewing a message is not possible. Therefore, cyberbullying samples should be collected from interactive dialogues that contain one title, one primary piece of content, and numerous response comments because at least two people must be involved to constitute cyberbullying behavior.

3.2. Cyberbullying Annotation

After data collection with the data crawler, certain content types (e.g., emojis and hyperlinks) were determined to be unsuitable for cyberbullying classification. The detailed processing steps were as follows:

(1).

Preliminary filtration: Following cyberbullying characteristics, repetitive content from numerous users was required. A set of dialogues was established with corresponding response comments. Each dialogue required at least twenty corresponding comments in general cases.

(2).

Cyberbullying dialogue samples filtration: The cyberbullying dialogue was filtered using the manual method because the goal of this paper is cyberbullying severity.

(3).

Feature scoring: Five experts annotated all the cyberbullying dialogues independently following the labeling criteria. Intent, repetition, and aggression toward the victim were the criteria for determining cyberbullying severity. Table 1 summarizes the labeling standards in detail. Each criterion was scored on a scale of 1 (strongly disagree) to 3 (strongly agree). Thus, each dialogue could obtain three possible scores for determining severity level.

(4).

Cyberbullying severity labeling: Cyberbullying dialogues were labeled as described in the following text. If more than three experts provided the same score, the mode value was used. Furthermore, to remove outlier scores, the scores provided by the five experts were averaged. Finally, dialogues with scores of 3, 4, and 5 were labeled as “slight,” dialogues with scores of 6 or 7 were labeled as “medium,” and dialogues with scores of 8 or 9 were labeled as “serious.”

Table 1. Labeling standards for cyberbullying.

Features	Labeling Rule
Intentional	Whether the content is targeted to a specific individual or a group, if so, further determines the number of people covered by the victimizer. The small number belongs to serious, and the larger number belongs to slight.
Repeated	The proportion of people attacking the same person in the message, the larger the proportion, the higher the repetition, and the degree of cyberbullying is relatively serious.
Aggressive	The content contains personal offensive remarks, belittle others’ personality, body, life, character, etc., or offensive comments on specific characteristics or sexual orientation.

Table 1. Labeling standards for cyberbullying.

Features	Labeling Rule
Intentional	Whether the content is targeted to a specific individual or a group, if so, further determines the number of people covered by the victimizer. The small number belongs to serious, and the larger number belongs to slight.
Repeated	The proportion of people attacking the same person in the message, the larger the proportion, the higher the repetition, and the degree of cyberbullying is relatively serious.
Aggressive	The content contains personal offensive remarks, belittle others’ personality, body, life, character, etc., or offensive comments on specific characteristics or sexual orientation.

3.3. Text Preprocessing

This section describes the dataset containing text and labels. Before model training, all text was converted into an index sequence for the HSAN model.

3.3.1. Text Cleaning

The text obtained from the website could contain invalid content, such as hyperlinks, webpage tags, and emoticons. This invalid content was removed in advance. The English text in the data was converted to lowercase to avoid excessively repeated words.

3.3.2. Sentence and Word Segmentation

The proposed model is a hierarchical encoding and prediction process; therefore, all the text in each dialogue was segmented into a set of sentences (including content and comments), and each sentence contained a set of words. In sentence segmentation, sentence text was split using punctuation, including commas, periods, question marks, exclamation marks, semicolons, and new line. Jieba Chinese word segmentation tools (Chinese text segmentation: built to be the best Python Chinese word segmentation module. https://github.com/fxsjy/jieba, accessed on 1 May 2021) were used to split sentences into words.

3.3.3. Text Sequence Generation to Input Data

Before model training, all the text was transformed into dense vectors to obtain indices for representing the corresponding terms. Two terms, namely “PAD” and “UNK”, were used for text encoding processing. “PAD” represents the extra words used to create a dense vector because the feature input of the neural network model required a matrix of batch samples. All the terms in the training set were used to construct a vocabulary dictionary and transform the terms to indices with dense vectors in the training, validation, and test sets. Moreover, an unknown term (not in the vocabulary dictionary) in the validation, and test sets were replaced with the “UNK” term. For example, consider a dialogue consisting of L sentences, where each sentence contains T words. Suppose that w is expressed as a word in a sentence x, where $x=\left\{w_1, w_2,\dots , w_T\right\}$. The sentence of title is $x_1^{TITLE}$, the sentences of content are $\left\{x_1^{CONTENT},\dots ,x_C^{CONTENT}\right\}$, and the sentences of comments are $\left\{x_1^{COMMENT},\dots ,x_M^{COMMENT}\right\}$. Thus, the total number of sentences is $L=1+C+M$, where C is the number of sentences of content and the M is the number of sentences of comment.

3.4. HSAN Model Building

Our proposed HSAN uses a hierarchical text-encoding method to obtain a dialogue-encoding vector. The final hidden feature of the dialogue consists of four parts: (1) a word encoder for extracting hidden word features, (2) a word SAM for applying different attention weights to words, (3) a sentence encoder for extracting hidden features of sentences, and (4) a sentence SAM for applying different attention weights to words. Figure 3 presents the model architecture of the proposed HSAN. The detailed processes of the proposed HSAN are described in the following text.

3.4.1. Word Encoder

This section describes the three text parts (i.e., title, content, and comments) input into the word encoder. The s is a word’s encoded vector by the word encoder $W^{encoder}$ for the three text parts. For example, the encoded title sentence $x_1^{TITLE}$ is $s_1$, the encoded content sentences $\left\{x_1^{CONTENT},\dots ,x_C^{CONTENT}\right\}$ are $\left\{s_2,\dots ,s_{C+1}\right\}$, and the encoded comment sentences $\left\{x_1^{COMMENT},\dots ,x_M^{COMMENT}\right\}$ are $\left\{s_{C+2},\dots ,s_L\right\}$:

$$s_1=W^{encoder}\left(x_1^{TITLE}\right)$$

(1)

$$\left\{s_2,\dots ,s_{C+1}\right\}=\{W^{encoder}\left(x_1^{CONTENT}\right),\dots ,W^{encoder}\left(x_C^{CONTENT}\right)\}$$

(2)

$$\left\{s_{C+2},\dots ,s_L\right\}=\{W^{encoder}\left(x_1^{COMMENT}\right),\dots ,W^{encoder}\left(x_M^{COMMENT}\right)\}$$

(3)

$W^{encoder}$ consists of the word encoding matrix $W_e,$ which is used to represent the word $w_t$ of each sentence $s_i$ in the word encoding ${x^{\prime }}_t$. The standard bidirectional GRU was used to obtain the hidden features of each word $h_t$ = $[{\overrightarrow{h}}_t, {\overleftarrow{h}}_t]$. The forward $\overrightarrow{GRU}$ indicates reading from the first word to the last word of the sentence, and the backward $\overleftarrow{GRU}$ indicates reading from the last word to the first word.

$${x^{\prime }}_t=W_e{w}_t,\hspace{1em}\hspace{1em}t\in \left[1, T\right]$$

(4)

$${\overrightarrow{h}}_t=\overrightarrow{GRU}\left({x^{\prime }}_t\right)$$

(5)

$${\overleftarrow{h}}_t=\overleftarrow{GRU}\left({x^{\prime }}_t\right)$$

(6)

Therefore, the $h_t$ obtained by connecting ${\overrightarrow{h}}_t$ and ${\overleftarrow{h}}_t$ represents the hidden features of the word $w_t$ learned after reading an entire sentence.

3.4.2. Word SAM

A GRU was used to estimate the hidden features for each word; however, each word exhibits different importance to the meaning of the sentence. Thus, a SAM was used to estimate the weight of each hidden feature of a word. If the SAM calculated the weighted sum of the hidden features $h_t$ of each word, the final sentence vector $s_i$ was obtained as follows:

$$u_t=tanh\left(W_w{h}_t\right)$$

(7)

$$l_t=\frac{\Vert u_t{\Vert }^2}{1+\Vert u_t{\Vert }^2}\frac{u_t}{\Vert u_t\Vert }$$

(8)

$${\alpha }_t=\Vert l_t\Vert$$

(9)

$$s_i={\displaystyle \sum }_t{\alpha }_t{h}_t$$

(10)

where $W_w$ is a learnable weight, and $u_t$ is used to measure the importance of each word $h_t$. $l_t$ uses squashing to compress $u_t$ so that its vector length falls between 0 and 1. The length of $l_t$ is used as the importance weight of the word ${\alpha }_t$, and the weighted sentence vector $s$ is obtained. The squashing-attention used is different from that of previous studies on the attention mechanism. For the weight value, a vector was used as an estimate instead of normalizing the value.

3.4.3. Sentence Encoder

The sentence vector $s_i, i\in \left[1, L\right],$ was obtained from the aforementioned word encoder. The bidirectional GRU was used to obtain the hidden features of the sentence ${h^{\prime }}_i$=$[{\overrightarrow{h^{\prime }}}_i$,${\overleftarrow{h^{\prime }}}_i]$.

$${\overrightarrow{h^{\prime }}}_i=\overrightarrow{GRU}\left(s_i\right), i\in \left[1, L\right]$$

(11)

$${\overleftarrow{h^{\prime }}}_i=\overleftarrow{GRU}\left(s_i\right), i\in \left[L, 1\right]$$

(12)

where ${h^{\prime }}_i$ is obtained by connecting ${\overrightarrow{h^{\prime }}}_i$ with ${\overleftarrow{h^{\prime }}}_i$ to represent the hidden features of the $i$th sentence, which constitutes learning of the contextual semantic relationship.

3.4.4. Sentence SAM

A SAM was used to estimate the attention weight of the hidden features of each sentence. If the attention weight ${{\alpha }^{\prime }}_i$ of the hidden feature ${h^{\prime }}_i$ of each sentence was estimated using the SAM, the final text vector $v$ was obtained as follows:

$${u^{\prime }}_i=tanh\left(W_S{h^{\prime }}_i\right)$$

(13)

$${l^{\prime }}_i=\frac{\Vert {u^{\prime }}_i{\Vert }^2}{1+\Vert {u^{\prime }}_i{\Vert }^2}\frac{{u^{\prime }}_i}{\Vert {u^{\prime }}_i\Vert }$$

(14)

$${{\alpha }^{\prime }}_i=\Vert {l^{\prime }}_i\Vert$$

(15)

$$v={\displaystyle \sum }_i{{\alpha }^{\prime }}_i{h^{\prime }}_i$$

(16)

where $W_S$ is a learnable weight and ${u^{\prime }}_i$ is used to measure the importance of the hidden feature ${h^{\prime }}_i$ of the sentence. Squashing was performed to compress ${u^{\prime }}_i$ so that its vector length was between 0 and 1. The length of the obtained ${l^{\prime }}_i$ value was calculated as the importance weight of the sentence ${{\alpha }^{\prime }}_i$. Finally, the hidden features $v$ of a dialogue were obtained after the weighting was determined.

3.4.5. Classification of Cyberbullying Severity

The text encoding vector $v$ of a dialogue was obtained for the classification of cyberbullying severity. The probability P of cyberbullying severity belonging to a certain class is expressed as follows:

$$P=softmax\left(W_{class}v+b_{class}\right)$$

(17)

where $W_{class}$ and $b_{class}$ are trainable weights and SoftMax is used for probability transformation. The severity levels of the proposed HSAN are classified into three categories: slight, medium, and serious. $P=\left(P_1,P_2,P_3\right)$ represent the probabilities of the three categories output by the proposed model. Cross-entropy was used as the loss function to calculate the loss value, and the final loss value is expressed as follows:

$$loss\left(\theta \right)=-{\displaystyle \sum }_{k=1}^3{y}_{k}log\left(P_k\right)$$

(18)

where $y_k$ represents the target labels of cyberbully severity; if the label category is $k$, then $y_k=1$; otherwise, $y_k=0$. The parameter $\theta$ denotes all the trainable parameters of the proposed HSAN model.

4. Experimental Results

This section describes the experimental data, implemented models, experimental design, and data analysis results.

4.1. Experimental Data

The experimental data (dialogues) were collected from the “gossip” board of the PTT forum (PTT Bulletin Board System: is the largest terminal-based bulletin board system (BBS) based in Taiwan. https://www.ptt.cc/, accessed on 1 January 2021) in Taiwan from January to June 2019. After data preprocessing, 5000 dialogues containing cyberbullying content were selected. All the dialogues were independently annotated for cyberbullying severity labeling by five experts.

Table 2. MAE of the five annotations and the final classification results.

Annotator	A	B	C	D	E	Average
MAE	0.51	0.53	0.48	0.60	0.48	0.52

presents the Mean Absolute Error (MAE) used to measure annotation consistency between the final label and label of each annotator. The average MAE of annotation consistency was 0.52.

Table 3. Cyberbullying data statistics.

Type	Number	Mean	Mode	Maximum	Minimum
Title	Word	10	10	21	1
Content	Sentence	64	100	1193	3
	word	5	1	54	1
Comment	Sentence	150	24	1500	20
	word	7	1	28	1

presents the details of the number of sentences and words in the three parts of the dialogue. For the title, the mean and mode of words were both 10. For content, the mean of sentences and words was 64 and 5, respectively, and the mode of sentences and words was 100 and 1, respectively. For comment, the mean of sentences and words was 150 and 7, respectively, and the mode of sentences and words was 24 and 1, respectively.

In total, 1838 samples belonged to the slight category, 1570 belonged to the medium category, and 1592 belonged to the serious category. The 5000 cyberbullying data samples were divided into 3500 training sets samples, 500 verification set samples, and 1000 test set samples in a ratio of 7:1:2.

Table 4. Experimental data statistics.

Data Sets	No. of Samples	Severity
		Slight	Medium	Serious
Training set	3500	1190 (34%)	870 (25%)	1440 (41%)
Validation set	500	201 (40%)	241 (48%)	58 (12%)
Test set	1000	447 (45%)	459 (46%)	94 (9%)
Total	5000	1838 (100%)	1570 (100%)	1592 (100%)

presents the experimental data statistics for the three cyberbullying severity labels.

4.2. Classification Models

In addition to the HSAN, the classification performance of several other models was determined. The proposed HSAN classification model was compared with two machine learning classification algorithms and ten deep learning classification models. All the classification models were rebuilt and trained by this study. The aforementioned models are described in the following text.

(1).

HSAN: Proposed model. This model has a hierarchical structure and uses a bidirectional GRU and squashing-attention network to encode text into hidden features. It uses the SoftMax function for classification.

(2).

GRU: This model uses a bidirectional GRU to encode text into hidden features. It has a nonhierarchical structure and uses the SoftMax function for classification.

(3).

GRU(Soft): This model is similar to the GRU model but includes a soft-attention network.

(4).

GRU(Squashing): This model is similar to the GRU model but includes a squashing-attention network.

(5).

HAN [13]: This model is a baseline model with a hierarchical structure.

(6).

SVM [46]: This baseline model uses the SVM algorithm. It uses TFIDF to transform documents into vectors.

(7).

RandomForest [47]: This baseline model uses the random forest algorithm. It uses TFIDF to transform documents into vectors

(8).

TextCNN [38]: This model uses word2vec pretraining word embeddings as input and multiple CNN kernels with max-pooling to extract multi-level features from the text.

(9).

TextRCNN [39]: This model uses bidirectional LSTM with max-pooling to extract critical features.

(10).

DPCNN [48]: This model is known as the deep pyramid CNN. It contains a word-level CNN with max pooling. The amount of collected data is halved to form a pyramid structure.

(11).

Transformer [49]: In this model, an encoder and a decoder are constructed through multilayer stacking and used with the self-attention mechanism.

(12).

CapsuleNet [45]: This model comprises neurons. In CapsuleNet, dynamic routing of the last layer of category capsules is conducted. A new squashing function is used to compress the vector length of the category, which represents the final category probability.

(13).

BERT: The model is known as the BERT classifier. It contains a pretrained BERT model with a final classification layer (linear layer). The study uses three pretrained models such as bert-base-chinese [40], distilbert-base-multilingual-cased [42], and allenai/longformer-base-4096 [41].

4.3. Experimental Design

In the experimental data, we propose that the splitting ratio was 7:1:2 for the training, validation, and test sets. The training set was used to train the model. Hyperparameter tuning was conducted in the validation set to avoid overfitting, and the superior model was selected. The test set was used for comparing the performance of the proposed model and other models. According to the cyberbullying data statistics for the training set, the maximum number of sentences for content and comments was 64 and 146, respectively. The maximum number of words for title, content, and comment was 10. The models’ parameter combinations were used to identify the best model according to the classification performance for the validation set.

Table 5. Parameter settings of the compared models.

Model	Parameter	Value
HSAN, HAN, GRU, GRU(Soft), GRU(Squashing), TextCNN, TextRCNN, DPCNN	Learning rate	1 × 10−3, 1 × 10−4, 1 × 10−5
	Epoch	50
	Hidden and embedding size	300, 500
	Dropout rate	0.1
SVM	Kernel	RBF, linear, poly, sigmoid
	Gamma	0.3, 0.5, 0.7, 0.9, 1
	Cost	1
Random Forest	Number of estimator	10, 20, 30, 40, 50, 60, 70, 80, 90, 100
Text CNN	filter sizes	(2, 3, 4)
	number of channels	300
DPCNN	filter size	3, 4, 5
	number of channels	300
CapsuleNet	routing	2, 5, 10
BERT	Learning rate Pretrained model	1 × 10−4, 1 × 10−5, 1 × 10−6 bert-base-chinese, distilbert-base-multilingual-cased and allenai/longformer-base-4096

presents the detailed parameter settings of the model.

4.4. Evaluation Metrics

The current study proposes a multi-class classification task for determining the severity of cyberbullying incidents, which are rated as slight, medium, and serious. Performance evaluation was conducted according to the metrics of precision ($precisio{n}_j$), recall ($recal{l}_j$), and F1 ($f{1}_j$) on each label j, and defined as follows:

$$ACC=\frac{numbe of labels correctely classified}{Total number of samples}$$

(19)

$$precisio{n}_j=\frac{number of label j correctly classified}{number of label j in gold standard}$$

(20)

$$recal{l}_j=\frac{number of label j correctly classified}{number of label j in model predicted}$$

(21)

$$f{1}_j=\frac{2\left(precisio{n}_j\times recal{l}_j\right)}{precisio{n}_j+recal{l}_j}$$

(22)

To evaluate multi-class problems, we used the macro and weighted averaging approaches to compute overall metrics for the three severity labels. The metrics $PR{E}_{macro}$ (macro precision), $PR{E}_{weighted}$ (weighted precision), $RE{C}_{macro}$ (macro recall), $RE{C}_{weighted}$ (weighted recall), $F{1}_{macro}$ (macro F1), $F{1}_{weighted}$ (weighted F1), and $F{1}_{overall}$ are defined as follows:

$$PR{E}_{macro}=\frac{1}{3}{\displaystyle \sum }_{j=1}^{3}precisio{n}_j$$

(23)

$$RE{C}_{macro}=\frac{1}{3}{\displaystyle \sum }_{j=1}^{3}recal{l}_j$$

(24)

$$F{1}_{macro}=\frac{1}{3}{\displaystyle \sum }_{j=1}^{3}f{1}_j$$

(25)

$$PR{E}_{weighted}=\frac{1}{N}{\displaystyle \sum }_{j=1}^{3}precisio{n}_j\ast n_j$$

(26)

$$RE{C}_{weighted}=\frac{1}{N}{\displaystyle \sum }_{j=1}^{3}recal{l}_j\ast n_j$$

(27)

$$F{1}_{weighted}=\frac{1}{N}{\displaystyle \sum }_{j=1}^{3}f{1}_j\ast n_j$$

(28)

$$F{1}_{overall}=\left(F{1}_{macro}+F{1}_{weighted}\right)/2$$

(29)

where N denotes the number of all samples and $n_j$ denotes the number of jth labels.

4.5. Classification Performance of the HSAN

Table 6. Performance of the HSAN for the three severity labels.

Label	Validation Set			Test Set
	$\mathit{P}\mathit{R}{\mathit{E}}_{\mathit{j}}$	$\mathit{R}\mathit{E}{\mathit{C}}_{\mathit{j}}$	$\mathit{F}{1}_{\mathit{j}}$	$\mathit{P}\mathit{R}{\mathit{E}}_{\mathit{j}}$	$\mathit{R}\mathit{E}{\mathit{C}}_{\mathit{j}}$	$\mathit{F}{1}_{\mathit{j}}$
Slight	63.43%	68.16%	65.71%	65.60%	73.38%	69.27%
Medium	62.55%	63.07%	62.81%	59.42%	57.73%	58.56%
Serious	68.29%	48.28%	56.57%	55.56%	31.91%	40.54%
Average macro	64.76%	59.84%	61.70%	60.19%	54.34%	56.12%
Average weighted	63.56%	63.40%	63.22%	61.82%	62.30%	61.76%
Overall	64.16%	61.62%	62.46%	61.01%	58.32%	58.94%

presents the average results obtained for three evaluation metrics of the HSAN under each severity label. In summary, for the validation set, the $PR{E}_{overall}$, $RE{C}_{overall}$, and $F{1}_{overall}$ values of the HSAN were 64.16%, 61.62, and 62.46%, respectively; for the test set, the $PR{E}_{overall}$, $RE{C}_{overall}$, and $F{1}_{overall}$ values of the HSAN were 61.01%, 58.32%, and 58.94%, respectively. The average recall of the “serious” severity label was lower than that of the other two labels, because the ratios of the samples in the training and validation sets were unbalanced. The number of samples labeled as “serious” constituted a larger proportion of the training set but a smaller proportion of the validation set. Therefore, the unbalanced data affected the classification performance. The aforementioned results were similar to those obtained for the validation set. The average recall of the “serious” severity label was lower than the other two labels, because the distributions of the validation and test sets were the same and the data were unbalanced.

4.6. Comparison of Learning Rates and Hidden Sizes

The $F{1}_{overall}$ values under the six parameter combinations of hidden size and learning rate in the validation set for the HSAN are presented in Figure 4. The value of 0.0001 represents the optimal learning rate, and the value of 300 represents the optimal hidden size and word embedding size according to $F{1}_{Overall}$.

4.7. Evaluation of Hierarchical Structure Versus Non-Hierarchical Structure

The current study compared the performance for the validation set between the squashing function in the HSAN and the SoftMax function in the HAN. The results indicate that the squashing function in the attention mechanism can achieve superior classification performance to the SoftMax function in the HAN (e.g., the squashing function had an 11.87% higher $F{1}_{overall}$ value than the SoftMax function did). In addition, the paper also applies the squashing-attention to the non-hierarchical model, such as the GRU(Squashing) model, the difference between GRU(Soft) and GRU(Squashing) in the $F{1}_{overall}$ is 5.82%, the difference between HSAN and GRU(Soft) in $F{1}_{overall}$ is 6.25%, and the different between HSAN and GRU in $F{1}_{overall}$ is 26.54%.

Table 7. Evaluation of the hierarchical and nonhierarchical structures.

Model	Hierarchical	Attention	$\mathit{F}{1}_{\mathit{m}\mathit{a}\mathit{c}\mathit{r}\mathit{o}}$	$\mathit{F}{1}_{\mathit{w}\mathit{e}\mathit{i}\mathit{g}\mathit{h}\mathit{t}\mathit{e}\mathit{d}}$	$\mathit{F}{1}_{\mathit{o}\mathit{v}\mathit{e}\mathit{r}\mathit{a}\mathit{l}\mathit{l}}$
GRU	No	No	30.94%	40.89%	35.92%
GRU(Soft)	No	Soft	53.50%	58.92%	56.21%
GRU(Squashing)	No	Squashing	61.36%	62.70%	62.03%
HAN	Yes	Soft	43.56%	57.62%	50.59%
HSAN	Yes	Squashing	61.70%	63.22%	62.46%

presents a comparison between the hierarchical and nonhierarchical structures when using the squashing-attention and soft-attention networks.

4.8. Comparison of the HSAN with Other Models

Table 8. Performance of HSAN with other popular models.

Model	ACC	$\mathit{F}{1}_{\mathit{m}\mathit{a}\mathit{c}\mathit{r}\mathit{o}}$	$\mathit{F}{1}_{\mathit{w}\mathit{e}\mathit{i}\mathit{g}\mathit{h}\mathit{t}\mathit{e}\mathit{d}}$	$\mathit{F}{1}_{\mathit{o}\mathit{v}\mathit{e}\mathit{r}\mathit{a}\mathit{l}\mathit{l}}$
SVM	54.50%	52.89%	52.10%	52.50%
RandomForest	46.40%	47.61%	38.68%	43.15%
TextCNN	60.00%	58.39%	58.75%	58.57%
TextRCNN	58.60%	58.60%	58.46%	58.53%
DPCNN	60.30%	57.16%	58.98%	58.07%
Transformer	55.40%	47.34%	53.80%	50.57%
CapsuleNet	45.25%	30.60%	41.47%	36.04%
BERT (bert-base-chinese)	57.40%	53.21%	51.48%	52.35%
BERT (distilbert-base-multilingual-cased)	55.80%	53.43%	54.58%	54.00%
BERT (allenai/longformer-base-4096)	49.00%	43.94%	45.03%	44.49%
HAN	58.70%	54.15%	55.77%	54.96%
HSAN	62.30%	56.12%	61.76%	58.94%

presents the evaluation results for all classification models in the test set. The HSAN model was superior to the two traditional ML models and nine DL models evaluated, with the $F{1}_{overall}$ being 58.94% by SHAN model. The labeling ratios of the training, validation and test datasets were unbalanced, and the labeling ratio of the training set versus the validation and test sets was inverted. Therefore, the $F{1}_{overall}$ measurement was used to evaluate other models on the test dataset. The performance of six models such as the TextCNN, TextRCNN, DPCNN, Transformer, CapsuleNet, and the HAN models were similar but not superior to those of the HSAN. The differences compared to the HSAN model in those six models in the $F{1}_{overall}$ values were 0.37%, 0.41%, 0.87%, 8.37%, 22.9%, and 3.98%. To compare with the three BERT classifiers (such as bert-base-chinese, distilbert-base-multilingual-cased, and allenai/longformer-base-4096) that our proposed HSAN model also obtains better performances, the differences in the $F{1}_{overall}$ were 6.59%, 4.94%, and 14.45%, respectively. The longformer pretrained model supports sequences of length up to 4096 words, but cannot obtain a better classification performance. In another hand, the HSAN model also outperforms machine learning models such as SVM and RandomForest, which the difference in the $F{1}_{overall}$ are 6.44% and 15.78%, respectively. In summary, the TextCNN model has the smaller difference in the $F{1}_{overall}$ compared with HSAN, and CapsuleNet has a larger difference in the $F{1}_{overall}$. In addition, the TextRCNN model outperforms the BERT and Transformer models. A reason is that the TextRCNN has considered all words (average 1380 words) of the input content (Lai et al., 2015); in this case, the pretrained BERT model has only considered 512 tokens (words) which is not enough to cover the whole input content (i.e., average 1380 words in a cyberbullying document). Therefore, the pretrained BERT model applied to the cyberbullying problem with the long document is not obtained the effective learning, because most of the words are not covered.

4.9. Comparison of the HSAN with Other Models on Other Public Datasets

In this section, we attempt to evaluate our proposed model, which learns the classification task of the English language. There are four public datasets with the English language to evaluate our proposed HSAN performances. All datasets use the under-sampling to reduce samples with random sample 10% data. The IMDB Review dataset was released by Mass et al. [50], and it belongs to a binary classification task that has 25000 training samples, 25000 testing samples, and balanced data. The Yahoo Answer dataset was released by Zhang et al. [51]; it belongs to a ten-class classification task that has 1,400,000 training samples, 60,000 testing samples, and balanced data. The Yelp Review Polarity dataset was released by Zhang et al. [51], and it belongs to a binary classification task that has 560,000 training samples, 38,000 testing samples, and balanced data. The Amazon Review Polarity dataset was released by Zhang et al. [51], and it belongs to a binary classification task that has 3,600,000 training samples, 400,000 testing samples, and balanced data.

Table 9. Performance on four public datasets with the English language.

Dataset	HAN				HSAN
	ACC	$\mathit{P}\mathit{R}\mathit{E}$	$\mathit{R}\mathit{E}\mathit{C}$	$\mathit{F}{1}_{\mathit{o}\mathit{v}\mathit{e}\mathit{r}\mathit{a}\mathit{l}\mathit{l}}$	ACC	$\mathit{P}\mathit{R}\mathit{E}$	$\mathit{R}\mathit{E}\mathit{C}$	$\mathit{F}{1}_{\mathit{o}\mathit{v}\mathit{e}\mathit{r}\mathit{a}\mathit{l}\mathit{l}}$
IMDB Review	65.23%	65.47%	65.35%	65.09%	66.54%	66.60%	66.54%	66.57%
Yahoo Answer	65.98%	65.87%	65.92%	65.72%	66.03%	65.84%	66.03%	65.93%
Yelp Review Polarity	83.60%	83.77%	83.68%	83.57%	84.92%	85.03%	84.92%	84.97%
Amazon Review Polarity	70.83%	74.53%	72.63%	73.57%	77.65%	78.62%	77.65%	78.13%

presents the evaluation results for HAN and HSAN classification models in the test set. Overall, the HSAN model was superior to the HAN model on four datasets. Therefore, the proposed squashing-attention mechanism also can improve the classification performance in the English language.

4.10. Discussion

The current study used the popular online PTT forum in Taiwan as the experimental data source to detect cyberbullying incidents in dialogue and determine their severity on three levels. According to the experimental results, compared with the HAN, the SAM in the HSAN reduced the severity of cyberbullying by 3.98% $F{1}_{overall}$ on the test dataset. When the SAM was applied to nonhierarchical structures in our experiments, the performance of the GRU(Squashing) model outperformed the GRU(Soft) model. Three BERT classifiers have a word length limitation which cannot cover very large content, such as multiple sentences in cyberbullying dialogue. Although using the longformertations model (input 4096 words) also cannot obtain improvement. In addition, the three experimental datasets were unbalanced. The training set contained larger samples related to the “serious” severity label than the other two labels, but in the test set, the “serious” severity label had fewer samples. The SAM improved the classification performance for the “serious” severity label in the inverse status. Cyberbullying cases are ubiquitous on social media; therefore, company owners and other stakeholders should play positive roles and advocate for increased corporate social responsibility. Companies can also further improve Internet content protection policies by establishing complaint and consultation channels for cyberbullying to help victims quickly obtain resources from possible partners, such as the Institute of Watch Internet Network, which facilitates the organization and implementation of extensive online public safety programs.

5. Conclusions

The HSAN model is proposed in this paper for the automatic detection of the severity of cyberbullying incidents in online dialogues on social media and the classification of cyberbullying severity. The paper has created a novel dataset for detecting that includes three levels for the severity of cyberbullying incidents, and it helped the social worker to collect their attention on cyberbullying cases. Our proposed HSAN model also has improved classification in the three-level cyberbullying problem. In addition, the proposed HSAN model can provide researchers to implement classification detection in a format of multiple sentences in threaded discussion. The results of the current study indicate that compared with the soft-attention mechanism, the SAM provided superior attention weight estimation results. In addition, implementing hierarchical networks for sentence- and word-level classification is suitable for hidden feature extraction and prediction in cyberbullying dialogues. For precisely estimating attention weight, the HSAN model applied higher attention weight to cyberbullying sentences than the HAN model did. Therefore, the HSAN model exhibited superior performance to the HAN model in the cyberbullying severity classification task. The three-level HSAN model had an $F{1}_{overall}$ value of 58.94% and outperformed the other eight prediction models. Furthermore, the experimental data was unbalanced, which resulted in the training set data having the highest number of “serious” severity labels; however, the number of samples in validation and test sets had the lowest proportions, which resulted in relatively poor model prediction for the “serious” category. Cyberbullying may occur in the form of pictures or videos; however, the current study did not account for these two factors, because text represents most cyberbullying cases on social media platforms.

In the future, the word embeddings in the HSAN model can be replaced with a pretrained Bidirectional Encoder Representations from Transformers (BERT)-based model. Since the BERT model has the restriction of sentence length, BERT could not obtain the effective features from the long length content, because the truncation approach is used to a fixed sentence length which is 512 tokens. However, the pretrained BERT model was formed with a considerable amount of textual data, and it is possible to get more precise semantic information. Therefore, the pretrained BERT model is used to replace the word encoder in the proposed model, because the maximum length (number of words) of a sentence in our experimental data is 28, so the pretrained BERT model may benefit the cyberbullying dataset. Moreover, a sentiment analysis can be used to extract additional sentimental features from text, because many negative words or sentences generated by a cyberbully may be used to present evidence of cyberbullying to someone other than the original victim.