**2. Related Work**

Several studies have proposed solutions to identify social situations through mobile and wearable devices to support mental health professionals [17]. In particular, studies have developed solutions to transform contextual data into sociability information. We categorize these studies according to their primary objectives [17]: solutions that aim to classify, predict, or associate social features to mental state, and solutions that focus on detecting and quantifying sociability levels. Table 1 presents the related works categorized by their primary objectives.


**Table 1.** Categorization of related works.

### *2.1. Detecting and Quantifying Sociability*

Some studies aim to develop solutions to identify social situations through passive detection to derive high-level information such as behavioral patterns and sociability levels. Exler et al. [21] designed a classification model capable of recognizing whether a person is alone or accompanied with an accuracy of 91.1%. This model uses location data, time of day, and activity information to perform this task. Barnett et al. [22] present a statistical approach to detect changes in sociability patterns by using phone calls and text messages, which were used to predict schizophrenic relapses. Harari et al. [23] identify patterns of stability and changes in social behavior (i.e., daily duration of conversations) of a student group over ten weeks. Bonilla et al. [24] found a set of patterns related to intensity functions

of all interactions in which patients were involved by analyzing data from the use of their phone calls and social applications.

Studies also aimed to explore the passive detection of social situations to quantify sociability. Eskes et al. [25] propose the use of context data produced by smartphones (e.g., call logs, GPS locations, and Bluetooth encounters) to capture social communication and social exploration (e.g., mobility patterns and social density). These social behaviors were used to develop a statistical approach able to generate a sociability score, in which higher scores represent greater social engagement. Wahle et al. [26] monitors participants' involvement in device-mediated communication (i.e., call logs and text messages) to quantify their sociability. Additionally, this solution recommends social exercises based on information on the intensity of social activities, time and location. Lane et al. [27] present a mobile application called *BeWell*, which has a classifier able to infer the human voice through microphone audio. This application calculates a sociability score by applying a linear regression on the total duration of conversations. In addition, *BeWell* provides feedback on the social engagemen<sup>t</sup> level of its users.

### *2.2. Associations between Mental State and Social Features*

Researchers also work on correlating social features (e.g., duration of phone calls, frequency of using social applications) with mental states (e.g., bipolar disorder, anxiety, depression). Mental states are typically identified using clinically validated self-report questionnaires. In general, researchers use correlation coefficients, as the Pearson and Spearman correlation coefficients [41,42], to calculate the degree of association between these variables.

In a study involving university students, Wang et al. [35] recognize that social routines (i.e., conversations and Bluetooth co-locations) of students presented correlations with their depression symptoms and stress levels. Results indicate that students who had lower frequencies and shorter duration periods in their daily social interactions also had higher levels of depression and stress. Chow et al. [36] identify temporal associations between depression, affective state and social anxiety with social isolation (i.e., home stay duration measured by GPS data). Boukhechba et al. [37] demonstrate that the social roles of visited places and communication patterns (i.e., phone calls and text messages) of the students show consistent associations with depressive states and social anxiety.

Some studies focus specifically on investigating association of social features with participants' stress levels. Wu et al. [30] found significant correlations between student stress levels with social features extracted from their social encounters measured through Bluetooth co-locations. Ono et al. [38] use a wearable device equipped with an infrared sensor to identify face-to-face interactions. This study found relationships between participants' stress levels with frequency, duration, and number of people involved in the social interactions. To do so, evaluation and validation methods were used to measure the performance of social models.

Some studies have as the main topic of social anxiety. Gong et al. [39] performed an association between participants' social anxiety levels and their physical behaviors, which were based on accelerometer-tracked body movement during device-mediated social interactions such as phone calls and text messages. Also, this study investigates whether the location where users performed technology-mediated communication influenced the social anxiety levels. Results indicate that people with higher levels of social anxiety exhibit more movement variations when making phone calls, especially in unfamiliar environments.

Other studies aim to correlate participants' social activities with their mood status. Servia-Rodriguez et al. [33] found associations between sociability patterns measured by analyzing phone calls and text messages of a large number of participants with their self-reported mood assessments. Matic et al. [40] found associations between time spent on speech activities (i.e., participation in verbal social interactions) and changes in positive affects.

### *2.3. Classifying and Predicting Mental State*

In this study category, researchers design solutions capable of classifying and predicting mental states. Specifically, these approaches train machine learning algorithms from social features.

Different social models have developed to classify mental states of individuals. Gu et al. [28] developed a wearable device equipped with a microphone capable of automatically identifying and analyzing paralinguistic features (e.g., Brightness\_sp and MFCC5\_sp) contained in the human voice during social interactions. These features were used to train the K-Means algorithm to classify the participants' anxiety level, which obtained an accuracy of 72.73%. Chen et al. [29] use the transfer learning technique [43] to identify autism symptoms through the analysis of speech features extracted from microphone data of the wearable device.

Solutions presented by the studies also developed social models to predict mental states of individuals. For this, Wu et al. [30] use features extracted from physical social interactions identified by smartphone Bluetooth encounters to train the Random Forest algorithm, which was able to predict participants' stress levels. Barnett et al. [22] developed and applied a statistical approach to recognize changes in patients' communication patterns. The proposed method can predict schizophrenic relapses at two weeks in advance.

Some solutions recognize other human behaviors (e.g., sleep, mood, mobility) combined with sociability to design solutions that can identify mental states. Researchers identified several types of behaviors that have implications for mental health, allowing them to design more appropriate features to develop mental state classification models and predictions. For example, these multimodal features are used to design machine learning models to identify and predict patients with depression [26,31], bipolar disorder [32,34], and mood states [33]. Therefore, these solutions represent potential tools for supporting digital phenotyping of mental health as they recognize and utilize various patient behaviors to perform this task.
