Next Article in Journal
Quantifiable Interactivity of Malicious URLs and the Social Media Ecosystem
Next Article in Special Issue
A Comparative Analysis of Wi-Fi Offloading and Cooperation in Small-Cell Network
Previous Article in Journal
High-Speed Rail-to-Rail Class-AB Buffer Amplifier with Compact, Adaptive Biasing for FPD Applications
Previous Article in Special Issue
Distributed Fog Computing for Internet of Things (IoT) Based Ambient Data Processing and Analysis
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Artificial Neural Network (ANN) Enabled Internet of Things (IoT) Architecture for Music Therapy

1
Department of Computer Science, DHA Suffa University, Karachi City 75500, Pakistan
2
School of Computing, Engineering and Physical Sciences, University of the West of Scotland, Glasgow G72 0LH, UK
3
Department of Computer Science, Institute of Business Administration, Karachi City 75270, Pakistan
4
Department of Anaesthesiology, Aga Khan University Hospital, Karachi City 74800, Pakistan
*
Author to whom correspondence should be addressed.
Electronics 2020, 9(12), 2019; https://doi.org/10.3390/electronics9122019
Submission received: 1 October 2020 / Revised: 22 November 2020 / Accepted: 24 November 2020 / Published: 29 November 2020
(This article belongs to the Special Issue Enabling Technologies for Internet of Things)

Abstract

:
Alternative medicine techniques such as music therapy have been a recent interest of medical practitioners and researchers. Significant clinical evidence suggests that music has a positive influence over pain, stress and anxiety for the patients of cancer, pre and post surgery, insomnia, child birth, end of life care, etc. Similarly, the technologies of Internet of Things (IoT), Body Area Networks (BAN) and Artificial Neural Networks (ANN) have been playing a vital role to improve the health and safety of the population through offering continuous remote monitoring facilities and immediate medical response. In this article, we propose a novel ANN enabled IoT architecture to integrate music therapy with BAN and ANN for providing immediate assistance to patients by automating the process of music therapy. The proposed architecture comprises of monitoring the body parameters of patients using BAN, categorizing the disease using ANN and playing music of the most appropriate type over the patient’s handheld device, when required. In addition, the ANN will also exploit Music Analytics such as the type and duration of music played and its impact over patient’s body parameters to iteratively improve the process of automated music therapy. We detail development of a prototype Android app which builds a playlist and plays music according to the emotional state of the user, in real time. Data for pulse rate, blood pressure and breath rate has been generated using Node-Red, and ANN has been created using Google Colaboratory (Colab). MQTT broker has been used to send generated data to Android device. The ANN uses binary and categorical cross-entropy loss functions, Adam optimiser and ReLU activation function to predict the mood of patient and suggest the most appropriate type of music.

1. Introduction

Various innovative uses of music have been discovered by researchers, apart from just being used for entertainment purposes. Music does not only positively affect the mood of healthy individuals but it also has a tendency to reduce the stress of patients [1]. Music therapy refers to a form of alternative medicine where music is used to impact the physical, social, emotional or cognitive health of individuals or groups. This therapy can involve various activities, such as listening to prerecorded or live vocal or instrumental compositions, listening to nature sounds such as waterfall or chirping birds, listening to religious recitations, playing an instrument or singing; all these activities can take place individually or in a group. When a patient listens to any form of music, it is referred to as passive music therapy, whereas when he participates in activities such as singing or playing an instrument, the therapy is referred to as active.
Although music cannot cure or prevent diseases, it has been known to reduce the stress, anxiety and possibly pain, in patients of critical illnesses. Past clinical studies have reported various forms of music therapy to positively influence over the well-being of individuals suffering from stress or anxiety due to cancer, insomnia, dental treatment, pre and post surgery phases, autism/disabilities and childbirth. Furthermore, there has also been evidence for the positive role of music for the elderly receiving end of life care. Despite the possible enormous advantage of music therapy, its use in clinical practice has been limited due to the lack of resources and lack of awareness/acceptance among the medical professionals and patients alike. Clearly, for conducting the physical music therapy sessions, the dedicated services of music experts and physicians are required. Furthermore, the patients are also required to visit the clinical facilities to receive these therapy sessions. Therefore, in case the process of music therapy can be automated using advanced information technologies, there is a high probability the service can be extended to a large number of populations without incurring heavy costs.
There have been various communication and computing technologies used in the domain of healthcare. Over the past decade, the focus of these technologies has widely been shifted for remotely monitoring and reporting the vital parameters of human body for urgent service delivery. In this context, there have been emerging proposals on using mobile, computing and wearable devices for analysis of health and emotional states of people in order to improve the health, well-being, social interactions and over all user experiences [2]. Two major technologies which have increasingly been integrated with the monitoring and diagnostics systems include Internet of Things (IoT) and Artificial Neural Networks (ANN). Body Area Network (BAN) or Wireless Body Area Network (WBAN) is a major domain of IoT which has been used for various e-health applications.
BAN or WBAN is defined as a wireless network of wearable mobile devices capable to collect physiological data from the human body and communicate the same to remote and/or mobile devices of healthcare service providers [3]. Recent advances in the area of BAN even propose to use smart clothing (e-textile) for collecting information about physiological parameters such as blood oxygen, temperature or electrocardiogram. ANN refers to a computation model that is designed based on biological neural network. ANN converges over time based on inputs and outputs and offers help in making optimal decisions for healthcare sector.
As described above, music therapy has not been automated as yet. There exists an opportunity to provide music therapy to the patients via mobile application which would reduce their anxiety level at minimum possible cost. Therefore, in this article, we propose a novel architecture to integrate the technique of music therapy with BAN and ANN. BAN would be deployed for sensing the body parameters; ANN will use this data for identifying the stress level or disease and would subsequently map the information to appropriate type of music which will be played on the patient’s hand-held device. Detailed Music Analytics will be performed by the ANN to identify the impact of music played over the patient’s physiological parameters and the choice of music shall be iteratively optimised. Finally, the information about patient’s condition shall be stored on the cloud platform and will also be transmitted to the medical service provider using BAN infrastructure.
To illustrate the proposed ANN enabled architecture, we developed a high level prototype in the form of an Android mobile app. In the present form of prototype, the test data (simulating human pulse rate, breath rate and blood pressure) has been generated via Node-Red and is transmitted to the Android app using MQTT Broker. The TensorFlow ANN running locally on the hand-held device has been designed to predict the user’s mood as either “Normal”, “Stressed” or “Exercising” in real time and suggest the music to be played. The Android app serves as an automated media player, changing the type of music played based on the user’s mood. Moreover, the app also allows the user to play music of his own choice in case his mood is detected as “normal”. Here, it is important to note that the developed Android app has the capability for replacing the existing media player applications that people generally have in their smart phones. To ensure this, we have added the music directories of patient preferred music, and music that should be played during the workout sessions, in addition to the music files specific to anxiety management. Source code for both the app and accompanying Node-Red/TensorFlow can be found on github at RoryNesbitt/MusicTherapy (https://github.com/RoryNesbitt/MusicTherapy).
In summary, the contributions of this article are as follows:
  • To conduct a detailed review of recent clinical studies to identify the impact of music therapy over patients of different diseases and to assess the role of BAN and ANN for the healthcare applications.
  • To propose IoT sensors for sensing and communicating diverse patient data to the remote stations.
  • To propose a novel IoT architecture for integration of BAN, ANN and music therapy to ensure timely delivery of music therapy in automated fashion.
  • To develop the ANN model to enhance the impact of automated music therapy by predicting the most appropriate type of music for each disease, customised based on the vitals of each patient.
  • To provide a working proof of concept that demonstrates how IoT, ANN and BAN can work together to provide an automated deliverance of Music Therapy in the form of an Android App.
The rest of this paper has been organised as follows: Section 2 describes the motivation behind developing an ANN enabled IoT architecture for music therapy; Section 3 presents a detailed review of music therapy techniques being used in the clinical settings and their impact; Section 4 describes the concepts and deployment of BAN for medical applications; Section 5 discusses the use of ANN for healthcare applications; Section 6 presents the proposed architecture integrating the music therapy technique with BAN and ANN; Section 7 details the design and high level prototype app developed to practically envision the proposed architecture; Section 8 describes some of the foreseen challenges for implementing the proposed architecture; Section 9 presents the discussion on state of the art of the developed prototype and the implications of such research and finally, Section 10 concludes the paper and offers the directions for future research.

2. Motivation

Although it has been proved by various past studies that music therapy, BAN and ANN, individually can provide numerous benefits to the patients and medical experts, to the best of our knowledge, no previous study has integrated these three techniques yet. For example, the authors in [4] developed an EEG-Based Brain Computer Interface to encourage participation of the patients in music-making process which could improve their effective state; however, no framework has been presented to automate the music therapy process for the patients of various diseases.
At present, there is no concept of automated music therapy at medical facilities. Mostly, the patients as an individual or group receive different forms of therapy at a certain time, regardless of their physiological status. Sometimes the patients gather at a dedicated room, where a certified therapist plays live music or facilitate the patients to sing or play the musical instruments themselves; otherwise, the therapy is arranged privately at the patient’s hospital room. In such settings, there is a need to hire certified music professionals, and the patients need to be physically present at the facility. However, there could be delays in providing the music therapy to patients or could be errors in detecting the most appropriate type of music for treating a specific disease as no measurements of stress levels are generally involved during these sessions. In addition, therapy is not customised in accordance with the patients’ needs as all the participants receive similar therapy regardless of the differences in the diseases and stress level among the group of patients.
Furthermore, the sustainability of using music therapy as a treatment option has remained hard to achieve. Many times, the patients receive music therapy while hospitalised but once they return home, they find no means of continuing this therapy. It is clearly not feasible for the majority of the patients to afford the dedicated service of music experts [5]. Therefore, if the music therapy can be integrated with emerging technologies of BAN, it would become possible to automate the delivery of this service and the patients will be enabled to get the treatment independently in a cost effective way.
In addition to proposing the use of BAN to automate the delivery of music therapy to the patients, the use of ANN has also been suggested in this paper. ANN offers the healthcare sector numerous benefits due to the training facilities it offers: the neural networks can learn from historical data of vital signs and can predict the future health state of patients. Based on this feature of ANN, it has been proposed in this article that the history of patient’s vitals should be taken as an input for the ANN; the ANN would generate output to identify whether the patient is “stressed”. This ANN output will then be used to play the music of most appropriate type. Initially, the selection of music shall be decided based on the findings of previous studies. Over time, the ANN will learn the impact of music played over the patient’s stress level and will develop the ability to predict the most appropriate music type and duration in accordance with the specific requirements of each patient. Hence, BAN in integration with ANN would ensure quality delivery of music therapy to the patients.

3. Studies on Impact of Music Therapy

This section briefly reviews the results of recent clinical studies reported by the reputed medical journals for the impact of music therapy on patients.

3.1. Impact on Mental Stress

The impact of nature sounds in combination with sedative music has been evaluated in [6] by using a music biofeedback system for healthy adults. The biofeedback is a mind-body technique where users learn how they can control their specific body processes to improve their emotional, mental and physical health. In this system, biofeedback instruments are used to collect data of vital parameters such as heart rate, respiration rate and blood pressure. In [6], a biofeedback system was used during relaxation exercises where nature sounds were aimed at responding to the physiological data of the users and music was targeted to induce calmness and to evoke positive emotions. The experiments were conducted by evaluating the stress level of participants after they had experienced some stressful activity. It was found that when nature sounds, sedative music and biofeedback systems are used in integration, the heart rate variability of the users improves.

3.2. Impact on Postsurgery Patients

Various studies have been conducted to identify the role of music therapy for the patients receiving postoperative care followed by open heart surgery. The study in [7] was administered by dividing 60 patients into control and experimental groups, who had been moved to the intensive care unit followed by the open-heart surgery. The patients from the experimental group were provided with headphones to listen to nonverbal music after 30 min of surgery, whereas no intervention was provided to the control group. The music played during the intervention was sedative which did not have any lyrics and had a sustained melodic quality. Furthermore, the rate of music was 60–80 beats per minute and it did not have percussion or strong lyrics. The music was selected by an expert considering the cultural aspects of the patients’ society.
The physiological parameters, oxygen saturation, respiratory rate, pulse rate, systolic and diastolic blood pressure of patients from both groups were measured. It was found that the experimental group had significant differences in the pulse rate, oxygen saturation and systolic blood pressure. Therefore, listening to favourite music postsurgery, the pain of patients may reduce.

3.3. Impact on Cancer Patients

Electroencephalography (EEG) data has been used in [8] to evaluate the role of music therapy for the emotional state of cancer patients at a palliative care facility. The study targeted assessing the emotional attributes of patients including anxiety, anger, stress and depression. The patient preferred music was played during the therapy sessions which belonged to both the categories of vocal and instrumental and also both genres of classic and popular were included.
The emotional response of the participants was collected to compare the impact of participating in a music therapy session versus being provided with company. It was found that the patients who participated in music therapy were found to be emotionally relaxed, whereas there was no change in the emotional state of the patients who received company for the same duration. Hence, it has come to the light that the music therapy can provide emotional support to the cancer patients.

3.4. Impact on Dental Anxiety

It has been shown by several studies that dental anxiety can be reduced by the use of music therapy techniques, as detailed in Table 1. A pilot study was conducted [9] to observe the impact of listening music over patients undergoing Minor Oral Surgery (MOS). Patients were provided with earphones and instrumental music was played during the oral treatment. The vital sign (heart rate) was measured and it was found that the heart rate reduced. It has been therefore found that music therapy can reduce the hindrances for dental care and management.

3.5. Affecting Patients of Insomnia

In a recent study [10], it was identified that Chinese Calligraphy handwriting (CCH) and traditional Chinese Guqin music may help the patients to deal with primary insomnia. The study subjects were classified as control group, Guqin group and calligraphy group. The control group remained in rest condition, whereas Guqin group received Chinese music therapy and the calligraphy group did calligraphy writing. Heart rate variability (HRV) and frontal midline (FZ) electroencephalographic signals were recorded for the subjects during the period of intervention. Heart rate of the patients from Guqin group was found to reduce significantly after intervention. FZ δ , FZ θ and FZ α waves showed enhanced effect in Guqin and Calligraphy group. The Calligraphy group also showed better performance as compared to the control group. Therefore, it was found that when the Chinese music is coupled with calligraphy, it provides the best results in terms of improving the mental and physical health of patients.

3.6. Effects on Cardiac Health

Music therapy intervention was provided to healthy Indian males for 30 days to assess its impact over cardiac health [1]. Eighty participants were divided into experimental and control groups. The music therapy intervention was based on Raga Todi of Indian Classical Music, and a 20-min long instrumental session was provided to the patients from experimental group; the control group remained at rest and did not receive any therapy. The impact of this intervention was monitored by comparing the blood pressure, heart rate and pulse rate of control and experimental groups. It was found that all the parameters measured showed a significant reduction at the end of intervention period for the experimental group. Therefore, it is concluded that Raga Todi of Indian Classical Music has the capability to positively impact cardiac health.

3.7. Impact for Study and Exercise

Although the main focus of music therapy is for emotional well-being, it has been stated in a recent study that music can also be a positive influence when performing a task such as study or exercise [11]. The authors concluded that an emotion-based recommendation system would have positive benefits; however, it is important that it takes into account the personal preference of the person it is recommending to, such as their energy level and situational context. Similarly, another study [12] reported that music therapy could play a positive role for improving social skills and emotional expressions, resolving psychological conflicts and enhancing the impulse control and motor function. Furthermore, music therapy also has a potential to improve cognitive functions such as memory and attention.

3.8. Types of Music and IoT Sensors for Diverse Diseases

In light of the review conducted, it has been identified that various types of music play a positive role for treating specific diseases. These findings have been summarised in Table 1 including the types of music and body parameters used to monitor the impact of music on patient. Furthermore, we also propose the potential sensors which can be integrated with BAN for monitoring each of the clinical scenario which the past studies have analysed. Moreover, we have suggested these sensors that are in line with existing BAN applications [3]. In addition, a smart phone shall be used wherever possible, for example, for monitoring sleep time. In addition, we have suggested to use Electrochemical Cortisol Sensor developed by [13] for detection of Cortisol which is a hormone used for detecting stress in the human body. The Cortisol sensor has not been integrated with BAN in the past and this would be an interesting direction of future research to integrate it with the BAN.
As seen from Table 1, different variations of music such as instrumental, vocal, popular, classical, nature sounds and religious recitations have been used to improve the recovery or relaxation rate of the patients. However, it is to be noted that for most of these studies, music experts selected the music which was culturally appropriate or preferred by the population groups. In addition, in some cases, the patients participated in the process of selecting music.

4. Body Area Networks in Medical Environment

Using BAN for healthcare applications has been receiving huge attention of healthcare practitioners and researchers alike. It has been observed by various studies that using BAN can significantly improve the quality of life as well as life expectancy of the patients through offering remote monitoring facilities. Some applications of BAN include monitoring body temperature, heart rate, physical activity level and to detect falls [3]. To support the BAN, other technologies such as mobile application development (to notify the users) and energy harvesting (to provide continuous power to the wearable devices) have been integrated to further optimise its functionality.
One of the recent proposals on using BAN for healthcare has been made in [44] with a focus on employee safety at the industrial workplace environment. A simple architecture has been developed by the authors where the technologies of WBAN, Low Power Wide Area Network (LPWAN) and IoT have been used for continuous monitoring and reporting of environmental parameters such as carbon dioxide presence, ultraviolet radiation, ambient temperature and relative humidity and physiological parameters (body temperature and heart rate) of the workers. The proposed BAN architecture can play a key role for improving the health and safety of industrial employees who often work in hazardous environments.
Prioritising data generated from WBAN has been proposed in [45] to efficiently deal with the emergency situations. The authors proposed a disease-centric health-care management system by integrating WBAN with multiple health-cloud service providers (H-CSP). The coexisting WBANs are categorised into different Disease-centric Patient Groups (DPGs) based on the level of urgency. It has been found that the proposed system can improve the throughput, delay and cost-effectiveness while guaranteeing the critical communication during emergency situations.

5. Artificial Neural Networks for Healthcare Applications

ANN has increasingly been used for various healthcare services such as diagnosis and informed decision making [46]. Some of the studies have also proposed to use ANN in combination with the wearable devices or BAN to improve the health and life quality of the users. In [47], Recurrent Neural Network (RNN) has been used for predicting the user’s level of physical activity based on the parameters of temperature, humidity, average activity level, sleep hours, mean heart rate, asthma and lung disease probability index, fine dust in the environment, distance traveled during the activity, cadence and skin disease probability index. Since all of the parameters selected by authors continuously keep on changing, the model developed predicts what level of physical activity the user will need each day.
Similarly, in [46], ANNs have been used for modelling the relationship between human body temperature, the skin temperature and the temperature of wearable noninvasive temperature monitoring device. The model has been successful to reduce the reaction time as compared to when only wearable temperature sensor was used. In another study [48], the authors used ANN to identify the presence of lung cancer in the human body. A range of patient’s symptoms have been used as input to the neural network including yellow fingers, anxiety, coughing, wheezing, fatigue, chronic diseases, shortness of breath, chest pain, difficulty in swallowing and allergy. The proposed ANN model was able to detect lung cancer with an accuracy of 96.7%.

6. ANN-Enabled IoT Architecture

This section presents the novel architecture for integrating the music therapy with BAN and ANN as graphically illustrated by Figure 1. Furthermore, the details of ANN architecture have been presented in Figure 2.
Figure 1 shows abstract view of the proposed architecture where the body parameters of patient are collected via wearable or implanted sensors forming the core BAN. These sensors continuously monitor the required physiological parameters such as heart rate, blood pressure, muscle movement, thoracic movement, brain activity, blood oxygen saturation and temperature. Any one or combination of these parameters may be monitored based on the requirement of the patients and suggestion of the medical experts. The threshold level for each patient would be decided based on various parameters such as age, past disease history, physical fitness level, etc. Predefined threshold values will be stored with the BAN cocoordinator (hand-held device/gateway node) and until the value remains below threshold, the sensors would continue to periodically monitor the data. As soon as a value above threshold is detected, the input shall be provided to the ANN architecture maintained over cloud which would in turn suggest the music to be played. The detailed working of ANN architecture will be described later in this section.
Subsequently, as indicated by Figure 1, wherever appropriate, an emergency alert or data log would be sent to the medical service provider over Internet, to ensure quick service delivery or contacting patients when necessary. The physicians may also be given a chance to supersede the decisions made by ANN by recommending music type other than what was suggested by ANN. The database would be maintained at the medical server for later access by experts or patients.
Furthermore, it is expected that ANN would converge over time by taking feedback from the BAN and performing detailed Music Analytics. This feedback will involve information about the impact of playing previously suggested music for the patient; in this context, the BAN will send details about the patient’s vital parameters followed by the playing of music. Based on this feedback, ANN would optimise its decision for future, focusing on the aspects such as genre/type of music to be played and the playing duration. As a result of this optimisation, it is expected that the correlation between disease/patient condition and the choice of music will even improve.

6.1. ANN Architecture

An Artificial Neural Network, such as proposed in this article, is made from several layers or neurons, a loss function and an optimiser. The proposed working of each component in ANN illustrated by Figure 2 is described as below:
Input Layer: This layer accepts the input from the BAN in the form of parameter sensed and forwards to the hidden layer.
Hidden Layer: This layer is divided into two sublayers: “Diseases” and “music”. The parameter is mapped to one of the most appropriate diseases and subsequently to the most appropriate type of music type/song. The identified music is provided to the output layer.
Output Layer: This layer provides the name of the music files to be played as an output to the BAN. Finally, the BAN coordinator will play the music available on the handheld device.
As seen from above, input data is sent to the first layer of ANN, from which it is sent through hidden layers and finally ends with a predicted output. During this process, the loss function provides the network with a path it needs to take in order to improve its knowledge of the input data, and the optimiser improves this by changing the weights of each neuron and then trying again to get a better prediction. Changing weights alters the extent to which the input neurons affect the final result; this implies that some parts of the input data may have a higher impact over output variables as compared to others.

6.2. ANN Loss Functions

The loss functions in ANN are used to compute the quantity that a model should seek to minimise during training. Binary Cross Entropy model used in this work performs binary classification tasks [49]. We initially used this model to identify whether the person has anxiety or not. Furthermore, when the multiclassification problem was solved where we needed to identify whether the person’s mood is normal, stressed or whether they are doing exercise, the Categorical Cross Entropy model was used since there were more than 2 classes [50]. The following equation has been used to compute the categorical cross entropy loss function, taken from [50]:
L o s s = i = 1 o u t p u t s i z e y i · log y ^ i
where y ^ i is the ith value in the scalar output of moods array and y i is the target value of mood, which has been set "normal" since we wanted to play the music such that the mood can return to normal. The size of output will be the number of values in the output model which is 3 for our trained model.
Furthermore, the binary cross entropy model is computed as the average of (1), as given below [49]:
L o s s = 1 o u t p u t s i z e i = 1 o u t p u t s i z e y i · log y ^ i + ( 1 y i ) · log ( 1 y ^ i )

6.3. ANN Optimiser

Optimisers in machine learning are used to update the attributes of neural network such as learning rate and weights to minimise the losses [51]. In this work, the Adaptive Moment Estimation (Adam) optimiser has been used to minimise the difference between the predicted mood and collected mood data. It updated the weights of the training data iteratively. Adam has been used because of the advantages it offers including straightforward and simplistic implementation, high computation efficiency, little memory requirements and adaptive learning rate for each weight. Adam works on the logic of reaching minimum slowly instead of jumping instantaneously to it. Therefore, the momentums of first and second order of the loss function are used, where Adam exponentially decays the average of past gradients. The parameter of momentum is used for reducing the high variance and softening the convergence. The equations for first moment m ^ t , and second moment v ^ t are given in (3) and (4) [51]:
m ^ t = m t 1 β 1 t
v ^ t = v t 1 β 2 t
where, m t and v t are moment vectors of first & second order and β 1 & β 2 are the exponential decay rates with the values, β 1 = 0.9 , and β 2 = 0.999 .
Here, (3) provides mean and (4) presents uncentered variance of the gradient for the loss function detailed earlier in (1) and (2).
Finally, the parameter (predicted mood in our case) is updated using (5) with θ t as the initial parameter vector and θ t + 1 as the resulting parameter vector:
θ t + 1 = θ t η v ^ t + ϵ m ^ t
where, η = learning rate, and ϵ = smoothing term to avoid division by zero = 10 8 .

6.4. ANN Activation Function

The activation function Rectified Linear activation function (ReLU) was used to translate the summed weighted inputs presented above, into outputs. ReLU has recently become a default function of may types of ANN due to the simplicity of training process and capacity to achieve better performance. ReLU is a nonlinear function that maps the same input to the output if input is above 0 and outputs 0 otherwise. Mathematically, ReLU is stated as [52]:
f ( x ) = m a x ( x , 0 )
For our system, the ReLU was used to compare the previous mood data with the currently collected one; this information was then used to inform the media player whether to continue the same music or if there is a need to change it.

7. Implementation Details

As a major contribution of this article, we developed an Android mobile application to illustrate the proposed ANN enabled architecture for music therapy. Human mood data was simulated to predict the levels of anxiety and the appropriate music was played accordingly. Each phase of implementation has been explained in this section.

7.1. Design of Android App

The Android application has been designed to have a simple yet effective interface. The app does not require any user intervention and would play the music in case stress is detected. On the other hand, the patient may also use the application as a usual music player, only when stress is not detected. The major screens designed for the prototype app including home, dashboard and controls are illustrated in Figure 3.
The home screen just shows a welcome message and the icon. The dashboard screen is set up to show all the vital data collected from the patient as well as the health state based on this data. Initially, the design included brain activity, but later it was changed to the breath rate considering the challenges associated with collecting the EEG data. The final dashboard design has been shown in Figure 4, where the vitals are displayed until the dashboard remains connected to the patient’s device and shows the status “disconnected” otherwise.
The third screen of app is for controlling the music. It displays the name of the song at the top with controls to mute the music or skip the song. There is no option to play the stopped song as the user’s mood may have changed since it was paused, meaning that particular track is no longer relevant to them. By forcing a skip, it allows for the app to check the current mood and rebuild the playlist if it has changed.

7.2. Generating Test Data

For developing the prototype, the automatic sensing from BAN has not been considered for the sake of avoiding human subjects and relevant ethical concerns. Hence, the first step of implementing the proof of concept app was to create appropriate test data, which could resemble the human vitals. We generated test data using Node-Red for three basic parameters heart rate, blood pressure and breathing rate. As already mentioned in Section 3, these parameters have widely been used by the clinical researchers for identifying the presence of anxiety or stress.
Although Music therapy can be useful for dealing with numerous medical conditions, we generated test data only to represent 3 states of patients; i.e., “normal”, “stressed” and “exercising”. The “normal” state represented the healthy status of the patient where he did not need the music therapy but he could still use the application for playing music, just as a usual media player; the “stressed” state is the major one as it indicates the anxiety and the proposed system has actually been designed to detect anxiety and play the appropriate type of music; finally, the “exercising” state represents that the patient is engaged in physical activity so the system could play the music which could support him while working out.
Here, it is important to note that each patient will have individual ranges for the physical parameters, representing the three states. These differences are often influenced by the demographic attributes such as age, physical fitness level, metabolic rate, etc. This implies that for the final system, ANN will be trained for each patient so that the customised music therapy solution could be developed according to the specific requirements of each patient. We considered average values of Diastolic Blood Pressure, Systolic Blood Pressure, heart rate and breath rate of healthy adult to generate the test data. The ranges used for defining each state based on the values of these physical parameters have been presented in Table 2.
Five thousand instances were generated for each state from which 80 percent data was used for training and the remaining 20 percent was used for validation/testing. The training data was then split again at a rate of 0.3, implying that of the remaining 80 percent data, 70 percent was used for training and 30 percent for testing. This proportion of training and testing data has been found to be efficient by the previous researchers working on ANN [53].
Different optimisers and activation functions were used to conduct the tests and study the variations in accuracy. It was observed that different activation functions did not affect the number of epochs required for reaching the maximum accuracy; however, the use of different optimisers influenced the number of epochs. Based on these tests, Adam and ReLU were used as they provided the fastest convergence. Initially, the system was able to achieve perfect accuracy at 10 epochs with 50 steps per each epoch. However, repeating experiments results in inconsistent performance as for some observations, 100 percent accuracy was even achieved for the fourth epoch. Clearly, the ANN could not be trained for this low number in order to avoid overfitting and also to maintain sustainability of the system in the longer run. When overfitting happened, the data was imported again, reshuffled and split to retrain, test and validate once again. Finally, 20 epochs were used for training as it was found that 100 percent accuracy was possible to achieve within 19 epochs (when not overfitting).

7.3. ANN Functionality

In the developed neural network, there was one input layer, one hidden layer and one output layer. The hidden layer had 10 neurons and the output layer also had one neuron to depict whether the patient has stress or not (represented by 0 and 1). Colaboratory, often called Google Colab, was used to create the neural network, as it offers good integration with TensorFlow and also allows for the user to access Google’s hardware such as GPU power through their web browser. This implies that using Colab, the network can be trained quickly from any device even if it does not have high processing power itself.
The loss function of binary cross entropy was used to predict the possibility of a positive result, in this case 1 represented the state "Stressed" for the patient. The purpose of ANN was to predict the most appropriate song based on the patient’s stress level (detected from the mood data) and to update the song being played based on the continuous monitoring of patient’s stress level. Hence, the goal was to minimise the difference between the predicted mood and current mood, as the song selection was based on the predicted mood and it must match with the actual mood data collected from the patient (from Node-Red for the prototype). The optimiser Adam and the activation function ReLU were used as described earlier in Section 6.3.
It is important to note that the binary cross entropy algorithm was initially used for training the ANN and computing the loss function for predicting the mood only. However, since more than two values were required at the output for treating a wide number of illnesses and emotional states, the limitation was overcome by adding a new section to Node-Red. Later, the binary cross entropy was modified to sparse categorical cross entropy so that more outputs can be obtained from the output layer of the implemented ANN. This allowed the system to enhance such that it could not only detect the mood but could also include details about physical activity being performed. However, with the larger neurons at output, the number of neurons and layers had to be adjusted to achieve 100 percent accuracy. It was found that sticking to one layer and increasing to 100 neurons or adding a second layer with both having 50 neurons realised the accuracy of 100 percent.

7.4. Data Communication and Management

Message Queuing Telemetry Transport (MQTT) broker was used to communicate the data between Node-Red and the Android app. This broker is hosted within the Node-Red and picked up with the Paho MQTT library for Android. The ANN was created with TensorFlow (using Keras API) and trained by saving test data with expected result over a period of time. Subsequently, the ANN became able to receive new input data and calculate the mood value, so that it can be included within the Android app to run locally.
Initially, the TensorFlow model was saved and run through TFLiteConvertor which saved the file as model .tflite. This file was downloaded and added to the assets directory in the Android project folder so that it could be used by the app; the structure of assets file has been shown in Figure 5. An ANN class was created to provide access to the Neural Network. On start-up it populated an array of mood names, so that the results from the tflite file can be displayed correctly to the user. The main function of ANN was however predict() which was called from the MQTT class whenever a message was received. This function loads the asset information for model .tflite as a MappedByteBuffer and sets it to the variable tflite. Subsequently, predict() inputs each of the tracked values as an array and gets the predicted value as the output. Since binary cross entropy was in use initially, the prediction is based on the likelihood of a 1 being the output, therefore result was run through the parsePrediction() function which returned an int value of 1 for a positive prediction and 0 for the negative. This number could then be saved as mood for use throughout the app.

7.5. Managing Music

Using the idea of expandability, the music used in the app was moved from the raw folder to assets/music, shown in Figure 5. Doing this allows to have subdirectories, unlike raw; therefore, the asset manager can read the contents of the directory, and the music can be dynamically read and added to the master playlist. Numbered sub directories were added to the music directory, each relating to the moods that the app can predict. Each of these directories were read and every track was added to the master playlist with the number of the directory as its mood value. The music files are named in the format of trackName.artist.extension allowing for the split command to be used.
With the inclusion of dynamic music building, the user choice was also included within each mood directory. For this purpose, Set 0 of directories was added to represent the choices. However, the users will be allowed to play the music of their own choice only when they are in a “normal” mood where they could choose from different genres of music. On the other hand, when the patient is under stress, the two allowed types shall include classical and ambient nature sounds, including in Set 1. These selections have been made in line with the previous research, where it has been found that classical music [54,55], particularly Mozart [56] can relieve stress and also, it has been shown that ambient nature sound can also have a calming effect on a person [57,58,59], due to which we made it the second choice. Finally, Set 2 in the file structure shown in Figure 5 has been included to facilitate the patient while performing exercise. According to the literature, fast paced music serves best during exercise, as noted by [60,61]; therefore, we chose electronics as one of the many genres. In addition, game themes have been selected. The games often create music that encourages the player to focus and increases heart rate to enhance the gaming experience. We have similar requirements when assisting a patient performing exercise, which implies that gaming music is expected to have a positive effect [62].

7.6. Consolidated Operation

Figure 6 presents the functionality of developed prototype, through demonstrating the connections between all the components. It has been shown that Android app worked based on the combination of receiving MQTT messages, running the ANN and controlling the media player to provide a smooth user experience for the patient. The three sections work independently and make calls to one another in order to maintain a seamless transition between the use of different music types.
The beginning of operation of Android App has been marked by the reception of an MQTT message (centre column of the flowchart depicted by Figure 6). First, the app confirms if it has received a relevant message, if not the message is discarded and otherwise, the relevant variable is updated. If an update has been made recently, the information is communicated to the ANN to make a new prediction for the current mood, as seen in the left portion of Figure 6. The ANN file when called, builds an input array using the values received from MQTT network. The ANN that processes these values and creates a new array of predictions as output. The app chooses the largest value of output array, as it indicates the most accurate predicted value. Once the ANN has made its prediction, the new data is saved and compared to the mood value for the current playing track. If the obtained value from the MQTT message and the one associated with the track being played are observed to be mismatched for more than 5 min, the skip function runs to deselect the track.
Finally, the right portion of the flowchart (Figure 6) illustrates functionality of the media player. This player runs independent of the other java files in the app; however, it uses the value of mood to build the playlist. A master playlist is generated dynamically from songs which are in the assets directory. These songs are assigned a mood and type value based on the directory structure and the name and artist is obtained from the filename. From this, the media player creates a working playlist with only songs for which the mood value matches the current mood value. If a user preference is also set, then songs for which the type matches with the user preference is selected. The media player then continues to play a track from this playlist and makes the check again; each time the song changes.

8. Expected Implementation Challenges

Due to the lack of music therapy practice in clinical settings and awareness in the general public, there would be several challenges when integrating this technique of alternative medicine with BAN and ANN. First, there may be issues with the use of BAN sensors for each patient due to medical conditions, personal perceptions and beliefs and more importantly, the ethical concerns that relate to patient’s confidentiality and anonymity. The patients as well as clinical experts may be reluctant to deploy the automated system as they may not believe in the efficiency of this therapy in the first place and they may also have questions regarding the usage of data which will be regularly transmitted to the base station. Therefore, all the stakeholders including the patient, caregiver/attendant and physicians must consent for the use of proposed music therapy prototype.
Second, it would be a challenge to develop a generic music sublayer in ANN due to differences in the music preferences of the patients. Third, the ANN algorithm needs to be smart enough to map the identified patient’s parameter with the appropriate type of music. Finally, there may not be a uniform impact of playing same music for different patients suffering from the same disease; hence, there would be a strong need to customise the system for each patient.

9. Discussions and Implications

The proposed ANN enabled architecture in the form of a mobile app shall facilitate the patients and therapist in a large number of clinical scenarios. As seen earlier in Table 1, there has been a long list of diseases and medical conditions where music therapy has already proved to bring positive patient outcomes. However, the findings of previous studies could never be realised at large scale until present due to the heavy resources that are required to provide the therapy. Due to the increasing usage of smart phone applications today, various healthcare services such as remote health monitoring, blood glucose level tracking, fitness and nutrition assistance, weight management and many others have already become a reality. Therefore, there remains a gap in the use of smart phones for delivering the alternative medicine services. Thus, based on the analysis of literature and existing prevalence of anxiety among the patients throughout the world, we proposed to deliver the service of music therapy using smart phone application.
We proposed an end-to-end architecture for providing the music therapy service to the patients on their smart phones. The architecture supports conventional remote health monitoring and management services in addition to the novel contribution of offering automated music therapy. The BAN sensors are easily available in the market and can easily be deployed to realise the proposed system. Similarly, ANN has already been used for various healthcare applications and could easily be modified for predicting the patient’s state and music which could have a positive impact over anxiety. Finally, the populations today already have familiarity with the Android applications and people from all age groups heavily use smart phones. Hence, the patients from different population sectors and age groups including the elderly being treated at care homes, all shall be able to benefit from the proposed system.
To illustrate the architecture developed in this work, we developed an Android app to realize the integration of different technologies envisioned in Figure 1. The first step of the architecture is the collection of data from the patient. To avoid the challenges associated with data collection from real patients, we used test data generated by Node-Red for developing the prototype; the data which could provide the insight into the state of patients (whether suffering from anxiety) was generated using the standard values of heart rate, blood pressure and breath rate. Keras API for Tensorflow was used for implementing the ANN, and MQTT server was used for communicating the data from Node-Red to the Android App. Three categories of music directories have been used to contain the music to be used by the patient or by the application. Hence, all the cutting-edge existing tools have been used to illustrate the functionality of automated music therapy system.
Based on the data collected from Node-Red and prediction made by ANN, a song/sound track is played and its impact is observed on the patient’s state. The fundamental goal of the system is to predict the patient’s state based on the vitals and play the appropriate music which could reduce the anxiety level. Another interesting point to note about the smart phone app developed in this work is that its use would not only be limited for delivering the music therapy services. Rather, it can also be used as a simple media player application where people can play music for leisure or during their workouts. This strategy would ensure that the patients are not burdened for having yet another mobile application and all their music needs can be fulfilled simply by using a single app.
In its present form, the developed prototype can be used for conducting the clinical trials. We will just need to replace the Node-Red data generation module with wearable device which could capture the vitals data of patients. Moreover, the categories of music (music sublayer) will be designed based on the further research about the impact of each music type on the patients suffering from specific diseases. The other components of system including MQTT broker and Tensorflow can still be used to convey the data to the Android application. The remote health monitoring service which has not been implemented in the present work will also be included by procuring services of Cloud infrastructure such as Google Cloud or Amazon AWS.
It is expected that the proposed architecture would aid the patients suffering from diverse range of diseases/medical conditions. The need for conducting physical and dedicated music therapy sessions will reduce, which would provide patients with more independence. In addition, since the anxiety of patients can be remotely managed simply through the use of advanced technologies, the healthcare resources used for dealing with anxiety related issues at present, would be significantly reduced. As a result, the healthcare funds for the patients, insurance providers and state all could be more strategically used for dealing with more serious diseases. In addition to the monetary issues, there could also be certain other factors which could act as a barrier for the provision of music therapy to patients in the present settings; for example, the patients could not move due to being elderly or physically impaired, the music therapists may not be available at the locations or the physical music therapy may not be allowed by the government due to religious/cultural factors. Moreover, the ongoing pandemic and the advisory of social distancing associated with it has also discouraged the clinical facilities from providing the elective or alternative medicine services. In all such situations, the patients could use automated music therapy mobile application simply from their home without any assistance of music expert, attendant or even physicians.

10. Conclusions

It has been identified by a wide range of recent clinical studies that music therapy has a strong tendency to bring improvements to the health of patients, particularly to mental health. However, the use of this therapeutic technique has been limited due to lack of awareness and trust of the patients and physicians. Furthermore, in the present scenario, the provision of music therapy services to the patients requires arranging physical sessions delivered by the music experts. Clearly, due to the requirement of extensive resources, it is not possible for the clinical facilities to provide regular music therapy to the patients, and also patients could not afford to attend these sessions. The immobility of patients, unavailability of music experts, high cost of therapy, and social distancing (at present) also contribute to limiting the usage of music therapy in the clinical settings.
To combat the above mentioned challenges, this article presented a novel architecture to integrate the advanced technologies of BAN and ANN with music therapy to facilitate automated delivery of this service. An Android App has been developed to illustrate the envisioned prototype in this work. Major components of the app include collection of data from the BAN sensors (achieved through dummy generated data) about the patients’ vitals and ANN’s prediction of patient’s mood and the type of music which matches best with the anxiety level. The music file suggested by ANN is played on media player via Android application on the patient’s handheld device. ANN has also been proposed to perform Music Analytics to identify the music type, duration and impact on patient’s parameters to improve the future decisions. Using the developed prototype, the music therapy service could be extended to large population sizes as it would not require any dedicated resources and music will just play on the smart phone application, based on the anxiety status of patients.
In future, the present work will be extended by collecting real patient’s data and including the remote health monitoring and management facilities. In this regard, we will need to take ethical approvals from the host universities and the clinical facilities. Initially, the facilities which already offer music therapy to their patients will be contacted to conduct clinical trials. The learning process of ANN will be optimised using the real data. The music files for each patient will be selected based on the patient’s customised preferences and the supporting literature detailing the type suitable for each disease. Thus, an interesting direction of future work would be to monitor the impact of playing different types of music such as vocal and instrumental pieces, sounds of nature and religious recitations for diverse range of diseases. Moreover, we plan to extend the high-level prototype developed in this system by transmitting the data in real-time to the cloud so that the patient’s information can timely be communicated to the physician. At present, the decision is locally taken and the music is played on the smart phone of patient, without informing the physicians. In future, another application interface will be designed to keep the physicians updated about the anxiety status of each patient. Using this information, the physicians will be able to contact the patients if they need to provide some immediate assistance. Furthermore, the physicians will also be provided with the option to suggest music for the patient, if required.

Author Contributions

S.S. proposed the major idea about developing an ANN-enabled architecture and wrote the major sections of this paper. R.N. developed the software prototype. M.Z.S. and N.R. supervised the work. A.A.K. (Anwar Ahmed Khan) developed the diagrams and end-to-end architecture. A.A.K. (Ausaf Ahmed Khan) assisted in the literature survey of clinical studies and K.K.K. suggested the sensors to be used for each clinical application. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Chatterjee, S.; Mukherjee, R. Evaluation of the Effects of Music Therapy Using Todi Raga of Hindustani Classical Music on Blood Pressure, Pulse Rate and Respiratory Rate of Healthy Elderly Men. J. Sci. Res. 2020, 64, 159–166. [Google Scholar] [CrossRef]
  2. Shu, J.; Chiu, M.; Hui, P. Emotion Sensing for Mobile Computing. IEEE Commun. Mag. 2019, 57, 84–90. [Google Scholar] [CrossRef]
  3. Chen, M.; Zhang, Y.; Qiu, M.; Guizani, N.; Hao, Y. SPHA: Smart Personal Health Advisor Based on Deep Analytics. IEEE Commun. Mag. 2018, 56, 164–169. [Google Scholar] [CrossRef]
  4. Daly, I.; Williams, D.; Malik, A.; Weaver, J.; Kirke, A.; Hwang, F.; Miranda, E.R.; Nasuto, S. Personalised, Multi-Modal, Affective State Detection for Hybrid Brain-Computer Music Interfacing. IEEE Trans. Affect. Comput. 2020, 11, 111–124. [Google Scholar] [CrossRef] [Green Version]
  5. Wilhelm, K.; Knight, A. Music Therapy Private Practice in the United States: Updated Financial and Marketing Recommendations. J. Music. Ther. 2020, tha011. [Google Scholar] [CrossRef]
  6. Yu, B.; Funk, M.; Hu, J.; Feijs, L.M.G. Unwind: A musical biofeedback for relaxation assistance. Behav. Inf. Technol. 2018, 37, 800–814. [Google Scholar] [CrossRef] [Green Version]
  7. Ajorpaz, N.M.; Mohammadi, A.; Najaran, H.; Khazaei, S. Effect of music on postoperative physiological parameters in patients under open heart surgery. J. Res. Health 2019, 9, 195–202. [Google Scholar]
  8. Ramírez, R.; Planas, J.; Escude, N.; Mercade, J.; Farriols, C. EEG-Based Analysis of the Emotional Effect of Music Therapy on Palliative Care Cancer Patients. Front. Psychol. 2018, 9, 254. [Google Scholar] [CrossRef] [Green Version]
  9. Gupta, A.; Ahmed, B. Experience of listening to music on patient anxiety during minor oral surgery procedures: A pilot study. Br. Dent. J. 2020, 228, 89–92. [Google Scholar] [CrossRef]
  10. Fung, M.Y.; Kao, H.R.; Lam, S.W.; Kao, T. Chinese guqin music and calligraphy for treating symptoms of primary insomnia. Chin. Med. Cult. 2019, 2, 48. [Google Scholar] [CrossRef]
  11. Rumiantcev, M.; Khriyenko, O. Emotion Based Music Recommendation System. In Proceedings of the Conference of Open Innovations Association FRUCT, Yaroslavl, Russia, 23–25 April 2020. [Google Scholar]
  12. Elliott, D.J.; Silverman, M.; McPherson, G.E. The Oxford Handbook of Philosophical and Qualitative Assessment in Music Education; Oxford University Press: Oxford, UK, 2019. [Google Scholar]
  13. Dhull, N.; Kaur, G.; Gupta, V.; Tomar, M. Highly sensitive and non-invasive electrochemical immunosensor for salivary cortisol detection. Sens. Actuators B Chem. 2019, 293, 281–288. [Google Scholar] [CrossRef]
  14. Akelma, F.K.; Altınsoy, S.; Arslan, M.T.; Ergil, J. Effect of favorite music on postoperative anxiety and pain. Anaesthesist 2020, 69, 198–204. [Google Scholar] [CrossRef] [PubMed]
  15. Kahna, M.; Belgat, W.G. The Contribution of Music Therapy to the Operating Room: A Randomized Control Study. In MAR Case Reports; Medical and Research Publications: Stoke-on-Trent, UK, 2020. [Google Scholar]
  16. Teckenberg-Jansson, P.; Turunen, S.; Pölkki, T.; Lauri-Haikala, M.-J.; Lipsanen, J.; Henelius, A.; Aitokallio-Tallberg, A.; Pakarinen, S.; Leinikka, M.; Huotilainen, M. Effects of live music therapy on heart rate variability and self-reported stress and anxiety among hospitalized pregnant women: A randomized controlled trial. Nord. J. Music. Ther. 2019, 28, 7–26. [Google Scholar] [CrossRef] [Green Version]
  17. Çetinkaya, F.; Aşiret, G.D.; Yilmaz, C.K.; İnci, S. Effect of listening to music on anxiety and physiological parameters during coronary angiography: A randomized clinical trial. Eur. J. Integr. Med. 2018, 23, 37–42. [Google Scholar] [CrossRef]
  18. Poquérusse, J.; Azhari, A.; Setoh, P.; Cainelli, S.; Ripoli, C.; Venuti, P.; Esposito, G. Salivary α-amylase as a marker of stress reduction in individuals with intellectual disability and autism in response to occupational and music therapy. J. Intellect. Disabil. Res. 2018, 62, 156–163. [Google Scholar] [CrossRef] [PubMed]
  19. Chennafi, M.; Khan, M.A.; Li, G.; Lian, Y.; Wang, G. Study of Music Effect on Mental Stress Relief Based on Heart Rate Variability. In Proceedings of the IEEE Asia Pacific Conference on Circuits and Systems (APCCAS), Chengdu, China, 26–30 October 2018. [Google Scholar]
  20. Luis, M.; Doss, R.; Zayed, B.; Yacoub, M. Effect of live oud music on physiological and psychological parameters in patients undergoing cardiac surgery. Glob. Cardiol. Sci. Pract. 2019, 2019, e201917. [Google Scholar] [CrossRef] [Green Version]
  21. Kahloul, M.; Mhamdi, S.; Nakhli, M.S.; Sfeyhi, A.N.; Azzaza, M.; Chaouch, A.; Naija, W. Effects of music therapy under general anesthesia in patients undergoing abdominal surgery. Libyan J. Med. 2017, 12. [Google Scholar] [CrossRef]
  22. Rahman, F.S.; Yahya, N.; Din, N.M.M.; Izaham, A.; Mat, W.R.W. The Comparative Effects of Listening to Prayer Recitation and Music Therapy Intraoperatively on Postoperative Pain. IIUM Med. J. Malays. 2018, 17. [Google Scholar] [CrossRef]
  23. Hasanah, I.; Mulatsih, S.; Haryanti, F.; Haikal, Z. Effect of music therapy on cortisol as a stress biomarker in children undergoing IV-line insertion. J. Taibah Univ. Med. Sci. 2020, 15, 238–243. [Google Scholar] [CrossRef]
  24. Demiray, A. Effects of Music Listening during Chemotherapy on Vital Signs and Anxiety Levels: A Randomized Controlled Trial. J. Med. Case Rep. Rev. 2020, 3, 462–476. [Google Scholar]
  25. Imran, S.; Moosabba, M.S.; Ancheril, A. Effectiveness of Music Therapy on Bio physiological and Psychological outcomes during Chemotherapy among Patients with Cancer-A pilot study. Scifed Nurs. Healthc. J. 2017, 1, 119–126. [Google Scholar]
  26. Mou, Q.; Wang, X.; Xiang, Q.; Li, J. Effects of Passive Music Therapy on Lung Cancer Patients in First PICC Catheterization: A Randomized Controlled Clinical Trial. Int. J. Med. Front. 2019, 3, 1–6. [Google Scholar]
  27. Spilioti, E.D.; Galanis, P.A.; Kalokairinou, A.G. The effects of music on cancer patients submitted to chemotherapy treatment. Int. J. Caring Sci. 2017, 10, 1465–1477. [Google Scholar]
  28. Jha, K.; Dubey, P.; Kumar, Y.; Singh, R.; Kumar, R. Effect of music of specific frequency upon the sleep architecture and electroencephalographic pattern of individuals with delayed sleep latency: A daytime nap study. J. Fam. Med. Prim. Care 2019, 8, 3915. [Google Scholar] [CrossRef]
  29. Jespersen, K.V.; Otto, M.; Kringelbach, M.L.; Van Someren, E.; Vuust, P. A randomized controlled trial of bedtime music for insomnia disorder. J. Sleep Res. 2019, 28, e12817. [Google Scholar] [CrossRef]
  30. Huang, C.-Y.; Chang, E.-T.; Hsieh, Y.-M.; Lai, H.-L. Effects of music and music video interventions on sleep quality: A randomized controlled trial in adults with sleep disturbances. Complement. Ther. Med. 2017, 34, 116–122. [Google Scholar] [CrossRef]
  31. Dixit, U.B.; Jasani, R.R. Comparison of the effectiveness of Bach flower therapy and music therapy on dental anxiety in pediatric patients: A randomized controlled study. J. Indian Soc. Pedod. Prev. Dent. 2020, 38, 71–78. [Google Scholar] [CrossRef]
  32. Jethani, B.; Narayana, I.H.; Dinesh, K.; Mathew, S. Influence of Music Therapy on Anxiety, Pain Perception, Heart Rate and Blood Pressure of Patients Undergoing Endodontic Treatment–A Randomized Control Trial. Int. J. Sci. Res. 2019, 8, 8–11. [Google Scholar] [CrossRef]
  33. Paul, D.; Peedikayil, F.C.; Soni, K.; Dhanesh, N. Comparison of brief relaxation and music distraction in the treatment of dental anxiety-a randomized controlled clinical trial. J. Res. Dent. 2018, 6, 109–117. [Google Scholar]
  34. Lakshmanan, R.; Packyanathan, J.S.; Jayashri, P. Effect of music therapy on anxiety levels on patient undergoing dental extractions. J. Fam. Med. Prim. Care 2019, 8, 3854. [Google Scholar] [CrossRef]
  35. Gaebel, C.; Rittner, S.; Stoffel, M.; Jarczok, M.; Aguilar-Raab, C.; Ditzen, B.; Warth, M. Study protocol of the MUSED study: A randomized controlled trial to evaluate the psychobiological effects of group music therapy in women with depression. Nord. J. Music. Ther. 2020, 1–26. [Google Scholar] [CrossRef]
  36. Ugur, H.G.; Aktaş, Y.Y.; Orak, O.S.; Saglambilen, O.; Avci, I.A. The effect of music therapy on depression and physiological parameters in elderly people living in a Turkish nursing home: A randomized-controlled trial. Aging Ment. Health 2017, 21, 1280–1286. [Google Scholar] [CrossRef] [PubMed]
  37. Antoniazza, B.; Pinto, M.P.; Ferraraccio, M.; Damini, M.; Sollami, A.; Marletti, G. Effects of music therapy on vital signs and anxiety: A study with terminally ill patients. J. Hosp. Palliat. Med. 2018, 1, 1–7. [Google Scholar]
  38. Ugur, H.G.; Orak, O.S.; Aktas, Y.Y.; Enginyurt, O.; Saglambilen, O. Effects of Music Therapy on the Care Burden of In-Home Caregivers and Physiological Parameters of Their In-Home Dementia Patients: A Randomized Controlled Trial. Complement. Med. Res. 2019, 26, 22–30. [Google Scholar] [CrossRef] [PubMed]
  39. Frye, S.; Weaver, C.; Dwiggins, A.; McMunn, A.; Hardy, A.; Batha, C.T.; Osman, M. Live Music Therapy impact on anxiety of patients receiving PET/CT scans. J. Nucl. Med. 2020, 61, 3034. [Google Scholar]
  40. Mallik, B.; Ferdousi, S.; Sultana, S. Effect of music therapy on heart rate variability in generalized anxiety disorder: A poincaré analysis. J. Bangladesh Soc. Physiol. 2020, 15, 39–45. [Google Scholar] [CrossRef]
  41. Akpinar, N.B.; Ceran, M.A.; Özkalp, B. The effect of classical Turkish and Western music on university students’ exam stress level, blood pressure and pulse rate: A randomized controlled trial. J. Health Sci. Med. 2020, 3, 216–220. [Google Scholar]
  42. Yakobson, D.; Arnon, S.; Gold, C.; Elefant, C.; Litmanovitz, I.; Beck, B.D. Music Therapy for Preterm Infants and Their Parents: A Cluster-Randomized Controlled Trial Protocol. J. Music. Ther. 2020, 57, 219–242. [Google Scholar] [CrossRef]
  43. Gallego-Gómez, J.I.; Balanza, S.L.-L.J.; García-Méndez, J.A.; Oliva-Pérez, J.; Doménech-Tortosa, J. Ṙ.-C.J.M. Effectiveness of music therapy and progressive muscle relaxation in reducing stress before exams and improving academic performance in Nursing students: A randomized trial. Nurse Educ. Today 2020, 84, 104217. [Google Scholar] [CrossRef]
  44. Wu, F.; Wu, T.; Yuce, M.R. An Internet-of-Things (IoT) Network System for Connected Safety and Health Monitoring Applications. Sensors 2018, 19, 21. [Google Scholar] [CrossRef] [Green Version]
  45. Misra, S.; Samanta, A. Traffic-Aware Efficient Mapping of Wireless Body Area Networks to Health Cloud Service Providers in Critical Emergency Situations. IEEE Trans. Mob. Comput. 2018, 17, 2968–2981. [Google Scholar] [CrossRef]
  46. Song, C.; Zeng, P.; Wang, Z.; Zhao, H.; Yu, H. Wearable Continuous Body Temperature Measurement Using Multiple Artificial Neural Networks. IEEE Trans. Ind. Inf. 2018, 14, 4395–4406. [Google Scholar] [CrossRef]
  47. Kim, J.C.; Chung, K. Prediction Model of User Physical Activity using Data Characteristics-based Long Short-term Memory Recurrent Neural Networks. KSII Trans. Internet Inf. Syst. 2019, 13. [Google Scholar] [CrossRef]
  48. Nasser, I.M.; Abu-Naser, S.S. Lung Cancer Detection Using Artificial Neural Network. Int. J. Eng. Inf. Syst. 2019, 3, 17–23. [Google Scholar]
  49. Peltarion A. Binary Cross-Entropy. Peltarion. 2020. Available online: https://peltarion.com/knowledge-center/documentation/modeling-view/build-an-ai-model/loss-functions/binary-crossentropy (accessed on 1 November 2020).
  50. Peltarion. Categorical Crossentropy. Peltarion. 2020. Available online: https://peltarion.com/knowledge-center/documentation/modeling-view/build-an-ai-model/loss-functions/categorical-crossentropy (accessed on 1 November 2020).
  51. Doshi, S. Various Optimization Algorithms For Training Neural Network. Towards Data Science. 13 January 2019. Available online: https://towardsdatascience.com/optimizers-for-training-neural-network-59450d71caf6 (accessed on 1 November 2020).
  52. Peltarion-R. “ReLU”. Peltarion. 2020. Available online: https://peltarion.com/knowledge-center/documentation/modeling-view/build-an-ai-model/activations/relu (accessed on 1 November 2020).
  53. Shahin, M.A.; Maier, H.R.; Jaksa, M.B. Data Division for Developing Neural Networks Applied to Geotechnical Engineering. J. Comput. Civ. Eng. 2004, 18, 105–114. [Google Scholar] [CrossRef]
  54. Umbrello, M.; Sorrenti, T.; Mistraletti, G.; Formenti, P.; Chiumello, D.; Terzoni, S. Music therapy reduces stress and anxiety in critically ill patients: A systematic review of randomized clinical trials. Minerva Anestesiol. 2019, 85, 886–898. [Google Scholar] [CrossRef]
  55. Hatwar, N.R.; Gawande, U.H. Can Music Therapy Reduce Human Psychological Stress: A Review. In Smart Trends in Computing and Communications; Zhang, Y.-D., Mandal, J.K., So-In, C., Thakur, N.V., Eds.; Springer (Smart Innovation, Systems and Technologies): Singapore, 2020; pp. 405–411. [Google Scholar] [CrossRef]
  56. Horden, P. Music as Medicine: The History of Music Therapy Since Antiquity; Routledge: London, UK, 2017. [Google Scholar]
  57. Chang, M.; Netzer, D. Exploring Natural Materials: Creative Stress-Reduction for Urban Working Adults. J. Creativity Ment. Health 2019, 14, 152–168. [Google Scholar] [CrossRef]
  58. Corazon, S.S.; Sidenius, U.; Poulsen, D.V.; Gramkow, M.C.; Stigsdotter, U.K. Psycho-Physiological Stress Recovery in Outdoor Nature-Based Interventions: A Systematic Review of the Past Eight Years of Research. Int. J. Environ. Res. Public Health 2019, 16, 1711. [Google Scholar] [CrossRef] [Green Version]
  59. Hedblom, M.; Gunnarsson, B.; Schaefer, M.; Knez, I.; Thorsson, P.; Lundström, J.N. Sounds of Nature in the City: No Evidence of Bird Song Improving Stress Recovery. Int. J. Environ. Res. Public Health 2019, 16, 1390. [Google Scholar] [CrossRef] [Green Version]
  60. Terry, P.C.; Karageorghis, C.I.; Curran, M.L.; Martin, O.V.; Parsons-Smith, R.L. Effects of music in exercise and sport: A meta-analytic review. Psychol. Bull. 2020, 146, 91–117. [Google Scholar] [CrossRef] [Green Version]
  61. Stork, M.J.; Karageorghis, C.I.; Ginis, K.A.M. Let’s Go: Psychological, psychophysical, and physiological effects of music during sprint interval exercise. Psychol. Sport Exerc. 2019, 45, 101547. [Google Scholar] [CrossRef]
  62. Fu, J.Z.X. The Influence of Background Music of Video Games on Immersion. J. Psychol. Psychother. 2015, 5. [Google Scholar] [CrossRef] [Green Version]
Figure 1. System level illustration and relevant components for integrating and ANN, BAN and music therapy.
Figure 1. System level illustration and relevant components for integrating and ANN, BAN and music therapy.
Electronics 09 02019 g001
Figure 2. Graphical illustration of proposed ANN enabled IoT Architecture for music therapy.
Figure 2. Graphical illustration of proposed ANN enabled IoT Architecture for music therapy.
Electronics 09 02019 g002
Figure 3. Screens of Android music therapy app.
Figure 3. Screens of Android music therapy app.
Electronics 09 02019 g003
Figure 4. Dashboard screen of Android app.
Figure 4. Dashboard screen of Android app.
Electronics 09 02019 g004
Figure 5. Architecture of assets directory.
Figure 5. Architecture of assets directory.
Electronics 09 02019 g005
Figure 6. Flow diagram illustrating the integration of ANN, MQTT and media player.
Figure 6. Flow diagram illustrating the integration of ANN, MQTT and media player.
Electronics 09 02019 g006
Table 1. State of the art: Diseases, Music Therapy, Monitored Parameters and Sensors.
Table 1. State of the art: Diseases, Music Therapy, Monitored Parameters and Sensors.
DiseaseStudied PopulationType of Music UsedParameter/s MonitoredStudy FindingsProposed Sensor(s)
Postoperative ManagementElective inguinal hernia surgery Patients [14]Patient preferred musicHeart rate, systolic and diastolic blood pressurePatient’s anxiety was reduced and satisfaction was improvedPulse rate sensor, Blood Pressure sensor
Elective Surgery Patients [15]Patient preferred musicHeart rate, systolic and diastolic blood pressureAnxiety, heart rate, respiratory rate and blood pressure reducedPulse rate sensor, Blood Pressure sensor
Hospitalized Pregnant Women [16]Humming with 2 Lyre instruments: Swedish and GermanHeart rate variabilityHRV and anxiety reduced, no impact observed on stressPulse rate sensor
Angiography Patients [17]Patient preferred sedative musicHeart rate, respiratory rate, systolic and diastolic blood pressureSystolic blood pressure and pain of the patients reducedPulse rate sensor, Flowmeter, Blood Pressure sensor
Patients with Autism/Disabilities [18]Composed pieces of musicSalivary CortisolStress levels of patients were reducedElectrochemical Cortisol Sensor
Healthy young Chinese males [19]Classic and self-selected relaxing musicHeart rate variabilityClassic music had a better influence on stress levelsPulse rate sensor
Open heart surgery patients [7]Sedative musicOxygen saturation, respiratory rate, pulse rate, systolic & diastolic blood pressureSystolic blood pressure, oxygen saturation and pulse rate were reducedPulse Oximetry Sensor, Pulse rate sensor, Blood Pressure sensor
Cardiac Surgery patients [20]live relaxing oud musicHeart rates, serum cortisol levelsPain and anxiety scores, respiratory and heart rates and Cortisol level reducesPulse rate sensor, Electrochemical Cortisol Sensor
Abdominal surgery patients [21]Instrumental MusicSystolic arterial blood pressureSystolic blood pressure reduced, and patients’ satisfaction score was higherBlood Pressure sensor
Open acute appendicitis surgery patients [22]Muslim Prayer & nature soundsBlood Pressure and Heart rateIntraoperative heart rates and postoperative pain scores were reducedBlood Pressure sensor, Pulse rate sensor.
CancerChildren suffering from Leukemia [23]Composed pieces of musicSalivary CortisolStress level reducedElectrochemical Cortisol Sensor
Patients Receiving Chemotherapy [24]Classical music and Turkish maqamsDiastolic blood pressureAnxiety levels and diastolic blood pressure reducedBlood Pressure sensor
Advanced Cancer Patients [8]Vocal & Instrumental Classic & PopularEEG-based emotion detectionAnxiety, tiredness and breathing difficulties were reducedBrain activity Sensor (Electrode)
Chemotherapy Patients [25]Patient preferred musicBlood pressure, heart rateAnxiety, blood pressure, heart and pain were reducedBlood Pressure sensor, pulse rate sensor
Lung Cancer patients [26]Slow-tempo & low-pitched musicSystolic blood pressures and respiration rateDiastolic blood pressure and heart rate reducedBlood Pressure sensor, flowmeter
Chemotherapy Patients [27]Instrumental, traditional, classical & religious musicPulse rate and blood pressureSystolic blood pressure reduced, insignificant impact on pulse rate and diastolic blood pressurePulse rate sensor & Blood Pressure sensor
InsomniaPatients with Primary Insomnia [10]Chinese Guqin MusicHeart rate variability & Event-Related evoked Potentials (ERPs)Physical and mental health was improved, and Insomnia conditions were reducedPulse rate sensor, Brain activity Sensor (Electrode)
Students with Delayed Sleep latency complaints [28]432 Hz musicEEG, ECG, electromyography (EMG), nasal airflow, thoracic movement, nasal saturation etc.Sleep latency was reducedBrain activity Sensor (Electrode), Pulse rate sensor, EMG Sensor, Flowmeter
Insomnia Disorder patients [29]Patient preferred Bedtime musicBlood Oxygen Saturation (BOS), Breathing Rate, Brain Activity, Heart RateInsomnia severity was reducedPulse rate sensor, Brain activity Sensor (Electrode), Flowmeter, Pulse Oximetry Sensor
Adults with Sleep disturbances [30]Buddhist musicBrain Activity and sleep timeSubjective total sleep time increasedBrain activity Sensor (Electrode), Smart phone
Dental AnxietyPediatric Dentistry Patients [31]Indian classical instrumental music (Raag Sohni)Pulse rate, systolic and diastolic blood pressurePulse rate, systolic and diastolic blood pressure reducedPulse rate sensor, Blood Pressure sensor
Minor Oral Surgery Patients [9]Instrumental MusicHeart rate variabilityAnxiety, pain, discomfort and heart rate reducedPulse rate sensor
Endodontic (Root Canal) Treatment Patients [32]Instrumental & Vocal MusicHeart rate, systolic and diastolic blood pressure)Physiological parameters and anxiety reducedBlood Pressure sensor, Pulse rate sensor
School going children during dental treatment [33]Mozart MusicHeart Rate and Oxygen SaturationAnxiety was reduced but no impact was observed on the oxygen saturationPulse rate Sensor and Pulse Oximetry sensor
Adults undergoing Dental Extraction [34]Indian Classical MusicSystolic, diastolic blood pressure and heart rateBlood pressure and heart rate reducedBlood Pressure sensor, Pulse rate sensor
Cardiac Risks, Stress & RelaxationHealthy Indian Males [1]Indian Classical Todi Raga MusicBlood pressure, pulse rate and respiratory rateBlood pressure, pulse rate and respiratory rate reducedBlood Pressure sensor, Pulse rate sensor
Healthy German Adults [35]Live Monochord MusicHeart rate variabilityHRV and associated depression level reducedPulse rate sensor
Healthy Young Adults from Netherlands [6]Nature sounds and Sedative musicHeart rate variabilityHRV, anxiety and arousal rate reducedPulse rate sensor
End of Life CareElderly at Turkish Nursing Home [36]Turkish Traditional & Sufi MusicSystolic blood pressureSystolic blood pressure and depression scores reducedBlood Pressure sensor
Hospice Care Italian Patients [37]Active or Passive Live Musical performance of patient preferred musicSystolic and Diastolic Blood Pressure and Blood Oxygen Saturation.Blood pressure, oxygen saturation and anxiety reducedBlood Pressure sensor, Pulse oximetry sensor
Turkish Dementia Patients [38]Turkish Classical MusicSystolic and diastolic blood pressure, heart rate, and respiration rateBlood pressure reducedBlood Pressure sensor, Pulse rate sensor, flowmeter
Misc.Patients receiving PET/CT scans [39]Live music by Trained music therapist and musicianBlood pressure, Pulse rate and respirations per minute (RPM)Respiration rate and anxiety reduced, whereas blood pressure and heart rate did notBlood Pressure, Pulse rate sensor and flowmeter
Generlized Anxiety Disorder Patients [40]Rabindra Sangeet based on Raga BhairabiHeart Rate Variability (HRV)Anxiety and heart rate was reducedPulse rate sensor
University Students with Exam Stress [41]Classical Western and Turkish musicPulse rate and Blood PressureBlood pressure and pulse rate reducedBlood Pressure and Pulse rate sensor
Preterm Infants [42]LullabyHeart Rate Variability (HRV)Nervous system stability of infants and anxiety scores of parents improvedPulse rate sensor
Women with depression [35]Patent Preferred MusicHeart Rate Variability (HRV), Salivary CortisolHRV and depressive symptoms reducedPulse rate sensor and Electrochemical Cortisol Sensor
Nursing Students with Exam Stress [43]Relaxing MusicBlood pressure, Heart rate, Salivary Cortisol, oxygen saturationBlood pressure, heart rate and Cortisol level reducedBlood Pressure, Pulse oximetry and pulse rate sensor and Electrochemical Cortisol Sensor
Table 2. Range of values used for training data for Node-Red.
Table 2. Range of values used for training data for Node-Red.
Patient’s StateParameter
Heart rateSystolic PressureDiastolic PressureBreath Rate
(Beats per Minute)(mmHg)(mmHg)(Breaths per Minute)
Normal60–65100–1208012–15
Stressed70–75110–1308015–20
Exercising90–120150–1708040–50
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Siddiqui, S.; Nesbitt, R.; Shakir, M.Z.; Khan, A.A.; Khan, A.A.; Khan, K.K.; Ramzan, N. Artificial Neural Network (ANN) Enabled Internet of Things (IoT) Architecture for Music Therapy. Electronics 2020, 9, 2019. https://doi.org/10.3390/electronics9122019

AMA Style

Siddiqui S, Nesbitt R, Shakir MZ, Khan AA, Khan AA, Khan KK, Ramzan N. Artificial Neural Network (ANN) Enabled Internet of Things (IoT) Architecture for Music Therapy. Electronics. 2020; 9(12):2019. https://doi.org/10.3390/electronics9122019

Chicago/Turabian Style

Siddiqui, Shama, Rory Nesbitt, Muhammad Zeeshan Shakir, Anwar Ahmed Khan, Ausaf Ahmed Khan, Karima Karam Khan, and Naeem Ramzan. 2020. "Artificial Neural Network (ANN) Enabled Internet of Things (IoT) Architecture for Music Therapy" Electronics 9, no. 12: 2019. https://doi.org/10.3390/electronics9122019

APA Style

Siddiqui, S., Nesbitt, R., Shakir, M. Z., Khan, A. A., Khan, A. A., Khan, K. K., & Ramzan, N. (2020). Artificial Neural Network (ANN) Enabled Internet of Things (IoT) Architecture for Music Therapy. Electronics, 9(12), 2019. https://doi.org/10.3390/electronics9122019

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop