*Article* **The Analgesic Efficacy of the Single Erector Spinae Plane Block with Intercostal Nerve Block Is Not Inferior to That of the Thoracic Paravertebral Block with Intercostal Nerve Block in Video-Assisted Thoracic Surgery**

**Sujin Kim 1, Seung Woo Song 1, Hyejin Do 1, Jinwon Hong 1, Chun Sung Byun <sup>2</sup> and Ji-Hyoung Park 1,\***


**Abstract:** This monocentric, single-blinded, randomized controlled noninferiority trial investigated the analgesic efficacy of erector spinae plane block (ESPB) combined with intercostal nerve block (ICNB) compared to that of thoracic paravertebral block (PVB) with ICNB in 52 patients undergoing video-assisted thoracic surgery (VATS). The endpoints included the difference in visual analog scale (VAS) scores for pain (0–10, where 10 = worst imaginable pain) in the postanesthetic care unit (PACU) and 24 and 48 h postoperatively between the ESPB and PVB groups. The secondary endpoints included patient satisfaction (1–5, where 5 = extremely satisfied) and total analgesic requirement in morphine milligram equivalents (MME). Median VAS scores were not significantly different between the groups (PACU: 2.0 (1.8, 5.3) vs. 2.0 (2.0, 4.0), *p* = 0.970; 24 h: 2.0 (0.8, 3.0) vs. 2.0 (1.0, 3.5), *p* = 0.993; 48 h: 1.0 (0.0, 3.5) vs. 1.0 (0.0, 5.0), *p* = 0.985). The upper limit of the 95% CI for the differences (PACU: 1.428, 24 h: 1.052, 48 h: 1.176) was within the predefined noninferiority margin of 2. Total doses of rescue analgesics (110.24 ± 103.64 vs. 118.40 ± 93.52 MME, *p* = 0.767) and satisfaction scores (3.5 (3.0, 4.0) vs. 4.0 (3.0, 5.0), *p* = 0.227) were similar. Thus, the ESPB combined with ICNB may be an efficacious option after VATS.

**Keywords:** video-assisted thoracic surgery; erector spinae plane block; paravertebral block; intercostal nerve block

#### **1. Introduction**

Although it is less invasive and damaging to tissue, video-assisted thoracic surgery (VATS) still causes moderate to severe postoperative pain [1]. Postoperative pain is an independent predictor of mortality and morbidity [2]. In addition, acute pain after thoracic surgery is associated with chronic pain; therefore, postoperative pain control is a main goal [3]. The thoracic epidural block (TEB) is the standard procedure for a regional block in thoracotomy; however, the paravertebral block (PVB) is known to show equipotent efficacy [4]. These are also widely used in VATS procedures. The erector spinae plane block (ESPB) is a simple fascial plane block that serves as an alternative to an epidural block, with fewer side effects [5]. It appears to be an effective analgesic technique at many levels and functions as an alternative when the PVB or epidural block is contraindicated [6]. However, some clinical trials have shown that ESPB has a lesser analgesic effect than PVB [7]; thus, no consensus has been reached. The intercostal nerve block (ICNB) is a multimodal analgesia technique following thoracic surgery, and its analgesic efficacy has been proven [8]. However, no study has compared the analgesic effects of ESPB and PVB when combined with ICNB.

**Citation:** Kim, S.; Song, S.W.; Do, H.; Hong, J.; Byun, C.S.; Park, J.-H. The Analgesic Efficacy of the Single Erector Spinae Plane Block with Intercostal Nerve Block Is Not Inferior to That of the Thoracic Paravertebral Block with Intercostal Nerve Block in Video-Assisted Thoracic Surgery. *J. Clin. Med.* **2022**, *11*, 5452. https://doi.org/10.3390/ jcm11185452

Academic Editor: Marco Cascella

Received: 28 July 2022 Accepted: 13 September 2022 Published: 16 September 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

We hypothesized that the ESPB would not be less effective than the PVB for attenuating pain when combined with ICNB after VATS. The primary outcome of this monocentric, single-blinded, randomized controlled noninferiority trial was the median difference in postoperative pain scores between the groups. The secondary outcomes were the cumulative postoperative analgesic consumption (calculated as oral morphine milligram equivalents, MME) and patient satisfaction scores.

#### **2. Materials and Methods**

#### *2.1. Study Population*

The study trial was approved by the Institutional Review Board of Yonsei University Wonju College of Medicine, Wonju, Korea, and the participants are listed at https://cris.nih.go.kr (accessed on 18 May 2021, KCT 0006271). We enrolled 52 patients with American Society of Anesthesiologists physical status I, II, or III, aged 19–80 years, who had undergone VATS between June 2021 and January 2022. Patients were excluded from the study for cognitive impairment, anticoagulant administration, coagulopathy, antiplatelet drug administration within 48 h, double antiplatelet therapy, surgical site infection, refusal of procedure, allergic reaction to local anesthetic, requiring therapeutic anticoagulant therapy after surgery, or pregnancy. Patients with comorbid diseases were excluded according to the judgment of the anesthesiologist (sepsis, anatomical thoracic deformity, empyema, increased intracranial pressure, etc.). Patients were randomly and evenly assigned to either the PVB with ICNB group or the ESPB with ICNB group by a computer-generated randomization table. Blinding of the group designation was maintained for the patients and attending anesthesiologists, except one practitioner (J.-H.P.) who performed the PVB or ESPB.

#### *2.2. Perioperative Management*

Anesthesia was induced by a bolus of propofol (1.5–2 mg/kg) and remifentanil (1 mcg/kg). Rocuronium (0.6 mg/kg) was used for tracheal intubation. The remifentanil infusion rate was adjusted by the attending anesthesiologist according to the overall hemodynamic data and the suggested intensity of surgical stimuli. The fraction of inhaled anesthetics was administered under BIS guidance. The surgeon performed the ICNB and placed a chest tube during the surgery. At the end of the surgery, the ESPB or PVB was performed in the lateral position. One anesthesiologist (J.-H.P.) performed the PVB and ESPB. The same experienced surgeon (C.S.B.) performed the ICNB. The intravenous patient-controlled analgesia (PCA) was used at the discretion of the anesthesiologist, and the dose was recorded in terms of morphine milligram equivalents (MME). The patients were extubated and transported to the postanesthetic care unit (PACU). The standard analgesic algorithm in PACU was intravenous nonopioid analgesics for visual analog scale pain scores of 4–6 [1] and intravenous fentanyl (50 μg) for VAS pain scores >6 [2]. The postoperative pain in the ward was controlled by the primary physician. Administered analgesic drugs were converted into MME and recorded in case record forms. The analgesics used in the ward were intravenous tramadol, intramuscular or subcutaneous meperidine, oral Ultracet® (tramadol 37.5 mg/acetaminophen 325 mg), and transdermal fentanyl patch.

#### *2.3. Paravertebral Block*

After surgery, each patient was placed in the lateral decubitus position, and skin preparation was performed. The patient was palpated at the T5 level, and the linear transducer was positioned in a vertical plane approximately 2.5 cm lateral to the palpated spinous process, obtaining a sagittal plane of the transverse process, superior costovertebral ligament, intertransverse ligaments, desired paravertebral space, pleura, and lung tissue. The paravertebral space was bordered by the vertebral body, intervertebral foramen, parietal pleura, and costovertebral ligament. A 21-gauge, 10 cm echogenic needle (Vygon SA; Ecouen, France) was placed in the paravertebral space using an in-plane approach to confirm that there was no blood aspiration. A small amount of local anesthetic was then

injected into the test dose in real time to reduce the anterior displacement of the pleura and spine. Ropivacaine (0.375%, 20 cc) was injected with aspiration every 5 cc to prevent intravascular injection.

#### *2.4. Erector Spinae Plane Block*

At the T5 level, the linear transducer was moved slowly from the midline to the lateral, and the transducer was moved approximately 3 cm until a transverse projection was observed. It was distinguishable from the ribs at that level: the transverse process is shallow and wide, whereas the ribs are deep and thin. The trapezius, rhomboid, and erector spinae muscles were then identified. An echogenic needle was inserted from the head to the foot, using an in-plane approach, and advanced toward the transverse process through the trapezius, rhomboid, and erector spinae muscles under ultrasound guidance. A small amount of local anesthetic was administered when the needle tip was located below the erector spinae muscle. The correct position of the needle was verified by visually confirming that the erector spinae muscle was separated from the transverse process. Ropivacaine (0.375%, 20 cc) was injected with aspiration every 5 cc to prevent intravascular injection.

#### *2.5. Intercostal Nerve Block*

At the end of the surgical procedure, a total of 10 cc of ropivacaine (2 cc per space) was injected into the intercostal space until swelling of the intercostal nerve at the T4–T8 levels occurred.

#### *2.6. Outcome Measures*

The primary endpoint of the present study was to assess the analgesic efficacy of ESPB compared with that of PVB when combined with ICNB by measuring the median differences between the groups in the VAS of pain at the PACU, as well as 24 and 48 h after surgery. The secondary endpoints were to investigate the total amount of analgesics administered to the patients in MME and the satisfaction score of patients using a five-point rating scale.

#### *2.7. Sample Size Calculation*

The standard deviation for PVB with ICNB in the pilot study was 2.29, and no significant difference was observed in the variance between the ESPB and PVB in a previous study [9]. When the noninferiority margin (delta value) was set to 2, it was determined based on the opinion of colleagues and a previous study [10]. The significance level was set to 0.05, and the power was set to 0.9; accordingly, the estimated number of patients required in each group was 23. Accounting for a dropout rate of 10%, we decided to enroll 26 patients in each group.

#### *2.8. Statistical Analysis*

All statistical analyses were performed using the IBM SPSS statistical software package (IBM SPSS Statistics for Windows, version 25, IBM Corporation, Armonk, NY, USA). Distribution of continuous variables was assessed using the Shapiro–Wilk test. Intergroup comparisons of the non-normally distributed variables were performed using the Mann–Whitney U test and are reported as the median (interquartile range). Intergroup comparisons of other variables that showed a normal distribution were tested using an independent *t*-test and are reported as the mean ± standard deviation (SD). For pain scores assessed at three timepoints, a post hoc Bonferroni correction was applied to adjust for multiple comparisons.

#### **3. Results**

In total, 52 patients were screened, and all of them were enrolled and assigned to the two groups. There was no dropout among the enrolled patients. Hence, all 52 patients were included in the final analysis.

The patient characteristics and types and durations of the surgeries were similar between the groups (Table 1).


**Table 1.** Patient characteristics and surgery details.

Values are displayed as the mean ± SD or *n* (%). PVB: paravertebral block, ESPB: erector spinae plane block, ICNB: intercostal nerve block, BSA: body surface area, OP: operation.

The visual analog scale (VAS) scores for each timepoint were not significantly different between the ESPB and PVB groups. The higher limit of the 95% CI for this difference (1.428 at PACU, 1.052 at 24 h, 1.176 at 48 h) was within the predefined noninferiority margin of 2 (delta). The total doses of rescue analgesics (110.24 ± 103.64 vs. 118.40 ± 93.52 MME, *p* = 0.767), the number of rescue analgesic events (5.88 ± 1.56 vs. 5.50 ± 1.45, *p* = 0.361), and satisfaction scores (3.5 (3.0, 4.0) vs. 4.0 (3.0, 5.0), *p* = 0.227) were not significantly different between the two groups (Figure 1). There were no significant differences in the intraoperative dose of remifentanil or the frequency of hypotension, bradycardia, or pleural puncture that occurred during the operation after the block (Table 2). The number of patients with moderate (VAS > 3) or severe (VAS > 6) pain was also similar between the two groups. We continuously identified the needle-tip position with ultrasound during the block. However, in the PVB-ICNB group, we recognized the pleural puncture, which was confirmed by the spread of the local analgesics into the pleura in two cases. The placement of the chest tube after the VATS procedure was performed as standard of care [11]. In both cases, the chest tube was removed and discharged without pneumothorax or other complications.

The hemodynamic data, including heart rate and mean arterial blood pressure during surgery, were also similar between the groups (Figure 2).

**Figure 1.** Visual analog scale (VAS) and satisfaction of patients: (**a**) VAS at PACU; (**b**) VAS at 24 h postoperatively; (**c**) VAS at 48 h postoperatively; (**d**) Satisfaction scores of patients. PACU: postanesthetic care unit, PVB: paravertebral block, ESPB: erector spinae plane block, ICNB: intercostal nerve block.

**Table 2.** Outcome measures.


Values are displayed as the mean ± SD, *n* (%), or median (interquartile range). The pain score was assessed using a visual analog scale (VAS) (0 = no pain, 10 = worst imaginable pain). The rescue analgesic requirement was calculated in morphine milligram equivalents (MME). The satisfaction score was assessed using a five-point numerical scale (1 = extremely dissatisfied, 5 = extremely satisfied). VAS: visual analog scale, PACU: postanesthetic care unit, PVB: paravertebral block, ESPB: erector spinae plane block, ICNB: intercostal nerve block, SD: standard deviation, CI: confidence interval.

**Figure 2.** Hemodynamic data: (**a**) Heart rate; (**b**) Mean arterial pressure.

#### **4. Discussion**

The results of the present trial suggest that ESPB was not inferior in analgesic efficacy to PVB when combined with ICNB for attenuating surgery-related pain. Moreover, this combination block was similar in terms of patient satisfaction, analgesic requirement, and the frequency of hemodynamic perturbations. In addition, although the difference was not significant, pleural puncture was absent in the ESPB-ICNB group but present in two cases in the PVB-ICNB group.

VATS is a minimally invasive procedure that results in minimal tissue trauma. Nevertheless, it also causes significant and intense acute pain, which may lead to post-thoracotomy pain syndrome [12]. The scope manipulation and use of rib retractors in VATS may cause intercostal nerve injury [13]. In addition, multiple muscle incisions and pleural irritation from chest tubes can cause moderate to severe pain [1].

TEB has long been the standard procedure for pain control after thoracotomy. However, side effects such as hypotension, bradycardia, and pruritus are common, and analgesic failure often occurs because of an incorrect target position. In addition, catastrophic side effects, such as epidural hematoma, require close attention [14]. A previous meta-analysis showed that PVB has an equipotent analgesic effect after thoracotomy compared with TEB [15]. However, the paravertebral space is narrow, and pleural puncture risks are possible [16].

ESPB involves the injection of local anesthetics into the fascial plane between the transverse process and erector spinae muscle [17]. In the case of VATS, the ports are positioned in the intercostal space of the mid-clavicular and post-scapular line, and an incision measuring 4–5 cm is positioned in the mid-axillary line [18]. Mostly, the intercostal nerves originating from the anterior rami of spinal nerves are responsible for sensory innervation [19]. However, since adjacent nerves branch out and perform various anastomoses with each other, sensory innervation is not well-associated with the segment level [20]. Therefore, it is important that the analgesics spread not only to the pathway of the intercostal nerve but also to adjacent segments. A previous literature review reported that the injectates spread to the ventral rami in 13 out of 16 cadaveric studies and to the paravertebral space in 12 studies through the thoracic ESPB. In all 16 studies, craniocaudal spread over three levels was observed [21]. In previous analyses, ESPB showed comparable efficacy to PVB in terms of opioid consumption and pain scores [22,23]. Controversial results indicating that PVB is superior to ESPB in terms of pain scores and opioid consumption have been reported [7,24].

Although there is no consensus on the analgesic effect, ESPB has several advantages over PVB. Firstly, it is technically easier to contact the transverse process than it is to fix a needle tip in the relatively narrow paravertebral space. Therefore, high skill is not required for the practitioners, and the difference in efficacy was shown to be little among the practitioners [19]. Moreover, since the paravertebral space is adjacent to the pleura, the puncture risk in PVB is higher than that in ESPB [19]. Thirdly, in terms of anticoagulation, ESPB has an advantage over PVB, which is considered a neuraxial block. Furthermore, the paravertebral space is a noncompressible area, but the target of the ESPB is compressible. However, unpredictable spread of injectates in ESPB was reported in some trials [25,26]. Therefore, the use of ICNB in conjunction with ESPB may enhance the analgesic effect. ICNB can be easily performed without complications while directly observing the spread. However, PVB provides better analgesic efficacy than ICNB [27]. In the present study, single ESPB combined with ICNB was not inferior to PVB combined with ICNB in terms of analgesic efficacy and opioid consumption. This combination block can be an effective and safe option to control pain after thoracoscopic surgery.

Our study had several limitations. Firstly, the analgesic effect was compared using a subjective index. However, there is no objective indicator that measures the pain index, and several previous trials have used the NRS as an effective tool for evaluating analgesic efficacy [28,29]. Secondly, we did not study patients for a long duration. According to the literature, moderate to severe pain is maintained postoperatively in approximately 10% of patients at 52 weeks [30]. Therefore, if we evaluated the patients for a longer period, the analgesic efficacy between the groups might have been better exhibited. Thirdly, our singlecenter setting and small sample size may have been insufficient for validating the secondary endpoints of the study. Fourthly, since both procedures were performed under general anesthesia, examination of the dermatomal level for analgesia, such as the pinprick test, could not be performed. Fifthly, the total dose and number of rescue analgesics exceeded those commonly used in VATS. This may have been a confounding factor in comparing the efficacy between the groups. However, in the previous study, 56% of patients complained of moderate to severe pain when only intravenous PCA was used [31]. Reducing the prevalence of moderate to severe pain and the VAS pain score in the present study showed the advantage of the combined block in both groups. Lastly, we did not include a control group of patients that only received ICNB. Therefore, it is difficult to determine to what extent the combination of regional block techniques in VATS is really advantageous.

In conclusion, the present study is the first trial to compare the effects of ESPB and PVB with ICNB. Both groups provided an adequate analgesic effect in VATS. However, compared to PVB, ESPB is easier to implement and has advantages in terms of safety due to no adjacent vulnerable structures. The present study provided evidence that ESPB with ICNB may be an efficacious analgesic option in VATS.

**Author Contributions:** Conceptualization, S.K. and J.-H.P.; methodology, S.K. and J.-H.P.; software, S.W.S. and J.-H.P.; validation, S.K., S.W.S., H.D. and J.-H.P.; formal analysis, H.D.; investigation, S.K.; resources, S.W.S., J.H. and C.S.B.; data curation, J.H.; writing—original draft preparation, S.K. and J.-H.P.; writing—review and editing, J.-H.P.; visualization, H.D.; supervision, J.-H.P.; project administration, C.S.B. and J.-H.P. All authors read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** This study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board of Yonsei University Wonju College of Medicine (IRB No. CR321026 on 18 May 2021).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data presented in this study are available upon request from the corresponding author.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


**Marco Cascella 1,2,\*, Sergio Coluccia 3, Federica Monaco 1, Daniela Schiavo 1, Davide Nocerino 1, Mariacinzia Grizzuti 1, Maria Cristina Romano <sup>1</sup> and Arturo Cuomo <sup>1</sup>**


**Abstract:** Background: The most effective strategy for managing cancer pain remotely should be better defined. There is a need to identify those patients who require increased attention and calibrated follow-up programs. Methods: Machine learning (ML) models were developed using the data prospectively obtained from a single-center program of telemedicine-based cancer pain management. These models included random forest (RF), gradient boosting machine (GBM), artificial neural network (ANN), and the LASSO–RIDGE algorithm. Thirteen demographic, social, clinical, and therapeutic variables were adopted to define the conditions that can affect the number of teleconsultations. After ML validation, the risk analysis for more than one remote consultation was assessed in target individuals. Results: The data from 158 patients were collected. In the training set, the accuracy was about 95% and 98% for ANN and RF, respectively. Nevertheless, the best accuracy on the test set was obtained with RF (70%). The ML-based simulations showed that young age (<55 years), lung cancer, and occurrence of breakthrough cancer pain help to predict the number of remote consultations. Elderly patients (>75 years) with bone metastases may require more telemedicinebased clinical evaluations. Conclusion: ML-based analyses may enable clinicians to identify the best model for predicting the need for more remote consultations. It could be useful for calibrating care interventions and resource allocation.

**Keywords:** telemedicine; telehealth; teleconsultations; predictive models; machine learning; cancer pain; random forest; gradient boosting machine; artificial neural network; LASSO–RIDGE algorithm

**1. Introduction**

Managing cancer-related pain typically requires a complex and multimodal approach [1]. One of the main challenges concerns the development of a useful pathway for addressing the multiple problems that can occur during the disease course [2–4].

Given that a model of care based on face-to-face visits requires an important commitment of resources, innovative strategies must be evaluated. Telemedicine may offer a variety of applications to re-evaluate pathways of care, including cancer pain management [5]. In this context, telemedicine-based strategies can have a paramount economic and organizational impact on healthcare systems [6], enhancing the quality of the care provided. A recent evidence-based analysis demonstrated that eHealth interventions are effective in improving pain management [7]. Although during the COVID-19 pandemic, different telemedicine approaches have been proposed [8], there is a need for establishing pathways that are valid beyond the emergency and routinely applied to clinical practice [9,10].

On the other hand, lacking the literature data from large-scale clinical experiences and precise directives from scientific societies, it is difficult to establish a model that provides

**Citation:** Cascella, M.; Coluccia, S.; Monaco, F.; Schiavo, D.; Nocerino, D.; Grizzuti, M.; Romano, M.C.; Cuomo, A. Different Machine Learning Approaches for Implementing Telehealth-Based Cancer Pain Management Strategies. *J. Clin. Med.* **2022**, *11*, 5484. https://doi.org/ 10.3390/jcm11185484

Academic Editor: Won Ho Kim

Received: 8 August 2022 Accepted: 14 September 2022 Published: 19 September 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

for the integration of telemedicine in the treatment process. Consequently, a proper strategy for the management of cancer pain through telemedicine should be fully designed.

The use of predictive models represents an important opportunity in medicine. The benefits of artificial intelligence (AI) and its branches such as machine learning (ML) are intended to enhance patient care, but also involve organizational processes and healthcare systems [11]. In the planning of care pathways, AI represents a valuable helpful resource to improve hospital workflows, identifying the activities that require priority and providing an adequate service to the patient's needs. Recently, for example, it was demonstrated that AI strategies such as natural language processing models can be a reliable guide to trigger early access for uncontrolled pain and other symptoms in palliative care [12].

In a recent cross-sectional investigation, we proposed a model of care and evaluated adherence to the telemedicine pathway [13]. This "hybrid" model provides for scheduled remote visits, but the patient can require other consultations. Additionally, in-person access is provided for emergencies or for diagnostic or clinical aims. For each patient, the number of telemedicine visits can vary depending on an unspecified number of reasons, and we have noticed that some patients required a greater number of remote consultations. On these premises, the purpose of this study is the development of data-driven predictive models for identifying those patients who may require more remote consultations. In the context of precision medicine for cancer pain management, we implemented ML algorithms to better customize treatment strategies. As pieces of evidence are needed to establish the most appropriate telemedicine pathways, the recognition of those patients who require a greater number of remote visits can stimulate the planning of ad hoc processes for managing multiple care needs and calibrating resource allocation.

#### **2. Materials and Methods**

#### *2.1. Study Population*

The study population included adult patients treated for cancer pain at the Istituto Nazionale Tumori, Fondazione Pascale, Italy.

A hybrid model of care was implemented. After the first in-person visit for a complete clinical and instrumental evaluation and for addressing legal and regulatory issues (consent acquisition), data collection, and training, a synchronous real-time video consultation was scheduled according to the clinical need. Further remote controls were programmed or required by the patients. Moreover, face-to-face visits were allowed to carry out minimally invasive procedures, for diagnosis, acute clinical motivations (e.g., drug side effects), or if requested by the patient [13].

The local Medical Ethics Committee approved this study (protocol code 41/20 Oss; date of approval, 26 November 2020), and all patients provided written informed consent. The investigation was conducted in accordance with the Declaration of Helsinki.

#### *2.2. Data Collection*

For each patient, 13 demographic, clinical, and therapeutic variables were collected to investigate the potential causes that may affect the number of remote consultations (Table 1). All the data were reported on a prospectively filled database and then registered on Zenodo [14]. The duration of the study was considered the time interval between the first and last remote consultations. The death of the patient and the occurrence of in-person visits or hospitalization were assumed as the conditions for the end of the observation period for data acquisition. The lack of further remote consultations for a two-month period was another condition for considering the observation closed.

The univariate analysis was performed to detect the main associations of selected features with the outcome variable (remote consultations: one or more).


**Table 1.** Data collection and variables.

Abbreviations: ECOG-PS, Eastern Cooperative Oncology Group Performance Status; MED, morphine-equivalent dose; NP, neuropathic pain; ROOs, rapid-onset opioids; PAMORAs, peripherally acting μ-opioid receptor antagonists; IV-Morphine, intravenous morphine. Legend: \* including cohabitation and marriage.

#### *2.3. Predictive Analysis*

#### 2.3.1. Preprocessing and Exploratory Data Analysis

After the loading, normalization, and standardization of the dataset (preparation process or preprocessing), as well as an exploratory data analysis aimed at discovering trends, the variables were selected. The expectation–maximization (EM) algorithm was used for the imputation of the missing data [15]. To facilitate model implementations and the interpretation of results, three age groups were obtained by categorizing the variable "age": ≤55 years old (called "younger patients"), 56–75 years old, and >75 years old ("older patients").

#### 2.3.2. Machine Learning Algorithms

Four ML-based algorithms were adopted as follows:


We chose these four ML-based prediction models for implementing different methods for regression or classification, such as bagging and boosting (RF and GMB, for additive regression models), and a strong learning method to compare different numerical approaches (LASSO–RIDGE, a binary regression model); ANN is one of the function algorithms that are largely used for classification (and regression) problems.

#### 2.3.3. Model Processing and Evaluation

Since a predictive analysis was performed in order to predict which cancer patient should need to have more than one remote consultation, the outcome variable was "the number of remote consultations" as dichotomized. Each classifier was optimized by repeated cross-validation (RCV) methods to focus on the best guess through the K-fold mean error calculus and to determine the hyperparameters that support the best guess and identify its structure. The sample was split into a training set (80% of the total size) to identify the hyperparameters and a test set for testing the models (20% of the total size).

A wide choice of hyperparameters was given to every algorithm to finally evaluate the best performance. In particular, each combination of hyperparameters was inserted as input for the algorithm. An 8-fold 5-repeated cross-validation method was adopted to find the best one, so the dataset was divided into 8 parts (20 individuals for any time), and the training and test parts were performed for each combination and for 5 times; the misclassification error rate was calculated upon 5 attempts (for a more precise managing of results). For each algorithm, the following features were applied:


Comparations were assessed through these models by calculating the accuracy and area under the receiver operating characteristic (ROC) curve (AUC). The AUC represents the sensibility (i.e., TP/(TP+FN)) and 1-specificity (1-FP/(TN+FP)) ratio. The AUC can easily be approximated with the measure of accuracy in the case of equidistribution between the modalities of the employee, but it is also suitable for solving problems of poorly distributed modalities. Each member is comparable to the observed value conditional correct classification rates, respectively. In other words, the AUC is equivalent to the probability that a random positive response is classified with respect to a random negative one. Another adopted goodness-of-prevision statistic parameter was the F1 score:

$$\mathbf{F1} = \mathbf{2} \text{ \* (precision \* recall)} / \text{(precision + recall)} \tag{1}$$

This measure considers the precision and recall of the test; precision is the number of true positives divided by the number of all the positive results, while recall is the number of true positives divided by the number of all the tests that should have been positive (i.e., true positives plus false negatives). The values of F1 scores range from 0 to 1.

Finally, Mathew's correlation coefficients (MCC) were calculated (from the confusion matrix) for each model, for obtaining a broad view of their predictive power and robustness (TP, true positive; TN, true negative; FP, false positive; FN, false negative):

$$\text{MCC} = \frac{\text{TP} \ast \text{TN} - \text{FP} \ast \text{FN}}{\sqrt{(\text{TP} + \text{FP})(\text{TP} + \text{FN})(\text{TN} + \text{FP})(\text{TN} + \text{FN})}} \tag{2}$$

#### 2.3.4. Risk Analysis

Based on the ML processes, the risk analysis for an increased number (>1) of remote consultations was assessed. We used an odds-ratio-like analysis that we indicated as the simulated odds ratios (SORs). Simulations were assessed in order to evaluate the risk of more consultations in target individuals. In particular, approximately 500 simulations were performed 150 times for creating a classification rate for the cases (target individuals) and control individuals. Subsequently, we calculated the odds ratio as the ratio of the effective odds for each individual typology and 95% credibility intervals (95% CIs) as the effective 2.5 and 97.5 percentiles for the SOR samples. Although wide possibilities were possible, the following four standard clinical conditions (targets) were established:


#### *2.4. Algorithmic Toolkit*

The data were analyzed using the *R* software version 4.1.3 (R Core Teams, R Foundation for Statistical Computing, Vienna, Austria). The toolkit included the mice package [20] for the imputation of the missing data. Caret was the main suite used for the implementation (creation, training) and evaluation (testing) of the classifiers [21]. Moreover, purr, pROC, and pRROC [22] were adopted for the construction and visualization of the ROC curves. The graphics packages included ggplot, ggpubr, and cowplot.

#### **3. Results**

A total of 267 patients were evaluated for cancer pain management through remote consultations between March 2021 and February 2022. Of these patients, 109 were excluded for not being available or having incomplete data; finally, the data from 158 patients were used for the descriptive and predictive analyses (Figure 1).

**Figure 1.** Flowchart of the study. Abbreviations: ML, machine learning; ECOG-PS, Eastern Cooperative Oncology Group Performance Status; MED, morphine-equivalent dose; PAMORAs, peripherally acting μ-opioid receptor antagonists; ROOs, rapid-onset opioids; IV-Morphine, intravenous morphine. Legend: the category "living with a partner" includes cohabitation and marriage.

#### *3.1. Descriptive Analysis*

The median age was 63 years old. Fifty-one percent were female. Just over half of the patients (53%) had more than one visit. The average number of visits was 2.27, with a standard deviation of 2.05 (Table 2).

The reasons for interruption of the telemedicine pathway (dropouts) were the patient's death (*n* = 63, 39.9%), the need for an invasive procedure (*n* = 15, 9.5%) or an in-person clinical assessment (*n* = 14, 8.9%). Six patients (3.8%) requested an in-person visit. Unplanned hospital admissions occurred in seven patients (4.4%). About a third of the patients (*n* = 53, 33.5%) were not evaluated (in person or remotely) for at least two months. These patients were contacted (email and telephone), and about half (*n* = 28) did not provide an answer; the remainder (*n* = 25) said they did not need further visits for cancer pain (Figure 2).

The univariate analysis was performed for evaluating the differences between the cohort of patients who underwent one remote consultation and those who received more telemedicine evaluations (Table 3).

**Figure 2.** Reasons for interruption of the telemedicine pathway (*n* = 158).

#### *3.2. Predictive Analysis*

Table 4 summarizes the results of the implemented ML methods. In our analyses, the accuracy, that is the proportion of the well-ranked parameters, relative to the training set, reached almost 100% for the RF and ANN algorithms. Nevertheless, the accuracy of the ANN on the test set was reduced by almost 50 percentage points. By contrast, RF showed an acceptable classification level (70% accuracy in the test) (*p* = 0.05) with an F1 score of 0.71.

The overall performance of a classifier, summarized over all the possible thresholds, is given by the area under the ROC curve (AUC). An ideal ROC curve will hug the top left corner: the larger the area under the curve, the better the classifier. Reducing the false-positive rate (FPR) and, at the same time, increasing the true-negative rate (TNR) is like finding a trade-off cut point between the error rates. A classifier that performs worse than a random classification has an AUC statistic of 0.5. Thus, an AUC value closer to 1 indicates a more adequate classification and a lower level of error: Its value is theoretically almost 1 as it is built. The AUC performances of the considered classifiers are reported in Figure 3.


**Table 2.** Data from the considered variables.

Abbreviations: \* *n* (%); ECOG-PS, Eastern Cooperative Oncology Group Performance Status; MED, morphineequivalent dose; ROOs, rapid-onset opioids; PAMORAs, peripherally acting μ-opioid receptor antagonists; NP, neuropathic pain; IV-Morphine, intravenous morphine.


**Table 3.** Univariate analysis for data exploration.

Legend: \* *n* (%); ˆ Wilcoxon rank-sum test; Pearson's chi-squared test; significance at 95%. Abbreviations: ECOG-PS, Eastern Cooperative Oncology Group Performance Status; MED, morphine-equivalent dose; ROOs, rapid-onset opioids; PAMORAs, peripherally acting μ-opioid receptor antagonists; NP, neuropathic pain; IV-Morphine, intravenous morphine.


**Table 4.** Performance comparison of the different classifiers for the developed machine learning models.

Abbreviations: GBM, gradient boosting machine; RF, random forest; LASSO, LASSO–RIDGE regression; ANN, artificial neural network; AUC, area under the receiver operating characteristic curve; ACC (tr), accuracy on training; ACC (tst), accuracy on test set; L and U, 95%CI lower and upper limits of test set accuracy statistic; *p*, accuracy on the test set and relative test for significance; Sens (tst), sensibility on the test; Spec (tst), specificity on the test. MCC, Mathew's Correlation Coefficient.

**Figure 3.** The area under the receiver operating characteristic (ROC) curve (AUC) of the considered models. False-positive rate (fpr) and true-negative rate (tpr) were considered. The plot shows the ROC curves calculated for each classifier over the entire dataset. RF and NN offer the best performance. Abbreviations: LASSO, LASSO–RIDGE regression; GBM, gradient boosting machine; ANN, artificial neural network; RF, random forest.

The confusion matrix for the two best models (RF and ANN) during the training phase is shown in Table 5.


**Table 5.** Comparison (confusion matrix) for the two best models.

Legend: one or more consultations were considered. Abbreviations: RF, random forest; ANN, artificial neural network.

#### *3.3. Risk Analysis*

The model with the best performance (i.e., RF) was implemented for assessing the risk analysis in different scenarios.

Condition 1: We calculated the risk of having repeated remote consultations for different cancer types. The other features were kept as randomly chosen. For those with lung neoplasm, there was a probability of 93.4% (92.6%, 94.2%) to receive multiple consultations and a higher risk (+172.2%, 95% CI = +70%, +301.1%) than cancer patients with no bone metastases; for those patients with colorectal neoplasm, the percentage was 88.8% (88%, 89.7%), and this risk was +92.2% (95%CI = +40.4%, +156.4%). For those affected by other cancers, the percentage was 71.9% (70.6%, 73.3%), and the risk was +55.5% (95%CI = +19.2%, +106.9%). For breast neoplasm with bone metastases, 90.1% (89.2%, 91.2%) of the patients were predicted to have multiple consultations, and +90.6% (95%CI = +32.5%, +158%) was their predicted risk for multiple consultations, compared with those without bone metastasis (Figure 4).

**Figure 4.** Simulation 1 refers to simulated odds ratios (SORs); percentages are labeled. This was performed for young patients (≤55 years old) with bone metastases and ROO use and young patients with bone metastases vs. those with no bone metastases. SORs for lung cancer were 2.72 (95%CI = 1.70–4.01); colorectal cancer 1.92 (95%CI = 1.40–2.56); other cancers 1.55 (95%CI = 1.19–2.07); breast cancer 1.91 (95%CI = 1.32, 2.58).

Condition 2: The same analysis was performed for older cancer patients (>75 years old), with and without bone metastases. For those patients affected by lung neoplasm, the risk for multiple remote consultations was 4.4 times (+335.5%, 95%CI = +209%, +529.6%) more than those with no bone metastasis, 2.9 times (+189.4%, 95%CI = +117.6%, +276.9%) for colorectal neoplasm, and 4.6 times (+357.9%, 95%CI = +252.6%, +495.5%) for other types of cancer. The expected probabilities were 88.7% (87.6%, 89.6%) for lung cancer, 82.9% (81.6%, 84%) for colorectal cancer, and 69.4% (68.0%, 70.8%) for other cancers. For those with breast neoplasm with bone metastasis, the simulated percentage of multiple remote consultations was 88.7% (87.8%, 89.6%) with a higher risk (+82.3%, 95%CI = +31.6%, +146.8%) of multiple remote evaluations than those without bone metastases (Figure 5).

Condition 3: The model demonstrated that male cancer patients can have an 11 times higher risk to receive multiple remote consultations than female cancer patients. The SORs were 11.3 (95%CI = 4.6, 24.1) for lung neoplasm and 11.1 (95%CI = 5.9, 20.6) for colorectal neoplasm. No statistical significance was found for other cancers (SOR = 0.97, 95%CI = 0.76, 1.23) (Figure 6).

**Figure 5.** Simulation 2 refers to simulated odds ratios (SORs); percentages are labeled. This was performed for older patients (>75 years old) with bone metastases vs. patients without bone metastases. SORs for lung cancer were 4.35 (95%CI = 3.90–6.30); colorectal cancer 2.89 (95%CI = 2.18–3.77); other cancers 4.58 (95%CI = 3.53–5.95); breast cancer 1.82 (95%CI = 1.32, 2.47).

**Figure 6.** Simulation 3 refers to simulated odds ratios (SORs); percentages are labeled. It was performed for young individuals (≤55 years old) with bone metastases: male vs. female SORs. Young male patients had a significantly higher risk to receive multiple remote consultations when affected by lung cancer (SOR = 11.30, 95%CI = 4.60, 24.10) and colorectal cancer (SOR = 11.1, 95%CI = 5.90, 20.60). No statistical significance was found for other cancers (SOR = 0.97, 95%CI = 0.76, 1.23).

Condition 4: An overall higher risk of having multiple telemedicine visits was found for young cancer patients than for male cancer patients, with SORs of +88.9% (95%CI = +16%, +182%) for lung cancer; +70.1% (95%CI = +24.1%, +143.3%) for colorectal cancer; and +16.5% (95%CI = −0.9%, +51.9%) for other cancers. Compared with older patients, for young female breast cancer patients, no significant risk was found (+19.5%, 95%CI = −20%, +65.2%) (Figure 7).

**Figure 7.** Simulation 4 refers to simulated odds ratios (SORs); percentages are labeled. This was performed for younger vs. older patients with bone metastases. Young patients had a significantly higher risk to receive multiple remote consultations for lung cancer. SORs were 1.89, 95%CI = 1.16, 2.82 for lung cancer and 1.70, 95%CI = 1.24, 2.43 for colorectal cancer. No statistical significance was found for other cancers (SOR = 1.16, 95%CI = 0.91, 1.52) and breast cancer (SOR = 1.19, 95%CI = 0.80, 1.65).

#### **4. Discussion**

In the setting of patients suffering from cancer pain, the applications of telemedicine strategies can enhance the effectiveness of clinical management [13] and lead to the optimization of resources [23]. Nevertheless, despite the growing use of telehealth methods, scientific evidence is still scarce to design care pathways.

Previously, we evaluated patient satisfaction with telemedicine and found high satisfaction rates with the care provided and the platform used. The dropout from the telemedicine pathway was investigated, and we found that approximately 10% of patients leave the telemedicine process due to unplanned clinic or hospital readmission or the need for non-pharmacological treatments [13]. Therefore, in this clinical setting, the development of telemedicine-based programs must consider multiple factors. The proposed model of care provides for a variable number of telemedicine visits by combining scheduled consultations and patient requests. In-person visits can be required to carry out minimally invasive procedures, diagnoses, or for other purposes. Furthermore, access to the hospital is provided for acute clinical conditions. Nevertheless, by following this approach, clinical practice has suggested that the careful planning of controls and the design of a safety pathway is a fundamental preliminary phase for validating our telemedicine-based model of care. The aim is to design a model of care that is generalizable while guaranteeing a patient-centered treatment.

In the clinical practice through telemedicine, we observed that many cancer patients had just one consultation. However, some individuals required a large number of closely remote visits. Consequently, we decided to evaluate the typology of cancer patients who may require more than one remote consultation. We searched for a more suitable strategy useful for achieving internal and external validation and translating the chosen model into the clinic [24]. For this aim, we adopted different ML models and decided to categorize the number of remote consultations as "one" or "more than one" remote consultation. Furthermore, the prediction of the number of remote consultations for new patients can involve several practical implications, including the design of personalized paths and optimal resource allocation. An increased number of remote consultations for cancer pain management may also reflect on ad hoc public or private healthcare/insurance health programs. For example, in Italy, the Ministry of Health released guidelines for the provision of telemedicine services, stimulating the design of paths for different care needs [25].

In ML analyses, preprocessing and exploratory data analysis (EDA) are the key elements of the process and take a large part of the time used for the whole analysis. The variable analysis is a crucial point for the modeling: It is part of the data quality and has a significant influence on the model's predictive power, robustness, and confidence. During these phases, it emerged that the variable "age" offered useful information for the model construction and understanding. This variable was categorized into three age groups (younger, mean age, and older patients). Consequently, in the univariate analysis, it was found that younger patients underwent more visits (*p* = 0.03). These data were used in the predictive analysis (simulations) for assessing, in target individuals, the risk of having multiple remote consultations. For example, the application of the chosen model (RF) showed that younger patients (≤55 years old) with bone metastases and ROO administration for BTcP treatment have an increased risk for more consultations, especially if affected by lung cancer. These data confirm what we previously highlighted in an analysis focused on the BTcP phenomenon. In a hierarchical classification, the worst phenotype of cancer pain patients was characterized by the presence of BTcP, younger age, and lung cancer [26].

In patients of advanced age (>75 years), the variable "bone metastasis" affected the prediction of the number of visits. Although this finding was confirmed above all for lung cancer, it concerns all cancer types. The combination of age and bone metastases identifies a particular class of fragile patients. These patients should be given greater attention by planning closer evaluations, also through telehealth strategies.

Male and female young patients (≤55 years old) with bone metastases were evaluated for their risk of needing multiple remote visits. The RF model showed that male patients had an 11 times higher risk to need multiple remote consultations for pain management than female patients, especially for lung and colorectal cancers. These data must be interpreted very carefully and not only based on the possible gender/age differences in pain perception [27]. This is probably due to the low sample size. The ongoing creation of a larger dataset will allow us to carry out multivariate analyses and define whether other variables such as the differences in the type and stage of the tumor, the impact of the disease on functionality, psychosocial aspects, any comorbidities, as well as the associated anticancer therapies, may influence the data. Based on the results of this predictive analysis, we will develop more accurate pathways for addressing the multiple management problems of cancer pain. The proposed model of care, in fact, also provides for a multidisciplinary approach with the simultaneous involvement of different professionals, such as psychologists, physical therapists, surgeons, general practitioners, etc.

This study has several limitations. The small size of the dataset is the most important limitation. With a higher sample size, some variables could be more representative within the predictive processes. Nevertheless, since not many predictors were present in the models, analyses could be performed despite the limited number of patients.

Numerical variables contain more information than their categorial transformations. For example, the ROO variable could be used as discrete. For these reasons, we chose to categorize the cancer patients' age into three categories (younger, middle-aged, and elderly patients). Another important limitation concerns the number of consultations per patient who is squeezed into the unit. Therefore, with a different distribution and the variables kept as numerical, better results would have been obtained.

The variables we adopted reflect our clinical practice. For example, although several opioids can be used for BTcP management [28], we usually prefer ROOs as the formulations licensed for this aim [29]. Moreover, this set of variables is not exhaustive. In cancer patients, for example, pain may not come from bone metastasis but often derives from invasion or abdominal metastasis, such as the peritoneal metastasis of colon cancer, and other causes. Probably, other variables such as cancer stage (e.g., TNM classification) should have been considered. In this regard, an improved dataset is planned in terms of its sample size and features. This will allow us to implement more sophisticated algorithms. The data from this study, however, can be useful for providing guidance for research in a field (telemedicine for cancer pain management) that is yet to be fully explored. The intervals between remote consultations, the needs for physical examinations, and the approaches for disease progression, as well as a careful definition of the process of early and simultaneous palliative care, are just some of the problems to be faced.

The methodological approach used in the simulations (i.e., SORs) has important potential that the clinician can exploit in predicting the outcome. On the other hand, SORs and simulations are obviously penalized by the sample size. In our analysis, the comparison between genders is an example of this gap. We highlight that their interpretation makes a practical and clinically useful sense if they are assessed through a good classifier.

Finally, we carried out only a few simulations as an example of the application of the evaluated model. The model, indeed, can be applied to a very large series of combinations of variables. Consequently, upon request, the dataset and model are available for further investigation.

#### **5. Conclusions**

The application of ML in telemedicine for pain management can enable physicians to make effective, real-time, and data-driven choices. This approach can be a key component in generating a better patient experience and improving health outcomes. A methodological approach to predictive analysis has great potential and could allow clinicians to provide important information to predict the outcome. Despite the important limitations of this study, in our analysis, the outcome (the number of remote consultations) was influenced by the selected variables such as the patient's age, the cancer type, and the occurrence of bone metastases. Further studies are needed to design and refine this model of care for cancer pain patients.

**Author Contributions:** Conceptualization, M.C. and S.C.; methodology, F.M.; software, SC.; validation, A.C., M.C.R. and D.N.; formal analysis, M.G.; investigation, F.M.; resources, D.S.; data curation, F.M.; writing—original draft preparation, M.C.; writing—review and editing, M.C.; visualization, D.N.; supervision, A.C.; project administration, A.C. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted in accordance with the Declaration of Helsinki and approved by the Ethics Committee of the Istituto Nazionale Tumori, Fondazione Pascale of Naples, Italy (protocol code 41/20 Oss; date of approval, 26 November 2020).

**Informed Consent Statement:** Informed consent was obtained from all the subjects involved in this study.

**Data Availability Statement:** All data were reported on a prospectively filled database and then registered on Zenodo [13].

**Acknowledgments:** We acknowledge Valeria Vicario, engineer of the Istituto Nazionale Tumori, Fondazione Pascale of Naples, Italy for her valuable advice.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

