Next Article in Journal
Post-Traumatic Growth during COVID-19: The Role of Perceived Social Support, Personality, and Coping Strategies
Next Article in Special Issue
Towards Facial Gesture Recognition in Photographs of Patients with Facial Palsy
Previous Article in Journal
User Engagement and Abandonment of mHealth: A Cross-Sectional Survey
Previous Article in Special Issue
An Efficient System for Eye Movement Desensitization and Reprocessing (EMDR) Therapy: A Pilot Study
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Healthcare
Volume 10
Issue 2
10.3390/healthcare10020223
healthcare-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Machine Learning for Predicting Risk of Early Dropout in a Recovery Program for Opioid Use Disorder

by
Assaf Gottlieb
1,
Andrea Yatsco
1,
Christine Bakos-Block
1,
James R. Langabeer
1,2 and
Tiffany Champagne-Langabeer
1,*
1
School of Biomedical Informatics, University of Texas Health Science Center at Houston, 7000 Fannin St., Houston, TX 77030, USA
2
McGovern Medical School, University of Texas Health Science Center at Houston, 6431 Fannin St., Houston, TX 77030, USA
*
Author to whom correspondence should be addressed.
Healthcare 2022, 10(2), 223; https://doi.org/10.3390/healthcare10020223
Submission received: 5 January 2022 / Revised: 23 January 2022 / Accepted: 24 January 2022 / Published: 25 January 2022
(This article belongs to the Special Issue Artificial Intelligence Developments for Medical Diagnosis and Monitoring)
Browse Figures
Versions Notes

Abstract

:
Background: An increase in opioid use has led to an opioid crisis during the last decade, leading to declarations of a public health emergency. In response to this call, the Houston Emergency Opioid Engagement System (HEROES) was established and created an emergency access pathway into long-term recovery for individuals with an opioid use disorder. A major contributor to the success of the program is retention of the enrolled individuals in the program. Methods: We have identified an increase in dropout from the program after 90 and 120 days. Based on more than 700 program participants, we developed a machine learning approach to predict the individualized risk for dropping out of the program. Results: Our model achieved sensitivity of 0.81 and specificity of 0.65 for dropout at 90 days and improved the performance to sensitivity of 0.86 and specificity of 0.66 for 120 days. Additionally, we identified individual risk factors for dropout, including previous overdose and relapse and improvement in reported quality of life. Conclusions: Our informatics approach provides insight into an area where programs may allocate additional resources in order to retain high-risk individuals and increase the chances of success in recovery.

1. Introduction

According to the U.S. Department of Health and Human Services (HHS), an estimated 10.1 million people aged 12 or older used opioids in the United States in 2019, and 1.6 million people had a diagnosed opioid use disorder (OUD) [1]. Of these, over 48,000 people died from synthetic opioid-involved overdoses. In 2017, the HHS declared a public health emergency and announced a five-point strategy to combat the opioid crisis.
The Houston Emergency Opioid Engagement System (HEROES) was established in 2018 and created rapid access to a long-term recovery pathway for individuals with OUD. The program includes access to board-certified physicians for initiation of an approved medication for opioid use disorder (buprenorphine), plus additional comprehensive care that incorporates peer recovery coaching, individual and group counseling, and navigation to other services and resources based upon need, including recovery housing, employment, and support for justice-involved individuals [2].
As has been previously demonstrated, longer treatment retention is associated with a greater likelihood of abstinence from opioids [3]. However, early dropout is common in substance use treatment settings and may lead to poorer outcomes relative to those completing a full course of treatment [4]. Protective factors against recurrence of use and overdose are higher for individuals who remain active in treatment [5]. According to the principles of treatment outlined by the National Institute on Drug Abuse, remaining in treatment for an adequate amount of time is critical, and a minimum of 90 days is recommended [6].
Previous studies have focused on identifying factors that reduce dropout rate. Two studies of 78 and 202 participants focused on short-term dropout (3 and 4 months, respectively) [4,7]. Both identified young age as the leading factor underlying dropout rate, with the second study also identifying opioid use in the first month. A recent study focused on multiple dimensions, including demographic and psychosocial variables, and discovered that patients with greater stress levels and a sustained history of emotional abuse were more likely to drop out of treatment [8]. Finally, two longer-term studies found that younger age, minority ethnicity, and unemployment status were also associated with poor retention [9,10]. The studies mentioned calculated single-factor statistics and not overall risk.
Machine learning methods for developing prediction models are data-driven and growing in their application to substance use treatment analysis [11]. For example, two machine learning methods have attempted to take the prediction task from individual factors to multi-factorial models based on administrative claims data. Acion et al. compared different machine learning methods for predicting completed treatments as an indication of treatment success [12], but their study did not take into account the time in which the treatment was discontinued. Hasan et al. focused on buprenorphine treatment and predicted buprenorphine discontinuation within one year [13]. However, this study relied only on available drug prescription, resulting in low performance. Furthermore, prediction for discontinuation after a year limits the ability to detect problems early and possibly intervene.
Therefore, we decided to utilize machine learning methods to improve retention rates in our program and to inform the literature on treatment retention. Leveraging this approach may offer program administrators and clinicians the opportunity to divert resources and engage participants at the highest risk for drop out early in the program. We introduced a machine learning model to a retrospective database of over 700 enrolled patients to identify individuals at higher risk of dropping from the program after 90 and after 120 days. The benefit of our approach over previous methods is that we are using a richer set of information on each participant in the program, including demographic, clinical, behavioral/mental health, and extensive history about prior substance use and criminal activity. Furthermore, as the program is ongoing, we can directly implement an intervention program to retain those at high risk for early dropout (after 90 and 120 days). Thus, our encouraging model performance in detecting the set of risk factors in these high-risk individuals provides a tool for our program and like-programs across the U.S. for retention and ultimately for improving recovery for individuals suffering with OUD, as they are retained longer in the program.

2. Materials and Methods

2.1. Data

As of May 2021 (here forth termed “index date”), the program database included 715 individuals that had joined more than 90 days before the index date and 691 individuals that had joined more than 120 days prior to the index date. The data are collected through self-reported questionnaires and through a health evaluation by a nurse practitioner or physician assistant. Data are stored in a Research Electronic Data Capture (REDCap) project database [14] (see Supplement, S1: BioPsychosocial_HEROES). Out of these individuals, 105 (15%) and 165 (24%) had dropped out of the program within less than 90 days and 120 days, respectively. Categorical factors (i.e., factors with more than two discrete values) were converted into dummy variables, resulting in 163 factors, including 10 continuous value factors and 153 dichotomous factors (binary) that describe these individuals. We further removed 55 factors with more than 50% missing values (Figure 1). We imputed each of the remaining 108 factors, including 1 continuous factor (age) and 107 dichotomous factors, using the average over the available information for each factor, which translates to imputing with the fraction of positive values in each factor.

2.2. Prediction Scheme

We tested several machine learning classifiers, including logistic regression, radial basis support vector machine, random forest (with 100 trees, and the default parameters in Matlab’s TreeBagger implementation), and five different variants of ensemble based on boosting algorithms based on decision trees, including Adaptive boosting (AdaBoost), Gentle Boost, LogitBoost, Robust Boosting, and totally corrective boosting [15,16,17,18]. We used 10-fold cross-validation and used the specificity and sensitivity metrics to evaluate our predictions, where for the boosting algorithm, we applied a nested cross-validation scheme that uses training, testing, and validation sets for hyperparameter tuning with Matlab’s automatic boosting algorithm, which uses Bayesian optimization. Due to the large imbalance between the positive (those who dropped out) and the negative (those who were retained) groups, we sampled from the negative group the same number of individuals as in the positive group and ran the cross-validation procedure, repeating the sub-subsampling process five times with different random sub-samples to verify the robustness of the results. Results provided are averaged over the five independent repetitions. All analyses were conducted using Matlab R2019a.

2.3. Factor Ranking

We estimated the predictor importance by permutation of out-of-bag predictor observations in the random forest classifier model. Additionally, we verified that these ranking were consistent with the association in statistical tests, where we used Mann–Whitney U test between the retained and dropped out groups for continuous factors and Fisher exact test for dichotomous factors. Significant factors with q-values (i.e., p-values adjusted for Benjamini-Hochberg (BH) false discovery rate) of .05 were selected.

3. Results

3.1. Machine Learning Framework to Identify Program Dropout

We evaluated the dropout rate in increments of 30 days (Figure 2), identifying that the dropout became substantial at 90- and 120-days periods reaching 15% (90 days) and 24% (120 days). Thus, we focused on predicting the characteristics of the patients likely to drop out of the program after 90 and 120 days.
We tested several machine learning classifiers to predict dropout (Methods, Table 1), selecting random forest as our optimal classifier (see Figure 3 for data analysis plan). Random forest had slightly better overall performance tradeoff than the rest of the classifiers, having the highest sensitivity (recall) with a comparable specificity to other classifiers (Table 1). We considered higher sensitivity more important than the prediction outcome, since having a low number of false negatives (fewer individuals who dropped out that were missed by the classifier) would be more critical for retention than possibly investing more resources in higher number of false positives (individuals we marked as high risk for dropout but would not drop out) with the current lower specificity. Thus, we chose random forest as our selected classifier. The classifier obtained sensitivity of 0.81 ± 0.02 and specificity of 0.65 ± 0.05 for the 90-day threshold and sensitivity of 0.86 ± 0.03 and specificity of 0.66 ± 0.02 for the 120-day threshold.

3.2. Risk Factors Associated with Program Dropout

We next identified factors associated with dropout using ranked predictor importance in the classification model using out-of-bag predictor observations technique and verified using statistical tests on each independent factor (Methods). There were 27 factors that passed the false discovery rate of 0.05 at both time points (Table S1).
Three factors were consistently ranked high in both the 90- and 120-day check dates, identified both by statistical test and predictor importance in the prediction model. These features include whether the individual have overdosed in the past (Benjamini-Hochberg false discovery rate (FDR)-adjusted, p < e−15), improvement in quality of life (FDR-adjusted, p < 4e−15), and whether they relapsed since joining the program ((FDR-adjusted, p < 3e−6), see Table 2. Interestingly, we did not observe significant differences in the ages of the individuals who dropped out and those who were retained as a previously published factor.
The top factor was whether the individual had overdosed in the past. Individuals who dropped out had 3.2 to 4.8 times higher incidence of past overdosing (i.e., the number of individuals with past dosing was 3.2 to 4.8 times higher in the dropout group, adjusted by group size). The second top factor was improvement in quality of life. More than 94% of the individuals who dropped out have reported improvement in their quality of life while only 64% (90 days)–59% (120 days) in the retained group reported improved quality of life. Finally, the group of individuals who dropped out had a 1.8–2 times higher incidence of reported relapses since joining the program.

4. Discussion

We have presented a model for predicting early dropout from a long-term recovery program for opioid use disorder at 90 and 120 days. Our model obtained good sensitivity and specificity and enables the identification of the top contributing factors. Overall, analyses of treatment dropout rates were low for program participants, with 85% active and engaged in treatment at 90 days and 76% active at 120 days post enrollment. Compared to a recent systematic and meta-analysis review of dropout rates that reports average dropout rates for targeted opioid treatment programs (39%) and target heroin treatment programs (26%), overall retention was high for participants in the analyzed program model [19]. While retention in the program was high overall, there were some distinguishable differences between those who dropped out of treatment versus those who remained in treatment.
Some variables were significantly related to retention probability. The top contributing factor was whether the individual had overdosed in the past. Individuals who dropped out of the program had overdosed in the past more than those who were retained, suggesting that individuals with an extensive history of substance use may need additional support maintaining engagement in an outpatient program. A recent study found individuals who experienced a non-fatal overdose were at increased risk of successive overdose and were more likely to engage in polysubstance use [20,21]. Moreover, polysubstance use may contribute to lifetime prevalence of overdose. Additional research from the National Epidemiologic Survey on Alcohol and Related Conditions III found that 98% of individuals who reported opioid use reported using other substances [22]. The correlation between opioid use, polysubstance use, and lifetime prevalence of overdose is well established by research, further supporting comprehensive services, including treatment for polysubstance use.
The second top factor was improvement in quality of life. More than 94% of those who dropped out reported an improvement in their quality of life, while only 64% (90 days)–59% (120 days) in the retained group reported improvements, per the quality-of-life survey. We estimate that individuals who had experienced improvements in their quality of life may hold the perception of doing well and feel that they do not need the same level of treatment to maintain their recovery, or they may have transitioned to longer-term programs with less intensity. This type of non-adherence to treatment or medication protocols when one is feeling better is not isolated to OUD. Similar studies focused on patients with depression have found that individuals stop taking their medications when they are feeling better [23]. While most research supports improvements in reported quality of life during substance abuse treatment, there is no consensus on the exact relationship between quality of life and treatment retention or dropout. Some research found quality of life increased early on in treatment from baseline, then decreased or diminished after leaving treatment [24,25,26,27,28]. Although research supports quality of life as an important factor in sustained abstinence, there are myriad of factors related to quality of life, and no standardized instrument to measure it among those with OUD [29,30].
Finally, individuals who dropped out had higher incidence of relapses since joining the program, which is likely an indication they needed increased accountability, additional or varied support, or would benefit from a referral to a higher level of care [31]. OUD is a chronic and relapsing condition, with relapse rates surpassing 90% [32,33]. Individuals with multiple psychosocial problems and instability are often hard to engage and retain in treatment [34]. Although MOUD is shown to reduce opioid-related mortality, some individuals receiving MOUD continue to use opioids and are at risk for a fatal overdose [35]. Another study found that individuals with an overdose-related ED encounter in the previous year were least likely to be abstinent from opioid use at baseline and more likely to use illicit benzodiazepines while enrolled in treatment [21]. Research on predictors of relapse has found that psychological factors, such as history of trauma and depression, and biological factors, such as cortisol levels and adrenal sensitivity, are associated with increased risk of relapse [36]. These data support an extensive approach to treatment that addresses biopsychosocial factors associated with relapse among those receiving MOUD.
Recovery from addiction is multifactorial and may take multiple attempts for an individual. Treatment that includes MOUD, behavioral health, and social support is widely accepted as the most effective treatment; yet, a considerable number of individuals receiving treatment perpetually relapse and often overdose, suggesting treatment resistance in some patients with OUD [37]. There are other treatment-resistant disorders and diseases, including mood disorders, cancer, and hypertension, which are widely accepted and supported by science; therefore, the idea of treatment-resistant OUD may be plausible and calls for further research [38,39,40]. We also found that individuals with past prescription opioid use, past legal issues, or a history of benzodiazepines were more likely to drop out at 90 days (Supplementary Table S1). Our findings support other research that has found use of benzodiazepines and current and past legal issues associated with increased dropout [10]. These types of activities may signal a lack of “readiness” for recovery or the ability to create change at the time of treatment [41,42].
Previous studies have identified younger age, cocaine and heroin use, lower doses of methadone, criminal activity/incarceration, and negative attitudes to methadone maintenance treatment as associated factors [43,44]. While age was not significant in our analysis, cocaine and heroin use and incarceration were statistically significant (p < 0.008), but did not reach the top ranked features listed in Table 2. Methadone use was not recorded in our data.
While we obtained good prediction performance, there could be factors that were not collected on these individuals that affect their choice to drop out of the program. One limitation of our model is that we use only structured factors. However, the program also collects free text information on participants that could potentially be converted to additional features that will enhance the model for future studies. Overall, our findings are consistent with previous research on treatment retention and support further exploration of treatment-resistant opioid use disorder.
The findings from this research support the concept of using a tiered risk approach, ranging from high to low risk of early dropout. A predictive model may be developed with a greater understanding of the factors that influence patient participation choices. The model allows for a system of classification; therefore, resources can be allocated efficiently, and treatment plans can be tailored to the individual. For example, participants in the highest risk group may receive more intensive follow-up from peer recovery coaches multiple times per week and might be required to attend additional group counseling and one-on-one counseling. Higher risk groups may also be required to submit urine drug tests on a regular basis to confirm abstinence from illicit substances, while participants in the lowest risk group may receive follow up from a peer recovery coach less often and be required to attend group counseling and one-on-one counseling every few weeks. By customizing the treatment and care provided to participants within each group, it becomes possible to provide participants with the greatest needs with the attention and resources required.

5. Conclusions

In this research, we analyzed variables for persistence in treatment for a cohort of individuals engaging in a program for OUD. We developed a data-driven prediction model using machine learning techniques and found that individuals with prior medication for OUD and multiple relapses may be considered higher risk and more likely to drop out of the program. Further qualitative research is needed to determine the optimal timing of interventions and the varied services needed for individuals to persist in treatment.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/healthcare10020223/s1, S1: BioPsychosocial_HEROES. Table S1. (This is the intake form used to collect participant data upon entry into the treatment program).

Author Contributions

A.G. and T.C.-L. conceived the study. T.C.-L., J.R.L. and C.B.-B. collected the data. A.G. analyzed the data. A.G., T.C.-L., C.B.-B. and A.Y. wrote the manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This research was partially funded by the Texas Targeted Opioid Response (TTOR), operated by the Texas Health and Human Services Commission (HHSC) through federal funding from the Substance Abuse and Mental Health Services Administration (SAMHSA).

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and was approved by the University of Texas Health Science Center at Houston (approval no. HSC-SBMI-17-1021). All participants provided written informed consent prior to enrollment in the study. Individuals in this study are further protected by a Confidentiality Certificate under Section 301(d) of the Public Health Services Act 42 U.S.C. Section 241(d) issued by the U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The datasets generated and/or analyzed during the current study are not publicly available due to them containing information that could compromise research participant privacy/consent.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

AdaBoostAdaptive Boosting
AUCArea Under the Curve
FDRFalse Discovery Rate
HEROESHouston Emergency Opioid Engagement System
OUDOpioid Use Disorder

References

  1. Substance Abuse and Mental Health Services Administration. Key Substance Use and Mental Health Indicators in the United States: Results from the 2019 National Survey on Drug Use and Health (HHS Publication No. PEP20-07-01-001, NSDUH Series H-55); Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration: Rockville, MD, USA, 2020. Available online: https://www.samhsa.gov/data/ (accessed on 2 January 2022).
  2. Langabeer, J.R.; Champagne-Langabeer, T.; Yatsco, A.J.; O’Neal, M.M.; Cardenas-Turanzas, M.; Prater, S.; Luber, S.; Stotts, A.; Fadial, T.; Khraish, G.; et al. Feasibility and outcomes from an integrated bridge treatment program for opioid use disorder. J Am. Coll. Emerg. Physicians Open 2021, 2, e12417. [Google Scholar] [CrossRef] [PubMed]
  3. Hser, Y.-I.; Evans, E.; Grella, C.; Ling, W.; Anglin, D. Long-term course of opioid addiction. Harv. Rev. Psychiatry 2015, 23, 76–89. [Google Scholar] [CrossRef] [PubMed]
  4. McHugh, R.K.; Murray, H.W.; Hearon, B.A.; Pratt, E.M.; Pollack, M.H.; Safren, S.A.; Otto, M.W. Predictors of dropout from psychosocial treatment in opioid-dependent outpatients. Am. J. Addict. 2013, 22, 18–22. [Google Scholar] [CrossRef] [PubMed]
  5. Krawczyk, N.; Mojtabai, R.; Stuart, E.A.; Fingerhood, M.; Agus, D.; Lyons, B.C.; Weiner, J.P.; Saloner, B. Opioid agonist treatment and fatal overdose risk in a state-wide US population receiving opioid use disorder services. Addiction 2020, 115, 1683–1694. [Google Scholar] [CrossRef]
  6. National Institute on Drug Abuse (NIDA). Principles of Effective Treatment; National Institute on Drug Abuse Website. Available online: https://www.drugabuse.gov/publications/principles-drug-addiction-treatment-research-based-guide-third-edition/principles-effective-treatment (accessed on 2 January 2022).
  7. Marcovitz, D.E.; McHugh, R.K.; Volpe, J.; Votaw, V.; Connery, H.S. Predictors of early dropout in outpatient buprenorphine/naloxone treatment. Am. J. Addict. 2016, 25, 472–477. [Google Scholar] [CrossRef] [Green Version]
  8. Panlilio, L.V.; Stull, S.W.; Kowalczyk, W.J.; Phillips, K.A.; Schroeder, J.R.; Bertz, J.W.; Vahabzadeh, M.; Lin, J.L.; Mezghanni, M.; Nunes, E.V.; et al. Stress, craving and mood as predictors of early dropout from opioid agonist therapy. Drug Alcohol Depend 2019, 202, 200–208. [Google Scholar] [CrossRef]
  9. Weinstein, Z.M.; Kim, H.W.; Cheng, D.M.; Quinn, E.; Hui, D.; Labelle, C.T.; Drainoni, M.-L.; Bachman, S.S.; Samet, J.H. Long-term retention in office based opioid treatment with buprenorphine. J. Subst. Abus. Treat. 2017, 74, 65–70. [Google Scholar] [CrossRef] [Green Version]
  10. O’Connor, A.M.; Cousins, G.; Durand, L.; Barry, J.; Boland, F. Retention of patients in opioid substitution treatment: A systematic review. PLoS ONE 2020, 15, e0232086. [Google Scholar] [CrossRef]
  11. Bharat, C.; Hickman, M.; Barbieri, S.; Degenhardt, L. Big data and predictive modelling for the opioid crisis: Existing research and future potential. Lancet Digit Health 2021, 3, e397–e407. [Google Scholar] [CrossRef]
  12. Acion, L.; Kelmansky, D.; van der Laan, M.; Sahker, E.; Jones, D.; Arndt, S. Use of a machine learning framework to predict substance use disorder treatment success. PLoS ONE 2017, 12, e0175383. [Google Scholar] [CrossRef]
  13. Hasan, M.M.; Young, G.J.; Shi, J.; Mohite, P.; Young, L.D.; Weiner, S.G. A machine learning based two-stage clinical decision support system for predicting patients’ discontinuation from opioid use disorder treatment: Retrospective observational study. BMC Med. Inform. Decis. Mak. 2021, 21, 1–21. [Google Scholar] [CrossRef] [PubMed]
  14. Patridge, E.F.; Bardyn, T.P. Research electronic data capture (REDCap). J. Med. Libr. Assoc. JMLA 2018, 106, 142. [Google Scholar] [CrossRef] [Green Version]
  15. Freund, Y.; Schapire, R.; Abe, N. A short introduction to boosting. J.-Jpn. Soc. Artif. Intell. 1999, 14, 1612. [Google Scholar]
  16. Friedman, J.; Hastie, T.; Tibshirani, R. Additive logistic regression: A statistical view of boosting (with discussion and a rejoinder by the authors). Ann. Stat. 2000, 28, 337–407. [Google Scholar] [CrossRef]
  17. Freund, Y. A more robust boosting algorithm. arXiv 2009, arXiv:0905.2138. [Google Scholar]
  18. Warmuth, M.K.; Liao, J.; Rätsch, G. Totally corrective boosting algorithms that maximize the margin. In Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, PA, USA, 25–29 June 2006; pp. 1001–1008. [Google Scholar]
  19. Lappan, S.N.; Brown, A.W.; Hendricks, P.S. Dropout rates of in-person psychosocial substance use disorder treatments: A systematic review and meta-analysis. Addiction 2020, 115, 201–217. [Google Scholar] [CrossRef] [PubMed]
  20. Shrestha, S.; Stopka, T.J.; Hughto, J.M.W.; Case, P.; Palacios, W.R.; Reilly, B.; Green, T.C. Prevalence and correlates of non-fatal overdose among people who use drugs: Findings from rapid assessments in Massachusetts, 2017-2019. Harm Reduct. J. 2021, 18, 93. [Google Scholar] [CrossRef] [PubMed]
  21. Au, V.Y.O.; Rosic, T.; Sanger, N.; Hillmer, A.; Chawar, C.; Worster, A.; Marsh, D.C.; Thabane, L.; Samaan, Z. Factors associated with opioid overdose during medication-assisted treatment: How can we identify individuals at risk? Harm Reduct. J. 2021, 18, 71. [Google Scholar] [CrossRef]
  22. Hasin, D.S.; Grant, B.F. The National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) Waves 1 and 2: Review and summary of findings. Soc. Psychiatry Psychiatr. Epidemiol. 2015, 50, 1609–1640. [Google Scholar] [CrossRef] [Green Version]
  23. Gabriel, A.; Violato, C. Knowledge of and attitudes towards depression and adherence to treatment: The Antidepressant Adherence Scale (AAS). J. Affect. Disord. 2010, 126, 388–394. [Google Scholar] [CrossRef]
  24. De Maeyer, J.; van Nieuwenhuizen, C.; Bongers, I.L.; Broekaert, E.; Vanderplasschen, W. Profiles of quality of life in opiate-dependent individuals after starting methadone treatment: A latent class analysis. Int. J. Drug Policy 2013, 24, 342–350. [Google Scholar] [CrossRef] [PubMed]
  25. Nosyk, B.; Guh, D.P.; Sun, H.; Oviedo-Joekes, E.; Brissette, S.; Marsh, D.C.; Schechter, M.T.; Anis, A.H. Health related quality of life trajectories of patients in opioid substitution treatment. Drug Alcohol Depend. 2011, 118, 259–264. [Google Scholar] [CrossRef] [PubMed]
  26. Nosyk, B.; Bray, J.W.; Wittenberg, E.; Aden, B.; Eggman, A.A.; Weiss, R.D.; Potter, J.; Ang, A.; Hser, Y.I.; Ling, W.; et al. Short term health-related quality of life improvement during opioid agonist treatment. Drug Alcohol Depend. 2015, 157, 121–128. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  27. Kelly, J.F.; Greene, M.C.; Bergman, B.G. Beyond Abstinence: Changes in Indices of Quality of Life with Time in Recovery in a Nationally Representative Sample of U.S. Adults. Alcohol. Clin. Exp. Res. 2018, 42, 770–780. [Google Scholar] [CrossRef] [PubMed]
  28. Mitchell, S.G.; Gryczynski, J.; Schwartz, R.P.; Myers, C.P.; O’Grady, K.E.; Olsen, Y.K.; Jaffe, J.H. Changes in Quality of Life following Buprenorphine Treatment: Relationship with Treatment Retention and Illicit Opioid Use. J. Psychoact. Drugs 2015, 47, 149–157. [Google Scholar] [CrossRef]
  29. Bray, J.W.; Aden, B.; Eggman, A.A.; Hellerstein, L.; Wittenberg, E.; Nosyk, B.; Stribling, J.C.; Schackman, B.R. Quality of life as an outcome of opioid use disorder treatment: A systematic review. J. Subst. Abus. Treat. 2017, 76, 88–93. [Google Scholar] [CrossRef] [Green Version]
  30. Strada, L.; Vanderplasschen, W.; Buchholz, A.; Schulte, B.; Muller, A.E.; Verthein, U.; Reimer, J. Measuring quality of life in opioid-dependent people: A systematic review of assessment instruments. Qual. Life Res. 2017, 26, 3187–3200. [Google Scholar] [CrossRef] [Green Version]
  31. Center for Substance Abuse Treatment. Substance Abuse: Clinical Issues in Intensive Outpatient Treatment; Treatment Improvement Protocol (TIP) Series 47. DHHS Publication No. (SMA) 06-4182; Substance Abuse and Mental Health Services Administration: Rockville, MD, USA, 2006.
  32. Strang, J.; Volkow, N.D.; Degenhardt, L.; Hickman, M.; Johnson, K.; Koob, G.F.; Marshall, B.D.L.; Tyndall, M.; Walsh, S.L. Opioid use disorder. Nat. Rev. Dis. Primers 2020, 6, 3. [Google Scholar] [CrossRef] [PubMed]
  33. Smyth, B.P.; Barry, J.; Keenan, E.; Ducray, K. Lapse and relapse following inpatient treatment of opiate dependence. Ir. Med. J. 2010, 103, 176–179. [Google Scholar]
  34. Chopra, N.; Marasa, L.H. The opioid epidemic: Challenges of sustained remission. Int. J. Psychiatry Med. 2017, 52, 196–201. [Google Scholar] [CrossRef]
  35. Ma, J.; Bao, Y.P.; Wang, R.J.; Su, M.F.; Liu, M.X.; Li, J.Q.; Degenhardt, L.; Farrell, M.; Blow, F.C.; Ilgen, M.; et al. Effects of medication-assisted treatment on mortality among opioids users: A systematic review and meta-analysis. Mol. Psychiatry 2019, 24, 1868–1883. [Google Scholar] [CrossRef] [PubMed]
  36. Sinha, R. New findings on biological factors predicting addiction relapse vulnerability. Curr. Psychiatry Rep. 2011, 13, 398–405. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  37. Patterson Silver Wolf, D.A.; Gold, M. Treatment resistant opioid use disorder (TROUD): Definition, rationale, and recommendations. J. Neurol. Sci. 2020, 411, 116718. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  38. Del Casale, A.; Sorice, S.; Padovano, A.; Simmaco, M.; Ferracuti, S.; Lamis, D.A.; Rapinesi, C.; Sani, G.; Girardi, P.; Kotzalidis, G.D.; et al. Psychopharmacological Treatment of Obsessive-Compulsive Disorder (OCD). Curr. Neuropharmacol. 2019, 17, 710–736. [Google Scholar] [CrossRef]
  39. Bergfeld, I.O.; Mantione, M.; Figee, M.; Schuurman, P.R.; Lok, A.; Denys, D. Treatment-resistant depression and suicidality. J. Affect. Disord. 2018, 235, 362–367. [Google Scholar] [CrossRef]
  40. Blum, K.A. Treatment-Resistant Hodgkin Lymphoma: Defining the Role of Autologous Transplantation. Cancer J. 2018, 24, 244–248. [Google Scholar] [CrossRef]
  41. Bozinoff, N.; Anderson, B.J.; Bailey, G.L.; Stein, M.D. Correlates of Stigma Severity among Persons Seeking Opioid Detoxification. J. Addict. Med. 2018, 12, 19–23. [Google Scholar] [CrossRef]
  42. Champagne-Langabeer, T.; Swank, M.W.; Langabeer, J.R. Routes of non-traditional entry into buprenorphine treatment programs. Subst. Abus. Treat Prev. Policy 2020, 15, 6. [Google Scholar] [CrossRef] [Green Version]
  43. Volkow, N.D.; Jones, E.B.; Einstein, E.B.; Wargo, E.M. Prevention and Treatment of Opioid Misuse and Addiction: A Review. JAMA Psychiatry 2019, 76, 208–216. [Google Scholar] [CrossRef]
  44. Oquendo, M.A.; Volkow, N.D. Suicide: A Silent Contributor to Opioid-Overdose Deaths. N. Engl. J. Med. 2018, 378, 1567–1569. [Google Scholar] [CrossRef]
Healthcare 10 00223 g001 550
Figure 1. Histogram of the fraction of missing values per factor. Factors with over 50% missing values were removed.
Figure 1. Histogram of the fraction of missing values per factor. Factors with over 50% missing values were removed.
Healthcare 10 00223 g001
Healthcare 10 00223 g002 550
Figure 2. Program retention rates at 30-day time intervals. Y-axis displays the fraction of individuals retained at each time point since joining the program.
Figure 2. Program retention rates at 30-day time intervals. Y-axis displays the fraction of individuals retained at each time point since joining the program.
Healthcare 10 00223 g002
Healthcare 10 00223 g003 550
Figure 3. Schematic of the data analysis pipeline. Information on the individuals that are enrolled in the program and longitudinal information is recorded in a REDCap database. A retrospective analysis generated a prediction model based on information collected prior to the index dates at 90 and 120 days.
Figure 3. Schematic of the data analysis pipeline. Information on the individuals that are enrolled in the program and longitudinal information is recorded in a REDCap database. A retrospective analysis generated a prediction model based on information collected prior to the index dates at 90 and 120 days.
Healthcare 10 00223 g003
Table
Table 1. Performance of different machine learning classifiers on the prediction of dropout.
Table 1. Performance of different machine learning classifiers on the prediction of dropout.
90 Days120 Days
MethodSpecificitySensitivitySpecificitySensitivity
Logistic Regression0.22 ± 0.30.9 ± 0.130.44 ± 0.270.81 ± 0.12
Radial Basis Support Vector Machines0.54 ± 0.020.63 ± 0.040.57 ± 0.030.66 ± 0.03
AdaBoost0.62 ± 0.050.82 ± 0.070.66 ± 0.010.83 ± 0.03
Gentle Boost0.63 ± 0.050.81 ± 0.020.67 ± 0.040.79 ± 0.03
Logit Boost0.64 ± 0.050.79 ± 0.040.68 ± 0.020.81 ± 0.04
Robust Boost0.62 ± 0.060.79 ± 0.020.69 ± 0.060.8 ± 0.05
Total Boost0.61 ± 0.030.81 ± 0.020.66 ± 0.030.84 ± 0.02
Random Forest0.65 ± 0.050.81 ± 0.020.66 ± 0.020.86 ± 0.03
Table
Table 2. Top contributing factors associated with individuals who dropped out at 90 and 120 days. Individuals with missing data are excluded.
Table 2. Top contributing factors associated with individuals who dropped out at 90 and 120 days. Individuals with missing data are excluded.
FactorIndividuals Who Dropped 90 Days
(%)		Individuals
Retained
90 Days
(%)		Individuals Who Dropped
120 Days
(%)		Individuals
Retained 120 Days
(%)		FDR-Adjusted
p-Values
(90 Days, 120 Days)
Have you overdosed?58 (56%)91 (17%)84 (52.2%)49 (11.0%)e−15, 2e−26
QoL improvement100 (95%)347 (64%)155 (93.9%)269 (59.0%)4e−15, 4e−25
Have you relapsed since joining?58 (55%)169 (31%)90 (54.5%)126 (27.6%)3e−6, 2e−11
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Gottlieb, A.; Yatsco, A.; Bakos-Block, C.; Langabeer, J.R.; Champagne-Langabeer, T. Machine Learning for Predicting Risk of Early Dropout in a Recovery Program for Opioid Use Disorder. Healthcare 2022, 10, 223. https://doi.org/10.3390/healthcare10020223

AMA Style

Gottlieb A, Yatsco A, Bakos-Block C, Langabeer JR, Champagne-Langabeer T. Machine Learning for Predicting Risk of Early Dropout in a Recovery Program for Opioid Use Disorder. Healthcare. 2022; 10(2):223. https://doi.org/10.3390/healthcare10020223

Chicago/Turabian Style

Gottlieb, Assaf, Andrea Yatsco, Christine Bakos-Block, James R. Langabeer, and Tiffany Champagne-Langabeer. 2022. "Machine Learning for Predicting Risk of Early Dropout in a Recovery Program for Opioid Use Disorder" Healthcare 10, no. 2: 223. https://doi.org/10.3390/healthcare10020223

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop