1. Introduction
Evidence suggests that despite undergoing a preoperative consultation and participating in the informed consent process, patients frequently exhibit limited understanding regarding potential postoperative complications [
1,
2,
3]. There is cumulative evidence in the literature to suggest that decision aids often increase patients’ understanding of proposed treatment and interventions and assist in the informed consent process [
4].
Hence, estimating the risk of postoperative complications is crucial in the shared decision-making process. It helps clinicians and multidisciplinary teams to plan preoperative management and postoperative care, including inpatient stay, and to predict likely duration of recovery.
Over the last decade, the burden of cancer incidence as well as mortality has grown rapidly worldwide and is reflected in the rates of gynaecological cancer increase, with cervical and uterine cancer incidence and mortality being the seventh most common cancers in women [
5]. With the increasing rates of GO operations, especially technically challenging minimally invasive surgery (MIS) [
6,
7] in obese patients and patients with multiple comorbidities, it is even more important to be able to accurately predict the likelihood of perioperative complications and involve patients in joint decision-making about their surgery. Iyer et al. were able to demonstrate, in their prospective multi-centre study, that the intraoperative rate of complications amongst patients undergoing surgery for cancer was 5.4%, whereas the postoperative complications rate was 27.1% [
8].
According to a recent review, cardiopulmonary exercise testing (CPET) is providing an objective evaluation of cardiorespiratory fitness and functional capacity resulting in individualised risk profiles, which could guide shared decision-making [
9]. However, although it is increasingly being used in the UK to evaluate preoperatively the postoperative morbidity, it is a very expensive method with limited capacity [
10].
Several online risk scoring tools are in clinical use, all of which attempt to predict the risk of postoperative morbidity and mortality during major surgical procedures. One such calculator, originally developed by Copeland et al. in 1991, is the Physiologic and Operative Severity Score for the Enumeration of Mortality and Morbidity (POSSUM) risk scoring [
11], subsequently modified to Portsmouth POSSUM (P-POSSUM) to provide a more accurate prediction [
12], which is widely accepted in the UK for postoperative mortality and morbidity risk prediction. P-POSSUM uses the preoperative physiological scores and intraoperative surgical scores of patients for generation of postoperative risk scores [
11]. However, published data across different specialties, including GO, suggest that it may overestimate risk, especially the mortality rates [
13,
14,
15,
16,
17], which may cause undue patient anxiety, influence the surgical decision-making and potentially prolong the length of stay (LOS). Moreover, P-POSSUM requires intraoperative parameters for morbidity and mortality calculation, which in turn makes the prediction less accurate.
The American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) surgical risk calculator was developed in 2013 based on the NSQIP database, which was designed to measure and improve the quality of surgical care [
18]. The database was derived from standardised prospective surgical data collection, across more than 700 hospitals and nine specialties in the USA, and further evaluation of the postoperative complications. The calculator derived from this database is a validated web-based tool requiring 19 preoperative risk factors for the prediction of LOS and 11 postoperative outcomes within 30 days of surgery (plus 2 outcomes in surgery involving gastrointestinal tract—ileus and anastomotic leak), which allows surgeons to receive a customised, patient-specific risk for their surgery [
19].
A number of studies have explored the applicability and efficacy of the ACS-NSQIP surgical risk calculator in the context of GO [
20,
21,
22,
23]. However, none of these studies have compared ACS-NSQIP to any other tools and there have not previously been attempts to analyse the tool in a cohort of patients undergoing RS.
The objective of our study was to explore the validity and suitability, as well as to compare the performance, of the P-POSSUM and ACS-NSQIP surgical risk calculators in the setting of GO minimally invasive robotic-assisted operations and hereby we present our pilot study that shows the performance of ACS-NSQIP compared to P-POSSUM.
2. Ethics and Approvals
This project was approved by the Audit and Quality Improvement department as part of the Research and Development office of the Royal Surrey NHS Foundation Trust (RSNFT) as part of the studentship with University of Brighton for a retrospective analysis of 1500 patients undergoing RS and 1500 patients undergoing open surgery.
3. Materials and Methods
3.1. Cohort Selection
Yearly operation lists were obtained from the designated electronic GO database of RSNFT. All patients were divided by the year of the operation (2009–2020) and by the type of surgery they had undergone (open/robotic/laparoscopic). Women who underwent other diagnostic procedures have been excluded from this study, as well as cases where there was not enough information for the ACS-NSQIP score calculation or the P-POSSUM score. While data collection is ongoing, the first 153 robotic cases were felt to be adequate for statistical analysis for this pilot and hence the first 153 robotic cases with complete data have been analysed.
In this report, we present the preliminary analysis of 153 patients who had undergone Da Vinci-assisted RS for suspected or confirmed gynaecological malignancies at RSNFT, based on the intention to treat analysis.
3.2. Data Collection
For all patients undergoing surgery at our cancer centre, data are recorded prospectively on a dedicated database. For each patient, the electronic database record was retrieved to collect P-POSSUM scores, along with the 19 pre-operative parameters required to calculate the ACS-NSQIP risk (
Table 1). The missing data were collected from paper patient records.
Following their surgery and discharge, patients have the option of contacting their local hospital, general practitioner (GP) or RSNFT for any concerns, and all patients at RSNFT are invited for a follow-up appointment within 30 days postoperatively when the course of recovery +/− further management/surveillance is discussed with the treating specialist. All readmissions and presentations to different trusts, visits to GP and treatments commenced within that period are discussed with the patient. Clinical nurse specialists are also contacted by other trusts when patients present/are admitted to other hospitals. These events are documented within the electronic database and/or patients’ records. The electronic database and paper records were also analysed to identify any actual postoperative 30-day complications and the LOS, which were recorded along with the rest of the data. Morbidity was defined by either the Clavien–Dindo score and/or the Postoperative Morbidity Survey (POMS) score within the 30 days after the surgery. All complications above Clavien–Dindo IIIa were classified as serious complications.
In addition to the variables listed above, a number of tumour characteristics were collected for each of the patients for general purposes, such as tumour site, neoplasm type and subtype, FIGO stage and grade.
The required parameters were entered into the online ACS-NSQIP surgical risk calculator tool [
24] for each patient, and the Common Procedural Terminology (CPT) code was selected as accurately as possible. In accordance with previous studies, if there was more than one procedure within the surgery, then more than one CPT code was selected and the scores for each were calculated; the highest percentage of each of the complications was selected for analysis [
20,
23]. Finally, the estimated LOS and risk of 11 postoperative outcomes were calculated (
Table 2): serious complication/any complication; (1) pneumonia; (2) cardiac complication; (3) surgical site infection (SSI); (4) urinary tract infection (UTI); (5) venous thromboembolism (VTE); (6) renal failure; (7) readmission; (8) return to the theatre; (9) death; (10) discharge to post-acute care; (11) sepsis.
3.3. Statistical Analysis
Firstly, baseline demographic variables were analysed to describe the selected cohort. Patients were categorised by age, ASA class, degree of obesity and types of comorbidities. Tumours were categorised as being malignant or benign, as well as by the site of neoplasm.
Statistical analysis was undertaken using Statistical Package for the Social Sciences (SPSS) version 24.0. The difference between each of the predicted postoperative outcomes, hence the definitiveness of the ACS-NSQIP surgical risk calculator, and the actual complications were measured using the Brier score. The closer the reliability term was to zero, the more reliable was the prediction of morbidity, mortality by P-POSSUM and ACS tools or specific complication by the ACS tool (
https://riskcalculator.facs.org/RiskCalculator/PatientInfo.jsp, accessed 2 March 2023).
Within the logistic regression model, we also determined the c-statistic standard measure of the predictive accuracy, which is equivalent to the area under the Receiver Operating Characteristic (ROC) curve when the outcome is binary [
25]. The ROC curve values range from 0 to 1, where 0 indicates an ineffective model and 1 means that the model is perfect for the prediction of complications [
26].
4. Results
In this report, we present results for the initial 153 patients. These patients had undergone a Da Vinci RS for gynaecological malignancy at RSNFT. Preoperative demographic data are presented in
Table 3. The vast majority of the patients were under 65 years of age (56.2%) and were classed as ASA 2 (66%). All patients had undergone planned procedures with no emergencies. Similar numbers of patients were of normal weight and overweight (30% and 27%, accordingly) and the majority of the patients were classed as obese (class 1–3) (43%).
In terms of comorbidities, a substantial proportion of the patients were found to have hypertension (35.3%), followed by 15% of smokers and 9.8% of non-insulin-dependent diabetes mellitus patients. Only a small number of patients were suffering from insulin-dependent diabetes mellitus (2.6%) and COPD (0.6%).
The majority of patients underwent surgery for malignancy (85%); 4.6% had borderline condition and 10.4% were found to have a benign tumour. A total of 64.7% of the tumours were located in endometrium, 29.4% in the cervix, 5.2% in the ovaries and one patient had a primary in another site (
Table 4).
The main cohort of patients had undergone hysterectomy with or without salpingoophorectomy and a staging procedure, lymphadenectomy and/or omentectomy with or without hysterectomy and/or salpingoophorectomy. The breakdown of the patients by the type of surgery is shown in
Table 4.
Of the 153 patients, 23 (15%) were found to have one or more complications and 13 (8.5%) had an event constituting a serious complication, as described previously. The breakdown of the actual postoperative complications is demonstrated in
Table 5. As shown, the most frequent complications in the sequence of their frequency are UTI (
n = 10), readmission (
n = 7), pneumonia (
n = 3) and SSI (
n = 3). Rarer complications included return to theatre (
n = 2), VTE (
n = 2), renal failure (
n = 1) and discharge to post-acute care facility (
n = 2). There were no deaths or cardiac complications observed in this cohort of patients.
When comparing the morbidity prediction ability of P-POSSUM and ACS-NSQIP surgical risk calculators, ACS-NSQIP showed a better predictive value than P-POSSUM with AUC 0.608 against 0.551 (
Figure 1). ACS-NSQIP was also found to statistically significantly predict postoperative complications better than P-POSSUM in patients with ASA class I and II.
The ACS-NSQIP calculator was found to perform best in predicting the risk of postoperative VTE (AUC = 0.793) and pneumonia (AUC = 0.657), followed by a worse prediction of readmission (AUC = 0.587) and UTI (AUC = 0.515). (
Figure 2).
When comparing the mortality predictions by ACS-NSQIP and P-POSSUM, the best results were shown by ACS-NSQIP (Brier (P-POSSUM mortality) = 0.0014; Brier (ACS mortality) = 0.000;
p = 0.084) (
Figure 3a) (
Table 6).
Given the lack of deaths in the cohort, it is evident that the P-POSSUM risk calculator is overestimating the mortality prediction compared to the ACS risk calculator, which has accurately predicted the mortality.
P-POSSUM was also found to statistically significantly overestimate morbidity, similarly to mortality risks (
p = 0.018), as shown on
Figure 3b and
Table 6.
Lastly, the performance of ACS-NSQIP in predicting individual complications was measured using the Brier score. The risk prediction tool showed the best accuracy in the prediction of major complications as follows: death, cardiac event, renal failure, VTE, return to theatre and SSI (
Figure 4) (
Table 7).
5. Discussion
To our knowledge, this study has been the first attempt to validate and compare the predictive value of surgical risk calculators, P-POSSUM and ACS-NSQIP, in the context of RS in GO. Previous studies have identified the lack of accuracy in the risk prediction of the P-POSSUM tool. It has been shown that P-POSSUM tends to overestimate mortality risk prediction in patients undergoing different types of surgery, specifically in GO [
13,
27], and our study has also confirmed that. This can affect the potential patients’ understanding of possible outcomes of surgery and influence the recovery, as the psychological aspect plays a vital role in cancer patient recovery [
28]. Moreover, we have been able to show that P-POSSUM overestimates morbidity along with mortality scores amongst our cohort of patients, which makes it an unreliable and potentially harmful tool to be used in this group of patients.
On the other hand, the results of our study show that ACS-NSQIP would have higher predictive value and would be a better tool to be used in GO. A study by Murray et al. with 142 patients had shown that ACS-NSQIP has a potential applicability in GO, especially since it could be used as a visual aid when counselling patients before the operation [
22]. ACS-NSQIP has been shown to be an accurate predictor, especially for death and VTE, which is in agreement with the study by Szender et al., which demonstrated that the Brier score approached the threshold for death, pneumonia, UTI and VTE in cancer patients, although the patient cohort of
n = 577 appear to have all had open surgery [
29]. Rivard et al. in their cohort of 1094 open cases were able to demonstrate a c-statistic value of >0.8 for death and 0.7 for cardiac events and renal failure [
20].
Thigpen et al. in their retrospective study, which included a cohort of patients who had undergone total abdominal or laparoscopic hysterectomy for benign gynaecological disease, found that the ACS-NSQIP calculator was effective in prediction of VTE and SSI but did not perform well in other categories of complications [
30]. These previous studies seem to be in concordance with our current research with regards to the efficacy of preoperative prediction of VTE during the postoperative period. In contrast, Teoh et al. report that the ACS-NSQIP risk calculator performs poorly in predicting 30-day postoperative complications in GO patients, especially compared to the general and colorectal surgical population [
21]. In their study, a mixed population of patients who had undergone MIS (laparoscopic or robotic) was reviewed in contrast to our pilot study, where we have reviewed exclusively robotic cases. This could influence the result, as patients selected for RS cases often have multiple comorbidities and are considered more complex cases. Manning-Geist et al. in their study (
n = 261) had also shown that ACS-NSQIP performed poorly in patients with ovarian cancer undergoing interval debulking surgery [
23].
Past studies have also shown that the presence of the following parameters increases the risk of postoperative complications in GO operations: diabetes, any comorbidity, increasing ASA class, age and BMI. The risk also depended on whether the approach was laparoscopic or open [
8]. These listed parameters are all included in the ACS-NSQIP surgical risk calculator. Nonetheless, Iyer et al. also showed that intra-operative factors also play a significant role in the prediction of postoperative complications. Those include estimated blood loss, duration of surgery and previous abdominal surgery. [
8] This could potentially be the reason the ACS-NSQIP tool has not shown such accurate predictive abilities in GO patients compared to the colorectal surgical cohort [
31]. Other reasons, as described by Rivard et al., could include the variety of complexity of procedures because of intraperitoneal disease burden, the need for a greater amount of CPT codes to be taken into account considering the complexity of the procedures and the frequent poor nutritional status of the GO patients weeks prior to the operations [
20].
6. Limitations and Insights
Being a pilot, retrospective data collection, the authors acknowledge that this poses a limitation to our study, potentially resulting in underreporting of postoperative complications due to inadequate follow-up or insufficient documentation in medical records.
All attempts were made to obtain complete records to obtain the results from both the ACS and P-POSSUM calculators for comparison. Nonetheless, data collection is currently underway with a much larger cohort of patients undergoing both open and minimally invasive surgery as well as validation in a prospective setting. It has been difficult to account for rare postoperative events, considering the number of participants, but plans exist to overcome this limitation in the future. The demographic profile of Surrey reveals a population characterised by relatively favourable ageing and affluence in comparison to broader demographics observed across the United Kingdom, though obesity as a risk factor prevails. In the ongoing data analysis, we strategically integrate multi-centre datasets to mitigate potential biases from specific demographic profiles. Lack of case selection has resulted in cases across all stages being incorporated, which the authors acknowledge is a limitation as there is evidence that the complexity of surgery is related to the disease burden as well as patient factors. Once again, being a pilot study, it does not allow for a stratified data analysis, which will be possible with larger datasets in the future.
This is the first study to analyse a separate cohort of patients who underwent Da Vinci robot-assisted operations, which has not been undertaken previously. Furthermore, a head-to-head comparison of the performance of two surgical risk calculators, P-POSSUM and ACS-NSQIP, in this cohort of patients, which is the main strength of this study, and has been undertaken at a tertiary referral cancer centre where patients are referred following a suspected or confirmed case of malignancy.
This preliminary analysis suggests a notable disparity in the prognostic accuracy between the ACS-NSQIP scoring system and the P-POSSUM model in the context of mortality prediction, discerning a distinct advantage in favour of the ACS-NSQIP scoring system, which demonstrated a null mortality rate, as opposed to the P-POSSUM model, which failed to anticipate such outcomes. This observation bears significant implications for our ongoing research endeavours, substantiating the heightened efficacy of the ACS-NSQIP scoring system in mortality prediction and thereby providing a more robust foundation for future investigations.
7. Conclusions
This pilot study, along with others, suggests that the P-POSSUM surgical risk tool overestimates morbidity and mortality scoring in GO patients undergoing RS and as such should not be the preferred risk calculator for this cohort of patients. The ACS-NSQIP surgical risk calculator informative value has been higher, especially in predicting major complications, and it may be used as an informed consent tool. However, future larger studies are necessary to further evaluate these prediction tools.
Author Contributions
Conceptualization, L.S., P.P. and T.K.M.; data curation, L.S., H.A. and T.K.M.; formal analysis, L.S., A.T., P.P. and T.K.M.; funding acquisition, L.S. and T.K.M.; investigation, L.S., H.A., M.E., D.H. and D.A.; methodology, L.S., A.T., P.W., P.P., M.S.F. and T.K.M.; project administration, L.S., M.S.F. and T.K.M.; resources, L.S., H.A., M.E., D.H., D.A. and T.K.M.; software, L.S., P.W. and T.K.M.; supervision, M.S.F. and T.K.M.; validation, L.S. and T.K.M.; visualization, L.S. and P.P.; writing—original draft, L.S., A.T., P.W., P.P., M.S.F. and T.K.M.; writing—review and editing, L.S., P.W., M.S.F. and T.K.M. All authors have read and agreed to the published version of the manuscript.
Funding
GRACE (Gynaecological Research and Clinical Excellence) Cancer Charity (Registered Charity No. 1189729) is funding the doctoral studies of Dr. Lusine Sevinyan.
Institutional Review Board Statement
This study was approved by the Research and Development office of the Royal Surrey NHS Foundation Trust (RSNFT) as part of the studentship with University of Brighton. Approval was granted for a retrospective analysis of 1500 patients undergoing RS and 1500 patients undergoing open surgery and this is part of that work. (RSNFT/2020/1164).
Informed Consent Statement
Not applicable.
Data Availability Statement
Data available upon request.
Acknowledgments
This research was supported by GRACE cancer charity. The authors would like to express their sincere gratitude to Simon Butler-Manuel for the encouragement and support. We also appreciate the constructive feedback from our peer reviewers, which significantly improved the quality of this manuscript.
Conflicts of Interest
The authors declare no conflicts of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.
Abbreviations
P-POSSUM | Portsmouth POSSUM |
POSSUM | Physiological and Operative Severity Score for the Enumeration of Mortality and Morbidity |
ACS-NSQIP | The American College of Surgeons National Surgical Quality Improvement Program |
NHS | The National Health Service |
GO | Gynaecological–Oncological |
RS | Robotic Surgery |
VTE | Venous Thromboembolism |
AUC | Area Under the Curve |
MIS | Minimally Invasive Surgery |
CPET | Cardio-Pulmonary Exercise Testing |
UK | The United Kingdom |
LOS | Length of Stay |
USA | The United States of America |
RSNFT | Royal Surrey NHS Foundation Trust |
ASA | American Society of Anesthesiologists |
SIRS | Systemic Inflammatory Response Syndrome |
COPD | Chronic Obstructive Pulmonary Disease |
BMI | Body Mass Index |
POMS | Postoperative Morbidity Survey |
FIGO | The International Federation of Gynecology and Obstetrics |
CPT | Common Procedural Terminology |
SSI | Surgical Site Infection |
UTI | Urinary Tract Infection |
PE | Pulmonary Embolism |
DVT | Deep Vein Thrombosis |
NPO | Nil Per Os |
NGT | NasoGastric Tube |
POD4 | Post Operative Day 4 |
GI | Gastro-Intestinal |
SPSS | Statistical Package for the Social Sciences |
ROC | Receiver Operating Characteristic |
EIN | Endometrial Intraepithelial Neoplasia |
CIN | Cervical Intraepithelial Neoplasia |
TLH | Total Laparoscopic Hysterectomy |
BSO | Bilateral Salpingo Oophorectomy |
RF | Renal Failure |
GP | General Practitioner |
References
- Kinnersley, P.; Phillips, K.; Savage, K.; Kelly, M.J.; Farrell, E.; Morgan, B.; Whistance, R.; Lewis, V.; Mann, M.K.; Stephens, B.L.; et al. Interventions to promote informed consent for patients undergoing surgical and other invasive healthcare procedures. Cochrane Database Syst. Rev. 2013, 2013, 1–247. [Google Scholar] [CrossRef] [PubMed]
- Perrenoud, B.; Velonaki, V.S.; Bodenmann, P.; Ramelet, A.S. The effectiveness of health literacy interventions on the informed consent process of health care users: A systematic review protocol. JBI Database Syst. Rev. Implement. Rep. 2015, 13, 82–94. [Google Scholar] [CrossRef] [PubMed]
- Bowling, A.; Rowe, G.; Lambert, N.; Waddington, M.; Mahtani, K.; Kenten, C.; Howe, A.; Francis, S. The measurement of patients’ expectations for health care: A review and psychometric testing of a measure of patients’ expectations. Health Technol. Assess. 2012, 16, 1–515. [Google Scholar] [CrossRef] [PubMed]
- Stacey, D.; Légaré, F.; Col, N.F.; Bennett, C.L.; Barry, M.J.; Eden, K.B.; Holmes-Rovner, M.; Llewellyn-Thomas, H.; Lyddiatt, A.; Thomson, R.; et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst. Rev. 2014, 2014, CD001431. [Google Scholar] [CrossRef]
- Sung, H.; Ferlay, J.; Siegel, R.L.; Laversanne, M.; Soerjomataram, I.; Jemal, A.; Bray, F. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J. Clin. 2021, 71, 209–249. [Google Scholar] [CrossRef] [PubMed]
- Conrad, L.B.; Ramirez, P.T.; Burke, W.; Naumann, R.W.; Ring, K.L.; Munsell, M.F.; Frumovitz, M. Role of minimally invasive surgery in gynecologic oncology: An updated survey of members of the Society of Gynecologic Oncology. Int. J. Gynecol. Cancer 2015, 25, 1121. [Google Scholar] [CrossRef] [PubMed]
- Scarpelli, E.; Armano, G.; Monfardini, L.; Valenti, A.; Barresi, G.; De Finis, A.; Rotondella, I.; Scebba, D.; Butera, D. Minimally invasive surgery in gynecological cancers: Update and systematic review. Clin. Exp. Obstet. Gynecol. 2022, 49, 88. [Google Scholar] [CrossRef]
- Iyer, R.; Gentry-Maharaj, A.; Nordin, A.; Burnell, M.; Liston, R.; Manchanda, R.; Das, N.; Desai, R.; Gornall, R.; Beardmore-Gray, A.; et al. Predictors of complications in gynaecological oncological surgery: A prospective multicentre study (UKGOSOC-UK gynaecological oncology surgical outcomes and complications). Br. J. Cancer 2015, 112, 475–484. [Google Scholar] [CrossRef]
- Otto, J.M.; Levett, D.Z.H.; Grocott, M.P.W. Cardiopulmonary Exercise Testing for Preoperative Evaluation: What Does the Future Hold? Curr. Anesthesiol. Rep. 2020, 10, 1–11. [Google Scholar] [CrossRef]
- Reeves, T.; Bates, S.; Sharp, T.; Richardson, K.; Bali, S.; Plumb, J.; Anderson, H.; Prentis, J.; Swart, M.; Levett, D.Z.H.; et al. Cardiopulmonary exercise testing (CPET) in the United Kingdom-a national survey of the structure, conduct, interpretation and funding. Perioper. Med. 2018, 7, 2. [Google Scholar] [CrossRef]
- Copeland, G.P.; Jones, D.; Walters, M. POSSUM: A scoring system for surgical audit. Br. J. Surg. 2005, 78, 355–360. [Google Scholar] [CrossRef] [PubMed]
- Prytherch, D.R.; Whiteley, M.S.; Higgins, B.; Weaver, P.C.; Prout, W.G.; Powell, S.J. POSSUM and Portsmouth POSSUM for predicting mortality. Br. J. Surg. 2003, 85, 1217–1220. [Google Scholar] [CrossRef] [PubMed]
- Das, N.; Talaat, A.; Naik, R.; Lopes, A.; Godfrey, K.; Hatem, M.; Edmondson, R. Risk adjusted surgical audit in gynaecological oncology: P-POSSUM does not predict outcome. Eur. J. Surg. Oncol. 2006, 32, 1135–1138. [Google Scholar] [CrossRef] [PubMed]
- Scott, S.; Lund, J.N.; Gold, S.; Elliott, R.; Vater, M.; Chakrabarty, M.P.; Heinink, T.P.; Williams, J.P. An evaluation of POSSUM and P-POSSUM scoring in predicting post-operative mortality in a level 1 critical care setting. BMC Anesthesiol. 2014, 14, 104. [Google Scholar] [CrossRef]
- Richards, C.H.; Leitch, F.E.; Horgan, P.G.; McMillan, D.C. A Systematic Review of POSSUM and its Related Models as Predictors of Post-operative Mortality and Morbidity in Patients Undergoing Surgery for Colorectal Cancer. J. Gastrointest. Surg. 2010, 14, 1511–1520. [Google Scholar] [CrossRef] [PubMed]
- Dutta, S.; Horgan, P.G.; McMillan, D.C. POSSUM and its related models as predictors of postoperative mortality and morbidity in patients undergoing surgery for gastro-oesophageal cancer: A systematic review. World J. Surg. 2010, 34, 2076–2082. [Google Scholar] [CrossRef] [PubMed]
- Second Patient Audit Report—The National Institute of Academic Anaesthesia. Available online: https://www.nela.org.uk/Second-Patient-Report-of-the-National-Emergency-Laparotomy-Audit (accessed on 6 April 2023).
- ACS NSQIP|ACS. Available online: https://www.facs.org/quality-programs/data-and-registries/acs-nsqip/ (accessed on 6 April 2023).
- Bilimoria, K.Y.; Liu, Y.; Paruch, J.L.; Zhou, L.; Kmiecik, T.E.; Ko, C.Y.; Cohen, M.E. Development and evaluation of the universal ACS NSQIP surgical risk calculator: A decision aid and informed consent tool for patients and surgeons. J. Am. Coll. Surg. 2013, 217, 833–842.e3. [Google Scholar] [CrossRef] [PubMed]
- Rivard, C.; Nahum, R.; Slagle, E.; Duininck, M.; Vogel, R.I.; Teoh, D. Evaluation of the performance of the ACS NSQIP surgical risk calculator in gynecologic oncology patients undergoing laparotomy. Gynecol. Oncol. 2016, 141, 281–286. [Google Scholar] [CrossRef] [PubMed]
- Teoh, D.; Halloway, R.N.; Heim, J.; Vogel, R.I.; Rivard, C. Evaluation of the American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator in Gynecologic Oncology Patients Undergoing Minimally Invasive Surgery. J. Minim. Invasive Gynecol. 2017, 24, 48–54. [Google Scholar] [CrossRef]
- Murray, I.; Lim, K.; Howells, R.; Jones, R.; Sharma, A.; Jones, S. The Utility of a Personalised Risk Calculator in Gynae-Oncology Surgery. Clin. Oncol. Res. 2020, 3, 7. [Google Scholar] [CrossRef]
- Manning-Geist, B.; Cathcart, A.M.; Sullivan, M.W.; Pelletier, A.; Cham, S.; Muto, M.G.; Del Carmen, M.; Growdon, W.B.; Sisodia, R.C.; Berkowitz, R.; et al. Predictive validity of American College of Surgeons: National Surgical Quality Improvement Project risk calculator in patients with ovarian cancer undergoing interval debulking surgery. Int. J. Gynecol. Cancer 2021, 31, 1356–1362. [Google Scholar] [CrossRef] [PubMed]
- Available online: https://riskcalculator.facs.org/RiskCalculator/PatientInfo.jsp (accessed on 24 May 2024).
- Austin, P.C.; Steyerberg, E.W. Interpreting the concordance statistic of a logistic regression model: Relation to the variance and odds ratio of a continuous explanatory variable. BMC Med. Res. Methodol. 2012, 12, 82. [Google Scholar] [CrossRef] [PubMed]
- Mandrekar, J.N. Receiver operating characteristic curve in diagnostic test assessment. J. Thorac. Oncol. 2010, 5, 1315–1316. [Google Scholar] [CrossRef] [PubMed]
- Mukherjee, S.; Kedia, A.; Goswami, J.; Chakraborty, A. Validity of P-POSSUM in adult cancer surgery (PACS). J. Anaesthesiol. Clin. Pharmacol. 2022, 38, 61–65. [Google Scholar] [CrossRef] [PubMed]
- Dunn, S.M.; Patterson, P.U.; Butow, P.N.; Smartt, H.H.; McCarthy, W.H.; Tattersall, M.H.N. Cancer by another name: A randomized trial of the effects of euphemism and uncertainty in communicating with cancer patients. J. Clin. Oncol. 1993, 11, 989–996. [Google Scholar] [CrossRef]
- Szender, J.B.; Frederick, P.J.; Eng, K.H.; Akers, S.N.; Lele, S.B.; Odunsi, K. Evaluation of the National Surgical Quality Improvement Program Universal Surgical Risk Calculator for a gynecologic oncology service. Int. J. Gynecol. Cancer 2015, 25, 512–520. [Google Scholar] [CrossRef]
- Thigpen, B.; Eppes, C.; Schaumberg, J.; Johnson, E.; Sangi-Haghpeykar, H.; Swaim, L.S. Does the ACS NSQIP Surgical Risk Calculator Accurately Predict Surgical Outcomes for Benign Hysterectomy? [06H]. Obstet. Gynecol. 2020, 135, 82S–83S. [Google Scholar] [CrossRef]
- Cohen, M.E.; Bilimoria, K.Y.; Ko, C.Y.; Hall, B.L. Development of an American College of Surgeons National Surgery Quality Improvement Program: Morbidity and mortality risk calculator for colorectal surgery. J. Am. Coll. Surg. 2009, 208, 1009–1016. [Google Scholar] [CrossRef]
| Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).