1. Introduction
Patients treated with radiotherapy require follow-up care to detect and treat acute, consequential and late side effects specific to radiotherapy, and monitor for recurrence [
1]. Acute side effects generally occur during treatment, persist up to a few weeks after treatment, and usually involve intermitotic cells (skin and mucosa). Consequential side effects occur when acute complications persist for longer durations and cause persistent damage [
2], whereas late side effects emerge months to years after radiotherapy treatment and usually involve postmitotic cells (liver, kidney, heart, muscle and bone) [
1]. The most common follow-up model of care to manage these side effects is the specialist-led model, where the radiation oncologist oversees the care, usually in a hospital setting. However, improved screening, earlier detection and increased incidence of cancer have led to an increased demand for cancer-related services [
3] and, subsequently, follow-up care [
4]. This increase in the demand for follow-up care poses a challenge to specialists and general practitioners to provide optimal follow-up care [
5].
It has been recommended that general practitioners take a greater role in cancer follow-up care [
6,
7,
8,
9] in the form of shared care. Shared care is a collaborative process through which different professional groups work together to improve health care quality [
10], in this case, the patient’s general practitioner and oncologist. The evidence for the benefits of shared cancer follow-up models of care is growing [
11,
12,
13,
14], with randomised controlled trials finding no difference in recurrence or quality of life when a general practitioner is involved in cancer follow-up care compared to an oncologist [
15,
16,
17,
18]. Yet, there are still barriers to implementing shared cancer follow-up care into practice.
A systematic review examined the factors that influence the translation of shared cancer care into practice [
19] and found that general practitioners were willing to take a greater role in cancer follow-up care, however, general practitioners sought specific follow-up clinical management guidelines to support them, which were based on best practice and preferably written by oncologists. The follow-up assessments could be in the form of a printable checklist or validated instruments and would reassure general practitioners that they are addressing aspects critical for the particular patients’ care [
20,
21,
22,
23]. However, some oncologists felt that general practitioners did not have the specialised knowledge of specific treatment side effects and how to manage the side effects [
24,
25,
26], thus posing a barrier in the move to shared cancer follow-up care.
Establishing concordance between health professionals is important as models of care adapt to the ever-changing demands of optimal patient management. However, there is limited research on the concordance (or discordance) of general practitioners and specialists in healthcare. It is important to understand the concordance to improve general practitioner confidence in providing follow-up care and reduce oncologists’ hesitancy to transfer some aspects of care for low-risk patients. Therefore, this study aimed to create a standardised follow-up clinical assessment for general practitioners and radiation oncologists to use on patients previously treated with radiotherapy and measure the level of agreement between general practitioners and radiation oncologists. To our knowledge, this is the first study of its kind.
2. Materials and Methods
This was a cross-sectional clinical practice study at the Illawarra Cancer Care Centre (regional) and the Shoalhaven Cancer Care Centre (rural), Australia. The radiation oncology services within these centres were familiar with a web-based health technology and had an existing system that could be integrated into the primary care setting. Ethics approval was received from the Joint University of Wollongong and the Illawarra Shoalhaven Local Health District Human Research Ethics Committee (2020/ETH00301). A detailed protocol is available [
27]; the trial was registered with the Australian New Zealand Clinical Trials Registry on 20 October 2020, ACTRN12620001083987.
2.1. Sample and Recruitment
Patients were eligible if they were scheduled for a radiation oncology follow-up consultation in 2021 and were three years post-radiotherapy treatment for breast, colorectal or prostate cancer. We purposely selected three years post-radiotherapy as it was expected that patients would have fewer toxicity issues. Radiation oncologists selected patients from their follow-up clinic list that they believed would suit a shared cancer follow-up model of care based on clinical considerations, including treatment type, prescription, and cancer staging.
The sample was taken from the overarching study with a recruitment aim of 20 patients. Patients were sent a participant information sheet informing them about the study aims, processes and inviting them to participate. Once patients consented in writing, their general practitioner was invited to participate.
2.2. General Practitioner Training
All general practitioners were visited by a radiation oncologist who provided a demonstration of how to access and complete the follow-up clinical assessment. In addition to this, the research team designed a training program that was approved for continuing professional development points. The training program included information on radiation oncology side effects and management, with a video demonstration on conducting a clinical assessment.
2.3. Web-Based Technology
The hospital’s PROsaiq
® platform [
28] was used to administer clinical assessments between oncologists and general practice via HealthPathways (described further below). PROsaiq consists of a webserver that uses the clinical assessments existing within the Oncology Information System (OIS), to produce a webpage in Xform format with a specific Uniform Resource Locator (URL) that can be shared. The webform contains placeholders for patient identifiers and survey assessment items. When the form is submitted, the webserver alters the returned survey from a JavaScript Object Notation (JSON) format into HL7 format and imports it into the OIS through the usual HL7 gateway. The submitted answers are stored and appear as if the survey had been completed entirely within the OIS. The PROsaiq system also included a module to monitor rejected incoming assessments to allow for manual correction of contained errors, for example, incorrect spelling of the surname, switching of first and last names, or incorrect medical record number. The first author monitored this portal.
The system was trialled for collecting quality of life-based patient-reported outcomes and deemed feasible in terms of use [
29,
30]. The PROsaiq platform acts only as a server of empty forms and a converter of returned forms, not storing any patient data and deliberately cleaning RAM after the completion of conversion and transfer.
2.4. Tool Development
At the time of writing, the cancer centres had an internal Dashboard system using PROsaiq that linked directly to the OIS for follow-up consultations. This allowed for the follow-up process to be streamlined internally. The assessments were compiled internally at the Illawarra and Shoalhaven Cancer Care Centres for follow-up of radiotherapy patients and were based on current practice. These clinical assessments review physical items on a scale from 0 to 3 or 4 (see
Table 1) for items specific to radiotherapy follow-up care, such as pain, fatigue, physical performance, bowel issues, urinary issues, and appetite. The included scales were sourced from the Radiation Therapy Oncology Group scales [
31] and the Common Terminology Criteria for Adverse Events, version 3.0 [
32].
Table 2 displays the items that apply for breast, prostate and colorectal follow-up care.
2.5. Data Collection
Radiation oncologist data—
Patients attended their standard scheduled radiation oncology follow-up appointment with their radiation oncologist. The radiation oncologists were provided with a list of patients before their clinic to remind them to enter the data at that point in time, instead of retrospectively. The radiation oncologist entered the clinical assessment data directly into Oncology Information System (OIS: Mosaiq®) as part of their standard consultation practice.
General practitioner data—
The patient was requested to visit their general practitioner the day before, the day of, or the day after their radiation oncologist follow-up appointment. This was to try and reflect contemporaneous results for the clinical assessments. The general practitioner accessed the clinical assessment via HealthPathways. HealthPathways is a web-based information portal being implemented in New Zealand, Australia, and the United Kingdom to help general practitioners make assessment, management, and specialist request decisions. Each ‘pathway’ is an agreement between the primary and specialist services, it is developed by specialists or hospital staff and is reviewed and approved by general practitioners.
To access the assessment, the general practitioner would first select the relevant cancer type based on the patient (see
Figure 1), enter the patient’s name and medical record, and complete the clinical assessment.
2.6. Data Analysis
The general practitioner and radiation oncologist data were extracted separately from the OIS, copied and pasted into the same Microsoft Excel spreadsheet for analysis. The data were matched using the patient’s medical record number, and de-identified. Percent agreement and Fleiss’ Kappa was used to determine the level of agreement of the clinical items outlined in
Table 1. A 75–90% per cent agreement demonstrates an acceptable level of agreement [
33]. Fleiss kappa accounts for chance and is used when the raters are different (four radiation oncologists were the non-unique raters). Fleiss kappa also shows the level of agreement within the variable for each category. Level of agreement is measured from 0 to 1 (slight agreement is <0.20; fair agreement 0.21–0.40; is moderate 0.41–0.60; substantial 0.61–80; and 0.81–1.00 is almost perfect agreement) [
34].
3. Results
Results were returned on 15 general practitioner-radiation oncologist dyads of data for analysis from the recruited sample of 19 (79%). For participant demographics, see
Table 3. Seven dyads were from the Shoalhaven (rural) and eight dyads from the Illawarra (regional); seven were breast cancer patients, seven were prostate patients, and one was a colorectal patient.
The percent agreement between general practitioners and radiation oncologists was above an acceptable level >75% for every variable. The Fleiss kappa ranged from moderate agreement to almost perfect agreement for the variables, see
Table 4, and sub analysis by demographic variable was therefore not required.
The clinical items with 100% percent agreement and almost perfect agreement (Fleiss Kappa) were Lymphoedema (all seven patients scored grade 0), recurrence (all seven patients scored grade 0), erectile dysfunction (all seven patients scored grade 3), proctitis and rectal haemorrhage. Fatigue and physical performance had an overall percent level of agreement of 87% and substantial Fleiss Kappa agreement (0.695, 0.0659). General practitioners reported seven patients as experiencing some level of fatigue, and radiation oncologists scored four patients with fatigue. For physical performance, two patients were scored as having some form of physical restriction by the general practitioner, compared to five patients being scored with physical restriction by the radiation oncologist. Appetite and unintended weight change agreement was moderate.
4. Discussion
There is limited research in understanding the concordance in knowledge, care patterns, and care outcomes between general practitioners and specialists for many health conditions. This knowledge of concordance (or discordance) is important when evaluating the feasibility of new models of care to manage both the clinical load and patients’ needs and preferences. For example, a study on the concordance of cardiac right bundle branch block diagnosis between cardiologists and general practitioners had substantial agreement, indicating general practitioners had scope to be more involved [
35]. However, a study on the diagnosis of skin conditions between general practitioners and dermatologists had moderate agreement, indicating more training was needed if the model was to change [
36].
Our study aimed to address two of the barriers to shared cancer follow-up care: the provision of clinical follow-up guidelines; and the perception that general practitioners may lack cancer specific skills and knowledge for shared cancer follow-up care. Our study has demonstrated a moderate to almost perfect agreement between the general practitioner and radiation oncologist completing a prescribed follow-up clinical assessment. The difference in reporting fatigue may be due to the radiation oncologist recording fatigue attributable to radiotherapy treatment from pre-treatment levels, compared to the general practitioner recording general tiredness level. Overall the results indicate general practitioners can adequately assess some aspects of cancer patients during follow-up for radiotherapy side effects and disease recurrence based on a prescribed clinical assessment.
The provision of a follow-up clinical assessment designed by the radiation oncologist is a key component of this research. Studies have suggested the need for specific and localised clinical follow-up guidelines or assessments [
20,
21]. There are a myriad of programs and software utilised across general practice clinics, and therefore the provision of clinical assessments needs to be easily accessible by general practitioners. Using specific follow-up guidelines that oncologists prepare should reassure both the general practitioner and the radiation oncologist that the general practitioner is addressing critical aspects of the patient’s care [
20,
23]. Given the moderate to almost perfect agreement we found, these results will be useful to bridge that gap. However, we did not evaluate if using the clinical assessment reassured general practitioners and improved their confidence.
One potential reason that oncologists feel uncertain about general practitioners providing shared follow-up care is that general practitioners rarely contact oncologists unless there is a problem. However, there is an absence in literature regarding general practitioners communicating with oncologists when changes in a patient’s overall condition, co-morbidities or concern for recurrence [
37]. Shared cancer follow-up care requires improved communication channels between general practitioners and oncologists in real-time [
24]. Our study successfully implemented a system that was easily accessible to general practitioners and allowed them to transfer the consultation results back to the radiation oncologist for review in real-time. Thus the radiation oncologist was not only able to maintain involvement in the patient’s care but could oversee the care. Although there is good concordance between the general practitioner and radiation oncologist, and the radiation oncologist could review the results from the shared care consultation in real-time, we do not know if this has improved the radiation oncologists’ acceptance and the likelihood of engaging in shared care models.
4.1. Limitations
This was the first study to our knowledge to evaluate the concordance between general practitioners’ and radiation oncologists’ clinical assessment in cancer follow-up care. A strength of the study was the use of a predefined protocol to minimise subjective clinical review. The limited sample size, which was partly due to the COVID-19 global pandemic and reduced face-to-face consultations, is acknowledged, and the results should be reviewed with caution. In addition, patients were three years post-treatment as it was expected that they would have fewer toxicity issues (as confirmed by our results), and these results cannot be generalised to patients on active treatment or earlier post-treatment. The reporting timeframe may be a factor to consider, as the level of pain or fatigue that a patient reports may be different in the timeframe between appointments of the radiation oncologist and general practitioner.
4.2. Future Implications
Future research should consider larger sample sizes, moving the post-treatment follow-up review to earlier in the follow-up phase, and expanding the assessments available for other cancers. Future research should also explore whether the provision of the clinical assessments resulted in improved general practitioner and radiation oncologist reassurance, and confidence that general practitioners can be involved in shared cancer follow-up model of care, and whether good concordance improves acceptance.
5. Conclusions
This study supports general practitioners taking a greater role in cancer follow-up care. Collaborative care between general practitioners and oncologists should be further explored. However, it is recognised that there are challenges to translating available evidence into systems to allow health professionals to work both independently and collaboratively to best meet the needs of patients. Systems for shared cancer follow-up care need to be integrated into both health settings and further development and analysis of specific clinical follow-up assessments. Further research with a larger sample, earlier in the post-treatment phase and qualitative analysis is warranted.
Author Contributions
T.S., H.S. and A.M. contributed to the study conception, design and methodology and analysis. Ethics was prepared and submitted by T.S. and H.S. T.S. performed the data collection. T.S. prepared the first draft of the manuscript and all authors contributed to subsequent revisions. All authors have read and agreed to the published version of the manuscript.
Funding
TS is a recipient of an Australian Government Research Training Program scholarship and a top-up scholarship by the Radiation Oncology Service from the Illawarra Shoalhaven Local Health District. Award/Grant number is not applicable.
Institutional Review Board Statement
The study was conducted in accordance with the Declaration of Helsinki, and approved by the Joint University of Wollongong and Illawarra Shoalhaven Local Health District Human Research Ethics Committee, 2020ETH00301.
Informed Consent Statement
The respondents were informed by a written formal cover letter about the study, including purpose, confidentiality, voluntariness, and anonymity. Written informed consent was obtained from all subjects involved in the study.
Data Availability Statement
The dataset generated during and/or analysed during the current study is available from the corresponding author upon reasonable request.
Conflicts of Interest
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.
References
- Majeed, H.; Gupta, V. Adverse Effects of Radiation Therapy; StatPearls Publishing: Tampa, FL, USA, 2022. Available online: http://www.ncbi.nlm.nih.gov/books/NBK563259/ (accessed on 26 April 2022).
- Dörr, W.; Hendry, J.H. Consequential late effects in normal tissues. Radiother Oncol. 2001, 61, 223–231. [Google Scholar] [CrossRef] [PubMed]
- Bray, F.; Ferlay, J.; Soerjomataram, I.; Siegel, R.L.; Torre, L.A.; Jemal, A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 2018, 68, 394–424. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Okera, M.; Baker, N.A.; Hayward, A.M.; Selva-Nayagam, S. Oncology workforce issues: The challenge of the outpatient clinic. J. Inter. Med. 2011, 41, 499–503. [Google Scholar] [CrossRef] [PubMed]
- McCabe, M.S.; Bhatia, S.; Oeffinger, K.C.; Reaman, G.H.; Tyne, C.; Wollins, D.S.; Hudson, M.M. American Society of Clinical Oncology Statement: Achieving High-Quality Cancer Survivorship Care. J. Clin. Oncol. 2013, 31, 631–640. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Institute of Medicine. Delivering High-Quality Cancer Care: Charting a New Course for a System in Crisis; National Academies Press: Washington, DC, USA, 2013; Available online: http://www.nap.edu/catalog/18359 (accessed on 21 September 2021).
- Dyer, S.; Yaganti, S.; Patel, J. Stratified follow-up care for people with prostate cancer. J. Prim. Health Care. 2016, 26, 24–30. [Google Scholar] [CrossRef]
- Cancer Australia. Shared Cancer Follow-Up and Survivorship Care; Cancer Australia: Sydney, Australia, 2019. Available online: https://www.canceraustralia.gov.au/clinicians-hub/shared-follow-care (accessed on 9 March 2022).
- Cancer Council Victoria and Department of Health Victoria. Optimal Cancer Care Pathways. 2021. Available online: https://www.cancer.org.au/health-professionals/optimal-cancer-care-pathways (accessed on 9 March 2022).
- Oosterink, J.J.; Oosterveld-Vlug, M.G.; Glaudemans, J.J.; Pasman, H.R.W.; Willems, D.L.; Onwuteaka-Philipsen, B.D. Interprofessional communication between oncologic specialists and general practitioners on end-of-life issues needs improvement. Fam. Pract. 2016, 33, 727–732. [Google Scholar] [CrossRef]
- Grunfeld, E. Cancer survivorship: A challenge for primary care physicians. Br. J. Gen. Pract. 2005, 55, 741–742. [Google Scholar]
- Grunfeld, E.; Levine, M.N.; Julian, J.A.; Coyle, D.; Szechtman, B.; Mirsky, D.; Verma, S.; Dent, S.; Sawka, C.; Pritchard, K.I.; et al. Randomized trial of long-term follow-up for early-stage breast cancer: A comparison of family physician versus specialist care. J. Clin. Oncol. 2006, 24, 848–855. [Google Scholar] [CrossRef]
- Lund, A.S.; Lund, L.; Jønler, M.; Graversen, P.; Bro, F.; Borre, M. Shared care in prostate cancer: A three-year follow-up. Scand. J. Urol. 2016, 50, 346–351. [Google Scholar] [CrossRef]
- Bro, F.; Lund, L.; Morten, J.; Peter, G.; Michael, B. “To be followed up in general practice”-developing and implementing a shared care model for prostate cancer. Int. J. Integ. Care. 2017, 17, 71–72. [Google Scholar] [CrossRef] [Green Version]
- Johnson, C.E.; Saunders, C.M.; Phillips, M.; Emery, J.D.; Nowak, A.K.; Overheu, K.; Ward, A.M.; Joske, D.J. Randomized Controlled Trial of Shared Care for Patients with Cancer Involving General Practitioners and Cancer Specialists. J. Oncol. Pract. 2015, 11, 349–355. [Google Scholar] [CrossRef] [PubMed]
- Ghezzi, P.; Magnanini, S.; Rinaldini, M.; Berardi, F.; Biagio, G.D.; Testare, F.; Tavoni, N.; Schittulli, F.; D’Amico, C.; Pedicini, T.; et al. Impact of Follow-up Testing on Survival and Health-Related Quality of Life in Breast Cancer Patients: A Multicenter Randomized Controlled Trial. JAMA 1994, 271, 1587–1592. [Google Scholar] [CrossRef] [PubMed]
- Grunfeld, E.; Fitzpatrick, R.; Mant, D.; Yudkin, P.; Adewuyi-Dalton, R.; Stewart, J.; Cole, D.; Vessey, M. Comparison of breast cancer patient satisfaction with follow-up in primary care versus specialist care: Results from a randomized controlled trial. Br. J. Gen. Pract. 1999, 49, 705–710. [Google Scholar]
- Grunfeld, E.; Mant, D.; Yudkin, P.; Adewuyi-Dalton, R.; Cole, D.; Stewart, J.; Fitzpatrick, R.; Vessey, M. Routine follow up of breast cancer in primary care: Randomised trial. BMJ 1996, 313, 665–669. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Sandell, T.; Schütze, H. Factors influencing the translation of shared cancer follow-up care into clinical practice: A systematic review. BMJ Open. 2022, 12, 8. [Google Scholar] [CrossRef] [PubMed]
- DiCicco-Bloom, B.; Cunningham, R.S. The experience of information sharing among primary care clinicians with cancer survivors and their oncologists. J. Cancer Surviv. 2013, 7, 124–130. [Google Scholar] [CrossRef] [PubMed]
- Anvik, T.; Holtedahl, K.A.; Mikalsen, H. “When patients have cancer, they stop seeing me”–the role of the general practitioner in early follow-up of patients with cancer-A qualitative study. BMC Fam. Pract. 2006, 7, 19. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Haq, R.; Heus, L.; Baker, N.A.; Dastur, D.; Leung, F.-H.; Leung, E.; Li, B.; Vu, K.; Parsons, J.A. Designing a multifaceted survivorship care plan to meet the information and communication needs of breast cancer patients and their family physicians: Results of a qualitative pilot study. BMC Med. Inform. Decis. Mak. 2013, 13, 76. [Google Scholar] [CrossRef] [Green Version]
- Walter, F.M.; Usher-Smith, J.A.; Yadlapall, S.; Watson, E. Caring for people living with, and beyond, cancer: An online survey of GPs in England. Br. J. Gen. Pract. 2015, 65, e761–e768. [Google Scholar] [CrossRef] [Green Version]
- Schütze, H.; Chin, M.; Weller, D.; Harris, M.F. Patient, general practitioner and oncologist views regarding long-term cancer shared care. Fam. Pract. 2018, 35, 323–329. [Google Scholar] [CrossRef]
- Potosky, A.; Han, P.; Rowland, J.; Klabunde, C.; Smith, T.; Aziz, N.; Earle, C.; Ayanian, J.Z.; Ganz, P.A.; Stefanek, M. Differences Between Primary Care Physicians’ and Oncologists’ Knowledge, Attitudes and Practices Regarding the Care of Cancer Survivors. J. Gen. Intern. Med. 2011, 26, 1403–1410. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Puglisi, F.; Agostinetto, E.; Gerratana, L.; Bozza, C.; Cancian, M.; Iannelli, E.; Ratti, G.; Cinieri, S.; Numico, G. Caring for cancer survivors: Perspectives of oncologists, general practitioners and patients in Italy. Future Oncol. 2017, 13, 233–248. [Google Scholar] [CrossRef] [PubMed]
- Sandell, T.; Schütze, H.; Miller, A. A Shared Cancer Follow-Up Model of Care Between General Practitioners and Radiation Oncologists for Patients With Breast, Prostate, and Colorectal Cancer: Protocol for a Mixed Methods Implementation Study. JMIR Res. Protoc. 2021, 10, 1. [Google Scholar] [CrossRef] [PubMed]
- Schuler, T.; Miller, A.A. PROsaiq: A smart device-based and EMR-integrated system for Patient-Reported Outcome measurement in routine cancer care. J. Radiat. Oncol. 2014, 6, 23111–23131. [Google Scholar] [CrossRef]
- Girgis, A.; Durcinoska, I.; Levesque, J.V.; Gerges, M.; Sandell, T.; Arnold, A.; Delaney, G.P. eHealth System for Collecting and Utilizing Patient Reported Outcome Measures for Personalized Treatment and Care (PROMPT-Care) Among Cancer Patients: Mixed Methods Approach to Evaluate Feasibility and Acceptability. J. Med. Internet Res. 2017, 19, e330. [Google Scholar] [CrossRef]
- Girgis, A.; Durcinoska, I.; Arnold, A.; Kaadan, N.; Miller, A.A.; Descallar, J.; Delaney, G.; PROMPT-Care Program Group. Phase III non-randomized controlled trial of PROMPT-Care, an eHealth intervention utilizing patient reported outcomes in routine clinical care: Impact on emergency department presentations. J. Clin. Oncol. 2019, 37, 6510. [Google Scholar] [CrossRef]
- Cox, J.D.; Stetz, J.; Pajak, T.F. Toxicity criteria of the Radiation Therapy Oncology Group (RTOG) and the European organization for research and treatment of cancer (EORTC). Int. J. Radiat. Oncol. 1995, 31, 1341–1346. [Google Scholar] [CrossRef]
- Trotti, A.; Colevas, A.; Setser, A.; Rusch, V.; Jaques, D.; Budach, V.; Langer, C.; Murphy, B.; Cumberlin, R.; Coleman, C. CTCAE v3.0: Development of a comprehensive grading system for the adverse effects of cancer treatment. Semin. Radiat. Oncol. 2003, 13, 176–181. [Google Scholar] [CrossRef]
- Stemler, S.E. A Comparison of Consensus, Consistency, and Measurement Approaches to Estimating Interrater Reliability. University of Massachusetts Amherst. Available online: https://scholarworks.umass.edu/pare/vol9/iss1/4/ (accessed on 13 March 2022).
- Shrout, P.; Fleiss, J. Intraclass correlations: Uses in assessing rater reliability. Psychol. Bull. 1979, 86, 2. [Google Scholar] [CrossRef]
- AL Ventosa, M.; Pera, G.; Roca Saumell, C.; Mengual Miralles, N.; Zamora Sanchez, M.V.; Gros Garcia, T.; Font, L.G.; Camps, M.B.; Francisco-Pascual, J.; Terradellas, J.B. Diagnosis of right bundle branch block: A concordance study. BMC Fam. Pract. 2019, 20, 58. [Google Scholar] [CrossRef]
- Clark, R.A.; Rietschel, R.L. The cost of initiating appropriate therapy for skin diseases: A comparison of dermatologists and family physicians. JAAD 1983, 9, 787–796. [Google Scholar] [CrossRef] [PubMed]
- Dossett, L.A.; Hudson, J.N.; Morris, M.A.M.; Lee, C.; Roetzheim, R.G.; Fetters, M.D.; Quinn, G.P.; Dossett, M.L.A. The Primary Care Provider (PCP)-Cancer Specialist Relationship: A Systematic Review and Mixed Methods Meta-synthesis. CA Cancer J. Clin. 2017, 67, 156–169. [Google Scholar] [CrossRef] [PubMed]
| Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).