Factors That Determine Completion Rates of Biomedical Students in a PhD Programme

Abstract

Purpose: The purpose of this retrospective study is to identify potential predictors of academic success or failure in Doctor of Philosophy (PhD) programmes in the field of biomedicine. Based on these, the policies and structure of academic programmes granting PhD degrees in biomedicine might be improved. Literature review (State of the art): At the present moment, most European and all of the EU doctoral education systems in biomedicine are regulated by the Salzburg principles of the Bologna process. Almost all the programmes formally comply with regulations, but the degree to which rules are applied varies greatly. The European Research Council (ERC) and various stakeholders’ associations, such as the Organisation for PhD Education in Biomedicine and Health Sciences in the European System (ORPHEUS), have recognised this and in their policies, they recommend regular evaluation of PhD programme structures. One such evaluation that was conducted at our institution motivated us to search for quantifiable factors that can help the process of PhD programme structural reform. Since the literature is scarce on this matter, we decided to conduct analysis of our own data and thus study the relationships between recommended EU policies and real-world data. Methods: Biology of Neoplasms is a PhD programme founded under Bologna process rules. It enrols students with Doctor of Medicine (MD), Doctor of Dental Medicine (DMD) or similar degrees in the biomedical field. A large portion of enrolled PhD students work full time in medical practices. A retrospective analysis was conducted on students who enrolled between 2006 and 2017. In order to quantify academic success, outcome measures of graduation (completion) rate, time to graduation, average impact factor of published papers comprising a PhD thesis and the ratio of the latter two were formed. Age, sex, employment institution, mentor experience and tuition subsidy were considered as potential predictors. Results: A total of 124 students were enrolled in the study—38% male. Out of the total, 21 (16.94%) students discontinued the study programme and 22 students graduated (17.7%). The average impact factor (IF) of published papers was 2.66 ± 1.51. Mentor experience (Odds ratio (OR) = 6.7) and student employment in academia (OR = 11.7) were significant predictors of successful graduation. Stricter criteria for graduation had no effect on graduation in newly enrolled students. Likewise, sex, tuition subsidy and age did not affect graduation rates. Surprisingly, time to graduation was not affected by any of the considered predictors. On the other hand, students that were mentored by experienced mentors and employed in academia outperformed their peers in terms of impact factors of publications related to their thesis. Conclusion: Characteristics such as gender, age at enrolment and even tuition paid by the institution do not have a significant impact on completion rate. Experienced mentors and employment in academic institutions seem to be the factors that predict a successful completion of a PhD programme. Furthermore, our results give a quantifiable support to the ORPHEUS and ERC recommendations and policies. These conclusions can be easily applied to any PhD programme formed under the tenets of the Bologna process.

1. Introduction

Development of doctoral studies began in 19th century Germany and were based on the so-called Humboldt model [1]. This model has so far influenced almost every form of doctoral education in the world. The model is based on an apprenticeship, meaning that doctoral candidate works for serval years under the supervision of a mentor on a specific research problem. After gathering a coherent body of knowledge on a problem, a candidate would publish a doctoral thesis, which would be usually evaluated by a committee of experts. During the 20th century, diversification of the Humboldt model by its adaptation to many specific national frameworks resulted in a variety of somewhat similar doctoral degree granting models [1].

Likewise, doctoral studies granting the Doctor of Philosophy (PhD) degree in the biomedical field are numerous and diverse through the world and EU. Criteria for completion of such studies commonly include research work, which is usually supervised by a PhD commission or council, and oftentimes they require a publication of an original scientific article [2]. However, within this general framework a great deal of heterogeneity exists. The Organisation for PhD Education in Biomedicine and Health Sciences in the European System (ORPHEUS) was founded in 2004 with the mission of synchronising criteria and ensuring the integrity and quality of PhD degrees in the biomedical field [3]. One year after, as a part of third cycle of the Bologna process, the Salzburg principles were adopted as a foundation for EU-wide doctoral degree synchronisation [4]. Since 2012, ORPHEUS publishes, in a spirit of Salzburg principles, a Best Practices e-textbook which addresses key issues in biomedical PhD education [5]. Among these, the issue of graduate school structure encompasses a need for a regular review and the updating of a graduate school structure.

In this vein, the international European Research Council (ERC) delegation visited the University of Split, School of Medicine and examined teaching conditions, study management, students and mentors in our Biology of Neoplasms (BN) PhD Programme. The delegation raised some serious concerns regarding our BN programme. Consequently, in an attempt to reform the structure of the PhD programme, we tried to identify factors that determine academic success and failure in biomedical PhD programmes by searching the MEDLINE database, the largest scientific literature database in the field of biomedicine. However, as others have also noted [6], quantitative information which would help guide our decision making regarding the PhD programme reform turned out to be rare and scarce, especially when it comes to European PhD programmes. Based on this, we decided to do a quantitative analysis of our PhD programme and identify factors that would predict academic success or failure, so that in the future structural reform of our PhD programme positive factors can be amplified.

2. Materials and Methods

2.1. Settings

The BN PhD Programme is a typical PhD programme developed to comply with the rules of the Bologna process. It has six semesters—semesters one to four are dedicated to teaching and research, while the last two semesters are exclusively dedicated to research. Since 2014, PhD programme have introduced stricter graduation criteria with the first authorship of two (previously one) articles on the thesis topic, published in a journal indexed in Web of Science with an impact factor (IF) >1 or one article with an IF ≥4. In our settings, only PhD students in full-time employment in academia have an obligation to graduate within 5 years of their enrolment in a PhD programme. Others do not have a specific time limit regarding graduation. Students employed in academia dedicate at least 50% of their working time to scientific duties, and the other part of their working hours might be dedicated to teaching and administrative tasks. Students not employed in academia must deal with scientific obligations outside of their standard working hours (40 h a week in Croatia). Furthermore, students that are employed in academia have their tuition fully subsidised by an academic institution, whereas others might or might not have their tuition subsidised depending on their employer. Admission to BN PhD programmes were liberal—i.e., a candidate having a Doctor of Medicine (MD), Doctor of Dental Medicine (MDM) or master’s degree in some field of biomedicine and with a grade point average of at least 3.5 in the Croatian grading system (5 being the maximum and 1 being the lowest grade) could enrol. Candidates were not required to have PhD mentors at the time of admission process. In this analysis, we included all of the students enrolled in the programme since its beginning in 2006. Students who enrolled after 2017 were excluded because they were unable to defend their PhD thesis before April 2019 when our study ended.

2.2. Outcome Measures

We tried to quantity academic success or failure trough fallowing outcome measures: the proportion of students who graduated, time to degree (from enrolment to defence of the PhD thesis), the number of published manuscripts on thesis topic, with the PhD student as first author, the IF of the journal where the PhD student was published as a first author, the ratio of the IF and time to graduation. Additionally, attrition rates and the reasons for the attrition were analysed.

2.3. Statistical Analysis

Data are presented as median and interquartile ranges for non-normally distributed data, mean and standard deviation for normally distributed data or as a proportion. Normality of data was tested by a Shapiro–Wilk test and normality probability plot correlation coefficient. To formalise differences between groups, we used the chi-squared, chi-squared for trend or Fisher’s exact test for nominal variables and t-test or ANOVA for continuous variables. Furthermore, multiple logistic regressions and model selections based on AIC (Akaiake Information Criterion) were used to find the optimal model for predicting outcome measures. For modelling purposes, i.e., for the purpose of identifying predicting factors, only analyses of 2007–2013 group were carried out. Inference was aided by point and interval (95% CI) estimates, goodness of fit measures, AIC and p values; the latter was interpreted in accordance with the ASA (American Statistical Association) Statement on p value.

3. Results

A comparison of the characteristics of the students in the two surveyed periods is shown in

Table 1. Characteristics of 124 students enrolled in Biology of Neoplasms Doctor of Philosophy (PhD) studies.

Variable		Period 2006–2013	Period 2014–2017	Total
Sex	male	34	14	48
	female	45	31	76
Age at enrolment	years (mean ± SD)	33.32 ± 8.13	28.61 ± 4.8	31.64 ± 7.45
Graduated student	number	21 (26.5%)	1 (2.2%)	22 (17.7%)
Profession	Doctor of Medicine or Dental medicine	72 (91.1%)	42 (93.3%)	114 (91.9%)
	other professions in biomedicine	7 (8.9%)	3 (6.7%)	10 (8.1%)
Employment	hospital	45 (57%)	27 (60%)	72 (58%)
	academia	13 (16.4%)	3 (7%)	16 (13%)
	primary health care, public health and private practice	15 (19%)	11 (24%)	26 (21%)
	other (industry)	6 (7.5%)	4 (9%)	10 (8%)
Financial status of tuition	self-paid	40 (50%)	41 (91%)	81 (65.2%)
	fully subsidised	15 (18.9%)	3 (7%)	18 (14.5%)
	partially subsidised	24 (30.3%)	1 (2%)	25 (20.2%)

. In total, there was a higher proportion of females—77:47 (62.1%). Students who enrolled in 2006–2013 were older than students who enrolled in 2014–2017 (33.32 ± 8.13 versus 28.61 ± 4.8 years). Students were mostly doctors of medicine or dental medicine (92%). These were mostly part-time students, fully staffed at the hospital or private practice (69%). Only 13% were “full time” students in various scientific projects at the School of Medicine or various research institutes.

3.1. Factors That Influence Graduation Rate

The cumulative graduation rate of students enrolled in the Biology of Neoplasms PhD programme was 17.7% (22/124) (Table 1). Most of the thesis topics were clinical. In order to discover predictors of graduation, we analysed the 2006–2013 dataset because it was richer in outcomes. Students who self-paid their tuition comprised 50% of the 2006–2013 group; their graduation percentage was 12.5%. Among students who were partially subsidised, graduation percentage was 20%, while among students who were fully subsidised by an institution, 73.3% graduated. These results show a clear positive trend between financial support and the proportion of graduating students (χ² = 16.79, p < 0.0001) (Figure 1a).

Furthermore, students employed by academic institutions had a graduation rate of 69%, followed by students in industry and private practice with 28.5% graduating. Only 13% of PhD candidates working in hospitals had graduated by the time data were collected (χ² = 9.64, p = 0.0081), (Figure 1b).

Of the 21 PhDs defended, 14 (67%) were mentored by experienced mentors (more than three PhDs theses defended prior to this mentorship), 6 (28%) were mentored by moderately experienced mentors (1 to 3 PhDs previously in mentorship), and 1 (5%) was led by an inexperienced mentor (so far he/she did not mentor doctoral dissertations). Thus, a trend can be found between the mentor ratio and the graduation rate (χ² = 9.71, p = 0.0018, Figure 1c).

Since in our setting all of the PhD students employed in an academic institution have their tuition fees fully subsidised, we decided to use model selection in order to find an optimal model which would predict the success of graduation. We considered age, sex, employment institution, mentor experience and tuition subsidy as potential predictors. After the model selection process, the most parsimonious model had only two variables—i.e., academia as institution (Odds ratio (OR) 11.66, 95% CI—1.78 to 126.9) and experienced mentor (OR 6.96, 95% CI—3.15 to 22.15). Prediction accuracy was excellent (

Table 2. Model selection.

Model No.	Variables	AUC (95% CI)	AICc
1	age, sex, academia as employment institution, mentor experience † and tuition subsidy	0.94 (0.90–0.99)	52.88
2 *	academia as employment institution, mentor experience †	0.94 (0.89–0.99)	47.51

* Optimal model; † mentor experience is ordinal variable coded as 0, 1, 2, 3 for no mentor, poorly, moderately and highly experienced mentors, respectively; AUC—area under curve; AICc—corrected Akaike information criterion.

and

Table 3. Odds ratio (OR) for variables from model No. 2.

Variable	Odds Ratio	95% CI
intercept	0.006692	0.0003167 to 0.04663
mentor experience †	6.956	3.147 to 22.15
academia as employment institution	11.66	1.784 to 126.9

† mentor experience is ordinal variable coded as 0, 1, 2, 3 for no mentor, poorly, moderately and highly experienced mentors, respectively; AUC—area under curve; AICc—corrected Akaike information criterion.

) and there were no issues with predictors correlating with each other.

3.2. Factors That Influence Time to Graduation

In total, the average time to doctoral degree was 79.1 months. Time to graduation did not depend on the type of institution in which the student was employed (ANOVA, R² = 1.5%, p = 0.8654), nor on the level of tuition subsidy (ANOVA, R² = 1.9%, p = 0.8346). Furthermore, the time to graduation of students with highly experienced mentors (85.54 ± 32.65 months) was not different compared to those with moderately experienced mentors (83.17 ± 30.35 months), (t-test, p = 0.8824).

3.3. Factors That Influence Academic Productivity

The average impact factor (IF) of journals in which the studies were published was 2.66 ± 1.51 and it was the highest for the first publication. The average number of publications included in the thesis was two, one student had three papers and another one had one paper (with IF > 4); more details are presented in

Table 4. Articles from PhD thesis and research topics.

Variable		Male	Female	Total	p Value
Impact factor	1st publication	2.34 ± 1.09	3.01 ± 1.76	2.81 ± 1.59	0.3679
	2nd publication	2.13 ± 1.00	2.60 ± 1.45	2.44 ± 1.31	0.4495
	3rd publication	1.28	/	1.14	/
Topics of thesis (No.)	basic	2	9	11	0.2307
	clinical	5	5	10
	public health	0	1	1

t-test; χ² test.

.

Students working in academic institutions had an average IF of 3.61 ± 1.79, those working in hospitals had an average IF of 1.85 ± 0.37 and finally students working in private practice or industry published in journals with an average IF of 1.74 ± 0.42. Comparing the average IF among of students working in academia with those who work in hospitals or in private practice and industry, the differences are characterised by fallowing p values (posthoc Welch t-test, p = 0.0193 and p = 0.0143, respectively).

Stratification of students by tuition subsidy does not seem to have any impact on the average IF (ANOVA, R² = 24.2%, p = 0.0822). Students with highly experienced mentors seem to have a higher average IF then those with moderately experienced mentors. (Welch t-test, mean difference = 1.227 ± 0.621, p = 0.069).

3.4. Attrition

During the study period, 21 (16.94%) students dropped out for a variety of reasons, the most common being personal factors (28.57%) (

Table 5. The reasons for attrition of students enrolled in PhD programme Biology of Neoplasms.

Reasons	Number of Students (%)
Problems with a mentor	4 (19%)
Personal factors	7 (33.3%)
The interference between work and private life	1 (4.7%)
Working pressure	1 (4.7%)
Lack of social support	3 (14.3%)
Practical research problems	1 (4.7%)
Different appointment	3 (14.3%)
Death	1 (4.7%)
Total	21

). Upon more detained analysis, not having a mentor turned out to be the most important factor in predicting student dropout probability, because none of the students who had a mentor dropped out (Fisher’s exact test p < 0.0001) (Figure 1b,c).

3.5. Effect of Changing Criteria for Graduation

In order to analyse the effect of stricter graduation criteria, we quantified academic success as a ratio of a candidate’s IF average and time to graduation. One candidate graduated from the group that enrolled after 2014, and her average IF to time to graduation ratio was at the 95th percentile of distribution derived from pre-2014 graduation data. Furthermore, the proportion of students who did not find a mentor decreased from 48% to 35% (χ² =1.836, p = 0.1755). The proportion of students with unexperienced mentors remained the same with 11%, the proportion of students with moderately experienced mentor increased from 16.5% to 35.5%, and finally the proportion of students with highly experienced mentors decreased from 48.1% to 35.5%; the p value for these changes was 0.1102. Fraction of students employed by academia dropped from 16.4% to 7% (χ² = 2.641, p = 0.4503).

4. Discussion

Our data imply that only one-fifth of the total enrolled students of Biology of Neoplasms have graduated. This low completion rate is consistent with similar biomedical PhD programs (Faculty of Medicine in Belgrade 16.6%, Faculty of Medicine in Izmir 26%, Faculty of Medicine, Public Health and Nursing University Gadjah Mada 27.94%; personal communication) as well as with the worldwide data [7]. On the other hand, an average completion rate of at least 66% has been reported in a recent European survey [8]. In the most successful countries such as the Netherlands, which has a graduation rate of 75%, candidates pursue PhDs in research grant funding [9]. They work as full-time students, less as candidates who pay their tuition independently. Dutch universities receive monetary rewards for each successfully completed PhD from their government. On the contrary, if the research fellowship is completed without PhD, the candidate is left without a salary and the only option is that institution pay for him/her to finish the programme [9].

A possible reason for the low graduation rate in our PhD programme is that most candidates are part-time students—i.e., they do not work in academia. In fact, more than 75% of them are medical practitioners at the beginning of their professional career. Second, the motivation of PhD candidates to enrol in the PhD programme is sometimes based on obtaining additional points for the residency admission in the hospitals, so it turns out that they are not committed to PhD study. These students were not intrinsically motivated to find a research topic and mentor. The similar objection was noted by a PhD programme in another Croatian University [10].

Furthermore, employment in academia can be considered as a proxy of how much working time is dedicated to scientific duties and it is not surprising that when combined with the mentor’s experience it dramatically increases odds (approximately 18-fold when expressed as natural log. of odds) for successful completion of the graduate programme. It is somewhat surprising that none of the considered predictors correlated with time to graduation.

When it comes to academic productivity expressed as average IF of papers that were published as a part of PhD thesis, with the student as a first author, PhD candidates with highly experienced mentors and those employed in academia outperform the others.

Attrition or dropout rates in our programme was 16.94%, which, interestingly, approximates the graduation rate. The main predictor of dropping out was an inability to find a PhD mentor.

If the results of our retrospective analysis are compared to ORPHEUS best practice recommendations, it is obvious that the strongest predictors such as employment in academia (i.e., time dedicated to PhD research), experienced mentors and inability to find a mentor correlate well with the content of the former dealing with the research environment and supervisors. Thus, it can be said that our study lends quantitative support to the recommendations.

Since this case study represents a retrospective study without validation cohort, its conclusions should be taken with some degree of reservation.

In conclusion, we were able to show that employment in academic institutions and experienced mentors are strong predictive factors of successful graduation in PhD programme; on the other hand, not finding a mentor was a strong predictive factor for dropping out of the graduate programme. Our results can be easily extended to other biomedical PhD programmes in Europe since a large majority of them are regulated by and adopted rules of the Bologna process; furthermore, our results represent a data-based confirmation of ORPHEUS best practices recommendations. Finally, one of the remaining problems that our study points to is the PhD education of practicing physicians, who by definition cannot dedicate themselves to a great extent to PhD research due to their workload. The solution to this problem may be in the establishment of MD/PhD combined programmes modelled after physician-scientist training programmes, which represent time-tested standards for combined MD and PhD education in the United States.