An Evaluation of the Accuracy of Peer to Peer Surgical Teaching and the Role of the Peer Review Process

Abstract

Background: Peer to peer learning is a well-established learning modality which has been shown to improve learning outcomes, with positive implications for clinical practice. Surgical students from across Ireland were invited to upload learning points daily while paired with their peers in a peer-reviewing process. This study was designed to assess content accuracy and evaluate the benefit of the review process. Method: A reflective content sample was selected from the database representing all gastrointestinal (GI) surgical entries. All questions and answers were double corrected by four examiners, blinded to the “review” status of the entries. Statistical analysis was performed to compare accuracy between “reviewed” and “non-reviewed” entries. Results: There were 15,569 individual entries from 2009–2013, 2977 were GI surgery entries; 678 (23%) were peer reviewed. Marked out of 5, accuracy in the reviewed group was 4.24 and 4.14 in the non-reviewed group. This was not statistically different (p = 0.11). Accuracy did not differ between universities or grade of tutors. Conclusion: The system of student uploaded data is accurate and was not improved further through peer review. This represents an easy, valuable and safe method of capturing surgical oral ward based teaching.

1. Introduction

Medical education is primarily based on the historical apprenticeship model, supported by didactic teaching [1]. Traditional methods of medical education and learning commonly centered on instructor-led sessions where participants i.e., students and teachers were required to be physically present at a given time and venue. The era of information technology has already seen significant changes to the manner in which education is delivered, with its further evolution and integration virtually guaranteed [1].

Surgent University [2] is an online clinical teaching repository which adopts the process of peer-assisted learning (PAL) among surgical trainees and students [3]. The term “peer-assisted learning” encompasses a variety of collaborative and cooperative modalities which include peer teaching, peer assessment and mentorship. PAL is commonly quoted as “people from similar social groupings who are not professional teachers helping each other to learn and learn themselves by teaching” and was coined by K.J. Topping in 1996 [4,5]. With particular emphasis on “by teaching”, one would be able to distinguish PAL from other forms of peer learning [5].

For years, PAL has played a major role in the training of medical professionals and its healthcare allies [6]. Previous studies have demonstrated the many benefits of PAL in medical education which include better academic achievements, improved study habits and attitudes while enhancing participants’ communication skills and self-confidence [7,8,9]; all of which are desirable in all medical graduates. The concept that both the student teacher and learner stand to benefit from the process is one that has driven the development and integration of PAL into modern day medical curricula [4,5,6,7,9,10,11,12].

In recent years, systems and learning theories have been developed for web-based learning in higher education [13]. This strategy has a confluence and is supported by a relatively newly emerging trend of learning theory, social constructivism [14,15,16]. Social constructivism is a branch of constructivism [14,15,16] which theorises that although learning requires that an individual is responsible for developing one’s own knowledge structure, learning a concept requires exchanging, sharing, and negotiation, as well as occasionally drawing on the expertise of more knowledgeable individuals. Learning involves both personal inner process and social aspect. Although many cognitive theorists have proposed models of inner learning process, social constructivism uniquely emphasizes the social aspect of learning.

This study describes a web-based strategy peer review that values critical feedback amongst peers and also allows for modification of information according to peer feedback. Over the past three years, medical students across universities in Ireland undergoing their surgical rotation or clerkship were invited to upload learning points on a daily basis. Students were paired with their peers in a peer reviewing process. The process encouraged reviewing facts uploaded by their paired partner, using evidence-based medicine to validate facts and providing relevant literature references.

The aim of this study was to determine if the accuracy of peer reviewed information was superior to non-peer reviewed information.

2. Experimental Section

2.1. Participants

All final-year medical students from three Irish universities;Royal College of Surgeons in Ireland (RCSI), University College Galway (UCG), and University College Cork (UCC) were invited to upload facts they had obtained from any member of a surgical team to which they were clinically rostered. They were randomly paired for the peer reviewing process as previously described. Editing of facts provided by their partners was encouraged if necessary.

2.2. Data Collection and Analysis

Facts representing all gastrointestinal (GI) surgical entries were selected from the online database. We excluded facts which were uploaded in error, duplicates and those with incomplete dataset. Facts were determined to be “reviewed” when peer reviewed and “non-reviewed” if otherwise.

A panel of experts established standard answers and a marking scheme for each fact. Four university lecturers/tutors double corrected all entries. Of a maximum of 5 marks, an average of marks for each entry was obtained. The experts and examiners were blinded to the “review” status of entries.

Student’s 2 tailed unpaired t-test was used to compare the accuracy of “reviewed” and “non-reviewed” facts. Better accuracy would be determined by higher average scores and statistical significance is determined by p-values ≤ 0.05.

3. Results and Discussion

3.1. Results

A total of 15,569 individual entries were uploaded to the website from 2009–2013. There were 2977 GI surgical entries of which 678 were “reviewed” (23%).

Students who uploaded entries were on their surgical rotation within a collection of 17 different hospitals around Ireland. Along with the hospital, the grade of members of the surgical team from whom they had obtained the piece of surgical information was also recorded (Table 1).

All selected entries had marks ranging from 1 to 5 and the total average mark for this was 4.17. Accuracy of “reviewed” group was 4.24 versus 4.14 in the “non-reviewed” group. With p-value of 0.11, it was determined that there was no statistical difference between the average scores and hence, accuracy of “reviewed” and “non-reviewed” facts (Table 2).

Accuracy between universities and between the 17 different hospitals did not statistically differ.

3.2. Discussion

Much of medical education is structured around a pre-clinical aspect which centres mainly on the basic sciences and a clinical component that draws from various medical fields and its respective practices. While academic institutions are under increasing pressure to produce graduates of a high standard capable of competing for post graduate training posts, educators are finding it increasingly difficult to deliver an ever expanding syllabus in a limited amount of time and resource [17]. E-learning has the potential to assist with the delivery of such an expanding syllabus while also supporting students in defining learning objectives within the structured assessment process [13].

Table 1. Breakdown of Entries Uploaded to the Surgent University Website.

Title	Title
Total Number of Entries from 2009–2013	15 569
Total Number of GI Surgery Entries	2 977 (19%)
Total Number of Peer-Reviewed GI Surgery Entries	678 (23%)
Percentage of GI Surgery Entries by Three Irish Universities
RCSI	80%
UCC	17%
UCG	3%
Percentage of GI Surgery Entries Provided by Different Tutor Grades
Intern	12%
Senior House Officer (SHO)	13%
Registrar	27%
Specialist Registrar (SPR)	12%
Consultant	36%

Table 1. Breakdown of Entries Uploaded to the Surgent University Website.

Title	Title
Total Number of Entries from 2009–2013	15 569
Total Number of GI Surgery Entries	2 977 (19%)
Total Number of Peer-Reviewed GI Surgery Entries	678 (23%)
Percentage of GI Surgery Entries by Three Irish Universities
RCSI	80%
UCC	17%
UCG	3%
Percentage of GI Surgery Entries Provided by Different Tutor Grades
Intern	12%
Senior House Officer (SHO)	13%
Registrar	27%
Specialist Registrar (SPR)	12%
Consultant	36%

Table 2. Mean Marks of Facts (Reviewed vs. Non-reviewed).

	Mean Marks	Standard Deviation
All GI Surgery Facts	4.17	0.93
Reviewed Facts	4.24	0.89
Non-reviewed Facts	4.14	0.94
p-Value from t-test 0.117

Table 2. Mean Marks of Facts (Reviewed vs. Non-reviewed).

	Mean Marks	Standard Deviation
All GI Surgery Facts	4.17	0.93
Reviewed Facts	4.24	0.89
Non-reviewed Facts	4.14	0.94
p-Value from t-test 0.117

The Surgent website was originally designed to be an interactive surgical teaching website with an online repository which facilitates access to e-learning materials [18]. It provides a platform upon which users are able to upload clinical based teaching materials, participate in learning activities and collaborate between other users. This online repository was highly utilized by medical students at their convenience and was demonstrated to be relevant and beneficial to their professional development [3]. The act of uploading facts onto the site and therefore sharing and teaching other students is one which mirrors the previously described PAL model [4,5,6,7].

Peer review has been shown to be beneficial as it promoted construction of knowledge through social sharing and competition [19]. According to social constructivism, students will better achieve deep learning in an authentic learning environment via social interaction [20]. The interactivity between students when participating in the peer review process mirrors the concept of collaborative learning. The works of Russian psychologist, Vygotsky describe the practice of collaborative learning as one with the potential for its participants to perform at “higher mental functions” when they work as a group as compared to when they work as individuals [21]. Also, the diversity that exists within groups of participants in terms of knowledge and experience naturally creates dialogue and discussion which in turn stimulates an appreciation and understanding of differing perspectives [21,22].

One of the criticisms of the PAL strategy is that user-generated content may not be accurate and is potentially unreliable as its very nature of “openness” allows anyone to contribute to the database, regardless of content validity and source of information [13]. The results of this study suggest otherwise. The fact that there was no statistical difference between the scores of reviewed and non-reviewed facts uploaded by students themselves would suggest that there was an element of accountability within the cohort of participants. The collaborative effort of the group where its peers are equally and highly motivated to create a library of clinical teaching materials which benefit all contributors, would tend to regulate each other’s performances [13]. It would also suggest that social normative pressures apply to students when they participate in the peer review process and this may be key in any PAL teaching modality.

Critics of peer to peer learning have also highlighted the decline of deep understanding of the subject matter among students as discussion shifted towards a more examination-focused one [23]. The concern then would be that students lack the ability to apply the knowledge towards everyday clinical practice. This is an understandable and reasonable concern and the authors feel the PAL strategy would benefit significantly through guidance from educational bodies in the form of defined and agreed learning objectives.

We also determined that there was no significant difference in the accuracy of facts provided by differing grades of senior tutors (Table 3). This seems to suggest that tutors were imparting knowledge which was suited to their current level of experience and was appropriate to the grade of those they were teaching. Such a system as this has the potential to allow audit of oral clinical based teaching. In a strategic fashion, it would enable educators to determine those specific areas of weakness in clinical teaching and initiate specific teaching interventions. Ultimately this may allow for the correlation of clinical teaching with patient outcomes.

Table 3. Mean Marks of Facts Provided by Different Grades of Tutors.

Grade of Tutor	Mean Marks	Mean Marks of “Reviewed” Facts	Mean Marks of “Non-Reviewed” Facts	p-Value
Intern	4.18	4.17	4.19	0.91
SHO	4.11	4.40	3.99	0.04
Registrar	4.13	4.18	4.10	0.52
SPR	4.29	4.44	4.21	0.15
Consultant	4.18	4.18	4.18	0.96

Table 3. Mean Marks of Facts Provided by Different Grades of Tutors.

Grade of Tutor	Mean Marks	Mean Marks of “Reviewed” Facts	Mean Marks of “Non-Reviewed” Facts	p-Value
Intern	4.18	4.17	4.19	0.91
SHO	4.11	4.40	3.99	0.04
Registrar	4.13	4.18	4.10	0.52
SPR	4.29	4.44	4.21	0.15
Consultant	4.18	4.18	4.18	0.96

Although no difference was found between the accuracy of those reviewed and non-reviewed facts, the overall accuracy of the system may be subject to criticism. A mean overall mark of 4.17 translates into an overall accuracy rate of 84%. While it remains a very high standard for a crowd sourced information repository, additional work is needed to improve this further.

4. Conclusions

The accuracy of online peer reviewed information is equivalent to non-peer reviewed information. We conclude that online PAL serves as an accurate and therefore valuable resource and supplement to the surgical teaching and training of students. Although the peer-reviewing process did not increase the accuracy of content, its role in promoting collaborative work and critical thinking is an important feature and should be examined further before it is removed.