Exploring Just-in-Time Teaching 3D Development as a Tool for Enhancing Knowledge and Understanding

Abstract

The integumentary system (skin) is the first line of defence in the body and part of the innate immune system. Within first year modules on Pharmaceutical Biology and Integrative Physiology in the Masters of Pharmacy degree at Robert Gordon University (RGU) several software tools were used to support both lecture and coursework material for the immune and integumentary systems. However, students had difficulty visualizing the various layers of the skin and how they become affected by different skin lesions. As a response to these identified learning difficulties, a just-in-time teaching 3-Dimensional elearning object was developed using free-to-use 3D CAD packages alongside common elearning software. The outcome was a virtualised human arm equipped to illustrate and label primary or secondary skin lesions whilst allowing spatial manipulation of the arm. This allowed students to manipulate and identify the specific skin layers involved. Evaluation of student engagement and learning was favourable, with students reflecting that they had a better understanding of the topic. Initial findings from this study highlight the benefits of quick, low-cost 3D production processes as just-in-time teaching elearning tools that have a positive impact on students’ performance.

1. Introduction

Although, the majority (~82%) of students entering the four year undergraduate MPharm at Robert Gordon University, are regarded as home students, i.e., coming from either England, Northern Ireland, Scotland, Wales or the European Union they have a diverse range of different qualification and educational backgrounds. As the University, increases internationalisation within its courses, student diversity in educational experience and expertise will only widen.

Currently, the only pre-requisite for entry to the undergraduate MPharm at RGU is Chemistry although an acceptable level of both English and Mathematics is required. However, as the pass rate for year 1 of the MPharm is high >90% all students are expected to complete and pass both an SCQF (Scottish credit and qualifications framework) level 7 Pharmaceutical Biology module and an extremely complex and detailed SCQF level 8 Integrated Physiology module. This particular module underpins many of the later topics in the MPharm therefore it is essential that students have the opportunity not only to pass the course but develop their knowledge and understanding of this key area. The pedagogical approach within the University is student centred learning [1], however, students often do not know what they need to know, why they need to know it and how to find the information to supply the correct answer. This paper highlights the role of in-house elearning developments and explores the coupling of rapid elearning tools with inexpensive 3D development as a method for enhancing knowledge and understanding.

A key milestone of the project was to introduce an additional elearning activity that could be accessed by students throughout the Integrated Physiology module to boost understanding and encourage engagement with the module resources.

1.1. Packaging Knowledge for Elearning

The immune system plays an underlying role in the majority if not all disease states and appreciating the cellular and physiological mechanisms involved is essential to treating these conditions. However, this vital area has limited coverage in year one of the MPharm and not covered again in depth until later in the course due to a full and busy curriculum. This has proven to be a major challenge and most students struggle not only to grasp the concepts involved but also to understand the expectations of the course.

Academics are not simply transmitters of knowledge but more facilitators guiding students learning. This constructivist view point draws on students' existing knowledge, beliefs, and skills building upon these and allowing the student to synthesize new understanding from prior learning and new information.

Greeno, Collins and Rednick identified three broad groupings or clusters to identify learning: the empiricist perspective (learning as activity); the cognitive perspective (learning as achieving understanding); and the situative perspective (learning as a social practice) [2]. Gagne’s hierarchy of behaviours fits into the empiricist perspective [3]. The cognitive approach encompasses the assumptions of constructivism indeed, Jean Piaget one of the forefathers of the constructivist approach postulated that knowledge assimilation occurs through the intellectual activity rather than the absorption of information [4]. Mark Walkers, article on Faraday’s lecture on candles is an excellent example of constructivism in use [5].

The basic cognitive competences that are required to be successfully demonstrated by 1st year MPharm students were described in 1956 in Bloom’s taxonomy and are knowledge, comprehension, application, analysis, synthesis and evaluation [6]. In the first semester of year 1, in the SCQF level 7 pharmaceutical biology module, the students only have one introductory 3 h coursework on immunology/the immune system the aim is for students to achieve the lower cognitive skills of knowledge and understanding (comprehension) of the basic concepts. In the SCQF level 8 Integrative Physiology course in the second semester students have three, one hour lectures (Power point) and one three hour coursework the expectations are for students to begin to develop the higher cognitive skills of application, analysis and synthesis.

In tandem with these cognitive skills the students are expected to be able to reflect on their own learning experience. One of the key goals in teaching is to provide the conditions for students to learn—then ensure that this learning is deep rather than simple regurgitation. From the very start of this project the intention was to instill a thirst for learning in the students. It is important for students, particularly healthcare professionals, such as pharmacists to develop ownership of their learning as this is likely to be a life-long process. By learning through a process of repetition and trial and error the student quickly appreciate their strengths and weaknesses. In addition, rapid feedback from the academic helps students formulate learning strategies and make sense of what they learn. Once the student has had that “Eureka” moment and understands the concept the academic has been trying to impart, they often deepen their own learning through communicating and explaining to other students. Formative assessment will deepen this process further. Paul Race’s “Ripples on a pond” model [7] provides the framework around which this project was developed.

The hypothesis on which this project was based was that provision of elearning software as an addendum to conventional teaching material could enhance student engagement and performance. A simple elearning activity was designed to encourage a deeper understanding of Integrative Physiology via trail-and-error exploration to help drive understanding of existing materials.

1.2. Development Workflows

In the field of Instructional design (ID) there are a number of frameworks that are useful when developing elearning content. These range from ID design approaches and implementations, to models of evaluation. A common elearning design approach used at RGU is the traditional stage-based ADDIE approach [8]. The workflow of an ADDIE project follows a step-wise workflow from Analysis, to design and development then implementation and finally evaluation. In this manner, ADDIE project progression is often depicted as water flowing serially down a single staircase. Development projects typically take place in response to an identified learning need and are staffed with elearning support personnel partnered with experts, commonly lecturers, in the target field.

The needs are diagnosed for the learner and a complete solution is implemented. In-house staff have found ADDIE-based projects to work well for large scale changes scheduled during student breaks or major course redevelopments, however it is less suited to term-time developments where staff resources are spread thin due to concurrent teaching and general learning support demands.

Whilst having reduced development time during semesters, there is a general high availability of potential test users. Utilising students as a resource, allows development projects to follow a rapid prototyping methodology [9]. This method involves iterative developmental steps that utilise early feedback received from both the student learners and tutor stakeholders. Development loosely follows ADDIE steps but in spiral workflow with each successive loop a self-contained output item with increasing functional complexity.

As a complement to the iterative nature of the development process, an adapted just-in-time teaching approach was chosen to develop a resource which met current needs, but inherently contained capacity to be extended. Just-in-time teaching is a combination of classroom and web-based training introduced by Physics Professor Gregor Novak as a solution to falling engagement within his science classes [10]. In order to maximise the efficiency of a classroom session with human instructors, Novak designed a web-based learning component intended for students to review outside the classroom prior to attending their lessons. Review can take place at any time convenient for the learner, and can therefore be subjectively viewed as “in my own time” or “just for me” learning by the learner [11]. The web component was split in to WarmUps, typically thought-provoking background information of a given classroom topic and Puzzles relating the background information presented [12,13]. As part of a feedback loop, students were invited to give their feedback on the web component to help the teacher build a so-called active learner classroom. Here, the teacher is expected to take the feedback onboard and adjust the teaching accordingly.

During our learning intervention, an engaging just-in-time teaching approach was employed using the web activity usage statistics, word-of-mouth and open-entry reflective comments as feedback to inform classroom learning.

2. Methodology: Project Development

There were 3 phases to this project; (1) design and development of the core elearing object, (2) completion and implementation of an elearning activity, and (3) overall evaluation of any change effects upon use of the module elearning content. The sample group was 146 first year undergraduate (2011/2012) MPharm students undertaking the SCQF 7 (Pharmaceutical Biology) and SCQF 8 (Integrative Physiology) modules.

2.1. Phase 1: Development of Core Elearning Elements—A 3D Virtual Arm

The open source 3D tool “MakeHuman” was used to create a gender-neutral human form (Figure 1) as a 3D mesh (MakeHuman.org, Giulianova, Italy) [14].

Using the 3D CAD package Blender 2.5 (Stichting Blender Foundation, Amsterdam, Netherlands) [15], the full human form was reduced to a core arm template The majority of the time allowance for core development was devoted to this stage of the project with no iterative feedback required. Expertise in 3D mesh editing and texturising was required as was anatomical expertise to reproduce a simplified internal skin structure depicting different skin layers.

From the complete limb, an arm was extracted from the 3D mesh and a representative cross section “block” imported into the free-to-use package Trimble SketchUp (Trimble Navigation, Sunnyvale, CA, USA) [16] formally Google SketchUp (Google, Mountain View, CA, USA). In both phases, development time was monitored at each stage.

Figure 1. 3D Virtual Arm template creation.

Figure 1. 3D Virtual Arm template creation.

2.2. Phase 2: Design and Implementation of the Learning Activity

The virtual arm 3D object was edited and formed into an elearning activity. Using organic modeling tools with the SketchUp application 3D features of various skin lesions were re-created from 2D photographs and each added and saved as separate instances of the main template. (Trimble Navigation, Sunnyvale, CA, USA) [16] Open-source rendering software Kerkythea was used to create a series of angular “shots” of each block section (Solid Iris Technologies, Attiki, Greece) [17].

Each set of shots were processed using commercial Adobe Flash software to create a 3D object tour [18], a common feature typically viewed on product websites to virtually rotate a given product. This allowed students to manipulate the area under review and to identify the specific skin layers involved. Areas of each lesion example were labeled and all objects embedded into the commercial Articulate™ elearning development package (Figure 2). Once added to the Articulate™ software (Articulate, New York, NY, USA) [19], further photographs, text descriptions with voice over, and quiz elements were added.

Prior to using the “virtual arm”, clear and concise instructions for using the presentation were provided. To ensure students had the opportunity to assess their learning they were provided with appropriate feedback and a quiz evolved including “fun” tasks such as a sequence, spell and visual test. If a student achieved a correct answer they were immediately “rewarded” with a “round of applause”. The students score was still provided at the end of the quiz together with the pass rate of 80%.

Figure 2. Skin legions activity created and embedded using presentation tool.

Figure 2. Skin legions activity created and embedded using presentation tool.

2.3. Phase 3: Evaluation

Kirkpatrick’s 1998 four-level model was adapted to help structure the qualitative and quantitative data collected for evaluation [20]. An updated description of Kirkpatrick’s model that includes elearning considerations and a fifth-level to examine return of investment or ROI [21] was used. Levels 1 to 3 (Reaction, Learning and behavior), embody a subjective form of evaluation which were combined with the more objective levels 4 to 5, namely Results and Return of Investment (ROI).

Level 1: Reaction. The first evaluation level of Kirpatrick’s model is Learner Reaction [20]. Data at this level gauges initial impressions using a quick form of feedback. Reaction to the material was gauged via discussions in-lecture and practical sessions. This also acted to verify the accessibility and functionality of the learning object’s final stage of prototyping.

Level 2: Learning. At level 2, our evaluation moves to a comparison between the personal learning experiences of the students and the intended goals. Views of personal learning were obtained from portfolio reflections following ethical approval from the RGU ethics committee. Students were asked on a voluntary basis to reflect on their learning experience as part of their e-portfolio and share these reflections.

Level 3: Behavior. Behavior seeks to determine the success in meeting the organisational goals, and identifying the strengths and weaknesses in the training process. Behavior was gauged by looking at how reflective comments such as those detailing how learner actions were effected by the presence of the Virtual Arm activity.

Level 4: Results. Results in the context of the Kirkpatick model produce the most clear quantitative evidence of meeting the training goals [20]. Data was obtained from overall Multiple Choice Question (MCQ) results produced from student assessment.

Level 5: Return of investment (ROI). The ROI is a performance measure which is best suited to financial comparisons between what is spent to and what is obtained from any given intervention. This metric helps stakeholders determine whether the project can be repeated efficiently. Whilst staff hours translating to a financial value were logged during the project, student outcomes are less easily quantifiable in cash terms at the departmental level. The equation to determine ROI in terms of financial efficiency is given as:

ROI = (Gain of investment (£) − Cost of investment (£))/Cost of investment

(1)

The authors propose an ROI that can measure efficiency based on use of existing and novel materials as measured by development time. The development and implementation of the 3D object was conducted in 2 phases, (1) the creation of the reusable core object, and (2) the customisation of the core object onto an elearning activity. Materials efficiency can then be measured as a percentage of the total time investment that can be reused for projects that require the same resources, therefore:

ROI ^materials = (Time spent creating reusable template/time spent creating entire object) × 100

(2)

Whilst this ROI approach does not deliver a generic cash-based comparison with other projects, standalone efficiency is still represented in terms of creating further developments from the existing core 3D object.

3. Results and Discussion

3.1. Use of the Virtual Arm alongside Existing Elearning

Pharmacy students require an appreciation of the structure and function of the integumentary system and not simply its role as a defence mechanism. A pilot learning object branded “The Virtual Arm” based on supportive immune and integumentary system notes for the SCQF level 8 (Integrative Physiology) courseworks was designed as described above and instigated as a new component during the courseworks. Articulate™ Presenter software was used to design a supportive presentation around material covered in the courseworks but also to introduce new material to the students. Supplemental to the 3D Virtual Arm object, Articulate™ Engage (Articulate, New York, NY, USA) was used to develop in-depth introductions to the immune and integumentary systems [19]. Articulate™ Quiz maker software (Articulate, New York, NY, USA) was used to design a revision quiz for both immune and the integumentary systems [19]. The pass rate for all quizzes was set at 80% and this information was shown to the students together with their own score at the end of the quiz. All questions were randomised as was the order in which the descriptors and key were presented to students. No restrictions were put on access and all e-learning materials were available to students either on or off campus throughout the course of the module.

An external software package, from the University of California, the Ouchterlony simulation was also used in this project [22]. Together with an adapted feedback presentation these illustrated one of the major concepts of the immune system, antigen-antibody interaction. Various mammalian and avian antibody/antigen combinations were used to place the information into context. In addition to the elearning materials appropriate articles from healthcare journals were used to enhance students learning [23].

In year one of the MPharm there were 146 students, from the data collected seven students failed to access any of the immune system information and eight failed to access any information pertaining to the integumentary system. The majority of students who did not access the material had previously passed the coursework component of the module and had no requirement to resit this aspect of the module. All activity report data from Moodle was collected, anonymised and entered into a password protected database. Several points were identified; certain material was not accessed by some students, whereas other students accessed the same material 19 times. This was particularly obvious with the interactive presentation file for the immune system [19]. The Ouchterlony simulation was also accessed on a frequent basis by a large number of students, yet some students did not engage with this programme [22]. Students’ generally accessed the immune system material to a greater extent than the integumentary system. Indeed only four students looked at the Integumentary coursework presentation [19].

3.2. Evaluation of Quantitative Data (Kirkpatrick Levels 4 and 5)

For students to pass the coursework component of Integrative Physiology they had to pass two equally weighted summative coursework assessments (a lab report based on the cardiovascular system and a 30 question multiple choice MCQ coursework test) which contribute 40% towards the final grade.

Students performed well in the coursework test with a mean pass of 66.5%. Only two students failed to pass the coursework test. This compared well with the laboratory report where the mean score was 62.9%. As with the MCQ test only two students failed the laboratory report, however these were not the same two students who failed the coursework test. Statistical analysis was performed on the SPSS database to ascertain whether any correlation existed between student engagement of the elearning material and performance in the MCQ test. Spearman’s Rho non-parametric test was used and this highlighted certain elearning activities as having a significant correlation with outcome in the MCQ test.

In brief, the immune coursework power point presentation (including an introduction to inflammation) and an article on the immune system (support material for lecture 2 and 3) showed a significant correlation at p = 0.01 with student performance in the MCQ test. In addition, the Interactive immune presentation file, Immune System Revision Quiz file (Articulate, New York, NY, USA) [19], all showed a positive correlation with performance in the MCQ test at a 0.05 level in a 2-tailed Spearman’s Rho test.

No correlation was observed between any of the other teaching material and performance in the MCQ test.

Development time was used to summarise a value representing the Return of investment for the hours spent developing the Virtual arm. A total of 7 hours was spent developing the core 3D object that was used to create the elearning activity in phase 2. Each Phase 2 skin element took an average of 3.5 h using in-house templates (excluding render time) to create, therefore:

ROI ^materials = (Time spent creating reusable template/time spent creating entire object) × 100 = 66%

(3)

This return of investment of a 66% reusable elearning resource is the subject of preliminary discussion centred upon further development of this tool.

3.3. Evaluation of Qualitative Data (Kirkpatrick Levels 1, 2 and 3)

Out of the 101 voluntary reflective posts on Integrative Physiology from students in year one of the MPharm 73 of these were reflections on the immune and integumentary systems, the subject of this project. Thematic analysis was used to identify three key recurrent themes (1) timetabling, (2) learning by doing, (3) reflecting on own needs. These are listed below, along with supporting quotes.

3.4. Timetabling

Due to timetabling issues, coursework sessions were organised prior to lectures: A key point when considering some of the student reflections.

• Student 1: “I do feel that I have understood the immune system slightly better thanks to having the coursework session before the lectures on this topic”.
• Student 2: “I feel that I was able to get more out of the lecture having previous been introduced to the material at the coursework”.

3.5. Learning by Doing

• Student 3: “The online virtual arm was a really useful resource. The Information was presented in a clear fashion and I felt my learning increased as a result of using it”.
• Student 4: “…the topics were difficult but they were made much easier by the way they were taught the lecturer had put a virtual arm on moodle along with quizzes which made it much easier the integumentary system as you were able to interact with parts of the system rather than just reading about it…”.
• Student 5: “…I feel especially the knowledge on the integumentary system will be of great benefit to my career in pharmacy…”

Students were also expected to work through a series of complex tasks to learn about antibody:antigen interactions as part of the coursework and then reflect on their learning by presenting the information in the form of a short one page report detailing results and putting these into context in a discussion. The overall view of this was favourable with students learning through trial and error.

• Student 6: “Have learnt that an epitope on an antigen can be recognised by more than one antibody and more importantly an antibody can detect more than one eptiope as shown by preciptitation lines in Oucterlony analysis. Learnt how to write up a results and discussions section correctly, importance of legends, labelling figures and tables and giving a brief description of what’s happening in the results section before expanding and explaining why in the discussion section”.
• Student 7: “Understanding the antibody/antigen interaction is important in understanding the immune response and learning this now will make it easier in later modules on immune system in 3rd year…”

3.6. Reflecting on Own Learning Needs

The combination of tasks provided during the coursework allowed students to highlight points of weakness and take ownership of their own learning needs.

• Student 8: “…By going through the notes, I did get a better understanding of the immune system but the quiz also showed me that I still need to go through the interactive powerpoint again”.
• Student 9: “Make notes from the interactive presentations and simulations that are available to me on moodule”.
• Student 10: “…there was good application containing information on skin lesion. This really helped as it was full of diagrams, which I find is is one of the best ways I can learn. …”
• Student 11: “…I have developed my knowledge of skin lesions and wound healing, finding the “virtual arm” resource on moodle a very helpful tool for this”.

Although, a voluntary request year one of the MPharm engaged and responded well when asked to reflect on their learning needs in relation to the immune and integumentary systems.

4. Discussion

The immune system had been removed from the Integrative Physiology module in previous years due to the large amount of material students were expected to cover in this SCQF level 8 module. However, due to a lack of space elsewhere within the MPharm the immune system was re-integrated back into the Integrative Physiology module in 2010/2011.

The premise in beginning this project was that if you provide students with the opportunity to assess and reflect on their own learning and provide the tools for them to answer their own areas of concern engagement with the learning process increases. It was also noted that rather than simply regurgitating information when the students are engaged with their own learning they start to apply their knowledge base to answer more in-depth questions.

The educational framework around which taught and support elements within these two topics was designed was Race’s “Ripple on a pond” model [7]. By providing students with a want to know and through a process of trial and error, feedback and assessment it was hoped students would gain a deeper knowledge and understanding of the immune system which they could take into later years of the MPharm.

The findings of this study found that the majority of students engaged with the elearning material in a meaningful way. It was apparent from the data collected however, that specific areas were revisited more often than others and these tended to be within the immune system topic. This was not unexpected, the integumentary system as the bodies first line of defence was initially introduced in the immune system. Therefore by the time the students came to the integumentary lectures and courseworks’ some of the material was already familiar to them and was reinforcing previous knowledge.

All material was made available for the duration of the module and this proved important as students were able to delve into particular aspects as they perceived their own learning needs

As the framework for teaching was the Paul Race’s “Ripples on a pond” model [7] it was reassuring to find that initial statistical analysis highlighted the importance of providing additional supportive material impacting performance in the MCQ coursework test. Particularly the immune interactive presentation file [19], Immune System Revision Quiz file [19], Immune System Coursework Power point presentation.

In providing extensive feedback to the students both verbally and written it was intended that students formulate their own learning strategies to make sense of the material they were studying. During this process several students had a “Eureka” moment, i.e., you could see that they had suddenly grasped the point being conveyed and were subsequently confident enough to “peer teach” less aware colleagues thus deepening their own learning through communicating and explaining to others.

Several forms of assessment were built into the coursework in the form of Articulate™ quizzes (Articulate, New York, NY, USA) and a formative mini laboratory report where students were required to write a results and discussion section on the Ouchterlony experiments [22]. This served two functions; it allowed formative feedback on new learning and—through deeper verbal discussion to the group as a whole—deepened students learning of the area. As the students were also required to write a cardiovascular report as part of the coursework component of the module it gave the students practice and formative assessment on writing a laboratory reports—many year one students have had limited practice in this type of assessment.

Surprisingly, neither the Ouchterlony simulation nor Ouchterlony feedback exhibited any correlation with the performance in the MCQ test. This may be because of unintentional bias as a result of providing verbal feedback on the Ouchterlony simulation during the integumentary system coursework.

Bias is a key area to consider when asking students to reflect on their learning experience. The potential for bias exists as students may reflect or skew their reflection simply to gain “positive attention” [24]. As 73 out of the 101 responders to the e-protfolio were on the immune and/or integumentary system positive social bias cannot be ruled out here.

Kolb’s experiential learning cycle places reflection in a central position pedagogically [25]. In this model students’ learn more effectively if they are actively involved in the learning process than if they are simply passive receivers. From the thematic analysis of the students’ reflection this certainly seems to be the case.

Once students accessed the material it was obvious that not all students accessed the model at the same stage but they all achieved some benefit through engagement with the material.

One unexpected finding with this project was that due to timetabling issues the immune coursework was timetabled prior to lectures on this area. This was not the preferred programme of activities envisaged at the beginning of this project. The expectation was that this would be problematical to all students; however, in light of the students’ comments this is something which will have to be reflected upon.

• Student 12: “I do feel that I have understood the immune system slightly better thanks to having the coursework session before the lectures on this topic. Although at the time during the coursework I did have some difficulties in understanding some of the material, I feel that I was able to get more out of the lecture having previous been introduced to the material at the coursework. This made me feel a bit more relaxed with this difficult topic”.

5. Conclusions

Initial findings from this study highlight complementary elearning tools as having a positive impact on students’ performance; in addition to providing access to colleagues and later years of the MPharm as supportive directed study. The intention of the authors is to further develop these tools as an interprofessional learning resource for pharmacists and other health care professionals in the area of skin, wound care and infection management.