*Article* **Experiences from COVID-19 and Emergency Remote Teaching for Entrepreneurship Education in Engineering Programmes**

**Chris G. Lambert 1,2 and Allan E. W. Rennie 1,\***


**\*** Correspondence: a.rennie@lancaster.ac.uk

**Abstract:** Education systems and institutions, often historically considered to be resolute, slowmoving entities transformed virtually overnight during the earlier stages of the COVID-19 pandemic, demonstrating nimbleness in adversity. This paper describes the first-hand experiences of teaching staff and students from a UK university which pivoted to emergency remote teaching for a core second-year module in engineering, focused on entrepreneurship. A range of methods are used including self-reflection, summative, formative, and focus-group student feedback. The paper provides an insight for readers who may be interested in the practical challenges associated with moving from an academic module typically delivered in a face-to-face learning environment accommodating a large student cohort (n = 177), to one that exists entirely in the digital domain. Our results show learning outcomes were fully met despite stark differences in quality of learning environments amongst students. Students reported benefits to remote learning because it offers a blended approach of both asynchronous content and synchronous sessions, with the latter enhancing engagement and providing structure to working weeks. Issues of presence emerged amongst group work: whilst it might be easier to confront some individuals for lack of contribution, it is also easier for those individuals to disengage. There was widespread support for the Microsoft Teams platform amongst students and staff but the former group reported this lacked a social environment in which relationships amongst team members could be nurtured informally, such as was experienced via social media.

**Keywords:** emergency remote teaching; engineering education; COVID-19; entrepreneurship education; distance learning; online learning

#### **1. Introduction**

The COVID-19 pandemic caused untold disruption and misery throughout 2020, affecting the lives of millions of people globally. The profound impacts of the virus have shocked economies and societies in ways that have not been witnessed by many in living memory and with consequences that are likely to be felt by future generations. Education systems have of course not been immune and have required dramatic changes at very short notice across all levels of the learning spectrum, from school children to postgraduate research students. The speed of the reactions of educational institutions and their members is an admirable reminder of the flexibility and responsiveness of teachers and students in ensuring that the provision of a high-quality learning environment is maintained despite the adversity faced. Universities closed their campuses and whilst some international students remained in accommodation, the reality of continuing to provide higher education teaching and associated assessment became a high priority, for teaching, administrative, and technical staff alike virtually overnight.

This article aims to report on the experiences of engineering teaching staff from a United Kingdom university in the summer term (April to June) of academic year 2019/20, at the height of movement restrictions during the first Government-imposed lockdown.

**Citation:** Lambert, C.G.; Rennie, A.E.W. Experiences from COVID-19 and Emergency Remote Teaching for Entrepreneurship Education in Engineering Programmes. *Educ. Sci.* **2021**, *11*, 282. https://doi.org/ 10.3390/educsci11060282

Academic Editor: Palitha Edirisingha

Received: 31 March 2021 Accepted: 2 June 2021 Published: 7 June 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

This paper provides an insight for readers who may be interested in the practical challenges associated with moving from an academic module typically delivered in a face-to-face learning environment to one that exists in the digital domain. The research aims are as follows:


Ambiguity surrounds the lexicon and definitions used when considering the differences between terms such as 'distance', 'online', 'digital', and 'remote learning', when taking account of characteristics of the learning environment and geographical interpretations of terms [1]. Distance learning is often considered to be a useful method of supporting adult education in particular due to the additional responsibilities adults may have in terms of parenting and employment. Distance learning is not a new concept of course and has been remarkably successful in education business models such as the Open University in the UK [2] for many decades, although distance learning is not without its reported barriers [3]. Some of these include "loss of student motivation due to the lack of face-to-face contact with teachers and peers, potentially prohibitive start-up costs and lack of faculty support" [3]. Further challenges have reportedly included the quality of instruction, cost effectiveness, misuse of technology, the role of technician and other support staff, and problems with operational efficiency of equipment [4].

The aforementioned terms generally refer to those where there is an implication of considerable planning in the design, execution, and evaluation of learning. For those who have pivoted to a different mode of delivery quite often in the space of a few weeks, such terms would be inappropriate. As others have posited [5–7], the terms adopted here refer to emergency remote teaching (ERT), which emphasises the urgency of the situation and the reactionary nature of dealing with a new mode of operation with relative quickness. ERT is a "temporary shift of instructional delivery to an alternative delivery mode due to crisis circumstances. It involves the use of fully remote teaching solutions for instruction or education that would otherwise be delivered face-to-face or in a blended or hybrid nature and that will return to that format once the crisis or emergency has abated" [7].

#### **2. Engineering Education**

The experiences of pivoting to ERT for university teaching in the context of COVID-19 are becoming increasingly more reported, helping others to benefit from a growing body of collective knowledge. This collection of topical research provides a rich and dichotomous snapshot of some of the effects of the pandemic. Many of the learning inequalities made acutely apparent as a result of COVID-19 have been widely reported [8,9] with the digital divide continuing to be a threat to addressing large scale equal access to education opportunities [6]. These social implications are grave and with increased prominence, it is hoped will motivate leaders and politicians to evoke widespread change. In keeping with the spirit of optimism, there have been considerable innovations in technology [10] as well as calls to make the future more socially responsible and environmentally just [11].

In terms of education of engineers, traditional curricula require tactile engagement through fabrication and testing, achieved by accessing workshops and laboratories with recent reviews describing the challenges associated with the student experience, quality assurance, assessment, and technologies [12] bought about by COVID-19. Student feedback has included fear and worry about their own health and the health of loved ones, difficulty in concentrating, disruption to sleeping patterns, decreased social interaction due to physical distancing, and increased concerns of academic performance [13]. This is shared in similar studies where causes of students' negative feedback of the situation include

network instability, unilateral interactions, and reduced concentration [14]. Consequences from mental health challenges during the pandemic could have long-term implications on student health and education [15].

Student responses to coping with what was the new education environment have been less well documented and have included strategies for self-discipline [16], use of appropriate technology to engage with instructors [17], and preferences over media type and length of sessions [18]. Authors have reported how teaching staff exercising compassionate and flexible pedagogy alongside effective communication has helped to support integrated engineering students [16]. The challenges with allied practical-based courses such as in the health professions present risks and opportunities [19] but the role of technology is commonly debated and often presented as having a lasting legacy [20,21], particularly the adoption of digital technologies [22]. Furthermore, practical course content has been successfully delivered through novel approaches to enable engineering students to be sent materials to undertake fabrication at home and fulfil learning outcomes [23]. Just as teachers have adapted, so too have students and successful mitigation strategies have been captured by overarching approaches to self-discipline.

A key feature of any undergraduate engineering programme should be that it helps enable people to work together by fostering and promoting teamwork. This aim has come about from both political influences as well as the increased demand by employers for graduates from technical disciplines to possess more transferable skills [24] and is a necessary component for accreditation awarded by the Professional Engineering Institutions (e.g., in the UK, the Institution of Mechanical Engineers and the Institution of Engineering and Technology) under the auspices of the Engineering Council [25]. The use of a range of strategies to achieve this has been investigated [26] and it was found that a combination of approaches brings benefits: embedded learning (where there is no direct reference to transferable skills), integrated learning (where skills are developed in parallel with core discipline knowledge) and bolt-on learning (where skills development is independent from core discipline). ERT may have an impact on the development of such transferable skills, especially where teamwork is a particular function of the mode of delivery. If students work together in groups online, does this better prepare them to be part of the global engineering workforce of the future?

#### *2.1. Entrepreneurship Education for Engineers*

Entrepreneurship education has seen significant growth in university curricula since the 1970s [27] with a recognition that the impact of capitalising on ideas will lead to economic growth at the company, regional, and national levels [28]. However, the role of entrepreneurship education is not without challenges, including measurement criteria [29] and impacts beyond courses such as its role in graduates, and their entrepreneurship [30]. Effective entrepreneurship education amongst engineering students has shown that this is a prime group for starting technology-related businesses at rates above the general population [31] and that entrepreneurial intention increases [32]. Authors have suggested that the role of entrepreneurship education in engineers should not be confined solely to reacting to economic growth and social returns but to provide engineers with the necessary skills to enhance their profile for a knowledge-based economy [33].

Within the Engineering degree courses offered at Lancaster University, a 15-credit, core-module undertaken by all Second-Year students (equivalent to UK FHEQ5) provides entrepreneurship education: Business Development Project (module mnemonic ENGR205). Taken from the Module and Programme Catalogue, the educational aims for subject-specific knowledge, understanding and skills are:

*To expose students to a rich mixture of experiential learning opportunities that develop a wide range of transferable skills in the context of engineering entrepreneurship and innovation, with a particular focus on the development and use of business plans and marketing strategies.*

The educational aims for general knowledge, understanding and skills are:

*To introduce students to a wide range of transferable skills in the context of entrepreneurship and innovation. To develop students' ability to think and argue critically, and plan and organise their work whilst being cognisant of team dynamics and operations.* The learning outcomes are provided in Figure 1.



**Figure 1.** Learning outcomes (subject-specific and general) for ENGR205: Business Development Project.

The module is delivered over an intensive five-week period during the summer term (April to June) and is made up of a number of lectures which include various guest speakers who share their experiences of business creation, entrepreneurial behaviours, and innovation management. The cohort size for academic year 2019/20 was 177. Students work in teams of between 10–12 which are assigned by the module convenor taking account of registered educational programme (e.g., degree discipline being studied) and level (i.e., Bachelors of Engineering (BEng Hons) or Masters of Engineering (MEng Hons) to ensure a semi-even distribution of skills, interests, and capabilities. In addition, there is also the option of selecting to participate in this module from outwith the core engineering student cohorts, including by students selecting engineering as a minor element of a Natural Sciences curriculum and international students undertaking a joint year abroad. This enables the benefits of non-self-selection such as knowledge spill over, cross-cultural learning relations, overcoming initial differences, and developing a strong team identity [34]. Such a method helps students prepare for workplace environments where being given a choice of colleagues with whom to work will not be an option for most. Moreover, it was found that students selecting their teammates has a very small impact on their level of satisfaction in the team [35]. Although the team size may be larger than optimum, the whole class size, available timetabling, and management and delivery over a short, intensive period of time by one module convenor within the final term of the academic year provides limited options to assign smaller team sizes.

The module is scheduled for the summer (UK) term of the academic year as assessment is 100% coursework-based, following the formal examination period completed in the weeks prior to delivery, providing the students with a somewhat different emphasis from their personal, focused study. Perhaps akin to the business and commercial world that the module emulates, it is conducted intensively over the course of four weeks, with a four-hour core session (with short breaks) scheduled each week for the whole class to participate. The fifth and final week is assigned for the completion and submission of assessed materials (see below). A structured lecture schedule is presented during the

weekly sessions, with the intention to provide the students with experiential learning, situational and commercial awareness, and innovation and entrepreneurial practice. Although the intention is to witness success—defined by the submission and pitching of a credible business proposition—by the completion of the module, it is important that through the formal lectures, many delivered by external speakers, emphasis is also placed upon 'failure', as this also constitutes a rich learning experience [36], developing resilience, tenacity and the motivation to try again [37]. Guest speakers do not therefore present solely on their professional acumen and business successes, but also on their journey through failure and disappointment. Lectures from guest speakers include, but are not limited to: 'Brilliant Business Ideas: Student Enterprise and Innovation'; 'Innovation in your Engineering Career'; 'Technopreneurship'; 'Pitching and Presentation Techniques'; 'Industry 4.0: A Once in a Generation Opportunity'; 'Patents vs Innovation'. These supplement lectures covering core learning and material from the module convenor: 'Idea Generation'; 'Venture Planning'; 'Business Model Generation'; 'Venture Creation'; 'Communication of Business Proposals'; 'Market Segmentation'; and 'The Marketing Process'.

In order to develop teamworking skills, the students are set weekly tasks to complete within their groups, for delivery to the class for feedback by the module convenor and the wider cohort at subsequent sessions. These include use of recognised management and entrepreneurial tools such as the Strategyzer Business Model Canvas (Figure 2), small group presentations, elevator pitches, and role playing (in the context of professional roles within a company).


**Figure 2.** The Business Model Canvas, modified (original available from strategyzer.com, accessed on 31 March 2021).

Students are assessed 100% by coursework which constitutes a business plan of ca. 4000 words (worth 60% of the module mark) detailing the group's idea or proposition and an opportunity to formally pitch the business idea via a presentation or video (worth 40% of the module mark) to an 'expert panel'. The panel consists of the module convenor, successful industrialists, and other academics (including from international universities), which comprises some of the guest speakers (to represent continuity in the module proceedings), and is concluded with a Question & Answer session which is used to further enhance learning, providing students with the opportunity to respond to questions, provide verbal clarifications, and to gain first-hand (constructive) feedback regarding their business proposition.

The module has been successfully delivered for several years with consistent positive student feedback, such that the basis of the module has been replicated for students in the Department of Biological and Life Sciences at this same university and for engineering students at a partner institution in China.

#### *2.2. Pivoting to ERT*

In the course of pivoting to ERT, the module convenor had very little time to consider either the wider implications for online teaching and learning or any opportunities being made available from the university for online delivery, a scenario faced by many educators globally [38]. Rather, the convenor had to find solutions that would allow the continuation of delivery and the ability to progress and conclude the academic year. Transitioning from a module that historically has benefitted from the intensive, face-to-face workshopstyle environment where visual and emotional behavioural cues help to engender team cohesiveness and relationship development, to one conducted entirely online, often with cameras switched off (in the case of the students) proved to be the biggest barrier to address. Added to this, despite being online, the geographical location of students played a key consideration, particularly as the majority of international students within the cohort felt it necessary to return to the support of family environments in their home countries, due to the longer-term uncertainty of the pandemic situation. As such, two 'international groups' were established to take account of time zones—one in South East Asia (primarily GMT+7) and one in Eastern Europe (primarily GMT+2), with the remainder of students being located in the UK and Western Europe (GMT+1). Weekly sessions were therefore scheduled for 10am start (within core hours for the majority of the class based in the UK), but within a reasonable time difference for access by others, particularly in South East Asia who essentially had to phase shift their working pattern to a later part of the day (also accounting for other university work)—additionally, these international groups could then set their own schedules for working collectively (e.g., preparing for the weekly tasks) at a time of day more conducive to their location. Feedback received was positive, emphasising that inclusion of all students had been considered, despite the difficulties of the (rapidly changing) situation faced. More generally, there was acceptance from within the cohort that there would be much wider implications affecting educational provision in the longer-term, including the likelihood of online learning being here to stay, a position felt not just at Lancaster University, but by undergraduate students more generally [38].

Guest speakers embraced the opportunity to participate virtually, primarily through synchronous delivery (in order to engage in real-time question and answers sessions—a source of rich learning from the 'experts'), but also through asynchronous content, made available from the module's dedicated virtual learning environment (VLE) site. All sessions were recorded and made available afterwards on the VLE, to allow the students to access this at a later time to reflect on the content delivered, recap on feedback provided from the weekly tasks (an opportunity never provided previously when the module is delivered real-time in-class) and to fully understand the task set for the week ahead (where ambiguity may exist). From a module convenor perspective, it was positive to note the students' use of the main Microsoft Teams group for the weekly sessions whilst simultaneously communicating with group peers within their private channels, as they discussed and established ways forward to tackle for example, the weekly tasks set, and to then confirm any misunderstandings within the main group before proceeding. Such interaction and engagement is not possible within the classroom environment, where team members can be spread throughout the room, relying on post-session discussions to clarify learning and discussing how they will proceed with the next task at hand.

Of particular note was the opportunity to capitalise on the situation of operating online through invitation of an international guest speaker, and senior business person who would not ordinarily have the time to dedicate for an in-person visit to the university, but were more than willing to join a short session online with the cohort. Being able to facilitate such opportunities brings significant added value to the learning experience of the students.

#### **3. Methodology**

A case study approach is an effective way of capturing the complexity of learning and teaching and the contexts and communities surrounding them [39], with particular regard in this situation to context; this approach exhibits close alignment with case study because of it being a bounded unit; it is located within a community, involves interactions and relationships between the case and the wider world, focuses on collecting rich data over a short period of time, and uses a variety of data collection tools [39]. Data legitimacy is provided by reporting on the perspectives of both teacher and student and using different data collection methods: formative student feedback, summative student feedback, reflection, and a thematic focus group. This triangulation helps to reinforce the quality of data collected [39], providing a thorough picture of ERT in the context of entrepreneurship education for undergraduate engineering students.

#### *3.1. Formative and Summative Student Feedback and Feedback to Students*

Formative student feedback was collected throughout the module via informal studentteacher interactions with the module convenor, mostly achieved through the Microsoft Teams platform. Immediate feedback is provided to the student groups (from the module convenor and student peers (engendering peer-to-peer learning [40])) following the delivery of weekly tasks set, but also on an ongoing basis (from the module convenor) through the use of channels (workspaces) within the Microsoft Teams platform, with each individual group operating within their own channel. Summative student feedback was captured by the end of module feedback questionnaire as required by standard university quality processes. Of the total class size (177), 38 responses were received, providing a 21.47% response rate. The sections of the evaluation which include written text were then analysed and coded based on the themes used in the focus group.

#### *3.2. Reflection from In-Module Student Engagement*

The opportunity to engage more closely with the student cohort (assigned to 15 independent groups of 10–12 students per group) using the online MS Teams platform, and in particular the digital trail that was created as each week passed, presented the module convenor with the chance to reflect, adapt, and reconfigure the module programme to ensure best advantage for all module participants (teaching staff, guest lecturers and students).

*"Group involvement over Teams was great and responsiveness and comments from [Module Convenor] were extremely helpful".*

#### *"I enjoyed coming up with a product and exploring the best way to implement it in a group setting as it allowed all decisions to be discussed".*

Reflecting on Kolb's Learning Cycle [41] directly in relation to online teaching delivery and the functionality of online platforms enabled the possibility to review, in real-time, students' engagement within sessions (which could still be viewed at a later date); this enabled appreciating the experience (Stage 1: Concrete Experience) of the students during these sessions, for example, as they developed their understanding of theoretical and practical material presented. Within the weekly tasks, students were able to reflect this in their own work, for example by taking best-practice approaches to completing tasks (presentations, elevator pitches, etc.; Stage 2: Reflective Observation). Through subsequent individual group workshops and full-cohort discussion sessions, students were able to discuss the likely implementation of this best practice incrementally into subsequent weeks' activities (Stage 3: Abstract Conceptualisation). This was evidenced and realised through the formal weekly deliverables where students would subsequently apply this knowledge (Stage 4: Active Experimentation) following peer discussion within groups, supporting

each other to the best effect. Ultimately, the benefits of such learning and reflection, across the range of engineering disciplines involved, supported the end-of-module assessment materials, where the formation of business plans and presentations/pitches were perfected based on this cyclic approach.

*"Getting to work with engineering students from different disciplines highlighted our different ways of thinking and problem solving, and made for some great teamwork and collaboration".*

Capitalising on using digital media more effectively, the module convenor was able to adopt Kolb's Learning Cycle and experiment with redefining aspects of the final assessment (whilst being cognisant of the need to adhere to the existing module learning outcomes). Students, having used video for aspects of their weekly deliverables, were therefore presented with the opportunity to create a 'multimedia promotional video' for their business proposition, as an alternative to a formal PowerPoint presentation.

#### *3.3. Thematic Focus Group*

A group of five students, two of whom were course representatives (participating on behalf of their respective engineering programmes) engaged in a post-module focus group to discuss their experiences of pivoting to online learning, generally in relation to the wider engineering curriculum, but specifically in relation to this entrepreneurially focussed module. This focus group aimed to elicit deeper feedback from students than those in the previous formative and summative student mechanisms. The focus group as a means of qualitative research has several strengths, including a cascading effect of discourse amongst participants and efficiencies of scale derived by interacting with several individuals simultaneously and serving as a mini-interaction laboratory [42]. Group interviewing can be particularly helpful when participants have been working together for some time with a common purpose [43], such as shared educational experiences. An interview schedule was devised which sought to extract feedback, congruent to the aims of the study. It was structured firstly with introductory remarks that included an emphasis on making improvements for the following academic year, which helped to create rapport and a sense that participants' input could have a lasting effect. The four principal sections devised for discussion were: Pivoting and Summer Term; Interaction/Software; Technology in the context of institutional platforms; and Outcomes. Table 1 provides the questions that were asked in each section.

It was conducted on Microsoft Teams in late July 2020 lasting approximately 1 h 40 min and led by the authors, recorded, and transcribed. The transcription was imported into NVivo 12 from where qualitative data analysis was undertaken. Firstly, to explore the data superficially, in-software visualisation tools were created such as word-clouds and hierarchy charts. Following this and using an inductive approach method, codes were generated where content emerged from the transcription; this resulted in the creation and use of 28 separate (grandparent, parent, and child) codes, which were referenced 175 times. Given the small number of participants, no case classifications (i.e., personal characteristics) were assigned. Pseudonyms are used to protect anonymity of those students participating. Using these four methods (formative student feedback, summative student feedback, reflection, and a thematic focus group) provided insights with those that contain multiple instances as forming the most pertinent points and hence the basis of our reported results.

This study has used student feedback via formative, summative, and focus group methods, on one course from a single institution, and as such the results may be relatively limited in terms of generalisability. The inductive nature of the work is intended to describe the emergent issues that have come from ERT early in the pandemic and as such, the reliability of the data may be limited.

**Table 1.** Breakdown of the questions (by section) used in the thematic focus group.

#### **Pivoting and Summer Term**

	- -Was there anything you felt anxious/apprehensive about?
	- -Was there anything you felt excited about?

#### **Interaction/Software**

	- -Did you use any platforms not prescribed by the lecturer (e.g., Messenger, WhatsApp, E-mail etc)?
	- - Did you 'meet' your team using video/call technology, e.g., MS Teams, Zoom, Skype, WhatsApp etc?
		- (i) If so, how many times?
		- (ii) Did the whole group attend?

#### **Technology in the Context of Institutional Platforms**

	- e.g., to access lecture recordings, access resources.
	- -What was your primary reason for using MS Teams before this point (learning, communicating, collaborating, etc)?
	- -What were the largest difficulties for you using MS Teams?

#### **Outcomes**


Students were then provided with the opportunity to make any other comments or general feedback.

#### **4. Results**

#### *4.1. Learning and Teaching Delivery*

Evidence of submitted work shows that learning outcomes were achieved with a recognition that the speed of pivoting to ERT was no easy accomplishment of the module convenor. Interestingly, rather than appearing anxious about this mode of delivery, students from the focus group reported a sense of excitement about the novel ways of learning, this being the first module taught to the year group entirely online since the move to ERT some weeks earlier. There appeared to be an openness to new learning brought about by the situation. This was somewhat quelled by uncertainty, especially of exams, which at Lancaster University typically take place for undergraduates at the start of summer term (April). As the first lockdown in the UK had begun, on the one hand there was anticipation of new ways of learning, yet on the other hand, students demanded certainty in what they might expect for the remainder of the academic year.

Students reported feeling more relaxed as a consequence of being in a familiar home environment rather than a lecture theatre. This had other benefits for individuals who had quietness and for example were able to utilise a desk to make notes and access to multiple dedicated computer hardware, all of which contributed to a conducive learning environment. However, this was not the case for all and one focus group participant did not have enough desk space at home for themselves and their siblings, meaning that this student completed all of their work on their bed. In the case of the former student, being on-campus would have presented a more favourable study environment.

There was no evidence in any of the student feedback that learning virtually was inferior to face-to-face teaching for this module. Students were actually keen to point out that lecture delivery improved in some ways, through the use of asynchronous recordings, where they could pause at any point to take notes, carry out online searches, or replay sections that were of particular interest or clarification was sought. This was highlighted when a focus group participant fed back that teaching staff during synchronous sessions were not able to recognise confused looks on the faces of students and had not prepared any ways of obtaining feedback or checking understanding in the course of delivery of another module. This was magnified when it was felt teaching staff had actually sped-up in terms of delivering content online, synchronously. There was support for synchronous sessions which was mostly to enable dialogue, by teaching staff responding to questions and providing structure to students' working weeks and working days.

#### *4.2. Teamwork and Group Dynamics*

It is clear that students valued group work with consistent positive comments from summative, formative, and focus group feedback. Students appeared to meet regularly in their groups (channels established in Microsoft Teams), multiple times weekly with some variance, and recognised that frequency changed when workload dictated. The oftencited frustration of students that are placed into teams that have members who contribute unequally unsurprisingly does not abate with doing this virtually. This was highlighted by 'David' in the focus group who commented on his observations of levels of contribution from different members:

*"I think there's some people who put more effort into the peer review than the report".*

The point of parity was given further consideration by 'Baris' who reflected on this compared to future professional working environments:

#### *"I think it is a sad reality that in the workplace some people will not pull the same weight as others".*

Whilst it might be discerned there is little difference to what is a ubiquitous issue amongst teams in virtually all types of environment, some aspects of engagement, delegation, and motivation appear to differ. The issue of presence was raised as a recurring point, in that individuals could appear to be present by joining a meeting yet contribute very little. Whilst a parallel can be drawn to the physical world of people who arrive at meetings and contribute little, the issue was felt to be more pronounced in the virtual context as individuals can join with extreme ease and could be doing other tasks at the same time. This is essentially epitomised by the almost universal decision of students to turn off their camera function, which has communication benefits of being able to see and respond to body language and non-verbal cues.

Having identified issues of performance amongst individuals, team leaders reported different experiences of addressing this, compared to doing it in-person. Some found it harder to do online and would have preferred to have dealt with underperformers in-person. Other people found it easier to address online (for instance by direct written communication) but interestingly found that the effect of this was less. In other words, although confronting someone face-to-face was harder, the effect of this was perceived to be greater. 'Raymond', a team leader captured the point in the following way:

*"Whilst it's easy to confront people about that, it's a lot easier for them to just sort of ignore you."*

#### *4.3. Technology and Online Platforms*

There was extremely widespread support for the use of Microsoft Teams as the preferred platform for engaging with formal content through synchronous sessions and asynchronous collaboration, with some students even reporting enjoying using it. This was used as the primary means of collaboration in the majority of teams that were represented in the focus group and positive feedback included ease of use, file sharing ability, updates from the developer, and use of the chat function. The chat function received particular praise as a way of asking questions during synchronous sessions that the same individual would not have done so, if in a lecture theatre or workshop environment. The ability to 'like' comments or questions was also seen as a particularly helpful means of agreeing with classmates, something that is not readily achievable in large-class situations. Students appeared to be uniformly in favour of Microsoft Teams as a platform for supporting learning by engaging with teaching staff and fellow students in a formal sense.

However, Microsoft Teams did not provide students with a social space where they could converse and "be friends" away from the more formalised platform used by the university and overseen by teaching staff. Students quite rightly placed a high value on this rapport-building aspect to their team development, which might include the sideline conversations that occur, further enriching interaction between group members. It became evident through the focus group that in order to achieve this alternative manner of connecting with fellow students, the Facebook Messenger platform was also used consistently and often comprehensively by some groups. Students felt that this was a place where they could put faces to names and ergo by inference, personalities to faces. It was also seen as a more casual channel in which punctuation, grammar, and spelling was less important than in using Microsoft Teams, which could be viewed at any time by university staff.

Other platforms which were referenced by students in helping to co-ordinate time and make decisions was Doodle Poll and Qualtrics, although it should be noted that the polls function in Facebook Messenger was cited as being used too. Google Drive was used to work collaboratively on documents where real time edits can be made by multiple users. Most student teams used Microsoft Teams for meetings. Discord was used to meet by one team represented in the focus group, but with generally negative feedback due to lack of features compared to other platforms and the team appeared to regret using it. No mention was made of other platforms such as WhatsApp, Skype, or Zoom. Use of the university's virtual learning environment (VLE), Moodle, was generally low with students reporting accessing it mainly to download module specific documents or obtain information (such as recorded sessions and the lecture slides) as well as upload the completed assessment materials.

#### **5. Discussion, Conclusions, and Implications**

It is evident from the formative feedback, supported by the focus group that students required certainty and clarity, which in times of unpredictable crisis can be difficult for university management and academic teaching staff to provide. This was mainly focused towards exams, and before announcements were made, it proved to be a point of concern and anxiety amongst the student demographic. The need for universities to not just communicate but take decisions early remains paramount to maintain student satisfaction.

Flexibility has long been seen as a contributing factor to students' decisions historically to opt for distance learning programmes [44], as this presents the beneficial opportunity of balancing study with other commitments. These advantages are only partially applicable when considering the forced nature of students to return to family, parental, or guardian homes where learning environments may have very stark and striking differences. An example of this is seen even in the small focus group which formed part of this study in which 'Raymond' had his own room with various computer hardware atop ample desk space whilst 'Julie' was working on her bed because her siblings were using the only available desk space within the family home. The authors present this as a necessary reminder that teaching staff need to have an appreciation of the likely differences in home learning environments, even if they are not intimately aware of each learner's personal circumstances.

As reported elsewhere [45], there are important access considerations when delivering material online, most patently that learners have internet access. Indeed, recent calls have been made by individuals such as Sir Tim Berners-Lee (who invented the World Wide Web) to make access to internet connectivity a universal right [46]. Computer infrastructure is an important pre-cursor to achieving such equity yet the idiosyncrasy presented by individual's personal circumstances need to be both recognised and where practicable, accommodated. Universities may not have the means to alter the learning environment but teaching staff should recognise that there are likely to be considerable differences amongst any given demographic. This may have profound impacts for issues such as quality of learning and knowledge retention.

Students need support in making group work as effective as possible [47], regardless of whether this is in-person or virtual. From the results presented, it appears this is an issue that requires teaching staff to reframe how teams are supported to be nuanced to the virtual world, if this is how the team will function. This requires specific guidance and scaffolding on team formation and performance, such as that which has been done in organisational development [48], as well as providing the infrastructure for teams to achieve optimally in virtual workspaces. Part of this is to consider how a climate of trust can build and sustain virtual teams [49]. A further recommendation for online group work informed by the results is, where practicable to do so, to use smaller teams, in the region of six instead of twelve participants. Such preparations are likely to have a side effect of preparing students to deal with a future professional working world where physical collaboration is less likely and where global virtual teams are more commonplace [50].

The demand for graduates to possess work-ready, transferable skills has increased by employers [24] and the pivotal role of teamwork within this is recognised universally. Our research shows that teamwork has presented different characteristics online, relative to face-to-face. It should be emphasised that the key feature here is not a reduction in attainment of learning outcomes but that there are features which are simply different. The most commonly reported was the issue of presence amongst team members, the significance of which is supported by previous research in online distance education courses [51]. Our own work shows the importance of students being present in team-based endeavours, which in very practical terms may include ensuring that they are actively contributing, turning cameras on, and leaving footprints or evidence of engaging within learning platforms. There are clear differences between temporary and ongoing distributed teams [52] and identified antecedents to conflict that are unique to distributed teams [53]. Being diligent, polite, and respectful members of a team in a distance learning environment will prepare students for when these very same qualities are required in a future online working environment.

From the research undertaken here, there are a number of implications for university teachers, educational institutions, and students which may be posited. For teachers of large undergraduate class courses, it is paramount that there is acknowledgement and a degree of sympathy with the variety of home environments that their students will be in. Some of these may be less conducive than traditional university studying environments and whilst changing those is unachievable, recognising and accommodating this should remain foremost in teacher's approaches. This includes thorough planning along with compassion [16], and instructor guidance and assistance, which it has been shown has a significant impact on students completing learning tasks [52]. Our own research suggests that instructors should consider the speed of their vocal delivery, especially in asynchronous content and by inviting feedback early-on from students during their course, and modify as appropriate.

For institutions, providing the software infrastructure to support learning in as consistent a way as possible is important. From our research, platforms such as MS Teams are

widely advocated by students and teaching staff, however any formal system such as this is not preferred for students to interact socially. The importance of learners' social context has been shown previously to be ignored by many educators [51], yet is a vital aspect to online learning. Teaching staff should recognise the social needs of students and instead of attempting to provide for this through formal methods, signpost students to initiate and take part in this outside of formal constructs. We found that social media platforms were reported by students as providing this essential connection and networks with their fellow students where they could exist as friends, not just as fellow students.

One of the potentially gravest and as yet unknown consequences from COVID-19 is the longer-term effects that have come from interrupted education, isolation, and grieving (at the loss of family and/or friends) or a combination of these experiences. As reported earlier, research [15] has been undertaken to consider the psychological impacts on university students, referred to as an increasingly vulnerable population. Whilst there are factors that contribute to higher levels of psychological impact that are out of the control of individuals (such as family income, race, and knowing someone infected with COVID-19), there are factors which reduce this impact such as spending at least two hours outside or less than eight hours on electronic screens [15]. Universities need to urgently develop intervention and prevention strategies to address psychological impacts that will affect long-term mental health [54].

We have reported a sense of excitement from the students initially about moving to a new way of learning, essentially bought about by the novelty of doing things differently. We believe this constitutes a "honeymoon" period and that the relative euphoria rapidly fades. Our experiences in this context show that some surveyed students have responded in an adaptable way to the emerging environment, closely resonating with themes of resilience and determination. Students that believe academic qualities can be taught (rather than being fixed) show higher achievement across challenging social and academic environments [55]. Furthermore, findings have demonstrated the positive correlation between determination and a range of academic performance indicators, underpinning the role that self-efficacy has in achieving such outcomes [56]. Linked to our own research, this supports the case for formal resilience training, as has been successfully used in other professional discipline education programmes [57] to become a feature of skills development in engineering courses. Such activity would be well-suited to entrepreneurship education given the significance it plays in company start-up success [58] and emerging business resilience frameworks for start-ups [59].

For students, it is clear from the work undertaken here and that of others [16] places a high value on structure and time management. Identifying daily, weekly, and termly routines that fit idiosyncratically into one's own personal circumstances is a fundamental step to gaining the most from learning at a distance. This forces students to consider their own self-discipline and transferable skills, such as time management, in a way that beforehand, was in part achieved for them, for instance with timetabled lectures and laboratory sessions. Asynchronous content inherently places responsibility on the student to actively engage with the material and having the supporting cognitive frameworks will help considerably. In a practical sense, this may mean developing and persevering with a timetable, regular study group sessions with peers, and the use of planning systems such as Gantt charts and work breakdown structures. Fundamentally, the benefits of sound planning, establishing a routine that works for individual circumstances, and actively managing time will most likely result in more positive learning.

**Author Contributions:** Conceptualization, A.E.W.R. and C.G.L.; methodology, A.E.W.R. and C.G.L.; software, C.G.L.; validation, A.E.W.R. and C.G.L.; formal analysis, A.E.W.R. and C.G.L.; investigation, A.E.W.R. and C.G.L.; resources, A.E.W.R. and C.G.L.; data curation, A.E.W.R. and C.G.L.; writing original draft preparation, A.E.W.R. and C.G.L.; writing—review and editing, A.E.W.R. and C.G.L.; visualization, A.E.W.R. and C.G.L.; project administration, A.E.W.R. and C.G.L.; funding acquisition, A.E.W.R. Both authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Acknowledgments:** The authors gratefully acknowledge the contribution and time from students who have provided feedback which has helped to produce the findings here and frame this narrative. We are particularly thankful to the five students who took part in the focus group following completion of the academic year.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

