For CS Educators, by CS Educators: An Exploratory Analysis of Issues and Recommendations for Online Teaching in Computer Science

Abstract

The COVID-19 pandemic has completely transformed the education sector. Almost all universities and colleges have had to convert their normal classroom teaching to online/remote or hybrid teaching during the COVID-19 pandemic. Online teaching has been found quite useful during an emergency situation. This switch to online teaching forced educators to come out of their comfort zone and learn new tools and techniques for online teaching. It is important, therefore, to analyse the problems faced by educators in online teaching because this has become the new normal. There are several studies identifying the issues faced by educators in online teaching but less is known about the issues faced by Computer Science (CS) educators. In this paper, we perform an exploratory study of the problems, questions, and associated responses from CS educators posted on popular Q&A forums, e.g., CS educators StackExchange. We identified six main challenges related to online teaching: platform recommendation, Q&A management, academic dishonesty, pair programming, and feedback mechanism. Several recommendations are provided by other CS educators in each of the categories, which are discussed in detail in this paper. This study will help organizations come up with better solutions to support their educators so that they can deliver better quality education and reduce the overall stress levels of staff.

1. Introduction

The COVID-19 pandemic started in December 2019, and by March 2020 it had spread across the globe. It is reported as one of the biggest pandemics of recent years [1]. COVID-19 is still having an impact. For example, in the UK, as per the government guidelines on 1 April 2022, adults and children with COVID-19 symptoms such as high temperature, should remain at home and avoid contact with other people for at least 5 days [2]. China has more recently announced the lockdown of some of its cities, including closing schools to stop the virus outbreak [3]. Hence, at present, it is difficult to say when the impact of COVID-19 will stop. Some researchers have predicted that the COVID-19 virus may be here for an indefinite time and will become endemic [4].

COVID-19 has impacted 87.6% of the world’s enrolled students [5]. It has caused 184 country-wide closures [5], with schools and universities being closed abruptly, either having to cancel classes or move to remote/online teaching methods. This kind of interruption in studies can have a long-term impact on the learning level of the students. A study by Baker [6] shows that 10 days or more of school closure can reduce student achievement, with this having the biggest impact on math courses. A study by Belot et al. [7] shows that school closure (due to strike) caused reduced educational attainment.

The COVID-19 lockdown was announced suddenly, and universities quickly came up with plans to manage education during the emergency. During that time, many students had to return to their hometowns to safeguard their lives. Countries announced their lockdown at different times, which caused problems in traveling. Many students were not able to go back to schools or colleges for months because of the travel bans imposed by various countries. In these, circumstances, online teaching was a boon. It helped learners continue their studies by attending classes remotely. Online video conferencing platforms such as MSTeams1, Zoom2, Google Meet3 etc., allowed teacher and students to deliver/join classes from anywhere in the world.

The COVID-19 pandemic has transformed the education sector, and now, online learning/teaching is an integral part of the new normal. A study shows that nearly 30% of US students are taking at least one online course [8]. Online education has the benefit that it can have an element of both synchronous and asynchronous teaching. Synchronous teaching is where teachers deliver live lectures/presentations to students, whereas in the asynchronous mode, lecturers upload their content on some online platforms, and the learners interact with it at their convenience.

In addition to during a pandemic crisis, online learning is quite beneficial in various other cases [9]. For example, first, it is helpful for individuals with work constraints/family constraints, i.e., individuals having caring responsibilities at home or working individuals who cannot attend traditional classroom teaching. Second, students who live in crowded cities want to avoid too much travel because of time constraints. Hence, it is important to give appropriate training to educators and students to make online instruction more effective. There are studies that show that educators and learners faced many issues with the sudden switch to online teaching/learning [10,11]. The United Nations has advised government(s) and various stakeholders to work on developing a coherent online teaching system for any other future disaster or emergency [12]. Prior literature discussed the issues faced by educators in online teaching [10,11]. There are some studies that compare the issues faced by educators based on the domain and level of the course [10,13]. It is important to focus on the domain of educators because educators teaching different courses can face different challenges in online teaching. For example, an educator teaching computer science has to conduct labs to give hands-on experience to the students. Hence, the educator may need to find a method for conducting online labs.

Some previous studies analyzed the issues faced by CS educators. Some of these studies show that CS educators are confident in remote learning/teaching [13], whereas other studies show that CS educators face issues [14] similar to educators in other domains. Computer literacy is one of the core skills needed today. It is so important that CS has become a required subject in some countries, for example, Israel, New Zealand, the USA, Eastern European countries, and the UK [15,16,17]. In 2016, the US started a ‘CS for All’ initiative with a vision to give access to computing education to all US students starting from kindergarten through high school [18]. Similar initiatives have been instituted the by the UK [17] and the European Union [19]. The ‘digital action plan’ by the European Union states that an introduction to computing at an earlier age can help in developing skills such as problem solving, creativity, and collaboration in students [19]. It is predicted that between 2016 and 2026, employment growth in the CS and mathematics field will be 13.6% [20]. It is therefore important to have a systemic approach to understand the difficulties faced by CS educators in online teaching as it can help in improving and planning future online teaching during emergency situations such COVID-19.

In this paper, we focus on issues faced by CS educators in online teaching sessions. We analyzed the Q&A posted by CS educators on the Computer Science Educators StackExchange (CSEd SE) forum4. The formal introduction of the forum is: “Computer Science Educators Stack Exchange is a question and answer site for those involved in the field of teaching Computer Science” [21]. CS educators facing any difficulty can post questions on this website, and the questions can be answered by other members of this website. Users can associate different tags with their questions, e.g., lesson ideas, student motivation, etc. These tags are helpful in describing and categorizing the questions, hence, making it easier for the forum’s members to search and answer the questions in their specialisations. The discussions presented on this forum provide a holistic glance of the CS educators’ views on online teaching methodology.

To the best of our knowledge, no other study has analysed the questions and answers related to ‘online’ and ‘distance learning’ tags posted on the CSEd SE webiste. We want to deeply understand the kind of issues that CS educators are facing in online teaching as well as various recommendations provided by other CS educators to overcome these issues. We identified six main challenges related to online teaching: platform recommendation, Q&A management, academic dishonesty, pair programming, and feedback mechanism. In this paper, we will discuss these issues and various recommendations provided by CS educators in detail. Throughout the paper, we used the female gender for discussion. In the context of this paper, we will consider CS educators as the users of the forum who posted questions on the CSEd SE website. This website is created for CS educators, and we believe that people posting questions on this website have some connection with CS education. Other researchers, who have used the dataset from this forum, have made a similar assumption [22]. However, this assumption has limitations which we discuss in Section 8 of this paper.

2. Related Work

In this section, we discuss work closely related to the work presented in this paper. We divided the related work into two main categories, i.e., remote teaching/learning in general and CS remote teaching/learning.

2.1. Remote Teaching/Learning in General

COVID-19 has caused a sudden shift from classroom teaching to online teaching. This sudden shift has caused lots of challenges. Several researchers analyzed challenges to online learning. Leech [10] et al. performed a survey of 831 K-12 teachers in Midwestern states and reported various challenges faced by these teachers during a shift to online teaching. Various educators reported issues such as low student engagement/response, the unfamiliarity of the students with required digital technology, and difficulty in translating the in-person teaching material to an online mode.

Ferri et al. [11] analysed data from an online discussion forum, conducted an interview, and identified several challenges with respect to online teaching. They also provide several suggestions to overcome these challenges. These challenges were mainly organized into three categories: technological challenges, social challenges, and pedagogical challenges. They report that Internet connectivity and access to workable devices to attend online lectures are the main technological challenges. Financial support from governments is needed to support families to provide them with suitable equipment for online education. Students lose interest in online teaching very easily; hence, retaining the attention of the student in online teaching is one of the big challenges. Therefore, innovative teaching methods are needed to meet this challenge. Additionally, appropriate training for using these digital devices is needed for teachers, students, and parents. The loss of human interaction between teachers and students is reported as a social challenge, and blended learning is proposed as a solution.

Guangul et al. [8] performed a survey of 50 educators. Educators responded that academic dishonesty, infrastructure, coverage of all the learning outcomes, and commitment of students toward assessment submission are the main challenges. Educators responded that to minimize academic dishonesty it would be good to prepare different questions for different students and also combine exams with projects/presentations. Having multiple assessments will give the examiner greater control to identify whether the work submitted by the students is original or not.

All the above studies used a survey-based approach to identify the challenges faced by educators/students in remote teaching.

2.2. Computer Science Remote Teaching/Learning

In this subsection, we review research that analysed the challenges faced by CS educators and students in remote teaching/learning. Crick et al. [13] analyzed the impact of COVID-19 on CS education in the UK. They conducted a large-scale survey of educators belonging to various sectors and found that CS educators felt more positive toward online education compared to educators teaching other subjects. CS educators raised concerns about teaching certain core subjects such as mathematical foundations and programming. There was concern about how more practical or collaborative topics such as robotics and group software projects can be taught online. Access to labs or specialised software at home was reported as a problem. This study only focuses on educators from the UK; hence, the results cannot be generalized to other parts of the world.

YeckehZaare et al. [23] analyzed how the pandemic affected the study of CS students. They analyzed the total number of interactions with eBooks and the number of days students used them as a measure of spacing. Their results show that students’ study was negatively impacted due to the pandemic. They found that on average, students had less interaction with eBooks and studied the eBooks for only a few days during the pandemic for an “Introduction to CS” course.

De et al. [14] conducted a large scale survey of 137 CS educators from 60 Brazilian institutes. They studied the impact of emergency education as well as the strategies used by educational institutes to combat the emergency education scenario. In their survey, many educators reported that they were facing issues in remote teaching as they did not have any prior experience. Due to this, they were facing difficulty in the adaption and creation of material suitable for remote learning. This study was focused on Brazilian universities, and the results cannot be generalized across the globe.

The above studies show that some CS educators were confident about remote teaching, whereas others struggled. Hence, more detailed studies are needed to find the actual issues that CS educators faced. Moreover, most of these studies are questionnaires based on specific groups from an institute or organization; hence, they may not be generalizable to other part of the world. The work presented in this paper complements the above studies by bringing another perspective to issues faced in online teaching. Surveys are good in identifying the challenges faced during online teaching, but often the challenges uncovered or recommendations provided are limited to the organization where the survey was conducted. POn the other hand, Q&A forums are open for all; hence they provide a wider scope to uncovering challenges and recommendations as they are not limited to a particular organization.

3. Methodology

All the data present on StackExchange websites are available to download in the form of XML files5. We downloaded all the data from the CSEd SE website on 15 August 2021. This dataset is available for researchers. There were 214 unique tags present on the website. We selected questions having “online”6 and “distance learning”7 tags relevant to this study and analyzed these questions and the associated answers.

3.1. Data Analysis

Figure 1 shows the data analysis steps. We use the process of content analysis as described by Kirppendorff [24]. This approach is used by other researchers for content analysis [22]. We use the sentence as the unit for analysis. The first author performed the coding of data and assigned four categories to it. Then in the second round, she expanded these four categories to seven categories, and in the third round finally, she created six categories. The second author then looked at the categories generated by the first author and performed independent coding. The two authors also brainstormed to see if there was any conflict in any category and identified a suitable category for such cases.

3.2. Privacy & Ethics of Internet Research

In this paper, we used data from CSEd SE website. The data on this website are publicly available and can be accessed by anyone without any login and password. Although all the data are public, we maintain the confidentiality of the users by not mentioning any name, complete quotes, or any other information that can identify participants using an Internet keyword search.

4. Dataset Details

In this section, we provide details related to the data present on the CSEd SE website. As of August 2021, 974 questions were asked on this website. Among these, 931 (95.6%) questions had at least one answer, and 352 (36.1%) questions had an accepted answer. On average, each question received 4.1 answers. There are 9273 registered users of this website. There were 5 questions consisting of the tag “online”, 9 questions having the tag “distance-learning”, and 2 questions having both tags, i.e., a total of 12 questions. These questions had an average view count of 369.3. Two users who asked questions were university professors, one was a high school teacher, one was a software engineer and details about the other were not available. Four were from the USA, two were from the UK, one was from Italy, and ibe was from Portugal. Other users had not specified their location. There were a total of 37 answers posted for these 12 questions. These questions were asked between 10 March 2020 to 17 April 2021, which is the time when various countries were going through COVID-19 lockdowns. After our initial screening of these questions, we removed three questions from the analysis. Two of these questions (QId-6903,6259) were off-topic and were not related to online or distance learning. The third question (QId-6595) did not receive any answers from the community (at the time of writing this paper). Hence, in this paper, we consider nine questions for detailed analysis.

Table 1. Statistical information about the questions analyzed: QId: question ID; score: number of upvotes a question received; answer count: number of answers a question received; comment count: number of comments a question received.

S.No	QId	Score	View Count	Answer Count	Comment Count
1	6278	4	74	3	2
2	6471	9	2296	5	16
3	6231	14	581	7	6
4	6234	5	179	4	0
5	6236	6	329	2	1
6	6242	2	110	4	13
7	6251	1	75	3	1
8	6294	4	119	3	1
9	6645	4	62	2	0
Sum		49	3825	33	40

shows the statistical details about the selected questions. These 9 questions received 49 upvotes (i.e., score), 3825 views, 33 answers, and 40 comments. Hence, these questions have quite a large audience, and analysis of these questions can provide important insights about online teaching from CS educators’ perspectives.

5. Results

In this section, we will discuss the various questions asked by CS educators and related answers/suggestions given by other educators on the CSEd SE website. Figure 2 shows the various categories obtained in this study and a brief summary of the suggestions given on CSE Ed website. All the questions related to online teaching and distance learning can be categorized as follows:

-

Topic 1: Platform Recommendation

-

Topic 2: Q&A Management;

-

Topic 3: Grading/Evaluation of Coding Assignments and Exams;

-

Topic 4: Academic Dishonesty Prevention;

-

Topic 5: Alternative to In-Person Pair Programming;

-

Topic 6: Feedback Mechanism.

In the following subsections, we will discuss each topic in detail.

Figure 2. Overview of the various categories analyzed in this study.

Figure 2. Overview of the various categories analyzed in this study.

5.1. Topic 1: Platform Recommendation (N = 7)

In this question (N = 7, N represents the number of answers, here and throughout this text), the educators were asking about tool recommendations for online classes (QID-6645, QID-6236, QID-6294). These recommendations can be categorized into two main parts: ‘platform recommendation for sharing course material’ and ‘platform recommendation for conducting online classes/meetings/group work’. The following is a detailed discussion about each category:

5.1.1. Platform Recommendation for Sharing Course Material

In this topic, the educator was interested in exploring the tools that are suitable for uploading, sharing, and organizing course material, for example, lectures, videos, slides, announcements, etc. Various options such as MSTeams, Moodle8, and a custom build website were suggested to the educator. GitHub9 was proposed to distribute and receive assignments. Additionally, Github, is also suggested for hosting the course website (that displays basic information about course) and displaying announcements related to the course. Discord10 was suggested to be used for class discussions. Google classroom11 was suggested for large classrooms to share course material. Educators stated that Google classroom has a clean and lightweight interface that is helpful. Respondents also suggested repl.it12 for sharing coding assignments, as it has a built-in feature for unit testing. Unpublished Youtube13 videos for sharing lectures were also suggested.

LMSs were not considered very helpful for short courses as they are designed for managing a school. They have features such as medical designation, school lunch designation etc., which are not required for short courses and can be overkill in such situations.

5.1.2. Platform Recommendation for Conducting Online Classes/Meetings/Group Work

Blackboard14 was suggested for online classes, meetings, etc., as it has features such as online testing, timed dissemination of resources, a messaging system, chat (both one-to-one and group), a grade book, broadcast messages for the instructor, personal contact control, and interface customization. However, some educators recommended using notebowl15 instead of blackboard, quoting that notebowl is an improved version of Blackboard. Several educators mentioned the use of Zoom for live lectures.

Discord was suggested as it has an informal feel to it that helps in maintaining the spark in online classes. It has options for sending reactions, gifts, important announcements, and code formatting for several programming languages. However, some educators mentioned that the screen cannot be shared on this platform; hence it is difficult to use in CS classes. Some educators mentioned that students also use WhatsApp, Facebook, and Telegram groups for informal meetings and chats with TAs. For the Linux system, the use of a GNU screen or tumx is suggested for online meetings.

5.2. Topic 2: Q&A Management (N = 4)

This discussion was concerned with how to help lecturers in online Q&A sessions (QId-6278). In this question, the lecturer was using Youtube live broadcast for lectures. She mentioned the problem of interruptions while giving live lectures on an online platform. The following are the four main suggestions given to her:

• To begin with, users suggested keeping her lectures short, for example, 10 min, and providing regular feedback to students through quizzes after each session.
• Second, she was advised to use Zoom, where she can divert all student questions to a chat window. She can scan the questions at the end of the lecture and can provide answers without interrupting the lecture session.
• Third, it was also suggested to follow a flipped classroom approach. Instead of creating all video lectures on her own, she can use shorter videos available online and ask students to view them before coming to a classroom. She can then use the lecture session more like a Q&A session rather than as a regular lecture session.
• Fourth, she was advised to make her course completely asynchronous. She can provide videos to the students and can create student and professor groups. Anyone can ask or answer the question in the group. This will decrease the burden on the lecturer for answering each question.
• Fifth, she can get help from a teaching assistant (TA) who can answer questions while she continues with the lecture.

5.3. Topic 3: Grading/Evaluation of Coding Assignments and Exams (N = 5)

This discussion relates to support for the educator in grading coding assignments or exams in an online environment. There was one question (QId-6471) asked in this category and five answers. Before the COVID-19 pandemic, the educator used to take hard copies of code submitted by the student and provide feedback. After moving to an online system, she was looking for efficient solutions to provide feedback to the students. There were many suggestions provided to her.

• Use of PDF Annotator Tools: She was advised to convert code to a PDF format and then use PDF readers, e.g., Acrobat Reader16,PDF Xchange viewer pro17, preview App (Mac)18, Microsoft OneNote19, and a keyboard or a mouse to provide comments. She was also advised to use smart devices such as a tablet computer with a pen that will help in drawing, scribbling, and writing on the student’s code. Tools such as speedgrader20 and Xournal21 which are specialized for grading were also recommended.
• Test Scripts and Code Review Tools: For a large class, manually checking code and providing feedback can be a tedious and a very time-consuming task. Hence, it is recommended to use more automated approaches such as automated testing tools or code review tools. These days automated test cases can be easily written in various programming languages such as Java. The educator can upload the test cases, and students can provide the results of those test cases after executing the program on their machine. Similarly, there are various code review tools available. For example, a code review tool integrated into Github22 is suggested.
• Live Code Share: Live code share can also be used to interact with the actual code base of the students and to provide them feedback on their code using tools such as Visual Studio Live Code Share23. This method can be used to give feedback at the points in the code where the students start to deviate from the actual requirement of the assignment. One educator suggested providing feedback in bullet points and conducting an online meeting with each student at the same time.
• Batch Scanner: Batch scanning is another option where the educator can grade the hard copy of transcripts and can use a batch scanning tool to convert it to a PDF to share it with the students. The educator can use a tool such as Pyzbar24 to assign QR codes to the graded sheets so that it is easier for the machine to recognize the student to whom the assignment/exam belongs.

5.4. Topic 4: Academic Dishonesty Prevention (N = 11)

In these questions (QID-6231, QId-6234), the educator was concerned about how to prevent cheating in online coding exams. There are already concerns raised about cheating in coding exams [25]. The transition to online classes during the COVID-19 pandemic has escalated this issue. The educator proposed the idea that she can create code with different inputs for each student so that it becomes difficult for the students to cheat. She asked for some recommendations about the tool(s) that can generate such code. The following is a summary of suggestions given to the educator:

• Specialized Tools: Some educators supported use of specialized tools to prevent online cheating in exams, for example, respondus lockdown browser25 and exam.net, Blackboard. Therespondus lockdown browser is a browser specifically designed to prevent online cheating. In this browser, the assessment is open on full screen. The student cannot open any other browser window, and features such as screen sharing, remote desktop, and virtual machine are disabled. Exam.net26 is another specialized tool suggested for conducting exams. This tool allows the teacher to set up the exam by typing the questions in it. They can also grant permission for tools allowed in the exam and can select security levels in the browser.
  Some educators suggested using existing software such as Moodle, Google Forms, and Microsoft Forms along with Zoom for conducting the exam. One educator suggested that if there is no specialized tool available, the educator can ask all the students to share their screens with the teacher and turn on their mics. The teacher can record the entire session so that students should be aware that if they cheat they will be noticed.
• Use of Plagiarism Detection Tools: Some educators suggested the use of plagiarism detection tools such as TurnitIn27 for detecting code or reporting similarity to identify whether the submitted work is copied or original.
• Generating a Set of Questions: Usually, educators give the same questions to all students in an exam. In this case, educators suggested using multiple questions in the exam and randomly assigning each student the questions. The easier approach suggested was to create three questions and randomly assign the question to each student. Hence, in that case, there will be less probability that the student and their friend will get the same question in the exam. Hence, it is not possible to prevent cheating in an online exam.

  -

  One educator shared their experience about using Blackboard for generating basic programming questions using the feature ‘calculated formula’. She shared that she is able to create some basic programming questions using this feature, for example, sequential assignment, logical operator, decision statement, etc. She also mentioned that Blackboard does not fully support the creation of such assignments, and she is struggling to create questions related to some areas of programming, for example, modulus operator, looping operator, etc.

  -

  Some educators also suggested using ongoing research from CUNY [26] and Croatia [27]. This ongoing research in the area of generating random programming questions indicates that the research community is already thinking in this direction. However, much work is still required in this area.

• Changing Nature of Exam: Some educators suggested changing the nature of the exam.

  -

  For example, instead of traditional exams, use group projects, assignments, oral presentations, etc., to assess the performance of the students. They say that in many studies it was found that project-based learning is more effective for most learners, especially, learners belonging to under-represented groups.

  -

  Educators suggested that group-based assessments are helpful in building skills such as teamwork, and interaction with other team members. Such skills are more useful in later stages of life (when students will join the workplace). Hence, it is more important to develop and test these skills and move to more authentic forms of assessment.

• Concept-Based Questions: One of the educators suggested not using questions that can be directly copied to an IDE but rather to go for questions that check the understanding of the given topic. For example, questions such as which algorithm is most suitable in a given scenario?
• Ethics/Honour Code Reminder: Some educators suggested reminding the student about moral values/ethics before attempting the exam. One of the educators shared the link of an experiment done by Dan on two student groups [28], where one group signed the honour code before the exam. In this experiment, it was found that the students who signed the honour code before the exam did not cheat.
• Impossible to Prevent Cheating: Some educators claim that if a student wants to cheat in an online environment, they can easily cheat, for example, using multiple screens and team viewer. There can be someone else in the room who can type the exam.

5.5. Topic 5: Alternative to In-Person Pair Programming (N = 4)

Pair programming is a method of coding where one person writes the code (director) and the other team member (navigator) sits nearby and verifies the code. Pair programming is effective and is found to be useful. However, because of the COVID-19 pandemic, pair programming became very challenging. In this question (QId-6242), the educator asked about tools that can be used to support pair programming. The following are some of the suggestions:

• Specialized IDE Plug-in: Xpairtise28 (Eclipse), Duckly29 (Intellij), AWS cloud930, Thonny (Python)31, and VS Live Share32 are suggested with voice or text channel, for example, MSTeams, Zoom, and Discord for communication between director and navigator.
• Other Tools: Zoom.us is suggested. The tools allow the person to share the screen with another team member, and the other team member can use a mouse and keyboard to edit the code.

In pair programming, there can be a concern about identifying who is doing the actual work. Hence, some educators also suggested using version control systems such as Git for code sharing and using GPG keys to identify who is performing the work.

5.6. Topic 6: Feedback Mechanism (Identifying the Problems That Students Are Facing) (N = 3)

In this question(QID-6251), the educator was working with students of aged 7–9 and 11–14 and was concerned about training the students to give good feedback. In an online environment, there is far less interaction between educator and students [29]; hence, it becomes difficult to find out what students are doing. Are they stuck at some point? Hence, in this question, the educator asked about some suggestions to train the students to provide clear feedback about their progress. Following are some of the suggestions given by the educators:

• Follow-Up Questionnaire: Give the students a questionnaire/Google form to fill in their feedback. Based on the feedback received, ask a follow-up question. For example, if a student said ‘this did not work’, the next question can be ‘what did you try?’ or ‘paste the code that did not work’.
• Distribution of Course-Related Handouts at the Beginning of the Semester: Educators suggested providing handouts to the students at the beginning of the semester. These handouts should explain to them the best practices about how to ask a question. This can help students ask good questions and share more details about their progress. However, other educators raised a concern that handouts or material given at the beginning of the course are often ignored by the students because they are overwhelmed by the information provided at the time.

6. Discussion

After the detailed analysis of questions asked by CS educators, we explored six main topics/issues that are faced by CS educators. Some of these issues apply to educators of other domains as well. In the following subsections, we discuss remote teaching issues that apply to all educators, remote teaching issues applicable to only CS educators, and the implications of this study for future research.

6.1. Remote Teaching Issues Applicable to All Educators

Topic 1 (Platform recommendation), Topic 2 (Q&A Management), topic 4 (Academic Dishonesty Prevention), and Topic 6 (Feedback Mechanism) are the issues that are applicable to all educators.

-

During the pandemic, all educators were confused about which platform to choose for online teaching, Q&A management, and feedback. Several great recommendations are uncovered in this study. For example, for the platform, the educators recommended first identifying the goal of why you would want to use the platform. Github, MSteams, Moodle, Google Classroom, etc., are suggested for sharing course material, whereas Blackboard, notebowl, Zoom, Discord, etc., are suggested for online classes/meetings. There was a trade-off regarding the use of Discord. Its informal interface can be useful in recreating the feel of a live lecture, but the inability to share a screen can make it difficult to use in some courses.

-

For Q&A management of an online class, it is suggested to have multiple lecturers (or TAs) present, one lecturing and the other answering questions. If it is not possible to have more than one lecturer (or TA), then the lecturer can choose to answer all the questions in the end, or she can use a flipped classroom-based approach where she can share the videos before the lecture, and the lecture session can be used as a Q&A session. Additionally, she can create a group/forum for the students where anyone can ask or answer questions.

-

Academic dishonesty prevention is one of the biggest challenges of online exams. Several solutions, such as the use of specialized tools (or browsers), concept-based questions, multiple questions, project-based learning, use of ethics, etc., are suggested. All these solutions are suitable and require further exploration in the context of online teaching.

-

In an online teaching environment, there is much less interaction between the student and the teacher. Hence, it becomes difficult for the teacher to find out which areas students are struggling with. In this context, the educators suggested having online forms with follow-up questions to find out details of which topics the student is having difficulty with. Additionally, the educator can give some guidance at the beginning of the course about “how to ask good questions in the course”.

6.2. Remote Teaching Issues Applicable Only to CS Educators

All the six topics/issues found in this study are applicable to CS educators. However, there are some issues that are faced only by CS educators. In this subsection, we will discuss such issues. Topic 1 (Platform recommendation), Topic 3 (Grading/Evaluation of Coding Assignments), Topic 4 (Academic Dishonesty Prevention), and Topic 5 (Alternative to In-Person Pair Programming) are the issues that are faced mainly by CS educators.

-

Finding a suitable platform for taking online classes or sharing course material is a general concern of all educators. However, we noticed that CS educators have additional needs. For example, sharing code assignments or code examples is an issue. Coding assignments may need formatting/styling requirements or support for unit testing which is not supported in many tools. Hence, CS educators should be careful when choosing a specific platform if the course is related to programming.

-

Evaluating/grading programming assignments and exams is one of the biggest challenges faced by CS educators in an online environment. Several suggestions are provided in this direction, for example, the use of test scripts, code review tools, PDF annotation tools, etc. However, it should be noted that all CS educators or students may not have the skills to handle these tools. Hence, appropriate training is required for both students and educators before using such tools.

-

Academic dishonesty prevention is a general concern as we have already discussed in the previous subsection. However, for programming assignments/exams this problem is much more severe because it is quite easy to cheat. Students can easily copy and paste the code and change variable names, etc. The recommendations provided by the educators such as the use of specialized tools, project-based learning, conceptual questions, etc., can be used by CS educators to overcome this challenge. However, as we mentioned earlier, more research is needed in this direction.

-

Pair programming is an important technique used in software development and is important if students get familiar with it at an early stage. How to simulate pair programming in an online environment is a unique challenge that is faced by CS educators. Several recommendations such as tools that allow sharing of the screen (with mouse and keyboard control) and specialized IDE are provided to help simulate pair programming.

6.3. Implications for Various Stakeholders

This study has discussed the online teaching issues faced by CS educators and various recommendations given by educators. This study has implications for the following stakeholders:

-

CS Educators: Educators often use the online tool recommended by the university. The COVID-19 pandemic started suddenly, and universities did not have time to explore multiple tools. Hence, most universities adapted their current tools for online teaching. However, the tools recommended by the universities might not match the requirements of all educators. After reading this paper, the educators will have knowledge about the various tools available for online classes, meetings, exams, etc. Hence, they may recommend the universities invest in tools that are more suitable for them.

-

Universities: This study can be helpful to universities in several ways. First, universities often adopt one tool for online teaching that is used by all the lecturers. However, this study reveals that different lecturers may have different needs, especially CS educators because they have to conduct online labs and online coding exams. Hence, different tools may be needed by the educators based on the type of subjects they are teaching (refer to Section 5). Second, universities should consider providing more resources to teachers during online teaching. Conducting Q&A during online classes can be a challenging task, and if students’ questions are not resolved appropriately, it can cause dissatisfaction among the students. Hence, universities should consider assigning TAs or more than one lecturer during Q&A sessions. Third, universities should consider providing training related to advanced features of various tools. The lecturer may explore the advanced features of the tools on their own, but it can result in lots of time wasted. Advanced training, such as how to generate a set of programming questions in blackboard, code review, test case generation, etc., can be quite beneficial to educators. Fourth, universities also need to provide appropriate hardware and software tools to educators. For example, equipment such as laptops, tablets, cameras, pentab, PDF annotators, etc, may be needed by the educators.

-

Research Community: This study is important for the research community in several ways. First, this study indicates that more studies are needed to understand the issues faced by CS educators in online teaching. As noted in Section 5, different issues are faced during classes (lectures), Q&A sessions, labs, and exams. Hence, there is a need to understand the unique issues faced during each of them and recommendations about them and to provide appropriate solutions to the educators. Second, there is a need to understand whether advanced programming concepts such as pair programming which used to happen offline can be replaced in the online setting. Does their efficacy remain the same in the online environment as well? If not, then appropriate tools need to be designed (refer to Section 5.5). Third, academic dishonesty during online lab exams is one of the major issues discussed by educators in our study (refer to Section 5.4 ). More research is needed to develop tools that can prevent cheating in online lab exams. Some suggestions for tools such as ’respondus lockdown browser’ are given. There is a need to explore the effectiveness of such tools.

6.4. Implications for Future Research

This paper has opened doors to several open and interesting research problems as discussed below:

-

Platform Selection: Traditionally, educators were reliant on whiteboards, marker pens, and PowerPoint to teach students. After the shift to online teaching, there are now several tools available to lecturers, with each tool having its strengths and weaknesses. We found that educators need help and training in using these online tools, such as Gsuite, office365, Google hangout Meets, Teams, Piazza, etc. It is important that universities carefully choose tools matching the needs of the educators. More research is needed about which tools are suitable for general classes and theory or programming courses, or do we need to consider a more formal or informal approach towards online classes.

-

Q&A in Live Lecture: Managing Q&A in live lectures is a difficult task. Hence, universities need to think of assigning multiple lecturers in the same course or providing TA support to the lecturer to help in managing Q&A in the live lectures.

-

Evaluation of Coding Exams: Online evaluation of coding exams is quite challenging. With the issue of academic dishonesty, the issue is further escalated. Hence, there is a need for more in-depth research on how to support educators in grading coding exams. If educators are using code review tools or automated test scripts, then appropriate training is required for students and educators.

-

Advance Programming Concepts: Teaching advanced programming/debugging concepts such as Pair programming can be quite challenging in an online environment. There is a need to develop tools that can make learning such advanced concepts in an online environment easier.

7. Conclusions

In this paper, we analyzed questions posted on the CSEd SE website to uncover challenges and recommendations with respect to online teaching in CS education. We analyzed Q&A having tags ‘online’ and ‘distance-learning’. We identified six main challenges related to online teaching: platform recommendation, Q&A management, academic dishonesty, pair programming, and feedback mechanism. Following are the main findings: first, we identified that different course instructors can have a different need for online teaching based on the type, of course they are teaching, for example, theory or labs. Hence, a single online teaching tool may not be sufficient for all the staff. Second, CS educators may need additional support for conducting and evaluating lab examinations such as tools to prevent academic dishonesty, facilitate automated testing, or review code. Third, we identified several recommendations in all six categories which consist of best practices and tool suggestions that can be helpful for educators and universities.

8. Limitations

• Followup Questions with the Participants: In this paper, we present an exploratory study of issues related to CS educators in online teaching. We analysed Q&A from an online website; hence, we were not able to complete any follow-up questions to the users. To overcome these shortcomings, we used the suggestions given in the literature [30] for conducting online qualitative research. We used multiple investigators or peer debriefing to create categories from the given data. This method is also used by other researchers [22] while conducting online qualitative research.
• Number of Questions Analyzed: In this paper, we consider nine questions for detailed analysis. This is a very small number to yield any conclusion about online issues faced by CS educators. However, Table 1 shows the statistical details about the selected questions. These nine questions received 49 upvotes (i.e., score), 3825 views, 33 answers, and 40 comments. Hence, these questions have quite a large audience, and analysis of these questions can provide important insights about online teaching from CS educators’ perspectives. We plan to extend this study by conducting a survey of CS educators about online teaching.
• Background of the CS Educators in This Paper: The CSEd SE website is created for CS educators. However, there are no strict rules defined about the background of the users. Anyone can ask a question or answer a question on the website. Hence, it is difficult to identify whether the person who is asking the questions is a CS educator or not. However, the quality of the questions and answers is moderated by the senior members who are experts in the domain. The low-quality questions and answers are either edited or deleted from the websites. The users of the website are given the choice of how much personal information they want to disclose on the website. For our study, we analyzed the public profiles of the users. Nine unique users asked the questions. Two users who asked these were university professors, one was a high school teacher, and one was a software engineer, and details about the other users were not available. In the future, we plan to expand this study by conducting an in-depth survey of CS educators having varied backgrounds to obtain more knowledge about online teaching issues.