Developing Prospective Teachers’ Beliefs about Digital Tools and Digital Feedback

Abstract

In classrooms today, teachers are asked to support their teaching with digital tools. For this purpose, teachers require not only technological knowledge but also corresponding beliefs about the advantages of digital tools. The development of those beliefs should already be embedded in the university education of teachers. To this end, we developed a university seminar aimed at fostering prospective teachers’ confidence in the utility of digital tools, using the digital tool STACK as an example. The seminar is based on learning mathematics with the digital tool STACK, independently designing digital tasks with said tool, and finally, reflecting on a teaching experiment with school students using STACK. To make the development of prospective teachers’ beliefs visible throughout the seminar, we worked with different qualitative methods. The results of this case study show that there are four developmental phases of prospective teachers’ beliefs which include an initial situation, a purely positive phase, a disillusionment, and a phase of differentiated beliefs. It becomes apparent that it is possible to develop prospective teachers’ beliefs about digital tools in a positive way.

1. Introduction

Increasing digitalization in our society [1] results in new requirements and standards for schools and thus correlates to a growing relevance of digital tools in mathematics education [2,3]. Digital tools can support school students in deepening their reasoning and visualization skills [4], acquiring content knowledge [5], avoiding misconceptions [6], and increasing their motivation [7,8]. Digital tools can also be used for learning mathematics in university contexts. For example, university students’ independent and active learning processes can be fostered by blended learning formats [9] and informative feedback [10]. In recent research, different systems were used to categorize digital tools independently from the specific situation in which they are used [11]. A global system of categories is provided by Hillmayr et al. [12], who divided digital tools into drill and practice programs, tutoring systems, intelligent tutoring systems, simulations, and hypermedia systems. Research has shown that the learning of mathematics can be improved by using digital tools [12] especially by providing interactive and adaptive scaffolds [6], elaborated and explanatory feedback [13], and the opportunity of pacing [14]. Intelligent tutoring systems combine these three design features [12], which is the reason why this paper focuses on intelligent tutoring systems. Such systems, which provide the opportunity for “feedback, activation of relevant knowledge, and adaption of learning content to prior student knowledge” ([12], p. 18), may have a great impact on school students’ learning. The open source digital tool STACK (System for Teaching and Assessment using Computer algebra Kernel, [15]) offers the possibility to design digital tasks, to give individualized feedback for each input; furthermore, it is possible to randomize tasks [15]. Thus, STACK could serve as a potentially promising intelligent tutoring system for learning mathematics, regardless of the specific situation in which it is used.

It has become apparent that digital tools do not unfold their potential on their own and can also have negative effects on school students’ mathematical understanding [16]. To avoid this, teachers need further training in how to implement digital tools didactically with care and to exploit their potential [12,16]. Research has shown that even prospective teachers do not seem to be adequately prepared to design and use these tools in the classroom [17]. For this reason, a main goal of university teacher training is to make prospective teachers aware of the didactically sensible use and design of digital tools.

An international comparison shows that teachers use digital tools rather sparsely [1]. Particularly in Germany, teachers use digital tools less frequently in the classroom, and also estimate the potential of digital tools to be lower than teachers in other countries [1]. One significant reason for this reluctance may be identified in the “wide variety of factors teachers referenced when taking instructional decisions, many of which could not be classified as knowledge” ([18], p. 364). For example, beliefs are one of these important factors for mathematics teachers’ professional life [19]. Skott [20] describes teachers’ beliefs as the default of their classroom practice and thus, teachers’ beliefs can be seen as a key factor in changing classroom practices. In particular, when designing lessons with digital tools, teachers’ specific beliefs about said tools seem to be crucial [21]. One factor that has a major influence on integrating digital tools into teaching is positive experiences with these in the teacher’s own school and university education [22,23]. The beliefs formed from these experiences at school and university serve as a filter for prospective teachers’ learning [24]. However, research on changing mathematics teachers’ beliefs about digital tools is, firstly, sparse [25], and secondly, has mostly yielded disappointing results that beliefs cannot be changed as desired [26]. For these reasons, it is a desideratum in mathematics education research to further investigate possible ways to influence mathematics teachers’ beliefs about digital tools in a positive way. We contribute to this desideratum by focusing on prospective mathematics teachers’ trajectories of beliefs about digital tools using the digital tool STACK as an example. In our research, we follow a case study approach that is “the dominant methodology used” in research on teaching mathematics teachers’ digital competencies ([26], p. 28). Thus, instead of investigating a bigger group of teachers, the aim of our study is to describe the existence of a development of beliefs by examining two prospective mathematics teachers’ beliefs in depth during an intervention.

2. Teachers’ Beliefs

2.1. Definition, Function, Importance

Fives and Buehl ([21], p. 471) stated that “research on teachers’ beliefs […] runs the gamut of research methodologies, theoretical perspectives, and identification of specific beliefs about any number of topics.” However, it is possible to outline specific aspects of teachers’ beliefs to locate our research approach [27].

According to Hannula [27], we regard beliefs from a psychological perspective primarily as a cognitive trait. Furthermore, we use the definition of Philipp [28], who suggests beliefs to be individual propositions that have a truth value. In addition, beliefs are understood to be organized in clusters, in which specific beliefs have different degrees of centrality [21,29]. Centrality means that central beliefs are strongly held by individuals, whereas peripheral beliefs are less strongly held [28]. Clusters of beliefs refer to “(a) self, (b) context or environment, (c) content or knowledge, (d) specific teaching practices, (e) teaching approach, and (f) students” ([21], p. 472). Beliefs about content or knowledge refer to mathematics or specific parts of mathematics [21]. Thus, beliefs about digital tools could be understood as a part of beliefs about content and knowledge. Furthermore, if digital tools are regarded as a means for learning mathematics, beliefs about digital tools could also be classified as representing beliefs about a teaching practice [21]. Finally, beliefs about self could refer to digital tools. Research on beliefs about self is often based on the construct of self-efficacy, according to Bandura [30]. In this theory, a teacher’s self-efficacy is his/her own belief about the individual “ability to plan and execute the skills necessary to produce a certain behavior” ([31], p. 591).

Beliefs are understood as a filter that shapes the way people receive information and how people act in a specific situation [32]. Thus, beliefs seem to shape what teachers learn [21] at university as prospective teachers and in their professional practice as a qualified teacher [23]. The function of beliefs as a filter is one reason why teachers’ beliefs have represented an important focus of mathematics education research in recent decades [20,28]. Furthermore, the current status of teachers’ beliefs seems to be related to their individual curricula [33], their enacted curricula, i.e., the teachers’ classroom practice [20,34] and, finally, teachers’ beliefs seem to have an impact on students’ learning [35,36]. For this reason, teachers’ beliefs about digital tools potentially influence the way students work with these tools in mathematics classrooms and how students gain knowledge and beliefs about digital tools.

2.2. Teachers’ Beliefs about Digital Tools

Using digital tools in the classroom not only requires knowledge about those tools but also corresponding beliefs about how digital tools can enhance the learning of mathematics [37]. The way digital tools are integrated in classroom practices depends crucially on teachers [22] and their beliefs [38,39]. Erens and Eichler [38] found in this context that teachers can be divided into different clusters. On the one hand, there are opponents of digital tools, who have strong doubts that they can support learning processes. On the other hand, the “technology supporters” ([38], p. 142) are positively convinced that digital tools can enhance mathematics learning. These beliefs seem to have an action-guiding effect because the teachers who have positive beliefs use digital tools more often in the classroom than the opponents [38].

Teachers’ beliefs include perceived positive and negative aspects of teaching and learning mathematics with digital tools [40]. Thurm et al. [41] developed categories for teachers’ beliefs about the advantages, disadvantages, and general issues of digital tools [41]. Beliefs that refer to advantages of the use of digital tools in the classroom may include the fact that digital tools are suitable as visualization instruments. A shift between different modes of representation (e.g., table, graph, function) can be supported through digital tools [41,42]. The significant amount of time required for the introduction of digital tools in the classroom and the resulting additional workload of the teacher is, instead, one disadvantage of the use of digital tools that a teacher may be convinced of [40,41]. However, there are also beliefs that can be classified as neutral, related to general issues; for example, beliefs about timing, such as at what point in a lesson a teacher should use digital tools [41,43]. Another dimension of teachers’ beliefs regarding digital tools is self-efficacy beliefs [44]. Thurm and Barzel [44] found that self-efficacy for designing lessons with digital tools (also for technical implementation) is an important factor when using digital tools in the classroom. As a result, within the teacher education program, there should be “a stronger focus on allowing teachers to gain specific mastery experience in implementing technology as this is the most powerful source of teachers self-efficacy” ([44], p. 58).

So far, to the best of our knowledge, we are not aware of research on beliefs about digital tools using STACK. However, beliefs about STACK may serve as an example of beliefs about intelligent tutoring systems and, more generally, beliefs about digital tools. For this reason, the aim of our study is to contribute to the research on teachers’ beliefs about digital tools by analyzing prospective teachers’ beliefs about STACK.

2.3. Development of Beliefs

Changing teachers’ beliefs as part of a development process is a crucial research topic in the field of teachers mathematics-related affect [24,45,46]. Particular emphasis is placed on three phases of teachers’ professional lives, namely, mathematics teachers’ belief growth during teacher education programs [47,48], the development of teachers’ beliefs in professional development [49,50], and the development of teachers’ beliefs as a result of their professional practice [21]. Within these three phases, Buehl and Fives [51] identified different sources for development (and thus changing beliefs), namely: formal education, teaching experiences, and self-reflection. Based on a literature review, Liljedahl et al. [46] stated that a strong impact through an intervention could result in a change of teachers’ beliefs. In this regard, research implies that even central beliefs need a strong impact for any change to be introduced, whereas newly-formed beliefs or peripheral beliefs are more likely to change or be modified [45,46].

Research into changes in mathematics teachers’ beliefs concerning digital tools is scarce. However, Thurm and Barzel [25] reported a change of teachers’ beliefs about digital tools during a professional development program concerning the use of digital tools in mathematics education. As a result of an experimental design with a treatment and a control group, the authors stated that “beliefs regarding teaching with technology developed more favorably in the experimental group” ([25], p. 1419). However, Hegedus et al. [26] reported overall disappointment regarding the results of changing teachers’ knowledge and beliefs through intervention.

3. Materials, Research Question, and Methods

3.1. The STACK Digital Assessment System

STACK (System for Teaching and Assessment using Computer algebra Kernel, [15]) is an open source digital tool with plugins available in the ILIAS (https://www.ilias.de/en/ (accessed on 3 June 2022)) and Moodle learning management systems (https://moodle.org/?lang=en (accessed on 3 June 2022)). Specific information about the STACK digital technology can be obtained by visiting the website of the project (https://www.ed.ac.uk/maths/stack (accessed on 3 June 2022)). STACK can be used to design digital mathematical tasks that provide procedural or conceptual knowledge [52] in different formats (symbolic, graphic, interactive). Thus, we refer to STACK as a digital tool that provides the possibility of designing digital STACK tasks. The digital tool STACK uses the computer algebra system Maxima [15], which enables an algebraic input that is not only matched with stored sample solutions, but also checked for mathematical properties. Task developers are able to create a potential response tree (PRT, [15]) in STACK which consists of different nodes. The potential response trees, in which the algebraic input is made, can be examined for specific mathematical property at each node, which allows individualized feedback for each user input (Figure 1).

3.2. Individualized Feedback as Potential Use of STACK

Particularly, the possibility of providing individualized feedback with STACK is crucial. However, as outlined above, the characteristics of feedback in STACK depend on those who develop digital STACK tasks and on how these developers perceive the possibilities and importance of feedback. For this reason, we will now briefly outline possible benefits and characteristics of feedback.

Feedback is considered information that focuses on aspects of performance and understanding [54]. Feedback is an effective intervention to support and optimize learning processes [10,55,56]. Furthermore, feedback can increase cognitive performance, motivation and the recipients’ willingness to make an effort [56]. However, it is important to distinguish between two different forms of feedback: “Feedback is more effective the more information it contains. Simple forms of reinforcement and punishment have low affect, while high-information feedback is most effective” ([56], p. 12). Feedback which contains information about mistakes made while processing, information about terminology used in the tasks or information about “self-regulation level” ([56], p. 12) can be defined as high-information feedback [56]. In this way, feedback can support learners individually and according to their potential. However, in the classroom, it can be assumed that feedback is undifferentiated due to the large number of students [57]. Within digital learning environments, there are numerous possibilities to provide feedback on learning processes [55,58,59], so that there is greater scope for teachers’ action. Teachers can decide whether the feedback is immediate or delayed (“time of occurrence”, [60]), whether it is textual, graphical or auditory (“way of occurrence”, [60]), which information it contains (“complexity”, [60]) and whether the feedback is directed at an individual school student or a group of school students (“target”, [60]). The individualized and differentiated feedback for each user input seems to be crucial [1], but often, even in digital learning environments, feedback tends to be simple and evaluative (categorization as right or wrong) [59].

3.3. Seminar

We developed a university seminar concept for prospective high school and vocational schoolteachers, focused on the digital tool STACK. The seminar is one of the elective modules in the teacher education program at the University of Kassel in Germany. This means that the students are asked to choose one of several seminars, one of which is this seminar addressing digital tools. For this reason, not every student in a cohort is enrolled in this seminar. The seminar was held for the first time in the winter semester 2020/2021 and was subsequently evaluated and optimized. On average, 15 prospective high school and vocational schoolteachers participated in each seminar cycle. The structure of the seminar is divided into four parts (Figure 2).

Learning: In the first part, prospective teachers themselves learn with the digital tool STACK. For this purpose, existing digital STACK tasks are provided for them to work and learn with. At this point, the prospective teachers should take on the learners’ perspective in order to experience the importance of learning mathematics with the digital tool STACK. This experience should have a strong impact to enable change and development of the prospective teachers’ beliefs [46].

Designing: In the second part, the participants change their point of view from the learner’s to the designer’s perspective, and independently design their own digital tasks with feedback in the digital tool STACK. Within this part, we address self-efficacy in technical implementation of digital STACK tasks [44]. The two parts of learning and designing represent formal education [51].

Teaching experiment: The third part includes the practical application of the digital STACK tasks which are designed by the prospective teachers. School students selected by the prospective teachers work with these tasks and comment on them, the digital format of the tasks, and the feedback given while working on the tasks. This teaching experiment is reduced in complexity, as the prospective teachers do not use their digital STACK tasks in regular lessons but in a reduced setting with only a few school students. In this way, we set out to address self-efficacy especially with regard to planning lessons [44,61] with the digital tool STACK, which can have significant effects due to reduced complexity [61].

Reflection: In the fourth part based on school students’ comments, the task, feedback, digital tool STACK, and teaching experiment are reflected on by the prospective teachers. Within the third and fourth part of the seminar, the prospective teachers take on the teachers’ perspective. Through their own teaching experience and self-reflection in particular [51], they should gain knowledge of the advantages and disadvantages of teaching and learning mathematics using the digital tool STACK [38,40].

In their final term paper, the prospective teachers repeat the second, third, and fourth steps when designing another digital STACK task with feedback, using it with school students and reflecting on the task and practical application.

3.4. Research Question

Within our study, we focus on the digital tool STACK which can be used to design digital STACK tasks, and in which individualized feedback can be considered an essential component. In this paper, we wish to investigate prospective teachers who attended a university seminar on the digital tool STACK and the development of their beliefs during the seminar. The university seminar includes the three components of learning with the digital tool STACK, designing digital STACK tasks, and reflecting upon the result of including the digital tool STACK in teaching experiments with school students. Our main research question (RQ) is as follows:

RQ: How does learning with, designing, and reflecting on one’s own digital STACK tasks in a university mathematics education seminar affect prospective teachers’ beliefs?

3.5. Participants

The seminar took place in the summer semester of 2021, the winter semester of 2021/2022, and the summer semester of 2022. In the three seminar cycles, an average of 15 participants were asked to participate in this study. An incentive for participation was offered: a fee as student workers for time spent on interviews, extended written comments, and for permission to use their final term papers for research. Furthermore, it was made clear that participation included a substantial time commitment. As a result, two prospective teachers in each seminar agreed to participate in this study. In this paper, we will focus only on the seminar in the summer semester of 2021. The data collections of the following seminar cycles (winter semester of 2021/2022 and summer semester of 2022) are not yet complete and should be used for additional research questions.

In this regard, we followed a case study approach aiming to explore the belief development of single prospective teachers and to describe this development in depth [62]. A total of 12 prospective teachers participated in this seminar cycle, with two prospective teachers who we call Maddison and Elizabeth choosing to contribute to our study (names are pseudonyms). Therefore, we will focus on the development of Maddison’s and Elizabeth’s beliefs about digital tools, using STACK as an example. Maddison and Elizabeth are two female prospective high-school teachers. Maddison is 26 years old and Elizabeth is 30 years old. Maddison began the teacher education program in 2018 and was in the 6th semester of study at the time of the seminar. Elizabeth was in the 8th semester of study. In

Table 1. General information about Maddison and Elizabeth.

Information	Maddison	Elizabeth
gender	female	female
age	26 years old	30 years old
type of school	high school	high school
semester of study	6th	8th

, we summarize the general information about Maddison and Elizabeth.

3.6. Methods

We worked with qualitative methods designed to collect beliefs and other variables of the prospective teachers (such as knowledge and motivation) during the seminar. We interviewed the prospective teachers before and after the seminar. We used these interviews as an open method for exploring a research field in which the research results are scarce so far [63]. Furthermore, we used written comments during the seminar, since writing comments or reflections during a university seminar is a common requirement for prospective teachers. The aim of using this triangulation of methods [64,65] is to obtain more detailed results and a more accurate overview of prospective teachers’ beliefs by investigating the phenomenon from different perspectives [66]. In addition, in order to highlight the development of the prospective teachers’ beliefs, we conducted a longitudinal study in the form of a pre-post-design using semi-structured interviews which took place twice in each seminar. The general design of our study is shown in Figure 3.

At the beginning of the seminar, the first interview takes place. At this point in the seminar, the prospective teachers have already been introduced to digital tasks in different formats (drop-down tasks, multiple-choice tasks, STACK tasks) to obtain an overview of different options for digital task design. After learning with the digital tool STACK within the seminar, the prospective teachers write a comment on a digital STACK task in which they list perceived advantages and disadvantages of using the digital tool STACK. After the steps of designing and reflection, the prospective teachers are also asked to write a comment on a digital STACK task. After submission of the final term paper at the end of the seminar, the final interview takes place.

The semi-structured interview guide includes different topics. The first topic refers to beliefs from the learner’s perspective. Therefore, this topic is about the prospective teachers’ school and university education, with reference to the use of digital tools therein, and perceived advantages or disadvantages of digital tools. The second and third topic refer to beliefs from the teachers’ perspective. Within the second topic, we ask for the prospective teachers’ self-efficacy beliefs while designing digital tools, respectively digital STACK tasks. The third topic concerns school and teaching in particular. We there ask for beliefs about digital tools, particularly STACK, from the teachers’ perspective and self-efficacy to design lessons with the digital tool STACK. In this context, we also collect beliefs about feedback in the school context. The pre- and post-interviews contain the same topics. In the pre-interview, the guiding questions outlined above do not refer to STACK specifically, but to digital tools in general. Since the prospective teachers have not yet worked intensively with the digital tool STACK at this point, they should express their beliefs about those digital tools they are already familiar with. In the post-interview, the guiding questions relate to digital tools in general, but also in particular to the digital tool STACK, which they have worked with intensively during the seminar.

Furthermore, we examine prospective teachers’ written comments on different digital STACK tasks. In the written comments, the prospective teachers should name any perceived advantages and disadvantages concerning the digital tool STACK. By using this method, we also wish to collect beliefs about the digital tool STACK and digital feedback.

We analyzed the data using qualitative content analysis [67]. For this purpose, the expressed beliefs are divided into different categories. For example, a quote in the interview (or the written comment) such as “The implementation of a digital STACK task can take a lot of time” was allocated to the deductive category “beliefs about the time expenditure” and the inductive subcategory “beliefs about the time expenditure while designing digital STACK tasks”. In the same way, we allocated inductive and deductive categories to both the interviews and the written comments. After the first data analysis, different categories of beliefs could be found in the interviews and the written comments. Our analyses revealed that some of the categories can be seen in both oral and written statements. Thus, the triangulation including the interviews and written comments consisted of the identification of categories addressed in both documents. For this reason, we combine the different methods in order to trace a development of the prospective teachers’ beliefs in this paper. In doing so, we restrict ourselves to strands of argumentation that belong to the different categories of beliefs found in the documents resulting from both methods. These categories are:

• Prospective teachers’ beliefs about burden and relief for a teacher created by using digital tools such as STACK.
• Prospective teachers’ beliefs about possibilities and limitations of STACK.
• Prospective teachers’ beliefs about digital feedback.

Additionally, various subcategories within these general categories were considered. Most of the subcategories in the category “burden and relief of a teacher created by using digital tools such as STACK” emerged inductively. An example of an inductive category is the teachers’ overview of students’ learning level, which can be coded either as an advantage (better overview through digital tools) or as a disadvantage (worse overview). The time expenditure category [40] is deductive, although two subcategories (designing and usage) were inductively identified. Subcategories of prospective teachers’ beliefs about possibilities and limits of STACK were found inductively (e.g., anticipation of typical mistakes) and deductively (adaptation and multiple use of existing digital tools [68]). The subcategories of prospective teachers’ beliefs about digital feedback are deductive (e.g., complexity of feedback or time of occurrence [60]). Finally, self-efficacy was a deductive category with the sub-categories of designing digital tools being, respectively, digital STACK tasks on the one hand and implementing the digital tool STACK in lessons on the other hand [44].

4. Results

In this section, we set out to trace the development of Maddison’s and Elizabeth’s beliefs about burden and relief created for a teacher by using digital tools such as STACK, beliefs about possibilities and limitations of STACK, and beliefs about digital feedback.

4.1. Beliefs about Burden and Relief Created for a Teacher by Using Digital Tools

Within the pre-interview, Maddison recognizes some advantages of using digital tools. For example, using explainer videos would lead to better time management within the classroom as teachers “cannot re-explain a topic again from scratch” to a school student due to time constraints. On the other hand, Maddison identifies disadvantages of using digital tools in the classroom:

“If I spend a long time graphically animating something to explain a tiny little fact that I could have explained in two minutes on the blackboard and all school students would understand just as well, that’s a negative effect I have.”

In this context, Maddison says that “digitalization can be a waste of time”. She obviously feels that the additional time required to design digital tools is a huge burden for teachers. We can conclude that Maddison’s self-efficacy on designing digital tools is rather low. At this point, Maddison emphasizes fairly general beliefs about the advantages of using (pre-existing) digital tools and about the disadvantages of designing digital tools on her own. This is the reason why we call this phase the “superficial initial situation”.

Elizabeth recognizes an advantage for teachers of using digital tools, namely, better time management for teachers. Alongside this advantage, Elizabeth states a disadvantage of digital tools. If digital materials are flawed, school students can build incorrect knowledge without the teacher noticing. However, this disadvantage is not specifically a disadvantage of digital tools. Elizabeth’s self-efficacy in designing digital tools, like Maddison’s, is rather low. She would like to learn about designing digital tools so as not to be “overwhelmed” like teachers in pandemic distance learning. We can see that Elizabeth’s beliefs are rather unspecific and thus she is in the “superficial initial situation”.

After learning with the digital tool STACK within the seminar, Maddison focuses on an advantage, specifically a better overview of the school students’ learning level:

“An advantage for the teacher is that he/she can check at a glance which school student has successfully completed the digital STACK task and which errors have occurred, which takes much longer within the analogue format.”

During the learning phase, Maddison and Elizabeth came to know the digital tool STACK from the learners’ perspective. This seemed to have a strong impact on Maddison’s beliefs because at this point, she only mentions benefits that a teacher enjoys by using the digital tool STACK. This is the reason why we call this phase the “high phase”.

Like Maddison, Elizabeth recognizes only advantages for the teacher and thus, is in the “high phase”. She focuses on school students’ self-directed learning enabled by the digital tool STACK, which eases a teacher’s burden and leads to better time management.

After designing a digital STACK task independently, using it with school students and reflecting on the teaching experiment, Maddison recognizes advantages as well as disadvantages:

“The implementation of a digital STACK task can take a lot of time, but afterwards it can be used as often as desired without any further effort.”

She focuses on the disadvantage of additional time required for designing digital STACK tasks but also identifies an advantage, which is their unrestricted use. In contrast to the “high phase”, a certain disillusionment happens in this step. Maddison realizes an additional burden for teachers after she has designed her first designed digital STACK task. For this reason, this stage is called “(partly) disillusionment”.

In contrast to Maddison, Elizabeth is not in the “(partly) disillusionment” phase. Elizabeth only mentions advantages of the digital tool STACK, which relieves teachers’ workload. For this, Elizabeth highlights the time-saving aspect of digital feedback. We can see that Elizabeth is still in the “high phase”.

In the post-interview, Maddison names several advantages of the digital tool STACK which facilitates the teacher’s task:

“Especially in the pandemic, when school students are at home and I as a teacher still want to see their progress, I then set a digital STACK task […] so that I am aware of school students’ performance, even though I cannot see them personally.”

Compared to the “high phase”, in which Maddison simply mentioned the advantage of a better overview, she is now able to relate this relief for the teacher to a specific circumstance, the pandemic distance learning context.

Elizabeth also names several advantages of the digital tool STACK for the teacher. For example, she addresses the aspect that a teacher in analogue lessons does not have the time to take a detailed look at each school student’s work. Thus, the digital tool STACK has the advantage of giving the teacher a more precise overview of a school student’s learning level but also of pointing out individual misconceptions through individualized digital feedback. We can see that Elizabeth’s beliefs have evolved, not only in her mentioning of the advantage of better time management but also by linking other advantages (better overview, pointing out individual misconceptions).

One disadvantage that Maddison mentions is the additional time required for designing:

“Digital STACK tasks are very time-consuming to design. And when you have spent an hour making your well-planned potential response tree—you have done everything nicely—and at the end something doesn’t work, then of course you’re annoyed.”

In contrast to the “(partly) disillusionment” phase, where Maddison generally said that technical implementation takes a lot of time, here she addresses a specific circumstance: the implementation of the potential response tree.

Elizabeth emphasizes the fact that digital STACK tasks have to be well-planned. School students should not be able to “trick” digital STACK tasks or to acquire incorrect knowledge through mistakes within the digital feedback. We can see that Elizabeth’s beliefs about errors in the process of designing digital STACK tasks have changed, as she is now able to identify which errors can occur specially in the context of the digital tool STACK.

Another aspect we can see within Maddison’s and Elizabeth’s statements is self-efficacy in designing digital STACK tasks. Maddison says that getting to know the digital tool STACK intensively “took the hurdle away” from her, giving her more freedom to use this digital tool later in her own lessons. Elizabeth now sees herself able to design well-planned digital STACK tasks and to consider specific errors that can occur while designing. Thus, we can see that Maddison’s and Elizabeth’s self-efficacy in designing digital STACK tasks has increased in contrast to the pre-interview.

Self-efficacy for using the digital tool STACK in classroom has also developed. Maddison is able to reason in a differentiated manner about which time use of the digital tool STACK would be appropriate:

“You have to pay close attention to where it makes sense to use STACK and where not.”

She says that, in her opinion, the digital tool STACK is not suitable as an introduction to a new topic, as in this situation, the “social aspect of learning” plays a major role. Elizabeth also considers the appropriate time to use the digital tool STACK which she perceives more as an opportunity for school students to practice. Elizabeth sees herself as being able to use STACK in class and explain to school students how to work with said tool.

We can see that Maddison’s and Elizabeth’s beliefs about burden and relief created for a teacher by digital tools have become differentiated primarily through designing, teaching experiences, and reflection of the second independently designed digital STACK task. This is the reason why we call this stage the “phase of differentiated beliefs”.

In

Table 2. Maddison’s and Elizabeth’s beliefs about burden and relief created for a teacher by using digital tools such as STACK during the semester.

Points of Time	Maddison	Elizabeth
pre-interview	time management (+) time required for designing (−) self-efficacy in designing (−)	time management (+) incorrect digital tools (−) self-efficacy in designing (−)
learning	overview (+)	time management (+)
designing and reflecting	time required for designing (−) unrestricted use (+)	time management (+) digital feedback (+)
post-interview	overview (+) time required for designing (−) self-efficacy in designing (+) self-efficacy in using (+)	time management (+), overview (+), individualized feedback (+) incorrect digital tools (−) self-efficacy in designing (+) self-efficacy in using (+)

, we summarize the beliefs that both prospective teachers express concerning burden and relief created for a teacher by using digital tools during the semester.

4.2. Beliefs about Possibilites and Limitations of STACK

Within the beliefs about possibilities and limitations of STACK, we can also identify the developmental phases that have occurred in the beliefs about burden and relief created for the teacher by using digital tools. In the beginning, both Maddison and Elizabeth are in the “superficial initial phase”, where Maddison recognizes some possibilities but also limitations of the digital tool STACK:

“The digital tool STACK offers a lot but is also limited in terms of functionality.”

She cites the possibility of STACK for providing feedback as an advantage. As a disadvantage, she states that individualized feedback is only given regarding typical errors:

“I cannot give individual feedback related exactly to this one school student, but I always have to […] think in advance what could go wrong while working on the task.”

At this point, Maddison has rather unspecific beliefs about the possibilities and limitations of STACK, as she only focuses on the function of providing feedback.

Elizabeth does not recognize any advantage of the digital tool STACK, but only the disadvantage that there is a lack of human interaction in the virtual working process. However, this fact is not a specific disadvantage of the digital tool STACK but a disadvantage of working with exclusively digital tools, as in the pandemic distance learning era.

Having learned with the digital tool STACK themselves, Maddison and Elizabeth enter the “high phase”. Maddison highlights the possibility of using STACK to randomize tasks:

“One advantage of a STACK task is that it can be randomized, […] so that many tasks are generated.”

In contrast to the “superficial initial situation”, in which she recognized a limitation, she now perceives no disadvantage of the digital tool STACK.

Elizabeth also identifies the possibility of randomization as an advantage of the digital tool STACK. In addition, Elizabeth addresses the advantage that once a digital STACK task is designed, it can be used in different classes without any further effort.

After designing and reflecting on their own digital STACK tasks, Maddison and Elizabeth can be placed in the “(partly) disillusionment” stage. Maddison identifies an advantage of STACK, namely its unrestricted use. She also focuses on the disadvantage that school students only receive individualized feedback to anticipated errors that have been included to the potential response tree:

“It is almost impossible to predict all possible student errors.”

Compared to the “high phase”, Maddison now recognizes a limitation which can be seen as a disadvantage of the digital tool STACK. This seems to be caused by the act of designing, using, and reflecting on her first digital STACK task.

Elizabeth also recognizes the disadvantage of not being able to anticipate all possible errors and further addresses the disadvantage that school students cannot ask anybody questions about their processing or the received feedback because of the virtual working process. Elizabeth’s belief about the lack of human interaction seems to have changed at this point, as she is now able to relate this fact directly to a specific situation.

The “(partly) disillusionment” is followed by the “phase of differentiated beliefs” in which Maddison and Elizabeth find themselves during the post-interview. Maddison focuses again on the possibility of randomizing tasks with STACK:

“An advantage is that a school student has infinite possibilities to repeat a task with other numerical values […] and that he/she can simply practice for himself/herself as a learner at home.”

In comparison to the “high phase” in which Maddison merely mentioned this advantage of STACK, she now relates the possibility to randomize tasks to a specific situation.

Elizabeth also elaborates on the advantage of the randomization function in the post-interview. She says that randomized STACK tasks are especially useful for building procedural knowledge and consolidating algorithms. Like Maddison, Elizabeth is now able to relate the previously named advantage to a specific situation and topic.

In addition to this advantage of the digital tool STACK, Maddison addresses a disadvantage, the lack of human interaction in the working process:

“I think for someone who is not quite as motivated, this personal contact is extremely important. A computer cannot fulfill the social component of learning at all.”

In this context, Maddison says that a teacher cannot be replaced by the digital tool STACK. Previously, Maddison did not address this aspect. Through designing, using, and reflecting on her second digital task, Maddison appears to have developed new beliefs about the possibilities and limitations of STACK.

Elizabeth also focuses on this disadvantage. She differentiates between students’ acceptance of feedback from a person in contrast to a computer. At this point, we can see that Maddison and Elizabeth recognize some advantages of STACK. Nevertheless, they emphasize the important role of the teacher in the school students’ learning process.

In

Table 3. Maddison’s and Elizabeth’s beliefs about possibilities and limitations of STACK during the semester.

Points of Time	Maddison	Elizabeth
pre-interview	feedback (+) typical mistakes (−)	lack of human interaction (−)
learning	randomization (+)	randomization (+) unrestricted use (+)
designing and reflecting	unrestricted use (+) typical mistakes (−)	typical mistakes (−) lack of human interaction (−)
post-interview	randomization (+) lack of human interaction (−)	randomization (+) lack of human interaction (−)

, we summarize the beliefs that both prospective teachers express about possibilities and limitations of the digital tool STACK during the semester.

4.3. Beliefs about Digital Feedback

The development of Maddison’s and Elizabeth’s beliefs about digital feedback resemble the developmental phases described above. At the beginning, Maddison and Elizabeth are in the “superficial initial phase”. Maddison and Elizabeth cite the time of occurrence of digital feedback. Maddison says:

“What is nice about the digital tool STACK is that school students get immediate feedback.”

We see that Maddison and Elizabeth generally hold beliefs about the advantages of digital feedback. However, these beliefs are rather unspecific as they only refer to the immediacy of feedback.

After learning with the digital tool STACK, Maddison and Elizabeth are in the “high phase”. Maddison focuses on the aspect of immediacy again and combines this aspect with the components of feedback:

“A school student does not only receive immediate information about whether the solution is correct, but also about the exact location of the error.”

She goes on to say that feedback should not provide the sample solution, but rather hints on how to improve the solution so that “a school student really deals with the task again.” Her beliefs seem to have evolved through learning with digital tasks.

Elizabeth also addresses immediacy and considers the components of digital feedback. In addition, Elizabeth declares greater fairness as another advantage, since the identification of a school student by handwriting is not possible.

In the next step of designing and reflecting on their own digital STACK task, Maddison and Elizabeth are in the “(partly) disillusionment” stage. Again, Maddison addresses the time of occurrence:

“One advantage of digital feedback is that it is given immediately after a solution is submitted. In contrast, analogue feedback takes time.”

Furthermore, she says that digital feedback is given to each student simultaneously, which is not possible in analogue lessons. Maddison’s beliefs about advantages of immediate feedback seem to have evolved, since she compares digital with analogue feedback for the first time. However, in contrast to this advantage, Maddison cites a disadvantage of digital feedback. Due to a perceived limitation of the digital tool STACK, Maddison expresses that feedback is always limited to previous anticipated, typical mistakes. Using this disadvantage, Maddison explains that digital feedback consequently is not always fair:

“If a school student has made a small mistake in solving the task which was not anticipated by the teacher, the solution will then probably be graded with 0 points, even though it is almost correct.”

In contrast to the “superficial initial situation” and the “high phase”, Maddison sees negative aspects of digital feedback for the first time.

Elizabeth also recognizes some advantages of digital feedback (immediacy and simultaneity) using comparisons to analogue lessons. In contrast to Maddison, she addresses the disadvantage of the absence of a teacher which means that school students cannot ask questions about their received digital feedback.

The last developmental phase is the “phase of differentiated beliefs” in which Maddison and Elizabeth find themselves in the post-interview. Maddison highlights the advantage of immediacy of digital feedback by comparing digital and analogue feedback:

“A school student can keep trying until the solution is correct without waiting for a teacher to finally arrive and say, “you got it wrong.””

Elizabeth also refers to immediate feedback and simultaneity again. She describes the aim of immediate and simultaneous feedback for the first time, which is that school students can work independently from the teacher which allows greater self-directed learning. Maddison’s and Elizabeth’s beliefs about immediacy and simultaneity as advantages of digital feedback seem to have consolidated.

Moreover, Maddison describes the aspect of individualization within digital feedback. She focuses on the components and aims of individualized digital feedback:

“Maybe I am missing some information at some point […] and then I get that information. Individual feedback is extremely important for this.”

The aim of individualized feedback is to inform students about certain missing facts and thereby increase motivation because a school student is “informed what to do next”. Another aspect Maddison addresses in this context is the length of individualized digital feedback:

“A text which is too long […] can also be quite exhausting for the learner. I think there is an optimum point, at which certain individuality is most conducive.”

We can see that Maddison’s beliefs about the individualization of digital feedback have developed. These statements about individualization cannot be categorized as advantageous or otherwise. Maddison now describes in a differentiated manner the aims of individualized feedback.

Elizabeth refers to the disadvantage of the absence of a human person. However, this time, she emphasizes mainly the communication among the school students. She also discusses the extent to which students are more accepting of feedback from a human than from a computer. We can see that Elizabeth’s beliefs about the absence of a human in giving feedback have become differentiated.

In

Table 4. Maddison’s and Elizabeth’s beliefs about digital feedback during the semester.

Points of Time	Maddison	Elizabeth
pre-interview	immediacy (+)	immediacy (+)
learning	immediacy (+) components (+)	immediacy (+) components (+) fairness (+)
designing and reflecting	immediacy (+) simultaneity (+) fairness (−)	immediacy (+) simultaneity (+) absence of a human (−)
post-interview	immediacy (+) individualization (o)	immediacy (+) simultaneity (+) absence of a human (−)

, we summarize the beliefs that both prospective teachers expressed about digital feedback during the semester.

5. Discussion

In this article, we set out to present how prospective teachers’ beliefs about digital tools using STACK as an example can be changed through a university seminar in the teacher education program.

5.1. Summary and Discussion

The results showed that within the prospective teachers’ beliefs about burden and relief created for a teacher by using digital tools such as STACK, beliefs about possibilities and limitations of the digital tool STACK and beliefs about digital feedback, a development has emerged. Although the development has slight, individual deviations, it generally includes the following four phases:

• The first phase, which can be temporally located right at the beginning of the seminar, is called the “superficial initial situation”. Although both prospective teachers expressed beliefs about digital tools in this phase, the beliefs were poorly reflected, unspecific, and superficial since the prospective teachers were not able to relate their beliefs to specific examples or situations. However, these beliefs serve as a default of the further development of the prospective teachers’ beliefs during the seminar. In their study, Liljedahl et al. [69] found that prospective teachers’ default beliefs “come from their lived experiences as mathematics students” ([69], p. 26). In relation to the present study, this means that the prospective teachers have only little and unreflective experiences with digital tools in their previous teacher education program, hence their beliefs are correspondingly unreflective and superficial.
• The second phase is called “high phase” and occurs directly after learning with the digital tool STACK. In this phase, the seminar showed a strong impact on the existing default beliefs of both prospective teachers (cf. also [46]) as only advantages of the digital tool STACK were perceived. This result could already be shown by a meta-analysis [70], which found that “technology-assisted, internet-based, and computer-assisted instruction or learning were related to positive belief appraisals” [70]. Marchisio et al. [71] were also able to show that digitally assisted alternative learning methods positively affect beliefs of university students. In another study, it was found that offering “alternative experiences as learners of mathematics” ([69], p. 28) can cause a belief change. It can be hypothesized that learning mathematics with digital tools is an alternative learning method that provides alternative experiences for the prospective teachers in this study. This seems to have a strong impact on their beliefs, thus enabled belief change. Furthermore, the prospective teachers developed new beliefs about the digital tool STACK as a result of formal teaching [21]. Both the existing default beliefs and the newly developed beliefs were associated by the prospective teachers with specific features of the digital tool STACK or situations. Thus, the default beliefs were enhanced with reference to content and concreteness.
• The third phase, which arises directly after the steps of designing and reflection, is called “(partly) disillusionment”. In this phase, the practical experience with school students seems to have a strong impact on both prospective teachers’ beliefs. This influence of practical experiences was often reported [21,24,72,73,74]. In our sample, it is apparent that the practical experience initially has a negative effect on prospective teachers’ beliefs who emphasize mainly disadvantages of the digital tool STACK that emerged through their teaching experience. An undesired development of beliefs due to a practice shock was also reported by Erens and Eichler [42]. Practice shock arises from confrontation with inconsistencies between prospective teachers’ expectations of teaching and reality [75]. One reason for this practice shock seems to be the lack of preparation within teacher education programs, so prospective teachers struggle to implement their theoretical knowledge into practice [75]. Therefore, an effective measure to avoid practice shock seems to be the inclusion of controlled practice experiences in the teacher education program [70], in which a combination of theoretical knowledge and practical experiences is provided [76]. Even though in the presented study the practical experiences were controlled, as the teaching experiment took place in a complexity-reduced setting and were combined with the previously acquired theoretical knowledge, a practice shock still occurred whereby the prospective teachers expressed negative beliefs about the digital tool STACK. However, the beliefs about the disadvantages of said tool were related to concrete examples or situations and were thus reflected beliefs.
• The fourth phase is called the “phase of differentiated beliefs” and can be temporally located at the end of the seminar after the final term paper. In this phase, the prospective teachers found a balance between advantages of the digital tool STACK expressed in the “high phase” and disadvantages that the prospective teachers emphasized directly after their teaching experiments in the “(partly) disillusionment” stage. The prospective teachers further elaborated their beliefs about advantages and disadvantages referring to situations that cause advantages and disadvantages. With regard to the phase of “(partly) disillusionment”, which arises directly after the first teaching experiment, we can hypothesize that the prospective teachers recovered from the initial practice shock by conducting a new teaching experiment. Thus, our results are consistent with findings of other studies that found that multiple practical experiences are “considered essential for ensuring that students in teacher preparation programs ‘are extraordinarily well prepared’” ([70], p. 21). It can be further assumed that in addition to teaching experiences, self-reflection [51] leads to the differentiation of previously-held beliefs. Moreover, Levin and Wadmany [77] reported a shift in teachers’ beliefs from simple to complex during a long-lasting professional development program. Although the intervention in this study was not as long as the aforementioned professional development program, the beliefs of the two prospective teachers showed the same development from simple default beliefs to complex and differentiated beliefs.

Indeed, previous research has already revealed individual elements that seem to have an impact on changes in prospective teachers’ beliefs. For example, alternative learning methods [69,71], teaching experiments that combine prospective teachers’ theoretical and practical knowledge [51,76], as well as subsequent self-reflection [51], appear to have an impact on changing prospective teachers’ beliefs. However, previous research has considered the impact on beliefs for each of these elements separately. To the best of our knowledge, the combination of these elements within one university seminar has not yet been investigated. As a result, the developmental phases and their temporal sequence identified in this study have not previously been reported in research.

5.2. Implications

These results showed that capturing beliefs related to the phases in which the digital tool STACK was introduced to prospective teachers was vital. This may also lead to the re-interpretation of existing results. For example, Hegedus et al. [26] reported disappointment about the change in teachers’ beliefs in intervention studies that focused on improving teachers’ knowledge and beliefs about digital tools. In our study, we would have gained the same disappointing results if we had restricted the data collection to the time directly after the first teaching experiments, thereby collecting only the beliefs within the “(partly) disillusionment” stage.

It can be speculated that the four phases of the development of prospective teachers’ beliefs would emerge also for an intervention concerning other digital tools for mathematics teaching. Thus, beliefs about STACK may serve as an example of beliefs about digital tools, as many of the expressed beliefs about STACK can be transferred to other digital mathematical tools. It could be further hypothesized that the four developmental phases are general phases when dealing with aspects of mathematics education through the three steps of the seminar. As we can see, the learning phase, the practical experiences during designing and teaching experiments, and the self-reflection stage have a strong influence on prospective teachers’ beliefs. Thus, it could be speculated that this strong impact of these three steps could also be transferred to other aspects of mathematics education. However, these hypotheses need to be tested in further research.

Moreover, we can conclude that parts of the prospective teachers’ beliefs were elaborated during the seminar, whereas other beliefs were developed ex novo. It can be assumed that the newly developed beliefs are not central beliefs [45,46] and thus may be more vulnerable than other beliefs. By contrast, the beliefs that the prospective teachers addressed in each of the four points of data collection may be understood as their central beliefs. These central beliefs changed in some nuances but have neither disappeared nor emerged during the semester. For example, Elizabeth referred consistently to the better time management as an advantage of the digital tool STACK. However, the way she describes this better time management develops during the seminar, from a simple to a more complex elaboration. Thus, the different measurements during the seminar and the triangulation of two methods led to an insight into the prospective teachers’ central and peripheral beliefs about digital tools such as STACK.

Self-efficacy also appears to increase through designing and teaching experiments [44,61] and seems to have an impact on differentiating prospective teachers’ beliefs. This result was expected. However, this result is of importance since research showed that self-efficacy was found to be a main affective predictor for academic success [31].

5.3. Limitations

Despite the new insights, our research has some limitations. First, we described the results of a case study [62] as a main methodological approach in the field of research on mathematics teachers’ digital competencies [26] that is based on two cases. These two cases served to examine the development of prospective teachers’ beliefs. Based on this exploration, we have been able to identify changes in beliefs and derive developmental phases. To support the hypotheses that these phases exist, the beliefs of other prospective teachers should be analyzed. In addition, the phases of development of prospective teachers’ beliefs could be further refined and differentiated by analyzing further prospective teachers’ beliefs, for example, by attributing detailed characteristics to the phases. For our specific project, the analysis of further data could also potentially explain more precisely the relation between the structure of the seminar and the developmental phases. In order to develop a resulting model from these findings, it would be important to replicate the results in a further study focusing on the development of prospective teachers’ beliefs about digital tools, such as STACK, in an intervention.

A second limitation is that the digital tool STACK was the only digital tool used within the intervention. This leads to two possible scenarios: on the one hand, prospective teachers’ beliefs about digital tools in general may differ from their beliefs about the digital tool STACK. On the other hand, it is conceivable that the expressed beliefs about digital tools in general refer to the digital tool STACK and thus, are not strictly separable.

A third limitation refers to the different methods that we used to analyze the development of prospective teachers’ beliefs. It is, of course, possible that prospective teachers are motivated to a different extent concerning the expression of beliefs in an interview or a written comment. To avoid this limitation, we attempted to restrict our analysis to those strands of argumentation that the prospective teachers addressed in both interviews and both written comments. Nevertheless, a future study with different measurements on the basis of the same method could challenge our assumption that it was possible to describe the development of teachers’ beliefs in a valid way.

6. Conclusions

In summary, this study showed that the development of prospective teachers’ beliefs about digital tools such as STACK is not straightforward but includes phases of purely positive beliefs, disillusionment, and reflected and differentiated beliefs. These findings suggest that it is entirely possible to develop prospective mathematics teachers’ beliefs about digital tools in a positive way, in terms of referring to advantages of digital tools and elaborating differentiated beliefs. Furthermore, these results imply that it is mandatory to develop prospective teachers’ beliefs about digital tools over a longer period of time. Short interventions may only lead to a “high phase”, while earlier interruption may result in only the “(partly) disillusionment” stage. Only the comprehensive teaching and data collection approach in the presented research revealed these findings. Further studies could reveal the existence of these developmental phases in relation to other digital tools, using the same data collection approach as this study.