‘Let’s Teach as We Preach’: The Design of a Professional Development Initiative to Support Teacher Educators’ Responsiveness to Diversity

Round 1

Reviewer 1 Report

 

I concur with you that developing effective PDI’s is important work. The model you have developed is thoughtful and sound. I appreciate both the aspiration to co-create such approaches and the recognition that such an approach is not always feasible. Likewise, you note ideas and realities throughout the article, inviting others to consider what would be necessary to embrace and enact in their own context. Your title is apt and serves as both description and injunction. Thank you for developing a model that could be adapted across contexts.

There are a few points where the English is not standard (word choice construction) which will, I suspect, be caught by the copyeditors. There are no major quality concerns.

Author Response

General remarks of the authors:

• Please note that sentences referring to the original manuscript are presented in italic.
• We are fully aware that the current study does not (yet) involve an impact evaluation of the professional development initiative (PDI) as presented in the manuscript. Even though this is certainly part of our future endeavours, it remains our deliberate intent for this manuscript to solely focus on the design. This exclusive focus on the design of the PDI was also made clear in the submitted abstract to the editors of this special issue of Education Science. It is because the editors responded positive to this proposal that we pursued the writing of this kind of manuscript and that we now will stick to this concept. Again, notwithstanding our next manuscript will focus on the impact evaluation of the PDI.

Reviewer 1

Remark 1
I concur with you that developing effective PDI’s is important work. The model you have developed is thoughtful and sound. I appreciate both the aspiration to co-create such approaches and the recognition that such an approach is not always feasible. Likewise, you note ideas and realities throughout the article, inviting others to consider what would be necessary to embrace and enact in their own context. Your title is apt and serves as both description and injunction. Thank you for developing a model that could be adapted across contexts.

Answer from the authors
We want to thank the reviewer for these kind words and acknowledgements. It really strengthens our belief that our model and the focus on the design of a PDI are of great value in education.

Remark 2
There are a few points where the English is not standard (word choice construction) which will, I suspect, be caught by the copyeditors. There are no major quality concerns.

Answer from the authors
We thank the reviewer for pointing this out. In reworking the manuscript, we took this comment into consideration to better the English where alterations were made based on other reviewers’ comments.

Reviewer 2 Report

 

Thank you for giving me the opportunity to review the manuscript titled "Let's Teach as We Preach."

 

The topic is in line with the topic of the Special Issue and it should be considered for publication. However, I think the article needs more work before publication and major revisions are required.

 

I truly hope my comments are welcome by the authors. My effort is directed towards helping them reinforce their paper and work towards a stronger justification of the relevance of their work. Academia and schools are very much in need of these kinds of initiatives. 

 

Before I proceed with my comments, I want to acknowledge that I am not an expert in the specific field presented in this article. Therefore, my comments should be considered in conjunction with those from scholars who have a deeper understanding of the subject matter and can provide more detailed feedback. 

 

It is also worth mentioning that my background does provide certain advantages for this review. Being unfamiliar with certain concepts and methods, I can offer insights on how useful this article might be for other scholars like myself who are interested in inclusive education. 

 

I would also like to mention that I have experience in developing systematic pedagogical interventions to address diversity and measuring their impact. However, I find this article difficult to read in its current form. I am uncertain whether I would be able to utilize the authors' theoretical framework and pedagogical approach. It would be beneficial if the authors made efforts to clarify the intervention by using a step-by-step approach, making it more reproducible.

 

Some specific comments:

 

1)    There are some aspects of the paper that are hard to follow. There is some incoherence between the theory and pedagogical methodology used. For example, the authors cite critical pedagogy at the beginning of the work and emphasize the importance of addressing structural inequalities; however, I don’t see this reflected in their conceptual framework and the pedagogical intervention. I think there is an issue of epistemology here that the authors have not addressed. Their model's implicit epistemology seems to be rooted in a different approach, which still addresses diversity and inclusion, but appears to be more coherent with methodological individualism or other frameworks. See, for instance, Table 3. Both the content focus and the pedagogical methods do not reflect structural inequalities and other aspects as cited at the beginning of the paper. The pedagogical methods also seem to refer to "deep learning" theory. This makes it difficult to support the idea that this model can produce a change in attitudes and behaviors of the participants towards more inclusive practices in the classroom.

2)    Some paragraphs need more work in terms of language. I had to read the paper more than once. There are several repetitions, and the argument is not always easy to follow. The text needs to be tightened up, and some sections need more work. For example, on page 1, the authors mention 5 clusters and then 3 clusters in the same paragraph. In the same section, on page 2, the authors repeat "creating an inclusive environment" (Cluster 1, under Cluster 2) This makes the reading confusing. There are other examples like this throughout the text. I would ask the authors to read the text again and make sure to address these problems throughout the text.  

3)    The creation of the education intervention is not supported with an assessment of the impact. The authors are aware of this problem and mention this (e.g. in the conclusion). However, the reason for presenting only its development and implementation needs to be further justified. 

4)    I also have concerns regarding the possible justification. If they have no evidence of how their intervention is experienced by participants and whether it is able to promote the intended outcomes, including the change of attitudes and practices of the targeted groups, it remains an open question.

5)    This is a problem because it makes their work difficult to be reproduced. Why should people reproduce something that we don't know is having the expected outcomes? There is a large literature that poses this issue both in educational science in general and in Transformative learning theory in particular.Also, research in psychology on intervention to reduce prejudice show that some of our interventions can be counterproductive, and even create a backlash among those participants who tend to have high levels of prejudices. 

6)    In the conclusion, the authors consider that presenting the development and implementation of the intervention adds to the literature despite not addressing its impact. However, I am very hesitant about this.

7)    Following this point, I am thinking that maybe a way to address this problem is to further clarify the methodology, and specifically the participatory method to develop the PDI, focusing on what insights have the authors used that can tell us that this educational intervention is relevant to produce change and can be easily reproduced? 

 

8)    A stronger explaination and systematization of the methodology concerning the development of the PDImay be a way to go. This could strengthen the paper and perhaps offer a justification for not assessing the impact. However, I am not sure whether this is a problem that can be addressed. This depends on whether their designing included a systematic approach or not. The authors claim it did, and this seems to be the case. However, in the article, this needs to be clarified.

9) I am not sure how the first part of the title “Let’s teach how we preach” is related to the article. 

 

10) All the participants were white, and most were women. Maybe something should be said also about how the composition of these groups affected the intervention. How could this intervention reproduced with different groups composition? 

11) Finally, another way to improve the article is to give us insights on the impact of the intervention during the implementaion.  How were participants reacting? what were they discussing and sharing? Although we dont know how the PDI impacted their lives, it would be very valuable to know if the PDI opened up space for deeper reflection, learning, transformation in the implementation phase. In order to create space for this, the authors could reduce other sections. Two possible sections that could be shorten to create for more empirical data are: 2. Methods and 3 Results (some sections could be reduced and/or combined) and 4. Discusssion. 

 

 

There are some repetitions and the text and argument are not always easy to follow. See my comments to the authors. 

Author Response

General remarks of the authors:

• Please note that sentences referring to the original manuscript are presented in italic.
• We are fully aware that the current study does not (yet) involve an impact evaluation of the professional development initiative (PDI) as presented in the manuscript. Even though this is certainly part of our future endeavours, it remains our deliberate intent for this manuscript to solely focus on the design. This exclusive focus on the design of the PDI was also made clear in the submitted abstract to the editors of this special issue of Education Science. It is because the editors responded positive to this proposal that we pursued the writing of this kind of manuscript and that we now will stick to this concept. Again, notwithstanding our next manuscript will focus on the impact evaluation of the PDI.

Reviewer 2

General remark
The topic is in line with the topic of the Special Issue and it should be considered for publication. However, I think the article needs more work before publication and major revisions are required. I truly hope my comments are welcome by the authors. My effort is directed towards helping them reinforce their paper and work towards a stronger justification of the relevance of their work. Academia and schools are very much in need of these kinds of initiatives. I would also like to mention that I have experience in developing systematic pedagogical interventions to address diversity and measuring their impact. However, I find this article difficult to read in its current form. I am uncertain whether I would be able to utilize the authors' theoretical framework and pedagogical approach.

Answer from the authors
We acknowledge the great effort put into this review. It really does feel constructive and appreciative of our work. Therefore, we want to express our gratitude. Additionally, we appreciate the expression of modesty by the reviewer. Reading the arguments for recommending a major revision, we can easily understand the reviewers’ line of reasoning and agree that the manuscript can be improved on a couple of points (see below for concrete adjustments).

 

Remark 1
There are some aspects of the paper that are hard to follow. There is some incoherence between the theory and pedagogical methodology used. For example, the authors cite critical pedagogy at the beginning of the work and emphasize the importance of addressing structural inequalities; however, I don’t see this reflected in their conceptual framework and the pedagogical intervention. I think there is an issue of epistemology here that the authors have not addressed. Their model's implicit epistemology seems to be rooted in a different approach, which still addresses diversity and inclusion, but appears to be more coherent with methodological individualism or other frameworks. See, for instance, Table 3. Both the content focus and the pedagogical methods do not reflect structural inequalities and other aspects as cited at the beginning of the paper. The pedagogical methods also seem to refer to "deep learning" theory. This makes it difficult to support the idea that this model can produce a change in attitudes and behaviors of the participants towards more inclusive practices in the classroom.

Answer from the authors
This is a very interesting comment which led to an insightful discussion in the team. Thank you for that. We were not aware that such an incoherence could be experienced by reading the article. For us, it had become so self-evident that every action towards addressing structural inequalities starts from individual actions; sometimes directly via these actions (e.g., being part of governance boards on all levels of policy) and sometimes indirectly by educating the next generation of teachers from a diversity-responsive point of view (e.g., modelling, challenging their frames of reference..). This is why the framework focuses on individual executable practices and the intervention on increasing these practices via challenging teacher educators’ own frames of reference. It is our reading from critical pedagogy that raising critical consciousness is inherently a form of addressing structural inequalities, since it leads to a challenging of the current educational system that does not (yet) serves each student. However, we now see that this might not be how all researchers understand critical pedagogy. Consequently, some clarifications were made throughout the beginning of the text to make sure the theory and methodology are perceived as more coherent. For example:

• (p.2) Yet, we believe teacher educators could potentially impact systems and structural inequalities even more.
• (p.2) Inevitably, these teacher educators’ individual practices - like advocating for responsiveness to diversity in education and modelling diversity-responsive practices to the future generation of teachers - can influence the direction of the educational system as a whole and, as such, tackle structural inequalities over time.
• (p.3) A first conclusion: while teacher educators feel responsible and willing to create qualitative learning environments for all and a more socially just world (i.e., responsiveness to diversity), (…)
• (p.4) This last finding ties in with earlier research on teacher educators, stating that some teacher educators have managed to develop a critical habit of mind, called an inquiry as stance, allowing them to challenge the status quo in education [36], and thus addressing structural inequalities (in)directly where possible.

Later in the text we refrained from adding such statements, because then, every time the concept of ‘responsiveness to diversity’ is used, the individual and structural are both implied.

Remark 2 and 12
Some paragraphs need more work in terms of language. I had to read the paper more than once. There are several repetitions, and the argument is not always easy to follow. The text needs to be tightened up, and some sections need more work. For example, on page 1, the authors mention 5 clusters and then 3 clusters in the same paragraph. In the same section, on page 2, the authors repeat "creating an inclusive environment" (Cluster 1, under Cluster 2) This makes the reading confusing. There are other examples like this throughout the text. I would ask the authors to read the text again and make sure to address these problems throughout the text.
12) There are some repetitions and the text and argument are not always easy to follow. See my comments to the authors.

Answer from the authors
Thank you for this comment. It is always a challenge to describe the framework in a concise and understandable manner, while also describing it in a different way than in former articles. We now reordered the numbering of the different clusters of the framework and avoided using a listing within a listing (cfr. 3 clusters of 5 clusters) in order to make it easier to read:

• (p.2) Based on concrete practices found in the literature, the framework distinguishes five main clusters of practices that any teacher educator can implement regardless of specific teaching or other assignments. Some of these practices occur in the context of their teaching assignment or when teacher educators interact with student teachers.

We also clarified how the conditional cluster of creating safe learning environments is connected to the first two clusters. As such, it is now more clear, why the first cluster is once again named after the second cluster:

• (p.3) Furthermore, as a precondition to both challenging student teachers’ frames of reference and the first cluster of creating inclusive learning environments, the literature pointed towards practices about creating safe learning environments.

The introduction of the result section was also rewritten to make it clearer that the results are descriptions of the phases of the participatory design process and to avoid repetition:

• (p.7) Within the scope of this article, the results of the first two phases of the participatory design process are presented. First, we summarise the information gathered via the data collection to identify the particularities in both institutional contexts, as well as both participant groups’ needs. Second, we introduce the design of the PDI and describe the overall and context-tailored design choices that were made. Deliberately leaving out the impact evaluation of the PDI in this article, allows us to go into more detail about the design itself and to create a more unique article for the design literature. Nevertheless, the impact evaluation will be conducted and reported on to finish the design process.

As with regard to the repetitions, we see how this could annoy the reader. Prior to the submitting manuscript, we had quite some discussions about the need to reintroduce concepts and tried to find a middle ground. Given this remark, we decided to delete the following repetitions:

• (p.5) The consultation of the literature, as discussed in the introduction of this article, facilitates distinguishing objectives and general design principles for a PDI to foster teacher educators’ diversity-responsive practices.
• (p. 12) We have argued how the overall design accommodate for the specific contexts, as well as prior-defined objectives: (1) foster an overall critical stance by permanently challenging teacher educators’ frames of reference; (2) raise awareness about diversity-responsive practices by providing a framework to analyse and acknowledge current practices; and (3) increase diversity-responsive practices by facilitating concrete inspiration [17].

Remark 3-6 and 11
3) The creation of the education intervention is not supported with an assessment of the impact. The authors are aware of this problem and mention this (e.g. in the conclusion). However, the reason for presenting only its development and implementation needs to be further justified.
4) I also have concerns regarding the possible justification. If they have no evidence of how their intervention is experienced by participants and whether it is able to promote the intended outcomes, including the change of attitudes and practices of the targeted groups, it remains an open question.
5) This is a problem because it makes their work difficult to be reproduced. Why should people reproduce something that we don't know is having the expected outcomes? There is a large literature that poses this issue both in educational science in general and in Transformative learning theory in particular. Also, research in psychology on intervention to reduce prejudice show that some of our interventions can be counterproductive, and even create a backlash among those participants who tend to have high levels of prejudices.
6) In the conclusion, the authors consider that presenting the development and implementation of the intervention adds to the literature despite not addressing its impact. However, I am very hesitant about this.
11) Finally, another way to improve the article is to give us insights on the impact of the intervention during the implementation.  How were participants reacting? what were they discussing and sharing? Although we dont know how the PDI impacted their lives, it would be very valuable to know if the PDI opened up space for deeper reflection, learning, transformation in the implementation phase. In order to create space for this, the authors could reduce other sections. Two possible sections that could be shorten to create for more empirical data are: 2. Methods and 3 Results (some sections could be reduced and/or combined) and 4. Discussion.

Answer from the authors
We share the concern of the reviewer that an impact evaluation must be conducted. However, we don’t see this as a loss for this particular article. As argued in the introduction (see general remarks of the authors) this was a deliberate choice on our part, which was deemed worthwhile by the editors of the special issue. Many design articles already mainly focus on impact descriptions, only briefly describing the interventions themselves. As such, we see great value for the literature to focus for once on the design itself. As such, without wanting to offend the reviewer, as a research team we have decided to stick to our initial plan without reworking the manuscript to an impact evaluation.

Furthermore, we want to stress that we do use evidence-informed design principles and make design choices catered to the context and needs of the participants, so it is with great probability that the PDI will be meaningful. We assume it will accommodate the three goals defined at the beginning of the article, but this is indeed no certainty without the upcoming assessment. We believe this will be published in the next year. To make future readers of the current article more aware of our position we incorporated it more explicitly in the introduction of the result section:

• (p.7) Within the scope of this article, the results of the first two phases of the participatory design process are presented. First, we summarise the information gathered via the data collection to identify the particularities in both institutional contexts, as well as both participant groups’ needs. Second, we introduce the design of the PDI and describe the overall and context-tailored design choices that were made. Deliberately leaving out the impact evaluation of the PDI in this article, allows us to go into more detail about the design itself and to create a more unique article for the design literature. Nevertheless, the impact evaluation will still be conducted and reported on to finish the design process.

We also reframed the introduction of the conclusion section addressing the lack of evaluation, to stress that we did build on evidence-informed design principles:

• (p.12) Since we build on evidence-informed design principles that have been supported by the literature, we assume a high probability that the PDI will be experienced as meaningful by the participants. Nonetheless, we acknowledge that to complete the design process, the PDI has yet to be implemented and evaluated.

Remark 7-8
7) Following this point, I am thinking that maybe a way to address this problem is to further clarify the methodology, and specifically the participatory method to develop the PDI, focusing on what insights have the authors used that can tell us that this educational intervention is relevant to produce change and can be easily reproduced?

8) A stronger explanation and systematization of the methodology concerning the development of the PDI may be a way to go. This could strengthen the paper and perhaps offer a justification for not assessing the impact. However, I am not sure whether this is a problem that can be addressed. This depends on whether their designing included a systematic approach or not. The authors claim it did, and this seems to be the case. However, in the article, this needs to be clarified.

Answer from the authors
This is an important very important remark for which we are grateful. The core methodology of this manuscript is the design process and in this case we followed a participatory design process. This is never a step-by-step process, but rather described in big phases. The first to phases are described in-depth in the manuscript (cfr. Identification and design). However, reading this comments we understand it might be insufficiently clear that these phases are ‘the systematic approach’. Consequently, we tried to address this issue in the new introduction to the result section as well:

• (p.7) Within the scope of this article, the results of the first two phases of the participatory design process are presented. First, we summarise the information gathered via the data collection to identify the particularities in both institutional contexts, as well as both participant groups’ needs. Second, we introduce the design of the PDI and describe the overall and context-tailored design choices that were made. Deliberately leaving out the impact evaluation of the PDI in this article, allows us to go into more detail about the design itself and to create a more unique article for the design literature. Nevertheless, the impact evaluation will be conducted and reported on to finish the design process.

It might feel counterintuitive, but the reproducibility of this process is not in the recreation of exactly the same intervention, but rather in engaging in the identification of the institutional contexts and participants groups’ needs to make an intervention context-specific. That’s why we argue in the discussion part: Second, we did not set out to create a blueprint that can be applied in any context or with any group of teacher educators. Rather, we suggest that careful contextualisation is needed if sustainable change in teacher educators’ professional development is desired.

Additionally, our conviction that this PDI leads to change is in fact rooted in the close contextualisation, combined with the use of evidence-informed design principles from the literature. Suggestions for the PDI were (in)directly made by the participants and teacher educators with prior experience in the topic. We hope that reemphasising these arguments help the reviewer understand our idea of the systematisation of the design process. It is quite common at our Department of Educational Sciences, so we do believe the wider field also acknowledges this idea.

Remark 9
I am not sure how the first part of the title “Let’s teach how we preach” is related to the article.

Answer from the authors
‘Teach as you preach’ is a common used adagium in the context of teacher educators, since modelling is a inevitable part of their profession. The PDI aims to foster their responsiveness to diversity so they will actually ‘teach more’ as they and the policy texts of teacher education programs often already ‘preach’ (cfr. section 1.1 and 3.1).

Remark 10
All the participants were white, and most were women. Maybe something should be said also about how the composition of these groups affected the intervention. How could this intervention reproduced with different groups composition?

Answer from the authors
Thank you for this suggestion. Just to clarify: we did not assume the participants were all white, but we did conclude that they presented as white. We think this is an important nuance, because we already had a prior experience in which people with migration background were assumed as white and felt overlooked. Nest to this clarification, a fair point is made which we thought was implied by this statement in the manuscript:

• For instance, the current PDI is tailored to participants who have a decent amount of prior knowledge and already feel responsible to invest in responsiveness to diversity. As this is not the case for all teacher educators, alterations would be needed for other starting positions.

However, it might indeed be worthwhile to make it more explicit, so we added the next sentence:

Similarly, both groups of participants presented as predominantly white and female, probably leading to different needs than groups with a different composition.

Reviewer 3 Report

A high quality paper reporting on a much needed area of research.

 

Referencing needs to be checked throughout.

Author Response

General remarks of the authors:

• Please note that sentences referring to the original manuscript are presented in italic.
• We are fully aware that the current study does not (yet) involve an impact evaluation of the professional development initiative (PDI) as presented in the manuscript. Even though this is certainly part of our future endeavours, it remains our deliberate intent for this manuscript to solely focus on the design. This exclusive focus on the design of the PDI was also made clear in the submitted abstract to the editors of this special issue of Education Science. It is because the editors responded positive to this proposal that we pursued the writing of this kind of manuscript and that we now will stick to this concept. Again, notwithstanding our next manuscript will focus on the impact evaluation of the PDI.

Reviewer 3

Remark
A high quality paper reporting on a much needed area of research. Referencing needs to be checked throughout.

Answer from the authors
We want to thank the reviewer to stress the importance of this work. We hope it might inspire others to further such research as well. Furthermore, we have checked the referencing to make sure it still aligns with the author instructions (https://www.mdpi.com/journal/education/instructions#references).This seemed to be the case.

Round 2

Reviewer 2 Report

The authors have made an effort to answer most of my concerns, but not all of them. Their effort is really appreciated, nonetheless . 

As I wrote in my first round of review, I am not too familiar with this specific literature, so some of my concerns may not be relevant for the scholarship the authors are referring to. My effort was directed to express some doubts based on an external reader's reaction to the content; a reader that is less familiar with the field. I hope the message came across clearly, as I do believe in collegial work and constructive feedback. 

I think the Editors are better placed to make a jugment concerning the changes that the authors have made based on my comments. So, if they decide to publish in its current form, I will accept their decision.