Gender Equality Impact Drivers Revisited: Assessing Institutional Capacity in Research and Higher Education Institutions

Round 1

Reviewer 1 Report

 

This is very important research examining the relationship between gender equality and institutional change in Highered. The Impact Driver model under the EU context is very informative in assessing institutional capacity. This study definitely has merits for publication. However, some of parts of the study need to be fixed to make the authors' model more understandable for the readers in general.

 

It seems to me that the authors should provide some discussion as to how we can apply this model to the greater picture. For example,  I would like to see how their model could explain institutional capacity and gender equality in different parts of the world, such as the United States, the Middle East, and the like.

 

Include a table before reviewing the literature that includes the following two information:

1) Source 

2) Briefly explain the earlier findings in 2 or 3 sentences at most.

This table is important for the reader who wants to see the summary of the reviewed works before delving into the details.

 

 

Figures 4 and 5 are not visible enough and look weird (just Excel print screens). Can you make it look more scholarly and bigger? I cannot even read what is written there. You can divide the figure into certain parts. It seems to me you can delete figure 5 and only include the hendecagon (institutionalization of impact drivers) as a separate figure. But it is your call at the end of the day to make it more readable and visible. I would like to see more understandable and, of course, readable figures in the revised version.

 

Please hypothetically mention the scale of the numbers from 0 to 1 in your hendecagon figure shown in figure 5. Then, you can create or mention certain scenarios focusing on explaining some random numbers in your scale. For example, explain the meaning of .2, .5, .7, and so on.  

 

Include a paragraph or two under the section “4.1. Considerations on the institutional use of the tool” that specifically focus on how the future research can use the “impact driver model.” Mention the directions for qualitative and quantitative (or even case study) methods and how your model can be utilized in these methodological approaches.

 

Given the contribution to the literature, I do believe that this is a great study and has potential for publication in Social Sciences. With that said, I decided to accept this work after minor revision.

 

 

Author Response

Dear editors and reviewers,

Thanks a lot for your careful reading and precious comments on our article.

Please find below our answers to Reviewer 1 comments.

Please note, too, that we have revised the order and indicators in the Impact Driver model, reflecting further the feedback received from the pilot.

Point 1: It seems to me that the authors should provide some discussion as to how we can apply this model to the greater picture. For example, I would like to see how their model could explain institutional capacity and gender equality in different parts of the world, such as the United States, the Middle East, and the like.

Response 1: Addressed in the final section of the article under recommendations for further research

Point 2: Include a table before reviewing the literature that includes the following two information:

1) Source 

2) Briefly explain the earlier findings in 2 or 3 sentences at most.

This table is important for the reader who wants to see the summary of the reviewed works before delving into the details.

Response 2: table included

Point 3: Figures 4 and 5 are not visible enough and look weird (just Excel print screens). Can you make it look more scholarly and bigger? I cannot even read what is written there. You can divide the figure into certain parts. It seems to me you can delete figure 5 and only include the hendecagon (institutionalization of impact drivers) as a separate figure. But it is your call at the end of the day to make it more readable and visible. I would like to see more understandable and, of course, readable figures in the revised version.

Response 3: The Excel print screens are intended as an illustration only; they depict the model used in the pilot. It has been revised after the pilot and will be published as such.

Point 4: Please hypothetically mention the scale of the numbers from 0 to 1 in your hendecagon figure shown in figure 5. Then, you can create or mention certain scenarios focusing on explaining some random numbers in your scale. For example, explain the meaning of .2, .5, .7, and so on.  

Response 4: An actual anonymised bar chart included in the paper with a brief comment on the stage of institutionalisation.

Point 5: Include a paragraph or two under the section “4.1. Considerations on the institutional use of the tool” that specifically focus on how the future research can use the “impact driver model.” Mention the directions for qualitative and quantitative (or even case study) methods and how your model can be utilized in these methodological approaches.

Response 5: Addressed in the final section of the article under recommendations for further research

Reviewer 2 Report

 

This article reports on the setting up of a model and a tool for the assessment of gender equality measures in research and higher education institutions.

The introduction and the discussion of the litterature are clear. The description of the model-building process is very interesting. It is however sometimes a little difficult to follow. I have made a number of suggestions for clarifications on the manuscript. They are all fairly minor. For instance, the impact drivers are not given explicitly at the beginning (they appear later in the paper) and their choice could be commented. The Actor mobilization model needs some more explanation (see questions on the manuscript) concerning the stages and the way an institution moves from one to the next. For example, is a transformational agent a single person or a group?

Globally, the model is clear and consistent and the tool seems well adapted. It would have been interesting to give more details on the feedback from the institutions that were testing it – did they find it useful, was the tool difficult to use…? Some interesting remarks are made in the paper, for example concerning the importance of tacit knowledge of the institution, but perhaps more details could be given.

I consider that the paper is solid and can be published with minor corrections.

 

 

Comments for author File: Comments.pdf

Author Response

Dear editors and reviewers,
Thanks a lot for your careful reading and precious comments on our article.
Please find below our answers to Reviewer 2 comments.

Please note, too, that we have revised the order and indicators in the Impact Driver model, reflecting further the feedback received from the pilot.

Point 1: The introduction and the discussion of the litterature are clear. The description of the model-building process is very interesting. It is however sometimes a little difficult to follow.

Response 1: We have included a table to improve the understanding of the process.

Point 2: I have made a number of suggestions for clarifications on the manuscript. They are all fairly minor. For instance, the impact drivers are not given explicitly at the beginning (they appear later in the paper) and their choice could be commented.

Response 2: A table reviewing the evolution of the model has been inserted. The table provides the list of impact drivers and how the list evolved during the process of building the model.

Point 3: The Actor mobilization model needs some more explanation (see questions on the manuscript) concerning the stages and the way an institution moves from one to the next. For example, is a transformational agent a single person or a group?

Response 3: Clarifications are added in the text

Point 4: Globally, the model is clear and consistent and the tool seems well adapted. It would have been interesting to give more details on the feedback from the institutions that were testing it – did they find it useful, was the tool difficult to use…?

Response 4: The information on the feedback is reflected implicitly in the text where the revisions done to the model are explained.

Point 5: Some interesting remarks are made in the paper, for example concerning the importance of tacit knowledge of the institution, but perhaps more details could be given.

Response 5: We consider this is out of scope for this paper.

In addition, we have integrated and answered  the points made by the reviewer in the article itself.

Reviewer 3 Report

please see attached

Comments for author File: Comments.pdf

Author Response

Dear editors and Reviewer 3,

Thank you for your careful reading and extensive comments on our article.

Please find below our response.

Please note, too, that we have revised the order and indicators in the Impact Driver model, reflecting further the feedback received from the pilot.

 

Point 1: Chapter 2 is titled “History and evolution of the Impact Driver model, with a theoretical framework”. Reading that heading of chapter 2 (and the title of the paper), one would assume a proper state-of-the-art section with a more detailed history, evolution and theoretical framework of more that just 2 impact driver models.

Response 1: We deleted 'with a theoretical framework' from that title and replaced with 'History of the development of the Impact Driver model'

Point 2:

Reading that heading of chapter 2 (and the title of the paper), one would assume a proper state-of-the-art section with a more detailed history, evolution and theoretical framework of more that just 2 impact driver models. And detailed explanation, why these 2 models had been chosen for developing the “new Impact Driver model”, is missing. Without such an explanation one might assume that the authors have developed these models and that they are just citing their own work – without reflection of other models, theoretical frameworks, research, methodology etc.

Response 2: We provided an explanation in the introduction that the ID model responds to the concern that evaluation of social change is a complex undertaking, requires a longer-term perspective than is usually possible and that focusing on processes is a good practice alternative, with references added. Furthermore, we add a statement on the M&E approaches in the structural change projects, including exchanges among M&E experts of different projects, which identified issues at stake such as the 'comparability' of results, the time needed before results become apparent and (in-)coherence of M&E approaches. We also explain that the two projects we focus on in this article followed an approach that responds to these concerns and allows comparison across institutions.

Point 3: Then, under chapter 2.3, it seams that these 2 models have been “integrated”. Does that mean, a new / third (?) model has been developed? Or is it even a fourth model that has been developed?

Response 3: We have added an overview table to clarify the evolution of the model.

Point 4: Why are the authors only referring to these 2 EU-funded projects? There are many more projects dealing with structural change in research organisations. A detailed explaining for the selection of these two projects is missing. (Again, one might assume that the authors only cite their own work – without reference to the body of literature.)

Response 4: See response to Point 2 above.

Point 5:  Consider revising the heading of chapter 2.3

Response 5: Given the explanations provided, we prefer not to revise the heading of section 2.3.

Point 6: Chapter 3, then, provides the “Description of the Impact Drivers model” - referring to the two before-mentioned EU-funded projects (= chapter 2.3). So, the impression is that the “new model” is built on the monitoring and evaluation approach/experience of 2 EU-funded projects. These approaches/experiences are hardly explained, whereas two other models – without explanation why – are in detail described and visualised under 2.1 and 2.2.

Response 6: With the overview table added to explain the evolution of the model, we maintain that the models are sufficiently presented in a balanced manner.

Point 7: Definition of “indicators” is not sufficient. • Table 2: Heading “indicator” is wrong; many of the items listed below in the table are description of phenomena, or concepts – and not precise indicators that really allow to measure both – quantitative and qualitative conditions and circumstances in organisations

Response 7: The indicators are qualitative indicators, which are generally used in policy evaluation (including institutions such as the UN or the ILO). See for example here, or here or here. 

Point 8:

• Add more explanation after the sentence (line 60-62). This sentence is more an introduction, and one waits for more details. Please provide some more of the variety in Europe. Or you consider that the explanations in the previous sentences are “enough” – than delete the respective sentence in line 60-62.

Response 8: We have slightly modified the sentences. We consider that further expanding on the variety in Europe exceeds the scope of this article.

Point 9:

• Paragraph starting with line 66: There are many more than just these 2 M&E approaches that have been applied. As the paragraph is formulated, one might assume that there are “only these 2 approaches” existing. I recommend rephrasing the paragraph and explain why you as authors are “just” referring to these 2 approaches? Justify your choice for these 2 models.

Response 9: Clarification added

Point 10:

• Line 87: what do you mean with “previous work”? Both models? Or just one? Or a synthesis? Please explain, be more specific.

Response 10: Revised to clarify and also the added table should provide the desired clarification

Point 11:

• Line 267, line 285 (and elsewhere): revise the wording “nothing is in place yet” – that is too colloquial. In each organisation at least “something” is in place. Name what you mean in terms of gender equality.

Response 11: We use the phrasing ‘nothing is in place yet’ in the guidance document that accompanies the tool to ensure understanding and accessibility. The assessment ‘nothing is in place yet’ is to be considered for each individual indicator and not across the board. As to what is in place “in terms of gender equality”, the individual rubrics define what it is that should be in place in that particular rubric.

Point 12:

• Section between line 319 and 328: Include information on how to assess? Which methods are applied? Survey? Interviews? Other?

Response 12: Information added on the approach used during the pilot. The guidance document for the use of the tool is not prescriptive in terms of methods to be used to gather the required information. A column is foreseen in the table to provide the source(s) of the information.

Point 13: 

• Figure 4 and 5: revise, both of the are not readable

Response 13: As we have responded before, in our feedback to reviewers 1 and 2, these visuals are mere illustrations of the tool which is an excel table.

Point 14:

• Line 344-347: include a – readable – figure of excel spreadsheet 2

Response 14: We have decided not to include a visual of spreadsheet 2 as we consider there is no added value in such a visual.

Point 15:

• Include a section with some insights dealing with the question: How to work with the ID tool.

Response: See Response 12

Point 16: General: Check the references carefully, e.g., SWG GRI 2020 (line 52) is missing in the “references”

Response 16: Checked and corrected

Point 17: When citing always provide full first names – that is only the case in reference 12 and 17.

Response 17: Unified

Point 18: When citing the work done within EU-funded projects, please name the authors. It is never a project (e.g., reference 4 “CASPER project”, reference 10) that writes a report or policy recommendation, but always people – with first and last names.

Response 18: Different projects take different approaches, especially when presenting policy advice. Both CASPER and GENDERACTION use collective authorship for policy advice delivered, meaning that the advice represents the opinions of the whole consortium. The Standing Working Group on Gender in Research and Innovation (SWG GRI), while appointing rapporteurs and task forces or sub-groups on particular tasks, also has collective ownership of its reports and advice as these undergo approval process by the Member States and Associated Countries. For these reasons, the authorship for the collective works listed will not be changed.

 

 

Point 19: Minor recommendations

Response 19: Thank you, integrated

Point 20: Minor typos

Response 20: Revised except for line 37 as there should be no space between the sign and the figure.