Technical Efficiency of Mexico’s Public Health System in the Delivery of Obstetric Care, during 2012–2018

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

I consider that the authors approached an interesting subject.

Nevertheless, I would add some suggestions.

Firstly, I think they should better define the inclusion and exclusion criteria for their study, as their current ones seem vague.

Secondly, since the main aim of this paper is to ´evaluate the technical efficiency of Mexico's public health system in the delivery of obstetric care from 2012 to 2018´, I consider the references numbers 30, 33, 36 to be inadequate. I would recommend the authors to focus better on the main goal when selecting the informative sources.

Furthermore, in my view, the sentence ´Globally and in Mexico, there is a misconception that public sector institutions are inefficient simply because they are public. ´ is not supported by enough literature data in this paper.

Lastly, I have noticed that the ´Conclusion´ section is missing, being somewhat blended with the ´Discussion` one. The authors might better highlight their conclusions.

Comments on the Quality of English Language

I would suggest the authors to revise their English language in this study. To avoid words repetition, they might vary the structure and length of their paragraphs. For example, I would shorten phrases and use connectors to improve the clarity of the paper´s content. Moreover, using synonyms could, as well, be helpful.

Author Response

I consider that the authors approached an interesting subject.

Thank you very much.

Nevertheless, I would add some suggestions.

Firstly, I think they should better define the inclusion and exclusion criteria for their study, as their current ones seem vague.

Thank you for your observation. A new section was added to specify the inclusion and exclusion criteria. (lines 125-135).

Secondly, since the main aim of this paper is to ´evaluate the technical efficiency of Mexico's public health system in the delivery of obstetric care from 2012 to 2018´, I consider the references numbers 30, 33, 36 to be inadequate. I would recommend the authors to focus better on the main goal when selecting the informative sources.

References number 30, 33, and 36 were removed.

Furthermore, in my view, the sentence ´Globally and in Mexico, there is a misconception that public sector institutions are inefficient simply because they are public. ´ is not supported by enough literature data in this paper.

Two additional references were included to provide a global context (line 101), and two other references describing the specific situation of technical efficiency in Mexico (line 104) were added.

Lastly, I have noticed that the ´Conclusion´ section is missing, being somewhat blended with the ´Discussion` one. The authors might better highlight their conclusions.

Done. A conclusions section was added (lines -384-405)

Comments on the Quality of English Language

I would suggest the authors to revise their English language in this study. To avoid words repetition, they might vary the structure and length of their paragraphs. For example, I would shorten phrases and use connectors to improve the clarity of the paper´s content. Moreover, using synonyms could, as well, be helpful.

The text was carefully revised to improve syntax, style, and clarity

 

Reviewer 2 Report

Comments and Suggestions for Authors

This paper addresses an important issue for healthcare systems – whether the allocation of resources within a system is efficient. The authors use an innovative approach to address the questions at hand. Additionally, the discussion is robust. I have several comments to help strengthen the manuscript and aid in the interpretation of findings and connection to information healthcare delivery, healthcare practice, and/or policy.

 

More information about the institutions is needed for readers not familiar with the Mexican healthcare system. This information should include some measures of whether the obstetrics patient population differed in terms of complexity or needs, as well as the number of physicians, delivery rooms, and hospitals.

 

More detail is needed about the data sources and variables of interest. This could be reported in a Table 1.

 

A more detailed description of how efficiency of each DMU and average technical efficiency were calculated and whether time effects were taken into account would aid in interpreting the data. I am not clear on how these calculations were made. For example, how was DMU weight was determined? What are the limitations of using state-level GDP as the only measure of socioeconomic status? How is the sophistication of healthcare resources taken into account? 

 

In addition to the figures, data tables would aid in interpreting the findings.

 

The assumption of homogeneity in the quality of outputs is an extremely strong assumption. How would taking into account quality of care affect the results? In the present model, the goal is resource minimization, ignoring quality of care and differences in severity of illness and/or at-risk pregnancies. It is likely that there are meaningful differences across DMUs and states that should be taken into account. Given the world-wide focus on maternal and infant mortality, measures of quality are needed. 

 

More interpretation of what a TE of 0.82 is needed for readers not familiar with the methodologic approach. Is the maximum TE only meaningful relative to the comparison groups, or does it have an interpretation from an absolute level? Similarly, more narrative about how to interpret Figure 2 is needed. 

Author Response

This paper addresses an important issue for healthcare systems – whether the allocation of resources within a system is efficient. The authors use an innovative approach to address the questions at hand. Additionally, the discussion is robust. I have several comments to help strengthen the manuscript and aid in the interpretation of findings and connection to information healthcare delivery, healthcare practice, and/or policy.

The authors are grateful for your valuable feedback and your clear commitment to assisting us in enhancing the manuscript and improving its relevance to healthcare delivery, practice, and policy.

More information about the institutions is needed for readers not familiar with the Mexican healthcare system. This information should include some measures of whether the obstetrics patient population differed in terms of complexity or needs, as well as the number of physicians, delivery rooms, and hospitals.

Thank you for your valuable feedback, with which we fully agree. Upon reviewing the document again, we recognized that the reported information was limited and challenging to comprehend for readers unfamiliar with the Mexican healthcare system. Therefore, we have included a detailed description of our healthcare system in the introduction to address this concern. (Lines 61-104).

  More detail is needed about the data sources and variables of interest. This could be reported in a Table 1.

A description of the data sources was expanded, and Table 1 was added.

 

A more detailed description of how efficiency of each DMU and average technical efficiency were calculated and whether time effects were taken into account would aid in interpreting the data. I am not clear on how these calculations were made. For example, how was DMU weight was determined? (lines 198-214). What are the limitations of using state-level GDP as the only measure of socioeconomic status? (Done:  Lines 357-363) How is the sophistication of healthcare resources taken into account? (Lines 145- 147)

We have included a paragraph that extensively describes how the efficiency of each DMU and average technical efficiency were calculated. Additionally, we provided a description of how time effects were managed. Furthermore, we expanded the description of limitations to include aspects related to GDP and the consequences of not incorporating quality into the analysis, among other factors.

In addition to the figures, data tables would aid in interpreting the findings.

Thank you for this valuable suggestion. A table displaying the technical efficiency results by DMU (Decision Making Unit) was included to enhance the clarity and comprehension of Figure 2.

The assumption of homogeneity in the quality of outputs is an extremely strong assumption. How would taking into account quality of care affect the results? In the present model, the goal is resource minimization, ignoring quality of care and differences in severity of illness and/or at-risk pregnancies. It is likely that there are meaningful differences across DMUs and states that should be taken into account. Given the world-wide focus on maternal and infant mortality, measures of quality are needed. 

We agree that the assumption of “homogeneity in the quality of outputs” is a large assumption. Nevertheless, this assumption was based on the contents of the Official Mexican Standard, which establishes the guidelines for obstetric care delivery in all Mexican health institutions. An explanation of this assumption was included in the manuscript (lines 200-2004). A discussion of the consequences of ignoring factors related to the quality of care and differences in severity of illness and/or at-risk pregnancies was added to the limitations section (lines 420-425).

More interpretation of what a TE of 0.82 is needed for readers not familiar with the methodologic approach. Is the maximum TE only meaningful relative to the comparison groups, or does it have an interpretation from an absolute level? (lines 220-224) Similarly, more narrative about how to interpret Figure 2 is needed. 

In the methods section (lines 201-207), we have incorporated a detailed description of the criteria and procedures utilized to establish the cut-off points as depicted in Figure 2. Furthermore, we have expanded the narrative on how to interpret these cut-off points for clarity and comprehension

Reviewer 3 Report

Comments and Suggestions for Authors

Data Envelopment Analysis is used to explore the technical efficiency (ratio of invested resources and healthcare delivery activities) in obstetrics care in Mexico.

 Introduction

-The link between social security and healthcare provision is not clear. Does social security (rather than health insurance) cover healthcare? Does the type of financial scheme match with specific healthcare providers? Some more description of the healthcare system is required.

-Please also describe some more on obstetric care in Mexico. Are deliveries usually in hospitals, in ambulatory settings, or both? What type of health professionals are involved? (midwives?)

-There are number of phrasings that may need revision

*collaboration scheme (p.2, l.9): suggest to omit scheme

*middle-high income countries (p.2, l.58): is this correct or it middle income countries?

 

Methods

-It is essential to describe the input and output variables in more detail: how were they selected, what is their validity, were they combined into single scales? Some more information on the databases used would also be helpful.

-Outputs are consultations and delivery services (p.3, l. 104), which suggests a focus on healthcare processes rather than health outcomes. This reflects a purely economic approach.

-The analytical approach is overall sound, but I cannot actually check whether the calculations are correct. Also, I would expect some sensitivity analyses in a study like this one.

-As the different financing schemes reflect different parts of the population, I wondered whether these differed regarding risk factors for obstetric outcomes (e.g. education level). Did the analysis adjust for such factors? Did you consider it in the interpretation of the differences?

 

Results

-A general comment is that I wondered whether the names of organisations should be presented in this publication, particularly in fig 3. For scientific purposes, it would be sufficient to keep these anonymous.

-Fig 2 is not clear to me, so I cannot assess it. It needs more explanation.

 

Discussion

-The discussion reflects on inequities, but this is limited by the absence of health outcomes in this study. Outputs are healthcare delivery activities, but more of those does not necessarily mean better health outcomes.

-One limitation of the study is mentioned, but this is rather brief. Further limitations are that the data were relatively old (up to 2018, so pre-covid-pandemic), that obstetric risk factors probably differed between populations of different providers but were not documented, and probably further issues.

Comments on the Quality of English Language

The language is generally fine but some light editing would improve the quality.

Author Response

Data Envelopment Analysis is used to explore the technical efficiency (ratio of invested resources and healthcare delivery activities) in obstetrics care in Mexico.

 Introduction

The link between social security and healthcare provision is not clear Does social security (rather than health insurance) cover healthcare? Does the type of financial scheme match with specific healthcare providers? Some more description of the healthcare system is required.

Thank you for your valuable feedback, with which we completely agree. Upon reviewing the document again, we recognized that the reported information was limited and challenging to understand for readers not familiar with the Mexican healthcare system. Therefore, we have included a detailed description of our healthcare system in the introduction (Lines 101-104).

Please also describe some more on obstetric care in Mexico. Are deliveries usually in hospitals, in ambulatory settings, or both? What type of health professionals are involved? (midwives?)

Done. (lines 119-123).

-There are number of phrasings that may need revision.

The text was carefully revised to improve syntax, style, and clarity.

*collaboration scheme (p.2, l.9): suggest to omit scheme.

The paragraph was modified to ensure comprehension of the concept.

*middle-high income countries (p.2, l.58): is this correct or it middle income countries?

Corrected

It is essential to describe the input and output variables in more detail: how were they selected, what is their validity, were they combined into single scales? Some more information on the databases used would also be helpful.

Excellent comment, thank you. In lines (137-154), we described the inputs and outputs. Additionally, in response to suggestions from another reviewer, we included a table summarizing the variables and their sources.

Outputs are consultations and delivery services (p.3, l. 104), which suggests a focus on healthcare processes rather than health outcomes. This reflects a purely economic approach.

Indeed, this article reflects the findings of a purely economic analysis. In societies like Mexico, the analysis of technical efficiency is of high interest for decision-making, which is always constrained by resource scarcity. This work contributes to identifying windows of opportunity for the reallocation and optimization of resources.

The analytical approach is overall sound, but I cannot actually check whether the calculations are correct. Also, I would expect some sensitivity analyses in a study like this one.

The methodological description has been expanded to clarify how the calculations were performed. Regarding sensitivity analysis, given the available data and the working hypothesis that posited segmentation and fragmentation as determinants of inefficiency in the Mexican healthcare system, it would rely on extreme and unrealistic assumptions. However, the authors are open to hearing suggestions.

As the different financing schemes reflect different parts of the population, I wondered whether these differed regarding risk factors for obstetric outcomes (e.g. education level). Did the analysis adjust for such factors? Did you consider it in the interpretation of the differences?

Thank you very much for your comment. We agree with your perspective. However, the available data does not allow for that level of analysis. In light of this, we aimed to crosscheck the efficiency analysis with economic level, as measured by state-level GDP. Theoretically, one would expect a differentiated distribution of risks and efficiencies based on economic level; however, the study's results did not show a relationship between efficiency and economic level. Risks were not analyzed due to the absence of population data. However, it is important to consider that the study's objective was to evaluate the technical efficiency of the Mexican public health system in delivering obstetric care from 2012 to 2018, categorized by economic condition at the state level.

Results

A general comment is that I wondered whether the names of organisations should be presented in this publication, particularly in fig 3. For scientific purposes, it would be sufficient to keep these anonymous

The names were omitted, and a nomenclature that maintains anonymity was used.

Fig 2 is not clear to me, so I cannot assess it. It needs more explanation.

Done. Lines (241-242) The graph was changed and the description of Figure 2 was expanded.

Discussion

The discussion reflects on inequities, but this is limited by the absence of health outcomes in this study. Outputs are healthcare delivery activities, but more of those does not necessarily mean better health outcomes.

We fully agree. Nevertheless, it is important to point out that measuring health outcomes fell outside of the scope of this study, which aimed to evaluate the technical efficiency of the Mexican public health system in the delivery of obstetric care during the period from 2012 to 2018, by economic condition at the state-level.

One limitation of the study is mentioned, but this is rather brief. Further limitations are that the data were relatively old (up to 2018, so pre-covid-pandemic), that obstetric risk factors probably differed between populations of different providers but were not documented, and probably further issues.

The limitations section was expanded, and a specific paragraph addressing COVID was included.

Comments on the Quality of English Language

The language is generally fine but some light editing would improve the quality.

The text was carefully revised to improve syntax, style, and clarity.

 

 

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

I think the changes you have made improve the coherence and quality of this paper.

Comments on the Quality of English Language

I have noticed the important revision of English language. However, there would be some observations:

1. Row 22 (abstract):

'Of the las' - probably a typing issue ('last' instead of 'las')

2. Rows 14-16:

'Using the results of the first stage, the average technical efficiency of each institution at the national level was estimated and compared. The results of the first stage were also used to estimate and compare the average efficiency of each state-level health system by socioeconomic characteristics'  

- I would rephrase them, avoiding repetitions.

3. Rows 27-28:

' ... rated (high efficiency scores) the majority have unused operational capacity. This unused capacity could be employed to meet the needs of vulnerable populations with restricted access to due to'

- 'restricted access due to' (without 'to' before 'due' - 'restricted access due to')

- I would rephrase these structures: 'have unused operational capacity. This unused capacity could be employed.

Author Response

Dear Reviewer

The authors appreciate the review and comments. The modifications are described below.

-----------------------

I have noticed the important revision of English language. However, there would be some observations:

1. Row 22 (abstract):

'Of the las' - probably a typing issue ('last' instead of 'las')

Done: This typo was corrected.

2. Rows 14-16:

'Using the results of the first stage, the average technical efficiency of each institution at the national level was estimated and compared. The results of the first stage were also used to estimate and compare the average efficiency of each state-level health system by socioeconomic characteristics'  

- I would rephrase them, avoiding repetitions.

Done: These sentences were restructured to eliminate repetitions. 

3. Rows 27-28:

' ... rated (high efficiency scores) the majority have unused operational capacity. This unused capacity could be employed to meet the needs of vulnerable populations with restricted access to due to'

Done: These sentences were rewritten to avoid repetition.

- 'restricted access due to' (without 'to' before 'due' - 'restricted access due to')