Assessing Police Technical Efficiency and the COVID-19 Technological Change from the Pact for Life Perspective

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

1. The paper is well written and applies a well-known and established empirical process. Although economists have more often applied stochastic frontier analysis over DEA, I am not convinced in this particular application that either process is going to provide a different result. However, the authors might mention in a footnote that there are other methods to empirically investigate the research question in this paper and that the authors have decided to use DEA.

2. One challenge in the DEA analysis is determining which unit is the most efficient and which is least efficient in absolute vs. relative terms. The authors might do a better job of explaining how these units are determined and whether there is a statistically significant difference between the most and least efficient units.

3. I am okay with the two year approach but it would seem better to have data that came from strictly before the pandemic and strictly after. The data from 2020 are a mix of pre-pandemic months of Jan-Mar, early pandemic months of Apr-Aug, and peak pandemic months of Sep-Dec 2020. Thereafter, in 2021 the pandemic is still active although during that year different countries are relaxing or strengthening their covid policies relative to population movement, interaction, etc. Would it be better to have data from 2021 or 2022 to use as the post-pandemic period?

4. The Pact for Life program is *very* specific and therefore the external validity of the empirical results is limited. The authors should do a better job in the conclusion to point out that the program has limited generality beyond Brazli. 

Author Response

1. The paper is well written and applies a well-known and established empirical process. Although economists have more often applied stochastic frontier analysis over DEA, I am not convinced in this particular application that either process is going to provide a different result. However, the authors might mention in a footnote that there are other methods to empirically investigate the research question in this paper and that the authors have decided to use DEA.

A: Thank you for the comment. We have added the following paragraph in the last part of section 2.2 as suggested:

“DEA models and developments offer valuable insights into measuring the efficiency and effectiveness of policing, focusing on the estimation of criminal costs [36] and the external environment's influence on police performance [35, 37]. Nevertheless, such measures are not exclusive for non-parametric frontier estimations and many applications of Stochastic Frontier Analysis (SFA) are similarly common in economic literature of crime and policing [38].”

2. One challenge in the DEA analysis is determining which unit is the most efficient and which is least efficient in absolute vs. relative terms. The authors might do a better job of explaining how these units are determined and whether there is a statistically significant difference between the most and least efficient units.

A: We have included two paragraphs to address this recommendation in the fourth section (Data analysis and discussion):

Efficiency is evaluated by comparing the performance of each Decision-Making Unit (DMU) relative to others in the sample, based on input-output relationships. Efficiency is typically defined as a DMU's ability to maximize outputs while minimizing inputs, and the most efficient units are those that operate on the efficient frontier, achieving an efficiency score of 1. To address the distinction between absolute and relative efficiency, our analysis reports the units that are efficient (score of 1) relative to their peers, acknowledging that these scores are relative to the data set being evaluated.

"Table 5 report an increase in the number of efficient Decision-Making Units (DMUs) in 2020 compared to 2019 (table 4). There are 8 eficient units during the pandemic compared to 4 before the pandemic. Efficient Integrated Security Areas (AIS) in 2019 are Jaboatão, Olinda, Arcoverde and Cabrobó. On the Other hand, efficient units in 2020 are Santo Amaro, Jaboatão, São Lourenço da Mata, Vitória, Palmares, Belo Jardim, Santa Cruz do Capibaribe and Cabrobó. The least eficiente units are also less ineficient in 2020 compared to 2019: considering the five more inefficient units in 2020, we have Salgueiro (0.60), Afogados (0.56), Garanhuns (0.55), Ouricuri (0.47) and Caruaru (0.46) compared to Boa Viagem (0.43), Petrolina (0.375), Afogados da Ingazeira (0.375), Limoeiro (0.27) and Garanhuns (0.25) in 2019. This finding suggest that the pandemic effect on crime and investigative policing was positive based on aggregate perspective."

3. I am okay with the two year approach but it would seem better to have data that came from strictly before the pandemic and strictly after. The data from 2020 are a mix of pre-pandemic months of Jan-Mar, early pandemic months of Apr-Aug, and peak pandemic months of Sep-Dec 2020. Thereafter, in 2021 the pandemic is still active although during that year different countries are relaxing or strengthening their covid policies relative to population movement, interaction, etc. Would it be better to have data from 2021 or 2022 to use as the post-pandemic period?

A: Thank you for the comment. This is a data limitation that we should consider expanding in future works for sure. We included this limitation and suggestion in the conclusion:

"An interesting extension for future work is considering a larger dataset including years that clearly distinguish between pre-and post-pandemic periods. While the current analysis utilizes data from 2019 and 2020, the year 2020 represents a transitional phase, with a mix of pre-pandemic (January-March), early pandemic (April-August), and peak pandemic months (September-December). To better assess the impact of the pandemic on efficiency and performance, it would be beneficial to include data from 2021 or 2022, which reflect more stable post-pandemic conditions. This would allow for a clearer comparison of the effects of the pandemic on policing efficiency, as 2021 and 2022 provide insight into how adjustments in policies, resource allocations, and societal behavior may have influenced crime rates and law enforcement outcomes. Such an approach would strengthen the analysis by offering a more distinct temporal separation between pre- and post-pandemic periods, providing a more robust understanding of long-term trends in efficiency."

4. The Pact for Life program is *very* specific and therefore the external validity of the empirical results is limited. The authors should do a better job in the conclusion to point out that the program has limited generality beyond Brazil.

A: Thank you. We included the following last paragraph in the conclusion to address this comment:

"Although the Pacto pela Vida program was designed for the specific context of Pernambuco, the insights and strategies it offers are highly relevant beyond Brazil. The adaptability demonstrated by many police units during the pandemic, such as Santo Amaro’s significant gains in efficiency and Jaboatão and Cabrobó resilience, highlight the importance of flexibility and responsiveness to external challenges. Strategies of this program, such as integrating the many domains of public security and implementing proactive crime prevention measures, are not limited to the local setting and can be applied to public safety efforts globally. The observed differences in how metropolitan and rural areas adapted to the pandemic in Pernambuco highlight the need for context-sensitive approaches to public security. These principles emphasize adaptable strategies that can be adapted to different environments to support public security initiatives in many regions and countries."

Reviewer 2 Report

Comments and Suggestions for Authors

The article addresses a question of great scientific and societal importance, which has not been the subject of a strong literature until now: the evaluation of the effectiveness of the police, particularly in contexts of technological change. It takes as its field a program implemented in the jurisdiction of Pernambuco (Brazil) to address rising crime rates, taking into account the impact of the Covid-19 pandemic on this effectiveness. The author can thus compare the differential effectiveness of the Civil Police of the State of Pernambuco in the 26 Integrated Security Zones (AIS) of the territory between 2019 and 2020. He does so by combining Data Envelopment Analysis (DEA) and an approach via the Malmquist Productivity Index (MPI) that not only allows to measure changes in the productivity of police investigations, but also to distinguish what, in these changes, is due to organizational changes and technological changes. He thus highlights both a general improvement in police effectiveness and strong differences between territories, demonstrating in particular that cities with a higher population density and greater economic and demographic dynamics, such as those close to the capital, have adapted more slowly to the demands of the pandemic in terms of public security compared to cities in the countryside or rural areas. Overall, the article is well conducted, well positioned in relation to the literature. The methodology is well explained and itself articulated with the few works that have already used homologous methods on close objects. A set of limitations are pertinently recalled in the conclusion. However, we will allow ourselves a few remarks:

- Mention is made of tables 3 and 4 line 383 while the subject refers to tables 4 and 5.

- The presentation of the results in part 4 is rather disjointed. We move from general statements to statements focused on certain geographical units, without this always being reported exactly to the figures that support these statements. It would be possible to start with general developments, and then come to developments specific to each (or some) geographical unit(s). More broadly, we do not understand the main guidelines of the conclusion: what are the factors that promote better productivity?

- The analysis is very convincing in highlighting the differential effectiveness of the police according to the different integrated security zones. It is much more allusive and hypothetical as to the explanatory factors of these differences. While these are the factors that would allow us to better understand efficient or inefficient practices

-It could have been useful to say more about the production of the statistics that are used. Since we know that these statistics sometimes say less about the reality of crimes and their resolution than about other more hidden phenomena.

Author Response

1. The article addresses a question of great scientific and societal importance, which has not been the subject of a strong literature until now: the evaluation of the effectiveness of the police, particularly in contexts of technological change. It takes as its field a program implemented in the jurisdiction of Pernambuco (Brazil) to address rising crime rates, taking into account the impact of the Covid-19 pandemic on this effectiveness. The author can thus compare the differential effectiveness of the Civil Police of the State of Pernambuco in the 26 Integrated Security Zones (AIS) of the territory between 2019 and 2020. He does so by combining Data Envelopment Analysis (DEA) and an approach via the Malmquist Productivity Index (MPI) that not only allows to measure changes in the productivity of police investigations, but also to distinguish what, in these changes, is due to organizational changes and technological changes. He thus highlights both a general improvement in police effectiveness and strong differences between territories, demonstrating in particular that cities with a higher population density and greater economic and demographic dynamics, such as those close to the capital, have adapted more slowly to the demands of the pandemic in terms of public security compared to cities in the countryside or rural areas. Overall, the article is well conducted, well positioned in relation to the literature. The methodology is well explained and itself articulated with the few works that have already used homologous methods on close objects. A set of limitations are pertinently recalled in the conclusion. However, we will allow ourselves a few remarks:

A: Thank you very much for your attention and comments. We hope we have addressed all your suggestions and we appreciate your review.

2. Mention is made of tables 3 and 4 line 383 while the subject refers to tables 4 and 5.

A: Corrected (now lines 407-408):

“Table 5 reports an increase in the number of efficient Decision-Making Units (DMUs) in 2020 compared to 2019 (table 4).”

3. The presentation of the results in part 4 is rather disjointed. We move from general statements to statements focused on certain geographical units, without this always being reported exactly to the figures that support these statements. It would be possible to start with general developments, and then come to developments specific to each (or some) geographical unit(s). More broadly, we do not understand the main guidelines of the conclusion: what are the factors that promote better productivity?

A: Thank you for the comment. We have adjusted as suggested the explanation in section 4 from general to specific and provide a few more comments in the conclusion. Please see the example of the following inclusions:

“Efficiency is evaluated by comparing the performance of each Decision-Making Unit (DMU) relative to others in the sample, based on input-output relationships. Efficiency is typically defined as a DMU's ability to maximize outputs while minimizing inputs, and the most efficient units are those that operate on the efficient frontier, achieving an efficiency score of 1. To address the distinction between absolute and relative efficiency, our analysis reports the units that are efficient (score of 1) relative to their peers, acknowledging that these scores are relative to the data set being evaluated.

Table 5 report an increase in the number of efficient Decision-Making Units (DMUs) in 2020 compared to 2019 (table 4). There are 8 eficient units during the pandemic compared to 4 before the pandemic. Efficient Integrated Security Areas (AIS) in 2019 are Jaboatão, Olinda, Arcoverde and Cabrobó. On the Other hand, efficient units in 2020 are Santo Amaro, Jaboatão, São Lourenço da Mata, Vitória, Palmares, Belo Jardim, Santa Cruz do Capibaribe and Cabrobó. The least eficiente units are also less ineficient in 2020 compared to 2019: considering the five more inefficient units in 2020, we have Salgueiro (0.60), Afogados (0.56), Garanhuns (0.55), Ouricuri (0.47) and Caruaru (0.46) compared to Boa Viagem (0.43), Petrolina (0.375), Afogados da Ingazeira (0.375), Limoeiro (0.27) and Garanhuns (0.25) in 2019. This finding suggest that the pandemic effect on crime and investigative policing was positive based on aggregate perspective.”

“ (…) the insights and strategies it offers are highly relevant beyond Brazil. The adaptability demonstrated by many police units during the pandemic, such as Santo Amaro’s significant gains in efficiency and Jaboatão and Cabrobó resilience, highlight the importance of flexibility and responsiveness to external challenges. Strategies of this program, such as integrating the many domains of public security and implementing proactive crime prevention measures, are not limited to the local setting and can be applied to public safety efforts globally. The observed differences in how metropolitan and rural areas adapted to the pandemic in Pernambuco highlight the need for context-sensitive approaches to public security. These principles emphasize adaptable strategies that can be adapted to different environments to support public security initiatives in many regions and countries.

The good performance of many units based on the Pacto pela Vida program during the pandemic suggests that adaptive and flexible strategies are crucial in times of crisis. Policymakers should institutionalize these adaptive measures, ensuring law enforcement agencies can quickly respond to changing circumstances without significant productivity losses. Integrating comprehensive data analytics into routine police work can provide real-time insights and predictive analytics, enabling proactive crime prevention measures. This approach can optimize resource distribution, improve response times, and enhance public safety. Continued research and policy development are es-sential to build on these insights, ensuring that public security efforts are practical and adaptive to changing circumstances.”

4. The analysis is very convincing in highlighting the differential effectiveness of the police according to the different integrated security zones. It is much more allusive and hypothetical as to the explanatory factors of these differences. While these are the factors that would allow us to better understand efficient or inefficient practices

A: Thank you. We agree with the reviewer that it could benefit from a deeper exploration of the factors that explain these differences. Because of the pandemic during the development of this project, much of the planned benchmarking was compromised. We have added in the limitations (conclusion section):

“An interesting extension for future work is considering a larger dataset including years that clearly distinguish between pre-and post-pandemic periods. While the cur-rent analysis utilizes data from 2019 and 2020, the year 2020 represents a transitional phase, with a mix of pre-pandemic (January-March), early pandemic (April-August), and peak pandemic months (September-December). To better assess the impact of the pandemic on efficiency and performance, it would be beneficial to include data from 2021 or 2022, which reflect more stable post-pandemic conditions. This would allow for a clearer comparison of the effects of the pandemic on policing efficiency, as 2021 and 2022 provide insight into how adjustments in policies, resource allocations, and societal behavior may have influenced crime rates and law enforcement outcomes. Such an approach would strengthen the analysis by offering a more distinct temporal separation between pre- and post-pandemic periods, providing a more robust under-standing of long-term trends in efficiency.

In addition, identifying the underlying causes behind the differential performance—whether related to resource allocation, leadership, community engagement, or other operational strategies—can provide more meaningful insights into the drivers of efficiency or inefficiency. Future work should focus on investigating these explanatory factors more rigorously, as understanding them would allow us to focus on best practices and develop targeted recommendations for improving police performance in both high- and low-efficiency zones. This can enhance the practical applicability of the findings and provide a more comprehensive understanding of the dynamics driving police effectiveness.”

5. It could have been useful to say more about the production of the statistics that are used. Since we know that these statistics sometimes say less about the reality of crimes and their resolution than about other more hidden phenomena.

A: Thank you. We have added an additional discussion on this as suggested:

“One common limitation in studies involving criminal data is the underlying dynamics that go beyond the crime reports and the resolution itself. Factors such as variations in data collection methods, regional reporting practices, and potential underreporting or misclassification of crimes can all influence the reliability of these figures. Additionally, clear-up rates, which mostly refer to the resolution of CVLI cases, may not always fully capture the complexities of criminal investigations, including factors like resource allocation, local policing strategies, or the broader socio-political environment. As such, while CVLI statistics are essential for assessing public security, they should be interpreted with caution, considering these hidden phenomena that may affect their accuracy and the real picture of crime and its resolution. For a better understanding of the police organization and crime statistics and context in Pernambuco, refer to other empirical applications in [41-45].”