Performance Analysis Method for Robotic Process Automation

Abstract

Recent studies show that decision making in Business Process Management (BPM) and incorporating sustainability in business is vital for service innovation within a company. Likewise, it is also possible to save time and money in an automated, intelligent and sustainable way. Robotic Process Automation (RPA) is one solution that can help businesses improve their BPM and sustainability practices through digital transformation. However, deciding which processes to automate with RPA technology can be complex. Consequently, this paper presents a model for selecting indicators to determine the profitability of shifting to RPA in selected business processes. The method used in this work is the Performance Analysis Method, which allows for predicting which processes could be replaced by RPA to save time and money in a service workflow. The Performance Analysis Method consists of collecting data on the speed and efficiency of a business process and then using that data to develop discrete event simulations to estimate the cost of automating parts of that process. A case study using this model is presented, using business process data from an international utility company as input to the discrete event simulation. The model used in this study predicts that this Electric Utility Company (EUC) will save a substantial amount of money if it implements RPA in its call center.

1. Introduction

Modern technological advances in simulation, robotics, machine learning and data analytics have recently been adapted for use in business and industry (often referred to as Industry 4.0 [1]). Currently, automation is based on robots, and at an industrial level, there is a demand for robots with increasingly advanced and robust capabilities. These requirements can be implemented using advances in control theories [2,3] and artificial intelligence (AI) [4]. Control theories help reduce trajectory errors and improve the precision of industrial robots [5] through controllers, as well as the use of intelligent systems to obtain high precision, for example, incorporating Fuzzy Logic when useful [6,7]. Moreover, AI is essential to train robots; their use is varied [8,9], whether in physical or software robots. On the other hand, the digitization and automation of processes in different business areas is a trend that evolves [10] and positively impacts customers’ well-being [11]. Companies use Business Process Management (BPM) approaches [12] to continuously improve their business processes and incorporate sustainability as an essential strategy [13,14,15].

One of the ways to improve and optimize business processes considering sustainability is by automating processes through Robotic Process Automation (RPA) technologies. For example, workforce efficiency can be improved by replacing repetitive technical work with automation, simulating different process workflows and analyzing the data to predict the most cost-effective organizational method [16]. The business processes best suited for RPA have low complexity, high error rates when performed by humans and are highly valuable to the enterprise [17,18]. RPA can be seen as an alternative to outsourcing, which reduces costs by shifting jobs to countries that can perform tasks with less effort. Therefore, RPA is often referred to as insourcing and can be seen as a less controversial and easier to control method of cost savings [19,20]. Ideally, RPA would allow companies to cut costs, reduce errors and reserve employees for more complex tasks and face-to-face customer interactions. Recent technological developments have unlocked the ability of RPA to handle more complex tasks that are not governed by well-defined business rules. However, there is still a gap in the literature related to when we want to decide what to automate [21].

Desai (2020) [22] indicates that many organizations need help deciding if they require RPA; this work proposes prerequisites for the organization to have RPA. These consist of selecting the primary business process and analyzing each level. These levels answer four questions: "Q1. Is the Business Process eligible for RPA?, Q2. How to asses RPA possibility and business process/use case?, Q3. How to assess if RPA is appropriate? and Q4. Can we allocate the assessment to categories if one can proceed/reject RPA?" A confusion matrix is used to evaluate the acceptance or rejection; if the levels’ results are equal to 1, the organization can select the RPA. In another work, Yi-Wei et al. (2019) [23] propose a cyclical method of eight steps to evaluate the benefits RPA offers to help decision-making in an organization. The first step is planning the project, evaluating and selecting resources, and labor for the execution. Although the method presents the design and implementation of RPA based on agile development, it does not present details on how to conduct this planning and decide which business processes to implement without taking significant risks. Asatiani’s (2023) [24] work notes that organizations must carefully determine which business processes to automate with RPA before implementation. Nevertheless, the model presented is operational, based on a decision checklist helping the model’s decision-making process that best suits the organization’s needs. This list focuses on the implementation of RPA, leaving an open question of how to determine which processes to automate.

Determining which tasks can be automated is often quite tricky, as decision making is required to identify and select the most beneficial plans. It usually helps to start with a simple filter to exclude processes that require complex decision making. The next step is to evaluate the potential performance of the RPA process. The process of evaluating performance is currently something of an art that requires the input of humans and extensive data gathering [25]. This is because there are many ways to evaluate performance, which change from task to task and depend on the organization’s priorities. Performance evaluations require a thorough knowledge of the processes involved and a careful selection of methods, volume constraints and analytical tools. With these constraints in mind, we attempt to evaluate the performance of a specific business process within a utility company. In this way, the work answers questions such as: Can we estimate the cost savings from switching to RPA for a specific business process using simulations based on employee performance and cost data?

It is often difficult to analyze the performance of real-world processes because they cannot be easily stopped or changed to gather data. This is one of the reasons why RPA capacity planning is essential to avoid financial losses, ensure user satisfaction and maintain a company’s external image. This often requires detailed studies of all business layers affected by a process, as the consequences of a process change cannot be foreseen directly. Thus, this work contributes to a method proposal (cyclical and evolutionary method) that establishes steps and models to be adopted in order to perform the selection of processes and performance analysis, identifying, for example, business objectives, metrics and performance indicators, the impact of the implementation of RPA, and predicting the performance of the implementation of RPA.

The objective of this work is to present a method of analysis of the performance of robotic process automation that should work for any generation of robots (G1, G2 and G3+ generation robots). This methodology establishes phases for process selection, techniques that can be used to filter out possible tasks and metrics and performance indicators that can be used to evaluate the impact of shifting to RPA for the selected processes, including barriers and bottlenecks that could limit efficiency. Figure 1 presents the robot generations based on the EY concepts—the first generation (G1) features rule-based automation of processes previously performed by humans. The second generation (G2) has improved process automation—robots can process unstructured data and use cloud technologies. The new generation of RPA, called G3+, can make decisions using artificial intelligence (AI) and machine learning (ML) to increase automation efficiency and scope [26,27,28,29,30]. The RPA G3+ architecture has a robotic intelligence component [31] that requires either a model that has been previously trained using data gathered for the business process in question or by linking the model to a database that can be quickly updated as new information emerges [32,33].

The organization of this paper is as follows: Section 2: Related Works; Section 3: Methodology; Section 4: Case study; Section 5: Result and Discussion. Lastly, the conclusions are discussed in Section 6.

2. Related Works

Simulation models represent a system abstraction, which aims to describe the system with a focus on performance analysis. These models can be classified according to their characteristics and purposes for which they are intended. The two types are the semi-formal descriptive models and the formal models. Semi-formal descriptive models describe the computational system without the need for simulation but with the fundamental concern of documentation and communication between different actors involved in the modeling. In this model, we have modeling languages, such as Icam Definition for Function Modeling (IDEF0), a functional modeling language for analysis, development, information integration, systems integration and business processes [34]; Unified Modeling Language (UML), which is a modeling language used on software engineering [35]; Business Process Model and Notation (BPMN)—a graphical notation to represent business processes [36]. This technique has the advantage of being standardized, thus allowing a unique understanding of complex business semantics and workflows, and Value Chain represents a set of activities to perform or deliver a service or product. It has an essential role in representing business processes in a global way of companies.

Formal models describe the computational system to allow its simulation in real time and dynamically. That is, their values change over time. Within this classification [25], we have the Discrete Simulation Model, which uses variables that assume discrete values in a certain period; Deterministic Simulation Model, in which the results of the models (variables) can be accurately determined; and the Stochastic or Probabilistic Simulation Model, in which the variables occur according to a probabilistic distribution. Vergidis et al. (2008) [37] review and classify the main techniques for modeling business processes concerning their analysis and optimization capabilities. Three primary groups are identified, and a selection of representative business process modeling techniques are classified based on them. The main contribution of the work is that it identifies which types of business process models are suitable for analysis and optimization.

Within the formal models focused on simulation, we can have [38] (1) Queuing Networks, a technique used for analytical modeling and simulation of computational systems. Through it, it is possible to evaluate the performance of a system, identify bottlenecks and estimate essential performance measures, such as predicting waiting times and queue sizes. (2) Petri Nets is a mathematical model for describing of distributed systems. A Petri Net is a directed bipartite graph in which the nodes represent transitions (events that may occur, represented by squares) and places (conditions, represented by circles). It has been applied in modeling and simulating dynamical systems and discrete events, for example, in software projects process automation, workflow management, data analysis, concurrent programming and reliability engineering).

A performance case study, considered relevant for the survey of this work, was conducted by Menascé (2002) [39]. Menascé presents a case study of modeling simulation of an e-commerce application, establishing four distinct and integrated performance models at the levels of business, services, user behavior and infrastructure. The business model uses metrics defined from the current financial profit of a fictitious company and the financial goal for the next year, discussing how this new goal should influence performance at other levels, particularly the impacts on the system’s infrastructure. The services responsible for generating the primary business goals are defined in the Service Model. For example, the services considered are sales of products and advertisements (banners) on the site. To represent the User Behavior Model, they used the Customer Behavior Model Graph (CBMG) model [40,41]. The graph nodes represent site pages, and the arcs are the navigation probabilities between the pages. The infrastructure model represents computational elements (web servers, networks, database servers, application servers) through Queue Networks. This model is integrated with the other models through performance goals (response time), evaluating which changes are necessary for the infrastructure to guarantee the business performance goals, including bottleneck analysis and the demands of each element, among others.

In the case of call centers, performance indicators are fundamental. It is essential to monitor the quality of this service and customer satisfaction. In addition, as there is an associated high cost, it is necessary to monitor the resources involved in detail. Effective management and optimization of resources are crucial to avoiding unnecessary costs. In general, the Queuing Network model is adopted, which allows its simulation, representing each system element (attendant, Audible Response Unit, computational support systems, among others) as a Queue [42]. Karakus et al. (2016) [43] define a Big Data infrastructure for call centers monitoring analytics, analyzing all recorded calls and measuring the performance of a customer service representative or agent.

In this work, unlike other works, the novelty is a proposed methodology for the selection of indicators and performance analysis of already existing robots/RPAs (monitoring) to specify possible implementations of new RPAs in a company. As a case study, we present the case of a large electric company in Brazil to guide the evaluation of the G1, G2 and G3+ robots. The method was defined based on previous work applied by Ernst & Young (EY) in the project. The predictions of this research were presented to the EUC and were taken into account according to the results obtained. In this way, they led to the decision to implement a "Proactive Notification" RPA that is capable of alerting with high precision customers with a higher probability of complaining; the work was published by Vajgel et al. (2021) [33].

EUC collected data on the number of customers using each of the services within the flowchart representing electrical damage reimbursement services, such as the average amount of employee time that must be spent servicing each customer at each step of the process, among other useful variables. Finally, a User Experience (UX) study with RPA after 18 months of adoption was conducted to capture the multifaceted experience of workers. RPA provided a positive user experience mainly because of the significant time saved. The work was recently published by Filgueiras et al. (2022) [44]. Thus, we can highlight the importance of the throughput analysis method for RPA.

3. Methodology

The methodology used consists of a set of steps (see Figure 2). These steps were developed based on a previous work with RPA system development by Ernst & Young (EY) [45] and on the related works in [25,40,46].

Figure 2 illustrates the USP/EY/EDP Performance Analysis Method, which comprises four steps. Step (1) Understand processes and architecture of RPAs; Step (2) Identify and select RPA processes: Step (3) Design model based on system infrastructure; and Step (4) Analyze and interpret results. Each of these steps is described below:

Step (1)

The RPA system must be studied in detail. The boundaries that delimit the system (main processes) and its interfaces with external systems, the leading current performance indicators and expected performance targets must be determined.

Step (2)

In this step, the target processes are listed, identifying their indicators, business metrics, computational infrastructure and personnel involved, according to Figure 3.

Figure 3 shows the process identification and selection step through a sequence. First, current and future performance (goals)—the indicators are based on the previous step’s business goals. Second, to map business goals into indicators and find their weights, the Tropos Model is used [47]. Third, mapping indicators into metrics is applied to the proposal by Barone et al. (2012) [48]. Furthermore, finally, the generated indicators, weights and metrics are used as parameters for the Ernst & Young Model (EY Model), which aims to support the preliminary selection of processes to be considered for robotization.

Step (3)

Using the information obtained in the previous steps, a document is elaborated with the conceptual model of the system that specifies the conception that one has of the system and everything that was assumed. The model details depend on project objectives, performance measures, data availability, issues of credibility of the information obtained, limitations of computing resources, opinions of experts in the business areas and time and financial limitations. In this stage, the Performance Analysis Models are defined using metrics obtained in the previous stages and the new measures collected in the existing computational systems or through estimates. The models considered are the Service Utilization Model Based on User Behavior (CBMG) and the Queuing Network Simulation Model. Experiments are planned and executed from the finished model to perform measurements on the models. The experiments should involve varying levels (or values) of factors that affect system performance, including the workload to be put through the system regularly and in peak situations.

Step (4)

The results of the experiments are random quantities that require a statistical analysis for their correct interpretation. The results will be used in decision-making processes if they are valid and credible.

RPA performance analysis cannot be performed in one step. Constant feedback and iterative optimization are required to achieve good results. The entire automation process must be thoroughly analyzed with two questions in mind: (1) “Where are we?” and (2) “Where do we want to go?”.

The first question helps determine which processes we want to automate, and the second question determines which indicators can be used to determine whether the automation method serves the company’s goals. Iterations of the process (Figure 2) may change the answers to these two questions somewhat, but the questions serve as a general guide for future refinements.

3.1. Performance Analysis Models in Consideration

The generalized cycle of performance analysis in Figure 2 can be refined to meet the purposes of the entire business, specific services, customers, or resources. As previously discussed, the computational systems’ modeling and capacity planning process is defined by several steps to generate Performance Analysis models. The generated models can be classified into four types, as outlined in Figure 4.

Figure 4 provides a graphic showing how the performance analysis applications are related. In this relationship, four models (business, service, customer and resource model) with approaches and metrics at the business and technological levels can be observed. The upward pointing arrows indicate a higher level of sophistication and interoperability of the components of a business process. Fully integrating and accounting for the impact of process changes requires consideration of more complex variables, such as user behaviour, the services offered by the business and the unique features of the business. Iterations of the performance analysis process should attempt to include these higher-level features. This can be accomplished by using standard metrics that serve as indicators of technical efficiency, user behaviour and desired business features.

The four types of applications for performance analysis are the following: (1) Business model, which provides a global vision of results that includes financial expenditures, social and environmental benefits, employee performance, and other factors relevant to the business as a whole. These metrics are fundamental for supporting mid and long-term process optimizations; (2) Service model, which addresses services that support the ultimate goals of the business, such as e-commerce, website efficiency, or service/product catalog for customers; (3) Customer/user model, which addresses issues of customer usage and satisfaction with business services and products; (4) Resource model, which defines the resource usage of the system in question and considers how to properly allocate those resources.

These models require the use of calculations and simulations that incorporate values based on service and usage data, such as the number of customers relative to the number of employees, customer service rates, error rates, and customer satisfaction rates. Statistical distributions with random variables and stochastic processes are used to represent the inputs and outputs of system processes about service time.

3.2. Metrics and Indicators for Process Selection

This subsection describes the metrics and indicators used for the performance analysis of a business process. Indicators refers to outcomes that allow evaluation of business processes (e.g., customer satisfaction, the efficiency of employee–customer interaction, product distribution), and metrics refers to the actual numerical values calculated for evaluation (e.g., percentage of satisfied customers, client interaction rate, employee error rate, products distributed per day). The indicators in this section are based on the work of Lacity et al. [49].

Table 1. Indicators for Process Automation based on Lacity et al.

Indicator	Description
Transaction volume	Frequency of task completion. It should be very high for RPA to be practical.
Access to multiple systems	The number of systems needed to complete a task should be high.
Stability of the environment	The environment should change only slightly.
Cognitive requirements	The task does not require creativity or interpretation.
Maturity of the process	The rules of the process can be easily and completely presented.
Decomposition of unambiguous tasks	The task can be divided into simple and measurable tasks.
Likelihood of human error	The error rate is high when performed by humans, but low when automated.
Number of exception rules	The task is highly standardized (i.e., the rate of anomalies is low).
Costs incurred	The cost and value of task performance are easily defined and the value of the task outweighs the cost.

lists the indicators to be considered for the first and second generations (G1 and G2) of the RPA process. The first column contains nine indicators, followed in the second column by a brief description.

Table 2. Indicators for selection of processes considered by Ernst & Young.

Indicator	Description
Based on rules	Does the process rely primarily on business rules?
Frequency	What is the frequency of execution of the process?
Level of standardization	Is the process standardized?
Data Sources	What are the process data inputs?
Data Types	Is the input data structured (i.e., tables and forms) or unstructured (i.e., sentences, emails, letters and contracts)?
Transaction Volume	Does the process have high transaction volumes?
Human Intervention	Does the process have frequent exceptions that require judgment and treatment via phone, email or other means of communication?
Time per transaction	What is the time spent per transaction?
Total number of FTEs	What is the total number of FTEs working on the process?
Business logic complexity	What are the complexity and variations of calculations required?
Number of Applications	How many applications does the process interact with?
Quality Impacts	Is the process prone to errors or data quality issues (i.e., an  improvement in data quality will significantly impact the process)?
Time-sensitivity	Is the process time-sensitive? Does the decrease in response time affect the subsequent process?
Process Documentation	Does the process have well-defined procedures? (i.e., work instruction).
Process Sensitivity	Is the process considered sensitive (i.e., information security, data privacy requirements, or does it involve compliance with legal/regulatory requirements)?

lists the indicators used in the suggested model for selecting Project Processes. Fifteen indicators are described; note that the indicators adopted in the Project Model correspond to the indicators for Process Automation for RPA G1 and G2 (see Table 1). However, the need to include cost indicators that can be used to assess the impact or even the Return on Investment should be emphasized. Another indicator for RPA G1 and G2, which are fundamental for G3, is the process reliability performance indicator.

These indicators are essential for evaluating the performance of machine learning algorithms. In the case of machine learning, the following indicators can be considered: Error rate and accuracy are two metrics commonly used for clustering algorithms and mean square error and average absolute distance are commonly used for regression algorithms [50].

Other indicators could be considered in later iterations within the Performance Analysis Model for G3+ RPA and refined based on the initial results. The ability of a robot to handle exceptions when processes require decisions is an example of a G3+ RPA indicator. Monitoring these indicators is essential for optimizing business process simulations. Tracking specific metrics during modeling is necessary to obtain information transferable to a real-world scenario. Business process data should inform these metrics. The robot-level metrics tracked in this work were velocity, reliability and availability.

• Velocity is a measure of how long it takes to complete a process. This metric can be subdivided into three parts:

  (a)

  Throughput is the number of executed requisitions per unit time. This unit of time depends on the request types that are sent to the system. For example, for a web server, the unit could be access requests per second;

  (b)

  Use refers to the percentage of time a resource set is busy. This is measured by comparing the time usage of a process to the time it takes to complete the entire process;

  (c)

  Response time is the time from the beginning to successful completion of an available service within a business process. The response time can be decomposed into service time and queue time:

  i.

  Service time refers to the time each resource (i.e., humans, robots or machines) is in use;

  ii.

  Queue time refers to the waiting time required to access each resource;

• Reliability is the probability of failure and the time interval between failures. A failure has occurred when a function of the business process could not be completed properly;
• Availability refers to times when the system cannot complete a service, which is called a fault. The types of faults depend on the process, but each one must be classified, monitored and assigned a probability in order to solve problems with the model.

Figure 5 shows a schematic representation of the workflow for the performance analysis process and associated metrics. The possible results for executing an order (request to a system) can be classified as follows [25,46]:

• Correct execution;
• Incorrect execution due to the occurrence of an error. An error occurs when the expected result of a particular functionality is not fulfilled due to a functional failure of the system;
• Lack of execution due to a failure occurring. A fault occurs when something unexpected happens in the environment, preventing the execution of a specific functionality or system.

Table 3. Services mapped to metrics of interest.

Service	Description	Used Metrics
Executed correctly	Package delivered on order	Response time Flow rate Processing time at origin/destination  Intermediate node time  Variance in response time
Executed Incorrectly	Out-of-order or duplicate packages	Probability of out-of-order or duplicate packets
Failure	Lost packages	Disconnection probability

organizes information about the metrics to be identified at different model levels.

Metrics applied to different levels of models are listed below:

• Hits/sec: number of requests for objects (infrastructure).
• Page Views/Day: number of pages per day (Infrastructure).
• Unique Visitor: how many different visitors accessed a site in a period (user behaviour).
• Revenue Throughput: measures return in monetary values per unit of time (business).
• Potential Loss Throughput: measured in monetary values per unit of time of purchases not made, that is, customers who gave up completing a purchase service (user behavior).
• User response time: total time the user waits for the service to run (user behaviour).
• Service probability: probability of a service being requested by a user (user behaviour).
• Full-Time Equivalent (FTE) or Whole Time Equivalent (WTE): indicates a person’s workload in order to make workloads comparable in various contexts (business).

3.3. Finding Indicators by Analyzing Company Data

The goal of an analytic process is to discover the patterns and anomalies that exist in a collection of data. An analytic process can be applied in any of the RPA development phases, including data preprocessing, model construction and RPA implementation. For example, analytics can be used for data preprocessing to extract features that can be used as input to the robotic process model. During construction, model analytics can be used to refine parameterization. In the production phase, analytics can be used to monitor the performance of each robot or the process to determine operational efficiency.

Analytics can also be applied to the entire RPA process to determine the potential for implementation in a broader business process. For performance monitoring that is intended to evaluate a single system, it is essential to communicate information to the data analyst through a dashboard of some sort so that metrics and systems can be changed or moved. Monitoring and diagnosing issues with the robot in the post-implementation phase is also an important step in the parameter refinement process.

Some important considerations for performing analytics during iterations of the RPA process are:

• Data availability, which refers to the proportion of data that can actually be systematically monitored and used in the modeling process. How much company data can be accurately monitored and how accurate are these data?
• Is it possible to process unstructured data? Can customer behavior, employee morale or word analysis be performed and used as the basis for a simulation?
• Input/output relationships—can the changes that the product or service undergoes be known at each stage of the process?
• What is the exception rate for this process? How often do exceptions to the rules of the system occur?
• When is judgment/decision making required? When and how often does the process require judgment, input and decision making by human or artificial intelligence?

4. Case Study

The purpose of this case study is to demonstrate the viability of the performance analysis method defined outlines in the previous sections. To this end, discrete process modeling and queueing theory were used to describe and investigate the potential of RPA implementation in the call center of an EUC and to outline future interaction goals between the EUC and the University of São Paulo (USP). We follow the iterative steps shown in Figure 2: (1) Determine performance objectives, (2) Understand process and architecture, (3) Identify and Select RPA Processes, (4) Design Model and (5) Analyze and Interpret results.

4.1. Determining Performance Objectives

The business process considered in this paper is used to compensate EUC’s customers for electrical damage to their property. The goals of EUC in optimizing this process are:

• Minimize in-person damage inspections through remote technical analysis with an equipment availability database and visit scheduling algorithms;
• Complete pricing analyses based on equipment availability surveys and replacement/repair costs;
• Approve compensation payments;
• Request new budgets from customers.

These improvements should cover more than 90% of the current workflow and directly impact the speed of the electrical damage compensation process, the number of customer complaints, the number of technical visits and the Full Time Equivalents (FTEs) required to provide each customer service. The estimated cost to implement this plan is $\$516,000$ for three months of RPA development, $\$971.21$ for monthly hardware and software costs and $\$1000$ for maintenance. EUC and EY helped develop three different scenarios to model the potential cost savings of investing in these improvements (

Table 4. Three example scenarios that could help predict the potential cost savings of an investment in RPA. Net Present Value (NPV) is the difference between the present value of cash inflows and the present value of cash outflows over a period of time.

Benefit	1	2	3
Reduce Hours (%)	$33\%$	$50\%$	$60\%$
Reduce Survey (%)	$20\%$	$40\%$	$60\%$
Payback (months)	31	19	16
Monthly Benefit ($)	21,256	36,642	47,086
NPV 5 years ($)	285,476	919,504	1,349,885

).

4.2. Understand Process and Architecture

A detailed flowchart of the electrical damage reimbursement process for the EUC call center and back office workers was provided as a guide for developing a discrete process model. A simplified version of the flowchart can be found in Figure 6. This flowchart provided the basic structure of the model.

Figure 6 shows the main services in the process and the percentage of customers using each service. The service that takes the longest time is estimated to be the one checking the cost estimate accuracy, which takes approximately one hour on average. The remaining services take approximately 40 min of employee time to complete. The client damage estimate represents the period when the client must determine how much replacement parts they need and provide that information to EUC. These estimates are based on data from EY, collected from monitoring EUC’s call center operations.

The EUC and EY also provided 2017 call center data that included an estimate of customer throughput (11,000 customers per year from the first state: State-A; 4000 from the second state: State-B) and overall customer service time. State-A and State-B each require two FTEs to complete these tasks. Employees from the EUC call center are paid an average of $\$5000$ per month. The cost to dispatch an electrical damage repair team is $\$60$ per day in State-A and $\$120$ per day in State-B. Some of the important statistics on customer service time for each service and the percentage of customers using each service are not yet known, but in many cases estimates have been provided.

4.3. Identify and Select RPA Processes

Most of the processes in this flowchart will be affected by the adoption of this RPA process to optimize business operations and reduce costs. The number of employees needed to complete the process will likely decrease for almost all services. For example, determining whether a customer needs service and estimating the damage and validity of the claim will be replaced largely, but some employees should remain for exceptional cases. The number of teams dispatched to inspect and repair damages for customers will decrease by $20\%$, $40\%$ and $60\%$ as a result of automation (Table 4).

4.4. Model Design

The model in this paper was developed using the discrete event simulation software Anylogic 8.0. Queues and delays were assigned to services that involve interaction between customers and employees. Delays were assigned to services that involve only system checks by computer software. The flowchart designed by the EUC and EY was used to structure this model, and statistics provided by the EUC and EY were inserted into the appropriate services to calculate predictions. The statistics for State-B and EY were calculated separately to account for the different service costs, service times and service rates of customers. The simulation ran until the relevant metrics reached equilibrium so that the mean approximation could be used to derive results comparable to the predictions in EUC’s scenarios from Table 4.

Data provided by EUC and EY were averaged over thousands of callers. In this work, a triangular distribution was used to model delay times for all services because the only inputs required were the mode and range of possible outcomes. For example, EUC reported an average service time for performing a price check after the customer estimated the cost of damages to be one hour. This value was used as the mode for this service and lower and upper bounds of 30 min and 2 h were set around this value. These values can be changed in future models to optimize results or reflect new information from the EUC.

The equations used to derive the cost savings of the three scenarios outlined by the EUC establish a relationship between the cost savings from reducing the number of FTEs and the number of surveys that must be sent out to inspect electrical damage. The equation used to calculate employee cost per unit time for the State-A and State-B models is given by Equation (1).

$$E_C=N{U}_w·S·t_T$$

(1)

Here, $E_C$ is the cost of employees per unit time, N is the number of employees, $U_w$ is the utilization of those employees, $t_T$ is the average time spent on work tasks and S is the salary paid to employees over time $t_T$.

The cost of paying workers to investigate the electrical damage claims of customers can be calculated using Equation (2).

$$F_C=C·S·t_S$$

(2)

where $F_C$ is the cost of paying the teams from State-A or State-B, C is the number of customers served by the workers, $t_S$ is the average time spent on the work tasks and S is the salary paid to the work teams over time $t_S$.

4.5. Analyze and Interpret Results

Since model optimization is an iterative process, this step will primarily serve to lay the groundwork for future models. The first model will provide guidance on which system metrics need to be changed and what data are needed from the EUC and EY. The data gathering and analysis process for this project is ongoing. EUC and EY are still negotiating the steps they will take to reduce costs and are gathering information about their call center infrastructure and the software and hardware that would be required to replace these services. The current model will essentially show whether the methodology in this work can be predictive for the complex processes used within EUC.

The data used for this first stage are from EUC during 2017 and 2018 on business characteristics, available services, customer behavior and infrastructure. In customer services, through the UCI call center, there are 263 thousand incoming calls/month, $55\%$ in Sao Paulo (State-A) and $45\%$ in Espirito Santo (State-B), of which $96\%$ of calls are answered. Service is also provided through 22 social media attendants (Facebook, Twitter, Instagram, Claim Here, Contact Us, EDP Online, SMS, and others). Monthly online media calls are Tablet Store—8306, Totem—108,546, Web Chat—16,689, Internet—1,245,950, Mobile—546,649, SMS—3570, and others—5767. There are 30 agencies in State-A and 69 agencies in State-B. The most requested services in State-A based on quantity are Shipping/Billing Issue, Change Data Partner, Change/Termination Contract, information, Payment agreement, New connection, reconnections; the most requested services in State-B are Shipping/Billing Issue, Change Partner data, Payment agreement, response change, Review Billing and information. Moreover, other services related to the call center are Electrical Damage Remedy (PID).

5. Result and Discussion

The results of this work were obtained from the first iteration of the model shown in Figure 2 and therefore should not be considered quantitative. These results need to be compared with further analysis performed by EUC and EY and then refined to make clear predictions. Results for the three scenarios described can be found in

Table 5. Predicted reduction in electric reimbursement costs over one year—A is Utilization (%), B is Employee Pay (R$), C is Survey Cost (R$) and D is Total Savings (R$).

	Original		1		2		3
	State-A	State-B	State-A	State-B	State-A	State-B	State-A	State-B
A	84	60	52	45	39	42	34	35
B	120,000	120,000	65,792	57,811	50,326	50,400	42,269	44,720
C	62,594	46,836	50,075	37,469	37,556	28,101	25,738	18,734
D	-	-	62,147	69,430	81,482	75,701	91,605	86,767

.

The EUC and EY predicted that it would take 31, 19 and 16 months, respectively, to make the RPA investments in Scenarios 1, 2 and 3. This model predicts that it will take 46, 35 and 32 months, respectively. These values suggest that the cost savings are significantly underestimated, but one should not place too much weight on the absolute values predicted by this preliminary version of the model. However, the proportion of money saved by State-A and State-B can be used to make predictions that could guide the EUC in their first implementation of RPA. The values of $120,000 per year for the initial employee salary are based on EUC’s estimated average salary data (Table 5). The values for Scenarios 1, 2 and 3 were calculated based on the percentage of hours spent on these tasks by the employees in the back office and the call center.

This model predicts that EUC is likely to save more money if it invests in RPA, particularly in the State-A branch. EUC stated that two FTEs are needed in State-A’s and State-B’s call centers, but State-B’s call center and back office have fewer callers per year. State-B reduces its costs more than State-A only for Scenario 1 because State-B has fewer inspections and repairs of electrical damage due to its higher price ($\$120$ per day). For Scenarios 2 and 3, State-A saves more money overall because employees have to spend less time cuts (Table 5). It should be noted that this difference in savings could be due to State-B employees performing other duties that are not included in this model.

It appears that Scenario 3 is likely to have the highest return on investment according to the model developed in this paper. This does not take into account the difficulties of integrating the RPA hardware and software with other services within this particular process. It may be advantageous for EUC to start using RPA slowly to prevent unforeseen challenges from adding costs. This model predicts that adding RPA to the suite of services offered by State-B will likely save EUC more money. Therefore, a modest introduction of RPA services in both states would likely be the best step for the first iteration.

The practical/professional consequences, according to the results of this study, have a corporate impact. With the insertion of the methodology, decision making for implementing RPAs can increase the assertiveness in its adoption and application. The economic impacts in the distribution will happen through the reduction of costs and minimizing investments. The operational efficiency results obtained can be replicated in other companies in the electricity sector since the same regulations permeate their processes. The decision to implement RPA could be the answer to more complex and costly business problems in the electricity sector, impacting customer satisfaction. Regarding the impact on the workforce, there will be the possibility of experiencing the ideal profile for the new area in practice, re-qualifying the leaders to manage and act with this new digital workforce.

From the socio-environmental point of view, the construction of successful RPAs will reduce the waste of inputs and greater availability of the electrical system to benefit the population. Proper routing of network services could help reduce $C{O}_2$ emissions. On the other hand, the academic consequences have a perspective of the evolution of a methodology for selecting impact activities in energy generation and distribution companies, oriented towards possible robotizations.

Consequently, periodic monitoring and critical analysis of the performance of the technologies is expected through the methodology developed to monitor market innovations, maintaining a minimum alignment of performance. This is also the case supporting the definition of policies, development standards, robot performance monitoring, governance, ethical behavior issues, and digital performance. The limitations and scenarios in which it could not be applicable at a corporate level are related to factors that may prevent its implementation, such as, for example, regulatory changes, the company’s cultural resistance to adopting new technologies, unavailability of corporate systems and the lack of cooperation.

6. Conclusions

This paper presented a method for indicator selection for business services modeling with G1, G2 and G3+ robots. The proposal was created through meetings and studies carried out by USP, EY and EDP teams with data provided by EUC and EY from 2017 and 2018. The method allows to establish scenarios and propose simulations for performance analysis to generate more detailed and accurate models for the different generations of RPA. In terms of sustainability, the method can be adapted for the Management of Information and Communication Technologies (ICT) Projects. The process of selecting indicators for the EUC call center was described, and the results of the first iteration were provided. This method allowed us to make reasonable predictions with a first-generation RPA model based on the direct evaluation.

Compared with other works from the literature, the novelty of this work is a method of indicator selection and RPA performance analysis to guide the evaluation of robots (robots G1, G2 and G3+). The methodology is described within a set of steps called the USP/EY/EDP Performance Analysis Method. The Performance Analysis Method is a cyclical and evolutionary process, allowing, in each of its iterations, to establish increasingly reliable models, indicators and metrics of business processes and computational systems, to support decision making. Iterative optimization of parameters through cyclic testing is sometimes referred to as a continuous evolution method, where each iteration can serve as input to the next set of parameters. The values, variables and system arrangement itself can be changed based on the results of one iteration of this process. The next step in this research will be to gather more detailed data on employee behavior and workflow efficiency to improve the current method of performance analysis. The speed of response of employees and the computer system to customer needs is critical to predicting which services could be replaced by automation.

The generalizability and adaptability of this performance analysis method are its most important characteristics. The process can be adapted to narrow, technical business processes or to comprehensive, company-wide services. The iterative nature of the process allows easy incorporation of data collected on employee behavior, total expenditure and customer satisfaction. In future iterations of the project, we hope to expand the scope to include business-wide processes that could be organized more effectively and what aspects of those processes could be automated.

Theoretically, the word “robot” used in this work means software, not physical robots. The result of the work is the explanation of the criteria suitable to the company for the future selection of robots (continuous scalability). The managerial and practical implications can be created as an “Observatory of Disruptive Technologies”, which will coordinate the development of robots of different generations for key activities of the company. This Observatory is responsible for following up and monitoring technological evolution, creating and monitoring the methodology for aligning and standardizing robotic development in the organization, and developing rules for information security and training, among other related responsibilities.

The selection of which process will be automated with RPA has application in company activities that impact the operation and are relevant to the company. This involves its customers, regulatory bodies and society, covering activities in the commercial areas, operation and asset management, among other possible and applicable areas. The Observatory will provide constant monitoring and evaluation of processes that may impact the company, providing a methodology for selecting activities, monitoring, and critical analysis of the selection performance through the methodology developed. The limitation of the work is that it are based on metrics generation strategies (performance measurements) obtained through simulation models, analytical modeling and measurements of the real system.

In future work, one should consider performance analysis under different orders of magnitude by combining the relevant system performance analysis experiments under different levels of request data volume in real application scenarios. Other simulation techniques could also be applied, for example, the Petri Net model, to simulate the conduct of customers/users in concurrent scenarios. It is also possible to incorporate different machine learning models, such as deep learning, within the methodology for data training and performance comparisons.