Incorporating KPIs in the Quantification of Overall Efficiency of Rolling Stock of Railways ^†

Abstract

The overall efficiency of the rolling stock of railways can be calculated by taking into account three parameters: availability, performance, and quality. The operator and maintainer (O&M) companies of railway engineering systems are contractually bound to deliver KPIs (key performance indicators) associated with the operation and maintenance of systems and sub-systems. O&M-level KPIs have links and overlaps with the three parameters of overall efficiency. The incorporation of KPIs is important for the assessment of the overall efficiency of equipment such as rolling stock (RS). This paper (1) presents modified definitions of the parameters of the overall efficiency of RS and (2) shows a method on how to combine the KPIs of RS of railways for the quantification of three parameters regarding the overall efficiency of RS. Using a new KPI-based method, the overall efficiency of the RS of a metro railway was calculated as 99.07%, which is realistically justifiable and verifiable for transit operators. Application of this method is shown according to a real-world system, and the advantages and limitations of the method are presented.

1. Introduction

The gold standard for measuring productivity is OEE (overall equipment efficiency). It determines the proportion of production time that is actually productive [1]. Measuring OEE can give important insights into how we can systematically improve the process by measuring individual parameters and identifying losses, enabling benchmarking progress and improving equipment productivity. Quantification and improvements in the efficiency of railway transit systems are always significant, due to fixed public funds and ever-increasing travel demands [2]. There exist certain methods to calculate the efficiency and performance of the public transport system or its sub-systems [3,4].

This paper adopts commonly used metrics in the production industry, redefines them, and then calculates them, assuming a train unit to be primary equipment in an urban rail transit system. The application of the OEE approach to the railway industry was first proposed by Mahboob et al. in 2012, wherein the overall efficiency of rolling stock (RS) for an urban railway transit system was calculated [5]. The term rolling stock, in the field of railway transportation, refers to railway vehicles such as locomotives, passenger coaches, and freight carriages. The OEE approach has been used in metro projects worldwide to evaluate the effectiveness of rolling stock. However, considering certain limitations with respect to priorities set by different transport owners, the methodology needs to be re-evaluated, based on the KPIs (key performance indicators) defined for O&M contractors as per the SLA (service level agreement) of a particular transport company. Furthermore, the O&M contractor also has an interest in carefully assessing and improving overall efficiency to avoid business- and service-related losses and extra costs. Therefore, their focus should remain on developing improved methods to assess and compare the parameters of efficiency, reliability, availability, maintainability, and safety of public transport systems [6,7]. Researchers have developed methods for the determination of the profitability and economic efficiency of rolling stock [8,9]. However, no research exists on the quantification of the overall efficiency of rolling stock by incorporating contractual KPIs that are strongly associated with operation and maintenance-related payments. The proposed method is novel and useful in the assessment of operation and maintenance costs and performance.

2. Overall Equipment Efficiency for Production and Urban Rail Transit System

An OEE score of 100% specifies that a company produces only good parts at maximum production capacity, with zero failure times. This translates to 100% quality (only acceptable parts), 100% performance (rapid production arrangements), and 100% availability (no downtime) [1,10]. OEE uses the following three parameters.

2.1. Availability

In a production environment, the ratio of actual run time to planned production time is used to calculate availability. Availability calculations include availability loss, which includes any events that disrupt planned production schedules for an extended period of time.

$$Availability={\displaystyle \frac{Total time available-Downtime}{Total time available}}\times 100\%$$

(1)

Unplanned interruptions (such as faulty equipment or shortages of raw material) and planned interruptions (such as transition time) are examples of things that cause availability loss. OEE analysis includes the transition time because this is time that could otherwise be used for good-quality production. While it is impossible to eliminate the transition time, it can be significantly reduced. After subtracting availability loss, the remaining time is known as run time.

2.2. Performance

Performance loss accounts for anything that causes the production process to run at less than the optimal design speed. Running at both slow cycles and with small stoppages is factored into OEE performance.

$$Performance={\displaystyle \frac{No. of units produced }{Total Possible number of units }}\times 100\%$$

(2)

Machine wear, substandard materials, poor operation and maintenance standards, misfeeds, and jams are all examples of things that cause performance loss. Net run time is the time remaining after subtracting performance loss.

2.3. Quality

Quality loss, which accounts for produced parts that do not meet the agreed quality standards, is factored into Quality.

$$Quality={\displaystyle \frac{No. of units produced-defective units}{Number of unit produced}}\times 100\%$$

(3)

Units that need to be reworked are examples of issues causing quality loss. Quality defines acceptable parts as parts that have successfully passed through the manufacturing process for the first time without requiring any reworking [1,11]. For further reading on the theory and application of OEE, readers are referred to Refs. [10,12,13]. The research conducted by Mahboob et al. in 2012 acknowledges the importance of avoiding the inter-relation of parameters to prevent redundancy in measurement, which can lead to incorrect conclusions [5]. However, the definitions of the parameters provided in the paper overlap. Earlier work defines the three OEE parameters for rolling stock as follows [5].

Availability corresponds to the number of train units (TUs) that are available for passenger service each day for mainline operations. When a train unit remains in the maintenance workshop for extra (i.e., other than planned) inspections, fault rectification, and checkups and, therefore, cannot be put into passenger service, this is referred to as an operational availability problem with the train unit. Performance of the RS is defined in reference to its required output for mainline operations. One of the key outputs of a train unit is the number of kilometers covered per unit of time. Higher kilometers per time refers to more travel places being visited per unit of time. A slower train unit will offer fewer spaces per unit of time. Quality of the RS can be defined as the ratio of the train sets that are actually working during mainline operations without facing any significant failure and the total number of good train units put into service.

2.4. Parameter Dependency

In order to obtain reliable results, the parameters must be independently defined; however, the performance parameter is dependent on the availability since, by tying train performance with train speed and the speed with kilometers traveled, it is inadvertently linked with availability, since more trains directly on the main line means that more kilometers are traveled. Similarly, the quality yield considers the time that the RS is in good condition (which also affects its availability); this creates a redundancy in measurement. Two of the parameters are dependent on the third; therefore, an accurate determination of OEE cannot be made. The dependency among the three parameters of the OEE increases if the KPIs associated with each parameter have shared effects among the OEE parameters.

3. Revisiting the Definitions of the Overall Equipment Efficiency of Rolling Stock

This paper aims to modify and improve the definition of the parameters of OEE for better applicability to metro transit projects, based on KPIs. Additionally, KPIs are key performance indicators that are set out in the service level agreement (SLA) by the employer, to quantify the performance of O&M contractors. These KPIs are based on factors that can affect the service standard. Some KPIs set down requirements for the availability of equipment, some concern performance, while others concern the quality of service. For this study of the Orange Line metro rail transit system (OLMRTS), the incorporation of KPIs in evaluating the OEE of rolling stock will be considered.

In OLMRTS, a total of 48 KPIs are related to the rolling stock department; out of these, 24 are related to the availability, performance, and quality of rolling stock (refer to

Table 1. Mapping of KPIs against OEE parameters.

Sr.#	KPI	Availability	Performance	Quality
1	First train two-way punctuality		Yes
2	Journey time regularity		Yes
3	Trip efficiency		Yes
4	Minor breakdowns (train reliability)		Yes	Yes
5	Major breakdowns (train reliability)	Yes	Yes	Yes
6	Train cleanliness			Yes
7	Poor-quality components or equipment			Yes
8	Not complying with maintenance and operation standards	Yes	Yes	Yes
9	Modification in the design or paintwork of the train			Yes
10	Damage, scratches, and discoloration			Yes
11	Damaged seats			Yes
12	Damaged armrest			Yes
13	Missing or broken gripper			Yes
14	Damaged window			Yes
15	Lack of lighting or incorrect layout			Yes
16	Damaged passenger access door			Yes
17	Manual door opening			Yes
18	Broadcast issue			Yes
19	Failure of fire extinguisher	Yes		Yes
20	Missing glass breaker			Yes
21	Air-conditioning system	Yes		Yes
22	Removing stagnant trains			Yes
23	Noise pollution			Yes
24	Lubricant leakages		Yes	Yes

). Each KPI is defined in the service contracts of the employers. For example, according to the employer of the Orange Line metro train system (OLMRTS) of Lahore, trip efficiency (Sr. # 03 in Table 1) is defined as the ratio between actual completed trips and total planned trips. For example, the daily planned trips of a metro line number 290, while the actually realized trips on a specific day number 285. In this way, the daily trip efficiency becomes 98.27%. Elaboration of the definitions of each KPI of Table 1 is beyond the scope of this paper. For detailed definitions of the KPIs, readers are referred to the service contract [14]. Table 1 presents only 24 KPIs that are relevant to this research work. Other KPIs are relevant to staff service standards or performance and do not affect the OEE parameters of the RS in terms of either maintenance or operational considerations.

Table 1, below, maps the relationship of different KPIs with the components of OEE defined earlier. When calculating the OEE based on KPIs, this mapping provides an understanding of and a basis for calculating individual parameters such as availability, performance, and quality for the particular RS of a metro railway. Using past data on KPI effects, experts from the O&M of RS have mapped the KPIs with the three parameters of the OEE. The overall efficiency of the RS has been calculated using the principles of OEE. All three parameters’ availability, performance, and quality yield have been redefined according to the equipment, i.e., RS, in order to establish the effects of the KPIs on each parameter of the OEE.

3.1. Availability

To accurately assess the availability of rolling stock, the concepts of technical availability and operational availability should be considered. (After assessment and as per contractual requirements, concerned authorities or research groups can decide on the utilization of either concept or the combined use of both concepts.) Technical availability corresponds to the actual number of unplanned breakdowns of TUs in relation to the total number of TUs in the fleet. This approach will provide details on actual technical availability, but it will also result in increased utilization of the resources that are actually required to manage the operation, thus increasing the overall running costs of the rail transit operation.

Depending upon the actual operational requirements of the TUs of a particular URTS, operational availability can be calculated based on the maximum required TUs for operation, as per the KPIs and SLA. Overall fleet size is not considered in this approach. In this article, the relationship between KPIs and OEE is being established; therefore, only operational availability will be considered, and moving on it will be referred to as simply “availability” in this work. The Orange Line metro rail transit system (OLMRTS) of Lahore comprises 27 train units in total. However, the daily operation requirements as per the operation plan vary, based on passenger service requirements throughout the week. Furthermore, the requirements vary according to other factors, like special operation orders for public holidays. For example, on normal days (from Monday to Saturday), the total number of TUs required is 20, and 2 spare TUs are required to cover the unexpected departure of trains to the main line. At weekends (only on Sundays), the total number of TUs required is 22, and 2 spare TUs are required to cover the unexpected departure of trains to the main line. Therefore, if any TU out of these 22 TUs is not available, the availability of the RS can be affected, given that there are replacement requirements for TUs during mainline operations. Except for these two requirements, there are also other factors that impact the number of TUs required for mainline operation. For example, the number of TUs required on different public holidays is different. Considering the operation requirements, we will evaluate the availability of the RS as the ratio between the total number of TUs available and the number of TUs required for daily operation.

$$Availabilty={\displaystyle \frac{No. of TUs available for operation}{No. of TUs required for operation}}\times 100\mathrm{\%}$$

(4)

A = Availability (%)

TU_{a, d} = Number of train units available for operation on “d” days of the year.

TU_{r, d} = Number of train units required for operation on “d” days of the year.

D = 1, …, 365 (all the days of a complete year)

$$A=\sum _{d=1}^D\left({\displaystyle \frac{{TU}_{a,d}}{{TU}_{r,d}}}\right)\times 100\%$$

(5)

Direct quantification is possible regarding the availability of RS, which already encompasses all the contractual requirements; thus, individual KPI integration is automatically achieved with the above approach. Availability-related KPIs are not directly associated with mainline operations, given that spare trains are provided on a timely basis in case of a faulty TU. In addition, it should be considered that the quantification of OEE will become increasingly challenging when there are requirements to consider technical availability with the KPIs that directly affect technical availability, while fleet size remains constant and the train unit’s operational age also increases over time, causing an increase in unplanned downtime.

3.2. Performance

Performance is calculated by the output of rolling stock on the main line. This output is measured in terms of the number of trips it performs on the main line. By completing the trip requirement on the main line, the RS will increase its output and, consequently, its performance. Similarly, if the RS is running more slowly due to faults caused by technical or non-technical reasons (e.g., due to any reason related to the train operator), this will reduce the output of the TUs, which results in missed trips and delayed trips, ultimately leading to low performance. Thus, the performance is defined as the ratio between the total number of trips completed on time and the number of trips planned.

$$\mathrm{P}\mathrm{e}\mathrm{r}\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{a}\mathrm{n}\mathrm{c}\mathrm{e}={\displaystyle \frac{\mathrm{P}\mathrm{l}\mathrm{a}\mathrm{n}\mathrm{n}\mathrm{e}\mathrm{d} \mathrm{T}\mathrm{r}\mathrm{i}\mathrm{p}\mathrm{s}-(\mathrm{D}\mathrm{e}\mathrm{l}\mathrm{a}\mathrm{y}\mathrm{e}\mathrm{d} \mathrm{T}\mathrm{r}\mathrm{i}\mathrm{p}\mathrm{s}+\mathrm{M}\mathrm{i}\mathrm{s}\mathrm{s}\mathrm{e}\mathrm{d} \mathrm{T}\mathrm{r}\mathrm{i}\mathrm{p}\mathrm{s})}{\mathrm{P}\mathrm{l}\mathrm{a}\mathrm{n}\mathrm{n}\mathrm{e}\mathrm{d} \mathrm{T}\mathrm{r}\mathrm{i}\mathrm{p}\mathrm{s}}}\times 100\mathrm{\%}$$

(6)

P = Performance (%)

T_p,d = Total planned trips on day “d”

T_KPI1,d = Trips where KPI 1 (First Train Two-way Punctuality) was not met on day “d”

T_KPI2,d = Trips where KPI 2 (Journey Time Regularity) was not met on day “d”

T_KPI3,d = Trips where KPI 3 (Trip Efficiency) was not met on day “d”

T_KPI4,d = Trips where KPI 4 (Minor Breakdown) was not met on day “d”

T_KPI5,d = Trips where KPI 5 (Major Breakdown) was not met on day “d”

D = 1, …, 365 (all days of a complete year)

$$P=\sum _{d=1}^D\left(1-{\displaystyle \frac{(T_{KPI1, d}+T_{KPI2, d}+T_{KPI3, d}+T_{KPI4, d}+T_{KPI5, d})}{T_{p,d}}}\right)\times 100\%$$

(7)

As shown in Table 1, some KPIs are related to more than one parameter. In order to avoid overlap and dependency, the remaining KPIs will be considered in the quality parameters.

3.3. Quality Yield of Rolling Stock

Quality is related to user experience or to the caliber of service provided to the passenger by the RS through the TUs. Therefore, any faults of the TUs that result in an inferior user experience will affect the quality yield. These can be monitored by their impact on the KPIs of the service level agreement. Therefore, quality is the proportion of total services or trips completed by the RS that meet predefined service-related key performance indicators (KPIs), which are specific, measurable aspects of service performance and customer satisfaction.

$$\mathrm{Q}\mathrm{u}\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}={\displaystyle \frac{\mathrm{N}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{t}\mathrm{r}\mathrm{i}\mathrm{p}\mathrm{s} \mathrm{m}\mathrm{e}\mathrm{e}\mathrm{t}\mathrm{i}\mathrm{n}\mathrm{g} \mathrm{K}\mathrm{P}\mathrm{I}\mathrm{s}}{\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{t}\mathrm{r}\mathrm{i}\mathrm{p}\mathrm{s} \mathrm{c}\mathrm{o}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e}\mathrm{t}\mathrm{e}\mathrm{d}}}\times 100\mathrm{\%}$$

(8)

Q = Quality (%)

T_{x, d} = Number of trips where KPI “x” was not met on day “d”, where term “x” corresponds to the KPIs related to quality in Table 1.

T_{c, d} = Total number of trips completed on day “d”

$$Q=\sum _{d=1}^D\left(1-{\displaystyle \frac{(T_{KPI6, d}+T_{KPI7, d}+T_{KPI8, d}+\dots +T_{KPI23, d}+T_{KPI24, d})}{T_{c,d}}}\right)\times 100\%$$

(9)

Considering the KPI definitions and the rectification time allowed for quality-related KPIs as per the SLA, generally, if the quality issue is resolved within the allowed time, then its effect on OEE may not be considered, as per the requirements, and any trip with such an issue should be included in the “Number of trips meeting KPIs”. Finally, the overall efficiency of RS becomes:

$$OEE=A\times P\times Q.$$

(10)

4. OEE of Rolling Stock of OLMRTS

To demonstrate the implementation of the parameters laid down in this paper, the OEE of the rolling stock in OLMRTS was calculated for the year 2023. The RS department maintains accurate data on the KPIs of the RS on an hourly basis. All the factual data of the KPIs in 2023 have been analyzed and utilized in the quantification of the OEE of RS. To restrict the length of this paper, the daily, weekly, and monthly achievements of KPIs are not shown here. The application of the method is shown through the yearly data.

4.1. Availability of Rolling Stock of OLMRTS

In 2023, OLMRTS fulfilled its mainline operation requirements, including the spare trains needed for operational adjustments due to various factors such as weather and special operation plans for public holidays. Overall, 8122 trains were required for mainline operations in 2023, and all the trains were made available for the time they were required on each day. Against the 8122 trains demanded for operational requirements in 2023, 9290 trains were planned to technically be available in 2023, with reference to the planned technical availability hours of each TU, as described in Figure 1, shown below. Therefore, 8122 trains correspond to an 87% (operational) availability requirement, considering that 9290 trains were planned to technically be available after considering planned maintenance activities. The rolling stock department of OLMRTS manages the technical availability of TUs by considering the actual availability of individual TUs in hours against the planned availability hours for each of the 27 TUs. These availability data are maintained on a daily basis and set against plans developed on a monthly basis. Planned availability time is calculated while considering the planned maintenance time required for each TU, based on the inspection data, maintenance history, and OEM recommendations. Figure 1, shown below, further describes the actual technical availability of each TU in hours for the year 2023, whereas Figure 2, shown later, describes the technical availability in percentage terms.

By considering the technical availability of trains in 2023, as explained in Figure 1 and Figure 2 above, the average technical availability of the OLMRTS TU fleet corresponds to 93.61%. Since the required daily availability of TUs is 86% for operational needs (derived from the daily timetable), whereas the actual daily technical availability of TUs is 93.16%, therefore, to meet a demand for 8122 TUs, 8122 TUs were made available in 2023, ensuring 100% operational availability for mainline operation. The Availability (A) of RS is calculated using the relationship in Equation (5):

$$A={\displaystyle \frac{8122}{8122}}\times 100\mathrm{\%}=100\mathrm{\%}$$

4.2. Performance of Rolling Stock of OLMRTS

Using the data for 2023, the performance (P) of the RS is calculated using the total trips planned against the total trips completed without a delay. In 2023, the trip efficiency and trip regularity-related KPIs (Sr. # 2, 3 of Table 1) were affected for 893 trips. Therefore, performance is calculated by putting the values extracted from Figure 3 and Figure 4 into the relationship in Equation (6):

$$\mathrm{P}\mathrm{e}\mathrm{r}\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{a}\mathrm{n}\mathrm{c}\mathrm{e}={\displaystyle \frac{105958-(837+56)}{105958}}\times 100\mathrm{\%}={\displaystyle \frac{105065}{105958}}\times 100\mathrm{\%}=99.15\mathrm{\%}$$

The total performance of the rolling stock for 2023 is 99.15%.

The above calculations use data on delayed trips and missed trips in 2023. These data also include issues caused due to several operational and technical reasons that were not directly attributable to RS. These include delayed trips and missed trips due to adverse weather conditions, and, depending upon inquiry and assessment, they may be exempt from penalization, as per the KPIs’ scope and the SLA. For simplification and to emphasize the importance of overall performance, all delayed trips and missed trips are considered in this calculation. Researchers can include or exclude the KPIs as per their requirements and contracts.

4.3. Quality of Rolling Stock of OLMRTS

As per the definition of quality discussed earlier, in total, 72 trips were affected by quality-related KPIs regarding rolling stock in 2023; thus, 99.93% service quality was achieved, as explained through these calculations and in Figure 5, shown below.

The quality of the RS of OLMRTS is calculated using the relationship in Equation (8):

$$\mathrm{Q}\mathrm{u}\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}={\displaystyle \frac{105830}{105902}}\times 100\mathrm{\%}=99.93\mathrm{\%}$$

It is important to note that in this parameter, to avoid redundancy and overlap, the KPIs related to train punctuality have not been included as these are already taken into account in the calculation for performance. Using the relationship in Equation (10), the OEE of the RS is calculated in the following way:

OEE = 100% × 99.14% × 99.93% = 99.07%

An OEE of 99.07% represents the world-class performance of the rolling stock of a metro rail transit system.

5. Conclusions

This research demonstrates the quantification of the OEE of the rolling stock of metro systems, which is primarily based on the KPI-based definitions of three parameters of OEE. Higher results for the availability (100%), performance (99.14%), and quality (99.93%) of the RS of OLMRTS are due to the fact that the train units are relatively new and their maintenance, operation, and performance meet the highest urban rail transit-related standards. Robust measures and strong controls have been implemented to achieve contractual KPIs.

Further improvements in the scope of the definitions presented in this research can be made. Availability is defined here in terms of operational requirements. Moving forward, this can be calculated in terms of overall equipment availability, considering both the planned and unplanned downtime of RS. Performance can be influenced by various factors, ranging from internal system failures to external conditions beyond the control of the operator. These factors include system failures within the rolling stock, adverse weather conditions, and external disruptions such as road strikes leading to the closure of operations. In this research, quality KPIs have been considered directly; however, since the impact of each KPI is different and penalization (by the system owner) is different, future works can consider incorporating the weighting of each KPI to more accurately represent its impact on performance. For example, a safety-related KPI that measures passenger injuries is more critical for service quality than a KPI that measures faded paint of saloon car-related incidents. This weightage can be taken directly from the penalizations laid out in the service contracts. The method presented in this work is not limited to the rolling stock and it can be applied to other systems and sub-systems of railways and their KPIs, such as signaling, communication, power supply, platform screen doors, civil engineering, and tracks and depot equipment. The OEE of a complete railway system can be quantified and assessed for use in different decisions. Once finalized, this method can be adopted, after modifications, for the comparison of the service contracts of metro systems worldwide.