UNC Charlotte Autonomous Shuttle Pilot Study: An Assessment of Operational Performance, Reliability, and Challenges

Abstract

This paper presents the findings from an autonomous shuttle pilot program conducted at the University of North Carolina at Charlotte between June and December 2023 as part of the North Carolina Department of Transportation’s Connected Autonomous Shuttle Supporting Innovation (CASSI) initiative. The shuttle completed 825 trips, transporting 565 passengers along a 2.2-mile mixed-traffic campus route. The study evaluates the shuttle’s operational performance, reliability, and challenges using data from onboard sensors, system logs, and operator reports. Key analyses are divided into four areas: service reliability, which assesses autonomy disengagements caused by signal loss, technical issues, and environmental factors; service robustness, focusing on the shuttle’s ability to maintain operations under adverse conditions; performance metrics, including average speed, autonomy percentage, and battery usage; and service usage, which examines the number of trips and passengers to gauge efficiency. Signal loss and battery-related issues were the primary causes of service interruptions, while environmental factors like weather and vegetation also affected shuttle performance. Recommendations include enhancing vehicle-to-infrastructure communication and optimizing battery management.

1. Introduction

Advancements in driverless technology have enabled the deployment of automated shuttles to improve mobility, especially in low-ridership areas and for first- and last-mile connectivity [1,2,3,4,5]. Commonly used for short trips in controlled environments like recreational parks, business districts, and university campuses [2,5,6], these shuttles effectively bridge gaps in public transit. Typically operating along predefined routes, they reach speeds of 9.3 to 28 mph (15 to 45 km/h) and carry 8 to 15 passengers [7]. With battery capacities ranging from 20 to 33 kWh, these shuttles offer operational times between 9 and 12 h, making them well-suited for small-scale transit applications [8].

According to the Society of Automotive Engineers (SAE), automated shuttles are classified as Level 4 vehicles, capable of full automation without human intervention under specific conditions [9]. However, despite being designed for Level 4 automation, these shuttles often operate at Level 3 in practice. While they possess high automation capabilities, they may still require a human safety operator or attendant to handle certain situations. For instance, the Navya Autonom Shuttle is equipped to function autonomously under ideal conditions but frequently operates under human oversight via an industrial controller and touch screens. This gap between intended design and actual operation underscores the complexities of deploying automated shuttles in real-world environments [10]. These complexities also extend to the broader ecosystem of stakeholders involved in the deployment and operation of automated shuttles. To gain public acceptance, road users, including commuters, pedestrians, and other drivers, must interact safely and comfortably with this new technology. On the other hand, shuttle designers focus on enhancing the technology to ensure it operates efficiently and safely. Finally, public organizations, such as transportation departments and transit agencies, are responsible for navigating the legal and regulatory landscape to ensure that shuttle deployments meet safety standards and local requirements [11,12,13,14].

The successful implementation of automated shuttles depends on stakeholder collaboration and where they are deployed. The environment—whether it is a neighborhood, campus, business park, hospital, or recreational area—plays a critical role in determining the shuttle’s effectiveness. Additionally, operational characteristics such as passenger capacity, speed, safety incidents, and user comfort can influence the overall success of these deployments. Road infrastructure, including the number of lanes, speed limits, traffic signals, and crosswalks, must also be considered when planning for automated shuttle integration. To this end, various Autonomous Shuttle (AS) pilots have been conducted to address a wide range of transportation challenges, including improving first- and last-mile transit gaps; navigating complex urban environments; providing mobility in rural and underserved areas; and testing the scalability, public acceptance, and regulatory frameworks required for the successful integration of AS technology into diverse transportation systems. These studies also explored the impact of ASs on specialized populations and showcased different collaborative and funding models while emphasizing the environmental and operational contexts in which AS technology must adapt [4,15,16,17,18,19,20,21,22,23,24]

For example, the city of Calgary launched a 2018 pilot connecting the Calgary Zoo LRT Station and Telus Spark Science Centre to evaluate public acceptance and AS integration into the city’s transit system [15]. This success led to a larger project in Beaumont, Alberta, where shuttles operated on mixed-traffic routes, testing navigation through intersections and interactions with vehicles and pedestrians [16]. Similarly, in Shenzhen, China, the Alphaba Smart Bus Demonstration targeted first- and last-mile gaps, deploying autonomous buses to address underserved areas [17]. Urban pilots have also played a significant role in testing AS technology. In Montreal, Quebec, two EasyMile shuttles initially ran isolated routes but expanded to connect with metro stations, navigating high-density urban roads and complex traffic environments [4]. Singapore’s three-month trial in Sentosa explored on-demand transit using a mobile app, while the WEpod Project in the Netherlands tested the scalability of both on-demand and fixed-route services [18,19]. In contrast, rural pilots like those in Nishikata, Japan, provided essential transport for elderly residents in underserved regions, while Perth, Australia’s RAC Intellibus pilot, offered insights into urban and semi-urban applications [20,21]. These projects have varied in scope and duration, ranging from short-term pilots like the Milo Project in Texas, which connected remote parking lots to event venues [22], to large-scale initiatives like the four-year CityMobil2 project in Europe, which tested AS technology across multiple cities and helped establish critical legal and technical frameworks for AS integration [23]. Across these studies, regulatory and legal frameworks were explored, with pilots like Candiac and Shenzhen testing shuttles on public roads under real traffic conditions, while controlled environments like Mcity at the University of Michigan and Haneda Airport provided safer testing grounds before transitioning to public roads [24,25].

Target populations for AS services varied as well. While pilots in Beaumont and Montreal engaged the general public to gauge attitudes toward autonomous vehicles, others, such as Nishikata’s focus on elderly residents and the WEpod project’s work with university communities, explored how AS technology could be adapted to meet specific needs [19,20]. The pilots demonstrated different funding and collaborative models. Public investment was central to projects like CityMobil2 and RAC Intellibus, which emphasized building regulatory frameworks and public trust, while industry-driven efforts in Europe and China highlighted the role of private investment in advancing commercialization and scalability [17,21,23]. Environmental factors, from simpler traffic conditions in rural and low-density areas to more complex urban environments, shape the challenges and opportunities of AS deployment [4,18,22].

To demonstrate the capabilities of connected and automated shuttles, the North Carolina Department of Transportation’s Connected Autonomous Shuttle Supporting Innovation (NCDOT-CASSI) program has piloted two different models of all-electric, low-speed automated shuttles across five projects [26]. NCDOT partnered with EasyMile to deploy their EZ10 Gen 3 shuttle in three pilots: a two-day demonstration at the N.C. Transportation Summit in 2020, a three-week deployment at Raleigh’s Centennial Campus, and a 13-week deployment at the Wright Brothers National Memorial. Additionally, NCDOT worked with Beep to test Navya’s Autonom shuttle in two longer projects: a 13-week deployment at Cary’s Bond Metro Park in 2023 and a 23-week deployment at UNC Charlotte. Our study, part of the UNC Charlotte pilot, focused on one of the most complex routes (2.2 miles/3.54 km) tested in the NCDOT-CASSI program. This route traversed heterogeneous traffic conditions, including private vehicles, buses, pedestrians, bicyclists, e-scooters, and golf carts, and it incorporated four traffic signals. It connected key parts of the campus, including residential buildings, academic spaces, the Student Union, and recreational areas, forming a core part of the campus transportation network. The UNC Charlotte pilot was the longest (six months) in North Carolina from June to December 2023. It covered various weather conditions— from summer heat to winter sleet—and provided valuable insights into shuttle performance under diverse visibility and traffic conditions. This paper focuses on evaluating the technological performance of the automated shuttle, with an emphasis on assessing its operational efficacy and identifying key challenges.

This paper is organized as follows: Section 2 presents the Operational Overview and Data Methodology, detailing the pilot study’s route, shuttle specifications, and data collection processes. Section 3 covers the Analysis and Results, focusing on key performance metrics such as autonomy disengagements, service reliability, battery usage, and operational efficiency. Section 4 concludes the paper with Recommendations to improve shuttle performance through targeted enhancements in signal management, battery optimization, and system robustness in complex traffic environments.

2. Operational Overview and Data Methodology

The NCDOT’s Integrated Mobility Division (IMD), in collaboration with UNC Charlotte and Beep, an autonomous shuttle company based in Florida, launched a pilot study through the CASSI program to assess the performance of automated shuttles in North Carolina [27]. From June to December 2023, an autonomous shuttle operated by Beep was deployed on the UNC Charlotte campus to evaluate its functionality and integration in a real-world environment.

2.1. Route Description

Figure 1 illustrates the route selected for the pilot study at UNC Charlotte. The route, shown in blue, spans 2.2 miles (3.54 km) and traverses a diverse traffic environment, sharing the road with private vehicles, buses, pedestrians, bicyclists, skateboarders, e-scooters, and golf carts. The shuttle operates along this route with seven designated stops in the following order: Greek Village 1 → Greek Village 8 → Greek Village 4 → Science Building → Student Union West → Student Decks → Light Rail East. These stops connect key residential, academic, and recreational areas on campus. Throughout the route, there are four traffic-light-controlled intersections and three stop sign intersections. These intersections represent critical points where the shuttle must navigate through mixed traffic while adhering to traffic regulations to ensure passenger safety. The route itself is a single lane with a speed limit of 20 miles per hour. The green line in Figure 1 indicates the path the shuttle takes to access the designated parking area, where it stays overnight and recharges in between morning and afternoon operational sessions.

2.2. Shuttle’s Specifications

For this study, Beep operated a Navya Autonom^® shuttle. The shuttle measures 15.68 feet in length, 6.89 feet in width, and 8.76 feet in height, providing a compact yet spacious design suitable for campus transportation and a capacity for 15 passengers. While the shuttle can reach a maximum speed of 45 mph (72 km/h), it was configured to operate at a maximum speed of 12 mph for this pilot study to comply with the 20 mph speed limit on the UNC Charlotte campus and to ensure safe operation in the complex, mixed-traffic environment, which included pedestrians and various forms of transportation. The shuttle’s empty weight is 5732 lbs, with a gross weight of 7716 lbs. The shuttle is an electric vehicle powered by a battery with a capacity of 33 kWh, which can be charged in two configurations: in 4 h using a 7.2 kW plug or in 9 h with a 3.6 kW plug. In addition, the shuttle is equipped with an array of advanced onboard sensors, including a Global Navigation Satellite System (GNSS), cameras, Light Detection and Ranging (LiDAR), odometry sensors, and an Inertial Measurement Unit (IMU) [28,29]. These sensors and their locations on the shuttle are shown in Figure 2. To ensure safety, the shuttle is managed by an onboard safety operator who can manually activate or deactivate the autonomous system as necessary.

2.3. Data Collection

This paper focuses on evaluating the technological performance of the automated shuttle, with an emphasis on assessing its operational efficacy and identifying key challenges. To achieve this, we analyzed data provided by Beep, gathered from the shuttle’s onboard sensors and reports from its operator. This dataset includes sensor logs, performance metrics, and fault diagnosis, providing a comprehensive overview of the shuttle’s daily operations.

To assess the shuttle’s efficacy and pinpoint key challenges, the collected data was divided into two primary categories: autonomy disengagement data and operational efficiency data. This categorization allowed us to evaluate the shuttle’s reliability and safety, as well as its operational performance, helping to identify both strengths and areas for improvement. Beep provided daily reports, and in cases of multiple disengagements, the specific hours during which these disengagements occurred were also documented.

2.3.1. Autonomy Disengagement Data Collection

The autonomy disengagement data focus on instances where the shuttle’s autonomous system was manually disengaged or experienced errors. These data encompass detailed incident information, such as the date and time of the incident, the week of the year, the number of weeks in the pilot program, the route, latitude and longitude of the incident, weather conditions, and vehicle speed at the time of the incident. Furthermore, the data outline the various causes that led to the disengagement of the automated shuttle. By examining these disengagements, these data serve as a valuable tool to pinpoint the challenges that prevent the shuttle from operating autonomously, offering key insights into its service robustness under different conditions. To better understand these disengagements, we classified the data into four main categories based on the identified causes: technical and signal issues, safety and interaction with other road users, environmental factors, and navigation and path deviations. Each of these categories was further divided into subcategories, as described below:

• Technical and Signal Issues refers to disengagements resulting from technical faults or signal problems. The specific causes of disengagement within this group are as follows:

  –

  Fault Code/Error Code: The shuttle encountered a fault or error code that prevented autonomous operation, prompting the shuttle operator to take manual control to navigate to the next safe stop location.

  –

  Signal Loss: The shuttle lost its 3G/GNSS/RTK signal, making it unable to continue in automated mode. The shuttle operator navigated manually until the signal was restored, then resumed automated mode.

  –

  Signalized Intersection: The shuttle operator manually navigated through a signalized intersection to avoid stopping at a green light and proceeded to the next safe stop location for troubleshooting.

• Safety and Interaction with Other Road Users refers to disengagements related to safety concerns and interactions with other road users, emphasizing the need for careful navigation in dynamic and unpredictable environments. The specific causes of disengagement within this group are as follows:

  –

  Other Road Users: A nearby vehicle was detected as an obstacle, causing the shuttle to switch to manual mode. The shuttle’s operator manually navigated around the vehicle and then returned to automated mode.

  –

  Blocked Station: The shuttle station was blocked, preventing autonomous operation into or out of the designated stop. The shuttle operator navigated manually to stop and then resumed automated mode.

  –

  Vulnerable Road Users: This includes cases involving children, elderly, or disabled individuals where manual control was necessary for safety.

  –

  Safety In/Out: This refers to situations where the shuttle encountered issues either while starting or completing its stop at a station. For instance, a “Safety Out” disengagement would occur if the shuttle faced challenges while departing from a station. Conversely, a “Safety In” disengagement would be recorded if there were difficulties when the shuttle arrived and stopped at a station.

• Environmental Factors refers to disengagements due to environmental conditions that affected the shuttle’s ability to operate autonomously. The specific causes of disengagement within this group are as follows:

  –

  Vegetation: Vegetation obstructed the shuttle’s path, preventing automated operation.

  –

  Obstacle Detection: An obstacle was detected in the shuttle’s path, preventing autonomous operation. The shuttle operator navigated around the obstacle in manual mode and then returned to automated mode.

  –

  Weather Conditions: Weather-related disengagements occur under adverse conditions like heavy rain or fog, which compromise the vehicle’s sensors and autonomous functions.

• Navigation and Path Deviation refers to disengagements caused by navigation errors or deviations from the approved route. The specific causes of disengagement within this group are as follows:

  –

  Priority Zone. An object, pedestrian, or vehicle was detected within a priority zone, halting autonomous operation. The shuttle operator manually navigated around the obstacle and then returned to automated mode.

  –

  Shuttle Manually Deviated from Approved Path: The shuttle operator manually operated the shuttle outside the National Highway Traffic Safety Administration (NHTSA) approved path.

2.3.2. Operational Efficiency Data Collection

The operational efficiency data include key metrics crucial for assessing both the efficiency and reliability of the automated shuttle during routine operations. The main metrics are categorized as follows:

• Service Reliability refers to the shuttle’s ability to operate according to its schedule. The key metrics used to evaluate service reliability include the following:

  –

  Suspension of Service captures periods when service operations were halted, providing context for any operational interruptions.

  –

  Scheduled and Operational Hours track the planned versus actual operational time of the shuttle, providing a measure of service consistency.

  –

  Uptime Percentage indicates the proportion of scheduled hours during which the shuttle remained operational, serving as a key indicator of reliability.

• Service Usage refers to the overall utilization of the shuttle service. The key metrics used to evaluate usage include the following:

  –

  Number of Round Trips Completed provides insight into service operational throughput.

  –

  Number of Passengers helps assess the shuttle’s usage and capacity.

  –

  Number of Passengers per Hour Operated provides insight to service demand and shuttle efficiency.

  –

  Number of Passengers per Round Trip provides insight into shuttle occupancy levels.

  –

  Number of Round Trips per Hour Operated offers a measure of operational efficiency.

• Service Performance evaluates the shuttle’s operational performance. The key metrics include the following:

  –

  Average and Maximum Vehicle Speeds measure the typical and peak speeds achieved daily, which are essential for evaluating the shuttle’s efficiency.

  –

  Autonomous Mode Percentage reflects the percentage of time the shuttle operated autonomously, providing insights into the shuttle’s capabilities and performance under various conditions.

  –

  Starting and Ending Battery Percentage measures the battery levels at the start and end of each operational period; these measurements are used to assess energy consumption.

  –

  Battery Percentage Used measures the difference between starting and ending battery percentages, providing insight into energy efficiency.

2.4. Data Processing

For data processing, we first excluded weekends and holidays to ensure that our analysis focused on regular operational days. Next, we removed any days when the shuttle service was suspended for the whole day to eliminate data that did not represent typical shuttle performance. We further processed the autonomy disengagement data using ArcGIS Pro 3.2 Online Mapping Software. First, we created the shuttle’s route in the software using the coordinates of the eight shuttle stops. After mapping the route, we added the disengagement coordinates. A 50-foot buffer zone was established on either side of each route segment, as shown in Figure 3, to include disengagements close to the shuttle’s path. Disengagement points outside this buffer were removed from the dataset. We further analyzed the data with thoroughness, ensuring that the causes of each disengagement matched the characteristics of the specific route segments. For example, disengagements due to signal loss were only retained if they occurred at intersections, as indicated by the mapped data. This thorough filtering process allowed for a focused analysis of disengagements along the shuttle route by aligning the data with precise route segments. This filtered dataset allowed for a more precise evaluation of the shuttle’s performance and reliability during its operational periods.

3. Analysis and Results

The data indicate that the shuttle successfully transported 565 riders on 825 individual trips, with 64 trips that had no riders. To thoroughly assess the performance of the automated shuttle, we analyzed both the autonomy disengagement data and the operational efficiency data, drawing insights from the key metrics collected.

3.1. Service Robustness: Autonomy Disengagement Analysis

The autonomy disengagement data can be an indicator of the robustness of the autonomous shuttle service. To this end, The processed data were examined to identify challenges hindering the shuttle’s autonomous operation under various spatio-temporal conditions, such as heavy traffic during rush hours or adverse weather conditions during specific seasons.

Throughout the pilot study, there were 244 incidents in which the autonomous system was disengaged due to system errors or manual intervention by the operator. Figure 4 provides a detailed breakdown of the reasons for the disengagement events involving the automated shuttle. The technical and signal issues category emerges as the leading cause, accounting for 55.8% of all incidents. Within this category, the fault codes represent 3.4% (9 incidents), signal loss contributes 7.9% (21 incidents), and signalized intersections account for a substantial 44.6% (119 incidents). The safety and interaction with others category is the second largest, comprising 25.8% of all disengagements. Within this group, disengagements due to other road users make up 18.4% (49 incidents), blocked station incidents account for 6.7% (18 incidents), and vulnerable road users and safety incidents in/out contribute 0.4% each (1 incident each). The category of environmental factors represents 14.2 of all disconnects, with obstacle detection at 4.9% (13 incidents), vegetation at 9% (24 incidents), and weather conditions at 0.4% (1 incident). Lastly, the category of navigation and path deviation comprises 4.1% of all disconnects. This includes priority zone disengagements at 3.4% (9 incidents) and cases where the shuttle manually deviated from the approved path at 0.7% (2 incidents).

Table 1. Variation in disengagement type by space and time.

Segment	Category	July	Aug.	Sep.	Oct.	Nov.	Dec.	Total
18 Student Union West–33 Student Union Deck	Environmental factors	-	-	-	1	1	1	3
	Navigation and path deviation	-	1	-	-	-	-	1
	Safety and interaction with others	3	-	3	4	2	3	15
	Technical and signal issues	13	8	9	6	3	24	63
	Segment total	16	9	12	11	6	28	82
33 Student Union Deck–34 Light Rail East	Environmental factors	-	3	-	-	-	1	4
	Safety and interaction with others	1	1	-	-	-	1	3
	Technical and signal issues	-	-	-	-	-	3	3
	Segment total	1	4	-	-	-	5	10
34 Light Rail East–Greek Village 4	Environmental factors	1	-	-	-	1	-	2
	Safety and interaction with others	-	-	-	1	-	-	1
	Technical and signal issues	1	3	7	-	7	2	20
	Segment total	2	3	7	1	8	2	23
Greek Village 1–Greek Village 8	Environmental factors	1	11	-	1	-	-	13
	Safety and interaction with others	1	11	-	2	1	-	15
	Technical and signal issues	1	-	1	-	-	-	2
	Segment total	3	22	1	3	1	-	30
Greek Village 4–Science Building	Environmental factors	5	2	-	2	1	-	10
	Navigation and path deviation	5	-	3	1	-	1	10
	Safety and interaction with others	2	3	3	7	4	3	22
	Technical and signal issues	2	15	12	-	4	3	36
	Segment total	14	20	18	10	9	7	78
Greek Village 8–Greek Village 4	Environmental factors	-	-	-	-	1	-	1
	Safety and interaction with others	-	-	-	1	-	1	2
	Segment total	-	-	-	1	1	1	3
Science Building–18 Student Union West	Environmental factors	5	-	-	-	-	-	5
	Safety and interaction with others	5	1	-	2	3	-	11
	Technical and signal issues	1	-	1	-	-	-	2
	Segment total	11	1	1	2	3	-	18
Total		47	59	39	28	28	43	244

provides an in-depth analysis of the spatio-temporal variation in autonomy disengagements by type. These data reveal critical insights into the operational challenges and trends that affect automated shuttle performance. The segment between 18 Student Union West to 33 Student Union Deck, with the highest number of disengagements, indicates it is the most problematic area for the automated shuttle. The primary cause in this segment was technical and signal issues, especially at signalized intersections where the shuttle struggled to maintain V2I communication, resulting in a manual takeover. The high frequency of disengagements in this segment suggests a need for targeted improvements in technical infrastructure and underscores the importance of enhanced training for handling these intersections.

Another critical segment is from Greek Village 4 to the Science Building, which saw significant disengagements (with 82 disengagements in total). The segment recorded a mix of disengagements due to technical faults, signal loss, and interactions during passenger boarding and alighting. The third most problematic segment is between Greek Village 1 and Greek Village 8 (with a total of 30 disengagements). This segment route experienced significant disengagements due to environmental factors and safety interactions with others. August showed a particularly high disengagement spike, which could be correlated with seasonal environmental changes, such as increased vegetation growth that affects sensor performance.

The segment between 34 Light Rail East to Greek Village 4 experienced moderate disengagements (with 23 total disengagements), primarily due to technical and signal problems. The segment between the Science Building and 18 Student Union West also experienced moderate disengagements (18 total), but these were mainly related to safety interactions with road users and environmental factors. The main reason for the disengagement in this segment was “vegetation”. The shuttle was not able to detect objects because of the thick vegetation. To address this issue, the campus trimmed overgrown branches along the entire pilot route in August. After this, the number of vegetation-based disengagements decreased significantly. The segment between 33 Student Union Deck and 34 Light Rail East recorded fewer disengagements (10 total disengagements), with issues primarily related to environmental factors and technical problems. Finally, the segment between Greek Village 8 and Greek Village 4 had the fewest disengagements (three total disengagements), suggesting it is one of the least problematic areas for the shuttle.

Figure 5 presents a monthly summary of the types of autonomy–disengagement by category. The total number of disengagements varied each month, peaking in August with 59 and reaching a low in October and November with 28. This variation suggests changing conditions and challenges, which seasonal factors, technical issues, or other variables may influence.

3.2. Service Reliability: Suspension Events, Operational Hours, and Uptime

The reliability of automated shuttle service was evaluated by examining key metrics, including suspension events, operational hours, and uptime. Suspension events provide information on periods when the shuttle was unable to operate, helping to identify recurring issues or external factors that may impact service. Operational hours track the shuttle’s performance relative to its scheduled availability, while uptime measures the percentage of time the shuttle remained operational as planned.

3.2.1. Temporal Variation in Service Suspension

The operational data from the automated shuttle, provided by Beep, showed that there were a total of 62 days with service suspensions, three of which were due to holidays. The other 59 incidents provide valuable insights into the most common factors disrupting the shuttle’s operations. We have categorized the reasons for service suspension into three main groups: Operator-Related Issues, Technical and Equipment Issues, and Environmental and External Factors.

Table 2. Service suspensions by month.

Category	Reason for Suspension	July	Aug.	Sep.	Oct.	Nov.	Dec.	Total
Attendant-Related Issues	Attendant attended mandatory training from Navya	-	-	-	-	1	-	1
	Attendant required to upload footage from incident that occurred overnight	-	-	-	-	1	-	1
	Attendant unable to complete shift due to illness	-	-	-	-	1	-	1
	Attendant was late for start of shift	-	-	1	-	-	-	1
Technical and Equipment Issues	Charger not working	-	-	1	-	-	-	1
	Controller reset	1	1	-	-	-	-	2
	GNSS signal loss and troubleshooting	3	5	3	3	1	2	17
	GNSS signal loss and troubleshooting; inclement weather	-	1	-	-	-	-	1
	GNSS signal loss and troubleshooting; insufficient battery	-	-	1	-	-	-	1
	Insufficient battery	3	9	3	-	1	-	16
	Motor bracket needed repair	-	-	1	1	-	-	2
	Motor bracket repaired by midday	-	-	-	1	-	-	1
	Navya was connected to vehicle at the start of shift	-	-	1	-	-	-	1
	Navya was connected to vehicle at the start of shift; GNSS signal loss and troubleshooting	-	-	1	-	-	-	1
	Vehicle did not charge after previous shift due to broken outlet	1	-	-	-	-	-	1
Environmental and External Factors	Inclement weather	-	4	-	2	2	1	9
	No service after 2 PM due to home football game	-	-	-	1	-	-	1
	Time needed to install equipment on vehicle for environmental data collection exceeded the charging break	-	-	-	-	1	-	1
Total		8	20	12	8	8	3	59

provides a detailed breakdown of the reasons for each category of service suspension and the number of occurrences from July to December.

The category “Technical and Equipment Issues” was the leading cause of service suspensions, with a total of 43 incidents. The most common issues were “GNSS signal loss and troubleshooting” and “Insufficient battery”, which together made up a significant share of all suspensions. Other technical problems included controller resets, charger failures, and necessary repairs to motor brackets. The high occurrence of these issues, especially in August and September, indicates ongoing challenges with equipment reliability and maintenance at the start of the pilot test.

There were four service suspensions due to attendant-related issues, including mandatory training, illness, uploading footage from incidents, and late arrivals for shifts. Environmental factors and external events led to fewer suspensions, totaling 11 during the period, with inclement weather being the most significant cause, resulting in 9 suspensions.

3.2.2. Scheduled vs. Operated Hours and Uptime Analysis

In addition to service suspension, we evaluated other metrics to assess the system’s reliability. The metrics include the comparison of the automated shuttle’s actual operational hours to its scheduled hours over time, as well as the uptime percentage, which represents the ratio of operated hours to scheduled hours. Figure 6 compares scheduled and operated hours throughout the study period. In this figure, the scheduled hours represent the planned operating hours for the shuttle each day, while the operated hours indicate the actual time the shuttle was in service.

The shuttle was initially scheduled to operate for 6 hours daily, excluding weekends and holidays, from July 17 to December 21. Its typical operating schedule ran from approximately 8:15 AM to 11:15 AM and 1:15 PM to 4:15 PM in the afternoon. However, beginning November 9, the schedule was extended on select days to include evening service from 5:30 PM to 8:30 PM, increasing total daily operation to 9 hours. Throughout the study, noticeable monthly variations were observed in scheduled and actual operating hours, particularly after the evening service was introduced on November 9.

Significant variations in shuttle operation can be observed in Figure 6. Since the shuttle began operating on July 13th, the initial period showed fewer scheduled hours, but there was a noticeable increase in the scheduled hours from August through November. The increase in November can be attributed to the shuttle extending its operation. Although the shuttle only operated for 15 days in December, including evening hours, the total scheduled and operated hours were similar to those in August and October. It is also evident that the actual operated hours consistently fell short of the scheduled hours throughout the observed period.

To better describe the operational efficiency of the automated shuttle, Figure 7 presents the uptime percentage. Uptime is the ratio of the actual hours the shuttle operated to the scheduled hours it was supposed to operate. A higher uptime percentage indicates that the shuttle was in service for most of its scheduled time, reflecting greater reliability and fewer disruptions. Figure 7 shows a general trend towards increasing operational efficiency over time. In July, the uptime percentages exhibited noticeable variability, reflecting inconsistent performance due to initial technical and operational adjustments. However, as the pilot period progressed, there was a trend toward higher median uptimes and narrower interquartile ranges, suggesting improved consistency in meeting scheduled hours. By October, the uptime percentages reached near-perfect levels, with a tight interquartile range around maximum efficiency, indicating that the shuttle’s operations had stabilized and ran smoothly. In November and December, the shuttle maintained high median uptimes, although a slightly wider interquartile range in November suggests some variability in operational performance, potentially due to seasonal factors or minor technical issues. December stands out with perfect uptime, demonstrating optimized performance and suggesting that any previous challenges were effectively addressed by the end of the pilot period. This trend indicates a positive trajectory in the ability of the shuttle to provide reliable service, with fewer interruptions and greater alignment with its scheduled hours.

Table 3. Effect of disengagement and service suspensions on uptime percentage.

Month	Number of Service Suspensions	Number of Disengagements					Average Uptime Percentage
		Enviromental Factors	Navigation and Path Deviation	Safety and Interaction with Others	Technical and Signal Issues	Total
July	8	12	5	12	18	47	81.24
August	20	16	1	16	26	59	76.17
September	12	0	3	6	30	39	86.08
October	8	4	1	17	6	28	90.00
November	8	4	0	10	14	28	88.38
December	3	2	1	8	32	43	96.01

shows that the average uptime percentage was inversely related to the number of service suspensions and disengagements. As suspensions and disengagements decreased, the average uptime increased. A correlation analysis confirmed this observation, revealing that average uptime was negatively associated with the number of service suspensions (correlation coefficient = −0.86) and disengagements (correlation coefficient = −0.65). Additionally, each type of disengagement (environmental factors, technical issues, safety and interaction with users, and navigation and path deviation) was also negatively associated with average uptime, indicating that increases in any disengagement led to a decrease in uptime. The data further suggest that service suspensions have a more pronounced effect, while disengagements impact uptime. For instance, in August, the month with the highest number of service suspensions (20) and disengagements (59), the average uptime was the lowest at 76.17%. In contrast, December, with only three service suspensions and 43 disengagements, saw a significant increase in average uptime to 96.01%. This shows that minimizing service suspensions is crucial for maintaining higher operational uptime, even if disengagements remain relatively high.

3.3. Service Usage: Number of Passengers and Trips

The usage of the automated shuttle service was evaluated by examining key operational metrics that provide information on the general demand and performance of the shuttle. These metrics include the number of completed round-trips and the number of passengers transported during the service period.

3.3.1. Number of Completed Round Trips

To gain a comprehensive understanding of the shuttle service’s usage patterns, we analyzed two key metrics: the number of round trips completed and the number of passengers transported by the shuttle.

Table 4. Summary of total number of round trips and passengers per month.

	July	August	September	October	November
Number of Round Trips	85	156	133	144	166
Number of Passengers	44	110	129	112	113

shows the total number of round trips and passengers for each month.

Figure 8 displays the number of round trips completed by the shuttle each day for each month. The large percentile range and the relatively higher median in July suggest that the shuttle service had days with a significantly high number of trips, likely due to the launch phase. From August to October, there was a period of stabilized usage, indicated by consistent medians and narrow percentile ranges, reflecting regular operations. In November, the data showed increased variability, possibly due to operational adjustments and the addition of additional evening shifts. In contrast, December was more consistent, with a higher median number of trips, suggesting optimized operations by the end of the year. There were significant outliers in August and November, indicating days with an unusually low number of trips, mainly due to service suspensions on those days.

Figure 9 shows the correlation between the number of trips made and the hours of operation. As expected, there was a positive correlation between these two variables. This correlation supports the higher number of trips observed in November and December, as shown in Figure 8. There was also a clustering of data points around 4 to 6 h of operation, with a range of 3 to 10 round trips, indicating that most days fell within this operating range and trip frequency. This suggests that the shuttle typically operated within these parameters. Fewer data points can be observed at less than 3 h of operation, which were mainly caused by service suspensions.

3.3.2. Number of Passengers

Figure 10 illustrates the number of passengers per hour transported for each month. At the beginning of the pilot, the low median values suggest limited passenger adoption or awareness, likely due to the school year not yet starting or because people were unfamiliar with the shuttle service. From August to October, there was an increase in both the median and variability of passenger numbers. This indicates that while overall demand was rising—likely because students were returning to school and becoming more familiar with the shuttle service—it remained inconsistent, with some days having significantly more or fewer passengers. This variability could be due to occasional service suspensions. In November and December, the data showed a trend toward stabilization. The median number of passengers per hour transported slightly decreased compared to the peak months of August to October, suggesting a leveling off in demand. This could be due to factors such as the shuttle’s slower speed compare to human-driven shuttles on campus or a decline in passenger interest. However, the narrower range of values during these months suggests that shuttle usage became more predictable and consistent. This indicates that shuttle operations and passenger demand had likely stabilized due to adjustments in the service schedule or more regular passenger behavior.

3.4. Service Performance: Speed and Autonomy Metrics

The operational performance of the automated shuttle service was evaluated by examining key metrics, including average and maximum speed, percent autonomy, and battery usage. Each of these metrics has a different perspective on how well the shuttle performs under normal conditions and whether its automated features are functioning optimally.

3.4.1. Average Speed and Maximum Speed

This section examines the monthly variations in the average and maximum speeds of the automated shuttle to assess its operational performance over time. The average speed reflects the typical velocity maintained during operation, whereas the maximum speed indicates the maximum velocity reached at any given moment. Figure 11 shows that average speeds generally remained stable but with some fluctuations. The median speeds were around 6.0 mph for most months, with noticeable increases in October and December, where the median speeds were higher. The relatively narrow interquartile ranges suggest moderate consistency within each month, although there was greater variability in August and December. Outliers in July, August, and November indicate occasional deviations from the standard speed range.

Figure 12 illustrates a general stability in maximum speeds across the months, with median values consistently around 11.4 mph. The narrow interquartile ranges indicate high consistency in the maximum speeds. Overall, the data suggest that both the average and maximum speeds were fairly stable. The consistency in the interquartile ranges points to reliable speed performance, although the increased variability observed in months such as August and December could benefit from further investigation.

3.4.2. Percent Autonomy

The percentage of autonomy reflects the degree to which automated driving systems, as opposed to the shuttle’s attendant or safety operator, control the shuttle’s decision making, steering, and movement. A higher percent of autonomy indicates that the automated systems are responsible for the shuttle’s operation most of the time. Figure 13 presents the monthly variation in the percentage of time the shuttle operated in autonomous mode, offering insights into the shuttle’s overall autonomy and the factors affecting its operational independence over the observed period.

As depicted in Figure 13, July exhibited a high-percent autonomy, with a median close to 95%, though a few outliers were noted. August showed increased variability, with a broader interquartile range, indicating a reduction in autonomous operation. The percentages stabilized from September to November, with medians around 90%, suggesting some reliance on manual operation. However, in December, the autonomy percentage slightly declined, with a median of around 85% and a few outliers. This fluctuation in autonomy can be correlated with the number of disengagements. A Pearson correlation analysis between the number of disengagements by type and percent autonomy was conducted, and the results are summarized in

Table 5. Correlation between disengagement type and percent autonomy.

Disengagement Type	Correlation with Percent Autonomy	p-Value
Environmental Factors	−0.08	0.40
Navigation and Path Deviation	0.01	0.88
Safety and Interaction with Others	0.10	0.31
Technical and Signal Issues	−0.49	<0.05
Total	−0.36	<0.05

, which presents the correlation coefficients and p-values for each type of disengagement.

Table 5 reveals that technical and signal issues had a significant negative correlation (−0.49) with the percentage of autonomy, with a p-value of <0.05, indicating a substantial and statistically significant impact on the reduction in shuttle autonomy. The general correlation for all combined types of disengagement came out to −0.36, with a p-value of <0.05, suggesting a moderate and statistically significant negative correlation. This indicates that while individual factors like environmental conditions and path deviations had minimal impacts, technical and signal issues significantly reduced the shuttle’s autonomous operation time. Collectively, all disengagements contributed to a noticeable decrease in percent autonomy.

3.5. Service Battery Performance

The automated shuttle is a battery-powered electric vehicle and was charged prior to each trip (the night before and from 11:30 am to 1:30 pm between the two shifts). For charging, the vehicle was taken to the parking lot of the transportation services. The data records include the percentage of battery at the beginning and at the end of each day, enabling the calculation of daily battery usage. Figure 14 illustrates monthly trends in the shuttle’s battery consumption, showing how the percentage of battery capacity used changed over time.

Figure 14 reveals a decreasing trend in battery consumption throughout the pilot program. In July, the shuttle had the highest battery usage, with a median of around 55% and significant variability, including multiple outliers. August showed slightly lower usage but maintained high variability and several outliers. From September onward, there was a notable decline in both the median and the interquartile ranges of battery usage, indicating more consistent and reduced consumption. In December, battery usage had decreased significantly, with the lowest median and a narrower interquartile range.

Figure 14 also highlights that battery usage variation was higher in the warmer months (July, August, and September) compared to the cooler months (November and December). This trend suggests that weather conditions significantly affect the shuttle’s battery efficiency. In hot weather, the increased demand for cooling systems and the increased internal resistance of the battery, needed to maintain passenger comfort and protect electronic components, lead to higher energy consumption. Continuous cooling requirements at high temperatures further exacerbate battery usage.

It is important to note that the variation in battery usage may not reflect actual usage, as the shuttle is charged between shifts. Moreover, although the percentage of battery is only recorded at the end of each day and does not directly correlate with operational hours, so there is an indirect relationship due to service suspension. On days when shuttle service is suspended due to low battery power, the operational hours were noticeably reduced, indicating that battery-related suspensions significantly affect shuttle daily operational time.

4. Conclusions and Recommendations

This study presented the potential of autonomous shuttles to operate efficiently within a university campus setting, though it also revealed several challenges that need to be addressed to enhance their overall reliability. Throughout the pilot, the shuttle showed a progressive improvement in operational efficiency, with higher uptime percentages and better consistency as the months progressed. However, disengagements, particularly due to technical and signal issues at intersections, emerged as a significant barrier to fully autonomous operations. Environmental factors such as weather conditions and vegetation, as well as interactions with other road users, also contributed to the challenges, though to a lesser extent. The battery performance, while generally reliable, experienced fluctuations that affected service reliability, particularly in warmer months. Despite these challenges, the shuttle maintained strong service robustness and autonomy for most of the pilot period, offering valuable insights into the feasibility of autonomous transportation in real-world, mixed-traffic environments. By addressing the identified issues, autonomous shuttles can become a reliable and safe alternative for campus transportation and beyond.

Future efforts should focus on several key areas to further enhance the performance and reliability of autonomous shuttles. First, improving GNSS signal reception and implementing robust Vehicle-to-Infrastructure (V2I) communication protocols at intersections will be critical to reducing signal-related disengagements. In addition, optimizing battery management systems and exploring higher-capacity batteries will extend operational hours and reduce the risk of service interruptions. Enhancing the shuttle’s obstacle detection and avoidance algorithms to better handle environmental challenges, such as weather and vegetation, will further improve reliability and safety. Finally, ongoing training for shuttle attendants to manage disengagements efficiently and refining interaction protocols with other road users will support smoother autonomous operations in dynamic environments. By addressing these areas, future deployments of autonomous shuttles can achieve greater reliability, safety, and operational efficiency, paving the way for broader adoption in public transportation systems.