Marker-Based 3D Position-Prediction Algorithm of Mobile Vertiport for Cabin-Delivery Mechanism of Dual-Mode Flying Car

Abstract

This paper presents an image-processing technique for cabin delivery employing local localization and docking in a mobile station, which is a mobile vertiport for the use of dual-mode flying cars. A dual-mode flying automobile with an aerial electric vehicle (AEV), a ground electric vehicle (GEV), and a cabin is a future method of transportation that can be used in both the air and on the ground. To enable AEVs to land in a specific position, a landing site is necessary. The proposed AEV uses vertical take-off and landing, and a vertiport landing site is required. As vertical take-off and landing sites require a lot of space and are challenging to operate in multiple positions, we suggest a mobile vertiport that can fit into a small space. A mobile station is appropriate for dual-mode flying cars since it includes critical activities such as transporting AEVs from the ground and charging as well as a cabin-delivery system. The mobile station can generate a path to the AEV by calculating the relative position using the markers attached to the AEV and estimating the position of the landing AEV. The mobile station detects a marker for precise positioning correction, followed by exact position correction for cabin delivery, to travel to the accurate position of the AEV. To increase the success rate of cabin delivery, docking markers are identified and the angle position error between the mobile station and cabin is computed and corrected to rectify the position between the cabin and the mobile station for cabin delivery. In addition, the experimental results revealed a mechanically correctable error range that encompassed all experimental values. Consequently, this study showed that image processing may be used to create a mobile station for dual-mode flying automobiles.

1. Introduction

Urbanization is advancing at a breakneck pace worldwide. According to the United Nations Economic and Social Council, since 2010, the global urban population has surpassed regional populations, with an urbanization rate of 55.5% as of 2018. The rural population has decreased since 2020, while the urban population is expected to increase continuously. Consequently, the global urbanization rate is expected to reach 68.4% by 2050. The UN’s analysis found that there are only 10 megacities with populations of more than 10 million people, but that number has risen to 33 million as of 2018 and is expected to rise to 43 million by 2030. This urbanization has resulted in a slew of issues in sectors such as transportation, housing, the environment, and energy. Cars that clog city streets in particular caused severe traffic congestion and environmental pollution, resulting in a significant socioeconomic loss. For example, Americans lose an average of 97 h per year due to traffic congestion, equating to a loss of USD 1348 per person. The urban traffic problem is more than just a traffic problem; it can lead to a slew of additional issues, including energy waste, pollution, and noise production. The combination of electric-powered autonomous cars with sharing-platform technologies is projected to help alleviate some of the city traffic congestion and environmental issues. However, given the current population concentration in cities, there is a limit to consider it as a fundamental solution. On the ground, autonomous driving and sharing platforms are seen as a better way to distribute urban traffic resources that are currently inefficiently managed through digital disruption [1]. It is necessary to distribute resources efficiently, but for a fundamental solution, a new space resource is required. Due to the saturation of ground and underground spaces, the urban traffic problem can no longer be managed to simply utilize two-dimensional plane space. To accomplish this, the National Aeronautics and Space Administration has designated the city’s short-distance air-transport ecosystem as the foundation for a new metropolis that will rely on low-altitude air for urban air transportation (UAM) [2,3,4,5,6,7,8,9]. The UAM is now widely used in a variety of industries and markets. The UAM has been investigated as a future model of transportation that uses the air to alleviate traffic congestion on the ground. Since it is difficult to establish a runway in the city center, the personal air vehicle (PAV) [10,11,12,13,14,15,16], which is a form of air transportation, has been researched as a form that can take off and land vertically. For vertical take-off and landing, the PAV requires a separate vertiport. Consequently, in addition to PAVs, a vertiport [17,18,19,20] is investigated.

Various transportation platforms and construction businesses in Korea and abroad are researching vertiports, the majority of which are fixed. A fixed-type vertiport is useful for passengers who manage and utilize a large number of PAVs, as well as for business expansion through value-added creation via connections to other modes of transportation. However, because of its size, it necessitates a significant investment as well as a considerable amount of space with unique buildings. Furthermore, a dual-mode flying car, which is a hybrid of aerial electric vehicle (AEV) and ground electric vehicle (GEV) platforms, is becoming closer to being a viable concept for moving people or goods between cities or inside cities in the future. A dual-mode flying car is a mode of transportation in which AEVs are used to travel between distant cities and GEVs are utilized to deliver passengers to their destinations within complex cities. It can be a transportation system that applies to passenger transportation and logistics and other services. In particular, because of the high population density of certain places such as the Asian regions, a vertiport that requires a big space may not be developed. This paper suggests a mobile vertiport for dual-mode flying automobiles as a solution to this challenge. A Special Vertiport is required for docking and undocking of passenger cabins on AEVs and GEVs, especially in a megacity, and in dense building environments, special buildings for Vertiports require space and space constraints. Since a mobile vertiport does not require a great amount of room and can fit into a small space, it can be put in a variety of positions around a city. Consequently, the number of urban stations available to passengers grows, allowing passengers to go closer to their chosen destination. A mobile vertiport or a mobile station is proposed in this paper. To travel to the AEV and GEV, a mobile station should be able to recognize a location. A target’s position should be recognized using cameras or sensors, and the positions of AEV and GEV are accurately detected using three-dimensional (3D) depth cameras to accurately locate a preferable object among many objects, barriers, and identical things. Using docking-image processing for precise position correction and cabin delivery using markers attached to AEV and GEV to move a cabin from the moving platform to the AEV or GEV while recognizing a preferred platform using markers attached to each platform and calculating a relative coordinate with markers, a mobile station for dual-mode flying cars was investigated. The study proposes a fixed Vertiport; the flying AEV must move to the correct Vertiport position, which must be increased in size for accuracy of landing, and it is difficult to control in order to land in the correct position of Vertiport depending on weather conditions. In addition, depending on the Vertiport’s accuracy, it is highly likely that the AEV will have to land in a different scenario, but the proposed Mobile Vertiport will reduce the burden of controlling the accuracy of the AEV because it can move to a large vacant lot or any mobile Vertiport. To this end, we propose Mobile Station, a Mobile Vertiport, and propose a location detection and docking system based on image data available in the proposed system and the image-processing algorithm of the mobile station, which is a mobile vertiport for dual-mode flying cars that can be used as a future transportation means. This image-processing algorithm proposes a precise position-correction algorithm based on markers to estimate the position of the platform and deliver a cabin.

2. Position-Prediction Technology for Cabin Delivery of the Mobile Station

2.1. Marker-Based AEV’s Position-Information Acquisition (Local Localization)

To utilize various functions such as cabin delivery and recharging, a mobile station should move to a spot where an AEV is landed. It is important to precisely detect the posi-tion of AEV for the mobile station to move. Therefore, the associated sensor’s data can be utilized to check the AEV position, but a sensor error occurs. Consequently, a mechanism for obtaining precise position data is required. This method can be accomplished using the suggested image-processing algorithm, which can obtain position information by detecting markers attached to the AEV’s body via the mobile station’s cameras.

The red circles in Figure 1 show the markers attached to the body, and these markers are recognized to obtain the position information of the AEV. The AEV’s markers are recognized using a depth camera attached to the mobile station, and a relative position between the mobile station and AEV is calculated by computing a distance and angle to the marker.

Figure 2 shows the mobile station model and the positions of the cameras to recognize the AEV markers. The mobile station employs a total of six cameras to estimate the position. Here, Cam1 and Cam2 are used to estimate the AEV’s position.

The distance and angle to the marker are calculated using two cameras, Cam1 and Cam2. The distance and angle are calculated at the center point of the camera using a trigonometric function based on the distance and angle measured using two depth cameras.

Figure 3 shows how to calculate the distance and angle to the marker from the center point of the camera.

Table 1. Definition of parameters in the stereo algorithm.

Parameter Name	Parameter Description
$\theta$	The angle between the camera object and the target
d	Distance between the camera and object
l	Distance between cameras
P	Pixel distance on the camera
α	The hypotenuse of focal length triangle
F	Camera’s focal length

describes the parameters used in Figure 3. A distance between the camera and object can be calculated using triangulation as follows:

$${\alpha }^2=p_1^2+F^2$$

(1)

$$F^2={\alpha }^2-p_1^2=p_1^2+{\alpha }^2-2p_1\alpha \cos {\theta }_1$$

(2)

θ can be summarized using the equation as follows:

$${\theta }_1={\cos }^{-1}(\frac{p_1}{\sqrt{p_1^2+F^2}})$$

(3)

${\theta }_2$ can also be calculated using the same method.

By rearranging the angles ${\theta }_1, {\theta }_2$, the following equations can be obtained:

$$\tan {\theta }_1=\frac{d}{l}$$

(4)

$$\tan {\theta }_2=\frac{d}{I-l}$$

(5)

By rearranging for d, the following equation can be obtained:

$$d=I\times \frac{\tan {\theta }_1\tan {\theta }_2}{\tan {\theta }_1+\tan {\theta }_2}$$

(6)

Using the d and θ obtained above, the AEV’s position can be calculated in the mobile station, and the mobile station can move to the AEV.

2.2. Precision Correction of the Mobile Station Based on Markers

As described in Section 2.1, the mobile station that moves to the AEV should be precisely aligned for docking with the AEV. For this, the markers attached to the AEV’s legs are used to precisely align the position, as shown in the red circles in Figure 4. Cams 3–6 in the mobile station in Figure 2 recognize the markers that indicate each position in the legs of the AEV, and distances and angles in the markers of the legs are recognized by each camera, calculating and correcting the AEV position within the mobile station.

Figure 5 depicts how the mobile station approaches the AEV based on the AEV’s position. Figure 5a depicts how the mobile station moves in response to the AEV position using the front camera. Figure 5b depicts the mobile station approaching the AEV while recognizing the red markers on the AEV’s rear side using Cams 3 and 5. The attachment of four markers and cameras in the AEV and mobile station allows the mobile station to see the AEV’s markers continuously without blank sections when approaching the AEV. The mobile station approaches the AEV while recognizing the marker on the rear side of the AEV and avoiding a collision with the AEV. Figure 5c shows that the mobile station uses cameras to check the markers at all times as it enters. Figure 5d depicts the mobile station docking with the AEV while checking the markers.

As shown in Figure 6, precise correction is complete once the position, distance, and angle of the four markers and depth cameras are precisely aligned with each other. Moreover, if there is a misalignment, the mobile station corrects the position.

If there is a position error between the mobile station and the AEV, it means that the AEV has been rotated or translated based on the mobile station. The position errors between the mobile station and the AEV are depicted in Figure 7. The left-hand figure depicts the translation error between the mobile station and the AEV, while the right-hand figure depicts the rotation error.

For the translation error, the mobile station moves in parallel while matching the distance and angle between the four cameras and markers. For the rotation error, each of the four points’ distance and angle is measured, and a rotation equal to the calculated angle is performed, assuming that the four points are connected by a rigid body. The rotated mobile station then checks the position error again and matches the position via translation movement.

2.3. Docking between the AEV and Cabin Using the Marker-Based Mobile Station

The mobile station is used for docking between the cabin and AEV. The cabin is precisely moved to the AEV’s docking connection area using the mobile station. In this case, a circular marker is used to determine whether the cabin is properly positioned in relation to the AEV’s docking area. A round-shape marker is used for the circular marker, as shown in Figure 8. The reason for using the circular marker is that the position or angle of the camera is highly likely to be misaligned when docking. In this case, circular markers are used because they are least affected by angle and position and are easy to correct. The color of the circular marker is used to classify the red, green, and blue (RGB) areas, resulting in the detection of a cycle for each color and the calculation of the center point of each circle. The reason for classifying the color of the marker is to secure an accurate reference point and origin by using the color position, and divide the color to compose the marker by dividing the front and back to calculate the correct position and angle. The angle between two line segments is calculated by drawing line segments formed by the center points of the circles and a rectangle connecting the outermost points of the circles.

Figure 9 shows an angle θ made between the yellow line segment created using the center point of the circle when the docking markers are rotated and the black rectangle made using the outermost points.

As the angular error occurs as much as θ based on the marker, the cabin is rotated to be docked. This process is iterated until it converges to a small value that is within the error range, and when the error converges, docking is performed.

3. Experiment

3.1. Experiment Environment

The experiment was conducted with the urban personal mobility vehicle, which included an AEV, a GEV, a cabin, and a mobile vertiport. The AEV, GEV, cabin, and mobile station used in the experiment are depicted in Figure 10. The cameras used in image processing were Intel D455 (Inter, USA) for AEV marker recognition and Intel 435i (Inter, USA) for precision-control-marker recognition. Nvidia Jetson Xavier AGX (Nvidia, USA) was used for the image-processing board, and Intel’s NUC7CJYH (Intel, USA) was used for docking-image processing. Iterative experiments were conducted to compare the image-processing error and the real error. Every time the experiment was run, an error was measured and compared to determine if it was a true iterative error.

3.2. Marker-Based AEV’s Position Information

The mobile station recognizes the marker attached to the AEV’s rear side to estimate the AEV’s early position. The position of the AEV’s marker is estimated using two internal cameras in the mobile station, and the relative position between the mobile station and the AEV is calculated using the coordinates estimated by the two cameras. In this manner, the mobile station can carve a path to the AEV position. The experiment to recognize the AEV’s rear marker revealed that AEV markers 4 m, 7 m, and 11 m away from the mobile station were recognized, as well as recognition results and errors.

Figure 11 shows the mobile station’s recognized results for the rear marker. This figure is the result of recognition from an RGB color image and a binarized image. There was no difference in recognition results between color and binarized images. Consequently, users can select image formats based on their needs, such as a greater representation of environmental information or computation capability.

Figure 12 depicts the recognized result of the AEV marker, which is 4 m from the camera’s center point in the mobile station. The position of the marker recognized by the left-side camera in the mobile station was 4.03 m at 5.4° from the center point of the two cameras, whereas the position of the marker recognized by the right-side camera was 4.11 m at 13.9°. This means that the marker identified by the two cameras can be calculated to be approximately 4.01 m from the center point.

Figure 13 and Figure 14 show the recognized AEV position at 7 m and 11 m distances from the mobile station, respectively. Figure 13 shows the recognized results at a distance of 7 m: 7.04 m at an angle of 6.4° from the left-side camera and 7.26 m at an angle of 16.5° from the right-side camera. Using the above results, the AEV’s position was calculated to be 6.96 m from the center point. Figure 14 shows that a distance of 11.35 m was measured at an angle of 15.3° from the left-side camera and 11.21 m was measured at an angle of 10.1° from the right-side camera, which was then calculated to be around 11.13 m from the center point. This result showed a similar distance value when compared to the actual position.

Table 2. Results of AEV’s distance estimation from the mobile station.

Distance (m)	AVG. Error (m)	Max Error (m)
2.5	0.05	0.08
4	0.07	0.08
7	0.11	0.13
11	0.16	0.19

shows the measurement results of the preceding experiment after 30 iterative measurements at each position. The errors generated by measuring a different angle while the AEV’s distance remained constant were also included. The average and maximum errors are displayed in the results. In the experiment, the AEV was set to not deviate from the camera by more than 30°. When the angle exceeds 30°, the camera barely sees the marker, making recognition impossible. Consequently, the experiment was limited to angles of less than 30°. All of the average recognition errors were less than 2%, and the maximum errors were also less than 3.2%.

3.3. Precision Correction of the Mobile Station Based on Markers

To recognize a marker, the position is estimated using images shot simultaneously. If images taken at different times are used, the mobile station may move based on the computed results. Consequently, images shot simultaneously are used in the global shutter computation. Figure 15 depicts images recognized by a total of four cameras: left, right, front, and rear cameras, with a translation error in a marker for precise recognition. The angular error was minimized at the target point by the center point of the marker recognized by each camera, but the distance error occurred. The marker was found about 2 cm to the left and 5 cm to the right. Consequently, the mobile station had to move to the left.

Figure 15 shows the angular error between the AEV and mobile station. To correct the angular error that occurred here, the mobile station should be rotated based on the position of the AEV. The angular error in Figure 16′s image indicates that the angle between the mobile station and AEV was tilted to the left. Because the angular error was approximately 1.3°, the mobile station was rotated to reduce the angular error. After checking the distance error, the mobile station moves again after completing the rotation to reduce the translation error.

Figure 17 shows the marker for precise recognition after completing the precision move, in which the angular error was less than 0.1° and all markers are included with the error range (0.5 cm) at a distance of 3.4 cm. Thus, the precision move was determined as completed.

The error measured for marker recognition was a mean value after iterating total 50 measurements. From the marker, the angle was within 1° and the distance error was measured within 1 cm.

3.4. Docking between the AEV and Cabin Using the Marker-Based Mobile Station

The marker, which comprises three circles, is recognized to perform docking. Figure 18 depicts the image recognizing the center of a circle in a marker made up of circles without an angular error.

Figure 19 depicts the recognition of the marker where position and angular errors occur. Once the marker has calculated the position and angular errors, the cabin should be moved in the direction of the error to reduce it. Since an error may occur between the mobile station and the AEV when moving the cabin to reduce the error, the error should be corrected using docking, and the precision position-recognition error mentioned in Section 3.3 should be rechecked.

Docking between the cabin and AEV begins once the error is reduced using the marker shown in Figure 13. After 20 iterative experiments, the error between the image processing and real positions of the marker at the normal position was within 0.3 cm with an angular error of 0.5° or less. Consequently, this result demonstrated that docking can be accomplished using markers.

4. Conclusions

Future transportation systems capable of moving in the air are being researched to improve a congested ground transportation system. The future transportation system will look like this: for large cities with dense populations, transportation means will be divided into two types: one that moves between cities and one that moves within cities. AEVs are used to move quickly between cities, and by using a vertiport in a city, users can move to their destinations using GEVs within cities. The study proposes a mobile vertiport station that can be used with dual-mode flying cars. The mobile station is required to be moved to an AEV landing position after identifying the position of the AEV for various tasks such as moving landed AEVs, recharging, and cabin delivery. To accomplish this, the mobile station must be localized to the AEV’s position and position-corrected. It also necessitates an operation with omnidirection for smooth translation and rotation movements, even in confined spaces. To enable a move, the AEV’s relative position is calculated by estimating the AEV’s position using markers attached to the AEV and depth cameras in the mobile station. To improve position accuracy between the AEV and mobile station, the position is precisely corrected using precision-correction markers attached to the AEV and cameras in the mobile station while eliminating blind spots, reducing the possibility of collision with the AEV when the mobile station moves. It also includes a docking system for cabin delivery, which is an important function of dual-mode flying cars. Cabin delivery can be accomplished between the AEV and GEV using an algorithm that calculates and corrects position and angular errors between the mobile station and cabin using docking markers attached to the AEV. After 50 repeated experiments in which the mobile station moves to the AEV, the average error was controlled within the error range of 1° and 1 cm, and accurate position control was performed. When the cabin repeatedly docked the experiment to the AEV 20 times, the error ranges for docking, marker recognition, and positional correlation were within 0.5° and 0.3 cm, respectively. Consequently, both results were within the previously defined error ranges, demonstrating that the algorithm can be used as a system for precise position correction and cabin delivery of the dual-mode flying car’s mobile vertiport. In the future, we plan to conduct a study based on the actual external environment and weather in which the AEV can fly through camera filters and correction algorithms to increase the recognizable outdoor environment. In addition, we will conduct research on effective system management through the linking of the mobile station algorithm and control systems.