A Study on the Non-Contact Artificial Intelligence Elevator System Due to the Effect of COVID-19

Abstract

Recently, IoT has been combined with artificial intelligence (AI) technology to develop into intelligent IoT (Artificial Intelligence of Things, AIoT) and can provide smart services to all industries. Generally, buildings with 30 or more floors are classified as high-rise apartments. There are high-rise apartments in many countries with small land areas and high population densities around the world, such as Hong Kong, Singapore, and Korea. Korea has many high-rise apartments and buildings with 30 or more floors, and residents there inevitably use elevators to move between floors. Because it is a high-rise apartment, it takes time for EVs to move, and a contactless method is needed when using EVs, especially due to COVID-19. In this study, we proposed a smart elevator system using AI that allows residents of high-rise apartments to conveniently shorten waiting and use times and use elevators without contact during rush hours due to the impact of COVID-19. The proposed system uses AI facial recognition to shorten EV waiting time and moving time and has a contactless floor selection function. As a result of the proposed system, it was confirmed that residents’ EV waiting time and travel time were shortened compared to existing EV systems.

1. Introduction

The concept of the Internet of Things (IoT), which started by connecting sensors through a network, is a core technology of the era of the fourth Industrial Revolution and a technology for realizing a hyper-connected society. Recently, IoT has developed into intelligent IoT (Artificial Intelligence of Things, AIoT) by combining with artificial intelligence (AI) technology, and intelligent services can be provided to all industrial areas [1,2,3].

The number of high-rise apartments may vary depending on the region or country. In general, buildings with more than 30 floors are classified as high-rise apartments. Globally, high-rise apartments exist in many countries with small land areas and high population densities, such as Hong Kong, Singapore, and the Republic of Korea. In Korea, many high-rise apartments exist mainly in large cities such as Seoul and Busan.

In the future, it is expected that the number of high-rise buildings over 30 floors will increase due to the development of architectural technology in Korea. Therefore, research on elevators, which are essential facilities, is actively being conducted. Refs. [4,5,6] proposed an elevator monitoring system based on IoT. Ref. [7] designed a smart elevator button that controls the elevator door using Arduino Uno as an IoT device. Ref. [8] designed an Internet of Things (IoT) device and proposed a contactless remote elevator control method using a mobile phone via Wi-Fi. Currently, smartphone apps using Bluetooth, touchless motion, and touchless foot methods are being developed as elevator call methods [9]. In the case of apartments where people live, many high-rise apartments over 30 floors are being constructed, and most of the residents use elevators. Because it is a high-rise apartment, it takes time for EVs to move, and in particular, non-contact methods are required when using EVs due to COVID-19. When residents use the parking lot (basement) or the elevator between the first floor and their floor, it is necessary to shorten travel time and select the number of floors without contact. Therefore, it is important to conveniently shorten the travel time when the residents use the elevator between the parking lot (underground) or the first floor and their residence floor.

For high-rise apartments, most of the elevator users (residents) travel to the first floor or parking lot (B1, B2, B3), with peak usage times occurring during the commute to and from work. Specifically, peak times for elevator use are from 7:30 to 8:30 a.m. and 4:00 to 8:00 p.m. Elevator use is more dispersed at other times and is reduced during late night hours.

In this study, due to the impact of COVID-19, we propose a smart elevator system using artificial intelligence that allows residents in high-rise apartments to conveniently shorten waiting and use times and use the elevator without contact during peak elevator usage times.

2. Related Works

2.1. Elevator Operation

2.1.1. Elevator Speed

Generally, the elevators are set at the maximum speed determined at the time of design, and in high-rise buildings, electric elevators use motors to move the cabin and are operated using electric motors and cable systems [9,10].

Electric elevators have no special speed limits, allowing designers and architects to adjust the speed to suit the purpose of the building and the needs of users. The taller the building, the more floors you have to travel, so if the speed is slow, the time between floors will be excessively long. High speeds are required to increase efficiency by reducing waiting times for building users. For example, in the case of Lotte World Tower (123 floors), a high-rise building, elevators operate at a maximum speed of 10 m/sec when ascending or descending. In general, elevators for apartments or general buildings usually have a speed of about 1 m/s to 2.5 m/s, while elevators for high-rise buildings or commercial buildings can have higher speeds of about 5 m/s to 20 m/s. Additionally, the elevator moves at a constant speed to ensure the safety of elevator occupants.

2.1.2. Elevator Opening and Closing Time

The time between waiting for the elevator door to open and closing varies depending on the manufacturer but is usually about 3 to 5 s. In the case of elevators for the disabled, it is required to wait with the doors open for more than 10 s. It takes approximately 4 s for three to four adults to board a typical 10-seat elevator.

2.2. Face Recognition

Facial recognition is an algorithm that uses object detection to automatically determine who the input image is. It is broadly divided into face detection, facial landmark detection, and facial feature extraction [11]. The existing famous CNN (Convolutional Neural Network)-based face recognition algorithm is DeepFace [12]. Using 4 million face images collected by FaceBook, CNN learns a network consisting of a total of eight layers, and by generating and merging multiple of them, it achieves a recognition rate of 97.25%, which is similar to the human recognition level of 97.53%. DeepID2 [13] used color images as input and increased the input resolution from 39 × 31 pixels to 55 × 47 pixels. In particular, it assumed SoftMax Loss as Identification Loss and assumed Loss using Euclidean Distance as Verification Loss and proposed a kind of multitask learning technique, achieving a final performance of 98.97%. FaceNet [14] used GoogLeNet for face recognition and proposed Triplet Loss for Face Verification, utilized it for face recognition, and achieved a recognition rate of 99.63% even with a single network.

Face recognition is a type of biometric recognition technology that extracts facial features from photos and videos taken based on AI deep learning technology, stores them in a database, and verifies identity by comparing the saved photo with the current photo [15,16,17,18,19,20]. Since it is a non-contact method that does not require human contact, it is hygienic and convenient, making it the simplest and most efficient security system.

Among biometric technologies, ‘facial recognition’ is a method of extracting, comparing, and analyzing key features of the face, such as the distance between the eyes, the length and width of the nose, and the length of the chin, in a format that compares face photo images in an existing database.

Recently, it has continued to evolve and develop based on deep learning technology using AI (artificial intelligence), which is used not only in security systems but also in various fields. Additionally, by analyzing the facial expressions or emotions of the person in question, functions tailored to each individual’s characteristics can be provided.

Facial recognition technology must continue to be researched and developed to be able to accurately identify a specific person, even in a variety of situations. To use convenient facial recognition technology safely, measures to strengthen security and protect personal information are also needed.

Table 1. Comparison of biometric technologies [21].

	Face Recognition	Fingerprint Recognition	Iris Recognition	Vein Authentication
Test sample data size	12 million people	Does not exist	More than 500,000 people	70,000 people
Error rate	0.0021%	0.0050%	0.0048%	* does not exist
Processing speed	Within 0.2 s	Within 0.5 s	Within 19.58 s	Within 2 s
User convenience	very convenient	convenient	somewhat inconvenient	commonly

* does not exist: the authors did not provide the error rate number.

shows a comparison of biometric recognition technologies, and Figure 1 shows the face authentication process.

2.3. OpenCV (Open Source Computer Vision Library)

OpenCV is an open-source library for AI image processing and computer vision [22]. It consists of over 2500 algorithms. It is used for:

(1)

Traditional algorithms related to image processing, computer vision, and machine learning;

(2)

Face detection and recognition, object recognition, object 3D model extraction, and 3D coordinate generation from stereo cameras;

(3)

Image stitching for high-resolution image generation, image search, red-eye removal, and eye movement tracking.

3. Proposed EVAI System

3.1. Elevator Operation

In this study, due to the impact of COVID-19, we propose a smart elevator system using artificial intelligence (EVAI) that allows users (occupants) in high-rise apartments to conveniently shorten elevator usage time and use the elevator contactless during concentrated elevator usage hours.

The following is the procedure for high-rise apartment residents to use the elevator before and after applying the proposed system.

Elevator Use Procedures before Application

Procedure for Using the Elevator after Application

① Generally, the user boarding the elavator approaches the front. ② The user touches and selects the moving direction (up and down) with the hand. ③ Elevator moves to the floor where the user is waiting. ④ If the elevator arrives and the door is held, the user selects the button of the floor after boarding the elevator. ⑤ The elevator moves to the selected floor after the door is closed.

① Even if the user is not near the elevator front, by using the 3 cameras of an elevator, it recognizes the face at a constant distance, and the EVAI recognizes the user. ② EVAI moves the elevator to the floor where the user waits. ③ At the same time, EVAI searches the elevator using the DB of the recognized user and extracts the user’s moving direction (up and down) and the destination floor from the day and time reference DB to set the auto-selection layer. ④ When the elevator arrives at the user’s floor and the door is opened, the user compares the selected floor with the intended floor after boarding. ⑤ If the selected layer is wrong, the user selectsthe correct floor. ⑥ The elevator moves to the selected floor after the door closes. ⑦ EVAI stores the user’s usage history (date, day, time, layer) in the DB.

Figure 2 shows the use of a non-contact EVAI.

3.2. EVAI System

The proposed system consists of a Raspberry Pi and a camera system using IoT, and EVAI applies facial recognition technology using OpenCV and classifies elevator occupants. Classified face information for each elevator user is stored and then searched and recommended by EVAI. EVAI automatically stores facial images and elevator usage information taken when the resident last used the elevator the day before. It is used for facial recognition and automatic selection of the destination floor the next day. Figure 3 and Figure 4 show the process by which EVAI collects, stores, and classifies facial images and movement information of high-rise apartment residents.

EVAI identifies apartment residents through facial recognition and automatically selects the number of floors the resident moves to use the learned elevator usage information. Additionally, the opening and closing times of the elevator are controlled and shortened.

Figure 5 shows EVAI’s selection of the number of moving floors based on information for each resident, and Figure 6 shows the EVAI operation algorithm.

4. Experiment and Results

4.1. Experimental Environment

The experiment used OpenCV for face recognition, and the experiment equipment is listed in Table 1. The method for detecting a human face in real time is to capture the video, create an image file, and use the image to check whether a human face exists through a face detection algorithm (Figure 7). This is repeated continuously while the camera records video. The face detection algorithm uses the Haar feature [23,24]. Haar feature is a method of dividing a video/image into several regions and then obtaining the characteristics of the object through the difference in brightness between each region. Since there are parts of the human face that are significantly different from the surrounding parts, such as hair, eyebrows, eyes, and lips, this method is fully effective. However, this method causes errors in facial recognition if part of the face is covered, so all residents must reveal their entire face.

We use OpenCV’s Haar feature-based cascade classifiers to identify residents’ faces in real time from images from the Pi camera in

Table 2. Experimental environment.

Environment	Device (H/W, S/W)
IoT device	Raspberry Pi 4
Camera	NOIR 8MP B0153
GPU	GeForce RTX 2080
AI deep-learning	CNN
Face recognition algorithm	OpenCV (Haar feature)

. The results of measuring the resident’s face using the direct touch time proposal system are shown in Figure 8 and

Table 3. Resident face recognition.

DB Resident Face Image	Current Face Image			Decision
	Front Face	Left Face	Right Face
				Same

.

4.2. Experimental Conditions

The conditions for the experiment of the proposed system were 240 residents in a 30-story high-rise apartment, assuming 2 households per floor and an average of 4 families, as shown in

Table 4. Experimental conditions.

Item		Condition
Number of residents		240 person
Apartment floor height		2.8 m (30th floor = 85 m, 3rd basement floor 15 m, total building distance 100 m)
EV speed		5 m/s
EV door opening and closing time		5 s
EV direct touch time		5 s
EV busy time	AM 07:30~09:30	Resident floor (1~30) -> Destination floor (1/B1/B2/B3)
	PM 04:00~08:30	Boarding floor (1/B1/B2/B3) -> Resident floor (1~30)

. The EV usage status of two houses (4-person family) on the 30th and 29th floors are shown in

Table 5. EV usage data of residents.

Apt. Number	Resident ID	The Day Before—Return Home (After Work, After School)				The Next Day—Going Out (Going to Work, School)
		Time	EV Current Floor	EV Waiting and Boarding Floor	EV Moving Floor	Time	EV Current Floor	EV Waiting and Boarding Floor	EV Moving Floor
3001	0001	PM 20:38	5	B2	30	AM 08:00	25	30	B2
	0002	PM 18:00	20	1	30	AM 08:10	18	30	1
	0003	PM 17:20	17	1	30	AM 08:15	10	30	1
	0004	PM 21:27	19	B2	30	AM 07:40	36	30	1
3002	0005	PM 17:00	22	B1	30	AM 07:50	17	30	B1
	0006	PM 16:30	9	1	30	AM 07:35	31	30	1
	0007	PM 17:17	20	1	30	AM 08:05	25	30	1
	0008	PM 19:08	14	1	30	AM 08:10	28	30	1
2901	0009	PM 18:20	8	B1	29	AM 08:00	30	29	B1
	0010	PM 17:04	22	1	29	AM 07:45	26	29	1
	0011	PM 16:03	28	1	29	AM 07:50	37	29	1
	0012	PM 18:30	16	B2	29	AM 07:55	21	29	B2
2902	0013	PM 16:11	7	1	29	AM 08:00	35	29	1
	0014	PM 21:41	10	B3	29	AM 08:10	28	29	B3
	0015	PM 17:27	18	1	29	AM 08:15	30	29	1
	0016	PM 22:06	24	B3	29	AM 07:40	36	29	B1

.

The following contents were confirmed in Table 5, and the results of EVAI are shown in

Table 6. EVAI classification results.

Resident Transportation	EV Uses Information of Residents Learned by EVAI				EVAI’s Floor Recommendation
	Return Home the Day Before		Go out the Next Day
	EV Boarding	Exit EV	Board EV	Exit EV
Walker	Floor 1	Living floor	Living floor	Floor 1	Success (Prior day, EV boarding = Next day, exiting EV)
Vehicle user	Underground parking area (B1 or B2 or B3)	Living floor	Living floor	Underground parking area (B1 or B2 or B3)
	Floor 1	Living floor	Living floor	Underground parking area (B1)	Fail (Prior day, EV boarding ≠ Next day, exiting EV)
	Floor 1	Living floor	Living floor	Underground parking area (B2)
	Floor 1	Living floor	Living floor	Underground parking area (B3)
	Underground parking area (B1)	Living floor	Living floor	Floor 1
	Underground parking area (B1)	Living floor	Living floor	Underground parking area (B2)
	Underground parking area (B1)	Living floor	Living floor	Underground parking area (B3)
	Underground parking area (B2)	Living floor	Living floor	Floor 1
	Underground parking area (B2)	Living floor	Living floor	Underground parking area (B1)
	Underground parking area (B2)	Living floor	Living floor	Underground parking area (B3)
	Underground parking area (B3)	Living floor	Living floor	Floor 1
	Underground parking area (B3)	Living floor	Living floor	Underground parking area (B1)
	Underground parking area (B3)	Living floor	Living floor	Underground parking area (B2)

:

-

Return home, EV, travel to floor = going out, EV, waiting, and boarding floor.

-

Return home, EV, waiting, and boarding floor ≠ going out, EV, travel to floor (ID 0004, 0016).

The EVAI floor recommendation classification results are shown in Table 6. If there are 12 instances of EVAI recommendation failure for each of the 240 residents, the failure rate is 5%. EVAI verifies the resident through facial recognition when the resident returns home, stores the floor the resident moves to in a database, and automatically recommends the floor the resident moves to the next day when the resident goes out and uses the EV. In addition, when the resident goes out, the floor the resident moves to is stored, and the floor the resident moves to is automatically recommended when the resident returns home and moves the EV. Suppose 12 cases of failure occur each among 240 residents using EVs during morning rush hour; in that case, the recommended fail rate of EVAI, which has learned the latest EV usage information of residents, is 5%.

Analysis of reasons for recommendation failure:

-

If the resident does not use the vehicle when going out;

-

When a resident has two or more vehicles and places them in parking lots on different floors, and they use them when going out.

If a resident (1 person) living on the 30th floor comes out of the house and touches the elevator directly to move the elevator from the 3rd basement floor to the 30th floor, the total travel time (waiting time) of the elevator is as Equation (1).

EV direct touch time (5 s) + EV moving time (20 s) + EV door opening and closing time (4 s) = 29 s

(1)

When applying the proposed system, the non-contact time and door opening/closing time of the elevator are reduced in the same environment, so the total travel time (waiting time) is equal to Equation (2).

EV non-contact time (0.2 s) + EV moving time (20 s) + EV door opening and closing time (1 s) = 21.2 s

(2)

When comparing Equations (1) and (2), we can see that the EV waiting time is reduced by 7.8 s in the same environment where the proposed system is applied.

If a resident (4 people) living on the 5th floor directly touches the elevator on the 30th floor and moves it to the 5th floor, the total time required is Equation (3).

EV direct touch time (5 s) + EV moving time (14 s) + EV door opening and closing time (4 s) = 23 s

(3)

When the proposed system is applied, the contact time is reduced in the same environment, but the elevator door opening/closing time remains unchanged at 4 s because there are 4 people. The total travel time (waiting time) is equal to Equation (4).

EV non-contact time (0.2 s) + EV moving time (14 s) + EV door opening and closing time (4 s) = 18.2 s

(4)

When comparing Equations (3) and (4), we can see that the EV waiting time is reduced by 4.8 s in the same environment where the proposed system is applied.

4.3. Experimental Results

The experimental results are shown in Figure 9, and the average reduction time of total travel time (waiting time) of the EV was reduced by 6.3 s. Because it compares the latest resident data photos with real-time photos, the resident face recognition rate was 99%, and the resident floor recommendation success rate was 95%. The probability of failure of EVAI recommendation for 240 residents was calculated using the cumulative distribution function (CDF) (Equation (5)) based on the binomial distribution, and the results are shown in Figure 10. If there are 6 or fewer residents among 240 who fail the EVAI recommendation, the probability is 4.2%.

$$\mathrm{P}\left(\mathrm{X}\le c\right)={\sum }_{x=0}^c\left({\displaystyle \genfrac{}{}{0pt}{}{n}{x}}\right)p^x{\left(1-p\right)}^{n-x}$$

(5)

n: 240; c: EVAI’s recommendation fail residents; p: 0.05.

5. Conclusions

Recently, IoT has been combined with artificial intelligence (AI) technology to develop into intelligent IoT (Artificial Intelligence of Things, AIoT) and can provide smart services to all industries.

Generally, buildings with 30 or more floors are classified as high-rise apartments. There are high-rise apartments in many countries with small land areas and high population densities around the world, such as Hong Kong, Singapore, and Korea. Korea has many high-rise apartments and buildings with 30 or more floors, and residents there inevitably use elevators to move between floors. Because it is a high-rise apartment, it takes time for EVs to move, and a contactless method is needed when using EVs, especially due to COVID-19.

As a result of the experiment, the recognition error rate for each user and the success rate of recommending the number of users by the user was 95%, confirming the excellence of the proposal system. Based on the results of this study, it was confirmed that in the case of high-rise apartments with 30 or more floors, residents can move more quickly by reducing the time used when using the elevator during peak usage times. Additionally, it would be appropriate for the proposed system to be configured with On-Device AI rather than Cloud-based AI to protect personal privacy and achieve fast face recognition speed.

The results of this paper target high-rise apartments, and they can be used to provide convenient elevators to elevator users in general high-rise buildings or special environments (hospitals, etc.). In fact, in environments where rapid patient transportation is important, such as large hospitals, there is a problem with the general public frequently using elevators for hospital staff and patients. In particular, the waiting time to board the elevator is a major obstacle in patient transport where golden time is important, so the proposed system will be able to shorten it.

Meanwhile, research and development on facial recognition technology must continue to be able to accurately identify specific people even in various situations, and measures to strengthen security and protect personal information are also needed, so we plan to conduct related research in the future.