Internet of Things-Based Intelligent Attendance System: Framework, Practice Implementation, and Application

Abstract

Tracking coronavirus patients and determining their close contacts (as part of COVID-19 mapping) have been huge challenges. In universities, in particular, there are many students and large gatherings who are at a higher risk of obtaining COVID-19. Many smart attendance management systems have been proposed that are based on RFID and fingerprint sensor modules, facial recognition, etc. However, these techniques operate with specific requirements, such as GPUs and large memories/datasets, or by combining recognizance and thermal cameras. To solve these issues and reduce costs, we designed an Internet of Things (IoT)-based intelligent attendance management system. In this paper, we compare the advantages/disadvantages of existing smart attendance management systems. We designed an IoT-based intelligent attendance management system based on the cloud, a web server, Google API, a non-contact body temperature sensor, and the Raspberry Pi 4 module (4G). We conducted a survey at a university and summarized the satisfaction levels of using our system.

1. Introduction

According to the World Health Organization (WHO) [1], there have been 521,920,560 confirmed cases (including 6,274,323 deaths) of coronavirus disease 2019 (COVID-19), an infectious disease caused by the SARS-CoV-2 virus. The virus has spread rapidly throughout the world and is transmitted by large droplets, aerosols, and fomites [2]. Moreover, it is not easy to know where the virus is and how it mutates [3]. Consequently, there is a risk of further spreading COVID-19, particularly in unvaccinated populations. The Centers for Disease Control and Prevention (CDC) [4] has advised people to keep a distance of at least six feet from others.

Countries throughout the world have implemented preventive public health measures to control and prevent the pandemic. To avoid the spread of SARS-CoV-2, popular methods include vaccines, physical distancing (a.k.a social distancing), and health declarations [5]. As part of socioeconomic recovery, many countries have transitioned from “national social distancing” to either “less restrictive social distancing” or “social openings” because of the effects on their economic and quality of life [6]

However, with society opening back up, many challenges need to be solved; for example, how to control the spread of the pandemic or how to notify close contacts about confirmed cases (including obtaining names and how to know who is vaccinated) [7]. Because a fever is the most common symptom of coronavirus, thermometers are important when assessing health conditions. There are many types of thermometers, such as digital thermometers, oral thermometers, digital ear (tympanic) thermometers, forehead (temporal) thermometers, app-based thermometers, pacifier thermometers, and mercury (liquid in glass) thermometer [8,9]. A thermometer can indicate if one’s body temperature is higher or lower than normal. The limitation surrounding the use of thermometers (in the prevention of COVID-19) is that people could spend more time in waiting rooms to have their temperatures checked.

Traditionally, general attendance system methods involve roll calls or signing attendance sheets. For example, in university, the teacher sequentially calls out the student names. This is time-consuming, especially for large classrooms. Moreover, the transformation from paper-based data to electronic forms has been challenging. Four main methods use reliable personal identification in attendance systems: (1) biometric fingerprint verification [10,11,12,13,14,15], (2) radio frequency identification (RFID)-based technology [16,17,18,19], (3) quick response (QR) code-based technology [20,21,22,23], and (4) facial recognition [24,25,26,27,28]. First, an RFID-based attendance system can immediately verify user attendance (e.g., students place their ID cards on the reader) [16,17,18,19]. This system requires users to carry RFID tags that contain unique information. For example, an RFID-based student attendance system requires name/ID number/class/section [16,17]. The limitation of this system is that the users must keep RFID tags (and not lose them). The price of an RFID tag is one of the challenges in using this system. Second, fingerprint recognition is the most reliable method for attendance system-based biometric recognition algorithms that consist of sensing, feature extraction, and matching modules. Biometric sensors scan the biometric traits of an individual (compared to the dataset created before) [15]. This system can result in a more efficient attendance system [12,13,14,15]. However, a fingerprint-based attendance system needs integration with other systems to adapt to the Internet-of-Things era. Third, a QR code-based attendance system operates by combining equipment containing QR codes, e.g., mobile applications, and a hosting machine for checking attendance [20]. The QR code-based attendance system provides quick statistical results as well as accurate and efficient data. Nevertheless, it requires the QR code of an individual and cannot detect fake QR codes. (With a fake QR code, the ‘faker’ will use the QR code picture of another person to access the attendance system). Finally, the attendance system using facial recognition operates similar to a fingerprint-based attendance system. This system consists of face detection methods (e.g., histogram of oriented gradients (HOG) [29]) and facial recognition techniques (e.g., convolutional neural network (CNN) [30]). To be more accurate, deep learning [30] or machine learning [31] are used to track facial changes in longer time periods. Consequently, when there is a large number of people, the attendance system using facial recognition takes longer. To address these issues in the COVID-19 situation, a facial recognition–QR code–body temperature–value-based attendance system was designed and tested in real-time environments to demonstrate the accuracy and efficiency of our system.

Our contributions are as follows.

• We investigated the current attendance systems. We classified them into four main attendance systems: (1) biometric fingerprint verification, (2) RFID-based technology, (3) QR code-based technology, and (4) facial recognition. We compared their advantages/disadvantages.
• We designed the framework of our proposal. Our system is based on facial recognition, QR codes, and non-contact body temperature sensors. Moreover, we used a database, web server, Google Mail, and an existing package in Python to design an intelligent system.
• Our proposal can check attendance in flexible ways, via facial recognition and QR codes. Moreover, people can conduct their attendance manually when facial recognition or QR codes cannot accurately operate.
• Our proposal is user-friendly and the attendance results can be shared via email. The manager can also collect all attendance results via his/her email.
• We also practiced implementation in real-time environments and collected our survey. The results show that all customers evaluated our system as highly accurate and efficient.

The rest of this paper is organized as follows: Section 2 provides a short survey of the smart attendance systems; Section 3 describes the proposed conceptual design of an IoT-based intelligent system; Section 4 presents the implementation phase of our system; Section 5 presents the performance evaluation, and Section 6 discusses our proposal. Finally, Section 7 concludes the paper.

2. Related Works

We classified the attendance system (according to the different equipment requirements and techniques) into four categories: (1) biometric fingerprint verification [10,11,12,13,14,15], (2) RFID-based technology [16,17,18,19], (3) QR code-based technology [20,21,22,23], and (4) facial recognition [24,25,26,27,28], as shown in Figure 1.

Using the Arduino, real-time clock (RTC) module, LCDs, and web-based applications [16,19], the system reads the RFID card and verifies it with the dataset in the microcontroller. However, the author did not clearly explain how to log and fetch data on the server/cloud; moreover, the microcontroller (from the 8051 families) had limited capacity. Therefore, it could not monitor student attendance on a large scale. Furthermore, regarding a Java Library to reduce the RFID tags collisions, the authors in [17] proposed a new anti-collision protocol by combining dynamically-framed slotted ALOHA (DFSA) and basic-framed slotted ALOHA (BFSA) methods. In [18], a low-cost event attendance system was designed by using a low-cost Wi-Fi chip with a full TCP/IP stack and capability, the ESP8266 module. The authors also proposed data analytics delivered by the system server.

Fingerprints have special characteristics that are unique to every person and do not change throughout one’s lifetime, barring an accident [15]. Thus, a fingerprint system is the best/fastest method for an attendance system. In [15], the authors combined a fingerprint reader and Arduino Wemos D1 R2; the system could access the database through Wi-Fi. This system could be implemented in large databases. On the other hand, the attendance system in [10] is a PC or server used to manage fingerprint devices and the attendance of students. To reduce the daily error rate, the fuzzy-based attendance system was proposed and implemented using the fingerprint device, .NET C# web service, Oracle database, and ZKTeco U260-C [14]. Moreover, the attendance system could integrate a Wi-Fi module to create an access point (in the case of no coverage) to increase the possibility of error-free attendance logging [13].

The QR code-based attendance system was developed for quick readability and greater storage capacities [20,21,22,23]. In the QR image, the data are presented in both horizontal and vertical components [23]. In [20,23], the authors developed the QR code-based attendance system by integrating the global positioning system (GPS) to avoid false registrations. Combined with the server, the identity of users can be verified and stored [21,22]. The facial images need to be stored in the presence of a fake student.

A facial recognition-based attendance system is considered an effective system in biometric verification based on pattern recognition and computer vision development [24]. Many algorithms are recognized, such as HOG [25], deep metric learning [26], a pre-trained Haar cascade model [27], and the eigenvalue face approach [28]. A facial recognition-based attendance system operates in the following steps: (1) capture the image, (2) detect the face, (3) recognize it with the database, and (4) mark the attendance. Complex facial recognition algorithms, such as deep learning and machine learning, require high CPU processing. Therefore, deployment of this system requires specific equipment, such as computers, laptops, and phones.

During the COVID-19 pandemic, we designed an IoT-based intelligent attendance system to provide a COVID-free workspace to employees and a ‘touchless’ check-in. We combined facial recognition–QR code–body temperature–value-based technologies and web applications to deploy our system. Cloud computing with database and data analytics is integrated into our system to provide effective and friendly operations. We will explain our system in detail in the following section.

3. Architecture of IoT-Based Intelligent System

With the development of the cloud-computing framework, there are many mobile cloud applications; for example, web browsing, email access, video playback, image editing, Google Maps, Gmail for iPhone, etc. [32,33,34,35] Following [32,33], the architecture of cloud computing is divided into three layers: infrastructure, platform, and application layers. Our proposal divides the cloud-based attendance systems into three-layer: end-user, wireless attendance device, and cloud computing layers, as shown in Figure 2.

In our architecture, the first layer involves end-users who use mobile devices or are assigned face-based marks. The next layer consists of wireless attendance devices, which can connect directly to Wi-Fi signals and integrate with a non-contact human body infrared IR temperature sensor module, such as MLX90614 [36]. The cloud computing layer supports the database, web server, application server, and file server in Figure 2. The end-users can access it to check their information and daily attendance marks. Our system allows the end-users to receive individual summation attendance results via their personal emails. Moreover, the manager can access the file server to calculate the salary for the current month

Based on upper architecture, we propose the system design of an IoT-based intelligent system in Figure 3. The end-users can flexibly choose their attendance checking options. If they registered facial recognition before, they could check by using facial recognition, otherwise, they must use a QR code. On the other hand, the non-contact human body infrared IR temperature sensor module will check body temperature immediately. After an end-user presses the “register” button, the system will check, save, or update new information. Moreover, our system can allow users to inquire about their attendance results. Moreover, they can send it to their personal emails.

4. Implementation of an IoT-Based Intelligent System

In this section, we discuss how to implement our design in real-time and adapt it with existing equipment.

4.1. QR Code

There are many methods used to generate encrypted QR codes with specific information [20,21,22,23]. One main method is via a web application. In the Python environment, it is easy to generate [37]. Our system requires specific information consisting of name, identification, birthday, gender, the number of vaccine doses, and phone number.

Governments throughout the world require health declarations in various ways, e.g., via web applications and mobile messages. In Vietnam, the government requires citizens to make health declarations via PC-COVID: Vietnam’s unified app for COVID-19 prevention and control [38]. It contains all COVID-19 requirement information and generates QR codes to track citizens. Citizens can scan QR codes before going to popular places, such as shopping malls, markets, and restaurants. It is a popular application that has around 56,000 download (iPhone) and 10,000,000 downloads (Google Play) (accessed 31 May 2022). Therefore, our system is based on the QR code as a source for the attendance system. In Figure 4, a QR code-generated PC COVID application consists of “001***374|NGUYEN VAN DUNG|1985-03-12|0|163313888001101602|<1<00<<094***949”. The main data we need are the individual’s name, e.g., NGUYEN VAN DUNG; ID, e.g., 001***374, birthday, e.g., 1985-03-12; gender, e.g., 0 (0: male; 1: female); the number of vaccinated doses, e.g., 1; and phone number, e.g., 094***949. Those who use our system can easily make QR codes by using the package in [37] or a web application.

However, QR detection and recognition do not operate well in complex scenes with more than one QR code skewed by perspective and rotation [39] because the detection and recognition of the codes are different. Therefore, users have to change their perspectives and rotations until the display shows the correct information of QR codes. Consequently, the QR code-based technology is time-consuming. To trade between efficiency and accuracy, we combined facial recognition systems and displayed the decoded information, described in the next section.

4.2. Facial Recognition System

There have been many methods used in facial recognition, such as deep learning, machine learning, and CNN [29,30,31]. A facial recognition system is recommended for a COVID-free workspace and ‘touchless’ check-in. Passive identification is one main advantage of the attendance system. It means a user cannot conduct an action for reporting identification. A facial recognition system captures frames via a live stream and recognizes them in the existing dataset. However, there are disadvantages, e.g., it requires a dataset for individual people. Therefore, it requires more memory to store the dataset. With complex facial recognition algorithms, it spends more time with large datasets. It delays the attendance system response when there is a large amount of people/employees/students.

According to [24], the facial detection–recognition rates do not have highly accurate levels. For example, the facial detection–recognition rates using face eigenvalue were 79% and 65% [28]. The facial recognition rate using the local binary pattern histogram algorithm (LBPH) algorithm is 77% [40]. To improve the facial detection–recognition rates, they need to integrate another method, i.e., the recognition library with the KNN algorithm gives a 97.35% precision rate [41]. However, it spends a long time processing and it cannot adapt to the low response times of the attendance system.

To reduce the facial detection–recognition rates and provide a response time for the attendance system, we used the facial recognition method (HOG) and retrieved information from the database. All information is displayed in a web application; the user needs to confirm his/her information before marking attendance by pushing the “Save” button or cancel by pushing the “No” button. If the information is in error, we require the QR code from the PC COVID application. We also include health declarations and require users to fill in the information.

4.3. Non-Contact Human Body Temperature Sensor Module

To control the COVID-19 pandemic, many companies and organizations require automated systems with integrated attendance functions and temperature measurements [42]. In particular, a non-contact human body infrared IR temperature sensor module must be used to prevent the spread of coronavirus [42,43,44,45]. Among these sensors, Melexis MLX90614 [36] (using inter-integrated circuit (I2C) protocols) is popular. It also has an application-specific standard product (ASSP) digital output. This module was chosen over other devices because of its small TO-39 package and suitable measurement ranges of −70 ${}^{\circ }$C to 380 ${}^{\circ }$C, which fits the measurements of humans [36,44], as shown in Figure 5. The operation can be summarized as follows. First, the system sends an I2C request message to the thermopile to initialize it. The MLX90614 module will measure it; when it is finished, it sends an ASSP message to the microprocessor via an I2C wire (SDA and SCL pins) [44].

We acknowledge the limitations of our study, including how high body temperatures (due to other reasons or infections) are distinguished. However, a fever is the most common symptom of coronavirus. If a high body temperature is sensed, the system will notify the patient. The patient has to go to the hospital for a COVID-19 test to obtain accurate results.

We summarize the integrated functions of facial recognition and temperature measurements in Figure 6. The system, after collecting personal information, combines body temperature measurements sent to the web application; they will be updated/saved in the cloud database.

4.4. Cloud Server

In our system, the cloud server is used to store the database and web applications. The cloud server allows for easier marking, checking, and inquiries. It consists of a web application, database, file server, and email server. The web application will contain the user’s information and her/his body temperature measurements. The user can choose options in our web application: (a) register, (b) clear, (c) manual, (d) check, (e) email, and (f) face dataset, as shown in Figure 7. We describe each option in

Table 1. Options in our attendance system.

Option	Description
Register	Mark or update attendance.
Clear	Clear information display.
Manual	Fill-up information to the system.
Check	Check attendance results to date.
Email	Send to private email.
Face dataset	Create dataset for facial recognition.

.

The database is used to store customer/student information in the cloud. Moreover, the file is created and updated for new attendance. These functions allow users to inquire about their attendance results. Moreover, the file can be created for managers to check and ‘make salaries’ for employees; the teacher could check all student attendance results. One such limitation of the cloud server is that obtaining separate emails for a large number of users, again and again, may be problematic for managers. In the future, we will embed the attendance reports into the student’s own page (on the university’s website).

Finally, we designed diagrams to mark attendance operations and send the results via email, as shown in Figure 8 and Figure 9. Figure 8 shows how to access and recognize QR codes or facial recognition (from the user to the cloud). The user information is marked autonomously and stored in the database. It helps to check and save the user’s attendance information, e.g., personal information and body temperature measurements. The user can choose to save/update/cancel his/her information in the attendance system.

Figure 9 shows the method to filter a specific user’s attendance. The user can check her/his attendance results and send the information back to her/his email to review. A file is also made and stored temporarily in the file server; if the user wants to send it, it will be sent and then deleted to protect the user. Otherwise, it will be deleted autonomously.

5. Evaluation of IoT-Based Intelligent System

In this section, we discuss our requirements and devices; we tested our system in a real-time environment. To make a low-cost attendance system comparable with existing attendance systems, we used the devices listed in

Table 2. Equipment using the attendance system.

Modules Used	Description
Raspberry Pi 4 Model B	8 GB LPDDR4-2400 SDRAM. Broadcom BCM2711, quad-core Cortex-A72 (ARM v8) 64-bit SoC 1.5 GHz [49].
Raspberry Pi 7″ Touchscreen display	The 800 × 480 display
Melexis MLX90614 [36]	Inter-integrated Circuit (I2C) protocol.
Language programming	Python version 3.9 [50]
Flask package [46]	Web development
MySQL packet [47]	Create database
HTTP servers [48]	Send F = files

. We used Python version 3.9 integrated into Raspberry Pi 4 Model B. To measure body temperature, we wrote our Python codes to adjust to real-time environments. Moreover, some packages embedded in Python were used, such as the Flask package [46], MySQL packet [47], and HTTP servers [48].

Figure 10 shows all functions of our design system. It also measures body temperature in real-time and reads QR codes/facial recognition by capturing the frames via the live stream. This figure shows the QR code extraction results. All needed information is shown on the left-side; the user can check it again.

After reading the QR code/facial recognition, the user checks her/his information again and pushes the “Register” button, the registration page will be displayed as in Figure 11. Our system will show the user’s information again to confirm after it is saved in the database. The user can choose the “Yes” option or the “No” option as shown in Figure 11.

To support a dynamic option and eliminate the errors from the QR code reader or facial recognition, we designed a “Manual” option, as shown in Figure 12. The user is required to fill out all information; then it will be checked again to be saved in the database. This option will spend more time filling-up, but the user will not miss attendance if the system operates in error.

To announce the attendance results, we show the results page, as shown in Figure 13. The user will know her/his status and the attendance mark stored in the database. Finally, the results page will return to the main page.

The user can check all of his/her attendance results by pressing the “Check” option on the main page in Figure 14. The check page will be displayed. On this page, the user will fill out her/his ID and click on search. Our system will find and filter results based on a given ID. The check results will be shown on the search results page, as shown in Figure 15. To save time, the user can send the current check results by filling out his/her email address. Afterward, the system will send the results file to his/her email if it exists.

To apply a facial recognition system, the user can create a dataset. Figure 16 shows three steps to creating the dataset. First, the user has to fill out all information in the left-side display. Afterward, the user presses the “submit” button to create his/her dataset information. On the middle page, the live stream camera is operating; the user has to move in order for the system to detect his/her face. If the face is detected, the user’s facial image is captured. Our system requires at least six photos to create a dataset because it will increase the face’s accuracy level.

Finally, we compare two methods—HOG [25] and CNN [26] algorithms—under Raspberry Pi 4 Model B [49], as shown in Figure 17. A database file used in the facial recognition algorithm is created by running the HOG or CNN methods. Under Raspberry Pi 4 processing, the CNN time is very long compared with the HOG algorithm. With 48 images, CNN takes 420 s to create an encoded faces file while HOG spends 64 s. Figure 8 demonstrates that the CNN does not adapt to Raspberry Pi 4 because of the efficient response time. Therefore, we recommend that facial recognition uses the HOG algorithm. Although the face detection–recognition rate is not a highly accurate level, our system allows the user to confirm his/her information before it will be stored in a database. Moreover, we combined QR code readers to increase the highly accurate levels of the attendance system.

To fairly evaluate and check the efficiency of the attendance system, we created a survey at TON DUC THANG University in Figure 18. We distributed 300 forms and collected 300 forms. After eliminating the forms with errors, we had 285 correct forms. We summarize our survey results in

Table 3. Summary of survey results.

	1	2	3	4	5
Communication with customers	1	2	3	4	5
Q (1.1) How satisfied are you with the display of our system?	0	0	17	178	90
Q (1.2) How was the response time of our system?	0	0	68	125	92
Q (1.3) How satisfied are you with the registration time of our system?	0	0	158	78	49
Q (1.4) How satisfied were you with checking your information again?	0	0	74	158	53
Performance	1	2	3	4	5
Q (2.1) How long did it take for you to finish the registration form?	0	124	62	87	12
Q (2.2) How satisfied were you with filling out the form in the manual option?	0	0	147	103	35
Q (2.3) How satisfied are you with the performance of the system?	0	0	0	87	198
Q (2.4) Did our system operate correctly when it scanned your QR code?	0	0	0	0	285
Novelty	YES		NO
(1) Have you used the same system?	285		0
(2) Is it comfortable in your university life, i.e., roll call. ?	285		0
(3) This system can help prevent the spread of COVID-19 by tracking all students who stayed in that room with a patient.	285		0
Security	1 (Beginner)	2 (Easy)	3 (Normal)	4 (Hard)	5 (Very hard)
Q (3.1) What is the security level of our system?	0	157	80	23	25
Q (3.2) Someone can check your absence, i.e., your parent, your friend?	0	114	145	15	11
Q (3.3) Someone can modify/delete your roll call?	0	0	0	275	10

. The bold text means the highest number is highlighted. Based on these results, we evaluated our system satisfaction level, as shown in Figure 19. Our system was overvalued in real-time tests. When communicating with customers, and regarding performance evaluation, most students felt ‘neutral’ with our system. Using QR codes, our system showed 100% correct information. Since all students can send emails to their email addresses and ‘check’ other people, the security level was evaluated as easy. In the future, we will create a log-in for managers or specific persons. If one displays his/her ID, a faker can access it and check the person’s attendance results. However, our system prevents someone from modifying or deleting a user’s attendance results. Finally, we will improve our system (e.g., pertaining to security) in future works.

6. Discussions

In this paper, we proposed the Internet of Things (IoT)-based intelligent attendance management system by combining facial recognition, QR code, and non-contact body temperature sensors. Moreover, we used a database, web server, Google mail, and an existing package in Python to design an intelligent system. Our proposal can check attendance in a flexible way, via either facial recognition or a QR code. Moreover, people can conduct attendance manually when facial recognition or QR codes cannot operate accurately. Our proposal is user-friendly and results can be shared via email. The manager also collects all attendance results via his/her email. We also practiced implementation in real-time environments and conducted a survey. The results show that all customers evaluate our system at high accurate and efficient levels.

However, there are open challenges that can be solved, as follows.

• Security: If a fake has a user’s ID, he can see the user’s attendance results. However, this problem can be solved by designing a two-step verification, such as confirmation via a Google account. On the other hand, the individual’s personal data can be protected based on the personal data protection act (PDPA) or general data protection regulation (GDPR).
• Accurate prediction: Our system is based on body temperature, e.g., to predict COVID-19 patients. However, the limitation of our study is how to confirm that the high body temperature is due to another reason (other than COVID). In this case, we can use the COVID-19 rapid test kit.
• Overload: Obtaining separate emails for a large number of users, again and again, may be a problem for managers. To solve this issue, we embedded the attendance reports into the student’s own page on the university’s website.
• Distance sensors: According to the datasheet for MLX90614, the sensor without any lenses could measure a distance maximum of up to 3 cm. To increase the distance of the measurement, we could use a copper lens hood.
• Accuracy: By comparing HOG and CNN, we observed that HOG outperforms CNN in terms of processing times. However, CNN provides more accurate results than HOG [51]. To increase our system performance, we could use more powerful hardware (NVIDIA and Google have some products, e.g., Edge TPU, and Jetson) to optimize CNN and reduce the response time.

7. Conclusions

In this paper, we designed an IoT-based intelligent attendance system that supports universities/companies that provide COVID-free workspaces and autonomous check-ins. Our system consists of attendance-taking, temperature measurements, and cloud computing, such as web applications, databases, and email servers. Our system can help managers/doctors control the COVID-19 pandemic by tracking/monitoring body temperatures (because a high temperature is a common symptom of COVID-19). Our system dynamically combines QR codes, facial recognition, and manual registration support for users. Moreover, all data can be stored, retrieved, filtered, and inquired easily via cloud computing. The evaluation results show our system is efficient, comfortable, and friendly to use in a real-time environment. In future work, we will apply AI-based analyses, prediction of infection, and contact tracing. For a large number of users, the optimal number of IoT devices used can be considered to satisfy the trade-off between the system performance (accuracy, latency, user satisfaction, etc.).