1. Introduction
The digital technologies that are deployed in smart home environments provide a lot of benefits for the residents, including efficiency and comfort enhancement [
1,
2]. The main objective of this platform is automation and energy management [
3,
4]; however, it can also provide different services for other applications, such as health and security. This platform uses different wireless sensors (that are mainly installed in the house), such as motion, power, humidity, and temperature, to continuously monitor residents and record their events. These recorded raw data will be used to recognize the residents’ activities and detect any abnormal event that requires further actions [
5].
There are several challenges in the activity recognition research area that are still the focus of researchers around the world. One of the main challenges that exists includes activity recognition in a smart home with multiple residents. When there is only one resident, all of the recognized activities can be easily assigned to a certain profile. However, when there are multiple residents living in the same home, their activities cannot be properly recognized and separated, as the sensors’ data do not provide any information about the identity of the person who triggers the sensor. Therefore, to have accurate activity recognition in a smart home with multiple residents, an approach is needed to distinguish between different residents, while helping to label the sensor data. Individual localization and identification technologies are suitable options to be applied for this purpose.
Several types of approaches that can provide identity (ID) information are introduced in the literature. These approaches include the following:
Vision-based approach: This approach uses cameras to monitor multiple residents while recognizing their activities [
6]. Biometric information, such as height and face, are the features that are used to identify multiple residents. The main issue with this approach is privacy.
Signature-based approach: In this type of approach, technologies such as microphone [
7], passive infrared (PIR) [
8], ultrasound [
9], and ultra-wide band (UWB) [
10] are used to generate a signature for residents. It is shown that this signature is unique and can be used to distinguish between residents while they are moving around the house. The accuracy and capability of this approach in a real smart home is still the focus of researchers’ attention.
Tag-based approach: In this approach, targets carry a tag as a unique ID, which uses different communication technologies, such as radio-frequency identification (RFID) [
11,
12,
13], Bluetooth [
14,
15,
16,
17,
18,
19,
20], ZigBee, and Wi-Fi [
21,
22,
23,
24], to broadcast their unique ID in a specific time interval. Additionally, there are scanners allocated in different areas that listen to the broadcasting packets to localize the tag and the individual.
For the application of these technologies in smart homes, the authors of [
11] use an RFID tag as the unique ID for the resident, while a received signal strength indication (
RSSI) index is used to find the resident’s location in a smart home. In a similar fashion, the authors of [
12] present the latest research progress in an effort to include RFID antenna gain patterns in model-based received signal strength (RSS) localization algorithms. In the same work, it is shown that real-world, non-isotropic gain patterns of RFID badges are not a problem to be ignored, but a means to improve localization accuracy. A demo on fine-grained RFID localization via UWB emulation is presented in [
13], where the proposed system allows users to move RFID-tagged objects to any location in line-of-sight, non-line-of-sight, and multi-path rich settings, and check that the system can accurately localize the objects. Another tag-based approach—based on the use of emerging Bluetooth low energy (BLE) devices (beacons), coupled with a mobile app and its intrinsic features—is outlined in [
15]. It is recommended by the authors that to ease the localization, appropriate filtering or data processing after the initial distance estimation should be carried out. The researchers in [
16] propose a hybrid real-time indoor localization architecture, based on smartphone and BLE beacons exploiting the complementary characteristics of the proximity (RSS method) and pedestrian dead-reckoning (PDR) approaches. Likewise, the authors of [
17] report the design, implementation, and evaluation of an Internet of Things (IoT)-based system for indoor localization using BLE technology. Although the proposed approach could estimate the position with good precision, it requires a long training process and the use of different machine learning classifiers, which makes it inefficient for application in big office halls or larger areas. The performance of BLE technology for indoor positioning systems in different transmission conditions is also discussed in [
18], where the localization method is implemented based on the
RSSI value. In the same subject area, the researchers in [
19] propose a method to render indoor localization and tracking practical, using only BLE sensors. In the same work, a distribution of
RSSI values (fingerprint measurements) for every position inside the region of interest is estimated, and a direct relationship between the accuracy of the radio map and the accuracy of localization and tracking is extracted. However, as in the proposed approach, initial scene analysis is a must because it is not practical to collect data from a high number of locations.
The authors of [
23] use the smartphone and its Bluetooth technology as the tag for the resident. The work also proposes that Bluetooth should be used rather than Wi-Fi, as it can provide good accuracy for room-level localization because of its limited energy level, which is usually limited to within a single room. On the other hand, the authors of [
24] present a positioning method with special wireless signal module configuration rules to reduce the complexity of the calculation process, and the running time. However, the proposed approach suffers from an error that is caused by the energy error due to the distant nodes’ positions.
As can be understood from the reviewed literature, there are plenty of methods and technologies to detect and monitor devices/people in an indoor environment, and each of them has its own pros and cons. However, in the case of healthcare systems, especially those applied for aged care, it is recommended to use technologies that fulfill requirements such as low cost, low power, high accuracy and reliability, real time operation, and good resistance to fading (caused by multipath propagation) [
25]. Based on these requirements, BLE is one of the promising technologies.
In this paper, BLE devices are used as the tag for multiple residents to localize and label their activities in a smart home. This solution not only provides an efficient way of transferring messages with low energy, but also gives high positioning accuracy, simple position estimation, and a lightweight and easily deployable tracking system. Compared to the reviewed literature, the contributions of this work can be summarized as follows:
Room-level accuracy is considered as the required accuracy for localization in smart home applications. In this system, the resident is carrying a BLE tag as his/her identity, which broadcasts a unique ID in a specific time interval.
To handle energy error issues, BLE scanners are optimally allocated in each room; they listen to the advertising packet and RSSI index to detect the presence of the resident in a given room.
A comprehensive study is conducted to examine the impacts of broadcasting parameters on RSSI and to design a high accuracy localization approach.
The possibility of knowledge extraction from other sensory networks for better localization is explored. As there are activity sensors installed in different rooms for the purpose of activity recognition, it is shown that the data from these sensors can also be used to improve the accuracy of localization and activity labeling.
2. Localization in Smart Homes Using BLE Technology
BLE technology is a promising communication technology for future smart homes because of its low energy consumption [
26]. This technology, due to its low data rate and low energy consumption, is applied in several studies for the purpose of indoor localization. To this end, the researchers in [
27] provide a comprehensive study on the performance of this technology for indoor localization, while the authors of [
28] make a comparative study with Wi-Fi for the purpose of indoor localization.
Figure 1 shows the implemented structure of the multi-residential localization approach in the Commonwealth Scientific and Industrial Research Organization (CSIRO) smart safer home (SSH) platforms [
29] using BLE technology. As shown in this figure, this structure includes two main parts. The first part includes BLE tags that are generic iBeacons carried by the residents. Tags have a MAC address as a unique ID, which is considered as the identity of the resident. Tags advertise this unique ID in a fixed time interval (advertising interval (
TA)).
The second part includes BLE Scanners, which are Android mobile phones in this case. As room-level accuracy is required, one scanner is considered for each room. A mobile application is developed, as the scanner, to listen to the BLE tags’ advertisements every second at the scanning time interval (
TS). During the scanning duration, if there is any BLE tag that is in the range of a BLE scanner, it has the possibility of being detected. For instance, in
Figure 2, BLE tag
i has the possibility of being detected by the scanner, while BLE tag
j cannot be detected. Beside this ranging criterion, there are other criteria that need to be met for the BLE tag to be detected by the scanner. The details of these criteria can be found in [
14].
Scanners record the data of each BLE tag, including the time of scanning, tag ID (MAC address), and RSSI, and send them to the CSIRO database for further processing. This set-up is implemented as part of the SSH platform to see how these data, and data from activity sensors, can be used for accurate activity labelling of multiple residents.
The objective of this paper is to localize the residents at each time step of T. In other words, the aim is to localize the residents in different rooms during each time period (T), if they are at home. The value of T depends on the minimum time that a resident is in a room. For instance, if the individuals are likely to stay in a room for at least 30 s before transitioning to different rooms, the maximum time step T for the localization should be 30 s, in order to have an acceptable localization accuracy.
As the scanning zones of the scanners may interfere with each other, one tag may be scanned by two or more scanners in the same time step.
RSSI will be used to associate the tag with the closest scanner and room. Additionally, as the advertising interval is usually less than
T, the scanner may record more than one
RSSI for each tag in each time step
T. Therefore, to decide on the location of the tag, the representative
RSSI value should be considered in each
T. Two representative indices are used in this paper. The first index is named as
p-
RSSI index that considers the maximum
RSSI as the representative
RSSI of the tag scanned by the scanner in each
T. If there are
k recorded
RSSI values for tag
i in a given time
T, then the representative
RSSI for this tag is the maximum
RSSI during this period, as given in
Figure 2.
The second index (which is named as
m-
RSSI) calculates the average of the
RSSIs based on their histogram, as shown in
Figure 3.
The algorithm used in the proposed structure includes two main parts to detect and localize each resident. The first part in this algorithm uses raw BLE data and the representative RSSI to localize the residents in each room during each T. This algorithm includes the following steps for each tag:
Step 1: For each time interval T, all of the RSSI for the tag, scanned by different scanners, are recorded.
Step 2: The representative
RSSI in each scanner is determined based on either the
p-
RSSI or the
m-
RSSI model, shown in
Figure 2 and
Figure 3.
Step 3: Based on the representative RSSI of the tag, the resident is in the room with the scanner with the maximum RSSI value.
The flowchart of localization using the
p-
RSSI index is shown in
Figure 4.
5. Multi-Residential Activity Labelling Using BLE Localization Technology
The proposed multi-residential localization approach has been trialed in a real smart home. There are two residents that are living in this home. The architecture of this smart home is shown in
Figure 10. Three types of smart home sensors—including five motion sensors, two switch sensors, and one power sensor—are used to monitor the residents’ activities. Additionally, five BLE scanners are used to localize the residents in the main monitoring area. The main monitoring area includes a Kitchen, Study room, Living room, Laundry, and Bedroom 1, as residents spend most of their time in these areas. For the purpose of this paper, if residents use the Bathroom or Storage, then they will be considered to be in Bedroom 1. Moreover, presence in the Office and Bedroom 2 will be associated with the room with the closest scanner, which is the Living room in this case.
The main aim of this trial is to study some of the main features of multi-residential activity labelling through BLE localization. The advertising interval for the tags is set to 1 s. The set-up is trialed for 1 week. The localization results for the whole week are shown in
Figure 11. To see more details, three sample days are shown.
Figure 12 shows the results of localization and activity labelling for the residents on the seventh day.
Figure 12a shows the localization results of the residents in different rooms during this day. It can be seen that resident 2 leaves Bedroom at 06:25 a.m., does some activities in the Laundry, Kitchen, and Living room, and then leaves the house at 11:50 a.m. While resident 1 wakes up at 09:17 a.m. and then stays in the home for the whole day. Based on the localization data, the events that are generated by the activity sensors are labelled using the individual’s location.
Figure 12b shows those activities that cannot be labelled based on the localization data. Based on this figure, there are six sensor activities (out of 239 events during this day) that cannot be labelled by any resident. This is because we cannot be 100% accurate in localizing and labelling the activities, as the
RSSI changes quickly or we may have missing
RSSI. Also, these two residents are seniors and they will have community service providers that are working at their home from time to time. These “activities” may be caused by those caregivers and thus cannot be correctly labelled through
RSSI. Based on these results, it can be said that the residents were carrying their tags during the whole day, as most of the sensor data can be labelled by the identity of the individuals.
Figure 13 shows the results for the fourth day. It can be seen that there are a lot of sensor events that cannot be correctly labelled based on localization data. It can be predicted that residents take off their tags at night, while forgetting to wear them in the morning. For instance, based on the tag localization, it can be said that resident 2 left the Bedroom at 06:28 a.m. However, there are some events that occur before this time which can be considered as attributable to the resident of interest as the other resident usually wakes up around 10:00 a.m. The same thing is happening with the second resident, as there are some events that occur before the tag leaves the Bedroom (10:59 a.m.), while the other resident is not at home. It can be seen that these two sequences of events can be associated with the residents to improve the performance of this approach. In conclusion, it can be said that forgetting the tag can clearly impact the performance of sensor data labelling.
Figure 14 shows a worst-case scenario, which is the sixth day. It can be seen that there are a lot of sensor events that cannot be labelled during this day. The missing events include two main parts. The first part is when resident 2 wakes up, which can be associated with this resident as the other resident is still asleep. However, it can be seen that after 10:00 a.m., resident 2 is in the Bedroom for the whole day. It can be said that this resident left the tag in Bedroom for the whole day. Therefore, there are a lot of activities that cannot be associated with the residents. In some cases (like the previous scenario), forgetting the tag can be detected and can be considered to improve the localization. However, in some cases, it is very difficult to associate the activities with the residents.