5.2. Experimental Setting
This experiment aims to evaluate the resulting performance of the emergency incident detection and monitoring mechanisms enabled by the adoption of the IoT2Edge data interoperability approach. For the evaluation of the proposed mechanisms, heterogeneous platforms that contribute to the information completeness enhancing the performance of the dynamic decision making engine are engaged. The decision engine is expected to automatically assess how critical the emergency situation is, based on a predefined scale (i.e., low, medium, high). It is foreseen that the confidence and promptness of the reasoning outcome will be higher and more accurate as the richness of the available information increases. In this approach, the following benefits are expected to be demonstrated: data interoperability of the IoT and social media services in support of cross application domain reasoning, through the homogenisation of information and automated identification of an emergency situation. The experiment envisages a semi-rural area where various information platforms are able to collect a wide variety of data related with local environmental parameters, social media activity, weather forecast, and fire danger index. In more detail, and as illustrated in
Figure 10, the following platforms are considered:
The intelligent Mobile Environmental Sensing Platform (MESP): The MESP aims to advance IoT platforms by introducing intelligent behavior and actuation in the process of data sensing. It consists of a unique central gateway realised on Raspberry Pi 3 B+ platform, called MESP-R, that acts as an aggregator of data collected from various Arduino platforms called MESP-A.
MESP-R: A Raspberry Pi 3 Model B (
https://www.arduino.cc/en/Main/Products) operates with the latest operating system “2018-11-13-raspbian-stretch-full”, while communication with the MESP-A is facilitated through a Xbee Zigbee S2 PRO 63 mW with an embedded wire antenna module. The Xbee module can achieve a range of 2 miles on a Line-of-sight, with a maximum bitrate of 250 kbps, which is more than sufficient for our experiment setting. In addition, a Raspberry Pi Camera IR-CUT is attached to the MESP-R, which is equipped with a 5 Megapixel OV5647 sensor. The sensor’s maximum resolution is 1080p and supports the connection of an IR LED, so that it can operate in a low lux (night vision) environment. Finally, an INA219 current sensor is attached for measuring voltage, current and power consumption utilised for the evaluation of the introduced mechanisms.
Figure 11 illustrates the setup of the MESP-R in detail.
MESP-A: The core component of the MESP-A is an Arduino MEGA 2560 R3 (
https://www.arduino.cc/en/Main/Products) which performs the aggregation, processing and communication of the sensed data. Attached there are 4 IR flame sensors, which operate on the IR spectrum range between 760 nm and 1100 nm, with a 60-degre detection angle. An MQ-2 Gas Sensor Module, which is sensitive to LPG, propane, hydrogen, and smoke, is also attached. Its output voltage boosts along with the concentration of the measured gases increases. MESP-A also consists of a DHT11 Sensor, which is used for detecting ambient temperature and humidity through a standard single-wire interface. For measuring soil temperature, we use the DS18B20 waterproof temperature sensor. Location information is recorded through the attached Adafruit Ultimate GPS Breakout version 3. Communication with the MESP-R is realised through an Xbee Zigbee S2 PRO 63 mW with an embedded wire antenna. Finally, a rocker ON-OFF switch is attached to the project box which enclosed the MESP-A. Energy is supplied to the system by a power bank 5.1Volt/2.5 Amperes with 6.900 mAh.
Figure 12 illustrates the setup of the MESP-A in more detail.
The rationale for constructing the MESP device is to have a multi-purpose micro-environmental sensing prototype useful for testing and evaluating various deployments and data management approaches. MESP demonstrates both the capabilities as well as the limitations of IoT platforms, hence it is considered as an appropriate test system for assessing different IoT-related mechanisms, e.g., the described data interoperability mechanisms. Within the scope of the wild-fire detection scenario, MESP operates as a sensing device deployed in the perimeter of a house located at a semirural area, i.e., a house surrounded by trees, close to a forest (as was the case in Rafina’s forest-fire and California’s camp fire incident). The camera controlled by MESP simulates a CCTV system monitoring the perimeter of the house. For the needs of the described experiment an IoT2Edge-Agent was deployed on MESP-R in order to retrieve, translate, process and disseminate the recorded environmental data.
OpenWeather Map API: The OpenWeatherMap (
https://openweathermap.org/) is an open platform that provides current and forecasted weather information through API calls. Within the scope of the conducted experiment, this API was utilised for accessing current weather data for locations specified through their GPS coordinates. OpenWeatherMap platform aggregates weather data from more than 40.000 stations deployed all over the planet while the results are provided in many forms (e.g., JSON, XML, HTML). For the needs of the described experiment, an IoT2Edge-Agent was connected to the OpenWeatherMap API in order to retrieve current and forecasted weather data for a specific location.
EFFIS API: The European Forest Fire Information System (EFFIS) [
68] offers updated and reliable information services about wildland fires in Europe. The EFFIS service provides maps for active fires and burnt areas for a specific number of past days, while it also provides fire danger forecasts. These forecasts are based on the Canadian Forest Fire Weather Index (FWI) System where past, current and future weather information are aggregated with location specific information (e.g., history of past fires, availability of fuels in the area, vegetation, etc.) and they are generated as daily maps using numerical fire danger predictions encoded in GeoTIFF maps. Fire danger is mapped in 6 classes (very low, low, medium, high, very high and extreme) with a spatial resolution of about 16 km. The EFFIS FWI data are retrieved by a dedicated IoT2Edge-Agent module through the respective API.
Twitter API: For the needs of this experiment the Twitter micro-blogging platform has been integrated. Twitter maintains a total number of 335 million monthly active users, who produce more than 500 million Tweets per day. The fact that 80 percent of Twitter users utilise the service through mobile devices, makes this social network an ideal platform for promoting the social sensing paradigm. In addition, Twitter provides an open, well specified API with almost unrestricted access to the publicly available, user-provided content and profile information. Data collection for the execution of this experiment is based on criteria expressed as hashtags and keywords associated with wildfires, combined with tags denoting geo-reference. Twitter also offers the option to geo-tag the provided Tweets, but this feature is not frequently used as it is a common practice for Twitter users to introduce their own tags in order to express the connection of their posts with a specific area. To this end, text analytics approaches have been employed as more appropriate. Again, an IoT2Edge-Agent has been deployed for retrieving and translating relevant with the experiment Tweets.
5.3. The Decision-Making Process and the Knowledge Extraction Services
Beyond the representational aspects, semantic computing supports reasoning on raw sensor data enabling the derivation of higher level abstractions; such abstractions form the basis of domain- and cross-domain knowledge which is the main enabler for decision making. To this end, and as it is described in
Section 4.1, the IoT2Edge-Agent “Inference Service” provides the necessary mechanisms in order to assign the appropriate domain concepts to the respective data collections. These rules are expected to be tailored to the needs of each application domain and the specifics of the collected data. For the needs of the wild-fire detection use case and for the realisation of the respective experiment, the following inference processes have been identified for each information platform:
MESP data classification: Sensors measurements and images collected by the MESP are combined in order to deduce higher level alert information for the potential identification of a fire ignition in the area. A custom image classification service is deployed as a native standalone application on MESP-R, in order to classify images as “fire-containing” or not. This service, which was initially developed for the purposes of the work presented in [
69], leverages Google’s Tensorflow deep learning framework. The “Inception v3” Deep Neural Network was retrained with a specific dataset [
70] containing either pictures of forest wildfires or plain forests. Images captured through the MESP-R camera are assigned a score ranging from [0,1], where 1 corresponds to the absolute confidence about the presence of a fire in the image. The numerical image classification scores are combined with the measurements derived from the infrared, temperature, gas monitoring and humidity sensors through data inference algorithms such as those presented in [
45,
47]. The aggregated outcome is normalised within the [0,1] range of values.
Table 3 contains the measurements range for each MESP sensor along with indicative thresholds denoting the presence of a fire.
The outcome of the inference process is modeled based on the 5 FIWARE Alert information classes (Informational, Low, Medium, High, Critical) [
63]. An example of the Alert JSON object produced by the MESP data classification engine is presented hereafter:
{
"id": "Alert:1",
"type": "Alert",
"category": "security",
"subCategory": "forestFire",
"severity": "high",
"location": {
"type": "Point",
"coordinates": [23.7319, 37.9877]
},
"dateIssued": "2018-07-23T09:25:55.00Z",
"description": "Potential forest fire identified by MESP",
}
Tweets Classification: The developed inference engine processes the messages contained in collection of Tweets aiming to identify time based changes on the utilisation of specific terms. In our case, the Twitter API is utilised in order to retrieve Tweets containing tags and keywords referring to the targeted locations along with fire-related keywords potentially denoting the ignition of a forest fire incident. Collected tweets are periodically processed at regular time intervals and an Alert class is assigned to them, based on textual content analysis. In a nutshell, the rationale of the algorithm that extracts the probability of a fire incident and the respective fire alert level based on information originating from Tweeter, is as follows: initially, the population
of the retrieved tweets posted in the latest window frame (
) that use keywords linked to the area of interest (this is expressed by the keyword set
L that results from a reverse geocoding process for the provided GPS coordinates, which uses external APIs such as the Google Maps API), is measured. For example, for the studied fire incident, the respective keyword set was as follows:
L = {
rafina,
mati,
penteli,
dionisos} Subsequently, the population
of the subset of
is measured. This counts the tweets posted in the latest window frame (
) that use both keywords linked to the area of interest, as well as keywords related to fire emergencies (this is expressed by the keyword set
F that is language dependent, aggregating fire emergency keywords expressed in the official language(s) of the country where the area of interest is located at). For example, for the studied fire incident, the respective keyword set was as follows:
F = {
pyrkagia,
fotia,
emprismos,
kapnos,
kaigomaste}. Finally, the ratio of the two aforementioned metrics is calculated:
, the value of which lies in the range
. The respective translation of
to a Twitter-based fire alert level
is executed as follows:
More details for the respective mechanism are presented in [
71]. An alert level based on the FIWARE Alert information model [
63] is associated with a location and a time interval. An example of the Alert JSON object produced by the Twitter data classification engine is presented hereafter:
{
"id": "Alert:2",
"type": "Alert",
"category": "security",
"subCategory": "forestFire",
"severity": "high",
"location": {
"type": " Polygon",
"coordinates": [23.7319, 37.9877]
},
"dateIssued": "2018-12-12T09:25:55.00Z",
"description": "Potential forest fire identified by Tweet analysis",
}
Weather Classification: Weather related measurements collected from data sources deployed in the area are processed and the respective fire weather index (FWI) is extracted, based on the Canadian forest fire danger rating system [
72], which is one of the most popular models used to link weather conditions with the probability and risk of wildfire incidents. The estimated FWI is normalised according to the FIWARE Alert information model severity classes [
63]. An example follows:
{
"id": "Alert:3",
"type": "Alert",
"category": "weather",
"subCategory": "fireRisk",
"severity": "high",
"location": {
"type": " Polygon",
"coordinates": [23.7319, 37.9877]
},
"dateIssued": "2018-12-12T09:25:55.00Z",
"description": "Increased fire risk indicated by weather data analysis",
}
As illustrated in
Figure 13, the Decision-Making Service (DMS) is deployed to the cloud as an IoT2Edge-Server component and is triggered by the MESP system when the fire-related average classification score of the images and the environmental sensors is above a predefined threshold, e.g., “Medium”, thus indicating a considerable probability of fire occurrence. The overall operation can be triggered periodically and runs continuously, meaning that the DMS may receive multiple fire alerts from various locations, where a decision should be extracted for each case. Once the DMS is triggered, it aggregates the various aforementioned data, in order to further evaluate the probability of a fire incident occurrence, the severity of the potential fire incident (e.g., in terms of human presence in the area, potential fire spreading, etc.). As it is already described, the alert levels derived from the diverse environmental monitoring and social media management platforms are modelled in uniform alert classes and are propagated to the cloud-based decision making service that is responsible for processing the respective values and generating an overall indication of the fire risk level.
Among the benefits of this approach is that no sensitive information (e.g., images) is disseminated from the respective distinct cyber-premises and only the alert levels are provided to the DMS, realising a privacy by design approach. In addition, the utilisation of common semantics through the enforcement of the data interoperability mechanisms facilitates the rapid cross-domain decision making.
In a nutshell, the DMS delivers the respective outputs based on the following steps that take place:
Step 1: The MESP system sends a fire alert notification (if the respective level is “MEDIUM” or higher), along with the associated location expressed in GPS coordinates to the DMS.
Step 2: The DMS executes a reverse geocoding process for the provided GPS coordinates, which uses external APIs (e.g., Google Maps API) in order to obtain a set of area names and location identifying keywords.
Step 3: The DMS triggers the Twitter-based Fire Detection Service with these keywords that collects and analyses the Twitter posts that are related to the indicated area, then identifies the ones that are potentially associated with an emerging wildfire incident and finally extracts the probability of a fire incident occurrence that is eventually returned to the DMS.
Step 4: The DMS then triggers the EFFIS-based Classification Service with the specific GPS coordinates, which retrieves GeoTIFF image provided by the EFFIS and extracts the Fire Danger Class for the provided coordinates that is then returned to the DMS.
Step 5: The DMS uses the Weather-based Fire Danger Classification Service that collects and analyses current and forecasted weather data that are linked with the indicated area (via the OpenWeather Map API) and uses them to extract the fire risk scores that are then returned to the DMS.
Step 6: The DMS processes and fuses the outputs of the four classifier services above in order to extract estimations for two fire-related metrics for the specified GPS coordinates: (i) the overall fire danger level (indicating the risk of fire ignition) and (ii) the probability of fire occurrence. More details on the internals of this step are provided in [
73].