Outdoor Node Localization Using Random Neural Networks for Large-Scale Urban IoT LoRa Networks^†

Comments

Responses

1.  The first paragraph in Intro is very long. It is better to summarize the state of arts localization approach for IoT, the limitations, and the key idea of the proposed approach, i.e., why the proposed approach achieve a better performance.
The current paragraph contains many low-level details that should be included in the related work.

- All the information given in the introduction seems relevant as it summaries the state of art of localization technologies & their limitations, localization approaches & their limitations.

-The proposed approach performs better with higher accuracy i.e., minimum mean localization error in meters (added in the introduction).

2. 4.1 dataset. More details should be introduced. From the results, a localization of 0.29m seems a relatively how accuracy for LoRa. So the deployment should be very dense. Details such as distance between nodes, received signal strength in the dataset should be introduced.

As seen in lines 166-175 on page 4, the used data was collected using 20 cars of Belgian postal services moving around in the dense Antwerp city provided with air quality sensors & IM880B-L RF LoRa modules and sending a message every minute whereby 130,343 messages were recorded [41].  Only RSSI values and GPS coordinates were recorded and reported by the researchers and since a large data was collected using moving vehicles not fixed nodes, the distance between nodes varies. Furthermore, RSSI based localization in our work considers RSSI values and position coordinates. Finally, figure 3 is added to show the distribution of RSSI values which varies between -122 dBm to -78 dBm.

3. 5.1 comparative performance analysis.
Authors show better performance of the proposed approch, but the key insight (why this approach is better) is still lost. Does this approach only works for dense IoT network? or other approaches are suitable for sparse networks. This should be more clearly discussed.

-Table 3 shows that proposed RNN based model outperforms models in other works with the minimum mean localization error in meters of 0.29 m which is the smallest of all!

- A paragraph is added in line 66-71 on page 2 explaining why RNN performance is better.

4. Since this paper is submitted to the algorithm journal. The concrete algorithm in psedocode  should be include in this paper.

RNN based localization algorithm added in line 230 to 242 on page 8

5. Recent LoRa localication papers in well-known conferences should be added as references. Also comparion to these works should be discussed.

e.g.,
-DeepLoRa: Learning Accurate Path Loss Model for Long Distance Links in LPWAN. INFOCOM 2021.
-"SateLoc: a Virtual Fingerprinting Approach to Outdoor LoRa Localization using Satelite Images", ACM/IEEE IPSN 2020.

-SateLoc paper is added to our related work. Our study considered LoRa localization using RSSI fingerprinting and compared our model with other RSSI based models that used the same Antwerp LoRa dataset.

-The scope of the DeepLoRa paper is estimating Path Loss Models.

Comments

Responses

1.  Please change reference 1 to the following references. Reference 1 is a domestic paper, and it is better to use the journal papers for better understanding.

· A sensor fusion framework for indoor localization using smartphone sensors and Wi-Fi RSSI measurements

 · Hybrid deep learning model based indoor positioning using Wi-Fi RSSI heat maps for autonomous applications.

-Recommended paper:’ A sensor fusion framework for indoor localization using smartphone sensors and Wi-Fi RSSI measurements’ is considered as reference 1.  

2. Please define LoRa in the introduction section.

LoRa defined in line 45 on page 2.

3. Please add a full stop at the end of the figure captions.

Full stops added at end of figure captions and tables.

4. In the related work section, Wi-Fi fingerprint algorithm: Performance analysis of fingerprint matching algorithms for indoor localization

Don’t understand this comment and what to change; however, the scope of our work is outdoor localization.

5. Please add a small paragraph at the end of the related work section and discuss how this paper is different from existing works. Please add some details like the proposed approach in the paper addresses some of the existing localization challenges.

- A paragraph is added in line 66-71 on page 2 explaining why RNN performance is better.

6. Please check the bracket alignment in line 168 of page 5.

Bracket well aligned.

7. Please discuss RNN architecture details in subsection 4.3. The structure of RNN and the hyperparameter details like the number of neurons, batch size, hidden layer, loss function, optimizer, etc

As seen in lines 166-175 on page 4, the used data was collected using 20 cars of Belgian postal services moving around in the dense Antwerp city provided with air quality sensors & IM880B-L RF LoRa modules and sending a message every minute whereby 130,343 messages were recorded [41].  Only RSSI values and GPS coordinates were recorded and reported by the researchers and since a large data was collected using moving vehicles not fixed nodes, the distance between nodes varies. Furthermore, RSSI based localization in our work considers RSSI values and position coordinates. Finally, figure 3 is added to show the distribution of RSSI values which varies between -122 dBm to -79 dBm.

8. Table 1 and Figure 5 indicate the same meaning, better to use single results.

Figure used for better comparison purposes and Table used to show exact values

9. Same as the previous comment, Table 2 and Figure 7 are the same.

Figure used for better comparison purposes and Table used to show exact values

10. Please check Table 3 comparison results. Some results show 500m, 358 m localization errors. However, these results are not acceptable for comparison. I think the unit is cm or something. Please check this point

Outdoor localization error reported in meters the SI unit for distance

11. If it is possible, please add some model comparison results with your approach. It is not clear why the RNN model gives better results than other models. Please justify your answers and show some validation results with other deep learning models.

-Table 3 shows that proposed RNN based model outperforms models in other works with the minimum mean localization error in meters of 0.29 m which is the smallest of all!

- A paragraph is added in line 64-81 on page 2 explaining why RNN performance is better.

-The scope of this work is on machine learning models

Comments

Responses

So, there are a few things that are not clear or missing. Most important is that the paper should provide an explanation about the way you measure the localization error regarding the coordinates (the dataset has geodetic coordinates). In equation (12) the formula calculating the Euclidean distance uses cartesian coordinates. You don’t explain at all if the geodetic coordinates of the dataset are converted to cartesian and how, and you do not provide the measurement units used. I suspect that there is a mismatch of units in the formula that leads to an extremely low error. So please provide details on how you convert from the GPS geodesic coordinates to the cartesian coordinates, used to calculate the Euclidean distance, including a few examples

This work considers deg2utm stand-alone function application to convert GPS coordinates to X, Y vector coordinates using MATLAB R2020b [53]; added in line 215 to 216.

% Example 1:
% Lat=[40.3154333; 46.283900; 37.577833; 28.645650; 38.855550; 25.061783];
% Lon=[-3.4857166; 7.8012333; -119.95525; -17.759533; -94.7990166; 121.640266];
% [x,y,utmzone] = deg2utm(Lat,Lon);
% fprintf('%7.0f ',x)
% 458731 407653 239027 230253 343898 362850
% fprintf('%7.0f ',y)
% 4462881 5126290 4163083 3171843 4302285 2772478

The authors should explain Figure 4. The paper does not explain at all, how each RSSI is matched with a specific lat, and long pair. As there are 72 RSSI values of the gateways, how does the matching algorithm work, and how it is related to the input you give to the network, given that the network has an input of 72 RSSI values and produces as output the position of the end node / target device? Some understanding may be inferred by fig. 4, but it is better if you explicitly describe the whole process.

The proposed RNN model uses seventy-two input layer neurons (72 receptions at 72 gateways receiving RSSI values from 130,343 positions), seventy-two hidden layer neurons and two output layer neurons (X, Y position coordinates). Please check in line 228 to 230. Please check RNN neural network working process in line 188 to 211 and more details in [36]

In line 232 what are the values RSSIxt, yt, M?

RSSIxt, yt, M means a RSSI value at position x,y collected  at a certain t in time slot M. Data was collected at different times in a period of 3 months.

In Line 266, I understand that you trained the same architecture for 1000, 3000, 5000, 15000, 134000 and the MAE wasn’t affected by increasing the training set (I suppose the test set remained the same 20% of the original data). Is my understanding correct? If yes, please provide a discussion of this finding and how you explain it. Personally, I would expect that accuracy would improve with the size of the training sets.

Using the same RNN architecture this is how different samples were used: For example, while analysing a sample of 1000 samples; 80% of 1000 was used for training and 20% of 1000 was used testing. The proposed RNN model runs k-folds; now added in line 228 to 230.

Comments

Responses

Comment 4 was the suggested reference paper that discusses different fingerprint approaches for localization. Even for outdoor localization, the concept of the fingerprint matching algorithm is the same. (This is not a major concern to the readers).   

The recommended paper only discusses for indoor localization this work is outdoor localization.

Comment 5 is not correctly addressed. The reviewer asked to discuss the details in the related work section, and the authors updated it in the introduction section

Updated in the related work section in line 151 to 157

I think the authors didn’t understand comment 7. The reviewer asked about the deep learning structure (RNN) and its parameter setting for training and testing. However, the author’s response is different for comment 7.

The proposed RNN model runs k-folds; now added in line 228 to 230.

Comment 10 discusses the localization error in the meter. If we consider a real system with 500 m or 358m localization error, the system is not acceptable. If the system has a 500 m localization error, it is not fair for comparison. Table 3 is not a proper comparison for a technical paper. Please imagine your system gives a 0.29 m localization error, and the techniques used in the Table 3 have 500 m or 398.4 m localization error. This comparison is not reasonable to the readers. Please find other conventional approaches which have lower localization error and compare it with your systems.  

Other conventional approaches which have lower localization error are added and compared with our system in Table 3.  

The authors didn’t understand comment 11. The reviewer asked about the comparison of different models for their proposed system. Please consider other models for your proposed system and compare the results with RNN results.

Different models of our proposed system were previously discussed in our published paper in [52]; a comment on that is also added in line 157 to 159 and in line 294 to 295. Also, Table 3 compares our model with other different models existing in literature. Furthermore, Fig 7 and fig 8 result compare results of our proposed model with different learning rates and samples.