Beyond Stochastic Gradient Descent for Matrix Completion Based Indoor Localization
Abstract
:1. Introduction
1.1. Related Works
1.2. Contribution
- Improving the localization accuracy of trilateration technique: We develop high accuracy fingerprint based indoor localization scheme which is based on sparse representation and matrix completion theories. As said before, trilateration technique is based on pairwise distances between the target node and the anchor nodes. Due to the fact that propagation conditions are not optimal in an indoor environment, several pairwise distances cannot be measured. So, our main contribution is to enhance the localization’s accuracy of trilateration by estimating all pairwise distances.
- Matrix completion resolution: We formulate the indoor localization scheme as a simple optimization problem which enables efficient and reliable algorithm implementation.
- Solving the proposed optimization problem: We develop closed-form algorithms, which can be reproduced by simple implementation, to solve the indoor localization problem. Specifically, we adopt recent methods like Nesterov accelerated gradient (Nesterov), adaptative Moment Estimation (Adam), Adadelta, Root Mean Square Propagation (RMSProp) and Adaptative gradient (Adagrad).
2. System Model
- Step 1: Refine and complete the squared distance matrix.
- Step 2: Once the matrix is completed, the coordinates of the node can be retrieved using the classic trilateration process.
3. Proposed Matrix Completion Based Localization
3.1. Problem Formulation
3.2. Matrix Completion: Optimization over GD and Its Variants
3.2.1. Gradient Descent (GD)
3.2.2. Nesterov Accelerated Gradient (NAG)
3.2.3. Adaptive Gradient (Adagrad)
- We set to be the gradient of the objective function with respect to the parameter
- we compute the memory of squared gradients over time as
- we modify the general learning rate at each time step t for every parameter based on the sum of the squares of the gradients that have been computed for up to time step t.
3.2.4. Root Mean Square Propagation (RMSProp)
- We compute the local average of previous as
- Then, we apply the update
3.2.5. Adadelta
- we compute gradient as in Equation (23).
- we compute the local average of previous
- we compute new term accumulating prior updates (Momentum: acceleration term)
- Then, we apply the update
3.2.6. Adaptive Moment Estimation (Adam)
- Compute second gradient moment with local accumulation (Adadelta/RMSProp)
- Compute the first gradient moment
- Compute bias-corrected first moment and second moment estimate
- Update parameters
4. Simulation Results and Discussion
4.1. Determining the Best Ratio between the Number of Unknown Nodes and the Number of Anchors
4.2. Simulation Setup
4.3. Verification of the Low Rank Property
4.4. Recovery Performance and Localization Accuracy
- EDM reconstruction error using the mean square error
- Localization error
4.5. Complexity
4.5.1. Analytical Expressions
4.5.2. Analysis
5. Method Verification Using Real Measurements
6. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Atzori, L.; Iera, A.; Morabito, G. The internet of things: A survey. Comput. Netw. 2010, 54, 2787–2805. [Google Scholar] [CrossRef]
- Hennebert, C.; Denis, B.; Le Gall, F.; Copigneaux, B.; Clari, F.; Sottile, F.; Mauro, F.; Smadja, P.; Pascali, S.; Preuveneers, D.; et al. uBiquitous, secUre inTernet-of-things with Location and contEx-awaReness. BUTLER project D 2, 1–134. Available online: www.iot-butler.eu (accessed on 5 July 2018).
- Dil, B.; Dulman, S.; Havinga, P. Range-based localization in mobile sensor networks. In Proceedings of the European Workshop on Wireless Sensor Networks, Zurich, Switzerland, 13–15 February 2006; pp. 164–179. [Google Scholar]
- Singh, S.P.; Sharma, S.C. Range free localization techniques in wireless sensor networks: A review. Procedia Comput. Sci. 2015, 57, 7–16. [Google Scholar] [CrossRef]
- Awad, F.; Al-Sadi, A.; Alsmady, A. Distributed and adaptive location identification system for mobile devices. EURASIP J. Adv. Signal Process. 2018, 2018, 61. [Google Scholar] [CrossRef]
- Kaplan, E.; Hegarty, C. Understanding GPS: Principles and Applications; Artech House: Norwood, MA, USA, 2005. [Google Scholar]
- Gu, Y.; Chen, M.; Ren, F.; Li, J. HED: Handling environmental dynamics in indoor WiFi fingerprint localization. In Proceedings of the IEEE Wireless Communications and Networking Conference (WCNC), Doha, Qatar, 3–6 April 2016; pp. 1–6. [Google Scholar]
- Sun, L.; Zheng, Z.W.; He, T.; Li, F. Multifloor Wi-Fi localization system with floor identification. Int. J. Distrib. Sens. Netw. 2015, 11, 131523. [Google Scholar] [CrossRef]
- Cheng, J.; Song, Z.; Ye, Q.; Du, H. MIL: A mobile indoor localization scheme based on matrix completion. In Proceedings of the IEEE International Conference on Communications (ICC), Kuala Lumpur, Malaysia, 23–27 May 2016; pp. 1–55. [Google Scholar]
- Yang, Z.; Liu, Y. Quality of trilateration: Confidence-based iterative localization. IEEE Trans. Parallel Distrib. Syst. 2010, 21, 631–640. [Google Scholar] [CrossRef]
- Norrdine, A. An algebraic solution to the multilateration problem. In Proceedings of the 15th International Conference on Indoor Positioning and Indoor Navigation, Sydney, Australia, 13–15 November 2012; p. 2012. [Google Scholar]
- Najib Yasmeen, N.A.; Daud, N.; Abd Aziz, A.; Razali, R. 3-Dimensional wireless sensor network localization: A review. AIP Conf. Proc. 2016, 080029. [Google Scholar] [CrossRef]
- Kumar, P.; Reddy, L.; Varma, S. Distance measurement and error estimation scheme for RSSI based localization in Wireless Sensor Networks. In Proceedings of the 5th IEEE Conference on Wireless Communication and Sensor Networks (WCSN), Allahabad, India, 15–19 December 2009; pp. 1–4. [Google Scholar]
- Nguyen, T.L.; Shin, Y. Matrix completion optimization for localization in wireless sensor networks for intelligent IoT. Sensors 2016, 16, 722. [Google Scholar] [CrossRef] [PubMed]
- Nguyen, L.; Kim, S.; Shim, B. Localization in Internet of Things network: Matrix completion approach. In Proceedings of the Information Theory and Applications Workshop (ITA), San Diego, CA, USA, 31 January–2 February 2016; pp. 1–4. [Google Scholar]
- Nguyen, L.; Shim, B. Localization of Internet of Things network via Euclidean distance matrix completion. In Proceedings of the IEEE/CIC International Conference on Communications in China (ICCC), Chengdu, China, 27–29 July 2016; pp. 1–4. [Google Scholar]
- Zhu, Y.; Jiang, A.; Kwan, H.K. ADMM-Based Sensor Network Localization Using Low-Rank Approximation. IEEE Sens. J. 2018, 18, 8463–8471. [Google Scholar] [CrossRef]
- Alfakih, A.Y.; Khandani, A.; Wolkowicz, H. Solving Euclidean distance matrix completion problems via semidefinite programming. Comput. Optim. Appl. 1991, 12, 13–30. [Google Scholar] [CrossRef]
- Ferreira, R.; Xavier, J.; Costeira, J.P.; Barroso, V. Newton algorithms for Riemannian distance related problems on connected locally symmetric manifolds. IEEE J. Sel. Top. Signal Process. 2013, 7, 634–645. [Google Scholar] [CrossRef]
- Cai, J.-F.; Candès, E.J.; Shen, Z. A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 2010, 20, 1956–1982. [Google Scholar] [CrossRef]
- Chen, M.; Ganesh, A.; Lin, Z.; Ma, Y.; Wright, J.; Wu, L. Fast convex optimization algorithms for exact recovery of a corrupted low-rank matrix. In Coordinated Science Laboratory Report no. UILU-ENG-09-2214; Coordinated Science Laboratory, University of Illinois at Urbana-Champaign: Champaign, IL, USA, 2009. [Google Scholar]
- Lin, Z.; Chen, M.; Ma, Y. The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv 2010, arXiv:1009.5055. [Google Scholar]
- Liu, C.; Shan, H.; Wang, B. Wireless Sensor Network Localization via Matrix Completion Based on Bregman Divergence. Sensors 2018, 18, 2974. [Google Scholar] [CrossRef] [PubMed]
- Xiao, F.; Liu, W.; Li, Z.; Chen, L.; Wang, R. Noise-tolerant wireless sensor networks localization via multinorms regularized matrix completion. IEEE Trans. Veh. Technol. 2018, 67, 2409–2419. [Google Scholar] [CrossRef]
- Grant, M.; Boyd, S.; Ye, Y. CVX: Matlab Software for Disciplined Convex Programming; CVX Research, Inc.: Austin, TX, USA, 2008. [Google Scholar]
- Guo, X.; Chu, L.; Sun, X. Accurate localization of multiple sources using semidefinite programming based on incomplete range matrix. IEEE Sens. J. 2016, 16, 5319–5324. [Google Scholar] [CrossRef]
- Guo, X.; Chu, L.; Ansari, N. Joint localization of multiple sources from incomplete noisy Euclidean distance matrix in wireless networks. Comput. Commun. 2018, 122, 20–29. [Google Scholar] [CrossRef]
- Chen, J.; Mitra, U. Unimodality-Constrained Matrix Factorization for Non-Parametric Source Localization. IEEE Trans. Signal Process. 2019, 67. [Google Scholar] [CrossRef]
- Nguyen, L.T.; Kim, J.; Kim, S.; Shim, B. Localization of IoT Networks Via Low-Rank Matrix Completion. IEEE Trans. Commun. 2019. [Google Scholar] [CrossRef]
- Zhang, H.; Liu, Y.; Lei, H. Localization from Incomplete Euclidean Distance Matrix: Performance Analysis for the SVD-MDS Approach. IEEE Trans. Signal Proces. 2019, 67. [Google Scholar] [CrossRef]
- Torgerson, W.S. Multidimensional scaling of similarity. Psychometrika 1965, 30, 379–393. [Google Scholar] [CrossRef]
- Tsai, M. Path-loss and shadowing (large-scale fading). Nat. Taiwan Univ. Oct. 2011, 20, 2011. [Google Scholar]
- Recht, B.; Fazel, M.; Parrilo, P.A. Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev. 2010, 52, 471–501. [Google Scholar] [CrossRef]
- Candès, E.J.; Recht, B. Exact matrix completion via convex optimization. Found. Comput. Math. 2009, 9, 717. [Google Scholar] [CrossRef]
- Duchi, J.; Hazan, E.; Singer, Y. Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 2011, 12, 2121–2159. [Google Scholar]
- Sebastian, R. An overview of gradient descent optimization algorithms. arXiv 2016, arXiv:1609.04747. [Google Scholar]
- Sutskever, I.; Martens, J.; Dahl, G.; Hinton, G. On the importance of initialization and momentum in deep learning. In Proceedings of the International Conference on Machine Learning, Atlanta, GA, USA, 16–21 June 2013; pp. 1139–1147. [Google Scholar]
- Wang, Y.; Kim, S.-P.; Principe, J.C. Comparison of TDNN training algorithms in brain machine interfaces. In Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN’05), Montreal, QC, Canada, 31 July–4 August 2005; pp. 2459–2462. [Google Scholar]
- Hinton, G.; Srivastava, N.; Swersky, K. Neural networks for machine learning lecture 6a overview of mini-batch gradient descent. Cited On 2012, 14, 31. [Google Scholar]
- Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
- Cheng, J.; Ye, Q.; Jiang, H.; Wang, D.; Wang, C. STCDG: An efficient data gathering algorithm based on matrix completion for wireless sensor networks. IEEE Trans. Wirel. Commun. 2013, 12, 850–861. [Google Scholar] [CrossRef]
- Xie, K.; Ning, X.; Wang, X.; Xie, D.; Cao, J.; Xie, G.; Wen, J. Recover corrupted data in sensor networks: A matrix completion solution. IEEE Trans. Mob. Comput. 2017, 16, 1434–1448. [Google Scholar] [CrossRef]
- Strassen, V. Gaussian elimination is not optimal. Numer. Math. 1969, 13, 354–356. [Google Scholar] [CrossRef]
- Benoit, A. Algorithmique I-Cours et Travaux Dirigés L3; Ecole Normale Supérieure de Lyon: Lyon, France, 2009. [Google Scholar]
- Iannazzo, B. On the Newton method for the matrix p th root. SIAM J. Matrix Anal. Appl. 2006, 28, 503–523. [Google Scholar] [CrossRef]
|
Adam | The Rest of Algorithms | ||
---|---|---|---|
Sigma shadowing = 0 | 0.5 | 0.5 | |
0.9 | 0.9 | ||
− | |||
− | |||
− | |||
− | |||
Sigma shadowing = 2 | 0.5 | 0.5 | |
0.9 | 0.9 | ||
− | |||
− | |||
− | |||
− | |||
Sigma shadowing = 5 | 0.5 | 0.5 | |
0.9 | 0.9 | ||
− | |||
− | |||
− | |||
− |
Algorithm | Localization Mean Error (m) | Number of Operations per Iteration Normalized by the Number of Operations per Iteration for GD | Number of Iterations to Reach the Convergence | Complexity of the Algorithm to Reach the Convergence Respect to the Complexity of GD | |
---|---|---|---|---|---|
Sigma shadowing = 0 | Without matrix completion | 4.1 | — | — | — |
GD | 0.56 | 1 | 5910 | 1 | |
NAG | 0.56 | 1.9839 | 3100 | 1.0406 | |
Adagrad | 1.8 | 1.2473 | 120,000 | 25.3261 | |
RMSProp | 1.8 | 1.5 | 43,750 | 11.1041 | |
Adadelta | 0.47 | 1.5054 | 14,150 | 3.6042 | |
Adam | 0.55 | 1.5 | 790 | 0.2005 | |
Sigma shadowing = 2 | Without matrix completion | 4.7 | — | — | — |
GD | 1.2 | 1 | 5535 | 1 | |
NAG | 1.2 | 1.9839 | 3942 | 1.4129 | |
Adagrad | 2.2 | 1.2473 | 125,900 | 28.3716 | |
RMSProp | 2.1 | 1.5 | 45,640 | 12.3686 | |
Adadelta | 1.2 | 1.5054 | 10,530 | 2.8639 | |
Adam | 1.2 | 1.5 | 775 | 0.21 | |
Sigma shadowing = 5 | Without matrix completion | 4.9 | — | — | — |
GD | 2.6 | 1 | 4435 | 1 | |
NAG | 2.6 | 1.9839 | 3281 | 1.4677 | |
Adagrad | 3.4 | 1.2473 | 121,900 | 34.2835 | |
RMSProp | 3.4 | 1.5 | 43,920 | 14.8546 | |
Adadelta | 2.7 | 1.5054 | 9742 | 3.3067 | |
Adam | 2.6 | 1.5 | 751 | 0.2540 |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Njima, W.; Zayani, R.; Ahriz, I.; Terre, M.; Bouallegue, R. Beyond Stochastic Gradient Descent for Matrix Completion Based Indoor Localization. Appl. Sci. 2019, 9, 2414. https://doi.org/10.3390/app9122414
Njima W, Zayani R, Ahriz I, Terre M, Bouallegue R. Beyond Stochastic Gradient Descent for Matrix Completion Based Indoor Localization. Applied Sciences. 2019; 9(12):2414. https://doi.org/10.3390/app9122414
Chicago/Turabian StyleNjima, Wafa, Rafik Zayani, Iness Ahriz, Michel Terre, and Ridha Bouallegue. 2019. "Beyond Stochastic Gradient Descent for Matrix Completion Based Indoor Localization" Applied Sciences 9, no. 12: 2414. https://doi.org/10.3390/app9122414
APA StyleNjima, W., Zayani, R., Ahriz, I., Terre, M., & Bouallegue, R. (2019). Beyond Stochastic Gradient Descent for Matrix Completion Based Indoor Localization. Applied Sciences, 9(12), 2414. https://doi.org/10.3390/app9122414