Next Article in Journal
Structural Changes in Boreal Forests Can Be Quantified Using Terrestrial Laser Scanning
Next Article in Special Issue
Bathymetric Inversion and Uncertainty Estimation from Synthetic Surf-Zone Imagery with Machine Learning
Previous Article in Journal
TCDNet: Trilateral Change Detection Network for Google Earth Image
Previous Article in Special Issue
Effective Training of Deep Convolutional Neural Networks for Hyperspectral Image Classification through Artificial Labeling
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Distributed Training and Inference of Deep Learning Models for Multi-Modal Land Cover Classification

by
Maria Aspri
1,2,
Grigorios Tsagkatakis
2,* and
Panagiotis Tsakalides
1,2
1
Institute of Computer Science, Foundation for Research and Technology-Hellas (FORTH), GR70013 Heraklion, Greece
2
Computer Science Department, University of Crete, GR70013 Heraklion, Greece
*
Author to whom correspondence should be addressed.
Remote Sens. 2020, 12(17), 2670; https://doi.org/10.3390/rs12172670
Submission received: 8 July 2020 / Revised: 14 August 2020 / Accepted: 17 August 2020 / Published: 19 August 2020
(This article belongs to the Special Issue Computer Vision and Deep Learning for Remote Sensing Applications)

Abstract

Deep Neural Networks (DNNs) have established themselves as a fundamental tool in numerous computational modeling applications, overcoming the challenge of defining use-case-specific feature extraction processing by incorporating this stage into unified end-to-end trainable models. Despite their capabilities in modeling, training large-scale DNN models is a very computation-intensive task that most single machines are often incapable of accomplishing. To address this issue, different parallelization schemes were proposed. Nevertheless, network overheads as well as optimal resource allocation pose as major challenges, since network communication is generally slower than intra-machine communication while some layers are more computationally expensive than others. In this work, we consider a novel multimodal DNN based on the Convolutional Neural Network architecture and explore several different ways to optimize its performance when training is executed on an Apache Spark Cluster. We evaluate the performance of different architectures via the metrics of network traffic and processing power, considering the case of land cover classification from remote sensing observations. Furthermore, we compare our architectures with an identical DNN architecture modeled after a data parallelization approach by using the metrics of classification accuracy and inference execution time. The experiments show that the way a model is parallelized has tremendous effect on resource allocation and hyperparameter tuning can reduce network overheads. Experimental results also demonstrate that proposed model parallelization schemes achieve more efficient resource use and more accurate predictions compared to data parallelization approaches.
Keywords: distributed deep learning; model parallelization; convolutional neural networks; multi-modal observation classification; land cover classification distributed deep learning; model parallelization; convolutional neural networks; multi-modal observation classification; land cover classification
Graphical Abstract

Share and Cite

MDPI and ACS Style

Aspri, M.; Tsagkatakis, G.; Tsakalides, P. Distributed Training and Inference of Deep Learning Models for Multi-Modal Land Cover Classification. Remote Sens. 2020, 12, 2670. https://doi.org/10.3390/rs12172670

AMA Style

Aspri M, Tsagkatakis G, Tsakalides P. Distributed Training and Inference of Deep Learning Models for Multi-Modal Land Cover Classification. Remote Sensing. 2020; 12(17):2670. https://doi.org/10.3390/rs12172670

Chicago/Turabian Style

Aspri, Maria, Grigorios Tsagkatakis, and Panagiotis Tsakalides. 2020. "Distributed Training and Inference of Deep Learning Models for Multi-Modal Land Cover Classification" Remote Sensing 12, no. 17: 2670. https://doi.org/10.3390/rs12172670

APA Style

Aspri, M., Tsagkatakis, G., & Tsakalides, P. (2020). Distributed Training and Inference of Deep Learning Models for Multi-Modal Land Cover Classification. Remote Sensing, 12(17), 2670. https://doi.org/10.3390/rs12172670

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop