*2.3. Interpretation Techniques*

In the context of image classification, interpretation techniques are intended to explain the predictions of trained models by visualizing the regions of the inputs that contributed to the final classification result. Thus, they can provide a coarse localization of target objects using image-level annotations.

Hereafter, a simplified explanation of the intuition behind the two gradient-based back-propagation techniques used in this paper (i.e., Grad-CAM and Grad-CAM++) is presented.

### 2.3.1. Gradient-Weighted Class Activation Mapping (Grad-CAM)

The Grad-CAM approach is based on the gradient information for the last convolutional layer of a trained network [50].

The gradients of the score for class *c* (*y<sup>c</sup>*) with respect to the feature maps *A<sup>k</sup>* of the convolutional layer are computed via back-propagation and then global-average pooled to obtain the weights *w<sup>k</sup> c* [50]:

$$w\_c^k = \frac{1}{Z} \sum\_i \sum\_j \frac{\partial y^c}{\partial A\_{ij}^k} \tag{1}$$

where, *Z* is the number of pixels in the activation map.

The weight w<sup>k</sup> c expresses the importance of feature map *k* for the class *c*. The class discriminative localization map Grad-CAM *LcGrad*-*CAM* is obtained by computing a weighted sum of the forward feature maps *A<sup>k</sup>* of the last convolutional layer [50]:

$$L\_{\text{Grad-CAM}}^c = ReLU\left(\sum\_k w\_k^c A^k\right) \tag{2}$$

where *ReLU* is the Rectified Linear Unit activation function. It is used to focus only on the features that have a positive influence on the target class [50].
