Automated Identification of Landfast Sea Ice in the Laptev Sea from the True-Color MODIS Images Using the Method of Deep Learning

Abstract

Landfast sea ice (LFSI) refers to sea ice attached to the shoreline with little or no horizonal motion in contrast to drifting sea ice. The LFSI plays an important role in the Arctic marine environmental and biological systems. Therefore, it is crucial to accurately monitor the spatiotemporal changes in the LFSI distribution. Here we present an automatic LFSI retrieval method for the Laptev Sea, eastern Arctic Ocean, based on a conditional generative adversarial network Pix2Pix using the true-color images of Moderate Resolution Imaging Spectroradiometer (MODIS). The spatial resolution of the derived product is 1.25 km, with a temporal interval of 7 days. Compared to the manually identified data from the true-color images of MODIS, the average precision of the LFSI area derived from LFSI mapping model reaches 91.4%, with the recall reaching 98.7% and F1-score reaching 94.5%. The LFSI coverage is consistent with the traditional large-scale LFSI products, but provides more details. Intraseasonal and interannual variations in LFSI area of the Laptev Sea in spring (March–May) during the period of 2002–2021 are investigated using the new product. The spring LFSI area in this region decreases at a rate of 0.67 × 10³ km² per year during this period (R² = 0.117, p < 0.01). According to the spatial and temporal changes, we conclude that the LFSI is becoming more stable while the area is shrinking. The method is fully-automatic and computationally efficient, which can be further applied to the entire Arctic Ocean for LFSI identification and monitoring.

1. Introduction

Climate warming in the Arctic is more than twice the global average because of the ice-ocean-atmosphere feedback mechanism, which is known as the “Arctic amplification” [1,2]. Satellite observations reveal that September Arctic sea ice extent was decreasing at a rate of about 12.6% per decade from 1979 to 2022 [3].

Landfast sea ice (LFSI) is a critical feature along the Arctic coast, accounting for 13% of the total sea ice area in the Northern Hemisphere [4]. It is defined as sea ice that forms and remains fast along the coast, where it is attached to the shore, to the ice shelf or tongue front, or between shoals or grounded icebergs. It can be formed in place from seawater or from sea ice which is drifted to the shore. Therefore, it generally exists in nearshore shallow waters where the depth is less than 25 m [5]. The distance from the LFSI edge to the coastline ranges from a few kilometers to several hundred kilometers due to different geography and seasons. Previous studies have documented that the Arctic LFSI area is decreasing overall [6,7].

LFSI is an essential component of the Arctic coastal ecosystem. The distribution of LFSI plays an important role in the nutrient supply of microorganisms and phytoplankton growth [1,8]. It provides shelters for vertebrates and microorganisms [9], and is a breeding and predation site for a variety of mammals and birds [10,11]. In addition, LFSI is a hunting platform and transportation route for local people [12].

The largest area of LFSI in the Arctic is located near the coasts of the East Siberian Sea, the Laptev Sea, and the Kara Sea. In Laptev Sea, the LFSI edge can extend up to 300–500 km offshore [13]. However, compared to other Arctic marginal seas such as the Kara Sea linked to snowstorm in northern Europe as or the Alaska Beaufort Sea relative to local indigenous communities and industry [14,15,16,17,18] only a few studies are carried out with a focus on the Laptev Sea. LFSI in Laptev Sea is generally one-year ice, forms in October and reaches to a basically unchangeable extent in January after several breakup and reformation processes. Little dynamics occur during winter and spring, except for the breakups caused by the shear effect between drift ice and LFSI [19].

At present, large-scale or regional-scale studies of the changes in LFSI area are mainly based on ice charts or remote sensing [6,7]. However, challenges are still need to overcome concerning large-scale, high resolution and continuous monitoring of LFSI. For the higher frequency changes in LFSI, such as breakup and reformation processes that can occur within several days, which is easy to be missed by traditional large-scale LFSI products (e.g., NIC ice chart) that were typically investigated at a relatively coarse time interval of >10 days. Fraser et al. [20,21] applied the edge detection method [22] to extract LFSI edge in Antarctica from consecutive cloud-free MODIS images, which still needs manual modification. In addition, the temporal resolution (15 days) requires improvements. Spatially, LFSI is a relatively narrow target in the remote sensing images, which is much smaller than the drifting pack ice even in the most stable conditions during winter [21]. Therefore, high-resolution images are needed to avoid misjudgment between LFSI and drifting ice. Studies using high-resolution images are already available based on optical or SAR imagery. Mahoney et al. [17] obtained the annual variation of LFSI in the Chukchi Sea and the Beaufort Sea through the analysis of 2500 SAR images from 1996 to 2008. Dammann et al. [23] extracted the LFSI edge of the Arctic region based on Synthetic Aperture Radar Interferometry (InSAR), which is very sensitive to the movement of sea ice. However, these methods are computationally intensive, and need to be modified visually after the automatic process.

Recent advances have been integrated with deep learning (DL) methods at LFSI detection. Kim et al. [24] used machine learning approaches-random forest (RF) and decision tree (DT) to map LFSI. Nevertheless, there are still lots of unavoidable misjudgments with RF, especially when images are cloud-contaminated. A comparison of several DL models found that Pix2Pix has excellent performance in detecting occluded edges obscured by clouds [25]. The Pix2Pix model [26] is a CGAN (conditional generative adversarial network) method of image-to-image translation model, which is effective for semantic segmentation [27]. Tsuda [28] and Chen et al. [25] also stated that the Pix2Pix (without discriminator) provides more information on branch paths, which is similar to the characteristics the of LFSI edge features. Therefore, we selected this model to retrieve LFSI since it has the potential to reduce the LFSI misclassification caused by clouds.

In this paper, we focus on Laptev Sea, and apply Pix2Pix on MODIS true-color imagery for LFSI identification, in order to facilitate the fully automatic and efficient identification of LFSI. Spring variations are investigated for the period 2002–2021 based on the derived LFSI extent. This method can be further applied to the entire Arctic Ocean for LFSI identification and monitoring.

2. Study Area and Data

In this study, we focus on the Laptev Sea (Figure 1), which is located on a wide continental shelf of eastern Arctic Ocean, between the Taymur Peninsula, the Northern Islands, Novosibirsk, and the Kotelny Islands. The LFSI in this region enters a relatively stable condition during spring (March–May) each year; therefore, this study focuses on intraseasonal changes in spring to identify the LFSI in the Laptev Sea.

MODIS true-color imagery provides excellent geographical coverage at 250-meter resolutions and daily timescales. Thus, the original data of this study are from MODIS sensors on both the Terra (MOD) and Aqua (MYD) satellites. True color images used in this study are acquired from the official website of National Aeronautics and Space Administration (NASA; https://worldview.earthdata-.nasa.gov/, accessed on 16 April 2021), which is composed of band 1 (620–670 nm), band 3 (459–479 nm), and band 4 (545–565 nm). Characteristics of the land surface, ocean, ice and clouds can be recognized from these images. The time series is available from 2002 to 2021.

Weekly sea ice concentration charts are used as a comparison of our study. The U.S. National Ice Center (NIC) provides ice concentration data along with total LFSI extent (https://nsidc.org/data/g10033/versions/1, accessed on 16 April 2021). The NIC LFSI extent is derived from multi-resources data, such as in-situ observation data and multi-resource quasi-real-time satellite remote sensing. The NIC ice chart is released weekly or biweekly, which indicates that the data represent the ice conditions over the time window.

3. Method

3.1. Pix2Pix Model

Pix2Pix is a CGAN method of image-to-image translation model. It consists of two independent networks, a generator and a discriminator. Parameters are set as follows: Pooling size = 50, batch size = 1, learning rate = 0.0002, and epochs = 100 [26], with both input and output sizes of 1024 pixels. In the process of adversarial training, the goal of the generator is to generate real pictures to the greatest extent possible to deceive discriminator. The goal of discriminator is to distinguish between the pictures generated by generator. When the generator is able to create results that can cheat the discriminator, the image-to-image translation is well-trained. As shown in Figure 2, the generator of Pix2Pix is an improved U-Net structure [29], which has eight downsampling blocks in the encoder and eight upsampling blocks to facilitate the extraction of image features. The length, width, and number of channels of the images for each layer are represented in brackets. The discriminator of Pix2Pix is a Patch GAN with three downsampling blocks followed by two convolutions with a stride of 1. The last layer of the discriminator includes a sigmoid function, sin order that the probability score can be [0, 1]. Specifically, the number approaching 1 suggests that the image is closer to the ground truth.

Mathematically, Pix2Pix learns a mapping function from an input image $X$ to the output image Y. The loss function $L_{pix2pix}$ is expressed as:

$$L_{pix2pix}=E\left[logD\left(X,Y\right)\right]+E\left[log\left(1-D\left(X,Y_{Pix2Pix}\right)\right)\right]+\lambda L_1$$

(1)

$$L_1=E\left[{‖Y-Y_{Pix2Pix}‖}_1\right]$$

(2)

where $Y_{pix2pix}$ is the image generated by $G$, λ is the weight parameter, which is set based on the magnitude of the data. $L_1$ is a traditional standard loss function, which is used to minimize the distance between the generated image $Y_{pix2pix}$ and the true output image $Y$ for minimized blurring effects. The objective function $G^{\ast }$ is:

$$G^{\ast }=arg \underset{G}{min} \underset{D}{max}\left\{L_{pix2pix}\right\}+\lambda E\left[{‖Y-Y_{Pix2Pix}‖}_1\right]$$

(3)

The objective function represents the distance between the detected value and the real value. For more accurate results of the LFSI identification, the parameters must be adjusted to minimize the objective function of G^∗.

3.2. Image Processing

Composite images. First, we use the consecutive images of 7 days to generate a composite image, and then perform quadratic enhancement on the image to improve the visualization of the LFSI edges, which may be contaminated by clouds for some individual image (Figure 3). Of note, we name the images as the last day of the 7 days. For example, the composite image from 1 to 7 March is marked as 7 March.

Label producing. Currently, there is no public LFSI training dataset available. Therefore, we manually extract LFSI areas and non-LFSI areas as training samples of deep learning. Non-LFSI areas include drifting pack ice, open water, leads, and polynyas. Figure 4 shows the composite image and corresponding label on 21 March 2010.

Training dataset. As Pix2Pix model requires, a one-to-one pair mapping between the original data and the corresponding label are used as the training data, as shown in Figure 4. Therefore, we select 108 pairs of composite images and corresponding labels in March, April, and May from 2010 to 2020 as the training dataset with an interval of 7 days.

Filter. Filtering is needed after the LFSI area is derived from the machine learning model. The whole filtering process includes three steps: Land mask, flood filling, and area control. As shown in Figure 5, land mask is used to eliminate the misjudgment of the model on the continent, which completes the conversion process from A to D (Figure 5b). The flood-filling algorithm is used to reduce the misjudgment of the model on the non-LFSI area, which completes the conversion process from B to E (Figure 5b). A threshold is set to eliminate abnormal areas to improve the ability of the model for LFSI recognition (C to F in Figure 5b). The filter process is fully automatic after the LFSI identification by the Pix2Pix model.

Testing dataset. Data obtained from years of 2002 to 2009 and in 2021 are used to validate the performance of Pix2Pix LFSI identification. The corresponding 108 labels (12 labels per year) are produced manually similar to the training dataset.

4. Results and Discussion

After Pix2Pix identification and filter processing, we obtain a new LFSI mapping model. I o verify the effectiveness of this model, we carry out accuracy verification, error analysis, and perform comparison experiments with public data sets. Furthermore, we generate spring LFSI products in the Laptev Sea over the past two decades and investigate its spatial and temporal variations.

4.1. LFSI Mapping Model Performance

The performance of the LFSI mapping model is quantified using the metrics of precision, Recall, and F₁-score.

Precision is the ratio of true positives to the sum of false positives and true negatives. It is also known as a positive predictive value, given as

$$Precision=\frac{TP}{TP+FP} ,$$

(4)

where TP (True Positive) is the number of samples, in which the true value is LFSI, and the classification result is correct.

FP (False Positive) refers to the number of samples, in which the true value is non-LFSI, whereas the classification result is LFSI.

Recall is the ratio of correctly predicted outcomes to all the actual LFSI values. It is also known as sensitivity or specificity and can be calculated as:

$$Recall=\frac{TP}{TP+FN}$$

(5)

where FN (False Negative) is the number of samples, in which true value is LFSI, whereas the classification result is non-LFSI.

F₁-score is the harmonic mean of precision and recall, with the value ranging from 0 to 1. It is a measure of classification quality. The method can be considered as more effective with the higher value of F₁-score, which is calculated as:

$$F_1-score=\frac{2TP}{2TP+FP+FN}$$

(6)

where FP (False Positive) is the number of samples, in which the true value is non-LFSI, whereas the classification result is LFSI.

The derived metrics are shown in

Table 1. Accuracy assessment results for the LFSI mapping model using the test dataset.

Year	Precision	Recall	F1-Score
2002	0.928	0.970	0.949
2003	0.903	0.988	0.951
2004	0.902	0.945	0.918
2005	0.910	0.977	0.941
2006	0.909	0.980	0.943
2007	0.901	0.988	0.942
2008	0.914	0.975	0.944
2009	0.920	0.985	0.951
2021	0.951	0.995	0.972
Average	0.914	0.979	0.945

. The LFSI mapping model identifies the LFSI area at an average precision of 91.4% and recall of 97.9%. Table 1 shows that recall is higher than precision for each year, which indicates that the LFSI area derived from the LFSI mapping model may be higher than the actual value. As a comprehensive index, the average F₁-score of 94.5% indicates that the model has good performance.

4.2. Retrieval of LFSI Area under Cloud Contamination

The LFSI mapping model identification results are mainly influenced by cloud. In the area covered by thin cloud, the model has a good effect on LFSI extraction, as shown in Figure 6. The LFSI edges can be extracted accurately in Figure 6b, compared with the LFSI edges visually inspected through the thin clouds in Figure 6a. However, the identification results in thick cloud areas are mainly misjudged by drifting pack ice, wrinkled clouds, water leads and the cloud itself. In a study by Fraser et al. [30], cloud-free image products were used to generate composite image, which leads to unnecessary “data holes” in the composite images, since the areas covered by thin clouds are also eliminated while the surface features of which are still discernible. Moreover, Fraser et al. [21] generated a complete library of gridded cloud mask granules afterwards to intelligently sort the cloud content over a period of 15 days to select the least cloudy scenes for subsequent manual and automated edge-detection purposes. Therefore, the extraction of LFSI edges using MODIS visible images is inevitably affected by cloud cover. Through our analysis, the difficulties of LFSI identification in thick-cloud areas include the following: (1) The LFSI mapping model is likely to misjudge the cloud shape contour as a LFSI edge; (2) the cloud obscures the edge features of LFSI; and (3) the LFSI mapping model is likely to overfit since because there are similar thick-cloud areas in local areas of the training set.

4.3. Comparison with NIC Ice Chart

The data from the LFSI mapping model are compared with the LFSI area extracted from the NIC ice chart with different periods considering the variability of LFSI. Although the NIC chart includes 14-days (or 7 days in some years) of sea ice distribution of the entire Arctic Ocean, its drawback is a relatively low spatial resolution of 10 km, while our LFSI mapping products derived from Pix2Pix has a spatial resolution of 1.25 km, which can provide more details on the Laptev LFSI conditions than the NIC chart.

Image obtained on 19 April 2021 is compared with the NIC ice chart in Figure 7. Typically, the identification of LFSI based on Pix2Pix matches the NIC data, as shown in Figure 7a. However, if the region is dominated by thick clouds over the 7-day time window, the results of Pix2Pix do not fit well with the NIC charts (Figure 7b). In case with thin clouds, the LFSI map-derived from the model has good perfomance as well (Figure 7c). For drifting pack ice, the LFSI mapping model matches with the NIC ice charts (Figure 7d).

4.4. Spatiotemporal Variation of LFSI in the Laptev Sea

Spring LFSI products of Laptev Sea are derived with a 7-day interval from 2002 to 2021. Based on the products, we examined the spatiotemporal variations of spring LFSI. The occurrence of LFSI from 2002 to 2021 is shown in Figure 8. It is worth noting that the higher the occurrence through the investigated season, the more stable the LFSI in this area. The ratio ($S$) of LFSI occurrence ($O$) larger than 0.5 ($O_{>0.5}$) and total LFSI area ($O_{>0}$) through the season, is introduced to represent the stability of a given year:

$$S=\frac{O_{>0.5}}{O_{>0}},$$

(7)

Based on LFSI stability, the study period is divided into three categories. The year when S > 97% or S < 92% is defined as an stable or unstable year, respectively. The remaining years are classified as relatively stable years. Therefore, unstable years of LFSI include years from 2003 to 2007; relatively stable years include years of 2002, 2008–2013, 2016, 2017, and 2020; and stable years include years of 2014, 2015, 2018, 2019, and 2021. This result infers that the spring Laptev Sea LFSI is transforming from unstable to stable.

Figure 9 shows the variation of LFSI area in the Laptev Sea in spring. We consider every 5 years as a node and divide them into four groups, i.e., 2002–2006, 2007–2011, 2012–2016, and 2017–2021. In the figure, the dotted line is the trend line, and the area with the same color with the dotted line is the upper and lower limit of the standard deviation. The seasonal trend is obtained based on a least squares method, which is −0.67 ± 5.04 × 10³ km² per day for 2002–2006, −0.57 ± 3.88 × 10³ km² per day for 2007–2011, 0.21 ± 1.47 × 10³ km² per day for 2012–2016, and 0.17 ± 1.03 × 10³ km² per day for 2017–2021, respectively. Seasonal variations are not consistent in these four groups, showing both positive and negative trends. This is due to the fact that the spring seasonal change is mainly influenced by dynamical processes such as collisions, shears, and breakups, rather than thermal factors. The standard deviation was decreasing gradually from 2002–2006 to 2017–2021, suggesting that the seasonal variations in the LFSI area in Laptev Sea are becoming smaller. This result is in accordance with the LFSI occurrence as shown in Figure 8. The fluctuation in LFSI extent is greater in March than May over the last two decades (Figure 9), which is due to the fact that the shear effect resulting from the drifting sea ice is stronger in March, which results in more breakups and reformation events of LFSI [19].

The maximum, minimum, and mean values of LFSI area for each spring are obtained. The average area of the Laptev Sea in spring over the last two decades is 260.85 × 10³ km². Compared with the study by Yu et al. [7], which reported that the LFSI in Laptev Sea is 224.29 × 10³ km² from 1976 to 2007 (January–May), our result is higher. This is partly explained by the fact that the NIC chart data underestimate the extent of LFSI. Besides, the study area of Laptev Sea we used is different, namely, 2002–2021 in our study versus 1976–2007 in the study by Yu et al. Spring LFSI area in Laptev Sea decreased at a rate of 0.67 × 10³ km² per year in the last two decades. Compared with the study by Yu et al. [7], whose result was −1.88 ± 0.31 × 10³ km² from 1976 to 2007, our trend of spring LFSI at Laptev was relatively smaller.

We extracted the dates when the spring LFSI area reached its maximum during these two decades, as shown in Figure 10. The LFSI reached its maximum in March in the spring of 2002, 2003, 2005, 2007, 2008, 2010, 2014, 2015, 2018, 2020, in April of 2004, 2006, 2009, 2011, 2013, 2016, 2019, and in May of 2012, 2017, 2021. We conclude that the date when the Laptev spring LFSI reached the maximum has been delayed over the past two decades. Since the LFSI forms near the coast, and the water depth is usually shallow, the atmosphere boundary layer condition dominates the duration of LFSI. Model experiments showed that the albedo, oceanic heat flux, air temperature, and snow accumulation affects the date of ice breakup [31,32]. The time when LFSI reaches the maximum extent in this region needs further analysis in combination with other data and numerical models.

Furthermore, relationships of LFSI formation and breakup events and wind are investigated. We calculated the regional-mean wind speed and direction of our study area. Figure 11a presented a formation event during 14–21 March 2004 in the southwest Laptev Sea. Northeastward expansion of LFSI can be observed. In the meantime, northeasterly wind is identified during 17–21, March. The southwestward wind drives new-formed ice or drift ice in the Laptev polynya to the shore, causing accumulation of ice at the outer edge of LFSI, and further resulting in its extension. On the other hand, the off-shore wind can cause LFSI to break, which is shown in Figure 11c,d. A big area of LFSI is broken down from the whole LFSI pack in the southeast Laptev Sea. Southerly wind especially the southeasterly wind from 7 to 14 March blows the LFSI at the outer edge to drift away from the shore. Note that wind is not the only factor that causes the LFSI to breakup, shear effect, ice thickness and strength, tides, and other factors are important as well. Here, we simply consider the influence of winds.

5. Conclusions

In this study, a new and improved model to identify spring Laptev Sea LFSI in Laptev Sea is introduced. To achieve a high spatio-temporal resolution, consecutive MODIS images are used for compositing and enhancing to obtain the LFSI edge line. Compared with the manual or semi-automatic extraction of LFSI by previous studies, this model can acquire spring Laptev Sea LFSI distribution in both clear-sky and thin-cloud cases while maintaining a high accuracy. In the meantime, it is fully automatic and more efficient. To prove the performance of the model, the precision evaluation is conducted, with an average precision of 91.4%, the recall of 98.7%, and F₁-score of 94.5%. The LFSI produced from our method matches with the NIC ice chart, but have higher resolution (1.25 km). Therefore, it provides more details of LFSI.

This model is used to generate LFSI coverage from March to May during the period of 2002–2021. Temporal and spatial changes in the LFSI distribution in the Laptev Sea in the past two decades are investigated based on our LFSI product. Stable, relatively stable, and unstable LFSI years are classified according to the seasonal occurrence. Together with the seasonal trend, our result suggests that the spring Laptev Sea LFSI is becoming more stable. In the meantime, the interannual study of LFSI area shows that the spring LFSI area in Laptev Sea decreased at a rate of 0.67 × 10³ km² per year in the last two decades. Specifically, the LFSI is becoming more stable while the area is becoming smaller.

Although this model achieves good results in identifying thin-cloud areas and cloud-free areas in most cases, it still does not perform well in thick-cloud areas. In the years with extreme LFSI areas, the previous studies suggest that important factors affecting the LFSI in this region are wind and air temperature. The influences of wind on LFSI formation and breakup are briefly discussed. However, what drives the seasonal and interannual changes in LFSI obtained in this study and how these factors work still requires further investigations.

In terms of the above problems, more research works are still needed to improve this method for LFSI identification and monitoring, which include: (1) The automatic detection of thick cloud areas by combining other remote sensing data, and (2) characterizing main factors that influences the seasonality of LFSI combining other environmental data, such as river runoff data, LFSI observations from coastal sites, atmospheric reanalysis data, polynya distribution, and distribution and movement of drifting ice.