Next Article in Journal
Effects of Atmospheric Correction and Image Enhancement on Effective Plastic Greenhouse Segments Based on a Semi-Automatic Extraction Method
Previous Article in Journal
Georeferencing Accuracy Assessment of Historical Aerial Photos Using a Custom-Built Online Georeferencing Tool
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Robust Watermarking Scheme for Vector Geographic Data Based on the Ratio Invariance of DWT–CSVD Coefficients

1
School of Geography Science and Geomatics Engineering, Suzhou University of Science and Technology, Suzhou 215000, China
2
National Marine Environmental Monitoring Center, Dalian 116085, China
*
Author to whom correspondence should be addressed.
ISPRS Int. J. Geo-Inf. 2022, 11(12), 583; https://doi.org/10.3390/ijgi11120583
Submission received: 5 September 2022 / Revised: 6 November 2022 / Accepted: 20 November 2022 / Published: 22 November 2022

Abstract

:
Traditional frequency-domain watermarking algorithms for vector geographic data suffer from disadvantages such as the random watermark embedding position, unpredictable embedding strength, and difficulty in resisting multiple attacks at the same time. To address these problems, we propose a novel watermarking algorithm based on the geometric invariance of the ratios of discrete wavelet transform (DWT) and complex singular value decomposition (CSVD) coefficients, which embeds the watermark information in a new embedding domain. The proposed scheme first extracts feature points from the original vector geographic data using the Douglas–Peucker algorithm, and then constructs a complex sequence based on the feature points set. The two-level DWT is then performed on the complex sequence to obtain the low frequency coefficients (L2) and high frequency coefficients (H2). On this premise, the CSVD algorithm is utilized to calculate the singular values of L2 and H2, and the ratio of the singular values of L2 and H2 is acquired as the watermark embedding domain. During the watermark embedding process, a new watermark sequence is created by the fusion of the original watermark index value and bits value to improve the recognition of the watermark information, and the decimal part at different positions of the ratio is altered by the new watermark sequence to control the watermark embedding strength. The experimental results show that the proposed watermarking algorithm is not only robust to common attacks such as geometric, cropping, simplification, and coordinate point editing, but also can extract watermark images with a high probability under random multiple attacks.

1. Introduction

The vector geographic data are widely utilized in urban planning, traffic navigation, disaster prevention and control, military applications, and other fields with a high socio-economic value [1,2,3,4]. With the rapid development of software, hardware, and internet technology, the sharing and transmission of vector geographical data is becoming simpler and more efficient, which may result in many copyright issues such as illegal storage, copying, and trafficking [4,5,6,7,8]. As a cutting-edge technology in the field of information security, digital watermarking is attracting increasing attention and, over time, has become an important technology for the copyright protection of vector geographical data [9,10].
Digital watermarking works by hiding meaningful (e.g., text or images containing producer information) or meaningless (e.g., randomly generated binary sequences) copyright information into the host data by preset information-concealing algorithms. In cases of copyright dispute, the information can only be extracted by re-producing the preset watermarking algorithms, and copyright can then be authenticated through an information comparison [9]. This characteristic determines that when digital watermarking is applied to vector geographic data, it is necessary to find a suitable watermark embedding domain that can meet the demand of using vector geographic data with a high accuracy, but also ensures that the watermarking algorithm is robust enough to resist complex editing of the vector geographic data. Generally speaking, digital watermarking algorithms for the vector geographic data are classified into spatial or frequency domain algorithms according to the position of the embedded watermark [11]. Cox et al. [12] directly encoded the coordinate values and embedded the watermark information into each vertex, and this was the first spatial domain watermarking approach for the vector data. Despite its lack of robustness, it has been a focus for further research. As the research has progressed, scholars have provided more solutions for improving the performance of the algorithm and have also mined increasing numbers of watermark embedding domains. Wang et al. [13] embedded watermark information by adjusting the value of the spatial topological metric to ensure the performance of the watermarking procedure, using the stability of the topological relations, which also guaranteed that the embedding of the watermark did not result in changes to the topological relations. Lee et al. [14] used relative and absolute position accuracy as constraints to embed watermarks in the perimeter of the polylines and the area of the polygons, ensuring that the vertices embedded in the watermark were within the allowable tolerance. This method could more successfully control the accuracy and topology changes after the watermark was embedded. Wu et al. [15] used polygonal similarity transformation, modulating watermark information in the feature set of spatial relationships of polygonal elements based on quantized index modulation (QIM), effectively maintaining the geometric features. Ren et al. [16] used the angular value between the line segments as the watermark embedding domain and embedded the watermark information using QIM, which had very little impact on the data fidelity. Generally, in the current research of spatial domain watermarking algorithms for vector geographic data, the geometric feature information or statistical information of various data are mined as the embedding domain of the watermark, and a good invisibility can be achieved by controlling the watermark embedding strength and method. However, in this type of algorithm, the watermark information is directly hidden in the coordinates of the vector geographic data, which can be easily removed via precision erase, and as a consequence this type of algorithm tends to be less secure and less practical [17].
Frequency domain watermarking algorithms are generally more robust and covert than spatial domain watermarking algorithms [10,18]. The watermark information is embedded by modifying the frequency domain transform coefficients of the coordinate values in the transformation domain scheme, which brings about a stronger concealment [19]. There are three classical frequency domain algorithms: (1) discrete Fourier transform (DFT), (2) discrete wavelet transform (DWT), and (3) discrete cosine transform (DCT), all of which have been widely studied in vector geographic data watermarking [20]. Xu et al. [21], Neyman et al. [20], and Zhang et al. [2] exploited the geometric invariance of the DFT coefficients to embed the watermark information. These algorithms were robust to geometric arracks but difficult in resisting vertex attacks such as vertex deletion, vertex insertion, or map cropping attacks. Zhang [22] enhanced the previous literatures by embedding watermarking information both in the DFT coefficients and spatial domain. The scheme was resistant to geometric attacks and vertex attacks but unable to resist combination attacks. Kitamura et al. [23] uniformly subdivided the vertexes into rectangular grids, and then taken each rectangular grid as the “pixel” of the grid image. Then, DWT for the images was utilized to embed the watermarks. Yang et al. [24] and Zope-Chaudhari et al. [25,26] achieved a copyright protection of the vector geographic data by embedding the watermarking information in the DWT coefficients. These schemes showed a strong robustness against noise, vertex deletion, and format transformation attacks. However, those methods were susceptible to geometric attacks and had a high impact on the data fidelity. Schemes [27,28] concealed information by fine-tuning the middle frequency of the DCT coefficients. The experimental results showed that their watermarking algorithm had a good resistance to geometric attacks but a poor invisibility. Wang et al. [29] embedded the watermark in the high-frequency coefficients of the DCT domain while also developing an error control method, which helps to reduce distortion after the watermark embedding. In summary, the frequency domain watermarking of the vector geographic data is gaining interest, but due to the nature of the algorithms, it is challenging to withstand multiple attacks at once, and the position of the watermark information during embedding is highly random, making it challenging to control the invisibility. These drawbacks seriously limit its potential for a practical application. To address these problems, we will dig deeper into the feature invariants in the DWT coefficients, aiming to design a digital watermarking scheme that can withstand multiple attacks while controlling the invisibility.
The objectives of this study are to: (1) propose a robust watermarking scheme for vector geographical data based on the ratio stability of the DWT and CSVD coefficients, and (2) to evaluate the performance of the proposed scheme through attack experiments and statistical analysis.

2. Methods

2.1. Ratio Invariance of DWT-CSVD Coefficients

The frequency transformation is implemented in the polyline which can be donated as L = E , V with an edge set E and a vertex set V = v 1 , v 2 , , v n , where v i = x i , y i , x i , y i . A polygon can be expressed as a closed polyline.
The proposed ratio invariance of DWT–CSVD transformation coefficients refers to the property that the ratio of the singular values, which are obtained from the CSVD algorithm performed on the low- and high-frequency coefficients of 2-level Haar wavelet-based DWT, is invariant under rotation, scaling, and translation (RST) operations. Details of the invariance are as follows: firstly, the complex sequence C = c 1 , c 2 , , c n is constructed from vertex set V according to Equation (1):
c k = x k + y k × i k = 1 , 2 , n
Then, the complex sequence C is transformed by Haar wavelet-based DWT to obtain the multi-resolution representations: L and H , where L is the low-frequency coefficient, also known as the approximate component which includes most energy of the original vertexes, and H is the high-frequency coefficient, also known as the detail component, which includes little energy. After that, perform Haar wavelet-based DWT on H to obtain the multi-resolution representations of H : L 2 and H 2 , where L 2 is the approximate component of H and H 2 is the detail component of H .
Based on L 2 and H 2 , the complex matrixes M a t L 2 and M a t H 2 are constructed, respectively. The CSVD algorithms performed on M a t L 2 and M a t H 2 are shown in Equations (2) and (3).
M a t L 2 = U L 2 × S L 2 × V L 2 H = U L 2 × s l × V L 2 H
M a t H 2 = U H 2 × S H 2 × V H 2 H = U L 2 × s h × V L 2 H
where U L 2 and V L 2 represent the unitary matrix consisting of the left and right singular vectors, respectively. V L 2 H is the Hermite matrix of V L 2 , and S L 2 is the singular value matrix. Because M a t L 2 is a one-dimensional matrix, after CSVD, the singular value matrix, S L 2 , has only one element, i.e., the singular value s l . Equation (3) follows the same principle as Equation (2). Finally, the ratio K of the DWT–CSVD transformation coefficients can be calculated by Equation (4) after acquiring the singular values of L 2 and H 2 .
K = s l s h
The ratio K will keep invariant, after the original vertexes are attacked by RST. The invariance outlined above will be proved theoretically by the following:
First, the one-dimensional Haar wavelet-based DWT can be expressed by the form of matrix W η ( η is an even number), which is shown in Equation (5) [30]:
W η × η = 2 / 2 2 / 2 0 0 0 0 0 0 2 / 2 2 / 2 0 0 0 0 0 0 2 / 2 2 / 2 2 / 2 2 / 2 0 0 0 0 0 0 2 / 2 2 / 2 0 0 0 0 0 0 2 / 2 2 / 2
The following is an expression for the transformation of the original signal using Haar wavelet-based DWT:
W η × x 1 + y 1 i x 2 + y 2 i x η + y η i = 2 2 × x 1 + x 2 + y 1 + y 2 i x 3 + x 4 + y 3 + y 4 i x η 1 + x η + y η 1 + y η i x 1 x 2 + y 1 y 2 i x 3 x 4 + y 3 y 4 i x η 1 x η + y η 1 y η i
H η / 2 = 2 2 × x 1 + x 2 + y 1 + y 2 i x 3 + x 4 + y 3 + y 4 i x η 1 + x η + y η 1 + y η i
G η / 2 = 2 2 × x 1 x 2 + y 1 y 2 i x 3 x 4 + y 3 y 4 i x η 1 x η + y η 1 y η i
where H η / 2 and G η / 2 denote the approximate component and detail component, respectively.
When the original vertexes are translated along the x-direction by α and along the y-direction by β , there is
W η × ( x 1 + α ) + ( y 1 + β ) i ( x 2 + α ) + ( y 2 + β ) i ( x η + α ) + ( y η + β ) i = 2 2 × x 1 + x 2 + 2 α + y 1 + y 2 + 2 β i x 3 + x 4 + 2 α + y 3 + y 4 + 2 β i x η 1 + x η + 2 α + y η 1 + y η + 2 β i x 1 x 2 + y 1 y 2 i x 3 x 4 + y 3 y 4 i x η 1 x η + y η 1 y η i
From the above equations, as can be seen, the detail component is unaffected when the original vertexes are translated. On this basis, it is straightforward to prove that the approximate and detail components of the 2-level DWT are invariant when the original signal is translated. Clearly, the ratio of the singular values will not change.
When the original vertexes are scaling by γ   ( γ > 0 ) , there is:
W η × ( x 1 × γ ) + ( y 1 × γ ) i ( x 2 × γ ) + ( y 2 × γ ) i ( x η × γ ) + ( y η × γ ) i = 2 2 × γ × x 1 + x 2 + y 1 + y 2 i x 3 + x 4 + y 3 + y 4 i x η 1 + x η + y η 1 + y η i x 1 x 2 + y 1 y 2 i x 3 x 4 + y 3 y 4 i x η 1 x η + y η 1 y η i
It shows that both the approximate component and detail component are γ times the original value. The following analyze the characteristics of the singular values. The singular value for a complex matrix A η × 1 can be used to calculate Equations (11) and (12).
d e t λ I A H × A = 0
s A = λ
where d e t is an operation of obtaining the determinant, λ A is the eigenvalue of matrix A H × A , I is a unit matrix, and s A is the singular value of A . In case of the matrix B = γ × A , there is:
d e t λ I γ 2 × A H × A = 0
s B = γ 2 λ = γ s A
where s B is the singular value of B . After scaling, the singular value is also scaled by the scaling ratio. The ratio of the singular values of the approximate component and detail component is identical to the original ratio since both the approximate component and detail component are scaled by the same ratio when the original vertexes are scaled.
When the original signal is rotated by θ , there is:
W η × x 1 × cos θ y 1 × sin θ + x 1 × sin θ + y 1 × cos θ i x 2 × cos θ y 2 × sin θ + x 2 × sin θ + y 2 × cos θ i x η × cos θ y η × sin θ + x η × sin θ + y η × cos θ i = 2 2 x 1 + x 2 × cos θ y 1 + y 2 sin θ + x 1 + x 2 × sin θ + y 1 + y 2 cos θ i x 3 + x 4 × cos θ y 3 + y 4 sin θ + x 3 + x 4 × sin θ + y 3 + y 4 cos θ i x η 1 + x η × cos θ y η 1 + y η sin θ + x η 1 + x η × sin θ + y η 1 + y η cos θ i x 1 x 2 × cos θ y 1 y 2 sin θ + x 1 x 2 × sin θ + y 1 y 2 cos θ i x 3 x 4 × cos θ y 3 y 4 sin θ + x 3 x 4 × sin θ + y 3 y 4 cos θ i x η 1 x η × cos θ y η 1 y η sin θ + x η 1 x η × sin θ + y η 1 y η cos θ i
Based on Equations (11) and (12), the singular values of the aforesaid approximate component and detail component are as follows:
s A p = 2 2 × k = 1 η 2 x 2 k 1 + x 2 k 2 + y 2 k 1 + y 2 k 2
s D e = 2 2 × k = 1 η 2 x 2 k 1 x 2 k 2 + y 2 k 1 y 2 k 2
where s A p and s D e are the singular values of the above approximate component and detail component, respectively. When performing the same option on the original approximate component and detail component, the singular values acquired are equal to the singular values above. Additionally, the ratios of the singular values are constant.
Since the ratio of the singular values of the approximate component and detail component is not altered by RST attacks, it is robust and provides a theoretical basis for digital watermarking algorithms for vector geographic data.

2.2. Watermark Information Generation

The watermark used in this paper is a binary image. Figure 1a shows a sample of a binary image with a size of 4 × 4 bits, and Figure 1b,c are the watermark index and watermark bits at the corresponding positions of the watermark image.
To enhance the security of the watermark, it is necessary to scramble the watermark using the Arnold transformation before embedding the watermark [31]. Arnold transformation, also called cat mapping which is proposed by Russian mathematician V.I. Arnold [32], works by mapping the original pixel position p , q to a new position p , q . Arnold transformation is now generalized by introducing two parameters a and b, as shown in Equation (18) [33].
p q = 1 a b 1 + a b p q m o d   N
where N is the size of the watermark image and m o d is the modulo operation, a , b + . The parameters a and b can be used as a secret key for watermark detection.
Inverse Arnold scrambling transformation can be expressed as:
p q = a b + 1 b a 1 p q m o d   N
To improve the recognition of the watermark information, the original watermark bits are mapped to new values according to Equation (20):
b i t m = 3 , b i t o = 0 7 , b i t o = 1
where b i t o is the original watermark bit and b i t m is the mapped bit. Obviously, the mapping process is reversible.
The final watermark information is created by combining watermark indexes with a specified fixed “length” and mapped watermark bits in order to ensure watermark synchronization. The “length” of the watermark index is the total number of digits, including the “0” value at the beginning. For example, suppose the fixed “length” is set to 3, the watermark index 16 will be expressed as “016” and suppose the fixed “length” is set to 4, the watermark index 16 will be expressed as “0016”. Take the watermark image in Figure 1 as the original watermark, the final watermark information is shown in Figure 2 when the fixed “length” is set to 3.

2.3. Watermark Embedding Rules

The ratio of the DWT–CSVD coefficients remains constant when rotating, scaling, and translating the vector geographic data, which is the basis of the robustness of the proposed watermarking algorithms. Based on this theory, the watermark information will be embedded into the coefficient ratio. The specific embedding rule is shown in Equation (21).
K = F K , 1 / ε , w
where ε > 0 ,   ε ,   1 / ε can be regarded as the watermark embedding strength, by function F , the watermark information w is embedded to the ε     th decimal place of the original DWT–CSVD transformation coefficients ratio, i.e., K , and K refers to the new ratio containing the watermark information. Figure 3 illustrates the embedding processing where the ε     th to the ( ε + p 1 )   th decimal places of K are replaced by w . For example, if K = 3.141592653589793 , ε = 4 , and w = 0157 , then K = 3.141015753589793 .

2.4. Process of Watermark Embedding and Detection

2.4.1. Process of Watermark Embedding

The proposed watermark embedding process is provided in Figure 4, and the specific algorithm steps are as follows:
Step 1: the Douglas–Peucker algorithm [34] is executed on the vertexes to obtain the feature points.
Step 2: the complex sequence of the feature points is constructed using Equation (1), then the 2-level DWT coefficients with a low frequency ( L 2 ) and high frequency ( H 2 ) are calculated from the complex sequence.
Step 3: the CSVD algorithm is applied for L 2 and H 2 , and the corresponding singular values are denoted as S L and S H , respectively. Then, their ratios K are calculated.
Step 4: generate the watermark information according to Section 2.2, and the watermark information is embedded into the ratio K by Equation (21), so as to obtain the watermarked ratio K .
Step 5: the watermarked singular value of the approximate component, denoted as S L , is calculated according to Equation (22).
S L = K × S L
Then, the watermarked approximate component, referred to as L 2 , is obtained using the inverse complex singular value decomposition (ICSVD) algorithm. The ICSVD algorithm is described as (23).
L 2 = U L 2 × S L × V L 2 H
where U L 2 and V L 2 H denote the unitary matrixes consisting of the left and right singular vectors of L 2 , respectively.
Step 6: the complex sequence C containing the watermark is computed using the inverse discrete wavelet transform (IDWT), as shown in Equations (24) and (25) where W H a 1 and W H a 2 are the inverse matrixes of the Haar wavelet-based DWT matrix. Then, the real and imaginary parts of C are separated to obtain the watermarked feature points coordinates. Finally, the obtained feature points coordinates are used to replace the original feature points coordinates to gain the watermarked vector geographical data.
H = L 2 H 2 × W H a 1
C = L H × W H a 2

2.4.2. Watermark Extraction and Detection

The distortion of the watermarked data is commonly used to assess the quality of the digital watermarking algorithms. When the watermark is implanted in the host data, there are inaccuracies between the original vector geographic data and the watermarked data. To evaluate the accuracy of the watermarked data, the maximum error (Max.Error) and mean square error (MSE) are employed as the error metrics to gauge the variability. The Max.Error and MSE equations are provided below.
M a x . E r r o r = max x i x i 2 + y i y i 2 i = 1 , 2 , , n
M S E = i = 1 n x i x i 2 + y i y i 2 n
where x i ,   y i represents the original coordinates, x i ,   y i represents the watermarked coordinates, and n denotes the number of vector geographic data vertexes.
Multiple attacks will be implemented on the watermarked data and then the robustness of the proposed scheme was verified by the extracted watermark images. The similarity between the extracted watermark image and the original watermark image is determined using normalized correlation ( N C ). The N C value ranged from 0 to 1. The higher the N C value, the closer the similarity between the two. Specifically, when the N C is equal to 1, it means that the extracted watermark is same as the original watermark. The N C equation is as follows [35]:
N C = i , j X N O R ( A i , j , A ˜ i , j ) p q
where A i , j and A ˜ i , j represent the bits of the original and extracted watermark which are the size of p × q , respectively, X N O R denotes the operator of the exclusive NOR.
The watermark extraction process is the inverse operation of the watermark embedding, and the detail process of the watermark extraction of the proposed algorithm is shown in Figure 5. The procedure for computing the singular value ratio in the watermark extraction algorithm is identical to that in the watermark embedding algorithm, hence it is not discussed here. The steps for extracting the watermark from the singular value ratio are as follows:
Step 1: the watermark index and mapped bit ( b i t m ) are obtained from the ratio K . The real watermark bit ( b i t r ) is calculated by Equation (29).
b i t r = 0 , b i t o < 5 1 , b i t o 5
Step 2: repeat Step 1 until all the watermark indexes and bits are extracted.
Step 3: through Steps 1 and 2, one watermark index may correspond to multiple watermark bits. A voting mechanism, expressed as Equation (30), is utilized to determine the final bit.
W i = 0 , a < b 1 , a b
where i denotes the watermark index, a denotes the number of bits with value 1 and index i , b denotes the number of bits with value 0 and index i . W i denotes the extracted watermark value, which is scrambled.
Step 4: the final extracted watermark image, W , is obtained by the inverse Arnold transformation of W .
Step 5: the NC value between the extracted image W and the original watermark image W is chosen to measure the watermark extraction effect.

3. Experiments and Results

3.1. Experimental Data

To verify the reliability of the proposed watermarking scheme, three data sets, rivers, roads, and village administrative division data were selected as the experimental data (Figure 6). These vector geographic data, including polylines and polygons, were in the ESRI shapefile format, which is commonly used in geographical information systems. For convenience, the experimental data were named as “rivers”, “roads”, and “boundary”, respectively. Table 1 provides the details of the experimental data.
The copyright graph used in the experiments is a binary image with a size of 16 × 16 bits. Figure 7 depicts the watermark image and scrambled watermark image.

3.2. Invisibility Evaluation and Parameters Settings

The embedding of the watermark information causes a certain amount of interference to the host data, influencing the accuracy of the original vector data and presenting the risk that the presence of the watermark information may be perceived by the data receivers. Invisibility as a technical feature of the vector data digital watermarking is a factor that must be considered when evaluating the performance of digital watermarking algorithms.
To evaluate the invisibility of the proposed scheme, Max.Error and MSE between the watermarked and original data were used with embedding strengths of 1/4, 1/5, 1/6, and 1/7. For comparison, two other schemes were also tested: (1) the scheme in ref. [25] which proposed a DWT-based algorithm with an embedding strength of 0.5, and (2) the scheme in ref. [36] which proposed a singular value decomposition-based algorithm with an embedding strength of 1 × 10−6. Table 2 shows the results, indicating that the smaller the watermark strength value, the stronger the imperceptibility of the watermark. Conversely, the larger the watermark strength value, the less imperceptible the watermark is. Furthermore, the experimental results shows that the proposed scheme, at a watermark embedding strength of 1/5, has better invisibility compared with the other two schemes. Figure 8 shows the overlay effect and detail enlargement between the river data with embedded watermark information and the original river data, and it can be seen that the proposed scheme causes little visual difference between the watermarked data and the original data.

3.3. Robustness Evaluation

To evaluate the robustness of the proposed scheme, various types of attacks were carried out at different levels on the watermarked data prior to the watermark extraction. The NC value was utilized to assess the difference between the original and extracted watermarks, and the copyright attribution was established by comparing the magnitude of the NC value and the predetermined NC threshold. In this paper, the NC threshold was set to 0.85, in other words, if the NC value of the original watermark and the extracted watermark was greater than 0.85, the watermark was considered to be successfully extracted from the vector geographic data, otherwise it failed to do so. For comparison, the robustness of the schemes in ref. [25,36] were also evaluated.

3.3.1. RST Attacks

RST refers to rotation, scaling, and translation, which are common geometric transformations in vector geographic data operations. To verify the robustness against the RST attacks, the vector geographic data to be inspected were rotated, translated, and scaled at different degrees and the corresponding extracted watermark images with NC values were recorded. The results are shown in Table 3.
As can be observed, the NC values between the extracted watermark and the original watermark are invariably one for both translational and rotational attacks at different scales, which implies that the proposed scheme is fully resistant to both translation and rotation. When scaling the watermarked data by 0.95 and 1.05, the NC values of all the experimental data are higher than the threshold, 0.85, and the copyright can be visually authenticated by the extracted watermark images. All the NC values are lower than the threshold when the scaling factors are 0.5 and 2. Because the proposed scheme, a blind watermarking scheme, cannot predict the scale degree of the experimental data to be evaluated, many of the feature points acquired by the Douglas–Peucker algorithm could not correlate to the feature points utilized for embedding the watermark as the scale degree increased. Therefore, the proposed scheme could identify a watermark under a minor scaling attack but cannot extract watermark information as the degree of scaling increased.

3.3.2. Simplification Attacks

Simplification is a common operation in geographical analysis. After simplification, the remaining points in the vector geographic data are called feature points, and the deleted points are called non-feature points. A good algorithm should resist attacks from simplification. To test the resistance of the proposed scheme to simplification attacks, the feature points were extracted and the non-feature points were deleted, with simplification rates ranging from 0% to 90%, by the Douglas–Peucker algorithm. Figure 9 depicts the experimental outcomes. The NC values of all the experimental data are invariably one when the simplification rate ranges from 0% to 70%. The outcomes result from that when the simplification rate is lower than the rate at which the original vector geographical data is simplified to extract the feature points, and the proposed scheme could accurately obtain the watermarked points so as to extract the watermark information without errors. When the simplification rate is higher than 70%, where only a few feature points are preserved, the NC values decrease below 0.75. Actually, if a high-intensity simplification attack is performed, the contour of the vector geographic data tends to be straight, and its application value is lost. Therefore, the proposed scheme is particularly resistant to simplification attacks.

3.3.3. Vertex Addition and Deletion Attacks

Vertex addition attacks interfere with the watermark information by introducing extra points into the vector geographic data. To verify the robustness of the scheme to attacks from the vertex addition, 10% to 200% of points were added into the watermarked data. The results are recorded in Figure 10. As can be seen, the NC values remain constant at one, although the adding ratio addition ratio reaches 200%. The experimental findings can be explained by the new points being excluded as non-feature points, which has no impact on the watermark extraction. Therefore, the proposed scheme can totally resist attacks from the vertex addition.
Vertex deletion attacks work by randomly deleting a certain percentage of vertexes, including the feature points, which are not removed by a simplification of the vector geographic data. Figure 11 illustrates the results of the deletion attacks with a deletion ratio ranging from 0% to 90%. As can be seen from the findings, the NC values are greater than 0.85 when the deletion rate is less than or equal to 30%. The NC values of the boundary and roads are higher than 0.85 when 30% to 40% of vertexes are eliminated, yet the watermark in the rivers cannot be detected. When the deletion rate reaches 50%, the proposed schema cannot correctly extract the watermark of any of the experimental data. As the deletion ratio climbs, more feature points carrying watermark information are removed, resulting in a watermark extraction failure. However, an excessively high deletion ratio would have a negative impact on the quality of the vector geographic data, and therefore this approach is robust within a fair range of deletion point attacks.

3.3.4. Cropping Attacks

To verify the robustness of the proposed scheme against map cropping, the watermarked data were cropped by 1/8, 1/4, 1/2, and 3/4, and the proposed scheme were compared with the schemes in refs. [25,36]. Table 4 shows the results of the NC values for the three schemes with different levels of cropping attacks. For rivers data, the schemes in refs. [26,36] fail to extract the watermark information even when the cropping area is 1/8, whereas the proposed scheme could resist the attack with a cutting area of 3/4. For the roads and boundary, the NC values of all the three schemes are one, when cropping off 1/8 and 1/4 of the area. The difference is that as the area of the cropped area increases, the proposed scheme is still able to extract the watermark information accurately, even when 3/4 of the area are cropped off, with NC values higher than 0.9, but the schemes in refs. [25,36] could not. Irregular cropping attacks have a high probability of changing the vertexes ordering, on which a watermark extraction of the schemes in refs. [25,36] rely, resulting in the failure of the watermark extraction. The proposed scheme successfully extracts the watermark bits and indexes by fusing the watermark index and bit values to create a new watermark sequence, guaranteeing a high accuracy for the detection of attacks from cropping.

3.3.5. Combination Attacks

A combination attack is a comprehensive attack that employs multiple attacks on the experimental data at the same time, making it more difficult to defend because of the unknown methods of the attacks and the fact that they are simultaneous. To examine the robustness of the proposed scheme against a combination attack, a 30% simplification attack (d1), 1/2 clipping attack (d2), and 60° rotation attack (d3) were chosen as the combination attack methods in this experiment. Table 5 displays the experimental results, where ✕ means extracting watermark images unsuccessfully and √ denotes extracting watermark images correctly. From Table 5, it can be seen that the proposed scheme could extract the watermark information accurately under a random combination of d1, d2, and d3 attacks, but neither the scheme in ref. [25] nor the scheme in ref. [36] could extract the watermark under the same combination attacks. It can be concluded that the proposed scheme performs well in terms of combination attacks.

3.4. Comparative Analysis

In this paper, we propose a new robust digital watermarking scheme based on the ratio invariance of DWT–CSVD coefficients. In this section, the algorithm performance was evaluated by being compared with prior schemes. In Table 6, √ indicates that the scheme is robust against an attack, and ✕ indicates that the scheme is not robust against that attack. Table 6 shows that the scheme in ref. [25] is robust to addition and deletion attacks, mainly due to the facts that the watermark is embedded in the DWT coefficients with a low frequency, but the scheme is not robust to common geometric attacks. The scheme in ref. [36] hides the watermark information in the eigenvectors of the SVD, which can better extract the watermark from the geometric attacks; however, the scheme cannot resist cropping, addition, and deletion attacks. Neither the scheme in ref. [25] nor the scheme in ref. [36] can effectively extract the watermark information from the combined attack. The proposed scheme can extract the watermark information more completely in both the above attacks and their combined attacks. In comparison, it can be concluded that the proposed scheme has a stronger robustness.

4. Conclusions

In this study, a novel watermarking scheme is designed based on the combination with DWT and CSVD for vector geographic data. The proposed scheme puts forward the ratio of the DWT–CSVD coefficients as the watermark embedding domain, which is invariable under geometric attacks. In the watermark embedding stage, a new watermark sequence formed by fusing the watermark index and bit values of the copyright image is embedded to the fractional bit of the ratio value to hide the watermark information. The experimental results show that the proposed scheme has a good invisibility and can extract reliable watermark images under different degrees of translation, rotation, compression, cropping, vertex deletion, and a vertex addition attack. Subjected to a random combination of the above attacks, the proposed scheme can also extract the watermark information with a high probability, demonstrating an excellent robustness. The scheme shows to be credible for the copyright protection of the vector geographic data as tested.

Author Contributions

Chengyi Qu and Xu Xi conceived and designed the experiments; Chengyi Qu wrote the original draft; Jinglong Du and Xu Xi reviewed and edited the paper; Tong Wu con-tributed framing, ideas, context, and wordsmithing. All authors have read and agreed to the published version of the manuscript.

Funding

This work is jointly funded by the National Natural Science Foundation of China (grant no. 42101420) and Postgraduate Research & Practice Innovation Program of Jiangsu Province (grant no. SJCX22_1560).

Data Availability Statement

The data used in this study can be accessible by request to the corresponding author.

Acknowledgments

The authors gratefully acknowledge the anonymous referees for their insightful criticism, which definitely enhanced the work.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Qiu, Y.; Gu, H.; Sun, J. High-payload reversible watermarking scheme of vector maps. Multimed. Tools Appl. 2018, 77, 6385–6403. [Google Scholar] [CrossRef]
  2. Zhang, L.; Yan, H.; Qi, J.; Zhang, Y. A Blind Watermarking Algorithm for Copyright Protection of Vector Geospatial Data Under Controllable Errors Based on DFT. Geomat. Inf. Sci. Wuhan Univ. 2015, 40, 990–994. [Google Scholar] [CrossRef]
  3. Peng, F.; Lin, Z.-X.; Zhang, X.; Long, M. Reversible data hiding in encrypted 2D vector graphics based on reversible mapping model for real numbers. IEEE Trans. Inf. Forensics Secur. 2019, 14, 2400–2411. [Google Scholar] [CrossRef]
  4. Li, Y.; Zhang, L.; Wang, X.; Zhang, X.; Zhang, Q. A Novel Invariant Based Commutative Encryption and Watermarking Algorithm for Vector Maps. ISPRS Int. J. Geo-Inf. 2021, 10, 718. [Google Scholar] [CrossRef]
  5. Pham, G.N.; Ngo, S.T.; Bui, A.N.; Tran, D.V.; Lee, S.-H.; Kwon, K.-R. Vector map random encryption algorithm based on multi-scale simplification and Gaussian distribution. Appl. Sci. 2019, 9, 4889. [Google Scholar] [CrossRef] [Green Version]
  6. Zhou, Q.; Ren, N.; Zhu, C.; Zhu, A.X. Blind Digital Watermarking Algorithm against Projection Transformation for Vector Geographic Data. ISPRS Int. J. Geo-Inf. 2020, 9, 692. [Google Scholar] [CrossRef]
  7. Wang, X.; Yan, H.; Zhang, L. Vector map encryption algorithm based on double random position permutation strategy. ISPRS Int. J. Geo-Inf. 2021, 10, 311. [Google Scholar] [CrossRef]
  8. Wang, X.; Yan, H.; Zhang, L.; Zhang, X.; Li, P. An encryption algorithm for vector maps based on the Gaussian random and Haar transform. J. Spat. Sci. 2021. Advance online publication. [Google Scholar] [CrossRef]
  9. Wang, X.; Shao, C.; Xu, X.; Niu, X. Reversible data-hiding scheme for 2-D vector maps based on difference expansion. IEEE Trans. Inf. Forensics Secur. 2007, 2, 311–320. [Google Scholar] [CrossRef]
  10. Zhu, C. Research Progresses in Digital Watermarking and Encryption Control for Geographical Data. J. Geod. Geoinf. Sci. 2017, 46, 1609–1619. [Google Scholar]
  11. Ren, N.; Zhao, Y.; Zhu, C.; Zhou, Q.; Xu, D. Copyright Protection Based on Zero Watermarking and Blockchain for Vector Maps. ISPRS Int. J. Geo-Inf. 2021, 10, 294. [Google Scholar] [CrossRef]
  12. Cox, G.; De Jager, G. A survey of point pattern matching techniques and a new approach to point pattern recognition. In Proceedings of the 1992 South African Symposium on Communications and Signal Processing, Cape Town, South Africa, 11 September 1992; pp. 243–248. [Google Scholar]
  13. Wang, C.; Peng, Z.; Peng, Y.; Yu, L.; Wang, J.; Zhao, Q. Watermarking geographical data on spatial topological relations. Multimed. Tools Appl. 2012, 57, 67–89. [Google Scholar] [CrossRef]
  14. Lee, S.-H.; Kwon, K.-R. Vector watermarking scheme for GIS vector map management. Multimed. Tools Appl. 2013, 63, 757–790. [Google Scholar] [CrossRef]
  15. Wu, B.; Wang, Y.; Wang, C. High-fidelity watermarking model for building polygons. Sci. Surv. Mapp. 2019, 44, 185–190. [Google Scholar]
  16. Ren, N.; Tong, D.; Cui, H.; Zhu, C.; Zhou, Q. Congruence and geometric feature-based commutative encryption-watermarking method for vector maps. Comput. Geosci. 2022, 159, 105009. [Google Scholar] [CrossRef]
  17. Xi, X.; Zhang, X.; Sun, Y.; Jiang, X.; Xin, Q. Topology-Preserving and Geometric Feature-Correction Watermarking of Vector Maps. IEEE Access 2020, 8, 33428–33441. [Google Scholar] [CrossRef]
  18. Li, Y.-M.; Wei, D.; Zhang, L. Double-encrypted watermarking algorithm based on cosine transform and fractional Fourier transform in invariant wavelet domain. Inf. Sci. 2021, 551, 205–227. [Google Scholar] [CrossRef]
  19. Abubahia, A.; Cocea, M. Advancements in GIS map copyright protection schemes—A critical review. Multimed. Tools Appl. 2016, 76, 12205–12231. [Google Scholar] [CrossRef] [Green Version]
  20. Neyman, S.N.; Pradnyana, I.N.P.; Sitohang, B. A new copyright protection for vector map using FFT-based watermarking. Telkomnika 2014, 12, 367. [Google Scholar] [CrossRef] [Green Version]
  21. Dehe, X.; Qisheng, W. The study of watermarking algorithm for vector geospatial data based on the phase of DFT. In Proceedings of the 2010 IEEE International Conference on Wireless Communications, Networking and Information Security, Beijing, China, 25–27 June 2010; pp. 625–629. [Google Scholar]
  22. Zhang, L.; Yan, H.; Zhu, R.; Du, P. Combinational spatial and frequency domains watermarking for 2D vector maps. Multimed. Tools Appl. 2020, 79, 31375–31387. [Google Scholar] [CrossRef]
  23. Kitamura, I.; Kanai, S.; Kishinami, T. Digital watermarking method for vector map based on wavelet transform. Proc. Geogr. Inf. Syst. Assoc. 2000, 9, 417–421. [Google Scholar]
  24. Yang, C.; Zhu, C. Watermarking algorithm for vector geo-spatial data on wavelet transformation. J. Zhengzhou Inst. Surv. Mapp. 2007, 1, 37–39. [Google Scholar]
  25. Zope-Chaudhari, S.; Venkatachalam, P. Protecting geospatial data using digital watermarking. In Proceedings of the 2012 International Conference on Computer and Communication Engineering (ICCCE), Kuala Lumpur, Malaysia, 3–5 July 2012; pp. 594–598. [Google Scholar]
  26. Zope-Chaudhari, S.; Venkatachalam, P.; Buddhiraju, K.M. Copyright protection of vector data using vector watermark. In Proceedings of the 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Fort Worth, TX, USA, 23–28 July 2017; pp. 6110–6113. [Google Scholar]
  27. Liang, B.; Rong, J.; Wang, C. A vector maps watermarking algorithm based on dct domain. ISPRS Congr. 2010, 38, 1–4. [Google Scholar]
  28. Wang, C.; Zhang, L.; Liang, B.; Zheng, H.; Du, W.; Peng, Y. Watermarking vector maps based on minimum encasing rectangle. In Proceedings of the 2011 Fourth International Conference on Intelligent Computation Technology and Automation, Shenzhen, China, 28–29 March 2011; pp. 1243–1246. [Google Scholar]
  29. Wang, X.; Huang, D.J.; Zhang, Z.Y. A DCT-based blind watermarking algorithm for vector digital maps. In Advanced Materials Research; Trans Tech Publications Ltd.: Wollerau, Switzerland, 2011; pp. 1053–1058. [Google Scholar]
  30. Van Fleet, P. Discrete Wavelet Transformations: An Elementary Approach with Applications; Wiley: Hoboken, NJ, USA, 2011. [Google Scholar]
  31. Satish, A.; Prasad, E.V.; Tejasvi, R.; Swapna, P.; Vijayarajan, R. Image scrambling through two level Arnold transform. In Proceedings of the Alliance International Conference on Artificial Intelligence and Machine Learning (AICAAM), Bangalore, India, 26–27 April 2019. [Google Scholar]
  32. Arnold, V.I.; Avez, A. Ergodic Problems of Classical Mechanics; Benjamin: New York, NY, USA, 1968. [Google Scholar]
  33. Shang, Z.; Ren, H.; Zhang, J. A block location scrambling algorithm of digital image based on Arnold transformation. In Proceedings of the 2008 the 9th International Conference for Young Computer Scientists, Zhangjiajie, China, 18–21 November 2008; pp. 2942–2947. [Google Scholar]
  34. Douglas, D.H.; Peucker, T.K. Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartogr. Int. J. Geogr. Inf. Geovisualiz. 1973, 10, 112–122. [Google Scholar] [CrossRef] [Green Version]
  35. Kandi, H.; Mishra, D.; Gorthi, S.R.S. Exploring the learning capabilities of convolutional neural networks for robust image watermarking. Comput. Secur. 2017, 65, 247–268. [Google Scholar] [CrossRef]
  36. Gaata, M.T. Robust watermarking scheme for GIS vector maps. Ibn AL-Haitham J. Pure Appl. Sci. 2018, 31, 277–284. [Google Scholar] [CrossRef]
Figure 1. Sample of watermark image. (a) Watermark image; (b) watermark index; (c) watermark bit.
Figure 1. Sample of watermark image. (a) Watermark image; (b) watermark index; (c) watermark bit.
Ijgi 11 00583 g001
Figure 2. Watermark with fused watermark indexes and bits. (a) Fixed length index; (b) mapped bits; (c) final watermark values.
Figure 2. Watermark with fused watermark indexes and bits. (a) Fixed length index; (b) mapped bits; (c) final watermark values.
Ijgi 11 00583 g002
Figure 3. Watermarking embedding rule.
Figure 3. Watermarking embedding rule.
Ijgi 11 00583 g003
Figure 4. Flow chart of watermark embedding algorithm.
Figure 4. Flow chart of watermark embedding algorithm.
Ijgi 11 00583 g004
Figure 5. Flow chart of watermark extraction algorithm.
Figure 5. Flow chart of watermark extraction algorithm.
Ijgi 11 00583 g005
Figure 6. The original vector geographic data: (a) rivers; (b) roads; (c) boundary.
Figure 6. The original vector geographic data: (a) rivers; (b) roads; (c) boundary.
Ijgi 11 00583 g006
Figure 7. Watermark images. (a) Original watermark image; (b) scrambled watermark image.
Figure 7. Watermark images. (a) Original watermark image; (b) scrambled watermark image.
Ijgi 11 00583 g007
Figure 8. Overlay of original data and watermarked data. We choose rivers for instance.
Figure 8. Overlay of original data and watermarked data. We choose rivers for instance.
Ijgi 11 00583 g008
Figure 9. The results of simplification attacks.
Figure 9. The results of simplification attacks.
Ijgi 11 00583 g009
Figure 10. The results of vertex addition attacks.
Figure 10. The results of vertex addition attacks.
Ijgi 11 00583 g010
Figure 11. The results of deletion attacks.
Figure 11. The results of deletion attacks.
Ijgi 11 00583 g011
Table 1. Detail of the experimental data.
Table 1. Detail of the experimental data.
DatasetsTypesCompression RatiosFeature Point CountsObject CountsVertexes Counts
RoadsPolyline70.44%84,00258,294429,463
RiversPolyline70.12%7301270336,724
BoundaryPolygon72.14%24,14420,799135,185
Table 2. Error analysis for the experiment data with different embedding strength.
Table 2. Error analysis for the experiment data with different embedding strength.
DatasetsIndicatorsProposed Scheme (m)Scheme in Ref. [25] (m)Scheme in Ref. [36] (m)
1/41/51/61/70.031 × 10−6
RoadsMax.Error0.600.055.7 × 10−33.7 × 10−40.170.3
MSE6.1 × 10−56.3 × 10−76.6 × 10−95.8 × 10−110.010.09
RiversMax.Error1.180.089.1 × 10−34.8 × 10−40.182.81
MSE1.5 × 10−31.5 × 10−59.9 × 10−85.3 × 10−100.010.94
BoundaryMax.Error0.690.062.9 × 10−32.8 × 10−40.1781.99
MSE1.3 × 10−41.4 × 10−61.2 × 10−81.2 × 10−100.010.60
Table 3. The results of RST attacks.
Table 3. The results of RST attacks.
DatasetsRotation (°)Translation (m)Scaling
106015010100050000.50.951.052
RiversIjgi 11 00583 i001Ijgi 11 00583 i002Ijgi 11 00583 i003Ijgi 11 00583 i004Ijgi 11 00583 i005Ijgi 11 00583 i006Ijgi 11 00583 i007Ijgi 11 00583 i008Ijgi 11 00583 i009Ijgi 11 00583 i010
NC = 1NC = 1NC = 1NC = 1NC = 1NC = 1NC = 0.49NC = 0.91NC = 0.90NC = 0.56
RoadsIjgi 11 00583 i011Ijgi 11 00583 i012Ijgi 11 00583 i013Ijgi 11 00583 i014Ijgi 11 00583 i015Ijgi 11 00583 i016Ijgi 11 00583 i017Ijgi 11 00583 i018Ijgi 11 00583 i019Ijgi 11 00583 i020
NC = 1NC = 1NC = 1NC = 1NC = 1NC = 1NC = 0.44NC = 0.91NC = 0.92NC = 0.49
BoundaryIjgi 11 00583 i021Ijgi 11 00583 i022Ijgi 11 00583 i023Ijgi 11 00583 i024Ijgi 11 00583 i025Ijgi 11 00583 i026Ijgi 11 00583 i027Ijgi 11 00583 i028Ijgi 11 00583 i029Ijgi 11 00583 i030
NC = 1NC = 1NC = 1NC = 1NC = 1NC = 1NC = 0.51NC = 0.89NC = 0.92NC = 0.48
Table 4. The results of cropping attacks.
Table 4. The results of cropping attacks.
Clip RatiosDatasetsScheme in Ref. [25]Scheme in Ref. [36]Proposed Scheme
1/8riversIjgi 11 00583 i031Ijgi 11 00583 i032Ijgi 11 00583 i033
NC = 0.52NC = 0.54NC = 1
roadsIjgi 11 00583 i034Ijgi 11 00583 i035Ijgi 11 00583 i036
NC = 1NC = 1NC = 1
boundaryIjgi 11 00583 i037Ijgi 11 00583 i038Ijgi 11 00583 i039
NC = 1NC = 1NC = 1
1/4riversIjgi 11 00583 i040Ijgi 11 00583 i041Ijgi 11 00583 i042
NC = 0.54NC = 0.50NC = 1
roadsIjgi 11 00583 i043Ijgi 11 00583 i044Ijgi 11 00583 i045
NC = 1NC = 1NC = 1
boundaryIjgi 11 00583 i046Ijgi 11 00583 i047Ijgi 11 00583 i048
NC = 1NC = 1NC = 1
1/2riversIjgi 11 00583 i049Ijgi 11 00583 i050Ijgi 11 00583 i051
NC = 0.48NC = 0.46NC = 1
roadsIjgi 11 00583 i052Ijgi 11 00583 i053Ijgi 11 00583 i054
NC = 0.44NC = 0.48NC = 1
boundaryIjgi 11 00583 i055Ijgi 11 00583 i056Ijgi 11 00583 i057
NC = 0.49NC = 0.5NC = 1
3/4riversIjgi 11 00583 i058Ijgi 11 00583 i059Ijgi 11 00583 i060
NC = 0.52NC = 0.44NC = 0.91
roadsIjgi 11 00583 i061Ijgi 11 00583 i062Ijgi 11 00583 i063
NC = 0.48NC = 0.51NC = 1
boundaryIjgi 11 00583 i064Ijgi 11 00583 i065Ijgi 11 00583 i066
NC = 0.55NC = 0.52NC = 1
Table 5. The results of combination attacks.
Table 5. The results of combination attacks.
DatasetsScheme in Ref. [25]Scheme in Ref. [36]Proposed
d1, d2d2, d3d1, d3d1, d2, d3d1, d2d2, d3d1, d3d1, d2, d3d1, d2d2, d3d1, d3d1, d2, d3
Rivers0.4220.4960.4570.4100.5390.5550.4570.4921111
Roads0.5470.4570.4490.5080.6320.5900.4230.4451111
Boundary0.4840.4960.4490.5000.6710.6130.5740.5081111
Table 6. Comparison between the proposed scheme and prior schemes.
Table 6. Comparison between the proposed scheme and prior schemes.
Attack TypesDatasetsProposedScheme in Ref. [25]Scheme in Ref. [36]
RotationRivers
Roads
Boundary
TranslationRivers
Roads
Boundary
ScalingRivers
Roads
Boundary
SimplificationRivers
Roads
Boundary
ClippingRivers
Roads
Boundary
DeletingRivers
Roads
Boundary
AddingRivers
Roads
Boundary
CombinationRivers
Roads
Boundary
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Qu, C.; Xi, X.; Du, J.; Wu, T. Robust Watermarking Scheme for Vector Geographic Data Based on the Ratio Invariance of DWT–CSVD Coefficients. ISPRS Int. J. Geo-Inf. 2022, 11, 583. https://doi.org/10.3390/ijgi11120583

AMA Style

Qu C, Xi X, Du J, Wu T. Robust Watermarking Scheme for Vector Geographic Data Based on the Ratio Invariance of DWT–CSVD Coefficients. ISPRS International Journal of Geo-Information. 2022; 11(12):583. https://doi.org/10.3390/ijgi11120583

Chicago/Turabian Style

Qu, Chengyi, Xu Xi, Jinglong Du, and Tong Wu. 2022. "Robust Watermarking Scheme for Vector Geographic Data Based on the Ratio Invariance of DWT–CSVD Coefficients" ISPRS International Journal of Geo-Information 11, no. 12: 583. https://doi.org/10.3390/ijgi11120583

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop