Assessing Robustness of Morphological Characteristics of Arbitrary Grayscale Images

Featured Application

Morphological analysis of various categories of grayscale arbitrary images; local and global feature extraction in grayscale arbitrary images recognition.

Abstract

In our previous work, we introduced an empirical model (EM) of arbitrary binary images and three morphological characteristics: disorder of layer structure (DStr), disorder of layer size (DSize), and pattern complexity (PCom). The basic concept of the EM is that forms of lines play no role as a morphological factor in any narrow area of an arbitrary binary image; instead, the basic factor is the type of line connectivity, i.e., isotropic/anisotropic connections. The goal of the present work is to justify the possibility of making the EM applicable for the processing of grayscale arbitrary images. One of the possible ways to reach this goal is to assess the influence of image binarization on the robustness of DStr and DSize. Images that exhibit high and low edge gradient are used for this experimental study. The robustness of DStr and DSize against the binarization procedure is described in absolute (deviation from average) and relative (Pearson’s coefficient correlation) terms. Images with low edge gradient are converted into binary contour maps by applying the watershed algorithm, and DStr and DSize are then calculated for these maps. The robustness of DStr and DSize were assessed against the image threshold for images with high edge gradient and against the grid size of contour maps and Gaussian blur smoothing for images with low edge gradient. Experiments with grayscale arbitrary patterns, such as the surface of Earth and Mars, tidal sand ripples, turbulent flow, a melanoma, and cloud images, are presented to illustrate the spectrum of problems that may be possible to solve by applying the EM. The majority of our experiments show a high level of robustness for DStr and DSize.

1. Introduction

In our previous work [1,2,3], we have justified that though the visual macro differences between patterns are significantly distinct, they nevertheless share a common feature: they have layers. A majority of arbitrary patterns could be described as being comprised of a very short layer system. Even when these short layers are detectable by the naked eye, they are not a visually dominant feature of an image and thus are not used as building blocks to assess that image’s morphology.

Studying the growth rates of fish scales [4,5] and lamellar bones [1,6,7] brought to our attention the fact that short layers could serve as a structural building block of arbitrary patterns. This resulted in the development of an empirical model (EM) of arbitrary binary images and three morphological characteristics: disorder of layer structure (DStr), disorder of layer size (DSize), and pattern complexity (PCom). The EM is comprised of an N-partite graph G(N), a Boolean function (BF), and table T_M,N, where G(N) and BF describe the structure of an arbitrary binary image and T_M,N describes the image size along transects R₁, …, R_j, …, R_N; EM = {G(N), BF, T_M,N}. A property of EM is that isotropic and anisotropic edges/vertices (i.e., isotropic/anisotropic connections) of G(N) serve as building blocks for the calculation of DStr. This means that forms of lines play no role as a morphological factor for very narrow areas of an image. Instead, types of line connectivity, i.e., isotropic/anisotropic connections are responsible for the description of an image’s structure.

This statement is the basic concept of our approach to the morphological analysis of arbitrary patterns. In addition, DStr, DSize, and PCom are dependent on an image’s orientation. Thus, DStr, DSize, and PCom represent the morphological characteristics of an image as a function of its rotation.

The focus of our previous work [1,2,3,5,6,7] was binary patterns and their analysis based on the EM. The goal of the present work is to justify the possibility of making the EM applicable to the processing of grayscale arbitrary images. One of the possible ways to reach this goal is to assess the influence of image binarization on the robustness of DStr and DSize.

Any grayscale image formed in nature and beyond, irrespective of its size and structure, is called a grayscale arbitrary image (GAI). Both images that exhibit high and low edge gradients are used for this experimental study. The robustness of the morphological characteristics of DStr and DSize against GAI binarization is described in both absolute (deviation from the average) and relative (Pearson’s coefficient [8] linear correlation) terms. The image global threshold is used to convert a GAI with a high edge gradient into binary mode and a watershed algorithm [9,10] is applied for the binarization of a GAI with a low edge gradient. The output of the watershed algorithm is a binary contour map. Various grid sizes of contour maps and the Gaussian blur radius are used to generate sets of contour maps in order to estimate the robustness of DStr and DSize against GAI binarization.

The paper is organized as follows. The proposed method is detailed in Section 2. Experimental results are provided in Section 3. A discussion is provided in Section 4. Concluding remarks are presented in Section 5.

2. Materials and Methods

2.1. Grayscale Images for Experiments

The image global threshold [11,12] and a watershed algorithm [10] are two distinct binarization methods that are used to assess the robustness of DStr and DSize. A global threshold scheme is applied for a GAI with a relatively high edge gradient (Figure 1). The watershed algorithm is used for the binarization of images with a very low edge gradient (Figure 2). This results in the presentation image under study, which takes the form of a binary contour map and can be used to calculate DStr and DSize [3].

2.2. Disorder of Layer Structure (DStr) and Size (DSize)

Let us first review the meaning of the DStr and DSize parameters and the basic elements of their calculation. The parameters of DStr and DSize are introduced to describe the morphological features of arbitrary patterns, with respect to the reference image, which are made up of layers that have an isotropic size and structure. Thus:

• The DStr serves as a measure of deviation of an arbitrary pattern that has an anisotropic structure from a pattern that has an isotropic structure;
• The DSize serves as a measure of deviation of an arbitrary pattern of an anisotropic size from a pattern of an isotropic size.

An isotropic structure implies that lines that comprise an image have no breaks and confluences. An isotropic size suggests that the distance between lines remains constant along R₁, …, R_N for any angles of an image rotation. Thus, an image with an isotropic size and structure consists of straight parallel lines that are at a constant distance from any nearby lines.

The notions of isotropic and anisotropic structure and size come from the formalization of growth layers (i.e., the growth increments) of living systems such as fish scales [4,5] and lamellar bone of humans and animals [2,6,7]. If the growth rate remains constant over a period of time, then the thickness of the growth increment remains constant across a 2-D area of growth, and growth increments contain no breaks and confluences in a layer’s structure. Thus, a constant growth rate results in the isotropic size and structure of growth layers. If the growth rate varies, then the thickness of growth lines also varies, and the structure usually exhibits breaks and confluences. Consequently, the values of DStr and DSize signal a deviation of the growth rate from the constant speed of layer formation. Overall, the EM provides tools to quantify the anisotropic growth rate across 2-D growth increments [1,5].

It has been shown [2,3] that breaks and confluences are inherent in various images formed in nature and beyond. Hence, it is possible to use the EM to create a morphological description of these images. Moreover, the breaks and confluences within a layer are the building blocks of any (i.e., arbitrary) binary patterns [3]. This means that the EM can be used to create a morphological description of an arbitrary binary image.

Let us consider the basic steps of calculating DStr and DSize. First, DStr and DSize are functions of GAI alignment. This is contrary to commonly used morphological characteristics of images, such as circularity, curvature, perimeter, convexity, and compactness. Thus, when the image under study rotates, DStr and DSize are calculated for each image position.

Second, the concept of isotropic and anisotropic edges/vertices (i.e., isotropic/anisotropic connectivity) of G(N) was introduced [2,3] in order to calculate DStr; the measure of DStr is the ratio between anisotropic vertices and all G(N) vertices. Third, the set of transects R₁, …, R_j, …, R_N is what is used to calculate DStr and DSize. Each transect is a straight line, and the distance between R_j−1 and R_j remains constant across R₁, …, R_N. The value of both DStr and DSize depends on the number of transects (i.e., sampling density) used in calculating them. The general principle used when choosing the number of transects is based on the fact that we do not know a priori how many transects will best describe the particular layered pattern. In these circumstances, our method is to examine many different versions of the transect set. The minimal number of transects is equal to two and the maximal number is defined by the minimal distance between two nearby transects, which cannot be less than one pixel. We calculated DStr and DSize for a normalized number of transects N_i(normalized) = N_i/Nmax in order to present the results of calculating for the scale [0, 1]. The function YStr = FStr(x), i.e., YStr = FStr (number of transect versions) was plotted, which describes dynamic changes in DStr when the number of transects tends to the maximum possible number. If all possible versions of the number of transects are used to calculate DStr, then YStr contains as many structural details as possible for the pattern being studied. The area bounded by YStr = FStr(x) and the axis x = 0 is the measure of DStr. Thus,

$$\mathrm{DStr}={{\displaystyle \int }}_0^1\mathrm{FStr}\left(\mathrm{x}\right)\mathrm{dx}$$

In a similar manner, the function YSize = FSize(x) is calculated, and the area bounded by YSize = FSize(x) and the axis x = 0 is the measure of DSize. Hence,

$$\mathrm{DSize}={{\displaystyle \int }}_0^1\mathrm{FSize}\left(\mathrm{x}\right)\mathrm{dx}$$

The source code of the program used to calculate these morphological parameters, the ReadMe file, and examples are available at ZENODO.

2.3. Absolute Robustness

Robustness (RB) in general terms can be defined as “… the capacity of a system to maintain a function in the face of perturbation” ([18] p. 169). In the present work, RB is the measure of the degree of variability for DStr and DSize as a function of GAI binarization. If RB = 1, then DStr and DSize remain unchanged. This suggests that the highest degree of robustness, RB = 0, indicates the lowest robustness level for DStr and DSize. Let us now consider the procedures of RB quantification in these scales.

The set of global thresholds denoted by tr₁, …., tr_k, …, tr_p allow us to convert a GAI into a set of binary images, BIm₁, …, BIm_k, …, BIm_p. The set of angles φ₁, …, φ_r, …, φ_q is used to calculate the morphological characteristics of a binary image as the function of its rotation. DStr(tr_k,φ_r), is the result of calculating DSt for tr_k and φ_r. The parameter DStr(tr_k, φ_r) is averaged over tr_k, k = 1,p for the angle of rotation φ_r, and is denoted by AvDStr(${\mathsf{\phi }}_{\mathrm{r}}$).

$$\mathrm{AvDStr}\left({\mathsf{\phi }}_{\mathrm{r}}\right)=\frac{1}{\mathrm{p}}{\displaystyle \sum }_{\mathrm{k}=1}^{\mathrm{p}}\mathrm{DStr}\left({\mathrm{tr}}_{\mathrm{k}},{\mathsf{\phi }}_{\mathrm{r}}\right) .$$

The absolute deviation of DStr from the arithmetical mean is denoted by (ΔDStr(tr_k,φ_r) for tr_k and φ_r and is equal to:

ΔDStr(tr_k,φ_r) = |DStr(tr_k,φ_r) − AvDStr(φ_r)|.

The average deviation of ΔDStr from the arithmetic mean for φ_r is equal to:

$$\mathrm{Av}\left[\mathsf{\Delta }\mathrm{DStr}\left({\mathsf{\phi }}_{\mathrm{r}}\right)\right]=\frac{1}{\mathrm{p}}{\displaystyle \sum }_{\mathrm{k}=1}^{\mathrm{p}}\mathsf{\Delta }\mathrm{DStr}\left({\mathrm{tr}}_{\mathrm{k}},{\mathsf{\phi }}_{\mathrm{r}}\right).$$

Finally, the robustness of DStr with respect to BIm₁, BIm₂, and BIm₃ on an absolute scale is equal to:

$$\mathrm{absRB}\left(\mathrm{DStr}\right)=1-\frac{1}{\mathrm{q}}{\displaystyle \sum }_{\mathrm{r}=1}^{\mathrm{q}}\mathrm{Av}\left[\mathsf{\Delta }\mathrm{DStr}\left({\mathsf{\phi }}_{\mathrm{r}}\right)\right]$$

(1)

The algorithm for calculating absRB(DSize) is identical to that for calculating absRB(DStr).

2.4. Relative Robustness

The intraclass correlation coefficient (ICC) is used as the measure of reliability [19] and robustness [20] of the morphological characteristics of images. The ICC has been applied in the present work as the measure of DStr and DSize’s relative robustness. Let us denote ICC(tr_i, tr_j) as the correlation coefficient between DStr of binary images BIm_i and BIm_j. So, the relative robustness of DStr for two images i and j is equal to:

relRB[DStr(BIm_i, BIm_j)] = ICC(BIm_i, BIm_j)

If the number of binary images equals p, then:

$$\mathrm{relRB}[\mathrm{DStr}({\mathrm{BIm}}_1,\dots ,{\mathrm{BIm}}_{\mathrm{p}})]=\frac{1}{\mathrm{M}}{\displaystyle \sum }_{\begin{array}{c}i,j=1\\ i\ne j\end{array}}^{\mathrm{M}}\left|\mathrm{ICC}\left({\mathrm{BIm}}_{\mathrm{i}},{\mathrm{BIm}}_{\mathrm{j}}\right)\right|,\mathrm{where}\mathrm{M}=\frac{{\mathrm{p}}^2-\mathrm{p}}{2}$$

(2)

The algorithm for calculating reRB(DSize) is identical to that for calculating reRB(DStr).

2.5. Example

Let us consider examples for calculating the absolute and relative robustness of DStr based on images and data presented in Figure 3. The result of the image binarization with thresholds tr₁ = 20, tr₂ = 110, and tr₃ = 200 (Figure 3a) are denoted by BIm₁, BIm₂, and BIm₃. All data used to calculate the absolute and relative robustness of DStr for BIm₁, BIm₂, and BIm₃ are presented in the table shown in Figure 3b.

Consider the sequence of steps in calculating the absolute robustness. Angles of rotation φ_r are responsible for the alignment of BIm₁, BIm₂, and BIm₃. Row r of the table is comprised of data for alignment φ_r which is used for the calculation of absRB(DStr(φ_r)). The value of the row 1, column 2 cell is equal to DStr(φ = 0°) = 0.866 for BIm₁. Consequently, the row 1, column 3 cell is equal to DStr(φ = 0°) = 0.774 for BIm₂ and the row 1, column 4 cell is equal to DStr(φ = 0°) = 0.666 for BIm₃. The following steps take place:

Step 1. Calculate the arithmetical mean of DStr(φ = 0°) of BIm₁, Bim_2, and BIm₃. The result of the calculation is shown in the row 1, column 5 cell.

Step 2. Calculate the deviation of DStr(φ = 0°) from the arithmetical mean, which is denoted by ΔDStr(φ = 0°). The result of the calculations is shown in the following cells: row 1, column 6; row 1, column 7; and row 1, column 8 for BIm1, BIm2, and BIm3, respectively.

Step 3. Deviations of ΔDStr(φ = 0°) for BIm₁, BIm₂, and BIm₃ are averaged and denoted by Av[ΔDStr(φ = 0°)]; row 1, column 9 cell = Av[ΔDStr(φ = 0°)] for BIm₁, BIm₂, and BIm₃.

Step 4. Steps 1–3 are repeated for φ = 10°, φ = 20°, …, φ = 180°. Column 9 comprises the results of calculating Av[ΔDStr(φ_r)].

Step 5. Equation (1) was used to calculate the absolute robustness of DStr with respect to binarization thresholds tr₁ = 20, tr₂ = 110, and tr₃ = 200; absRB[DStr(Bim₁, BIm₂, BIm₃)] = 0.938.

Figure 3c shows the high level of correspondence (i.e., ICC values) and the consequently high level of relative robustness of DStr with respect to tr₁, tr₂, and tr₃.

3. Results

This section presents the results of the experiments to estimate for the robustness of DStr and DSize with respect to grayscale image binarization. High and low gradient grayscale images are used to assess the absolute and relative robustness of DStr and DSize. Data used in this article are available online at Supplementary Directory.

3.1. Robustness of DStr and DSize of High Edge Gradient Images

Thirteen grayscale images (Figure 1) are used as test samples to explore both the absolute and relative robustness of DStr and DSize. The experiment was organized as follows. The global threshold was used to convert the grayscale image into a binary mode. Eight thresholds, tr₁ = 20, tr₂ = 50, tr₃ = 80, …, tr₈ = 230, were used to assess the robustness of DStr and DSize. All images were rotated for φ = 0°, φ = 10°, …, φ = 180°. The parameters of DStr and DSize were calculated for the position of each image rotation. Equations (1) and (2) were used to calculate the absolute and relative robustness of DStr and DSize. All images, except the DSize of tidal ripples, have absolute and relative robustness of more than 0.85 (Figure 4).

3.2. From a Low Edge Gradient Image to a Contour Map

Many images formed in nature and beyond have fuzzy edges that are difficult or even impossible to differentiate visually from one another (Figure 2). One of the potential ways to present these edges in binary mode is to use the concept of watershed segmentation [9,10,21]. In this case, an image is regarded as a topographical map, where the brightness of each pixel represents its height. In a 2-D plane, a topographic map exhibits contour lines of equal brightness. The procedure for converting a grayscale image to a contour map consists of several basic steps: (a) converting an image into an array Arr(X,Y,Z) in the comma-separated values format; (b) calculating a grid based on Arr(X,Y,Z); and (c) using the grid to construct a contour map (Figure 5, blur radius equal to zero).

The first step in constructing the contour map is to calculate Arr(X,Y,Z). This array is comprised of three columns, X, Y, and Z. The first two columns are the x and y coordinates of a pixel and the third column is its brightness. Various tools within geographical information systems have been developed in order to automate the process of plotting contour maps. The commercially available program, SURFER [22], is used for this purpose in the present work.

3.3. Robustness of DStr and DSize of Low Edge Gradient Images

The robustness of DStr and DSize for contour maps is assessed against grid size and Gaussian blur radius. The grid size is used due to the fact that it is one of the factors which substantially influences the shape of map contours [23]. The Gaussian blur is applied due to its ability to reduce the many negligible details in a grayscale image (Figure 5).

The experiment was organized as follows. The image brightness was normalized and presented using the relative scale [0, 1]. Wide ranges for grid size and Gaussian blur radius (

Table 1. Binary contour maps. Setting for assessing the robustness of DStr and DSize against grid size and Gaussian blur radius.

	Grids Size (Relative Scale)	Gaussian Blur Radius
Aircraft	1, 0.5, 0.25	0, 3, 6, 9, 12, 15
Melanoma	1, 0.5, 0.25	0, 2, 4, 6, 8
Cloud	1, 0.5, 0.25	0, 2, 4, 6, 8, 10, 12, 14

) were used to access the robustness of DStr and DSize for Boeing engine condensation trails, a melanoma, and a cloud. The identical program settings were used to calculate DStr and DSize for all experiments: the minimal number of transects is equal to three, the minimal number of pixels between transects is equal to three, and the differences in the number of transects for two sets of transects, S_j and S_j+1, is equal to six (ReadMe file, ZENODO).

Equations (1) and (2) were used to calculate RB(DStr) and RB(DSize) (Figure 6). The absolute and relative robustness of DStr and DSize against a different grid size varies from 0.86 to 0.98 (Figure 6a) for all images. This was also true for different Gaussian blur factors, except for the relRB(DSize) of the melanoma and cloud (Figure 6b).

4. Discussion

The goal of our study was to estimate the degree of robustness of DStr and DSize against image binarization without focusing on the specific problem statement of morphological image analysis. It has transpired that the results of the experiments we performed do not guarantee that applying the EM to the morphological analysis of any grayscale images will always result in a high level of robustness for DStr and DSize. Thus, when the EM is applied, the robustness of DStr and DSize has to be assessed. Moreover, it was demonstrated that the watershed algorithm and Gaussian blur smoothing are tools that can be used to create the binarization of a grayscale image with low edge gradient and make it possible to apply the EM for such image analysis.

Areas of application of EM and its limitations with respect to analysis of binary images are described in our previous work [1,2,3], which is also applicable for GAI in addition to two new limitations. It is also necessary to point out that the robustness of DStr and DSize depends on various program settings, which have not been included in our experiments. This is a limitation of the present work. Another limitation is that the contours which make up the contour map are manually chosen to quantify DStr and DSize, since some contours have noise and negligible image details. Gaussian blur is one of the possible tools that allows us to reduce noise in a contour map.

Overall, we consider the results of the present work as the first step toward justifying the applicability of the EM to the processing of a GAI. Different known and unknown assumptions and limitations still have to be explored with respect to various image categories and the goals of each category’s morphological analysis.

5. Conclusions

This study shows that the EM is potentially applicable for the morphological analysis of grayscale arbitrary images. The robustness of DStr and DSize must be assessed against different parameters, such as the program setting, grid size, and Gaussian blur radius, in order to improve the consistency and reproductivity of the morphological analysis. Furthermore, the EM complements methods of the morphological analysis of images based on pixel manipulation.

6. Patents

Smolyar, I.V. System and Method for Quantification of Size and Anisotropic Structure of Layered Patterns. U.S. Patent 8,755,578, 17 June 2014.

Smolyar, I.V. System and method for encryption/decryption of 2-D and 3-D arbitrary images. U.S. Patent 10,819,881, 27 October 2020.