A Small UAV Based Multi-Temporal Image Registration for Dynamic Agricultural Terrace Monitoring

Abstract

Terraces are the major land-use type of agriculture and support the main agricultural production in southeast and southwest China. However, due to smallholder farming, complex terrains, natural disasters and illegal land occupations, a light-weight and low cost dynamic monitoring of agricultural terraces has become a serious concern for smallholder production systems in the above area. In this work, we propose a small unmanned aerial vehicle (UAV) based multi-temporal image registration method that plays an important role in transforming multi-temporal images into one coordinate system and determines the effectiveness of the subsequent change detection for dynamic agricultural terrace monitoring. The proposed method consists of four steps: (i) guided image filtering based agricultural terrace image preprocessing, (ii) texture and geometric structure features extraction and combination, (iii) multi-feature guided point set registration, and (iv) feature points based image registration. We evaluated the performance of the proposed method by 20 pairs of aerial images captured from Longji and Yunhe terraces, China using a small UAV (the DJI Phantom 4 Pro), and also compared against four state-of-the-art methods where our method shows the best alignments in most cases.

1. Introduction

China has more than 70% of its land made up of mountains and hills, while the main agricultural terraces are located in southwest and southeast China. Therefore, agricultural terraces have become the most important agricultural land-use type and support the main agricultural production in these areas. Furthermore, it also plays an important role in reducing flood runoff, changing terrain slope and maintaining water, soil and conservation fertilizer.

However, due to Chinese agricultural policy and regional characteristics, most agricultural terraces in southwest and southeast China are farmed by smallholders, and have small sizes, scattered distributions, complex terrains and other characteristics. Such issues have increased difficulty in land use management and planting management of local governments, as well as regular cropping and planting area monitoring of smallholder farmers. Meanwhile, illegal land occupations, natural disasters, water and wind erosion are also causing a drastic decrease in agricultural terraces, and these further aggravate the soil erosion and endanger the smallholder production. Consequently, a light-weight and low cost dynamic monitoring of agricultural terraces has become a serious concern for smallholder production systems in China.

Agricultural terrace monitoring includes two major aspects: (i) crop monitoring and (ii) planting area monitoring. In this work, we mainly focus on the planting area monitoring, which generally depends on aerial remote sensing and image processing techniques. In parallel to the technological advances of aerial remote sensing, achievements in image processing have promoted the development of sophisticated algorithms using aerial images, such as the landform classification, which is one of the effective ways for dynamic monitoring of land-use changes. Traditional landform classification methods can be divided into non-supervised classification methods and supervised classification methods [1]. Basically, non-supervised methods consist of manual classification methods and automated classification methods [2]. Manual classification methods are relatively time-consuming and the results depend on subjective decisions of the interpreter and are, therefore, neither transparent nor reproducible [3,4]. Automated classification methods [5,6] make use of the unsupervised nature and automation of the change analysis process. However, they are unfavored by difficulties in identifying and labeling change trajectories [7], and the lack of information on calibration of the ground [7]. Artificial neural network (ANN) constitutes a key component of supervised classification methods [8,9]. It is a non-parametric method that is capable of estimating the properties of data based on the training samples. However, ANN suffers from the long training time, the sensitivity to the amount of training data used, as well as the applicability of ANN functions in the common image processing softwares [7].

Subsequent to the landform classification, classified images can be adopted to monitor land use changes. Over the last few years, in order to analyze agricultural terraces, different and new technologies have been used. Satellite data of high spatial resolution and advanced image processing techniques, have opened up a new insight for mapping landscape features, such as terraces. This has given the opportunity for quantitative assessment of farming practices as an indicator in water pollution risk assessment [10], soil erosion risk assessment [11] and landslide boundary monitoring [11,12,13,14]. In addition, airborne LiDAR (light detection and ranging), which has been developed to collect and subsequently characterize vertically distributed attributes [15], and the derived digital elevation model (DEM) [16,17,18,19,20] or digital terrain model (DTM) [12,14] are becoming standard practices in spatial related areas. Recently, the use of unmanned aerial vehicle (UAV) for civil applications has emerged as an attractive and flexible option for the monitoring of various aspects of agriculture and environment [21]. For example, Diaz-Varela et al. [21] proposed an automatic identification of agricultural terraces through object-oriented analysis of high resolution digital surface models and multi-spectral images obtained from UAVs. Deffontaines et al. [22] monitored the active inter-seismic shallow deformation of the Pingting terraces by using UAV high resolution topographic data combined with InSAR time series. Yang et al. [23] proposed a multi-viewpoint remote sensing image registration method that provided an accurate mapping between different viewpoint images for ground change detections.

Compared with satellite and other aerial remote sensing, using small UAVs for agricultural terrace monitoring has a strong mobility, high efficiency, low cost and other advantages. However, the following issues still exist: (i) Due to the payload capacity, small UAVs usually can only carry a light-weight visible light camera, such as CCD or CMOS cameras that limit available image information while increasing difficulty in monitoring algorithms compared with using multi-spectral imaging. (ii) When collecting multi-temporal images for the same location (e.g., a planting area in terraces), the imaging perspective of small UAVs is often easily affected by wind speed/direction, complex terrain, battery capacity (e.g., flying distance), aircraft posture (pitch, roll, yaw), flying height and other human factors. These factors cause the captured scenes (i.e., the same location in a pair of multi-temporal images) to not be in the same coordinate system, while image geometric distortions, low image overlapping, brightness changes and color changes may also be produced in such multi-temporal images.

The above issues have led to the fact that multi-temporal images of the same scene captured by small UAVs may not be directly used to detect changes for dynamic agricultural terrace monitoring, and a reliable multi-temporal image registration, which can transform the images into one coordinate system, is necessary in order to be able to subsequently compare or integrate the data obtained from the multi-temporal images.

In this work, we focus on planting areas of agricultural terraces, and present a small UAV based multi-temporal image registration method for dynamic agricultural terrace monitoring. The major contributions of the proposed method includes: (i) the guided image filtering for agricultural terrace image preprocessing is first designed to enhance terrace ridges in multi-temporal images, (ii) the multi-feature descriptor is then applied to combine the texture feature and the geometric structure feature of terrace images for improving the description of feature points and rejecting outliers, (iii) the multi-feature guided model provides an accurate guiding for feature point set registration, and (iv) the feature points based image registration finally registers the terrace images accurately.

2. Methodology

The proposed small UAV based multi-temporal image registration method has four major sequential processes: (i) image preprocessing; (ii) feature extraction and combination; (iii) feature point set registration; and (iv) image registration. In this section, we first introduce the proposed method followed by analyzing the computational complexity and discussing the implementation details.

2.1. Guided Image Filtering Based Agricultural Terrace Image Preprocessing

Given a gray agricultural terrace image $\mathbf{I}$ with $x\times y$ pixels and intensity $\mathbf{I}$ = 0.3R + 0.59G + 0.11B from the colorized image captured by a small UAV. We first define a preprocessing method to strengthen the identifiability of terrace ridges in multi-temporal images, and then extract salient features of terrace images along the enhanced ridges in the second step. The main reason is that accuracy and validity of an image registration are not only controlled by the performance of feature point set registration, but also determined by the number and the distribution density of feature points, because of the image transformation constructed by abundant feature points.

In this work, the preprocessing method improves the contrast ratio between terraces and their ridges using the Guided Image Filtering (GIF) [24]. In order to extract large and quality feature points from terrace ridges, we first adopt the GIF, which has the edge-preserving smoothing and the gradient preserving, to preprocess input images. The GIF employs a guidance image to construct a spatially variant kernel and is also related to the matting Laplacian matrix [25].

Firstly, a linear translation-variant guided filtering process in a square window ${\pi }_k$ centered at a pixel k is defined by:

$${\mathsf{\Lambda }}_i={\alpha }_k{\mathbf{I}}_i+{\beta }_k,\forall i\in {\pi }_k,$$

(1)

where i is a pixel index, $\mathsf{\Lambda }$ is the output image, ${\alpha }_k=\frac{\frac{1}{\left|\pi \right|}{\sum }_{i\in {\pi }_k}{\mathbf{I}}_i{g}_i-{\gamma }_k{\overline{g}}_k}{{\delta }_k^2+\epsilon }$ and ${\beta }_k={\overline{g}}_k-{\alpha }_k{\gamma }_k$ are two parameters of the minimal cost function $\underset{{\alpha }_k,{\beta }_k}{argmin}{\sum }_{i\in {\pi }_k}({({\alpha }_k{\mathbf{I}}_i+{\beta }_k-g_i)}^2+\epsilon {\alpha }_k^2)$. Here, g is the input image that is identical to the guidance image $\mathbf{I}$, ${\gamma }_k$ and ${\delta }_k^2$ are the mean and the variance of $\mathbf{I}$ in ${\pi }_k$, $\left|\pi \right|$ is the number of pixels in ${\pi }_k$, $\epsilon$ is a regularization parameter preventing ${\alpha }_k$ from being too large, and ${\overline{g}}_k=\frac{1}{\left|\pi \right|}{\sum }_{i\in {\pi }_k}{g}_i$ is the mean of g in ${\pi }_k$.

Secondly, we apply the linear model to all local windows in the entire image:

$$\begin{array}{cc}\hfill {\mathsf{\Lambda }}_i& =\frac{1}{\left|\pi \right|}\sum _{k:i\in {\pi }_k}({\alpha }_k{\mathbf{I}}_i+{\beta }_k)\hfill \\ & ={\overline{\alpha }}_i{\mathbf{I}}_i+{\overline{\beta }}_i,\hfill \end{array}$$

(2)

where ${\overline{\alpha }}_i=\frac{1}{\left|\pi \right|}{\sum }_{k\in {\pi }_k}{\alpha }_k$ and ${\overline{\beta }}_i=\frac{1}{\left|\pi \right|}{\sum }_{k\in {\pi }_k}{\beta }_k$. An example of agricultural terrace image enhancement ($250\times 150$ pixels) using GIF is given in Figure 1.

An agricultural terrace is constructed by cropland and terrace ridges. Ridges can be considered as a salient feature that provides the features of geometric contours and surface textures for feature based terrace image registration. However, the flat color of terrace ridge is similar with croplands and detecting cropland and terrace ridges becomes difficult when crops are growing in the early stage. Thus, the extracted feature points along ridges are more helpful than points distributed on the cropland. Mathematically, the exponential function can change the density of the data distribution; therefore, we expand the gray value distribution of the common terrace via a natural exponential function formed as:

$${\mathbf{I}}_i^{new}=\left\{\begin{array}{cc}1,\hfill & \hfill \mathrm{if}exp(-{\mathsf{\Lambda }}_i)\ge 0.7,\\ exp(-{\mathsf{\Lambda }}_i),\hfill & \hfill \mathrm{otherwise}.\end{array}\right..$$

(3)

Finally, we can obtain the preprocessed gray agricultural terrace image ${\mathbf{I}}^{new}$ with a high contrast ratio, smoothing edges, and prominent ridges. The preprocessing step gives an opportunity to extract quality features from these preprocessed terrace images.

2.2. Features Extraction and Combination

Feature points are selected by using the good-feature-to-track criterion [26], similar to the Harris detector, based on the second moment matrix [27]. The selection specifically maximizes the quality of tracking, and is therefore optimized by construction, as opposed to more ad hoc measures of texturedness. The selected feature point set $\mathbf{P}={\left\{{\mathbf{p}}_t\right\}}_{t=1}^T$ belongs with the geometric coordinate of a input agricultural terrace image pixel, where ${\left\{{\mathbf{p}}_t\right\}}_{t=1}^T\in {\mathbb{Z}}^{+}$. There is an example of feature points extraction from a preprocessed agricultural terrace image ($500\times 300$ pixels) as shown in Figure 2.

2.2.1. Local Texture Feature Descriptor

A local texture (LT) feature descriptor is designed to describe the texture features around each feature point according to the dominant rotated local binary patterns (DRLBP) proposed by Mehta [28]. Given a gray image $\mathbf{I}$ with $x\times y$ pixels. The DRLBP operates in a local circular region by taking the difference of the central pixel with respect to its neighbors. It is defined as:

$$DRLB{P}_{R,L}^{\mathbf{I}}(x,y)=\sum _{l=0}^{L-1}m(i(x,y),i(a_l\left(x\right),b_l\left(y\right)))·2^{mod(l-D,L)},$$

(4)

where

$$m(i(x,y),i(a_l\left(x\right),b_l\left(y\right)))=\left\{\begin{array}{cc}1,\hfill & i(a_l\left(x\right),b_l\left(y\right))\ge i(x,y),\hfill \\ 0,\hfill & i(a_l\left(x\right),b_l\left(y\right))<i(x,y).\hfill \end{array}\right.,$$

$i(x,y)$ and $i(a_l\left(x\right),b_l\left(y\right))$ are the gray values of central pixel and its neighbor in image $\mathbf{I}$, respectively, $(x,y)$ and $(a_l\left(x\right),b_l\left(y\right))$ are the geometric coordinate of the central pixel and its lth neighbor. Note that $a_l\left(x\right)=x+Rcos(2\pi l/L)$ and $b_l\left(y\right)=y-Rsin(2\pi l/L)$, R is the radius of the circular neighborhood and L is the number of the neighbors. The $mod$ indicates the modulus operator, and $D=\underset{l\in \{0,1,\dots ,L-1\}}{argmax}|i(a_l\left(x\right),b_l\left(y\right))-i(x,y)|$. In this paper, the ${\{DRLB{P}_{R,L}^{\mathbf{I}}(x,y)\}}_{x=0,y=0}^{X-1,Y-1}$ are held in binary form. The DRLBP descriptor $DRLB{P}_{R,N}^{\mathbf{I}}$ of $\mathbf{I}$ is denoted by the DRLBP histogram ${\mathbf{H}}_{R,L}(DRLB{P}_{R,L}^{\mathbf{I}})\in {\mathbb{R}}^{2^L}$. The $mod$ operator circularly shifts the weights with respect to the dominant direction because of the weight term $2^{mod(l-D,L)}$ depends on D in the above definition. Therefore, the DRLBP is a rotation invariance and computationally efficient texture descriptor.

Before giving the LT for each feature point, the image is first weighted for each feature point based on its geometric coordinates, the weighting matrix for each point ${\mathbf{p}}_t$ is defined as:

$${ϵ}_{xy}^t=exp(-\frac{\parallel {\mathbf{I}}_{xy}-{\mathbf{p}}_t{\parallel }^2}{2{\tau }^2}),$$

(5)

where ${\mathbf{I}}_{xy}$ is the geometric coordinate of the pixel $\mathbf{I}(x,y)$, ${\mathbf{p}}_t$ is the tth point of $\mathbf{P}$, and $\tau$ is a parameter that controls the window size of LT. ${ϵ}^t$ is a $x\times y$ weighting matrix of the tth feature point, and the tth weighting gray image is obtained by:

$${\mathbf{I}}^t(x,y)={ϵ}_{xy}^t\times \mathbf{I}(x,y),$$

(6)

where the weighting gray image ${\mathbf{I}}^t$ has the same size with the source gray image. Let ${\mathbf{I}}^t(x,y)=0$ when ${\mathbf{I}}^t(x,y)\le {10}^{-4}$. Figure 3 shows an example of weighting gray image.

We define the LT of the tth feature point via DRLBP in the weighting gray image:

$$\mathcal{LT}{\left({\mathbf{p}}_t\right)}_{R,L}={\mathbf{H}}_{R,L}(DRLB{P}_{R,L}^{{\mathbf{I}}^t}),$$

(7)

where $\mathcal{LT}{\left(\mathbf{P}\right)}_{R,L}$ is a LT feature set with T vectors both size of $1\times 2^L$.

2.2.2. Local Geometric Structure Feature Descriptor

A local geometric structure (LGS) feature is designed for each feature point in $\mathbf{P}$ by a local vector weighting method defined by:

$$\mathcal{LGS}\left({\mathbf{p}}_t\right)=\sum _{k=1}^K{\eta }_{tk}\overrightarrow{{\mathbf{p}}_t{\mathbf{p}}_{t_k}},$$

(8)

where $t_k$ is the index of kth neighbor of ${\mathbf{p}}_t$, and K is the number of neighbors. The LGS descriptor employs the K neighbors to give a LGS description for each feature point ${\mathbf{p}}_t$ but ignores outlier points. Hence, the value of K and the weight term ${\eta }_{tk}$ play a crucial role for the performance of LGS. We use an outlier score to define the weight term of $\overrightarrow{{\mathbf{p}}_t{\mathbf{p}}_{t_k}}$:

$${\eta }_{tk}=\frac{1}{2\pi {\sigma }_{\mathcal{LGS}}^2}exp(-\frac{{\mathsf{\Delta }}_{tk}^{\mathcal{LT}}}{2\pi {\sigma }_{\mathcal{LGS}}^2}),$$

(9)

where ${\sigma }_{\mathcal{LGS}}^2$ is the variance of ${\left\{{\mathsf{\Delta }}_t^{\mathcal{LT}}\right\}}_{t=1}^T$, and the outlier score ${\mathsf{\Delta }}_{t_k}^{\mathcal{LT}}$ is computed by the LT distance between ${\mathbf{p}}_{t_k}$ and the point that has the most similar LT feature with ${\mathbf{p}}_{t_k}$ as:

$${\mathsf{\Delta }}_{tk}^{\mathcal{LT}}=min\parallel \mathcal{LT}{\left({\mathbf{p}}_{t_k}\right)}_{R,L}-\mathcal{LT}{\left({\mathbf{p}}_{t(t\ne t_k)}\right)}_{R,L}{\parallel }^2.$$

(10)

2.2.3. Multi-Feature Descriptor

Different types of feature descriptors have their own advantages and limitations. This motivates us to make the respective advantages of LT and LGS descriptors complementary to each other. The multi-feature (MF) is designed to combine the local texture information and the local geometric structure information for improving the identifiability of each feature point. However, a fixed falseness does no good for guiding point registration throughout the iterations. Thus, the MF descriptor is defined as:

$$\mathcal{MF}\left(\mathbf{P}\right)=\mathbf{P}+T_1\mathcal{LGS}\left(\mathbf{P}\right)+T_2\mathcal{LT}{\left(\mathbf{P}\right)}_{R,L},$$

(11)

where $T_1$ and $T_2$ are annealing parameters for the LGS and the LT features, respectively. The instantiation of Equation (11) is given in the implementation details section.

2.3. Multi-Feature Guided Point Set Registration Model

For feature based image registration, two sets of feature points are extracted from a pair of multi-temporal agricultural terrace images (i.e., a sensed image and a reference image), respectively. The extracted feature points contain a large number of outliers that limit the performance of current non-rigid point set registration algorithms [29,30,31]. For this issue, a robust multi-feature guided model is designed—given two point sets $\mathbf{A}={\left\{{\mathbf{a}}_n\right\}}_{n=1}^N$ (i.e., the source point set) and $\mathbf{B}={\left\{{\mathbf{b}}_m\right\}}_{m=1}^M$ (i.e., the target point set) which are extracted from the sensed image and the reference image, respectively. The proposed point set registration model is first (i) to estimate correspondences between $\mathbf{A}$ and $\mathbf{B}$ by the proposed MF descriptor at each iteration, and then (ii) to update the location of $\mathbf{A}$ using a non-rigid transformation built by the recovered correspondences. The steps (i) and (ii) are iterated such that the $\mathbf{A}$ can gradually and continuously approach the target point set $\mathbf{B}$ , and finally match the exact corresponding points in $\mathbf{B}$ .

2.3.1. Correspondence Estimation

In the first step, the Gaussian mixture model (GMM) is applied to estimate correspondences by measuring the similarity of the MF between two point sets, and the correspondence estimation problem is considered as a GMM probability density estimation problem. Let the MF of ${\mathbf{a}}_n$ be the centroid of the nth Gaussian component, and the MF of ${\mathbf{b}}_m$ be the mth data. The GMM probability density function (PDF) is therefore obtained as:

$$s\left(\mathcal{MF}\left({\mathbf{b}}_m\right)\right)=(1-\omega )\sum _{n=1}^N{\mathcal{P}}_{mn}\varphi \left(\mathcal{MF}\left({\mathbf{b}}_m\right)\right|\mathcal{MF}\left({\mathbf{a}}_n\right))+\frac{\omega }{M},$$

(12)

where $\varphi \left(\mathcal{MF}\left({\mathbf{b}}_m\right)\right|\mathcal{MF}\left({\mathbf{a}}_n\right))=\frac{1}{2\pi {\sigma }^2}exp(-\frac{\parallel \mathcal{MF}\left({\mathbf{b}}_m\right)-\mathcal{MF}\left({\mathbf{a}}_n\right){\parallel }^2}{2{\sigma }^2})$ with the equal isotropic covariances ${\sigma }^2$ of MF, ${\mathcal{P}}_{mn}=\frac{1}{N}$ are non-negative equal quantity with ${\sum }_{n=1}^N{\mathcal{P}}_{mn}=1$, which are called the priors of GMM. $\frac{\omega }{M}$ is an additional uniform distribution with a weighting parameter $\omega$, $0<\omega <1$ for outlier dealing.

Once we have the PDF of GMM that is guided by the similarity of MFs, we can estimate correspondence by the posterior probability of GMM via Bayes’ rule:

$$s_{nm}\left(\mathcal{MF}\left({\mathbf{a}}_n\right)\right|\mathcal{MF}\left({\mathbf{b}}_m\right))=\frac{exp(-\frac{\parallel \mathcal{MF}\left({\mathbf{b}}_m\right)-\mathcal{MF}\left({\mathbf{a}}_n\right){\parallel }^2}{2{\sigma }^2})}{{\sum }_{i=1}^{N}exp(-\frac{\parallel \mathcal{MF}\left({\mathbf{b}}_m\right)-\mathcal{MF}\left({\mathbf{a}}_i\right){\parallel }^2}{2{\sigma }^2})+\frac{2\pi {\sigma }^2\omega N}{M(1-\omega )}},$$

(13)

by which we obtain an one-to-many fuzzy correspondence matrix ${\mathbf{S}}_{N\times M}$ guided by the similarity of MFs. Meanwhile, the corresponding target point set is obtained by:

$$\overline{\mathbf{B}}=\mathbf{S}\times \mathbf{B}.$$

(14)

The proposed correspondence estimation method imitates the process of human practice, which measures the similarities of geometric structure feature and local texture feature. Generally, the process for humans to estimate the corresponding point of the source point a in terrace image consists of two parts: (i) searching for a region in a reference image that has a similar geometrical location and structure compared to the region that surrounds source point a, and (ii) finding a point within this region that has similar color features (LT in this paper) to the source point set. An example is shown in Figure 4, where only 100 feature points are shown for visual convenience. The two feature points a and b both have the similar pattern of LT histogram and the similar geometric feature.

2.3.2. Transformation Estimation

We model the non-rigid displacement function f by requiring it to lie within a specific functional space, namely a vector-valued reproducing kernel Hilbert space (RKHS) [32,33]. The Gaussian kernel, which is in the form $\mathbf{G}({\mathbf{a}}_{n1},{\mathbf{a}}_{n2})=exp(-\frac{1}{2{\alpha }^2}\parallel {\mathbf{a}}_{n1}-{\mathbf{a}}_{n2}{\parallel }^2)$ and of size $N\times N$, is chosen to be the associated kernel for the RKHS, where $\alpha$ is a constant to control the spatial smoothness. The function f can be defined by:

$$f\left(\mathbf{A}\right)=\mathbf{A}+\mathbf{G}W.$$

(15)

Thus, the transformation estimation boils down to finding a finite parameter matrix W.

Before a direct parameter estimation, we first illustrate a rule by which a reliable transformation parameter is obtained in the estimation process.

• Regularizing the transformation estimation process.
  The adopted Tikhonov regularization framework [34,35,36,37] is one of the most common forms of regularization. It minimizes an energy function in an RKHS $\mathcal{H}$ to regularize a function f, and can be written as:

  $$\mathcal{R}\left(f\right)={\parallel f\parallel }_{\mathcal{H}}^2.$$

  (16)
  In this paper, the function f is defined in Equation (15).

As shown in Figure 5a, the regularized function (denoted by black line) is more reasonable than its non-regularized counterpart (denoted by blue curve). The transformation of an iterative registration method is such a procedure that slowly displaces the source point set so that the correspondence estimation is easier and more reliable. In other words, regularizing the transformation is necessary to accomplish the iterative registration. Figure 5b,c indicate that the ill-posed problem will exist if the transformation is not regularized. Note that as the number of points increases, the increasing arbitrariness of the transformation will lead to more severe ill-posed problems.

The multi-feature guided fuzzy correspondence matrix $\mathbf{S}$ contains $N\times M$ probabilities, hence a reliable transformation will produce a larger expectation of probabilities. Therefore, the solution of transformation estimation is detected by maximizing a likelihood function that is formed as ${\mathsf{\Pi }}_{m=1}^{M}s\left(\mathcal{MF}\left({\mathbf{b}}_m\right)\right)$, or equivalent to minimizing the negative log-likelihood function, which is formed as:

$$E(W,{\sigma }^2)=-\sum _{m=1}^{M}log(1-\omega )\sum _{n=1}^N{\mathcal{P}}_{mn}\varphi \left(\mathcal{MF}\left({\mathbf{b}}_m\right)\right|\mathcal{MF}\left({\mathbf{a}}_n\right))+\frac{\omega }{M}.$$

(17)

We use the maximizing expectation (M-step) of the expectation maximization (EM) algorithm [29] to estimate the transformation. The idea of the EM algorithm is first to guess the values of parameters (“old” parameter values) via computing the posterior probability by Equation (13) (E-step), and then to find the “new” parameter values via minimizing the expectation of the complete negative log-likelihood function (M-step), which is formed as:

$$Q(W,{\sigma }^2)=\frac{1}{2{\sigma }^2}\sum _{m=1}^M\sum _{n=1}^N{s}_{nm}{\parallel \mathcal{MF}\left({\mathbf{b}}_m\right)-\mathcal{MF}\left(f\left({\mathbf{a}}_n\right)\right)\parallel }^2+N_{\mathbf{S}}log{\sigma }^2+\frac{\mu }{2}\mathcal{R}\left(f\right),$$

(18)

where $N_{\mathbf{S}}={\sum }_{m=1}^M{\sum }_{n=1}^N{s}_{nm}\le M$ (with $M=N_{\mathbf{S}}$ only if $\omega =0$), $\mathcal{R}$ is the regularization of the transformation, and $\mu$ is a weighting parameter controlling the strength of the regularization. Furthermore, with an initialized deterministic annealing parameter ${\sigma }^2$, the parameter W is obtained by $\underset{W}{argmin}Q$. The mathematical solution is detailed in Section 2.5.

2.4. Feature Points Based Image Registration

Let $\mathbf{I}$ and ${\mathbf{I}}^t$ be the sensed and reference images, where the source point set $\mathbf{A}$ and target point set $\mathbf{B}$ are extracted from $\mathbf{I}$ and ${\mathbf{I}}^t$, respectively. $x(·)\times y(·)$ denotes the size of one image. Our goal is to obtain the transformed image $\widehat{\mathbf{I}}$.

After the transformed source point set $\widehat{\mathbf{A}}$ is obtained, a mapping function can be estimated based on the corresponding set that is constructed by $\mathcal{C}=\{\mathbf{A},\widehat{\mathbf{A}}\},$ and then the image registration can be realized. There are two types of mapping: (i) forward approach: directly transforming the sensed image $\mathbf{I}$ using the mapping function, and (ii) backward approach: determining the transformed image $\widehat{\mathbf{I}}$ from $\mathbf{I}$ using the grid of the reference image ${\mathbf{I}}^t$ and the inverse of the mapping. Due to the discretization and rounding, (i) is complicated to implement, as it can produce holes and/or overlaps in the output image, and we use the backward approach for image transformation.

We employ the TPS (thin plate spline) transformation model which obtained by:

$${\mathbf{E}}^{TPS}={\left(\begin{array}{cc}\mathcal{K}& \mathsf{\Phi }\\ {\mathsf{\Phi }}^T& \mathbf{O}\end{array}\right)}^{-1}\left(\begin{array}{c}\mathbf{A}\\ \mathbf{0}\\ \mathbf{0}\\ \mathbf{0}\end{array}\right),$$

(19)

where the TPS model ${\mathbf{E}}^{TPS}$ is of size $(N+3)\times 3$, $\mathbf{O}$ is a $3\times 3$ matrix of zeros and $\mathsf{\Phi }$ is the $N\times 3$ matrix with the nth denoting $(1,{\widehat{\mathbf{a}}}_n)$, and the $N\times N$ TPS kernel ${\mathcal{K}}_{ij}=\parallel {\widehat{\mathbf{a}}}_i-{\widehat{\mathbf{a}}}_j{\parallel }^{2}log\parallel {\widehat{\mathbf{a}}}_i-{\widehat{\mathbf{a}}}_j\parallel$.

A regular grid ${\Theta }_{Z\times 2}^t={{\left\{{\theta }_z^t\right\}}_{z=1}^Z}^T$ is obtained by a pixel-by-pixel indexing process on the reference image ${\mathbf{I}}^t$, where $Z=X({\mathbf{I}}^t)\times Y({\mathbf{I}}^t)$. Letting grid ${\Theta }^t$ be the source point set, and ${\mathbf{E}}^{TPS}$ the TPS transformation model, the transformed grid is obtained by first computing

$${\widehat{\Theta }}_{Z\times 3}^t=\left(\begin{array}{cc}\overline{\mathcal{K}}& \overline{\mathsf{\Phi }}\end{array}\right)\mathbf{E},$$

(20)

then restoring the dimension of the grid to 2 by ${\widehat{\Theta }}^t\leftarrow \left(\begin{array}{cc}{\widehat{\Theta }}_{(·,1)}^t& {\widehat{\Theta }}_{(·,2)}^t\end{array}\right)$, where the $Z\times N$ kernel ${\overline{\mathcal{K}}}_{ij}=\parallel {\theta }_i^t-{\widehat{\mathbf{a}}}_j{\parallel }^{2}log\parallel {\theta }_i^t-{\widehat{\mathbf{a}}}_j\parallel$, $\overline{\mathsf{\Phi }}$ is the $Z\times 3$ matrix with the zth row denotes $(1,{\theta }_z^t)$ and ${\widehat{\Theta }}_{(·,i)}^t$ denotes the ith column of ${\widehat{\Theta }}^t$. Let $\Theta$ be the grid obtained on $\mathbf{I}$, we have

$$\widehat{\Theta }={\widehat{\Theta }}^t\cap \Theta .$$

(21)

Finally, the transformed image $\widehat{\mathbf{I}}$ is obtained by resampling intensities from the sensed image $\mathbf{I}$ based on $\widehat{\Theta }$, setting the rest of pixels to black. Note that the bicubic interpolation is used to improve the smoothness of $\widehat{\mathbf{I}}$; to be more precise, the intensities of each pixel in $\widehat{\mathbf{I}}$ is determined by summing the weighted neighbor pixel intensities within a $4\times 4$ window.

2.5. Implementation Details

The instantiation of Equation (11) is impeded by LT descriptor $\mathcal{LT}$, which is constructed by a $2^L$ dimension histogram. Actually, the goal of Equation (11) is to measure the distance of MFs, which means that instantiating

$${\mathsf{\Gamma }}_{nm}=\parallel \mathcal{MF}\left({\mathbf{a}}_n\right)-\mathcal{MF}\left({\mathbf{b}}_m\right)\parallel$$

is equivalent to instantiating Equation (11). Analogically, ${\mathsf{\Gamma }}_{nm}=\parallel ({\mathbf{a}}_n-{\mathbf{b}}_m)+(\mathcal{LGS}\left({\mathbf{a}}_n\right)-\mathcal{LGS}\left({\mathbf{b}}_m\right))+(\mathcal{LT}\left({\mathbf{a}}_n\right)-\mathcal{LT}\left({\mathbf{b}}_m\right))\parallel$ has two-dimensional terms and one $2^L$ dimension term, hence, respectively instantiating

$${\mathcal{G}}_{nm}(\mathbf{B},\mathbf{A})=({\mathbf{b}}_m-{\mathbf{a}}_n)+(\mathcal{LGS}\left({\mathbf{b}}_m\right)-\mathcal{LGS}\left({\mathbf{a}}_n\right))$$

(22)

and

$${\mathsf{\Psi }}_{nm}(\mathbf{B},\mathbf{A})=\mathcal{LT}\left({\mathbf{b}}_m\right)-\mathcal{LT}\left({\mathbf{a}}_n\right)$$

(23)

is equivalent to instantiating Equation (11). The distance of geometric and similarity of LT are denoted by Equations (22) and (23), in which the instantiation of $\mathcal{G}$ can be commonly realized by Euclidean distance computation. Furthermore, in general, a human similarity estimation of LT is to differentiate the pattern of the histogram, as we discussed in Section 2.3.1. Therefore, we instantiate $\mathsf{\Psi }$ by firstly normalizing each histogram in $[0,1]$, and then computing the quadratic distance sum of each dimension in the histogram, which is formed as:

$$\begin{array}{cc}\hfill {\mathsf{\Psi }}_{nm}& =\mathcal{LT}\left({\mathbf{a}}_n\right)-\mathcal{LT}\left({\mathbf{b}}_m\right)\hfill \\ & =\sum _{i=1}^{2^L}{(\mathcal{LT}({\mathbf{a}}_n,i)-\mathcal{LT}({\mathbf{b}}_m,i))}^2,\hfill \end{array}$$

(24)

where $\mathcal{LT}({\mathbf{a}}_n,i)$ denotes the ith column of the histogram of ${\mathbf{a}}_n$.

Once Equation (11) is instantiated, we can respectively rewrite Equations (13) and (18) by:

$$s_{nm}=\frac{exp(-\frac{\parallel {\mathcal{G}}_{nm}{(\mathbf{B},\mathbf{A})\parallel }^2+{\mathsf{\Psi }}_{nm}(\mathbf{B},\mathbf{A})}{2{\sigma }^2})}{{\sum }_{i=1}^{N}exp(-\frac{\parallel {\mathcal{G}}_{im}{(\mathbf{B},\mathbf{A})\parallel }^2+{\mathsf{\Psi }}_{im}(\mathbf{B},\mathbf{A})}{2{\sigma }^2})+\frac{2\pi {\sigma }^2\omega N}{M(1-\omega )}},$$

(25)

and

$$Q=\frac{1}{2{\sigma }^2}\sum _{m=1}^M\sum _{n=1}^N{s}_{nm}(\parallel {\mathcal{G}}_{nm}(\overline{\mathbf{B}},f\left(\mathbf{A}\right)){\parallel }^2+{\mathsf{\Psi }}_{nm}(\mathbf{B},\mathbf{A}))+N_{\mathbf{S}}log{\sigma }^2+\frac{\mu }{2}\mathcal{R}\left(f\right);$$

(26)

thus, we can complete the M-step of EM algorithm for implementing the agricultural terrace image registration. The matrix form of Equation (26) to simplify the derivative is written as:

$$\begin{array}{cc}\hfill Q& =\frac{1}{2{\sigma }^2}(Tr({\mathcal{G}}_{\overline{\mathbf{B}}}^T{\mathbf{S}}_{\overline{\mathbf{B}}}{\mathcal{G}}_{\overline{\mathbf{B}}})-2Tr({\mathcal{G}}_{f\left(\mathbf{A}\right)}^T\mathbf{S}{\mathcal{G}}_{\overline{\mathbf{B}}})+Tr({\mathcal{G}}_{f\left(\mathbf{A}\right)}^T{\mathbf{S}}_{\mathbf{A}}{\mathcal{G}}_{f\left(\mathbf{A}\right)})-2Tr\left(W^T\mathbf{GS}{\mathcal{G}}_{\overline{\mathbf{B}}}\right)\hfill \\ & +2Tr\left(W^T{\mathbf{GS}}_{\mathbf{A}}{\mathcal{G}}_{f\left(\mathbf{A}\right)}\right)+Tr\left(W^T{\mathbf{GS}}_{\mathbf{A}}\mathbf{G}W\right)+Tr\left({\mathsf{\Psi }}^T(\mathbf{B},\mathbf{A})\mathbf{S}\mathsf{\Psi }(\mathbf{B},\mathbf{A})\right))+N_{\mathbf{S}}log{\sigma }^2+\frac{\mu }{2}Tr\left(W^T\mathbf{G}W\right),\hfill \end{array}$$

(27)

where $Tr$ denote trace operate, ${\mathbf{S}}_{\mathbf{A}}=dig\left(\mathbf{S}\mathbf{1}\right),{\mathbf{S}}_{\mathbf{B}}=dig\left({\mathbf{S}}^T\mathbf{1}\right)$, $\mathbf{1}$ is a column vector with all ones. ${\mathcal{G}}_{\mathbf{P}}=\mathbf{P}+{\mathbf{U}}^T\left(\mathbf{P}\right)\mathbf{P}$, where operator $\mathbf{U}\left(\mathbf{P}\right)$ is defined by

$${\mathbf{U}}_{ij}\left(\mathbf{P}\right)=\left\{\begin{array}{cc}{\eta }_{ik}-K,\hfill & \hfill \overrightarrow{{\mathbf{p}}_i{\mathbf{p}}_j}\in {\{\overrightarrow{{\mathbf{p}}_i{\mathbf{p}}_{i_k}}\}}_{k=1}^K,\\ -K,\hfill & \hfill \overrightarrow{{\mathbf{p}}_i{\mathbf{q}}_j}\notin {\{\overrightarrow{{\mathbf{p}}_i{\mathbf{q}}_{i_k}}\}}_{k=1}^K,\end{array}\right.$$

with $\mathbf{P}={\left\{{\mathbf{p}}_i\right\}}_{i=1}^I$ denoting a non-representational point set containing I points, $i_k$ is the index of the kth neighbor of the ith point, K is the number of neighbors, and ${\eta }_{ik}$ is the weight of LGS defined by Equation (9). The partial derivative of Equation (27) with respect to the parameter W is obtained by:

$$\frac{\partial Q}{\partial W}=-\frac{\mathbf{GS}{\mathcal{G}}_{\overline{\mathbf{B}}}}{{\sigma }^2}+\frac{{\mathbf{GS}}_{\mathbf{A}}{\mathcal{G}}_{\mathbf{A}}}{{\sigma }^2}+\frac{{\mathbf{GS}}_{\mathbf{A}}\mathbf{G}W}{{\sigma }^2}+\mu \mathbf{G}W.$$

(28)

Setting Equation (28) to zero, the parameter W is obtained by:

$$W={({\mathbf{GS}}_{\mathbf{A}}\mathbf{G}+\mu {\sigma }^2\mathbf{G})}^{-1}(\mathbf{GS}{\mathcal{G}}_{\overline{\mathbf{B}}}-{\mathbf{GS}}_{\mathbf{A}}{\mathcal{G}}_{\mathbf{A}}).$$

(29)

Parameter Setting

For evaluating image features, four groups of parameters—(i) R, the radius of the circular neighborhood in DRLBP; (ii) L, the number of the neighbors in DRLBP; (iii) $\tau$, the window size controlled parameter in LT; and (iv) $T_1$ and $T_2$, two annealing parameters for LGS and LT—are used. We set $R=1$, $L=8$, $\tau =10$, $T_1=exp(-iter/10)$ and $T_2=exp(-iter/50)$.

For registering extracted feature points, six parameters—(i) $\omega$, outlier weighting parameter; (ii) $\alpha$, a constant to control the spatial smoothness; (iii) ${\sigma }^2$, the equal isotropic covariances of MF; (iv) W, the parameter of point set transformation; (v) $\mu$, the weighting parameter of regularization and (vi) $ite{r}_{max}$, the max number of iteration—are used. We set $\omega =0.7$, $\alpha =2$ and $ite{r}_{max}=50$, and initialize W as a matrix with all zeros. Moreover, ${\sigma }^2$ and $\mu$ are first initialized by

$${\sigma }^2=\frac{NTr\left({\mathbf{A}}^T\mathbf{A}\right)-2\left(\mathbf{1}\mathbf{A}\right){\left(\mathbf{1}\mathbf{B}\right)}^T+MTr\left({\mathbf{B}}^T\mathbf{B}\right)}{2NM}$$

and $\mu =8$, then annealed as

$${\sigma }^2\leftarrow \frac{|Tr\left({\mathbf{A}}^T{\mathbf{S}}_{\mathbf{A}}\mathbf{A}\right)-2Tr(f{\left(\mathbf{A}\right)}^T\mathbf{S}\overline{\mathbf{B}})+Tr({\overline{\mathbf{B}}}^T{\mathbf{S}}_{\mathbf{B}}\overline{\mathbf{B}})|}{2N_{\mathbf{S}}}$$

and

$$\mu \leftarrow \frac{{(ite{r}_{max}^4-ite{r}^4+1)}^{\frac{1}{4}}}{ite{r}_{max}}\times \mu ,$$

respectively.

The pseudo-code of our feature points based agricultural terrace image registration is outlined in Algorithm 1.

Algorithm 1: Local texture feature and geometric structure feature guided agricultural terrace image registration

2.6. Computational Complexity

The computational complexity of each part in our feature points based agricultural terrace image registration is as follows:

• Image preprocessing:
  time complexity: $\mathcal{O}(X_1{Y}_1+X_2{Y}_2)$, space complexity: $\mathcal{O}(X_1{Y}_1+X_2{Y}_2)$.
• Feature extraction:
  time complexity for feature points selection : $\mathcal{O}(X_1{Y}_1+X_2{Y}_2)$, space complexity: $\mathcal{O}(N+M)$;
  time complexity for computing LT: $\mathcal{O}(N{X}_1^2{Y}_1^2+M{X}_2^2{Y}_2^2)$, space complexity: $\mathcal{O}(N{X}_1{Y}_1+M{X}_2{Y}_2)$;
  time complexity for computing LGS: $\mathcal{O}(N^2+M^2)$, space complexity: $\mathcal{O}(N+M)$.
• Point set registration:
  time complexity for correspondence estimation: $\mathcal{O}\left(NM\right)$, space complexity: $\mathcal{O}\left(NM\right)$;
  time complexity for transformation estimation: $\mathcal{O}(N^3+M^3)$, space complexity: $\mathcal{O}(N^2+M^2+NM)$.
• Image registration:
  time complexity: $\mathcal{O}(N^2+X_1{Y}_{1}N+X_1{Y}_1+M^2+X_2{Y}_{2}M+X_2{Y}_2)$, space complexity: $\mathcal{O}(N^2+X_1{Y}_{1}N+X_1{Y}_1+M^2+X_2{Y}_{2}M+X_2{Y}_2)$,

where $X_1,X_2,Y_1,Y_2$ are the widths and heights of the source image and sensed image, respectively; $N,M$ are numbers of the feature points extracted from the source and sensed image. Overall, the time complexity of the proposed method is $\mathcal{O}(N{X}_1^2{Y}_1^2+M{X}_2^2{Y}_2^2)$, and the space complexity is $\mathcal{O}(N{X}_1{Y}_1+M{X}_2{Y}_2)$.

3. Experiments and Results

3.1. Experiments Design

CPD (coherent point drift) [29], GLMDTPS (global and local mixture distance with thin plate spline transformation) [30], SIFT (scale invariant feature transform) [38] and SURF (speeded-up robust features) [39], four state-of-the-art methods, are compared against our method in the following experiments. SIFT and SURF methods used the open source $VLFeat$ toolbox with the threshold 1 and the Matlab open source $OpenSURF$ function with the default setting, respectively. We design two series of experiments: (i) due to the employment of the same feature point sets, the quantitative comparison on feature point matching is carried out on CPD, GLMDTPS and our method using the precision ratio (PR) [40]; (ii) quantitative comparison and qualitative demonstration on image registration are carried out on all the methods using the root of mean square error (RMSE), mean absolute error (MAE) and standard deviation (SD). The experimental dataset includes 20 pairs multi-temporal (4–5 month time interval) and multi-viewpoint agricultural terrace images ($500\times 300$ pixels) captured from Longji and Yunhe terraces, China (see Figure 6). All agricultural terrace images were obtained by a small UAV (the DJI Phantom 4 Pro (SZ, China), store homepage: [41]) with a CMOS camera. The small UAV basically maintained the same flight height (around 50–70 m) for collecting multi-temporal images of the same locations, but appropriately changed the imaging perspective for generating different geometric distortions, and different overlapping degrees of image pairs. All experiments are tested on a PC with 2.60 GHz Intel CPU and 16 GB memory.

3.2. Evaluation Criterion

The PR is usually employed to estimate the accuracy of feature point matching and defined as

$$PR=\frac{TP}{TP+FP},$$

(30)

where $TP$ and $FP$ denote the true positive and the false positive, respectively. The positive indicates inliers and the negative indicates outliers.

The RMSE, MAE and SD are usually used to quantify image registration accuracy. We manually determine 20 pairs of landmarks between the sensed image and the reference image as ground truth, and all the landmarks are well-distributed and selected in the easily identified places around agricultural terraces. The related formulations and the definitions in statistics are as follows:

$$RMSE=\sqrt{\frac{1}{N^l}\sum _{n=1}^{N^l}{\parallel a_n^l-b_n^l\parallel }^2},$$

(31)

$$MAE=\frac{{\sum }_{n=1}^{N^l}\parallel a_n^l-b_n^l\parallel }{N^l},$$

(32)

$$SD=\sqrt{\frac{1}{N^l}\sum _{n=1}^{N^l}{(d(a_n^l,b_n^l)-RMSE)}^2},$$

(33)

where $a_n^l$ and $b_n^l$ are the nth pair of corresponding landmarks in the sensed image and the reference image, respectively. $N^l$ is the total number of selected landmarks, and the operator $d(·,·)$ denotes the distance.

3.3. Results of Feature Matching

All agricultural terrace image pairs contain viewpoint changes and were captured with a 4–5 month time interval. By using the proposed preprocessing method, each image pair has 624 to 1513 feature points. For the quantitative comparison and visual demonstration, we evenly selected 100 feature points with more than 50% outlier (negative) to data rate for calculating the PR and visualizing the matching results as shown in

Table 1. Experimental results on series (i). Quantitative comparisons on the mean PR (precision ratio). Bold fonts indicate the best results. All units are in percentages. CPD (coherent point drift) denotes the method [29] and GLMDTPS (global and local mixture distance with thin plate spline transformation) denotes the method [30].

Method	CPD	GLMDTPS	Ours
PR	1.45%	10.43%	63.1%

and Figure 7. CPD and GLMDTPS gave poor performances since they estimated correspondences only using the Euclidean distance and the mixed geometric features, respectively, although CPD applied the motion coherent based geometric constraint to regularize the displacement field and GLMDTPS applied the annealing scheme to gradually change the transformation from rigid to non-rigid during registration. Our method gave the best matching performance in all image pairs since the local texture feature and the local geometric structure feature are combined and very complementary.

3.4. Results of Image Registration

For CPD, GLMDTPS and our method, all extracted feature points were used for image registration. Feature points for SIFT and SURF were extracted by their default setting. The quantitative comparison using the mean RMSE, MAE and SD are shown in

Table 2. Experimental results on series (ii). Quantitative comparisons on image registration measured using the mean RMSE (root of mean square error), MAE (mean absolute error) and SD (standard deviation) are carried out. $0\le \theta \le 20$ denotes the number of the failed registrations (can not manually identify landmarks). Bold fonts indicate the best results, and all units are in pixels. CPD (coherent point drift) denotes the method [29], GLMDTPS (global and local mixture distance with thin plate spline transformation) denotes the method [30], SIFT (scale invariant feature transform) denotes the method [38] and SURF (speeded-up robust features) denotes the method [39].

Method	$\mathit{\theta }$	RMSE	MAE	SD
CPD	0	41.78	25.18	50.54
GLMDTPS	5	21.35	6.95	24.51
SIFT	20	-	-	-
SURF	17	11.51	3.51	14.30
Ours	0	29.95	10.70	37.89

. The transformed images and the checkboards in four typical image registration examples are shown in Figure 8. Registering images using sparse correspondence has the advantages of being potentially faster and easily maintains a high feature point matching ratio. However, the image registration accuracy might not be high since a desired image transformation with acceptable accuracy can only be established based upon more dense correspondence (e.g., more feature points). This can be explained since the image transformation is basically interpolated from correspondences of feature points, thereby an adequate number of correspondences play a crucial role in yielding detailed transformation.

In this experiment, SIFT and SURF, which extract a relative small number of feature points, failed all and 17 registrations, respectively, although SURF performed well in the other three registrations (see Table 2). The reason was that the image registrations with a small number of feature points are sensitive to mismatching. Although CPD and GLMDTPS employed the same number of feature points with our method, the geometric features used in CPD and GLMDTPS were sensitive to outliers and similar neighborhood structures in multi-temporal images. Therefore, they also gave the relatively poor registration accuracies (e.g., GLMDTPS failed five registrations). In our method, the local texture feature and the local geometric structure feature of terrace images are combined well to improve the feature description of points, while helping to reject outliers. The outliers were rejected in the point matching step, but used to yield detailed transformation in the image registration step. Therefore, our method gave the best registration performance.

4. Conclusions

Due to soil erosion, illegal land occupations, agricultural land management practices and low purchasing power in China’s southeast and southwest mountain area, smallholder farmers as well as local governments require a light-weight and low cost technology such as DJI Phantom UAVs to monitor their planting area in terraces. However, multi-temporal images of the same planting area captured by small UAVs have only visible image information and are always accompanied by viewpoint changes, image geometric distortions, low image overlapping, brightness changes and color changes such that the images may not be directly used for dynamic agricultural terrace monitoring. Thus, transforming multi-temporal images into one coordinate system is necessary in order to be able to subsequently compare or integrate planting area information for dynamic agricultural terrace monitoring.

In this work, we have presented a small UAV based multi-temporal image registration method. The proposed method first designed a guided image filtering to enhance terrace ridges in multi-temporal images, and a multi-feature descriptor was applied to combine the texture feature and the geometric structure feature of terrace images for improving the description of feature points and rejecting outliers. The multi-feature guided model then provided an accurate guiding for feature point set registration, and the feature points based image registration finally gave an accurate image registration. Experiments on 20 pairs of multi-temporal terrace images captured by a DJI Phantom 4 Pro demonstrated that our method gives the best registration performance, and outperforms four state-of-the-art methods. To fully realize the dynamic agricultural terrace monitoring, future work will focus on core decision rules of automatic change detection algorithms between the registered images. For registering other landscape elements, the image enhancement preprocessing, and multi-feature extraction and combination steps (as described in Section 2.1 and Section 2.2), should focus on object-specific features that are invariant to various imaging perspectives, color and brightness changes.