Adaptive Processing for EM Telemetry Signal Recovery: Field Data from Sichuan Province

Abstract

This paper deals with the study of multi-channel adaptive noise cancellation with a focus on its application in electromagnetic (EM) telemetry. We presented new variable step-size least mean square (LMS) techniques: regularized variable step-size LMS and regularized sigmoid variable size LMS, for electromagnetic telemetry data processing. Considering the complexity and spatial distribution of environmental noise, algorithms with multiple reference signals were used to retrieve transmitted EM signals. The feasibility of the regularized variable step size LMS algorithms with numerical simulation was analyzed and presented. The adaptive processing techniques were applied in the recovery of frequency and binary phase shift key modulated signal. The proposed multi-channel adaptive technique achieves fast convergence speed, low mean squared error and is shown to have good convergence characteristics compared to conventional methods. In addition to attaining good results from the multi-channel adaptive filter and performing the signal analysis in real-time, we implemented combined fast effective impulse noise removal techniques. The combination of median and mean filters was effective in removing a wide range of impulsive noises without distorting any other data points. Further, electromagnetic telemetry data were acquired during a drilling operation in Sichuan province, China, for real field application. Data processing workflow was designed for EM telemetry data processing based on the noise characteristics, simulation results and expected result for demodulation. To establish a comprehensive overview, a performance comparison of the acquisition array system is also provided. Conclusively, the introduced multichannel adaptive noise canceling techniques are very effective in recovering transmitted EM telemetry signals.

1. Introduction

Over the past decade, bi-directional transmission of data from Bottom Hole Assembly (BHA) to the rig floor has played a vital role in reducing the risk involved in geosteering, well landing and saving considerable drilling time. In electromagnetic telemetry (EMT), downhole measured data are transmitted as coded electromagnetic waves by installing an electrical insulator sub along the drill string. Surface receivers are used in measuring the voltage difference between the drilling string and an earth antenna fixed into the ground at tens to hundreds of meters away from the rig. One of the fundamental challenges affecting the performance of EMT is the impact of noise on the measured signal. When the transmitted signals are from a transceiver at great depths, EM telemetry signal strength measured at the surface is very low due to attenuation, and noisy due to majorly surface interference. The noise could either distort or mask the transmitted signal and are caused by system nonlinearity, noise interference from adjacent environment and more.

Geophysical signal processing is either carried out post-data recovery or pre-data recovery from receiver unit, where a trade-off between time and data processing quality is considered. Digital signal processing (DSP) techniques use ranges from simple data filtering and smoothening by moving the average method to a more complex workflow in the neural network method [1,2,3,4,5,6,7,8,9]. However, recent practices have relied on high-resolution information to be applied or accessed in real-time. Conventional communication systems use filters to cancel noise in a received signal. Based on the previous study [5,6] and confirmation from our findings, noise sources on or near the drill rig usually have large power spectral densities over a broad frequency band including the transmitting frequency. Very low signal to noise ratio data, with noise and main signals operating in the same frequency, require highly effective data processing techniques. This explains why there are many difficulties associated with the retrieval of transmitted current signals from recorded EMT signals. Therefore, to maximize the acquired information and optimize the data recovery procedure, a significant and efficient data processing system is essential for accurate analysis and signal interpretation.

Two classes of noise affect the performance of electromagnetic telemetry systems: coherent harmonic noises from mainly industrial power supply equipment, which are simple to filter out either by spectral analysis or notch filtering and the random noises from atmospheric activities (e.g., thunderstorms), power grid switching, telluric, pipe scraping, etc. Therefore, EMT noise can be characterized as a combination of both random and impulsive noise. The random impulsive noises are mostly created by the power grid switching and by thunderstorms. In comparison to related fields, limited study relating to EMT (EM MWD) data processing has been published. Therefore, part of the review will reference other fields with similar signal-noise characteristics.

Noise suppression techniques have been introduced in several signal processing studies with similar noise characteristics, where the magnitude of noise recorded is higher than recorded signals and or data with very low signal to noise ratio is recovered. This includes adaptive filtering techniques such as least mean square, recursive least square, normalized least mean square, multichannel acoustic echo canceler, nonlinear adaptive filtering and other noise reduction techniques, spectral subtraction method and neural network techniques [5,6,10,11,12,13,14].

However, most of these methods achieve better noise cancellation only under certain conditions or by combining several denoising strategies; for example, spectral subtraction method used by reference [6] in EM MWD data processing resulted in improvement of the signal to noise ratio in the presence of typical rig surface noise. However, the improvement was limited to effective suppression of receiver-filtering related noise, with limitations faced in the retrieving of the coded signal. Reference [8], in their study of the effectiveness of Wiener filtering and adaptive noise canceling techniques in multi-channel magnetic resonance sounding (MRS) signal processing, showed/implied that different filters must be used for effective noise cancellation of different noise events as the two techniques performed less efficiently with signals containing spikes. In addition, neither methods reacted to the random phase shift or the difference in sampling frequencies, which led to a reduction in the noise canceling efficiency.

Furthermore, the study on least mean square (LMS) and normalized LMS (NLMS) techniques in noise cancellation shows that these techniques work well for medium to high signal to noise ratio signals but results in either noisy recovered signals which are post-processed or over-processed signal with information lost [12,15,16] when the signal to noise ratio ranges from low to very low. Reference [17] demonstrated that data collected from different sites show the line noise increases as the site falls within the power grid. In contrast, the extremely low frequency (ELF) noise power changes slowly as the site location is changed. This phenomenon is highly favorable to the application of an adaptive method. However, this study is focused on the application of single-channel LMS filter. With an improvement in adaptive noise cancellation algorithms, we present a comparison of the noise cancellation effect from 4 different adaptive methods with a multi-channel approach.

In general, the performance of nonlinear adaptive filtering algorithms is better than linear adaptive filtering algorithms [18,19,20]. However, the computation cost and the number of hardware required for these techniques are higher. Therefore, for practical engineering application, linear adaptive filtering algorithms such as LMS algorithm, whose calculations are smaller and hardware requirements are lower, are frequently used [21]. However, this paper aims at presenting the study on the application of nonlinear multichannel adaptive noise attenuation techniques and novel fast processing workflow in EMT to detect EM signals at the well surface.

In addition, to attain the best results from the multi-channel adaptive filter and perform the signal analysis in real-time, we introduced a combined fast and effective impulse noise removal technique. An ideal filter for impulsive noise removal is one that removes only the impulses without distorting any other data points.

When impulsive noise is present in digital data, median filters are often used as a remedy. The median filters run a sliding window over the data and at each step replace the point in the middle of the window by the median of the points inside the window [22,23,24,25]. However, study has shown the median filter technique to introduce distortion in the recovered data [26]. A more recent instantaneous approach involves applying a modified Z score to locate spikes and a simple moving average filter to remove the located spikes [26]. The Z scores are calculated using the detrend differentiated series. However, the technique is only effective at annihilating localized linear and slow-moving curve linear trends. Other techniques, such as generalized and modified wavelet methods [24], and spectral method [27,28] have been used effectively, but at the expense of obtaining timely results. Likewise, several techniques have been effectively applied in the attenuation of salt and pepper noise, which include new switching based median filter (NSMF) [29], adaptive weight median filter (AWMF) [30,31] and iterative adaptive fuzzy filter using alpha-trimmed mean [32]. References [33,34] demonstrated the application of maximum correntropy criterion-based sparse adaptive filtering algorithms in eliminating large impulsive noise. [33] indicates that the exponential term in the MCC updating equation is responsible for the elimination of large errors, which turns the algorithm robust against impulsive-like noises as observed in the pattern of the MSD training curves. Reference [35] introduced a technique that combines multilevel weighted graphs technique with induced generalized order weighted average (IGOWA) to remove salt and pepper noise from images. In addition, these techniques are highly effective in removing salt and pepper noise but are less time-efficient as they employ an iterative process in filter weight estimation.

As part of the developmental stage in geophysical equipment design, signal retrieval using effective noise cancellation tools are considered in this manuscript, with focus on the adaptive noise removal and inclusion of combined impulse noise removal technique. In this study, we compare results obtained from different adaptive noise cancellation techniques (NLMS, SVSLMS) with multi-reference array techniques and two new forms of sigmoidal variable step-size adaptive processing techniques. Signals generated from frequency shifting and phase-shifting form of signal modulation techniques (FSK and PSK) were considered. We considered the improvements in the algorithms for adaptive noise canceling techniques. Several reference dipoles, physically displaced from the primary dipole, are used to describe the noise characteristics in the drilling area. The high degree of noise correlation between distant locations makes adaptive noise canceling a good choice for noise removal. In addition, an adaptive filtering system can modify its parameters to adjust to the different noise conditions.

2. General Theory for Adaptive Noise Canceling

The adaptive noise-canceling technique has been an essential tool in the field of signal processing. Signal retrieval using effective noise cancellation tools, with a focus on the nonlinear adaptive noise removal, has a very wide application, including borehole studies. For EM telemetry, the data recorded at the surface, as shown in (1), are described as a combination of input current/voltage source S_v with specified waveform, earth filter S_e and noise such as industrial noise, cultural noise, telluric and external/extraterrestrial noise S_n (receiver function excluded).

$$S_d{=S}_v{+S}_e{+S}_n,$$

(1)

where S_d is the signal recorded at the receiver. In additive characteristics, the earth response and other noise characteristics are combined, and the simplified noise elimination expression is given as

$$S_v{=S}_d{-S}_n$$

(2)

However, we cannot achieve optimal filtering signal with fixed coefficient filter when random, unpredictable signal and noise characteristics are recorded. Therefore, nonlinear adaptive noise cancellation technique is essential. This technique requires the measurement of at least a single reference datum point that will adequately define the noise characteristics S_n for the specified dataset S_d. To collect the environmental noise without the transmitted signal being present, the reference dipole is placed colinear but away from the primary dipole and close to noise sources as much as possible. However, a single reference channel circumstance can only be achieved if there is only one and uniformly distributed noise source. This could be less effective in the situation of complexity and non-homogeneous spatial noise distribution. Therefore, our study will focus on using multiple reference signals.

In general, the ANC system has two parts: a primary channel with the main signal, and reference signals referred to as the noise signal (Figure 1). The recorded primary signal is represented as a time-harmonic signal by

$$S_d{\left(n\right)=A}_s·\mathit{cos}(\omega t{+n}_1)$$

(3)

$n=\left[1,2,3,\dots \dots ,m\right]$, while the reference signals are given as

$$S\left(n\right)=\left[S_1\left(n\right),S_2\left(n\right),S_3\left(n\right),\dots \dots ,S_k\left(n\right)\right]$$

(4)

$$S_k{\left(n\right)=A}_r·\mathit{cos}(\omega t{+n}_k)$$

(5)

and their respective filter weights are given as

$$W_i\left(n\right)=\left[W_1\left(n\right),W_2\left(n\right),W_3\left(n\right),\dots \dots ,W_k\left(n\right)\right]$$

(6)

such that the average reference signal is given as

$$S_a\left(n\right)={\displaystyle {\sum }_{i=1}^K\left[W_i\left(n\right)S_i\left(n\right)\right]}$$

(7)

Based on (2), the estimated error signal can be written as follows:

$$E\left[e\right]=E\left[S_v{+S}_n{-S}_a\right]$$

(8)

To adjust the parameters in a linear adaptive system, the least mean square (LMS) algorithm is used as a fast search algorithm using a gradient estimate [15,36]. The basic idea of the LMS algorithm is that it makes mean square error between the derived noise signal and desired signal to be minimum by adjusting the weight coefficients. This is expressed as

$$E{\left|e(n)\right|}^2=E{\left|S_v(n){+S}_n(n){-S}_a(n)\right|}^2$$

(9)

Consequently, the general expression for calculation of weight coefficients for the different LMS technique is given as,

$$W_i\left(n\right)=W_i(n-1)+2\mu \left(n\right)e\left(n\right)S_i\left(n\right)$$

(10)

where μ(n) is the step size and e(n) is the estimated error signal. For a nonlinear adaptive system, the filtering process requires continuous adjustment of the average reference signal with the filter weight coefficient. Its application efficiency is owned to the ability to estimate the noise characteristics from provided reference data.

2.1. Fixed and Step-Size LMS Algorithms

Four classes of adaptive algorithms are considered in this paper. The adaptive algorithms are: least mean square (LMS), normalized LMS (NLMS), sigmoid variable size LMS (SVSLMS) [37,38], and regularized variable step size (VSS) techniques: regularized variable step size LMS (RVSSLMS) and regularized sigmoid variable size LMS (RSVSLMS). The expression for the calculation of the weight coefficients for the different LMS techniques to be considered is given as,

LMS:

$$W\left(n\right)=W(n-1)+2\mu e\left(n\right)S_d\left(n\right)$$

(11)

that is, μ is constant.

NLMS:

$$W_i\left(n\right)=W_i(n-1)+2\left(\raisebox{1ex}{$\alpha $}\!\left/ \!\raisebox{-1ex}{$\beta +S_d^T\left(n\right)S_d\left(n\right)$}\right.\right)e\left(n\right)S_i\left(n\right)$$

(12)

where $0<\beta <1$, and $1<\alpha <10$.

β in (12) is a small positive constant used to avoid division by zero when the input vector S_d is zero and α is the adaptation positive constant.

SVSLMS:

$$\begin{array}{l}{W}_i\left(n\right)=W_i(n-1)+\\ 2\left(\beta ·\mathit{min}\left(abs\left(e(n)\right)·\left[1-\mathit{exp}(-\alpha {\left|e(n)\right|}^2\right],\left[1-\mathit{exp}(-\alpha {\left|e(n)\right|}^2\right]\right)\right)e\left(n\right)S_i\left(n\right)\end{array}$$

(13)

where $0<\beta <\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{${\lambda }_{\mathit{max}}$}\right.$, and $1<\alpha <10$.

α is a parameter controlling the shape of the sigmoid function, while β controls the range of the sigmoid function.

RVSSLMS:

$$\begin{array}{l}{W}_i\left(n\right)=W_i(n-1)+\\ 2\left(\begin{array}{l}\beta ·\mathit{min}\left(abs(e(n)\right)·\left[1-\mathit{exp}(-\alpha {\left|e(n)\right|}^2\right]\\ +\left(\lambda \left[{\left|\mu (n-1)\right|}^2\right]\right)\end{array}\right)e\left(n\right)S_i\left(n\right)\end{array}$$

(14)

where $0<\beta <\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{${\lambda }_{\mathit{max}}$}\right.$, $1<\alpha <10$, and $0.1<\lambda <1$.

λ is a parameter controlling the trade-off between the shape of sigmoid function and range of the sigmoid function.

RSVSLMS:

$$\begin{array}{l}{W}_i\left(n\right)=W_i(n-1)+\\ 2\left(\begin{array}{l}\left(\beta \ast \lambda \right)·\mathit{min}\left(abs(e(n)\right)·\left[1-\mathit{exp}(-\alpha {\left|e(n)\right|}^2\right]\\ +\left(\left(\beta /\lambda \right)·\left[1-\mathit{exp}(-\alpha {\left|e(n)\right|}^2\right]\right)\end{array}\right)e\left(n\right)S_i\left(n\right)\end{array}$$

(15)

where $0<\beta <\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{${\lambda }_{\mathit{max}}$}\right.$, $1<\alpha <10$, and $0.1<\lambda <1$.

β in (13) to (15) controls the convergence rate, and both α and λ control the error when the algorithm was almost steady. In addition, λ controls the error at the initial state for the RSVSLMS technique. This ensures a possible shift in the function characteristics to ensure a faster convergence at the early stage.

LMS is the most commonly used technique due to its simplicity. For each iteration, the LMS algorithm requires 2N additions and 2N+1 multiplications. However, using a constant step size, the filter is modified by an amount relative to the immediate estimate of the gradient of the error. This could lead to very slow convergence and eventual settling for a less accurate result, especially when dealing with random noise. The normalized least mean square algorithm (NLMS) technique is an extension of the conventional LMS technique which solves the above-mentioned issue by selecting a different step size value during the iteration process. In the NLMS algorithm, the step size varies as a function of the input energy and hence gives enhanced convergence properties [10,14]. The value of β in (12) is a small positive constant, and it is introduced in the algorithm to avoid division by zero when the values of the input vector are zero. The robustness against low SNR could be improved by reusing coefficient schemes [39,40], particularly the case for non-stationary (time-varying) noise signals, and where a sudden rise in the noise power may occur. SVSLMS algorithm, on the other hand, is a variable step-size LMS algorithm based on sigmoid function and are controlled by two constants, α and β. The two constants control the shape and size of the step size function, respectively, and in-turn the error in the steady state and the convergence rate. The condition for the convergence of this algorithm is $0<\mu \left(n\right)<1/{\epsilon }_{\mathit{max}}$, where ${\epsilon }_{\mathit{max}}$ is the maximum eigenvalue of the autocorrelation matrix of input signals [10]. Since β controls the value range and initial value of the step size, its maximum limit is the same as the inverse of the maximum eigenvalue of the autocorrelation matrix of input signals. The algorithm has a faster convergence rate, but μ(n) varies too fast when e(n) approaches zero.

Similarly, in regularized VSS techniques, RVSSLMS and RSVSLMS, α affects the shape, while λ acts to regularize both the shape and the initial step size, and in turn the rate of convergence. This guides against instability in SVSLMS, which is related to the choice of β (too high or too low) with respect to the given initial value.

2.2. Analysis of Regularized VSS Techniques’ Efficiency

For the analysis of the efficiency of the RSVSLMS technique and selection of appropriate value for λ, we presented plots of the change of step size with a change in error function at various values of α and λ (Figure 2a–d). From the plot, we deduce that the value of λ within the range of 0.1–1 gives an effective convergence rate with the value of α ranging from 1 to 10 and effective value of 5.

Figure 2 indicates the shape of μ(n) changes a lot with a change in the values of λ. A rapid convergence rate is expected with the obtained function curve at the initial adaptive phase, meaning big value of μ(n). However, as e(n) closes to 0, there is a small detuning at the later phase, meaning μ(n) changes slowly. In general, the following description describes the influence of λ on the shape of the function plot, vis-a-vis the convergence rate;

λ ≡ 0.1: The rate of change of the step size at the initial phase is slow, keeping the step size high and the convergence rate fast.

λ ≡ 1: The rate of change of the step size at the initial phase is higher, having a quick transition from the value of high step size to a low value and the convergence rate reduced. This is expected to be beneficial in case of the moderately noisy signal.

In extreme cases,

λ < 0.1: The step size at the initial phase is constant and very high, keeping the step size high and the convergence rate fast.

λ > 1: The rate of change of the step size is fast and almost linear, and there is smaller detuning at the steady-state phase that means μ(n) changes slowly when e(n) closes to 0. The function curve is similar to the RVSSLMS curve (Figure 3). Based on its characteristics at the steady-state phase, it is expected to be less efficient in wide range noise canceling when α is set higher than 5.

Therefore, for the comparison of the function curve from other SVSLMS algorithm, we set α and λ to be 5 and 0.5, respectively, considering the demands of larger initial step size and a steady change when e(n) is close to 0. Figure 3 shows the curves of the step size, μ(n) changing with the error function, e(n), adopting methods from the modified SVSLMS algorithm (blue dot/scattered plot) for comparison [38]. The proposed RSVSLMS algorithm has a large but slowly varying step size in the initial phase to ensure rapid convergence rate, and after convergence, the step size varies slowly in a steady-state phase. The update parameter, λ, has a stable value for the initial step size, which is determined by β, within the specified effective range, while the shape changes with change in λ, thereby increasing or reducing the rate of change of the step size to accommodate the difference in noise level, convergence rate and time. When large, the update parameter λ controls both the amplitude and shape of the step size if its range is wider. Therefore, β and α are kept constant while only the update parameter is changed for utmost performance, ensuring a perfect trade-off between the function curve shape and magnitude of the step size.

RVSSLMS function curve, on the other hand, has a constant shape, characterized by a relatively linear decay of step size. However, in comparison with the variable step size algorithm presented by Reference [41], there is a small detuning at the steady-state phase as e(n) closes to 0, meaning μ(n) changes slowly within the steady-state phase, thereby resulting in steady convergence.

3. Implementation and Simulation

In this section, synthetic data with defined noise characteristics, running as an infinitely continuous signal are denoised by implementing adaptive filtering. The results are presented in the section for a period of 0.1 s. The design for the adaptive filter simulation in developed Matlab Simulink for real-time noise suppression using adaptive filters; NLMS, RVSSLMS and RSVSLMS, are shown in Appendix A. This generates synthetic data with random noise. The interpolated (digital FIR) low pass filter was introduced to generate the variation in noise characteristics by varying the cut of frequency between 50 Hz and 350 Hz and introducing delay time to the signal to represent reference noise (Figure 4d). Continuous moving average was used to generate the variation in noise characteristics to represent reference noise acquisition (Figure 4c). The design allows an operator to change parameters at any time in the design to achieve optimization filter, and observe and compare the results from designed filters for the adaptive techniques with LMS or NLMS filter using a built-in Matlab function.

We illustrate the performance of the proposed algorithms by comparing the output of the computer simulations in a system with an identical scenario and technique parameters given in

Table 1. Adaptive noise-canceling technique parameters.

Technique	β	α	λ	σ	μ	Convergence Time (*0.05 s)
NLMS	NA	NA	NA	0.01	0.001	550
SVSLMS	0.00001	5	NA	NA	NA	500
RVSSLMS	0.00001	5	0.5	NA	NA	120
RSVSLMS	0.00001	5	0.5	NA	NA	80

. The adaptive filter and the unknown channel are assumed to have the same number of taps. The input signals are obtained by filtering the coded signal with: (1) random sequence and (2) random sequence and Gaussian noise (Figure 4d). Two reference noises with a time lag of 0.1 s and 0.25 s were added to S(n), which are independent of the input signal.

Firstly, the frequency shift key (FSK) modulation was selected in data coding with a sample rate of 2400 Hz and a cycle period of 0.05 s. The coded signal with a carrier frequency of 60 Hz is shown in Figure 4b. The input signals are obtained by filtering the coded signal with; (1) random sequence and (2) random sequence and Gaussian noise. Two reference noise with a time lag of 0.1 s and 0.25 s were added to S(n), which are independent of the input signal.

The left-hand side plots in Figure 5 shows the comparison of the overlay of the error signal on the input signal for all the techniques. These show the convergence speeds of the newly introduced algorithms (RVSSLMS and RSVSLMS) are higher than those of the old ones (NLMS and SVSLMS). Conversely, considering a measured signal, with random noise having the same sample time as the recorded signal, the proposed algorithms are faster, more accurate and efficient in noise cancellation than the regular varying step size NLMS algorithm. After a short period, this algorithm convergence enters a steady-state, and its detuning is smaller, which means that the fluctuation/step size changed within a smaller range. It also can be seen that the performance of SVSLMS is limited as the rate of convergence of the technique is slower, and it becomes unstable if the initial step size is reduced by using a value of β = 0.000001.

Furthermore, we carry out a comparison of the performance between the algorithms in the presence of random sequence with a sample time of 0.001 s and Gaussian noise with a sample time of 0.1 s. Both primary channel and reference channels contain a certain ratio of Gaussian noise running at a sample time different from that of the main signal. For a better representation of observed field scenarios, we added impulsive noise to the reference signal (Figure 6) and included the impulse noise reduction technique in the noise reduction workflow.

Later in this paper, we applied a two-stage method to suppress the impulsive noise in the field data but right now, only the second stage is applied to the simulated data. However, the complete two-stage method is introduced. Thus, in the first phase, we used the mean filter to remove generalized more repetitive impulsive signals. This is done by following the given steps.

Let Y = Y₁, …, Y_j, …, Y_n, represent the values of the data within a given time window. $Y_j$ and $Y_n$ represent the mid-time and the end-tine data for the given time window, respectively. The mean $M_{nt}$ mean(Y) of the series is calculated.

The obtained mean $M_{nt}$ is compared with previous mean $M_{nt-1}$, and the lowest of both is selected as $M_{nt}$.

The difference between the mean value ($M_{nt}$) and mid-time value ($Y_j$) is then calculated; $Z_m=M_{nt}-Y_j$. $Z_m$ value is then compared against a set of upper and lower thresholds, which represents the thresholds for background noise in the area which are acquired prior to the start of the field data.

Interpolated value $Y_j$ is then obtained for the mid-time data by calculating the mean of its immediate neighbors, using the following mathematical expression

$$Y_j=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$n$}\right.{\displaystyle \sum _{i=1}^n{Y}_i}\times I\left({\left|Z_m\right|}_{>\tau l}^{<\tau h}\right)$$

(16)

where $I(u)$ is an indicator function taking value one if the condition u is satisfied and 0 otherwise, and $Y_j$ remains unchanged. The width of the moving average neighborhood is controlled by the parameter i; which was set in our applications to 8.

In the second phase, we implemented the modified Z median filter [35]. However, the update for the interpolated data considers both the upper and lower thresholds using (8) above. This technique is effective at annihilating localized linear and slow-moving curve linear trends. The method proposed here requires low computation time and still reduces the total error considerably. This algorithm is computationally efficient and inexpensive, easy to execute and should be of great utility in a wide range of impulse noise removal. In this section, the criterion $\left|Z_m\right|>1$ was used as a guideline based on an outlier-labeling rule with the objective of screening large datasets. In practice, the scientist will have immediate control over this threshold parameter.

Figure 6 shows the synthesized main signal with added impulsive noise at a time of 2.25 s, the combined reference signal with added noise at time 3.75 s, and their respective denoised signals. The frequency spectrum for the signals (Figure 6a) indicates that the largest noise amplitude is concentrated within the first 50 Hz, which is within the range of the transmitted signal and that combined reference signal record higher amplitude at higher frequencies. The most notable difference between noisy and filtered data is observed in the combined references at high frequencies. In addition, no notable spike is recorded within the range of frequencies displayed. However, the effectiveness of the modified Z mean technique in removing localized impulsive noise and the effect of this processing step is established in this section.

Figure 7 demonstrates the processing results obtained from the different adaptive techniques. By considering the recovered signal from the impulsive noisy signal, Figure 7c,d shows the introduced algorithms (RVSSLMS and RSVSLMS) at a given time range performs well and better than the traditional varying step size NLMS algorithm and SVSLMS algorithm. The signal waveform extracted by the RSVSLMS algorithm is the closest to the ideal signal waveform with less noise. Although the result obtained with the NLMS technique reflects a much lesser distortion from the introduced impulsive noise, the recovered signal, in general, is very noisy and shows no good representation of the transmitted data at the given time frame.

The effectiveness of the modified Z median filter is observed by comparing the results obtained from noisy data with those obtained after the impulse noise has been filtered out (Figure 8a,b), using the RSVSLMS technique. The effect of impulse noise is observed as a distortion in the recovered transmitted signal at times 2.25 s, 3.75 s and 3.84 s. However, these distortions were effectively corrected, and the true transmitted waveform was restored in the recovered transmitted signal from median filtered data. In addition, the impulsive noise was more pronounced in Figure 8a where the noise was added to the main signal than in Figure 8b where the impulsive noise was added to only one of the reference signals at 3.75 s. The application of multi-channel reference signals resulted in the reduction of the effect of the impulsive noise until it became relatively insignificant. However, when the impulsive signal exists in all the reference signals at 3.84 s, the effect is notable, so also is the effectiveness of the median filter.

Furthermore, the binary phase shift key (BPSK) modulation was selected in data coding with a sample rate of 2400 Hz and a cycle period of 1 s. The coded signal with a central frequency of 10 Hz is shown in Figure 9b. The selected modulation technique and parameter is similar to the field transmitted current/voltage signal from the bottom hole assembly to the surface but with periodic cycle and larger sample rate. We illustrate the performance of the new algorithms by comparing the output of the computer simulations in a given system, using the parameters given in

Table 2. Adaptive noise-canceling technique parameters.

Technique	β	α	λ	μ
LMS	NA	NA	NA	0.00001
SVSLMS	0.00005	5	NA	NA
RVSSLMS	0.00005	5	0.5	NA
RSVSLMS	0.00005	5	0.5	NA

. The adaptive filter and the unknown channel are assumed to have the same number of taps.

Figure 10 shows comparisons of the denoised signal obtained by using LMS, SVSLMS, RVSSLMS, RSVSLMS techniques over a time range of 4 s. Figure 10b–d shows the introduced algorithms and the traditional varied step size LMS algorithm performed very well and better than the traditional LMS algorithm. The signal waveform extracted by these algorithms were close to the ideal signal waveform in Figure 9b. Figure 11 shows the comparison of the mean squared error plot for the various adaptive techniques. The mean squared error plots show that the result obtained from the variable step size techniques were close, and are more accurate than the conventional LMS technique. A high convergence rate characterizes the LMS technique result, but with low accuracy and a steady mean squared error value of 2.

Amongst the variable step size techniques, the RSVSLMS technique had the highest convergence rate with a mean squared error value of about 0.2. The traditional SVSLMS technique, on the other hand, had the largest convergence rate with a significant initial drop in MSE error for a period of about 10 s. However, the SVSLMS shows a continuous steady decrease in MSE error over a long period at the steady-state phase. This results in the lowest MSE value of about 0.01 for this study. However, in EM telemetry, a trade-off between convergence rate and MSE value is required to ensure real-time processing of the transmitted signal.

Similarly, we simulated synthetic signal using a combination of random and white noise but with a frequency difference of range of 10, that is, 10 and 20 Hz, respectively. Similar to previous results (Figure 10), the LMS technique had the highest convergence rate but with a very low MSE value, which is indicative of noisy retrieved signal as observed in Figure 12. The MSE plot (Figure 13) also shows a close similarity in the accuracy of the retrieved signal from the RSVSLMS and RVSSLMS techniques. In comparison with results from less complex noise characteristics, the retrieved signal quality for LMS and SVSLMS techniques decreased significantly, and their convergence rate increased by a large margin. Thus, we can infer that with more complex noise data from field measurement, the above techniques would be less effective in real-time EM telemetry signal recovery than introduced RVSSLMS and RSVSLMS techniques.

In general, the obtained results show that the newly introduced techniques are very effective in complex noise suppression. The proposed algorithms are faster, more accurate and efficient than the regular varying step size NLMS algorithm within the short time frame expected for EM telemetry signal recovery. After a short period, these algorithms convergence enters a steady-state, and its detuning is smaller, which means that the fluctuation changes within a smaller range. The rapid and then steady change in step size and changes in function curve for RSVSLMS algorithm ensures the stability of the introduced techniques over a wide range of applications. Although the computation cost and the number of hardware required for these techniques are higher, considering the convergence rate and accuracy of the results obtained within the first 5 to 10 s, the newly introduced RVSSLMS technique is expected to be most effective in adaptive signal retrieval for EM telemetry. The fast convergence rate makes this technique applicable in real-time engineering application with high output data quality irrespective of the complexity of the noise characteristics. Lastly, the multi-channel adaptive noise canceling performed above on the synthetic data gave an insight into the characteristics of the retrieved phase shift signal.

4. Field and Simulation Study

Electromagnetic telemetry data were acquired during a drilling operation in Sichuan province, China. The starting depth of the downhole transceiver was about 2000 m. Phase shift coded signal with a frequency of 10 Hz was transmitted from the insulating gap along the drill string, which was outside the borehole casing. The electrode array used in EM telemetry signal measurement is shown in Figure 14. The voltage difference between dipoles (L1 to L10), which varies in length from 30 m to 80 m was recorded using the DRU 45 system developed by Institute of Geology and Geophysics, Chinese Academy of Sciences at a sampling frequency of 2400 Hz.

Firstly, we considered the performance of the techniques if the frequency shift signal had to be the modulation type by convolving the field noise signal with the generated frequency shift signal. The noise spectrum from dipoles L2, L4, L5 and L7, were used in this study (Figure 15a). The spectrum shows noise signals with an amplitude range of the order of 10-1 to 10-5 mv for the frequency range of interest, with the largest magnitude recorded by L2. The processed main signal is obtained by convolving the noise signal from the dipole L5 (blue) with simulated frequency shift key modulated signal (Figure 15b). For this study, lines L2, L4 and L7 were used as reference signals. The frequency shift signal was generated using a sample rate of 2400 Hz, signal frequencies of 10 and 5 Hz and a period of 0.5 sec. The transmitted signal amplitude was varied between 0.001 V and 0.0002 V to allow for an assessment of the processing technique under different signal to noise ratio (SNR). The plot of the amplitude spectrum of the generated data and combined reference noise within the frequency range of interest is presented in Figure 15c. The frequency spectrum (Figure 15c) indicates that the combined reference signal amplitude is generally higher, especially at the low frequencies. In addition, a significantly high amplitude spectrum was recorded around frequencies 200 Hz, 400 Hz and 600 Hz. The performance of the variable step size techniques was compared by analyzing the output (obtained error signal) signals over specified time ranges, the mean squared error plots and bar chart of SNR and the power of the output signal in dB.

Figure 16 and Figure 17 demonstrates the processing results obtained by the variable step size techniques. The obtained results show that the introduced algorithms (RVSSLMS and RSVSLMS) performed better than the traditional varying step size LMS algorithm at both early time and late time in this study. The inclusion of the impulsive noise filter in the workflow resulted in the removal of major impulsive noise signals as recorded between the time band of 135 s and 139 s (Figure 15b and Figure 16a). This is also evident in the MSE plot.

The signal waveform extracted by the SVSLMS algorithm still contained the footprint of high and low-frequency noise, even though the signal recovered still indicate the alternating form of the input signal in Figure 16b is close to the ideal signal waveform. The recovered signal from the regularized techniques, considering RSVSLMS technique as an example, show a decrease in quality with a decrease in signal to noise ratio from −55.5 dB to 69.4 dB. At the current state, a good correlation with the original signal was observed when the SNR of the input/main signal is higher than −69 dB. The power of the output signal, on the other hand, remain mostly constant and unhinged by the change in the main signal’s SNR (Figure 18). The recovered signal from RSVSLMS technique has the highest signal power with a value of −25 dB. From the results, it is evident that the proposed regularized LMS techniques achieve better noise tracking by providing an overall quality improvement of the processed synthetic EMT signal over the regular SVSLMS method.

Furthermore, the mean squared error plots show a very fast initial convergence for the regularized LMS techniques and a much slower convergence for the SVSLMS technique (Figure 19). However, after the first 170 s, the SVSLMS technique records the lowest MSE value until all the techniques converge at a later time. The smoothness of the regularized LMS technique’s MSE values represents a consistent, smooth and well denoised output signal. On the other hand, the MSE plot for the SVSLMS technique is more erratic, and it represents a noisy output signal. In all, the RSVSLMS technique with very fast convergence, the lowest long-range MSE value and highest recovered signal power, is expected to perform the best in EMT data processing with the combined impulse noise filtering included in the processing workflow.

Lastly, the acquired EMT data was processed using the newly introduced adaptive techniques. The noise characteristics for the area and individual dipole were considered in selecting the reference dipole, while the main signal was selected based on expected maximum telemetry signal from previous simulation studies. The noise spectrum from dipoles L1 to L7, are shown in Appendix B. The spectrum shows general noise signals with an amplitude range of the order of 10⁻¹ to 10⁻⁵ v.

For the frequency range of interest (Appendix B), the measured voltage ranges from 10⁻¹ to 10⁻³ v with the largest magnitude recorded by L3 for electrodes set colinear to the main dipole L1, and L7 for electrodes orthogonal to the main dipole. This shows that the magnitude of data noise recorded increases as the offset electrode moves closer to the drilling rig, especially with electrodes set orthogonal to the main dipole. The ELF noise power changes slowly as the site location is changed. This phenomenon is highly favorable to the application of an adaptive method. Further, several high amplitude signals were recorded over the broad frequency spectrum, which includes 50 Hz industrial/power noise. The main input signal is obtained from dipole L1 with a dipole length of 80 m. Several sets of reference noise were used to identify the most effective dipole array for the estimation of overall main data noise. However, only the results from the sets of reference data, set 1: L2, L8 and L9, and set 2: L4, L5 and L7, will be presented in this publication.

Figure 20 demonstrates the data processing workflow adopted in this study considering the noise characteristics, simulation results and expected result for phase shift demodulation. The primary input to the signal processor is the signal received from the well by the antenna(s) connected to the wellhead. The primary input contains the desired EMT signals plus all the noise that is to be filtered out. The secondary inputs are used as the references for the adaptive filter and contain only the surface noise.

To achieve the best results and still perform the signal processing in real-time, the number of data points for a given time frame is reduced by setting the new sampling rate between 512 Hz and 200 Hz, which is larger than the anti-aliasing filter frequency. These settings provide the system with enough information to process the data with acceptable accuracy in real-time. Equiripple (digital FIR) low pass filter was introduced before the impulsive noise canceling to reduce the excessive frequency bandwidth to a reasonable range with a crossover frequency of 80 Hz.

When impulsive noise is present in digital data, combined impulsive noise filter runs a 4 point sliding window over the data and at each step replaces the data with new values. The data are scanned and only changed if impulses are detected. The impulse data are replaced by a piecewise-linear estimated value calculated from previous and current data points. The used techniques meet the requirement of an ideal filter for impulsive noise removal, which is one that removes the impulses without distorting any other data points. In addition, the adaptive noise counselling result and time-continuous sine wave, with the same frequency as the transmitted signal (10 Hz), were combined to generate the final processing output to enhance the phase shifts for easy demodulation process.

The performance of the regularized variable step-size LMS techniques were compared by analyzing the output signals over specified time ranges and the mean squared deviation plots. Figure 21, Figure 22, Figure 23 and Figure 24 demonstrate the processing results obtained by the selected techniques. The recovered signal between 50 to 60 s and 80 to 90 s of the data is presented for comparison. Figure 21 and Figure 22 show that the introduced techniques are highly effective in EMT data processing even in the presence of complex field noise with wide frequency band. The results obtained from the regularized techniques are very similar to each other. However, the signal waveform extracted by the RVSSLMS algorithm still contained the footprint of low-frequency noise. This may result in masking of the phase shift signal in the final processing output even though the signal recovered is close to the ideal signal waveform.

Furthermore, the results obtained from single and multiple reference signals using the RSVSLMS technique are compared in Figure 23. Though the two results are similar in most cases, few differences such as the high amplitude distortion between 137 s and 138 s, and distortions between the time 132 s and 134 s caused by high-frequency signal interference shows the single reference technique could present less accurate result over time. In another study, the effect of change in the reference signals’ orientation was considered. The result obtained from the different sets of reference signals is presented in Figure 24. Result obtained from the RSVSLMS technique shows a significant decrease in accuracy when the orthogonal dipole signals were used for data processing. Therefore, the selection of reference signal array in EM telemetry data processing when phase shift modulation is used should be limited to colinear dipoles and dipoles within 300 of the main signal.

Lastly, the mean squared deviation (MSD) plots show a significant correlation between the processing output and the MSD value (Figure 25). The two techniques, RSVSLMS in green and RVSSLMS in blue, converged significantly differently from the conventional SVSLMS and NLMS techniques. The MSD of NLMS displays fast convergence but has the highest MSD value at an average value above 10⁻⁴. The SVSLMS technique, on the other hand, had the slowest convergence rate with lowest MSD value about 10-5. The MSD plot for RSVSLMS and RVSSLMS results display the fastest convergence and the lowest MSD value. The MSD plot becomes steady at an average value of about 10⁻⁷. The characteristics of the MSD plot is in agreement with the level of accuracy of signal retrieved from these techniques. In all, the RSVSLMS technique shows more consistent good performance in noise-canceling and MSD value at the steady-state. The correlation between the performance of the regularized LMS techniques, both with synthetic and real field data, signifies its effectiveness in EM telemetry signal processing even when the recorded data have a low signal to noise ratio. The obtained results also show that the newly introduced technique can adapt effectively, even in the presence of several changes in noise characteristics. In comparison to the phase modulated signal, the adaptive processed results from the frequency modulated signal, in general, show a readily definite representation of the input data.

5. Conclusions

Novel variable step-size algorithms, regularized SVSLMS and VSSLMS, were proposed in this study for EM telemetry signal processing. Analysis of the simulation results and MSE error show an increase in convergence rate and steady-state detuning at the later phase. The function curve and error curve illustration show that the regularized variable step-size LMS algorithms combine the virtues of a trade-off between the shape of the step size function curve and the step size initial value, which leads to fast convergence rate and lower final misalignment. The regularized SVSLMS and VSSLMS algorithms show more advantages in convergence rate during the initial phase and steady-state detuning than a traditional variable step NLMS algorithm and SVSLMS algorithm. Through comparative analysis between the proposed algorithms with multi-channel reference noise signal and previous LMS techniques, it is concluded that the new algorithms perform very well on signal waveform retrieval and data fitting error. The electromagnetic telemetry test data demonstrate that a good denoising effect can be obtained with multiple reference channels and also, the dipole orientation has a significant effect on the shift signal recovery.

The effectiveness of the designed regularized LMS algorithm and reference dipole arrays to extract the EM telemetry signal from field data with complex noise estimated based on the progressive weight calculation from multiple colinear and near colinear reference channels was established. The regularized variable step size approach significantly improved the separation performance level and could increase the reach of EM telemetry. The proposed processing workflow with combined impulsive noise filter and the multi-channel adaptive technique method can be used in developing timely computerized automating EMT signal retrieval. Based on our findings, further study on the application of neural network for further processing and effective demodulation is proposed.