Sampling by Difference as a Method of Applying the Sampling Kantorovich Model in Digital Image Processing

Abstract

In this paper, the connections between the Sampling Kantorovich model and the sampling process are highlighted and exploited. Based on the theoretical framework of the Sampling Kantorovich operators, a sampling paradigm, here named Sampling Kantorovich by Difference (SKD), is introduced. In line of principle, SKD allows for overcoming the technical limitation due to the fact that the resolution of a signal/image is strictly connected with the size of the used sensors. We analyze the paradigm in the case of a simulated super resolution type problem. The same mathematical model, being extendable to other signal reconstruction procedures, suggests a theoretical way for new technical solutions in the sampling procedures.

1. Introduction

Every aspect of our daily life is strongly connected with the quality and quantity of information we can perceive and elaborate. Information is contained in signals, transmitted in different forms and via different physical mediums. To measure these signals and extract useful information from them, specific techniques have been developed. The main problem of reconstructing signals, starting from the knowledge of their samples, has been tackled and mathematically solved by the following classic Whittaker–Kotelnikov–Shannon (WKS) theorem.

Theorem 1. 

(WKS Sampling Theorem [1]). Let $f\in L^2\left(\mathbb{R}\right)$ such that $supp\widehat{f}\subseteq [-\pi w,\pi w]$, $w>0$, where $\widehat{f}$ denotes the Fourier transform of the function f. Then, f can be reconstructed on the whole real axis by the following formula

$$\sum _{k\in \mathbb{Z}}f\left(\frac{k}{w}\right)sinc(wt-k)=f\left(t\right),t\in \mathbb{R}.$$

The WKS sampling theorem states the sufficient conditions for the exact reconstruction of a function $f\in L^2\left(\mathbb{R}\right)$, which is band-limited, by means of the above interpolation formula given its samples values $f\left(\frac{k}{w}\right),$$k\in \mathbb{Z},w>0$ calculated in the, so-called nodes $\frac{k}{w}$. For an interesting discussion about the above formula, see, e.g., Ref. [2]. From an application point of view, the WKS theorem states that, given an infinite countable set of samples $f\left(\frac{k}{w}\right)$, it is possible to reconstruct the original signal without any errors if the Nyquist condition ${\nu }_s\ge 2{\nu }_{max}$ is satisfied ([3,4]), where ${\nu }_s$ is the sampling frequency and ${\nu }_{max}$ is the maximum frequency of the sampled signal. Although the WKS theorem had a very strong impact as in the mathematical literature as in the applications, its hypotheses hide both theoretical as practical limitations. From the theoretical point of view, to require that $f\in L^2\left(\mathbb{R}\right)$ and $supp\widehat{f}\subseteq [-\pi w,\pi w]$, $w>0$, means that f is the restriction to the real axis of an entire function of exponential type (see the Paley–Wiener theorem [5]).

Moreover, from the application point of view, the above interpolation formula requires the knowledge of the samples values $f\left(\frac{k}{w}\right)$ over whole $\mathbb{R}$, being $k\in \mathbb{Z}$, i.e., the sampling of the signal on infinite time nodes: consequently, the evaluation of the signal should be made both in the past of a fixed instant $t\in \mathbb{R}$ as in the future, which means making a prediction (causality condition is not respected). Furthermore, in concrete application problems, signals are duration-limited (i.e., with compact support) and this is not compatible with the requested band limitation, as stated by the Heisenberg uncertainty principle. Moreover, in applications, the value of ${\nu }_{max}$ is generally fixed by the degree of precision requested in the measurement and not by the signal itself: in fact, it is not possible to precisely state a priori what is the maximum frequency of an unknown function. For all these reasons, the hypotheses of the WKS sampling theorem need to be weakened.

Relaxing the assumptions determines the loss of an exact reconstruction and consequently introduces an approximation error.

A class of operators, called generalized sampling operators, has been formulated with the aim to solve the mentioned critical issues connected with the WKS theorem (see, e.g., Refs. [6,7,8,9]).

For a function $\phi :\mathbb{R}\to \mathbb{R}$ with $\phi \in C_c\left(\mathbb{R}\right)$ (where $C_c\left(\mathbb{R}\right)$ is the space of continuous functions with compact support on $\mathbb{R}$) and a bounded function $f:\mathbb{R}\to \mathbb{R},$ the generalized sampling operators are defined as

$$\left(S_w^{\phi }f\right)\left(x\right):=\sum _{k\in \mathbb{Z}}f\left(\frac{k}{w}\right)\phi (wx-k),x\in \mathbb{R},w>0,$$

for every $x\in \mathbb{R},$$\phi$ being a suitable kernel function (see, e.g., Refs. [9,10]).

The above operators are bounded linear operators mapping $L^{\infty }\left(\mathbb{R}\right)$ into $L^{\infty }\left(\mathbb{R}\right)$ and $C\left(\mathbb{R}\right)$ into itself, with respect to the $L^{\infty }\left(\mathbb{R}\right)$ and $C\left(\mathbb{R}\right)$ norms, respectively, (here $L^{\infty }\left(\mathbb{R}\right)$ denotes the space of essentially bounded functions, while $C(\mathbb{R}$) is the space of bounded and uniformly continuous functions on $\mathbb{R}$).

The operator norm is $\parallel S_w^{\phi }{\parallel }_{\left[C\right(\mathbb{R}),C(\mathbb{R}\left)\right]}={sup}_{u\in \mathbb{R}}{\sum }_{k\in \mathbb{Z}}\left|\phi (u-k)\right|$, where $m_0\left(\phi \right)={sup}_{u\in \mathbb{R}}{\sum }_{k\in \mathbb{Z}}\left|\phi (u-k)\right|$ defines the absolute discrete moment of order zero of the function $\phi$.

Moreover, it is possible to prove (see, e.g., Ref. [9]) that $\left(S_w^{\phi }f\right)\left(x\right)$ converges to $f\left(x\right)$ as w tends to $+\infty ,$ when the kernel $\phi$ satisfies the following condition:

$${\displaystyle \sum _{k\in \mathbb{Z}}\phi (x-k)=1,}$$

for every $x\in \mathbb{R},$ pointwise in every point x of continuity for f bounded and uniformly with respect to $x\in \mathbb{R},$ when f is bounded and uniformly continuous.

The generalized sampling formulation has, compared to the classical WKS sampling theorem, the advantages to require only a finite number of samples, that in addition, can be taken only in the past with respect to the point to reconstruct (this last result being valid assuming $\phi \in C_c\left({\mathbb{R}}^{+}\right)$). As previously mentioned, the weakening of the hypotheses has its downside, represented by the introduction of a reconstruction error. In the setting of the generalized operators, controlling this error is important to quantify the goodness of the reconstruction: for these reasons, besides results of convergence, also results about the order of approximation, saturation order, and inverse results, have been given (see, e.g., Ref. [11]). For the extension of the theory to a multidimensional setting and to other functional spaces, see ([12,13,14]). Note that thanks to the generalized formulation, the problem concerning the not causality condition of the reconstruction process is avoided (for $\phi \in C_c\left({\mathbb{R}}^{+}\right)$).

From a practical point of view, the generalized formulation maintains a pointwise mathematical model, in opposition to the not-pointwise nature of the samples in the measurement process. Indeed, a certain time is required to perform an acquisition, but neither the WKS nor the generalized formulation models take into account of this aspect. As we see in Section 2, the Sampling Kantorovich mathematical model will overpass this limitation.

Although the theory of reconstruction using these operators has solved the drawbacks connected with the classical sampling formulation, they are not very suitable to be studied in the context of not continuous functions/signals. In fact, it is not difficult to show (see the counterexample in [15]) that they do not map $L^p\left(\mathbb{R}\right)$ into itself, their expression showing how strongly they are related to the function values on single nodes $\frac{k}{w},k\in \mathbb{Z},w>0.$ A more realistic way to model the effective sampling process is to introduce, in place of the pointwise samples $f\left(\frac{k}{w}\right)$, an averaged version in a neighborhood of the nodes. From a theoretical point of view, to allow the reconstruction of a broader class of signals (therefore not necessarily restricted to $C\left(R\right)$ or to $C^0\left(R\right)$-space of continuous functions), in [15] the Sampling Kantorovich (SK) operators have been introduced and studied. On the other hand, from an application point of view, the new operators mathematically model, concretely, the sampling process. Thanks to this model, we are able to define a sampling paradigm that we call Sampling Kantorovich by Difference (SKD), that is applied to a super-resolution-type problem.

The goal of this work is to analyze the SK model and its behavior when applied to a sampling type problem, related, e.g., to a super resolution problem by multiple acquisitions. In particular, as summarized in Section 4, the proposed SK model is able to theoretically manage more correctly (i.e., avoiding the ideal pointwise approach and introducing an integral to formalize the sampling procedure) multiple acquisitions for super resolution, with respect to the WKS and the generalized sampling formulations. This results, in line of principle, in a reconstruction without errors.

Other approximation and quasi-approximation operators are also available for image processing and reconstruction (e.g., Refs. [16,17]). Saturation results, proving the rate of convergence, e.g., of the wavelets and SK operators are available in [18,19]. Moreover, in this work, the SKD is considered in an uniform (equi-spaced) sampling case, but, as for other methods (e.g., Refs. [20,21]), it can be extended to the more general, non-uniform, paradigms.

In addition, the non-linearity of the Digital to Analog (DA) and Analog to Digital (AD) converters are not taken into consideration in what follows; the model mathematical and the experimental data purely being achieved via algorithmic simulations. Only the case of ideal converters, i.e., without distortions of any kind, is considered in this work. The paper is organized as follows: in Section 2, the main results concerning the SK operators are summarized; in Section 3, the new concept of SKD is introduced; in Section 4, the results of numerical simulations, applied to a super-resolution-type problem, are shown; finally, in Section 5, the conclusions and some remarks follow.

2. Sampling Kantorovich (SK) Mathematical Model

To weaken the assumptions connected with the generalized sampling theorem and to extend the convergence results to the class of functions belonging to $L^p$ spaces, the Sampling Kantorovich (SK) operators have been introduced in [15] as more suitable in this setting.

In the one dimensional case, let $\mathsf{\Pi }={\left(t_k\right)}_{k\in \mathbb{Z}}$, a sequence of real numbers such that:

(i)

$-\infty <t_k<t_{k+1}<+\infty$;

($ii$)

${\displaystyle \underset{k\to \pm \infty }{lim}{t}_k=\pm \infty }$;

($iii$)

there exists $\Delta ,\delta >0$ such that $\delta \le t_{k+1}-t_k:={\Delta }_k\le \Delta ,$ for every $k\in \mathbb{Z}.$

We call $\mathsf{\Pi }$ a subdivision of $\mathbb{R}$ and it is fixed throughout the paper.

A function $\chi :\mathbb{R}\to \mathbb{R}$ is called a kernel for the SK operator if:

(${\chi }_1$)

$\chi \in L^1\left(\mathbb{R}\right)$ and it is bounded in a neighborhood of the origin;

(${\chi }_2$)

${\sum }_{k\in \mathbb{Z}}\chi (u-t_k)=1$, for each $u\in \mathbb{R}$;

(${\chi }_3$)

there exists $\beta >0$, such that

${\displaystyle m_{\beta ,\mathsf{\Pi }}\left(\chi \right):=\underset{u\in \mathbb{R}}{sup}\sum _{k\in \mathbb{Z}}|\chi (u-t_k)\left|\right|u-t_k{|}^{\beta }<+\infty }$, i.e., the discrete absolute moments of order $\beta$ are finite.

Definition 1. 

Given a kernel χ, the family

$$\left(K_{w}f\right)\left(x\right):={\sum }_{k\in \mathbb{Z}}\chi (wx-t_k)\left[{\displaystyle \frac{w}{{\Delta }_k}{\int }_{\frac{t_k}{w}}^{\frac{t_{k+1}}{w}}f\left(u\right)du}\right]$$

for $x\in \mathbb{R}$, $w>0$ and where $f:\mathbb{R}\to \mathbb{R}$ is a locally integrable function such that the above series converges for each $x\in \mathbb{R}$, defines the sampling Kantorovich operators.

It is possible to extend the same family of operators in a multidimensional setting ([22]). The Definition 1 is given in the general context of non-uniform sampling schemes, but it includes as a particular case the equi-spaced sampling, often used in the applications.

Examples of well-known kernels satisfying the above assumptions and used in the implementation of the SK operators, in one-dimensional as well in multidimensional settings, are given in [10,11,13,23,24].

SK are approximation operators, able to reconstruct pointwise, continuous, and bounded functions, and uniformly, functions which are uniformly continuous and bounded, as $w\to +\infty$, (see, e.g., Refs. [15,22,25,26]).

Convergence theorems stand (see, e.g., Ref. [22]).

Theorem 2. 

Let χ be a kernel for the operators defined in Definition 1 and let $f:{\mathbb{R}}^n\to \mathbb{R}$ be a continuous and bounded function. Then, for every $\underline{x}\in {\mathbb{R}}^n$,

$$\underset{w\to +\infty }{lim}\left(K_{w}f\right)\left(\underline{x}\right)=f\left(\underline{x}\right).$$

Moreover, if $f\in C\left({\mathbb{R}}^n\right)$, then

$$\underset{w\to +\infty }{lim}{\parallel K_{w}f-f\parallel }_{\infty }=0,$$

where ${\parallel ·\parallel }_{\infty }$ denotes the usual sup-norm.

Moreover, the operators $K_w$ are suitable also to reconstruct not necessarily continuous signals, as happens, e.g., in $L^p$-setting. For this aim, it is possible to formulate the following theorem.

Theorem 3. 

Let χ be a kernel for the operators defined in Definition 1. For every $f\in L^p\left({\mathbb{R}}^n\right)$, $1\le p<+\infty$, there holds

$$\underset{w\to +\infty }{lim}{\parallel K_{w}f-f\parallel }_p=0,$$

where ${\parallel ·\parallel }_p$ denotes the usual $L^p$-norm.

For the above $SK$ operators, results concerning rate of approximation, saturation order, and inverse theorems, can be found in [18,27,28,29]. Moreover, the above operators have been implemented for image reconstruction and enhancement and used for various applications (see, e.g., Refs. [24,30]), exploiting their low pass filtering properties (see, e.g., Ref. [31]).

3. Sampling Kantorovich by Difference

From now on, our model focuses on the averages of the sampled values and not on the kernels of the operators. In fact, the proposed method, which uses multiple measurements, allows us to reconstruct the signal/image, working only on the sampled values of the signal.

With the expression Sampling Kantorovich by Difference (SKD), we design the sampling process that measures the differences between mean values in repeated shifted acquisition. Using the formalism introduced in the previous section, we start working in a one-dimensional setting, introducing two sequences of the domain of the signal $f,$ i.e., ${\mathsf{\Pi }}_A={\left(t_{k_A}\right)}_{k_A\in \mathbb{Z}}$, ${\mathsf{\Pi }}_B={\left(t_{k_B}\right)}_{k_B\in \mathbb{Z}}$, such that $t_{k_B+h}<t_{k_A+h},$ for every $k_A,k_B,h\in \mathbb{Z}$ (see Figure 1).

In the equi-spaced case, with

$$t_{k_A+h}=\frac{t_{k_B+h+1}+t_{k_B+h}}{2}$$

and

$$t_{k_B+h}=\frac{t_{k_A+h}+t_{k_A+h-1}}{2},$$

for a locally integrable function f, the following equality stands

$$\frac{w}{t_{k_B+1}-t_{k_A}}{\int }_{\frac{t_{k_A}}{w}}^{\frac{t_{k_B+1}}{w}}f\left(u\right)du=2\left(\frac{w}{\Delta k_B}{\int }_{\frac{t_{k_B}}{w}}^{\frac{t_{k_B+1}}{w}}f\left(u\right)du\right)-\frac{w}{t_{k_A}-t_{k_B}}{\int }_{\frac{t_{k_B}}{w}}^{\frac{t_{k_A}}{w}}f\left(u\right)du,$$

(1)

where the first term on the left side is the mean value of f in $[\frac{t_{k_A}}{w},\frac{t_{k_B+1}}{w}]$, the first term on the right side is the mean value of f in $[\frac{t_{k_B}}{w},\frac{t_{k_B+1}}{w}]$ (multiplied by the constant 2) and the last term, the mean value of f in $[\frac{t_{k_B}}{w},\frac{t_{k_A}}{w}].$

We denote by

$$M_{{\mathsf{\Pi }}_A}\left(\frac{t_{k_A}}{w}\right):=\frac{w}{{\Delta }_{k_A}}{\int }_{\frac{t_{k_A}}{w}}^{\frac{t_{k_A+1}}{w}}f\left(u\right)du,$$

$$M_{{\mathsf{\Pi }}_B}\left(\frac{t_{k_B}}{w}\right):=\frac{w}{{\Delta }_{k_B}}{\int }_{\frac{t_{k_B}}{w}}^{\frac{t_{k_B+1}}{w}}f\left(u\right)du,$$

where ${\Delta }_{k_A}=\frac{t_{k_A+1}-t_{k_A}}{w}$ and ${\Delta }_{k_B}=\frac{t_{k_B+1}-t_{k_B}}{w},$ and by

$$M_0:=M_0\left(\left[\frac{t_{k_B}}{w},\frac{t_{k_A}}{w}\right]\right)=\frac{w}{t_{k_A}-t_{k_B}}{\int }_{\frac{t_{k_B}}{w}}^{\frac{t_{k_A}}{w}}f\left(u\right)du.$$

From now on, for simplicity of notation, we denote $M_{{\mathsf{\Pi }}_A}\left(\frac{t_{k_A}}{w}\right)=M_A\left(\frac{t_{k_A}}{w}\right)$ and $M_{{\mathsf{\Pi }}_B}\left(\frac{t_{k_A}}{w}\right)=M_B\left(\frac{t_{k_B}}{w}\right).$

The values $M_A$ and $M_B$ are the results of a measurement process, in which the acquired values are means of the input. To a certain extent, in the most of the practical cases, the measurement process is reduced to measure a mean of the input, in space or in time. Now, by $\left(1\right),$ if $M_0$ is known, it is possible to recursively calculate, by difference, the integral mean of f in the intervals $[\frac{t_{k_A}}{w},\frac{t_{k_B+1}}{w}],[\frac{t_{k_B+1}}{w},\frac{t_{k_A+1}}{w}]$, and so on, such to achieve a resolution of the measurements depending on the shifting distance $\epsilon$ between the samples of the two subdivisions grouped together. Similar procedures are commonly used in the framework of super resolution type techniques ([32]).

A simple example of SKD is given by multiple measurements performed with the same sensor (built by contiguous cells of sizes d) shifted in space. We define a sensor by using an integral model, such that, given an input, the sensor returns to the output the mean value of the given input. Moreover, in our case we assume to be in an ideal situation, i.e., without any distortion or non-linearity. For this reason, given the aim of the work is the theoretical investigation of the SK model, we will use simulated data in Section 4. Between the two measurements, the sensor is shifted by $\frac{d}{2}$. We also assume the measured values of f depending on d, in a similar way to what happens with light intensity in photographic cameras. The process is schematized in Figure 2: the first measurement is performed with the sensor in position B (upper row); then the sensor is shifted to the right by $\epsilon =\frac{d}{2}$ and another measurement is performed in position A (second row). These two measurements provide the values of $M_B$ and $M_A$, respectively. The goal is to achieve a doubled resolution with respect to the size d.

Then,

$$M_1\left(\left[\frac{t_{k_A}}{w},\frac{t_{k_B+1}}{w}\right]\right)=2M_B\left(\frac{t_{k_B}}{w}\right)-M_0\left(\left[\frac{t_{k_B}}{w},\frac{t_{k_A}}{w}\right]\right),$$

$$M_2\left(\left[\frac{t_{k_B+1}}{w},\frac{t_{k_A+1}}{w}\right]\right)=2M_A\left(\frac{t_{k_A}}{w}\right)-\left[2M_B\left(\frac{t_{k_B}}{w}\right)-M_0\left(\left[\frac{t_{k_B}}{w},\frac{t_{k_A}}{w}\right]\right)\right].$$

Same equations can be achieved iterating, obtaining, for $p\in \mathbb{N},$

$$M_p\left(\left[\frac{t_{k_B+p/2}}{w},\frac{t_{k_A+p/2}}{w}\right]\right)=$$

$$=2M_A\left(\frac{t_{k_A+p/2-1}}{w}\right)-M_{p-1}\left(\left[\frac{t_{k_A+(p-2)/2}}{w},\frac{t_{k_B+p/2}}{w}\right]\right),\mathrm{i}\mathrm{f} p \mathrm{i}\mathrm{s} \mathrm{e}\mathrm{v}\mathrm{e}\mathrm{n}$$

$$M_p\left(\left[\frac{t_{k_A+(p-1)/2}}{w},\frac{t_{k_B+(p+1)/2}}{w}\right]\right)=$$

$$=2M_B\left(\frac{t_{k_B+(p-1)/2}}{w}\right)-M_{p-1}\left(\left[\frac{t_{k_B+(p-1)/2}}{w},\frac{t_{k_A+(p-1)/2}}{w}\right]\right),\mathrm{i}\mathrm{f} p \mathrm{i}\mathrm{s} \mathrm{o}\mathrm{d}\mathrm{d}$$

with $M_0$ as above defined, i.e., the applicability of the method is represented by the knowledge of $M_0.$

Indeed, in practice, $M_0$ should be measured by a sensor of size d. This implies that $M_0$ is known if, e.g., f is constant on the interval $[\frac{t_{k_B}}{w},\frac{t_{k_B+1}}{w}]$ or the mean of f in $[\frac{t_{k_B}}{w},\frac{t_{k_A}}{w}]$ and in $[\frac{t_{k_A}}{w},\frac{t_{k_B+1}}{w}]$ is equal to $M_0$.

Therefore, from now on, we will define the following “far field” condition.

Definition 2. 

The signal f satisfies the far field condition on the interval $I\subset \mathbb{R}$, with $I=I_1\cup I_2$, $I_1^{\circ }\cap I_2^{\circ }=\varnothing$, if either of the two statements below hold true:

• $f\left(x\right)=C,C\in \mathbb{R},$ for every $x\in I,$
• $\frac{1}{|I_1|}{\int }_{I_1}f\left(x\right)dx=\frac{1}{|I_2|}{\int }_{I_2}f\left(x\right)dx,$

where by $|·|$ we denote the measure of the considered interval.

Note that the above far field condition consists in knowing the numeric value for $M_0,$ from which the calculation of $M_A$ and $M_B$ follows.

From the application point of view the far field conditions can be always achieved, using suitable technical solutions. Note that the above condition can be formulated in multidimensional settings, extending the reasoning in more variables. Moreover, we highlight that in the present setting, the partitions are not necessarily constituted by integers numbers, i.e., in general $t_k\ne k,k\in \mathbb{Z},$ as could happen in practical situations.

Remark 2. 

We remark that, since the two sequences ${\mathsf{\Pi }}_A={\left(t_{k_A}\right)}_{k_A\in \mathbb{Z}}$, ${\mathsf{\Pi }}_B={\left(t_{k_B}\right)}_{k_B\in \mathbb{Z}}$, are equally spaced each other, we could have used a single partition and achieve a unique expression for $M_p$. We preferred to formalize the model by two distinct partitions to clearly distinguish the starting sequence and the one obtained by shifting the sensor. In this way, we focused more the attention on the measurement procedure which, in some cases (as happen in this one), can be based only on two distinct positions A and $B.$

Further, from a mathematical point of view, the single sequence formulation does not simplify too much the final expression but allows to achieve a more elegant result, expressed by a single recursive formula for $M_p$.

To use a single partition, for every fixed $k\in \mathbb{Z}$, we define the following sequence

$${\left(t_{k+p/2}\right)}_{p\in \mathbb{N}}\mathrm{with}{\Delta }_p:=t_{k+p/2}-t_{k+(p-1)/2}=\Delta ,for everyp\in \mathbb{N},$$

i.e., it is an equi-spaced partition.

We now denote by

$${\displaystyle M\left(\frac{t_{k+p/2}}{w}\right):=\frac{w}{2\Delta }{\int }_{\frac{t_{k+p/2}}{w}}^{\frac{t_{k+p/2+1}}{w}}f\left(u\right)du,}$$

and by

$$M_0\left(\left[\frac{t_k}{w},\frac{t_{k+1/2}}{w}\right]\right):=\frac{w}{\Delta }{\int }_{\frac{t_k}{w}}^{\frac{t_{k+1/2}}{w}}f\left(u\right)du.$$

Then, we may write

$$M_p\left(\left[\frac{t_{k+p/2}}{w},\frac{t_{k+(p+1)/2}}{w}\right]\right)=2M\left(\frac{t_{k+(p-1)/2}}{w}\right)-M_{p-1}\left(\left[\frac{t_{k+(p-1)/2}}{w},\frac{t_{k+p/2}}{w}\right]\right),$$

for every $p\in \mathbb{N}.$

4. Application of the SKD Model to a Super-Resolution-Type Problem

In this section, the proposed mathematical model is applied to a super resolution type problem in images, and we will always consider multiple acquisitions shifting the sensor by $\epsilon =d/2.$ The super resolution problem is an ill-posed problem whose goal is to obtain Higher Resolution (HR) images starting from Lower Frequency (LR) acquisitions. In case of super resolution from multiple images, the missing information is retrieved from repeated acquisitions of the same static scene. The data are then merged together on the same grid and some interpolation or quasi-interpolation methods, with their own kernels, are used to perform the final reconstruction [33] (see Figure 3).

Once the method and the kernel have been chosen, the reconstruction depends exclusively on the measured values.

In the following, our interest is focused on the measured values with different measurement methods: pointwise or average. We assume that the cells of which the measuring sensor is made, have square shape, as it happens in practice. From now on, we refer to the classic and generalized sampling models as the WKSG model. The main difference between the SKD and the WKSG models is that the first one uses the ${\displaystyle w{\int }_{\frac{k}{w}}^{\frac{k+1}{w}}f\left(u\right)du}$ mean values while the second one employs the point values $f\left(\frac{k}{w}\right)$ (in this application $t_k=k,k\in \mathbb{Z}$). Of course, the application of the results achieved in the one-dimensional case previously investigated, can be extended to the two-dimensional case, with the same procedure of shifting along the x and y axes.

All the results below presented have been achieved writing codes in Python, vers. 3.9.13. One set of six HR images of artworks (resolution $H\times W$) has been used. To avoid the non-linearity of the Analog to Digital (A/D) converters, the measurement data are simulated, such that the HR images are halved in size, performing a mean over each $2\times 2$ square of not overlapping pixels. The mean process then, artificially generates four different images, each one of size $\frac{H}{2}\times \frac{W}{2}$, (see Figure 4). We denote the values of the pixels in the new smaller images with $M_R(i,j),M_B(i,j),M_G(i,j),M_Y(i,j)$, where the letters $R,B,G,Y$ are connected with the color of the grid used in the meaning (M) process (see Figure 4 again).

The halving process, to which the WKSG model is applied, is made through the following equations, which link the original image (HR) and the halved image (LR).

Here, they follow,

$$ifioddandjodd,M_R\left(\frac{i+1}{2},\frac{j+1}{2}\right)=\frac{I(i,j)+I(i,j+1)+I(i+1,j)+I(i+1,j+1)}{4},$$

$$ifioddandjeven,M_B\left(\frac{i+1}{2},\frac{j}{2}\right)=\frac{I(i,j)+I(i,j+1)+I(i+1,j)+I(i+1,j+1)}{4},$$

$$ifievenandjodd,M_G\left(\frac{i}{2},\frac{j+1}{2}\right)=\frac{I(i,j)+I(i,j+1)+I(i+1,j)+I(i+1,j+1)}{4},$$

$$ifievenandjeven,M_Y\left(\frac{i}{2},\frac{j}{2}\right)=\frac{I(i,j)+I(i,j+1)+I(i+1,j)+I(i+1,j+1)}{4},$$

where I is the $W\times H$ pristine version of a generic low-frequency image in the set. The previous equalities assume that the nodal values (point values) are given by the means: this approximation is tacitly assumed in application models. The implementation of the WKSG model in terms of measured values highlights, by comparison, the errors coming from this assumption, as in Figure 5c, with the indication of the $PSNR$ (see, e.g., Ref. [24]), calculated compared to the original HR corresponding image. In the test set, a constant frame is surrounding the image of the paint, such that the “far field” condition is satisfied. If any approximation (rounding) is avoided in the mean process, the SKD model provides exact results ($MSE=0,PSNR=+\infty$); see Figure 5d,e and Figure 6e,f.

Indeed, each equation given in the system of Section 4 needs a division by 4 (since it is a mean): this is a source of error, because the result of the division could be a non-integer number affecting, the same way, the values calculated for I after the reconstruction process. On the other hand, I must assume integer values (e.g., 256 levels in grayscale images). To avoid this source of error, in our specific case, it is enough to multiply by 4 the given equations: this is equivalent to request that the measurement equipment (e.g., a photographic camera) has a number of quantization levels L (i.e., a precision in the measurements, e.g., the number of gray levels) four times the resolution of the native instrument. E.g., in the case here examined, where $P=256$ (number of gray levels), L must be chosen such that $L=4P$.

As expected, the main differences between the two models are more evident in the areas of the image where the signal exhibits higher variations, while they are null where the signal is constant (see Figure 5f). Further examples are provided in Appendix A.

We finally remark that it is understood that the above considerations can be subsequently applied to the corresponding operators, for example to further rescale the image, and obviously the measurement methods will have their impact on the operators, e.g., on the rescaling process.

5. Conclusions

The introduction of the SK operators formulation to model the sampling procedure in the applications allows to define the concept of Sampling Kantorovich by Difference, thanks to which the resolution of a measurement in a sampling procedure can be connected not only to the geometrical size of the sensor cells, but also to its shifting, assuming the “far field” condition to be satisfied. The most important part of the SK operators, compared with the WKS sampling theorem and the generalized approach, stands in the introduction of the integral averages.

This average offers a modeling more descriptive of the sampling procedure if compared with the ones following from the WKS and the generalized sampling formulation. In fact, these two models assume that the result of a sampling procedure is a pointwise value that is going to be extended as a constant in the given pixel. Even if this assumption is justified by technological reasons (e.g., to each pixel only a single value of brightness can be associated, exploiting the limited resolution power of the human eye), it does not seem to be the most suitable one, being the measurement a meaning process. Eliminating the above assumption in the model, i.e., using the SKD, brings to better reconstructions of the unknown sampled function in the super resolution from multiple acquisitions problem.

The SKD way of re-thinking the sampling process could open new technical applications in the sampling scenario.