Neural-Network-Based Curve Fitting Using Totally Positive Rational Bases

Abstract

This paper proposes a method for learning the process of curve fitting through a general class of totally positive rational bases. The approximation is achieved by finding suitable weights and control points to fit the given set of data points using a neural network and a training algorithm, called AdaMax algorithm, which is a first-order gradient-based stochastic optimization. The neural network presented in this paper is novel and based on a recent generalization of rational curves which inherit geometric properties and algorithms of the traditional rational Bézier curves. The neural network has been applied to different kinds of datasets and it has been compared with the traditional least-squares method to test its performance. The obtained results show that our method can generate a satisfactory approximation.

1. Introduction

The problem of obtaining a curve that fits a given set of data points is one of the fundamental challenges of Computer Aided Geometric Design (CAGD), and it has become prevalent in several applied and industrial domains, such as Computer-Aided Design and Manufacturing (CAD/CAM) systems, Computer Graphics and Animation, Robotics Design, Medicine and many others. To face this issue, several families of bases of functions have been considered. There is a large body of literature on this topic and there are numerous methods to solve this issue, such as several least-squares techniques and different progressive iterative approximation methods (see [1,2,3] and the references therein).

Given a system $(u_0,\dots ,u_n)$ of linearly independent functions defined on an interval $I\subseteq \mathbb{R}$ and $P_0,\dots ,P_n\in {\mathbb{R}}^k$, we can define a parametric curve as $\gamma \left(t\right)={\sum }_{i=0}^n{P}_i{u}_i\left(t\right)$, $t\in I$. The polygon $P_0\cdots P_n$, formed by the ordered sequence of points $P_i\in {\mathbb{R}}^k$, $i=0,\dots ,n$, is called the control polygon of γ and the points $P_i$, $i=0,\dots ,n$, are named the control points of $\gamma$ with respect to $(u_0,\dots ,u_n)$. A matrix is totally positive (TP) if all its minors are nonnegative (see [4]). A system of functions $(u_0,\dots ,u_n)$ defined on I is TP if all its collocation matrices ${\left(u_j\left(t_i\right)\right)}_{i,j=0\dots ,n}$ with $t_0<\cdots <t_n$ in I are TP. A TP system of functions on I is normalized (NTP) if ${\sum }_{i=0}^n{u}_i\left(t\right)=1$, for all $t\in I$. A basis provides shape-preserving representations if the shape of the curve imitates the shape of its control polygon. Normalized totally positive bases provide shape-preserving representations. The normalized B-basis of a given space is an NTP basis such that the matrix of change of basis of any NTP basis with respect to the normalized B-basis is TP and stochastic. This property implies that the control polygon of a curve with respect to the normalized B-basis can be obtained by a corner cutting algorithm from its control polygon with respect to any other NTP basis. Thus, the control polygon with respect to the normalized B-basis is closer in shape to the curve than the control polygon with respect to any other NTP basis. Furthermore, the length of the control polygon with respect to the normalized B-basis lies between the length of the curve and the length of its control polygon with respect to any other NTP basis. Similar properties hold for other geometric properties such as angular variation or number of inflections (see [5,6,7]). So, the Normalized B-basis has the optimal shape-preserving properties among all NTP bases of the space. The Bernstein bases and the B-spline bases are the normalized B-bases of their corresponding generated spaces.

It is well known that the bases obtained by rationalizing Bernstein bases are also the normalized B-bases of the generated spaces of rational functions. These spaces are made up of rational polynomial functions where the denominator is a given polynomial. Rational Bernstein bases add adjustable weights to provide closer approximations to arbitrary shapes and have become a standard tool in CAGD since they allow the exact representation of conic sections, spheres and cylinders. In [8], the generalization of rational Bernstein bases obtained when replacing the linear polynomial factors by trigonometric or hyperbolic functions or their mixtures with polynomials were analyzed. The generated rational curves inherit geometric properties and algorithms of the traditional rational Bézier curves and so, they can be considered as modeling tools in CAD/CAM systems.

As mentioned before, the weights of rational bases can be used as shape parameters. However, it is well known that the effect of changing a weight in a rational basis is different from that of moving a control point of the curve (see Figure 1). Thus, the interactive shape control of rational curves through adjusting weights is not a straightforward task and it is not easy to design algorithms to obtain the appropriate weights (see [9], Chapter 13).

Some recent papers have shown that the application of Artificial Intelligence (AI) techniques can achieve remarkable results regarding the problem of obtaining rational curves that fit a given set of data points. To face this issue, in [10], a bio-inspired algorithm was applied through the use of rational Bézier curves. Besides, in [11,12], evolutionary algorithms were applied to rational B-spline curves. As a novelty, in this paper, we define a one-hidden-layer neural network using the general class of rational bases with optimal shape-preserving properties proposed in [8]. In that work, the authors presented evaluation and subdivision algorithms. However, this is the first time that the problem of obtaining a rational fitting curve using these general class of totally positive rational bases is modeled by a neural network and its weights and control points optimized using a training algorithm. In this paper, we extend [8] for their application in curve fitting training the neural network with a recent stochastic learning process, the AdaMax algorithm [13], to find suitable weights and control points. In this approximation process, the rational basis is a hyperparameter and can be changed by substituting the linear factors by polynomial, trigonometric or hyperbolic functions, thus expanding the potential range of applications to include more difficult shapes.

The layout of the paper is as follows. In Section 2, we recall several concepts regarding CAGD and we present a general class of rational bases which are normalized B-bases. Then, in  Section 3, we present a one-hidden-layer neural network based on the rational bases presented in the previous section to approximate a given set of data points. This neural network is trained with an optimization algorithm to update the weights and control points used to construct a curve that approximates the given set of data points, while decreasing a loss function. In Section 4, several experiments are provided illustrating the use of the neural network with different normalized B-bases to test its performance giving an approximation of different kinds of sets of data points. Moreover, the proposed method has been compared with the traditional least-squares method. Finally, conclusions and future work are presented in Section 5.

2. Shape-Preserving and Rational Bases

Let us suppose that $I\subseteq \mathbb{R}$ and $f,g:I\to \mathbb{R}$ are nonnegative continuous functions. Then, for $n\in \mathbb{N}$, we can define the system $(u_0^n,\dots ,u_n^n)$ where:

$$u_k^n\left(t\right)=\left(\genfrac{}{}{0pt}{}{n}{k}\right)f^k\left(t\right)g^{n-k}\left(t\right)\text{such}\text{that}t\in I,k=0,\dots ,n.$$

(1)

For any positive weights $w_i^n$, $i=0,\dots ,n$, let us define ${\omega }^n\left(t\right)={\sum }_{i=0}^n{w}_i^n{u}_i^n\left(t\right)$ and denote by $({\rho }_0^n,\dots ,{\rho }_n^n)$ the rational basis described by

$${\rho }_i^n\left(t\right)=w_i^n\frac{1}{{\omega }^n\left(t\right)}{u}_i^n\left(t\right),i=0,\dots ,n,$$

(2)

where $(u_0^n,\dots ,u_n^n)$ is defined in (1). Clearly, this system spans a space of rational functions with denominator ${\omega }^n\left(t\right)$,

$${\mathcal{R}}^n=\mathrm{span}\{{\rho }_i^n\left(t\right)\mid i=0,\dots ,n\}=\{u\left(t\right)/{\omega }^n\left(t\right)\mid u\left(t\right)\in {\mathcal{U}}^n\},$$

(3)

where ${\mathcal{U}}^n$ is the space generated by $(u_0^n,\dots ,u_n^n)$.

The following result corresponds to Corollary 4 of [8] and provides the conditions characterizing that the system given in (2) has optimal shape-preserving properties.

Proposition 1.

The system of functions given in (2) is the normalized B-basis of the space ${\mathcal{R}}^n$ defined in (3) if and only if the function $f/g$ defined on $I_0=\{t\in I\mid g\left(t\right)\ne 0\}$ is increasing and satisfies

$$inf\left\{\frac{f\left(t\right)}{g\left(t\right)}\mid t\in I_0\right\}=0,sup\left\{\frac{f\left(t\right)}{g\left(t\right)}\mid t\in I_0\right\}=+\infty .$$

(4)

Let us see several choices of functions f and g satisfying the conditions of Proposition 1. Let us consider the functions

$$f\left(t\right)=\frac{t-a}{b-a},g\left(t\right)=\frac{b-t}{b-a},t\in [a,b].$$

(5)

It is well known that the corresponding rational basis (2), which is the rational Bernstein basis, is the normalized B-basis of its generated space (3).

We can also consider the functions

$$f\left(t\right)=t^2,g\left(t\right)=1-t^2,t\in [0,1].$$

(6)

The corresponding rational basis (2) spans the space

$${\mathcal{R}}^n=\mathrm{span}\{u\left(t\right)/{\omega }^n\left(t\right)\mid u\left(t\right)\in {\mathcal{U}}^n,{\omega }^n\left(t\right)=\sum _{i=0}^n{w}_i^n{u}_i^n\left(t\right)\},$$

where the system $(u_0,\dots ,u_n)$ given in (1) spans the space $\langle 1,t^2,\dots ,t^{2n}\rangle$ of even polynomials defined on $[0,1]$ of degree less than or equal to $2n$.

Trigonometric and hyperbolic bases are attracting a lot of interest, for instance in Isogeometric Analysis (cf. [14]). Let $0<\Delta <\pi /2$. Define

$$f\left(t\right)=sin\left((\Delta +t)/2\right) \mathrm{and} g\left(t\right)=sin\left((\Delta -t)/2\right) \mathrm{for} t\in I=[-\Delta ,\Delta ].$$

(7)

Let us notice that the functions f and g satisfy $f\left(t\right)>0$ and $g\left(t\right)>0$ for all $t\in (-\Delta ,\Delta )$. Moreover, it can be checked that

$${\left(\frac{f\left(t\right)}{g\left(t\right)}\right)}^{\prime }={\left(\frac{sin\left(\frac{\Delta +t}{2}\right)}{sin\left(\frac{\Delta -t}{2}\right)}\right)}^{\prime }=\frac{1}{2}\frac{sin(\Delta )}{{sin}^2\left(\frac{\Delta -t}{2}\right)}>0,\forall t\in (-\Delta ,\Delta ).$$

(8)

Therefore, for any $0<\Delta <\pi /2$, the function $f/g$ is a strictly increasing function on $(-\Delta ,\Delta )$ and f and g satisfy the conditions of Proposition 1. The corresponding rational basis (2) spans the space

$${\mathcal{R}}^n=\mathrm{span}\{u\left(t\right)/{\omega }^n\left(t\right)\mid u\left(t\right)\in {\mathcal{U}}^n,{\omega }^n\left(t\right)=\sum _{i=0}^n{w}_i^n{u}_i^n\left(t\right)\},$$

where, for a given $n=2m$, the system $(u_0,\dots ,u_n)$ given in (1) is a basis that coincides, up to a positive scaling, with  the normalized B-basis of the space $\langle 1,cost,sint,\dots ,cosmt,sinmt\rangle$ of trigonometric polynomials of degree less than or equal to m on I (see Section 3 of [15]).

Finally, let $\Delta >0$. Define

$$f\left(t\right)=sinh\left((\Delta +t)/2)\right) \mathrm{and} g\left(t\right)=sinh\left((\Delta -t)/2\right) \mathrm{for} t\in I=[-\Delta ,\Delta ].$$

(9)

Clearly, $f\left(t\right)>0$ and $g\left(t\right)>0$ for all $t\in (-\Delta ,\Delta )$. Moreover, it can be checked that

$${\left(\frac{f\left(t\right)}{g\left(t\right)}\right)}^{\prime }={\left(\frac{sinh\left(\frac{\Delta +t}{2}\right)}{sinh\left(\frac{\Delta -t}{2}\right)}\right)}^{\prime }=\frac{1}{2}\frac{sinh\Delta }{{sinh}^2\left(\frac{\Delta -t}{2}\right)}>0,\forall t\in (-\Delta ,\Delta ).$$

(10)

Therefore, for any $\Delta >0$, $f/g$ is a strictly increasing function on $(-\Delta ,\Delta )$ and f and g satisfy the conditions of Proposition 1. The corresponding rational basis (2) spans the space

$${\mathcal{R}}^n=\mathrm{span}\{u\left(t\right)/{\omega }^n\left(t\right)\mid u\left(t\right)\in {\mathcal{U}}^n,{\omega }^n\left(t\right)=\sum _{i=0}^n{w}_i^n{u}_i^n\left(t\right)\},$$

where, for $n=2m$, the system $(u_0,\dots ,u_n)$ given in (1) spans the space $\langle 1,e^t,e^{-t},\dots ,e^{mt},e^{-mt}\rangle$ of hyperbolic polynomials of degree less than or equal to m on I.

In Figure 2, we illustrate two examples of the rational basis (2) of degree 3. Let us observe the effect on the shape of the functions of the basis as weights change.

Section 4 will show examples of approximations of given sets of data points using all the above mentioned normalized B-bases. Moreover, the neural network presented in the following section will be used to compute the optimal weights and control points of their corresponding fitting curves.

3. Curve Fitting with Neural Networks

It is well known that curve fitting is the process of constructing a curve, or mathematical function, that has the best fit of a given set of data points. A related topic is Regression Analysis in Machine Learning. In the literature (see [16] (Chapter 11)), a regression problem is the problem of predicting a real value for each input data. Let us consider an input space $X\subset {\mathbb{R}}^m$ and target values $Y\subset {\mathbb{R}}^k$, and a distribution over $X\times Y$, denoted by $\mathcal{D}$. Then, a regression problem consists of a set of labeled samples $S={\left\{(x_i,y_i)\right\}}_{i\in \{0,\cdots ,\ell \}}\in X\times Y$ drawn according to $\mathcal{D}$  where $y_i$ are the target real values we want to predict. There exists a huge variety of regression algorithms, i.e., algorithms solving regression problems, such as Linear Regression, Decision Trees, Support Vector Regression, and Neural Networks, among others. The quality of the prediction of an algorithm or model depends on the difference between the target (i.e., the true value) and the predicted one, and it is measured using a loss function. Then, given a set $\mathcal{H}$ (also called “hypothesis”) of function mappings X to Y, the aim of the regression algorithm is to use S to find $h\in \mathcal{H}$ such that the expected loss is small.

Specifically, the problem that we want to solve can be stated as follows. Suppose that f and g are functions defined on $[a,b]$ satisfying the conditions of Proposition 1. Consider a set of parameters $a\le t_0<\cdots <t_{\ell }\le b$ and a sequence of data points $s_0,\dots ,s_{\ell }\in {\mathbb{R}}^k$, where each parameter $t_i$ is associated with a data point $s_i$. For some $n\le \ell$, we want to obtain a rational curve

$$c\left(t\right)=\sum _{i=0}^n\frac{w_i^n\left(\genfrac{}{}{0pt}{}{n}{i}\right)f^i\left(t\right)g^{n-i}\left(t\right)}{{\sum }_{i=0}^n{w}_i^n\left(\genfrac{}{}{0pt}{}{n}{i}\right)f^i\left(t\right)g^{n-i}\left(t\right)}{P}_i,t\in [a,b],$$

(11)

to approximate the set of data points $s={\left(s_i\right)}_{i=0}^{\ell }$. Therefore, the goal is to obtain the weights $w_0^n,\cdots ,w_n^n$ and the control points $P_0,\cdots ,P_n$ of the rational curve (11) that best fits the set of data points. In order to compute them, we have used a stochastic optimization process to train a neural network that models the rational curve $c\left(t\right)$.

The problem to be solved can be interpreted then as a regression problem where the set of labeled samples is composed of the input data, X, that is the set of parameters $a\le t_0<\cdots \le t_{\ell }\le b$ and the target set of data points $Y=s={\left(s_i\right)}_{i=0}^{\ell }$.

Then, the expression in (11) can be represented as a hierarchical computational graph with just one hidden layer that we will denote as ${\mathcal{N}}_{w,P}:\mathbb{R}\to {\mathbb{R}}^k$ where the computations are organized as in Figure 3. The obtained curve ${\mathcal{N}}_{w,P}\left(t\right)$ is the rational curve $c\left(t\right)$ that approximates the given set of data points and we denote as the fitting curve.

The key idea is to iteratively change the input weights $w={\left(w_i^n\right)}_{i=0}^n$ and control points $P={\left(P_i\right)}_{i=0}^{\ell }$ of the active curve ${\mathcal{N}}_{w,P}\left(t\right)$, and so it deforms towards the target shape represented by the set of data points $s={\left(s_i\right)}_{i=0}^{\ell }$ (see Figure 4).

Then, we apply an adaptive learning rate optimization algorithm to train the neural network to find the weights and control points, which can be, for example, the Adaptive Moment Estimation (Adam) algorithm or its variant Adaptive Moment Estimation Maximum (AdaMax) algorithm based on infinity norm. These methods are used for stochastic optimization, to solve the supervised learning problem and to find the parameters where a minima is located. In this paper, we have mainly used the AdaMax variant (see Algorithm 1) because of its stability and simplicity [13]. However, the Adam method can be useful depending on the shape of the set of data points to be approximated and the choice of the loss function. The stochastic objective function, also called the loss function, measures the goodness of the fitting curve. Let us notice that there exist different loss functions such as the mean absolute error, the cross-entropy loss, the mean squared error, among others (the different loss functions implemented in Tensorflow can be consulted in the tensorflow documentation), that can be chosen depending on the problem. In our case, we have considered the mean absolute error as the loss function because of the choice of the training algorithm, given by the following expression:

$$E(w,P)=\frac{{\sum }_{i=0}^{\ell }\left|s_i-{\mathcal{N}}_{w,P}\left(t_i\right)\right|}{\ell +1}.$$

(12)

The Adam and the AdaMax algorithms are stochastic gradient-based optimization algorithms and, as previously mentioned, they update the weights and the control points iteratively. The step size is a real number that measures how much the weights and the control points are updated upon each iteration. Besides, the Adam algorithm uses the first and the second moment estimate to update the weights and the control points which are updated following exponential decay rates (${\beta }_1$ and ${\beta }_2$). Finally, as AdaMax is a variation of Adam using infinity norm, the second moment estimate has a simple recursive formula which will be denoted in Algorithm 1 as exponentially weighted infinity norm. See [13] for a detailed description of the both Adam and AdaMax algorithms.    

Algorithm 1: The AdaMax algorithm [13] adapted to our context.

The number of units (i.e. weights and control points) is a hyperparameter and is determined based on the complexity of the shape to be approximated. Besides, the step size, $\alpha$, can be changed depending on the state of the convergence of the training procedure, for example, when the loss values (i.e., the evaluation of the loss function) gets stuck or the update of the parameters is too big. Then, it is useful to increase or reduce, respectively, the step size according to the values of the loss function.

4. Experiments Results

In order to show the performance of the neural network ${\mathcal{N}}_{w,P}$, we have taken different sets of data points $s={\left(s_i\right)}_{i=0}^{\ell }$. They have been chosen to reflect the variety of situations where the proposed neural network can be applied. The first set of data points belongs to a closed conic curve, the second one belongs to a transcendental curve, the third one is a curve with a twisted shape and, finally, the fourth one is a noisy set of data points from a baroque image.

In all cases, we have taken different functions f and g satisfying the conditions of Proposition 1 and allowing that the corresponding rational bases (2) have the optimal shape-preserving properties.

Remark 1.

One of the requirements for a rational basis to be the normalized B-basis of its generated space is the positivity of the weights. This makes it necessary to apply the absolute value to the weights in the weight update step of Algorithm 1. However, in the experiments shown, we opted to avoid this choice because we have observed that, although in the intermediate steps the weights could be negative, at the end of the training, the weights were positive and the convergence was faster. Nevertheless, we can add the restriction depending on the needs.

Let us notice that we have used an uniform distribution of the parameters $t_i$, $i=0,\dots ,\ell$, in all the examples. This choice of parameters does not have to be the optimal but we have preferred it because of its simplicity. Besides, a normalization of the data points, $s={\left(s_i\right)}_{i=0}^{\ell }$, between 0 and 1, was applied in order to facilitate the training procedure following the formula:

$${\widehat{s}}_i=\frac{s_i-minS}{maxS-minS},\mathrm{for}i\in \{0,\cdots ,\ell \}.$$

(13)

The AdaMax algorithm was applied to solve the minimization problem with the mean absolute error loss function (12) with the following choice of hyperparameters: $\alpha =0.0001,{\beta }_1=0.9,{\beta }_2=0.999,\mathrm{and}\epsilon ={10}^{-7}$. Then, the number of iterations of the algorithm depends on the desired accuracy of the fitting curve to the set of data points. In this case, we have used between 3000 to 5000 iterations of the algorithm but, with more iterations and a better tuning of the parameters, the results provided here may be improved. Finally, in order to reach a better approximation, the first and the last control point of the fitting curve were fixed to be the same as the first and the last point of the set of data points, thus the obtained fitting curve is always exactly at those points. We can see in

Table 1. Loss values of the mean absolute error (12) for different fitting curves of degree n with $f\left(t\right)=t$, $g\left(t\right)=1-t$, $t\in [0,1]$ (Basis 1), $f\left(t\right)=t^2$ and $g\left(t\right)=1-t^2$, $t\in [0,1]$, (Basis 2), $f\left(t\right)=sin\left((\Delta +t)/2\right)$ and $g\left(t\right)=sin\left((\Delta -t)/2\right)$, $\Delta <\pi /2$, $\Delta <\pi /2$, $t\in [-\Delta ,\Delta ]$, (Basis 3) and finally $f\left(t\right)=sinh\left((\Delta +t)/2\right)$ and $g\left(t\right)=sinh\left((\Delta -t)/2\right)$, $\Delta <\pi /2$, $t\in [-\Delta ,\Delta ]$, (Basis 4). They were all trained with 4000 iterations, $\alpha =0.0001$, ${\beta }_1=0.9$, ${\beta }_2=0.999$, $\epsilon ={10}^{-7}$. The process was repeated 5 times, with the loss values provided being the best values reached.

n	Basis 1	Basis 2	Basis 3	Basis 4
Circle
3	3.3946 × ${10}^{-2}$	3.7129 × ${10}^{-2}$	7.0468 × ${10}^{-2}$	3.6438 × ${10}^{-2}$
4	2.1757 × ${10}^{-3}$	1.5338 × ${10}^{-2}$	3.1678 × ${10}^{-3}$	2.5582 × ${10}^{-3}$
5	1.7333 × ${10}^{-4}$	9.2269 × ${10}^{-3}$	2.8083 × ${10}^{-4}$	2.2488 × ${10}^{-3}$
Cycloid
8	1.0849 × ${10}^{-3}$	3.6855 × ${10}^{-4}$	3.6017 × ${10}^{-4}$	3.1674 × ${10}^{-4}$
9	4.6163 × ${10}^{-4}$	3.6855 × ${10}^{-4}$	3.6017 × ${10}^{-4}$	2.4914 × ${10}^{-4}$
10	3.3944 × ${10}^{-4}$	3.6855 × ${10}^{-4}$	3.6017 × ${10}^{-4}$	2.4914 × ${10}^{-4}$
Archimedean spiral
11	1.5982 × ${10}^{-3}$	1.0474 × ${10}^{-2}$	2.2349 × ${10}^{-2}$	7.8109 × ${10}^{-4}$
12	1.5982 × ${10}^{-3}$	7.8916 × ${10}^{-3}$	5.7801 × ${10}^{-3}$	7.8109 × ${10}^{-4}$
13	1.4106 × ${10}^{-3}$	5.2853 × ${10}^{-3}$	5.7801 × ${10}^{-3}$	7.8109 × ${10}^{-4}$

, a summary of the loss values from different fitting curves. Let us observe that the value n is the degree of the fitting curve and it depends on the complexity of the shape to be approximated. Moreover, let us notice that the proposed neural network is able to obtain a suitable accuracy with low degrees and, as a generalization of other methods, we can choose, depending on the shape of the set of data points, the basis that best fits. Note that, in CAGD, it is important to properly address the problem of curve fitting, finding a balance between accuracy and degree of the curve since high-degree curves are computationally expensive to evaluate. The AdaMax algorithm has been selected because it is computationally efficient with little memory requirements, suited for problems with large data or parameters. In 

Table 2. Time of execution of the proposed algorithm measured in seconds for different numbers of units and iterations. The values provided are the mean of 5 repetitions with a set of data points of size 100.

$\mathit{n}+1$	Number of Iterations
	1	25	50	100	3000
5	0.1259	1.4284	2.8381	5.7259	189.5386
10	0.0989	2.0781	4.1325	10.2672	268.8726
15	0.1244	2.7142	5.3781	10.9886	347.6139
50	0.6479	8.2589	13.3576	27.4398	850.6713
100	1.1624	14.3999	32.6576	65.3298	1521.3971

, the time of execution of the Algorithm 1 using different numbers of units (i.e., weights and control points) and numbers of iterations is provided.

The implementation (the code of the experimentation can be found in https://github.com/Cimagroup/Curve-approximation-NN) was developed using TensorFlow 2.0 [17] allowing developers to easily use it to build and deploy Machine Learning powered applications. All experiments were ran on a Quad-Core Intel Core i7 CPU, 2.8 GHz with 16 GB RAM. Let us see a detailed description of the experiments.

4.1. Circle Curve

Circles and conic sections play a relevant role in curve design and have been approximated in several works (see [18,19]). Let us show different approximations to the circle given by the parametric equations

$$\left\{\begin{array}{c}{x}_c\left(t\right)=cos\left(t\right),\hfill \\ y_c\left(t\right)=sin\left(t\right),\hfill \end{array}\right.$$

(14)

$t\in [0,2\pi ]$. First, let us see an approximation obtained performing the neural network ${\mathcal{N}}_{w,P}$ using polynomial functions.

We have taken the following sequence of points $s={\left(s_i\right)}_{i=0}^{99}$ on the circle (14):

$$s_i=(x_c\left(2\pi t_i\right),y_c\left(2\pi t_i\right)),i=0,\dots ,99,$$

being the parameters $t_i=i/99$, $i=0,\dots ,99$.

For $n=5$, we have approximated the set s by the fitting curve $({\mathcal{N}}_{w,P_x}\left(t\right),{\mathcal{N}}_{w,P_y}\left(t\right))$ considering the functions $f\left(t\right)=t$ and $g\left(t\right)=1-t$ at the vector nodes $t={\left(t_i\right)}_{i=0}^{99}$ with $t_i$ as above. After training the neural network ${\mathcal{N}}_{w,P}$, performing the Algorithm 1 for 3000 iterations, we have obtained the following weights:

$$\begin{array}{cc}\hfill & w_0^5=1.253390550613403320,w_1^5=0.6930422186851501465,w_2^5=0.7461462020874023438,\hfill \\ \hfill & w_3^5=0.7428795099258422852,w_4^5=0.7499830722808837891,w_5^5=1.511020541191101074,\hfill \end{array}$$

and the following control points:

$$\begin{array}{cc}\hfill & P_0=(1,0),P_1=(0.9514569678202488,2.265164366524486),\hfill \\ \hfill & P_2=(-2.041393043066488,1.823594836649781),P_3=(-2.573274681480492,-1.594942712567735),\hfill \\ \hfill & P_4=(0.9104298578262079,-2.558222347419337),P_5=(1,0).\hfill \end{array}$$

We can see in Figure 5 the obtained fitting curve of degree 5 and its corresponding control polygon.

In order to analyze the performance of the obtained approximation to the circle, we depict some geometric approximations of the approximation error. In Figure 6, we plot the radial and curvature errors given, respectively, by

$${ϵ}_r\left(t\right)=r\left(t\right)-1,{ϵ}_k=k\left(t\right)-1,$$

where $r\left(t\right)=\sqrt{{\mathcal{N}}_{w,P_x}^2\left(t\right)+{\mathcal{N}}_{w,P_y}^2\left(t\right)}$ and $k\left(t\right)=({\mathcal{N}}_{w,P_x}^{\prime }\left(t\right){\mathcal{N}}_{w,P_y}^{″}\left(t\right)-{\mathcal{N}}_{w,P_x}^{″}\left(t\right){\mathcal{N}}_{w,P_y}^{\prime }\left(t\right))/{\left(\sqrt{{\left({\mathcal{N}}_{w,P_x}^{\prime }\left(t\right)\right)}^2+{\left({\mathcal{N}}_{w,P_y}^{\prime }\left(t\right)\right)}^2}\right)}^3$.

We can observe that the radial error vanishes at $t=-\Delta ,\Delta$ because the approximation is exact for the initial and the last points of the curve.

Let us see another approximation example of the unit circle performing the neural network ${\mathcal{N}}_{w,P}$ using trigonometric functions. We have the following set of data points $s={\left(s_i\right)}_{i=0}^{99}$ on circle (14):

$$s_i=(x_c\left((t_i/\Delta +1)\pi \right),y_c\left((t_i/\Delta +1)\pi \right),i=0,\dots ,99,$$

being the parameters $t_i=-\Delta +2\Delta i/99$, $i=0,\dots ,99$, with $0<\Delta <\pi /2$. For $n=5$, we have approximated the set s by the fitting curve $({\mathcal{N}}_{w,P_x}\left(t\right),{\mathcal{N}}_{w,P_y}\left(t\right))$ using the functions $f\left(t\right)=sin\left((\Delta +t)/2\right)$ and $g\left(t\right)=sin\left((\Delta -t)/2\right)$ at the vector nodes $t={\left(t_i\right)}_{i=0}^{99}$ with $t_i$ as above. After training the neural network ${\mathcal{N}}_{w,P}$, performing Algorithm 1 for 3000 iterations, and with $\alpha =0.001,{\beta }_1=0.9,{\beta }_2=0.999,and\epsilon ={10}^{-7}$, we have obtained the following weights:

$$\begin{array}{cc}\hfill & w_0^5=2.193439722061157227,w_1^5=0.6315788030624389648,w_2^5=0.4818322956562042236,\hfill \\ \hfill & w_3^5=0.4559116661548614502,w_4^5=0.6407876610755920410,w_5^5=2.386862277984619141,\hfill \end{array}$$

and the following control points:

$$\begin{array}{cc}\hfill & P_0=(1,0),P_1=(1.094536581177390,2.759880482316126),\hfill \\ \hfill & P_2=(-3.104044853918599,2.85189370210171),P_3=(-3.702726956127786,-2.817369554402328),\hfill \\ \hfill & P_4=(1.085284693582619,-3.000573791799708),P_5=(1,0).\hfill \end{array}$$

We can see in Figure 7 the fitting curve of degree 5 obtained and its control polygon.

Figure 8 shows its corresponding radial and curvature errors. We can observe that the radial error vanishes at $t=-\Delta ,\Delta$ because the approximation is exact for the initial and the last points of the curve.

Finally, for different values of n, Figure 9 shows the history of the loss function given in (12) through the training process on 3000 iterations of the neural network ${\mathcal{N}}_{w,P}$ using the polynomial functions and the trigonometric functions. We can observe, in both cases, that the convergence to the circle of the fitting curves is faster as n increases.

4.2. Cycloid Curve

Cycloids are commonly used in manufacturing applications (e.g. gear tooth geometry). The cycloid is a transcendental curve and thus, it cannot be expressed by polynomials exactly. Creating a complex curve in a CAD/CAM system is not always straightforward. In [20], it is shown the need to new methods for approximating this curve. Let us show different approximations to the cycloid by several fitting curves obtained by training the neural network ${\mathcal{N}}_{w,P}$ using polynomial, trigonometric and hyperbolic functions f and g that satisfy the conditions of Proposition 1.

The cycloid is given by the parametric equations

$$\left\{\begin{array}{c}{x}_{cc}\left(t\right)=t-sint,\hfill \\ y_{cc}\left(t\right)=1-cost,\hfill \end{array}\right.$$

(15)

$t\in [0,2\pi ]$. We have taken the following sequence of points $s={\left(s_i\right)}_{i=0}^{99}$ on the cycloid (15):

$$s_i=(x_{cc}\left(2\pi t_i\right),y_{cc}\left(2\pi t_i\right)),i=0,\dots ,99,$$

being the parameters $t_i=i/99$, $i=0,\dots ,99$.

For $n=10$, we have approximated the set s by two fitting curves at the vector nodes $t={\left(t_i\right)}_{i=0}^{99}$ with $t_i$ as above. One of the fitting curves is obtained using the functions $f\left(t\right)=t$ and $g\left(t\right)=1-t$, and the other fitting curve is obtained using the functions $f\left(t\right)=t^2$ and $g\left(t\right)=1-t^2$.

We can see in Figure 10, the fitting curves and their corresponding fitting error. The fitting error is given by the Euclidean norm between the approximated curve—in this case, the cycloid curve—and the obtained fitting curve. Observe that the cycloid curve is better approximated by the fitting curve obtained using the functions $f\left(t\right)=t^2$ and $g\left(t\right)=1-t^2$ than by the fitting curve obtained using the functions $f\left(t\right)=t$ and $g\left(t\right)=1-t$.

Let us see two more approximations of the cycloid. We have taken the following sequence of points $s={\left(s_i\right)}_{i=0}^{99}$ on the cycloid (15):

$$s_i=(x_{cc}\left((t_i/\Delta +1)\pi \right),y_{cc}\left((t_i/\Delta +1)\pi \right),i=0,\dots ,99,$$

being the parameters $t_i=-\Delta +2\Delta i/99$, $i=0,\dots ,99$, with $0<\Delta <\pi /2$.

For $n=10$, we have approximated the set s by two fitting curves at the vector nodes $t={\left(t_i\right)}_{i=0}^{99}$ with $t_i$ as above. One of the fitting curves is obtained by training the neural network ${\mathcal{N}}_{w,P}$ using the trigonometric functions $f\left(t\right)=sin\left((\Delta +t)/2\right)$ and $g\left(t\right)=sin\left((\Delta -t)/2\right)$ and the other one using the hyperbolic functions $f\left(t\right)=sinh\left((\Delta +t)/2\right)$ and $g\left(t\right)=sinh\left((\Delta -t)/2\right)$.

We can see in Figure 11, the fitting curves and their corresponding fitting error. Let us observe that the cycloid is better approximated by the fitting curve obtained using the hyperbolic functions than the fitting curve obtained using the trigonometric functions.

4.3. Archimedean Spiral Curve

Finally, let us show different approximations of the Archimedean spiral given by the parametric equations

$$\left\{\begin{array}{c}{x}_{sa}\left(t\right)=tcos\left(t\right),\hfill \\ y_{sa}\left(t\right)=tsin\left(t\right),\hfill \end{array}\right.$$

(16)

$t\in [0,4\pi ].$ First, let us see an approximation obtained training the neural network ${\mathcal{N}}_{w,P}$ using polynomial functions. We have taken the following sequence of points $s={\left(s_i\right)}_{i=0}^{99}$ on the Archimedean spiral (16):

$$s_i=(x_{sa}\left(4\pi t_i\right),y_{sa}\left(4\pi t_i\right)),i=0,\dots ,99,$$

being the parameters $t_i=i/99$, $i=0,\dots ,99$.

For $n=11$, we have approximated the set s by the fitting curve using the functions $f\left(t\right)=t$ and $g\left(t\right)=1-t$ at the vector nodes $t={\left(t_i\right)}_{i=0}^{99}$ with $t_i$ as above. Figure 12 shows the fitting curve of degree 11 and its corresponding fitting error.

Let us see other examples in which the Archimedean spiral is approximated by the neural network using hyperbolic functions. We have taken the following sequence of points $s={\left(s_i\right)}_{i=0}^{99}$ on the Archimedean spiral (16):

$$s_i=(x_{sa}((t_i+\Delta )2\pi /\Delta ),y_{sa}((t_i+\Delta )2\pi /\Delta ),i=0,\dots ,99,$$

being the parameters $t_i=-\Delta +2\Delta i/99$, $i=0,\dots ,99$. For $n=11$, we have approximated the set s by the fitting curve using the functions $f\left(t\right)=sinh\left((\Delta +t)/2\right)$ and $g\left(t\right)=sinh\left((\Delta -t)/2\right)$ at the vector nodes $t={\left(t_i\right)}_{i=0}^{99}$ with $t_i$ as above.

Figure 13 shows the fitting curve of degree 11 and its corresponding fitting error.

We can see in Figure 14, the convergence of the two above fitting curves. Let us observe that the fitting curve corresponding to the hyperbolic rational basis has a faster convergence. Therefore, once again, it seems that the choice of the functions f and g, hence the choice of the rational basis, is relevant to the approximation.

4.4. Comparison of Least-Squares Fitting and the Neural Network ${\mathcal{N}}_{w,P}$

In this section, the proposed neural network is compared with the least-squares method and the regularized least-squares method. It is well known that least-squares fitting is a common procedure to find the best fitting curve to a given set of data points by minimizing the sum of the squares of the data points from the curve ([9,21,22]). The least-squares method is sensitive to small perturbations in data and, in those cases, regularization methods can be applied such as the regularized least-squares method (see [23]). Therefore, two experiments have been developed. The set of data points used in the first experiment is a non-noisy parametrization of known curves, so the least-squares method has been applied. In the second experiment, we used an image to obtain a noisy set of data points and applied the regularized least-squares method.

The problem that we want to solve is stated as follows. Suppose that f and g are functions defined on $[a,b]$ satisfying the conditions of Proposition 1. Consider a set of parameters $a\le t_0<\cdots \le t_{\ell }\le b$ and a sequence of data points $s_0,\dots ,s_{\ell }\in {\mathbb{R}}^k$, where each parameter $t_i$ is associated with a data point $s_i$. For some $n\le \ell$, we want to compute a rational curve

$$c\left(t\right)=\sum _{i=0}^n\frac{w_i^n\left(\genfrac{}{}{0pt}{}{n}{i}\right)f^i\left(t\right)g^{n-i}\left(t\right)}{{\sum }_{i=0}^n{w}_i^n\left(\genfrac{}{}{0pt}{}{n}{i}\right)f^i\left(t\right)g^{n-i}\left(t\right)}{P}_i,t\in [a,b],$$

(17)

minimizing the sum of the squares of the deviations from the set of data points $s={\left(s_i\right)}_{i=0}^{\ell }$, that is, ${\sum }_{i=0}^{\ell }{(s_i-c\left(t_i\right))}^2$. In order to compute the control points $P={\left(P_i\right)}_{i=0}^n$ of the fitting curve, we have to solve, in the least-squares sense, the overdeterminated linear system $AP=s$, where the matrix A is

$$\begin{array}{c}A={\left(\frac{w_i^n\left(\genfrac{}{}{0pt}{}{n}{i}\right)f^i\left(t_j\right)g^{n-i}\left(t_j\right)}{{\sum }_{i=0}^n{w}_i^n\left(\genfrac{}{}{0pt}{}{n}{i}\right)f^i\left(t_j\right)g^{n-i}\left(t_j\right)}\right)}_{0\le i\le n;0\le j\le \ell .}\end{array}$$

In this fourth experiment, let us see two examples comparing the fitting curves obtained applying this traditional method and the fitting curves obtained by training the neural network ${\mathcal{N}}_{w,P}$. We have seen previously an approximation of the circle (14) obtained with the neural network ${\mathcal{N}}_{w,P}$ for $n=5$ using the polynomial functions $f=t$ and $g=1-t$, $t\in [0,1]$. Besides, we have seen an approximation of the Archimedean spiral (16) obtained with the neural network ${\mathcal{N}}_{w,P}$ for $n=11$ using the polynomial functions $f=t$ and $g=1-t$, $t\in [0,1]$. Using the same parameters and sets of data points, we solved the corresponding overdeterminated linear systems $AP=s$ to obtain the control points of the curve that best fits the given data points in the least-squares sense. For this purpose, following the steps shown in Section 6.4 of [1], we have obtained the solutions of $AP=s$, firstly, by using the Matlab command SVD and, secondly, by using the Matlab command mldivide.

Since the weights are unknown and are necessary for solving the linear system $AP=s$ in the least-squares sense, these have been randomly chosen within a positive range. Let us observe that one of the advantages of the neural network ${\mathcal{N}}_{w,P}$ with respect to the least-squares method is that the neural network ${\mathcal{N}}_{w,P}$ not only finds suitable control points, but it also finds suitable weights to fit the given set of data points.

In

Table 3. For different number of weights and control points, the time of execution of the least-squares method using the Matlab commands mldivide and SVD, and the time of execution of the Algorithm 1 for 3000 iterations are provided. The values have been measured in seconds with a set of data points of size 100 and they are the mean of 5 repetitions.

$\mathit{n}+1$	Least-Squares Method		Algorithm 1
	mldivide	SVD	3000 Iterations
5	63.3014	65.6015	189.5386
10	167.0402	237.0386	268.8726
15	309.5811	457.8987	347.6139
20	478.1141	774.3472	429.2819
30	961.2860	978.6830	568.5647
50	2552.4064	4097.9278	850.6713

, the execution time of the least-squares method using a different number of weights and control points is provided. We can see that, although for low-degree fitting curves, the least-squares method is faster than the method proposed in this paper, for the fitting curves of degree greater than $n=18$, the proposed method is faster than the least-squares method.

We can see in Figure 15 and Figure 16 that the fitting curves obtained by training the neural network are more accurate than the fitting curves obtained by applying the traditional least-squares method.

In the last experiment, we have taken 31 points from a baroque image as a set of data points (see Figure 17). We have approximated them by two fitting curves of degree 8 with hyperbolic functions $f\left(t\right)=sin\left((\Delta +t)/2\right)$ at the vector nodes $t={\left(t_i\right)}_{i=0}^{30}$, such that $t_i=-\Delta +2\Delta (i-1)/30$, $0<\Delta <\pi /2$. The first fitting curve was obtained by training the neural network ${\mathcal{N}}_{w,P}$ during 1000, reaching a final mean absolute error on the set of data points of $9\times {10}^{-3}$. Besides, this example shows the robustness of the proposed method to noisy data points. The second fitting curve was obtained with random vector weights and the regularized least-squares method by using an adaptation of the Matlab library available in [24]. This last fitting curve reached a mean absolute error of $1.95\times {10}^{-2}$. Visually, in Figure 17, it is appreciated that the fitting curve obtained by training the neural network ${\mathcal{N}}_{w,P}$ achieves better curvature than the regularized least-squares method (which is more evident at the beginning and at the end of the curve). Finally, we would like to highlight that we have observed in all the executed experiments a good performance of the neural network for small values of n, in spite of the complexity of the set of data points.

5. Conlusions and Future Work

In this work, we have tackled the problem of finding a rational curve to fit a given set of data points. To solve this issue, we have proposed a one-hidden-layer neural network based on a general class of totally positive rational bases belonging to spaces mixing algebraic, trigonometric and hyperbolic polynomials, thus expanding the potential range of applications to include more difficult shapes. In order to obtain the weights and control points of the rational curve to fit the data points, the neural network is trained with an optimization algorithm to update the weights and control points while decreasing a loss function. The fitting curves of the numerical experiments show that for certain curves, the use of particular rational bases provides better results.

In future work, we wish to extend the neural network to obtain a suitable parametrization of the data points. The choice of the parameters can help to improve the approximation. Additionally, we plan to apply our method to curves, surfaces and high-dimensional data, and analyze, as in [25] for Bezier curves, its application to industrial software, CAD/CAM systems (such as Blender (https://www.blender.org/), Maya (https://www.autodesk.es/products/maya/overview) or Solid Edge (https://solidedge.siemens.com/en/)), and other real-world problems. Finally, we plan to explore the applicability of our approach to the resolution of linear differential equations (cf. [26,27]).