Photogrammetric Precise Surveying Based on the Adjusted 3D Control Linear Network Deployed on a Measured Object

Featured Application

The elaborated method can be applied to objects dimensioning, volume calculations, or displacement measurements. The approach uses popular and commonly available instruments, making the solution universal.

Abstract

In surveying engineering tasks, close-range photogrammetry belongs to leading technology by considering different aspects like the achievable accuracy, availability of hardware and software, accessibility to measured objects, or the economy. Hence, constant studies on photogrammetric data processing are desirable. Especially in industrial applications, the control points for close-range photogrammetry are usually measured using total stations. In the case of smaller items, more precise positions of control points can be obtained by deploying and adjusting a three-dimensional linear network located on the object. This article analyzes the accuracy of the proposed method based on the measurement of the linear network using a professional tape with a precision of ±1 mm. It is shown what accuracy of object feature dimensioning can be obtained based on the proposed innovative network method for photo-point measurement, using only the minimum required number of two stereo-images. The photogrammetric 3D model derived from them and captured with a non-metric camera is characterized by the highest possible precision, which qualifies the presented approach to accurate measurements used in the surveying engineering. The authors prove that the distance between two randomly optional points derived from the 3D model of a dimensioned object is equal to the actual distance measured directly on it with one-millimeter accuracy.

1. Introduction

Close-range photogrammetry (CRP) is a versatile surveying technology able to create object models of different sizes, from small industrial elements to large buildings [1,2]. The accuracy of CRP mostly depends on the size of the pixel, the distance between subsequent camera positions, and the respective distances to the photographed item. Moreover, the essential factor is the distribution, density, and accuracy of the control points. Such photo-points deployed on engineering objects and captured with CRP or terrestrial laser scanning (TLS) are usually measured using classical methods by utilizing total stations, for example [3,4,5,6,7]. The CRP techniques are used in various engineering tasks related to studying the geometry of structures [8], both on the macro and micro scale [9,10]. According to the current state-of-the-art in land surveying and the capacities offered by modern computers, it is possible to apply CRP for capturing static and dynamic objects in motion. For example, in publication [11], the authors focused on the self-developed photogrammetric system consisting of dedicated hardware and employing a unique adjustment strategy to determine the accurate position of the mapped elements. They proved that such an integrated approach significantly increases the positioning accuracy allowing for fast registration of the existing scenes. The GNSS satellite receiver and the IMU inertial unit were used as additional devices supporting the integrated surveying. Another work [12] discusses precisely capturing geometrical changes in 2D using CRP. The proposed research method allows for high accuracy in determining displacements of the object’s control points expressed, even in sub-millimeter values. The presented method consists of a geometric system utilizing different camera positions and is mainly dedicated to indoor applications. The high accuracy was confirmed by a comparative study evaluating the displacements of the single-name points placed in a test stand using micrometers with a resolution of 0.01 mm. The root-means-square (RMS) values of the differences between the predefined and evaluated displacements in the two directions (local X and Y) are 0.11 and 0.02 mm, which closely predisposes this approach to metrological tasks. However, relatively small entities placed in a limited space were assessed in this case. A comparative study on the influence of control points number on CRP positioning is explained in [13]. Nevertheless, the studied solution concerns aerial photogrammetry using unmanned aerial vehicles (UAV). The obtained accuracies vary within a few centimeters, allowing for the solution to map the displacements of natural objects of considerable size (for example, open-pit mines, geotechnical structures, slopes, or embankments) [14]. All previously cited works, together with [15], confirm the usefulness of photogrammetric methods in industrial solutions, which require the exact determination of the examined geometry elements. Recently, low-cost solutions have also become very popular in the precise dimensioning of things [16]. This approach uses inexpensive devices and technologies, competing with well-known, classic solutions provided by recognized manufacturers. The study [17] presented a low-cost CRP system for imaging building façades. Such a system uses a specially designed measuring set employing a camera with a laser distance meter, making it possible to conduct measurements with millimeter-level accuracy. The authors used the self-developed software to assess the impact of various uncertainties (including camera calibration errors) on the surveying results. In this case, a crucial factor was recognizing the measurement set’s geometrical parameters, which determines the absolute accuracy of the derived object model. CRP is also successfully used together with other techniques for remote detection of object geometry [18], such as TLS. An example of such integration in archaeological applications is shown in [19]. The presented approach allows for accurately mapping objects in 3D, treating precisely determined geometric data (distances between points) as a significant enhancement of a point cloud. A similar integration of data from remote sensing imaging and numerical terrain models is presented in [20]. In the accurate and reliable determination of the geometric parameters of engineering objects using photogrammetry, a crucial role is played not only by the instrumental aspect but, perhaps above all, the methodological attitude. Developing an optimal strategy for acquiring, processing, and analyzing measurement results is the pivotal problem of modern geomatics. The issues mentioned above motivated the authors to develop and test a CRP method based on adjusting the spatial control network located either on a test object or in its proximity. The article analyzes the accuracy of the proposed method based on previous surveying, such as a linear control network using a measuring tape with a precision of ±1 mm. Such accuracy can hardly be obtained by employing a common total-station, especially since targeting and measuring distances to object edges, corners, and other characteristic points are always problematic, especially while executing reflectorless surveys. We have demonstrated the accuracy of object dimensioning based on the proposed innovative photo-point network deployment using only the minimum required number of two stereo images.

2. Materials and Methods

The developed procedure is based on the classic stereo-pair of pictures taken for a test site using a popular camera. The control network consisting of linear values has been marked directly on the object, the points of which represent its characteristic elements. The control points gain Cartesian coordinates expressed in the local system encompassing the pictured entity as well as the pixel coordinates determined for the left and right photograph. Then, the measured linear elements of the control network related to the local coordinates of the selected points are adjusted by the least-squares method employing the Gauss-Markov model [21]. The adjusted positions of the control points are then used to set out the orientation and distortion parameters both for the left and right image, respectively. For the resulting center projections, the accuracy assessment is carried out considering the estimated errors in determining their parameters. As a result, the photogrammetrically derived linear elements of the control network are compared with the actual measures derived from the direct object survey.

To verify the developed method, we took control measurements in the university classroom, assuming an office desk with some auxiliary elements. The control network oriented in the local XYZ coordinate encompassed all its characteristic points (object vertices). All linear components were measured three times with a surveying tape, thus determining their values with ± 1 mm precision. The 3D linear network established on the investigated object consisted of 12 points joined by 40 distances. Then, using a professional mirror-reflex camera, a Canon EOS 500D [22], two photos were taken covering the item from its left and right side (Figure 1 and Figure 2).

The key technical parameters of the camera used are presented in

Table 1. Key technical parameters of the camera Canon EOS 500D.

Parameter	Description
Type	Digital AF/AE SLR camera with built-in flash
Recording media	SD memory card, SDHC memory card
Image type	JPEG, RAW (14-bit, Canon original)
Image sensor type	CMOS sensor
Image sensor size	22.3 × 14.9 mm
Resolution	Effective pixels: approx. 15.10 MPx Total pixels: approx. 15.50 MPx
Color filter system	RGB primary color filter
Low-pass filter	Fixed position in front of the image sensor
Focus modes	Servo mode; Auto-focus; Manual focus
Exposure control	Full Auto, Portrait, Landscape, Close-up, Sports, Night Portrait, Flash Off, Creative Auto, Program), shutter-priority AE, aperture-priority AE, auto depth-of-field AE, manual exposure, E-TTL II autoflash, movie shooting

[23].

It should be added that the camera is commonly available on the market, emphasizing the low-cost feature of the presented method. For the experiment, the photos were taken using the classic indoor shooting program at short distances to the subject (portrait function at a fixed focal length, f/2 aperture, shutter speed 1: 200 s, ISO 400, no flash).

All mathematical operations were performed using Mathcad v. 15.0 software (PTC—https://www.mathcad.com–accessed on 28 April 2022; previously known as Mathsoft) by programming appropriate calculation worksheets.

3. Results

As is shown in Figure 1 and Figure 2, the reference frame X, Y, Z is connected with the network, assuming the Z-axis passes through point 1 perpendicularly to the plane with points 1, 2, and 4. The axis Y is parallel to the section |1–2|, and the third orthogonal axis X is parallel to |1–4|. Hence, the three connecting reference sections with network points 1, 2 and 4 obtained the following coordinates: 1 (0, 0, h), 2 (0, |1–2|, h), 4 (|1–4|, 0, h). The sections |1–2| and |1–4| represent measured distances between points 1 and 2 as well as 1 and 4, respectively. The value h is the approximate height of the object. The approximate coordinates X, Z, Y of all remaining points 3, 5, …, 12 are computed by solving the network consisting of 40 measured distances between all points.

To evaluate the proposed approach, we consider an example 3D network presented in Figure 3, which exemplifies a standard spatial network used in land surveying. The observational linear equation of the measured spatial distances between any two spatial network points P (X_P, Y_P, Z_P) and Q (X_Q, Y_Q, Z_Q) is based on the well-known formula used in three-dimensional surveying [24]:

$$\begin{array}{ll}v=& -\frac{X_Q-X_P}{s_0}d{X}_P-\frac{Y_Q-Y_P}{s_0}d{Y}_P-\frac{Z_Q-Z_P}{s_0}d{Z}_P\hfill \\ & +\frac{X_Q-X_P}{s_0}d{X}_Q+\frac{Y_Q-Y_P}{s_0}d{Y}_Q+\frac{Z_Q-Z_P}{s_0}d{Z}_Q-(s-s_0),{\sigma }_s\hfill \end{array}$$

(1)

where: dX_P, dY_P, dZ_P, and dX_Q, dY_Q, dZ_Q are corrections of the approximate coordinates X_P, Y_P, Z_P and X_Q, Y_Q, Z_Q of the points P and Q, respectively, v is the random error of distance measurement s with zero value expectation and known standard deviation equaling σ_s = 1 mm, s₀ is an approximate value of the measured distance s:

$$s_0=\sqrt{{(X_Q-X_P)}^2+{(Y_Q-Y_P)}^2+{(Z_Q-Z_P)}^2}$$

(2)

The Gauss-Markov observational model [21,25] for all measured spatial distance observations s₁, s₂, …, s_n, n = 40, (Figure 1 and Figure 2) is completed regarding the Equation (3):

$$v=Ax-(s-s_0),{\Sigma }_s^{}$$

(3)

where: x—vector of unknown corrections: dY₂, dX₃, dY₃, dZ₃, dX₄, dY₄, dX₅, dY₅, dZ₅, …, dX₁₂, dY₁₂, dZ₁₂, v—vector of unknown observational error residuals v₁, v₂, …, v_n, A-known design matrix, s—vector of observations s₁, s₂, …, s_n, s₀—vector of approximate values of observations s_0,1, s_0,2, …, s_0,n, σ_s—diagonal covariance matrix of observations s, composed from standard deviations σ_s,1, σ_s,2, …, σ_s,n.

In the adjustment, the network is connected with the reference frame by three coordinates, X₁, Y₁, Z₁ of point 1, two coordinates X₂, Z₂ of point 2, and one coordinate Z₄ of point 4. These coordinates are constant; it means dX₁ = dY₁ = dZ₁ = 0, dX₂ = dZ₂ = 0, dZ₄ = 0.

The least-squares solution of the Gauss-Markov model (8): $v^T{{\sum }^{ }}_s^{-1}v=min$ is given by well-known formulas [21,26]:

$$x={\Sigma }_x^{}{A}^T{\Sigma }_s^{-1}(s-s_0),$$

(4)

where the equations:

$${\Sigma }_x^{}={(A^T{\Sigma }_s^{-1}A)}^{-1}$$

(5)

$${\Sigma }_v^{}={\Sigma }_s^{}-A{\Sigma }_x{A}^T$$

(6)

represent the covariance matrices of x and v, respectively.

The maximal value of the standard deviations σ_X, σ_Y, σ_Z of the adjusted coordinates Y₂ + dY₂, X₃ + dX₃ Y₃ + dY₃, Z₃ + dZ₃, X₄ + dX₄, Y₄ + dY₄, X₅ + dX₅, Y₅ + dY₅, Z₅ + dZ₅… X₁₂ + dX₁₂, Y₁₂ + dY₁₂, Z₁₂ + Z₁₂, taken from diagonal elements of the covariance matrix (5) is equal to 1.2 mm.

The adjusted positions of the network point 1, 2, …, 12 are used as so-called control points for the determination of the orientation and distortion parameters of the left and right stereo photos in the object reference frame (X, Y, Z).

3.1. Accuracy of the Photogrammetric Intersection

The Photogrammetric Central Projection

The photogrammetric central projection is a projection of the three-dimensional object space (X, Y, Z) onto a two-dimensional image plane (x, y) (Figure 1 and Figure 2). The central projection equations (X, Y, Z) $\to$(x, y) are given by following [26,27]:

$$\begin{array}{ll}x=x_0-& f_s\frac{r_{11}(X-X_0)+r_{12}(Y-Y_0)+r_{13}(Z-Z_0)}{r_{31}(X-X_0)+r_{32}(Y-Y_0)+r_{33}(Z-Z_0)}\hfill \\ & -[k_1({(x-x_0)}^2+{(y-y_0)}^2)+k_2{({(x-x_0)}^2+{(y-y_0)}^2)}^2](x-x_0)\hfill \\ & -k_3[3{(x-x_0)}^2+{(y-y_0)}^2]-2k_4(x-x_0)(y-y_0)\hfill \end{array}$$

(7)

$$\begin{array}{ll}y=y_0+& f_s\frac{r_{21}(X-X_0)+r_{22}(Y-Y_0)+r_{23}(Z-Z_0)}{r_{31}(X-X_0)+r_{32}(Y-Y_0)+r_{33}(Z-Z_0)}\hfill \\ & -[k_1({(x-x_0)}^2+{(y-y_0)}^2)+k_2{({(x-x_0)}^2+{(y-y_0)}^2)}^2](y-y_0)\hfill \\ & -2k_3(x-x_0)(y-y_0)-k_4[{(x-x_0)}^2+3{(y-y_0)}^2]\hfill \end{array}$$

(8)

where X, Y, Z are the coordinates of an object point P; x, y are the pixel image coordinates of the projected point P; Y₀, Y₀, Z₀ are the coordinates of the central projection point O; x₀, y₀ are the image pixel coordinates of the central point O; k₁, k₂, k₃, k₄, are parameters of the camera distortion, fs = f/s, f is the focal length of the camera, s is the CCD cell size of the camera, whilst:

$$\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]=\left[\begin{array}{ccc}\cos \varphi \cos \kappa & \cos \omega \sin \kappa +\sin \omega \sin \varphi \cos \kappa & \sin \omega \sin \kappa -\cos \omega \sin \varphi \cos \kappa \\ -\cos \varphi \sin \kappa & \cos \omega \cos \kappa -\sin \omega \sin \varphi \sin \kappa & \sin \omega \cos \kappa +\cos \omega \sin \varphi \sin \kappa \\ \sin \varphi & -\sin \omega \cos \varphi & \cos \omega \cos \varphi \end{array}\right]$$

(9)

is the matrix of rotation (ω, φ, ϰ) of the object reference frame (X, Y, Z) related to the camera reference frame (Y_k, Y_k, Z_k). The frame’s origin (Y_k, Y_k, Z_k) is the central projection point O, the axes Y_k, Y_k run parallel to the sections of the CCD frame, and Z_k is perpendicular to the CCD frame. The six parameters Y₀, Y₀, Z₀, ω, φ, ϰ define the external orientation of the camera reference frame (Y_k, Y_k, Z_k) concerning the object reference frame (X, Z, Y). The three parameters x₀, y₀, f = f_ss define the internal orientation of the image referring to the camera reference frame (Y_k, Y_k, Z_k).

3.2. The Photogrammetric Spatial Resection

The photogrammetric spatial resection method allows for determining orientation (Y₀, Y₀, Z₀, ω, φ, ϰ, x₀, y₀, f_s) and distortion (k₁, k₂, k₃, k₄) parameters of an image based on the known control points deployed on the object [28]. In the case of the left image, the 13 orientation and distortion parameters were computed by solving the set of 22 Equations (7) and (8) for eleven visible control points 1, 2, …, 12 (Figure 1) by the least-squares method. The camera’s determined position (Y₀, Y₀, Z₀) in the left (L) is shown in Figure 3. The pixel coordinates x_comp, y_comp of the control points computed after adjustment according to the Equations (7) and (8) are not right equal to the measured values x, y. The residual mean squares (RMS) of the obtained pixel coordinates residuals ε_x = x − x_comp, ε_y = y − y_comp are equal to 1.8 and 2.0 in pixels, respectively.

In the case of the right image, the 13 orientation and distortion parameters were computed by solving a set of 22 Equations (7) and (8) for eleven visible control points 1, 2, …, 12 (Figure 2) by the least-squares method. The camera’s determined position (Y₀, Y₀, Z₀) in the right (R) is shown in Figure 4. The RMS for obtained pixel coordinates residuals ε_x = x − x_comp, ε_y = y − y_comp are equal 1.3 and 1.6 in pixel values, respectively.

Due to the nonlinear least-squares problem, the approximate values of the parameters were computed using the Direct Linear Transformation (DLT) method [29].

3.3. The Photogrammetric Triangulation

The photogrammetric triangulation is a method of simultaneous determination of the orientation and distortion parameters of the stereo or multi-images as well as the coordinates X, Y, Z of new so-called tie points located on the object [30]. In our experiment, we chose only one tie point A (Figure 1 and Figure 2). In this case, the 26 orientation and distortion parameters of the left and right stereo images plus three coordinates X_A, Y_A, Z_A of the tie point A were computed by solving the set of 48 Equations (7) and (8) by the least-squares method [31]: 24 equations for twelve visible points on the left image 1, 2, …, 12, A and 24 equations for twelve visible points on the right image 1, 2, …, 12, A. The RMS values for the obtained pixel coordinates residuals ε_x, ε_y are equal to 1.5 and 2.1 in pixel values, respectively. The determined positions of the camera in the left and right positions concerning the 12 network points are shown in Figure 3.

3.4. The Accuracy of the Photogrammetric Intersection

The photogrammetric intersection is a method of computing the coordinates of object points based on the spatially oriented stereo or multi-images referenced in the object reference system X, Y, Z. For example, the coordinates X_B, Y_B, Z_B of the new object point B (Figure 1 and Figure 2) are computed by solving four Equations (7) and (8) by the least-squares method: two equations for measured pixel coordinates of the projected point B on the left image (Figure 1) and two equations for projected point B on the right image (Figure 2). The computed spatial distance between points A (X_A, Y_A, Z_A) and B (X_B, Y_B, Z_B) is equal to [(X_B − X_A)² + (Y_B − Y_A)² + (Z_B − Z_A)²]^1/2 = 0.210 m. The distance between relevant points measured by tape is also equal to 0.210 m. So, in this case, the error of the photogrammetrically measured distance between two points A and B on the object is equal to 0.000 [m].

In the second example, the distance between points C and D is computed (Figure 1 and Figure 2). The coordinates of points C and D were computed in the same way as those of point B. The computed spatial distance between photogrammetrically determined points C (x_C, y_C, z_C) and D (x_D, y_D, z_D) is equal [(X_D − X_C)² + (Y_D − Y_C)² + (Z_D − Z_C)²]^1/2 = 0.989 m. The distance between these points measured by tape equals 0.990 m. In this experiment, an error of the photogrammetrically measured distance between two points C and D on the object equals 1 mm. For clarity, the aforementioned results are summarized in

Table 2. The accuracy of the photogrammetric intersection expressed as the comparison between subsequent control points measured in different methods.

From Point	To Point	Tape Measured Distance [m]	Photogrammetric Measured Distance [m]	Residual Value σ [m]
A	B	0.210	0.210	0.000
C	D	0.990	0.989	0.001

.

4. Discussion

Close-range photogrammetry is used in many engineering fields. Employing it for object dimensioning and assessing displacement values requires developing and testing an appropriate strategy. In the case of small pieces, more precise control points can be set out by founding and adjusting a three-dimensional linear network on the object. Our experiment shows that the adjusted positions of the control points are determined with high accuracy of about ±1 mm. Furthermore, such a high accuracy refers to the entire 3D photogrammetric model derived from the control points deployed directly on the examined structure. Hence, it is possible to achieve desirable measurement uncertainty related to only two images taken with the non-metric camera Canon EOS 500D.

The proposed approach is progressive and can be freely modified to be integrated with other surveying technologies. For example, it can be utilized while dimensioning indoor spaces for the area and volume calculations. Another possibility is to use it in displacement monitoring, control measurements, or finishing works on a construction site.

In contemporary geomatics, one of the deciding factors when choosing a particular surveying method is its availability and time consumption. In the discussed case, both conditions are successfully fulfilled. Using a non-metric, commonly used camera makes the solution low-cost. Also, data processing is not time-consuming, and the algorithm can be run on a standard computer. The application is possible for each object, where a three-dimensional linear network can be established. Such a point deployment can be possible in different ways by selecting characteristic object points (for example, its vertices and edges) or projecting light spots using a common laser emitter. The network can be measured using a tape or hand-held laser distance meter as well as, in case of lesser object availability, a standard total station offering a reflectorless distance surveying option. In such a case, however, one should count with a final accuracy lower than 1 mm.

The limitation in using the developed method may be poor lighting conditions on the site, insufficient resolution of the camera, and the inability to cover the object with a control network. It can be mentioned that the location of network points may refer to both characteristic ends of the measured object as well as points marked using available laser projectors. These items are the subject of further research conducted by the authors.