6.1.1. Self-Calibration

Performing self-calibration, in which the camera is calibrated without ground control points (GCP), but only with measured image point coordinates of a (flat) concrete surface, leads to unsatisfactory results. The reason for this lies primarily in the lack of spatial distribution of the points and causes the parameters of the interior orientation of the camera to correlate with each other. Table 3 shows the dependencies of the parameters in the form of the correlation coefficients after the self-calibration procedure.

**Table 3.** Dependencies of the parameters of interior orientation after self-calibration.


The table shows that the radial distortion parameters (*k*1–*k*3) especially correlate with focal length (*f*). The subsequently reconstructed point cloud using these parameters shows a conspicuous curvature (Figure 12). To visualize the curvature, a single profile line was extracted from the reconstructed point cloud. This profile line, 10-fold scaled in the height direction, is shown at the bottom of Figure 12. The histogram of the height distribution of the point cloud also indicates a smeared normal distribution (Figure 12, right).

**Figure 12.** Reconstructed point cloud with conspicuous curvature after self-calibration (**top**, **left**), extracted profile line (**bottom**, 10-fold scaled in height), and histogram of the height values (**right**).

### 6.1.2. Calibration of the Test-Field

In order to determine the GCP coordinates of the custom-built 3D calibration test-field, we attached it to a tripod with a rotatable plate and took images using a digital single-lens reflex camera (DSLR). The DSLR was mounted on a ball joint, which can be moved on two axes parallel to the calibration field. The measurement setup is shown in Figure 13. A total of 69 images of the calibration field were taken systematically from different directions and subsequently processed using the photogrammetric software PHIDIAS (Version 2.21, Langerwehe, North Rhine-Westphalia, Germany) [20]. The targets were measured in the images and afterwards, the 3D world coordinates were determined using bundle adjustment. Bundle adjustment also provided information about image measurement accuracy, which is 0.57 μm or 0.12 pixels in the *x*-direction and 0.59 μm or 0.12 pixels in the *y*-direction.

**Figure 13.** Measurement setup for the calibration of the 3D calibration test-field.

### 6.1.3. Calibration of the Industrial Camera

For the calibration of the interior orientation of the camera, we took a total of 30 images evenly distributed over the object. Subsequently, bundle adjustment with image measurements of the targets was performed in PHIDIAS, with only the parameters of the interior orientation included as unknowns. A cross validation of the adjusted point coordinates and subsequent calculation of the root mean square (RMS) of the point deviations leads to an object measurement accuracy of 3.40 μm for the *x*-coordinate, 4.19 μm for the *y*-coordinate, and 18.42 μm in depth. The conspicuously larger deviation for the *z*-component is related to the recording configuration and the associated poor intersection geometry of the optical rays. For the parameters of the interior orientation, we obtained the values and standard deviations given in Table 4. All parameters successfully passed a significance test with an error probability of 5%.

**Table 4.** Parameters of the interior orientation after calibration using the 3D calibration test-field.


Parameter estimation using least squares adjustment based on the Gauss-Markov-Model also provided the correlations of the parameters in the form of the correlation coefficients. These are listed in Table 5. The previously occurring correlations of focal length and radial distortion parameters have been significantly reduced. The remaining correlations between the *k*-parameters or between *p*1 and *cx* as well as *p*2 and *cy* are justified in the mathematical-functional models and can therefore not be completely avoided (see e.g., [34]).

**Table 5.** Correlation coefficients of the parameters of interior orientation after calibration using the 3D calibration test-field.


The visual impression of the point cloud reconstructed with the new set of parameters for the interior orientation confirms the significantly improved result as well. A curvature of the point cloud can no longer be noticed after reconstruction. For illustration, the point cloud and a profile line extracted from it and 10-fold scaled in height are shown in Figure 14. The histogram of the height distribution of the point cloud now approximates a normal distribution (Figure 14, right).

**Figure 14.** Point cloud (**top**, **left**), extracted profile line (**bottom**, 10-fold scaled in height), and histogram of the height values (**right**) after reconstruction with the new parameter set.

### *6.2. System Assessment*

### 6.2.1. Test Objects

We used a set of 18 concrete specimens with plainly different surface textures as test objects. The specimens have a size of 40 cm × 40 cm × 10 cm. For each of them, there are reference values for the roughness, determined by the sand patch method, the laser triangulation method, and the paste method [8]. A selection of three specimens with different roughness is shown in Figure 15.

**Figure 15.** Selection of three concrete specimens with different roughness of the investigated 18.

### 6.2.2. Measurement Procedure

After calibration of the industrial camera using the custom-built 3D calibration testfield, we examined all 18 specimens with our proposed measurement system. In the following, the measurement procedure is described based on a single specimen.

The measurement system is centrally placed onto the concrete specimen and initially a general functionality testing of the measurement system is performed. After switching on the system, the lighting is adjusted so that the surface to be captured is well illuminated. Following, the procedure for capturing images of the concrete surface is started by a controlling software. The images are captured with the following configuration: Five images in the *x*-direction and six images in the *y*-direction resulting in the images having an overlap of about 78% in x-direction and 69% in y-direction. The image capture of the surface with that particular configuration is finished in about 5 min. During capture, the images are transferred continuously to the mobile measurement computer (laptop).

After finishing the capture, the evaluation software is started and reads in the image dataset as well as the camera calibration data. In the first main step, the camera pose is refined using SfM. As a by-product, a sparse point cloud of the object surface is provided as well. The total time for this step is about 1 min. Figure 16 (top, left) shows the estimated camera poses with the sparse point cloud. Subsequently, a dense point cloud of the object surface is generated. Since this is the most compute-intensive step of the reconstruction pipeline, it takes about 6 min to complete. The generated dense point cloud and a zoomin are presented at the top right corner and at the bottom in Figure 16. Finally, the reconstructed dense point cloud of the concrete surface is used to derive a roughness parameter. For this exemplary specimen, we obtain the value *Ra* = 1.065 mm.

(**c**) 

**Figure 16.** 3D point cloud reconstruction of a particular concrete specimen: (**a**) Reconstructed camera poses including the sparse point cloud after SfM procedure, (**b**) generated dense point cloud after Dense Image Matching (DIM), and (**c**) zoom-in of the dense point cloud.

### **7. Results and Discussion**

### *7.1. GPU Acceleration of SGM*

The GPU-accelerated implementation of SGM was evaluated regarding the runtime. In order to determine the speed-up of the GPU acceleration, we also developed in C++ programming language a pure CPU-based implementation. In both cases, the same algorithms were used.

As a processing platform, we used Nvidia's graphics card GeForce GTX 1080 Ti. This high-end GPU contains 28 Streaming Multiprocessors (SM) with 128 CUDA Cores per SM and therefore provides a total of 3584 CUDA cores for parallel processing. The graphic card's throughput is around 11.70 TFLOPS and it has a memory space of around 11 GB.

As test data, we used stereo images of a concrete specimen previously captured by the measurement system. The area for dense image matching of the stereo images is restricted to the part of the images that is visible in both. Hence, we investigated our pipeline of SGM with grayscale images with a size of 2887 × 2652 pixels. The disparity search range was fixed to 80 pixels.

The comparison of the total runtime between the pure CPU implementation and the GPU-accelerated implementation is shown in Figure 17. We ran both implementations 10 times and charted the average. The runtime of the CPU implementation is therefore around 209 s, whereas the GPU-implementation achieves 4.4 s for the execution. Hence, with GPU acceleration, SGM is completed in only 2.1% of the time needed by the pure CPU implementation.

**Figure 17.** Comparison of the total runtime between the pure CPU implementation and the Graphics Processing Unit (GPU)-accelerated implementation.

For a more detailed comparison, we further plotted the runtime of every main algorithm of the SGM pipeline in Figure 18. The chart shows that basically each algorithm benefits from GPU acceleration. This is because the algorithms used by SGM are suitable for parallelization.

**Figure 18.** Runtime comparison of the single algorithms of the Semi-Global Matching (SGM) pipeline.

#### *7.2. Comparison of the Results of Our Measurement System with the Sand Patch Method*

The 18 concrete specimens were used to compare the results of our proposed measurement system with those obtained using sand patch method. For this purpose, we plotted our estimated results for the parameter *Ra* and the reference values for the *MTD* parameter determined by the sand patch method in Figure 19.

**Figure 19.** Comparison between *Ra* estimated by our measurement system and the references values for *MTD* determined by the sand patch method.

The values for the parameter *Ra* estimated by our measurement system are lower than the reference values for the *MTD* parameter. However, that was to be expected: Although both parameters represent the mean distance of the actual surface to a reference plane, the two reference planes used by the two methods are different. To be specific, *Ra* refers to the mean plane estimated through all points of the surface, while *MTD* refers to the plane placed onto the uppermost peaks (which leads in general to higher distances between the surface and the reference plane).

Furthermore, a closer look at the measurement values exposes some correlation between both methods. To show this more clearly, we plotted the values in a scatterplot with *MTD* in the *x*-axis and *Ra* in the *y*-axis (Figure 20). The Pearson correlation coefficient of both measurement series leads to 0.9681, which indicates a strong linear relationship.

**Figure 20.** Correlations between *Ra* estimated by our measurement system and the reference values for *MTD* determined by the sand patch method.

#### *7.3. Area- vs. Line-Based Estimation of the Roughness*

To demonstrate the necessity of an area-based roughness determination, we conducted further examinations of the concrete specimens. In particular, we chose a specimen appearing to be very irregular and coarse to carry out further analyses.

Calculating, e.g., the roughness parameter *Ra* for that specimen in the area-based way leads to 0.905 mm. For comparison with the line-wise calculation, we extracted 11 equally spaced lines both in the horizontal and vertical directions from the same point cloud. The extracted lines have a spacing of 4 cm. Calculating the line-based roughness parameter leads to the following measurement series:

The individual values vary substantially as shown in Figure 21. For the horizontal lines, the arithmetic mean is *xh* = 0.889 mm and the standard deviation is *sh* = 0.249 mm. For the vertical lines, the mean is *xv* = 0.891 mm and the standard deviation is *sv* = 0.199 mm. Setting up a 95% confidence interval for the horizonal lines results in

$$P\{0.722 \le \mu\_{\text{l}} \le 1.056\} = 95\% \tag{13}$$

and for the lines in the vertical direction, it results in

$$P\{0.757 \le \mu\_{\upsilon} \le 1.025\} = 95\%.\tag{14}$$

**Figure 21.** Calculated *Ra* for the extracted lines and for the whole area.

A normally distributed measurement series of the lines would lead to 5% of the values falling outside the 95% confidence interval (meaning either no line or just 1 line should fall outside the interval). However, in our case, six of the horizontal lines and four of the vertical lines do not fall into the calculated interval for the expected mean of the measurement series. Accordingly, the assumption that the individual profile lines are only subjected to normally distributed noise does not apply. Thus, the deviations of the lines can no longer be justified just stochastically, and this brings us to the conclusion that a single line is unsuitable to specify the roughness of an entire surface. An area-based determination, on the other hand, enables a reliable estimation of the roughness since all measuring points of the entire surface are concerned in the calculation.
