An Initial Dot Encoding Scheme with Significantly Improved Robustness and Numbers

Abstract

Coded mark point, as an important part of automatic matching, is widely used in stereo vision and photogrammetry. An encoding scheme named the Initial Dot encoding scheme, has been designed based on the programme of concentric rings, and which is able to solve restrictions on the number of codes and promote the stability of decoding. The designed coding pattern includes three functional points: locating point, start point and coded point. The maximum coding number of the Initial Dot scheme can reach 262,144 when the number of code bits is 18. The recognition experiments show that the Initial Dot encoding scheme can achieve 100% recognition rate at a 75 degree angle, which is very difficult for other coding schemes. By comparing experiments with the Schneider encoding scheme for a large flexible panel, the recognition rate of the proposed scheme is 99%, which is higher than the recognition rate of 90% of the Schneider scheme. The experimental results indicate that the Initial Dot encoding scheme can be successfully applied on flexible solar panels under a 1000 mm × 500 mm field of view.

1. Introduction

Vision measurement with its advantages of non-contact, high precision, high efficiency and good stability has a wide range of applications in large field-of-view measurements [1,2]. Mark points, as important parts of machine vision, play an important role in visual measurement. For a large field of view, encoding the marker points can effectively distinguish each other and can complete automatic matching [3,4,5]. However, most existing coded mark points schemes have two problems such as insufficient coding and low robustness in a large field of view. Therefore, it is necessary to design a coding scheme which can satisfy the coding number and satisfy the decoding stability to meet the measurement requirements under a large field of view [6].

At this stage, the coded mark points have a good application prospect in measurement [7,8]. Zhao Junqiao applied the coded mark points to the automatic parking system to match the related information [9]. In building up 3D measurement model for large scale object, the using of coded mark points not only solves the cumulative error of the stitching, but also greatly reduces the difficulty of matching the non-coded points [10,11]. Based on the stability of the marker point recognition, some people use the coded marker point on the measurement of the attitude angle of the drone [12]. At the same time, some people used a special coding pattern to design a target for underwater [13].

The coded marks are divided into three groups according to the features of the graph: dotted distribution, concentric ring and digital symbol. The most typical dotted distribution coded mark point is Hattori coded scheme which adopted by V-STARS system(Video-Simultaneous Triangulation and Resection System) [13]. In this scheme, the three outer circles are used to confirm the coordinate system of the coded mark points, and the three small circles in the middle are used to encode the mark points. Accordingly, the advantage of this scheme is that the mark points are simple, and only the circular patterns are identified as features, and it has been applied to industrial production widely. In the case of large field of view, due to the influence of affine transformation, the central point and encoded points of Hattori coded scheme eventually affect the stability of decoding and reconstruction results. Based on this plan, Yang Jian designed an error correcting coded point scheme [14]. With the support of error correcting points, the system has higher stability in recognition. While, the location of the image is affected by the method of edge detection [15]. At the same time, the decoding number is affected by the angle, and the code error rate is prone to occur when the number of coded points are different [16]. Lu Chengjing designed a point coordinate encoding which based on cross-ratio invariance [17]. Based on the basis of dotted distribution coded mark, through the characteristic of cross ratio, this scheme provides robustness of decoding. But, this scheme is still limited by the number of codes. In Lu’s scheme, when the number of bits is 20, the maximum possible number of codes can only reach 900.

As a classical method of the concentric ring, the Schneider coded mark is widely spread by GOM company from German [18], which consists of a dot and surrounding rings. Therefore, the advantage of the Schneider coded scheme is simple in construction and easy to recognize. Due to the limitation of structure, the number of coding points is much less than the point distribution under the same number of code bits, so it is difficult to afford the needs of a large field of view. Shortis MR and Seager JW designed a square coded mark which was applied in underwater calibration [13]. Each coded target consists of a central dot, surrounded by a different combination of sides that may be any of the six segments, and then each coded target must be rotationally unique to all other coded targets. Therefore, 1296 kinds of coded values can be obtained at most. Because the decoding value is affected by the linear length of the encoded image, the decoding stability is much lower under the large field of view. Ma Yangbiao and Zhong Yuexian designed 8 bit encoding circular markers, which have the advantage of dotted distribution coded mark and concentric ring type coded scheme [19]. In order to keep the rotation invariance of the coded mark points, the scheme used the method of encoding circularly cyclic decoding for minimum. Although the scheme has good robustness and recognition efficiency, the number of codes is as small as Schneider codes and does not satisfy the required for large field of view.

In addition, the digital symbol scheme also included numbers or other binary symbols, such as the coded scheme proposed by Wong and the coded scheme proposed by Caesar [20]. They usually used binary and digital symbols to encode the marked points and these programs are composed of positioning and symbol [21]. The advantage is that the expansibility of the number of coded is better, but the drawback is that the identification of symbols is difficult and the recognition efficiency is low. In the case of large field of view, the more complex the encoded image, the worse the decoding stability.

In summary, the above researchers have focused on the following two aspects in the study of the coded mark points: coded number and robustness. However, in order to match the suitable marked points in a large field and overcome the limitation of the coded numbers, there is an urgent need for a coded mark point that can afford both the number of codes and the rashness of the decoding. This paper introduces a dotted distribution type encoding that has initial information mark called the Initial Dot scheme. It is remarkable that a starting point is added to the coded scheme to increase the number of codes, as well as the full using dots improve the robustness of recognition. The aim of this paper is to design a new coded mark to support expansion, which can solve the limitation of the insufficient number of traditional coded points and the influence of affine transformation on robustness. In Section 2, a detailed introduction of this programme is given. A stability experiment of this scheme is introduced in Section 3. In Section 4, this paper mainly introduces the experiment of applying an Initial Dot scheme to flexible solar panels. Finally, the conclusions are in Section 5.

2. Coded Mark Design and Detection

2.1. Coded Mark Design

The shape of the mark points is varied, and the most common ones are circles, crosses and dot arrays. When designing coded points, the following requirements should provide three aspects [22]:

• Coded size is appropriate
• Coded graphics are unique.
• Good stability, less affected by rotation.

As shown above in Figure 1, the coded mark points are composed of three layers: the innermost layer is the positioning layer which consists of a pure black circle, and the function is to locate the coordinates of mark point; the secondary layer is the starting layer, consisting of a small solid ring that determines the starting point of the coded layer; the outermost layer is the coded layer, consisting of several solid or hollow circles, which are used for the encoding of mark points.

2.2. Decoding

The coded layer can be divided into N parts by angle, each of which is $360/N$ degrees. In the case of a starting point, the value nearest the starting point is first determined, and the point can be solid or hollow. When the change is a solid round, the decoded value is 1, when the point is a hollow circle, the value is 0, if the points do not have any circle, then the decoding failure pattern, it is non-default point encoding. The decoding direction is clockwise, and a binary code value of twelve bits is obtained. Compared to the Schneider coded scheme, the coded scheme’s outermost layer of this paper is the point instead of the ring, which more recognizable in detection. Particularly, this scheme has a starting point, which can greatly increase the number of coded marks. According to the following

Table 1. Comparison of the number of coded points.

Bit of Coded Points	4	6	8	10	12	18
Hattori scheme	0	2	13	38	88	206
Schneider scheme	5	13	35	107	325	14,601
Mayangbiao’s scheme	5	13	35	107	325	14,601
Initial Dot scheme	16	64	256	1024	4096	262,144

: When the initial information is added, in the case that the number of codes is 12, the encoding scheme with the start information is 11.7 times the coded scheme without the starting information.

2.3. Coded Mark Detection

The process of image preprocessing is actually debugging the image information, highlighting the required parts, and eliminating the need for partial impact. Because the encoding points used in this paper are characterized by the circle, the central location of the circle is a critical step in decoding [23,24].

First of all, in order to quickly detect the feature information of images, this paper used the method of local binaryzation. The idea of binaryzation is by setting a certain threshold T, Data greater than T can be passed, its value changed to 1, less than T data cannot passed, and its value changed to 0. The threshold segmentation method can be divided into two kinds: global threshold method and local threshold method. Under the uneven illumination condition, the method of local threshold segmentation is better than the global threshold segmentation method. The basic idea of the algorithm is that the image is divided into two parts by threshold, one is the background, the other is the object graph. At the same time, the minimum variance within the class and the maximum variance between classes are needed to make the two sets of grey values. Specific algorithms are as follows: assuming that the gray range of the image is from 1 to L, at this point the threshold k is used to divide it into two groups: $C_0=\{1\sim k\}$ and $C_1=\{k+1\sim L\}$

The probability distribution of the two groups is

$${\omega }_0=\sum _{i=1}^k{P}_i.$$

(1)

$${\omega }_1=\sum _{i=1+k}^L{P}_i$$

(2)

Then the gray average of the two sets is

$${\mu }_0=\sum _{i=1}^k\frac{i{P}_i}{{\omega }_0}$$

(3)

$${\mu }_1=\sum _{i=1+k}^L\frac{i{P}_i}{{\omega }_1}$$

(4)

The average gray value of the whole image is shown

$$\mu ={\mu }_0{\omega }_0+{\mu }_1{\omega }_1$$

(5)

The inter class variance between the two groups as follow

$${\sigma }^2\left(k\right)={\omega }_0{\omega }_1({\mu }_1-{\mu }_0)$$

(6)

After the binaryzation of the image, in order to carry out the next step of feature extraction, segmentation needs to be carried out. Then the method used in this paper is the region growing method. Select a seed point to be detected the number of regions in the image, and then search in the eight neighborhood this seed point. If the gray value of a growth point value and seed point value is less than a threshold, it shows that the growth point and its seeds are in the same area.

The most important part is the identification of the coded layer, the starting layer, and the marker layer. Since the coded schemes in this paper are composed of dots or circles, the information in the image can be judged by roundness, so that it can be used to distinguish whether it is the part of the coded mark. The criterion of roundness is as follows:

$$C=\frac{P^2}{4\pi A}$$

(7)

P stands for perimeter, and A stands for area, C is the roundness of an image, the more nearly 1, the closer to the circle. The difference between the starting point and the positioning point is mainly the size of the area. Then we can get a single coded mark point information by clustering the distance between the location of the encoding layer and the distance between the central points. First detected is the starting point information, whose diameter is designed to be one-third of the center point. Then, the diameter of the surrounding code points is one-half of the center point, by which the number of code points can be determined.

The final step is to decode the number of the encoded marker point by the position information of the code point. Taking a 12-bit code point as an example, as shown above Figure 2, since we have determined the number of coded bits and the location of the code point. The difference between the position information of the starting point and the position information of the center point can decode the rotation angle of the marking point, which is used to realize the rotation invariance of the article coding scheme. The first bit of the code point is in the same line as the start point and is closest to the start point. Then, the number of the marker points is calculated clockwise. Here, the decoded value is composed of a 12-bit binary number, wherein if the code point is a hollow dot, it is 1 and the solid dot is 0. Finally, we decode the data represented by the coded marker points for subsequent matching and reconstruction.

3. Verification Experiments of Coded Mark Points

In order to verify the stability and advantage of the designed coded mark points, the three aspects included illumination conditions, shooting angle and the size of marker points was analyzed, and some conclusions are summarized by this experiment. Here, the same preprocessing and edge detection are performed in this paper with schneider’s coding scheme to prevent the detection results from being affected by different algorithms. In this experiment, As shown in Figure 3, 12-bit coded marks of the proposed scheme are compared with the Schneider scheme to verify the advantages and disadvantages of the scheme compared with the Schneider scheme.The camera (IO industries 2M360-CL) used f = 25 mm lenses in the experiment, and the resolution of the camera has reached 2088 × 1024 pixel.

Firstly, taking into account the impact of illumination on the mark point, the experimental method and the decoding result of the Schneider scheme are taken under the average gray value of 70–220. As shown above in Figure 4, under the influence of different illumination, especially under the condition of over saturation or under saturation illumination, the two coded schemes all show good stability.

Secondly, under the same illumination conditions, by adjusting the shooting angle, comparing the stability of the scheme and the Schneider coded scheme, we can verify the advantages and disadvantages of the experimental scheme. As shown in Figure 5, from the above decoding results we can see that the coded marks designed in this paper can be decoded very well under the shooting angle of 0–75 degrees, however, the decoding stability of Schneider scheme becomes worse when the shooting angle is greater than 45 degree angle. Therefore, the coded scheme designed in this paper is more robust than the Schneider coded scheme in the case of a large shooting angle.

Thirdly, The size of the marking points of the two schemes is discussed under the condition that both the illumination condition and the shooting angle are the same. In the experiment, the size of the mark point are 10, 15 and 20 mm, and the experiment is carried out on the condition that the average gray value is 160. As the resolution of the picture is 1280 × 1024, the percentage of the pixels are 0.21%, 0.34% and 0.55% when the mark points are 10, 15 and 20 mm.

As shown in Figure 6, in the case of size 10 mm, the coded scheme of this paper is not effectively detected; in the case of 15 and 20 mm, the two schemes can be decoded. By analyzing the experimental procedures, the reason for this problem is that the starting point is too small to detect. Therefore, the Schneider coded scheme has some advantages over the size of coded mark points. Lastly, by comparing the decoding stability of the coded scheme of this paper and the Schneider encoding scheme, we can obtain the results shown in the following table. Here, the standard of evaluation is the bit error rate.

Through

Table 2. Comparison coded points’ angle.

The Recognition Rate under the Shooting Angle (%)	$0^{\circ }$	${30}^{\circ }$	${45}^{\circ }$	${60}^{\circ }$	${75}^{\circ }$
Pixel ratio 0.21% of Schneider	100	100	35	17	8
Pixel ratio 0.34% of Schneider	100	100	58	17	17
Pixel ratio 0.55% of Schneider	100	100	92	25	17
Pixel ratio 0.21% of Initial Dot	0	0	0	0	0
Pixel ratio 0.34% of Initial Dot	100	100	100	25	17
Pixel ratio 0.55% of Initial Dot	100	100	100	100	100

, it shows that the angle of the shooting affect the recognition of the coded mark points and the size of the coded mark point also affects the stability of the detection. The larger the size of the coded mark point, the more easily it is to recognize the coded mark point; the larger the angle of the shot, the worse the stability of the coded mark recognition. When the size of the coded mark point is 15 and 20 mm, the recognition rate of this coded scheme is generally higher than that of the Schneider coded scheme. In particular, when the size of the coded mark point is 20 mm, the recognition rate of the coded mark point below 75 degrees can reach one hundred percent. However, the coded mark of this paper is very poor when the size is 10 mm. In general, the robustness of the Initial Dot coding scheme is stronger than the robustness of the Schneider.

4. Experiment and Result Analysis

4.1. Verification Experiment about Mark Points on Flexible Solar Panel

The coded mark points designed in this paper have a great increase in the number of coded and the angle of measurement compared to the Schneider coded scheme. In view of the three-dimensional measurement of flexible solar panels, because the size of flexible solar panels is too large, we can use the coded mark points designed in this paper. Here, this paper designs a measurement scheme based on binocular stereo vision. As shown in Figure 7. This system includes a flexible solar panel, 572 coded mark points, two CMOS cameras (IO industries 2M360-CL) with f = 12 mm lenses, a light and a computer.

The experimental procedures are as follows:

• Calibrate The rotation matrix and the translation matrix for the two cameras.
• Shoot the encoding marks on flexible solar panel and decode mark points.
• Move the camera and repeat the above b steps 6 times.
• Complete splicing according to the repeated marks.

Result Analysis

As shown in Figure 8, this is the actual experimental device of two methods. After decoding and recognition of the encoding point, the results are shown in

Table 3. The decoding of six parts.

Serial Number	Number of Recognition	Total Number	Recognition Rate (%)
A	172	180	95.6%
B	167	180	92.8%
C	178	180	98.9%
D	173	180	96.1%
E	174	180	96.7%
F	172	180	95.6%

. The average recognition rate of six surfaces is 96.0%. Experiments show that the recognition effect is stable and can distinguish the influence of interference points. The seven parameter method can be used to stitch the six surfaces, because the two adjacent surfaces have many coincident points. Finally, the three-dimensional reconstruction results of the six times experiments are as follows Figure 9:

The data in the figure above are calculated separately from the six measurements, and each part has the ability to facilitate the stitching of 3D data because of repeated coded marks. As can be seen from the diagram, the surface of each part of the middle part is relatively neat and can identify completely, while the part of the edge is not recognized. The illumination and camera shooting angle have great influence on the experiment. As can be seen from the diagram, the points received by the six surfaces are approximately similar to one plane, and there is a partial error on the splicing of the flexible plate. There are 572 points on the flexible board, 562 points are identified, and the recognition rate is 98.3%.

After that, the flexible solar panels are scanned using a scanner in order to verify the reliability of the scheme. Then, we fit the two sets of data to the same coordinate system. Since there is no corresponding matching point here, the software is used to match the corresponding approximation. As shown in

Table 4. The evaluation parameters of fitting result.

Evaluation Parameters	Average Error	R-Square
	3.217 mm	0.998

, the R-Square values obtained by fitting the two faces to each other are above 0.995, and the average error of fitting is 3.217 mm. R-Square indicates the correlation between two face types. The closer the value is to 1, the higher the correlation. Because the R-S value measured in this paper is 0.998, it is shown that the results of the two methods are basically similar, thus proving that the reconstructed results are reliable. In this experiment, due to the process of coordinate splicing, the measurement error is accumulated and finally reflected above the average error. However, this experiment can basically prove that this program can help the surface measurement of large flexible solar panels. In general, this experiment can prove that the scheme has good robustness in decoding, and can perform coordinate splicing according to these points, which has great significance for local position tracking.

4.2. Comparative Experiment

In order to compare the differences of the Schneider method, this paper has done some comparative experiments. The experiment will measure a three-dimensional surface of the wall of about 1000 mm × 500 mm, using the wall scanned by the three-dimensional scanner as a real value, and simultaneously measuring the area using a 10-bit Schneider, a 12-bit Schneider and a 10-bit Initial Dot. According to the coding principle, the maximum number of codes that can be encoded by the 10-bit Schneider coding scheme is 107, and 104 kinds of experiments are used, which basically reach its maximum value. Similarly, the 12-bit Schneider uses 315 code points, which basically reaches its maximum value of 325. However, due to the large number of 10-bit Initial Dot encoding schemes, only 525 code points can be used in the same field of view, which is half of the maximum code value. Then, we use the following three types of marker points to perform the following steps:

• Paste the marker points and calibrate the parameters of the two cameras.
• Shoot the encoding marks on the wall and decode mark points.
• Move the camera and repeat the above two steps 4 times.
• Complete splicing according to the repeated marks.

Result Analysis

After decoding and recognition of the encoding point, the results are shown in

Table 5. Analysis of decoding results of 3 experiments.

Target	Number of Recognition	Total Number	Recognition Rate (%)
10-bit Schneider	94	104	90.38%
12-bit Schneider	300	315	95.24%
10-bit Initial Dot	525	530	99.05%

and Figure 10. In the range of 1000 mm × 500 mm field of view, the 10-bit Initial Dot encoding scheme can fill the entire field of view and only use 525 points, However, the 10-bit Schneider encoding scheme can only cover part of the field of view. In the same field of view, the number of 10-bit Initial Dot is the most, and the recognition is best. This experiment proves that the numbers of codes of Initial Dot are more than that of the Schneider of the same bit, and the decoding effect is better than that of the Schneider scheme under the same conditions. Then, the scatter is reconstructed in three dimensions and spliced into the same coordinate system.

As shown in Figure 11, by fitting results of a three-dimensional scatter and scanning, the results are shown in

Table 6. Fitting results of the three scattered points.

Target	Average Error	R-Square
10-bit Schneider	2.932 mm	0.995
12-bit Schneider	2.117 mm	0.995
10-bit Initial Dot	1.324 mm	0.998

. Since the result of the fitting is affected by the three-dimensional stitching, the more points between the coordinates, the better the fitting effect is obtained. Therefore, the number of Initial Dot encoding schemes is larger than that of the Schneider encoding scheme, the effect of three-dimensional coordinate stitching is better, and the average error is also lower. The fitting results show that the Initial Dot coding scheme is more effective than the Schneider scheme. Due to the limitations of the Schneider coded scheme, there will be duplicate points in the case of a larger field of view, which in turn will affect automatic matching. However, the Initial Dot coded scheme will hardly show a similar situation.

5. Conclusions

In this paper, a new encoding scheme was designed and applied to flexible solar panels to measure large three-dimensional shapes. Compared with the traditional coded scheme, the number of the coded mark of Initial Dot scheme is improved by about ten times. In addition, through the comparison between the scheme and the Schneider scheme, it is proved that the scheme has better stability than the Schneider scheme when the coded size is above 15mm, especially when the shooting angle is greater than 45 degrees. Then, another experiment was used to reconstruct the surface of the flexible solar panel through 572 coded mark points. When fitting the results of the 3D scanner with the experimental results, it turns out to be the case that the experimental scheme is related to the scanner, so this result is feasible. The last experiment proves that the Initial Dot encoding scheme has more code numbers under the same conditions than the Schneider encoding scheme, and the recognition effect is better, reaching 99.05%. In the end, because the Initial Dot encoding scheme has more coded points, it has a better reconstruction effect than the Schneider encoding scheme.