Using the hybrid network proposed in this paper to detect the piston errors of sub-mirror consists of three steps. Firstly, establish the data sets for network training. Within the coherent length range of the input broadband light, multiple groups of piston errors are randomly generated and loaded onto the corresponding sub-mirrors. By taking the piston errors to the established simulation segmented optical system, the corresponding focal plane degraded images are generated for training the Resnet network, which is used to detect the signs of piston errors. Then, taking the absolute value of the piston errors and the optical system parameters into Equation (10), the values of the MTF sidelobes can be obtained to train the BP network. Secondly, train the hybrid network. The generated focal plane degraded images and the signs of the corresponding piston errors are used as the input and output of the Resnet network, respectively, and with the cross-entropy loss function used as the index function of the network optimization, the Resnet network can be well trained through a gradient-based stochastic optimization algorithm: Adam. Then, the calculated values of the MTF sidelobes and the absolute values of the corresponding piston errors are used as the input and output of the BP network, respectively, and by setting up a specific training algorithm, the BP network can also be well trained. Finally, after the hybrid network training is finished, by taking a degraded focal plane image into the Resnet network, the signs of all sub-mirrors’ piston errors can be obtained. Then, performing the Fourier transform on the focal plane image obtains the values of the MTF sidelobes which can be input to the BP network; the absolute values of all sub-mirrors’ piston errors can be solved. One thing to note here is that the corresponding relationship between the MTF sidelobe and its related sub-mirror should be established in advance, as the absolute values of all the sub-mirrors’ piston errors can be measured simultaneously by one CCD broadband image. Thus, the measurement of the piston error with high precision and a wide detection range can be realized by combining the output of the two networks.
We first conducted simulation experiment analysis on the segmented telescope system composed of two hexagonal sub-mirrors, as shown in
Figure 2. Then, the simulation experiments were processed to multiple sub-mirrors (
n > 2) segmented telescope system which was composed of four hexagonal sub-mirrors. In the end, we perform some comparation work between our method and the work published by Ma Xiafei et al. in paper [
17], since they also used a single wide-band image of a point source to perform piston sensing by a neural network.
3.1. Simulation on Two Sub-Mirrors Segmented Telescope System
MATLAB software was used to build the simulation optical system model consisting of two hexagonal sub-mirrors as, shown in
Figure 2. The specific system parameters were the same as those set in
Section 2.1. During the simulation experiment, the left sub-mirror of the system was set as the reference sub-mirror, and a series of piston errors were introduced on the right sub-mirror. Here, we firstly generated 60,000 sets of piston errors in [−10~10] µm randomly and 60,000 sets of focal plane degradation images were obtained by MATLAB simulation. Then, by taking the corresponding piston errors into Equation (10), we obtained 60,000 sets of the MTF sidelobes values. The generated data sets were divided into three groups, namely the training set, verification set, and test set. The proportion of the three parts was 65%:20%:15%, namely 39,000 groups for training, 12,000 groups for verification, and 9000 groups for testing.
Then, the network training was be processed based on the obtained data sets. Here we built a Pytorch deep learning environment on an Ubuntu server equipped with Intel i7 CPU and Nvidia GeForce 2080 GPU both from the City of Santa Clare, CA, USA, to achieve the training of the hybrid networks. The Resnet network was first trained to predict the signs of piston error. The focal plane degradation images shown in
Figure 3 were used as the input of the Resnet network, and the signs of the piston errors were used as the output of the Resnet network, with the label ‘0’ representing positive and the label ‘1’ representing negative. Each network was trained with 300 epochs, the batch size was set as 32, and the cross-entropy loss function was used as the index function. The network parameters were updated by back-propagation and the evaluation of the trained network was realized through the verification set. At the same time, in order to improve the network efficiency, a kind of strategy known as batch normalization was used between convolutional layers to prevent gradient disappearance.
Four Resnet networks with different depths, including Resnet18, Resnet34, Resnet50, and Resnet101, were trained here to test their prediction accuracy of the piston error signs.
Figure 9 shows the loss functions of the training set and the verification set, where the horizontal axis represents the number of trainings and the vertical axis represents the cross-entropy loss function. We can see that after 300 rounds of training, the loss function gradually declined and finally reached a stable state. From the loss function curve of the training set, the Resnet networks fitting degree gradually increased with the deepening of the network, but from the loss function curve of the verification set, with the deepening of the network, the generalization ability of the networks Resnet50 and 101 were much lower than that of Resnet18 and 34, resulting in the loss function oscillation of the verification data set. Then, we use the test data set to verify the piston error sign prediction accuracy of the four Resnet networks with different depths. The test result is given in
Figure 10 where the prediction accuracy of the piston error signs is given by the number of correct predictions in the test set divided by its total number. From
Figure 9 and
Figure 10, we can see that the Resnet34 network had the highest generalization ability and the highest prediction accuracy. Therefore, the optimal model for the prediction of the signs of piston error for the two sub-mirror segmented system is the Resnet34 network.
Then, the BP neural network was trained to solve the absolute value of the piston errors. The values of the MTF sidelobes shown in
Figure 5 were taken as the input of the network, and the corresponding absolute values of the piston error were taken as the output of the network. The number of neurons in the hidden layer of the BP network was set as 10, the node transfer function of the hidden layer was logsig function, the node transfer function of the output layer was purelin function, and the learning training function was traindx, which is a variable learning rate momentum algorithm. The training results of the network are shown in
Figure 11.
Figure 11a shows the loss function changing with the number of iterations and
Figure 11b provides the error distribution between the expected value and the actual network output value in the form of a histogram. According to the training results, the RMSE between the expected value and the actual output of the network in the training set, the verification set, and the test set are 2.235 × 10
−5 μm, 2.734 × 10
−5 μm, and 1.873 × 10
−5 μm, respectively. It was proved that the BP neural network proposed here can be used to calculate the absolute value of piston error with high precision.
After the hybrid network is well trained, the actual network performance can be tested. Here, another 500 new focal plane images were generated from the simulation optical system for testing. In order to approximate the actual imaging environment, Gaussian distribution noise with a mean of 0 and a variance of 0.05 was introduced into the simulated PSF images. Furthermore, since tip–tilt errors cannot be completely corrected, the tip–tilt errors were also added to each sub-mirror during the generation of the focal plane degradation image, where the total RMSE value of the added tip–tilt errors was 0.01 λ. Since we only considered the co-phase errors like Keck where all sub-mirrors were assumed to have the same shape and be perfect without high-order aberrations except pistons and tip–tilts, higher order aberrations of each sub-mirror were not considered here.
Figure 12 shows 30 groups of the experiment results randomly selected from the whole 500 groups. The piston error detection accuracy was given by the difference between the measured piston error and the set piston error. According to the error analysis, the RMSE of the difference values of the 30 experimental results is 1.26 nm where the RMSE is defined as
;
N is the number of experiment groups.
Figure 13 shows the piston error detection results of all the 500 groups of simulation experiments, among which the difference in the detection results within 494 groups is 10 nm, and the RMSE of the 494 groups is 1.76 nm. The left 6 groups of difference values of the piston error detection results are very large, far greater than the mean difference values of the 494 groups of piston error detection results. This is because the signs of piston errors were predicted wrongly by the Resnet network, and the measurement error was basically twice of the set piston error.
Figure 13a shows the piston error measurement results of all the 500 groups of simulation experiments,
Figure 13b shows the measurement errors of these 494 groups in the form of a scatter plot, and
Figure 13c takes the absolute value of measurement error as the horizontal coordinate to give the statistical results of the 500 groups of piston error detection experiments. It can be seen that the detection range of this method is very wide, ranging from −10 μm to 10 μm, reaching the whole coherence length of the input broadband light. The measurement accuracy is very high, and the probability of measurement error less than or equal to 10 nm is as high as 98.8%. Under the characteristics of our workstation, the time consumed by the Resnet 34 network to detect the signs of piston error from a degraded focal plane image was 3.1 ms, the time consumed of the BP network to detect the absolute value of one piston error from the MTF sidelobe obtained by Fourier transformation of the degraded focal plane image was 0.45 ms. So, the detection time of the proposed algorithm is quite fast as long as the hybrid network is well-trained.
3.2. Simulation on Four Sub-Mirrors Segmented Telescope System
In this part, we use a four sub-mirror segmented telescope system as an example to test the performance of the proposed algorithm on detecting the piston errors of a multiple sub-mirrors (
n > 2) segmented telescope system at one time. When using the Resnet network to detect the signs of piston errors of the multi-submirrors, the output of the Resnet network was set to 2
(N−1), which corresponds to the piston error signs of all sub-mirrors (except the reference sub-mirror). When the BP network was used to solve the absolute value of the piston errors of multiple sub-mirrors through the MTF sidelobes, according to paper [
16], we used a segmented telescope system composed of
N sub-mirrors, there are
N2 sub-MTFs. In the spatial frequency domain, the
N sub-MTFs overlapped at the position where the center spatial frequency was zero to form the central peak, while the other
N(N − 1) sub-MTFs distributed around the central peak to form the sidelobes. Every pair of sub-mirrors produced a pair of MTF sidelobes; the sidelobes symmetrically distributed on both sides of the central peak. If all of the sidelobes did not overlap, their amplitudes could be obtained at the same time by one CCD image, hence the absolute value of the piston errors of all sub-mirrors can be retrieved at the same time by inputting the peak height of the sub-MTFs corresponding to each sub-mirror into the trained BP network. Combining the outputs of the two networks, the piston errors of all the segmented sub-mirrors can be solved at one time using a focal plane degradation image.
The established simulation model of a four sub-mirrors segmented telescope system in MATLAB is shown in
Figure 14. The reason why the system model was built like this was to prevent the MTF sidelobes from overlapping. Here, the wavelength of the input broadband light was also centered at 632.8 nm with a 20 nm bandwidth. The No. 1 sub-mirror was set as reference pupil and the piston errors randomly generated between −10 µm and 10 µm were introduced on sub-mirrors No. 2, 3, and 4, thus the degraded focal plane images with different piston errors could be obtained.
Figure 15 shows several degraded images on the focal plane corresponding to the several sets of introduced piston errors on multiple sub-mirrors.
During the simulation experiment, 60,000 sets of piston errors were randomly generated and introduced into the simulation optical system; 60,000 groups of focal plane degradation images can be obtained. The focal plane degradation image was used as the input of the Resnet network, and the signs of the piston errors of the 3 sub-mirrors were used as the output. Here, the Resnet network output was divided into eight categories. When constructing the labels of network outputs, we used 0 to indicate a positive sign and 1 to indicate a negative sign, which is similar to the binary encoding process and is shown in the
Table 1. The other setting parameters and training process of the Resnet network were the same as those of the two sub-mirror system, which will not be repeated here.
When training the BP network, we took the number of sub-mirrors,
n = 4, the wavelength of the input broadband light, and the 60,000 sets of piston errors generated randomly into Equation (10). We used the modulus values of the
MTF sidelobes calculated directly from Equation (10) as the input matrix, while the corresponding piston errors were used as the output matrix of the training network.
Figure 16 shows the relationship between the modulus values of the
MTF sidelobe directly calculated by Equation (10) and the piston error of sub-mirror in the form of a curve. The obtained 60,000 data sets were used to train the BP network. The parameter setting and the training process of the BP network were the same as that of the two sub-mirrors segmented optical system, which also will not be repeated here.
In order to solve the absolute value of the piston errors of the multiple sub-mirrors simultaneously from one focal plane image based on the well-trained BP network, the mapping relationship between sub-
MTFs and their corresponding sub-mirrors had to be established in advance. The system
MTF of the four sub-mirrors segmented optical system in this experiment included a central main peak and 12 (N (N − 1) = 4 × (4 − 1) = 12) sidelobes, which is shown as
Figure 17 with color-marks. When sub-mirror No. 1 was set as the reference mirror, sub-mirror No. 2 produced the red sub-
MTFs, sub-mirror No. 3 produced the green sub-
MTFs, and sub-mirror No. 4 produced the yellow sub-
MTFs. The six peripheral blue sub-
MTFs were modulated by the piston errors of the other two sub-mirrors simultaneously, except the reference mirror, which cannot be used to measure the piston error of each sub-mirror. In order to ensure the effectiveness of the proposed method, one of the most important things was to avoid any overlap of the
MTF sidelobes, which were formed by each pair of sub-waves sampled by the corresponding pair of the sub-apertures during the establishment of the mapping relationship. Thus, the arrangement of multiple apertures should be designed scientifically. The detailed arrangement rule can refer to paper [
16]. It should be noted that the relationship between the
MTF sidelobes and the absolute value of the piston error of each sub-mirrors was the same (as shown in
Figure 16), thus it was not necessary to conduct training for all three sub-mirrors at one time, but to use one data set to train a single BP network. Then, by inputting the modulus of sub-
MTF corresponding to each sub-mirror directly, the piston error absolute value of each sub-mirror could be obtained. This can reduce the difficulty of network training and improve the detection accuracy.
After the hybrid network is well trained, the actual performance of the network should be tested. We randomly generated multiple sets of piston errors in the range of [–10~10] um and introduced them into sub-mirrors No. 2, 3, and 4 separately, then the focal plane degradation image could be obtained. In order to be closer to the real imaging situation, Gaussian noise with mean 0, variance 0.05, and tip–tilt errors with RMSE 0.01 λ were added to the generated the focal plane degradation images.
We also generated 500 focal plane images of the system for testing, and the test results are shown in
Figure 18. The piston error detection results of sub-mirror No. 2, 3, 4, are shown in
Figure 18a–c, respectively,
Figure 18d shows the distribution histogram of the RMSE values of all three sub-mirrors’ piston errors detection results in the 500 groups. It can be seen that with the increasement in the number of sub-mirrors, the classification number of the Resnet network became greater, and the detection accuracy of the signs of the piston errors decreased, but the detection accuracy of the BP network had no change. The probability of a measurement error less than or equal to 10 nm could still be maintained above 85%.
3.3. Comparation Work
Finally, we compared our method with the work published by Ma Xiafei et al. in paper [
17], because they also used a single wide-band image of a point source to perform piston sensing by neural network, where a DCNN network was used to directly learn the mapping relationship between the focal plane degradation images and the piston errors of sub-mirrors. The typical architecture of DCNN comprises one input layer, several convolutional layers, pooling layers, fully connected layers, and one output layer, among which convolutional layers together with pooling layers play a role as feature extractors. The DCNN Ma et al. used has 26 layers, the detailed structure of which can be referred to in
Figure 2 of paper [
17]. The main difference between Ma’s method and ours is that they did not establish the precise theoretical relationship between
MTF and piston error based on Fourier optics, thus the piston error detection was realized based on a single neural network. This increased the training difficulty of the network and could not guarantee the high piston error detection accuracy.
The comparison experimental results aimed for the two-pupil segmented systems and the four-pupil segmented systems are shown in
Figure 19. It should be noted that the relevant experimental parameters must be set to the same during the comparison. Here, we used the same parameters as Ma’s work for both the two-pupil segmented system and the four-pupil segmented system, including the 500–600 nm broadband input light, 500 mm focal length, and 1.67 μm pixel size of the CCD. It can be seen that the piston error detection accuracy of the method proposed by us is generally higher than that of the method proposed by Ma et al. This is because we constructed the theoretical relationship between the system
MTF and the piston errors and used the modulus of the
MTF sidelobes as the network input, while Ma et al. directly used the focal plane image as the input of the network. However, the maximum value of piston error detection difference of our proposed method is much larger than that of Ma’s method. This is because when using the Resnet network to detect the signs of the piston error wrongly, the detection difference value was almost twice that of the set piston error value. So, how to improve the detection accuracy of the signs of piston error is quite important in our next work.