Star-Based Calibration of the Installation Between the Camera and Star Sensor of the Luojia 1-01 Satellite

Abstract

Ground control points (GCPs) are generally used to calibrate the installation between the camera and star sensor of a satellite in orbit and improve the geometric positioning accuracy of the satellite. However, the use of GCPs for high-frequency calibration is difficult, and it is particularly difficult to acquire accurate GCPs for the image of a nightlight satellite. In this study, we developed a camera-star sensor installation calibration method that eliminates the need for GCPs. In the proposed method, the camera and star sensor lenses are simultaneously pointed at the star, and the camera-star sensor installation is accurately calibrated by processing the star map obtained by the camera and star sensors. Reference data such as road network and Moon position data were used to verify the proposed method and evaluate its positioning accuracy. The results of the application of the method to the positioning of the Luojia 1-01 satellite indicated an accuracy within 800 m, which is comparable with that of the traditional method.

1. Introduction

The stress release that occurs during the launch of a satellite and the change in the physical environment when the satellite enters orbit alter the installation between the satellite’s camera and star sensor, resulting in reduced geometric positioning accuracy of images obtained by the satellite. Generally, the orbit determination accuracy may be better than the level of sub-meters, indicating that the main determinant of the positioning accuracy is the attitude accuracy [1,2,3]. Therefore, after sending a satellite into orbit, it is usually necessary to calibrate the installation between the camera and the star sensor. Multiple studies have been performed on in-orbit geometric calibration using ground control points (GCPs). This GCP-based calibration method acquires the precise determination of the parameters of the camera-star sensor installation, thereby improve the image positioning accuracy.

Through a GCP-based method, the Advanced Land Observing Satellite (ALOS) launched from Japan in 2006 achieved an image absolute positioning accuracy of 8 m (3.2 pixels, 2.5 m resolution) [4,5,6]. Similarly, the Pleiades 1A satellite launched from France in 2011 achieved an absolute positioning accuracy of 5.6 m (8 pixels, 0.7 m resolution) [7]. In 2012, the Ziyuan 3-01 satellite was launched from China and achieved an image absolute positioning accuracy of 10 m (4.8 pixels, 2.1-m resolution) by GCP-based method [8,9,10]. The SPOT 6/7 satellite launched from France in 2012/2014 similarly achieved an image absolute positioning accuracy of 7.8 m (3.9 pixels, 2 m resolution) [11]. The Worldview-3/4 satellite launched in 2014/2016 achieved an image absolute positioning accuracy of 2.7 m (8.7 pixels, 0.31-m resolution) [12,13,14]. To eliminate the need for GCPs, Kubik et al., adopted the extremely agile maneuvers of the Pleiades satellite and developed an auto-calibration imaging mode that enables calibration of the camera-star sensor angle error without the use of GCPs [15,16,17,18]. Jiang similarly developed an autonomous calibration method based on agile-maneuver imaging and verified it by simulation [19,20].

However, single calibration cannot guarantee the long-term accuracy of satellite positioning without the use of GCPs. The seasonal illumination changes and temperature alternation between warm and cold that occur during an orbital cycle cause the attitude angle of the camera-star sensor to continuously change over a long period, eventually invalidating the attitude angle or installation obtained by a single calibration, resulting in reduced positioning accuracy. Degradation of the attitude determination accuracy of the ALOS is due to the relative installation changes that occur between its star sensors, which have periodic terms for an orbital cycle and exhibit very long-term drift. Hence, calibration activities are ongoing on GCP-based methods for improving positioning accuracy [21,22]. It has also been found that the positioning accuracy of the Ziyuan 3 satellite deteriorates with increasing time interval between the acquisitions of the calibration and verification images [23,24,25].

To address the problem of the installation change that occurs over an extended period, it is necessary to calibrate the camera-star sensor angle at short time intervals. However, the process of the acquisition of high-precision GCPs is expensive and difficult, and more so for global control points. Moreover, the occurrence of suitable imaging weather conditions without cloud and rain is limited. This makes the use of the existing GCP-based method for repeated calibration at short time intervals challenging. Further, in the case of night imaging satellites, the lack of control data makes it more difficult to achieve calibration through traditional methods. In addition, the auto-calibration method, which similar to that used by the Pleiades satellite, does not require control data, but strong agility of the satellite for the acquisition of a couple of images from the same orbit with 180° direction. The method is therefore poor applicability for the present purpose.

In the present study, we developed a non-GCP method for the calibration of the camera-star sensor of the Luojia 1-01 nightlight satellite. In the proposed method, a camera and star sensor are simultaneously used to obtain images of stars, and the star map so acquired is processed and used to accurately calibrate the installation of the camera-star sensor.

Figure 1A illustrates the conventional imaging model of the Luojia 1-01 satellite. The camera C is the optical axis pointing to a ground target, while the star sensor S is the optical axis pointing to stars. In an inertial system, the attitude of the star sensor is obtained from the ascension and declination of the stars acquired by the star sensor, while the attitude of the camera is obtained by pointing it to a known target (GCP). This enables the determination of the installation angle between the camera and the star sensor. The process corresponds to that of conventional geometric calibration, referred to here as the GCP-based method. The method proposed in this paper is illustrated in Figure 1B. As the satellite maneuvers with some combination of roll, pitch and yaw at a safe speed, it maintains its inertial motion when it assumes a stable positioning direction. The attitudes of the camera C and star sensor S in the inertial system are obtained when they simultaneously photograph stars. The angular relationship between the star sensor and the camera is then deduced and calibration is performed, referred to here as the Star-based method. Because there are no clouds in outer space, the satellite does not have to maneuver in a specific direction, but only needs to keep its camera and star sensor pointed at the star. Most satellites are able to satisfy this requirement.

The Luojia 1-01 satellite was successfully launched on 2 June 2018. The satellite was an experimental nightlight remote sensing satellite developed at the Wuhan University, China, with a resolution of 129 m at a sub-satellite point [26,27]. Star data collected by the camera of the Luojia 1-01 satellite were used in the present study. The details of the satellite and its camera are presented in

Table 1. Luojia 1-01 satellite parameters.

Parameter	Value/Description
Orbit type	Sun-synchronous orbit
Orbit height	645 km
Ground sampling distance	129 m (sub satellite point)
Ground swath	264 km × 264 km
Spectral range	480–800 nm
Camera focal length	55 mm
Camera pixel size	2048 × 2048
Camera detector size	11 μm × 11 μm
Camera field of view (FOV)	32.3°
Star sensor sampling frequency	1 Hz
Star sensor FOV	90°
Imaging mode	Nightlight mode/daytime mode/Moon mode/star mode
Attitude maneuverability	Pitch axis > 0.9°/s
Total satellite mass	< 20 kg
Envelope size	520 mm × 870 mm × 390 mm

. Regarding the imaging mode in the table, the nightlight and daytime modes respectively indicate that the camera is pointing to the earth at night and during the day while imaging; the Moon mode indicates that the camera is pointing to the Moon while imaging; and the star mode indicates that the camera is photographing stars.

2. Materials and Methods

2.1. Star-Based Calibration Principle and Verification Methods

After photographing stars with the camera and star sensor, the star map is processed to determine the attitude of the camera and star sensor in the inertial system. This specifically includes the coordinate extraction of star points from the star map, acquisition of star position in the celestial coordinate system, determination of the attitude in the inertial system by the star sensor and camera, and installation calibration between the star sensor and camera. All these processes include one or two smaller processes, as presented in the calibration methods part of Figure 2. The updated installation parameters of Luojia 1-01 are subsequently acquired. In the present experiment, the geometric positioning accuracies of other images were used to verify the effectiveness of the parameters. Two verification method were considered. In the first method, which is the traditional verification method, check points (CPs) were acquired through the homonymy points between the ground reference images and nightlight images. A ground geometric model and installation parameters were then used to calculate the coordinates of different points. The offset of the calculated coordinates relative to the coordinates of the CPs is a measure of the positioning accuracy. The employed model utilized CPs and did not require the use of GCPs, and the process can thus be referred to as “Geometric positioning accuracy verification without GCPs.” In the second considered verification method, the Moon center was used to replace the CPs. This was a novel approach. The integrated flow chart in Figure 2 details the calibration and verification methods utilized in this study. The images obtained by Luojia 1-01 and other external data used in the study are introduced in the next subsection. Further details are presented farther in the paper.

2.2. Introduction to the Experimental Data

2.2.1. Luojia 1-01 Images

Eleven images obtained by Luojia 1-01 were used in this study. The star mode image was used for calibration and acquisition of the installation parameters from the other images. The nightlight mode and Moon mode images were used for verification. The effective duration of the star data obtained in this experiment was consisted of 146 scenes of star maps which imaging approximately 15 s (0.1 s exposition time per scene). The data details are presented in

Table 2. Luojia 1-01 experiment data.

Cumulative Day	Imaging Start Time	Imaging Mode	Imaging Angle/° (Roll, Pitch, Yaw)
8	2018-06-10T08:04:10.65	Moon mode	−84.81, −20.98, −13.07
25	2018-06-27T23:43:24.15	Moon mode	−85.09, −13.96, 18.77
26	2018-06-28T11:20:46.18	Daytime mode	1.06, −12.62, 5.42
174	2018-11-23T22:45:05.29	Moon mode	−120.84, −28.84, 29.07
241	2019-01-29T14:56:37.23	Nightlight mode	−11.39, −0.37, −1.56
241	2019-01-29T14:57:02.26	Nightlight mode	−11.39, −0.37, −1.56
282	2019-03-11T14:56:18.15	Nightlight mode	−7.55, −0.21, −1.69
282	2019-03-11T14:56:38.17	Nightlight mode	−7.53, −0.23, −1.68
294	2019-03-22T17:40:19.18	Moon mode	−141.02, −43.05, 53.39
303	2019-04-01T03:33:15.69	Star mode	−98.13, −45.44, 15.71
353	2019-05-21T15:03:00.15	Moon mode	163.62, −44.94, 142.04

.

2.2.2. External Data

The employed star catalog data was obtained by the Smithsonian Astrophysical Observatory (SAO) in the J2000 epoch. Based on the limiting magnitude of the SAO catalog, which contains about 300,000 stars, stars with a magnitude of less than five were selected. Based on the proper motion correction algorithm, which depends on the position of the stars, approximately 1500 navigation stars were selected from the experimental catalog data [28].

The Earth orientation parameters (EOP) of the Earth reference system relative to the celestial reference system were obtained from the International Earth Rotation and Reference Systems Service. The parameters indicate the precession, nutation, and pole shift of the Earth and can be used to calculate the rotation matrix for the J2000 coordinate system to the WGS84 coordinate system at any epoch since 1960 [29].

The Ephemeris DE421 published by the Jet Propulsion Laboratory describe the motion of large bodies in the solar system, including the Moon and Earth. The data were acquired by different methods such as laser and radar ranging and are applicable to 1900–2050. They can be used to calculate the position and velocity of various celestial bodies in the solar system relative to the Earth at any given time [30].

The verification reference images were open access images obtained from Landsat, while the elevation reference data were obtained by a 90-m shuttle radar topography mission digital elevation model (SRTM DEM). The Landsat images were 30-m resolution and were acquired during the daytime. CPs could be acquired from the homonymy points between the verification images and Landsat images. The Landsat images and SRTM DEM data were with well-characterized geometric accuracy, which are adequate for positioning accuracy verification of the 129-m resolution images acquired by Luojia 1-01 [31].

2.3. Star Coordinate Acquisition from the Star Map

Theoretically, the background of the star photographs acquired by Luojia 1-01 is black, and the star point is relatively bright in the star map. The coordinates of the star point can thus be extracted. However, due to sunlight on the side of the camera lens or stray light from the earth, the actual star map has a strong background noise. The multi-scene star map of the photographing period can be accumulated, and the background noise value can be determined by averaging the value for each pixel. The star points can therefore be extracted after eliminating the influence of the background noise [32]. The brightness of the star map is expressed as $f\left(x,y\right)$, and the background noise $B\left(x,y\right)$ extracted from the $k$-scene of the star map can be calculated as follows:

$$B\left(x,y\right)=\frac{{{\displaystyle \sum }}_1^{k}f\left(x,y\right)}{k}.$$

(1)

If $t$ represents the background threshold, threshold segmentation of the original image can be conducted, and the image $F\left(x,y\right)$ obtained after the thresholding can be expressed as follows:

$$F\left(x,y\right)=\{\begin{array}{rr}f\left(x,y\right),& f\left(x,y\right)\ge B\left(x,y\right)+t\\ 0,& f\left(x,y\right)<B\left(x,y\right)+t\end{array}.$$

(2)

Connected domain marks are used to identify pixels belonging to a given same-star point. The initial information regarding the position and brightness of each star point can thus be obtained. Using the grayscale square weighted centroid method [33], the centroid sub-pixel coordinates of the $i$th star, $\left(x_i,y_i\right)$, can be obtained as follows:

$$x_i=\frac{{{\displaystyle \sum }}_{m=m_1}^{m=m_2}{{\displaystyle \sum }}_{n=n_1}^{n=n_2}{\left[F\left(x_m,y_n\right)\right]}^{2}x}{{{\displaystyle \sum }}_{m=m_1}^{m=m_2}{{\displaystyle \sum }}_{n=n_1}^{n=n_2}{\left[F\left(x_m,y_n\right)\right]}^2},y_i=\frac{{{\displaystyle \sum }}_{m=m_1}^{m=m_2}{{\displaystyle \sum }}_{n=n_1}^{n=n_2}{\left[F\left(x_m,y_n\right)\right]}^{2}y}{{{\displaystyle \sum }}_{m=m_1}^{m=m_2}{{\displaystyle \sum }}_{n=n_1}^{n=n_2}{\left[F\left(x_m,y_n\right)\right]}^2}.$$

(3)

where $\left(m_1,n_1\right)\mathrm{and}\left(m_2,n_2\right)$ represent the start and end points of the connected domain marks of a given star, respectively.

2.4. Acquisition of Star Position in the Celestial Coordinate System

2.4.1. Star Map Recognition

Following star point extraction, it is necessary to identify the corresponding star in the star catalog. This process is referred to as star map recognition and generally involves comparing the pattern formed by the stars captured by camera with the star catalog [34]. Before star recognition is undertaken, the camera direction is calculated by a laboratory installation. Based on the FOV of camera, the stars can be chosen from the star catalog so that the number of stars is decrease dramatically. Then Luojia 1-01 uses the grid method to recognize the star map [35,36]. The unknown star and its nearest star constitute the x-axis in the development of a plane coordinate system. A rotation and binarization process is performed on the star map using a grid of size $g\times g$. The grid elements are assigned values, with 1 indicating the presence of a star, and 0 indicating the absence of a star. The binarization pattern is regarded as the recognition feature of the star. A bit comparison with each pattern in the catalog is used to determine the best match with the catalog patterns. Agreement between two patterns is established by determining the total number of shared on-grid cells. For the sensor pattern $p_s$ and catalog pattern $p_c$, the number of shared grids is given by

$${\displaystyle \sum }_{i=1}^{g\times g}{p}_{si}\wedge p_{ci},$$

(4)

where $\wedge$ is the logical ”and” of the two bits. The closest matching catalog pattern $p_c$ is considered to be the one that produces the largest sum from the above expression, implying that the sensor star $s$ should be paired with the catalog star $i$.

Based on the results of the recognition, the $i$th star in the conventional celestial coordinate system can be accessed with the right ascension and declination $\left({\overline{\alpha }}_i,{\overline{\mathsf{\delta }}}_i\right)$. The actual photographed stars must be corrected for their proper motion in order to accurately locate the position of the stars in the instantaneous celestial coordinate system.

2.4.2. Proper Motion Correction

Each star is in motion relative to the solar system, with the motion distances of different stars differing, hence the notion of star proper motion. The anniversary movement of most stars is less than 0.1″, although those of a few stars may reach a few arcseconds, with those of yet much fewer stars reaching 10″. There is thus the need to correct the displacement caused by the star proper motion [37]. Assuming that the star proper motion is regarded as constant over a short time, the influence of the proper motion on the right ascension and declination of stars over $t$ years can be expressed as follows:

$${\alpha }_i={\overline{\alpha }}_i+\mathsf{\Delta }\alpha \times t,\hspace{1em}\hspace{1em}{\mathsf{\delta }}_i={\overline{\mathsf{\delta }}}_i+\mathsf{\Delta }\mathsf{\delta }\times t,$$

(5)

where ${\overline{\alpha }}_i$ and ${\overline{\mathsf{\delta }}}_i$ are respectively the right ascension and declination of the $i$-th star during a conventional epoch, $\mathsf{\Delta }\alpha$ and $\mathsf{\Delta }\mathsf{\delta }$ are respectively the proper angles of the annual right ascension and declination, and ${\alpha }_i$ and ${\mathsf{\delta }}_i$ are respectively the right ascension and declination of the $i$-th star in the instantaneous epoch.

2.5. Attitude Determination of the Camera/Star Sensor

2.5.1. Attitude Determination

According to the ideal pinhole imaging model, there should be a one-to-one correspondence between a star vector in the star catalog and a star vector in the camera/star sensor coordinate system. As shown in Figure 3, in the instantaneous celestial coordinate system of J2000, the declination and right ascension of the $i$-th star are denoted by $\left({\alpha }_i,{\delta }_i\right)$. The observation vector can be expressed as follows:

$${\widehat{\mathit{Z}}}_{Bore}^i=\left[\begin{array}{c}cos{\alpha }_{i}cos{\delta }_i\\ sin{\alpha }_{i}cos{\delta }_i\\ sin{\delta }_i\end{array}\right]$$

(6)

After imaging, the coordinates of the $i$-th star in the star map coordinate system O-X_CCDY_CCD are $\left(x_i,y_i\right)$, and the measurement vector converted to the camera/star sensor coordinate system O-XsYsZs can be expressed as follows:

$${\widehat{\mathit{W}}}_i=\frac{1}{\sqrt{\left({\left(x_i-x_0\right)}^2+{\left(y_i-y_0\right)}^2+f^2\right)}}\left[\begin{array}{c}{x}_i-x_0\\ y_i-y_0\\ -f\end{array}\right],$$

(7)

where $f$ is the focal length of the camera/star sensor’ optical lens, and $(x_0,y_0)$ is the intersection point between the optical axis of the lens and the image plane. When the camera/star sensor in the J2000 system is known and expressed as a certain attitude matrix ${\mathit{R}}_{J2000}^{Cam/Star}$, the $i\mathrm{th}$ star’s measurement vector ${\widehat{\mathit{W}}}_i$ and its observation vector ${\widehat{\mathit{Z}}}_{Bore}^i$ are related as follows:

$${\widehat{\mathit{W}}}_i={\mathit{R}}_{J2000}^{Cam/Star}{\widehat{\mathit{Z}}}_{Bore}^i.$$

(8)

when using the TRIAD algorithm, only two stars are needed to obtain the attitude matrix ${\mathit{R}}_{J2000}^{Cam/Star}$. However, multiple stars are generally recognized in the field of view of a camera/star sensor and redundant observations are established for the multiple stars. The determination of the satellite triaxial attitude may be considered as equivalent to Wahba’s problem [38] and can be expressed as follows:

$$J\left({\mathit{R}}_{J2000}^{Cam/Star}\right)=\frac{1}{2}{\displaystyle \sum }_{i=0}^n{a}_i\Vert {\widehat{\mathit{W}}}_i-{\mathit{R}}_{J2000}^{Cam/Star}{\widehat{\mathit{Z}}}_{Bore}^i{\Vert }^2$$

(9)

where $J$ is the minimum loss function, and $a_i\ge 0$ is the weighting factor. The purpose is to find a ${\mathit{R}}_{J2000}^{Cam/Star}$ that minimizes the sum of the residual errors for all the stars $i$ ($i=0,1\dots .n$). Using the Davenport q-method, quaternion estimator (QUEST) method, singular value decomposition (SVD) method, fast optimal attitude matrix (FOAM) method, estimator of the optimal quaternion (ESOQ) method, and other algorithms [39], the attitude can be obtained in quaternions such as $\mathit{q}={\left[\begin{array}{cccc}{q}_1& q_2& q_3& q_4\end{array}\right]}^T$. Thus, the attitude rotation matrix ${\mathit{R}}_{J2000}^{Cam/Star}$ can be expressed as follows:

$${\mathit{R}}_{J2000}^{Cam/Star}=\left[\begin{array}{ccc}{q}_1^2-q_2^2-q_3^2+q_4^2& 2\left(q_1{q}_2+q_3{q}_4\right)& 2\left(q_1{q}_3-q_2{q}_4\right)\\ 2\left(q_1{q}_2-q_3{q}_4\right)& -q_1^2+q_2^2-q_3^2+q_4^2& 2\left(q_2{q}_3+q_1{q}_4\right)\\ 2\left(q_1{q}_3+q_2{q}_4\right)& 2\left(q_2{q}_3-q_1{q}_4\right)& -q_1^2-q_2^2+q_3^2+q_4^2\end{array}\right]$$

(10)

2.5.2. Aberration Correction

In celestial body observation, the observation angle of an observer at rest and an observer in motion observed in the same celestial body are inconsistent. This phenomenon is referred to as the aberration effect. As shown in Figure 4A, when the observer is at rest, the star is located in the direction S. When the observer is moving, the star light source observed by the observer always shifts toward the destination point, which is in the direction A, as shown in Figure 4B.

The angle between the directions of the motion and the light source is denoted by $\theta$, while $\mathsf{\alpha }$ is the aberration migration angle, $v$ is the observer’s motion speed, and $c$ is the speed of light. The aberration formula can therefore be expressed as follows:

$$\mathsf{\alpha }=\frac{v}{c}sin\theta$$

(11)

The aberration effect is similar to a bias with a magnitude and direction dependent on the relative velocity between the observer and the observed object. Because Luojia 1-01 moves fast in the true of date (TOD, an inertial coordinate system defined at the correct epoch) coordinate system, the aberration effect must be considered. In the case of star or stellar observation, the earth’s orbital motion must also be considered. The size of the aberration rate of the satellite is related to the satellite velocity within the heliocentric inertial frame and the optical axis orientation of the observation instrument [15,40,41]. Because the three coordinate axes of the heliocentric or geocentric celestial coordinate system of J2000 have the same direction, the satellite velocity can be directly added as follows:

$${\mathit{V}}_{\mathit{H}}={\mathit{V}}_{sc}+{\mathit{V}}_E$$

(12)

where ${\mathit{V}}_{\mathit{H}}$ is the resultant velocity of the satellite in the heliocentric celestial coordinate system of J2000, ${\mathit{V}}_{sc}$ is the velocity of the satellite in the geocentric celestial coordinate system of J2000, and ${\mathit{V}}_E$ is the velocity of the earth in the heliocentric celestial coordinate system of J2000. The optical axis direction of the camera in the celestial coordinate system of J2000 is denoted by ${\mathit{Z}}_{Bore}$, and the deviation angle caused by an aberration is defined as follows:

$$\mathsf{\alpha }=\frac{\left|{\mathit{V}}_{\mathit{H}}\times {\mathit{Z}}_{Bore}\right|}{c}.$$

(13)

According to the deviation angle and rotation axis, the transformation relation can be expressed in the form of quaternions using the axis/angle formula. The aberration correction quaternion $\Delta \mathit{q}$ is expressed as follows:

$$\Delta \mathit{q}=[\Delta q_1,\Delta q_2,\Delta q_3,\Delta q_4]=[\Delta {\overline{\mathit{q}}}^T,\Delta q_4]$$

(14)

$$\Delta {\overline{\mathit{q}}}^T={\left({\mathit{V}}_{\mathit{H}}\times {\mathit{Z}}_{Bore}\right)}_{norm}sin(\frac{\mathsf{\alpha }}{2})$$

(15)

$$\Delta q_0=\cos (\frac{\mathsf{\alpha }}{2})$$

(16)

where $()norm$ indicates normalization after cross product. Thus, the revised quaternion $\widehat{\mathit{q}}$ of the camera/star sensor can be obtained from the initial quaternion $\mathit{q}$ and time $\Delta \mathit{q}$, as follows:

$$\widehat{\mathit{q}}=\mathit{q}\otimes \Delta \mathit{q}.$$

(17)

2.6. Calibration of Camera-Star Sensor Installation

The least squares method was used to calibrate the installation between the star sensor and camera. For example, under the inertia frame of J2000 at time $t_1$, the attitude rotation matrix of the star sensor is ${\left({\mathit{R}}_{J2000}^{Star}\right)}_{t_1}$, the attitude rotation matrix of the camera is ${\left({\mathit{R}}_{J2000}^{Cam}\right)}_{t_1}$, and the camera-to-the star sensor installation matrix is ${\left({\mathit{R}}_{Cam}^{Star}\right)}_{t_1}$. The following formula is thus established:

$${\left({\mathit{R}}_{J2000}^{Star}\right)}_{t_1}={\left({\mathit{R}}_{Cam}^{Star}\right)}_{t_1}{\left({\mathit{R}}_{J2000}^{Cam}\right)}_{t_1},$$

(18)

where ${\mathit{R}}_{J2000}^{Star}$ can be expressed as follows:

$${\mathit{R}}_{J2000}^{Star}=\left[\begin{array}{ccc}{a}_1& a_2& a_3\\ a_4& a_5& a_6\\ a_7& a_8& a_9\end{array}\right],$$

(19)

where $a_1\dots a_9$ are the nine parameters of the camera/star sensor rotation matrix at time $t_1$, similar to ${\mathit{R}}_{J2000}^{Cam}$. For multiple measurements, the least squares solution of ${\mathit{R}}_{Cam}^{Star}$ can be obtained. For the values observed between times $t_1$ and $t_n$, the installation parameter between the camera and star sensor can be expressed as follows:

$${\mathit{R}}_{Cam}^{Star}={\mathit{R}}_{J2000}^{Star}{\mathit{R}}_{J2000}^{Cam}{}^T{\left({\mathit{R}}_{J2000}^{Cam}{\mathit{R}}_{J2000}^{Cam}{}^T\right)}^{-1}$$

(20)

where

$${\mathit{R}}_{J2000}^{Star}={\left[\begin{array}{ccc}{\left({\mathit{R}}_{J2000}^{Star}\right)}_{t_1}& \dots & {\left({\mathit{R}}_{J2000}^{Star}\right)}_{t_n}\end{array}\right]}_{3\times n}.$$

(21)

A similar expression can be obtained for ${\mathit{R}}_{J2000}^{Cam}$.

2.7. Geometric Positioning Model and Verification Methods

2.7.1. Verification Model Using Ground Reference

The camera-star sensor installation obtained by calibration was used to construct a ground geometric positioning model, and the image positioning accuracy without using GCPs was verified by the following formula:

$${\left[\begin{array}{c}X\\ Y\\ Z\end{array}\right]}_{WGS84}={\left[\begin{array}{c}{X}_s\\ Y_s\\ Z_s\end{array}\right]}_{WGS84}+m{R}_{J2000}^{WGS84}{\mathit{R}}_{Star}^{J2000}{\mathit{R}}_{Cam}^{Star}\left[\begin{array}{c}x-x_0\\ y-y_0\\ -f\end{array}\right].$$

(22)

where ${\mathit{R}}_{Cam}^{Star}$ is the camera-star sensor installation parameter; ${\left[\begin{array}{ccc}X& Y& Z\end{array}\right]}_{WGS84}^T$ denotes the position of any point in the WGS84 coordinate system; ${\left[\begin{array}{ccc}{X}_s& Y_s& Z_s\end{array}\right]}_{WGS84}^T$ denotes the center of the satellite in the WGS84 coordinate system $;m$ is the scale factor determined by the orbital altitude; $R_{J2000}^{WGS84}$ is the matrix of the coordinate rotation from J2000 to WGS84, obtained by combining the EOP parameters with the imaging time$;{\mathit{R}}_{Star}^{J2000}$ is the attitude of the inertial system measured by the star sensor; $\left(x,y\right)$ denotes the numbers of row and column pixels in the image; $\left(x_0,y_0\right)$ denotes the center of the focal plane of the camera; and $f$ is the focal length of the camera.

For verification using a ground reference, all the parameters on the right side of Equation (22) can be acquired through the satellite platform. Hence, the ground coordinates ${\left[\begin{array}{ccc}\tilde{X}& \tilde{Y}& \tilde{Z}\end{array}\right]}_{WGS84}^T$ of any point in the image yet to be verified can be calculated using Equation (22). By comparison with the coordinates ${\left[\begin{array}{ccc}\overline{X}& \overline{Y}& \overline{Z}\end{array}\right]}_{WGS84}^T$ of the homonymy point in the reference image, the positioning accuracy can be determined. However, if ${\left[\begin{array}{ccc}X& Y& Z\end{array}\right]}_{WGS84}^T$ in Equation (22) is known, the coordinates can be acquired as GCPs from the corresponding homonymy point. The precision of ${\mathit{R}}_{Cam}^{Star}$ can thus be calculated using the equation, which is the basic equation for GCP-based calibration.

2.7.2. Verification Model Using Moon Reference

As shown in Figure 5, the centers of the Earth, $O_e$, satellite, $O_s$ and Moon, $O_m$, have the following geometric relationship in the J2000 coordinate system:

$$\stackrel{\rightharpoonup }{O_e{O}_m}=\stackrel{\rightharpoonup }{O_e{O}_s}+\stackrel{\rightharpoonup }{O_s{O}_m}$$

(23)

that is,

$${\left[\begin{array}{c}{X}_m\\ Y_m\\ Z_m\end{array}\right]}_{J2000}={\mathit{R}}_{WGS84}^{J2000}\left[\begin{array}{c}{X}_s\\ Y_s\\ Z_s\end{array}\right]+m{\mathit{R}}_{Star}^{J2000}{\mathit{R}}_{Cam}^{Star}\left[\begin{array}{c}x-x_0\\ y-y_0\\ -f\end{array}\right]$$

(24)

where ${\left[\begin{array}{ccc}{X}_m& Y_m& Z_m\end{array}\right]}_{J2000}^T$ denotes the coordinates of the Moon center in the J2000 coordinate system. The other notations are as previously described for the ground geometric positioning model.

When using Moon reference for verification, the coordinates of the Moon center can be acquired using the Ephemeris DE421 and the imaging time. Other parameters can also be acquired through the satellite platform. Hence, using the model, the coordinates of the Moon center can be obtained as $\left(\tilde{x},\tilde{y}\right)$, and those of the actual Moon center in the image as $\left(\overline{x},\overline{y}\right)$, enabling the determination of the positioning accuracy.

3. Results

3.1. Calibration and Verification of the Camera-Star Sensor Installation

Figure 6 shows the results at various steps of processing the star map. Figure 6A shows the first scene of the star map acquired by Luojia 1-01. Stripe noise is visible in the original image. Figure 6B shows the star map after background denoising, while Figure 6C shows the extraction of a star, with the center at the centroid of the star point. The position of each star in the star map is acquired after this step. Figure 6D shows the result of the recognition using the SAO star catalog, while Figure 6E shows the star extraction result for the first star map. Owing to the sensitive magnitude of the Luojia 1-01 camera (less than fourth magnitude), only twelve stars were extracted from the first scene. There were insufficient control points for inner element calibration of the camera, though sufficient for the calibration of exterior elements such as the installation parameter. After this step, the coordinates of each star in the J2000 coordinate system was acquired. Combining the results in Figure 6C,E, the installation parameter between the camera and star sensor was acquired by the calibration method, as previously described.

After compensation of the installation parameter, the new position of each star in the star map was calculated. The difference between the new position of each star and the previously extracted position is the calibration error. The error includes the star extraction error, star proper motion error, aberration correction error, and camera inner element error. The calibration errors of the different stars are shown in Figure 6F. The root mean square (RMS) error was determined to be 0.4 pixels, which meets the requirements for installation calibration between the camera and star sensor of Luojia 1-01. At the same time, the error reflected that the consistency of each pixel in Luojia-1’s camera is so fine that four GCPs or even one GCP were used could reflect the absolute positioning accuracy when verification.

3.2. Verification of Camera-Star Sensor Installation

3.2.1. Verification Using Ground Reference

In this experiment, two groups of installation parameters after calibration were compared and verified. The groups of parameters are as follows:

• Installation parameters acquired by the GCP-based method, as shown in Figure 1A. The experimental images were obtained in the daytime imaging mode of Luojia 1-01 on 28 June 2018 [26].
• Installation parameters acquired by the star-based method, as shown in Figure 1B. The experimental images were obtained in the star imaging mode of Luojia 1-01 on 1 April 2019.

Figure 7 shows the relationship between the nighttime images obtained by Luojia 1-01 and the reference daytime images of differing scales obtained from Landsat. Four homonymy points were identified, as shown in Figure 7A,D. The coordinates acquired from Figure 7D were considered as the CPs of the nighttime images. The positions of the four points in the nightlight images were calculated after compensation of the installation parameter using the GCP-based method (Equation (22)) or star-based method. Figure 7B,E respectively show nightlight and daytime images of Guangzhou City. The red circles in the figures indicate homonymy points. The position (latitude and longitude) of the red circle in Figure 7C, calculated after compensation of the installation parameter, is at the cross hairs point in Figure 7F. The length of the arrow in the figure represents the positioning accuracy. The overall statistical results of positioning accuracy are presented in

Table 3. Positioning accuracy comparison of Luojia 1-01 nightlight images under two method.

Cum. Day	Imaging Date	GCP-Based Method		Star-Based Method
		Image (pixel)	Ground (m)	Image (pixel)	Ground (m)
241	29 January 2019	6.3	820	6.5	850
241	29 January 2019	6.9	900	6.8	890
282	11 March 2019	5.3	690	4.3	560
282	11 March 2019	4.4	570	5.2	680
RMS		5.80	748	5.78	745

.

3.2.2. Verification Using Moon Reference

In the use of Moon reference for the verification, the laboratory measurement parameter, conventional GCP-based calibration parameter, and proposed star-based calibration parameter were used to construct the Moon geometric positioning model with the aid of Equation (24). Based on the Ephemeris data DE421 containing the position of the Moon, the positioning model was used to calculate the Moon position in the Luojia 1-01 image. By comparing the result with the actual extracted barycentric position of the Moon in the image, the positioning accuracy was determined. The overall positioning accuracy results for the proposed star-based calibration method are presented in Figure 7. The cross hairs point indicates the Moon center in the image, calculated by the positioning model, while the end of the arrow indicates the Moon center. The Sobel-Zernike operator is used to detect the edge of Moon, and searching algorithm based on circularity detection is used to find the Moon edge points for extracting the Moon center [42].

Figure 8A illustrates the Moon positioning accuracy, with the length of the arrow representing the positioning error of the Moon center. Figure 8B–F show five Moon center offset conditions. Because the images were obtained at different times, waxing and waning of the Moon occurred. Moreover, although the Moon was not full, the Moon center could still be accurately determined. The Moon positioning accuracies of the three calibration methods are presented in

Table 4. Moon position under different calibration method.

Cum. Day	Imaging Data	Calibration Method	Calculated Moon Center (Pixel)	Measured Moon Center (Pixel)	Positioning Accuracy
					Pixel	Ground/m
8	10 June 2018	Lab.	(1133.54, 679.72)	(1105.25, 606.67)	78.34	10183.76
		GCPs	(1102.25, 603.89)		4.09	531.70
		Stars	(1099.96, 599.89)		8.60	1117.94
25	27 June 2018	Lab.	(438.85, 1054.01)	(411.07, 977.75)	81.16	10551.10
		GCPs	(408.44, 981.20)		4.34	563.96
		Stars	(405.75, 976.58)		5.45	708.13
174	23 November 2018	Lab.	(1050.06, 1001.15)	(1018.46, 918.70)	88.30	11478.76
		GCPs	(1019.89, 926.02)		7.46	969.59
		Stars	(1017.29, 921.97)		3.47	451.49
294	23 March 2019	Lab.	(1037.89, 972.96)	(1003.77, 892.53)	87.37	11357.83
		GCPs	(1007.62, 897.87)		6.58	855.81
		Stars	(1005.05, 893.81)		1.81	235.33
353	21 May 2019	Lab.	(1019.58,959.01)	(984.65,881.27)	85.23	11079.90
		GCPs	(989.25,883.98)		5.34	694.20
		Stars	(986.69,879.90)		2.46	319.80
RMS		Lab.			84.16	10941.31
		GCPs			5.71	742.32
		Stars			5.00	650.04

.

4. Discussion

As can be observed from Table 3 regarding the two images obtained on cumulative day 241, the positioning accuracy is approximately 900 m on the ground for both the GCP-based and Star-based methods. Regarding the two images obtained on cumulative day 282, the calibration parameter obtained through the star-based method affords a positioning accuracy of approximately 5–6 pixels on the image, and approximately 700 m on the ground. The RMS value indicated an accuracy within 800 m for star-based methods, which are comparable with the positioning accuracy of the GCP-based method.

As can be observed from Table 4, before the calibration of the star sensor/camera in orbit, the geometric positioning error only reaches approximately 80 pixels, when only the laboratory calibration parameters are used. This implies a ground positioning accuracy of 80 × 129 m ≈ 10 km. After GCP-based calibration, the positioning accuracy is approximately 6 pixels on the image, corresponding to approximately 780 m on the ground. The star-based calibration images were captured on 1 April 2019, which is far from the data cumulative days 8 and 25, for which it afforded positioning accuracies of 700–1000 m. However, for the relatively close data cumulative day 174, the positioning accuracy is approximately 450 m. When the time difference between the calibration date and the data cumulative day is only approximately one week (cumulative day 294), the positioning accuracy reaches 235 m, or approximately 1.8 pixels. The positioning accuracy deteriorates with further increase of the cumulative day, being 2.46 pixels for cumulative day 353, which is about 50 days after the calibration date.

It is obvious from Figure 9 that a combination of the GCP-based and star-based calibration methods tends to improve the positioning accuracy when the Moon center changes, with the accuracy further increasing with decreasing time interval between the verification and calibration dates. Obviously, the Luojia 1-01 could be suggested a star-based calibration once a month which will maintain the positioning accuracy of less than three pixels. Considering that the positioning accuracy of optical satellites generally decreases over an extended time, the method developed in this study enables independent completion of the camera and star sensor calibration. Moreover, there is no strict satellite maneuverability requirement because the camera can be pointed to an arbitrary star. In other words, the satellite does not need to execute any special maneuver during the imaging besides maintaining its inertial motion. The calibration frequency depends on the required positioning accuracy of the satellite, and the geometric positioning remains good over an extended time. Compared with the ground calibration method, the requirement for control points in the proposed method is greatly reduced. The utilized control points are automatically obtained from stars, affording the additional advantage of significantly increased calibration speed.

5. Conclusions

Stellar photography was used to develop a method for calibrating the installation between the camera and star sensor of a satellite. The proposed method involves star extraction, recognition, aberration correction, and attitude determination. Experimental results demonstrated that the calibration accuracy was comparable with that of GCP-based methods. The proposed method was verified by using it to position the center of the Moon. It was observed that the shorter the time interval between the calibration and verification images, the higher the uncontrolled positioning accuracy.

The proposed calibration method can also be integrated with in-orbit processing to perform the calibration processes during the non-working hours of the satellite. Compared with the GCP-based method, the proposed method has significant advantages, including a broad imaging time and location range, which facilitate a high calibration frequency. The method offers a new approach to the geometric calibration of optical satellites. In its present demonstration, images of twelve stars were acquired and used to calibrate the installation parameters between the satellite’s camera and star sensor, which are external elements. The imaging of more stars would require a higher sensitivity level of the star magnitude. Further study is planned to apply the proposed method to the calibration of the internal elements of the camera for other high-resolution satellite.