Focus-Adjustable Head Mounted Display with Off-Axis System

Featured Application

Head mounted display; display optical system.

Abstract

An off-axis system refers to an optical system in which the optical axis and the normal vector at the vertex of each surface do not match. An off-axis optical system can be applied in order to construct a thin and light optical system. In particular, the optical system used for a see-through head-mounted display (HMD) must be designed asymmetrically, with respect to the optical axis. Because the vision of a human is different for each individual, HMD requires focus adjustment. The effective focal length (EFL) of the optical system must be calculated to obtain the focus adjustment. However, the off-axis optical system cannot be calculated by conventional methods. In this study, the EFL was calculated by rotating the coordinates of the rays near the optical axis by the angle of reflection or refraction at the intersection of each surface, with the rays coinciding with the optical axis. The magnitude of movement of the micro-display for focus adjustment was obtained from the calculated EFL, for a see-through type HMD.

1. Introduction

In general, an imaging system can be categorized into an asymmetric optical system and a rotationally symmetric optical system, with respect to the optical axis [1,2]. The rotationally symmetric optical system has been applied to many products and the lenses used in the symmetric optical system are also rotationally symmetric, so it has the advantage of easy lens fabrication and assembly [3,4,5]. However, owing to recent developments in optical component processing technology, asymmetric component processing is also possible. These asymmetric components are integrated into new products to reduce their size and weight [6].

Figure 1a presents an optical layout for patent JP2012-093478 [7]; this optical system is an eyepiece used in cameras. As exhibited in Figure 1a, the axial ray passing through the center of the stop is a straight line and there is rotational symmetry about this ray. Figure 1b presents the optical layout for patent KR 10-2015-0094891 [8]. This optical system is used in head-mounted displays (HMDs). In this optical system, the axial ray passing through the center of the stop is not a straight line, and there is asymmetry about this ray [9]. This is called an off-axis optical system. It describes an optical system in which the axial ray coinciding with the optical axis do not coincide with a normal vector at the vertex of the surface [10].

Figure 1 displays the optical layout of both optical systems on the same scale. The off-axis system depicted in Figure 1b is thinner than the on-axis system presented in Figure 1a. In Figure 1, the blue, red, and green lines represent rays for +1.0-field, 0.0-field, and −0.1-field, respectively. There is a reduction in the assembly time and cost reduction becomes possible when an asymmetric aspheric prism is used instead of multiple lenses [11,12,13]. The asymmetric aspherical surface is made of a light plastic material and the size of the optical system can be made short and small, primarily by using a reflective surface [14,15]. In particular, it is impossible to superimpose an image of an external landscape and a micro-display with a rotationally symmetric optical system [4]. Moreover, the off-axis system can suppress stray light, has a wider angle of view than a rotationally symmetric system, and can prevent ghost images [16]. In the past, these off-axis optical systems have been applied to astronomical telescopes. Recently, they have been predominantly used for HMDs or head-up displays [17]. In an astronomical telescope with an off-axis system, focusing is not required, so there is no need to calculate the first orders. However, since the HMD optical system in this study requires focusing, the first orders should be calculated. Two types of HMDs are depicted in Figure 2.

Figure 2a displays an HMD that visually removes the external world and presents a virtual reality (VR) and Figure 2b depicts an HMD that presents an augmented reality (AR), allowing simultaneous observation of a virtual object and an external scene [19,20]. The optical system applied to the see-through type HMD in Figure 2b must be designed with asymmetry about the optical axis for AR.

The size of the global AR/VR market in 2022 is estimated to be USD 105 billion, and the augmented reality market is expected to grow more than six times than that of VR [21,22,23]. It is expected that the demand for see-through AR HMDs will increase.

An afocal optical system, such as an HMD, serves to enlarge the image of a small display into a virtual image [17,24]. Because the virtual image distance is very large compared to the size of the optical system, it is represented by the reciprocal of the distance (in meters); this is referred to as the diopter (D) [22,25,26]. HMDs require focus adjustment because the uncorrected vision of a person varies among individuals [27]. The focus adjustment of the off-axis optical system used in the see-through type HMD is performed by moving the micro-display [28]. In the optical system with the infrared region, 2D materials or metasurfaces are about to be commercialized and many studies are being conducted in the visible region. However, since the main purpose of this study is to propose a method to calculate the first order of an off-axis system, the correlation can be applied irrespective of whether 2D materials or metasurfaces are utilized.

The amount of micro-display movement of the off-axis optical system used for a see-through type HMD was obtained by rotating the axial rays passing through the center of the stop in a straight line and using the system matrix to calculate the first order [29]. The calculation results were verified by gradually lowering the height of the axial ray and the angle of the chief ray to confirm whether the EFL converged to a constant value. In addition, the amount of image movement according to the change in object distance of the off-axis optical system was determined to confirm the possibility of a focus adjustment of the see-through type HMD.

2. Materials and Methods

In paraxial ray tracing, the ray has a height on each surface; therefore, the sag value on each surface can be disregarded [30,31,32]. Thus, each surface can be approximated as a flat surface. When this assumption is made, the optical system becomes ideal, without aberration. In an ideal optical system, the ray coincides with the chief ray in the image plane, regardless of its initial height [33,34,35]. Therefore, paraxial ray tracing can be used as a design criterion for optical systems.

The system matrix can be used in a general rotationally symmetric optical system, as illustrated in Figure 3. The ray slope, u, and the ray height, h, of the ray can be calculated from Equation (1) [36,37].

$$\left[\begin{array}{c}{h}_i\\ n_i{u}_i\end{array}\right]=\left[\begin{array}{cc}A& B\\ C& D\end{array}\right]\left[\begin{array}{c}{h}_0\\ n_0{u}_0\end{array}\right]$$

(1)

where n is the refractive index and the subscript i of each variable indicates the surface number. The 1st orders are calculated using Equations (1)–(5) [25,38].

$$EFL=-\frac{h_0}{u_i}=-\frac{n_i}{C}, BFL=-\frac{h_i}{u_i}=-\frac{A·n_i}{C}$$

(2)

$$H_2=EFL-BFL=\frac{n_i(1-A)}{C} (\mathrm{from} H_2 \mathrm{to} \mathrm{last} \mathrm{surface})$$

(3)

$$EFL=-\frac{h_i}{u_0}=-\frac{n_0}{C}, FFL=-\frac{h_0}{u_0}=-\frac{n_0·D}{C}$$

(4)

$$H_1=FFL-EFL=\frac{n_0(D-1)}{C} (\mathrm{from} \mathrm{first} \mathrm{surface} \mathrm{to} H_1)$$

(5)

Equations (2) and (3) yield the effective focal length (EFL), back focal length (BFL), and the second principal surface (H₂) when a collimated beam (solid red line in Figure 3) is sent from left to right in the optical system. Equations (4) and (5) generate the EFL, front focal length (FFL), and first principal surface (H₁) when the collimated beam (red dotted line in Figure 3) is sent from right to left in the optical system. It is observed that the EFL retains the same value regardless of the direction of the collimated beam [36].

The conventional paraxial ray tracing method cannot be applied in off-axis optical systems [29]. Therefore, a rotational transformation is performed such that the axial ray is in a straight line, based on the intersection point between each surface and axial ray, obtaining the intersection point and the refraction or reflection angle on each surface. Figure 4 illustrates the process of straightening the optical axis by rotating it based on the intersection of the optical axis and each surface. Figure 4a depicts the original optical system. Figure 4b demonstrates that the entire optical system is rotated so that the axial ray on the optical axis refracted from the first surface becomes a straight line. Figure 4c indicates the result of similarly rotating the axial ray on the optical axis to a straight line from the second surface to the last surface. In addition, the axial ray and the chief ray are rotationally converted based on the rotation center point and the rotation angle of the axial ray. In Figure 4, the black line represents the optical surface before rotation, and the blue line represents the optical surface after rotation conversion. The 1st order of the off-axis optical system can be calculated using Equations (2) and (5), obtained from A, B, C, and D for each surface [13]. Here, a smaller axial ray height or chief ray incident angle results in a smaller sag of the optical surface. Therefore, the accuracy of the result was determined by confirming whether the EFL calculated by this method converged.

Adjusting the focus moves the image surface as the object distance changes [39,40]. Figure 5 displays the change in the position of the image surface according to the change in the object distance. If the object is at infinity, the image is formed at the focal point of the lens. If the object moves from infinity to a finite distance, the image is shifted by Δf in the direction of the optical axis. Therefore, it can be observed that the object distance changes with the image distance.

Equation (6) is a Gaussian imaging equation, when the image and the object are in the air. Here, l is the object distance, i.e., the distance from the object to the first principal point; l’ is image distance, i.e., the distance from the second principal to the image; and f is the focal length of the optical system [25,41,42]. Therefore, l and l’ are positive.

$$\frac{1}{l}+\frac{1}{l^{\prime }}=\frac{1}{f}$$

(6)

The movement of the image surface (Δf) can be derived from Equation (6); this is expressed by Equation (7) [25]:

$$\Delta f=\frac{f^2}{l-f}$$

(7)

Figure 6 reveals that the focus is adjusted by shifting the micro-display with respect to the optical system presented in Figure 1b. The figure also indicates that the object distance of the optical system is infinite at 0 D, and the axial rays (red) toward the object are collimated. Figure 6a displays the optical layout when the object distance is changed from infinity to 200 mm, which is a negative diopter. Figure 6b exhibits an optical layout when the object distance is +200 mm, a positive diopter [13]. In modern life, the images of many people are captured in front of the retina and their vision is corrected with a meniscus lens with negative refractive power [43,44]. Therefore, products that are mounted on the eyes, such as HMDs, are designed with a negative average diopter value for an afocal system.

3. Results and Discussion

Table 1. The 1st order calculation result for KR 10-2015-0094891.

	HMY	UMY	HCY	UCY	Rotation Angle (°)
STO	4.00000	−0.00200	0.00000	0.21256	0.00000
2	4.07905	−0.00127	4.30023	0.13571	6.35697
3	−4.00114	0.13132	−5.61107	0.04270	152.69759
4	2.46681	−0.13133	5.04222	−0.04271	−102.10941
5	1.43497	−0.21807	4.55093	−0.08075	−9.82334
IMG	0.04677	−0.21807	3.89760	−0.08075	0.00000
EFL(Marginal ray)		18.34265
EFL(Chief ray)		18.33674
EFL(System matrix)		18.82512
BFL(Marginal ray)		6.58027
H1 to S1		−5.74583
Si-1 to H2		−12.07202

presents the first order results calculated by a program written in the internal programming language of optical design S/W for this study; the rotation angle was used to straighten the axial rays passing through the center of the stop. HMY and UMY are the rotation and rotation angles of the axial rays that do not pass through the center of the stop on each surface. It is the height and angle after the rotation conversion of the axial ray. Similarly, HCY and UCY refer to the height and angle of the chief ray, respectively. These values (HMY, UMY, HCY and UCY) were applied to Equations (2) and (5) to obtain the first orders of the off-axis optical system. Because the ray displayed in Table 1 uses the ray above the tangential plane, the x component of the ray coordinates is always 0, and the height of the ray is smaller than the size of the optical system; thus, the sag corresponding to the z component of the ray height can be ignored.

Therefore, HMY and HCY correspond to the height of the axial ray and the chief ray, respectively. In addition, UMY and UCY are obtained by dividing the y component of the direction cosine of each ray by the z component. The units of HMY and HCY are the length units (mm) of the optical system, and UMY and UCY are tangent of the ray angle, so UMY and UCY are dimensionless [45]. Furthermore, the EFLs in Table 1 are calculated when the initial incidence height of the axial ray is 4 mm and the main ray is incident at 12°.

To determine the accuracy of the first orders obtained from this method, it was confirmed that the EFL converges to a constant value as the incident height of the axial ray and the incident angle of the chief ray decrease. Figure 7 graphically represents the change in EFL according to the normalized pupil height for the axial ray and the normalized field ratio for the chief light. From the graph, it can be observed that the EFL converges to approximately 19 mm as the normalized pupil height and field ratio tend to zero.

Table 2. Calculation of the amount of movement of the image surface according to the change in object distance.

Object Distance	+1 D	−2 D	−4 D	∞ (0 D)
∆f(theory)	0.372280	0.673348	1.265977	0
∆f (optimization)	0.284736	0.579862	1.173932	0
error	0.087544	0.093486	0.092045	0

presents the movement of the image surface according to the object distance. In Table 2, Δf (theory) is the theoretical image surface displacement calculated using Equation (7) according to the change in the object distance. The spot can be minimized using the optimization function of the optical design software, according to the change in the object distance. The position of the minimum spot obtained through this is the actual movement, Δf (optimization), of the image surface. The difference in the amount of movement of these two values was indicated as an error.

The optical system used was designed with a fixed object distance. Because there is significant variation in the aberration with object distance, it appears that an error occurred owing to this effect [46,47]. Therefore, when designing an off-axis optical system with diopter adjustment, the zoom must be applied to the object distance. Here, this error corresponds to the vertical movement of the upper surface and thus becomes a vertical aberration. In afocal optical systems, such as HMDs, when the longitudinal aberration is within ±0.5 D, there is no problem during actual use. At this time, the virtual image distance corresponding to 0.5 D is 2 m. The amount of focus shift (Δf) exhibited in Figure 7 into Equation (7) for this virtual image distance can be calculated to be approximately 0.179 mm. Therefore, it is evident that the error in Table 2 is smaller than 0.5 D.

4. Conclusions

The see-through type HMD is designed as an off-axis system as it is an augmented reality product that allows simultaneous observation of external scenes and virtual objects. In addition, because the HMD is a device mounted on the head, it should be light and small for commercialization; thus, using an asymmetrical aspherical surface is advantageous, as it results in a thin and light optical system. Human vision is different for each individual, and hence HMDs require focus adjustment. In this study, the coordinates of the axial rays passing through the center of the stop were rotated according to the example of patent KR 10-2015-0094891, which uses an asymmetric aspherical surface. Through this, the axial rays were adjusted so that they are in a straight line so that conditions similar to those of the paraxial and rotational symmetry optical system were realized. Therefore, the first orders were calculated using the system matrix for the height and angle of the ray on each surface. The accuracy of the calculation results was confirmed by gradually reducing the height of the axial ray and the angle of the chief ray to confirm the convergence of the focal length. The calculated focal length was used to obtain the necessary movement of the image surface, according to the change in the object distance from +1 to −4 D.