Considerations of Curvature for a Near-Eye Holo-Video Display

Abstract

Most 3D augmented reality, near-eye displays suffer from a low field of view. Leaky-mode lithium niobate devices are often used to steer, multiplex, or otherwise control the path of light in a system. Such devices are ideal candidates for creating such displays, and, in this paper, we demonstrate a path to increasing their view angle to 180° by curving the substrate, which will steer the output light toward the user’s eye. We also model the tradeoff between field of view, allowable interaction length, and draw distance for these curved leaky-mode devices.

1. Introduction

Leaky-mode lithium niobate (LiNbO₃) devices show promise for near-eye applications. The flat, clear material is a perfect candidate for a low-profile display. The devices are direct-view, monolithic, and highly transparent, in contrast to other proposed technologies [1,2]. They have low relative fabrication complexity and no backplane (i.e., no backing material required for reflection, absorption, etc.). Furthermore, they potentially support very high bandwidth [3] and do not suffer from quantization error of pixelated devices (i.e., error caused by pixelized spatial sampling) [4,5,6]. They can rotate the polarization of light for low noise operation, where the unused light can be eliminated from the final image with a simple polarizer [7]. Finally, they can selectively modulate light based on color in the frequency domain [8,9].

Leaky-mode surface-acoustic-wave (SAW) LiNbO₃ devices can be categorized into two groups: edge-exit and face-exit (as illustrated in Figure 1). Edge-exit devices have a wide deflection angle [10] and have various potential applications, notably a holo-video monitor [7,11]. Face-exit leaky devices, however, suffer from a reduced deflection angle. The main reason to use face-exit devices for a near-eye display is that the output of the display is angled toward the user and requires little to no external redirection (which would not be the case for an edge-exit device) [10,12]. Another notable reason includes removing the zero-order light from the image without the need for a polarizer, which the edge-exit device requires. Finally, by using the entire face, these devices remove the limit to the numerical aperture created by the edge-exit variety.

Achieving a large field of view (FOV) has been a challenge for many augmented reality (AR), near-eye technologies, with most managing to achieve somewhere around 20–80° [13,14,15,16,17]. Previous 3D display technologies have demonstrated this capability (i.e., alcove holograms), but these were not near-eye transparent displays [18]. Microlens arrays show promise for virtual reality (VR) displays, but these rely on opaque screens to project an image (and thus are not suited to AR) [19] or some external projector to shine light on to the lens array [20]. Some near-eye displays use complex lens systems and/or external cameras to augment the user’s FOV, but these are either bulky or add distortions that must then be accounted for [15,21].

Our goal is to achieve a full 180° FOV, while minimizing the sacrifice to angular resolution. To this end, in this paper, we explore models that optimize the FOV parameter. In addition to FOV, we also consider the limitations of co-linear, SAW-driven optical deflection with respect to maximum allowable angle. We also explore various geometries; this allows us to rearrange the boundaries of what we can achieve with this device. Finally, we present one potential instantiation of a binocular display. Our discussion is limited to increasing FOV in one direction (horizontal), since curving the crystal in one dimension is fairly straightforward, whereas bending in two dimensions requires additional complexities that we are unwilling to challenge in this paper.

2. Flat Screen

2.1. LiNbO₃ Flat Screen Device Review

A primitive design for a LiNbO₃ near-eye device might resemble something shown in Figure 2. Light is introduced into a surface proton-exchanged waveguide by either prism or grating coupling. In the opposite direction, a chirped SAW is launched from comb transducers fabricated on the surface of the device. The SAW is guided by the waveguide and interacts with the incoming light for some unspecified interaction length. Using k-vector analysis [22], it can be shown that the momentum of the light will change according to the opposing acoustic k-vectors. The bounds of this interaction are defined by the strength of the acoustic frequency chirp: The highest spatial frequency available defines the greatest deflection possible, and the lowest defines the minimum the light must deflect in order to escape the waveguide.

We provide a brief overview of the SAW–light interaction in Figure 2. A single-mode waveguide carries polarized light at some initial angle, ${\theta }_1$, until the light encounters a surface acoustic wave traveling along the top of the waveguide. These waves will interact [22], and the light will escape, or leak from, the waveguide at some angle ${\theta }_2$. With the correct setup, the leaky light will eventually encounter the back face of the substrate, where a surface-relief grating allows it to escape the substrate at exit angle or “maximum deflection angle”, ${\theta }_{MAX}$. The user’s eye would then be set behind this back plane at some set distance, and the total viewing area (or the total area that the user can look into and see an image) would be determined by the region over the device that allows for acousto-optic interaction.

2.2. Device Limitations

In order to properly analyze our device, we must consider the various limitations imposed by the system as it is currently envisioned. For example, the LiNbO₃ acousto-optic (AO) modulator is only capable of deflecting light (${\theta }_{MAX}$ in Figure 2) by a few degrees (the current model achieves ${\theta }_{MAX}=2°$ of deflection for the fabrication parameters described in Laughlin et al.) [10]. For the sake of this paper, we consider this angle to be 2°, although we will also allow for a −2° of deflection (by firing light down the waveguides in both directions), which gives a total of 4° of focus to our device (this is only a maximum deflection angle, and a smaller angle is used to draw points further in space). In addition, we primarily limit our analysis to a geometric consideration of the system, since that is what is required to show that the new geometry (using currently existing technology) will achieve a high field of view.

2.3. Viewzone and Field of View

For the purposes of this paper, we here consider the viewzone of the device to be the radial distance viewable by the user at any given point in image space. In contrast, the FOV will be the full angle of viewable area available to the user. This is illustrated in Figure 3.

To determine the viewzone for a flat-screen device, we consider some point drawn by the active region of our device at some distance, $d_1+d_2=d$. This point is created by some converging phase front created out of an aperture, a; this can be defined by the angle underneath the virtual point, which we will call $2\theta$ (see Figure 4a). As the point is drawn along the radial distance chosen, the aperture must likewise move along the available active region space (while some addition deflection is possible while maintaining a fixed aperture location, it will always be of a lower quality than the symmetrical, normal case, so here we ignore this).

When the virtual points are at their maximum distance, such that the eye can see both of them, the uppermost back-cast ray of one point and the lowest of the other will intersect at the eye, as seen in Figure 4b. The angle subtended by the virtual points and the eye will be the FOV for this device.

It should be noted that, as the points are drawn closer to the eye, the maximum deflection angle will no longer be sufficient to converge to the chosen point; the closest drawable point that the display is capable of will be directly proportional to the aperture size chosen for the device, and the slope (as pictured above) will be $m=\tan \left({\theta }_{FOV}°/2\right)$. Therefore, this limitation can be overcome by choosing a smaller aperture size. For the aperture size we will choose in the future, however, this point will occur closer to the eye than the eye is capable of resolving, so this can be ignored. In contrast, as the virtual points are drawn further from the eye, the angle, $\theta$, will necessarily shrink. This conforms to standard optical methods.

For a flat-screen device with ${\theta }_{MAX}=2°$, a FOV of 4°, or $2·{\theta }_{MAX}$, is achievable. With an aperture size of approximately $2a=$ 17.5 mm, the device can draw points up to the limit of normal adult human vision, around 25 cm from the eye [23].

3. Circular Screen

3.1. Purpose and Initial Concept

Our goal in creating a curved display is to increase the FOV while sacrificing the maximum aperture size allowed. Specifically, if the display has a radius of curvature (ROC) equal to its distance from the eye, then the FOV will automatically be 180° (or at least the angle subtended by the actual curved device). This behavior is demonstrated visually in Figure 5a.

Both the flat and curved devices use the same underlying technology, and the flat case is already a proven method [7,8,9,10]. Curved or circular LiNbO₃ has been shown to propagate SAWs at similar levels to those achieved in flat cases [24]. Furthermore, we show that, at the radius of curvatures and interaction lengths available, the curved device operates in a very similar fashion to the flat case.

The major consideration for this setup is the allowable aperture for drawing the virtual point. In contrast to the flat-screen case, where aperture and allowable draw distance has a linear relationship, the curved display adds a complication; that is, as the display curves, so does the ${\theta }_{MAX}=2°$ off the normal (see Figure 5b). This means that, in order to achieve the maximum deflection possible, and therefore the closest point possible, the aperture size must be reduced. Figure 6a shows the relationship between aperture size and closest drawable point for a curved display of this type. Note that, as the arclength used increases (for a fixed ROC), the closest drawable points approaches infinity. The display can no longer converge to a point at any distance, since, with the distance along the curved surface increased, and therefore the angular offset between the endpoints increased, the ${\theta }_{MAX}=2°$ of deflection is no longer sufficient to converge to a point. Again, refer to Figure 5b to understand this limitation.

In order to facilitate the design for this near-eye display, we choose to limit the allowable aperture to one size for all points drawn (for a given ROC). In practice, a variable aperture size would allow for greater control over points drawn further in space, but this would complicate the design outside the scope of a paper. Thus, we choose to limit the aperture to be whatever gives us a closest drawable point of 25 cm, which is the closest point the average human eye can focus to. For the device parameters shown in Figure 6a, then, the aperture size chosen will be approximately 1 mm, which is much smaller than the allowable aperture for the flat-screen design.

As we design the flat-screen display, we endeavor to keep the aperture size as large as possible. The allowable aperture directly correlates to angular resolution and the point spread function of the focused beam; and although this falls outside the scope of this paper, it still informs our decisions regarding device design.

3.2. Theory

We now outline the principles and equations defining our system. The curved surface is placed in front of the eye, and the radius of curvature (ROC $r$) is somewhere behind the eye (anywhere between infinity and directly at the center of the eye). The flat case is the same as the curved case with the ROC set at infinity. The angles and distances for the setup are defined in Figure 7.

With a known radius of curvature and arclength, the closest drawable distance can be found. A similar method can be used to find the other two parameters. The equations, listed below, fall out directly from the Pythagorean Theorem, the Law of Cosines, and the Law of Sines. We use these equations to create Figure 6, Figure 8 and Figure 9 by defining one parameter (ROC $r$ or the closest drawable point, for example) and solving for the rest.

$${\theta }_3=\frac{a}{r}$$

(1)

$$l=a·\frac{\sin {\theta }_3}{{\theta }_3·\sin {\theta }_2}$$

(2)

$${\theta }_2=\frac{\pi -{\theta }_3}{2}$$

(3)

$${\theta }_1=\pi -{\theta }_2-{\theta }_{MAX}·\frac{\pi }{180}$$

(4)

$${\theta }_4=\pi -{\theta }_2$$

(5)

$${\theta }_5=\pi -{\theta }_4-{\theta }_1$$

(6)

$$d=l·\frac{\sin {\theta }_1}{\sin {\theta }_5}$$

(7)

The variables in these equations are as follows: $d$, as previously decided, is the closest drawable point, assuming we are using the maximum deflection allowed by the system. $l$ is the chord underneath the length $a$, which runs along the region subtended by ${\theta }_3$ and represents half the allowable aperture of our system (given this closest drawable point and predetermined deflection angle). $2{\theta }_3$ is equivalent to the FOV for the system, and $r$ is the radius of curvature. All other angles have no practical real-world meaning and serve only to find the final variables we seek.

4. Radius of Curvature vs. Field of View

We now briefly consider the case between the flat-screen and the circular design with the end of the ROC placed at the location of the eye (at distance $r$). Everything between these two edge cases can likewise be described in terms of a circle with an ROC somewhere behind the eye (keeping the distance between the display and the eye fixed). Indeed, even the flat-screen case is only an iteration of the curved case, where the ROC is infinite.

Figure 10 shows how the FOV is determined for a display with a given ROC. By constraining the aperture size and draw distance to some predetermined values, we can find the relationship of ROC and FOV. This relationship will run from a FOV of 4° (in the case of the flat screen) to 180° (when the radius of curvature = the distance from the display to the eye). The results for this scenario are shown in Figure 8a. Note that the FOV in Figure 8a is shown to extend slightly beyond 180°; this is because the simulation extends an extra ${\theta }_{MAX}=2°$ beyond the 180° arbitrary limit (on each side) imposed by the theoretical LiNbO₃ curved display. In reality, the FOV will be limited by the amount of useable hardware in the final instantiation of the product.

As we can see from Figure 8a, choosing any radius other than one that falls on the eye gives a much smaller field of view. The slope of this falloff can be changed by choosing a larger aperture size, but that will necessitate placing the device much further from the eye in order to achieve a similar ability to converge, but because of the distance from the viewer, the virtual objects will have a proximity penalty. In addition, the improvement to the slope by increasing the aperture size is negligible when compared to the other compromises required to achieve it.

We also ask the question of whether we could move the ROC center past the eye by, one day, achieving a deflection angle greater than ${\theta }_{MAX}=2°$. Figure 8b considers this possibility. As we can see, even if this were the case, the device still fails to produce results much different from the ${\theta }_{MAX}=2°$ case until we reach the 90° case, which obviously would produce the full FOV regardless of curvature. In this way, we see that the device performance depends almost exclusively on the correct curvature and eye placement and has very little to do with angular deflection capability.

5. Design Graph for a Curved Near-Eye Display

In the circular case shown above (Figure 6), the ROC was chosen to be 1.5 cm. We chose this distance because it is roughly similar to most eyewear currently on the market. As we can see from Figure 8, choosing any radius other than one that falls on the eye gives a much smaller field of view; therefore, we opt to stick with the full curvature.

Another option available is to increase the ROC such that the display is further from the eye. The ROC, when compared to either the closest drawable point (from the eye) or the aperture size (holding the other fixed in either case), can be seen in Figure 9.

These graphs indicate that increasing the ROC is beneficial, but only to a point which we now consider. Depending on the parameters chosen, this point will fluctuate, but two main observations can be made. The first is that the closest drawable point from the eye will improve as the display becomes flatter, but, after a certain point, the effects of flattening the display converge to the flat-screen case, and the linear effects of placing the display further from the eye, to compensate for the increased ROC increase. We generally want to avoid this linear region, not only because the drawable points sharply decrease, but also because the display ought to be reasonably close to the user in order to be considered a near-eye display.

The second item of note is that, as ROC increases, the allowable aperture will likewise increase, but this begins to slow as the curvature increases. Thus, for the example chosen in Figure 9, anywhere past approximately ROC = 0.5 m, the aperture size will see diminished returns for the tradeoff of placing the display further from the user. Again, we want to keep the display close to the user, but because the space in which we do want to place the display shows such a large variation, we will try to place the display as far from the user as possible.

We have made the code used for this graph available for readers as Code 1 [25].

6. Preferred Instantiation

The ideal situation, given the previous information, would be to place the display as far from the user as possible. In practice, however, this display will be binocular, which means that if both eyepieces are to have an equal and full FOV, this will limit how far out the semicircular displays can be placed (if the eye is still to be at the center of the circle).

In addition, because the human eye has a maximum of 160° FOV, we can elect to crop parts of the circle to allow the display to gain further distance from the user. This will maximize our aperture size, and the closest drawable point will be 25 cm from the eye, with an aperture size of 2.2 mm. For an aperture of this size, the path curves a maximum of 65 nm, which is an order of magnitude less than the wavelength of the light, and multiple orders smaller than the SAW wavelength. Essentially, each interaction region can be treated as a miniature flat display. A sketch of this setup is shown in Figure 11. The chosen parameters shown here conform to the design graph in Figure 9.

The distance between the eyes is chosen to be 64 mm—the average interpupillary distance for a human adult male [26].

7. Conclusions

We presented an analysis of curved-screen near-eye display technology using SAW-driven LiNbO₃. This curved display has the potential to increase the viewzone and FOV for leaky-mode face-exit devices substantially. After considering the tradeoffs between FOV, ROC, aperture, and closest drawable point, we have put forth one possible instantiation, with a monocular FOV matching that of the human eye, and a binocular FOV surpassing 180°. We also offer a design graph for others that may be interested in pursuing this method of creating a high-FOV near-eye display.