Recording a Hologram Transmitted over a Communication Channel on One Sideband

Abstract

The paper presents experimental results on the recording and restoration of 3D holographic frames suitable for transmitting 3D holographic images with the frame rate required for TV images and a resolution of the Full HD standard and higher. The Patent RF No. 2707582 proposed a method for compressing holographic information and transmitting it using a procedure similar to SSB over conventional communication channels. In this work, the holographic information of a 3D portrait of a person, transmitted and received via the Wi-Fi communication channel, was restored in the form of a rainbow hologram, as one of a variety of holograms, by the computer addition of the carrier spatial frequency, and then hologram was actually produced on a photoresist. This technology can be used to create a holographic phototelegraph or, if there is a dynamic holographic display, to create a holographic television and 3D augmented reality.

1. Introduction

The problems related to holographic TV infringe on the interests of experts both in the area of holography and in image coding. For this reason, the Joint Photographic Experts Group (JPEG) has announced a competition of the proposals in holographic coding [1,2,3]. In its standard ISO/IEC 21794 [4], JPEG proposed solutions for coding more complete light field data [5] through creating a 4D array of the data on the light wave, with subsequent purely mathematical elimination of redundancy, similar to Huffman codes, for an increase in the entropy of an encoded message. High requirements in terms of the amount of information processed in this manner and the processing rate are also mentioned. This is due to the fact that each frame, which is a 3D array of the object holographing data, should change at a frame rate accepted by the major TV standards. This approach is implemented at present within the technologies of point cloud [6,7], polygonal [8,9] and voxel [10,11] representation of 3D images. Transmission of the real-time 3D images requires not only high computer productivity, but also the immense throughput capacity of the communication channel. Indeed, the density of holographic data records is limited by diffraction effects and is proportional to the spatial frequencies in a frame, inversely proportional to the size of the Airy spot [12,13,14]. In other words, the number of points on a hologram is inversely proportional to the size of the Airy spot, and for the focused image holograms, their number is inversely proportional by the order of magnitude to the wavelength. Within the visible range, for a hologram of a portrait image of A4 size (210 mm × 297 mm), which corresponds to a television screen of 14 inches diagonal, at λ = 0.63 µm, the number of points on the hologram reaches n = 1.6 × 10¹¹. For the binary record, this corresponds to the same amount of information measured in data bits. This is approximately n = 2 × 10¹⁰ bytes ≈ 20 GB, which is equivalent to the information contained in an hour-long film of the Full HD standard. For transmission along a radio channel with a frame rate of 25 frames/s, this will take a bandwidth of about 500 GHz and will overlap the whole radio range available for mankind, pushing out all other radio and television channels. A way out of the formed contradiction is either a substantial decrease in the size of the holographic image, which does not correspond to the requirements of augmented reality, or the compression of the holographic information so that its amount would not exceed reasonable limits—that is, the possibilities of the modern communication channels 3G and 4G.

To solve this problem, we chose another method of transmitting the holographic information, as presented in [14,15] and covered by RF Patent [16]. The essence of this method is in recognizing that it is not necessary to transmit the whole amount of holographic information for the creation of a 3D holographic image at the receiver end of the communication channel; it is sufficient to only transmit the part responsible for carrier modulation. In spite of the fact that the modulation in holography is more complicated, this is very similar to the SSB method, because the spatial frequencies of the carrier are already input at the receiving end of the communication channel. The difference between the proposed method and the SSB is the fact that the side spatial frequencies are not transmitted, but are still created at the receiving end of the communication channel on the basis of the two transmitted image modes: depth map + surface texture of the 3D holographic object. So, a 4D file of the holographic content considered in [4,5], which is a time-base sweep of 3D images, is replaced by the time-base sweep of two 2D images, to which JPEG compression has already been applied. The first of these two images is the surface texture of the 3D object, similar to a common photographic image or a series in a video. The second is a map of the 3D surface onto which the texture is superimposed, thus forming an image of the 3D object, or a mask of the object. Thus, extremely large amounts of information contained in holograms and preventing transmission along conventional communication channels may be compressed without any damage to the acceptable size, comparable with a generally accepted television frame of the Full HD or 4K standard. In this case, transmission of holographic information (Figure 1) resembles the single sideband modulation (SSB) known in radio engineering, because the carrier spatial frequency responsible for the spatial separation of the zero and minus first diffraction order carrying the information on the 3D holographic object is absent in this case.

The holograms of the single sideband of spatial frequencies, transmitted along the channel, were synthesized by returning the carrying spatial frequency. It is rather difficult to reproduce these holograms using the classical method of repeating holographing technology according to the scheme proposed by E. N. Leith and J. Upatnieks. This can be implemented in a much easier manner using the technology based on S. Benton’s recording scheme. The first hologram transmitted along a conventional communication channel in real time was recorded using this technology. The parallax of the image restored by this hologram was measured. The outlooks for the use of these holograms are discussed.

The direct SSB transmission was performed along the Wi-Fi channel in the packages containing 500 pairs of images, as in those shown in Figure 1a,b, with the measurement of transmission rate [17].

An experimental transmission of the sequence of signal pairs of this kind (Figure 1a,b) was carried out over a wireless Wi-Fi channel to imitate a 3D video. For this purpose, a File Transfer Protocol (FTP) was installed at the transmitting device, while the FileZilla software for working with FTP servers was installed at the receiving device. Each transmitted frame of a 3D image was the sum of two 2D frames: a texture (2000 × 2000 pixels) and a mask (1000 × 1000 pixels). To imitate the transmission of a video sequence, the packages of 500 double frames was transmitted simultaneously. The time of the frame package transmission measured using FileZilla software during reproduction in real time was 12 s for the frames formed in the JPEG file format, and 51 s for the frames formed in the PNG format, which is equivalent to transmitting 41.67 and 9.8 double frames per second, respectively. This means that, at least for the JPEG format, the actual rate of 3D signal transmission within a narrow bandwidth of the Wi-Fi channel (IEEE 802.11n standard) was sufficient to provide the TV frame rate standard. On this basis, it may be concluded that the transmission of a full image of a 3D object in real time, with a frame rate higher than 25 frames per second, is quite accomplishable with this device configuration.

The hologram is synthesized at the receiving end of communication channel with the help of the digital method using the transmitted information on the 3D holographic object. This hologram is a material analog of a holographic display [14] reconstructing the image of this object. This hologram may be of any type; however, virtual synthesis of the most widespread holographic displays, according to the scheme proposed by Yu. N. Denisyuk [18], is hindered during the transformation of a virtual hologram into its material analog. Because of this, in our first paper presented at the HoloExpo 2021 conference Features of the synthesis of SSB holograms (SSBH) [19], we chose the synthesis of holograms at the receiving end of the communication channel, according to the scheme proposed by E. N. Leith and J. Upatnieks. However, the work with this type of hologram involves several problems that are difficult to solve. These problems are first of all connected with the necessity to make fast Fresnel transforms describing the propagation of the complex amplitude of the electromagnetic field in the space from the object to the hologram, and from the hologram to the observation plane within the scalar approximation of diffraction theory [20], as well as with the practical difficulties in the realization of point-by-point recording of large data arrays. In addition, this type of hologram is not reconstructed with white light—firstly, because the spectral blooming Δy of the points in the image, which is proportional to spectrum width Δλ and the distance Z from the hologram plane to the point of holographed image, is characterized by substantial extension at a large distance from the hologram, as shown by Equation (1), where θ_R is the angle of reference beam inclination:

Δy_λ ≈ (Δλ/λ)∙Z∙θ_R,

(1)

Secondly, the angular size θ of white light sources is much larger than that of a laser source, which also leads to blurring. See Equation (2):

Δy_θ ≈ θ∙Z.

(2)

For a white light source (Δλ = λ), with the angular size of the luminous body θ = 1/200, placed at a distance of best viewing distance (Z = 250 mm), these values will be Δy_λ = 250 mm and Δy_θ = 1.2 mm, respectively.

One can see in Equations (1) and (2) that the values of both types of blurring are proportional to the distance from the image to the hologram, so it is convenient to consider so-called focused image holograms [21], because blurring is minimal in this case. A dynamic monitor operating in the real time mode can recover the image by laser radiation, and then three lasers will be necessary for a full-color image (RGB color coding); otherwise, white color may be used (Yu. N. Denisyuk’s thick hologram and S. Benton’s thin rainbow holograms). We chose the holograms recovered with white light, and among them, S. Benton’s holograms [22], because it is very difficult to synthesize Denisyuk’s material holograms from their digital models. In addition, in rainbow holograms, the entropy decreases due to the elimination of the vertical parallax, which causes a decrease in the amount of information in the hologram; information overload not only hinders the transmission along communication channels [23], but also brings complications into computer synthesis. Moreover, the horizontal parallax is in much higher demand than the vertical one, so at the first stage we chose Benton’s rainbow hologram for experimental recording.

2. Experimental Recording of a Rainbow Hologram at the Receiving End of the Communication Channel

A single-step scheme of recording a rainbow hologram is presented in Figure 2, while its version for recording flat colored images is shown in Figure 3. If several flat images are recorded according to the latter scheme, rotating the photographic plate around the vertical axis between exposures, each image will be reconstructed at its own angle during reconstruction, thus providing the parallax effect.

The image of holographed object 2, backlit by beam 1, which is coherent with 6, is transmitted to the photographic plate 5 through the filter 4 of vertical spatial frequencies so that, according to [24], S. Benton’s rainbow hologram is formed. Its speckle structure is extended along the vertical axis and has a complete set of horizontal spatial frequencies (Figure 4).

This hologram contains less information on the object because the vertical parallax is absent from it, and in the white light, it recovers a monochrome image, which is iridescent to the onlooker’s eyes moving vertically. Its appearance suggests that this hologram may be synthesized with the help of a set of columns composed of short, differently directed segments.

A colored image of the object may be obtained, for example, according to the scheme shown in Figure 3, where a slit source 3 with a random phase mask 2 forms an object wave on hologram 5, with the spatial modulation by mask 4. The latter is chosen from different spectral regions of the flat image to be recorded in accordance with Bragg angle θ_s, so that the reconstruction angle would be identical for any of the recorded waves [25].

Not only one, but several flat images taken at different angles may be recorded on a holographic plate (Figure 5).

These holograms, when irradiated with a beam analogous to the reference one, but with a wide spectral range, will reconstruct the images perceptible as those having a horizontal parallax; they may be considered one of the possible versions of colored 3D displays.

It is known that the development of a number of complicated optical–mechanical schemes implementing these colored displays on the basis of S. Benton’s rainbow holograms have been carried out over some time [26]. However, as a rule, a record of this kind would possess low angle selectivity for thin holograms and low diffraction efficiency because of the rapid exhaustion of the dynamic range of photoresponse. These disadvantages may be eliminated using the DotMatrix technology, in which a photographic plate is divided into a series of so-called holopixels, so that each of them is, in turn, composed of a set of j regions corresponding to the number of recorded incidence angles. A flat image of the 3D image at only one incidence angle is recorded on each j region in all holopixels. Irradiation of the obtained rainbow focused image holograms produces a stereo effect, providing a horizontal parallax of a series of approach angles of the colored images of the initial 3D object.

For the experimental recording of an SSB hologram in the rainbow hologram technology, we chose a KineMax MASTERING SYSTEM (Polish Holographic Systems, Warsaw, Poland) (Figure 6), in which a complicated set of optical elements was maximally replaced by the computer synthesis of the corresponding diffraction gratings.

Under objective 14, there is an area-mobile objective table onto which the photoresist is placed. The image of several gratings formed by the LC-SLM (Figure 7) is projected through objective 9 onto the corresponding region of the photoresist.

The hologram, as a whole, is recorded by moving the table and projecting a specific calculated set of spatially oriented gratings onto every region; the hologram is composed of a set of holopixels, each containing a region for each of the j images to be recorded (Figure 8).

One can see that the structures obtained through the calculation and composition of the vertical segments with the bands oriented transversely at different angles (Figure 8) are very similar to those obtained in the classical S. Benton’s scheme (Figure 4).

3. Results and Discussion

A general view of the hologram recorded with the KineMax device is shown in Figure 9.

The parallax observed in the image reconstructed from this hologram is presented in Figure 10.

One can see in Figure 10 that the reconstructed images actually exhibit the parallax effect. The value of the parallax may be measured directly with the help of porting out the hologram-reconstructed image using a telescopic system (Figure 11).

The parallax is observed in the space into which the hologram-reconstructed image is ported (See Video S1). The test structures were two needles placed apart from each other at a distance equal to the object depth along the optical axis of the stand of the image reconstruction by the hologram. The upper needle is closer to the hologram along the optical axis, and the lower one is at a longer distance. The positions of the needles were chosen so that their tips remained almost non-shifted from the selected reference point on the object with a change in the angle of rotation of the reconstructed image. The distance between the needles was measured and was found to be equal to 5.85 mm; the telescopic system had a 9-fold longitudinal magnification, which corresponds to the depth of the hologram-reconstructed image equal to 0.65 mm.

4. Conclusions

Other representations of the 3D holographic signal that are widely developed today, such as the point cloud technology mentioned above [6,7], polygonal [8,9] and voxel [10,11] representations of three-dimensional images, have extensive bibliography, which cannot be reflected in our article. We can only address review papers, for example [27,28]. In

Table 1. The main stages that we relied on.

	Main Stages	Title of Main Publication	Publication of the Basic Content
1.	Mapping of the patient’s body surface with structured light.	Mikhailov V. P. & Co. Method for detecting postural disbalance (Computer optical topography).	[29]
2.	The problem is formulated of model 3D holographic signals for the purpose of their dynamic transmission.	Shoydin S. A. & Co. Modeling of diffraction on volumetric bodies in the MatLab software environment.	[30]
3.	A method to record 3D holographic information suitable for transmission along conventional radio channels is chosen.	Shoydin S. A. Method of holographic recording remote formation.	[16]
4.	A holographic system is proposed for registering 3D information by another method, using a ToF camera with surface sensing, followed by the formation of a point cloud, with the ability to create 3D images with a frame frequency of 20 Hz and low resolution in the depth of the object.	Yanagihara H. & Co. Real-time three-dimensional video reconstruction of real scenes with deep depth using electro-holographic display system.	[31]
5.	The possibility to transmit 3D holographic information over a conventional communication channel and its main features are described.	Shoydin S. A. & Co. Synthesis of holograms received by a communication channel.	[14]
6.	The transmission of a 3D holographic signal of Full HD standard with TV frame rate is performed experimentally.	Pasoev A. L. & Co. Transmission of holographic information on a single sideband.	[17]
7.	A hologram has been experimentally recorded from 3D information transmitted over a conventional radio channel.	Shoydin S. A. & Co. Recording a hologram transmitted over a communication channel on one sideband.	Present paper

, we presented some of, we believe, the key stages of our development based on the creation of a mask using structured light, and the parallel development of our foreign colleagues based on the creation of a mask (surface map) using a time-of-flight (ToF) camera. We believe that the low resolution in the depth of the recorded scene is inherent in the ToF technology, since the operation of the electronics serving it must exceed 300 GHz per pixel for the high-quality registration of a 3D image of a human portrait. At the same time, the technology we are developing for using structured light does not have such strict limitations, and is much better adapted for transmitting 3D holographic images over distance. This also applies to portrait images in the TV standard of spatial resolution, with continuous parallax of the image depth, as well as with the TV frame rate.

In the present paper, the data from the holographic 3D object were actually transmitted by us through a Wi-Fi channel in the form of two frames (a mask and a texture, as shown in Figure 1), so it may be concluded that the cycle involving the recording of a 3D image of the object; its transmission through a conventional radio channel with the accepted TV frame rate; the synthesis of a rainbow focused image hologram, which is equivalent to recording flat holograms at 15 incidence angles; the reconstruction of a 3D image of the object; and the observation of the parallax in the image were all carried out successfully. This allows us to conclude that it is possible to transmit holograms in real time through conventional communication channels using a method similar to SSB. Thus, the possibility of implementing a holographic phototelegraph was confirmed experimentally. The authors hope that the data described in this paper will be useful for the developers of dynamic holographic 3D displays to solve the problems of visual reality and holographic TV.