Interactive Geological Data Visualization in an Immersive Environment

Abstract

Underground flow paths (UFP) often play an important role in the illustration of geological data by geologists, especially in illustrating geological data and revealing stratigraphic structures, which can help domain experts in their exploration of petroleum information. In this paper, we present a new immersive visualization tool to help domain experts better illustrate stratigraphic data. We use a visualization method based on bit-array-based 3-D texture to represent stratigraphic data. Our visualization tool has three major advantages: it allows for flexible interaction at the immersive device, it enables domain experts to obtain their desired UFP structure through the execution of quadratic surface queries, and supports different stratigraphic display modes, as well as switching and integration geological information flexibly. Feedback from domain experts has shown that our tool can contribute more for domain experts in the scientific exploration of stratigraphic data, compared to the existing UFP visualization tools in the field. Thus, experts in geology can have a more comprehensive understanding and more effective illustration of the structure and distribution of UFPs.

1. Introduction

Geological refraction data, referred to as geological data, can help illustrate geological formations using data obtained from the propagation of geological waves in different parts of the subsurface. Scientific data visualization based on geological data, especially geological volume visualization, plays an integral role in oil and gas exploration. Compared to traditional volume data, such as CT scan volume data and medical volume data, geological data has a noise problem in the results acquired due to current limitations in survey technology. This affects the clear interpretation of stratigraphic data by experts in the field. According to the domain knowledge provided by our geologists, the underground flow path (UFP) plays an important role in understanding the stratigraphic structure.

The quadratic surface-based query can help experts explore the structure of UFPs and local regions. Domain-specific languages (DSLs), such as set operator expressions, provide a high degree of customizability to quadratic surface queries. Liu et al.  [1,2] provided methods to illustrate 3-D seismic data and obtain the visualization results that reveal the distribution and geostructures of UFP. In addition, they also designed lightweight compilers based on set data visualization [3,4] to parse DSLs. Their interactive design brings great inspiration. We have referred to these and designed a compiler to reveal the UFP, which is based on parsing the set operator expressions.

Although there are many tools [1,2] to analyze stratigraphic data, the collaborating geologists say that there are still some challenges in existing graphical geological data visualization methods for interactive exploration. They are as follows. First, the user’s observation is limited because the traditional display device is restricted by the limited size of the screen. Most of the traditional geological data visualizations are presented on PC platforms, which prevents users from immersive experience and intuitive mid-air interaction of the data exploration. Second, due to the limitations of traditional interaction tools, such as 2-D screens, keyboard, mouse, and multi-touch devices, users are limited to operate in the 2-D space and have troubles with depth-direction moving and hybrid 3-D interactions of data exploration  [5]. Third, in order to help geologists study the structure and UFP distribution information of geological data, they need to be given different types of interactive functions and different stratigraphic display modes. Examples are armchair display modes, palisade display modes, and sometimes they need to combine them, i.e., cross-shaped display modes [2].

Moreover, the domain experts also require the integration of UFP results and the combination of various stratigraphic display methods, and even the integration of other query results, such as quadratic surface queries based on set operator expressions. In addition, there is a need to efficiently switch between different stratigraphic display modes. In response to these challenges, we propose an immersive stratigraphic data visualization tool for creating and customizing petroleum data explorations. We designed a new immersive interface to visualize seismic data, aiming to help domain experts explore the data intuitively and obtain insight into understanding the data through Touch Handle interaction. The main advantages of our tools are as follows.

First, our tool provides users to explore data in an immersive environment, compared with some currently existing works [6,7] that explore stratigraphic data in a completely virtual environment.zc We blended data visualization with real environments, which takes full advantage of immersive devices.

Second, according to the needs of domain experts in petroleum data exploration, we thus designed three stratigraphic display modes in an environment, i.e., armchair display modes, palisade display modes, and cross-shaped display modes.

Third, the quadratic surface query based on set expressions allows users to freely explore stratigraphic data of different shapes, which is adapted to the actual needs of exploring stratigraphic data.

We tested our tool by using three datasets provided by the Northwest Branch of the China Institute of Petroleum Exploration and Development. Further, according to the peculiar structure of each, we also refer to the three as planar (Stratigraphic Datasets I), flat and long (Stratigraphic Datasets II), and vertical (Stratigraphic Datasets III). Rendering tasks were assigned to immersive devices that users could explore the geological data in the immersive environment. It provides many opportunities for enhanced free human interaction with 3-D objects and visualizations. The evaluation of the design and implementation includes user studies and case studies to show the usability and effectiveness of the proposed tool. In addition, the different ways of displaying strata in the immersive environment can help domain experts to explore geological strata from different perspectives and scales.

In this paper, we first review the background of this work in Section 2 and present our approach in Section 3, which describes in detail the three main parts of our approach: Section 3.1 describes the interaction design of the immersive visualization tool, Section 3.2 describes the interactive integration and switch, Section 3.3 describes the different modes of stratigraphic display in the immersive environment, and Section 3.4 describes the immersive environment in which quadratic surface-based queries are performed on the underlying data. In Section 4, we give the results and discuss them separately, the domain experts gave us a lot of valuable feedback, most of which are positive. We conclude in Section 5 and  Section 6.

2. Background and Related Work

In this paper, we review the related work on applications of immersive visualization, devices of immersive visualization, data-centric transfer functions, geological data illustration, and visualization to show the background of our work.

2.1. Applications of Immersive Visualization

Immersive visualization can be customized for various visualization types, but currently the visualization types that can be built are the most common types, including bar-shaped bodies, chair-shaped bodies, and crosshair functions. We use a technique called join-and-complement to perform queries. AR and VR can achieve better spatial perception and increase authenticity, and are usually used as a data-driven presentation medium for exploration and experience. Augmented Reality (AR) and Virtual Reality (VR), which are usually used as a data-driven presentation medium for exploration and experience, can achieve better spatial perception and increase authenticity, and can also present a data-driven process of exploration and experience. Narrative visualization in immersive environments [8] can help users better understand domain knowledge and provide users with an intuitive experience of interactively exploring scientific data. With large screens, more examples can be displayed and zoomed in, and immersive technologies can be explored by combining with AR  [9] or VR [7]. There are many immersive interactive tools, including handles, mice, and keyboards. This kind of interaction can promote user exploration because it is very convenient and natural with the immersive display. Scientific visualization mainly focuses on the visualization of three-dimensional (3-D) phenomena, which is an interdisciplinary application field in science.

2.2. Data-Centric Transfer Functions

Transfer function, which is an interactive analysis method, enables users to select RGBA values of different hardness values, so as to obtain the final interpretation effect [10]. According to the terms of dimensionality, we can easily classify data-centric transfer functions into one-dimensional, two-dimensional, and higher-dimensional forms. Igouchkine et al. [11] implemented a 1-D transfer function. It can assign different properties to subregions and their boundaries. Liu et al. [1,12] used 1-D transfer functions to visualize geological volume data.

To help users explore clusters and evaluate selected features, Ma et al. [13] proposed a semi-automatic transfer function (TF) scheme in the 1D/2D TF domain. Compared with the traditional scheme, this scheme can effectively understand and evaluate the selected features.

With the development of science and technology, more data contains multiple variables, so other parameterized expressions are needed. Wang et al. [14] proposed a method that can fit data distribution through the Gaussian mixture model, and it also can generate a Gaussian transfer function. Volume data of different colors can be used as multi-variable volume data. Wang et al. [15] designed a parallel volume rendering method to visualize large scale volume data. Ebert et al. [16] used the color distance gradient dot product transfer functions and applied it to help users find the desired tissue information. Furthermore, the interactive dynamic coordinate system [17] can be used to project the multi-variable data into the 2-D space to facilitate users to select features. In addition, Guo et al. [18,19] put forward a kind of transfer function design interface that is based on the combination of dimensional projections and row coordinates. Moreover, Lawton et al. [6] proposed to turn the visualization of seismic data in the experimental environment into a customizable 3-D data visualization. Similarly, in order to ensure the construction of an effective 3-D geological model, Amorim et al. [20] mimic expert interpretation and create geological structure models based on sketches. This method can avoid the shortcomings of separate modeling. Rocha et al. [21] also used a new sampling strategy to enhance different geological attributes. The high-dimensional composition also has many advantages, it can provide not only its goals and backgrounds but also the best number and combination of attributes [22].

2.3. Geological Data Illustration

Geological data description can be divided into three categories: Horizon extraction, fault detection, and geological data interpretation. Horizon extraction usually uses surface detection technology to detect the horizon in 3-D geological data. The geological horizon indicates that the rock properties have changed and played a central role in the interpretation of Geosciences  [23]. Based on the surface detection technology, Faraklioti et al. [24] also used six-connectivity to detect horizon fragments. In addition, they proposed an automatic recognition algorithm that can analyze the layers in seismic volume data. In addition, the combination of 2-D and 3-D minimum cost path and minimum cost surface tracking is introduced to extract the horizon with little user input. Sketches can help to extract geological horizons from geological data  [25,26]. The 3-D geological discontinuities or faults have important applications in 3-D structural and stratigraphic analysis. Geological coherence or faults act on the geological data itself, so they are not affected by the interpreter or automatic picker biases [27]. Conventional amplitude time slices are usually helpful in viewing faults. Then, a more robust coherence algorithm based on similarity is proposed [27], which reduces the mixing of upper-layer or lower-layer features. The automatic fault detection algorithm is also developed through the Highest Confidence First (HCF) merging strategy [28] and double Hough transforms  [27]. A more effective geological fault detection system is further developed by using a graphics processor  [29]. In addition, the number of attributes based on consistency and texture can significantly improve the efficiency and quality of 3-D Ground-Penetrating Radar (GPR) interpretation [30], especially for complex data collected across active fault zones. Geological interpretation is of great significance to reveal the structural information displayed in the data. To improve the annotation of geological structures, Patel et al.  [31,32,33] used deformed textures, line transfer functions, texture transfer functions, and various graphical methods to track the horizon and interpret geological data. They further put forward the new technology of knowledge-aided annotation and computer-aided interpretation of geological data for oil and gas exploration  [33]. Domain knowledge about the structure and topology of geological features in geological data can also guide dynamic surfaces into these features [34]. Sketch-based methods can greatly improve the interactivity of illustrations. For example, Natali et al. [35] proposed a sketch-based method to create 3-D illustrative models in geological textbooks. In addition, they designed a method based on the composition of two synchronous data structures for processing and rendering  [36]. Volume rendering is one of the most common methods for interactive rendering of geological data. It can provide a lot of information about geological continuity and statistics to help visualize the data for analysis [37]. Gradient-based methods are sensitive to high-frequency noise. A real-time gradient-free method  [23] is proposed to present results similar to high-quality global illumination. The slice-based method is used to extract the upper channel and salt dome from the geological data. In addition, Liu et al.  [38] presented a method based on volume slicing, which extracts UFP through interactive slicing, but it is semi-automatic. It still needs the participation of users, which will reduce the extraction accuracy and exploration efficiency.

2.4. Geological Data Visualization

We program a domain-specific visualization system to extract the stratigraphic structures according to seed point tracing. Further, it can also realize the exploration of geological graphic data through human operation of VR equipment. Users are enabled to adjust the recommended seeds by fine-tuning them with visual interactions. The seed is automatically generated by the density gradient calculation based on the kernel function and the link information can be obtained through the weighting algorithm to construct the graph. By different sorts of nodes in the graph, we improve the extracted UFP structures to give the users a more intuitive feeling. Compared to the traditional measures, we can take the following approach: At first, we use the continuous scale-space theory [39,40,41] to analyze the multi-scale features [39,40,41] of the recommended seeds. Based on the theory, we propose a novel kernel-function-based density gradients computation approach to recommend multi-seed [2] automatically. Users intuitively explore the structure of the extracted UFP by fine-tuning the seeds. In order to explore the local area efficiently, we also use a quadratic-surface distance query scheme [42].

3. Interactive Geological Data Illustration

In order to help geologists or other oil exploration experts better understand the distribution and related information of UFP, we propose a method of interactive description of 3-D stratigraphic data and other exploration data in the immersion device. Figure 1 shows the overall flow of our method. First of all, we have obtained a number of different stratigraphic exploration datasets from petroleum exploration experts. At the same time, the transfer function adjusted on the PC devices will be transferred to the immersive device through the cloud server. Then, the dataset is preprocessed, and the original geological exploration data are sliced to generate 3-D texture visualization of the original geological exploration data on the GPU of the immersive environment. The 3-D texture based on the bit-array will be used for slice query interaction in different modes, so as to observe the organizational structure of each stratum. Simultaneously, the quadratic surface-based query method allows experts to further explore geological data at different stratigraphic levels, for example, oil exploration experts can better determine the position of oil data. Then different stratigraphic display modes are used to visualize the extracted surface, well, or other features. Finally, several illustration results are mixed together to reveal the structure and distribution of UFP.

3.1. Interaction Design of the Immersive Visualization Tool

In fact, we can use Oculus Quest as our immersive device in our experiment. The movement of the view is controlled by the tracking system of Oculus Quest, and other interaction methods are realized through the Touch Handle and our defined event respond script.

Basic Interactions. Based on the Oculus Touch Handle, we offer some basic interactions. In order to interact with the rendering result, we can bind basic interactions such as rotation, translation, and scaling (zoom-in and zoom-out) to the buttons and joystick on the Touch Handle (Figure 2). In addition, we have expanded more flexible interaction methods. For example, the user can adjust the thickness of the stratum slices by clicking on the buttons “A” and “B” on the handle, as shown in

Table 1. The design of Touch Handle interaction in an immersive environment.

Press Buttons	Move Parts	Display Mode	Function
NULL	Thumbstick	Whole data	Rotation
HandTrigger	Handle	Whole data	Translation
HandTrigger	Thumbstick	Whole data	Scaling
Button A	Handle	Palisade	Adjusting the interval between slices
Button B	Handle	Armchair	Adjusting the interval between slices
Button X	Thumbstick	Palisade	Adjusting the thickness of slices
Button Y	Thumbstick	Armchair	Adjusting the thickness of slices
Button X	Handle	Palisade	Adjusting the start position of slices
Button Y	Handle	Armchair	Adjusting the start position of slices

. Our tool detects the trajectory strokes of the Touch Handle, taking full advantage of the multiple degrees of freedom of the immersive environment. Users can adjust the position and interval of the armchair or palisade by drawing strokes left and right or back and forth (Figure 3).

Transfer Function Design. The traditional color selection interface based on the WIMP paradigm in an immersive environment is not suitable for natural interaction techniques and the user is not able to select colors efficiently in interaction [43]. The domain experts we asked also said they were more familiar with the traditional WIMP-based color picker on the PC side; therefore, we program a transfer function editor on the traditional PC side, equipping keyboard and mouse interaction devices. We edit the transfer function of a scientific volume data online on the PC side in a visual steering scheme because of the easiness of manipulating the traditional 2-D transfer function editor in a 2-D visualization space. Users can add control points by selecting special intensity values in the editor and then set the color and opacity values for the control points. The transfer function editor includes an adjustment view (Figure 4, left) and a preview view in the immersive device (Figure 4, right).

Based on a grid coordinate system, the adjustment view is designed. The x-axis stands for the intensity value of the volume data, and the y-axis represents the opacity of the intensity on the x-axis. Simultaneously, users can alter the opacity of the y-axis unit grid. Users can select the appropriate color in the color table dialog box by clicking the middle mouse button to obtain the best color value. Afterward, users can adjust the control points in the grid coordinate system of the transfer function editor, forming various peaks, and achieve the change of color and opacity for a certain intensity value of the volume data. After obtaining a suitable preview effect, users can save the transfer function data and render immersively with the data shared into the immersive device. As a result, users can adjust the appropriate transfer function which reflects the stratigraphic information in a collaborative mode. The collaborative mode is two people collaborating to adjust the transfer function, with one person modifying the control points on the GUI interface of the PC side, and the other user wearing an HMD to give real-time feedback on the real-time rendering result.

3.2. Interaction Organization and Switch

All the geological survey data we use are provided by geologists who collaborate with us. The original geological data is obtained by reflection of seismic waves and is then integrated into volume data.

According to the discussion with experts in geology, we need an intuitive and convenient interactive design to extract the UFP within a certain range in the immersive environment. In addition, domain experts need a variety of interactive stratigraphic display methods in 3-D volume illustration, and the interaction of the method needs to be as effective as possible. Therefore, we proposed a method with immersive stratigraphic display modes, which include armchair display mode, palisade display mode, and cross-shaped display mode. This method allows the user to observe the distribution of UFP in multi-views in the immersive environment. It also supports the integration of UFP results into various stratigraphic display modes. Moreover we summarize some general immersive interaction designs from the feedback of experts, which requires the system to meet two requirements. First, different types of interactions can occur simultaneously in different strata or voxels. Second, different immersive interaction results can be assigned to a given voxel.

In this paper, the 3-D texture based on bit-array is used to organize different interactions, such as different display modes and quadratic surface queries surfaces. The results of the different interactions are integrated into each voxel with a display value. The display value consisting of 0 and 1 indicates whether the voxel is displayed or not. A 3-D texture is designed to organize all interaction types assigned to the geological volume.

3.3. Different Stratigraphic Display Modes

In order to meet the needs of different domain experts, the visualization system should be used in different stratigraphic display modes, i.e., armchair display mode, palisade display mode, cross-shaped display mode, sometimes a combination of them is needed. Figure 3 shows three stratigraphic display modes, which are used in the visualization system. The users can adjust the start position of a group of stratigraphic layers, the depth of a single stratigraphic layer, and the interval between the stratigraphic layers. With the adjustment, the distribution visualization of the UFP and other objects, the users are interested in, can be better revealed.

Moreover, they also integrate the UFP results into many stratigraphic display modes. The straightforward method to visually display the geological data with different display modes is to cut the original volume data before rendering. It is tedious and inefficient because the immersive device should frequently process the data. In the worst case, the data should be preprocessed frame by frame during rendering; however, it is not flexible, and it is difficult to switch between different stratigraphic display modes, because the results of different display modes are very different, as shown in Figure 3.

To increase the flexibility and customization of using different stratigraphic display modes to visualize stratigraphic data, we designed a slice parameter equation, which is determined by the start position of a slice, the interval between slices, and the thickness of the slice, as shown in Equation (1) ( K is used to obtain the spacing between the Kth slice and the starting slice, and W is used to obtain the rendering range of each slice). In our method, the slice equation is the general form after transformation. It is just the specification that defines the slice group. The user can change the parameters of the slice by performing the interaction of different stratigraphic display modes, such as the start position of the front slices, the interval slices, and the depth of each slice.

$$\begin{array}{c}\hfill palisadeSlices=\left\{\right(x,y,z\left)\right|x=startPosition+interval\times \mathit{K}+\mathit{W},\\ \hfill \mathit{K}\in N,-\frac{thickness}{2}<\mathit{W}<\frac{thickness}{2}\}\left(palisadedisplaymode\right)\\ \hfill armchairSlices=\left\{\right(x,y,z\left)\right|y=startPosition+interval\times \mathit{K}+\mathit{W},\\ \hfill \mathit{K}\in N,-\frac{thickness}{2}<\mathit{W}<\frac{thickness}{2}\}\left(armchairdisplaymode\right)\end{array}$$

(1)

As described in Section 3.1, 3-D texture based on bit-array helps to support different stratigraphic display modes.

3.4. Quadratic Surface Query and Exploration

To extract UFP information in a certain range, we define several quadratic surfaces for exploration. We use quadratic surfaces as a visual metaphor for oil reservoirs. According to the experts’ recommendations, we pre-define the size and position of the five spherical surfaces. Simultaneously, users can add any query spheres, and the position of the query sphere and the parameters of the quadratic surface can be modified by scripting. Each query sphere has a special meaning, e.g., the coverage area mapped to the real oil reservoirs. As shown in Figure 5b, we design a virtual keyboard in an immersive environment where the user can click on buttons through the joystick to generate the set operation expressions. We have asked for suggestions from domain experts of geology and the response was that there are many single operator set queries (e.g., A∪C) in geological exploration and there is a need for queries with multiple operators combined. Adopting the suggestions of the experts, we used the five most common operators to achieve the majority of our geological exploration query requirements. The Figure 5a shows the immersive results of queries made by setting several spherical surfaces in a specific range via the virtual keyboard.

Algorithm 1 represents the details of the implementation process of the quadratic surface. The 3-D texture based on bit-array is conducive to querying the surface set. In our method, we regard the query process with different surfaces as different interactions. First, we perform a set operation on the pre-defined query spheres. Then, the GPU of the immersive environment generates a 3-D texture map according to the query results, and renders it in an immersive environment by a ray casting algorithm. Users can find the query spheres associated with the query results when looking at the quadratic surface-based query results in an immersive environment. It is convenient to explain each process of quadric surface query, any combination of multiple surface queries, such as their union, intersection, or complementarity.

Algorithm 1QUADRATIC_SURFACE_QUERY() Function

function OPERAND_2_BOOLEAN(operand, IntersectQueue, UnionQueue)

returnValue = False //The type of variable returnValue is Boolean

if operand is quadratic_index then

//If the operand is the number of the quadratic surface

returnValue = position in quadratic_surfaces[operand] //The type of variable returnValue is Boolean

else

//If the operand of the expression is intersection or union, the head boolean value of the corresponding queue is popped

if operand is intersection then

returnValue = IntersectQueue.Pop()

else

returnValue = UnionQueue.Pop()

end if

end if

return returnValue

end function

function    QUADRATIC_SURFACE_QUERY(position,    query_expression, quadratic_surfaces)

expressionTree = BuildExpressionTree(query_expression)

//Syntax analysis of expressions and building expression trees

IntersectQueue, UnionQueue = new Queue<Boolean>()

//Create queues to store the results of the expression sub-tree of intersection and union

while expression in expressionTree do //Iterate through all subtrees of the expression tree

X = OPERAND_2_BOOLEAN(expressopn.x)

Y = OPERAND_2_BOOLEAN(expressopn.y)

//Push the results of this subtree into the queue

if expression.operator is intersection then

IntersectQueue.Push(X AND Y)

else

UnionQueue.Push(X OR Y)

end if

end while

//When the expression tree iteration is finished, the top value of the two queues indicates whether the position is rendered or not

if IntersectQueue NOT Queue.Empty then

return IntersectQueue.Pop()

end if

if UnionQueue NOT Queue.Empty then

return UnionQueue.Pop()

end if

end function

4. Results

4.1. Implementation

We used three different stratigraphic datasets provided by experts to evaluate the effectiveness and practicability of our method. The experiments were run on the immersive device Oculus Quest. The immersive device is equipped with a Snapdragon 835 Processor, running 4 GB Rama and OLED screen with a 72 HZ refresh rate. The rendering component of stratigraphic data in immersive visualization was developed based on GPU rendering and CG libraries. The proposed method is tested using three datasets provided by field scientists, i.e., Dataset I, Dataset II, and Dataset III.

After data preprocessing and integration, the whole dataset is rendered on the immersive device. We integrate the results of all interactive operations including quadric surface query and three stratigraphic display modes, into a 3-D texture based on a bit-array. The immersive device then renders the dataset by determining the display value of each voxel of the 3-D texture. The immersive environment obtains the color and opacity values of each voxel according to the pre-defined transfer function and mixes them into the final rendering result. Algorithm 1 shows the process of quadric surface query in the immersive environment. Equation (1) shows the slicing equation in the stratigraphic display mode.

We recorded the data preprocessing time in HMD and the display frame rate of the interface when HMD ran the three stratigraphic datasets, as shown in

Table 2. Three stratigraphic datasets are used to test the processing performance of our tools in the HMD. The time of the data preprocessing and the size of display frame rate in HMD shows the performance of our tool for different stratigraphic data.

Dataset Type	The Size of Dataset (MB)	Data-Preprocessing Time (s)	Frame Rate (fps)
Dataset I	57.9	19.13	15.56
Dataset II	18.1	10.92	18.43
Dataset III	46.3	18.58	14.44

. The data preprocessing time includes loading the data in the HMD, loading the transfer functions, and loading the information of the query sphere 3-D texture, as shown in Figure 6. Currently, because of the poor processing performance of the HMD used in our tests, it takes a long time to process the data and runs at a low frame rate, which affects the user’s exploration efficiency and experience to some extent. In addition, we found that there is still some noise in the results from the secondary surface query, which interferes with the user’s exploration of the target data.

4.2. Case 1: Stratigraphic Dataset I

The first case is an immersive exploration of Dataset I. In this case, the user transmits the stratum Dataset I to the VR device. After rendering in the immersive device, the user can first view the overall geological information of the stratum of Dataset I. Then, the user inputs the set operation expression by clicking the virtual keyboard, and clicks the query button to query the corresponding stratigraphic query information that matches the set operation result. The result of the first example is shown in Figure 7. The user queries the union of query sphere A and query sphere C to determine the possibilities covered by the reservoirs mapped by A and C. The range of river channels contains oil. The user can judge the intensity of the river channel in the area by comparing the river’s color with the customized transfer function, so as to evaluate the difficulty of oil extraction. The color scheme of the Dataset I color map visualization is specified by the transfer function editor. Orange indicates that it is difficult to mine with high intensity, blue indicates that it is easy to mine with low intensity, and fully transparent indicates the river channel information that is not required to be rendered and queried, which is other noise information. We can set the color and transparency to make the different types of data, we want to view clearer. In addition, users can also combine the different stratum display modes shown in Figure 3 with the quadric surface query results shown in Figure 7a to finally obtain the stratigraphic data interpretation results shown in Figure 7c. Users can perform basic interactions on geological data through processing (such as rotation, zoom, etc.). For example, by shaking the “Thumbstick” described in Figure 2, the user can rotate the rendering result to view other perspective information based on the river channel data of the formation Dataset I. In order to accurately reflect the stratigraphy profiles, the collected data can be displayed in different stratigraphic display modes to show the stratigraphy profiles required by the domain experts. The stratigraphy profiles obtained through interaction in this way are very clear and easy to understand. They truly and intuitively display their complex geological structures and realize the query of UFP information. The combination of different stratum display modes and quadric surface query results allows the user to see the overall UFP distribution clearly, and also to see different positions and different thicknesses of the stratigraphy profiles.

4.3. Case 2: Stratigraphic Dataset II

The second case is an immersive exploration of the flat strip stratigraphic Dataset II. Similarly, users can transfer PC-edited transfer functions and stratigraphic Dataset II to the immersive device. The query sphere can be pre-defined in the script and placed in locations that contain important riverway information by users. In this case, users explore the intersection of query sphere A and B. The red and brown river channel information in Figure 8 is mainly distributed in the strata falling in both query sphere A and query sphere C. Additionally, the river channel structure is visualized by the transfer function, which makes the river channel in deep underground clearer. Users can explore the detailed substructure of a particular range by virtue of intersection operations. For example, if the user is only interested in a small portion of river channel information, the corresponding observations can be made across multiple river channels, as shown in Figure 8. Under the circumstances, visual clutter generated by other strata can be removed. As a result, users can easily explore critical river channel information, avoiding interference from noise.

4.4. Case 3: Stratigraphic Dataset III

The third case for immersive exploration is the vertical stratigraphic Dataset III. The volume data mainly shows information about the stratigraphy at depth, which has a more complex structure than the two cases above. In this case, we chose to use multiple set operations for expression queries. Further, to obtain a better rendering, we chose to show the main river structures in blue and purple for immersive exploration, as shown in Figure 9c. As represented in Figure 9d, the user combined the cross-shaped display mode of this stratigraphic data with the results of the expression query B∩!A for a synthetic interpretation of the stratigraphic information.

4.5. Domain Experts Feedback

We have consulted with two domain experts from the Northwest Branch of China Institute of Petroleum Exploration and Development who have discussed with us and gave some suggestions frequently throughout our work. These experts had already learned about some of the previous PC-based work on petroleum data exploration. They evaluated whether our data exploration approach in an immersive environment would give them a better and more intuitive understanding of the data by comparing it to traditional methods. Domain experts also provide some practical requirements for oil exploration and assess whether our overall approach can meet the requirements.

Much affirmative feedback from domain experts of geology has been given to us. Experts said that our tool provides an effective method for geologists to study geological phenomena and petroleum observation in 3-D space. Using visualization technology can describe complex geological structures more accurately and intuitively.

4.6. User Study

In addition to this informal feedback, we also conducted a user study to assess the usability and the utility of our tool. We recruited 10 participants majoring in geography from the university. Each participant was asked to experience pre-loaded petroleum datasets in an immersive environment, three formation display modes, and quadric surface query results based on set operation expressions, respectively. We encouraged them to freely interact and explore data through the Touch Handles. Investigators provided necessary operation instructions nearby. Lastly, we conducted a questionnaire for users and made a brief interview to collect comments about our tool. Most of the feedback was positive, which demonstrate the usability of the proposed approach as shown in Figure 10.

The questions in the questionnaire relate to involvement (Q1), usability, (Q2) and practicability of our tool (Q3–Q9). In subsequent interviews, participants told what impressed them most and gave some valuable suggestions for our tools. The results of Q1 ($\mu$ = 4.90, 95% CI = (4.67, 5.13)) show that almost every participant felt concentrated on the study. Our tool is for domain experts without programming experience. Participants considered that our interface is intuitive and the operation of interaction is easy to understand, which reflects the usability of our tool. (Q2 ($\mu$ = 4.90, 95% CI = (4.67, 5.13))). For Q3 ($\mu$ = 5.00, 95% CI = (5.00, 5.00)), Q4 ($\mu$ = 4.80, 95% CI = (4.50, 5.10)) and Q9 ($\mu$ = 4.80, 95% CI = (4.50, 5.10)), all participants agreed that our tool gives users a sense of immersion and allows users to explore data more vividly than traditional exploration tools based on the computer screen. P6 said “3-D immersion has a more comprehensive understanding of the stratum and can better understand the details of the stratum”. In addition, participants gave some positive comments on our query methods(Q5 ($\mu$ = 4.80, 95% CI = (4.35, 5.25)), Q6 ($\mu$ = 4.80, 95% CI = (4.35, 5.25)), Q7 ($\mu$ = 4.80, 95% CI = (4.50, 5.10)), Q8 ($\mu$ = 4.70, 95% CI = (4.35, 5.05))), i.e., three stratigraphic display modes (armchair display mode, palisade display mode, and cross-shaped display mode) and quadric surface query based on set operation expression. In particular, most of participants were interested in the stratigraphic display modes. e.g., “The data of the palisade display mode is clearly observed and the topographic and geological structure is well reconstructed” (P4). “Palisade display mode is clearer to help experts understand the details of stratum” (P8).

5. Discussion

Our approach to petroleum data visualization in an immersive environment effectively helps domain experts to explore stratigraphic data, better reveal geological structures, and illustrate the distribution of geological materials they are interested in. In our approach, exploring data through mid-air interactions allows users to have a high degree of freedom in their operations. Since mid-air interaction can easily cause fatigue, we reduce the type of the hand motions in mid-air interaction by combining the interaction with the Touch Handle buttons. Feedback from domain experts who have tested our tools is mostly positive; however, there are still some limitations of our tool that require further refinement and improvement.

Initially, in the process of quadratic surface query, our tool only allows the user to pre-set the location information of the query surface in a script and then import it into the immersive device for interactive exploration. Compared to some traditional methods with much unnecessary interaction, our query function based on set-operator expressions enables users to query the stratigraphic information of interest freely. There are still some barriers for users to set up the surface information, for example, altering the parameter information via scripting is not intuitive enough. For this reason, we will add a preview model of quadratic surface-based query in the immersive environment, which enables users to interact directly with the settings via the Touch handle in the immersive environment.

In addition, our tools currently only support programs that rely on virtual reality technology. VR still has limitations for the user with regards to data authenticity, which is not the best result; therefore, we would like to enable the tool to support AR in future work. AR is based on an overlay of digital images of the real world environment, with some motion tracking and visual feedback technology. In this way, the user will further enhance the interaction between the device and reality in realistic scenarios, thus enhancing the authenticity and immersion in the exploration of scientific data.

A higher frame rate can improve the smoothness of data during exploration and increase the efficiency of user exploration. There are two main factors affecting the frame rate at present: on the one hand, the poor processing performance of the used immersive devices, to a certain degree, lead to a low frame rate; on the other hand, the complexity of the currently used ray casting algorithm in the immersive visualization is high, and later work can improve the frame rate by optimizing the ray casting algorithm in the immersive visualization.

Finally, we can also add visual customization tools to provide the user to freely annotate the illustration of the visualization results. For example, a lasso tool could be added to help users select the oil data that they need to display. Other components such as brushing, erasing, keying, sketching, and drawing will all be conducive to the data illustration.

6. Conclusions

Our tool currently allows for the completion of immersive exploration of different stratigraphic datasets. Compared with traditional stratigraphic visualization tools based on virtual environments, our tool integrates data visualization with real environments. We use the proposed bit-array-based 3-D textures to organize the type of interaction with the geological data. This helps field scientists to better understand the distribution of UFP or other geological materials of interest. Currently, users can transfer query spheres and stratigraphic datasets edited on a PC to an immersive device. The transfer function allows visualization of the structure of the river channel, making the deep subsurface channel much clearer. In addition, users can change the different stratigraphic display modes and enter specific set operator expressions to explore the stratigraphic structures of their interest by a query. The different types of queries can be integrated to form the final interpreted visualization. Finally, the geological data visualization system in an immersive environment also supports interactive slice-and-dice analysis using handles, allowing users to explore the data through Touch Handle interaction.

Discussions were held with collaborating domain experts and much positive feedback was received; however, there are still some limitations of the tool. For example, our work does not currently support AR and we cannot move the technology to the real world, which would reduce the adaptability of our work and shrink the authenticity of the user experience.

We will accept the experts’ suggestions and continue to improve our work based on what is discussed in Section 5. For later improvements, we will continue to improve and extend our work in the future to address some of the issues in Section 5, including the modification of parameter information for quadratic surfaces.