**1. Introduction**

With the continuous development and advancement of geographic information systems (GIS), it has become challenging for traditional two-dimensional GIS to meet the needs of geospatial visualizations and analysis [1,2]. With the widespread application of GIS in society, the application of three-dimensional (3D) scenes in smart cities is becoming deeper and wider [3–5]. The traditional two-dimensional managemen<sup>t</sup> model has been unable to meet the actual needs of the pipe network and pipe big data information analysis, expression, and application. The three-dimensional pipe network occupies a large part of the application scenarios. The performance-based city planning includes data management, data modeling, data analysis and data visualization. Due to the large-scale data size, wide range and various pipeline networks and pipelines, both on the ground and underground, and the spatial distribution is complex and varied, high-performance modeling of 3D pipe networks remains challenging. It is therefore essential to study the 3D pipe network construction and high-performance rendering.

> 35

Geodesign is proposed in order to visualize models for city planning [6,7]. With the development of Building Information Modeling (BIM), a three-dimensional model of construction project or facility's physical and functional characteristics, and a shared knowledge has gradually become a common data expression in the field of architecture. Resources provide a decision-making basis for the managemen<sup>t</sup> of the life cycle of a project or a facility. At di fferent stages of the life cycle, various stakeholders modify information through BIM to coordinate operations. BIM data is an important data source for 3D GIS [8,9]; it enables 3D GIS to move from macro to micro, and at the same time to achieve refined management—especially for pipe networks.

Utility networks are modeled using graph-theory abstractions [10,11], because the structure properties of graph provide network nodes and links. Most utility networks are based on two dimensional modeling design [12], but in some cases, applications such utility networks operate in 3D [13–15]. Pipe network is one type of utility network, the construction of a 3D pipe network is based on network science and GIS. As the BIM-oriented network model can be used in indoor network applications [16], a 3D pipe network should be used in both outdoor and indoor GIS. It is challenging to design 3D pipe networks by considering visualization, checking topology, data management, and 3D network analysis.

The 3D pipe network includes the pipe point portion and the pipeline portion. In the existing three-dimensional systems, the pipeline portion is normally constructed by vector line lofting, and the pipe point is generally built by manual modeling and then imported into the scene. The workflow of these pipe point models requires a lot of manpower and material resources. At the same time, for 3D pipe network data, the user's data source is often vector point and line data. If the model's display mode is adopted, data redundancy and extra workload are unfortunately inevitable. The maintenance of vector seed of point, line data and pipe point models is required as well. In addition, in some enterprise-level scenarios, pipe network data in 3D city models is extremely dense, and thus rendering these dense pipe networks requires a lot of hardware resources. Therefore, improving the carrying capacity and rendering performance of massive pipe network data can be the answer to current capacity and rendering performance limits.

GIS-based methods and modeling are widely used in urban planning [17–21]. GIS and BIM are located in two di fferent sectors for building a smart city. BIM provides the data foundation, and GIS provides a spatial reference, spatial analysis and decision making [22–24]. The combination of BIM and GIS contributes to urban planning, water conservancy projects, railway information modeling, underground pipe network information modeling, ancient building modeling and other fields [25,26]. The integration of GIS and Computer-aided design (CAD) supplies new data sources for building a smart city [27,28]. A hybrid system for expanding 2D GIS into the 3D scene is an e ffective way to consider the integration of 2D GIS and 3D GIS [29]. There are fewer studies that focus on the integration of spatial data modeling for GIS, BIM, CAD, and oblique photography. It is critical to study the integration model for spatial 3D data.

The computing-intensive framework plays an essential part in high-performance based smart city applications [30]. Parallel computing is an e ffective way to visualize 3D city models [31,32]. 3D city modeling requires e ffective algorithms [33,34]. NVIDIA, inventor of the graphics processing unit (GPU), proposes a parallel computing framework based on computing unified device architecture (CUDA), which can be used to speed up the processing e fficiency of geospatial applications [35–37]. The multi-threaded parallel processing based on multi-core central processing unit(CPU)s provides extensive computational capabilities [38,39]. The scientific computing for building city modeling is shifting from a CPU-centric central processing model to a collaborative processing model in which CPUs and GPUs work together [40].

In this paper, we propose a hybrid framework for high-performance modeling of 3D pipe networks. This paper is organized as follows: Section 2 introduces a pipe network data model, Section 3 shows hybrid high-performance modeling of 3D pipe networks, Section 4 describes the experiments and the results, and the last section is the conclusion and further work.

#### **2. Pipe Network Data Model**

The network data model is an essential part of high-performance modeling of 3D pipe networks.

#### *2.1. Pipe Network Data Structure*

A variety of pipe networks are normally represented hierarchically [41]. A network data set is used to describe a type of pipe network information. The data volume is less redundant and contains topology information, which can be used for subsequent spatial network analysis, such as burst analysis and connectivity analysis. Each pipeline data set consists of two sub-datasets: lines and nodes. The line sub-dataset represents pipeline objects, and the node sub-dataset represents point objects, such as elbows, tees, crosses, and valves.

Table 1 shows the main structure of the attribute table of the line sub-dataset, where the ID field is the ID of the line object itself, the FromNode and ToNode fields record the ID of the node object connected to both ends of a line object, and the PrevPoint and NextPoint fields recording the geometric coordinates of the other endpoints of the other line objects connected at both ends can facilitate the subsequent construction of the visualized point model.(*xi*, *yi*, *zi*) is the *i*-th point coordinate of geometry information. In addition, the geometric coordinates of the line object itself and some attribute information are stored.



Table 2 shows the main storage structure of the node sub-dataset, wherein the ID field is the ID of the pipe point object, the ID also corresponds to the FromNode and ToNode fields in the line sub-dataset, and a field for storing geometric information and attribute information. In these two associated data sets, the topology information is stored in the online data set.


**Table 2.** The node structure in the network data set.

#### *2.2. Construction of Pipeline Model*

#### 2.2.1. Coordinate Computation of Pipeline Network Model

After building the acquired pipeline data as a network dataset, it will be necessary to create a 3D pipeline network model for rendering. We use the pipe segmen<sup>t</sup> corresponding to the line segmen<sup>t</sup> between the two coordinates on the line geometry object of the pipeline as a rendering unit, and then combine these pipe segments to form the entire pipeline.

Each pipe segmen<sup>t</sup> consists of two sections, and if the position of the vertices on the two sections on the pipe section can be determined, an entire pipeline can be determined. We take the center of each section as the origin, the *X*-axis along the pipeline, and the Cartesian coordinate system on the *Y*-axis in the section (as shown in Figure 1). Since the cross-sectional shape of the pipeline is known, the position of the point on the pipe cross section in the local coordinate system can be determined. The position *Ps* of the center of the section in the world coordinate system is stored in the geometric information of the online object so that the translation matrix ( *Mt*) of the center of the section can be obtained.

**Figure 1.** Local coordinate system of pipe section.

To make the section perpendicular to the pipeline, and to make the upper direction of the pipeline section coincide with the upper direction in the world space, a rotation is also required. It is known that the world coordinate pointing to the center point of a given section is *W*1, and the coordinate pointing to the center point of the next section along the strike direction is *W*2, and the direction of the strike is → *V*sec = *W*1 − *W*2, corresponding to the X-axis in the local coordinate system of the section, and the upper direction in the world space is → *Vup* = (0, 0, <sup>1</sup>), corresponding to the Z axis in the local coordinate system and the *Y* axis being → *Vr* = → *V*sec × → *Vup*, the rotation matrix *Mrot* can be obtained. The coordinates of the final section vertices in the world coordinate system are defined as formulation (1).

$$P\_w = M\_{rot} \cdot M\_l \cdot P\_s.\tag{1}$$

It is worth noting that for a longitudinal tube, the up direction is the same or opposite to the direction. In this case, *V*r calculated from the previous section can be used as the *V*r of the current section, or any direction perpendicular to the direction can be specified as *V*r.

#### 2.2.2. Smoothing of Pipeline Inflection Points

When the pipeline is an inflection point, it is necessary to process the cross-section rotation matrix of the connected pipe segments before and after the inflection point so that the front and rear sections can be connected. The inflection point smoothing effect can be achieved by inserting several sections with smooth transitions, while the original two sections need to be retracted a distance to meet the smooth section, as shown in Figure 2.

**Figure 2.** Construction of smooth tube.

In this study, the following algorithm was used to calculate the cross-sectional matrix smoothed by the inflection point: Project the two lines that meet the inflection point into the plane where the two lines are located. Make a circle in the plane tangent to the two lines. As shown in Figure 3, the arc *AB* is the position of the center-line of the transition section. The more transition sections you take on this section, the smoother the inflection point. The length of the point *A* to the inflection point *O* is the length *M* of the pipeline section that needs to be retracted. By the constraint that the radius of the circle cannot be greater than the width *W* of the pipeline, we can ge<sup>t</sup> the value of *M* using formulation (2):

$$M = \tan \theta \ast \mathcal{W}.\tag{2}$$

**Figure 3.** The result of the inflection point projected onto the plane.

The vector *OA* is defined as → *L*1 and *OB* is defined as vector → *L*2. Then, the center coordinate (*xc*, *yc*, *zc*) can be obtained by the following method:

$$
\overrightarrow{P}\_{l} = \overrightarrow{L}\_{1} \times \overrightarrow{L}\_{2} \,\tag{3}
$$

$$\begin{array}{l}k\_1 = \mathring{P}\_{\dot{l}} \dot{\mathcal{O}} \\ k\_2 = \mathring{L}\_1 \hat{A} \\ k\_3 = \mathring{L}\_2 \mathring{\mathcal{B}} \end{array} \tag{4}$$

$$\begin{array}{l}D = \mathring{P}\_{l} \mathtt{x} \ast \mathring{\mathsf{L}}\_{1} \mathtt{y} \ast \mathring{\mathsf{L}}\_{2} \mathtt{z} + \mathring{P}\_{l} \mathtt{y} \ast \mathring{\mathsf{L}}\_{1} \mathtt{z} \ast \mathring{\mathsf{L}}\_{2} \mathtt{x} + \mathring{P}\_{l} \mathtt{z} \ast \mathring{\mathsf{L}}\_{1} \mathtt{x} \ast \mathring{\mathsf{L}}\_{2} \mathtt{y} \\ -\mathring{P}\_{l} \mathtt{z} \ast \mathring{\mathsf{L}}\_{1} \mathtt{y} \ast \mathring{\mathsf{L}}\_{2} \mathtt{x} - \mathring{P}\_{l} \mathtt{y} \ast \mathring{\mathsf{L}}\_{1} \mathtt{x} \ast \mathring{\mathsf{L}}\_{2} \mathtt{z} - \mathring{P}\_{l} \mathtt{x} \ast \mathring{\mathsf{L}}\_{1} \mathtt{z} \ast \mathring{\mathsf{L}}\_{2} \mathtt{y} \end{array} \tag{5}$$

$$\begin{array}{l} \mathbf{x}\_{\varepsilon} = (k\_1 \ast \mathbf{\hat{L}}\_1, \mathbf{y} \ast \mathbf{\hat{L}}\_2 \, \mathbf{z} + k\_3 \ast \mathbf{\hat{P}}\_1 \mathbf{y} \ast \mathbf{\hat{L}}\_1 \, \mathbf{z} + k\_2 \ast \mathbf{\hat{P}}\_1 \mathbf{z} \ast \mathbf{\hat{L}}\_2 \mathbf{y} \\\ -k\_3 \ast \mathbf{\hat{P}}\_1 \mathbf{z} \ast \mathbf{\hat{L}}\_1 \mathbf{y} - k\_2 \ast \mathbf{\hat{P}}\_1 \mathbf{y} \ast \mathbf{\hat{L}}\_2 \mathbf{z} - k\_1 \ast \mathbf{\hat{L}}\_1 \mathbf{z} \ast \mathbf{\hat{L}}\_2 \mathbf{y}) / D \end{array} \tag{6}$$

$$\begin{array}{l} y\_c = (k\_2 \ast \hat{P}\_l.x \ast \hat{L}\_2.z + k\_1 \ast \hat{L}\_1.z \ast \hat{L}\_2.x + k\_3 \ast \hat{P}\_l.z \ast \hat{L}\_1.x \\ -k\_2 \ast \hat{P}\_l.z \ast \hat{L}\_2.x - k\_1 \ast \hat{L}\_1.x \ast \hat{L}\_2.z - k\_3 \ast \hat{P}\_l.x \ast \hat{L}\_1.z) / D \end{array} \tag{7}$$

$$\begin{array}{l} \mathbf{z}\_{\mathbf{c}} = (k\_3 \ast \mathsf{P}\_l \mathbf{x} \ast \mathsf{L}\_1, \mathbf{y} + k\_2 \ast \mathsf{P}\_l, \mathbf{y} \ast \mathsf{L}\_1 \mathbf{x} + k\_1 \ast \mathsf{L}\_1 \mathbf{x} \ast \mathsf{L}\_2, \mathbf{y} \\\ -k\_1 \ast \hat{\mathsf{L}}\_1, \mathbf{y} \ast \hat{\mathsf{L}}\_2 \mathbf{x} - k\_3 \ast \hat{\mathsf{P}}\_l, \mathbf{y} \ast \hat{\mathsf{L}}\_1 \mathbf{x} - k\_2 \ast \hat{\mathsf{P}}\_l \mathbf{x} \ast \hat{\mathsf{L}}\_2 \mathbf{y} \Big) / D \end{array} . \tag{8}$$

From the coordinates of the center of the circle, the position of the transition section inserted in the arc can be easily obtained, and then the translation and rotation matrix of the section of the transition section can be calculated by the previous method.

#### *2.3. Construction of the Pipe Point Model*

Pipe points such as elbows, tees, and crosses are essential parts of the pipeline scenario. In the previous pipeline information managemen<sup>t</sup> systems, such a pipe point is normally constructed by a method manually and created in the modeling software, and then the modeled pipe point is imported into the scene. There are two main issues in the way of constructing the pipe point. The first is that the construction of the pipe point model is carried out in its own local coordinate system. Therefore, after the pipe point is imported into the scene, it is necessary to manually adjust its position direction so that the pipe points can be matched with the pipeline. The second is that due to the accuracy of the pipeline data collection, the angle between the pipe and the pipe point is di fferent from the standard value. Therefore, the standard three-way and four-way models cannot be used to match the pipeline. It is necessary to construct a pipe point model for each angle, which greatly increases the workload of modeling.

#### 2.3.1. Constructing the Pipe Point Model

We propose a new method for constructing the pipe point model, which can quickly construct the elbows, tees, and crosses that match the connected pipelines. Among them, the elbow is a pipe point connected to two pipes. The following paragraph is an example of the construction of such a pipe point connected to two or more pipes by taking a multi-pass pipe point as an example.

A multi-pass point can be broken down into a combination of multiple half-pipe segments, as shown in Figure 4. The shape of each half pipe segmen<sup>t</sup> is determined by the center point of the pipe point and the direction of the pipe connected to it, so that one half-pipe segmen<sup>t</sup> becomes HalfPipePair. Its structure contains three points including first point (FirstPt), center point (CenterPt), and second point (SecondPt).

**Figure 4.** The division of multi-way pipe.

We define that the half pipe pair is made up of a section on the X > 0 side of the pipeline, and its direction is FirstPt->CenterPt->SecondPt.

Use the following algorithm to find the half pipe pairs needed to make up the multi-pass point.


#### 2.3.2. Matrix Computation of Half-Pipe

After a set of half-pipe pairs constituting the multi-pass pipe point are obtained, they need to be decomposed into pipe segmen<sup>t</sup> units. As shown in Figure 5, one half pipe pair can be broken down into two pipe segmen<sup>t</sup> units. The calculation of the section matrix that is connected to the pipeline is given in Section 2.2.1. The matrix calculation of the intersection of two pipe segments is described below. It is easy to know that the interface is the angle bisector of <sup>∠</sup>*AOB*, and the trend is → *V*sec = −→ *AO* + −→ *OB*. Therefore, its rotation matrix can be calculated by the method in Section 2.2.1.

**Figure 5.** The plane projection of half tube.

Let ∠*AOB* be α, the normal vector of *AOB* plane be *P*, and the angle between *P* and upper direction → *Vup* be β. Then, the scaling ratios *ScaleY* and *ScaleZ* of the *Y* and *Z* axes in the local coordinate system and the scaling matrix *MScale* can be calculated. The vertex position on the half pipe segmen<sup>t</sup> is *PW* = *MScale*·*Mrot*·*Mt*·*Ps*.

$$\text{Scale}\_{Y} = \sqrt{\sin^{2}\beta / \sin^{2}(\alpha \* 0.5) + \cos^{2}\beta},\tag{9}$$

→

→

$$\text{Scale}\_{\mathbb{Z}} = \sqrt{\cos^2 \beta / \sin^2 (a \ast 0.5) + \sin^2 \beta},\tag{10}$$

$$M\_{\rm Scale} = \begin{pmatrix} 1 & 0 & 0 & 0 \\ 0 & \text{Scale}\_Y & 0 & 0 \\ 0 & 0 & \text{Scale}\_Z & 0 \\ 0 & 0 & 0 & 1 \end{pmatrix} . \tag{11}$$

#### *2.4. Integration Framework for 2D GIS and 3D GIS*

The integration framework for 2D GIS and 3D GIS contains spatial data managemen<sup>t</sup> integration, application integration, function module integration, expression symbol system integration, and analysis function integration. The essential part is that the spatial data model and spatial data structure of 2D and 3D data objects are kept in the same design. All the two-dimensional data can be directly used with high-performance modeling in the 3D scene without any spatial data conversion. 2D maps and 3D scenes can easily be generated based on the integration framework for 2D GIS and 3D GIS (Figure 6). The 2D and 3D integration of spatial data managemen<sup>t</sup> solves the problem of spatial data compatibility of different dimensions and reduces the cost and complexity of the system construction to meet the needs of various applications. Spatial analysis is supported in both 2D map and 3D scene. 3D network analysis is widely used in pipeline network management. GPU graphics hardware acceleration provides a powerful support for 3D analysis functions and brings users a good experience with high performance.

**Figure 6.** Pipe network models in 2D GIS and 3D GIS.

#### **3. High-Performance Modeling for 3D Pipe Networks**

The framework for high-performance modeling of 3D pipe network contains a spatial 3D model, instantiation, adaptive rendering, and combinational parallel computing.

#### *3.1. Spatial 3D Model (S3M)*

Spatial 3D model (S3M) is proposed for spatial data transmission, exchange and high-performance visualization of massive and multi-source 3D spatial data [42]. It meets the requirements of users in three-dimensional data transmission and analysis between different terminals (mobiles, browsers, and desktops) through an efficient and scalable data specification. S3M supports multi-source data, including pipelines, BIM, laser point clouds, vector, terrain, dynamic water surfaces, and 3D grids. S3M provides the ability to efficiently visualize a large amount of data using the level of details (LOD), batch rendering, and instantiation to improve the rendering performance.

The S3M includes description files, index tree files, data files, and attribute files. The file organization of each object storage is shown in Table 3. S3M TileTreeSet object is the basic element in S3M. TileTreeSetInfo is its description information, which is an overall description of the data. If the TileTreeSet is constructed based on the point, line, polygon or model dataset, there may be an AttributeInfos that represents attribute description information for each data set. Space division of three-dimensional data are in a specified spatial range, each spatial division corresponding to a tree structure organization tile collection is represented by a TileTree object. IndexTree is index information of its tree structure. AttributeData representing attribute data are recorded in TileTree attribute data for each object. Each TileTree is subdivided step by step from top to bottom, and each space partition corresponds to one tile, represented by a Tile object.



The description file (.scp) and data folder are the basic components. The description file contains the path filename (.json) for each TileTree. The index file is a description of the tree structure of the tile data and can obtain the bounding box of the tile file of each layer, the switching information of the LOD and the attached child node file without loading the actual data. The main role is to accelerate the efficiency of tile file retrieval; the attribute data includes one attribute description file (attribute.xml) and a .xml file storing each tile attribute data in each TileTree. All tiles in TileTree form a tree-like logical structure.

To facilitate sharing of multi-source heterogeneous geospatial data, the open geospatial database connectivity (OGDC) was proposed to connect different types of geospatial data (Figure 7). OGDC provides a unified entry to geospatial data [43]. The unified data specifications, such as I3S [18] and S3M, can be used for high-performance, consistent access to data in the system, such as various GIS terminal applications, and user APPs.

**Figure 7.** The heterogeneous geospatial data fusion framework.

#### *3.2. Instantiation Rendering of Pipe Networks*

Instantiated rendering refers to multiple renderings of a grid using different parameters at different locations. Instantiation is often applied to the rendering of static mesh objects such as leaves, grass, etc. in a large number of scenes. All instances share a vertex buffer, which is used to store a single grid of data that will be instantiated multiple times. The instance buffer is used to store instance data for each object, including the information such as transformation matrices, color data, and lighting data. The rendering process is combined with a vertex buffer and an instance buffer for rendering.

The pipeline is in fact composed of pipe sections that are substantially similar and have slightly different positions and directions. In order to render these slightly different pipe segments, the traditional method is to store each pipe segmen<sup>t</sup> separately, which consumes a lot of memory resources; or it needs to switch frequently between rendering states, such as rotating and panning each pipe segment, which significantly affects rendering performance. By instantiating the grid of reused pipe segments, reducing the number of calls and memory requirements, and giving most of the rendering work to the GPU, reducing the CPU load is a good way to improve rendering performance.

For the instantiation of the pipe point pipeline, only two sets of grids need to be established. One set is a complete section pipe section for rendering the elbows in the pipeline and the pipe points, and the other set is a half-section pipe section for rendering the multi-pass pipe points. As shown in Figure 8, the different tube segmen<sup>t</sup> examples differ in the position of the cross-section at both ends, in the size of the zoom, and in the direction. These can be represented by two transformation matrices MBottom and MTop. The matrix stores 18 floating point numbers representing the three rows and three columns of the two matrices into the instance buffer, and the grid data of the pipe segmen<sup>t</sup> is stored in the vertex buffer. In addition, the vertex needs to be stored in the vertex attribute of the vertex buffer, to which the matrix belongs before and after. Finally, the vertex position of the pipeline can be calculated by multiplying the vertices of the static mesh with the matrix in the vertex shader.

**Figure 8.** The grid of section and half cross section.

#### *3.3. Adaptive Rendering for Pipe Network Data*

The BIM data structure includes a spatial data model and attributes data. The spatial data model includes spatial information and appearance shape. The attribute data includes design parameters, construction parameters and operation, and maintenance parameters. The 3D GIS covers the data structure of BIM (spatial data and attribute data), covering the data representation of BIM (3D model), covering BIM data objects (BIM for architectural objects, and GIS covers a wide range, including architectural objects), and it overlaps with the BIM function (information managemen<sup>t</sup> and spatial analysis). The 3D underground pipeline scene based on BIM-GIS consists of three-dimensional pipe points and three-dimensional pipelines. The three-dimensional pipeline includes round pipes, square grooves, pipe blocks, and vertical pipes. Three-dimensional pipe points include three types of feature points, wells and ancillary facilities: feature points such as elbows, straight throughs, three-way, cover plugs, and pipe caps. Wells include square wells, round wells, and rain rafts. Ancillary facilities include valves and water meters. Based on the BIM technology, this study uses linear symbols to construct three-dimensional pipelines and adaptive pipe point symbols to construct three-dimensional pipe points. Some special feature points, wells, and ancillary facilities are displayed by model symbols.

BIM model symbols can be used to visualize the pipe points with complex shapes, such as valves. However, in the three-dimensional scene, the model symbols cannot adaptively adjust the angle according to the direction of the pipeline in the *X*, *Y*, and *Z* directions. Due to the influence of the model angle, the pipe point deviates from the scene, for example, the pipe at the bottom of the valve cannot be connected with the pipeline and the valve switch is covered by a pipe. We can convert the BIM model to 3D pipe network using S3M (Figure 9).

**Figure 9.** From BIM model to 3D pipe network data model.

An adaptive rendering model is proposed with the combinations of the data structure of the BIM model and the spatial structure characteristics of 3D GIS. The adaptive rendering model contains a pipe layer setting method and a symbol matching method. The pipe point layer was shown in a custom thematic map, specifying a property field as the symbol style of the thematic map. After setting the parameters of the symbol model, the symbol matching method adjusts the angle according to the pipeline orientation, including the pipe at the bottom of the valve in the pipeline direction, and the valve switch perpendicular to the pipeline direction (Figure 10).

**Figure 10.** The adaptive rendering map in 3D GIS.

#### *3.4. Combination Computing Framework with GPU and OpenMP*

Rendering large-scale 3D pipe networks requires intensive computational time. The combinational parallel computing framework with GPU and OpenMP can greatly improve computing intensive pipe network-based applications. Considering that the spatial data exchange between the memory and the memory is time-consuming, and the large pipe network data cannot be copied to video memory at one time. We used the method of reading the pipe network data into the video memory. At the same time, we introduce the multi-thread processing strategy based on OpenMP to minimize the time occupation of GPU computing and data exchange to achieve the purpose of hiding data exchange time as much as possible.

Each CUDA core is only responsible for calculating the pixel result value of one specific location at the same time, and the kernel function specifies the specific calculation method. The specific workflow of a kernel function is to obtain the location of the pixel according to the built-in thread ID, and obtain the neighboring pixel at the location of the pixel (Figure 11). Then, the cell value is calculated based on the specific operator. Among them, the temporary variable corresponding to the neighboring pixel value is stored by using the shared memory inside the CUDA thread block, which can effectively improve the analysis performance. The computational resources were recycled when the combinational parallel computing framework was finished.

**Figure 11.** The workflow of the combinational parallel computing framework with GPU and OpenMP.
