Size-Adaptive Texture Atlas Generation and Remapping for 3D Urban Building Models

Abstract

A high-fidelity 3D urban building model requires large quantities of detailed textures, which can be non-tiled or tiled ones. The fast loading and rendering of these models remain challenges in web-based large-scale 3D city visualization. The traditional texture atlas methods compress all the textures of a model into one atlas, which needs extra blank space, and the size of the atlas is uncontrollable. This paper introduces a size-adaptive texture atlas method that can pack all the textures of a model without losing accuracy and increasing extra storage space. Our method includes two major steps: texture atlas generation and texture atlas remapping. First, all the textures of a model are classified into non-tiled and tiled ones. The maximum supported size of the texture is acquired from the graphics hardware card, and all the textures are packed into one or more atlases. Then, the texture atlases are remapped onto the geometric meshes. For the triangle with the original non-tiled texture, new texture coordinates in the texture atlases can be calculated directly. However, as for the triangle with the original tiled texture, it is clipped into many unit triangles to apply texture mapping. Although the method increases the mesh vertex number, the increased geometric vertices have much less impact on the rendering efficiency compared with the method of increasing the texture space. The experiment results show that our method can significantly improve building model rendering efficiency for large-scale 3D city visualization.

1. Introduction

High-precision 3D urban models are an important source of data for smart city systems [1] which has been widely used in facility management, development planning, emergency decision making, and so on. As the 3D urban model from UAV oblique photography still has the disadvantages of structural adhesion, white holes, mesh distortions, and texture missing, a manual 3D model is still necessary [2,3]. The high-precision manual modeling method augments the reality of the scene at the cost of the rapid increase in the amount of data. Real-time rendering of a detailed 3D model is still a challenge for computer graphics [4]. As for a complex textured urban model, the amount of texture data is much larger than the geometric data [5]. It usually has dozens or hundreds of textures, which are characterized by large amounts but small individuals. Since each texture can be loaded and rendered very quickly, it is of little significance to simplify and compress a single texture, but for graphics processing unit (GPU) rendering, large quantities of small textures significantly increase the costs of loading and rendering batches, slowing down the rendering efficiency. Rendering a large-scale urban model scene directly is prone to slow model loading, untimely texture display, and rough roaming. Therefore, packing multiple textures into atlases for a 3D urban model can solve the rendering limitation.

Over the past few years, a great amount of research has been conducted on 3D model simplification. Some researchers studied geometric mesh simplification, which is mostly beneficial for finely tessellated and geometrically complex scenes [6,7,8]. As a 3D urban building model has the characteristics of non-manifold edges, footprint borders, and complex roofs, the mesh simplification method has to consider these features to preserve corner sharpness [9,10,11]. The Google Draco library is widely used for 3D model simplification, which compresses the mesh geometry and remaining texture mapping [12]. Other researchers studied texture simplification and composition. Since the amount of texture data is much larger than that of the geometric data for the complex urban model, texture simplification of the model is of great significance for data compression. Texture compression by reducing the texture size and quality can compress the amount of data [13], while it has a poor effect on the rendering efficiency and the visualization effect of the multi-texture urban model. Texture composition by packing all the textures into an atlas is widely used to reduce loading and rendering batches for the GPU [14]. For 3D urban models, tiled textures are more frequently used than non-tiled textures [15], such as walls, windows, and roofs. To achieve texture remapping, the texture atlas is usually processed in three ways. One way is to apply Render to Texture (RTT) technology to generate a projected surface atlas [16], which tends to reduce the texture accuracy. Another way is to add a repeating processed texture into the atlas, which greatly increases the amount of texture data. The third way is to pack the original textures into an atlas, which is the most widely used method. Some scholars studied repeated texture mapping for atlases through Shader programming [17,18,19]. These methods need more storage space between individual textures to avoid visual repeated line effects. In the meantime, the repeating method of the texture atlas by Shader programming is affected by the 3D graphic API, and the processed model can only be used for specific 3D software [5]. On the other hand, the traditional methods of the packed atlas size are uncontrollable [20]. For typical graphics hardware, the maximum texture size is limited to a supporting role. The texture atlas must be compressed below the maximum supported texture size, which inevitably reduces the model accuracy. In summary, the above texture composition methods cannot achieve the optimal combination for the texture atlas in terms of accuracy, size, and storage space.

In this paper, the challenges are addressed to realize the texture composition for an urban building model. A size-adaptive texture atlas generation and remapping method for a 3D urban building model is introduced, which can pack all the textures of a model adaptively without losing accuracy and increasing extra storage space. First, the individual textures of a 3D urban model are classified into non-tiled textures and tiled textures. Then, the multiple textures are packed into one or more texture atlases adaptively, depending on the maximum supported texture size of the graphics processing unit card. At last, the mesh of the model is reconstructed and mapped according to the original texture type of each triangle. For the triangle with the original non-tiled texture, new UV coordinates in the texture atlases are calculated, but as for the triangle with the original tiled texture, it is clipped into many unit triangles to apply the texture mapping method. Although the method increases the mesh vertex number, the increased geometric vertices have much less impact on the rendering efficiency compared with the method of adding extra texture storage space. Meanwhile, the Google Draco algorithm can be applied to compress the mesh geometry and remaining texture mapping. To test the performance of our multi-texture composition method, comprehensive experiments using real data are conducted. Single building models and large-scale urban scenes are tested for their rendering efficiency and effect. The experiment results show that our method can pack all the textures of a 3D urban model with lossless accuracy and significantly improve the rendering efficiency of the large-scale urban scene.

This paper is organized as follows. In Section 2, an overview of the texture atlas processing framework is presented. In Section 3, the texture atlas generation and remapping methods are explained in detail. In Section 4, we conduct a complete experiment for this method and analyze the experimental results. Finally, conclusions are drawn and future work is planned in Section 5.

2. System Framework

Figure 1 shows the workflow of the texture atlas generation and remapping method. The core idea of the method is to pack all the textures with optimal storage space and maintain the texture accuracy. For a typical urban model, the specific processes are as follows:

(1)

Extract model data and classify the textures. The textures and geometric mesh are extracted from the model, and all the textures are classified into non-tiled textures and tiled textures;

(2)

Set the maximum texture size and pack all the textures. The maximum supported rendering texture size is acquired according to the graphics processing unit card, and all the textures are packed into one or more texture atlases by the improved lowest horizontal search and simulated annealing algorithms;

(3)

Remap the texture atlases onto the geometric mesh. For the triangle with the original non-tiled texture, the new texture coordinates in the texture atlases can be calculated and set to the mesh directly. However, for the triangle with the original tiled texture, the corresponding mesh triangle is clipped into many unit triangles, and each one is remapped using texture mapping as the triangle with the original non-tiled texture;

(4)

Export the new model with the texture atlases.

3. Methodology

The main processing steps of the method include texture atlas generation and texture atlas remapping.

3.1. Texture Atlas Generation

The texture atlas generation problem is known to be NP-complete [21,22]. Computational geometry methods [23,24,25] perform well in terms of unused space, but they are not efficient enough for large urban data sets. The lowest horizontal search algorithm is relatively simple and efficient, but it does not take into account the rectangular size difference, which tends to leave blank space [26]. Considering the texture features of 3D urban building models, the improved lowest horizontal search and simulated annealing algorithms are combined to pack all the textures into one or more texture atlases optimally.

3.1.1. Multi-Texture Packing

The lowest horizontal search algorithm refers to updating the horizontal line to create a strip and insert the textures into the strip, as shown in Figure 2. Considering the limitation of the maximum texture size, the algorithm is improved to pack textures into one or more atlases adaptively.

As for a building model, the original textures are described as $\left\{Ti\right|i=1,2\dots ,m\}$. The width and height of the i-th texture T_i are w_i and h_i, respectively, so the total texture area is $\mathrm{s}={\displaystyle {\sum }_{i=1}^{i=m}\mathrm{wi}h}i$. According to the maximum size of the supported texture of the graphics processing unit card, the textures are packed into n texture atlases, described as $\left\{A_j\right|j=1,2\dots ,n\}$. The width and height of the i-th texture atlas A_i are W_i and H_i, respectively, so the total area of the texture atlases is $\mathrm{S}={\displaystyle {\sum }_{j=1}^{j=n}{W}_j{H}_j}$. The following equation is for obtaining the maximum proportion value:

$$P(X)={\displaystyle {\sum }_{i=1}^{i=m}\mathrm{wi}h}i/{\displaystyle {\sum }_{j=1}^{j=n}{W}_j{H}_j}$$

(1)

As shown in Figure 3, the specific steps of the texture packing algorithm include the following:

(1)

Traverse the height value of all the textures and sort them in descending order. For textures with the same height, sort them in descending order based on the width value;

(2)

Get the maximum supported atlas size and create a blank image. The initial size is set to be the maximum supported texture size, such as 4096;

(3)

Set the initial horizontal line and iteratively insert the textures into the atlas. The height of the first horizontal line is set to be the height of the first sorted texture. For the i-th texture T_i, it is inserted into the atlas if possible. Otherwise, other textures are checked and inserted into the atlas. $\{(Ti,Aj)|i=1,2\dots ,m;j=1,2\dots ,n\}$ is used to update the packing state, where T stands for the unpacked textures and A stands for the texture atlases;

(4)

If no texture can be inserted into the current strip, the height of the horizontal line is updated. It is equal to the horizontal height plus the height of the current texture to be merged. Then the processing step goes to step 3 to constitute another stripe;

(5)

If the atlas cannot be inserted and the unpacked texture exists, a new blank atlas is created. Then, the processing step goes to step 2 to constitute another atlas;

(6)

The height of the last atlas is set to be the height of the horizontal line;

(7)

Export the texture atlases.

3.1.2. Atlas Optimization

Since the lowest horizontal search algorithm only achieves the local optimum, the simulated annealing algorithm is combined with it to make the packed texture atlases reach the global optimum. Simulated annealing (SA) is a stochastic approach that simulates the statistical process of growing crystals using the annealing process to reach its global minimum internal energy configuration [27]. The temperature formula is set to be $T=rT$, the initial temperature is T₀, and the minimum temperature is T_min. The specific steps of the simulated annealing algorithm include the following:

(1)

Attain the packed texture atlases by the lowest horizontal search algorithm as the initial solution X₀;

(2)

Randomly swap the orders of two textures in step 1 in Section 3.1.1, pack the textures using the lowest horizontal search algorithm, and attain the current solution X₁;

(3)

Calculate the value difference $\Delta P=P(X_1)-P(X_0)$. If $\Delta P\ge 0$, X₁ is accepted as a more optimal solution for texture packing. Otherwise, check $\exp (\Delta P/T)\ge Random(0,1)$. If this is true, X₁ is accepted and set to $X_0=X_1$. If this is not true, proceed to step 4;

(4)

Set $T=rT$ and repeat steps 2 and 3 until $T\le T_{\min }$;

(5)

Export the optimal texture atlases.

The parameter r is an empirical value ranging from 0 to 1. If r is large, the probability of searching for the global optimal solution is higher, but the searching process time is longer. Otherwise, if r is small, the searching process will be fast, but it may eventually reach a local optimum.

3.2. Texture Atlas Remapping

As all the textures of a building model are packed into one or more texture atlases, the texture atlases must be remapped onto the geometric mesh. The texture atlas remapping problem is calculating the new texture coordinates for each triangle mesh. Figure 4 shows the texture atlas remapping sketch map.

3.2.1. Non-Tiled Texture Remapping

For the geometric triangle corresponding to the original non-tiled texture, the new texture coordinate for the atlases can be calculated directly. The relationship of the original textures and the atlases is described as $\{(Ti,Aj)|i=1,2\dots ,m;j=1,2\dots ,n\}$. As shown in Figure 5, the original texture coordinate of a vertex in the geometric triangle ABC is (U₀, V₀). The original texture is packed into the atlas in the position of (u, v). The width of the original texture is w, and the height is h. The width of the texture atlas is W, and the height is H. Therefore, the new texture coordinate of the part in the atlas for the triangle ABC can be calculated as follows:

$$\left\{\begin{array}{l}U=(u+U0*w)/W\\ V=(v+V0*h)/H\end{array}\right.$$

(2)

The coordinate (U, V) is the new texture coordinate of the vertex corresponding to the atlas. The texture coordinates for the three vertices of triangle ABC can be computed, which realizes non-tiled texture remapping.

3.2.2. Tiled Texture Remapping

A tiled texture is also called a repeated texture, which indicates that the texture coordinate is larger than one. The current 3D graphics API is designed to repeat the entire texture. Once the original tiled texture is packed into the texture atlas, the triangle cannot repeat the texture anymore. In order to keep the combined texture size optimal, the geometric triangle corresponding to the original tiled texture is clipped into many unit triangles for texture mapping.

Rectangle Clipping

Figure 6 shows the method of clipping the geometric triangle using the unit rectangle. O-XYZ is the 3D global geometric Cartesian coordinate system, the triangle in the global coordinate system is ABC, and its vertex coordinates are (X_a, Y_a, Z_a), (X_b, Y_b, Z_b), and (X_c, Y_c, Z_c). Meanwhile, o-uv is the 2D local texture coordinate system, the triangle in the local coordinate system is abc, and its texture coordinates are (u_a, v_a), (u_b, v_b), and (u_c, v_c). The triangle is clipped into many unit triangles in the 2D texture coordinate system, and the geometric vertices of the unit triangles are calculated in the 3D Cartesian coordinate system. The specific steps of the triangle clipping algorithm include the following:

(1)

Extract the geometric coordinates and original texture coordinates of each vertex in the triangle;

(2)

In the 2D texture coordinate system, the unit rectangle is used iteratively to cut triangle abc. It is clipped into many small triangles. The texture coordinate of each new vertex can be calculated, which is described as $\{(Ui,Vi)|i=1,2\dots ,k\}$. The vertex p is the created vertex in the 2D texture coordinate system, and the texture coordinate corresponding to the original texture can be divided into two cases;

(a)

The vertex p is located inside triangle abc, so the texture coordinate of vertex p corresponding to the original texture can be calculated:

$$\left\{\begin{array}{l}u=m\\ v=n\end{array}\right.$$

(3)

(b)

The vertex p is located on the edge of triangle abc. Assuming that vertex p is located on the edge ab, it can be divided into two situations. One situation is that vertex p is located on the m-th column line, so the texture coordinate of vertex p corresponding to the original texture can be calculated as follows:

$$\left\{\begin{array}{l}u=m\\ v={ \mathrm{v}}_a+\frac{m-u_a}{u_b-u_a}*(v_b-v_a)\end{array}\right.$$

(4)

Another situation is that vertex p is located on the n-th row line, so the texture coordinate of vertex p corresponding to the original texture can be calculated as follows:

$$\left\{\begin{array}{l}u={ \mathrm{u}}_a+\frac{n-v_a}{v_b-v_a}*(u_b-u_a)\\ v=n\end{array}\right.$$

(5)

(3)

Calculate the geometric coordinates of the created vertices. The geometric coordinates of the created vertices are described as $\left\{\mathrm{P}i\right|i=1,2\dots ,k\}$. As we know, the 2D texture coordinates can be projected to the 3D Cartesian coordinate system to calculate the geometric coordinates, with geometric vertex P corresponding to texture vertex p. A Bursa-Wolf Seven-Parameter Transformation [28] is applied to compute the geometric coordinates of the new vertices. The geometric coordinates of vertex P can be calculated as follows:

$$\overrightarrow{P}=\overrightarrow{O}+U*\overrightarrow{X}*scaleX+V*\overrightarrow{Y}*scaleY$$

(6)

where $\overrightarrow{O}$ is the geometric coordinates corresponding to the texture coordinate system origin o, U and V are the texture coordinates of vertex P, $\overrightarrow{X}$ and $\overrightarrow{Y}$ are the direction factor, and scaleX and scaleY are the scale factors.

Texture Mapping

Since the geometric triangle is clipped into many unit triangles, texture mapping can be applied to the texture atlases. As shown in Figure 7, vertex P is the new vertex clipped by the n-th row and m-th column of the unit rectangle. The original texture coordinate of vertex P is $(\mathrm{u},v)$, so the new texture coordinate of vertex P for the texture atlas can be calculated as follows:

$$\left\{\begin{array}{l}U=u-m\\ V=v-n\end{array}\right.$$

(7)

Since the texture coordinates and the geographic coordinates of vertex P and Q are acquired, the new triangle APQ is built. Other new vertices and triangles can be constructed in the same manner. By clipping triangles and adding geometric vertices, the problem of repeated texture mapping is solved. The texture atlases are remapped in this manner.

4. Experiment and Discussion

To test the performance of our multi-texture composition method, comprehensive experiments using real data were conducted. The test area was in the city of Huaibei in China. The longitude of the extent ranges from 116.7632° to 116.8391°, and the latitude ranges from 33.9397° to 33.9823°. The original format of the experiment data was a Max file, and the amount of data was 12.1 GB. The size-adaptive multi-texture composition tool was programmed in C++ and C#. It applied the Assimp [29] library to load and convert the model files. The urban models were converted to GLB format and organized into the 3DTiles format [30]. For the large-scale urban model scenes, Tomcat was used to publish these model data, and the open-source web 3D library Cesium [31] was applied to render the urban models. The Google Chrome browser was used to explore the model scenes. The reported results were measured on an Intel Core i5-6400 @2.70 GHz (4 CPUs) PC with 8GB of core memory and an NVIDIA GeForce GT 730 with 1GB of memory.

4.1. Experimental Results

In order to efficiently load and render the massive high-precision building models, the files were processed and organized appropriately. The processing workflow is shown in Figure 8, the specific steps were as follows:

(1)

Extract the urban scene from the Max file;

(2)

Split the scene into many individual building models and convert them into OBJ format;

(3)

Pack the multi-textures into the texture atlases and remap them onto the geometric mesh for each building model;

(4)

Convert the models into GLB format and organize them by the 3DTiles format;

(5)

Load and render the urban scene in Cesium.

Figure 9 shows the textures of a building model before and after processing. Figure 9a shows 95 textures of the original model, including non-tiled textures and tiled textures. The maximum texture size was set to be 2048, so all the textures were packed into 4 texture atlases as shown in Figure 9b. The number of textures was significantly reduced by texture packing. The original geometric mesh is shown in Figure 10a, and it was clipped for texture atlas remapping as shown in Figure 10b. Through the comparison, it can be seen that the number of geometric vertices increased after the texture remapping. Figure 11a is the visual effect of the original model, and Figure 11b is the visual effect of the processed model. Compared with the original model, the visual accuracy of the texture-packed model was almost the same.

The texture-packed building model was loaded and visualized in Cesium as shown in Figure 12. It can be seen from the figure that the rendering effect of the model after texture processing was good. The visual repeated line effect did not exist on the 3D building model’s surface. The visual effect of the texture-packed model met the actual demand. This shows that the method in this paper can achieve texture packing and remapping for 3D urban building models.

Table 1. Comparison of single models.

Model Name	Texture Processing	Texture Number	Vertex Number	Draco Compressed File Size (MB)
HB15b085	Original	16	541	1.82
	Packed	1	4659	0.54
HB15b121	Original	37	19,500	5.04
	Packed	2	36,779	1.08
HB15b001	Original	95	4372	5.14
	Packed	4	47,585	2.57

shows the data comparison results before and after texture packing for three urban models of different complexities. The original and processed models were compared in GLB format. The multi-textures were packed without image compression, and the geometric mesh was clipped for texture atlas remapping. By comparing the data before and after processing of the same 3D urban building model, it can be found that the number of textures decreased, and the number of geometric vertices increased after texture packing, but the storage size of the model file decreased. Although more vertices were generated after the geometric mesh was clipped, the vertex storage was efficient, and the increased storage space was limited. Meanwhile, the geometric mesh of the model could be further compressed by the Google Draco algorithm. The processed model with added vertices was simplified more efficiently than the original model. As a result, the processed model’s file was smaller than the original model’s file.

Since a single building model file size is relatively small, the large-scale urban scene was tested for its rendering frame rate. All the building models were processed and organized into 3D Tiles format by a quadtree index, which was loaded in Cesium as shown in Figure 13. Compared with the original urban scene, the texture-packed urban scene was loaded and rendered faster. The reasons for this include two points: (1) the building model was packed with fewer textures, which decreased the texture loading and rendering batches, and (2) the file size of the processed model was smaller, which decreased the loading time. When comparing the rendering smoothness of the urban scene, the rendering of the original 3D scene was not smooth, while the rendering FPS of the processed urban scene was above 30, which is relatively smooth. Through the processing method in this article, the rendering effect of a large-scale urban model was realistic, and the rendering result was smooth.

4.2. Discussion

When compared with the texture composition method [5], our method provides three main advantages: (1) the method generates texture atlases with less storage space in the case where the textures are not compressed; (2) the packed atlas size is controllable, and the multiple textures are packed into one or more texture atlases adaptively depending on the maximum supported texture size of the graphics processing unit card; and (3) the texture-packed building model can be loaded in different 3D visualization software, which is not affected by the 3D graphic API.

The method was designed from the beginning to generate texture atlases adaptively and efficiently. Although the number of geometric vertices increased for texture remapping, it is worth generating 3D building models with adaptive texture atlases for improving the rendering efficiency. As for complex outdoor and indoor building models, the number of geometric vertices increased fast, so both the geometric and texture data should be further considered for model compression.

5. Conclusions and Future Work

This paper presents a size-adaptive texture atlas generation and remapping method for urban building models. The multi-textures are packed into one or more texture atlases adaptively, depending on the maximum supported texture size of the graphics processing unit card. Then, the mesh of the model is reconstructed and mapped according to the original texture type of each triangle. As for the triangle with the original non-tiled texture, new UV coordinates in the texture atlases are calculated, while the triangle with the original tiled texture is clipped into many unit triangles to apply the texture mapping method. Single building models and large-scale urban scenes were tested for their rendering efficiency and effect. The experimental results show that the method could pack all the textures of an urban model without losing accuracy and significantly improve the rendering efficiency of the large-scale urban scene.

This paper provides a solution for users to pack multiple textures and export a textured building model. Since this method achieves texture remapping at the cost of increasing the geometric vertices, it can be further improved. Meanwhile, with the development of KTX texture compression technology [32], the data volume of the 3D building model can be further compressed. In the future, 3D building model compression methods that take into account both the geometric and texture accuracy will be studied for smart city visualization.