Numerical Evaluation of Hot Air Recirculation in Server Rack

Response 1: Thank you for pointing this out. We agree with this comment. Therefore, we have updated the summary. [Page 1, line 23-26 of the abstract is updated to summaries the finding]   

“[The server with series configured fan system experienced an increase of 9% average inlet air temperature when com-pared to the original server under similar condition. The current work serves as a base for integrating liquid and air-cooling systems to form hybrid cooling systems for high-density racks in legacy data centers]”

Comments 2: [- I believe that the introduction should focus more on the purpose of the presented work, clearly indicating the research hypothesis, showing what has already been done in the world and what constitutes the authors' contribution to the development of the topic.]

Response 2:  We agree. While it is important to include the purpose of the presented work in the introduction, it is also important to consider the potential limitations and alternative perspectives to provide a well-rounded view of the topic. We wanted to highlight recent studies concerning hot recirculation in data-centers and the typical type of servers that are used and whose operation and nature are available in literature.

The purpose of the study was addressed in the last paragraph of the introduction section.

Comments 3: [-Section 3.1 can be omitted as it is the absolute basis of how CFD works.]

Response 3: Agree. I/We have, accordingly, done/revised/changed/modified…..to emphasize this point. This section shows the CFD approach we took to reach the result. It is important to be explicit in this regard as it would allow other researchers to replicate the study. This section of the CFD process facilitates transparency and knowledge transfer as well.

Comments 4: [-Fig.10 - unclear unit (wc)]

Response 4: Agree. We have, accordingly, revised Figure 10 caption to emphasize this point. [Figure 10 on page 13 is modified to have two sections (a) and (b). The first section shows the exact literature results, and the latter shows the CFD results. The units are corrected]

“[inches wc (water column)]”

Comments 5: [-Chapter 4 - what turbulence solver was used?]

Response 5: Thank you for pointing this out.

[ Page 8, section 3.1, line 224]

“[Governing Equation and the Large-Eddy Simulation Turbulence Model]”

Comments 6: [-No real discussion of the results.]

Response 6: Agree. We have accordingly revised and added table to emphasize this point. [Table 6 added on page 18, section 4.2, line 465-467, which shows the inlet and outlet air temperature difference for both fan system configuration. The table shows the impact fan system configuration impact on the cooling effectiveness.]

“[As shown in Figure 13(b), the server fans were reconfigured to a series arrangement with four additional server fans added behind the original server fans, but overall server configuration remains unchanged. All the openings, internal compartments, boundary conditions, and fan set speed remained unchanged. As shown in Figure 11, the results showed an increased system available pressure when the system was running against a 10 Pa hot aisle back pressure. Based on the proportionality be-tween the fan static pressure and the airflow rate through the system, the supply inlet temperature is expected to decrease. The increase in system pressure is expected to lower hot air recirculation at a backpressure of 10 Pa in comparison to the parallel-configuration fan set.”

“Figure 12 shows the system available pressure at free delivery pressure (0pa backpressure). The results showed no improvement in the available pressure head of the system. The series configuration consumes significant power with no significant improvement in air distribution. These results also reveal that the series fan configuration is not ideal for servers that run on a low IT load (computation workload), which is prevalent in cloud computation servers]”

Comments 7: [- Conclusion 1 - "The 3D scanning technology can be used to reproduce detailed CAD models, which can subsequently be used for detailed CFD modelling." This statement is obvious and does not result directly from research. I understand that the authors did not intend to explore this topic.]

Response 7: Agree. We have, accordingly, modified it to emphasize this point. [Page 18,section 5 we have removed the statement and rectified the percentage difference between CFD and experimental results]

“[The server-supply inlet temperature was used to validate the CFD model based on the experimental results. Based on the results, there was an average of 3% between the experimental data and CFD model results under similar environmental and operational conditions. This study represents a significant advancement towards real-life modeling of complex configurations.]”

Comments 8: [- Conclusion 3 - "The change in the server fan configuration from a parallel to a series arrangement increased the available static pressure". This is essentially the basis of ventilation as an engineering issue.]

Response 8 Agree. We have, accordingly, modified it to emphasize this point. [Page 18,section 5 we have removed the statement and rectified the inlet air temperature percentage difference]

“[Increasing the server fan head does not address the issue of hot air recirculation in the server, which aggravates the situation. The server inlet air temperature increased by an average of ~5% from -20 Pa to -5 Pa backpressure, and by 19% at the free delivery point (0 Pa). The inlet air temperature was further increased by 10% at back pressures of 5–20 Pa. The outlet air temperature decreased by 1%, signifying poor cooling effectiveness owing to high static pressure]”

Comments 9: [- Conclusion 4 - "This also highlights correctly sizing the server fans". Yes, but what guidelines do the authors propose here? Can the presented results be generalized, for example, to computer hardware manufacturers?]

Response 9: Agree. We have, accordingly, modified it to emphasize this point [Page 18, section 5, line 508-512, we have updated the point to include the proposed server fan system sizing guideline]

“[This also highlights the importance of correctly sizing the server fans, which can significantly impact the overall server thermal performance, even under low-load conditions. It was established that a positive net system pressure does not necessarily result in the elimination of hot-air recirculation. Server fan sizing or fan set configuration should be dictated by the required server component temperature i.e. CPU]”

Comments 10: [-The list of references should be edited under the guidelines.]

Response 10: Agree. I/We have, accordingly, done/revised/changed/modified…..to emphasize this point. Discuss the changes made, providing the necessary explanation/clarification. Mention exactly where in the revised manuscript this change can be found – page number, paragraph, and line.]

“[updated text in the manuscript if necessary]”

4. Response to Comments on the Quality of English Language

Point 1:

Response 1: We have updated all the issues raised as per response 1-10 on this report  

5. Additional clarifications

Comments 1: [Line 201 – the Authors have stated: “To reproduce all major server components, neglecting smaller components that would have warranted a much smaller mesh size…” – in the paper should be added a discussion and explanations how neglecting smaller components will influence obtained numerical model results. Sometimes, neglecting smaller components can be highly influential on the obtained results and on their accuracy and precision in relation to the real exploitation conditions.]

Response 1: Thank you for pointing this out. We agree with this comment. Therefore, added this comment:

It needs to be mentioned that a mesh of greater fineness would be necessary to capture the intricate geometry and fluid dynamics of smaller components. Yet, employing a finer mesh raises the count of cells or elements within the computational domain, consequently increasing the computational burden. Therefore, accommodating highly refined mesh for smaller components presents challenges in parallelization efficiency [23]. Our study attempted to balance the workload between processors which proved to be more difficult, potentially undermining the overall efficiency of the computation. However, a grid sensitivity in section 3.2 mitigated this problem.

Comments 2: [In the paper should be added a Nomenclature inside which will be listed and explained in one place all abbreviations, symbols and markings used throughout the paper text. I believe that the Nomenclature will be helpful to many readers.]

Response 2: Thank you for pointing this out. We agree with this comment. Therefore, we have added the nomenclature. [A nomenclature is added on page 19, line 528-563, which includes all the abbreviations and symbols used throughout the article.]

Comments 3: [Throughout the paper can be found many obvious and typing mistakes – please perform careful check and corrections. For example – in the paper occur two Subsections 3.1.1; Equations are not properly numerated (after equations 8 and 9 follow equations 8 and 9), etc. All such mistakes must be removed.]

Response 3: Agree. We have accordingly updated the equations and sections numbering. [Section 3.1.1 Virtual disk model: Actuator disk methods on page 9, line 255 is updated to 3.1.2. Equation 8 and 9 on page 11, line 295-296 are updated to equation 10 and 11 respectively.]

“[3.1.2 Virtual disk model: Actuator disk methods, GCI_"fine " =(1.25|ε|)/(r^p-1)  (10), ε=(f_2-f_1 )/f_1     (11)  ]”

Comments 4: [Table 3 is totally unclear and it is not explained at all. This table as well as all the values presented in this Table should be properly explained (it is not clear what they actually present, where are they used and what exact meaning of the presented data is). In addition, in the text above this Table which explains Table heading is written: “efficiency (Eta)” – but in the Table is used “Tau” – some obvious problems and confusion occur with this Table, corrections and explanations are required.]

Response 4: Agreed. We have, accordingly revised that the paragraph to describe the fan performance curves and how they were adopted/model to give the virtual disk its body force propeller. It presents fan effects.

The fan effects were modeled via the body force propeller method, part of the Star CCM+ virtual disk model. Fluid dynamic forces on the virtual disk, resulting from fluid interaction, were represented using the Wageningen B-series [28] propeller performance curves as shown in Table 3. Table 3 is the data imported to Star CCM which matches the nondimensional advance ratio (J) thrust (Kt), torque (Kq), and efficiency (Eta) as properties with their respective columns.

Comments 5: [Validation (Figure 10):

- First of all, in the Legend of a figure should stand that experimental results are from literature [10], so the exact literature is required.

-Secondly, only a Temperature against Backpressure is not fully reliable for the validation. The proper validation should contain at least few more operating parameters. So, the validation results must be expanded with direct comparison of more measured and calculated results – two parameters only are surely not sufficient for the conclusion that the validation is properly performed. A special attention should be placed on expanding validation results – at the moment they are not sufficient (at least in my opinion).

- Thirdly, the Authors have stated in the paper: “there is an approximate 10% difference for each data point between the experimental data and CFD model results” – such difference should be properly explained. Also, the Authors should explain which difference between measured and calculated values can be considered as appropriate. In many engineering applications, the 10% difference is not acceptable, so the detail explanations and discussions are required.]

Response 5: Thank you for pointing this out. We agree with this comment. Therefore, we have modified figure 10 to show the exact literature and added a table showing individual percentage difference between the CFD and experimental results. The summation of the percentage difference was also rectified. [Figure 10 on page 13 is modified to have two sections ((a) and (b)). The first section shows the exact literature results and latter showing the CFD results. Table 5: Experimental IPMI and CFD inlet air temperature results are added on page 13-14, line 348 showing individual data point percentage difference. The percentage difference was also rectified by averaging them, which is currently 3% and within the acceptable engineering range.]

“[Figure 10(a) shows the impact of varying server back pressure on the IPMI (onboard sensors) and Dis(discrete/cold aisle sensor) inlet air temperature experimental results from Khalili et al. [10], experimental study on the impact of internal design on the efficiency of IT equipment in a hot aisle containment system. Figure 10(b) shows the CFD inlet air temperature results under the same experimental conditions, where the fan set speed was maintained constant at different back-pressure magnitudes.]”

Comments 6: [Some calls on the Figures throughout the paper are missing or they are wrong – what can be really frustrating during the paper reading. For example – Lines from 383 up to 393 – where is callon Figure 11 to which this text is related? Lines 399 and 400 – 0Pa backpressure results are presented in Figure 12, not 11; Line 405 – it is written: “Figure 12 presents the velocity streamlines” –Figure 13 presents the velocity streamlines, etc. Corrections are required throughout the paper.]

Response 6: Thank you for pointing this out. We agree with this comment. Therefore, we have updated section 4.1 to reference the correct figures 11,12 and 13. [Section 4.1 on page 14-15, line 401-422, was updated to reference the correct figures.]

“[As shown in Figure 13(b), the server fans were reconfigured to a series arrangement with four additional server fans added behind the original server fans, but overall server configuration remains unchanged. All the openings, internal compartments, boundary conditions, and fan set speed remained unchanged. As shown in Figure 11, the results showed an increased system available pressure when the system was running against a 10 Pa hot aisle back pressure. Based on the proportionality be-tween the fan static pressure and the airflow rate through the system, the supply inlet temperature is expected to decrease. The increase in system pressure is expected to lower hot air recirculation at a backpressure of 10 Pa in comparison to the parallel-configuration fan set.”

“Figure 12 shows the system available pressure at free delivery pressure (0pa backpressure). The results showed no improvement in the available pressure head of the system. The series configuration consumes significant power with no significant improvement in air distribution. These results also reveal that the series fan configuration is not ideal for servers that run on a low IT load (computation workload), which is prevalent in cloud computation servers”]”

Comments 7: [Figure 13 title is wrong – this Figure presents velocity streamlines, not system head pressure comparison.]

Response 7: Thank you for pointing this out. We agree with this comment. Therefore, we have updated Figure 13. [Figure 13 on page 16, line 435-436, was updated.].

“[Figure 13: (a) Velocity magnitude streamlines comparison between parallel and (b) series fan configurations at 6800 rpm speed and 0 Pa backpressure]”

Comments 8: [Lines from 449 to 452 – this description is valid for a series or parallel fan arrangement? Please, be more specific in the description of Figure 15 (as well as in the descriptions of other Figures) and clearly highlight the differences in operating parameters between series and parallel fan configurations.

As this description is also placed in the Conclusions, it is absolutely not clear what the Authors mean by this description. Itis written:

“The server inlet air temperature increased by 5% from -20 Pa to-5 Pa backpressure, and by 23% at the free delivery point (0Pa).” – according to Figure 15, this is valid for series fan arrangement. “The inlet air temperature was further increased by11% at back pressures of 5–20 Pa.” - according to Figure 15, this is valid for series fan arrangement. “The outlet air temperature decreased by 1%, signifying poor cooling effectiveness owing to high static pressure.” – decrease in outlet air temperature is seen for parallel fan arrangement only.]

Response 8: Agree. We have accordingly revised and added table to emphasize this point and updated Figure 15 description. [Table 6 added on page 18, section 4.2, line 465-467, which shows the inlet and outlet air temperature difference for both fan system configuration. The table shows the impact fan system configuration impact on the cooling effectiveness.]

“[Figure 15: Inlet and outlet air temperature comparison between parallel and series fan configurations with variation in backpressure. The server fan system speed is kept at 6800rpm]”

Comments 9: [Scientific novelty and paper contribution to the existing research field are not clear and obvious (at least to me). Maybe proper explanations and removing so many mistakes in the paper will properly highlight paper novelty and contribution – at the moment they are not clear.]

Response 9: Thank you for pointing this out, we have updated all the mistakes as correctly pointed out. [we have updated all the issues raised as per response 1-9 on this report.]

Comments 10: [The List of the References can be enlarged with more recent literature from this research field.]

Response 10: Thank you for pointing this out, we have checked all the recent relevant literature and there is not a significant shift with regards to this subject.  

Comments 11: [The English is understandable, but it can be improved in many sentences or whole paper parts.]

Response 11: Thank you for pointing this out, we have updated all the mistakes as correctly pointed out.

4. Response to Comments on the Quality of English Language

Point 1:

Response 1:

5. Additional clarifications

[Here, mention any other clarifications you would like to provide to the journal editor/reviewer.]

Comments 1: [The introduction are detailed enough, the materials and methods shows the making of the geometry, and converting to it into CFD model. However, I missing a Table or some graphical implementation of the materials used, and the boundary conditions.]

Response 1: Thank you for pointing this out. We agree with this comment. Therefore, we have added boundary conditions.

Page 12, section 3.3.1, line 340-345, we have added boundary conditions.

Comments 2: [During the mesh independence study, the 1 mesh is shown twice. Why is that? There are no changes in magnitude of the mesh? With the decrease of the base size, the error should be lower? How, that error value was calculated. Have the authors used boundary layers (inflation)?]

Response 2: Agree. I/We have, accordingly, done/revised/changed/modified…..to emphasize this point.

Thank you for pointing out this error. Base 5.5 mm was mistakenly indicated twice. The simulation was not necessarily repeated twice. We have deleted it from the table. The calculation of the percentages were updated as follows:

To provide clarity on the inflation layers modelled. This paragraph that describes the method used to extract the surface and volume with prism layer refinement was added.

The internal volume of the server was extracted using the surface wrapper feature of Star CCM+. The volume of interest is the volume of the surface wrapper extracted from the self-contained volume of the server. In other words, all outer surfaces of the internal parts of the server were surface-wrapped to form a volume within the server. The method was used to create seed points placed at two defined locations within the server and extract the volume. A surface mesh wrapper was used concurrently with trimmer volume meshes (hexahedral mesh with a minimal cell skewness angle close to zero to target orthogonal meshes around concave cells). The prism layer with a total thickness of 1E-4 m from the surface and seven inflation layers, produces near-wall cells of y^+=5 the values used. The subsequent objective was to determine the optimal mesh grid size that would prevent a significant decrease in the model's accuracy resulting from the discretization of the computational domain [29].

Comments 3: [During the model validation a numeric error value between the measured and simulated values should be calculated. The curve in Figure 10 ins continuous, or contains discrete points? If the latter please change the Figure accordingly.]

Response 3: Agree. We have, changed the figure.to emphasize this point. The error bars were added and the points turned to discrete points. Please see below:

Figure 10: (a)Variation of IPMI IAT with backpressure; permissions republished with permission from Khalili et al. [10]. ASME (2018) published the impact of internal design on the efficiency of IT equipment in a hot aisle containment system - an experimental study. (b) Variation of inlet temperature with backpressure CFD results with 5% error bars.

Comments 4: [Figure 11 is there a stationary work point? Why there are only results for 60 second are shown?]

Response 4: We have found 60 seconds suitable for our chosen timestep. It is computationally expensive to model longer periods using the same timestep. A timestep could be increased to model longer periods, i.e., hours. However, a model would need to have its stability over the increased timestep which could be further expensive with not much change in the results.

The noticeable working point of the two systems is that after 22 seconds the fans in parallel provide a reduced pressure in the system.

Comments 5: [Similar comment should be applied for all the result graphs.]

Response 5: Agreed. We have included this analyses to underline this point.

Figure 12 shows the system’s available pressure at free delivery pressure (0 Pa backpressure). The series configuration consumes significant power with no significant improvement in air distribution. It provides not further noticeable contribution to the system after 41 seconds. These results also reveal that the series fan configuration is not ideal for servers that run on a low IT load (computation workload), which is prevalent in cloud computation servers.

Comments 6: [How the changes in rpm changes the performance of the system?]

Response 6: Thank you for pointing this out.

The analysis is carried out in a constant fan speed conditions to minimize the number variable that can have an impact of the air flowrate.

Comments 7: [Why not use higher performance ventilator instead of the same one?]

Response 7: Thank you for pointing this out this point.

We wanted to maintain the same equipment frame, which would make it easier to retrofit

Comments 8: [How the incomplete cooling should effect beyond the computer.]

Response 8: Thank you for pointing this out this point.

Insufficient cooling of server rack or IT equipment leads to high power consumption and computation downtime. It has a huge impact on the global economy and human life as a general.

4. Response to Comments on the Quality of English Language

Point 1:

Response 1:    (in red)

5. Additional clarifications

[Here, mention any other clarifications you would like to provide to the journal editor/reviewer.]

Response 1: Thank you for pointing this out. We agree with this comment. Therefore, added this comment:

Figure 10 was modified to include a second validation of air flow vs static pressure.

Comments 2: In the first paper version, the Authors have highlighted that 10% difference between measurement results and CFD calculations are obtained. It seems that the Authors have repeated calculations, because now that difference is 3%.

It is still unknown which difference between measurements and CFD results can be considered as sufficiently accurate and precise - that should be added and explained in the paper. Without this explanation, even 3% difference can be too high (what surely is for some applications and some operating parameters).

Response 2: Thank you for pointing this out. We agree with this comment. Therefore, we have added this explanation related to accuracy of the model. An additional validation of the numerical model was conducted for the static pressure across the chassis (inlet baffle) of the server in relation to the airflow rate through it. The numerical model showed a discrepancy within 15% of the experimental results. This deviation is attributed to the flow impedance being proportional to the square of the flow stream, which mandates that the surface resistance of the chassis model be highly accurate. It is postulated that the accuracy of the virtual disk is conditional to the precision of the fitting curve, the mesh density, and the numerical efficiency of the solver.