To Cache or Not to Cache

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The authors propose a new non-datapath caching algorithm, Fear Of Missing Out (FOMO), that enables selective updates to the cache. This design can be generally augmented with existing datapath cache policies. FOMO operates by transitioning between two states: Insert and Filter, allowing it to selectively disable the cache update process without relying on complex control parameters. The simplicity and effectiveness of the algorithm design are impressive. Below are some specific comments and questions:

In the Filter state, the difference between FOMO and conventional datapath cache policies is that the cache will be updated only if the item incurs two cache misses: one to be recorded in the Miss-History and one to trigger the actual cache update, regardless of the interval between the two misses. I am wondering if this is reasonable, as an adversarial request list can still easily trigger a lot of cache updates.

Will there be a size limitation for the Miss-History queue? It seems that the queue can grow infinitely, causing performance issues due to the time required to check if an item is in the queue.

Is HR_{cache} and HR_{Miss-History} the hit rate? It seems the notations are not explained in the manuscript. The transition condition, i.e., HR_{cache} vs. HR_{Miss-History}, and HR_{Miss-History} > 5%, seems very empirical. Is this valid for all workloads? Can we make these conditions tunable?

How can we decide the underlying policy to incorporate with FOMO for different workloads?

Author Response

Reviewer # 1

 

In the Filter state, the difference between FOMO and conventional datapath cache policies is that the cache will be updated only if the item incurs two cache misses: one to be recorded in the Miss-History and one to trigger the actual cache update, regardless of the interval between the two misses. I am wondering if this is reasonable, as an adversarial request list can still easily trigger a lot of cache updates.

Thank you for this comment and yes, it is possible that an adversarial workload can trigger a lot of cache updates. In fact, adversarial workloads are a challenge for all well-defined caching algorithms. To consider how commonly such adversarial workloads occur in the wild, we evaluated FOMO using over 200 production storage workloads with 6 different configurations each, totaling over 1200 unique setups. We find that FOMO is able to adequately function within its parameters to favorably impact the vast majority of these setups. 

Will there be a size limitation for the Miss-History queue? It seems that the queue can grow infinitely, causing performance issues due to the time required to check if an item is in the queue.

The Miss-History queue size is also equal to the cache size. This is clarified in the paper (Section 3). 

Is HR_{cache} and HR_{Miss-History} the hit rate? It seems the notations are not explained in the manuscript. The transition condition, i.e., HR_{cache} vs. HR_{Miss-History}, and HR_{Miss-History} > 5%, seems very empirical. Is this valid for all workloads? Can we make these conditions tunable?

Yes, these are hit-rates and now also defined in the paper (Section 2). The constants used in the transition conditions were indeed empirically derived and seemed to work well with the vast majority of the over 1200 setups (workload/cache-size configs) that we evaluated FOMO with. 

How can we decide the underlying policy to incorporate with FOMO for different workloads?

While choosing the underlying algorithm is not the focus of this work, we believe that an empirical approach is likely to be the easiest. However, workload analysis (e.g., frequency vs. recency profiles) and working set sizes are also informative and have been used in past work. 

Reviewer 2 Report

Comments and Suggestions for Authors

The paper is promising and has good writing qualities. However, the main concern is the comparison among the related work in the field. It is rather limited, and it places itself to sound incremental. It may not be incremental in the end, but I am not completely confident in its current form, because it doesn't mention existing similar techniques like bypassing, and references are relatively old.

 

The plots are generally not very helpful. Even the motivation figure is probably past results/ and  a bit unexplained, and does not explain how the hit rate losses for small periods of time translate to considerable performance losses. There is overuse of violin plots, and is difficult to compare numbers. In the related writing, I would always prefer to see more specific numbers. Maybe you also need to consider using a timing simulator rather than a hit rate simulator.

 

For a major revision, I would recommend to revisit related work, and to disassociate with mARC and other replacement policies in general, at least in introduction etc., and improve most figures. The distinction for "non-datapath" and "general solution" is not consistent throughout the paper, and may limit the reach of the paper. I am not 100% sure this is possible within the journal time limits, but it will either uncover contribution issues, or it will greatly improve the paper.

 

Other

. Figures 3,4 too simplistic to be helpful/explanatory. It is just boxes with numbers (insert arrows etc.)

. Making the cache simulators/experiments open-source would increase the contribution and confidence of the paper

Author Response

The paper is promising and has good writing qualities. However, the main concern is the comparison among the related work in the field. It is rather limited, and it places itself to sound incremental. It may not be incremental in the end, but I am not completely confident in its current form, because it doesn't mention existing similar techniques like bypassing, and references are relatively old.

The updated version of the paper now includes discussion of recently published papers including S3FIFO, a FIFO-based cache replacement algorithm and CacheSack, a coarse-grained cache admission policy. These are papers published within the last year. Both works are orthogonal to FOMO. S3FIFO is a novel FIFO-based datapath cache replacement algorithm that is orthogonal to FOMO and can work with FOMO to create a non-datapath version. CacheSack sets a static admission policy at the granularity of an entire workload. Finally, we also discuss the recently-published work by Ajdari et al. The focus of this work is on optimizing accesses to I/O caches which are non-datapath caches and not on managing admissions. The updates are annotated in the paper.  

The FOMO work builds upon the findings and learnings from the mARC and LARC work, but it uses an entirely novel  design relative to these prior works to address fundamental limitations with their designs. The updated version now focuses the presentation on the novelty of the FOMO in the introduction and motivation sections, making the descriptions entirely self-contained. The new version moves the prior discussion regarding the differences from the mARC and LARC contributions to the related work section. We agree that this provides a much clearer description of the novelty of FOMO as well as provide the necessary comparisons to the closely related work. 

The plots are generally not very helpful. Even the motivation figure is probably past results/ and  a bit unexplained, and does not explain how the hit rate losses for small periods of time translate to considerable performance losses. There is overuse of violin plots, and is difficult to compare numbers. In the related writing, I would always prefer to see more specific numbers. Maybe you also need to consider using a timing simulator rather than a hit rate simulator.

1. We have included specific numbers where possible in our discussions as has been highlighted in the updated document. The motivation figure (Figure 2) is an illustrative representation capturing the impact of delayed workload shift detection in state-of-the-art non-datapath caching algorithm, mARC. It is  included for establishing context for FOMO. 
2. The use of violin plots is a result of the large number of data points generated within our experiments. Figures 6 and 7 involved reporting and comparing over 1056 numerical values with each caching algorith + admission policy combination. We believe that reporting the distribution instead of mean/median/variance values provide a more complete picture. In Figures 8 and 9, we had comparisons of up to 106 numerical values per configuration and this prompted us to report entire distributions as well. Each violin plot reports the median and the two extremes as well as the shape (distribution) of all the values. 
3. The use of a timing simulator is outside the scope of this work since this work focuses on improving cache hit-rate and cache write-rate. How the cache hit-rate improvement would impact the ultimate performance perceived by a workload (e.g., latency/throughput) would depend greatly on the actual system deployment, including the performance of storage devices vs. cache devices, parallelism, memory, compute power, etc. at the I/O controllers, etc. There are other recent works such as that of Adjari et al. (which we cite in the paper) that extensively address the complexity of optimizing for latency/throughput in such non-datapath caches. 

For a major revision, I would recommend to revisit related work, and to disassociate with mARC and other replacement policies in general, at least in introduction etc., and improve most figures. The distinction for "non-datapath" and "general solution" is not consistent throughout the paper, and may limit the reach of the paper. I am not 100% sure this is possible within the journal time limits, but it will either uncover contribution issues, or it will greatly improve the paper.

The motivation section has been revised to include recent papers on caching. Our related work section addresses contributions including recent ones from allied areas such as caching for CDNs. 

The paper has been rewritten so that FOMO can be read more independently of mARC while also acknowledging that mARC’s design approach serves as an inspiration for the challenges involved in building an admission policy for non-datapath caches. 

The paper has been revised to use non-datapath and “general solution” more consistently throughout the text. 

 

Other

. Figures 3,4 too simplistic to be helpful/explanatory. It is just boxes with numbers (insert arrows etc.)

These figures now include additional annotations (bubbles and arrows) providing visual illustration of the concepts involved. 

. Making the cache simulators/experiments open-source would increase the contribution and confidence of the paper

We will make all the code used to generate data for the paper open source at the time of publication and include all pointers to relevant datasets via a public GitHub repository. 

 

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

The article is improved and can be published. I do not agree with the firm stance of the reply on the numerous violin plots, timing simulator being out of scope, incremental presentation etc., but I do not think these are deal breakers. Further disassociating from specific solutions could have more impact, but this is according to how much time you have available. The diff has a bug that double-prints the removed text (not related to the actual manuscript). The open sourcing would be a contribution, so it would be best to have the link in the next minor version, even if the repository is under construction. Figure 7 misses x labels. Consider moving text from figure captions as bubbles in the actual figures.

Author Response

The article is improved and can be published. I do not agree with the firm stance of the reply on the numerous violin plots, timing simulator being out of scope, incremental presentation etc., but I do not think these are deal breakers. Further disassociating from specific solutions could have more impact, but this is according to how much time you have available. The diff has a bug that double-prints the removed text (not related to the actual manuscript). The open sourcing would be a contribution, so it would be best to have the link in the next minor version, even if the repository is under construction. Figure 7 misses x labels. Consider moving text from figure captions as bubbles in the actual figures.

Thank you for the additional feedback. The X-axis label for Figure 7 is the same across all 15 plots and is specified now in the caption.  The open source FOMO repository information is now included in the draft and the repository will be updated with public code once the paper is published.