Skip to main content

Overview

Website Crates.io Version docs.rs

What is foyer?​

foyer, just as its slogan, is a hybrid cache library for the Rust programming language. 🦀

What is hybrid cache?​

A hybrid cache is a caching system that utilizes both memory and disk storage simultaneously.

hybrid cache

It is commonly used to extend the insufficient memory cache for the system uses Object Store Service (OSS, e.g. AWS S3) as its primary data storage1 to improve performance and reduce costs2.

Why we need a hybrid cache?​

More and more systems are using OSS as their primary data storage. OSS has many great features, such as low storage cost, high availability and durability, and almost unlimited scalability.

However, there are also downsides with OSS. For example, the latency is high and uncontrollable, and its price increases with each accesses. The downsides will be further amplified in a large working set because of more data exchange between cache and OSS.

exchange

With a hybrid cache, the ability to cache the working set can be extended from memory only to memory and disk. This can reduce data exchange between cache and OSS, thereby improving performance and reducing costs.

exchange hybrid cache

Highlights of foyer​

As a hybrid cache, foyer provides the following highlighted features:

  • Hybrid Cache: Seamlessly integrates both in-memory and disk cache for optimal performance and flexibility.
  • Plug-and-Play Algorithms: Empowers users with easily replaceable caching algorithms, ensuring adaptability to diverse use cases.
  • Fearless Concurrency: Built to handle high concurrency with robust thread-safe mechanisms, guaranteeing reliable performance under heavy loads.
  • Zero-Copy Abstraction: Leveraging Rust's robust type system, the in-memory cache in foyer achieves a better performance with zero-copy abstraction.
  • User-Friendly Interface: Offers a simple and intuitive API, making cache integration effortless and accessible for developers of all levels.
  • Out-of-the-Box Observability: Integrate popular observation systems such as Prometheus, Grafana, Opentelemetry, and Jaeger in just ONE line.

Why use foyer, when you have 'X'?​

Unfortunately, there is currently no general proposed hybrid cache library in the Rust community. If you have a need for hybrid cache, foyer would be your best choice.

CacheLib3 provides a Rust binding. However, it only provides limited interfaces. You still need to patch the C++ codebase if you have requirements such as logging, metrics, or tracing supports. Besides, foyer provides a better optimized storage engine implement over CacheLib. You can try both foyer and CacheLib to compare the benchmarks.

For the needs as an in-memory only cache, foyer also provides compatible interfaces and competitive performance. Benchmarks4 show that foyer outperforms moka5 and is only second to quick-cache6.

benchmarks

What's next?​

  • Learn how to use foyer in your project, goto Tutorial.
  • Learn how to solve various challenging situations with foyer, goto Topic.
  • Learn how other projects use foyer, goto Case Study.
  • Learn the design of foyer, goto Design.

Acknowledgement​

foyer draws inspiration from CacheLib3, a well-known cache library written in C++, and Caffeine7, a widely-used in-memory cache library in Java, among other projects like moka5, intrusive-rs8, etc.

Thank you for your efforts! 🥰

References​

  1. Systems using OSS as its primary data storage: RisingWave, Chroma Cloud, SlateDB, etc. ↩

  2. How streaming database RisingWave use foyer to improve performance and reduce costs: Case Study - RisingWave. ↩

  3. FaceBook/CacheLib. ↩ ↩2

  4. Benchmark with mokabench9: PR comments. ↩

  5. moka-rs/moka. ↩ ↩2

  6. arthurprs/quick-cache. ↩

  7. ben-manes/caffeine. ↩

  8. Amanieu/intrusive-rs. ↩

  9. moka-rs/mokabench. ↩