The 100 Terabyte Problem
Cloudflare recently executed an astonishing memory heist, freeing 100 terabytes of operational capacity across its global network. This remarkable optimization is equivalent to adding the entire RAM capacity of 130 modern Gen 13 servers to their infrastructure, all without deploying a single new piece of hardware. Such a massive reclamation underscores the profound impact of low-level software efficiency.
Powering critical services like 1.1.1.1, Gateway DNS, and DNS Firewall, Cloudflare's Big Pineapple platform manages an enormous scale. At any given moment, this system stores over 250 billion DNS cache entries, each representing a crucial piece of internet routing information, facilitating billions of global requests daily. This immense scale transforms even minor inefficiencies into significant operational and cost challenges.
The core challenge for Cloudflare lay in the cumulative effect of seemingly tiny overheads. With 250 billion DNS entries, a single wasted byte per entry multiplies into hundreds of gigabytes of superfluous memory across the global fleet. This microscopic inefficiency, scaled across such a vast system, does not just consume valuable resources; it can also impact performance and increase operational costs, demanding a fundamental re-evaluation of data structures.
Five Cuts, One Massive Payoff
Cloudflare executed five precise Rust-level optimizations, each a surgical strike against memory bloat, culminating in a massive 100TB payoff. Foremost, engineers replaced Vec<T> and String with Box<[T]> and Box<str> for immutable cache entries. This eliminated pre-allocated capacity and unnecessary heap overhead, immediately freeing 64 bytes per entry and over 15 TB fleet-wide.
A second critical change involved storing DNS records as raw wire-format bytes, rather than fully parsed Rust structs. This streamlined processing, as the system now avoids redundant parsing and serialization. It also significantly improved cache locality, packing more relevant data into fewer cache lines.
Further layout ingenuity prevented memory waste. Cloudflare boxed large enum variants, such as NAPTR within the RecordData enum, to stop smaller, more common record types (like A and AAAA) from reserving excessive padding. This ensured memory allocation matched actual data size more closely.
Engineers also merged three separate DNS response record lists (answer, authority, additional) into a single, unified list. Two compact 2-byte u16 offsets now delineate the sections, replacing larger pointer/length pairs and saving 28 bytes per entry. This consolidation significantly tightened the data structure.
Finally, conditional owner storage represented the fifth cut. Cloudflare now stores the owner name only when it differs from the queried domain, inferring it from the cache key otherwise. Combined, these granular optimizations reduced the per-entry footprint by 56%, from 953 bytes to 420 bytes, transforming Cloudflare's memory landscape.
Faster by Getting Smaller
Shrinking data structures often implies a performance trade-off, but Cloudflare's optimizations delivered a rare double win. Cache insert throughput surged by an impressive 43%, climbing from 625,000 to 893,000 entries per second. Concurrently, lookup latency dropped significantly, improving by 19% from 828 ns to 670 ns.
This acceleration isn't magic; it's a direct consequence of memory density. By reducing the memory footprint per DNS cache entry by 56% (from 953 bytes to 420 bytes), Cloudflare's data structures became more compact. Smaller data occupies less space, allowing more cache entries to fit into the CPU's faster L1, L2, and L3 caches, enhancing CPU cache locality.
Think of it like stocking a pantry: a smaller package means you can fit more frequently used items on the easily accessible front shelves. Processors spend less time fetching data from slower main memory (RAM) and more time processing it directly from these high-speed caches, drastically cutting down on wait states.
Such a profound improvement in both resource consumption and operational speed is an engineering rarity, defying the typical compromises between memory usage and execution time. Cloudflare not only freed 100 terabytes of memory but also made their DNS resolver demonstrably faster. For a deep dive into the technical specifics, read Cloudflare's own account: How we saved 100 terabytes of memory by optimizing 1.1.1.1's DNS cache.
Enjoying this? Get one like it in your inbox each morning.
one email a day · unsubscribe in two clicks · no third-party tracking
The Ripple Effect of Efficiency
Cloudflare will not let this 100TB of freed memory sit idle. Instead, it directly reinvests this substantial capacity into its global network, specifically expanding the DNS cache for services like 1.1.1.1. This strategic allocation maximizes the impact of the memory recovery, turning a technical triumph into an immediate operational upgrade.
A larger DNS cache translates directly into tangible benefits for millions of users worldwide. Increased cache hit rates mean faster DNS responses, as Cloudflare's servers can resolve more queries from local memory rather than initiating upstream lookups. This directly enhances the responsiveness and resilience of the internet experience.
Cloudflare's achievement offers a crucial industry lesson: deep systems optimization remains a powerful lever for efficiency. As the demand from AI workloads drives server memory costs upward, optimizing existing infrastructure to reclaim 100 terabytes of memory without hardware changes is not merely an engineering feat; it's a strategic imperative. This meticulous Rust-level work underscores the enduring value of precision in systems programming.
Frequently Asked Questions
How much memory did Cloudflare actually save?
Cloudflare saved approximately 100 terabytes (TB) of RAM across its global server fleet by optimizing its 1.1.1.1 DNS cache, equivalent to the memory in 130 of its servers.
What programming language was used for the optimization?
The optimizations were implemented in Rust, which was chosen for its memory efficiency, performance, and low-level control over data structures.
Did saving memory slow down the DNS resolver?
No, counterintuitively, the optimizations also improved performance. Cache insert throughput increased by 43% and lookup latency dropped by 19%.
How was the memory saved?
The savings came from five main changes to how DNS data was stored, including eliminating unused memory capacity in vectors, merging data lists, and storing records more efficiently as raw bytes.
What will Cloudflare do with the saved 100TB of memory?
The freed memory will be reinvested to increase the size of the DNS cache. This is expected to improve cache hit rates, reducing upstream queries and making the 1.1.1.1 service even faster for users.