Cloudflare's Code Just Deleted 100TB

Cloudflare's global DNS infrastructure, powered by the aptly named Big Pineapple platform, operates at an astonishing scale. It constantly manages over 250 billion DNS cache entries, underpinning critical services like 1.1.1.1, Gateway DNS, and DNS Firewall for countless users worldwide.

Despite this immense operational footprint, Cloudflare engineers uncovered a pervasive, hidden inefficiency. A typical DNS cache entry, benchmarked at 953 bytes, contained a significant proportion of memory overhead dedicated not to DNS data itself, but to internal data structure bookkeeping. Rust's default Vec and String types, for example, store a pointer, a length, and a capacity field.

For static cached DNS records, which do not grow, this capacity field became an unnecessary burden, consuming precious memory. This seemingly minor inefficiency compounded dramatically across the vast network. Even a single wasted byte per entry on 250 billion records translates to over 250 gigabytes of superfluous RAM.

This extensive memory waste equated to the operational cost of entire servers, highlighting how small, overlooked details in data structure design can balloon into massive infrastructure expenses. Cloudflare's revelation underscored the critical importance of fine-grained memory management even in high-level languages like Rust.

Rust's data structures often prioritize flexibility. Standard types like Vec and String efficiently manage dynamic data by allocating more memory than immediately needed, reserving extra capacity for future growth. Each instance implicitly stores three pieces of information: a pointer to its data, its current length, and this pre-allocated capacity.

This strategy is highly effective for mutable data, but it represents pure waste for immutable records. Cloudflare's 250 billion DNS cache entries are fixed once stored; they never expand or contract. The additional capacity field, therefore, served no functional purpose within "Big Pineapple," yet consumed valuable system resources.

Recognizing this inefficiency, engineers replaced Vec and String with boxed slices (Box<[T]>) and boxed strings (Box<str>). These specialized Rust types allocate only the exact memory required for the data, eliminating the capacity field entirely. This seemingly small change saved 64 bytes per individual cache entry.

Across Cloudflare's vast global fleet, this optimization instantly reclaimed over 15 TB of RAM. It demonstrates how a deep understanding of data structure semantics, combined with the specific requirements of the application, can unlock immense efficiency gains, turning dormant memory into active, usable resources.

Cloudflare moved beyond individual pointers for DNS sections like answer, authority, and additional, embracing a strategy of tightly packed data. They consolidated these sections into a single contiguous memory block, then used tiny two-byte offsets to navigate precisely to each part. This eliminated pointer overhead, contributing significantly to the 56% reduction in cache entry size, dropping from 953 bytes to a mere 420 bytes per entry.

Further optimizations targeted redundancy and structural inefficiencies within the cache. Cloudflare stopped storing the owner name if it duplicated the query, reconstructing it from the cache key only when necessary. Addressing Rust enum sizing, which defaults to the largest variant, they implemented boxing for large, rare record types such as NAPTR. This ensured smaller, more common records like A and AAAA could occupy significantly less memory.

The final major change involved storing record data as raw, length-prefixed wire-format bytes, instead of fully parsed Rust structs. This bypasses the memory overhead of Rust's rich type system for data that doesn't require constant parsing, further shrinking the in-memory footprint across Big Pineapple's 250 billion cache entries. For a deeper dive into these ingenious memory-saving techniques, readers can consult How we saved 100 terabytes of memory by optimizing 1.1.1.1's DNS cache | Cloudflare Blog.

Meticulous data re-engineering culminated in a striking outcome: Cloudflare reduced its typical DNS cache entry size from 953 bytes to a mere 420 bytes. This represents a substantial 56% reduction in memory footprint per entry. Across Cloudflare's global fleet, which manages over 250 billion cache entries, this optimization reclaimed an astounding 100 terabytes of total memory without adding a single new server.

Crucially, these memory savings did not come at a performance cost; instead, operations significantly accelerated. Cache inserts became 43% faster, leaping from 625,000 to an impressive 893,000 entries per second. Similarly, lookups saw a 19% speed improvement, with latency dropping from 828 nanoseconds to a rapid 670 nanoseconds. This rare win-win scenario delivered both significant resource efficiency and enhanced operational speed, defying conventional tradeoffs.

Cloudflare will strategically deploy the freed 100 terabytes of memory to expand its DNS cache capacity. A larger, more robust cache enables the "Big Pineapple" platform to resolve an even greater proportion of queries locally. This directly reduces the need to fetch data from upstream servers, translating into faster DNS resolution for users worldwide and visibly improving overall internet speed for everyone.

How much memory did Cloudflare save with this optimization?

Cloudflare reclaimed approximately 100 terabytes (TB) of RAM across its global fleet, equivalent to the memory of about 130 servers, without adding new hardware.

Did optimizing for memory make Cloudflare's DNS slower?

No, the optimizations had the opposite effect. Cache inserts became 43% faster, and lookups became 19% faster, disproving the common tradeoff between memory usage and speed.

What programming language did Cloudflare use to achieve this?

Cloudflare used the Rust programming language, taking advantage of its features for fine-grained memory control to rewrite how DNS cache entries are structured.

What was the main change Cloudflare made to save memory?

A key change was replacing standard Rust data types like Vec and String with Box<[T]> and Box<str>. This eliminated an unnecessary 'capacity' field for cached data that never changes size, saving 64 bytes per entry.

How did Cloudflare reduce the DNS cache entry size by 56%?

They implemented five key Rust-level changes: eliminating capacity fields, packing data sections into a single block with offsets, removing redundant information, boxing large enum variants, and storing data in its raw wire format.