Codex Broke Git. OpenAI Rebuilt the Engine

Coding agents are fundamentally shifting the cadence of development. Where human developers might push a handful of commits daily, AI agents can generate bursts of branch updates and CI activity that far exceed what traditional Git infrastructure was built to handle. OpenAI experienced this firsthand, seeing commits to its main branch surge by 1,378% by late 2025, with projections of 2,400% by year-end.

This sheer volume of concurrent operations creates a bottleneck. Unrelated branch updates compete for shared server resources, transforming concurrency into debilitating contention. At OpenAI, this dynamic caused monorepo reliability to plummet below 90%, sometimes hitting 87.1% due to timeouts and reference conflicts.

The problem isn't the individual code changes, but the system's inability to process them in parallel. If hundreds of agents push changes to different branches simultaneously, all those pushes can stall, even if the changes are entirely independent. This cascades into stalled pushes and unreliable source control, slowing every agent and human developer in the loop.

This operational friction highlights a critical scaling challenge. Git servers couldn't't keep up with Codex's demands, demonstrating that as agentic development matures, the underlying infrastructure must evolve to match its unprecedented velocity and throughput.

OpenAI’s solution, Pando, is a new backend engine for Git, not a new version control system. Engineers and Codex agents continue to use familiar git push and git pull commands. This backward compatibility was crucial, minimizing disruption to existing developer workflows, CI pipelines, and extensive tooling while radically changing the underlying infrastructure.

Key to Pando’s design is its handling of write paths. Unlike traditional Git servers that often serialize updates or lock the entire repository, Pando processes updates to separate branches independently. This means a push to branch A no longer waits for unrelated pushes to branch B, eliminating shared bottlenecks that previously degraded monorepo reliability to below 90%.

Pando also aggressively caches Git packfiles, which are compressed bundles of Git objects. When dozens of CI jobs request the same base code, the server doesn't regenerate identical data repeatedly. This caching achieves over a 99% hit rate, drastically reducing compute overhead and saturating multi-gigabit network links, accommodating peaks exceeding 57 terabytes of Git packfiles per hour.

This architectural shift delivered significant performance gains. Median push latency dropped by nearly 80%, representing a roughly 5x speedup. Pando now serves almost 1 petabyte of Git data daily, a testament to its scalability. Designed primarily in Rust by a single engineer, with significant implementation by Codex agents, Pando went from kickoff to production in just 10 weeks.

Core to Pando's performance is a sophisticated caching strategy for Git packfiles. These compressed bundles efficiently package Git objects—like commits, trees, and blobs—for transfer between repositories and clients. In a high-throughput environment, repeated CI requests for the same code base would ordinarily force Git servers to continually re-generate identical packfiles, consuming significant compute cycles and time.

Pando addresses this by aggressively caching these packfiles. When dozens of CI jobs request the same code, the system reuses pre-generated packfiles instead of recalculating them. This dramatically reduces server computation, serving data much more efficiently and at network saturation speeds.

This caching, combined with Pando's decoupled branch updates, has yielded impressive results. OpenAI reports median push latency fell by nearly 80%, a fivefold speedup that transformed their development experience. Pando now serves close to a petabyte of Git data per day, accommodating peak demands exceeding 57 terabytes of packfiles per hour. For further technical details, explore Scaling Git at OpenAI.

Faster writes come with a consistency bill. Pando achieves its impressive throughput by making a crucial architectural trade-off: reads across multiple branches are not guaranteed to reflect one perfectly synchronized, repository-wide snapshot. While individual branch histories remain consistent, the system prioritizes immediate write availability over atomic multi-branch read consistency.

This compromise works for OpenAI’s agent-driven development, where hundreds of parallel tasks frequently push changes to isolated branches, and the immediate state of a single branch is often more critical than a perfectly synchronized, global view. However, systems expecting atomic, repository-wide reads across disparate branches would need to account for this eventual consistency model.

Pando’s rapid development — designed by one engineer, implemented largely by Codex agents, and moving from kickoff to production in just 10 weeks — underscores a broader lesson. Agent-scale development is pushing developer infrastructure choices into the spotlight, demanding a fundamental rethinking of tools that were designed for human-centric workflows. Pando's ability to serve almost a petabyte of Git data every day, while cutting median push latency by nearly 80%, demonstrates both the opportunity and the pressure to innovate.

What is Pando?

Pando is OpenAI’s proprietary Git backend, built to handle heavy concurrent reads and writes while preserving familiar Git commands and workflows.

Does Pando replace Git?

No. Pando replaces the backend serving Git data; developers and agents can continue using standard commands such as git push and git fetch.

How did Pando improve Git performance?

It processes updates to different branches independently and caches Git packfiles, reducing contention and repeated work for CI requests.

What trade-off does Pando make?

Reads spanning multiple branches may not show one perfectly synchronized repository-wide view, trading that guarantee for more available, concurrent writes.