Opens in a new tab
Explainer

Anthropic Scion puts source control behind a cache hierarchy

In brief

An agent can reuse a nearby copy of a source file instead of fetching it again. Scion speeds that path; a fleet starting without its saved copies asks more of the shared service.

5 min read

Is this the right starting point?
Before you read
Experience with Git/CI jobs and basic distributed-service capacity and failure testing.
What you will take away
Evaluate source-control cache benefits against cold-start, regional-failure and write-reconciliation behavior.
Sources
A large paper code archive delivers only a few selected sheets through three coral channels to small blue task containers.
On this page4 sections

Coding agents can prepare changes faster than the machinery around them can check and merge them. Anthropic's October 13, 2026 account puts numbers on that pressure: daily merges grew from about 2,500 in May to more than 30,000 in September, and the company moved its monorepo to an internal system called Scion.

That moves part of the waiting into source control. Scion makes it a shared online service, with caches along the route to files an agent needs. The operational question is how that route behaves when many jobs ask together, especially when their nearby copies are empty.

A job can ask for less of the repository

Imagine an agent needing one deployment configuration file. Scion can fetch that file without making a complete working copy of unrelated directories. This selective copy is a sparse checkout; a virtualized checkout goes further by producing a file only when it is needed. A CI job asking only what changed can use an API and skip the working copy altogether. Commits, merges, rebases and history queries run on Scion’s servers, while the client keeps a limited local cache.

The configuration file may already be on the agent’s machine from an earlier job. A background process, the node daemon, keeps those shared copies. If its copy is missing, it asks a cache in the same region. If that cache misses too, the request reaches the authoritative Scion servers, which can read Spanner or cloud storage. The same file takes either a short route nearby or a longer route through shared services.

Anthropic says it keeps requests within those designated layers. Otherwise losing the regional copy service could send every waiting job straight to the authoritative servers and overwhelm them. That explains the design’s purpose; it is not an independently demonstrated recovery result. The engineering account describes the arrangement.

Conceptual Scion request path through a node cache, regional cache and authoritative server. A failed regional cache is shown as a containment boundary, not permission for clients to bypass it.
Requests move through designated tiers. The failure annotation describes the reason for containment, not a measured outage or a documented failover procedure.

A second hypothetical agent needs only a patch, while the first needs the configuration and some deployment history. Giving both the full repository does the same preparation for two different jobs. Selective retrieval lets each ask for the files its investigation actually uses.

A quick checkout can mean less requested data or more reuse of data already present. Record requested objects and bytes, local reuse and time to the first useful file. A newly started fleet may have fewer nearby copies, so record its starting cache state rather than assuming yesterday’s reuse.

The warm path and the recovery path are different tests

Warm-cache results describe the tested setup, not automatically its recovery after caches disappear. Compare representative reads and a cold start before treating that speed as a capacity promise.

Take an illustrative cache handling 10,000 requests per second. A 99% hit rate sends 100 upstream; 90% sends 1,000; no hits sends all 10,000. These are example calculations, not Scion measurements. They describe request counts, not bandwidth or CPU multipliers, which also depend on sizes and costs.

Dropping just nine percentage points in hit rate multiplied misses tenfold. A restart wave can therefore change the shared tiers' job dramatically: each empty cache requests work it previously absorbed. The warm path and recovery path need separate examination.

Proposed experimentWhat to observeWhat the result answers
A new agent node starts with an empty cache.Time to first useful file, fetched bytes, upstream concurrency, and completed tasks.Whether the workflow can recover without assuming a warm node.
Many nodes start together.Aggregate misses, queued requests, retry counts, and tasks that cannot start.Whether ordinary per-node demand becomes a fleet-wide bottleneck.
A regional cache is unavailable.Which requests fail or wait, which permitted routes remain, and whether upstream load is contained.Whether the failure stays regional rather than moving unchecked to shared services.
A source-control write loses its response.The destination's changeset or operation record before another submission.Whether the write was accepted, rejected, or remains unresolved.

End the experiment with the right input in the agent's hands. For the incident example, compare the retrieved configuration against its known revision and inspect the report's citation. A cache helps only if it supplies the version this task needs.

Treat writes separately. A timeout after a server-side commit does not reveal whether it took effect. This article establishes no Scion retry or idempotency contract; obtain one before automating resubmission. The write-reconciliation guide follows that lost-response distinction.

What a smaller team can borrow

Scion is an internal service for Anthropic's scale, not a generally available replacement for Git. Its results do not show that a smaller repository needs the same migration.

First locate repeated setup work. If jobs fetch files they never read, reduce the inputs. Git's shallow, sparse and filtered clone options address different costs: history, working-tree files and deferred objects. Check the history and files still needed for blame, merge-base calculations and reproducible builds.

If tests dominate and checkout is already small, redesigning source control will not shorten the tests. An agent requesting the wrong files also remains unhelpful through a fast API. Measure progress toward the useful task so omitted necessary work cannot masquerade as speed.

For a smaller team, begin with the wait before the first relevant file. Trimming history or materialized files may resolve it. If shared caching is worthwhile, also test the morning when machines start empty. That morning and the warm afternoon belong to the same development service.

Sources and scope

This is documentation-based analysis of Anthropic's October 13 engineering report, discovered through the ClaudeAI discussion. The cache calculation and experiments are original illustrative teaching examples, not executed Scion tests. Git option descriptions were checked against its official clone reference. Scion's reported performance and availability remain Anthropic's claims.

Source context

Source links appear within this article. Read them alongside the author’s analysis and evaluate the guidance against your environment.

Report an error or outdated detail

More on Anthropic

All Anthropic coverage

Keep reading

Where to go next