Zero-day retention, and what it cost us to build

The feature that is mostly deletion

Zero day retention means that once your response is returned, we keep nothing. No prompt, no completion, no files. Traces retain metadata only: which model answered, token counts, latency, cost, and the eval score if one ran. It is available on Pro and above, and it took nine months.

Nine months is an absurd number for a feature whose entire description is to do less. Here is where the time went.

Storage is not one place

When we started, request content existed in more places than anyone could list from memory. It was in the trace record, which is the obvious one. It was in the queue that fed the trace writer. It was in the replay buffer that makes streaming failover possible. It was in the error reporter, because a stack trace from a schema validation failure carried the payload that failed validation. It was in two caches. It was in the logs of three services that logged request bodies at debug level, and debug level was on in one region because of an incident that nobody had turned off.

The first six weeks were an inventory. We wrote a test that submits a request containing a unique string and then searches every system we operate for that string. We ran it weekly. It kept finding new places for two months.

The parts that were hard

Replay was the hardest. Streaming failover works by holding what has been sent so far, so that when a provider dies mid stream we can resume against a second model without the client seeing a partial answer twice. That buffer is request content by definition. Under zero day retention it now lives only in memory, is scoped to the call, and is dropped when the call ends, which meant rewriting the failover path so that nothing outside that scope could hold a reference.

Error reporting was next. We strip payloads before they leave the process, which is straightforward, and we lost the ability to reproduce a schema failure from the report, which is not. The trade is real. We took it because a customer cannot audit our good intentions.

Backups were the slowest. A backup that contains content you promised to drop is still content you kept, so the promise has to include a horizon. Ours is 35 days, which is how long a backup lives before it rolls off. We say so on the Trust Center, because the alternative is to say deleted and mean deleted eventually.

The switch is not a setting

We shipped zero day retention as a per project setting with a per route override, and the override is the part people use. Most teams do not want nothing kept everywhere. They want nothing kept on the two routes that touch regulated data, and thirty days everywhere else, because thirty days is what makes debugging possible.

That shape surprised us. We built it as an all or nothing product decision and customers immediately used it as a per route compliance tool. The Enterprise version goes further: retention is set per project, down to zero, and pinned to a single region along with the processing and the backups.

What we gave up

Support got harder. When a customer on zero day retention says a response was wrong, we cannot look at the response. We can see that the call happened, which model answered, how many tokens it produced, and what the eval said. Most of the time that is enough to find the problem. When it is not, we ask the customer to reproduce it with retention on for one route, which is a worse experience than seeing it from here.

Our own debugging got harder in the same way, and it improved the product. Every tool we built to work without content, the metadata views, the eval score history, the per route latency and cost breakdowns, is a tool that works for every customer now, including the ones who never turn retention off.

What it cost, precisely

Nine months of one to three engineers, most of a rewrite of the trace pipeline, one permanent reduction in what our support team can see, and a 35 day qualifier on the word deleted.

What it bought is a shorter conversation with a legal team. Tessera moved from a seven month estimate to six weeks, and the part of that which belongs to this feature is the part where nobody had to negotiate about what we keep, because the answer is nothing.

If you are building this

Start with the inventory test, not the design. Write the thing that searches every system for a known string and run it weekly from the first day. Every design you write before that test exists will be wrong, because it will be based on where you believe content lives.

Then decide your backup horizon before you write any marketing copy, because that number is the only part of the promise a customer can check.

What we tell auditors

The question an auditor asks is not whether you delete content. It is how you know. Our answer is the inventory test, run weekly, with its results retained: a unique string goes in through the public API, and a job searches every store, queue, index, log, and backup we operate for that string twenty four hours later. The test either finds nothing or it opens a ticket.

That test is the only reason we are comfortable with the claim. A design document says where content is supposed to live. A weekly search says where it does. The two disagreed four times in the first year, always because a new service logged something a developer thought was metadata.

We also publish the backup horizon rather than burying it, because 35 days is the honest qualifier on the word deleted and a security team will find it anyway. Finding it in our own documentation is a very different conversation from finding it in a questionnaire response.

The metadata line

Zero day retention keeps metadata: model, token counts, latency, cost, eval score, and whatever you attach as metadata yourself. That last part is the one to watch. If you attach a user email as trace metadata, we keep the user email, because you told us to and we cannot tell it apart from a deploy identifier.

We considered scanning metadata for things that look like personal data and refusing them. We decided against it: a scanner that is wrong in either direction is worse than a clear rule, and the clear rule is that metadata is yours and we keep exactly what you send. It is documented in three places for that reason.

Would we do it again

Yes, and sooner. The nine months were not spent building the feature. They were spent discovering how many systems had quietly become content stores, and that discovery would have cost less at a tenth of the scale.

The advice to an earlier version of us: decide on day one which systems are allowed to hold request content, write the test that enforces it, and run the test from the first week. Not because you will ship zero day retention soon, but because when a customer asks for it you will be answering a question about configuration rather than starting an archaeology project.