openai-python v3.40.0 adds prompt cache breakpoint TTLs, and that is the whole story

OpenAI shipped openai-python v3.40.0 on October 27, and the release notes are almost comically thin. One feature. One bug fix. A stack of dependency bumps. But the feature, tucked under the api scope in pull request #4188, is “add prompt cache breakpoint TTL options,” and that clause deserves more attention than the release it arrived in. It is a small knob on a developer SDK. It is also a tell about how OpenAI expects people to build with its models over the next year.

The rest of v3.40.0 is housekeeping. The CI fix in #4185 pins an explicit browser demo dependency. The chores log a “completed generated-code separation” (#4190), a griffe bump from 1.14.0 to 2.3.0 (#4069), and routine action-version bumps across actions/create-github-app-token (3.2.0 to 3.3.0), actions/download-artifact (8.0.1 to 8.0.2), actions/upload-artifact (7.0.1 to 7.0.2), github/codeql-action/analyze (4.38.2 to 4.39.0), and openai/codex-action (1.12 to 1.13). None of that changes what a developer can do. The TTL option does.

What a cache breakpoint TTL actually is

Prompt caching is the practice of letting a provider keep the processed state of a long, repeated prefix of a request so it does not have to be recomputed on every call. You pay less, and you wait less. OpenAI has exposed cache behavior to developers through its API for a while. What v3.40.0 adds is control over how long a cache breakpoint lives, expressed as a TTL option on the breakpoint itself.

Read that carefully. A breakpoint is a marker you place in a prompt to signal where the cacheable region ends. A TTL on that breakpoint means the developer decides, per request, how long the cached prefix should persist before the provider is free to evict it. That is a meaningful shift in who controls a cost-relevant parameter. Previously the cache lifetime was largely the provider’s call. Now the client can say, in effect, “hold this for me.”

The release notes do not specify the allowed TTL values, the default, or the pricing delta. Those details live in the API reference, not the changelog, and Tessera has not independently confirmed them. {/* TODO: confirm allowed TTL values, default lifetime, and any price differential from OpenAI’s API docs or the #4188 diff */} What we can say from the release itself is that the option exists and is exposed in the Python SDK.

Why this matters more than it looks

The economics of a long-running agent are dominated by repeated context. A coding assistant that re-reads the same repository scaffold on every turn. A support agent that carries the same policy document through a thousand conversations. A retrieval pipeline that prepends the same forty-page manual to every query. In each case the expensive part is the prefix, and the prefix is identical across calls. Caching it is the difference between a product that pencils out and one that does not.

Giving developers a TTL turns caching from a background optimization into a design decision. If you know your agent will fire twenty calls in the next ninety seconds, you want a short, aggressive TTL that keeps the prefix hot and then lets it go. If you are running a batch job that revisits the same context every few hours, you want a longer lifetime and you are willing to pay for the residency. The v3.40.0 change lets you express both without changing the rest of your request shape.

There is a second-order effect that matters for anyone building on top of OpenAI. Cache TTLs are a lever on unit economics, and unit economics are what determine whether an agent product has a margin. A startup selling an agent at a fixed subscription price lives or dies on the cost of the tokens behind it. Being able to tune cache residency per workload is the kind of control that lets a small team keep a thin margin instead of watching it evaporate under a spike in context length.

The generated-code separation is the quiet part

Buried in the chores is the more structurally interesting item: “record completed generated-code separation” (#4190), alongside “update generated-code verification and preserve SDK helpers” (#4186). OpenAI’s SDKs are largely generated from an API spec, with hand-written helpers layered on top. Separating generated code from hand-maintained code is unglamorous plumbing, but it is the plumbing that determines how fast the SDK can track the API.

If the generated surface is cleanly separated, a change like the TTL option can flow from the spec to the SDK with less human review and fewer regressions. The preserve SDK helpers clause in #4186 is the tell that the team is being careful about the boundary: generated code churns, hand-written ergonomics should not. For developers, the payoff is that new API capabilities show up in the Python client closer to when they show up in the API, rather than trailing by a release cycle.

That is the unspoken promise of a release like this one. The headline feature is small. The machinery underneath it is being rebuilt so that the next feature is not.

What this means for AI builders

The TTL option is a signal that OpenAI expects caching to be a first-class part of how applications are built, not an implementation detail. That has consequences for how you architect an agent. If your prompts are structured so the stable prefix is at the front and the variable part is at the back, you can exploit a breakpoint TTL. If your prompts interleave context and instructions, you cannot, and you will pay full freight on every call.

The practical advice is to look at your request shapes before you look at the new parameter. A cache breakpoint only helps if there is a clean, repeated region to mark. The teams that benefit most from v3.40.0 are the ones that already separate their system prompt, their retrieved context, and their user turn into distinct blocks. The teams that will be disappointed are the ones that concatenate everything into one string and hope the provider sorts it out.

We asked OpenAI for the TTL values, the default lifetime, and whether cache residency is priced differently from cache writes. {/* TODO: comment sought from OpenAI on TTL semantics and pricing */} The release notes do not say, and the changelog is not the place to find out. Until the API reference catches up, treat the TTL option as a capability you can wire up but not yet cost out.

The larger point stands regardless of the missing numbers. A provider that hands developers a TTL on cache breakpoints is a provider telling you where it thinks the workload is going: toward long, repeated, agentic context that has to be cheap to be viable. The model launches get the press. The knobs that make the models affordable are the ones that decide which products survive.

The next thing to watch is whether the TTL values and their pricing show up in the API reference before the next SDK minor, or whether developers are left to reverse-engineer the semantics from the client library. Either way, the breakpoint is now in the code, and the cost curve of a long-running agent just became something a developer can tune.