The tragedy of unbounded retry
Optimistic clients retry on 429, then retry harder. One runaway job becomes a denial-of-service against your own community.
If docs do not publish the window, operators invent the worst one.
Design the budget in the open
Show remaining budget. Cap concurrency. Prefer queueing over silent failure.
Inside the lab: fuse first, cleverness second.
Field steps
- Publish soft and hard limits where humans can find them.
- Add client-side jittered backoff by default.
- Cap concurrent tool calls per run.
- Alert on spend spikes before the card declines.
Long view
As AI products commoditize, reliability storytelling beats feature lists. Budgets are how trust shows up in the API.