When Two Providers Fail the Same Way, Check Your Own Gateway
Five of six subagents stopped on what looked like a provider's rate limit. When it happened again, the logs showed upstream was fine and my own gateway was refusing new requests while long-running streams held every slot. The fix was to make bursts wait instead of fail.

Within two minutes of starting, five of my six subagents had stopped with the same message: "You hit a rate limit and stopped." The next fan-out failed the same way, and that one I could trace. The limit wasn't a provider's. It was mine.
I run my Capy threads through my own gateway in front of the models I use. It gives me one place for routing, limits and logs. It also means that when something says no, I have to check whether it was me.
Five of six
I'd started an orchestrator thread with a wide fan-out. It launched six analysis subagents on Opus 5.5, and five of them stopped almost immediately.
The orchestrator moved them to another model and kept going. Its report blamed Anthropic rate limits.
I pushed back. They shouldn't have been hitting Anthropic rate limits. The orchestrator's next explanation was plausible: Anthropic and OpenAI were both returning rate-limit errors, it said, probably from too many long-context runs at once.
That explanation should have been the clue. When two different providers reject the same burst at the same moment, the problem is usually something in front of both of them.
The gateway's own logs
When the next fan-out hit the same wall, the gateway's own logs had the answer.
This wasn't the first time my layer had spoken in a provider's voice. Twice in the week before, a refusal from the gateway had reached the client as a provider error: once for a request just over its size limit, and once for an upload that timed out and came back as invalid JSON. After the first of those, the gateway started logging a fixed reason whenever it refuses a request.
That paid off. In a 21-minute window around the second fan-out, the gateway had refused 24 requests because its concurrency cap was full. All 170 upstream calls that completed in the same window returned HTTP 200.
Why a small cap failed
The cap was a small global limit on requests in flight, across every provider and every thread. Agent traffic broke it in three ways.
Agent requests are long-lived streams, and some held a slot for well over a minute. The cap was shared, so the orchestrator itself and unrelated conversations held slots the subagents needed. And when a request was refused, the client retried every few seconds and ran out of retries long before a slot came free.
Put together, a burst of long requests against a small shared pool with fast client retries doesn't degrade gracefully. It fails, and it fails for whatever arrives while long streams hold the slots.
Wait instead of fail
The problem was what happened at the limit: a full gateway said no instantly, and an instant no to an agent is a failed task.
So I had my coding agent replace rejection with admission control. The gateway now runs twice as many requests at once, and the rest wait in a bounded first-in, first-out queue for a limited time. The per-minute request budget makes requests wait the same way instead of refusing them. The details:
- Requests wait before their bodies are parsed, so the queue holds connections, not large payloads.
- A waiting request that disconnects, is cancelled or times out gives its place back, including the race where it's cancelled just as a slot is granted.
- If the queue is full, or a request waits out its time, it still gets a 429 with a Retry-After header, and provider errors pass through unchanged. The client will still label that 429 as the provider's, which is outside the gateway's control. What changed is that a refusal now means the request already waited its turn, and the gateway's logs say which side refused.
- The logs now distinguish queued, admitted, released and cancelled requests, with how long each one waited.
The tests cover a streaming burst of subagents through the slots, FIFO fairness, queue bounds and timeouts, disconnected clients, and provider 429s passing through untouched. A live burst of simultaneous Opus 5.5 streams came back clean.
A queue doesn't make capacity infinite. A burst big enough to keep every slot busy for the whole wait still gets refused, and one global queue means a large fan-out can delay everything behind it.
The rollout couldn't drop live streams either. A temporary copy of the fixed gateway took new requests while the old process finished its in-flight streams, and then the normal service restarted with the fix. Shutdowns now give active requests time to finish. The second fan-out's subagents all finished.
Your own layer speaks in other voices
Any layer you put between agents and providers will, sooner or later, say no in someone else's voice. The agents can't tell the difference, and an orchestrator will give you a reasonable story about why the provider did it.
Fan-out is where this shows up first. Cursor's Projects delegates tasks to "thousands of subagents." Claude Code Projects runs threads in parallel. A limit that never mattered for one interactive session becomes a wall within seconds when an orchestrator starts six.
The trade-off is latency. A burst that used to fail in a second can now wait in line before it starts. A subagent that starts late still finishes. One that's refused doesn't. Provider rate limits are real, and they still apply. But when two providers fail the same way at the same moment, the shared layer in front of them is the first place to look.
