Cross-provider failover: the 429 always comes at launch

Rate limits have a sense of timing. Your product runs for months on a comfortable fraction of quota, and then the launch works — a feature lands on a big account, traffic multiplies by twenty over an afternoon, and your media provider starts answering with the only response that matters: 429 Too Many Requests. The one day you most need capacity is, by construction, the day you exceed yours.

Not all failures deserve a second chance

Before talking about failover, you have to split failures into two families, because doing the wrong thing with either family costs real money.

• Provider-side failures — rate_limited, provider_error, timeout, and the honest unknown. The request was fine; the machine answering it wasn't. These are exactly the failures another provider can absorb.
• Input-caused failures — malformed parameters, content-policy rejections, an image the model can't parse. The request itself is the problem. These must never fail over: the same input fails identically everywhere, so retrying elsewhere buys you the same error at double the price — or worse, on a metered provider, several billed attempts at generating nothing.

This classification has to be conservative and it has to live in code, not in a runbook. Every provider reports errors differently — one vendor's 400 is another's 422 is another's 200 with an error field — and mapping that mess into a stable taxonomy is most of the unglamorous work of multi-provider reliability.

Failover inside one company is half a failover

The aggregators know all this, and the good ones retry across their own fleet. What they cannot do, structurally, is fail over past themselves. If the aggregator's API is what's down — or its upstream capacity for a model class is what's saturated — every one of its internal fallbacks shares the same fate. You've concentrated your availability in one company's control plane and called it redundancy.

Real failover crosses a company boundary. Anything less shares a blast radius with the thing it's protecting you from.

The generative media market makes this unusually practical: the same capability — text-to-image, image-to-video — is served by genuinely independent companies with independent infrastructure, pricing, and failure modes. fal going down tells you nothing about whether Replicate is up. That independence is free reliability, if your routing layer can reach across it.

What honest cross-provider failover requires

Crossing the boundary is more than a second API key in the config:

• Requests named by capability, not vendor. A job that asks for text-to-image can be re-planned; a job hardcoded to one provider's endpoint and parameter dialect cannot. (This is the same property that saves you when a model retires.)
• Input re-normalization. The next candidate speaks a different schema — different parameter names, different ranges, different upload semantics. The router owns that translation; the caller shouldn't know it happened.
• A recorded plan. The ordered candidate list should be written down when the job is accepted, so "why did this run on provider B?" has an answer months later.
• The failover as evidence. Each failover should land in the job's ledger with its cause preserved — which provider failed, with what error, at what time. That record is how you renegotiate rate limits, catch a degrading vendor early, and explain the bill.

Through Wakemark, the whole story is one request:

curl -s https://api.wakemark.ai/v1/jobs \
  -H "Authorization: Bearer $WAKEMARK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "capability": "text-to-image",
    "input": { "prompt": "a lighthouse at dusk" }
  }'
# The candidate plan is recorded in the job's first ledger event.
# If the top candidate 429s mid-launch, the failover lands in the
# ledger (cause preserved) and the same job completes elsewhere.

The 429 will still come at launch. The difference is what your users see when it does: an error page, or a generation that took four seconds longer than usual because it quietly completed on a different company's GPUs.