AI gateway in front of OpenAI

You already built the seam

There is a config value in your codebase that nobody has thought about since the first commit. The base URL. It has been sitting in your client constructor the whole time, treated as a constant because it has never once needed to be anything else. It is, in practice, an interception point for every model call your application will ever make, and you got it for free by using the SDK the normal way.

Point it at Ringside and every call now passes through something that can count, cap, retry and reroute before it reaches a provider. Your call sites do not change, because the wire format does not change. Chat, embeddings, Assistants and Threads behave as the OpenAI SDK expects them to. The migration people put off for a quarter is a string.

What failover does, and what it will not do

Pass model as an ordered array and Ringside walks the chain, OpenAI to Anthropic to Mistral, in the order you wrote. Escalation happens on 502, 503 and 504, on upstream_unavailable, and when a response fails schema validation. It deliberately does not happen on every 5xx. A bare 500 usually means your request was the problem, and retrying a bad request against a second provider just spends money twice to get the same answer.

Two limits worth knowing before you plan around this, because both are rejected at parse time rather than failing quietly at runtime. A model array with stream: true returns fallback_with_stream_unsupported, so streaming calls name a single model. Pairing a chain with an Idempotency-Key is refused for the same reason: a chain has more than one possible outcome, and an idempotency key promises exactly one.

The other two problems

• • The $800 night. Give each Customer a monthly_budget_usd and the cap is checked on the request path, so a runaway loop hits 402 customer_budget_exceeded instead of running until someone wakes up. Per-Customer RPM and TPM limits cap the rate as well as the total.
• • The renewal call. /v1/customers/:id/margin pairs what you billed against what the calls cost, grouped by day, model or customer. Add FC-Customer to your existing calls and the report starts filling in from that moment. It cannot backfill the year you already ran without it, which is the argument for adding the header on the same day you change the base URL.
• • You get told. The wallet.low, customer.budget_exceeded and run.failed webhooks push the events that matter to you rather than waiting for you to go looking.

One thing on cost. Pooled credits carry a per-model markup, so the catalog rate is what your wallet is charged and that margin is how the platform earns. You are trading a slightly higher per-token rate for the caps, failover and reporting, rather than paying a separate platform fee on top of provider prices.

Architecture

In code

# Before:
# client = OpenAI(api_key=OPENAI_KEY)

# After:
from openai import OpenAI
client = OpenAI(
    api_key=RINGSIDE_BEARER_KEY,
    base_url="https://api.fightclub.pro/v1",
)

# Every call site below is unchanged from what you already shipped.
resp = client.chat.completions.create(
    model="fc:openai/gpt-4o",     # or keep "gpt-4o", it resolves
    messages=messages,
    extra_headers={"FC-Customer": user.id},  # optional, but this is what
                                             # makes the reports per-user
)

# An ordered fallback chain. Non-streaming calls only.
resp = client.chat.completions.create(
    model=["fc:openai/gpt-4o", "fc:anthropic/claude-sonnet-4.6"],
    messages=messages,
)
# Escalates on 502, 503, 504, upstream_unavailable, or a response that
# fails schema validation. A plain 500 or a 400 does not escalate.
# Sending stream=true with an array is rejected at parse time
# (fallback_with_stream_unsupported), as is pairing it with
# an Idempotency-Key.

Cross-links