Error handling and resilience

The mental model

Every Ringside call can fail for one of two reasons. Either something about your request is wrong and will be wrong every time you send it, or the platform (or an upstream provider) hit a transient condition that will likely clear on its own. The whole job of a resilient client is to tell those two apart and react differently. Stop and fix the first. Retry the second.

Ringside makes that classification deterministic. Every error response carries a stable, lowercase code string. That code, not the HTTP status alone and never the prose message, is the value you switch on.

The error envelope

Every non-2xx response from https://api.fightclub.pro/v1 returns the same JSON shape. This is the OpenAI-compatible envelope, so an existing OpenAI client deserializes it unchanged.

{
  "error": {
    "type": "rate_limit_error",
    "code": "rate_limit_exceeded",
    "message": "Rate limit reached for this customer.",
    "retry_after_seconds": 2
  }
}

type

Always present. The coarse class of failure, one of: authentication_error, invalid_request_error, permission_error, rate_limit_error, conflict_error, not_found_error, wallet_error, internal_error, api_error.

code

Always present. The stable, fine-grained identifier you branch on (for example customer_budget_exceeded, invalid_model_ref, thread_locked).

message

Always present. Human-readable. For logs and support tickets only. Never parse it.

param

Present only on field-validation errors. Names the offending field so you can point the user at the right input.

retry_after_seconds

Present only on rate-limit errors. The number of seconds to wait before retrying. Mirrors the Retry-After response header.

Retryable versus terminal

The table below is the core of any retry policy. Retryable errors are safe to send again after a backoff. Terminal errors mean the request is malformed, unauthorized or out of money, and retrying the identical request just burns latency and gets the same answer back.

Retrying writes safely with idempotency

Retrying a read is free. Retrying a write is dangerous. If the first attempt actually succeeded but the response got lost on the wire, a naive retry double-charges the wallet and double-creates the resource. The fix is an idempotency key.

Send the idempotency-key header (exactly that, lowercase) on any write you might retry. Generate one UUID per logical operation and reuse it across every retry of that operation. Ringside stores the first successful response and replays it byte-for-byte on any repeat within the replay window, so the retry returns the original result without re-executing the work.

Header

idempotency-key, lowercase. Same name on every endpoint that accepts one.

Replay window

24 hours. After that the key is forgotten and a request with it executes fresh.

Scope

Keyed on (developer, customer, key). The customer dimension is the attribution target, so two different customers using the same key string under one developer never read each other's cached response.

What is cached

Only successful (2xx) responses. A 4xx or 5xx is not stored, so a retry after a failure runs for real rather than replaying the error.

Provider resilience with a model cascade

For chat completions, the model field accepts either a single ref or an array. An array is a cost-ordered cascade: Ringside tries the first ref, and on a retryable upstream failure it falls through to the next, and so on, charging you only for the call that actually succeeded.

{
  "model": [
    "fc:openai/gpt-4o-mini",
    "match:anthropic/sonnet>=4",
    "fc:mistral/mistral-large"
  ],
  "messages": [{ "role": "user", "content": "Summarize this contract." }]
}

When a cascade fires, the response headers tell you exactly what happened. X-Ringside-Model-Used is the ref that served the request, X-Ringside-Models-Tried lists the refs attempted in order and X-Ringside-Fallback-Triggered is set when the first choice was skipped. Log these so a quiet degradation (your primary model being down for an hour) is visible rather than invisible.

Backoff with jitter

When you do retry, do not retry immediately and do not retry on a fixed interval. A fixed interval synchronizes every failing client into a thundering herd that hits the endpoint at the same instant the moment it recovers. Use exponential backoff with full jitter: each attempt waits a random duration between zero and a ceiling that doubles per attempt.

There is one exception that overrides the formula. On a 429, Ringside has told you the exact wait via Retry-After (integer seconds) and retry_after_seconds. Honor that value rather than your own backoff curve, because the platform knows when the window resets and your guess does not.

• ·Cap total attempts (3 to 5 is sane). An infinite retry loop turns a brief outage into a self-inflicted one.
• ·Cap the per-attempt ceiling (for example 20s) so the backoff does not grow without bound.
• ·Always honor Retry-After on a 429 in preference to your computed delay.
• ·Only retry the retryable classes from the table above. Never retry a 400, 401, 402, 403 or 404.

Surface the request id everywhere

Every response carries an X-Request-Id header. If the client sends its own X-Request-Id (matching ^[a-zA-Z0-9_-]{1,64}$) Ringside threads that value straight through, so you can join your distributed traces to ours. If you do not send one, the platform mints a req_<hex> id and returns it.

Put this id in two places. First, in every structured log line for the call, success or failure, so you can grep your own logs by it. Second, in any support ticket you open. With the id we can pull the exact request server-side in seconds; without it we are guessing.

level=error msg="ringside chat failed" req_id=req_a1b2c3d4e5f6a7b8 status=503 error_code=upstream_unavailable customer=cus_42 model_tried="fc:openai/gpt-4o-mini" attempt=2

A handler that puts it together

The two examples below implement the full policy: classify on error.code, honor Retry-After on a 429, back off with jitter on 5xx, never retry a terminal error and reuse one idempotency key across retries of the same write. Note that the cascade and the idempotency key are not combined, per the restriction above.

import os, time, random, uuid, requests

BASE = "https://api.fightclub.pro/v1"
KEY = os.environ["RINGSIDE_KEY"]  # ko_<64 hex>

TERMINAL = {400, 401, 402, 403, 404}

def chat(messages, customer_id, max_attempts=4):
    idem = str(uuid.uuid4())  # one key, reused across retries
    headers = {
        "Authorization": f"Bearer {KEY}",
        "FC-Customer": customer_id,        # e.g. "cus_42" or "ext:acme-1"
        "idempotency-key": idem,
    }
    body = {"model": "fc:openai/gpt-4o-mini", "messages": messages}

    for attempt in range(1, max_attempts + 1):
        resp = requests.post(f"{BASE}/chat/completions", json=body, headers=headers)
        req_id = resp.headers.get("X-Request-Id")

        if resp.ok:
            return resp.json()

        err = resp.json().get("error", {})
        code = err.get("code")

        if resp.status_code in TERMINAL:
            raise RuntimeError(f"terminal {code} (req {req_id}): {err.get('message')}")

        if resp.status_code == 429:
            wait = err.get("retry_after_seconds") or int(resp.headers.get("Retry-After", "1"))
        else:  # 5xx: exponential backoff with full jitter, capped at 20s
            wait = random.uniform(0, min(20, 2 ** attempt))

        print(f"retryable {code} (req {req_id}), attempt {attempt}, sleeping {wait:.1f}s")
        time.sleep(wait)

    raise RuntimeError(f"exhausted {max_attempts} attempts")

Edge cases worth handling

409 thread_locked

Only one active run per thread. If you get this, the previous run has not finished. Poll its status to completion (or cancel it) before starting another, rather than blind-retrying.

402 customer_budget_exceeded

The customer crossed monthly_budget_usd. This is a business event, not a transient one. Route it to your billing flow or raise the budget, do not retry.

400 with a param

Field validation. The param field names the input that failed. Surface it to whoever supplied that field instead of retrying the same payload.

Lost response on a write

If a POST times out with no response, you do not know whether it landed. This is exactly what the idempotency key protects: retry with the same key and you get the original result if it landed, or a fresh execution if it did not.

Stale or unknown model ref

400 invalid_model_ref means the ref lacks a valid prefix (fc:, match:, slot: or dyn:) or carries an unsupported @<region> suffix. Fix the ref. It is terminal.

Defaults and limits to keep in mind

• ·Idempotency replay window: 24 hours, scoped to (developer, customer, key).
• ·Rate-limit headers: X-RateLimit-Remaining on every response, Retry-After (integer seconds) only on a 429. There is no X-RateLimit-Limit or X-RateLimit-Reset.
• ·Four rate-limit layers enforce independently (edge per-IP, per-client-token, per-developer, per-customer); the lowest wins, so a clean dev quota does not guarantee a clean customer quota.
• ·Chat request body serializes to at most 100,000 bytes; oversized payloads are a terminal 400, not a retry.
• ·A model:[] cascade is the only built-in cross-provider failover; with a single model string you own failover yourself.

Get the classification right once, wrap it in a helper and every call site inherits the same behavior. The platform gives you stable codes, an explicit Retry-After, an idempotency primitive and a request id on every response. Wire those four into your client and transient failures become a non-event.