Batch processing

What batch processing is

A batch is a single object (batch_...) that wraps thousands of independent requests against one endpoint. Instead of opening one HTTP connection per request and waiting on each round trip, you hand Ringside a file of requests up front and walk away. The platform queues the work, runs the rows on its own dispatch loop and writes the results to a downloadable output file. Nothing streams back to you in real time.

The trade you are making is latency for cost and throughput. A batch can take minutes or hours to drain. In exchange every billed row inside a qualifying batch is charged at half the synchronous markup, and you stop managing connection pools, retries and rate-limit backoff by hand for bulk jobs.

When to use batch vs sync

Batch supports exactly three endpoints: chat completions, embeddings and moderations. Each line in the input file targets the one endpoint you declared on the batch. Moderation rows are free (they meter at $0), so the discount multiplier is a billing no-op there, but you can still batch them to avoid managing the request loop yourself.

The lifecycle

A batch moves through an eight-state machine. You drive only two of the transitions (create and, optionally, cancel); the background worker handles the rest.

validating --> in_progress --> finalizing --> completed
     |              |                          
     |              +--> (expires_at < now) --> expired
     |              |
     +--------------+--> cancelling --> cancelled
     |
     +--> failed   (bad input file: empty, oversized, malformed JSONL)

validating

Initial state. The worker is parsing the input JSONL and checking the row count and shape. A bad input file moves the batch to failed.

in_progress

Validation passed. request_counts.total is now set and the worker is dispatching child requests in windows. This is where most of the wall-clock time goes.

finalizing

Every row has been processed. The worker is assembling the canonical output and error files before flipping to completed.

completed

Terminal success. output_file_id (and error_file_id, if any rows failed) are populated and downloadable.

failed

Terminal. The input file was rejected as a whole: empty, over the 50,000-row cap or containing any malformed JSONL line. The reason is written to error_file_id.

cancelling

You called cancel while the batch was validating or in_progress. The worker is flushing whatever child output already ran.

cancelled

Terminal. Partial output from rows that completed before cancel is preserved in output_file_id.

expired

Terminal. The 24-hour completion window elapsed before all rows finished. Any partial output is preserved; errors carries batch_expired.

Building the input file

The input is a JSONL file: one JSON object per line, newline-delimited. Each line is a self-contained request with a custom_id you choose, the HTTP method, the target endpoint URL and the request body you would otherwise POST synchronously.

{"custom_id":"row-1","method":"POST","url":"/v1/chat/completions","body":{"model":"fc:openai/gpt-4o-mini","messages":[{"role":"user","content":"Classify the sentiment of: shipping was slow but the product is great"}]}}
{"custom_id":"row-2","method":"POST","url":"/v1/chat/completions","body":{"model":"fc:openai/gpt-4o-mini","messages":[{"role":"user","content":"Classify the sentiment of: absolutely love it"}]}}
{"custom_id":"row-3","method":"POST","url":"/v1/chat/completions","body":{"model":"fc:openai/gpt-4o-mini","messages":[{"role":"user","content":"Classify the sentiment of: never buying from here again"}]}}

• ·A batch may carry at most 50,000 rows. An input file over that cap fails the whole batch with too_many_requests.
• ·An input file with zero valid rows fails with empty_input_file.
• ·Each line has a 1 MB cap. A line larger than that is treated as a malformed row.

End-to-end walkthrough

Four moves: upload the file with purpose=batch, create the batch, poll until terminal, then download the output. The example uses curl plus the OpenAI Python SDK pointed at the Ringside base URL. Every child request bills your developer wallet; batch create does not read an FC-Customer header, so there is no per-customer attribution on a batch in v1.

1. 1
   Upload the JSONL with purpose=batch

   Files are uploaded as multipart/form-data. The purpose must be batch or the create call rejects the file. The 512 MB file-size cap applies. The response gives you a file_... id.

   curl

   Copy

   curl https://api.fightclub.pro/v1/files \
     -H "Authorization: Bearer ko_<key>" \
     -F purpose=batch \
     -F file=@requests.jsonl
   
   # => { "id": "file_3f9a...", "object": "file", "purpose": "batch", ... }

2. 2
   Create the batch against one endpoint

   Pass the input_file_id, the endpoint and completion_window. The only accepted window is 24h. The batch is created in validating; the worker picks it up on its next tick.

   curl

   Copy

   curl https://api.fightclub.pro/v1/batches \
     -H "Authorization: Bearer ko_<key>" \
     -H "Content-Type: application/json" \
     -d '{
       "input_file_id": "file_3f9a...",
       "endpoint": "/v1/chat/completions",
       "completion_window": "24h",
       "metadata": { "job": "sentiment-backfill-2026-06" }
     }'
   
   # => { "id": "batch_8c1d...", "object": "batch", "status": "validating",
   #      "request_counts": { "total": 0, "completed": 0, "failed": 0 }, ... }

3. 3
   Poll until the status is terminal

   Fetch the batch by id and read status plus request_counts. Stop when status is completed, failed, cancelled or expired. A poll every 30-60 seconds is plenty; the worker ticks on a 30-second cadence.

   curl

   Copy

   curl https://api.fightclub.pro/v1/batches/batch_8c1d... \
     -H "Authorization: Bearer ko_<key>"
   
   # => { "id": "batch_8c1d...", "status": "in_progress",
   #      "request_counts": { "total": 3, "completed": 2, "failed": 0 },
   #      "output_file_id": null, "error_file_id": null, ... }

4. 4
   Download the output (and error) files

   On completed, output_file_id is set. If any rows failed, error_file_id is also set. Both are downloaded through the files content endpoint.

   curl

   Copy

   # successful responses
   curl https://api.fightclub.pro/v1/files/file_out9b.../content \
     -H "Authorization: Bearer ko_<key>" -o output.jsonl
   
   # failed rows (only present if error_file_id is set)
   curl https://api.fightclub.pro/v1/files/file_err4c.../content \
     -H "Authorization: Bearer ko_<key>" -o errors.jsonl

import time
from openai import OpenAI

client = OpenAI(
    base_url="https://api.fightclub.pro/v1",
    api_key="ko_<key>",
)

f = client.files.create(file=open("requests.jsonl", "rb"), purpose="batch")

batch = client.batches.create(
    input_file_id=f.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
    metadata={"job": "sentiment-backfill-2026-06"},
)

TERMINAL = {"completed", "failed", "cancelled", "expired"}
while batch.status not in TERMINAL:
    time.sleep(30)
    batch = client.batches.retrieve(batch.id)
    print(batch.status, batch.request_counts)

if batch.output_file_id:
    out = client.files.content(batch.output_file_id)
    open("output.jsonl", "wb").write(out.read())
if batch.error_file_id:
    err = client.files.content(batch.error_file_id)
    open("errors.jsonl", "wb").write(err.read())

Reading the output

The output file is JSONL, one line per processed row. Each line carries the custom_id you set so you can join it back to your input. A row that ran carries a response object with the HTTP status_code and the endpoint body; a row that threw carries an error object instead.

{"id":"batch_req_a1...","custom_id":"row-1","response":{"status_code":200,"request_id":"req_5e2c...","body":{"id":"chatcmpl-...","choices":[{"message":{"role":"assistant","content":"mixed"}}]}},"error":null}
{"id":"batch_req_b2...","custom_id":"row-2","response":{"status_code":200,"request_id":"req_7a91...","body":{"id":"chatcmpl-...","choices":[{"message":{"role":"assistant","content":"positive"}}]}},"error":null}

Partial failures and request_counts

There are two distinct failure layers. Whole-batch validation failures are different from per-row failures, and they land in different places.

Whole-batch failure (validation)

If the input file itself is wrong, the batch never starts dispatching and goes straight to failed. This happens when the file is empty, exceeds 50,000 rows or contains any malformed JSONL line. In that case error_file_id holds a single explanation line and no rows are billed.

Per-row failure (partial success)

Once a batch is in_progress, rows are independent. A row whose underlying call throws is recorded with an error object on its output line rather than aborting the batch. The batch still reaches completed, and request_counts tells you the split.

request_counts.total

Number of valid rows the worker accepted at validation. Set once the batch leaves validating.

request_counts.completed

Rows that produced a response object in the output file.

request_counts.failed

Rows that produced an error instead of a response, plus any malformed rows counted into the error file.

A healthy completed batch has completed + failed == total. If failed is non-zero, read the failed lines: rows that errored mid-dispatch appear in the output file with a non-null error and a null response, while structurally bad rows are written to error_file_id. Re-run the failures by filtering on custom_id and submitting a new, smaller batch.

Cancelling a batch

You can cancel a batch while it is validating or in_progress. Cancel moves it to cancelling; the next worker tick flushes any child output that already ran and flips it to cancelled. Output from rows that completed before the cancel is preserved.

curl -X POST https://api.fightclub.pro/v1/batches/batch_8c1d.../cancel \
  -H "Authorization: Bearer ko_<key>"

# => { "id": "batch_8c1d...", "status": "cancelling", ... }

Cancelling an already-cancelling or cancelled batch is idempotent: it returns the current row. Cancelling a terminal batch (completed, failed, cancelled or expired) returns 400 invalid_state_transition.

Listing batches

List your batches newest-first with cursor pagination. The response is an {object:"list", data, has_more, next_cursor} envelope. limit defaults to 20 and caps at 100. Pass the previous next_cursor as after to page. An optional status filter narrows to one of the eight states.

curl "https://api.fightclub.pro/v1/batches?limit=20&status=completed" \
  -H "Authorization: Bearer ko_<key>"

# => { "object": "list",
#      "data": [ { "id": "batch_8c1d...", "status": "completed", ... } ],
#      "has_more": false, "next_cursor": null }

Limits, defaults and errors

Billing notes

• ·Each child request deducts from your developer wallet. Batch create does not read FC-Customer, so there is no per-customer attribution on a batch in v1.
• ·The 0.5x discount applies per qualifying batch (10+ rows). Smaller batches are billed at the standard 1.0x markup.
• ·Moderation rows are free regardless of batch size; the discount multiplier is audit-only there.
• ·An empty developer wallet fails the affected rows the same way a synchronous call would, and those rows land in the failed count.