Batch processing strategies

Benefits and limits of the Message Batches API

50% cost savings, a processing window of up to 24 hours, and no guaranteed latency SLA. Those three facts decide every question in this task.

Latency tolerance is the deciding factor

• Appropriate: non-blocking, latency-tolerant work. Overnight reports, weekly audits, nightly test generation.
• Inappropriate: anything blocking. A pre-merge check that gates a pull request cannot wait an unbounded number of hours for a 50% saving.

The trap is treating the discount as a default. It's a discount for work that genuinely doesn't care when it finishes.

Ground shipping costs less than overnight: the right trade for a textbook you need next month, the wrong one for a passport you need Thursday. Unlike a carrier, though, batch gives an outer bound and no estimate inside it.

Tool use and continuation

Message Batches supports tool use, multi-turn message histories, and the server-side agentic loop. Server tools — web search, web fetch, code execution, MCP connectors — run inside that loop, so their results come back within the same batch. A client tool is one your own application executes, such as a script or a test runner, which is why it has to come back to you. A client tool still returns a tool_use request for your application to execute; continuing from that result requires another batch or synchronous request. Iterative client-tool workflows are therefore possible but can accumulate asynchronous round trips and deadline risk.

Sizing submissions against an SLA

Because processing can take up to 24 hours, the submission interval is what you control.

For a 30-hour target with a 24-hour processing window, submitting at least every six hours reserves the arithmetic budget; a four-hour cadence leaves two hours of additional margin. But cadence alone cannot create a hard guarantee: a batch can expire or fail. A real SLA needs monitoring, retry or synchronous fallback, and explicit handling for expired requests.

Handle batch failures and control costs

Not every failure is worth another attempt unchanged. A document that exceeded the context limit will exceed it identically on resubmission, and each round costs another processing window. Read what the failure was before resubmitting: some items need a modified input, not a repeat.

Every request carries a custom_id that links it to its response. Results can return in any order, so do not match them by list position. Do not ask the model to copy a document identifier either. Use the exact custom_id supplied by your application.

One entry in the submitted batch. custom_id is yours to assign; params is an ordinary request:

{
  "custom_id": "invoice-2026-08-0117",
  "params": {
    "model": "claude-opus-5",
    "max_tokens": 1024,
    "messages": [{ "role": "user", "content": "Extract the fields." }]
  }
}

The results show why each request needs a unique custom_id:

{"custom_id": "invoice-2026-08-0119", "result": {"type": "succeeded"}}
{"custom_id": "invoice-2026-08-0117", "result": {"type": "succeeded"}}
{"custom_id": "invoice-2026-08-0118", "result": {"type": "errored"}}

Three results, and none of them is where you put it. Reading 0118 as the answer to your second request would be silently wrong, and nothing about the output would look broken.

On partial failure, resubmit only the failed items, identified by that id, with whatever modification the failure implies — chunking a document that exceeded context limits, for instance.

And before committing a large volume: refine the prompt on a sample first. First-pass success rate is what determines how many expensive resubmission rounds you pay for.