This guide was prepared with AI assistance using the linked documentation. Code examples are illustrative; verify them against your environment and current API documentation before use.
Choose by the shape of the work
You have 5,000 independent product descriptions to classify overnight. A batch is a natural fit. You have one research request that may take several minutes. A background response may fit better.
Both are asynchronous, but their inputs, result handling, limits, and retention characteristics differ. Start with the user's deadline and how you will recover unfinished work.
Before selecting either path, write down the deadline, how users will learn that the job is complete, and what happens to individual failures. If a workflow requires a fast interactive response, an asynchronous queue may need a different product design rather than a hidden change behind the same button.
Use Batch for independent requests
The Batch API accepts a JSONL input file: one request per line, each with a unique custom_id. OpenAI's current guide lists supported endpoints and states that each input file uses a single model. Results are matched by custom_id rather than input order.
Check the guide for current completion windows, pricing, upload limits, and eligibility. Asynchronous processing is appropriate only if the product can wait and handle individual failed requests.
Batch work needs a manifest that survives a worker restart. Track which records were submitted, the batch identifier, the configuration version, and the final result or error for each custom_id. This lets you reconcile an output file without relying on the order in which the provider returns lines.
Prepare identifiers before submission
Use an identifier that lets your application map a result back to the right record without exposing unnecessary customer information. Save your input manifest and task state separately; an output file is not a substitute for your application's record of what was submitted.
This single JSONL line illustrates the required shape. Model access and supported parameters must be checked in the current documentation.
{"custom_id":"item-001","method":"POST","url":"/v1/responses","body":{"model":"gpt-6-luna","input":"Classify this synthetic example."}}Use background mode for one long response
A background Responses request starts one asynchronous generation and returns a response object that your application can poll. The current guide says to continue polling while the status is queued or in_progress, then handle the terminal state.
Persist the response ID and give the user a way to leave and return. A worker should use bounded polling and handle failure, cancellation, and incomplete output rather than assuming every terminal state is success.
A background response is useful when a single task may outlast the original browser request. It does not remove the need for an application job record. Tie the response identifier to the authenticated user and enforce authorization again when the result is retrieved. Do not expose a raw identifier as a substitute for an access check.
Plan streaming and retention explicitly
Background mode can be combined with streaming when created with streaming enabled. The documentation describes resuming from a sequence number; do not assume an ordinary non-streaming background response can later be converted into a new stream.
Background responses are temporarily stored to support polling, including requests with store=false. Retention and eligibility depend on the feature and project policy. Check the data-controls page before sending sensitive content.
A user may cancel a task after a worker has already started downstream processing. Design the job states so that cancellation requests, confirmed cancellation, completion, and unknown outcomes remain distinguishable. The interface should reflect the state your system has actually confirmed, not assume that closing a browser tab stopped the job.
Reconcile the results
For a batch, join output and error records to the submitted manifest using custom_id. Identify missing, failed, and completed records. Retry only the subset that needs another attempt and make any downstream writes idempotent.
For a background response, save the terminal state and the result or error according to your retention policy. Show users whether their request is queued, running, completed, or needs attention. Never call a job finished solely because the browser stopped waiting.
Exercise an expired or unavailable result, a failed row, duplicate custom IDs rejected before submission, and a worker restart between polling and persistence. Verify that your application reports which items need attention. If retrying produces a customer-facing write, protect that write independently from the model request.
Example: process a catalog overnight
Create a manifest with a stable ID for each item and the instruction version. Validate records before submission and check that the user may process them. When results arrive, join output and error records to the manifest using custom_id; do not assume result order matches input order.
For a single background response, save the response ID with the user’s job and poll from a worker. This lets a person leave and return without submitting the same work again. Distinguish completed, failed, and cancelled jobs.
Sources and further reading
Documentation reviewed November 9, 2026. Verify current model support, limits and pricing before implementation.