This guide was prepared with AI assistance using the linked documentation. Code examples are illustrative; verify them against your environment and current API documentation before use.
A rate limit is not a reason to repeat everything
A user asks your assistant to prepare a report. The API returns a 429, and your application retries. That is reasonable if the attempt was rejected before doing work. It is dangerous if the previous attempt already created a ticket or sent an email through a tool.
Treat model requests and external actions as separate operations. A retry policy can improve availability, but each write needs its own protection against duplicate execution.
- Identify the request and the user-visible task separately.
- Classify the error before retrying.
- Record whether an external action already completed.
Find the limit you are actually hitting
OpenAI applies limits at organization and project levels and they vary by model. Shared model-family limits may be consumed by more than one model. Check the limits page for the project receiving the errors rather than assuming the issue belongs to a single user.
Requests per minute and tokens per minute are different constraints. Adding concurrency can exhaust either one, and cached input still counts toward the relevant rate limits. Rate and spending limits are also separate controls.
Build a queue around the workload you control. Limit concurrent work per project and model, and make the queue visible to operators. If a customer can submit a large job, apply product-level quotas as well: provider limits are shared resources, not a substitute for per-user controls.
Honor Retry-After and add jitter
When a temporary limit returns a retry delay, wait at least that long. Add randomness to prevent many workers from returning at the same instant. Use a bounded attempt count and a deadline; eventually tell the user that the task could not be completed and offer a safe way to try again.
The official SDKs already retry some eligible failures. Check the installed SDK's behavior before adding another loop, or the two policies may multiply attempts.
Keep the retry loop bounded by both elapsed time and attempts. A request that has already exceeded the user’s deadline should not continue consuming resources indefinitely. Distinguish a temporary rate limit from an account or quota problem that another immediate retry will not resolve.
// Illustrative delay calculation only; parse and classify the actual error separately.
function waitMs(attempt: number, retryAfterMs = 0): number {
const ceiling = Math.min(30_000, 500 * 2 ** attempt);
return retryAfterMs + Math.floor(Math.random() * ceiling);
}Protect writes with a durable operation key
Before the first attempt, assign a stable identifier to the intended business action. Store the key with the action status in your database and pass it to downstream services that support idempotency. If a worker restarts, look up the existing status before creating another order or notification.
The key must identify the intended action, not the network attempt. A new key on every retry cannot prevent duplication. Also decide how to resolve an unknown outcome: a timeout after submission may require querying the downstream system before retrying.
Store enough state to distinguish an action that has not started from one in progress, one confirmed complete, and one whose outcome is unknown. A durable record matters because process memory disappears on a restart. Do not mark the action successful solely because you sent the request.
Streaming needs its own failure path
An error can occur after a stream has started and after the user has seen partial text. OpenAI's rate-limit guidance distinguishes pre-stream HTTP errors from errors delivered during a stream. Do not silently replay a partially consumed response as if nothing happened.
Keep the partial result visible, label it incomplete, and let the user retry when appropriate. If tools have already run, reconcile their state first. A retry should not repeat a completed side effect.
Test and operate the policy
Test temporary 429 responses, a non-retryable request error, an unavailable downstream service, a timeout after a write, and a stream interrupted after output. Confirm the user sees a useful status and an operator can distinguish failures from completed work.
Record attempt counts, error classes, elapsed time, and the final task outcome. Avoid logging raw prompts or secrets by default. Revisit caps when workload or model limits change.
Before a release, simulate two workers receiving the same job and a crash immediately after the downstream service accepts a write. Inspect the final business records, not just application logs. The expected result is one intended action or an explicit unresolved state, not a second action created by the test harness.
Example: a write with an unknown outcome
A service times out after receiving a request to create a ticket. The timeout alone does not tell you whether the ticket exists. Mark the action as having an unknown outcome and look it up using the stable operation key or a supported downstream reconciliation mechanism.
If the ticket exists, return the existing result. If the service confirms it does not, retry using the same key. If the outcome cannot be established, route the case to a person instead of creating a duplicate. Give the user a status that reflects this uncertainty.
Sources and further reading
Documentation reviewed November 9, 2026. Verify current model support, limits and pricing before implementation.