Rate Limits

Overview

Rate limits are enforced per account, across all API keys belonging to it — creating extra keys does not raise your ceiling. Two dimensions are checked on every request, before any generation starts.

Your current limits are shown in the console. Need more headroom? Contact us with your expected traffic pattern.

The 429 contract

When either dimension is exceeded, the API responds with HTTP 429 and a JSON error body. The response always carries Retry-After (in seconds) plus the four x-ratelimit-* headers, so you can act on it without guessing.

HTTP/1.1 429 Too Many Requests
Retry-After: 3
x-ratelimit-limit-requests: 8
x-ratelimit-remaining-requests: 0
x-ratelimit-limit-tokens: 400000
x-ratelimit-remaining-tokens: 0
Content-Type: application/json

{
  "error": {
    "code": "rate_limited",
    "message": "Concurrency limit reached for your account. Retry after 3s.",
    "type": "rate_limit_error"
  }
}

Header values above are illustrative. Read them from the actual response — do not hard-code them.

A 429 means nothing was generated and nothing was billed: the check runs before the request reaches the model. It is always safe to retry a request that failed with 429.

Rate limit headers

These headers are present on both successful responses and 429s, so you can steer traffic before you get throttled.

Retry strategy

Retry 429s with exponential backoff and jitter. Retrying immediately, or retrying every client at the same interval, turns a brief spike into a sustained outage.

1. Prefer the Retry-After value when present — it reflects the real reset window.
2. Otherwise back off exponentially: 1s, 2s, 4s, 8s … capped at roughly 60s.
3. Add full jitter (sleep a random amount between 0 and the computed delay) so concurrent clients do not resynchronise.
4. Cap the number of attempts (5–6 is plenty) and surface the failure instead of retrying forever.

import random
import time

import openai
from openai import OpenAI

client = OpenAI(base_url="https://api.apodex.ai/v1", api_key="YOUR_API_KEY")

MAX_ATTEMPTS = 6
BASE_DELAY = 1.0   # seconds
MAX_DELAY = 60.0

def create_with_retry(**kwargs):
    for attempt in range(MAX_ATTEMPTS):
        try:
            return client.chat.completions.create(**kwargs)
        except openai.RateLimitError as err:  # HTTP 429
            if attempt == MAX_ATTEMPTS - 1:
                raise
            # Retry-After 是服务端给的权威值，优先用它；没有再退指数退避
            retry_after = (err.response.headers or {}).get("retry-after")
            if retry_after:
                delay = float(retry_after)
            else:
                delay = min(BASE_DELAY * 2 ** attempt, MAX_DELAY)
            # full jitter：避免所有客户端在同一毫秒一起重试
            time.sleep(random.uniform(0, delay))

resp = create_with_retry(
    model="apodex-1.1",
    messages=[{"role": "user", "content": "Hello"}],
    max_tokens=512,
)

Reducing throttling

• Bound your own concurrency to the value in x-ratelimit-limit-requests with a client-side semaphore, instead of firing everything and absorbing 429s.
• Watch x-ratelimit-remaining-tokens and slow down as it approaches zero, rather than waiting for the 429.
• Trim prompts and set a realistic max_tokens — TPM counts both prompt and generated tokens.
• Send a stable X-Session-Id so repeated turns hit the prefix cache; see Core Models.