Skip to main content

Retry only transient failures

Use bounded exponential backoff, respect Retry-After, and stop after a fixed attempt or time budget.

Retry conditions

Do not retry authentication, billing, validation, missing-model, or malformed-request errors until the request is corrected.

Wait for a cold model

A model that has not served for a while goes cold. The next call starts it and gets 425 until it is ready, which can take several minutes. To wait instead of retrying, send X-Fastino-Wait-For-Model: true on POST /v1/chat/completions, /v1/messages, /v1/responses, /v1/systemone, or /v1/gliner-2. The request waits up to 150 seconds for the model, then runs. If the model is still starting after that, you get the usual 425 with Retry-After, and the model keeps starting for your retry. The header accepts true, 1, or yes in any case; any other value is ignored. Streaming requests ignore this header.
Set your client read timeout above 170 seconds so it does not give up while the request waits.

Check and warm a model before use

  • GET /v1/model-status/{model_id} returns state (ready, starting, or cold) and retry_after_seconds, which is null once the model is ready.
  • POST /v1/model-status/{model_id}/warm starts the model without running inference and returns 202. It is not billed, and repeated calls share one start.
Warm the model before a benchmark or a traffic burst, then poll the status every retry_after_seconds until it is ready. Each API server reports what it has seen, so right after a warm call the status can still say cold for a short time; keep polling.

Safe retry checklist

  • Use an idempotency key for training-job creation and other supported mutations.
  • Keep the same request body when replaying an idempotent request.
  • Apply jitter and a maximum delay.
  • Log the final status and request ID, but never the API key.
See the complete error response shapes and resolution guidance in Errors.