> ## Documentation Index
> Fetch the complete documentation index at: https://docs.fastrun.si/llms.txt
> Use this file to discover all available pages before exploring further.

# Errors and retries

> What each error means and which ones to retry.

Errors use the OpenAI format, with an HTTP status and a JSON body:

```json theme={null}
{
  "error": {
    "message": "all backends for this model are at capacity",
    "type": "rate_limit_error",
    "code": 429
  }
}
```

| Status | `type` | Meaning | Retry? |
| - | - | - | - |
| 400 | `invalid_request_error` | The body is not a JSON object, or `model` is missing. | No. Fix the request. |
| 400 | (from the model) | The model rejected the request, for example a prompt longer than the context window. | No. Fix the request. |
| 401 | `authentication_error` | The key is missing, unknown or revoked, or the account is not approved yet. | No. See [Authentication](https://docs.fastrun.si/authentication.md). |
| 429 | `rate_limit_error` | Every GPU serving the model is busy. | Yes, after `Retry-After`. |
| 502 | `upstream_error` | The GPU servers failed to answer. | Yes, with backoff. |
| 503 | `unavailable` | No GPU server for the model is up right now, or keys cannot be checked for a moment. | Yes, after `Retry-After`. |

Before answering with 429, 502 or 503, fastrun has already tried the other GPU servers that serve the model. When
these errors persist, wait longer between retries rather than retrying faster.

## Retrying

Retry 429, 502 and 503 with exponential backoff, and wait at least as long as the `Retry-After` header says. OpenAI
SDKs do this for you: they retry these statuses twice by default and follow `Retry-After`. You can raise the count:

```python Python theme={null}
client = OpenAI(
    base_url="https://api.fastrun.si/v1",
    api_key=os.environ["FASTRUN_API_KEY"],
    max_retries=5,
)
```

```typescript TypeScript theme={null}
const client = new OpenAI({
  baseURL: "https://api.fastrun.si/v1",
  apiKey: process.env.FASTRUN_API_KEY,
  maxRetries: 5,
});
```

Don't retry 400 and 401: the same request fails the same way.

## Errors in a stream

A streamed response can start with status `200` before the request is sure to succeed. A failure after that point
arrives as an event with an `error` field. See [Streaming](https://docs.fastrun.si/guides/streaming.md#errors-during-a-stream).

## Timeouts

A streamed request can run for up to 60 minutes. A request without streaming sends nothing until the whole answer is
ready and must be complete within 10 minutes, so set your client's timeout to cover the longest answer you expect. For
long answers, use [streaming](https://docs.fastrun.si/guides/streaming.md), which sends tokens and keep-alive comments as it goes.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.