> ## Documentation Index
> Fetch the complete documentation index at: https://docs.fastrun.si/llms.txt
> Use this file to discover all available pages before exploring further.

# Prompt caching

> Pay less and wait less when requests share a beginning.

When a request starts with the same tokens as a recent one, the GPU reuses the work it already did for that beginning
instead of reading it again. This happens automatically; there is nothing to turn on.

Cached input is cheaper ($0.10 instead of $0.50 per million tokens for Gemma 4 31B) and makes the first token arrive
sooner, which matters most for long system prompts, documents and conversations.

## How to get cache hits

* **Put what stays the same first.** System prompt, tool definitions and reference documents go at the start; the
  part that changes goes at the end.
* **Keep earlier turns unchanged.** In a conversation, send the previous messages exactly as before and add the new
  ones at the end. Editing an early message makes everything after it uncached.
* **Use the same API key for a conversation.** fastrun sends the requests of one conversation (the same key, model,
  system prompt and first user message) to the same GPU server, where its earlier turns are cached.

## Seeing cache hits

`usage.prompt_tokens_details.cached_tokens` tells you how many input tokens came from the cache:

```json theme={null}
"usage": {
  "prompt_tokens": 5200,
  "completion_tokens": 120,
  "total_tokens": 5320,
  "prompt_tokens_details": { "cached_tokens": 4992 }
}
```

The cache is best effort. Entries are dropped when space is needed, and a request can land on a server that has not
seen the beginning yet, for example when the usual one is full. Cached tokens are counted in whole blocks of tokens, so
the last few tokens of a shared beginning may not count as cached.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.