Models - fastrun

Gemma 4 31B Instruct

google/gemma-4-31b-it

Google’s Gemma 4 31B, instruction-tuned, with FP8 weights. It runs on vLLM with multi-token-prediction speculative decoding, which speeds up generation without changing the output distribution.


Context window	131,072 tokens (input and output together)
Maximum output	16,384 tokens
Input	Text only
Output	Text
Quantization	FP8
Region	United States

Features

Feature	Support
Streaming	Yes
Tool calling	Yes
Structured outputs (`response_format`)	Yes
Reasoning	Yes, off by default
Prompt caching	Yes, automatic
Images, audio, video	No

Sampling parameters

temperature (0–2), top_p (0–1), top_k (0–1000), min_p (0–1), frequency_penalty (−2–2), presence_penalty (−2–2), repetition_penalty (0–2), max_tokens, stop (up to 4), seed, logprobs and top_logprobs (0–20). See Create a chat completion for details.

Listing models

GET /v1/models returns the models that can serve requests right now.

curl https://api.fastrun.si/v1/models \
  -H "Authorization: Bearer $FASTRUN_API_KEY"

{
  "object": "list",
  "data": [
    { "id": "google/gemma-4-31b-it", "object": "model", "owned_by": "fastrun" }
  ]
}