Skip to content
RelayRouter
On this page

Inference

Chat completions

The main endpoint. [OI]-compatible request and response shapes, with relay-specific reliability fields.

POST/v1/chat/completionsAPI key (JWT also resolves)

Aliases: /api/v1/chat/completions, /playground/run, and /playground. The body is passed through to the upstream: messages, tools, temperature, top_p, stop, response_format, and so on.

Request

curl -X POST https://api.relayrouter.org/v1/chat/completions \
  -H "Authorization: Bearer $RELAY_API_KEY" \
  -H 'content-type: application/json' \
  -d '{
    "model": "deepseek-v4.1-flash-1017",
    "messages": [
      { "role": "system", "content": "be terse" },
      { "role": "user", "content": "hello" }
    ],
    "max_tokens": 256,
    "temperature": 0.7,
    "stream": false
  }'
FieldTypeDescription
no_fallbackbooleanNever switch models for this request. Same-model retries still happen.
fallback_modelsstring[]Explicit failover chain for this request.
streambooleanReturn a server-sent event stream instead of one JSON body.

Response

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "created": 1792287604,
  "model": "deepseek-v4.1-flash-1017",
  "choices": [{
    "index": 0,
    "finish_reason": "stop",
    "message": { "role": "assistant", "content": "hi" }
  }],
  "usage": {
    "prompt_tokens": 23,
    "completion_tokens": 2,
    "total_tokens": 84,
    "prompt_cache_hit_tokens": 0,
    "prompt_cache_miss_tokens": 87,
    "prompt_cache_write_tokens": 0,
    "cache_read_input_tokens": 0,
    "cache_creation_input_tokens": 0,
    "cached_tokens": 0,
    "completion_thinking_tokens": 0,
    "completion_tokens_details": { "reasoning_tokens": 0 },
    "cost": 0.00001
  }
}
  • total_tokens may exceed prompt + completion because of hidden prompt overhead or reasoning. Billing and quota use the provider total when present.
  • Reasoning models may return message.reasoning_content.completion_tokens includes thinking tokens.
  • usage.cost is the provider's number, not the amount RelayRouter charged. Do not display it as your bill.
  • A 200 response containing an error object, or an empty completion with no text, reasoning, or tool calls, is treated as an upstream failure and is not billed.

Tools and JSON mode

Tool definitions and response_format are passed through unchanged. Tool calls appear in message.tool_calls and count toward completion tokens as reported by the provider.

Billing and errors

The serving model is billed at its own rate. A fallback hop is never billed at the cheaper requested model's rate. Failed upstream calls are never billed. See Errors for status codes and provider codes.

PreviousModelsNextStreaming