Skip to main content
POST
/
v1
/
count_tokens
Count Tokens
curl --request POST \
  --url http://localhost:8080/v1/count_tokens \
  --header 'Content-Type: application/json' \
  --data '
{
  "model": "<string>",
  "input": [
    {
      "role": "user",
      "content": "<string>"
    }
  ],
  "fallbacks": [
    "<string>"
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "<string>",
        "arguments": {}
      }
    }
  ],
  "stream": true,
  "stream_format": "<string>",
  "background": true,
  "conversation": "<string>",
  "include": [
    "<string>"
  ],
  "instructions": "<string>",
  "max_output_tokens": 123,
  "max_tool_calls": 123,
  "metadata": {},
  "parallel_tool_calls": true,
  "previous_response_id": "<string>",
  "prompt_cache_key": "<string>",
  "reasoning": {
    "effort": "<string>"
  },
  "safety_identifier": "<string>",
  "service_tier": "<string>",
  "stream_options": {
    "include_usage": true
  },
  "store": true,
  "temperature": 123,
  "text": {
    "max_tokens": 123
  },
  "top_logprobs": 123,
  "top_p": 123,
  "tool_choice": "auto",
  "truncation": "<string>"
}
'
{
  "object": "<string>",
  "model": "<string>",
  "input_tokens": 123,
  "input_tokens_details": {
    "text_tokens": 123,
    "audio_tokens": 123,
    "image_tokens": 123,
    "cached_tokens": 123
  },
  "tokens": [
    123
  ],
  "token_strings": [
    "<string>"
  ],
  "output_tokens": 123,
  "total_tokens": 123,
  "extra_fields": {
    "provider": "openai",
    "request_type": "list_models",
    "model_requested": "<string>",
    "model_params": {
      "temperature": 0.7,
      "top_p": 0.9,
      "top_k": 40,
      "max_tokens": 1000,
      "stop_sequences": [
        "\n\n",
        "END"
      ],
      "presence_penalty": 0,
      "frequency_penalty": 0,
      "tools": [
        {
          "type": "function",
          "function": {
            "name": "get_weather",
            "description": "Get current weather for a location",
            "parameters": {
              "type": "object",
              "description": "<string>",
              "properties": {},
              "required": [
                "<string>"
              ],
              "enum": [
                "<string>"
              ]
            }
          },
          "id": "<string>"
        }
      ],
      "tool_choice": {
        "type": "auto",
        "function": {
          "name": "get_weather"
        }
      },
      "parallel_tool_calls": true
    },
    "latency": 1234,
    "billed_usage": {
      "prompt_tokens": 123,
      "completion_tokens": 123,
      "search_units": 123,
      "classifications": 123
    },
    "raw_response": {}
  }
}

Body

application/json
model
string
required

Model identifier in 'provider/model' format

input
object[]
required

Messages to tokenize

Minimum array length: 1
fallbacks
string[]

Fallback models in 'provider/model' format

tools
object[]
stream
boolean
stream_format
string
background
boolean
conversation
string
include
string[]
instructions
string
max_output_tokens
integer
max_tool_calls
integer
metadata
object
parallel_tool_calls
boolean
previous_response_id
string
prompt_cache_key
string
reasoning
object
safety_identifier
string
service_tier
string
stream_options
object
store
boolean
temperature
number
text
object
top_logprobs
integer
top_p
number
tool_choice
Available options:
auto,
any,
none,
required,
tool
truncation
string

Response

200 - application/json

Token count response

object
string

Response object type

model
string
input_tokens
integer
input_tokens_details
object

Detailed input token breakdown. Only populated for Vertex and Gemini providers.

tokens
integer[]

Array of token IDs

token_strings
string[]

Array of token strings (if available)

output_tokens
integer | null
total_tokens
integer | null
extra_fields
object