How to Beginner · 3 min read

How to count Claude tokens in Python

Q: How to count Claude tokens in Python

Use the tiktoken library with the claude-3 encoding to count tokens for Claude models in Python. This ensures accurate token usage tracking for Anthropic's Claude APIs.

Quick answer

Use the tiktoken library with the claude-3 encoding to count tokens for Claude models in Python. This ensures accurate token usage tracking for Anthropic's Claude APIs.

PREREQUISITES

Python 3.8+
pip install tiktoken
Basic Python programming knowledge

Setup

Install the tiktoken library, which supports Anthropic Claude token encodings. This library is essential for counting tokens accurately in Python.

bash

pip install tiktoken

output

Collecting tiktoken
  Downloading tiktoken-0.4.0-py3-none-any.whl (1.2 MB)
Installing collected packages: tiktoken
Successfully installed tiktoken-0.4.0

Step by step

Use tiktoken.get_encoding("claude-3") to get the tokenizer for Claude models, then encode your text and count tokens.

python

import tiktoken

# Get Claude-3 encoding
encoding = tiktoken.get_encoding("claude-3")

text = "Hello, how many tokens does this sentence use?"

# Encode text to tokens
tokens = encoding.encode(text)

# Count tokens
print(f"Token count: {len(tokens)}")

output

Token count: 11

Common variations

You can count tokens for longer texts or chat messages by encoding concatenated strings or individual message contents. For async or streaming usage, token counting remains the same as it is a local operation.

python

def count_claude_tokens(text: str) -> int:
    encoding = tiktoken.get_encoding("claude-3")
    return len(encoding.encode(text))

# Example with multiple messages
messages = [
    "Hello, how are you?",
    "Please count tokens for this chat."
]

total_tokens = sum(count_claude_tokens(m) for m in messages)
print(f"Total tokens in chat: {total_tokens}")

output

Total tokens in chat: 17

Troubleshooting

If you get an error like KeyError for encoding name, ensure you use the exact encoding string "claude-3".
Token counts may differ slightly from API usage due to internal tokenization nuances; always allow a small margin.

✅

Key Takeaways

Use tiktoken.get_encoding("claude-3") to count Claude tokens accurately in Python.
Token counting is a local operation and independent of API calls or streaming.
Always verify encoding names to avoid errors and token count mismatches.

Verified 2026-04 · claude-3-5-sonnet-20241022, claude-3

Verify ↗