How to analyze image with Claude
Quick answer
Use the
claude-3-5-sonnet-20241022 model with the anthropic Python SDK and enable the computer-use-2024-10-22 beta along with the computer_20241022 tool to analyze images. Upload the image as base64 or URL in the message content and request analysis in natural language.PREREQUISITES
Python 3.8+Anthropic API keypip install anthropic>=0.20
Setup
Install the anthropic Python SDK and set your API key as an environment variable.
pip install anthropic>=0.20Step by step
This example shows how to send an image to Claude for analysis using the computer-use-2024-10-22 beta and the computer_20241022 tool. The image is encoded in base64 and included in the message content.
import os
import base64
import anthropic
client = anthropic.Client(api_key=os.environ["ANTHROPIC_API_KEY"])
# Load and encode image as base64
with open("image.png", "rb") as f:
image_b64 = base64.b64encode(f.read()).decode("utf-8")
# Prepare tool for computer use
tools = [{
"type": "computer_20241022",
"name": "computer",
"display_width_px": 1024,
"display_height_px": 768
}]
# Compose message asking Claude to analyze the image
messages = [{
"role": "user",
"content": f"Analyze this image:\ndata:image/png;base64,{image_b64}"
}]
response = client.messages.create(
model="claude-3-5-sonnet-20241022",
max_tokens=1024,
system="You are a helpful assistant that can analyze images.",
messages=messages,
tools=tools,
betas=["computer-use-2024-10-22"]
)
print(response.content[0].text) output
The assistant provides a detailed analysis of the image content, describing objects, colors, or text detected.
Common variations
- Use image URLs instead of base64 by including the URL in the message content.
- Adjust
max_tokensfor longer or shorter responses. - Use async calls with
asyncioandanthropicif needed.
Troubleshooting
- If you get an error about missing betas, ensure you include
betas=["computer-use-2024-10-22"]in the request. - Check your API key and environment variable
ANTHROPIC_API_KEY. - Make sure the image file path is correct and the image is not too large (keep under a few MB).
Key Takeaways
- Use the Anthropic SDK with the computer-use beta to enable image analysis in Claude.
- Encode images as base64 or provide URLs in the user message content for multimodal input.
- Always include the correct beta flag and tool definition to avoid request errors.