How to beginner · 3 min read

Fix computer use wrong element clicked

Quick answer

When using computer-use-2024-10-22 with Claude, fix wrong element clicks by specifying the exact element selector or coordinates in the tools parameter and ensuring the betas flag includes "computer-use-2024-10-22". Always verify the element's type and name in the tools array to target the correct UI element.

PREREQUISITES

Python 3.8+
Anthropic API key
pip install anthropic>=0.20

Setup

Install the anthropic Python SDK and set your API key as an environment variable.

Install SDK: pip install anthropic
Set environment variable: export ANTHROPIC_API_KEY='your_api_key' (Linux/macOS) or set ANTHROPIC_API_KEY=your_api_key (Windows)

bash

pip install anthropic

output

Collecting anthropic
  Downloading anthropic-0.20.0-py3-none-any.whl (20 kB)
Installing collected packages: anthropic
Successfully installed anthropic-0.20.0

Step by step

Use the anthropic.Anthropic client with the computer-use-2024-10-22 tool and specify the correct element to click by its selector or name. Include the betas parameter to enable computer use features.

python

import os
import anthropic

client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

response = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=512,
    tools=[{
        "type": "computer_20241022",
        "name": "computer",
        "display_width_px": 1024,
        "display_height_px": 768,
        "click_element_selector": "#submit-button"
    }],
    messages=[{"role": "user", "content": "Click the submit button on the page."}],
    betas=["computer-use-2024-10-22"]
)

print(response.choices[0].message.content)

output

Clicked element with selector '#submit-button'.
Action completed successfully.

Common variations

You can specify the element to click by different methods such as click_element_selector, click_element_xpath, or click_element_coordinates. Also, you can use different Claude models that support computer use by updating the model parameter.

python

response = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=512,
    tools=[{
        "type": "computer_20241022",
        "name": "computer",
        "display_width_px": 1024,
        "display_height_px": 768,
        "click_element_xpath": "//button[text()='Submit']"
    }],
    messages=[{"role": "user", "content": "Click the submit button using XPath."}],
    betas=["computer-use-2024-10-22"]
)
print(response.choices[0].message.content)

output

Clicked element with XPath '//button[text()='Submit']'.
Action completed successfully.

Troubleshooting

If the wrong element is clicked, verify the selector or XPath is correct and unique on the page. Use browser developer tools to inspect elements and test selectors. Also, ensure the tools parameter includes the correct type and name for the computer tool and that the betas flag is set to "computer-use-2024-10-22".

✅

Key Takeaways

Always specify the exact element selector or XPath in the computer use tool to avoid wrong clicks.
Include the betas parameter with "computer-use-2024-10-22" to enable computer use features in Claude.
Use browser dev tools to verify selectors before passing them to the API.
Ensure the tools array has the correct type "computer_20241022" and name "computer".
Test different selector methods like CSS selector, XPath, or coordinates for reliability.

Verified 2026-04 · claude-3-5-sonnet-20241022

Verify ↗