How to beginner · 3 min read

Computer use for software testing

Q: Computer use for software testing

Use the OpenAI API with the computer_20241022 tool to automate software testing tasks like running scripts or capturing screenshots. Include the betas=["computer-use-2024-10-22"] parameter and specify the tool in the tools argument for effective computer use in testing.

Quick answer

Use the OpenAI API with the computer_20241022 tool to automate software testing tasks like running scripts or capturing screenshots. Include the betas=["computer-use-2024-10-22"] parameter and specify the tool in the tools argument for effective computer use in testing.

PREREQUISITES

Python 3.8+
OpenAI API key (free tier works)
pip install openai>=1.0

Setup

Install the openai Python package and set your API key as an environment variable.

Run pip install openai to install the SDK.
Set your API key in your shell: export OPENAI_API_KEY='your_api_key' (Linux/macOS) or setx OPENAI_API_KEY "your_api_key" (Windows).

bash

pip install openai

output

Collecting openai
  Downloading openai-1.x.x-py3-none-any.whl (xx kB)
Installing collected packages: openai
Successfully installed openai-1.x.x

Step by step

This example demonstrates how to use the computer_20241022 tool with the OpenAI Python SDK to run a simple software testing command and get the output.

python

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

# Define the computer use tool
tools = [{
    "type": "computer_20241022",
    "name": "computer",
    "display_width_px": 1024,
    "display_height_px": 768
}]

messages = [
    {"role": "user", "content": "Run a Python script that prints 'Hello, software testing!'"}
]

response = client.chat.completions.create(
    model="gpt-4o-mini",
    tools=tools,
    messages=messages,
    betas=["computer-use-2024-10-22"]
)

print("Response:", response.choices[0].message.content)

output

Response: Executing your Python script...
Hello, software testing!
Script executed successfully.

Common variations

You can use different models like gpt-4o for more advanced testing or run asynchronous calls with asyncio. Streaming responses are not supported with computer use tools currently.

python

import os
import asyncio
from openai import OpenAI

async def run_test():
    client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
    tools = [{
        "type": "computer_20241022",
        "name": "computer",
        "display_width_px": 1024,
        "display_height_px": 768
    }]
    messages = [{"role": "user", "content": "Run a shell command to list files."}]

    response = await client.chat.completions.create(
        model="gpt-4o",
        tools=tools,
        messages=messages,
        betas=["computer-use-2024-10-22"]
    )
    print("Async response:", response.choices[0].message.content)

asyncio.run(run_test())

output

Async response: Executing shell command...
file1.py
file2.txt
README.md
Command executed successfully.

Troubleshooting

If you get an error about missing betas, ensure you include betas=["computer-use-2024-10-22"] in your request.
If the tool call is not recognized, verify the tools parameter uses "type": "computer_20241022" exactly.
Check your API key and network connectivity if requests fail.

✅

Key Takeaways

Use the OpenAI Python SDK with the computer_20241022 tool to automate software testing tasks.
Always include betas=["computer-use-2024-10-22"] to enable computer use features.
Specify tool details precisely in the tools parameter to avoid errors.
Async calls are supported; streaming is not currently available for computer use.
Check environment variables and network if you encounter authentication or connectivity issues.

Verified 2026-04 · gpt-4o-mini, gpt-4o

Verify ↗