How to use Together AI for AI agents
Quick answer
Use the
openai Python SDK with base_url="https://api.together.xyz/v1" and your TOGETHER_API_KEY to create chat completions with Together AI models. Call client.chat.completions.create() with your chosen model and messages to build AI agents.PREREQUISITES
Python 3.8+Together AI API key (set TOGETHER_API_KEY in environment)pip install openai>=1.0
Setup
Install the openai Python package and set your Together AI API key as an environment variable.
- Install SDK:
pip install openai - Set environment variable:
export TOGETHER_API_KEY="your_api_key"(Linux/macOS) orset TOGETHER_API_KEY=your_api_key(Windows)
pip install openaiStep by step
Use the OpenAI-compatible SDK with Together AI's base URL to create a simple AI agent chat completion.
import os
from openai import OpenAI
client = OpenAI(api_key=os.environ["TOGETHER_API_KEY"], base_url="https://api.together.xyz/v1")
messages = [
{"role": "user", "content": "Hello, how can you assist me today?"}
]
response = client.chat.completions.create(
model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
messages=messages
)
print(response.choices[0].message.content) output
Hello! I can help you with a wide range of tasks including answering questions, generating text, and more.
Common variations
You can use different Together AI models by changing the model parameter. For asynchronous calls, use async with await. Streaming responses are supported by setting stream=True in chat.completions.create().
import asyncio
import os
from openai import OpenAI
async def async_chat():
client = OpenAI(api_key=os.environ["TOGETHER_API_KEY"], base_url="https://api.together.xyz/v1")
messages = [{"role": "user", "content": "Tell me a joke."}]
stream = await client.chat.completions.acreate(
model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
messages=messages,
stream=True
)
async for chunk in stream:
delta = chunk.choices[0].delta.content or ""
print(delta, end="", flush=True)
asyncio.run(async_chat()) output
Why did the scarecrow win an award? Because he was outstanding in his field!
Troubleshooting
- If you get authentication errors, verify your
TOGETHER_API_KEYenvironment variable is set correctly. - For model not found errors, confirm the model name is valid and available on Together AI.
- Timeouts may require retry logic or checking network connectivity.
Key Takeaways
- Use the OpenAI-compatible
openaiSDK with Together AI'sbase_urlfor seamless integration. - Set your API key in the environment variable
TOGETHER_API_KEYto authenticate requests. - Support for async and streaming enables responsive AI agent implementations.
- Switch models easily by changing the
modelparameter to fit your agent's needs.