How to beginner · 3 min read

How to use Fireworks AI with LangChain

Q: How to use Fireworks AI with LangChain

Use the OpenAI-compatible OpenAI client with base_url="https://api.fireworks.ai/inference/v1" and your FIREWORKS_API_KEY. Then instantiate ChatOpenAI from langchain_openai with the same base URL and API key to run chat completions seamlessly within LangChain.

Quick answer

Use the OpenAI-compatible OpenAI client with base_url="https://api.fireworks.ai/inference/v1" and your FIREWORKS_API_KEY. Then instantiate ChatOpenAI from langchain_openai with the same base URL and API key to run chat completions seamlessly within LangChain.

PREREQUISITES

Python 3.8+
Fireworks AI API key
pip install openai>=1.0 langchain_openai

Setup

Install the required packages and set your environment variable for the Fireworks AI API key.

Install packages: openai and langchain_openai
Set FIREWORKS_API_KEY in your environment

bash

pip install openai langchain_openai

output

Collecting openai
Collecting langchain_openai
Successfully installed openai-1.x langchain_openai-0.x

Step by step

This example shows how to create a LangChain ChatOpenAI client configured to use Fireworks AI's OpenAI-compatible API endpoint. It sends a simple chat message and prints the response.

python

import os
from openai import OpenAI
from langchain_openai import ChatOpenAI

# Initialize OpenAI client with Fireworks AI endpoint
client = OpenAI(
    api_key=os.environ["FIREWORKS_API_KEY"],
    base_url="https://api.fireworks.ai/inference/v1"
)

# Create LangChain ChatOpenAI instance using the same API key and base_url
chat = ChatOpenAI(
    model="accounts/fireworks/models/llama-v3p3-70b-instruct",
    openai_api_key=os.environ["FIREWORKS_API_KEY"],
    openai_api_base="https://api.fireworks.ai/inference/v1"
)

# Run a chat completion
response = chat.invoke([{"role": "user", "content": "Hello from Fireworks AI with LangChain!"}])
print("Response:", response.content)

output

Response: Hello! How can I assist you today?

Common variations

You can use async calls, streaming, or switch models by adjusting the ChatOpenAI parameters. Fireworks AI models always start with accounts/fireworks/models/.

python

import asyncio
from langchain_openai import ChatOpenAI

async def async_chat():
    chat = ChatOpenAI(
        model="accounts/fireworks/models/llama-v3p3-70b-instruct",
        openai_api_key=os.environ["FIREWORKS_API_KEY"],
        openai_api_base="https://api.fireworks.ai/inference/v1"
    )
    response = await chat.ainvoke([{"role": "user", "content": "Async hello!"}])
    print("Async response:", response.content)

asyncio.run(async_chat())

output

Async response: Hello! How can I assist you asynchronously?

Troubleshooting

If you get authentication errors, verify your FIREWORKS_API_KEY is set correctly in your environment.
For model not found errors, ensure the model name starts with accounts/fireworks/models/.
Check your network connectivity to https://api.fireworks.ai/inference/v1.

✅

Key Takeaways

Use the OpenAI-compatible OpenAI client with Fireworks AI's base URL for direct API calls.
Configure ChatOpenAI in LangChain with openai_api_key and openai_api_base to integrate Fireworks AI.
Fireworks AI model names always start with accounts/fireworks/models/.
Async and streaming calls are supported by LangChain's ChatOpenAI with Fireworks AI.
Always set your API key in environment variables to avoid authentication issues.

Verified 2026-04 · accounts/fireworks/models/llama-v3p3-70b-instruct

Verify ↗