How to beginner · 4 min read

How to use Groq with LangChain

Q: How to use Groq with LangChain

Use the openai Python SDK with base_url="https://api.groq.com/openai/v1" to connect to Groq's API, then wrap the client with LangChain's ChatOpenAI class by specifying the client parameter. This enables you to use Groq models like llama-3.3-70b-versatile seamlessly within LangChain workflows.

Quick answer

Use the openai Python SDK with base_url="https://api.groq.com/openai/v1" to connect to Groq's API, then wrap the client with LangChain's ChatOpenAI class by specifying the client parameter. This enables you to use Groq models like llama-3.3-70b-versatile seamlessly within LangChain workflows.

PREREQUISITES

Python 3.8+
Groq API key (set as GROQ_API_KEY environment variable)
pip install openai>=1.0 langchain_openai

Setup

Install the required Python packages and set your Groq API key as an environment variable.

Install packages: pip install openai langchain_openai
Set environment variable: export GROQ_API_KEY='your_api_key' (Linux/macOS) or set GROQ_API_KEY=your_api_key (Windows)

bash

pip install openai langchain_openai

Step by step

This example shows how to create an OpenAI-compatible client for Groq, then use it with LangChain's ChatOpenAI to generate chat completions.

python

import os
from openai import OpenAI
from langchain_openai import ChatOpenAI

# Initialize Groq client with base_url override
client = OpenAI(api_key=os.environ["GROQ_API_KEY"], base_url="https://api.groq.com/openai/v1")

# Wrap Groq client in LangChain ChatOpenAI
chat = ChatOpenAI(model_name="llama-3.3-70b-versatile", client=client)

# Define prompt messages
messages = [{"role": "user", "content": "Explain the benefits of LangChain."}]

# Generate response
response = chat.invoke(messages)
print(response.content)

output

LangChain enables easy integration of LLMs with chains, agents, and memory, simplifying complex workflows and accelerating development.

Common variations

You can use async calls with LangChain by importing AsyncChatOpenAI and awaiting invoke_async. To use streaming, LangChain supports streaming tokens with callbacks. You can also switch Groq models by changing model_name to other Groq-supported models like llama-3.1-8b-instant.

python

import asyncio
from langchain_openai import AsyncChatOpenAI
from openai import OpenAI

async def main():
    client = OpenAI(api_key=os.environ["GROQ_API_KEY"], base_url="https://api.groq.com/openai/v1")
    chat = AsyncChatOpenAI(model_name="llama-3.3-70b-versatile", client=client)
    messages = [{"role": "user", "content": "What is Groq?"}]
    response = await chat.invoke_async(messages)
    print(response.content)

asyncio.run(main())

output

Groq is a high-performance AI hardware and software platform optimized for large language models and fast inference.

Troubleshooting

If you get authentication errors, verify your GROQ_API_KEY environment variable is set correctly.
For connection errors, ensure your network allows HTTPS requests to https://api.groq.com.
If the model is not found, confirm you are using a valid Groq model name like llama-3.3-70b-versatile.

✅

Key Takeaways

Use the OpenAI SDK with base_url set to Groq's endpoint to access Groq models.
LangChain's ChatOpenAI accepts a custom client for seamless Groq integration.
Async and streaming calls are supported via LangChain's async interfaces.
Always set your Groq API key in the environment variable GROQ_API_KEY.
Verify model names and network connectivity to avoid common errors.

Verified 2026-04 · llama-3.3-70b-versatile, llama-3.1-8b-instant

Verify ↗