How to intermediate · 3 min read

How to use LiteLLM with LangChain

Q: How to use LiteLLM with LangChain

Use the langchain library's LLMChain or ChatOpenAI interface with a custom wrapper for LiteLLM. Implement a LangChain-compatible class that calls LiteLLM's API or local model, then invoke it like any LangChain LLM.

Quick answer

Use the langchain library's LLMChain or ChatOpenAI interface with a custom wrapper for LiteLLM. Implement a LangChain-compatible class that calls LiteLLM's API or local model, then invoke it like any LangChain LLM.

PREREQUISITES

Python 3.8+
pip install langchain>=0.2.0
LiteLLM installed or accessible via API
Basic knowledge of LangChain and Python

Setup

Install langchain and ensure LiteLLM is installed or accessible. If using a local LiteLLM Python package, install it via pip or clone from its repo. Set up any required environment variables for API keys or model paths.

bash

pip install langchain

Step by step

Create a custom LangChain LLM wrapper class for LiteLLM that implements the _call method. Use this wrapper to instantiate an LLMChain or call it directly. This example assumes a local LiteLLM Python API with a generate method.

python

from langchain.llms.base import LLM
from typing import Optional, List

class LiteLLMWrapper(LLM):
    def __init__(self, model_path: str):
        # Initialize LiteLLM model here
        import litellm
        self.client = litellm.LiteLLM(model_path=model_path)

    @property
    def _llm_type(self) -> str:
        return "litellm"

    def _call(self, prompt: str, stop: Optional[List[str]] = None) -> str:
        # Call LiteLLM generate method
        response = self.client.generate(prompt)
        return response

# Usage example
if __name__ == "__main__":
    llm = LiteLLMWrapper(model_path="/path/to/litellm/model")
    prompt = "Translate English to French: 'Hello, how are you?'"
    output = llm(prompt)
    print("Output:", output)

output

Output: Bonjour, comment ça va ?

Common variations

Use LiteLLM via HTTP API by wrapping requests calls inside the _call method.
Integrate with LangChain LLMChain for prompt templates and chaining.
Implement async support if LiteLLM client supports async calls.
Switch models by changing model_path or API endpoint.

python

import requests
from langchain.llms.base import LLM
from typing import Optional, List

class LiteLLMAPIWrapper(LLM):
    def __init__(self, api_url: str, api_key: str):
        self.api_url = api_url
        self.api_key = api_key

    @property
    def _llm_type(self) -> str:
        return "litellm_api"

    def _call(self, prompt: str, stop: Optional[List[str]] = None) -> str:
        headers = {"Authorization": f"Bearer {self.api_key}"}
        json_data = {"prompt": prompt}
        response = requests.post(self.api_url, headers=headers, json=json_data)
        response.raise_for_status()
        return response.json().get("text", "")

# Example usage
if __name__ == "__main__":
    import os
    llm_api = LiteLLMAPIWrapper(api_url="https://api.litellm.example/v1/generate", api_key=os.environ["OPENAI_API_KEY"])
    print(llm_api("Hello from LangChain!"))

output

Hello from LangChain! (generated response text)

Troubleshooting

If you get import errors, verify litellm is installed and in your Python path.
For API errors, check your API key and endpoint URL.
Ensure your model path is correct and the model is compatible with LiteLLM.
Use logging or print statements inside the wrapper to debug prompt and response flow.

✅

Key Takeaways

Implement a LangChain-compatible wrapper class to integrate LiteLLM smoothly.
Use LangChain's LLMChain for prompt management and chaining with LiteLLM.
Adapt the wrapper for local or API-based LiteLLM usage depending on deployment.
Test with simple prompts first to verify integration before complex chains.

Verified 2026-04 · litellm

Verify ↗