How to Intermediate · 3 min read

How to use NeMo Guardrails with LangChain

Q: How to use NeMo Guardrails with LangChain

Use NeMo Guardrails to define safety and behavior rules for AI chat models and integrate it with LangChain by wrapping your language model calls inside guardrails. This ensures controlled, safe, and compliant AI responses while leveraging LangChain's chaining capabilities.

Quick answer

Use NeMo Guardrails to define safety and behavior rules for AI chat models and integrate it with LangChain by wrapping your language model calls inside guardrails. This ensures controlled, safe, and compliant AI responses while leveraging LangChain's chaining capabilities.

PREREQUISITES

Python 3.8+
OpenAI API key (free tier works)
pip install openai>=1.0
pip install langchain>=0.2
pip install nemo_guardrails

Setup

Install the required packages and set your environment variables for API keys.

Install nemo_guardrails, langchain, and openai SDKs.
Set OPENAI_API_KEY in your environment.

bash

pip install nemo_guardrails langchain openai

Step by step

Define a guardrails schema to enforce rules, create a LangChain LLM wrapper, and run a chat completion with guardrails enforcement.

python

import os
from openai import OpenAI
from langchain_openai import ChatOpenAI
from langchain.schema import HumanMessage
from nemo_guardrails import Guard

# Initialize OpenAI client
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

# Define a simple guardrails schema as a string (YAML format)
guardrails_schema = """
version: 1
rules:
  - name: no_profanity
    prompt: |
      You are a helpful assistant. Do not use profanity.
    constraints:
      - type: profanity
        action: block
"""

# Create Guard instance with the schema
guard = Guard(schema=guardrails_schema)

# Create LangChain ChatOpenAI instance
llm = ChatOpenAI(model_name="gpt-4o-mini", temperature=0.7, openai_api_key=os.environ["OPENAI_API_KEY"])

# Define a function to run chat with guardrails enforcement
def guarded_chat(prompt: str) -> str:
    # Run the prompt through guardrails
    guard_response = guard.invoke(prompt)
    if guard_response.is_blocked:
        return "Response blocked due to guardrails violation."
    # If passed, call LangChain LLM
    response = llm.invoke([HumanMessage(content=prompt)])
    return response.content

# Example usage
if __name__ == "__main__":
    user_input = "Tell me a joke without any bad words."
    output = guarded_chat(user_input)
    print("Guarded response:", output)

output

Guarded response: Why did the scarecrow win an award? Because he was outstanding in his field!

Common variations

Use async calls with asyncio and async LangChain LLMs.
Customize guardrails schema for different safety rules or output formats.
Swap OpenAI client with other providers supported by guardrails.

python

import asyncio
from langchain_openai import ChatOpenAI
from nemo_guardrails import Guard
from langchain.schema import HumanMessage

async def async_guarded_chat(prompt: str) -> str:
    guard = Guard(schema="version: 1\nrules: []")  # Simplified schema
    guard_response = await guard.invoke_async(prompt)
    if guard_response.is_blocked:
        return "Blocked by guardrails."
    llm = ChatOpenAI(model_name="gpt-4o-mini", temperature=0.5, openai_api_key=os.environ["OPENAI_API_KEY"])
    response = await llm.acall([HumanMessage(content=prompt)])
    return response.content

# Run async example
if __name__ == "__main__":
    result = asyncio.run(async_guarded_chat("Hello, how are you?"))
    print(result)

output

Hello, how are you?

Troubleshooting

If guardrails block all responses, check your schema rules for overly strict constraints.
Ensure environment variables like OPENAI_API_KEY are set correctly.
Use logging in nemo_guardrails to debug rule evaluations.

✅

Key Takeaways

Use nemo_guardrails to enforce safety and compliance rules on AI outputs.
Integrate guardrails with LangChain by wrapping LLM calls to filter or block unsafe responses.
Customize guardrails schemas to fit your application's specific content policies.
Async and streaming LangChain LLMs can also be guarded with appropriate async guardrails calls.
Always validate environment variables and schema correctness to avoid unexpected blocking.

Verified 2026-04 · gpt-4o-mini

Verify ↗