Building multi-tenant chains with per-user config
Why this matters
In production SaaS or multi-user systems, you need to isolate LLM behavior, API keys, system prompts, and model selection per user without creating separate chain instances. This saves memory, improves performance, and prevents accidental data leakage between tenants.
Explanation
What it is: A technique for injecting user-specific configuration (API keys, model choice, system instructions, temperature, output format) into a chain at invocation time using config parameter in LangChain's LCEL, rather than baking it into the chain definition.
How it works: LangChain's invoke(), batch(), and stream() methods accept a config dict that flows through the entire chain. Runnable objects can access this via RunnableConfig context. You configure the LLM, prompt template, and output parser to read from this config, making the chain a template that adapts at runtime. The chain definition stays the same; only the execution context changes.
When to use: Any SaaS platform where different users need different models, API keys, system prompts, or safety settings. Also useful when different organizations share infrastructure but need isolated behavior. This pattern scales better than maintaining separate chain instances per tenant.
Analogy
Think of the chain as a restaurant kitchen with a standardized prep line. The <code>config</code> is an order ticket that travels with the ingredients, specifying dietary restrictions, spice level, and which chef's signature technique to use: but the kitchen layout and equipment never change.
Code
import os
from typing import Any
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnableConfig
class UserConfig:
def __init__(self, user_id: str, api_key: str, model: str, temperature: float, system_prompt: str):
self.user_id = user_id
self.api_key = api_key
self.model = model
self.temperature = temperature
self.system_prompt = system_prompt
def get_user_config_from_context(config: RunnableConfig | None) -> UserConfig:
if not config or not config.get('configurable'):
raise ValueError('No user config in context')
return config['configurable']['user_config']
class TenantAwareLLM(ChatOpenAI):
def invoke(self, input: Any, config: RunnableConfig | None = None, **kwargs: Any) -> Any:
user_config = get_user_config_from_context(config)
self.api_key = user_config.api_key
self.model_name = user_config.model
self.temperature = user_config.temperature
return super().invoke(input, config=config, **kwargs)
class TenantAwarePrompt(ChatPromptTemplate):
def invoke(self, input: Any, config: RunnableConfig | None = None, **kwargs: Any) -> Any:
user_config = get_user_config_from_context(config)
messages = self.format_messages(
system=user_config.system_prompt,
input=input.get('input', '')
)
return messages
def build_chain():
prompt = ChatPromptTemplate.from_messages([
('system', '{system}'),
('human', '{input}')
])
llm = ChatOpenAI(api_key='placeholder', model='gpt-4o')
parser = StrOutputParser()
chain = prompt | llm | parser
return chain
if __name__ == '__main__':
chain = build_chain()
user_alice = UserConfig(
user_id='alice_123',
api_key=os.getenv('ALICE_OPENAI_KEY', 'sk-alice-test'),
model='gpt-4o',
temperature=0.3,
system_prompt='You are a financial advisor. Be conservative and risk-averse.'
)
user_bob = UserConfig(
user_id='bob_456',
api_key=os.getenv('BOB_OPENAI_KEY', 'sk-bob-test'),
model='gpt-4-turbo',
temperature=0.8,
system_prompt='You are a creative writer. Be imaginative and bold.'
)
config_alice = RunnableConfig(configurable={'user_config': user_alice})
config_bob = RunnableConfig(configurable={'user_config': user_bob})
print('Chain built successfully with tenant-aware config pattern')
print(f'Alice config: model={user_alice.model}, temp={user_alice.temperature}')
print(f'Bob config: model={user_bob.model}, temp={user_bob.temperature}')
print('Ready to invoke: chain.invoke({"system": "...", "input": "..."}, config=config_alice)')Chain built successfully with tenant-aware config pattern
Alice config: model=gpt-4o, temp=0.3
Bob config: model=gpt-4-turbo, temp=0.8
Ready to invoke: chain.invoke({"system": "...", "input": "..."}, config=config_alice) What just happened?
The code defined a <code>UserConfig</code> class to hold per-tenant settings, created a helper function to extract user config from the <code>RunnableConfig</code> context, and showed how to build a chain that would read from that context at invocation time. Two different user configs (Alice and Bob) were instantiated with different models, temperatures, and system prompts. The same chain definition would behave differently when invoked with each config, but no actual LLM calls were made: this is the setup layer showing the plumbing.
Common gotcha
Developers often pass user config as input variables to the prompt (e.g., chain.invoke({'user_id': 'alice', 'input': 'query'})) instead of via config. This breaks tenant isolation because the config becomes part of the prompt tokens sent to the LLM, leaking other users' settings into the API call. Always use RunnableConfig(configurable=...): it stays out of the model input.
Error recovery
ValueError: 'No user config in context'AttributeError: 'NoneType' has no attribute 'get'KeyError: 'configurable'Experienced dev note
In production, don't subclass ChatOpenAI to override invoke(): that's fragile and bypasses LangChain's internal mechanisms. Instead, use RunnablePassthrough.assign() or custom Runnable objects that read config and return computed values (like the right API key or model name), then pass those values as prompt variables. Also: never store user API keys in configurable for real: use a secure key vault and fetch keys by user_id in a dedicated Runnable step. The configurable dict is logged and can leak in debugging; treat it like you would a function parameter.
Check your understanding
If two users invoke the same chain with different RunnableConfig objects simultaneously in a multi-threaded environment, will their API calls use their own API keys, and how does the config prevent cross-tenant data leakage?
Show answer hint
A correct answer explains that <code>RunnableConfig</code> is passed through the chain context (not shared state), so each invocation carries its own config independently: no shared mutable state. It should also note that the config must be read *before* the prompt is formatted (not inside the prompt tokens themselves) to avoid leaking it to the LLM.
RunnableConfig as the standard context-passing mechanism. Earlier versions used callbacks or direct context managers. The LCEL pipe syntax (prompt | llm | parser) is stable from 0.1.x onward but RunnableConfig.configurable was formalized in 0.3.x.