InMemoryChatMessageHistory vs persistent storage
Why this matters
Every production chatbot needs to remember conversations: either within a session or across restarts. Choosing wrong means either losing conversation context or blowing through memory on long-running services.
Explanation
InMemoryChatMessageHistory is a simple in-RAM message store that holds conversation history for the lifetime of your Python process. When the process exits, all history vanishes. Persistent storage (PostgreSQL, SQLite, Redis, or any custom database) survives process restarts and allows sharing conversation state across multiple instances.
Mechanically, InMemoryChatMessageHistory appends messages to a list in memory via add_message(). On retrieval, it returns that list instantly: O(1) lookup. Persistent storage writes to disk/network, then reads back on demand. This is slower but survives shutdown and scales across machines.
Use InMemoryChatMessageHistory for demos, tests, or single-session prototypes. Use persistent storage whenever a human might close their browser tab and return later, or when multiple users share conversation threads.
Analogy
InMemoryChatMessageHistory is like jotting notes on a whiteboard during a conversation: fast and convenient, but erased when you leave the room. Persistent storage is like writing in a notebook you keep forever.
Code
from langchain_core.chat_history import InMemoryChatMessageHistory
from langchain_core.messages import HumanMessage, AIMessage
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.runnables.history import RunnableWithMessageHistory
import json
from datetime import datetime
print("=== InMemoryChatMessageHistory Demo ===")
memory = InMemoryChatMessageHistory()
memory.add_message(HumanMessage(content="What is the capital of France?"))
memory.add_message(AIMessage(content="The capital of France is Paris."))
memory.add_message(HumanMessage(content="And Germany?"))
memory.add_message(AIMessage(content="The capital of Germany is Berlin."))
print("\nMessages in memory:")
for msg in memory.messages:
print(f" {msg.__class__.__name__}: {msg.content}")
print(f"\nTotal messages: {len(memory.messages)}")
print(f"Memory object ID: {id(memory)} (will be same until process exits)")
print("\n=== Persistent Storage Simulation (SQLite-style) ===")
class SqliteChatHistory:
"""Simulates persistent storage behavior"""
def __init__(self, filepath: str, session_id: str):
self.filepath = filepath
self.session_id = session_id
self.messages = []
self._load_from_disk()
def _load_from_disk(self):
"""In real code, this reads from SQLite/PostgreSQL"""
try:
with open(self.filepath, 'r') as f:
data = json.load(f)
self.messages = data.get(self.session_id, [])
print(f"Loaded {len(self.messages)} messages from disk for session {self.session_id}")
except FileNotFoundError:
print(f"No existing history file, starting fresh")
self.messages = []
def add_message(self, role: str, content: str):
msg = {"role": role, "content": content, "timestamp": datetime.now().isoformat()}
self.messages.append(msg)
self._save_to_disk()
def _save_to_disk(self):
"""In real code, this writes to SQLite/PostgreSQL"""
try:
with open(self.filepath, 'r') as f:
all_data = json.load(f)
except FileNotFoundError:
all_data = {}
all_data[self.session_id] = self.messages
with open(self.filepath, 'w') as f:
json.dump(all_data, f, indent=2)
def get_messages(self):
return self.messages
persistent = SqliteChatHistory(filepath="/tmp/chat_history.json", session_id="user_123")
persistent.add_message(role="user", content="What is Python?")
persistent.add_message(role="assistant", content="Python is a programming language.")
persistent.add_message(role="user", content="Is it good for AI?")
persistent.add_message(role="assistant", content="Yes, Python dominates AI/ML development.")
print("\nMessages in persistent storage:")
for msg in persistent.get_messages():
print(f" [{msg['role']}]: {msg['content']}")
print(f"\nAll messages saved to disk at: /tmp/chat_history.json")
print("If this process crashes, messages are still there.")
print("\n=== Key Differences ===")
print(f"InMemory - Lookup: O(1), Survives restart: No, Multi-process: No")
print(f"Persistent - Lookup: O(n), Survives restart: Yes, Multi-process: Yes") === InMemoryChatMessageHistory Demo === Messages in memory: HumanMessage: What is the capital of France? AIMessage: The capital of France is Paris. HumanMessage: And Germany? AIMessage: The capital of Germany is Berlin. Total messages: 4 Memory object ID: 140234567890123 (will be same until process exits) === Persistent Storage Simulation (SQLite-style) === No existing history file, starting fresh Messages in persistent storage: [user]: What is Python? [assistant]: Python is a programming language. [user]: Is it good for AI? [assistant]: Yes, Python dominates AI/ML development. All messages saved to disk at: /tmp/chat_history.json If this process crashes, messages are still there. === Key Differences === InMemory - Lookup: O(1), Survives restart: No, Multi-process: No Persistent - Lookup: O(n), Survives restart: Yes, Multi-process: Yes
What just happened?
The code demonstrated two message storage patterns: (1) InMemoryChatMessageHistory added 4 messages to a list in RAM and retrieved them instantly. (2) SqliteChatHistory simulated persistent storage by writing each message to a JSON file on disk, then loading that file on initialization: demonstrating that data survives process restarts. The output shows both store the same data, but with different durability guarantees.
Common gotcha
Developers often assume InMemoryChatMessageHistory is 'good enough' for production because it works in dev/test. Then a container restarts, the conversation history vanishes, and the user gets a confusing blank slate. Always check your deployment model: if the process can restart or scale horizontally, use persistent storage.
Error recovery
RuntimeError: Chat history not foundFileNotFoundError in persistent storageDatabase connection timeoutExperienced dev note
The hidden cost of InMemoryChatMessageHistory in production is memory bloat on long-running services. Each conversation accumulates tokens: 10 messages × 500 tokens/message = 5K tokens. Scale to 1,000 concurrent users × 100 conversations each = 500M tokens in RAM. This kills your service. Also, InMemoryChatMessageHistory doesn't scale across load balancers: user A's conversation on server 1 is invisible to server 2. Always start with persistent storage if you're deploying to Kubernetes, serverless, or anything multi-instance.
Check your understanding
You have a chatbot running on Kubernetes with 3 replicas. A user talks to the bot on pod-1, then their next request lands on pod-2. Which storage option loses the conversation history, and why?
Show answer hint
The correct answer recognizes that InMemoryChatMessageHistory stores data in pod-1's process memory only. Pod-2 has no access to it. Persistent storage solves this because all pods read/write to the same backend (database).