Code Intermediate medium · 6 min

InMemoryChatMessageHistory vs persistent storage

What you will learn
InMemoryChatMessageHistory stores conversation in RAM and disappears on restart; persistent storage keeps it forever.

Why this matters

Every production chatbot needs to remember conversations: either within a session or across restarts. Choosing wrong means either losing conversation context or blowing through memory on long-running services.

Skip if: Don't use persistent storage for stateless APIs that process one request at a time. Don't use InMemoryChatMessageHistory for multi-user systems or services running longer than a few hours.

Explanation

InMemoryChatMessageHistory is a simple in-RAM message store that holds conversation history for the lifetime of your Python process. When the process exits, all history vanishes. Persistent storage (PostgreSQL, SQLite, Redis, or any custom database) survives process restarts and allows sharing conversation state across multiple instances.

Mechanically, InMemoryChatMessageHistory appends messages to a list in memory via add_message(). On retrieval, it returns that list instantly: O(1) lookup. Persistent storage writes to disk/network, then reads back on demand. This is slower but survives shutdown and scales across machines.

Use InMemoryChatMessageHistory for demos, tests, or single-session prototypes. Use persistent storage whenever a human might close their browser tab and return later, or when multiple users share conversation threads.

Analogy

InMemoryChatMessageHistory is like jotting notes on a whiteboard during a conversation: fast and convenient, but erased when you leave the room. Persistent storage is like writing in a notebook you keep forever.

Code

python
from langchain_core.chat_history import InMemoryChatMessageHistory
from langchain_core.messages import HumanMessage, AIMessage
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.runnables.history import RunnableWithMessageHistory
import json
from datetime import datetime

print("=== InMemoryChatMessageHistory Demo ===")
memory = InMemoryChatMessageHistory()

memory.add_message(HumanMessage(content="What is the capital of France?"))
memory.add_message(AIMessage(content="The capital of France is Paris."))
memory.add_message(HumanMessage(content="And Germany?"))
memory.add_message(AIMessage(content="The capital of Germany is Berlin."))

print("\nMessages in memory:")
for msg in memory.messages:
    print(f"  {msg.__class__.__name__}: {msg.content}")

print(f"\nTotal messages: {len(memory.messages)}")
print(f"Memory object ID: {id(memory)} (will be same until process exits)")

print("\n=== Persistent Storage Simulation (SQLite-style) ===")

class SqliteChatHistory:
    """Simulates persistent storage behavior"""
    def __init__(self, filepath: str, session_id: str):
        self.filepath = filepath
        self.session_id = session_id
        self.messages = []
        self._load_from_disk()
    
    def _load_from_disk(self):
        """In real code, this reads from SQLite/PostgreSQL"""
        try:
            with open(self.filepath, 'r') as f:
                data = json.load(f)
                self.messages = data.get(self.session_id, [])
                print(f"Loaded {len(self.messages)} messages from disk for session {self.session_id}")
        except FileNotFoundError:
            print(f"No existing history file, starting fresh")
            self.messages = []
    
    def add_message(self, role: str, content: str):
        msg = {"role": role, "content": content, "timestamp": datetime.now().isoformat()}
        self.messages.append(msg)
        self._save_to_disk()
    
    def _save_to_disk(self):
        """In real code, this writes to SQLite/PostgreSQL"""
        try:
            with open(self.filepath, 'r') as f:
                all_data = json.load(f)
        except FileNotFoundError:
            all_data = {}
        
        all_data[self.session_id] = self.messages
        
        with open(self.filepath, 'w') as f:
            json.dump(all_data, f, indent=2)
    
    def get_messages(self):
        return self.messages

persistent = SqliteChatHistory(filepath="/tmp/chat_history.json", session_id="user_123")

persistent.add_message(role="user", content="What is Python?")
persistent.add_message(role="assistant", content="Python is a programming language.")
persistent.add_message(role="user", content="Is it good for AI?")
persistent.add_message(role="assistant", content="Yes, Python dominates AI/ML development.")

print("\nMessages in persistent storage:")
for msg in persistent.get_messages():
    print(f"  [{msg['role']}]: {msg['content']}")

print(f"\nAll messages saved to disk at: /tmp/chat_history.json")
print("If this process crashes, messages are still there.")

print("\n=== Key Differences ===")
print(f"InMemory - Lookup: O(1), Survives restart: No, Multi-process: No")
print(f"Persistent - Lookup: O(n), Survives restart: Yes, Multi-process: Yes")
Output
=== InMemoryChatMessageHistory Demo ===

Messages in memory:
  HumanMessage: What is the capital of France?
  AIMessage: The capital of France is Paris.
  HumanMessage: And Germany?
  AIMessage: The capital of Germany is Berlin.

Total messages: 4
Memory object ID: 140234567890123 (will be same until process exits)

=== Persistent Storage Simulation (SQLite-style) ===
No existing history file, starting fresh

Messages in persistent storage:
  [user]: What is Python?
  [assistant]: Python is a programming language.
  [user]: Is it good for AI?
  [assistant]: Yes, Python dominates AI/ML development.

All messages saved to disk at: /tmp/chat_history.json
If this process crashes, messages are still there.

=== Key Differences ===
InMemory - Lookup: O(1), Survives restart: No, Multi-process: No
Persistent - Lookup: O(n), Survives restart: Yes, Multi-process: Yes

What just happened?

The code demonstrated two message storage patterns: (1) InMemoryChatMessageHistory added 4 messages to a list in RAM and retrieved them instantly. (2) SqliteChatHistory simulated persistent storage by writing each message to a JSON file on disk, then loading that file on initialization: demonstrating that data survives process restarts. The output shows both store the same data, but with different durability guarantees.

Common gotcha

Developers often assume InMemoryChatMessageHistory is 'good enough' for production because it works in dev/test. Then a container restarts, the conversation history vanishes, and the user gets a confusing blank slate. Always check your deployment model: if the process can restart or scale horizontally, use persistent storage.

Error recovery

RuntimeError: Chat history not found
Your InMemoryChatMessageHistory instance was garbage-collected or you're accessing a different instance. Store it as an instance variable, not a local variable in a function.
FileNotFoundError in persistent storage
The file path doesn't exist. Create the directory first: `os.makedirs(os.path.dirname(filepath), exist_ok=True)` before writing.
Database connection timeout
Your persistent storage backend (PostgreSQL, Redis) is unreachable. Implement exponential backoff retry logic and fall back to InMemory temporarily, or fail fast with a clear error message.

Experienced dev note

The hidden cost of InMemoryChatMessageHistory in production is memory bloat on long-running services. Each conversation accumulates tokens: 10 messages × 500 tokens/message = 5K tokens. Scale to 1,000 concurrent users × 100 conversations each = 500M tokens in RAM. This kills your service. Also, InMemoryChatMessageHistory doesn't scale across load balancers: user A's conversation on server 1 is invisible to server 2. Always start with persistent storage if you're deploying to Kubernetes, serverless, or anything multi-instance.

Check your understanding

You have a chatbot running on Kubernetes with 3 replicas. A user talks to the bot on pod-1, then their next request lands on pod-2. Which storage option loses the conversation history, and why?

Show answer hint

The correct answer recognizes that InMemoryChatMessageHistory stores data in pod-1's process memory only. Pod-2 has no access to it. Persistent storage solves this because all pods read/write to the same backend (database).

VERSION InMemoryChatMessageHistory is part of langchain-core 0.3.x (April 2026). In langchain < 0.3.0, message history was built into older chain patterns. Always use the LCEL + RunnableWithMessageHistory pattern, not the deprecated AgentExecutor + initialize_agent pattern.
NEXT

Learn how to integrate chat history into your LLM chain using <code>RunnableWithMessageHistory</code> to automatically manage context between turns.

Community Notes

No notes yetBe the first to share a version-specific fix or tip.