Agent scratchpad: how the agent thinks
Why this matters
Understanding scratchpad mechanics lets you debug why agents make wrong decisions, design better prompts for agent reasoning, and know when an agent is getting confused or stuck in loops.
Explanation
An agent scratchpad is the internal working memory that LangGraph and agent loops use to track an agent's chain of thought. As an agent thinks step-by-step: deciding what tool to call, observing the result, deciding the next action: all of this gets written into the scratchpad. The scratchpad is then passed back to the LLM in the next iteration so the agent can see what it has already tried and reason about what to do next.
Mechanically, the scratchpad accumulates text as AgentAction (what the agent decided to do) and AgentObservation (what the tool returned). Before each LLM call, the scratchpad is prepended to the prompt so the LLM has full context of the conversation and prior attempts. This is why longer agent runs produce longer context windows: the scratchpad grows with each step.
You should care about scratchpad content when debugging agent behavior. If an agent repeats the same action twice, or misses an obvious next step, looking at the scratchpad reveals exactly what the agent saw and what reasoning led to that decision. Modern LangGraph agents handle scratchpad automatically, but you can inspect and even customize it for advanced use cases.
Analogy
Imagine a scientist working through a lab experiment. The scratchpad is their lab notebook: each observation gets written down, each failed attempt is recorded, so when they flip back through the notebook they see the full chain of what they've tried. Without the notebook, they might try the same failed experiment twice. The LLM is the scientist; the scratchpad is the notebook.
Code
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain.agents import AgentExecutor, create_tool_calling_agent
from langchain_core.tools import tool
import json
@tool
def add_numbers(a: int, b: int) -> int:
"""Add two numbers together."""
return a + b
@tool
def multiply_numbers(a: int, b: int) -> int:
"""Multiply two numbers together."""
return a * b
tools = [add_numbers, multiply_numbers]
llm = ChatOpenAI(model="gpt-4o", temperature=0)
prompt = ChatPromptTemplate.from_messages([
("system", "You are a math assistant. Solve math problems step by step using the available tools."),
("user", "{input}"),
MessagesPlaceholder(variable_name="agent_scratchpad"),
])
agent = create_tool_calling_agent(llm, tools, prompt)
executor = AgentExecutor(agent=agent, tools=tools, verbose=True, max_iterations=5)
result = executor.invoke({
"input": "What is (5 + 3) multiplied by 2?"
})
print("\n--- Final Answer ---")
print(result["output"])> Entering new AgentExecutor...
> Entering new AgentExecutor...
Invoking: add_numbers with {'a': 5, 'b': 3}
response_metadata={'token_usage': {'completion_tokens': 26, 'prompt_tokens': 152, 'total_tokens': 178}, 'model_name': 'gpt-4o', 'system_fingerprint': 'fp_...'}
8
Invoking: multiply_numbers with {'a': 8, 'b': 2}
16
> Finished AgentExecutor...
--- Final Answer ---
(5 + 3) multiplied by 2 equals 16. What just happened?
The agent executor ran two iterations. First, the agent's scratchpad was empty, so it saw the original question and decided to call <code>add_numbers(5, 3)</code>. That result (8) got added to the scratchpad as an observation. On the second iteration, the scratchpad now contained the observation from step 1, allowing the agent to see that 5 + 3 = 8 and then decide to multiply 8 * 2. The scratchpad grew with each step, keeping the agent's context intact.
Common gotcha
Developers often expect the scratchpad to persist between separate executor.invoke() calls, but it doesn't. Each invocation starts with an empty scratchpad. If you need multi-turn agent conversations, you must manually pass message history through the prompt's MessagesPlaceholder, not rely on scratchpad state surviving across invocations.
Error recovery
Agent gets stuck in a loopagent_scratchpad variable not foundScratchpad context is hugeExperienced dev note
The scratchpad is not magic: it's just text concatenation. When you see an agent 'thinking,' what you're really seeing is the LLM reading back its own prior outputs (actions and observations) formatted into the prompt. This means two things: (1) the agent's reasoning is only as good as the clarity of what you put in the scratchpad, so tool output should be informative, not verbose, and (2) you can customize scratchpad formatting if the default agent loop doesn't fit your use case. Most developers miss that they can inject custom formatting logic into the scratchpad to steer agent behavior.
Check your understanding
Why would an agent benefit from having a scratchpad, and what specific problem occurs if you remove it from the prompt template entirely?
Show answer hint
A correct answer must explain that without scratchpad, the LLM loses context of prior tool calls and observations, leading to either repeated actions or inability to build on prior results. The key insight is that the scratchpad is the mechanism by which the agent remembers its own past within a single run.
AgentExecutor with initialize_agent() and managed scratchpad implicitly through chain objects. The current pattern (langchain 1.2.x) uses create_tool_calling_agent + AgentExecutor with explicit MessagesPlaceholder for scratchpad, which gives you more control over formatting.