skip to content

Memory & State

5 roadmaps5 questions

You will learn how conversation state carries between turns: buffer, summary and token-buffer memories, message stores, wiring history into LCEL chains with RunnableWithMessageHistory, and persistence backends. Interviewers ask because memory is really a context-window budget problem, and they want to hear how you trade fidelity against tokens and latency.

part ofAI agent & RAG frameworksoverview, primer and where to startread it →
on this pageshow

explore

questions

5

In LangChain, what does a BaseChatMessageHistory store, and when does in-memory storage break?

level: juniorimportance: must knowfreq 62%

answer

  1. one conversation, one log
  2. messages, add_messages, clear
  3. session id is the key
  4. in-memory dies with the process
  5. replicas do not share a Python dict

basics

~20 s

A chat message history holds one conversation's ordered messages, keyed by a session id, through the messages property plus add_messages and clear. InMemoryChatMessageHistory keeps that list in process RAM, so a restart or a second replica loses the conversation.

solid answer

~40 s

`BaseChatMessageHistory` in `langchain_core.chat_history` is the interface for a single conversation's message log: a `messages` property returning `list[BaseMessage]`, `add_messages()` to append, helpers `add_user_message()` / `add_ai_message()`, and `clear()`, with async variants alongside. `InMemoryChatMessageHistory` is the reference implementation — a plain Python list on the instance, usually behind a module-level dict keyed by session id. Fine for notebooks and tests, but it dies with the process, is invisible to any other replica behind a load balancer, and grows without bound in RAM. For real traffic you swap in a backed implementation such as `SQLChatMessageHistory` or `RedisChatMessageHistory` from `langchain-community` and partner packages, or subclass the interface yourself. The abstraction never trims, summarizes, or scopes by user: it is an append-only message log and nothing more.

code

python · 13 lines
python
from langchain_core.chat_history import InMemoryChatMessageHistory
from langchain_core.messages import AIMessage, HumanMessage, messages_to_dict

store: dict[str, InMemoryChatMessageHistory] = {}

def get_session_history(session_id: str) -> InMemoryChatMessageHistory:
    return store.setdefault(session_id, InMemoryChatMessageHistory())

history = get_session_history("user-1")
history.add_messages([HumanMessage("hi"), AIMessage("hello")])
print([m.content for m in history.messages])
print(messages_to_dict(history.messages))
history.clear()

go deeper

for a junior

Know that a chat message history is one conversation's message list keyed by session id, and be able to name messages, add_messages and clear.

for a middle

Explain why the store holds message objects rather than text, and why an in-process dict fails on restart and across replicas.

for a senior

Discuss bounding reads, concurrent appends on one session, and deriving the session key from the authenticated user rather than the request body.

for a principal

Own the retention side: chat logs are user data, so TTL, encryption and a reachable delete path are design inputs, not afterthoughts.

## The abstraction `BaseChatMessageHistory` (in `langchain_core.chat_history`) is LangChain's smallest memory unit: the message log of one conversation. The contract is deliberately tiny. - `messages` — property returning `list[BaseMessage]` in chronological order (`SystemMessage`, `HumanMessage`, `AIMessage`, `ToolMessage`). - `add_messages(messages)` — append a batch. - `add_user_message(text)` / `add_ai_message(text)` — convenience wrappers that build the message for you. - `clear()` — drop the conversation. - async counterparts (`aget_messages`, `aadd_messages`, `aclear`) so a network-backed store does not block the event loop. Everything else in LangChain's memory story — wiring history into a chain, trimming it, summarizing it — sits on top of this one interface. ## Why messages, not strings History is stored as message objects, not concatenated text. That matters because a turn can carry more than words: an `AIMessage` may hold `tool_calls`, a `ToolMessage` carries a `tool_call_id`, and any message can carry `additional_kwargs` and `response_metadata`. If you flatten to `str(message)` you lose the structure the model provider needs on the next request. When you serialize, use `messages_to_dict()` and `messages_from_dict()` from `langchain_core.messages` so the round trip is lossless. ## InMemoryChatMessageHistory and its three failure modes The in-process implementation stores a list on the object. Typical demo code keeps a `dict[str, InMemoryChatMessageHistory]` at module scope and looks up by session id. It fails in production for three separate reasons. 1. **Restart amnesia.** Deploys, crashes and autoscaler evictions wipe every live conversation. 2. **Replica affinity.** With two or more pods behind a load balancer, turn two can land on a pod that has never seen turn one. The user sees a model that forgot the last sentence, intermittently — the hardest kind of bug to reproduce. 3. **Unbounded growth.** Nothing evicts sessions. A long-running process accumulates every conversation it ever served until the container is OOM-killed. ## Backed implementations Persistent stores live outside `langchain-core`. `langchain-community` and partner packages ship implementations such as `SQLChatMessageHistory` (any SQLAlchemy URL), `RedisChatMessageHistory`, `FileChatMessageHistory` and document-store variants. They take the session id plus connection details in the constructor and satisfy the same interface, so switching backend is a one-line change at the point where you construct the history object — the chain above it does not change. ## Writing your own Subclass `BaseChatMessageHistory`, implement `messages`, `add_messages` and `clear`, and serialize with `messages_to_dict`. Two design points people miss: give reads a bound (fetch the last N rows, not the whole table, or a two-year-old session will pull thousands of rows on every turn), and think about concurrency — two browser tabs on one session id can interleave appends, and the interface offers no locking. ## What the interface deliberately does not do It does not trim to a token budget, does not summarize, does not scope by user (session id is an opaque string — if you let a client send it raw, one user can read another's conversation), and does not distinguish this conversation from anything the user said last week. Those are all layers you add above it.

  • Why store BaseMessage objects rather than a plain list of strings?
    Because a turn carries structure a string drops: an AIMessage can hold tool_calls, a ToolMessage carries its tool_call_id, and messages carry additional_kwargs and response_metadata. Providers need that structure back on the next request. Serialize with messages_to_dict and messages_from_dict so the round trip preserves it.
  • What goes wrong if the client supplies the session id directly?
    Session id is an opaque string with no authorization attached, so a client that can send an arbitrary value can read someone else's conversation. Derive the key server-side from the authenticated user, and scope the conversation id under it rather than trusting whatever the browser sent.

saying these in an interview costs you the question

  • Thinks the in-memory dict is shared across replicas
  • Stores history as concatenated strings, losing tool calls
  • Assumes the history object trims itself to a token budget
  • Lets the client choose its own session id
  • Cannot name a persistent backend for chat history

context

resolving links…

open as a page

In LangChain, how do buffer, summary and token-buffer conversation memories differ?

level: middleimportance: must knowfreq 60%

basics

~20 s

Buffer memory replays every turn verbatim, so fidelity is perfect and tokens grow without limit. Summary memory keeps a running LLM-written summary, so the prompt stays flat but each turn costs an extra model call and loses detail. Token-buffer memory keeps the newest messages that fit a token limit.

open as a page

How does RunnableWithMessageHistory feed prior turns into a LangChain LCEL chain?

level: middleimportance: must knowfreq 68%

basics

~20 s

RunnableWithMessageHistory wraps a runnable. On each call it uses the session id from config['configurable'] to fetch that conversation's history, prepends the stored messages to the input, invokes the inner chain, then appends the new input and output messages back to the store.

open as a page

Your LangChain chat sessions overflow the context window; how do you bound history?

level: seniorimportance: must knowfreq 56%

basics

~20 s

Insert trim_messages from langchain_core.messages as a step before the prompt, so every invocation cuts the loaded history to a token budget. Keep the system message pinned, trim from the oldest end, and start the kept window on a human message so tool replies are never orphaned.

open as a page

How do you choose where LangChain conversation history lives for a multi-replica service?

level: principalimportance: should knowfreq 40%

basics

~20 s

Treat the get_session_history factory as the seam and pick the backend from the requirements behind it: latency and TTL push toward a cache, durability and audit toward a database. Key by authenticated user plus conversation, bound each read, and set retention and deletion policy alongside.

open as a page