How do you make Spring AI chat memory durable in production using JDBC or Cassandra ChatMemoryRepository, and what operational concerns arise?

## Why the default isn't production-grade Out of the box `MessageWindowChatMemory` uses `InMemoryChatMemoryRepository` — a `ConcurrentHashMap`. That means: - **Volatile**: every conversation is lost on restart/redeploy. - **Not shared**: instance A can't see a conversation started on instance B (breaks load-balanced/multi-instance deployments). - **Unbounded**: no cross-conversation cap or expiry; memory leaks as conversations accumulate. Production replaces the repository with a durable `ChatMemoryRepository`. ## JDBC persistence — `JdbcChatMemoryRepository` Add the Boot starter: ``` spring-ai-starter-model-chat-memory-repository-jdbc ``` Spring Boot auto-configuration builds a `JdbcChatMemoryRepository` on top of your existing `DataSource`/`JdbcTemplate`. Messages are stored in a relational table (default name **`SPRING_AI_CHAT_MEMORY`**) with columns for conversation id, message content, message type/role, and a timestamp for ordering. **Schema initialization** is controlled by: ```properties spring.ai.chat.memory.repository.jdbc.initialize-schema=always | embedded | never ``` - `always` — run the bundled DDL on startup (handy for dev). - `embedded` — only for embedded databases (H2, etc.). - `never` — you manage DDL yourself (typical for prod, where Liquibase/Flyway owns schema). Spring AI ships dialect-specific schema scripts. In a Liquibase/Flyway shop you'd set `never` and add the table to your migrations so schema is versioned and reviewed. ## Cassandra persistence — `CassandraChatMemoryRepository` Add: ``` spring-ai-starter-model-chat-memory-repository-cassandra ``` This auto-configures a `CassandraChatMemoryRepository`, a wide-column store well suited to append-heavy, high-write conversational workloads partitioned by conversation id. Its standout feature is native **TTL (time-to-live)**: ```properties spring.ai.chat.memory.repository.cassandra.time-to-live=PT24H ``` Rows automatically expire after the TTL, so abandoned conversations self-clean — no batch pruning job needed. This is a real operational advantage over JDBC. ## Wiring it in Whichever repository you pick, it plugs into the same policy layer: ```java ChatMemory chatMemory = MessageWindowChatMemory.builder() .chatMemoryRepository(jdbcChatMemoryRepository) // or Cassandra .maxMessages(20) .build(); ``` Windowing (policy) and persistence (storage) stay independent — you get durable, shared storage *and* a bounded prompt. ## Operational concerns (the principal-level meat) 1. **Unbounded growth / retention.** The window caps what's *replayed*, not what's *stored*. JDBC keeps every persisted message forever unless you prune. You need a retention policy: a scheduled delete, or use Cassandra TTL. Otherwise the table grows without bound. 2. **Horizontal scale.** Persistent, shared storage removes the need for sticky sessions — any instance can resume any conversation by id. This is often the primary reason to persist. 3. **Schema management across envs.** Decide `initialize-schema` per environment; in prod prefer versioned migrations (`never` + Liquibase/Flyway) over auto-DDL for auditability and rollback. 4. **Privacy / PII.** Transcripts often contain personal data. Consider encryption at rest, access controls, a retention limit, and honoring erasure requests via `ChatMemory.clear(conversationId)`. Don't log full transcripts. 5. **Latency on the hot path.** Every turn does a read (load history) and a write (persist new messages). A slow DB adds latency to each model call; size/connection-pool accordingly and keep the window modest. 6. **Concurrency & ordering.** Interleaved requests on one conversationId can race; the timestamp/order column and your id-per-request discipline matter. 7. **Consistency model.** Cassandra is eventually consistent by default — tune consistency level if a just-written turn must be readable immediately by another node. ## When to choose which - **JDBC**: you already run a relational DB, want transactional/queryable transcripts, and will manage retention yourself (or volumes are modest). - **Cassandra**: very high write throughput, many conversations, and you want automatic expiry via TTL and easy horizontal scale. - **In-memory**: only demos, tests, or single-instance ephemeral use.