Memory Patterns: Structured and Procedural Memory
Intent
Beyond episodic records, agents benefit from semantic memory (facts in databases or knowledge graphs queried by tools) and procedural memory (reusable skills).
Introduction
While episodic memory deals with retrieving *what happened* in past runs, sophisticated agents require distinct architectural stores for *factual knowledge* (semantic memory) and *know-how capabilities* (procedural memory). In cognitive science, these divisions map to how the human brain organizes facts separate from skills like riding a bike. The Structured and Procedural Memory pattern optimizes agent architecture by matching memory storage to the specific access patterns the model demands: semantic facts are queried using structured queries, while procedural skills are called by name and executed.
Matching Memory to Access Patterns
An optimized cognitive agent structures its externalized memory along three distinct lines:
- Semantic Memory (Structured Facts): Facts, definitions, schemas, and relational data. These are stored in relational databases, document stores, or knowledge graphs. The agent queries them exactly using tools (e.g., executing SQL or Cypher) rather than relying on loose vector similarity search.
- Procedural Memory (Reusable Skills): Code libraries and verified executable actions. When an agent creates a successful workflow (e.g., writing a specific script to scrape data), it writes this capability to a skill library. Future runs retrieve this verified skill by name and execute it in a sandbox.
Why Skill Libraries (Procedural) Maximize Efficiency
In standard agents, when a task requires writing code to solve a problem (e.g. converting 100 PDFs into Markdown files), the model writes a new Python script from scratch. This introduces multiple round-trips as the agent encounters execution syntax errors, runs debug traces, and makes code fixes. This is highly inefficient.
Under a procedural framework like Voyager [1], once the agent writes a working PDF-to-Markdown script, the harness saves this block in a verified skill library. When the next task requires parsing PDFs, the agent does not rewrite the code. Instead, it retrieves the verified script by name, passes the arguments, and executes it directly inside a sandbox, skipping coding errors and round-trips entirely.
Trade-offs
Dividing memory systems along cognitive science lines yields high performance, but introduces structural complexity:
High Retrieval Precision
Matching the query pattern (exact lookup for databases, by-name execution for skills) eliminates retrieval noise and irrelevant summaries.
Reduced Round-Trips
Invoking pre-verified code skills from procedural memory bypasses the need to write and debug scripts repeatedly.
High Development Overhead
Setting up relational/graph databases, defining strict tool schemas, and maintaining code libraries requires significant engineering overhead.
Execution Security Risks
Storing and running arbitrary code skills from procedural memory mandates secure execution sandbox environments to prevent system compromise.
Known Uses
Structured and procedural architectures power specialized lifelong learning agents:
- Voyager: An open-ended Minecraft agent that continuously explores, learns, and commits verified JavaScript skills to a library [1].
- Cognitive Architectures: Classical AI setups (such as SOAR or ACT-R) and contemporary LLM equivalents mapping memory types onto cognitive counterparts [2].
References
-
[1]
Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., & Anandkumar, A. (2024). Voyager: An Open-Ended Embodied Agent with Large Language Models. Transactions on Machine Learning Research (TMLR).
https://arxiv.org/abs/2305.16291 -
[2]
Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2024). Cognitive Architectures for Language Agents. Transactions on Machine Learning Research (TMLR).
https://arxiv.org/abs/2309.02427