MemPalace: Revolutionizing AI Memory Beyond RAG

MemPalace: A New Approach to AI Memory

What is MemPalace?

MemPalace presents itself as an innovative solution for memory management in artificial intelligence systems. Unlike conventional methods, this open-source system focuses on the local storage of conversations and project data in their original form. Each message is treated as a distinct memory unit, allowing for organized and durable recall.

The structure of MemPalace is inspired by the mnemonic model of the "memory palace." This hierarchical model consists of several levels: Wings for people or projects, Rooms for subjects, Halls for types of memory, and Drawers for transcriptions, with Cabinets for summaries.

A Distinct Approach from Traditional Memory Systems

Traditional memory systems, such as RAG pipelines or vector databases, prioritize retrieval efficiency, often at the expense of contextual richness. These systems segment data, create embeddings, and retrieve similar segments during inference.

MemPalace adopts a different approach by preserving the entirety of information in its original form, rather than limiting itself to embeddings. This hierarchical method enhances contextual understanding and employs a combination of symbolic structure and vector search to connect different knowledge systems. As a result, MemPalace offers superior reasoning capabilities and better traceability compared to conventional memory systems.

The Main Idea: Verbatim Memory vs. Summary

Most memory tools for AI agents use a language model (LLM) to summarize or extract key facts from conversations. Tools like Mem0 and Zep analyze discussions to produce concise reports, but this approach can lead to a loss of contextual information and subtle details. Indeed, an LLM must determine what is "important" and discard the rest.

MemPalace takes the opposite strategy by choosing to "store everything." The system retains a complete record of all messages exchanged between users and assistants, without summarization or deletion. This raw storage method has several significant advantages:

• Complete Context: By maintaining full access to all conversation details, the AI can reconstruct the entirety of the dialogue.

• Superior Recall: With its comprehensive database, MemPalace achieves exceptional accuracy in information retrieval, boasting a 96.6% recall@5 on LongMemEval, which includes 500 questions.

• Traceability: By retaining everything, the system allows users to verify responses against the original discussion logs.

Exploring the Architecture of MemPalace

The design of MemPalace is based on the ancient mnemonic method of loci, creating a multi-level framework that facilitates the location and access of stored memories. Here’s an overview of its hierarchical structure and data processing system.

The Hierarchical Design of the "Palace" Memory

• Wings (Project-Level Segmentation): Wings define main divisions that encompass entire domains or projects, allowing for the separation of personal memories from team memories. Subjects within a wing are organized into specific Rooms after the wings are defined.

• Rooms (Subject-Level Organization): Rooms serve as spaces that connect all existing subjects within a wing. For example, the "Work" wing may contain rooms named "Meetings," "Projects," and "Emails." Each document or conversation is assigned to a specific combination of wing and room.

• Halls (Types of Memory: Facts, Events, Preferences): Across all wings, common halls classify types of memory. MemPalace defines halls as hall_facts, hall_events, hall_discoveries, hall_preferences, and hall_advice. For instance, a project decision is stored in the hall_facts of its room, while a meeting summary goes into hall_events.

• Drawers (Raw Verbatim Storage): Each memory piece is stored in a specific drawer, containing a text file with the complete transcription of a discussion or email, or a code file as it was recorded. Drawers function as unaltered archives, preserving their content in its original form.

• Cabinets (Compressed Representations): Cabinets contain the compressed AAAK (or "summary") representation of each drawer. They direct users to the original content of their drawer, functioning as a compact index.

Storage and Retrieval Pipeline

The MemPalace pipeline consists of two main components: write memory for ingestion and read memory for retrieval at query time.

• Verbatim Storage (Ingestion): When a conversation or file is extracted, MemPalace writes each message as a new drawer entry in its database. The text is directly stored in a vector store (default: ChromaDB) without filtering by LLM. Metadata, such as wing, room, and hall tags, are attached to allow for subsequent contextual queries.

• Vector Search with ChromaDB: For retrieval, MemPalace uses semantic vector search. Each drawer is embedded and stored in ChromaDB. During a query, the system vectorizes the request and finds the most similar drawers by cosine similarity, typically returning matches in milliseconds.

• Metadata Layer (Knowledge Graph): In addition to raw text, MemPalace builds a temporal knowledge graph in local SQLite. Each fact is stored with validity windows, including temporal relationships and contextual dependencies.

Compression Mechanism (AAAK)

MemPalace offers an optional compression feature called AAAK. This abbreviation system allows for the storage of extensive information with minimal token usage. Lossy compression uses regular expressions to transform words into abbreviations and select key phrases, reducing tokens by approximately 30 times.

• Lossless Compression Strategy: The goal of AAAK is to achieve "lossless" content compression, allowing for the reconstruction of each factual assertion. AAAK aims to provide complete evidence of actions taken, without using proprietary tokenizers or embeddings.

• Token Efficiency and Context Injection: The objective is to maintain efficient compression while allowing for a faithful reconstruction of contextual information.

How MemPalace Works (End-to-End Flow)

MemPalace enables AI agents to maintain permanent memory elements, accessible at any time. The system transforms spoken dialogue into vector representations, stored in ChromaDB. The agent accesses its essential memories when specific information is needed, rather than using its entire memory database.

Data Ingestion (Conversation Extraction)

Data ingestion constitutes the first step. MemPalace listens to each turn of a conversation, capturing user messages, AI responses, and metadata, then prepares this raw text for storage.

• Chunking: MemPalace divides long messages into pieces of 512 tokens with 64 token overlaps, thus avoiding the loss of context at the boundaries of chunks.

• Metadata Tagging: Each piece receives a role (user or assistant), a turn number, a session ID, and a timestamp.

• Deduplication: MemPalace uses deterministic IDs like session-turn-N, allowing the same turn to be rewritten by simply overwriting the existing entry.