One question, two rankings: agent memory with and without HeapDB

A question we keep getting: what does HeapDB actually do for agent memory that a vector store or BM25 index doesn't? This post answers it by running one assistant session twice over the same store and showing both rankings.

The short version: HeapDB is not the memory store and it is not the search. It is a re-ranking step between "find candidate memories" and "put them in the prompt". It decides which of the matching memories go into the context window, says why in one line, and learns from which memories the agent ended up using.

Where it sits in the loop

user turn
   │
   ▼
candidates      BM25 (+ embeddings, optional): top 50 memories by text match
   │
   ▼
re-rank         relevance × recency × importance × learned usefulness × rules     ← HeapDB
   │
   ▼
top k → prompt → model answers, citing the memories it used
   │
   ▼
feedback        mark_used / mark_ignored  →  updates the usefulness links          ← HeapDB

A plain memory layer stops after the first box: the top k by similarity go straight into the prompt, and the last box doesn't exist because there is nowhere to put the signal.

The scenario

A personal assistant has been talking with Maya for a month. Among its stored memories:

Today Maya asks: "When is my flight to Lisbon?" The assistant has room for three memories. The answer is lisbon-booked, which is also the one with the fewest words in common with the question.

Without HeapDB: plain search

The store ranks by text match alone. From the example below:

search  1. 1.000  I have always wanted to see Lisbon, the flights there look amazing
        2. 0.944  Flights to Lisbon usually run $400-700 round trip
        3. 0.583  Booked the flight: SFO to LIS, departs March 14, 9:40am

The answer makes the cut, in third place, behind a wish and a price quote that both say "Lisbon" and "flights". A model will usually still find it. But with k=2, or a few more Lisbon chats in the store, it falls out of the context and the assistant answers from the price quote or says it doesn't know.

More important is what happens next: nothing. Whether the assistant answered well or badly, the next "when is…" question is ranked by exactly the same rule. Similarity measures does this memory talk about the same things? It has no notion of has a memory like this helped with a question like this?

With HeapDB, day one

heapdb-memory takes the same candidates and scores each one:

score = relevance × recency × importance × usefulness × rules

• relevance: the BM25 (and cosine, if configured) score, normalised over the candidates. The same signal search uses.
• recency: decay on time since the memory was last used, with a floor. On by default (30-day half-life).
• importance: a per-memory dose, set explicitly or from the kind (fact 1.0, preference 1.3, tool result 0.6, …).
• usefulness: learned from feedback. A noisy-OR over links P(useful | feature), where the features describe the memory's shape (kind, source, has a date, has a number, is itself a question, …) crossed with the question type (when, who, how many, …). Words are never features; they only go to the BM25 index.
• rules: a superseded memory (same key, newer version stored) is multiplied by 0.1.

With no feedback anywhere yet, every candidate has the same usefulness (0.14, the prior), so the ranking only moves because of importance:

heapdb  1. 0.085  I have always wanted to see Lisbon, the flights there look amazing
        2. 0.066  Booked the flight: SFO to LIS, departs March 14, 9:40am
        3. 0.064  Flights to Lisbon usually run $400-700 round trip

The booked flight (a fact, importance 1.00) passes the price quote (a tool result, 0.60). That is a fixed heuristic, not learning, and it is roughly what any store with a "kind" weight would do. On day one, HeapDB is search plus a few priors.

The feedback loop

After answering, the client reports which recalled memories the answer used:

res = mem.recall("When is my dentist appointment?", k=4)
# ... the model answers from one of them ...
mem.feedback(used=[answer_id])        # the others shown are marked ignored

Over MCP this is the mark_used / mark_ignored pair; the server's instructions tell the model to call it after it answers.

For each memory in the feedback, heapdb-memory inserts a small retrieval event into HeapDB. It carries no text, only the memory's shape features crossed with the question type, for example qt:when&date:yes, qt:when&num:yes, kind:fact, q:no. A used memory confirms its event for the effect useful; an ignored one rejects it. HeapDB credits each feature in proportion to its current strength and updates a Beta posterior per (feature, useful) link.

Links are kept per namespace (one user) with a pooled global layer. A namespace with little feedback of its own borrows from the pool, so a new user starts with what the assistant has learned from everyone else.

With HeapDB, after feedback

In the example, 40 other users of the same assistant each ask three "when is my …?" or "how many people are coming to …?" questions. Their stores have the same kinds of decoys as Maya's (a chat that repeats the topic word, a question, an assistant tip). After each answer the client reports the memory it used. Maya herself gives no feedback, and nobody else ever asks about flights to Lisbon.

search  1. 1.000  I have always wanted to see Lisbon, the flights there look amazing
        2. 0.944  Flights to Lisbon usually run $400-700 round trip
        3. 0.583  Booked the flight: SFO to LIS, departs March 14, 9:40am
heapdb  1. 0.300  Booked the flight: SFO to LIS, departs March 14, 9:40am
        2. 0.201  Flights to Lisbon usually run $400-700 round trip
        3. 0.136  I have always wanted to see Lisbon, the flights there look amazing

Search hasn't changed: it can't. HeapDB now puts the booking first, and the EXPLAIN line says why:

rel 0.58 (bm25 0.58) x recency 0.81 (age 20.0d) x importance 1.00 x useful 0.63
  [kind:fact=0.23, num:yes=0.23, date:yes=0.08]

It is the weakest text match of the three (rel 0.58), and it wins on learned usefulness (0.63). The pooled links it learned:

kind:fact              P(useful) 0.309  (n 120)
qt:howmany&num:yes     P(useful) 0.119  (n 66)
qt:when&date:yes       P(useful) 0.101  (n 54)

This is the part plain search cannot do. The links describe what kind of memory answers what kind of question, not topics, so they transfer to a question about a flight that no one asked about before, for a user who never gave feedback.

Read the links critically

The strongest link is kind:fact, not qt:when&date:yes. In this simulation every answer happened to be stored as a fact, so HeapDB learned that shortcut too. Likewise num:yes is high because the answers contained clock times; it is part of why the price quote ($400-700) is second. Learned links are associations in the feedback you give, including its accidents. EXPLAIN makes them visible, which is the point of printing them, but it does not make them right. With real traffic the remedy is the usual one: varied feedback, and checking explain_usefulness() before trusting a link.

A failure HeapDB can't fix

The first draft of this scenario stored the booking as "Booked it: SFO to LIS, departs March 14, 9:40am". It shares no word with "When is my flight to Lisbon?" (LIS is not Lisbon to BM25), so it was never a candidate, and no amount of learning could rank it. HeapDB only re-orders what candidate generation returns. Abbreviations, paraphrases and other languages need embeddings in the candidate step (heapdb-memory takes an optional embedder), or better text at write time.

The example

The full script and its real output, run with heapdb-memory 0.1.0 from PyPI. The output is deterministic (fixed clock, seeded simulation, explicit ids).

=== 1. Day one: no feedback anywhere yet ===
Q: When is my flight to Lisbon?
  search  1. 1.000  I have always wanted to see Lisbon, the flights there look amazing
          2. 0.944  Flights to Lisbon usually run $400-700 round trip
          3. 0.583  Booked the flight: SFO to LIS, departs March 14, 9:40am
  heapdb  1. 0.085  I have always wanted to see Lisbon, the flights there look amazing
          2. 0.066  Booked the flight: SFO to LIS, departs March 14, 9:40am
          3. 0.064  Flights to Lisbon usually run $400-700 round trip

=== 2. After 120 sessions of feedback from other users (none from Maya) ===
Q: When is my flight to Lisbon?
  search  1. 1.000  I have always wanted to see Lisbon, the flights there look amazing
          2. 0.944  Flights to Lisbon usually run $400-700 round trip
          3. 0.583  Booked the flight: SFO to LIS, departs March 14, 9:40am
  heapdb  1. 0.300  Booked the flight: SFO to LIS, departs March 14, 9:40am
          2. 0.201  Flights to Lisbon usually run $400-700 round trip
          3. 0.136  I have always wanted to see Lisbon, the flights there look amazing

EXPLAIN for heapdb's top result:
  rel 0.58 (bm25 0.58) x recency 0.81 (age 20.0d) x importance 1.00 x useful 0.63 [kind:fact=0.23, num:yes=0.23, date:yes=0.08]

Strongest learned links, all users pooled (prior mean 0.010):
  kind:fact              P(useful) 0.309  (n 120)
  qt:howmany&len:short   P(useful) 0.119  (n 66)
  qt:howmany&num:yes     P(useful) 0.119  (n 66)
  qt:when&date:yes       P(useful) 0.101  (n 54)
  qt:when&ent:yes        P(useful) 0.101  (n 54)

"""The same assistant memory, ranked two ways.

One heapdb-memory store; every question is recalled twice from it:

  search   relevance alone (BM25): what a plain memory search returns.
  heapdb   relevance x recency x importance x learned usefulness (default weights).

Run: pip install heapdb-memory==0.1.0 && python with-and-without-heapdb.py
"""

import random

from heapdb_memory import Memory, ScoringConfig

DAY = 86400.0
T0 = 1_780_000_000.0                       # fixed clock: reproducible output
SEARCH = ScoringConfig(w_recency=0, w_importance=0, w_usefulness=0)

mem = Memory(clock=lambda: T0)


def show(label, res):
    for r in res:
        print(f"  {label:<7} {r.rank}. {r.score:.3f}  {r.text}")
        label = ""


def compare(ns, question, now):
    print(f"Q: {question}")
    show("search", mem.recall(question, k=3, namespace=ns, now=now, config=SEARCH, track=False))
    res = mem.recall(question, k=3, namespace=ns, now=now)
    show("heapdb", res)
    return res


# Maya's memories from a month of chats. The answer is "lisbon-booked".
for mid, text, day, kind, source in [
    ("lisbon-wish", "I have always wanted to see Lisbon, the flights there look amazing", 8, "episode", "maya"),
    ("lisbon-price", "Flights to Lisbon usually run $400-700 round trip", 9, "tool_result", "assistant"),
    ("lisbon-q", "What is a good neighborhood to stay in Lisbon?", 9, "episode", "maya"),
    ("lisbon-booked", "Booked the flight: SFO to LIS, departs March 14, 9:40am", 10, "fact", "maya"),
    ("jordan", "Jordan says Lisbon was great but way too hilly", 12, "episode", "jordan"),
]:
    mem.add(text, kind=kind, source=source, id=mid, namespace="user:maya", created_at=T0 + day * DAY)

ASK = "When is my flight to Lisbon?"
NOW = T0 + 30 * DAY

print("=== 1. Day one: no feedback anywhere yet ===")
compare("user:maya", ASK, NOW)

# Other users of the same assistant, over a few weeks. Each asks a "when" or
# "how many" question; the assistant answers from one memory and reports it
# with feedback(used=[...]). Plain search has nowhere to put this signal.
TOPICS = ["dentist", "haircut", "vet", "passport", "car service", "concert",
          "interview", "rent", "physio", "flight", "dinner", "lease"]
rng = random.Random(7)
sessions = 0
for u in range(40):
    ns, t = f"user:{u:02d}", T0 + rng.uniform(0, 25) * DAY
    for topic in rng.sample(TOPICS, 3):
        w = topic.split()[-1]
        mem.add(f"I really need to sort out the {topic}, the {w} keeps slipping", namespace=ns,
                source="user", kind="episode", created_at=t)
        mem.add(f"Do you know a good {topic} place near the office?", namespace=ns,
                source="user", kind="episode", created_at=t)
        mem.add(f"A {topic} booking is easy to move online", namespace=ns,
                source="assistant", kind="tool_result", created_at=t)
        if rng.random() < 0.5:
            q = f"When is my {topic}?"
            day = rng.choice(["Monday", "Tuesday", "Friday"])
            ans = mem.add(f"The {topic} is confirmed for {day} at {rng.randint(9, 17)}:00",
                          namespace=ns, source="user", kind="fact", created_at=t)
        else:
            q = f"How many people are coming to the {topic}?"
            ans = mem.add(f"{rng.choice(['Four', 'Six', 'Nine'])} confirmed for the {topic}",
                          namespace=ns, source="user", kind="fact", created_at=t)
        res = mem.recall(q, k=4, namespace=ns, now=t + DAY)
        if ans.id in res.ids:
            mem.feedback(used=[ans.id], recall_id=res.recall_id, now=t + DAY)
            sessions += 1

print(f"\n=== 2. After {sessions} sessions of feedback from other users (none from Maya) ===")
res = compare("user:maya", ASK, NOW)
print("\nEXPLAIN for heapdb's top result:")
print(f"  {res[0].explain()}")

print("\nStrongest learned links, all users pooled (prior mean 0.010):")
rows = [r for r in mem.explain_usefulness("*") if r.feature.startswith(("qt:", "qsrc:", "kind:", "q:"))]
for r in sorted(rows, key=lambda r: -r.strength)[:5]:
    print(f"  {r.feature:<22} P(useful) {r.strength:.3f}  (n {r.n:.0f})")

What the benchmark says

The example is constructed to show the mechanism, so it says nothing about how much this helps in general. For that we measured evidence retrieval on LoCoMo (10 long conversations, 1982 questions, leave-one-conversation-out, oracle feedback from the evidence turns). From the committed RESULTS.md:

• Learned usefulness is the main gain: BM25 plus learned usefulness vs BM25 alone, ΔMRR +0.033 [+0.026, +0.039] on held-out conversations.
• It learned the same patterns as above: qt:when&date:yes P(useful) 0.0300 (n 1403) against 0.0006 for qt:when&date:no; qt:howmany&num:yes 0.0278.
• Recency hurt on that benchmark: the default 30-day decay cost −0.043 MRR [−0.052, −0.035], and tuning turned it off (w_recency=0) in 18 of 20 choices. LoCoMo asks about any point in a months-long history and never re-uses a memory, so it cannot test reinforcement.
• Untuned defaults do not beat BM25: 0.421 vs 0.459 MRR. The gains need feedback and weights that fit the workload.

The full tables and ablations are on the agent memory use-case page, and the gain holds with realistic noisy feedback: +0.035 MRR over the tuned hybrid with half the citations missed, 10% false citations and half the questions giving feedback (noisy-feedback results).

Summary

So the honest pitch: HeapDB turns "the agent used this memory" into a signal that improves the next recall, for every user, and it explains every recall. It is a ranking and transparency layer on top of your search, not a faster or larger store, and on day one, without feedback, it is close to the search it sits on.