Learning loop and explanations

Feedback turns "the last thing I ate" into "the last thing I ate that usually does this to me."

Feedback with proportional credit

A confirmation does not give every feature of the event full credit. HeapDB splits it by each feature's share of the noisy-OR, the standard EM update for a noisy-OR model. For a confirmed event with features F and current strengths s:

$$r_f = \frac{s_f}{1 - \prod_{g \in F} (1 - s_g)}, \qquad \alpha_f \mathrel{+}= r_f, \qquad \beta_f \mathrel{+}= 1 - r_f$$

• Confirm: a feature that was already strong takes most of the credit, so a weak feature that happened to be on the same plate stops riding along. Under the old uniform rule (α += 1 for every feature) a weak rider converges to the same strength as the real cause; under proportional credit it stays low.
• Reject: each feature gets β += 1. In a noisy-OR every feature independently failed to cause the effect.
• Counts are fractional, so Beta parameters are real numbers, not tallies. The uniform rule remains available per effect (credit="uniform").

Priors: per effect and shared across entities

Each link's belief is built from three layers:

1. The registered prior for the effect (beans → bloating starts at Beta(3, 1)).
2. Global evidence pooled across all entities, capped at a few pseudo-observations (default 5) so it guides without overriding.
3. The entity's own counts.

A new user gets answers shaped by what is typically true for everyone; as their own feedback accumulates, it dominates. Borrowing can be switched off per effect.

Inseparable causes

Features that always appear together (beans and onion in every burrito) cannot be told apart by any amount of feedback. Rather than crediting one arbitrarily, HeapDB reports them as one joint cause (beans+onion) with its own joint link, created only for feature sets that actually occurred on judged events. Scoring still uses per-feature links; the joint link is what the explanation shows, flagged inseparable until the features are seen apart.

Plausibility rules

Domain knowledge enters as rules that can only lower a score: add_rule(effect, name, predicate, multiplier) with a multiplier between 0 (hard block) and 1. For example, a transaction under $10 is a weak suspect for a dispute (× 0.3). Multipliers above 1 are rejected so that pruning bounds stay valid. A type-level filter (cause_types) restricts which event types can cause an effect at all, so a meal can never explain a latency spike, and every kernel is zero for negative lag, so an event after the effect never scores.

EXPLAIN

Every result can carry an EXPLAIN record: lag and kernel value, each feature's link strength and its share of the noisy-OR, the dose (weight), the rules that applied, joint causes, the entity, and a confounding note when relevant. A ranked cause without a reason is not trustworthy, so explanation is part of the API, not a debugging aid.

Calibration

Ranking is not enough once scores are compared across entities or shown as likelihoods. Every feedback call records the score each judged event had before learning, plus the outcome. calibration_report(effect) (or CALIBRATION OF effect) returns, per score bin, the mean predicted score against the observed confirm rate, plus a Brier score. Automatic recalibration is on the roadmap.

Feedback updates ranking weights; see Design scope for how to read a score and which tools to pair HeapDB with.

You can see the loop at work in the demo: every popped result shows its EXPLAIN line, and “That was it” / “Not it” (or, for code changes, “Reverted” / “Cleared in postmortem”) update the Beta counts.