Connectors

Our connectors link your real event streams to HeapDB. There are two ways to run them:

• In HeapDB Cloud, we run them for you. Turn on Sentry, PostHog, HTTPS ingest, Kafka or OpenTelemetry per database in the console, and events land in that database. See Connect from HeapDB Cloud.
• Self-run, next to heapdb-core in your own process. Three work today and are available on request: GitHub, PostHog and a generic alert webhook. We're also planning a community connectors repository.

The GitHub connector powers the code-changes use case. The generic alert webhook covers Sentry, PagerDuty and Datadog until they get dedicated connectors. The PostHog connector powers product analytics.

Connect from HeapDB Cloud

Every database in HeapDB Cloud has a Connectors tab. Pick a kind, fill in the form, and HeapDB keeps the connector running: it polls or receives your data, maps each record to a HeapDB event, and writes it through the Data API with the connector's own API key. Writes count against the database's plan like any client, so quotas, usage and billing work the same.

Free includes 1 connector per database, Pro 10, and Team is set up with you (plans).

Add one

1. In the console, open a database and choose Connectors → Add connector.
2. Pick the kind and fill in the form. For HTTPS ingest and Kafka, the mapping editor shows the HeapDB event your sample record becomes as you type.
3. After you create it, the console shows the endpoint URL, the ingest token and the signing secret once, with copy-paste snippets. Copy them then; you can rotate them at any time.
4. Use Test connection: for Sentry, PostHog and Kafka, HeapDB checks the credentials against the source; for HTTPS ingest and OpenTelemetry, it runs a sample record through the mapping and checks the connector's access to the database, without writing anything. The result shows within seconds.

Each connector shows its state (healthy, syncing or error), the last sync, the events sent and the last error. You can disable, edit, rotate and delete it from the same list.

HTTPS ingest

curl -X POST https://ingest.heapdb.com/v1/ingest/<connector_id> \
  -H "Authorization: Bearer $HEAPDB_INGEST_TOKEN" \
  -H "Content-Type: application/json" \
  -d '[{"service":"checkout","timestamp":"2026-10-22T14:03:00Z","kind":"deploy","tags":["pr:1234"]}]'

• Body: one JSON object, a JSON array, or NDJSON (application/x-ndjson), up to 1000 records and 5 MB per request (gzip allowed).
• Auth: the bearer token, or instead an HMAC signature X-HeapDB-Signature: t=<unix time>,v1=<hex HMAC-SHA256 of "<t>.<body>" with the signing secret>, accepted within 5 minutes of t. Use the signature when the sender can sign but shouldn't hold a token.
• Responses: 202 {"accepted": n}; 400 with the first records that could not be mapped and why; 401; 404 for an unknown or disabled connector; 413; 429 with Retry-After when the database is over its quota.
• Delivery is at-least-once. Records with the same external_id are written once; without one, HeapDB derives an id from the record, so retries are safe.

Mapping records

A record that is already a HeapDB event ({entity, time, type, features, weight?, attrs?, external_id?}, time in Unix seconds) needs no mapping. For any other JSON, a mapping names where each part comes from:

{
  "entity":      {"path": "service"},
  "time":        {"path": "timestamp", "format": "iso8601"},
  "type":        {"path": "kind", "default": "event"},
  "features":    [{"path": "tags", "as": "list"}, {"path": "env", "prefix": "env:"}],
  "weight":      {"path": "lines_changed", "transform": "log1p", "default": 1.0},
  "external_id": {"path": "id"},
  "effect":      {"when": {"path": "kind", "equals": "error_spike"}, "name": "error_spike"}
}

• path is dotted, with array indexes: a.b.0.c. Missing values use default.
• time formats: unix (seconds), unix_ms and iso8601 (with an offset, like 2026-10-22T14:03:00Z). Without time, the receive time is used.
• features take a single value or, with "as": "list", every item of a list, each with an optional prefix. Duplicates are dropped.
• weight is the value as is, log1p of it, or a constant ("transform": "const" with a default).
• effect: records whose value at when.path equals when.equals are recorded as effect occurrences (observations for observational learning) instead of events.

OpenTelemetry

Add an otlphttp exporter to your collector's logs pipeline:

exporters:
  otlphttp/heapdb:
    logs_endpoint: https://ingest.heapdb.com/v1/otlp/<connector_id>/v1/logs
    headers:
      Authorization: "Bearer ${env:HEAPDB_INGEST_TOKEN}"
    compression: gzip

The endpoint answers like any OTLP/HTTP receiver: 200 with an ExportLogsServiceResponse, and records that could not be mapped are reported in partial_success. Each log record becomes an event: service.name is the entity, the event.name attribute (or the body's type) is the type, and the allow-listed attributes become key:value features (by default deployment.environment, service.version, vcs.ref and code.filepath). Log bodies are not stored. OTLP/HTTP accepts protobuf and JSON.

Kafka

HeapDB joins the consumer group heapdb-<connector_id> and commits offsets only after the Data API has stored the write. Grant the SASL user read access to the topic and to that group. Brokers must be reachable from the internet on port 443 or 9092–9096, over TLS (SASL_SSL or SSL).

How credentials are kept

• Credentials you enter are encrypted before they are stored, each with its own data key from AWS KMS. They are write-only: the console can replace them but never shows them again, and the website cannot decrypt them. Only the connector runner can.
• Ingest tokens are 256-bit random secrets, shown once; HeapDB stores only their SHA-256 hash.
• Each connector has its own API key for its database, created and revoked with the connector.
• Creating, editing, enabling, disabling, rotating and deleting a connector are written to the audit log.

Self-run connectors

The rest of this page covers the self-run connectors (available on request).

Architecture

 Connector (GitHub, PostHog, alert webhook, ...)
     │  verifies and parses webhooks, backfills history
     ▼
 Normalized records
     EventRecord     a change happened      (PR #4107 merged to checkout, features, dose)
     EffectRecord    something went wrong   (error_spike on checkout at 15:00)
     FeedbackRecord  what really caused it  (revert of #4107 confirms; #4108 cleared)
     CandidateRecord an open PR to score    (features per service, replaced on each push)
     │
     ▼
 Bridge  ── use-case profile (effects, kernels, priors, rules, feature extractors)
     │  insert / top_causes / feature_risk / feedback
     ▼
 HeapDB (heapdb-core)
     │
     ▼
 Sinks: GitHub check run, PR comment, log, stdout (dry run)

• Connectors know one source. They verify signatures, map payloads to normalized records and never call HeapDB directly.
• Normalized records are plain dataclasses, so a Sentry spike and a failing GitHub check are the same EffectRecord shape. CandidateRecord holds an open PR's features so the review queue can replace its entry when new commits arrive.
• The bridge owns the HeapDB instance. It applies a use-case profile, writes records, asks questions when an effect arrives, and hands answers to sinks. It snapshots to JSONL.
• A use-case profile holds the domain knowledge: effect types and their lag kernels, priors, rules (docs_only is hard-blocked), and how to extract features and dose. The built-in code-changes profile implements the feature list with failing_main_check, error_spike and incident.
• Sinks publish answers. The GitHub sink writes a check run or comment naming the likely culprits, and a PR comment with the pre-merge risk score.

GitHub connector

Outputs:

• Culprit check run / comment when an effect arrives: the top 3 likely culprit PRs with EXPLAIN chips (merged 40 min before, touches payments/retry, migration, approved after 2 minutes).
• PR risk comment on open PRs: the risk score (highest across the PR's services and the three effects, naming which one) and the features driving it, so reviewers can spend their attention where it matters. Scores rank PRs; they are not probabilities and can exceed 1 for large changes.
• Backfill: imports recent merged PRs, reverts and incident issues so links start from your history instead of priors alone.

GitHub App permissions

To evaluate what the GitHub connector needs: it runs as a GitHub App (organization or user) that receives webhooks and writes check runs and comments.

Webhook events: Pull request, Pull request review, Check run, Workflow run, Deployment status, Issues, Issue comment. Paths map to services (for example services/checkout/ → checkout), so a PR touching several services becomes one event per service. A dry-run mode logs what it would post instead of writing to GitHub.

PostHog connector

PostHog's event stream feeds observational learning: no one labels causes, so outcomes (converted, churned, rage_click, exception) give soft credit to the events before them, and windows that pass without the outcome count against.

• Live events arrive through a PostHog HTTP Webhook destination (Data pipeline → Destinations) that posts each event, with a shared-secret header, to the connector.
• History is imported with PostHog's HogQL query API, using a read-only key, so links start from your data.
• Mapping: PostHog events ($pageview, $rageclick, $exception, your own events, feature-flag exposures) become HeapDB events and effects per person. Only allow-listed properties are kept; personal-data keys are always refused.
• Output: a feature-effect report per outcome (strength, 90% interval, lift, support) with confounder warnings, and the top likely causes for one person's outcome.

Generic alert webhook

POST /hooks/alert accepts a small JSON body and turns it into an EffectRecord. It requires a shared-secret header X-HeapDB-Token:

{"service": "checkout", "effect": "error_spike", "time": "2026-10-19T11:25:00Z",
 "id": "sentry-4821", "title": "TypeError in split_order", "url": "https://..."}

service and effect are required (service may be a list to query a group), time defaults to now, and id deduplicates repeats. Point a Sentry alert rule, a PagerDuty webhook or a Datadog monitor notification at it (each tool can template the body) to ask "which merged PR likely caused this spike?" from your alerting stack.

Writing your own connector

Until the community connectors repository exists, you can build the same loop on heapdb-core: map each source payload to insert (a change happened), top_causes (an effect arrived), feedback (a revert or postmortem) or observe / settle (outcomes without labelled causes). Keep features about the change, never about the person.

Roadmap

Want a connector for your stack, or access to the connectors? Post it on the use case board.