Engram Alpha

Runtime architecture ​

Engram is one binary, but at runtime it is one heavy process and any number of deliberately light ones. This page is the map: which processes exist and why, how they find each other and talk, and what happens between the first serve and the final stop. Everything binds 127.0.0.1 — nothing ever listens on an external interface.

The processes ​

 assistant (stdio MCP)      assistant (stdio MCP)          browser / IDE
         │                          │                            │
  engram-alpha mcp           engram-alpha mcp                    │
   (bridge, light)            (bridge, light)                    │
         │ streamable HTTP          │ streamable HTTP            │ HTTP + SSE
         └──────────┬───────────────┘                            │
                    ▼                                            ▼
       ┌──────────────────────────────────────────────────────────────┐
       │              the machine core · 127.0.0.1:8787               │
       │  every store lock · the local models · the pane · MCP · SSE  │
       └──────────────────────────────────────────────────────────────┘
                    ▲                                ▲
     engram-alpha serve                 status / doctor / stop / hooks
     (ensure core + register repo,      (one-shot REST clients)
      print pane URL, exit)

The machine core is the one process per machine that holds everything heavy: every open store (and, on the TepinDB backend, their exclusive locks), the three local models, the pane's web server, and every MCP session. It always holds the home graph — that is what its /health advertises — and opens registered project stores lazily on first access (a pane switch, a bridge connecting). You never start it by hand: serve and mcp spawn it detached when the machine has none, with its output going to ~/.engram/core.log. It runs as engram-alpha core, an internal subcommand hidden from the help listing, and it never exits on its own — only engram-alpha stop (or you) ends it.

engram-alpha serve is a launcher, not a server. It makes sure the core is running (spawning it if needed), registers the current repository with it, refreshes the repo-local .engram/daemon.json advertisement, prints the pane URL, and exits — the core keeps serving after the command returns. Run it from anywhere, as often as you like; concurrent runs converge on exactly one core.

engram-alpha mcp is the stdio MCP server your assistant launches — and it is always a bridge: it never opens a store itself, on any backend. It proxies the client's stdio session to the core's MCP endpoint over streamable HTTP, holds a census lease so the core can name it, and exits on its own when the client disconnects or the core goes away. If no core answers it starts one; if the core can't start, the session fails with a clear error in .engram/mcp.log rather than silently opening the store in-process.

Everything else is a thin client. status, doctor, stop, the session-brief hook, brief, export, and import are one-shot REST calls against the core (with a direct-open fallback only when no daemon owns the store). The IDE plugins and the browser just point at the pane URL.

How processes find each other ​

Discovery is two JSON files plus a health check — no process-name matching, no port scanning:

  • ~/.engram/daemon.json — the machine core's advertisement (port, pid, version, home-graph path), written when the core binds its port and removed by its shutdown. Every client reads it and verifies the port over GET /health before trusting it, so a stale file is harmless.
  • <repo>/.engram/daemon.json — a repo-local pointer at the same core, written by serve. This is what the IDE plugins and the Claude Code plugin scrape the pane port from. The core removes these on shutdown too.
  • ~/.engram/registry.json — the machine's project registry: which repositories have graphs. serve, mcp, and setup register; the core opens registered stores lazily.

How they communicate ​

  • MCP (streamable HTTP). Bridges forward each stdio session to /projects/{id}/mcp, which binds the session to that project; /mcp is the same endpoint for the home graph. A direct streamable-HTTP MCP client can skip the bridge and connect to these URLs itself.
  • REST + SSE. The pane, status, doctor, the hooks, and stop use the core's HTTP API; live graph updates stream over SSE.
  • Census leases. Each bridge registers itself with the core (POST /clients → a lease: pid, kind, project root, the MCP client's own name from its initialize — claude-code, mcp-go, … — and connected-since), renews the lease on its existing 15-second heartbeat, and deregisters on clean exit. A lease that stops renewing expires after 45 seconds — a crashed client disappears from the census within three missed beats. The census is pure observability: a failed registration never blocks bridging. It is exactly what engram-alpha status and the pane's Settings → System process list render — including which client bound where, and via what.
  • Liveness and reconnects. The same heartbeat pings the core over MCP. Since 0.10.4 a missed ping is checked against the core's /health: a core that answers there is busy, not gone, and the session stays (up to two minutes of missed pings). A core that doesn't answer, or a session it closed, sends the bridge into a 90-second wait for a core to come back at the same address. It never spawns one itself, so stop stays a stop. When a core returns, the bridge keeps its session if the same process answers, or opens a new one and sends the client notifications/tools/list_changed if a new core took over. Only a core that stays gone for the whole window ends the bridge. Every slow ping, likely machine sleep, reconnect and exit is logged in <repo>/.engram/mcp.log; the core logs runtime stalls over 5 seconds (ENGRAM_STALL_WARN_SECS) to ~/.engram/core.log.

Which project an MCP session serves ​

A bridge binds its session to a project in this order (each rung is logged, so mcp.log always says which one bound the session):

  1. An explicit --db pins the project; the bridge is a verbatim passthrough.
  2. The client's MCP roots (no --db): after the handshake the bridge asks for roots/list and binds to the first file:// root. On a roots/list_changed notification it re-resolves and rebinds — one long-lived, db-less config entry follows the client across project switches. This is what makes single-global-config clients like Windsurf work.
  3. The bridge's working directory, when the client doesn't advertise roots (or advertises them and never answers), as long as it can host a project. The filesystem root and your home directory never can (0.9.8): the Windsurf JetBrains plugin launches from both, and a writable ~ used to register itself as a project with a second graph beside the home graph.
  4. The default agent project — a machine-level setting for sessions with no folder signal at all (an IDE that spawns the bridge from / and never answers roots/list). Set it in the pane's Settings → System info → Default agent project, or over the core's loopback-only GET/POST /settings (~/.engram/settings.json underneath). The bridge reads it at bind time: changing it affects future sessions, never ones already connected. Unset, the ladder continues to
  5. The home graph — the session binds the core's /mcp endpoint rather than dying, reads only: nothing chose this graph, so every write addressed to "this project" is refused with an error that names the client, the reason, and the rebind call, until the agent binds the session itself (below) — a write aimed at an explicit project still passes. A later roots/list_changed naming a real workspace still rebinds away normally.

The bridge tells the core which rung bound the session (its upstream clientInfo carries bound_by=<rung> and the end client's name), so the session census on /system and the pane's Processes panel show both.

The agent can rebind itself: brief with project. When a client advertises roots but never answers them (Windsurf and Devin CLI in the field) and has no usable working directory either, the agent is the only party that knows the real workspace. Passing project to the brief tool — name, id, or any absolute path inside a registered root — rebinds the running session to that project and returns its brief in the same call (0.8.9 shipped this as a separate set_project tool; 0.8.11 folded it into brief). Sessions stranded on rungs 4 or 5 see a one-line hint atop their brief pointing at it, and the brief's first line always names the graph it describes (# Engram brief — project 'x' (/path) or … — the home graph), so "am I bound to the right project?" is one line to read. Every write verdict names the project it landed in; a session on rung 4 (the configured default) says so on its first write. One line in the client's rules file (e.g. AGENTS.md) makes it automatic:

At the start of a session, call engram's brief with project set to the absolute path of the workspace, then follow the brief it returns.

A resumed conversation, an IDE restart, or an MCP reload spawns a new bridge and therefore a new session, back on the fallback rung — whatever the conversation remembers binding earlier. That is why rung 5 refuses writes rather than warning: the agent's next write fails with the rebind instruction instead of landing silently in the wrong graph (issue #11).

The rebind is session-scoped: it refuses unregistered paths (listing the known projects — register a repo by running engram-alpha serve in it once) and never touches the default-agent-project setting. The pane's Processes census shows each live session's current binding under Sessions.

Lifecycle ​

First run. The first serve — or the first assistant session, via its bridge — spawns the core detached. Model provisioning can dominate a first run, so the spawner waits up to 180 seconds for a healthy advertisement; progress and failures land in ~/.engram/core.log.

Convergence. Any number of serve, mcp, and plugin launches converge on one core. If two core starts race for the port, the loser recognizes the winner by its /health — it converges only with a daemon advertising this user's home graph; a foreign engram daemon (another user on the machine, a test sandbox) is just a taken port to walk past.

Idle unload. When the core has had zero connected bridges, no counted HTTP activity, and no model use for 15 minutes, it drops all three ONNX model sessions — measured, that takes a working core from ~845 MB resident to ~150 MB (the OS reclaims the memory in stages), while the core itself stays resident and instantly reachable. The next demand from any path — a search, a background sweep, a model swap — reloads lazily and exactly once: 0.1–0.5 s measured. A connected-but-idle assistant keeps the models resident by design. Health polls, status reads, and lease pings are deliberately exempt from the activity clock, so watching the core never keeps it warm. ENGRAM_IDLE_UNLOAD_SECS tunes the window; 0 disables the unload.

Stop. engram-alpha stop asks the core to shut down over a loopback-only POST /shutdown: MCP sessions close (bridges wait 90 seconds for a core to return, so a stop-then-serve restart keeps assistants connected, then exit), every engine is drained so any store operation that had started has committed, every store lock is released, the daemon files are removed, and the process exits. Since 0.8.13 stop waits on the process, not just the health endpoint — "stopped" means the PID is gone and every store lock with it — escalating SIGTERM → SIGKILL only if the orchestrated path stalls, and warning if the port is still occupied afterwards. Legacy per-repo daemons from pre-0.8.8 binaries are discovered and stopped the old way.

Version handshake (0.8.13). Every serve, bridge bind, and rebind compares the running core's /health version against its own binary: a core older than the binary is stopped and respawned from the newer one, so an update propagates to the core on the next touch instead of leaving stale sessions converging on old code forever. A newer core is always converged on — nothing ever downgrades it. If the core's store was deleted or replaced on disk (a wiped .engram/), the core notices on the next access and reopens — creating a fresh store — instead of serving the deleted file from its cache.

Deprecated: serve --http-only. The pre-0.8.8 foreground shape. It still works — it ensures the core exactly like plain serve and then stays in the foreground while the core is healthy (back-compat for process supervisors), exiting when the core dies — but plain serve is the current form.

Watching it run ​

engram-alpha status reads the core's /system and /projects live: core pid, version, uptime, and port; model residency (loaded or unloaded-idle, and since when); every registered project with whether the core currently holds its store open; and every connected client with pid, project folder, and connection age. --json emits the same for scripts. The pane shows the same census in Settings → System → Processes.