fix(telegram,conversations): attachments by day with sweep in both modes, merge renames the topic, gone topics unbind, failures visible

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

.claude/CLAUDE.md

@@ -6,11 +6,9 @@
 - **NEVER write comments** - code should be self-documenting
 
 ## Layout
-- `src/beaver_gateway` - python, unopinionated primitives (no vault paths, no agent names, no Russian): one package per capability - `agents`, `backends`, `conversations`, `jobs`, `frontends`, `mcp`, `vault`, `security`, `events`, `storage`; `config.py` loads a setup, `app.py` builds the runtime, `cli.py` is the entrypoint
-- every string shown to a model or a user lives in a `*Texts` dataclass with English defaults (`conversations/texts.py`, `frontends/telegram/texts.py`); a setup overrides them
+- `src/beaver_gateway` - python, unopinionated primitives (no vault paths, no agent names)
 - `ui` - SvelteKit SPA (admin + Obsidian panel), served by the gateway from `ui/build`
 - `docs/MISSION.md`, `docs/PRD.md`, `docs/PLAN.md`, `docs/PROGRESS.md` - keep PROGRESS.md current
-- docstrings: one line per module saying what it is for, one line per public configuration field; no section references, phases or history
 
 ## Checking commands
 After writing code, always run (or `make check`):

.env.example

@@ -28,7 +28,3 @@ RAYCAST_BEARER=
 # токен от @BotFather, свой user id - @userinfobot
 TELEGRAM_BOT_TOKEN=
 TELEGRAM_USER_ID=
-
-# Откуда тулза send_file имеет право брать файлы (через ':').
-# Папка входящих вложений разрешена всегда, её дописывает сам фронтенд.
-SEND_FILE_ROOTS=/mnt/hole/shared

README.md

@@ -1,74 +1,2 @@
-# beaver-gateway
-
-A gateway for personal agents built on the Claude Agent SDK: conversations
-with a lifecycle, windows to talk through, jobs, memory hooks, and a policy
-boundary - without an opinion about what the agent is for. A setup is one
-`config.py` (or a package next to it) that assembles a `Gateway`; this repo
-holds the primitives, [beaver-agent](https://git.kotikot.com/beaver/beaver-agent)
-is a full setup built on them.
-
-![The admin: what is in motion, what waits for an answer, the context and the quotas, the graph around the master](preview/shots/admin-now.webp)
-
-## What it gives a setup
-
-- **agents/** - what an agent is: a Claude agent with a prompt per
-  conversation kind, skill sets per kind, gateway tools, a `PreToolUse`
-  policy; a Raycast agent for cheap fast answers.
-- **backends/** - how a turn runs: the Claude Agent SDK subprocess with a
-  session pool, transcripts and session stores; the Raycast wire.
-- **conversations/** - the model of a conversation: kinds (master, branch,
-  deep, job, fork), rows and windows, the inject queue with priorities,
-  seeds, turns, questions, closing through a distiller, master rotation,
-  the envelope, the in-process gateway tools.
-- **jobs/** - cron, webhook and event jobs on pgqueuer, deferred injects,
-  the subscription budget.
-- **frontends/** - the windows: Telegram (master = General, topic = branch,
-  files out of the agent through `send_file`),
-  markdown files in a vault, `/api` + the admin SPA, an Anthropic-compatible
-  `/anthropic/v1/messages`, MCP re-exposure at `/mcp/<name>`, and
-  `WebhookFrontend` for a window a setup declares itself - its own request
-  and response schema, its own agent, its own token scope
-  (`docs/FRONTEND-PLUGINS.md`).
-- **mcp/** - MCP servers a setup declares (stdio, http, python tools),
-  aggregated in-process and handed to agents by name.
-- **vault/** - watching a directory of notes for the envelope.
-- **security/** - bearer tokens, the audit log, credential redaction.
-- **events/** - the in-process bus every frontend taps, and the stream
-  event protocol backends emit.
-- **storage/** - SQLModel tables and the Postgres session store.
-
-`config.py` defines `Gateway` and loads a setup; `app.py` builds the runtime
-from it; `cli.py` is the entrypoint. The admin SPA in `ui/` is the browser
-window on all of it: conversations by day, the thread with the tree of tool
-calls, the context of the last turn, memory, jobs, usage, tokens and the audit.
-
-![A conversation in the admin: the thread, the queue, the details](preview/shots/admin-thread.webp)
-
-![The Context tab: what the model held on the last turn - harness, prompt granules, skills, the notes the thread reached](preview/shots/admin-context.webp) Every string the gateway puts in front of
-a model or a user has an English default and is overridable
-(`ConversationTexts`, `TelegramTexts`).
-
-## Run
-
-```sh
-cd examples
-CLAUDE_CODE_OAUTH_TOKEN=$(claude setup-token) docker compose up --build
-```
-
-`examples/config.py` is the smallest setup that starts: one agent, one
-python tool, one cron job, the built-in frontends on port 8000; the compose
-file next to it adds Postgres. Without Docker: `uv sync`, build the SPA with
-`cd ui && bun install && bun run build`, then run `uv run beaver-gateway` with
-`DATABASE_URL` and `CONFIG_PATH` set. A setup's own secrets come from `.env`
-next to its config (`load_dotenv` runs before the config is executed).
-
-## Develop
-
-```sh
-make check   # ruff format --check, ruff check, ty, pytest
-make fix
-```
-
-No comments in code: modules carry a one-line docstring saying what they are
-for, public configuration fields carry a one-line docstring saying what they
-mean; everything else is the code.
+# Beaver Gateway
+[![AI Slop Inside](https://sladge.net/badge.svg)](https://sladge.net)

examples/config.py

@@ -1,64 +1,204 @@
-"""The smallest setup: one agent, one tool, one job, the built-in frontends.
-
-The loader execs this file with ``ClaudeAgent``, ``RaycastAgent``,
-``McpServer``, ``ExposedMcp`` and ``Gateway`` already bound; the imports
-below are for type checkers. A real setup grows into a package next to
-this file (see beaver-agent).
-"""
-
-import logging
+# Sample user config — grows alongside the implementation phases.
+#
+# Loader (beaver_gateway/config_loader.py) execs this file with
+# ClaudeAgent, RaycastAgent, McpServer, ExposedMcp, Gateway already
+# bound — so importing them is optional. We import explicitly here so
+# IDEs and type-checkers see real symbols instead of free variables.
 import tempfile
-from datetime import UTC, datetime
+from datetime import date
 from pathlib import Path
 
 from beaver_gateway.agents.base import ExposedMcp
-from beaver_gateway.agents.claude import ClaudeAgent, Prompts
-from beaver_gateway.config import Gateway
+from beaver_gateway.agents.claude import ClaudeAgent
+from beaver_gateway.agents.raycast import RaycastAgent, RemoteTool, UserPreferences
+from beaver_gateway.core.registry import Gateway
+from beaver_gateway.core.turn_record import slugify
 from beaver_gateway.frontends.admin import AdminFrontend
 from beaver_gateway.frontends.anthropic import AnthropicMessagesFrontend
-from beaver_gateway.frontends.api import ApiFrontend
 from beaver_gateway.frontends.markdown import MarkdownFrontend
 from beaver_gateway.frontends.mcp_server import McpServerFrontend
-from beaver_gateway.jobs.scheduler import Job, JobRun
 from beaver_gateway.mcp.types import McpServer
 
-log = logging.getLogger("example")
 
-VAULT = Path(tempfile.mkdtemp(prefix="beaver-vault-")).resolve()
-PROMPT = VAULT / "assistant.md"
-PROMPT.write_text(
-    "You are a concise assistant. When asked the time, call `current_time`.\n"
-)
+def chat_path(title: str, agent: str, vault: Path) -> Path:  # noqa: ARG001
+    """Where a new chat file lands in the vault.
+
+    Called by ``MarkdownFrontend`` for every conversation that needs a
+    file: a ``deep`` chat spawned by the dispatcher, a ``/v1/messages``
+    chat (``title`` = first user message), or with ``log_all_chats=True``
+    the archive of a stateless agent's turns. Return value can be
+    absolute or relative; relative paths are anchored under ``vault``.
+
+    Layout below: ``<vault>/<YYYY-MM>/<YYYY-MM-DD>_<topic>.md``.
+    """
+    today = date.today()
+    return vault / f"{today:%Y-%m}" / f"{today:%Y-%m-%d}_{slugify(title, maxlen=40)}.md"
 
 
 def current_time() -> str:
-    return datetime.now(UTC).astimezone().isoformat()
+    """Return the current local time as an ISO-8601 string.
 
+    Trivial demo tool for the Phase 2.1 internal MCP aggregator —
+    confirms a ``python_tool`` namespace is reachable on
+    ``http://127.0.0.1:<INTERNAL_MCP_PORT>/mcp/time``.
+    """
+    from datetime import datetime
 
-async def heartbeat(run: JobRun) -> None:
-    master = await run.master()
-    log.info("heartbeat: master=%s", master.external_id if master else None)
+    return datetime.now().astimezone().isoformat()
 
 
 gateway = Gateway(
     agents=[
+        # Phase 2.2 — ClaudeCodeBackendAdapter routes this agent's
+        # ``/v1/messages`` calls through ``claude-code-api``. The
+        # ``time`` MCP gets exposed as ``mcp__time__current_time`` to
+        # the subscription claude session via
+        # ``BackendOptions.mcp_servers`` pointing at the internal
+        # aggregator on ``127.0.0.1:INTERNAL_MCP_PORT/mcp/time/``.
+        #
+        # Fresh empty tempdir (not a hardcoded ``/tmp``) for two
+        # reasons: claude-code-api derives the JSONL project-key from
+        # ``cwd``, but claude itself writes the JSONL using the cwd's
+        # realpath — on macOS ``/tmp`` and ``/var/folders/...`` are
+        # both ``/private/...`` symlinks, so unresolved cwds make
+        # ``JsonlWatcher`` time out waiting on the wrong path. The
+        # explicit ``.resolve()`` collapses the symlink before claude
+        # ever sees the dir, and ``mkdtemp`` guarantees the directory
+        # is empty so claude does not pick up leftover files.
         ClaudeAgent(
-            name="assistant",
-            model="claude-sonnet-5",
-            cwd=VAULT,
-            prompts=Prompts(master=(PROMPT,), branch=(PROMPT,), deep=(PROMPT,)),
-            gateway_tools=("spawn", "say", "schedule"),
+            name="stub",
+            model="claude-sonnet-4-6",
+            # ``system_prompt`` is appended to claude's built-in agent
+            # prompt (via ``--append-system-prompt``) — so it adds the
+            # agent's identity on top of claude-code's baseline tool
+            # knowledge, rather than replacing it. Same shape as the
+            # RaycastAgent's ``system_prompt → additional_system_instructions``
+            # mapping. For full ``BackendOptions`` knobs (timeouts,
+            # extra_args, history mode, etc.) import ``ClaudeCodeOptions``
+            # and pass ``options=ClaudeCodeOptions(...)``.
+            system_prompt=(
+                "You are a stub agent used to validate the Phase 0 skeleton.\n"
+                "If asked the current time, call the `current_time`"
+                " MCP tool instead of guessing."
+            ),
+            cwd=Path(tempfile.mkdtemp(prefix="beaver-stub-cwd-")).resolve(),
             expose_mcps=(ExposedMcp(name="time"),),
-        )
+        ),
+        # Phase 1.2 — a RaycastAgent the AnthropicMessagesFrontend will
+        # route via RaycastBackend. Phase 1.5 added the per-agent knobs
+        # (`temperature` / `additional_system_instructions` / etc.) —
+        # only `model`, `system_prompt`, and at least one of the others
+        # is mandatory.
+        RaycastAgent(
+            name="research",
+            model="Gemini 3.1 Flash Lite",
+            system_prompt=(
+                "You are a research assistant. "
+                "Reply in the user's language. Cite URLs when you use web search."
+            ),
+            temperature=0.5,
+            available_native_tools=(RemoteTool.WEB_SEARCH, RemoteTool.READ_PAGE),
+            # Lambda so today's date is rebuilt on every request while
+            # locale/timezone stay pinned. ``True`` would give the same
+            # fresh-date behaviour but would also auto-pick host locale
+            # and timezone (``en-US`` / system tz), which isn't what we
+            # want here.
+            user_preferences=lambda: UserPreferences(
+                locale="en-GB",
+                timezone="Europe/Berlin",
+                current_date=date.today().isoformat(),  # noqa: DTZ011 — local date is intended
+            ),
+        ),
+    ],
+    mcps=[
+        # Phase 2.1 — bundle of plain Python callables exposed as one
+        # FastMCP namespace. The internal aggregator mounts it under
+        # ``/mcp/time`` on ``127.0.0.1:INTERNAL_MCP_PORT``; Phase 2.2's
+        # ClaudeCode adapter forwards that URL into
+        # ``BackendOptions.mcp_servers``. Phase 3's ``McpServerFrontend``
+        # reverse-proxies the same internal URL out to external clients.
+        McpServer.python_tool(name="time", tools=[current_time])
+        # Phase 3 — illustrates the ``lenient`` flag. Real-world stdio MCPs
+        # sometimes print "Processing..." or other chatter to stdout before
+        # their actual JSON-RPC frames; the default mcp client forwards
+        # those parse failures downstream as warnings (visible in
+        # Cursor/Cline). With ``lenient=True`` we silently drop non-JSON
+        # lines, so downstream UIs see clean JSON-RPC only. The command
+        # below is just a placeholder — replace with whatever stdio MCP
+        # you actually want gateway to ingest (e.g. an obsidian-mcp).
+        #
+        # Commented out by default: example users won't have the binary
+        # installed and an unreachable command makes ``docker compose up``
+        # surface a confusing "command not found" line at first request.
+        # Uncomment after pointing ``command`` at a real stdio MCP.
+        #
+        # McpServer.stdio(
+        #     name="obsidian",
+        #     command=["uvx", "mcp-obsidian"],
+        #     env={"OBSIDIAN_API_KEY": "..."},
+        #     lenient=True,
+        # ),
     ],
-    mcps=[McpServer.python_tool(name="time", tools=[current_time])],
     frontends=[
+        # Phase 1.4 — expose the agents as `model=<name>` on an
+        # Anthropic-compatible Messages endpoint. Auth comes from
+        # `BOOTSTRAP_TOKENS` in the env (`name1:value1,name2:value2`).
+        #
+        # Every HTTP frontend is mounted under its own path on the one
+        # gateway port (`Gateway.port`, 8000 here): `/anthropic/v1/messages`.
+        # Behind a reverse proxy set `Gateway(public_url="https://domain.com")`
+        # so advertised endpoints use the outside origin; the proxy just
+        # forwards everything to the gateway, no prefix stripping.
         AnthropicMessagesFrontend(),
+        # Phase 3 — re-exposes every declared `McpServer` outside the
+        # gateway with bearer auth + audit log. Each namespace lives at
+        # `/mcp/<name>/`; a flat bundle is published at `/mcp/all/`.
+        # Discovery page (HTML, auth-gated) at `/mcp/` with copy-pastable
+        # Cursor / Claude Desktop snippets. Auth re-uses `BOOTSTRAP_TOKENS`.
         McpServerFrontend(),
+        # Phase 4.3 — browser admin UI. Creds come from
+        # `ADMIN_USER`/`ADMIN_PASS`; the session cookie is signed with
+        # `SESSION_SECRET`. Use it to mint tokens (Argon2-hashed in
+        # the DB), revoke them, and watch the audit log. Scope is
+        # enforced on the bearer frontends: tokens minted with scope
+        # `messages` only work on `/v1/messages`; `mcp` only on
+        # `/mcp/<name>`; `*` works everywhere. Served at `/admin/`; `/`
+        # redirects there.
         AdminFrontend(),
-        ApiFrontend(master_agent="assistant", branch_agent="assistant"),
-        MarkdownFrontend(vault_path=VAULT / "chats", default_agent="assistant"),
+        # Obsidian-vault chat frontend. Each `.md` is one conversation
+        # (User/Assistant turn pairs). The Obsidian companion plugin
+        # POSTs `{filename, content?}` to `/chat` — the frontend reads
+        # the file, runs the agent if the last turn is `user`, and
+        # appends the assistant reply back. With `log_all_chats=True`
+        # *every* turn (from Anthropic Messages too) is mirrored into
+        # `{vault}/_logs/<agent>/` so the vault is the central archive.
+        #
+        # `vault_path` here points at a per-restart tempdir so the
+        # example boots cleanly; in real deployments mount the
+        # Obsidian-sync container's vault volume to a stable path and
+        # pass that instead.
+        # Mounted at `/md` (`/md/chat`, `/md/chat/stream`).
+        MarkdownFrontend(
+            # Point at the dedicated chats subdir of your real Obsidian
+            # vault — the gateway has no idea (and no need) about other
+            # notes outside it. Path resolution / vault-escape checks
+            # are anchored here, so absolute-path attempts (and ``..``
+            # tricks) can't reach notes alongside it.
+            #
+            #   vault_path=Path("/Users/me/Obsidian/Personal/chats"),
+            #
+            # Per-restart tempdir kept here so the example boots even
+            # without a real vault on the host.
+            vault_path=Path(tempfile.mkdtemp(prefix="beaver-vault-")).resolve(),
+            default_agent="research",
+            log_all_chats=True,
+            # ``chat_path`` (optional) overrides the default
+            # ``{vault}/_logs/<agent>/<date>_<slug>.md`` layout. Heads up:
+            # any custom path forces ``warm_index`` to scan the entire
+            # vault on startup so the fingerprint→file map of archived
+            # stateless chats survives a restart no matter where you put
+            # files.
+            chat_path=chat_path,
+        ),
     ],
-    jobs=[Job("heartbeat", heartbeat, cron="0 * * * *")],
-    tz="UTC",
 )

examples/docker-compose.yml

@@ -1,36 +1,31 @@
-services:
-  postgres:
-    image: postgres:16-alpine
-    environment:
-      POSTGRES_USER: beaver
-      POSTGRES_PASSWORD: beaver
-      POSTGRES_DB: beaver
-    volumes:
-      - postgres-data:/var/lib/postgresql/data
-    healthcheck:
-      test: ["CMD-SHELL", "pg_isready -U beaver -d beaver"]
-      interval: 5s
-      timeout: 3s
-      retries: 20
+# Minimal compose for the Phase 0 DoD: build the image, mount a
+# single-file user config, watch the process print its summary line
+# and exit cleanly.
+#
+# Real deployments add a postgres service, raycast-config.json bind,
+# ~/.config/claude bind, etc. — Phase 5 lives in a separate example
+# repo (see PLAN §5.1).
 
+services:
   gateway:
     build:
       context: ..
-    depends_on:
-      postgres:
-        condition: service_healthy
+      dockerfile: Dockerfile
     environment:
-      CONFIG_PATH: /config/config.py
-      DATABASE_URL: postgresql+psycopg://beaver:beaver@postgres:5432/beaver
-      ADMIN_USER: admin
-      ADMIN_PASS: change-me
-      SESSION_SECRET: change-me-too
-      BOOTSTRAP_TOKENS: dev:change-me:*
-      CLAUDE_CODE_OAUTH_TOKEN: ${CLAUDE_CODE_OAUTH_TOKEN}
+      DATABASE_URL: "sqlite:///./gateway.db"
+      ADMIN_USER: "admin"
+      ADMIN_PASS: "change-me"
+      SESSION_SECRET: "dev-secret-change-me"
+      # Phase 1.4 — comma-separated `name:value` pairs. The frontend
+      # 401s every request until at least one token is set.
+      BOOTSTRAP_TOKENS: "cursor:dev-token-change-me"
+      # RAYCAST_BEARER / RAYCAST_DEVICE_ID + a raycast.json bind go
+      # here once you wire up an actual RaycastAgent. The example
+      # config.py declares one, so set these (or remove the agent)
+      # before exposing port 8000.
     ports:
+      # The one gateway port: /anthropic, /mcp, /admin, /md under it
+      # (change ADMIN_USER/ADMIN_PASS/SESSION_SECRET before exposing).
       - "8000:8000"
     volumes:
       - ./config.py:/config/config.py:ro
-
-volumes:
-  postgres-data:

src/beaver_gateway/agents/base.py

@@ -17,8 +17,9 @@ from pydantic import BaseModel, ConfigDict
 class ExposedMcp:
     """Reference to an ``McpServer`` (by name) exposed to a single agent.
 
-    ``tools`` allowlists tool names (``None`` = all); ``deny`` removes
-    ``fnmatch`` patterns on top, e.g. ``("delete_*",)``.
+    ``tools`` is an allowlist of tool names (``None`` = every tool);
+    ``deny`` is a tuple of ``fnmatch`` patterns removed on top of that,
+    e.g. ``("delete_*",)``.
     """
 
     name: str

src/beaver_gateway/agents/claude.py

@@ -1,21 +1,25 @@
 """Claude agent definition, backed by the Claude Agent SDK.
 
-``system_prompt`` is used verbatim, or per-kind granules from ``prompts``
-are assembled via ``agents/prompts.py``. ``skill_sets`` are
-``<skill>/SKILL.md`` directories, each becoming a local SDK plugin.
+The system prompt is ``system_prompt`` verbatim or, per conversation kind,
+the granules named in ``prompts`` assembled at every session spawn (see
+``core/prompt.py``). ``skill_sets`` are directories of ``<skill>/SKILL.md``
+folders; each becomes a local SDK plugin, either the same tuple for every
+kind or a ``SkillSets`` with a tuple per kind (§4.3: the master never sees
+what a branch opens). Nothing from disk is loaded otherwise: the adapter
+runs with ``setting_sources=[]``.
 """
 
 from __future__ import annotations
 
-from collections.abc import Mapping  # noqa: TC003
-from pathlib import Path  # noqa: TC003
+from collections.abc import Mapping  # noqa: TC003 - pydantic runtime
+from pathlib import Path  # noqa: TC003 - pydantic runtime
 
 from pydantic import BaseModel, ConfigDict, Field, model_validator
 
 from beaver_gateway.agents.base import BaseAgent
-from beaver_gateway.agents.policy import PolicyRule  # noqa: TC001
-from beaver_gateway.agents.prompts import PromptSource  # noqa: TC001
-from beaver_gateway.conversations.kinds import KINDS, Kind
+from beaver_gateway.core.kinds import KINDS, Kind
+from beaver_gateway.core.policy import PolicyRule  # noqa: TC001 - pydantic runtime
+from beaver_gateway.core.prompt import PromptSource  # noqa: TC001 - pydantic runtime
 
 __all__ = ["ClaudeAgent", "ClaudeOptions", "Prompts", "SkillSets"]
 
@@ -46,10 +50,10 @@ class ClaudeOptions(BaseModel):
 
 
 class Prompts(BaseModel):
-    """Prompt assembly per conversation kind: the granules, in order.
+    """Prompt assembly per conversation kind (§3.12): the granules, in order.
 
-    A kind left ``None`` is not served; ``ClaudeAgent.kinds`` follows from
-    the kinds set here.
+    A kind left ``None`` is not served by the agent; ``ClaudeAgent.kinds``
+    follows from the kinds set here.
     """
 
     model_config = ConfigDict(frozen=True)
@@ -90,12 +94,14 @@ class ClaudeAgent(BaseAgent):
 
     prompts: Prompts = Field(default_factory=Prompts)
     kinds: tuple[Kind, ...] = ()
-    """Conversation kinds this agent serves; ``create``/``spawn`` reject the rest.
-    Defaults to what ``prompts`` covers, or ``("deep",)`` for a verbatim prompt."""
+    """Conversation kinds this agent serves; ``create``/``spawn`` reject the
+    rest. Defaults to the kinds ``prompts`` covers, or ``("deep",)`` for a
+    verbatim ``system_prompt``."""
 
     skill_sets: tuple[Path, ...] | SkillSets = ()
     """Skill-set directories, each a local plugin: one tuple for every kind,
-    or a ``SkillSets`` giving each kind its own (``None`` = no skills)."""
+    or ``SkillSets`` to give each kind its own (a kind left ``None`` gets
+    no skills)."""
 
     gateway_tools: tuple[str, ...] = ()
     """Gateway tools exposed in-process (``read_conversation``, ``spawn``,
@@ -103,8 +109,8 @@ class ClaudeAgent(BaseAgent):
 
     options: ClaudeOptions = Field(default_factory=ClaudeOptions)
     policy: tuple[PolicyRule, ...] = ()
-    """``PreToolUse`` rules, run in order on every tool call; the first
-    ``Deny`` is what the model reads back. See ``agents/policy.py``."""
+    """``PreToolUse`` rules (§3.7), run in order on every tool call; the
+    first ``Deny`` is what the model reads back. See ``core/policy``."""
 
     @model_validator(mode="after")
     def _kinds_follow_prompts(self) -> ClaudeAgent:

src/beaver_gateway/agents/raycast.py

@@ -1,12 +1,31 @@
 """Raycast agent definition.
 
-Field set is the union of ``raycast_api.ChatAPI.stream`` parameters useful
-as per-agent defaults; per-request values override them where both exist.
+Field set is the union of ``raycast_api.ChatAPI.stream`` parameters that
+make sense as **per-agent defaults**. Per-request values (currently:
+``temperature``) win when both are set; the rest fall back to whatever
+the agent declared, then to Raycast's own defaults.
+
+``BaseAgent.system_prompt`` maps onto Raycast's wire field
+``additional_system_instructions`` (the slot the real client uses for
+*content*); the wire field ``system_instructions`` stays at the Raycast
+source default — ``"markdown"`` for ``AI_CHAT``, ``"plain"`` otherwise.
+We don't expose that wire dichotomy to the user — they get one
+conceptual "system prompt".
+
+Excluded on purpose:
+
+* ``buffer_id``/``message_id``/``current_date`` — per-call ephemeral
+* ``provider`` override — escape hatch for non-catalog models, no clear
+  use case yet (revisit in PRD §14 when discovery lands)
+* ``locale`` — process-wide via ``Settings.raycast_locale`` because we
+  only spin up one ``raycast_api.Client`` per gateway
+* ``system_instructions`` (wire) — that's a format marker, not content;
+  the SDK fills it from the source default and we let it
 """
 
 from __future__ import annotations
 
-from raycast_api import (  # noqa: F401
+from raycast_api import (  # noqa: F401  — UserPreferences re-exported for user configs
     RemoteTool,
     Source,
     UserPreferences,
@@ -17,21 +36,44 @@ from beaver_gateway.agents.base import BaseAgent
 
 
 class RaycastAgent(BaseAgent):
-    """Agent backed by ``raycast-api``."""
+    """Agent backed by ``raycast-api``.
+
+    ``available_native_tools`` is the closed set of Raycast's server-side
+    "remote tools" (``web_search``, ``search_images``, ``read_page``) —
+    typed as ``RemoteTool`` so config-time IDE completion lists exactly
+    the three valid values. Pydantic also coerces string literals, so
+    ``("web_search", "read_page")`` keeps working unchanged.
+
+    ``user_preferences`` toggles the auto-generated ``<user-preferences>``
+    block Raycast prepends to ``additional_system_instructions``:
+
+    * ``True`` (default) → auto from host locale/timezone/today, rebuilt
+      every request so the date stays fresh;
+    * ``False`` → omit the block entirely;
+    * ``UserPreferences(...)`` instance → used verbatim (frozen at the
+      time the agent was loaded, so the date won't auto-update);
+    * ``Callable[[], UserPreferencesArg]`` → re-invoked on every request.
+      Use this for the common case "fresh date but custom
+      locale/timezone": ``user_preferences=lambda:
+      UserPreferences(locale="ru-RU", timezone="Europe/Berlin",
+      current_date=date.today().isoformat())``. Callables may nest
+      (a lambda returning a lambda…) but there's no real reason to.
+
+    The library uses this block for date/locale-aware formatting, not
+    for personalisation/memory — those are out of scope upstream.
+
+    ``reasoning_effort`` values vary by model: GPT-5 takes
+    ``"minimal"|"low"|"medium"|"high"``; Anthropic exposes nothing here
+    (Claude reasoning lives in a separate ``…-reasoning`` model variant
+    in the catalog). Unknown effort for the chosen model is ignored
+    server-side, so we stay loose as ``str | None``.
+    """
 
     streaming: bool = True
     available_native_tools: tuple[RemoteTool, ...] = ()
-    """Raycast's server-side "remote tools": ``web_search``, ``search_images``,
-    ``read_page``."""
-
     source: Source = Source.AI_CHAT
 
     temperature: float | None = None
     reasoning_effort: str | None = None
-    """Model-specific (e.g. GPT-5: ``"minimal"``/``"low"``/``"medium"``/``"high"``);
-    ignored server-side if the model doesn't support it."""
-
     tool_choice: str | None = None
     user_preferences: UserPreferencesArg = True
-    """Auto-fills the ``<user-preferences>`` block from host locale/timezone/date;
-    ``False`` omits it, or pass ``UserPreferences(...)`` / a zero-arg callable."""

src/beaver_gateway/app.py

@@ -1,447 +0,0 @@
-"""Build the runtime from a ``Gateway`` and run every part of it until shutdown."""
-
-from __future__ import annotations
-
-import asyncio
-import functools
-import logging
-from contextlib import AsyncExitStack
-from pathlib import Path
-from typing import TYPE_CHECKING, Any
-
-import psycopg
-import uvicorn
-from pgqueuer import PsycopgDriver
-from raycast_api import Client as RaycastClient
-from raycast_api.config import Config as RaycastConfig
-
-from beaver_gateway.agents.claude import ClaudeAgent
-from beaver_gateway.agents.raycast import RaycastAgent
-from beaver_gateway.backends.claude_sdk import (
-    ClaudeSdkBackend,
-    RunnerConfig,
-    UsageEvent,
-)
-from beaver_gateway.backends.raycast import RaycastBackend
-from beaver_gateway.backends.sessions import SessionPool
-from beaver_gateway.conversations.envelope import Envelope
-from beaver_gateway.conversations.rotation import Rotation, RotationPolicy
-from beaver_gateway.conversations.service import Conversations
-from beaver_gateway.conversations.tools import build_tool_server
-from beaver_gateway.events.bus import EventBus
-from beaver_gateway.frontends.base import GatewayRuntime
-from beaver_gateway.frontends.bearer import require_token
-from beaver_gateway.frontends.root import build_root_app
-from beaver_gateway.jobs.scheduler import Scheduler
-from beaver_gateway.mcp.internal_app import build_internal_app
-from beaver_gateway.security.auth import TokenStore, scopes_with
-from beaver_gateway.storage import (
-    Database,
-    PostgresSessionStore,
-    Usage,
-    append_audit,
-    append_usage,
-)
-
-if TYPE_CHECKING:
-    from collections.abc import Iterable, Iterator
-
-    from claude_agent_sdk import McpSdkServerConfig
-    from fastmcp import FastMCP
-    from fastmcp.tools.base import Tool as FastMCPTool
-    from starlette.applications import Starlette
-    from starlette.requests import Request
-    from starlette.types import ASGIApp
-
-    from beaver_gateway.agents.base import BaseAgent
-    from beaver_gateway.agents.policy import ToolAudit
-    from beaver_gateway.backends.base import Backend
-    from beaver_gateway.config import Gateway
-    from beaver_gateway.mcp.types import McpServerT
-    from beaver_gateway.settings import Settings
-
-__all__ = ["AgentRegistry", "McpRegistry", "run"]
-
-_log = logging.getLogger("beaver_gateway.app")
-
-
-class AgentRegistry:
-    def __init__(self, agents: Iterable[BaseAgent]) -> None:
-        self._agents: dict[str, BaseAgent] = {}
-        for a in agents:
-            if a.name in self._agents:
-                msg = f"duplicate agent name: {a.name!r}"
-                raise ValueError(msg)
-            self._agents[a.name] = a
-
-    def __getitem__(self, name: str) -> BaseAgent:
-        return self._agents[name]
-
-    def get(self, name: str) -> BaseAgent | None:
-        return self._agents.get(name)
-
-    def __iter__(self) -> Iterator[BaseAgent]:
-        return iter(self._agents.values())
-
-    def __len__(self) -> int:
-        return len(self._agents)
-
-    def __contains__(self, name: object) -> bool:
-        return name in self._agents
-
-
-class McpRegistry:
-    def __init__(self, mcps: Iterable[McpServerT]) -> None:
-        self._mcps: dict[str, McpServerT] = {}
-        for m in mcps:
-            if m.name in self._mcps:
-                msg = f"duplicate mcp name: {m.name!r}"
-                raise ValueError(msg)
-            self._mcps[m.name] = m
-
-    def __getitem__(self, name: str) -> McpServerT:
-        return self._mcps[name]
-
-    def get(self, name: str) -> McpServerT | None:
-        return self._mcps.get(name)
-
-    def __iter__(self) -> Iterator[McpServerT]:
-        return iter(self._mcps.values())
-
-    def __len__(self) -> int:
-        return len(self._mcps)
-
-    def __contains__(self, name: object) -> bool:
-        return name in self._mcps
-
-
-async def run(gateway: Gateway, settings: Settings) -> None:
-    agents = AgentRegistry(gateway.agents)
-    mcps = McpRegistry(gateway.mcps)
-    db = Database(settings.database_url)
-    await db.create_all()
-    token_store = TokenStore(
-        db,
-        bootstrap=TokenStore.parse_bootstrap(settings.bootstrap_tokens),
-        bootstrap_scopes=TokenStore.parse_bootstrap_scopes(settings.bootstrap_tokens),
-    )
-
-    async with AsyncExitStack() as stack:
-        stack.push_async_callback(db.dispose)
-        await token_store.start()
-        stack.push_async_callback(token_store.stop)
-        internal_app, internal_urls, mcp_servers = _build_internal_mcp(
-            gateway.mcps, settings=settings
-        )
-        mcp_tools = await _prefetch_mcp_tools(mcp_servers)
-
-        pool = SessionPool()
-        bus = EventBus()
-        late = _LateConversations()
-        session_store = PostgresSessionStore(db)
-        backends = await _build_backends(
-            settings=settings,
-            agents=agents,
-            stack=stack,
-            db=db,
-            session_store=session_store,
-            mcp_internal_urls=internal_urls,
-            mcp_servers=mcp_servers,
-            mcp_tools=mcp_tools,
-            pool=pool,
-            late=late,
-        )
-        conversations = Conversations(
-            db=db,
-            agents=agents,
-            backends=backends,
-            bus=bus,
-            pool=pool,
-            store=session_store,
-            texts=gateway.texts,
-            frontends=gateway.frontends,
-            envelope=Envelope(
-                watch=gateway.watch, tz=gateway.tz, recall=gateway.recall
-            ),
-            distiller=gateway.distiller,
-            user_sink=gateway.user_sink,
-        )
-        late.conversations = conversations
-        scheduler = Scheduler(
-            conversations=conversations,
-            jobs=gateway.jobs,
-            driver=await _pgqueuer_driver(settings.database_url, stack),
-            budget=gateway.budget,
-            rotation=Rotation(
-                conversations, gateway.rotation or RotationPolicy(tz=gateway.tz)
-            ),
-            tz=gateway.tz,
-        )
-        conversations.scheduler = scheduler
-
-        runtime = GatewayRuntime(
-            agents=agents,
-            mcps=mcps,
-            backends=backends,
-            token_store=token_store,
-            db=db,
-            mcp_internal_urls=internal_urls,
-            admin_user=settings.admin_user,
-            admin_pass=settings.admin_pass,
-            session_secret=settings.session_secret,
-            frontends=tuple(gateway.frontends),
-            conversations=conversations,
-            bus=bus,
-            pool=pool,
-            scheduler=scheduler,
-            public_url=gateway.public_url.rstrip("/") if gateway.public_url else None,
-            scopes=scopes_with(fe.scope for fe in gateway.frontends),
-            send_file_roots=_roots(settings.send_file_roots),
-        )
-        for fe in gateway.frontends:
-            fe.configure(runtime)
-
-        _log.info(
-            "beaver-gateway: loaded %d agents, %d mcps, %d frontends",
-            len(agents),
-            len(mcps),
-            len(gateway.frontends),
-        )
-        if not gateway.frontends:
-            return
-
-        await conversations.start()
-        stack.push_async_callback(conversations.stop)
-        await scheduler.start()
-        stack.push_async_callback(scheduler.stop)
-        hooks = scheduler.app(
-            functools.partial(_authorize_hook, runtime=runtime, scope="api")
-        )
-        async with asyncio.TaskGroup() as tg:
-            tg.create_task(pool.reap_loop())
-            if internal_app is not None:
-                tg.create_task(_serve_internal_mcp(internal_app, settings=settings))
-            tg.create_task(_serve_root(gateway, extra={"/hooks": hooks}))
-            if gateway.watch is not None:
-                tg.create_task(gateway.watch.run())
-            for fe in gateway.frontends:
-                tg.create_task(fe.serve())
-
-
-async def _authorize_hook(
-    request: Request, *, runtime: GatewayRuntime, scope: str
-) -> str:
-    return await require_token(request, runtime, scope=scope)
-
-
-async def _pgqueuer_driver(url: str, stack: AsyncExitStack) -> PsycopgDriver | None:
-    plain = _plain_postgres_url(url)
-    if plain is None:
-        return None
-    conn = await psycopg.AsyncConnection.connect(plain, autocommit=True)
-    stack.push_async_callback(conn.close)
-    return PsycopgDriver(conn)
-
-
-def _plain_postgres_url(url: str) -> str | None:
-    for prefix in ("postgresql+psycopg://", "postgresql://", "postgres://"):
-        if url.startswith(prefix):
-            return "postgresql://" + url[len(prefix) :]
-    return None
-
-
-async def _serve_root(gateway: Gateway, *, extra: dict[str, ASGIApp]) -> None:
-    app = build_root_app(gateway.frontends, extra=extra)
-    config = uvicorn.Config(app, host=gateway.host, port=gateway.port, log_level="info")
-    _log.info(
-        "gateway on http://%s:%d - %s",
-        gateway.host,
-        gateway.port,
-        ", ".join([*(fe.path for fe in gateway.frontends if fe.path), *extra])
-        or "no http frontends",
-    )
-    await uvicorn.Server(config).serve()
-
-
-def _build_internal_mcp(
-    mcps: list[McpServerT], *, settings: Settings
-) -> tuple[Starlette | None, dict[str, str], dict[str, FastMCP]]:
-    if not mcps:
-        return None, {}, {}
-    return build_internal_app(mcps, host="127.0.0.1", port=settings.internal_mcp_port)
-
-
-async def _prefetch_mcp_tools(
-    servers: dict[str, FastMCP],
-) -> dict[str, list[FastMCPTool]]:
-    out: dict[str, list[FastMCPTool]] = {}
-    for name, server in servers.items():
-        try:
-            out[name] = list(await server.list_tools())
-        except Exception:  # noqa: BLE001
-            _log.exception("failed to list tools for MCP %r, skipping", name)
-            out[name] = []
-    return out
-
-
-async def _serve_internal_mcp(app: Starlette, *, settings: Settings) -> None:
-    config = uvicorn.Config(
-        app,
-        host="127.0.0.1",
-        port=settings.internal_mcp_port,
-        log_level="warning",
-        loop="uvloop",
-    )
-    _log.info(
-        "internal MCP aggregator on http://127.0.0.1:%d/mcp/<name>",
-        settings.internal_mcp_port,
-    )
-    await uvicorn.Server(config).serve()
-
-
-def _roots(value: str) -> tuple[Path, ...]:
-    return tuple(Path(p) for p in value.split(":") if p.strip())
-
-
-class _LateConversations:
-    conversations: Conversations | None = None
-
-    def server(
-        self, key: str, _kind: str, names: tuple[str, ...]
-    ) -> McpSdkServerConfig | None:
-        if self.conversations is None or not names:
-            return None
-        return build_tool_server(self.conversations, conversation_key=key, names=names)
-
-    async def ask(self, key: str, payload: dict[str, Any]) -> str:
-        if self.conversations is None:
-            msg = "conversations service is not up yet"
-            raise RuntimeError(msg)
-        answer = await self.conversations.ask(key, payload)
-        return self.conversations.answer_text(answer)
-
-
-async def _build_backends(
-    *,
-    settings: Settings,
-    agents: AgentRegistry,
-    stack: AsyncExitStack,
-    db: Database,
-    session_store: PostgresSessionStore,
-    mcp_internal_urls: dict[str, str],
-    mcp_servers: dict[str, FastMCP],
-    mcp_tools: dict[str, list[FastMCPTool]],
-    pool: SessionPool,
-    late: _LateConversations,
-) -> dict[str, Backend]:
-    backends: dict[str, Backend] = {}
-
-    raycast_agents = [a for a in agents if isinstance(a, RaycastAgent)]
-    if raycast_agents:
-        client = await _try_open_raycast_client(settings, stack)
-        if client is not None:
-            raycast_backend = RaycastBackend(
-                client, mcp_servers=mcp_servers, mcp_tools=mcp_tools
-            )
-            for a in raycast_agents:
-                backends[a.name] = raycast_backend
-
-    runner = RunnerConfig(user=settings.claude_runner_user, home=settings.claude_home)
-    mcp_tool_names = {
-        name: [t.name for t in tools] for name, tools in mcp_tools.items()
-    }
-
-    async def record_usage(event: UsageEvent) -> None:
-        row = Usage(
-            agent_name=event.agent_name,
-            conversation_id=event.conversation_id,
-            session_id=event.session_id,
-            model=event.model,
-            effort=event.effort,
-            input_tokens=event.usage.input_tokens,
-            output_tokens=event.usage.output_tokens,
-            cache_read_tokens=event.usage.cache_read_tokens,
-            cache_creation_tokens=event.usage.cache_creation_tokens,
-            context_tokens=event.usage.context_tokens,
-            cost_usd=event.usage.cost_usd,
-            duration_ms=event.usage.duration_ms,
-            num_turns=event.usage.num_turns,
-            model_usage=event.usage.model_usage,
-        )
-        try:
-            async with db.session() as session:
-                await append_usage(session, row)
-        except Exception:  # noqa: BLE001
-            _log.exception("usage write failed for %s", event.agent_name)
-
-    async def record_tool(event: ToolAudit) -> None:
-        detail = {
-            "conversation": event.conversation,
-            "kind": event.kind,
-            "tool": event.tool,
-            "decision": event.decision,
-            "reason": event.reason,
-            "brief": event.brief,
-        }
-        try:
-            async with db.session() as session:
-                await append_audit(
-                    session,
-                    actor=f"agent:{event.agent}",
-                    kind="tool_call",
-                    agent_name=event.agent,
-                    detail=detail,
-                )
-        except Exception:  # noqa: BLE001
-            _log.exception("tool audit write failed for %s", event.agent)
-
-    for a in agents:
-        if isinstance(a, ClaudeAgent):
-            adapter = ClaudeSdkBackend(
-                agent=a,
-                mcp_internal_urls=mcp_internal_urls,
-                session_store=session_store,
-                mcp_tool_names=mcp_tool_names,
-                runner=runner,
-                usage_sink=record_usage,
-                pool=pool,
-                tool_server=functools.partial(late.server, names=a.gateway_tools),
-                asker=late.ask,
-                audit_sink=record_tool,
-            )
-            await stack.enter_async_context(adapter)
-            backends[a.name] = adapter
-
-    return backends
-
-
-async def _try_open_raycast_client(
-    settings: Settings, stack: AsyncExitStack
-) -> RaycastClient | None:
-    if not settings.raycast_bearer:
-        _log.warning(
-            "RaycastAgent present but RAYCAST_BEARER is unset; those agents 503"
-        )
-        return None
-    if not settings.raycast_device_id:
-        _log.warning(
-            "RaycastAgent present but RAYCAST_DEVICE_ID is unset, those agents 503 "
-            "(generate once with `python -c 'import secrets; "
-            "print(secrets.token_hex(32))'`)"
-        )
-        return None
-    if not settings.raycast_config_path.exists():
-        _log.warning(
-            "RaycastAgent present but %s is missing, those agents 503",
-            settings.raycast_config_path,
-        )
-        return None
-
-    config = RaycastConfig.load(settings.raycast_config_path)
-    client = RaycastClient(
-        config=config,
-        bearer_token=settings.raycast_bearer,
-        device_id=settings.raycast_device_id,
-        locale=settings.raycast_locale,
-    )
-    return await stack.enter_async_context(client)

src/beaver_gateway/backends/__init__.py

@@ -1,6 +1,6 @@
 """Backend adapters.
 
 Each backend wraps a provider SDK (``raycast-api``, ``claude-agent-sdk``)
-and yields the unified :class:`~beaver_gateway.events.stream.MessageStreamEvent`
+and yields the unified :class:`~beaver_gateway.core.events.MessageStreamEvent`
 family. The Anthropic-style frontend serialises events straight to SSE.
 """

src/beaver_gateway/backends/base.py

@@ -1,8 +1,19 @@
 """Backend protocol.
 
 A backend turns an Anthropic-style turn (``messages`` + agent definition)
-into a stream of :class:`~beaver_gateway.events.stream.MessageStreamEvent`
-records; provider quirks are translated here, not in the frontend.
+into a stream of :class:`~beaver_gateway.core.events.MessageStreamEvent`
+records. The frontend serializes whatever comes out straight to SSE, so
+backends are the only place where provider quirks are translated.
+
+Implementations are plain :class:`typing.Protocol` conformers - no ABC
+subclassing - to keep them swappable in tests with bare async generators.
+
+``**options`` is the one extension point. Known keys, all optional and
+ignored by backends that don't keep state: ``conversation_id`` (stable id
+the backend may pin a live session to), ``session_id`` (backend session to
+resume when nothing is live), ``capture`` (a
+:class:`~beaver_gateway.core.turn_capture.TurnCapture` the backend fills
+after the stream closes).
 """
 
 from __future__ import annotations
@@ -15,7 +26,7 @@ if TYPE_CHECKING:
     from anthropic.types import MessageParam
 
     from beaver_gateway.agents.base import BaseAgent
-    from beaver_gateway.events.stream import MessageStreamEvent
+    from beaver_gateway.core.events import MessageStreamEvent
 
 
 class Backend(Protocol):
@@ -29,13 +40,5 @@ class Backend(Protocol):
         system: str | None = None,
         **options: Any,
     ) -> AsyncIterator[MessageStreamEvent]:
-        """Yield Anthropic stream events for one turn against ``agent``.
-
-        ``**options`` keys are optional and ignored by backends that don't
-        keep state: ``conversation_id`` (stable id the backend may pin a
-        live session to), ``session_id`` (backend session to resume when
-        nothing is live), ``capture`` (a
-        :class:`~beaver_gateway.backends.capture.TurnCapture` the backend
-        fills after the stream closes).
-        """
+        """Yield Anthropic stream events for one turn against ``agent``."""
         ...

src/beaver_gateway/backends/claude_sdk.py

@@ -1,7 +1,29 @@
 """Claude Agent SDK backend adapter.
 
-One :class:`ClaudeSdkBackend` per :class:`ClaudeAgent`; its sessions live in
-a shared :class:`~beaver_gateway.backends.sessions.SessionPool`.
+One :class:`ClaudeSdkBackend` per :class:`ClaudeAgent`. A live session is
+one ``ClaudeSDKClient`` (one claude subprocess) and runs one turn at a
+time; the sessions of every agent live in one shared
+:class:`~beaver_gateway.core.sessions.SessionPool` that owns TTL and
+memory-pressure eviction. Sessions are keyed by ``conversation_id`` when
+the caller passes one or by a text-only fingerprint of ``messages[:-1]``
+for stateless callers (``/v1/messages``). Without a live session the
+adapter resumes ``session_id`` from the session store (after closing any
+``tool_use`` left open by a crash), or seeds the incoming history into the
+store via ``core/transcript`` and resumes that.
+
+Per-turn ``**options`` beyond the protocol's: ``kind`` (conversation kind,
+picks the prompt assembly and the pool TTL), ``pinned`` (never evicted),
+``tools=False`` (no MCP at all - forks and jobs), ``observer`` (callback
+receiving every raw SDK message, subagent ones included), ``turn_id``.
+
+Events on the wire are the Anthropic ``MessageStreamEvent`` family: one
+``message_start``/``message_stop`` envelope per turn, block indices
+rebased across the API calls claude makes inside the turn.
+
+Process isolation: claude is spawned through a small exec wrapper that
+drops every inherited environment variable outside a whitelist and, when
+``RunnerConfig.user`` is set, switches to that uid before exec (done in the
+wrapper rather than via ``subprocess(user=...)``, which uvloop rejects).
 """
 
 from __future__ import annotations
@@ -37,7 +59,6 @@ from claude_agent_sdk import (
     PermissionResultDeny,
     ResultMessage,
     StreamEvent,
-    TaskNotificationMessage,
     TextBlock,
     ThinkingBlock,
     ToolResultBlock,
@@ -46,18 +67,9 @@ from claude_agent_sdk import (
     project_key_for_directory,
 )
 
-from beaver_gateway.agents import policy as policy_mod
-from beaver_gateway.agents import prompts as prompt_assembly
-from beaver_gateway.backends.capture import TurnCapture, TurnUsage
-from beaver_gateway.backends.sessions import Session, SessionClient, SessionPool
-from beaver_gateway.backends.transcript import (
-    build_entries,
-    close_open_tool_uses,
-    fingerprint,
-    text_of,
-)
-from beaver_gateway.conversations.kinds import as_kind
-from beaver_gateway.events.stream import (
+from beaver_gateway.core import policy as policy_mod
+from beaver_gateway.core import prompt as prompt_assembly
+from beaver_gateway.core.events import (
     StopReason,
     build_content_block_stop,
     build_input_json_delta,
@@ -71,16 +83,18 @@ from beaver_gateway.events.stream import (
     build_thinking_delta,
     build_tool_use_block_start,
 )
+from beaver_gateway.core.kinds import as_kind
+from beaver_gateway.core.sessions import Session, SessionClient, SessionPool
+from beaver_gateway.core.transcript import (
+    build_entries,
+    close_open_tool_uses,
+    fingerprint,
+    text_of,
+)
+from beaver_gateway.core.turn_capture import TurnCapture, TurnUsage
 
 if TYPE_CHECKING:
-    from collections.abc import (
-        AsyncIterator,
-        Awaitable,
-        Callable,
-        Coroutine,
-        Iterable,
-        Sequence,
-    )
+    from collections.abc import AsyncIterator, Awaitable, Callable, Iterable, Sequence
 
     from anthropic.types import MessageParam
     from claude_agent_sdk import (
@@ -93,12 +107,14 @@ if TYPE_CHECKING:
 
     from beaver_gateway.agents.base import BaseAgent
     from beaver_gateway.agents.claude import ClaudeAgent
-    from beaver_gateway.conversations.kinds import Kind
-    from beaver_gateway.events.stream import MessageStreamEvent
+    from beaver_gateway.core.events import MessageStreamEvent
+    from beaver_gateway.core.kinds import Kind
 
 
 _log = logging.getLogger("beaver_gateway.backends.claude_sdk")
 
+# §3.7: in bypass the callback only ever sees AskUserQuestion, and that is
+# exactly the one we want - the SDK's warning about the rest is noise here.
 warnings.filterwarnings("ignore", category=CanUseToolShadowedWarning)
 
 ASK_TOOL = "AskUserQuestion"
@@ -160,7 +176,8 @@ AuditSink = "Callable[[policy_mod.ToolAudit], Awaitable[None]]"
 Asker = "Callable[[str, dict[str, Any]], Awaitable[str]]"
 """``(conversation_key, AskUserQuestion input) -> text the model reads as the
 tool result``. The only channel an answer has in bypass mode is
-``PermissionResultDeny.message``: ``updated_input`` never reaches the model."""
+``PermissionResultDeny.message`` (spike S1, s05): ``updated_input`` never
+reaches the model."""
 
 
 @dataclass(frozen=True, slots=True)
@@ -209,16 +226,8 @@ class ClaudeSdkBackend:
         tool_server: Callable[[str, str], McpSdkServerConfig | None] | None = None,
         asker: Callable[[str, dict[str, Any]], Awaitable[str]] | None = None,
         audit_sink: Callable[[policy_mod.ToolAudit], Awaitable[None]] | None = None,
-        waker: Callable[[str, str], Coroutine[Any, Any, None]] | None = None,
     ) -> None:
-        """One backend per Claude agent.
-
-        ``waker(key, note)`` is called when a session starts a turn by itself
-        (a background task reported back); the caller then attaches.
-        """
         self._agent = agent
-        self.waker = waker
-        self._tasks: set[asyncio.Task[None]] = set()
         self._audit_sink = audit_sink
         self._store = session_store
         self._runner = runner or RunnerConfig()
@@ -272,62 +281,13 @@ class ClaudeSdkBackend:
     def live(self, key: str) -> Session | None:
         return self._pool.get(key)
 
-    def has_pending(self, key: str) -> bool:
-        """A turn the session started on its own is waiting for ``attach``."""
-        live = self._pool.get(key)
-        return live is not None and bool(live.pending)
-
-    def pending_note(self, key: str) -> str | None:
-        live = self._pool.get(key)
-        return live.note if live is not None else None
-
-    async def attach(
-        self,
-        *,
-        key: str,
-        capture: TurnCapture | None = None,
-        observer: Callable[[Any], None] | None = None,
-        turn_id: str | None = None,
-    ) -> AsyncIterator[MessageStreamEvent]:
-        """Consume the turn a live session started by itself as a normal turn.
-
-        The CLI runs one when a background task (a subagent) reports back:
-        no prompt goes out, the events are what the model said in reply to
-        the report, the turn ends on that result.
-        """
-        live = self._pool.get(key)
-        if live is None or not live.pending:
-            return
-        message_id = f"msg_{uuid.uuid4().hex}"
-        yield build_message_start(message_id=message_id, model=self._agent.model)
-        turn = _Turn()
-        async with live.lock:
-            live.running_turn = turn_id or message_id
-            live.last_used = time.monotonic()
-            async for event in self._run_turn(live, None, turn, observer, capture):
-                yield event
-            live.turns += 1
-            live.last_used = time.monotonic()
-            live.running_turn = None
-        usage, _ = await self._after_turn(
-            live, turn, conversation_id=key, history=[], capture=capture
-        )
-        yield build_message_delta(
-            stop_reason=turn.stop_reason, usage=_wire_usage(usage)
-        )
-        yield build_message_stop()
-
-    async def repair_session(
-        self, session_id: str, *, text: str = "interrupted"
-    ) -> int:
+    async def repair_session(self, session_id: str) -> int:
         """Close ``tool_use`` blocks a crash left without a result; count added."""
         key = self._store_key(session_id)
         entries = await self._store.load(cast("Any", key))
         if not entries:
             return 0
-        fixes = close_open_tool_uses(
-            cast("list[Mapping[str, Any]]", entries), text=text
-        )
+        fixes = close_open_tool_uses(cast("list[Mapping[str, Any]]", entries))
         if fixes:
             await self._store.append(cast("Any", key), cast("Any", fixes))
             _log.warning(
@@ -342,7 +302,7 @@ class ClaudeSdkBackend:
         *,
         agent: BaseAgent,
         messages: Iterable[MessageParam],
-        system: str | None = None,  # noqa: ARG002
+        system: str | None = None,  # noqa: ARG002 - the agent owns its prompt
         conversation_id: str | None = None,
         session_id: str | None = None,
         reseed: bool = False,
@@ -352,21 +312,8 @@ class ClaudeSdkBackend:
         tools: bool = True,
         observer: Callable[[Any], None] | None = None,
         turn_id: str | None = None,
-        **options: Any,  # noqa: ARG002
+        **options: Any,  # noqa: ARG002 - per-request knobs are not supported
     ) -> AsyncIterator[MessageStreamEvent]:
-        """Run one turn against ``agent``, yielding Anthropic stream events.
-
-        Sessions are keyed by ``conversation_id`` when given, else a
-        text-only fingerprint of ``messages[:-1]``. Without a live session,
-        resumes ``session_id`` (closing any ``tool_use`` a crash left open),
-        or seeds ``messages[:-1]`` via ``backends/transcript.py`` and resumes
-        that. ``kind`` picks the prompt assembly and pool TTL; ``pinned``
-        sessions are never evicted; ``tools=False`` mounts no MCP servers
-        (forks and jobs); ``observer`` sees every raw SDK message.
-
-        A resumed session that dies before any event reached the caller is
-        reseeded from history and retried once.
-        """
         if agent.name != self._agent.name:
             msg = f"backend bound to {self._agent.name!r}, got {agent.name!r}"
             raise ValueError(msg)
@@ -374,7 +321,7 @@ class ClaudeSdkBackend:
         if not history or history[-1].get("role") != "user":
             msg = "the last message must be a user turn"
             raise ValueError(msg)
-        prompt = _prompt(history[-1].get("content"))
+        prompt = _prompt_text(history[-1].get("content"))
         prior = history[:-1]
         key = conversation_id or fingerprint(prior)
         spec = _SessionSpec(kind=kind, pinned=pinned, tools=tools)
@@ -388,6 +335,8 @@ class ClaudeSdkBackend:
             live.running_turn = turn_id or message_id
             live.last_used = time.monotonic()
             try:
+                # Events go out as the CLI produces them: the frontends
+                # stream text and thinking live, the turn is not buffered.
                 async for event in self._run_turn(
                     live, prompt, turn, observer, capture
                 ):
@@ -395,6 +344,8 @@ class ClaudeSdkBackend:
             except Exception:
                 live.running_turn = None
                 await self._pool.close(key)
+                # A dead resume can be reseeded from history, but only
+                # while nothing of this turn has reached the caller yet.
                 if not (live.resumed and live.turns == 0) or turn.events:
                     raise
                 _log.exception(
@@ -465,71 +416,70 @@ class ClaudeSdkBackend:
     async def _run_turn(
         self,
         live: Session,
-        prompt: str | list[dict[str, Any]] | None,
+        prompt: str,
         turn: _Turn,
         observer: Callable[[Any], None] | None = None,
         capture: TurnCapture | None = None,
     ) -> AsyncIterator[MessageStreamEvent]:
-        """Run one prompt, or attach to a self-started one, yielding events.
+        """Run one prompt, yielding wire events as they arrive.
 
         ``turn`` is filled in place (result, synthesized history, count of
         events already yielded) so the caller can finish bookkeeping - and
         decide whether a retry is still possible - after a failure. The
         session id lands in ``capture`` with the first frame, so a turn cut
         by a restart still leaves a resumable session behind.
-
-        A result whose ``origin`` is not the human's ends a turn the CLI ran
-        on its own; inside a prompted turn it is passed over, the events it
-        came with stay in this turn rather than getting lost.
         """
+        streaming = self._agent.options.include_partial_messages
         raw: list[Any] = []
-        cursor = _Cursor()
-        live.observer = observer
-        live.attached = True
-        for queued in live.pending:
-            live.inbox.put_nowait(queued)
-        live.pending.clear()
-        try:
-            if prompt is not None:
-                await live.client.query(
-                    prompt if isinstance(prompt, str) else _stream(prompt)
+        next_index = 0
+        offset = 0
+        await live.client.query(prompt)
+        async for message in live.client.receive_response():
+            if observer is not None:
+                observer(message)
+            session_id = getattr(message, "session_id", None)
+            if isinstance(session_id, str) and session_id and live.session_id is None:
+                live.session_id = session_id
+                if capture is not None:
+                    capture.session_id = session_id
+            if isinstance(message, MirrorErrorMessage):
+                live.dirty = True
+                _log.error(
+                    "session %s: mirror error, marked dirty: %s",
+                    live.session_id,
+                    message.error,
                 )
-            while True:
-                message = await live.inbox.get()
-                if message is _END:
-                    if prompt is not None:
-                        msg = f"claude exited before answering ({live.session_id})"
-                        raise RuntimeError(msg)
-                    break
-                if isinstance(message, BaseException):
-                    raise message
-                if observer is not None:
-                    observer(message)
-                if not self._note(live, message, capture):
+                continue
+            if getattr(message, "parent_tool_use_id", None) is not None:
+                continue
+            if isinstance(message, StreamEvent):
+                event = message.event
+                if event.get("type") == "message_start":
+                    offset = next_index
+                    turn.context_tokens = _context_of(event.get("message"))
                     continue
-                if isinstance(message, ResultMessage):
-                    if prompt is not None and _self_started(message):
-                        _log.info(
-                            "session %s: a self-started turn ended inside a "
-                            "prompted one, folded in",
-                            live.session_id,
-                        )
-                        continue
-                    turn.result = message
-                    turn.stop_reason = _STOP_REASONS.get(
-                        message.stop_reason or "", "end_turn"
-                    )
-                    break
-                for out in self._emit(message, turn, cursor, raw):
-                    turn.events += 1
-                    yield out
-            if turn.events == 0:
-                for out in _emit_whole(raw, cursor):
-                    turn.events += 1
-                    yield out
-        finally:
-            live.attached = False
-            live.woken = False
+                index = event.get("index")
+                if isinstance(index, int):
+                    next_index = max(next_index, offset + index + 1)
+                    if streaming:
+                        for out in _emit_stream_event(event, offset + index):
+                            turn.events += 1
+                            yield out
+            elif isinstance(message, AssistantMessage):
+                raw.append(message)
+                if not streaming:
+                    for block in message.content:
+                        for out in _emit_block(block, next_index):
+                            turn.events += 1
+                            yield out
+                        next_index += 1
+            elif isinstance(message, UserMessage):
+                raw.append(message)
+            elif isinstance(message, ResultMessage):
+                turn.result = message
+                turn.stop_reason = _STOP_REASONS.get(
+                    message.stop_reason or "", "end_turn"
+                )
         turn.synthesized = synthesize_turn_messages(raw)
         _log.info(
             "turn: agent=%s session=%s events=%d synthesized=%d stop=%s",
@@ -540,88 +490,6 @@ class ClaudeSdkBackend:
             turn.stop_reason,
         )
 
-    @staticmethod
-    def _note(live: Session, message: Any, capture: TurnCapture | None) -> bool:
-        """Session bookkeeping for one message; False if not the main thread's."""
-        session_id = getattr(message, "session_id", None)
-        if isinstance(session_id, str) and session_id and live.session_id is None:
-            live.session_id = session_id
-            if capture is not None:
-                capture.session_id = session_id
-        if isinstance(message, MirrorErrorMessage):
-            live.dirty = True
-            _log.error(
-                "session %s: mirror error, marked dirty: %s",
-                live.session_id,
-                message.error,
-            )
-            return False
-        if isinstance(message, TaskNotificationMessage):
-            live.note = _note_of(message)
-            return False
-        return getattr(message, "parent_tool_use_id", None) is None
-
-    def _emit(
-        self, message: Any, turn: _Turn, cursor: _Cursor, raw: list[Any]
-    ) -> Iterable[MessageStreamEvent]:
-        streaming = self._agent.options.include_partial_messages
-        if isinstance(message, StreamEvent):
-            event = message.event
-            if event.get("type") == "message_start":
-                cursor.offset = cursor.next_index
-                turn.context_tokens = _context_of(event.get("message"))
-                return
-            index = event.get("index")
-            if isinstance(index, int):
-                cursor.next_index = max(cursor.next_index, cursor.offset + index + 1)
-                if streaming:
-                    yield from _emit_stream_event(event, cursor.offset + index)
-        elif isinstance(message, AssistantMessage):
-            raw.append(message)
-            if not streaming:
-                for block in message.content:
-                    yield from _emit_block(block, cursor.next_index)
-                    cursor.next_index += 1
-        elif isinstance(message, UserMessage):
-            raw.append(message)
-
-    async def _read(self, live: Session) -> None:
-        """Read the client for the session's whole life.
-
-        During a turn every message goes to the turn's inbox. Between turns
-        a subagent still running is shown to the last observer, and the
-        first frame of the main thread means the CLI started a turn by
-        itself: it is buffered in ``pending`` and the waker is told once.
-        """
-        try:
-            async for message in live.client.receive_messages():
-                if live.attached:
-                    live.inbox.put_nowait(message)
-                    continue
-                if isinstance(message, TaskNotificationMessage):
-                    live.note = _note_of(message)
-                if _main_thread(message):
-                    live.pending.append(message)
-                    if not live.woken:
-                        live.woken = True
-                        self._wake(live)
-                elif live.observer is not None:
-                    live.observer(message)
-            live.inbox.put_nowait(_END)
-        except asyncio.CancelledError:
-            raise
-        except Exception as exc:  # noqa: BLE001
-            _log.error("session %s: reader died: %s", live.session_id, exc)
-            live.dirty = True
-            live.inbox.put_nowait(exc)
-
-    def _wake(self, live: Session) -> None:
-        if self.waker is None:
-            return
-        task = asyncio.create_task(self.waker(live.key, live.note or ""))
-        self._tasks.add(task)
-        task.add_done_callback(self._tasks.discard)
-
     async def _acquire(
         self,
         key: str,
@@ -685,7 +553,7 @@ class ClaudeSdkBackend:
             spec.tools,
             self._runner.user,
         )
-        live = Session(
+        return Session(
             key=key,
             agent=self._agent.name,
             kind=spec.kind,
@@ -694,8 +562,6 @@ class ClaudeSdkBackend:
             resumed=resume is not None,
             pinned=spec.pinned,
         )
-        live.reader = asyncio.create_task(self._read(live))
-        return live
 
     def _default_factory(self, options: ClaudeAgentOptions) -> SessionClient:
         return _RunnerClient(options, uid=self._uid)
@@ -756,12 +622,6 @@ class ClaudeSdkBackend:
     ) -> Callable[
         [str, dict[str, Any], ToolPermissionContext], Awaitable[PermissionResult]
     ]:
-        """Build the ``can_use_tool`` callback for one session.
-
-        In bypass mode the SDK only ever calls this back for
-        ``AskUserQuestion``; the warning about the rest is suppressed at
-        import time.
-        """
         asker = self._asker
 
         async def can_use_tool(
@@ -784,7 +644,7 @@ class ClaudeSdkBackend:
     def _hooks(
         self, key: str, spec: _SessionSpec
     ) -> dict[HookEvent, list[HookMatcher]] | None:
-        """One in-process ``PreToolUse`` hook: policy rules, then audit."""
+        """§3.7: one in-process ``PreToolUse`` hook - policy rules, then audit."""
         agent = self._agent
         if not agent.policy and self._audit_sink is None:
             return None
@@ -844,12 +704,6 @@ class ClaudeSdkBackend:
         return plugins
 
     def _exec_wrapper(self, *, extra_keep: tuple[str, ...]) -> Path:
-        """Path to a generated wrapper that execs claude with a whitelisted env.
-
-        Drops every inherited variable outside the whitelist and, when
-        ``RunnerConfig.user`` is set, switches to that uid before exec - done
-        here rather than via ``subprocess(user=...)``, which uvloop rejects.
-        """
         if self._wrapper is not None:
             return self._wrapper
         keep = sorted({*ENV_KEEP, *self._agent.options.env_keep, *extra_keep})
@@ -885,15 +739,6 @@ class _SessionSpec:
     tools: bool
 
 
-_END = object()
-
-
-@dataclass
-class _Cursor:
-    next_index: int = 0
-    offset: int = 0
-
-
 @dataclass
 class _Turn:
     events: int = 0
@@ -1000,54 +845,14 @@ def _mcp_disallowed(
     return out
 
 
-def _prompt(content: Any) -> str | list[dict[str, Any]]:
-    """Plain text when the turn is text only; the block list when images ride along."""
+def _prompt_text(content: Any) -> str:
     text = text_of(content)
     if not text:
         msg = "user message has no text content"
         raise ValueError(msg)
-    if isinstance(content, list) and any(
-        isinstance(b, dict) and b.get("type") != "text" for b in content
-    ):
-        return [dict(b) for b in content if isinstance(b, dict)]
     return text
 
 
-async def _stream(content: list[dict[str, Any]]) -> AsyncIterator[dict[str, Any]]:
-    yield {
-        "type": "user",
-        "message": {"role": "user", "content": content},
-        "parent_tool_use_id": None,
-    }
-
-
-def _emit_whole(raw: Iterable[Any], cursor: _Cursor) -> Iterable[MessageStreamEvent]:
-    """Blocks of a turn that streamed no deltas, e.g. one reported whole."""
-    for message in raw:
-        if isinstance(message, AssistantMessage):
-            for block in message.content:
-                yield from _emit_block(block, cursor.next_index)
-                cursor.next_index += 1
-
-
-def _self_started(result: ResultMessage) -> bool:
-    origin = result.origin
-    return origin is not None and origin.get("kind") != "human"
-
-
-def _main_thread(message: Any) -> bool:
-    if getattr(message, "parent_tool_use_id", None) is not None:
-        return False
-    return isinstance(
-        message, (StreamEvent, AssistantMessage, UserMessage, ResultMessage)
-    )
-
-
-def _note_of(message: TaskNotificationMessage) -> str:
-    summary = message.summary or message.data.get("description") or message.task_id
-    return f"{summary} ({message.status})"
-
-
 def synthesize_turn_messages(raw: Iterable[Any]) -> list[dict[str, Any]]:
     out: list[dict[str, Any]] = []
     for message in raw:

src/beaver_gateway/backends/raycast.py

@@ -1,8 +1,32 @@
 """Raycast backend adapter.
 
-Translates between Anthropic's ``/v1/messages`` wire vocabulary and the
-``raycast-api`` SDK, splicing MCP tools into Raycast's ``tool_calls`` and
-routing them back in-process where the agent declares ``expose_mcps``.
+Translates between Anthropic's ``/v1/messages`` wire vocabulary (incoming
+``MessageParam`` history, outgoing ``MessageStreamEvent`` SSE) and the
+``raycast-api`` SDK (``Message`` history, ``ChatStreamChunk`` SSE).
+
+Two halves live here:
+
+* :func:`_to_raycast_messages` — pure conversion of an Anthropic message
+  list into ``list[raycast_api.Message]``. ``tool_result`` blocks carry no
+  tool name in Anthropic; we recover it by remembering each ``tool_use``
+  id we saw upstream.
+* :meth:`RaycastBackend.complete` — opens a ``client.chat.stream`` and
+  walks chunks through a tiny block-state machine. The state machine
+  exists only because Raycast streams ``tool_calls`` in three phases
+  (open with id+name, deltas with empty id, final summary with the full
+  ``arguments``) — Anthropic wants one ``content_block_start`` → deltas
+  → ``content_block_stop`` per block, so we de-duplicate the final
+  summary against the per-delta increments already emitted.
+
+MCP wiring: Raycast has no native MCP concept, so when an agent declares
+``expose_mcps`` we splice each MCP's tools into the wire request as
+``Tool.local(name=f"{mcp}__{tool}", ...)`` and run a gateway-internal
+loop. Every time the model emits a tool_call for one of those local
+tools we route it back to the underlying MCP in-process, append the
+result as a ``tool`` message, and re-issue the stream — all inside one
+Anthropic envelope (one ``message_start`` … one ``message_stop``). The
+tool_use blocks DO surface to the caller (mirrors what
+``ClaudeSdkBackend`` does), but tool_results stay internal.
 """
 
 from __future__ import annotations
@@ -18,7 +42,7 @@ from raycast_api import Message as RaycastMessage
 from raycast_api import RemoteTool, Tool, ToolCall
 
 from beaver_gateway.agents.raycast import RaycastAgent
-from beaver_gateway.events.stream import (
+from beaver_gateway.core.events import (
     StopReason,
     build_content_block_stop,
     build_input_json_delta,
@@ -41,7 +65,7 @@ if TYPE_CHECKING:
     from raycast_api import ChatStreamChunk, Client
 
     from beaver_gateway.agents.base import BaseAgent
-    from beaver_gateway.events.stream import MessageStreamEvent
+    from beaver_gateway.core.events import MessageStreamEvent
 else:
     from collections.abc import Mapping
 
@@ -52,9 +76,11 @@ __all__ = ["RaycastBackend"]
 _log = logging.getLogger("beaver_gateway.backends.raycast")
 
 
+# Cap on consecutive tool-call turns inside one Anthropic envelope.
+# Real conversations rarely chain more than a handful; the limit only
+# fires on a model that loops, and surfaces as a clean ``end_turn``
+# with an error tool_result rather than a hang.
 _MAX_TOOL_TURNS = 20
-"""Cap on consecutive tool-call turns inside one Anthropic envelope; a model
-that loops hits this and gets a clean ``end_turn`` instead of a hang."""
 
 
 _RAYCAST_TO_ANTHROPIC_STOP: dict[str, StopReason] = {
@@ -366,7 +392,7 @@ class _BlockState:
 
     def __init__(self) -> None:
         self.index: int = -1
-        self.kind: str | None = None
+        self.kind: str | None = None  # "text" | "thinking" | "tool_use" | None
         self.tool_id_to_block: dict[str, int] = {}
         self.tool_idx_to_id: dict[int, str] = {}
 
@@ -402,14 +428,11 @@ class RaycastBackend:
         self._client = client
         self._mcp_servers: Mapping[str, FastMCP] = mcp_servers or {}
         self._mcp_tools: Mapping[str, list[FastMCPTool]] = mcp_tools or {}
+        # Cached per-agent catalog — agents are immutable, so a single
+        # render at first use covers the gateway's lifetime.
         self._agent_catalog: dict[str, _AgentToolCatalog] = {}
 
     def _catalog_for(self, agent: RaycastAgent) -> _AgentToolCatalog:
-        """Build (or reuse) the tool catalog for ``agent``.
-
-        Agents are immutable, so a single render at first use covers the
-        gateway's lifetime.
-        """
         cached = self._agent_catalog.get(agent.name)
         if cached is not None:
             return cached
@@ -425,26 +448,31 @@ class RaycastBackend:
         system: str | None = None,
         **options: Any,
     ) -> AsyncIterator[MessageStreamEvent]:
-        """Run one turn against ``agent``, yielding Anthropic stream events.
-
-        On the wire Raycast uses ``system_instructions`` as a format marker
-        (``"markdown"``/``"plain"``, filled from the source default when we
-        pass ``None``) and ``additional_system_instructions`` for the actual
-        prompt content; ``system`` (or ``agent.system_prompt``) flows into
-        the latter.
-        """
         if not isinstance(agent, RaycastAgent):
             msg = f"RaycastBackend requires RaycastAgent, got {type(agent).__name__}"
             raise TypeError(msg)
 
         raycast_messages = _to_raycast_messages(messages)
         catalog = self._catalog_for(agent)
+        # Native remote tools + spliced MCP locals. ``None`` keeps the
+        # SDK from sending a ``tools`` field at all when neither is
+        # declared.
         tools_arg: list[Tool | RemoteTool | str] | None = (
             list(catalog.tools) if catalog.tools else None
         )
 
+        # On the wire Raycast uses ``system_instructions`` as a format
+        # marker (``"markdown"`` for AI_CHAT, ``"plain"`` otherwise —
+        # filled in by the SDK from the source default when we pass
+        # ``None``) and ``additional_system_instructions`` as the actual
+        # prompt content. So our ``system_prompt`` (or the per-request
+        # Anthropic ``system``, if present) flows into the *additional*
+        # slot. The SDK still prepends ``<user-preferences>`` to whatever
+        # we hand it via ``_build_preamble``.
         prompt_content = system if system is not None else agent.system_prompt
 
+        # Per-request options win over agent defaults; agent defaults
+        # win over Raycast SDK defaults. ``None`` means "fall back".
         async for event in self._stream(
             agent=agent,
             raycast_messages=raycast_messages,
@@ -471,16 +499,12 @@ class RaycastBackend:
         reasoning_effort: str | None,
         tool_choice: str | None,
     ) -> AsyncIterator[MessageStreamEvent]:
-        """Yield one Anthropic envelope spanning one or more ``chat.stream`` calls.
-
-        A tool_call not in ``mcp_routing`` is left to bubble out as a
-        regular ``tool_use`` block instead, for the caller to answer with a
-        ``tool_result`` the usual way.
-        """
         message_id = f"msg_{uuid.uuid4().hex}"
         yield build_message_start(message_id=message_id, model=agent.model)
 
         state = _BlockState()
+        # ``working_messages`` is the rolling history fed back to
+        # Raycast as we resolve MCP tool_calls turn by turn.
         working_messages = list(raycast_messages)
         last_usage: dict[str, int] | None = None
         last_finish: str | None = None
@@ -493,6 +517,9 @@ class RaycastBackend:
                 model=agent.model,
                 messages=working_messages,
                 source=agent.source,
+                # ``system_instructions=None`` → SDK substitutes the
+                # source default (``"markdown"`` / ``"plain"``). Real
+                # prompt goes into ``additional_system_instructions``.
                 additional_system_instructions=prompt_content,
                 user_preferences=agent.user_preferences,
                 tools=tools,
@@ -513,16 +540,27 @@ class RaycastBackend:
             if acc.usage:
                 last_usage = acc.usage
 
+            # Figure out which (if any) tool_calls land on our MCP
+            # routing table. Anything not in the table is left to bubble
+            # out of the envelope as a regular tool_use block — the
+            # Anthropic caller can then respond with a tool_result the
+            # usual way, and the next ``complete`` invocation will
+            # carry it back in.
             pending_mcp_calls = [
                 tid for tid in acc.tool_order if acc.tool_names.get(tid) in mcp_routing
             ]
             if not pending_mcp_calls:
                 break
 
+            # Close whatever block is still open before we step into
+            # tool execution — the next turn's chunks start a fresh
+            # block sequence.
             if state.kind is not None:
                 yield build_content_block_stop(state.index)
                 state.kind = None
 
+            # Echo the assistant turn so Raycast sees its own reply +
+            # tool_calls in subsequent context.
             assistant_text = "".join(acc.text_parts)
             assistant_tool_calls = [
                 ToolCall(
@@ -538,6 +576,10 @@ class RaycastBackend:
                 )
             )
 
+            # Dispatch each MCP-routed call and append a ``tool``
+            # message. Calls not in the routing table get a placeholder
+            # error so the model can correct itself rather than the
+            # gateway hanging the conversation.
             for tid in acc.tool_order:
                 tool_name = acc.tool_names.get(tid, "")
                 args_str = "".join(acc.tool_args.get(tid, [])) or "{}"
@@ -546,12 +588,16 @@ class RaycastBackend:
                         tool_name=tool_name, args_json=args_str, mcp_routing=mcp_routing
                     )
                 else:
+                    # Non-MCP tool — shouldn't really happen because we
+                    # haven't surfaced any other locals, but defend.
                     result_text = f"Tool {tool_name!r} is not handled by the gateway."
                 working_messages.append(
                     RaycastMessage.tool(
                         tool_call_id=tid, name=tool_name, result=result_text
                     )
                 )
+
+            # Loop: re-stream with the new history.
         else:
             _log.warning(
                 "raycast tool-call loop hit %d-turn cap for agent %r; "
@@ -560,6 +606,7 @@ class RaycastBackend:
                 agent.name,
             )
 
+        # Close whatever block is still open before the final delta.
         if state.kind is not None:
             yield build_content_block_stop(state.index)
             state.kind = None
@@ -669,12 +716,6 @@ class RaycastBackend:
         the final-summary chunk's arguments string is dropped because the
         deltas already streamed it.
 
-        Providers disagree on how ``arguments`` arrives: streaming ones
-        (GPT) send deltas across chunks and then restate the full string in
-        a final summary chunk (skip it, already streamed); non-streaming
-        ones (Gemini) send it only in that final summary (emit it once,
-        since nothing streamed it first).
-
         Side-effects on ``acc`` mirror what gets emitted to the wire so
         the gateway can rebuild a full ``ToolCall`` for the next Raycast
         turn (it needs the joined ``arguments`` JSON string, which the
@@ -687,6 +728,9 @@ class RaycastBackend:
             raw_tc = raw_tcs[i] if i < len(raw_tcs) else {}
             idx_field = raw_tc.get("index") if isinstance(raw_tc, dict) else None
 
+            # Resolve this entry to a tool-id key, mirroring
+            # `raycast_api.ChatResult._merge_tool_calls`. Phase 1 carries
+            # id+index, phase 2 only index, phase 3 only id.
             tool_id: str | None = None
             if tc.id:
                 tool_id = tc.id
@@ -699,6 +743,7 @@ class RaycastBackend:
 
             block_idx = state.tool_id_to_block.get(tool_id)
             if block_idx is None:
+                # New tool_use block. Close any open text/thinking block first.
                 if state.kind is not None:
                     events.append(build_content_block_stop(state.index))
                 state.index += 1
@@ -711,11 +756,22 @@ class RaycastBackend:
                         block_idx, tool_use_id=tool_id, name=tc.name or ""
                     )
                 )
+                # Streaming providers (GPT) deliver arguments as deltas
+                # across chunks and re-state the full string in a final
+                # summary chunk; non-streaming-args providers (Gemini)
+                # only send args in the final summary. Emit args in
+                # both cases when this is the first appearance — final
+                # summary then IS the full string.
                 if tc.arguments:
                     events.append(build_input_json_delta(block_idx, tc.arguments))
                     acc.add_tool_args(tool_id, tc.arguments)
                 continue
 
+            # Existing block. Final summary chunks restate the full args
+            # string; if we already streamed deltas, that restatement is
+            # a duplicate (skip). If we streamed nothing (the streaming
+            # provider didn't send mid-arg deltas — Gemini path again),
+            # the summary IS the args — emit it once.
             if is_final_summary:
                 if tc.arguments and not acc.tool_args.get(tool_id):
                     events.append(build_input_json_delta(block_idx, tc.arguments))

src/beaver_gateway/cli.py

@@ -1,41 +1,533 @@
-"""Process entrypoint: logging, signals, ``.env``, the config, then ``app.run``."""
+"""Process entrypoint.
+
+Phase 1.4 — async ``main``: install uvloop, load the user config, build
+registries + per-agent backends (only ``RaycastBackend`` so far), wire
+each frontend with a ``GatewayRuntime``, and run all
+``frontend.serve()`` coroutines concurrently. Without any frontends we
+still print the Phase 0 DoD line and exit cleanly so the bare skeleton
+keeps working.
+
+Phase 2.1 — when the user declares any ``McpServer``, we additionally
+build the internal MCP aggregator app and run it on
+``127.0.0.1:INTERNAL_MCP_PORT`` as another task inside the same
+TaskGroup. URLs are surfaced through ``GatewayRuntime.mcp_internal_urls``
+so Phase 2.2's ClaudeCode adapter can find them.
+
+Phase 3 — the same aggregator backs the external ``McpServerFrontend``;
+``cli`` doesn't have to know that the frontend reverse-proxies into it,
+it just keeps the aggregator running for anyone who needs it.
+"""
 
 from __future__ import annotations
 
 import asyncio
 import contextlib
+import functools
 import logging
 import signal
+from contextlib import AsyncExitStack
+from typing import TYPE_CHECKING, Any
 
+import psycopg
+import uvicorn
 import uvloop
 from dotenv import load_dotenv
+from pgqueuer import PsycopgDriver
+from raycast_api import Client as RaycastClient
+from raycast_api.config import Config as RaycastConfig
 
-from beaver_gateway import app, config
-from beaver_gateway.security.redact import install as install_redaction
-from beaver_gateway.security.redact import load_secrets as load_secrets_to_mask
+from beaver_gateway import config_loader
+from beaver_gateway.agents.claude import ClaudeAgent
+from beaver_gateway.agents.raycast import RaycastAgent
+from beaver_gateway.backends.claude_sdk import (
+    ClaudeSdkBackend,
+    RunnerConfig,
+    UsageEvent,
+)
+from beaver_gateway.backends.raycast import RaycastBackend
+from beaver_gateway.core.auth import TokenStore
+from beaver_gateway.core.bus import EventBus
+from beaver_gateway.core.conversations import Conversations
+from beaver_gateway.core.envelope import Envelope
+from beaver_gateway.core.gateway_tools import build_tool_server
+from beaver_gateway.core.redact import install as install_redaction
+from beaver_gateway.core.redact import load_secrets as load_secrets_to_mask
+from beaver_gateway.core.registry import AgentRegistry, Gateway, McpRegistry
+from beaver_gateway.core.rotation import Rotation, RotationPolicy
+from beaver_gateway.core.scheduler import Scheduler
+from beaver_gateway.core.sessions import SessionPool
+from beaver_gateway.frontends._auth import require_token
+from beaver_gateway.frontends.base import GatewayRuntime
+from beaver_gateway.frontends.root import build_root_app
+from beaver_gateway.mcp.internal_app import build_internal_app
 from beaver_gateway.settings import Settings
+from beaver_gateway.storage import (
+    Database,
+    PostgresSessionStore,
+    Usage,
+    append_audit,
+    append_usage,
+)
+
+if TYPE_CHECKING:
+    from claude_agent_sdk import McpSdkServerConfig
+    from fastmcp import FastMCP
+    from fastmcp.tools.base import Tool as FastMCPTool
+    from starlette.applications import Starlette
+    from starlette.requests import Request
+    from starlette.types import ASGIApp
+
+    from beaver_gateway.backends.base import Backend
+    from beaver_gateway.core.policy import ToolAudit
+    from beaver_gateway.mcp.types import McpServerT
+
+
+_log = logging.getLogger("beaver_gateway.cli")
 
 
 def main() -> None:
+    """Sync wrapper: uvloop loop factory + asyncio.run."""
     logging.basicConfig(
         level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s"
     )
     install_redaction()
-    _sigterm_as_interrupt()
-    asyncio.run(_run(), loop_factory=uvloop.new_event_loop)
+    _install_sigterm_handler()
+    asyncio.run(_async_main(), loop_factory=uvloop.new_event_loop)
 
 
-async def _run() -> None:
-    load_dotenv(override=False)
-    load_secrets_to_mask()
-    settings = Settings()  # ty: ignore[missing-argument]
-    gateway = config.load(settings.config_path)
-    await app.run(gateway, settings)
+def _install_sigterm_handler() -> None:
+    """Turn SIGTERM into a normal interpreter exit.
 
+    Python's default SIGTERM disposition kills the process outright, so
+    neither ``AsyncExitStack`` unwinding nor ``atexit`` hooks run. That
+    matters because every ``claude`` we spawn lives in its own session
+    (ptyprocess calls ``setsid``), which makes it immune to the signal
+    that took us down — a hard SIGTERM leaves one orphaned CLI per live
+    session, each holding hundreds of MB. Raising ``KeyboardInterrupt``
+    instead routes ``docker stop`` / ``systemctl stop`` through the same
+    shutdown path as Ctrl-C, which does reap them.
+
+    Only installed when we own the main thread's signal handlers; under
+    an embedding host that isn't ours to take.
+    """
 
-def _sigterm_as_interrupt() -> None:
     def _raise_interrupt(_signum: int, _frame: object) -> None:
         raise KeyboardInterrupt
 
     with contextlib.suppress(ValueError, OSError):
         signal.signal(signal.SIGTERM, _raise_interrupt)
+
+
+async def _async_main() -> None:
+    # Populate ``os.environ`` from ``.env`` before anything else so the
+    # user's ``config.py`` can read its own secrets via ``os.environ[...]``
+    # (Firefly PAT, third-party MCP creds, etc.). ``Settings`` already
+    # reads ``.env`` independently via pydantic-settings, but that path
+    # populates Settings fields, not the process environment.
+    # ``override=False``: real env vars (Docker, systemd) win over .env.
+    load_dotenv(override=False)
+    # Only now does the process environment hold the credentials the
+    # redactor masks literally (in Docker they arrive via ``env_file``).
+    load_secrets_to_mask()
+
+    settings = Settings()  # ty: ignore[missing-argument]
+    gateway = config_loader.load(settings.config_path)
+
+    agents = AgentRegistry(gateway.agents)
+    mcps = McpRegistry(gateway.mcps)
+
+    # Phase 4.1 — open the async DB and run create_all once. Engine
+    # pool is process-wide; ``dispose()`` after the TaskGroup unwinds.
+    db = Database(settings.database_url)
+    await db.create_all()
+
+    # Phase 4.2 — TokenStore now reads from the DB (in-memory cache
+    # primed at start, TTL-refreshed, last_used_at flushed by a
+    # background task). BOOTSTRAP_TOKENS layers on top so first-run /
+    # examples still work without DB writes.
+    token_store = TokenStore(
+        db,
+        bootstrap=TokenStore.parse_bootstrap(settings.bootstrap_tokens),
+        bootstrap_scopes=TokenStore.parse_bootstrap_scopes(settings.bootstrap_tokens),
+    )
+
+    async with AsyncExitStack() as stack:
+        stack.push_async_callback(db.dispose)
+        await token_store.start()
+        stack.push_async_callback(token_store.stop)
+        # Internal MCP URLs must exist before we construct any
+        # ClaudeSdkBackend - adapters bake the URLs into their
+        # ``mcp_servers`` at construction time. The
+        # ``mcp_servers`` map is used by the Raycast backend, which
+        # needs in-process ``list_tools`` / ``call_tool`` access (the
+        # Raycast wire has no native MCP concept).
+        internal_app, internal_urls, mcp_servers = _build_internal_mcp(
+            gateway.mcps, settings=settings
+        )
+
+        # Prefetch tool catalogs for every MCP so RaycastAgent requests
+        # don't pay a per-turn list_tools roundtrip and so a broken MCP
+        # surfaces at startup instead of mid-conversation.
+        mcp_tools = await _prefetch_mcp_tools(mcp_servers)
+
+        pool = SessionPool()
+        bus = EventBus()
+        late = _LateConversations()
+        session_store = PostgresSessionStore(db)
+        backends: dict[str, Backend] = await _build_backends(
+            settings=settings,
+            agents=agents,
+            stack=stack,
+            db=db,
+            session_store=session_store,
+            mcp_internal_urls=internal_urls,
+            mcp_servers=mcp_servers,
+            mcp_tools=mcp_tools,
+            pool=pool,
+            late=late,
+        )
+        conversations = Conversations(
+            db=db,
+            agents=agents,
+            backends=backends,
+            bus=bus,
+            pool=pool,
+            store=session_store,
+            texts=gateway.texts,
+            frontends=gateway.frontends,
+            envelope=Envelope(
+                watch=gateway.watch, tz=gateway.tz, recall=gateway.recall
+            ),
+            distiller=gateway.distiller,
+            user_sink=gateway.user_sink,
+        )
+        late.conversations = conversations
+        scheduler = Scheduler(
+            conversations=conversations,
+            jobs=gateway.jobs,
+            driver=await _pgqueuer_driver(settings.database_url, stack),
+            budget=gateway.budget,
+            rotation=Rotation(
+                conversations, gateway.rotation or RotationPolicy(tz=gateway.tz)
+            ),
+            tz=gateway.tz,
+        )
+        conversations.scheduler = scheduler
+
+        runtime = GatewayRuntime(
+            agents=agents,
+            mcps=mcps,
+            backends=backends,
+            token_store=token_store,
+            db=db,
+            mcp_internal_urls=internal_urls,
+            admin_user=settings.admin_user,
+            admin_pass=settings.admin_pass,
+            session_secret=settings.session_secret,
+            frontends=tuple(gateway.frontends),
+            conversations=conversations,
+            bus=bus,
+            pool=pool,
+            scheduler=scheduler,
+            public_url=gateway.public_url.rstrip("/") if gateway.public_url else None,
+        )
+
+        for fe in gateway.frontends:
+            fe.configure(runtime)
+
+        _log.info(
+            "beaver-gateway: loaded %d agents, %d mcps, %d frontends",
+            len(agents),
+            len(mcps),
+            len(gateway.frontends),
+        )
+
+        # Keep the Phase 0 DoD line on stdout for grep-friendly smoke
+        # tests, in addition to the structured log line above.
+        print(
+            f"beaver-gateway: loaded {len(agents)} agents, "
+            f"{len(mcps)} mcps, {len(gateway.frontends)} frontends"
+        )
+
+        if not gateway.frontends:
+            # No external listeners → nothing to serve. The internal
+            # MCP app has no consumer on its own, so we skip running
+            # it in this path and exit cleanly (Phase 0 DoD).
+            return
+
+        await conversations.start()
+        stack.push_async_callback(conversations.stop)
+        await scheduler.start()
+        stack.push_async_callback(scheduler.stop)
+        hooks = scheduler.app(
+            functools.partial(_authorize_hook, runtime=runtime, scope="api")
+        )
+        async with asyncio.TaskGroup() as tg:
+            tg.create_task(pool.reap_loop())
+            if internal_app is not None:
+                tg.create_task(_serve_internal_mcp(internal_app, settings=settings))
+            tg.create_task(_serve_root(gateway, extra={"/hooks": hooks}))
+            if gateway.watch is not None:
+                tg.create_task(gateway.watch.run())
+            for fe in gateway.frontends:
+                tg.create_task(fe.serve())
+
+
+async def _authorize_hook(
+    request: Request, *, runtime: GatewayRuntime, scope: str
+) -> str:
+    return await require_token(request, runtime, scope=scope)
+
+
+async def _pgqueuer_driver(url: str, stack: AsyncExitStack) -> PsycopgDriver | None:
+    """A dedicated autocommit connection for pgqueuer's LISTEN/NOTIFY."""
+    plain = _plain_postgres_url(url)
+    if plain is None:
+        return None
+    conn = await psycopg.AsyncConnection.connect(plain, autocommit=True)
+    stack.push_async_callback(conn.close)
+    return PsycopgDriver(conn)
+
+
+def _plain_postgres_url(url: str) -> str | None:
+    for prefix in ("postgresql+psycopg://", "postgresql://", "postgres://"):
+        if url.startswith(prefix):
+            return "postgresql://" + url[len(prefix) :]
+    return None
+
+
+async def _serve_root(gateway: Gateway, *, extra: dict[str, ASGIApp]) -> None:
+    app = build_root_app(gateway.frontends, extra=extra)
+    config = uvicorn.Config(app, host=gateway.host, port=gateway.port, log_level="info")
+    _log.info(
+        "gateway on http://%s:%d - %s",
+        gateway.host,
+        gateway.port,
+        ", ".join([*(fe.path for fe in gateway.frontends if fe.path), *extra])
+        or "no http frontends",
+    )
+    await uvicorn.Server(config).serve()
+
+
+def _build_internal_mcp(
+    mcps: list[McpServerT], *, settings: Settings
+) -> tuple[Starlette | None, dict[str, str], dict[str, FastMCP]]:
+    """Build the aggregator app + URL map + server map, or empty equivalents.
+
+    The URL map is always handed out (frontends may still introspect
+    ``runtime.mcp_internal_urls`` even if nothing is configured); the
+    app is ``None`` when there are no MCPs to mount, so the caller
+    skips the uvicorn task entirely. The server map is the in-process
+    handle the Raycast backend needs to splice MCP tools into its
+    requests — empty when no MCPs are configured.
+    """
+    if not mcps:
+        return None, {}, {}
+    return build_internal_app(mcps, host="127.0.0.1", port=settings.internal_mcp_port)
+
+
+async def _prefetch_mcp_tools(
+    servers: dict[str, FastMCP],
+) -> dict[str, list[FastMCPTool]]:
+    """Eagerly enumerate tools per MCP so the Raycast loop has a static catalog.
+
+    Each underlying proxy is allowed to fail independently — a broken
+    MCP shouldn't take down the whole gateway. The result has one entry
+    per MCP that responded; agents that ``expose_mcps`` a missing entry
+    will simply expose no tools from it (logged once per request).
+    """
+    out: dict[str, list[FastMCPTool]] = {}
+    for name, server in servers.items():
+        try:
+            out[name] = list(await server.list_tools())
+        except Exception:  # noqa: BLE001 — proxy can raise any transport error; we degrade per-MCP rather than fail the whole gateway
+            _log.exception("failed to list tools for MCP %r — skipping", name)
+            out[name] = []
+    return out
+
+
+async def _serve_internal_mcp(app: Starlette, *, settings: Settings) -> None:
+    """Run the internal MCP aggregator on loopback.
+
+    Bound to ``127.0.0.1`` (never EXPOSE'd) — only the in-process
+    ClaudeCode subprocess reaches it. Logged at ``warning`` level so
+    we don't drown the gateway's own logs in per-request noise.
+    """
+    config = uvicorn.Config(
+        app,
+        host="127.0.0.1",
+        port=settings.internal_mcp_port,
+        log_level="warning",
+        loop="uvloop",
+    )
+    server = uvicorn.Server(config)
+    _log.info(
+        "internal MCP aggregator on http://127.0.0.1:%d/mcp/<name>",
+        settings.internal_mcp_port,
+    )
+    await server.serve()
+
+
+class _LateConversations:
+    """Backends need a tool-server factory before the service that backs it exists."""
+
+    conversations: Conversations | None = None
+
+    def server(
+        self, key: str, _kind: str, names: tuple[str, ...]
+    ) -> McpSdkServerConfig | None:
+        if self.conversations is None or not names:
+            return None
+        return build_tool_server(self.conversations, conversation_key=key, names=names)
+
+    async def ask(self, key: str, payload: dict[str, Any]) -> str:
+        if self.conversations is None:
+            msg = "conversations service is not up yet"
+            raise RuntimeError(msg)
+        answer = await self.conversations.ask(key, payload)
+        return self.conversations.answer_text(answer)
+
+
+async def _build_backends(
+    *,
+    settings: Settings,
+    agents: AgentRegistry,
+    stack: AsyncExitStack,
+    db: Database,
+    session_store: PostgresSessionStore,
+    mcp_internal_urls: dict[str, str],
+    mcp_servers: dict[str, FastMCP],
+    mcp_tools: dict[str, list[FastMCPTool]],
+    pool: SessionPool,
+    late: _LateConversations,
+) -> dict[str, Backend]:
+    """Construct one backend per agent name.
+
+    The Raycast ``Client`` is shared across every ``RaycastAgent``
+    (bearer + device-id are process-wide), so we open it lazily - only
+    when at least one ``RaycastAgent`` is present - and close it via
+    the caller's exit stack.
+
+    Each :class:`ClaudeAgent` gets its own :class:`ClaudeSdkBackend`
+    (own live-session pool, own prompt and MCP set); all of them share
+    the session store and the usage sink.
+    """
+    backends: dict[str, Backend] = {}
+
+    raycast_agents = [a for a in agents if isinstance(a, RaycastAgent)]
+    if raycast_agents:
+        client = await _try_open_raycast_client(settings, stack)
+        if client is not None:
+            raycast_backend = RaycastBackend(
+                client, mcp_servers=mcp_servers, mcp_tools=mcp_tools
+            )
+            for a in raycast_agents:
+                backends[a.name] = raycast_backend
+
+    runner = RunnerConfig(user=settings.claude_runner_user, home=settings.claude_home)
+    mcp_tool_names = {
+        name: [t.name for t in tools] for name, tools in mcp_tools.items()
+    }
+
+    async def record_usage(event: UsageEvent) -> None:
+        row = Usage(
+            agent_name=event.agent_name,
+            conversation_id=event.conversation_id,
+            session_id=event.session_id,
+            model=event.model,
+            effort=event.effort,
+            input_tokens=event.usage.input_tokens,
+            output_tokens=event.usage.output_tokens,
+            cache_read_tokens=event.usage.cache_read_tokens,
+            cache_creation_tokens=event.usage.cache_creation_tokens,
+            context_tokens=event.usage.context_tokens,
+            cost_usd=event.usage.cost_usd,
+            duration_ms=event.usage.duration_ms,
+            num_turns=event.usage.num_turns,
+            model_usage=event.usage.model_usage,
+        )
+        try:
+            async with db.session() as session:
+                await append_usage(session, row)
+        except Exception:  # noqa: BLE001
+            _log.exception("usage write failed for %s", event.agent_name)
+
+    async def record_tool(event: ToolAudit) -> None:
+        detail = {
+            "conversation": event.conversation,
+            "kind": event.kind,
+            "tool": event.tool,
+            "decision": event.decision,
+            "reason": event.reason,
+            "brief": event.brief,
+        }
+        try:
+            async with db.session() as session:
+                await append_audit(
+                    session,
+                    actor=f"agent:{event.agent}",
+                    kind="tool_call",
+                    agent_name=event.agent,
+                    detail=detail,
+                )
+        except Exception:  # noqa: BLE001
+            _log.exception("tool audit write failed for %s", event.agent)
+
+    for a in agents:
+        if isinstance(a, ClaudeAgent):
+            adapter = ClaudeSdkBackend(
+                agent=a,
+                mcp_internal_urls=mcp_internal_urls,
+                session_store=session_store,
+                mcp_tool_names=mcp_tool_names,
+                runner=runner,
+                usage_sink=record_usage,
+                pool=pool,
+                tool_server=functools.partial(late.server, names=a.gateway_tools),
+                asker=late.ask,
+                audit_sink=record_tool,
+            )
+            await stack.enter_async_context(adapter)
+            backends[a.name] = adapter
+
+    return backends
+
+
+async def _try_open_raycast_client(
+    settings: Settings, stack: AsyncExitStack
+) -> RaycastClient | None:
+    """Open a ``raycast_api.Client`` if creds are available, else warn + skip.
+
+    Skipping is gentler than failing startup: the user can bring up the
+    gateway, list their agents, and still hit a ``ClaudeAgent``; affected
+    ``RaycastAgent`` instances 503 with a specific message at request time.
+    Missing ``raycast.json`` falls into the same bucket — first-run
+    users won't have it yet.
+    """
+    if not settings.raycast_bearer:
+        _log.warning(
+            "RaycastAgent present but RAYCAST_BEARER is unset — those agents will 503"
+        )
+        return None
+    if not settings.raycast_device_id:
+        _log.warning(
+            "RaycastAgent present but RAYCAST_DEVICE_ID is unset — those agents "
+            "will 503 (generate once with `python -c 'import secrets; "
+            "print(secrets.token_hex(32))'`)"
+        )
+        return None
+    if not settings.raycast_config_path.exists():
+        _log.warning(
+            "RaycastAgent present but %s is missing — those agents will 503",
+            settings.raycast_config_path,
+        )
+        return None
+
+    config = RaycastConfig.load(settings.raycast_config_path)
+    client = RaycastClient(
+        config=config,
+        bearer_token=settings.raycast_bearer,
+        device_id=settings.raycast_device_id,
+        locale=settings.raycast_locale,
+    )
+    return await stack.enter_async_context(client)

src/beaver_gateway/config_loader.py

@@ -1,10 +1,20 @@
-"""``Gateway`` - the one object a setup's ``config.py`` assembles - and its loader."""
+"""Load the user's ``/config/config.py``.
+
+The config file is regular Python. We ``exec`` it in a namespace seeded
+with the public surface the user is expected to use (``ClaudeAgent``,
+``RaycastAgent``, ``McpServer``, ``ExposedMcp``, ``Gateway``). The file
+must assign a top-level ``gateway = Gateway(...)``.
+
+Element-level validation already happens at construction time — agents
+and ``McpServer`` factories are pydantic models that reject garbage.
+What we *can't* validate at construction is the ``Gateway`` container
+itself (deliberately a plain dataclass per PRD), so we type-check its
+contents here before handing it back.
+"""
 
 from __future__ import annotations
 
-import re
 import sys
-from dataclasses import dataclass, field
 from typing import TYPE_CHECKING, Any
 
 from croniter import croniter
@@ -12,60 +22,18 @@ from croniter import croniter
 from beaver_gateway.agents.base import BaseAgent, ExposedMcp
 from beaver_gateway.agents.claude import ClaudeAgent
 from beaver_gateway.agents.raycast import RaycastAgent
-from beaver_gateway.conversations.texts import ConversationTexts
+from beaver_gateway.core.conversations import ConversationTexts
+from beaver_gateway.core.registry import Gateway
+from beaver_gateway.core.scheduler import Job
 from beaver_gateway.frontends.base import Frontend
-from beaver_gateway.jobs.scheduler import Job
 from beaver_gateway.mcp.types import HttpMcp, McpServer, PythonToolMcp, StdioMcp
 
 if TYPE_CHECKING:
-    from collections.abc import Awaitable, Callable
     from pathlib import Path
 
-    from beaver_gateway.conversations.distill import Distiller
-    from beaver_gateway.conversations.envelope import RecallContext
-    from beaver_gateway.conversations.rotation import RotationPolicy
-    from beaver_gateway.conversations.texts import UserSaid
-    from beaver_gateway.jobs.scheduler import Budget
-    from beaver_gateway.mcp.types import McpServerT
-    from beaver_gateway.vault.watch import VaultWatch
-
-__all__ = ["ConfigError", "Gateway", "load"]
-
-_SCOPE = re.compile(r"[a-z][a-z0-9_-]*")
-
-
-@dataclass(slots=True)
-class Gateway:
-    agents: list[BaseAgent] = field(default_factory=list)
-    mcps: list[McpServerT] = field(default_factory=list)
-    frontends: list[Frontend] = field(default_factory=list)
-    texts: ConversationTexts | None = None
-    """Every string the gateway says to a model; ``None`` = the English defaults."""
-    jobs: list[Job] = field(default_factory=list)
-    """Cron, webhook and event jobs run by ``jobs.scheduler``."""
-    rotation: RotationPolicy | None = None
-    """When a master is replaced by a fresh one; ``None`` keeps the defaults."""
-    watch: VaultWatch | None = None
-    """Directory watcher feeding the envelope; ``None`` = no change block."""
-    recall: Callable[[RecallContext], str | None] | None = None
-    """Envelope lookup on the user's text: pointers the setup derives from its files."""
-    user_sink: Callable[[UserSaid], Awaitable[None] | None] | None = None
-    """Sees every user message entering a master or branch turn."""
-    budget: Budget | None = None
-    """Subscription utilisation past which non-critical jobs wait."""
-    distiller: Distiller | None = None
-    """Who closes deep chats and where digests and the index live."""
-    tz: str = "UTC"
-    """Zone for the envelope clock, cron expressions and the rotation hour."""
-    host: str = "0.0.0.0"  # noqa: S104
-    port: int = 8000
-    """The one listener; every HTTP frontend is mounted under its ``path``."""
-    public_url: str | None = None
-    """Origin the reverse proxy shows the world; ``None`` derives it per request."""
-
 
 class ConfigError(Exception):
-    pass
+    """User config file is missing, unreadable, or structurally wrong."""
 
 
 _PUBLIC_NAMES: dict[str, Any] = {
@@ -81,7 +49,7 @@ _McpInstance = StdioMcp | HttpMcp | PythonToolMcp
 
 
 def load(path: Path) -> Gateway:
-    """Execute a setup's ``config.py`` and return its top-level ``gateway``."""
+    """Execute ``path`` and return its top-level ``gateway`` object."""
     try:
         source = path.read_text(encoding="utf-8")
     except FileNotFoundError as exc:
@@ -92,11 +60,12 @@ def load(path: Path) -> Gateway:
         raise ConfigError(msg) from exc
 
     code = compile(source, str(path), "exec")
+    # Siblings of the config (``policy.py``, ``mcps/``) import by name.
     parent = str(path.resolve().parent)
     if parent not in sys.path:
         sys.path.insert(0, parent)
     namespace: dict[str, Any] = {"__file__": str(path), **_PUBLIC_NAMES}
-    exec(code, namespace)  # noqa: S102
+    exec(code, namespace)  # noqa: S102 - exec'ing user config is the feature
 
     try:
         gw = namespace["gateway"]
@@ -138,12 +107,6 @@ def _validate(gw: Gateway, path: Path) -> None:
                 f"got {type(f).__name__}"
             )
             raise ConfigError(msg)
-        if f.scope is not None and not _SCOPE.fullmatch(f.scope):
-            msg = (
-                f"{path}: gateway.frontends[{i}] has scope {f.scope!r}; "
-                f"a frontend scope is a lowercase name like 'voice', never '*'"
-            )
-            raise ConfigError(msg)
     names: set[str] = set()
     for i, j in enumerate(gw.jobs):
         if not isinstance(j, Job):

src/beaver_gateway/conversations/__init__.py

@@ -1 +0,0 @@
-"""Conversations: rows, queue, seeds, turns, questions, closing, rotation, envelope."""

src/beaver_gateway/conversations/closing.py

@@ -1,322 +0,0 @@
-"""Ending conversations: the distiller, the line cap, the master handover."""
-
-from __future__ import annotations
-
-import asyncio
-import contextlib
-import inspect
-import logging
-from dataclasses import dataclass
-from datetime import UTC, datetime, timedelta
-from typing import TYPE_CHECKING, Any, cast
-
-from beaver_gateway.conversations.distill import (
-    Digest,
-    DistillContext,
-    LineCap,
-    append_index,
-    check_digest,
-    find_digest,
-    index_line,
-    trim_summary,
-    written_paths,
-)
-from beaver_gateway.conversations.questions import Questions
-from beaver_gateway.conversations.state import aware
-from beaver_gateway.conversations.texts import NewDayContext
-
-if TYPE_CHECKING:
-    from beaver_gateway.conversations.rotation import HandoutContext
-    from beaver_gateway.storage.models import Conversation, InjectQueueItem
-
-__all__ = ["Closing", "DistillResult"]
-
-_log = logging.getLogger(__name__)
-
-CLOSE_WAIT = 0.25
-CLOSE_TRIES = 40
-CAP_TRIES = 3
-
-
-@dataclass(frozen=True, slots=True)
-class DistillResult:
-    conversation: Conversation
-    fork: Conversation
-    text: str
-    digest: Digest | None
-    error: str | None
-    trimmed: bool
-
-
-class Closing(Questions):
-    async def close(self, conv: Conversation) -> Conversation:
-        row = await self.set_status(conv, "closed")
-        with contextlib.suppress(LookupError):
-            await self._backend(conv.agent_name).close(conv.external_id)
-        return row
-
-    async def request_close(self, conv: Conversation) -> Conversation:
-        """``close_chat`` from inside a turn: the chat closes once the turn ends."""
-        if conv.kind != "deep":
-            msg = f"only deep chats close this way, {conv.external_id} is {conv.kind}"
-            raise ValueError(msg)
-        return await self.set_flags(conv, {"close_requested": True})
-
-    async def idle(
-        self,
-        *,
-        kind: str,
-        days: int,
-        since: datetime | None = None,
-        limit: int | None = None,
-    ) -> list[Conversation]:
-        """Open conversations of ``kind`` with a session, quiet for ``days``."""
-        now = datetime.now(UTC)
-        cutoff = now - timedelta(days=days)
-        out: list[tuple[datetime, Conversation]] = []
-        for conv in await self.find(status="open", kind=kind, limit=10_000):
-            if conv.session_id is None:
-                continue
-            last = aware(conv.last_activity_at or conv.created_at)
-            if last > cutoff or (since is not None and last < since):
-                continue
-            out.append((last, conv))
-        out.sort(key=lambda pair: pair[0])
-        rows = [conv for _, conv in out]
-        return rows[:limit] if limit is not None else rows
-
-    async def distill(
-        self, conv: Conversation, *, reason: str = "api"
-    ) -> DistillResult:
-        """Fork under the distiller: digest checked and indexed, merge to the master."""
-        if self._distiller is None:
-            msg = "no distiller configured (Gateway(distiller=...))"
-            raise RuntimeError(msg)
-        if conv.kind != "deep":
-            msg = f"only deep chats are distilled, {conv.external_id} is {conv.kind}"
-            raise ValueError(msg)
-        row = await self.get_row(cast("int", conv.id)) or conv
-        if row.status != "open":
-            msg = f"conversation {row.external_id} is {row.status}"
-            raise ValueError(msg)
-        if await self.busy(row):
-            msg = f"conversation {row.external_id} is busy"
-            raise RuntimeError(msg)
-        memory = bool(row.flags.get("memory", True))
-        chat_name = await self.chat_name(row)
-        ctx = DistillContext(
-            conversation=row,
-            title=await self.implied_title(row),
-            source=await self.window_of(row),
-            chat_name=chat_name,
-            memory=memory,
-            reason=reason,
-            day=datetime.now(UTC).astimezone().date(),
-        )
-        prompt = await self._distill_prompt(ctx)
-        started = datetime.now(UTC)
-        self._bus.publish(
-            "distill.start",
-            conversation_id=row.external_id,
-            reason=reason,
-            memory=memory,
-        )
-        result = await self.fork(
-            row,
-            prompt,
-            strip_tools=True,
-            agent=self._distiller.agent,
-            title=f"digest: {chat_name}",
-        )
-        text, trimmed = trim_summary(result.text)
-        digest: Digest | None = None
-        error: str | None = None
-        if memory:
-            written = written_paths(result.capture.synthesized_messages)
-            path = find_digest(self._distiller, since=started, written=written)
-            if path is None:
-                error = self._texts.digest_missing
-            else:
-                checked = check_digest(path, self._distiller)
-                if isinstance(checked, str):
-                    error = f"{path.name}: {checked}"
-                else:
-                    digest = checked
-                    append_index(self._distiller, index_line(digest, chat_name))
-        if error is not None:
-            _log.warning("distill of %s: %s", row.external_id, error)
-        master = await self.open_master()
-        if master is not None and text:
-            note = self._texts.closed.format(
-                chat=chat_name,
-                digest=(
-                    self._texts.closed_digest.format(digest=digest.path.stem)
-                    if digest
-                    else ""
-                ),
-                text=text,
-            )
-            await self.inject(master, note, urgency="normal", origin="digest")
-        await self.close(row)
-        row = await self.set_flags(
-            row,
-            {
-                "close_requested": None,
-                "closed_reason": reason,
-                "digest": str(digest.path) if digest else None,
-                "digest_error": error,
-            },
-        )
-        self._bus.publish(
-            "conversation.distilled",
-            conversation_id=row.external_id,
-            fork=result.conversation.external_id,
-            reason=reason,
-            memory=memory,
-            digest=str(digest.path) if digest else None,
-            error=error,
-            text=text,
-            trimmed=trimmed,
-            master=master.external_id if master is not None else None,
-        )
-        return DistillResult(
-            conversation=row,
-            fork=result.conversation,
-            text=text,
-            digest=digest,
-            error=error,
-            trimmed=trimmed,
-        )
-
-    async def _distill_prompt(self, ctx: DistillContext) -> str:
-        source = self._texts.distill
-        if source is None:
-            template = (
-                self._texts.distill_prompt
-                if ctx.memory
-                else self._texts.distill_prompt_no_memory
-            )
-            return template.format(
-                chat=ctx.chat_name, reason=ctx.reason, day=ctx.day.isoformat()
-            )
-        produced: Any = source(ctx)
-        return await produced if inspect.isawaitable(produced) else produced
-
-    async def _close_after_turn(self, conv: Conversation) -> None:
-        for _ in range(CLOSE_TRIES):
-            if await self.busy(conv):
-                await asyncio.sleep(CLOSE_WAIT)
-                continue
-            try:
-                await self.distill(conv, reason="close_chat")
-            except RuntimeError as exc:
-                _log.info("closing %s: %s, retrying", conv.external_id, exc)
-                await asyncio.sleep(CLOSE_WAIT)
-                continue
-            except Exception:  # noqa: BLE001
-                _log.exception("closing %s after its turn failed", conv.external_id)
-            return
-        _log.warning("closing %s: still busy, giving up", conv.external_id)
-
-    async def before_turn(self, conv: Conversation) -> str | None:
-        cap = LineCap.from_flags(conv.flags.get("line_cap"))
-        if cap is None:
-            return None
-        try:
-            return cap.path.read_text(encoding="utf-8") if cap.path.exists() else ""
-        except OSError:
-            _log.exception("line cap: cannot read %s", cap.path)
-            return None
-
-    async def after_turn(self, conv: Conversation, before: str | None) -> None:
-        row = await self.get_row(cast("int", conv.id))
-        if row is None:
-            return
-        if row.kind == "deep" and row.flags.get("close_requested"):
-            self._track(asyncio.create_task(self._close_after_turn(row)))
-        cap = LineCap.from_flags(row.flags.get("line_cap"))
-        if cap is not None and before is not None:
-            await self._enforce_cap(row, cap, before)
-
-    async def _enforce_cap(self, conv: Conversation, cap: LineCap, before: str) -> None:
-        if not cap.path.exists():
-            return
-        after = cap.path.read_text(encoding="utf-8")
-        lines = sum(1 for line in after.splitlines() if line.strip())
-        if lines <= cap.max_lines:
-            return
-        if before:
-            cap.path.write_text(before, encoding="utf-8")
-        else:
-            cap.path.unlink()
-        attempts = int(conv.flags.get("line_cap_attempts", 0) or 0) + 1
-        await self.set_flags(conv, {"line_cap_attempts": attempts})
-        self._bus.publish(
-            "line_cap.bounced",
-            conversation_id=conv.external_id,
-            path=str(cap.path),
-            lines=lines,
-            max_lines=cap.max_lines,
-            attempt=attempts,
-        )
-        _log.warning(
-            "line cap: %s came back with %d lines (cap %d), restored; attempt %d",
-            cap.path,
-            lines,
-            cap.max_lines,
-            attempts,
-        )
-        if attempts > CAP_TRIES:
-            return
-        await self.inject(
-            conv,
-            self._texts.too_long.format(
-                name=cap.path.name, lines=lines, max_lines=cap.max_lines
-            ),
-            urgency="urgent",
-            origin="cap",
-            interrupt=False,
-        )
-
-    async def handout(self, conv: Conversation, ctx: HandoutContext) -> str:
-        """The closing master's last turn."""
-        source = self._texts.handout
-        if isinstance(source, str):
-            prompt = source.format(day=ctx.day.isoformat(), reason=ctx.reason)
-        else:
-            produced: Any = source(ctx)
-            prompt = await produced if inspect.isawaitable(produced) else produced
-        self._bus.publish(
-            "handout.start", conversation_id=conv.external_id, day=ctx.day.isoformat()
-        )
-        try:
-            text, _ = await self.run_text_turn(conv, prompt, origin="handout")
-        except Exception:  # noqa: BLE001
-            _log.exception("handout turn on %s failed", conv.external_id)
-            text = ""
-        self._bus.publish(
-            "handout.end",
-            conversation_id=conv.external_id,
-            day=ctx.day.isoformat(),
-            text=text[:2000],
-        )
-        return text
-
-    async def new_day(
-        self, conv: Conversation, *, reason: str = "night", moved: int = 0
-    ) -> InjectQueueItem:
-        """The new master's first inject."""
-        ctx = NewDayContext(
-            day=datetime.now(UTC).astimezone().date(), reason=reason, moved=moved
-        )
-        source = self._texts.new_day
-        if isinstance(source, str):
-            text = source.format(day=ctx.day.isoformat(), reason=ctx.reason)
-        else:
-            produced: Any = source(ctx)
-            text = await produced if inspect.isawaitable(produced) else produced
-        if moved:
-            text += self._texts.moved_injects.format(moved=moved)
-        return await self.inject(
-            conv, text, urgency="urgent", origin="rotation", interrupt=False
-        )

src/beaver_gateway/conversations/context.py

@@ -1,246 +0,0 @@
-"""What a conversation's model holds: prompt granules, skills, tools, files touched."""
-
-from __future__ import annotations
-
-import re
-from collections import Counter
-from pathlib import Path
-from typing import TYPE_CHECKING, Any
-
-if TYPE_CHECKING:
-    from collections.abc import Iterable, Mapping
-
-    from beaver_gateway.agents.claude import ClaudeAgent
-    from beaver_gateway.conversations.kinds import Kind
-
-__all__ = ["bash_paths", "compose", "files_touched", "granules", "skill_sets"]
-
-CHARS_PER_TOKEN = 3.2
-READ_TOOLS = frozenset({"Read", "NotebookRead"})
-WRITE_TOOLS = frozenset({"Write", "Edit", "MultiEdit", "NotebookEdit"})
-SEARCH_TOOLS = frozenset({"Glob", "Grep"})
-PATH_KEYS = ("file_path", "notebook_path", "path", "file", "filename")
-_FRONTMATTER = re.compile(r"^---\s*\n(.*?)\n---", re.DOTALL)
-# A shell command names files as quoted strings or bare words; only notes
-# and plain text files count, code and binaries are the agent's own business.
-_SHELL_TOKEN = re.compile(r'"([^"\n]+)"|\'([^\'\n]+)\'|([^\s"\'`;|&()<>]+)')
-_NOTE_SUFFIXES = (".md", ".canvas", ".txt", ".csv", ".json", ".yaml", ".yml")
-_SHELL_CD = re.compile(r'(?:^|&&|;|\|\|)\s*cd\s+("([^"]+)"|\'([^\']+)\'|(\S+))')
-_SHELL_WRITES = re.compile(
-    r"write_text|write_bytes|>>?\s*[\"\']?[^\s&|]|\btee\b|\bsed\s+-i|\bmv\b|\brm\b"
-    r"|\btouch\b|\bcp\b|open\([^)]*[\"\'][wa]"
-)
-
-
-def bash_paths(command: str, cwd: Path | None) -> tuple[list[str], Path | None, bool]:
-    """Files a shell command names, where it leaves the shell, whether it writes.
-
-    Relative names resolve against ``cwd``, or against the directory of a
-    leading ``cd`` in the same command. Absolute names stay as they are.
-    """
-    here = cwd
-    for match in _SHELL_CD.finditer(command):
-        target = match.group(2) or match.group(3) or match.group(4) or ""
-        target = target.strip()
-        if not target:
-            continue
-        here = Path(target) if target.startswith("/") else (here or Path()) / target
-    found: list[str] = []
-    for match in _SHELL_TOKEN.finditer(command):
-        raw = (match.group(1) or match.group(2) or match.group(3) or "").strip()
-        if not raw.lower().endswith(_NOTE_SUFFIXES) or "*" in raw:
-            continue
-        path = Path(raw) if raw.startswith("/") else (here or Path()) / raw
-        text = str(path)
-        if text not in found:
-            found.append(text)
-    return found, here, bool(_SHELL_WRITES.search(command))
-
-
-def _tokens(chars: int) -> int:
-    return round(chars / CHARS_PER_TOKEN)
-
-
-def _relative(path: str, cwd: Path | None) -> str:
-    if cwd is None:
-        return path
-    try:
-        return str(Path(path).relative_to(cwd))
-    except ValueError:
-        return path
-
-
-def _tool_uses(entries: Iterable[Mapping[str, Any]]) -> Iterable[tuple[str, Any, str]]:
-    for entry in entries:
-        message = entry.get("message")
-        if entry.get("type") != "assistant" or not isinstance(message, dict):
-            continue
-        content = message.get("content")
-        if not isinstance(content, list):
-            continue
-        stamp = str(entry.get("timestamp") or "")
-        for block in content:
-            if isinstance(block, dict) and block.get("type") == "tool_use":
-                yield str(block.get("name") or "?"), block.get("input"), stamp
-
-
-def files_touched(
-    entries: Iterable[Mapping[str, Any]], *, cwd: Path | None = None
-) -> tuple[list[dict[str, Any]], dict[str, int]]:
-    """Files named in tool inputs, with how they were touched, plus tool counts."""
-    files: dict[str, dict[str, Any]] = {}
-    counts: Counter[str] = Counter()
-    # The shell keeps its directory between calls; follow it.
-    shell_cwd = cwd
-
-    def touch(raw: str, how: str, stamp: str) -> None:
-        key = _relative(raw, cwd)
-        row = files.setdefault(
-            key, {"path": key, "reads": 0, "writes": 0, "other": 0, "last_at": stamp}
-        )
-        row[how] += 1
-        row["last_at"] = max(row["last_at"], stamp)
-
-    for name, tool_input, stamp in _tool_uses(entries):
-        counts[name] += 1
-        if not isinstance(tool_input, dict):
-            continue
-        if name == "Bash" and isinstance(tool_input.get("command"), str):
-            paths, shell_cwd, writes = bash_paths(tool_input["command"], shell_cwd)
-            for raw in paths:
-                touch(raw, "writes" if writes else "reads", stamp)
-            continue
-        raw = next(
-            (v for k in PATH_KEYS if isinstance(v := tool_input.get(k), str) and v),
-            None,
-        )
-        if raw is None or (name in SEARCH_TOOLS and "file_path" not in tool_input):
-            continue
-        if name in READ_TOOLS:
-            touch(raw, "reads", stamp)
-        elif name in WRITE_TOOLS:
-            touch(raw, "writes", stamp)
-        else:
-            touch(raw, "other", stamp)
-    ordered = sorted(files.values(), key=lambda r: r["last_at"], reverse=True)
-    return ordered, dict(counts)
-
-
-def granules(agent: ClaudeAgent, kind: Kind) -> dict[str, Any]:
-    """The system prompt as its sources: tag, path, size, token estimate."""
-    sources = agent.prompt_for(kind) if agent.serves(kind) else None
-    rows: list[dict[str, Any]] = []
-    if sources is None:
-        chars = len(agent.system_prompt)
-        if chars:
-            rows.append(
-                {
-                    "tag": None,
-                    "path": None,
-                    "bytes": chars,
-                    "tokens_est": _tokens(chars),
-                }
-            )
-    else:
-        for source in sources:
-            tag, raw = source if isinstance(source, tuple) else (None, source)
-            path = Path(raw)
-            try:
-                chars = len(path.read_text(encoding="utf-8"))
-            except OSError:
-                chars = 0
-            rows.append(
-                {
-                    "tag": tag,
-                    "path": _relative(str(path), agent.cwd),
-                    "bytes": chars,
-                    "tokens_est": _tokens(chars),
-                }
-            )
-    total = sum(r["bytes"] for r in rows)
-    return {"bytes": total, "tokens_est": _tokens(total), "granules": rows}
-
-
-def _skill_meta(skill_md: Path) -> dict[str, str]:
-    try:
-        text = skill_md.read_text(encoding="utf-8")
-    except OSError:
-        return {}
-    match = _FRONTMATTER.match(text)
-    meta: dict[str, str] = {}
-    if match:
-        for line in match.group(1).splitlines():
-            key, sep, value = line.partition(":")
-            if sep:
-                meta[key.strip()] = value.strip()
-    return meta
-
-
-def skill_sets(agent: ClaudeAgent, kind: Kind) -> list[dict[str, Any]]:
-    """Skill-set directories the kind loads, with each skill's name and description."""
-    out: list[dict[str, Any]] = []
-    for directory in agent.skills_for(kind):
-        skills = []
-        try:
-            children = sorted(p for p in directory.iterdir() if p.is_dir())
-        except OSError:
-            children = []
-        for child in children:
-            skill_md = child / "SKILL.md"
-            if skill_md.is_file():
-                meta = _skill_meta(skill_md)
-                skills.append(
-                    {
-                        "name": meta.get("name") or child.name,
-                        "description": meta.get("description", ""),
-                        "path": _relative(str(child), agent.cwd),
-                    }
-                )
-        out.append(
-            {
-                "set": directory.name,
-                "path": _relative(str(directory), agent.cwd),
-                "skills": skills,
-            }
-        )
-    return out
-
-
-def compose(
-    agent: ClaudeAgent,
-    kind: Kind,
-    entries: Iterable[Mapping[str, Any]],
-    *,
-    context_tokens: int,
-) -> dict[str, Any]:
-    files, counts = files_touched(entries, cwd=agent.cwd)
-    prompt = granules(agent, kind)
-    skills = skill_sets(agent, kind)
-    skills_chars = sum(
-        len(s["name"]) + len(s["description"])
-        for group in skills
-        for s in group["skills"]
-    )
-    return {
-        "agent": {
-            "name": agent.name,
-            "model": agent.model,
-            "effort": agent.options.effort,
-        },
-        "kind": kind,
-        "context_tokens": context_tokens,
-        "prompt": prompt,
-        "skills": skills,
-        "skills_tokens_est": _tokens(skills_chars),
-        "tools": {
-            "allowed": list(agent.options.tools) if agent.options.tools else None,
-            "disallowed": list(agent.options.disallowed_tools),
-            "gateway": list(agent.gateway_tools),
-            "mcps": [m.name for m in agent.expose_mcps],
-        },
-        "files": files,
-        "tool_counts": counts,
-        "history_tokens_est": max(
-            0, context_tokens - prompt["tokens_est"] - _tokens(skills_chars)
-        ),
-    }

src/beaver_gateway/conversations/messaging.py

@@ -1,156 +0,0 @@
-"""Putting words into a conversation: a message, an inject, ``say``, a file."""
-
-from __future__ import annotations
-
-import logging
-from typing import TYPE_CHECKING, Any, cast
-
-from beaver_gateway.conversations.turns import Turns
-from beaver_gateway.frontends.files import SendFileError
-
-if TYPE_CHECKING:
-    from datetime import datetime
-
-    from beaver_gateway.conversations.injects import Priority
-    from beaver_gateway.storage.models import Conversation, InjectQueueItem
-
-__all__ = ["Messaging"]
-
-_log = logging.getLogger(__name__)
-
-
-class Messaging(Turns):
-    async def post(
-        self,
-        conv: Conversation,
-        text: str,
-        *,
-        origin: str = "user",
-        attachments: list[dict[str, Any]] | None = None,
-    ) -> InjectQueueItem:
-        item = await self._queue.push(
-            conversation_id=cast("int", conv.id),
-            priority="user",
-            origin=origin,
-            text=text,
-            attachments=attachments,
-        )
-        await self.touch_user(conv)
-        self._bus.publish(
-            "message.queued",
-            conversation_id=conv.external_id,
-            item=item.id,
-            origin=origin,
-        )
-        self._ensure_worker(cast("int", conv.id))
-        return item
-
-    async def inject(
-        self,
-        conv: Conversation,
-        text: str,
-        *,
-        urgency: Priority = "normal",
-        origin: str = "system",
-        interrupt: bool = True,
-    ) -> InjectQueueItem:
-        if conv.kind == "master" and conv.status != "open":
-            live = await self.open_master()
-            if live is None:
-                _log.error(
-                    "inject (%s) for closed master %s: no open master, it stays there",
-                    origin,
-                    conv.external_id,
-                )
-            else:
-                _log.info(
-                    "inject (%s) for closed master %s goes to %s",
-                    origin,
-                    conv.external_id,
-                    live.external_id,
-                )
-                conv = live
-        item = await self._queue.push(
-            conversation_id=cast("int", conv.id),
-            priority=urgency,
-            origin=origin,
-            text=text,
-        )
-        self._bus.publish(
-            "inject.queued",
-            conversation_id=conv.external_id,
-            item=item.id,
-            priority=urgency,
-            origin=origin,
-        )
-        if urgency == "urgent" and interrupt:
-            backend = self._backend(conv.agent_name)
-            if await backend.interrupt(conv.external_id):
-                _log.info(
-                    "conversation %s: interrupted for urgent inject", conv.external_id
-                )
-                await self._queue.mark_interrupting(item)
-        self._ensure_worker(cast("int", conv.id))
-        return item
-
-    async def interrupt(self, conv: Conversation) -> bool:
-        """Cut the running turn; the reply so far still lands."""
-        try:
-            backend = self._backend(conv.agent_name)
-        except LookupError:
-            return False
-        return await backend.interrupt(conv.external_id)
-
-    async def say(self, conv: Conversation, text: str) -> dict[str, Any]:
-        runner = self._runners.get(cast("int", conv.id))
-        _log.info("say[%s]: %s", conv.external_id, text[:200])
-        return self._bus.publish(
-            "say",
-            conversation_id=conv.external_id,
-            text=text,
-            turn_id=runner.turn_id if runner is not None else None,
-        )
-
-    async def send_file(
-        self, conv: Conversation, path: str, *, caption: str = "", method: str = "auto"
-    ) -> str:
-        """Hand a file to the human: this conversation's window, else the master's."""
-        for target in (conv, await self.open_master()):
-            if target is None:
-                continue
-            for fe in self._frontends:
-                if target.kind in fe.kinds and fe.sends_files:
-                    note = await fe.send_file(
-                        target, path, caption=caption, method=method
-                    )
-                    _log.info("send_file[%s]: %s", target.external_id, note)
-                    return note
-        msg = (
-            "nowhere to send: no frontend that sends files shows this "
-            "conversation, and there is no open master"
-        )
-        raise SendFileError(msg)
-
-    def file_note(self) -> str:
-        """What the ``send_file`` tool tells the model about roots and size."""
-        sender = next((f for f in self._frontends if f.sends_files), None)
-        return sender.file_note() if sender else "no frontend here sends files"
-
-    async def schedule(
-        self,
-        conv: Conversation,
-        at: str,
-        text: str,
-        *,
-        urgency: Priority = "wake",
-        dedupe_key: str | None = None,
-    ) -> tuple[int | None, datetime]:
-        if self.scheduler is None:
-            msg = "no scheduler; `schedule` is unavailable"
-            raise RuntimeError(msg)
-        return await self.scheduler.schedule(
-            conv, at, text, urgency=urgency, dedupe_key=dedupe_key
-        )
-
-    async def schedules(self, conv: Conversation | None = None) -> list[dict[str, Any]]:
-        return await self.scheduler.scheduled(conv) if self.scheduler else []

src/beaver_gateway/conversations/questions.py

@@ -1,85 +0,0 @@
-"""``AskUserQuestion`` from inside a turn: answered or timed out."""
-
-from __future__ import annotations
-
-import asyncio
-import contextlib
-from typing import TYPE_CHECKING, Any, cast
-from uuid import uuid4
-
-from beaver_gateway.conversations.spawning import Spawning
-from beaver_gateway.conversations.state import Question
-
-if TYPE_CHECKING:
-    from beaver_gateway.storage.models import Conversation
-
-__all__ = ["Questions"]
-
-
-class Questions(Spawning):
-    async def ask(self, key: str, payload: dict[str, Any]) -> str | None:
-        """Show the question and wait; ``None`` when nobody answered in time."""
-        conv = await self.get(key)
-        if conv is None:
-            return None
-        runner = self._runners.get(cast("int", conv.id))
-        question_id = f"q_{uuid4().hex[:10]}"
-        pending = Question(
-            conversation_id=key,
-            turn_id=runner.turn_id if runner is not None else None,
-            questions=list(payload.get("questions") or []),
-            answer=asyncio.get_running_loop().create_future(),
-        )
-        self._questions[question_id] = pending
-        await self._set_pending_question(conv, value=True)
-        self._bus.publish(
-            "question",
-            conversation_id=key,
-            turn_id=pending.turn_id,
-            question_id=question_id,
-            questions=pending.questions,
-            timeout=self._question_timeout,
-        )
-        try:
-            async with asyncio.timeout(self._question_timeout):
-                answer = await pending.answer
-        except TimeoutError:
-            self._bus.publish(
-                "question.timeout",
-                conversation_id=key,
-                turn_id=pending.turn_id,
-                question_id=question_id,
-            )
-            return None
-        finally:
-            self._questions.pop(question_id, None)
-            await self._set_pending_question(conv, value=False)
-        self._bus.publish(
-            "question.answered",
-            conversation_id=key,
-            turn_id=pending.turn_id,
-            question_id=question_id,
-            answer=answer,
-        )
-        return answer
-
-    def answer(self, question_id: str, answer: str) -> bool:
-        pending = self._questions.get(question_id)
-        if pending is None or pending.answer.done():
-            return False
-        pending.answer.set_result(answer)
-        return True
-
-    def answer_text(self, answer: str | None) -> str:
-        if answer is None:
-            return self._texts.unanswered.format(
-                minutes=round(self._question_timeout / 60)
-            )
-        return self._texts.answered.format(answer=answer)
-
-    async def _set_pending_question(self, conv: Conversation, *, value: bool) -> None:
-        async def apply(row: Conversation) -> None:
-            row.pending_question = value
-
-        with contextlib.suppress(LookupError):
-            await self._update(conv, apply)

src/beaver_gateway/conversations/rows.py

@@ -1,563 +0,0 @@
-"""The conversation rows: create, find, bind to windows, flags, status, history."""
-
-from __future__ import annotations
-
-import json
-import logging
-from datetime import UTC, datetime
-from typing import TYPE_CHECKING, Any, cast
-from uuid import uuid4
-
-from sqlalchemy import String, func
-from sqlalchemy import cast as sa_cast
-from sqlmodel import col, select
-
-from beaver_gateway.backends.transcript import (
-    messages_from_entries,
-    render_messages,
-    text_of,
-)
-from beaver_gateway.conversations.kinds import KINDS, Kind
-from beaver_gateway.conversations.state import State, iso
-from beaver_gateway.frontends.markdown.history import load_messages
-from beaver_gateway.storage.models import (
-    Conversation,
-    ConversationBinding,
-    ConversationMessage,
-    RateLimit,
-    TranscriptEntry,
-    Usage,
-)
-
-if TYPE_CHECKING:
-    from collections.abc import Awaitable, Callable, Iterable
-
-    from beaver_gateway.frontends.base import Frontend
-
-__all__ = [
-    "MASTER_ALIAS",
-    "PARENT_ALIAS",
-    "STATUSES",
-    "TITLE_MAX",
-    "Rows",
-    "context_of",
-    "implied_title",
-]
-
-_log = logging.getLogger(__name__)
-
-MASTER_ALIAS = "master"
-PARENT_ALIAS = "parent"
-STATUSES = ("open", "merged", "closed", "archived")
-TITLE_MAX = 80
-
-
-class Rows(State):
-    async def create(
-        self,
-        *,
-        kind: Kind,
-        agent: str,
-        parent: Conversation | None = None,
-        title: str | None = None,
-        origin: str = "api",
-        session_id: str | None = None,
-        flags: dict[str, Any] | None = None,
-    ) -> Conversation:
-        if kind not in KINDS:
-            msg = f"unknown conversation kind {kind!r}"
-            raise ValueError(msg)
-        if not self._claude_agent(agent).serves(kind):
-            msg = f"agent {agent!r} does not serve kind {kind!r}"
-            raise ValueError(msg)
-        now = datetime.now(UTC)
-        row = Conversation(
-            frontend=origin,
-            external_id=str(uuid4()),
-            agent_name=agent,
-            kind=kind,
-            parent_id=parent.id if parent is not None else None,
-            title=title,
-            session_id=session_id,
-            flags=dict(flags or {}),
-            last_activity_at=now,
-        )
-        async with self._db.session() as session:
-            session.add(row)
-            await session.commit()
-            await session.refresh(row)
-        self._bus.publish("conversation.created", **self.public(row))
-        return row
-
-    async def get(self, public_id: str) -> Conversation | None:
-        async with self._db.session() as session:
-            result = await session.exec(
-                select(Conversation).where(Conversation.external_id == public_id)
-            )
-            return result.first()
-
-    async def get_row(self, row_id: int) -> Conversation | None:
-        async with self._db.session() as session:
-            return await session.get(Conversation, row_id)
-
-    async def resolve(
-        self, key: str, *, origin: Conversation | None = None
-    ) -> Conversation | None:
-        """By public id, or ``master`` / ``parent`` relative to ``origin``."""
-        key = key.strip()
-        if key == MASTER_ALIAS:
-            return await self.open_master()
-        if key == PARENT_ALIAS:
-            if origin is None or origin.parent_id is None:
-                return None
-            return await self.get_row(origin.parent_id)
-        return await self.get(key)
-
-    async def open_master(self) -> Conversation | None:
-        masters = await self.find(kind="master", status="open", limit=1)
-        return masters[0] if masters else None
-
-    async def find(
-        self,
-        *,
-        status: str | None = None,
-        kind: str | None = None,
-        parent: Conversation | None = None,
-        limit: int = 200,
-    ) -> list[Conversation]:
-        stmt = select(Conversation).order_by(col(Conversation.id).desc()).limit(limit)
-        if status is not None:
-            stmt = stmt.where(Conversation.status == status)
-        if kind is not None:
-            stmt = stmt.where(Conversation.kind == kind)
-        if parent is not None:
-            stmt = stmt.where(Conversation.parent_id == parent.id)
-        async with self._db.session() as session:
-            return list((await session.exec(stmt)).all())
-
-    async def search(self, query: str, *, limit: int = 20) -> list[Conversation]:
-        needle = f"%{query}%"
-        by_title = select(Conversation.id).where(col(Conversation.title).ilike(needle))
-        by_first = select(ConversationMessage.conversation_id).where(
-            ConversationMessage.seq == 0,
-            col(ConversationMessage.content_json).ilike(needle),
-        )
-        by_entry = (
-            select(Conversation.id)
-            .join(
-                TranscriptEntry,
-                col(TranscriptEntry.session_id) == col(Conversation.session_id),
-            )
-            .where(sa_cast(col(TranscriptEntry.entry), String).ilike(needle))
-        )
-        stmt = (
-            select(Conversation)
-            .where(col(Conversation.id).in_(by_title.union(by_first, by_entry)))
-            .order_by(col(Conversation.id).desc())
-            .limit(limit)
-        )
-        async with self._db.session() as session:
-            return list((await session.exec(stmt)).all())
-
-    async def bindings(self, conv: Conversation) -> list[ConversationBinding]:
-        async with self._db.session() as session:
-            result = await session.exec(
-                select(ConversationBinding)
-                .where(ConversationBinding.conversation_id == conv.id)
-                .order_by(col(ConversationBinding.id))
-            )
-            return list(result.all())
-
-    async def bind(
-        self,
-        conv: Conversation,
-        *,
-        frontend: str,
-        external_id: str,
-        visible: bool = True,
-    ) -> ConversationBinding:
-        if conv.kind not in self.frontend(frontend).kinds:
-            msg = f"frontend {frontend!r} does not show kind {conv.kind!r}"
-            raise ValueError(msg)
-        async with self._db.session() as session:
-            existing = list(
-                (
-                    await session.exec(
-                        select(ConversationBinding).where(
-                            ConversationBinding.conversation_id == conv.id,
-                            ConversationBinding.frontend == frontend,
-                        )
-                    )
-                ).all()
-            )
-            row = next((b for b in existing if b.external_id == external_id), None)
-            if visible:
-                for other in existing:
-                    if other is not row and other.visible:
-                        other.visible = False
-                        session.add(other)
-                same_window = await session.exec(
-                    select(ConversationBinding).where(
-                        ConversationBinding.frontend == frontend,
-                        ConversationBinding.external_id == external_id,
-                        ConversationBinding.conversation_id != conv.id,
-                        col(ConversationBinding.visible).is_(True),
-                    )
-                )
-                for other in same_window.all():
-                    other.visible = False
-                    session.add(other)
-            if row is None:
-                row = ConversationBinding(
-                    conversation_id=cast("int", conv.id),
-                    frontend=frontend,
-                    external_id=external_id,
-                    visible=visible,
-                )
-            else:
-                row.visible = visible
-            session.add(row)
-            await session.commit()
-            await session.refresh(row)
-        self._bus.publish(
-            "conversation.bound",
-            conversation_id=conv.external_id,
-            frontend=frontend,
-            external_id=external_id,
-            visible=visible,
-        )
-        return row
-
-    async def find_bound(
-        self, *, frontend: str, external_id: str
-    ) -> Conversation | None:
-        async with self._db.session() as session:
-            result = await session.exec(
-                select(Conversation)
-                .join(
-                    ConversationBinding,
-                    col(ConversationBinding.conversation_id) == col(Conversation.id),
-                )
-                .where(
-                    ConversationBinding.frontend == frontend,
-                    ConversationBinding.external_id == external_id,
-                    col(ConversationBinding.visible).is_(True),
-                )
-                .order_by(col(Conversation.id).desc())
-            )
-            return result.first()
-
-    async def last_binding(
-        self, *, frontend: str, kind: str
-    ) -> ConversationBinding | None:
-        """The window ``frontend`` last used for ``kind``; outlives a rotation."""
-        async with self._db.session() as session:
-            result = await session.exec(
-                select(ConversationBinding)
-                .join(
-                    Conversation,
-                    col(Conversation.id) == col(ConversationBinding.conversation_id),
-                )
-                .where(
-                    ConversationBinding.frontend == frontend, Conversation.kind == kind
-                )
-                .order_by(col(ConversationBinding.id).desc())
-            )
-            return result.first()
-
-    async def window_of(self, conv: Conversation) -> str | None:
-        for binding in await self.bindings(conv):
-            if binding.visible:
-                return binding.external_id
-        return None
-
-    async def set_flags(
-        self, conv: Conversation, flags: dict[str, Any]
-    ) -> Conversation:
-        async def apply(row: Conversation) -> None:
-            row.flags = {**row.flags, **flags}
-
-        return await self._update(conv, apply)
-
-    async def set_status(self, conv: Conversation, status: str) -> Conversation:
-        if status not in STATUSES:
-            msg = f"unknown status {status!r}"
-            raise ValueError(msg)
-
-        async def apply(row: Conversation) -> None:
-            row.status = status
-
-        return await self._update(conv, apply)
-
-    async def set_title(self, conv: Conversation, title: str) -> Conversation:
-        async def apply(row: Conversation) -> None:
-            row.title = title
-
-        return await self._update(conv, apply)
-
-    async def touch_user(self, conv: Conversation) -> Conversation:
-        async def apply(row: Conversation) -> None:
-            row.last_user_activity_at = datetime.now(UTC)
-
-        return await self._update(conv, apply)
-
-    async def reparent(self, conv: Conversation, parent: Conversation) -> Conversation:
-        async def apply(row: Conversation) -> None:
-            row.parent_id = parent.id
-
-        return await self._update(conv, apply)
-
-    async def _update(
-        self, conv: Conversation, apply: Callable[[Conversation], Awaitable[None]]
-    ) -> Conversation:
-        async with self._db.session() as session:
-            row = await session.get(Conversation, conv.id)
-            if row is None:
-                msg = f"conversation {conv.external_id} vanished"
-                raise LookupError(msg)
-            await apply(row)
-            row.updated_at = datetime.now(UTC)
-            session.add(row)
-            await session.commit()
-            await session.refresh(row)
-        self._bus.publish("conversation.updated", **self.public(row))
-        return row
-
-    def public(self, conv: Conversation) -> dict[str, Any]:
-        return {
-            "id": conv.external_id,
-            "kind": conv.kind,
-            "agent": conv.agent_name,
-            "title": conv.title,
-            "status": conv.status,
-            "parent_row": conv.parent_id,
-            "session_id": conv.session_id,
-            "running_turn": conv.running_turn,
-            "pending_question": conv.pending_question,
-            "flags": conv.flags,
-            "origin": conv.frontend,
-            "created_at": iso(conv.created_at),
-            "last_user_activity_at": iso(conv.last_user_activity_at),
-            "last_activity_at": iso(conv.last_activity_at),
-        }
-
-    async def describe(self, conv: Conversation) -> dict[str, Any]:
-        out = self.public(conv)
-        out["title"] = await self.implied_title(conv)
-        out["context_tokens"] = await self.context_tokens(conv)
-        parent = await self.get_row(conv.parent_id) if conv.parent_id else None
-        out["parent"] = parent.external_id if parent is not None else None
-        out["bindings"] = [
-            {"frontend": b.frontend, "external_id": b.external_id, "visible": b.visible}
-            for b in await self.bindings(conv)
-        ]
-        live = self._pool.get(conv.external_id)
-        out["live"] = live is not None
-        out["busy"] = live.busy if live is not None else False
-        runner = self._runners.get(cast("int", conv.id))
-        out["turn"] = runner.snapshot() if runner is not None else None
-        pending = self.pending_question(conv.external_id)
-        out["question"] = (
-            {"id": pending[0], "questions": pending[1]} if pending else None
-        )
-        return out
-
-    def pending_question(self, key: str) -> tuple[str, list[dict[str, Any]]] | None:
-        for question_id, pending in self._questions.items():
-            if pending.conversation_id == key and not pending.answer.done():
-                return question_id, pending.questions
-        return None
-
-    async def rate_limits(self, *, limit: int = 100) -> list[RateLimit]:
-        async with self._db.session() as session:
-            result = await session.exec(
-                select(RateLimit).order_by(col(RateLimit.id).desc()).limit(limit)
-            )
-            return list(result.all())
-
-    async def context_tokens(self, conv: Conversation) -> int:
-        async with self._db.session() as session:
-            row = (
-                await session.exec(
-                    select(Usage)
-                    .where(Usage.conversation_id == conv.external_id)
-                    .order_by(col(Usage.id).desc())
-                    .limit(1)
-                )
-            ).first()
-        return context_of(row)
-
-    async def context_tokens_by_id(self, ids: Iterable[str]) -> dict[str, int]:
-        wanted = list(ids)
-        if not wanted:
-            return {}
-        latest = (
-            select(func.max(Usage.id))
-            .where(col(Usage.conversation_id).in_(wanted))
-            .group_by(Usage.conversation_id)
-        )
-        async with self._db.session() as session:
-            rows = (
-                await session.exec(select(Usage).where(col(Usage.id).in_(latest)))
-            ).all()
-        return {
-            cast("str", row.conversation_id): context_of(row)
-            for row in rows
-            if row.conversation_id
-        }
-
-    async def usage_tokens(self, since: datetime) -> int:
-        async with self._db.session() as session:
-            rows = (
-                await session.exec(
-                    select(Usage).where(
-                        col(Usage.ts) >= since.astimezone(UTC).replace(tzinfo=None)
-                    )
-                )
-            ).all()
-        return sum(
-            r.input_tokens + r.output_tokens + r.cache_creation_tokens for r in rows
-        )
-
-    async def busy(self, conv: Conversation) -> bool:
-        row = await self.get_row(cast("int", conv.id)) or conv
-        if row.running_turn or row.pending_question:
-            return True
-        live = self._pool.get(row.external_id)
-        if live is not None and live.busy:
-            return True
-        pending = await self._queue.pending(cast("int", row.id))
-        return any(i.priority in ("user", "urgent", "wake") for i in pending)
-
-    @property
-    def frontends(self) -> list[Frontend]:
-        return list(self._frontends)
-
-    def frontend(self, name: str) -> Frontend:
-        for fe in self._frontends:
-            if fe.name == name:
-                return fe
-        msg = f"unknown frontend {name!r}"
-        raise ValueError(msg)
-
-    def default_agent(self, kind: Kind) -> str | None:
-        for fe in self._frontends:
-            if kind in fe.kinds and (agent := fe.agent_for(kind)):
-                return agent
-        return None
-
-    async def materialize(self, conv: Conversation) -> ConversationBinding | None:
-        for fe in self._frontends:
-            if conv.kind not in fe.kinds:
-                continue
-            binding = await fe.materialize(conv)
-            if binding is not None:
-                return binding
-        return None
-
-    async def mark_closed(self, conv: Conversation) -> bool:
-        marked = False
-        for fe in self._frontends:
-            if conv.kind in fe.kinds:
-                try:
-                    marked = await fe.mark_closed(conv) or marked
-                except Exception:  # noqa: BLE001
-                    _log.exception("%s could not mark %s", fe.name, conv.external_id)
-        return marked
-
-    async def read(self, conv: Conversation, *, window: int | None = None) -> str:
-        return render_messages(await self.history(conv), window=window)
-
-    async def history(self, conv: Conversation) -> list[dict[str, Any]]:
-        if conv.session_id is None:
-            async with self._db.session() as session:
-                return await load_messages(
-                    session, conversation_id=cast("int", conv.id)
-                )
-        return messages_from_entries(
-            cast("Any", await self.entries(conv)), with_ts=True
-        )
-
-    async def entries(self, conv: Conversation, *, subpath: str = "") -> list[Any]:
-        if conv.session_id is None:
-            return []
-        key = {**self._store_key(conv), "subpath": subpath}
-        return list(await self._store.load(cast("Any", key)) or [])
-
-    async def subpaths(self, conv: Conversation) -> list[str]:
-        if conv.session_id is None:
-            return []
-        return list(await self._store.list_subkeys(cast("Any", self._store_key(conv))))
-
-    async def first_user_texts(self, ids: Iterable[int]) -> dict[int, str]:
-        wanted = list(ids)
-        if not wanted:
-            return {}
-        async with self._db.session() as session:
-            rows = (
-                await session.exec(
-                    select(ConversationMessage).where(
-                        col(ConversationMessage.conversation_id).in_(wanted),
-                        ConversationMessage.seq == 0,
-                        ConversationMessage.role == "user",
-                    )
-                )
-            ).all()
-        return {
-            r.conversation_id: text_of(json.loads(r.content_json)).strip() for r in rows
-        }
-
-    async def implied_title(self, conv: Conversation) -> str | None:
-        if conv.title:
-            return conv.title
-        text = (await self.first_user_texts([cast("int", conv.id)])).get(
-            cast("int", conv.id)
-        )
-        return implied_title(text)
-
-    async def chat_name(self, conv: Conversation) -> str:
-        """What a ``[[wikilink]]`` to the chat says: the file's stem when it has one."""
-        window = await self.window_of(conv)
-        if window and window.endswith(".md"):
-            return window.rsplit("/", 1)[-1][: -len(".md")]
-        return await self.implied_title(conv) or conv.external_id
-
-    async def adopt(self, *, kind: Kind, first_user_text: str) -> Conversation | None:
-        """The one unbound, session-less conversation whose history starts here."""
-        text = first_user_text.strip()
-        if not text:
-            return None
-        bound = select(ConversationBinding.conversation_id).where(
-            col(ConversationBinding.visible).is_(True)
-        )
-        async with self._db.session() as session:
-            rows = (
-                await session.exec(
-                    select(Conversation).where(
-                        Conversation.kind == kind,
-                        Conversation.status == "open",
-                        col(Conversation.session_id).is_(None),
-                        col(Conversation.id).not_in(bound),
-                    )
-                )
-            ).all()
-        firsts = await self.first_user_texts(cast("int", r.id) for r in rows)
-        hits = [r for r in rows if firsts.get(cast("int", r.id)) == text]
-        return hits[0] if len(hits) == 1 else None
-
-
-def implied_title(text: str | None) -> str | None:
-    if not text:
-        return None
-    line = text.strip().splitlines()[0].strip()
-    return line if len(line) <= TITLE_MAX else line[: TITLE_MAX - 1] + "…"
-
-
-def context_of(row: Usage | None) -> int:
-    """The last API call's input, or the per-call average for older rows."""
-    if row is None:
-        return 0
-    if row.context_tokens:
-        return row.context_tokens
-    total = row.input_tokens + row.cache_read_tokens + row.cache_creation_tokens
-    return round(total / max(row.num_turns or 1, 1))

src/beaver_gateway/conversations/seeds.py

@@ -1,71 +0,0 @@
-"""How a new conversation starts: the seed rendered into its first prompt."""
-
-from __future__ import annotations
-
-import inspect
-from datetime import UTC, datetime
-from typing import TYPE_CHECKING, Any, cast
-
-from beaver_gateway.conversations.kinds import as_kind
-from beaver_gateway.conversations.rows import Rows
-from beaver_gateway.conversations.texts import SeedContext
-
-if TYPE_CHECKING:
-    from beaver_gateway.storage.models import Conversation
-
-__all__ = ["SEEDS", "Seeds"]
-
-SEEDS = ("clean", "morning", "copy", "brief")
-
-
-class Seeds(Rows):
-    async def pending_seed(self, conv: Conversation) -> str | None:
-        """A seed nobody has spoken after yet: rendered now, spent once."""
-        seed = conv.flags.get("seed")
-        if not seed:
-            return None
-        parent = await self.get_row(conv.parent_id) if conv.parent_id else None
-        ctx = SeedContext(
-            kind=as_kind(conv.kind),
-            seed=str(seed),
-            agent=conv.agent_name,
-            parent=parent,
-            text=None,
-            title=conv.title,
-        )
-        window = conv.flags.get("seed_window")
-        text = await self.seed_text(
-            ctx, window=window if isinstance(window, int) else None
-        )
-        await self.set_flags(conv, {"seed": None, "seed_window": None})
-        return text
-
-    async def seed_text(self, ctx: SeedContext, *, window: int | None) -> str:
-        texts = self._texts
-        stamp = datetime.now(UTC).astimezone().strftime("%Y-%m-%d %H:%M")
-        head = texts.seed_head.format(
-            seed=ctx.seed,
-            kind=ctx.kind,
-            title=f" «{ctx.title}»" if ctx.title else "",
-            stamp=stamp,
-        )
-        body: str | None = None
-        if texts.seed is not None:
-            produced: Any = texts.seed(ctx)
-            if inspect.isawaitable(produced):
-                produced = await produced
-            body = cast("str | None", produced)
-        if body is None:
-            if ctx.seed == "brief":
-                body = ctx.text
-            elif ctx.seed == "copy":
-                scope = (
-                    texts.seed_copy_window.format(window=window)
-                    if window
-                    else texts.seed_copy_all
-                )
-                body = texts.seed_copy.format(scope=scope)
-            elif ctx.seed == "morning":
-                body = texts.seed_morning_missing
-        parts = [head, body, ctx.text if ctx.seed != "brief" else None]
-        return "\n\n".join(p for p in parts if p)

src/beaver_gateway/conversations/service.py

@@ -1,158 +0,0 @@
-"""``Conversations`` - the service every frontend, job and gateway tool talks to.
-
-Built as layers, one file each: rows → seeds → turns → messaging →
-spawning → questions → closing; this file adds start, stop and restart
-recovery. A turn started by a user message streams back to whoever asked;
-a turn started by an inject streams nowhere and can only speak via ``say``.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import contextlib
-import logging
-import re
-from datetime import UTC, datetime, timedelta, tzinfo
-
-from sqlmodel import col, select
-
-from beaver_gateway.conversations.closing import Closing, DistillResult
-from beaver_gateway.conversations.kinds import KINDS
-from beaver_gateway.conversations.rows import context_of, implied_title
-from beaver_gateway.conversations.seeds import SEEDS
-from beaver_gateway.conversations.spawning import ForkResult
-from beaver_gateway.conversations.state import aware
-from beaver_gateway.conversations.texts import (
-    ConversationTexts,
-    NewDayContext,
-    SeedContext,
-    UserSaid,
-)
-from beaver_gateway.storage.models import Conversation
-
-__all__ = [
-    "KINDS",
-    "SEEDS",
-    "ConversationTexts",
-    "Conversations",
-    "DistillResult",
-    "ForkResult",
-    "NewDayContext",
-    "SeedContext",
-    "UserSaid",
-    "context_of",
-    "implied_title",
-    "parse_at",
-]
-
-_log = logging.getLogger(__name__)
-
-_RELATIVE = re.compile(r"^\+(\d+)\s*([smhd])$")
-_UNITS = {"s": 1, "m": 60, "h": 3600, "d": 86400}
-
-
-class Conversations(Closing):
-    async def start(self) -> None:
-        await self.recover()
-        for row_id in await self._queue.conversations_with_pending():
-            self._ensure_worker(row_id)
-        self._idle_task = asyncio.create_task(self._idle_loop())
-
-    async def stop(self) -> None:
-        tasks = list(self._tasks)
-        if self._idle_task is not None:
-            tasks.append(self._idle_task)
-        for task in tasks:
-            task.cancel()
-        for task in tasks:
-            with contextlib.suppress(BaseException):
-                await task
-        self._tasks.clear()
-        self._idle_task = None
-
-    async def recover(self) -> list[Conversation]:
-        """Repair the transcripts of turns a restart cut and tell each conversation."""
-        async with self._db.session() as session:
-            result = await session.exec(
-                select(Conversation).where(col(Conversation.running_turn).is_not(None))
-            )
-            cut = list(result.all())
-        for conv in cut:
-            fixed = 0
-            if conv.session_id is not None:
-                backend = self._backend(conv.agent_name)
-                try:
-                    fixed = await backend.repair_session(
-                        conv.session_id, text=self._texts.interrupted
-                    )
-                except Exception:  # noqa: BLE001
-                    _log.exception("repair of %s failed", conv.session_id)
-            turn_id = conv.running_turn
-
-            async def clear(row: Conversation) -> None:
-                row.running_turn = None
-                row.pending_question = False
-
-            await self._update(conv, clear)
-            note = self._texts.cut_by_restart.format(turn_id=turn_id)
-            if fixed:
-                note += self._texts.repaired_tools.format(
-                    fixed=fixed, interrupted=self._texts.interrupted
-                )
-            await self.inject(conv, note, urgency="normal", origin="system")
-            _log.warning("conversation %s: %s", conv.external_id, note)
-        for item in await self._queue.interrupted():
-            _log.warning(
-                "queue item #%s (%s) was running at restart; marked interrupted",
-                item.id,
-                item.priority,
-            )
-        return cut
-
-    async def _idle_loop(self) -> None:
-        while True:
-            try:
-                await self._emit_idle()
-            except Exception:  # noqa: BLE001
-                _log.exception("idle watcher failed")
-            await asyncio.sleep(self._idle_interval)
-
-    async def _emit_idle(self) -> None:
-        if not self._idle_days:
-            return
-        now = datetime.now(UTC)
-        for conv in await self.find(status="open", limit=10_000):
-            last = aware(conv.last_activity_at or conv.created_at)
-            days = int((now - last).total_seconds() // 86400)
-            due = [d for d in self._idle_days if days >= d]
-            if not due:
-                continue
-            notified = int(conv.flags.get("idle_notified", 0) or 0)
-            if due[-1] <= notified:
-                continue
-            await self.set_flags(conv, {"idle_notified": due[-1]})
-            bindings = await self.bindings(conv)
-            self._bus.publish(
-                "conversation.idle",
-                conversation_id=conv.external_id,
-                kind=conv.kind,
-                agent=conv.agent_name,
-                days=due[-1],
-                bindings=[
-                    {"frontend": b.frontend, "external_id": b.external_id}
-                    for b in bindings
-                    if b.visible
-                ],
-            )
-
-
-def parse_at(at: str, tz: tzinfo = UTC) -> datetime:
-    raw = at.strip()
-    match = _RELATIVE.match(raw.replace(" ", ""))
-    if match:
-        amount, unit = match.groups()
-        return datetime.now(UTC) + timedelta(seconds=int(amount) * _UNITS[unit])
-    parsed = datetime.fromisoformat(raw)
-    if parsed.tzinfo is None:
-        parsed = parsed.replace(tzinfo=tz)
-    return parsed.astimezone(UTC)

src/beaver_gateway/conversations/spawning.py

@@ -1,203 +0,0 @@
-"""New conversations from old ones: spawn with a seed, fork a copy, merge back."""
-
-from __future__ import annotations
-
-import logging
-from dataclasses import dataclass
-from typing import TYPE_CHECKING, Any, cast
-
-from claude_agent_sdk import fork_session_via_store, project_key_for_directory
-
-from beaver_gateway.backends.transcript import strip_tool_entries, window_entries
-from beaver_gateway.conversations.messaging import Messaging
-from beaver_gateway.conversations.seeds import SEEDS
-from beaver_gateway.conversations.texts import SeedContext
-
-if TYPE_CHECKING:
-    from beaver_gateway.backends.capture import TurnCapture
-    from beaver_gateway.conversations.kinds import Kind
-    from beaver_gateway.storage.models import Conversation
-
-__all__ = ["ForkResult", "Spawning"]
-
-_log = logging.getLogger(__name__)
-
-
-@dataclass(frozen=True, slots=True)
-class ForkResult:
-    conversation: Conversation
-    text: str
-    capture: TurnCapture
-
-
-class Spawning(Messaging):
-    async def spawn(
-        self,
-        *,
-        kind: Kind,
-        agent: str | None = None,
-        seed: str = "clean",
-        parent: Conversation | None = None,
-        text: str | None = None,
-        title: str | None = None,
-        window: int | None = None,
-        origin: str = "api",
-        binding: tuple[str, str] | None = None,
-        flags: dict[str, Any] | None = None,
-        attachments: list[dict[str, Any]] | None = None,
-    ) -> Conversation:
-        """Create a conversation in a window and queue its seed.
-
-        ``binding`` reuses a window that already exists instead of asking the
-        home frontend for one. Without ``text`` the seed waits in ``flags``
-        and opens the first turn, so a fresh window costs nothing until
-        someone speaks.
-        """
-        if seed not in SEEDS:
-            msg = f"unknown seed {seed!r}"
-            raise ValueError(msg)
-        if seed == "brief" and not text:
-            msg = "seed=brief needs text"
-            raise ValueError(msg)
-        if agent is None and kind == "branch" and parent is not None:
-            agent = parent.agent_name
-        agent = agent or self.default_agent(kind)
-        if agent is None:
-            msg = f"no default agent for kind {kind!r}; pass `agent`"
-            raise ValueError(msg)
-        session_id: str | None = None
-        if seed == "copy":
-            if parent is None or parent.session_id is None:
-                msg = "seed=copy needs a parent with a session"
-                raise ValueError(msg)
-            session_id = await self._copy_session(
-                parent, window=window, strip_tools=False
-            )
-        conv = await self.create(
-            kind=kind,
-            agent=agent,
-            parent=parent,
-            title=title,
-            origin=origin,
-            session_id=session_id,
-            flags=flags,
-        )
-        if binding is not None:
-            await self.bind(conv, frontend=binding[0], external_id=binding[1])
-        else:
-            await self.materialize(conv)
-        ctx = SeedContext(
-            kind=kind, seed=seed, agent=agent, parent=parent, text=text, title=title
-        )
-        if text is None:
-            return await self.set_flags(conv, {"seed": seed, "seed_window": window})
-        await self._queue.push(
-            conversation_id=cast("int", conv.id),
-            priority="user",
-            origin=f"seed:{seed}" if seed == "brief" else origin,
-            text=await self.seed_text(ctx, window=window),
-            attachments=attachments,
-        )
-        self._ensure_worker(cast("int", conv.id))
-        return conv
-
-    async def fork(
-        self,
-        conv: Conversation,
-        prompt: str,
-        *,
-        window: int | None = None,
-        strip_tools: bool = False,
-        title: str | None = None,
-        agent: str | None = None,
-    ) -> ForkResult:
-        """Copy the history into a one-off session, run ``prompt`` on it, close it."""
-        agent = agent or conv.agent_name
-        session_id = await self._copy_session(
-            conv, window=window, strip_tools=strip_tools, agent=agent
-        )
-        child = await self.create(
-            kind="fork",
-            agent=agent,
-            parent=conv,
-            title=title or f"fork: {conv.title or conv.external_id}",
-            origin="system",
-            session_id=session_id,
-        )
-        try:
-            text, capture = await self.run_text_turn(
-                child, prompt, origin="fork", tools=False
-            )
-        finally:
-            await self._backend(agent).close(child.external_id)
-        child = await self.set_status(child, "closed")
-        return ForkResult(conversation=child, text=text, capture=capture)
-
-    async def merge(self, conv: Conversation) -> ForkResult:
-        if conv.parent_id is None:
-            msg = "merge needs a parent conversation"
-            raise ValueError(msg)
-        parent = await self.get_row(conv.parent_id)
-        if parent is None:
-            msg = "parent conversation vanished"
-            raise LookupError(msg)
-        result = await self.fork(
-            conv,
-            self._texts.merge_prompt,
-            title=f"merge: {conv.title or conv.external_id}",
-        )
-        if result.text.strip():
-            await self.inject(parent, result.text, urgency="normal", origin="merge")
-        await self.set_status(conv, "merged")
-        await self.mark_closed(conv)
-        self._bus.publish(
-            "conversation.merged",
-            conversation_id=conv.external_id,
-            parent=parent.external_id,
-            fork=result.conversation.external_id,
-        )
-        return result
-
-    async def _copy_session(
-        self,
-        conv: Conversation,
-        *,
-        window: int | None,
-        strip_tools: bool,
-        agent: str | None = None,
-    ) -> str:
-        if conv.session_id is None:
-            msg = f"conversation {conv.external_id} has no session to copy"
-            raise ValueError(msg)
-        live = self._pool.get(conv.external_id)
-        if live is not None and live.dirty:
-            msg = f"conversation {conv.external_id} has a mirror gap; not forking"
-            raise RuntimeError(msg)
-        source = self._claude_agent(conv.agent_name)
-        target = self._claude_agent(agent) if agent else source
-        forked = await fork_session_via_store(
-            self._store, conv.session_id, directory=str(source.cwd)
-        )
-        source_key = {
-            "project_key": project_key_for_directory(str(source.cwd)),
-            "session_id": forked.session_id,
-        }
-        target_key = {
-            "project_key": project_key_for_directory(str(target.cwd)),
-            "session_id": forked.session_id,
-        }
-        if window is not None or strip_tools or target_key != source_key:
-            entries = await self._store.load(cast("Any", source_key)) or []
-            trimmed = window_entries(cast("Any", entries), window=window)
-            if strip_tools:
-                trimmed = strip_tool_entries(trimmed)
-            await self._store.delete(cast("Any", source_key))
-            await self._store.append(cast("Any", target_key), cast("Any", trimmed))
-        _log.info(
-            "forked session %s -> %s (window=%s, strip_tools=%s)",
-            conv.session_id,
-            forked.session_id,
-            window,
-            strip_tools,
-        )
-        return forked.session_id

src/beaver_gateway/conversations/state.py

@@ -1,191 +0,0 @@
-"""What every layer of the conversations service shares: wiring and lookups."""
-
-from __future__ import annotations
-
-import asyncio
-from dataclasses import dataclass, field
-from datetime import UTC, datetime
-from typing import TYPE_CHECKING, Any, cast
-
-from claude_agent_sdk import project_key_for_directory
-
-from beaver_gateway.conversations.injects import InjectQueue
-from beaver_gateway.conversations.texts import ConversationTexts
-
-if TYPE_CHECKING:
-    from collections.abc import Awaitable, Callable, Sequence
-
-    from claude_agent_sdk import SessionStore
-
-    from beaver_gateway.agents.claude import ClaudeAgent
-    from beaver_gateway.app import AgentRegistry
-    from beaver_gateway.backends.claude_sdk import ClaudeSdkBackend
-    from beaver_gateway.backends.sessions import SessionPool
-    from beaver_gateway.conversations.distill import Distiller
-    from beaver_gateway.conversations.envelope import Envelope
-    from beaver_gateway.conversations.texts import UserSaid
-    from beaver_gateway.events.bus import EventBus
-    from beaver_gateway.frontends.base import Frontend
-    from beaver_gateway.jobs.scheduler import Scheduler
-    from beaver_gateway.storage.db import Database
-    from beaver_gateway.storage.models import Conversation
-
-__all__ = ["Question", "Runner", "State", "aware", "iso"]
-
-
-@dataclass
-class Runner:
-    lock: asyncio.Lock = field(default_factory=asyncio.Lock)
-    wake: asyncio.Event = field(default_factory=asyncio.Event)
-    task: asyncio.Task[None] | None = None
-    turn_id: str | None = None
-    origin: str | None = None
-    text: str | None = None
-    started_at: datetime | None = None
-    tools: dict[str, dict[str, Any]] = field(default_factory=dict)
-
-    def snapshot(self) -> dict[str, Any] | None:
-        if self.turn_id is None:
-            return None
-        return {
-            "id": self.turn_id,
-            "origin": self.origin,
-            "text": self.text,
-            "started_at": iso(self.started_at),
-            "tools": list(self.tools.values()),
-        }
-
-
-@dataclass(frozen=True, slots=True)
-class Question:
-    conversation_id: str
-    turn_id: str | None
-    questions: list[dict[str, Any]]
-    answer: asyncio.Future[str]
-
-
-class State:
-    _db: Database
-    _agents: AgentRegistry
-    _backends: dict[str, Any]
-    _bus: EventBus
-    _pool: SessionPool
-    _store: SessionStore
-    _texts: ConversationTexts
-    _frontends: list[Frontend]
-    _normal_window: float
-    _idle_days: tuple[int, ...]
-    _idle_interval: float
-    _question_timeout: float
-    _envelope: Envelope | None
-    _distiller: Distiller | None
-    _user_sink: Callable[[UserSaid], Awaitable[None] | None] | None
-    _queue: InjectQueue
-    _runners: dict[int, Runner]
-    _questions: dict[str, Question]
-    _tasks: set[asyncio.Task[None]]
-    _idle_task: asyncio.Task[None] | None
-    scheduler: Scheduler | None
-
-    def __init__(
-        self,
-        *,
-        db: Database,
-        agents: AgentRegistry,
-        backends: dict[str, Any],
-        bus: EventBus,
-        pool: SessionPool,
-        store: SessionStore,
-        texts: ConversationTexts | None = None,
-        frontends: Sequence[Frontend] = (),
-        normal_window: float = 3600.0,
-        idle_days: Sequence[int] = (2,),
-        idle_interval: float = 3600.0,
-        question_timeout: float = 600.0,
-        envelope: Envelope | None = None,
-        distiller: Distiller | None = None,
-        user_sink: Callable[[UserSaid], Awaitable[None] | None] | None = None,
-    ) -> None:
-        self._db = db
-        self._agents = agents
-        self._backends = backends
-        self._bus = bus
-        self._pool = pool
-        self._store = store
-        self._texts = texts or ConversationTexts()
-        self._frontends = [f for f in frontends if f.name]
-        self._normal_window = normal_window
-        self._idle_days = tuple(sorted(idle_days))
-        self._idle_interval = idle_interval
-        self._question_timeout = question_timeout
-        self._envelope = envelope
-        self._distiller = distiller
-        self._user_sink = user_sink
-        self._queue = InjectQueue(db)
-        self._runners = {}
-        self._questions = {}
-        self._tasks = set()
-        self._idle_task = None
-        self.scheduler = None
-        for backend in backends.values():
-            if hasattr(backend, "waker"):
-                backend.waker = self._woken
-
-    async def _woken(self, key: str, note: str) -> None:
-        raise NotImplementedError
-
-    @property
-    def db(self) -> Database:
-        return self._db
-
-    @property
-    def queue(self) -> InjectQueue:
-        return self._queue
-
-    @property
-    def bus(self) -> EventBus:
-        return self._bus
-
-    @property
-    def pool(self) -> SessionPool:
-        return self._pool
-
-    def _backend(self, agent: str) -> ClaudeSdkBackend:
-        backend = self._backends.get(agent)
-        if backend is None or not hasattr(backend, "repair_session"):
-            msg = f"agent {agent!r} has no Claude SDK backend"
-            raise LookupError(msg)
-        return cast("ClaudeSdkBackend", backend)
-
-    def _claude_agent(self, name: str) -> ClaudeAgent:
-        agent = self._agents.get(name)
-        if agent is None or not hasattr(agent, "cwd"):
-            msg = f"unknown Claude agent {name!r}"
-            raise LookupError(msg)
-        return cast("ClaudeAgent", agent)
-
-    def _store_key(self, conv: Conversation) -> dict[str, str]:
-        agent = self._claude_agent(conv.agent_name)
-        return {
-            "project_key": project_key_for_directory(str(agent.cwd)),
-            "session_id": cast("str", conv.session_id),
-        }
-
-    def _runner(self, row_id: int) -> Runner:
-        runner = self._runners.get(row_id)
-        if runner is None:
-            runner = Runner()
-            self._runners[row_id] = runner
-        return runner
-
-    def _track(self, task: asyncio.Task[None]) -> None:
-        self._tasks.add(task)
-        task.add_done_callback(self._tasks.discard)
-
-
-def aware(value: datetime) -> datetime:
-    return value if value.tzinfo is not None else value.replace(tzinfo=UTC)
-
-
-def iso(value: datetime | None) -> str | None:
-    return aware(value).isoformat(timespec="seconds") if value is not None else None

src/beaver_gateway/conversations/texts.py

@@ -1,129 +0,0 @@
-"""Every string the gateway puts in front of a model; English defaults, overridable."""
-
-from __future__ import annotations
-
-from dataclasses import dataclass, field
-from typing import TYPE_CHECKING
-
-from beaver_gateway.conversations import injects
-
-if TYPE_CHECKING:
-    from collections.abc import Awaitable, Callable
-    from datetime import date, datetime
-
-    from beaver_gateway.conversations.distill import DistillContext
-    from beaver_gateway.conversations.kinds import Kind
-    from beaver_gateway.conversations.rotation import HandoutContext
-    from beaver_gateway.storage.models import Conversation
-
-__all__ = [
-    "ConversationTexts",
-    "EnvelopeTexts",
-    "NewDayContext",
-    "SeedContext",
-    "UserSaid",
-]
-
-
-@dataclass(frozen=True, slots=True)
-class SeedContext:
-    kind: Kind
-    seed: str
-    agent: str
-    parent: Conversation | None
-    text: str | None
-    title: str | None
-
-
-@dataclass(frozen=True, slots=True)
-class NewDayContext:
-    """``reason`` is ``night``, ``age`` or ``context``; only ``night`` is a new day."""
-
-    day: date
-    reason: str
-    moved: int
-
-
-@dataclass(frozen=True, slots=True)
-class UserSaid:
-    conversation_id: str
-    kind: str
-    title: str | None
-    text: str
-    at: datetime
-    agent: str = ""
-    """Agent whose turn it is; a setup with several tells them apart by it."""
-
-
-@dataclass(frozen=True, slots=True)
-class EnvelopeTexts:
-    header: str = (
-        "[envelope - background signal, not a message; react only if it bears "
-        "on the question]"
-    )
-    time: str = "time: {stamp} ({zone})"
-    changed: str = "vault, changed {since}: {names}"
-    since: str = "since {time}"
-    since_start: str = "since start"
-    truncated: str = "… (envelope cap)"
-    file_header: str = "--- {path}, added lines only ---"
-    more_lines: str = "+ … {count} more"
-
-
-@dataclass(frozen=True, slots=True)
-class ConversationTexts:
-    merge_prompt: str = (
-        "This branch is closing. Write a merge note for the master: what was "
-        "decided, what was done, what was not and why, open questions. "
-        "Identifiers and links verbatim. Brief, past tense."
-    )
-    inject_header: Callable[[injects.InjectContext], str] = injects.inject_header
-    bundle_header: str = "[injects accumulated since {since}; not the user]"
-    task_prompt: str = "[subagent reported back: {note}]"
-    """Stands in for the user's words in mirrors of a turn a subagent started."""
-    interrupted: str = "interrupted"
-    answered: str = "The user answered: {answer}"
-    unanswered: str = (
-        "The user did not answer within {minutes} min. The question was shown "
-        "to them as text; finish the turn now, the answer comes as the next "
-        "message."
-    )
-    seed: Callable[[SeedContext], Awaitable[str | None] | str | None] | None = None
-    """Body of a seed by mode; ``None`` from it falls back to the defaults below."""
-    seed_head: str = "[seed: {seed}] {kind}{title}, {stamp}."
-    seed_copy: str = "The parent's history is copied ({scope}); continue in it."
-    seed_copy_window: str = "last {window} turns"
-    seed_copy_all: str = "whole history"
-    seed_morning_missing: str = "No handout arrived."
-    handout: Callable[[HandoutContext], Awaitable[str] | str] | str = (
-        "This master is closing ({reason}). Write the handout for {day}: a "
-        "briefing for the morning, not a task list - past tense, no imperatives."
-    )
-    new_day: Callable[[NewDayContext], Awaitable[str] | str] | str = (
-        "The master was replaced ({reason}); the handout for {day} is written."
-    )
-    moved_injects: str = " {moved} queued injects moved over from the old master."
-    distill: Callable[[DistillContext], Awaitable[str] | str] | None = None
-    """The distiller fork's first message; ``None`` uses the two templates below."""
-    distill_prompt: str = (
-        "Deep chat «{chat}» is closed ({reason}), today is {day}. Write the "
-        "digest as a file and the merge as your reply: up to 5 lines, third "
-        "person."
-    )
-    distill_prompt_no_memory: str = (
-        "Deep chat «{chat}» is closed ({reason}), today is {day}. Memory is off "
-        "for it: write no file, only the merge as your reply - up to 5 lines, "
-        "third person."
-    )
-    closed: str = "Deep chat [[{chat}]] closed{digest}.\n{text}"
-    closed_digest: str = ", digest [[{digest}]]"
-    digest_missing: str = "the digest file did not appear"
-    too_long: str = (
-        "`{name}`: {lines} lines against a cap of {max_lines}. The write was "
-        "rejected and the file restored. Shorten and rewrite."
-    )
-    cut_by_restart: str = "turn {turn_id} was cut by a gateway restart"
-    repaired_tools: str = (
-        "; {fixed} open tool calls received tool_result «{interrupted}»"
-    )
-    envelope: EnvelopeTexts = field(default_factory=EnvelopeTexts)

src/beaver_gateway/conversations/turns.py

@@ -1,555 +0,0 @@
-"""Running turns: one worker per conversation over its queue, the backend call."""
-
-from __future__ import annotations
-
-import asyncio
-import base64
-import contextlib
-import inspect
-import logging
-from datetime import UTC, datetime
-from pathlib import Path
-from typing import TYPE_CHECKING, Any, cast
-from uuid import uuid4
-
-from claude_agent_sdk import (
-    AssistantMessage,
-    RateLimitEvent,
-    ResultMessage,
-    StreamEvent,
-    ToolResultBlock,
-    ToolUseBlock,
-    UserMessage,
-)
-
-from beaver_gateway.backends.capture import TurnCapture
-from beaver_gateway.backends.transcript import text_of
-from beaver_gateway.conversations import injects
-from beaver_gateway.conversations.seeds import Seeds
-from beaver_gateway.conversations.state import Runner, aware, iso
-from beaver_gateway.conversations.texts import UserSaid
-from beaver_gateway.frontends.accumulate import StreamAccumulator
-from beaver_gateway.storage.models import Conversation, InjectQueueItem, RateLimit
-
-if TYPE_CHECKING:
-    from collections.abc import AsyncIterator, Callable, Sequence
-
-    from beaver_gateway.events.stream import MessageStreamEvent
-
-__all__ = ["Turns", "content_of", "image_block"]
-
-_log = logging.getLogger(__name__)
-
-
-class Turns(Seeds):
-    async def turn(
-        self,
-        conv: Conversation,
-        *,
-        messages: Sequence[Any],
-        origin: str,
-        capture: TurnCapture | None = None,
-        session_id: str | None = None,
-        use_session: bool = True,
-        tools: bool = True,
-        turn_id: str | None = None,
-        item_origin: str | None = None,
-        shown: str | None = None,
-        attach: bool = False,
-    ) -> AsyncIterator[MessageStreamEvent]:
-        """Run one turn under the conversation's lock; the only path to the backend.
-
-        ``shown`` is what frontends display as the turn's prompt (the user's
-        words, not the envelope). ``attach`` sends nothing: the turn is one
-        the live session started by itself, a subagent reporting back.
-        """
-        row_id = cast("int", conv.id)
-        runner = self._runner(row_id)
-        backend = self._backend(conv.agent_name)
-        turn_id = turn_id or f"turn_{uuid4().hex[:12]}"
-        capture = capture or TurnCapture()
-        resume = session_id if session_id is not None else conv.session_id
-        async with runner.lock:
-            runner.turn_id = turn_id
-            runner.origin = origin
-            runner.text = shown if shown is not None else _prompt_preview(messages)
-            runner.started_at = datetime.now(UTC)
-            runner.tools = {}
-            await self._mark_running(conv, turn_id)
-            before = await self.before_turn(conv)
-            self._bus.publish(
-                "turn.start",
-                conversation_id=conv.external_id,
-                turn_id=turn_id,
-                origin=origin,
-                item_origin=item_origin,
-                text=runner.text,
-            )
-            stop = "error"
-            cut = False
-            try:
-                observer = self._observer(conv, runner, turn_id, origin)
-                events = (
-                    backend.attach(
-                        key=conv.external_id,
-                        capture=capture,
-                        observer=observer,
-                        turn_id=turn_id,
-                    )
-                    if attach
-                    else backend.complete(
-                        agent=self._claude_agent(conv.agent_name),
-                        messages=messages,
-                        conversation_id=conv.external_id,
-                        session_id=resume if use_session else None,
-                        reseed=not use_session,
-                        capture=capture,
-                        kind=conv.kind,
-                        pinned=conv.kind == "master",
-                        tools=tools,
-                        observer=observer,
-                        turn_id=turn_id,
-                    )
-                )
-                async for event in events:
-                    yield event
-                stop = "interrupted" if capture.interrupted else "end_turn"
-            except asyncio.CancelledError:
-                cut = True
-                raise
-            finally:
-                runner.turn_id = None
-                await self._mark_done(conv, capture, cut=cut)
-                if stop != "error":
-                    try:
-                        await self.after_turn(conv, before)
-                    except Exception:  # noqa: BLE001
-                        _log.exception("after-turn hook on %s failed", conv.external_id)
-                self._bus.publish(
-                    "turn.end",
-                    conversation_id=conv.external_id,
-                    turn_id=turn_id,
-                    origin=origin,
-                    item_origin=item_origin,
-                    stop=stop,
-                    usage=_usage_dict(capture),
-                )
-
-    async def run_text_turn(
-        self,
-        conv: Conversation,
-        text: str,
-        *,
-        origin: str,
-        tools: bool = True,
-        turn_id: str | None = None,
-        item_origin: str | None = None,
-        attachments: Sequence[dict[str, Any]] | None = None,
-        shown: str | None = None,
-        attach: bool = False,
-    ) -> tuple[str, TurnCapture]:
-        capture = TurnCapture()
-        acc = StreamAccumulator()
-        agent = self._claude_agent(conv.agent_name)
-        async for event in self.turn(
-            conv,
-            messages=[{"role": "user", "content": content_of(text, attachments)}],
-            origin=origin,
-            capture=capture,
-            tools=tools,
-            turn_id=turn_id,
-            item_origin=item_origin,
-            shown=shown,
-            attach=attach,
-        ):
-            acc.feed(event)
-        message = acc.finalize(model=agent.model)
-        reply = "\n\n".join(
-            getattr(b, "text", "")
-            for b in message.content
-            if getattr(b, "type", "") == "text"
-        ).strip()
-        return reply, capture
-
-    async def before_turn(self, conv: Conversation) -> str | None:  # noqa: ARG002
-        return None
-
-    async def after_turn(self, conv: Conversation, before: str | None) -> None:  # noqa: ARG002
-        return
-
-    def turn_origin(self, conv: Conversation) -> str | None:
-        runner = self._runners.get(cast("int", conv.id))
-        return runner.origin if runner is not None and runner.turn_id else None
-
-    async def _woken(self, key: str, note: str) -> None:
-        """The live session of ``key`` started a turn by itself; run it next."""
-        conv = await self.get(key)
-        if conv is None:
-            _log.warning("self-started turn on unknown conversation %s: %s", key, note)
-            return
-        _log.info("conversation %s woke by itself: %s", key, note)
-        self._ensure_worker(cast("int", conv.id))
-
-    async def _run_pending(self, conv: Conversation) -> None:
-        turn_id = f"turn_{uuid4().hex[:12]}"
-        note = self._backend(conv.agent_name).pending_note(conv.external_id) or ""
-        try:
-            text, _ = await self.run_text_turn(
-                conv,
-                note,
-                origin="task",
-                turn_id=turn_id,
-                item_origin="task",
-                shown=note,
-                attach=True,
-            )
-        except Exception:  # noqa: BLE001
-            _log.exception(
-                "self-started turn %s on %s failed", turn_id, conv.external_id
-            )
-            return
-        if not text.strip():
-            return
-        self._bus.publish(
-            "reply",
-            conversation_id=conv.external_id,
-            turn_id=turn_id,
-            item=None,
-            item_origin="task",
-            source="task",
-            prompt=self._texts.task_prompt.format(note=note),
-            user_text=None,
-            text=text,
-        )
-
-    def _ensure_worker(self, row_id: int) -> None:
-        runner = self._runner(row_id)
-        runner.wake.set()
-        if runner.task is None or runner.task.done():
-            runner.task = asyncio.create_task(self._worker(row_id))
-            self._track(runner.task)
-
-    async def _worker(self, row_id: int) -> None:
-        runner = self._runner(row_id)
-        while True:
-            conv = await self.get_row(row_id)
-            if conv is not None and self._has_pending(conv):
-                await self._run_pending(conv)
-                continue
-            items = await self._queue.pending(row_id)
-            batch, wait = self._pick(items)
-            if batch is None:
-                runner.wake.clear()
-                if wait is None:
-                    return
-                with contextlib.suppress(TimeoutError):
-                    await asyncio.wait_for(runner.wake.wait(), timeout=wait)
-                continue
-            conv = await self.get_row(row_id)
-            if conv is None:
-                await self._queue.finish(batch, status="failed")
-                return
-            await self._run_batch(conv, batch)
-
-    def _pick(
-        self, items: list[InjectQueueItem]
-    ) -> tuple[list[InjectQueueItem] | None, float | None]:
-        if not items:
-            return None, None
-        head = items[0]
-        if head.priority == "urgent":
-            return [head], None
-        tail = [i for i in items if i is not head and i.priority in ("wake", "normal")]
-        if head.priority in ("user", "wake"):
-            return [head, *tail], None
-        age = (datetime.now(UTC) - aware(head.created_at)).total_seconds()
-        if age >= self._normal_window:
-            return [head, *tail], None
-        return None, max(self._normal_window - age, 1.0)
-
-    async def _run_batch(
-        self, conv: Conversation, batch: list[InjectQueueItem]
-    ) -> None:
-        head = batch[0]
-        turn_id = f"turn_{uuid4().hex[:12]}"
-        await self._queue.start(batch, turn_id)
-        if head.priority == "user":
-            origin = "user"
-            prompt = head.text
-            await self._note_user(conv, head.text)
-            envelope = self._envelope_for(conv, head.text)
-            if envelope:
-                prompt += "\n\n" + envelope
-            if len(batch) > 1:
-                prompt += "\n\n" + self._bundle(batch[1:])
-        else:
-            origin = "inject"
-            prompt = "\n\n".join(
-                f"{self._texts.inject_header(injects.context_of(i))}\n{i.text}"
-                for i in batch
-            )
-        seed = await self.pending_seed(conv)
-        if seed:
-            prompt = f"{seed}\n\n{prompt}"
-        try:
-            text, capture = await self.run_text_turn(
-                conv,
-                prompt,
-                origin=origin,
-                turn_id=turn_id,
-                item_origin=head.origin,
-                attachments=head.attachments if origin == "user" else None,
-                shown=head.text if origin == "user" else None,
-            )
-        except Exception:  # noqa: BLE001
-            _log.exception("turn %s on %s failed", turn_id, conv.external_id)
-            await self._queue.finish(batch, status="failed")
-            return
-        await self._queue.finish(
-            batch, status="interrupted" if capture.interrupted else "done"
-        )
-        if origin == "user":
-            self._bus.publish(
-                "reply",
-                conversation_id=conv.external_id,
-                turn_id=turn_id,
-                item=head.id,
-                item_origin=head.origin,
-                source="queue",
-                prompt=prompt,
-                user_text=head.text,
-                text=text,
-            )
-
-    def _has_pending(self, conv: Conversation) -> bool:
-        try:
-            return self._backend(conv.agent_name).has_pending(conv.external_id)
-        except LookupError:
-            return False
-
-    def _bundle(self, items: Sequence[InjectQueueItem]) -> str:
-        lines = [self._texts.bundle_header.format(since=iso(items[0].created_at))]
-        lines.extend(f"- [{i.origin}] {i.text}" for i in items)
-        return "\n".join(lines)
-
-    def _envelope_for(self, conv: Conversation, text: str = "") -> str | None:
-        if self._envelope is None:
-            return None
-        if conv.kind == "master":
-            return self._envelope.build(text=text, kind="master", agent=conv.agent_name)
-        if conv.kind == "branch":
-            return self._envelope.recall_only(
-                text=text, kind="branch", agent=conv.agent_name
-            )
-        return None
-
-    async def _note_user(self, conv: Conversation, text: str) -> None:
-        if self._user_sink is None:
-            return
-        message = UserSaid(
-            conversation_id=conv.external_id,
-            kind=conv.kind,
-            title=conv.title,
-            text=text,
-            at=datetime.now(UTC),
-            agent=conv.agent_name,
-        )
-        try:
-            result = self._user_sink(message)
-            if inspect.isawaitable(result):
-                await result
-        except Exception:  # noqa: BLE001
-            _log.exception("user sink failed for %s", conv.external_id)
-
-    def _observer(
-        self, conv: Conversation, runner: Runner, turn_id: str, origin: str
-    ) -> Callable[[Any], None]:
-        conversation_id = conv.external_id
-
-        def observe(message: Any) -> None:
-            parent = getattr(message, "parent_tool_use_id", None)
-            if isinstance(message, RateLimitEvent):
-                self._observe_rate_limit(conv, message)
-            elif isinstance(message, StreamEvent):
-                self._bus.publish(
-                    "stream",
-                    conversation_id=conversation_id,
-                    turn_id=turn_id,
-                    origin=origin,
-                    parent_tool_use_id=parent,
-                    event=message.event,
-                )
-            elif isinstance(message, AssistantMessage):
-                for block in message.content:
-                    if isinstance(block, ToolUseBlock):
-                        event = self._bus.publish(
-                            "tool",
-                            conversation_id=conversation_id,
-                            turn_id=turn_id,
-                            origin=origin,
-                            parent_tool_use_id=parent,
-                            tool_use_id=block.id,
-                            name=block.name,
-                            input=block.input,
-                        )
-                        runner.tools[block.id] = {
-                            "tool_use_id": block.id,
-                            "name": block.name,
-                            "input": block.input,
-                            "parent_tool_use_id": parent,
-                            "started_at": event["ts"],
-                            "ended_at": None,
-                            "is_error": None,
-                            "content": None,
-                        }
-            elif isinstance(message, UserMessage):
-                blocks = message.content if isinstance(message.content, list) else ()
-                for block in blocks:
-                    if isinstance(block, ToolResultBlock):
-                        event = self._bus.publish(
-                            "tool.result",
-                            conversation_id=conversation_id,
-                            turn_id=turn_id,
-                            origin=origin,
-                            parent_tool_use_id=parent,
-                            tool_use_id=block.tool_use_id,
-                            is_error=bool(block.is_error),
-                            content=_result_preview(block.content),
-                        )
-                        node = runner.tools.get(block.tool_use_id)
-                        if node is not None:
-                            node["ended_at"] = event["ts"]
-                            node["is_error"] = event["is_error"]
-                            node["content"] = event["content"]
-            elif isinstance(message, ResultMessage) and parent is None:
-                self._bus.publish(
-                    "result",
-                    conversation_id=conversation_id,
-                    turn_id=turn_id,
-                    origin=origin,
-                    subtype=message.subtype,
-                    is_error=message.is_error,
-                    num_turns=message.num_turns,
-                )
-
-        return observe
-
-    def _observe_rate_limit(self, conv: Conversation, message: RateLimitEvent) -> None:
-        info = message.rate_limit_info
-        row = RateLimit(
-            window=info.rate_limit_type or "unknown",
-            status=info.status,
-            utilization=info.utilization,
-            resets_at=_from_unix(info.resets_at),
-            overage_status=info.overage_status,
-            overage_resets_at=_from_unix(info.overage_resets_at),
-            agent_name=conv.agent_name,
-            session_id=message.session_id,
-            raw=dict(info.raw),
-        )
-        self._bus.publish(
-            "rate_limit",
-            conversation_id=conv.external_id,
-            window=row.window,
-            status=row.status,
-            utilization=row.utilization,
-            resets_at=iso(row.resets_at),
-            overage_status=row.overage_status,
-        )
-        self._track(asyncio.create_task(self._record_rate_limit(row)))
-
-    async def _record_rate_limit(self, row: RateLimit) -> None:
-        try:
-            async with self._db.session() as session:
-                session.add(row)
-                await session.commit()
-        except Exception:  # noqa: BLE001
-            _log.exception("rate limit write failed")
-
-    async def _mark_running(self, conv: Conversation, turn_id: str) -> None:
-        async def apply(row: Conversation) -> None:
-            row.running_turn = turn_id
-            row.last_activity_at = datetime.now(UTC)
-
-        await self._update(conv, apply)
-
-    async def _mark_done(
-        self, conv: Conversation, capture: TurnCapture, *, cut: bool = False
-    ) -> None:
-        async def apply(row: Conversation) -> None:
-            if not cut:
-                row.running_turn = None
-            row.last_activity_at = datetime.now(UTC)
-            if capture.session_id is not None:
-                row.session_id = capture.session_id
-
-        await self._update(conv, apply)
-
-
-IMAGE_MAX_BYTES = 5 * 1024 * 1024
-
-
-def content_of(
-    text: str, attachments: Sequence[dict[str, Any]] | None
-) -> str | list[dict[str, Any]]:
-    """Plain text, or text plus the images that ride with it."""
-    blocks = [b for a in attachments or () if (b := image_block(a)) is not None]
-    if not blocks:
-        return text
-    return [{"type": "text", "text": text}, *blocks]
-
-
-def image_block(attachment: dict[str, Any]) -> dict[str, Any] | None:
-    media_type = str(attachment.get("media_type") or "")
-    path = Path(str(attachment.get("path") or ""))
-    if not media_type.startswith("image/") or not path.is_file():
-        return None
-    if path.stat().st_size > IMAGE_MAX_BYTES:
-        return None
-    data = base64.b64encode(path.read_bytes()).decode("ascii")
-    return {
-        "type": "image",
-        "source": {"type": "base64", "media_type": media_type, "data": data},
-    }
-
-
-def _prompt_preview(messages: Sequence[Any], limit: int = 400) -> str | None:
-    if not messages:
-        return None
-    text = text_of(messages[-1].get("content"))
-    return text[:limit] if text else None
-
-
-def _from_unix(value: int | None) -> datetime | None:
-    return datetime.fromtimestamp(value, tz=UTC) if value is not None else None
-
-
-def _result_preview(
-    content: str | list[dict[str, Any]] | None, limit: int = 400
-) -> str:
-    if content is None:
-        return ""
-    text = (
-        content
-        if isinstance(content, str)
-        else "\n".join(
-            str(part.get("text", ""))
-            for part in content
-            if isinstance(part, dict) and part.get("type") == "text"
-        )
-    )
-    return text if len(text) <= limit else text[:limit] + "…"
-
-
-def _usage_dict(capture: TurnCapture) -> dict[str, Any] | None:
-    usage = capture.usage
-    if usage is None:
-        return None
-    return {
-        "input": usage.input_tokens,
-        "output": usage.output_tokens,
-        "cache_read": usage.cache_read_tokens,
-        "cache_creation": usage.cache_creation_tokens,
-        "cost_usd": usage.cost_usd,
-        "duration_ms": usage.duration_ms,
-    }

src/beaver_gateway/core/__init__.py

@@ -0,0 +1,7 @@
+"""Cross-cutting machinery: registries, event protocol, auth, sessions."""
+
+from __future__ import annotations
+
+from beaver_gateway.core.registry import AgentRegistry, Gateway, McpRegistry
+
+__all__ = ["AgentRegistry", "Gateway", "McpRegistry"]

src/beaver_gateway/core/audit.py

@@ -0,0 +1,94 @@
+"""Single entry point for writing :class:`AuditLog` rows.
+
+Every frontend ends up needing the same three-line pattern — open a DB
+session, append a row, swallow failures so the user-visible request
+still succeeds. Phase 4.3 inlined that pattern in the admin frontend
+under a private ``_audit()`` helper; Phase 4.4 lifts it here so the
+Messages and MCP frontends can call the same function and so the
+swallow-and-log policy lives in one place.
+
+The contract:
+
+* ``log(runtime, actor=..., kind=...)`` is fire-and-forget. It awaits
+  the DB write (so callers can ``await`` it before responding and get
+  ordering), but never raises — if the audit insert fails, the function
+  emits an ``exception`` log line and returns.
+* ``actor`` is a free-form string. By convention: ``"token:<name>"``
+  for bearer-authenticated traffic, ``"admin:<user>"`` for admin-UI
+  actions, ``"anon"`` for failed-auth paths we still want to record.
+* ``kind`` is a short tag — see :data:`KNOWN_KINDS` for the set the
+  current frontends emit; new tags don't need a code change here, the
+  column is free-form.
+* ``**detail`` is JSON-serialised by :func:`append_audit`. Keep it
+  small: paths, methods, status codes — not request bodies. Anything
+  passed here lands in ``AuditLog.detail_json`` verbatim.
+
+Why a thin wrapper rather than ``append_audit`` directly: callers want
+"write if you can, otherwise carry on", and pulling the try/except into
+every frontend was already starting to drift (admin had it, bearer
+frontends would have copy-pasted). One module, one policy.
+"""
+
+from __future__ import annotations
+
+import logging
+from typing import TYPE_CHECKING, Any
+
+from beaver_gateway.storage import append_audit
+
+if TYPE_CHECKING:
+    from beaver_gateway.frontends.base import GatewayRuntime
+
+
+_log = logging.getLogger("beaver_gateway.audit")
+
+
+# Tags currently emitted by the gateway. The set is informational —
+# ``AuditLog.kind`` is free-form so new code can introduce new tags
+# without touching this list — but listing them here gives the admin UI
+# and any downstream log consumers one canonical reference.
+KNOWN_KINDS: frozenset[str] = frozenset(
+    {
+        "messages",  # POST /v1/messages accepted
+        "mcp_call",  # /mcp/<ns>/... proxied
+        "tool_call",  # a model's tool call seen by the PreToolUse hook
+        "login_ok",
+        "login_failed",
+        "logout",
+        "token_create",
+        "token_revoke",
+    }
+)
+
+
+async def log(
+    runtime: GatewayRuntime,
+    *,
+    actor: str,
+    kind: str,
+    agent_name: str | None = None,
+    **detail: Any,
+) -> None:
+    """Best-effort audit insert. Never raises.
+
+    Opens its own short-lived :class:`AsyncSession` so callers don't
+    have to thread one through. If the DB hiccups (table missing,
+    disk full, connection drop), we log and move on — the audit trail
+    is observability, not a hard precondition for serving the request.
+    """
+    try:
+        async with runtime.db.session() as session:
+            await append_audit(
+                session,
+                actor=actor,
+                kind=kind,
+                agent_name=agent_name,
+                detail=detail or None,
+            )
+    except Exception:  # noqa: BLE001
+        _log.exception(
+            "audit write failed: actor=%s kind=%s agent=%s", actor, kind, agent_name
+        )
+
+
+__all__ = ["KNOWN_KINDS", "log"]

src/beaver_gateway/core/auth.py

@@ -1,7 +1,32 @@
-"""Bearer-token verification: a DB-backed store with an in-memory cache.
+"""Bearer-token verification (Phase 4.2 — DB-backed with in-memory cache).
 
-Optional ``BOOTSTRAP_TOKENS`` env entries are checked first. Verify takes
-a raw token or ``Authorization`` header and returns a :class:`TokenIdentity`.
+The store is fed by two sources:
+
+1. **DB** (``Token`` table from Phase 4.1) — the primary source. Rows
+   carry Argon2id hashes; the admin UI (Phase 4.3) will be the only
+   writer at steady state.
+2. **`BOOTSTRAP_TOKENS`** env — a name→plaintext map kept around for
+   first-run, disaster-recovery, and ``examples/`` smoke tests. These
+   entries live alongside DB rows in the cache and are never persisted.
+
+Hot path is in-memory: at :meth:`start` we pull every non-revoked DB
+row and stash it in a list; subsequent :meth:`verify` calls re-load
+when the cache is older than ``ttl_seconds``. ``last_used_at`` updates
+are coalesced into a small dict and flushed by a background task every
+``flush_interval`` seconds — one transaction per flush rather than one
+per request.
+
+We can't index DB rows by a derived plaintext key because Argon2 salts
+are random — so verify does a linear scan over candidates, calling
+``argon2.PasswordHasher.verify`` on each. N is small by design (single
+operator, ~10 tokens at most); the cost is irrelevant. The scan runs
+through ``asyncio.to_thread`` to keep the event loop free of the ~50ms
+KDF block.
+
+The module knows nothing about HTTP frameworks. It takes a raw token
+(or a verbatim ``Authorization`` header value) and returns a
+:class:`TokenIdentity` (name + scope + db-id), or ``None`` for a miss.
+Frontends own the 401 response shape.
 """
 
 from __future__ import annotations
@@ -21,7 +46,7 @@ from argon2.exceptions import InvalidHashError, VerifyMismatchError
 from beaver_gateway.storage import list_active_tokens, touch_token
 
 if TYPE_CHECKING:
-    from collections.abc import Iterable, Mapping
+    from collections.abc import Mapping
 
     from beaver_gateway.storage import Database
 
@@ -35,8 +60,8 @@ class TokenStoreError(ValueError):
     """Malformed ``BOOTSTRAP_TOKENS`` value or duplicate token."""
 
 
-BUILTIN_SCOPES: frozenset[str] = frozenset({"*", "messages", "mcp", "admin", "api"})
-"""The scopes the gateway's own frontends define.
+VALID_SCOPES: frozenset[str] = frozenset({"*", "messages", "mcp", "admin", "api"})
+"""The scopes a ``Token.scope`` may hold (Phase 4.3 admin UI enforces).
 
 * ``*`` — wildcard, may use any frontend
 * ``messages`` — Anthropic Messages frontend only
@@ -44,25 +69,17 @@ BUILTIN_SCOPES: frozenset[str] = frozenset({"*", "messages", "mcp", "admin", "ap
 * ``api`` — conversations API (``/api``) and its SSE
 * ``admin`` — reserved for programmatic admin access; the AdminFrontend
   itself authenticates via session cookies, not bearer tokens, so this
-  scope is unused today.
-
-A setup's own frontend adds its scope on top; see :func:`scopes_with` and
-``GatewayRuntime.scopes``, which is what ``POST /api/tokens`` validates.
+  scope is unused today and kept for forward compatibility.
 """
 
 
-def scopes_with(extra: Iterable[str | None]) -> frozenset[str]:
-    """The builtin scopes plus every frontend's own; ``None`` entries drop out."""
-    return BUILTIN_SCOPES | {s for s in extra if s}
-
-
 @dataclass(frozen=True, slots=True)
 class TokenIdentity:
     """What :meth:`TokenStore.verify` resolves to on success.
 
     ``token_id`` is the DB row id for persisted tokens, or ``None`` for
     an env-bootstrap match (those have no DB row to touch). ``scope``
-    gates which frontend the token may hit (see :data:`BUILTIN_SCOPES`);
+    gates which frontend the token may hit (see :data:`VALID_SCOPES`);
     bootstrap tokens implicitly get ``"*"``.
     """
 
@@ -75,8 +92,8 @@ class TokenIdentity:
 
         ``"*"`` is the wildcard; an exact match satisfies a single scope.
         Unknown ``required`` values intentionally fall through to a
-        strict equality check — callers should pass a scope the runtime
-        knows (:data:`BUILTIN_SCOPES` plus every frontend's own).
+        strict equality check — callers should pass one of
+        :data:`VALID_SCOPES`.
         """
         return self.scope in ("*", required)
 
@@ -91,23 +108,33 @@ class _CachedToken:
     hashed_value: str
 
 
+# Default Argon2id parameters from ``argon2-cffi`` are fine for our scope.
+# They target ~50ms on a modern CPU — enough to make a stolen-hash brute
+# force expensive, cheap enough to verify a handful per request.
 _HASHER = PasswordHasher()
 
 
 def hash_token(plaintext: str) -> str:
-    """Return an Argon2id hash for ``plaintext``, for admin/seed use."""
+    """Return an Argon2id hash for ``plaintext`` (admin / seed-only path).
+
+    Phase 4.3 will call this when the admin creates a token; Phase 4.2
+    exposes it so smoke scripts can seed the DB without re-implementing
+    the same line.
+    """
     return _HASHER.hash(plaintext)
 
 
 class TokenStore:
-    """DB-backed verifier with an in-memory cache, TTL, and batched touches.
+    """DB-backed verifier with in-memory cache + TTL + batched touches.
 
-    Call ``await store.start()`` after construction to prime the cache
-    and start the flusher; ``await store.stop()`` drains the touch queue.
+    Construct in ``cli.main`` after :class:`Database` is up, then
+    ``await store.start()`` to prime the cache and spin up the flusher
+    task. ``await store.stop()`` on shutdown drains the touch queue.
 
-    Bootstrap entries (``BOOTSTRAP_TOKENS``) are checked first and stay
-    usable even if the DB is unreachable; they never get flushed to
-    ``last_used_at`` since they have no DB row.
+    Bootstrap entries (from ``BOOTSTRAP_TOKENS``) sit alongside DB rows
+    in the same lookup path; we check them first, in constant time, so
+    they remain usable even if the DB is unreachable. They never appear
+    in ``last_used_at`` flushes because they have no DB row.
     """
 
     __slots__ = (
@@ -132,11 +159,8 @@ class TokenStore:
         ttl_seconds: float = 30.0,
         flush_interval: float = 5.0,
     ) -> None:
-        """Index bootstrap tokens for O(1) verify lookup.
-
-        Each one's scope defaults to ``"*"`` unless narrowed via
-        ``name:value:scope``.
-        """
+        # Bootstrap is keyed by value internally so verify is O(1) over
+        # plaintext. Each value also keeps its name for audit lines.
         by_value: dict[str, str] = {}
         for name, value in (bootstrap or {}).items():
             if not name or not value:
@@ -150,6 +174,9 @@ class TokenStore:
                 raise TokenStoreError(msg)
             by_value[value] = name
         self._bootstrap_by_value: dict[str, str] = by_value
+        # Bootstrap entries are ``"*"`` unless the env narrows them
+        # (``name:value:scope``) - a webhook token in a URL should not be
+        # an admin token.
         self._bootstrap_scopes: dict[str, str] = {
             name: (bootstrap_scopes or {}).get(name, _BOOTSTRAP_SCOPE)
             for name in by_value.values()
@@ -165,6 +192,8 @@ class TokenStore:
         self._touch_queue: dict[int, datetime] = {}
         self._flusher_task: asyncio.Task[None] | None = None
 
+    # ---- bootstrap parsing (kept for `cli` / tests) ---------------------
+
     @staticmethod
     def parse_bootstrap(raw: str) -> dict[str, str]:
         """Parse ``name1:value1,name2:value2[:scope]`` (``BOOTSTRAP_TOKENS``)."""
@@ -181,9 +210,15 @@ class TokenStore:
 
     @classmethod
     def from_env(cls, raw: str, db: Database | None = None) -> TokenStore:
-        """Bootstrap-only (or bootstrap + db) convenience constructor."""
+        """Legacy entrypoint: bootstrap-only (or bootstrap + db).
+
+        Phase 1.3 call sites still expect a one-liner; we keep the
+        classmethod so they don't have to learn the new constructor.
+        """
         return cls(db, bootstrap=cls.parse_bootstrap(raw))
 
+    # ---- lifecycle ------------------------------------------------------
+
     async def start(self) -> None:
         """Prime the cache and (if a DB is attached) start the flusher loop."""
         await self._refresh()
@@ -207,19 +242,19 @@ class TokenStore:
         self._bootstrap_scopes[name] = scope
 
     async def invalidate(self) -> None:
-        """Force the next verify to re-read from DB."""
+        """Force the next verify to re-read from DB (Phase 4.3 admin hook)."""
         self._loaded_at = 0.0
 
-    async def verify(self, token: str | None) -> TokenIdentity | None:
-        """Return the matching identity, or ``None`` for unknown/empty tokens.
+    # ---- verify path ----------------------------------------------------
 
-        Bootstrap entries are checked first via constant-time compare;
-        the DB cache list is swapped rather than mutated on refresh, so
-        a concurrent reload can't affect an in-progress scan.
-        """
+    async def verify(self, token: str | None) -> TokenIdentity | None:
+        """Return the matching identity, or ``None`` for unknown/empty tokens."""
         if not token:
             return None
 
+        # Bootstrap first: constant-time compare per entry, never hits DB.
+        # `compare_digest` is overkill for a name→value lookup but cheap
+        # and removes one timing variable for free.
         for value, name in self._bootstrap_by_value.items():
             if hmac.compare_digest(token, value):
                 return TokenIdentity(
@@ -233,6 +268,9 @@ class TokenStore:
 
         await self._ensure_fresh()
 
+        # Snapshot the cache reference so a refresh mid-scan doesn't
+        # surprise us. List itself is immutable per refresh (we swap,
+        # not mutate).
         cache = self._cache
         for entry in cache:
             try:
@@ -268,18 +306,17 @@ class TokenStore:
     def __bool__(self) -> bool:
         return bool(self._cache) or bool(self._bootstrap_by_value)
 
-    async def _ensure_fresh(self) -> None:
-        """Reload the cache when stale.
+    # ---- internals ------------------------------------------------------
 
-        Double-checked under the lock so concurrent callers don't all
-        trigger a reload.
-        """
+    async def _ensure_fresh(self) -> None:
         if self._db is None:
             return
         now = time.monotonic()
         if now - self._loaded_at <= self._ttl:
             return
         async with self._lock:
+            # Re-check under the lock — first arrival reloaded, others
+            # should fall through.
             now = time.monotonic()
             if now - self._loaded_at <= self._ttl:
                 return
@@ -294,6 +331,8 @@ class TokenStore:
         next_cache: list[_CachedToken] = []
         for row in rows:
             if row.id is None:
+                # Defensive: SQLModel will assign an id on insert; a
+                # None here would mean someone handed us an unsaved row.
                 continue
             next_cache.append(
                 _CachedToken(
@@ -314,31 +353,30 @@ class TokenStore:
                 await self._flush_now()
         except asyncio.CancelledError:
             raise
-        except Exception:  # noqa: BLE001
+        except Exception:  # noqa: BLE001 — never let the flusher die silently
             _log.exception("token flusher crashed; touches will stop")
 
     async def _flush_now(self) -> None:
-        """Flush queued touches in one transaction.
-
-        Detaches the queue first so concurrent ``verify()`` writes land
-        in the next flush.
-        """
         if self._db is None or not self._touch_queue:
             return
+        # Detach the queue so concurrent verify() writes don't bleed
+        # into the in-flight transaction.
         pending, self._touch_queue = self._touch_queue, {}
         async with self._db.session() as session:
             for token_id in pending:
+                # We don't pass the timestamp through — `touch_token`
+                # stamps `now` itself, and we'd rather have one source
+                # of truth than reconcile clocks.
                 await touch_token(session, token_id=token_id)
         _log.debug("flushed %d token touch(es)", len(pending))
 
 
 __all__ = [
-    "BUILTIN_SCOPES",
+    "VALID_SCOPES",
     "TokenIdentity",
     "TokenStore",
     "TokenStoreError",
     "hash_token",
-    "scopes_with",
 ]
 
 

src/beaver_gateway/core/bus.py

@@ -1,7 +1,10 @@
 """In-process event bus: what the gateway does, as a stream frontends can tap.
 
-Events are plain dicts. Publishers never block; a subscriber that falls
-behind loses its oldest events rather than stalling the publisher.
+Events are plain dicts ``{"type", "seq", "ts", ...}``. Publishers never
+block; a subscriber that falls behind loses its oldest events rather than
+stalling the turn that produced them. ``/api/events`` serialises the
+stream as SSE, the panel builds its subagent tree from ``stream`` events
+carrying ``parent_tool_use_id``.
 """
 
 from __future__ import annotations

src/beaver_gateway/core/conversation_store.py

@@ -1,7 +1,41 @@
-"""Stored conversation history for the markdown frontend.
+"""Stateful conversation history for the markdown frontend.
 
-Aligns the file the user is editing against the DB-stored Anthropic-shape
-message history, splicing text edits or forking on structural changes.
+The gateway used to be stateless about identity: claude-code-api's
+in-memory session pool was keyed by a fingerprint of the messages the
+gateway forwarded, and on a fingerprint miss the same fingerprint was
+used to seed a fresh PTY's JSONL transcript. That worked as long as
+the frontend could round-trip the *exact* content blocks the live
+session had observed. The markdown frontend can't — the parser strips
+``[!tool]-`` callouts because the human is allowed to edit the prose,
+and the rendered tool callouts don't carry the canonical ``tool_use``
+block fields anyway. So a continuation hit was *only* reliable for
+turns that never used a tool; once tools entered the picture, every
+subsequent turn missed the cache and reseeded from a tool-less
+transcript, leading to "assistant doesn't remember the tool calls it
+just made."
+
+This module makes the gateway stateful for the markdown frontend (and
+any other frontend that wants in). The DB stores the full
+Anthropic-shape message list — text blocks, ``tool_use`` blocks,
+``tool_result`` blocks, thinking signatures — exactly as
+claude-code-api would have seen on the wire. Before each turn we
+align the file the user is editing against the stored history:
+
+* If the user just appended a new user turn at the bottom, we feed
+  the backend our stored-plus-new history and the fingerprint hits.
+* If the user edited the *text* inside an assistant turn but left the
+  tool callouts alone, we splice the new text into the stored
+  ``tool_use`` blocks and feed *that* — the fingerprint misses (text
+  differs), claude-code-api reseeds with a full transcript (tools and
+  all), the new live session has memory of the prior tool calls.
+* If the user changed the *structure* (added/removed/reordered a tool
+  callout, edited an old user turn, etc.) we fork: take stored history
+  up to the divergence, take incoming text-only past the divergence.
+  The fingerprint misses; claude-code-api reseeds with a clean
+  truncated history; downstream turns continue from there.
+
+"Divergence point" is found by walking the file's turns and the
+stored display turns in lockstep. See :func:`diff_and_fork`.
 """
 
 from __future__ import annotations
@@ -33,12 +67,21 @@ __all__ = [
 ]
 
 
+# ---- types --------------------------------------------------------------
+
+
 @dataclass(frozen=True, slots=True)
 class ForkOutcome:
     """Result of aligning the incoming file against stored history.
 
-    ``messages`` is the backend input; ``persist_messages`` is history to
-    store before the new reply; ``divergence_index`` is where they diverged.
+    ``messages`` is what the gateway feeds to the backend (already
+    includes the new user prompt at the tail). ``persist_messages``
+    is the canonical conversation state the gateway should hold in
+    the DB *up to but not including* the new assistant reply — the
+    caller appends the synthesized turn from the backend onto this
+    and writes the result back. ``divergence_index`` is the
+    display-turn index at which incoming first disagreed with stored
+    (``None`` if everything matched; the new tail is appended cleanly).
     """
 
     messages: list[MessageParam]
@@ -52,6 +95,9 @@ class ForkOutcome:
         return self.divergence_index is None and not self.edited
 
 
+# ---- public store API ---------------------------------------------------
+
+
 async def load_conversation(
     session: AsyncSession, *, frontend: str, external_id: str
 ) -> Conversation | None:
@@ -69,7 +115,9 @@ async def mint_conversation(
 ) -> Conversation:
     """Create a fresh conversation row with a new uuid for external_id.
 
-    Caller must persist the returned ``external_id`` so future requests can find it.
+    Caller is responsible for persisting the returned ``external_id`` on
+    the frontend side (frontmatter, response header, …) so future
+    requests can find this conversation again.
     """
     row = Conversation(
         frontend=frontend, external_id=str(uuid.uuid4()), agent_name=agent_name
@@ -96,8 +144,9 @@ async def load_messages(
 ) -> list[dict[str, Any]]:
     """Return stored messages ordered by ``seq`` ascending.
 
-    Each entry is a canonical Anthropic ``MessageParam`` dict, the same
-    shape fed to the backend on continuation.
+    Each entry is a canonical Anthropic ``MessageParam`` dict — ``role``
+    plus ``content`` (string or list of block dicts). The same shape
+    we feed to the backend on continuation.
     """
     stmt = (
         select(ConversationMessage)
@@ -115,8 +164,10 @@ async def load_messages(
 def _sanitize_content(content: Any) -> Any:
     """Strip wire-illegal fields from stored Anthropic content blocks.
 
-    Drops ``tool_result.is_error: null`` — the API rejects null there
-    though the field is optional.
+    Older capture code emitted ``"is_error": null`` on ``tool_result``
+    blocks; the Anthropic API rejects null there (the field is optional
+    but, when present, must be boolean). We omit the key on read so
+    historical rows don't break continuation.
     """
     if not isinstance(content, list):
         return content
@@ -137,17 +188,27 @@ def _sanitize_content(content: Any) -> Any:
 async def rewrite_messages(
     session: AsyncSession, *, conversation_id: int, messages: list[dict[str, Any]]
 ) -> None:
-    """Replace the conversation's stored messages (full overwrite, no branch history).
+    """Replace the conversation's stored messages with ``messages``.
 
-    Deletes and flushes before inserting — SQLAlchemy's default INSERT-before-DELETE
-    flush order would otherwise collide with ``uq_msg_conv_seq``.
+    The user said no branch history — we overwrite on fork. Cheap at
+    our volume; if it ever matters we can switch to soft-delete +
+    branch pointers.
     """
+    # Bulk-delete and flush before inserting the new sequence: SQLAlchemy's
+    # unit-of-work flushes INSERTs before DELETEs by default, which would
+    # trip ``uq_msg_conv_seq`` when the new rows reuse the same seq numbers
+    # as the soon-to-be-deleted ones.
+    # SQLModel descriptors resolve to ColumnElement at runtime but to bare
+    # ``int`` in ty's stubs; the select-path at line 135 lives behind sqlmodel's
+    # own ``select`` overloads that hide it, but ``sqlalchemy.delete().where``
+    # uses the raw stubs.
     await session.execute(  # ty: ignore[deprecated]
         delete(ConversationMessage).where(
             ConversationMessage.conversation_id == conversation_id  # ty: ignore[invalid-argument-type]
         )
     )
     await session.flush()
+    # Insert the new sequence.
     for seq, m in enumerate(messages):
         session.add(
             ConversationMessage(
@@ -157,6 +218,7 @@ async def rewrite_messages(
                 content_json=json.dumps(m["content"], separators=(",", ":")),
             )
         )
+    # Bump conversation.updated_at.
     conv = await session.get(Conversation, conversation_id)
     if conv is not None:
         from datetime import UTC, datetime
@@ -166,11 +228,21 @@ async def rewrite_messages(
     await session.commit()
 
 
+# ---- alignment ----------------------------------------------------------
+
+
 @dataclass(frozen=True, slots=True)
 class _StoredDisplayTurn:
     """A "display turn" reconstructed from stored raw messages.
 
-    Parser-equivalent view used to diff the file against the DB.
+    ``role`` is ``"user"`` (single user-prompt message) or
+    ``"assistant"`` (one or more assistant messages, optionally
+    interleaved with user-only-tool_result messages). ``messages`` is
+    the slice of stored raw messages this display turn covers, in
+    order. ``spoken_text`` and ``skeleton`` are the
+    parser-equivalents for diff purposes; ``text_segment_count`` lets
+    us refuse a splice when the user edited across a tool boundary in
+    a way we can't safely undo.
     """
 
     role: str
@@ -183,8 +255,10 @@ class _StoredDisplayTurn:
 def _group_display_turns(stored: list[dict[str, Any]]) -> list[_StoredDisplayTurn]:
     """Walk raw stored messages, group them into Obsidian-visible turns.
 
-    A user-prompt message (no ``tool_result`` blocks) opens a new turn;
-    tool-result-only messages roll into the current assistant turn.
+    A user-prompt message (``role=user`` with string content, or list
+    content with no ``tool_result`` blocks) opens a user display turn.
+    Otherwise it's a tool-result follow-up and rolls into the current
+    assistant display turn.
     """
     out: list[_StoredDisplayTurn] = []
     i = 0
@@ -203,6 +277,7 @@ def _group_display_turns(stored: list[dict[str, Any]]) -> list[_StoredDisplayTur
             )
             i += 1
             continue
+        # Assistant display turn: collect consecutive non-prompt messages.
         group: list[dict[str, Any]] = []
         while i < len(stored):
             m = stored[i]
@@ -224,16 +299,14 @@ def _group_display_turns(stored: list[dict[str, Any]]) -> list[_StoredDisplayTur
 
 
 def _is_user_prompt(content: Any) -> bool:
-    """A user message is a *prompt* unless its content carries tool_result blocks.
-
-    Unknown content shapes are conservatively treated as a prompt.
-    """
+    """A user message is a *prompt* unless its content carries tool_result blocks."""
     if isinstance(content, str):
         return True
     if isinstance(content, list):
         return not any(
             isinstance(b, dict) and b.get("type") == "tool_result" for b in content
         )
+    # Unknown shape — be conservative, treat as prompt.
     return True
 
 
@@ -255,9 +328,13 @@ def _summarize_assistant_group(
 ) -> tuple[str, list[str], int]:
     """Compute (spoken_text, tool_skeleton, text_segment_count) for a display group.
 
-    Must mirror ``parser.parse_assistant_structure``: text blocks collapse into
-    one segment, tool_use becomes a skeleton entry, tool_result/thinking are invisible.
+    Mirrors what ``parser.parse_assistant_structure`` would produce when
+    re-parsing the rendered version of this group: consecutive text
+    blocks across assistant messages collapse into one text segment;
+    tool_use blocks become skeleton entries; tool_result messages and
+    thinking blocks are invisible.
     """
+    # See ``diff_and_fork`` for why the parser-type imports are deferred.
     from beaver_gateway.frontends.markdown.parser import TextSegment, ToolSegment
 
     segments: list[TextSegment | ToolSegment] = []
@@ -274,6 +351,7 @@ def _summarize_assistant_group(
 
     for msg in group:
         if msg["role"] == "user":
+            # tool_result message — boundary for text but emits no segment.
             _flush()
             continue
         content = msg.get("content")
@@ -290,6 +368,7 @@ def _summarize_assistant_group(
             elif btype == "tool_use":
                 _flush()
                 segments.append(ToolSegment(name=str(blk.get("name", ""))))
+            # thinking: skip silently
     _flush()
     spoken_chunks = [s.text for s in segments if isinstance(s, TextSegment)]
     spoken = "\n\n".join(c for c in spoken_chunks if c).strip()
@@ -298,14 +377,30 @@ def _summarize_assistant_group(
     return spoken, skeleton, text_count
 
 
+# ---- the core algorithm -------------------------------------------------
+
+
 def diff_and_fork(
     *, stored: list[dict[str, Any]], incoming: list[ParsedTurn]
 ) -> ForkOutcome:
     """Align the incoming parsed file against stored history.
 
-    ``incoming`` must end with a user turn (the new prompt); raises otherwise.
-    Segment-class imports below are deferred to avoid a cycle with ``parser``.
+    ``stored`` is the raw Anthropic-shape message list from the DB
+    (one entry per ``ConversationMessage`` row). ``incoming`` is the
+    user-visible turn list from the markdown parser. The last
+    ``incoming`` entry must be a user turn — that's the new prompt
+    triggering this request.
+
+    Returns a :class:`ForkOutcome` whose ``messages`` is what the
+    backend should run on and whose ``persist_messages`` is the
+    canonical history to store in the DB once the backend's
+    synthesized cycle is appended.
     """
+    # ``parser`` lives under ``frontends/markdown/`` whose ``__init__``
+    # eagerly loads ``frontend.py``, which in turn imports this module
+    # — pulling the parser at module-import time creates a cycle. The
+    # helpers below import the segment classes lazily inside their own
+    # function bodies to break it.
     if not incoming or incoming[-1].role != "user":
         msg = (
             "diff_and_fork expects incoming to end with a user turn "
@@ -322,10 +417,16 @@ def diff_and_fork(
 
     if divergence is None and len(prior_incoming) < len(stored_groups):
         if _file_lags_store(stored_groups, len(prior_incoming), new_user_turn):
+            # Not a deletion — the file simply never received turns we
+            # already ran. Adopt the stored tail verbatim so history stays
+            # structured and its fingerprint still matches the live
+            # session's. See ``_file_lags_store`` for why this matters.
             spliced_groups.extend(
                 list(g.messages) for g in stored_groups[len(prior_incoming) :]
             )
         else:
+            # Incoming truncated stored (user deleted some past turns).
+            # Truncate stored to match.
             divergence = len(prior_incoming)
 
     backend_msgs, persist_msgs = _assemble_tail(
@@ -347,9 +448,26 @@ def _file_lags_store(
 ) -> bool:
     """Is the shorter incoming file a stale view rather than a deletion?
 
-    Tell: if the DB already holds the submitted prompt at the position the
-    file stops at, it's a lagging render, not a deletion — misjudging this
-    forks history, breaks the fingerprint, and respawns the backend session.
+    A file with fewer display turns than the DB has two possible causes,
+    and they need opposite handling:
+
+    * the user deleted trailing turns — we should truncate to match;
+    * the turn ran, was persisted, but its reply never made it back into
+      the ``.md`` (the render lost a race with the user's next prompt, or
+      the reply rendered to nothing visible). The file is simply behind.
+
+    The tell is the prompt the user is submitting right now: if the DB
+    already holds it at exactly the position the file stops at, this is a
+    re-submission of a turn we've already run, not a deletion. Nobody
+    deletes a turn and immediately retypes it verbatim.
+
+    Getting this wrong is expensive and self-sustaining. Forking here
+    flattens every post-divergence turn into plain text (losing tool_use /
+    tool_result structure), persists that flattened history, and changes
+    the conversation fingerprint — so the backend's session pool misses,
+    spawns a fresh ``claude``, reseeds it from a multi-MB JSONL, and
+    strands the previous process. The file still lags afterwards, so the
+    next turn does it again.
     """
     if prior_len >= len(stored_groups):
         return False
@@ -362,8 +480,12 @@ def _walk_prefix(
 ) -> tuple[list[list[dict[str, Any]]], int | None, bool]:
     """Walk incoming vs stored side-by-side until first divergence.
 
-    An empty incoming skeleton (no tool callouts rendered) means prose
-    alone decides the match; a mismatch forces a fresh backend session.
+    Returns the spliced/matched group list (one entry per matched
+    display turn, each carrying the raw messages we'll feed to the
+    backend for that turn), the divergence index (``None`` if all
+    of ``prior_incoming`` matched) and whether any assistant prose
+    was spliced in from the file - a rewritten reply keeps the
+    structure but must not resume the session that said otherwise.
     """
     from beaver_gateway.frontends.markdown.parser import TextSegment, ToolSegment
 
@@ -384,6 +506,8 @@ def _walk_prefix(
             s.name for s in inc.structure if isinstance(s, ToolSegment)
         )
         inc_text_count = sum(1 for s in inc.structure if isinstance(s, TextSegment))
+        # Files rendered without tool callouts (§3.10) carry no skeleton:
+        # prose alone decides whether the turn matched.
         if inc_skeleton and inc_skeleton != st.skeleton:
             return spliced_groups, i, edited
         if inc.text == st.spoken_text:
@@ -452,10 +576,21 @@ def _splice_in_place(
 ) -> list[dict[str, Any]] | None:
     """Copy the stored messages, swapping only their text block contents.
 
-    Preserves message/thinking-block boundaries that a rebuild would lose
-    (and that a cache-hit continuation depends on); returns ``None`` if
-    text blocks aren't 1:1 with incoming segments, so the caller rebuilds instead.
+    Rebuilding a turn from the file loses everything the markdown never
+    carried — thinking blocks, and the message boundaries claude chose.
+    Both matter: an ``assistant[thinking] + assistant[text]`` pair (what
+    claude emits for a reasoning turn) collapses into a single message,
+    so the history is one message shorter than the one the backend
+    pooled its live session under, and the next turn misses the cache
+    and respawns. Substituting in place keeps the message count and the
+    invisible blocks exactly as stored.
+
+    Returns ``None`` when stored text blocks and incoming text segments
+    aren't one-to-one — consecutive text blocks merge into a single
+    rendered segment, so there'd be no way to know how to split the
+    edited prose back apart. The caller then rebuilds instead.
     """
+    # See ``diff_and_fork`` for why the parser-type import is deferred.
     from beaver_gateway.frontends.markdown.parser import TextSegment
 
     new_texts = [
@@ -491,9 +626,18 @@ def _splice_by_rebuild(
 ) -> list[dict[str, Any]] | None:
     """Rebuild an assistant display turn with new text + stored tool_use blocks.
 
-    Matches tool_use blocks to incoming ``ToolSegment``s by position; returns
-    ``None`` (caller forks) if a matching stored ``tool_result`` is missing.
+    Walks the incoming structure; for each ``TextSegment`` emits a
+    text block into the current assistant message; for each
+    ``ToolSegment`` consumes the next stored ``tool_use`` block (by
+    position), closes the current assistant message, emits the
+    matching ``tool_result`` user message, and opens a new assistant
+    message. Final ``TextSegment`` closes the last assistant message.
+
+    Returns ``None`` if we can't find a matching tool_result for some
+    tool_use (stored history is malformed) — caller falls back to
+    fork.
     """
+    # See ``diff_and_fork`` for why this import is deferred.
     from beaver_gateway.frontends.markdown.parser import TextSegment
 
     tool_uses, tool_results_by_id = _harvest_tool_blocks(stored_group)
@@ -520,6 +664,8 @@ def _splice_by_rebuild(
     if current_asst:
         spliced.append({"role": "assistant", "content": current_asst})
     elif not spliced:
+        # Defensive: assistant turn with no text and no tools makes no
+        # sense; caller will treat as fork.
         return None
     return spliced
 

src/beaver_gateway/core/distill.py

@@ -1,8 +1,13 @@
-"""Closing a deep chat: the digest, the index, the file line cap.
+"""Closing a deep chat: the digest, the index, the file line cap (§6.4, §8.4).
 
-``Distiller`` says where digests and the index live; the gateway checks the
-digest's frontmatter and caps the merge text. ``LineCap`` is the same idea
-for a file a job rewrites: too long, and it is bounced back unchanged.
+The gateway knows no path by itself (§0.8): ``Distiller`` from ``config.py``
+names the agent, says where digests land and where the index lives, and
+the distiller writes the file on its own. What the gateway does is check that a file
+with a valid frontmatter appeared under ``Distiller.dir`` during the fork
+turn, put one line into the index, and cap the merge text at
+``SUMMARY_LINES``. ``LineCap`` is the same idea for a file a job rewrites
+(``состояние.md``): a result longer than the cap is bounced - the file
+goes back to what it was and the job is told to shorten.
 """
 
 from __future__ import annotations
@@ -34,7 +39,7 @@ __all__ = [
 ]
 
 SUMMARY_LINES = 5
-"""A merge into the master is at most this many lines."""
+"""A merge into the master is at most this many lines (§6.4)."""
 
 
 @dataclass(frozen=True, slots=True)
@@ -44,10 +49,10 @@ class Distiller:
     agent: str
     dir: Path
     index: Path
-    type: str = "digest"
+    type: str = "выжимка"
     """Value the ``type`` frontmatter key must carry."""
 
-    index_header: str = "# index\n\none line per digest: chat → its digest.\n"
+    index_header: str = "# индекс\n\nстрока на выжимку: чат → его выжимка.\n"  # noqa: RUF001
 
 
 @dataclass(frozen=True, slots=True)
@@ -145,13 +150,13 @@ def check_digest(path: Path, digests: Distiller) -> Digest | str:
     try:
         post = frontmatter.load(str(path))
     except (OSError, ValueError) as exc:
-        return f"unreadable: {exc}"
+        return f"не читается: {exc}"
     meta = post.metadata
     if meta.get("type") != digests.type:
-        return f"`type` must be `{digests.type}`, not {meta.get('type')!r}"
+        return f"`type` должен быть `{digests.type}`, не {meta.get('type')!r}"
     source = meta.get("source")
     if not isinstance(source, str) or not source.strip():
-        return "`source` is empty"
+        return "`source` пустой"
     when = meta.get("date")
     if isinstance(when, datetime):
         when = when.date()
@@ -159,11 +164,11 @@ def check_digest(path: Path, digests: Distiller) -> Digest | str:
         try:
             when = date.fromisoformat(when.strip())
         except ValueError:
-            return f"`date` is not a date: {when!r}"
+            return f"`date` не дата: {when!r}"
     if not isinstance(when, date):
-        return "`date` is missing"
+        return "`date` отсутствует"
     if not post.content.strip():
-        return "empty body"
+        return "тело пустое"
     return Digest(path=path, source=source.strip(), date=when)
 
 

src/beaver_gateway/core/envelope.py

@@ -1,32 +1,42 @@
-"""The envelope: a background block under the user's text - clock, changes, recall."""
+"""The envelope (§3.3): a background block after the user's text.
+
+Assembled when the turn starts, never when the message is queued: the
+time and what changed in the vault since the last envelope (added lines
+for the ``full`` files, names and counts for the rest). Ceilings keep it
+a signal, not a document; the injects that ride along are bundled below
+it by the queue, with their own header.
+"""
 
 from __future__ import annotations
 
 import logging
-from dataclasses import dataclass, field
+from dataclasses import dataclass
 from datetime import UTC, datetime
 from typing import TYPE_CHECKING
 from zoneinfo import ZoneInfo
 
-from beaver_gateway.conversations.texts import EnvelopeTexts
-
 if TYPE_CHECKING:
     from collections.abc import Callable, Sequence
 
-    from beaver_gateway.vault.watch import Change, VaultWatch
+    from beaver_gateway.core.watch import Change, VaultWatch
 
 __all__ = ["Envelope", "RecallContext", "render"]
 
 _log = logging.getLogger(__name__)
 
+HEADER = (
+    "[конверт - фоновый сигнал, не обращение; "
+    "реагируй, только если относится к вопросу]"
+)
+
 
 @dataclass(frozen=True, slots=True)
 class RecallContext:
+    """What the setup's ``recall`` hook sees: the user's text and where it landed."""
+
     text: str
     kind: str
     now: datetime
-    agent: str = ""
-    """Agent whose turn it is; a setup with several tells them apart by it."""
 
 
 @dataclass(slots=True)
@@ -38,16 +48,13 @@ class Envelope:
     names_only_within: float = 600.0
     last_at: datetime | None = None
     recall: Callable[[RecallContext], str | None] | None = None
-    """Setup-side lookup on the user's text; its lines go under the change block."""
-    texts: EnvelopeTexts = field(default_factory=EnvelopeTexts)
+    """Setup-side lookup run on the user's text at turn start: pointers into
+    the vault (a person's card, the agent's own notes, due dates) that the
+    gateway cannot know the paths of. Its lines go under the vault block;
+    a failure is logged and the envelope goes out without them."""
 
     def build(
-        self,
-        *,
-        now: datetime | None = None,
-        text: str = "",
-        kind: str = "master",
-        agent: str = "",
+        self, *, now: datetime | None = None, text: str = "", kind: str = "master"
     ) -> str:
         now = now or datetime.now(UTC)
         changes = self.watch.take() if self.watch is not None else []
@@ -63,29 +70,23 @@ class Envelope:
             names_only=names_only,
             max_lines=self.max_lines,
             per_file=self.per_file,
-            texts=self.texts,
         )
         self.last_at = now
-        block = self.recall_block(text=text, kind=kind, now=now, agent=agent)
+        block = self.recall_block(text=text, kind=kind, now=now)
         return f"{out}\n{block}" if block else out
 
     def recall_only(
-        self, *, text: str, kind: str, now: datetime | None = None, agent: str = ""
+        self, *, text: str, kind: str, now: datetime | None = None
     ) -> str | None:
-        block = self.recall_block(
-            text=text, kind=kind, now=now or datetime.now(UTC), agent=agent
-        )
-        return f"{self.texts.header}\n{block}" if block else None
+        """The recall lines under the header, without the vault diff (branches)."""
+        block = self.recall_block(text=text, kind=kind, now=now or datetime.now(UTC))
+        return f"{HEADER}\n{block}" if block else None
 
-    def recall_block(
-        self, *, text: str, kind: str, now: datetime, agent: str = ""
-    ) -> str | None:
+    def recall_block(self, *, text: str, kind: str, now: datetime) -> str | None:
         if self.recall is None or not text.strip():
             return None
         try:
-            block = self.recall(
-                RecallContext(text=text, kind=kind, now=now, agent=agent)
-            )
+            block = self.recall(RecallContext(text=text, kind=kind, now=now))
         except Exception:  # noqa: BLE001
             _log.exception("recall hook failed")
             return None
@@ -101,52 +102,38 @@ def render(
     names_only: bool,
     max_lines: int = 120,
     per_file: int = 30,
-    texts: EnvelopeTexts | None = None,
 ) -> str:
-    texts = texts or EnvelopeTexts()
     zone = ZoneInfo(tz)
     stamp = now.astimezone(zone)
-    lines = [
-        texts.header,
-        texts.time.format(stamp=f"{stamp:%Y-%m-%d %H:%M}", zone=_zone_label(tz)),
-    ]
+    lines = [HEADER, f"время: {stamp:%Y-%m-%d %H:%M} ({_zone_label(tz)})"]
     ordered = sorted(changes, key=lambda c: (not c.full, c.path))
     since_label = (
-        texts.since.format(time=f"{since.astimezone(zone):%H:%M}")
-        if since is not None
-        else texts.since_start
+        f"с {since.astimezone(zone):%H:%M}" if since is not None else "со старта"  # noqa: RUF001
     )
     if ordered:
         names = ", ".join(f"{c.path} (+{c.added_count})" for c in ordered)
-        lines.append(texts.changed.format(since=since_label, names=names))
+        lines.append(f"vault, изменено {since_label}: {names}")
     if not names_only:
-        _append_diffs(
-            lines, ordered, max_lines=max_lines, per_file=per_file, texts=texts
-        )
+        _append_diffs(lines, ordered, max_lines=max_lines, per_file=per_file)
     return "\n".join(lines[:max_lines])
 
 
 def _append_diffs(
-    lines: list[str],
-    changes: Sequence[Change],
-    *,
-    max_lines: int,
-    per_file: int,
-    texts: EnvelopeTexts,
+    lines: list[str], changes: Sequence[Change], *, max_lines: int, per_file: int
 ) -> None:
     budget = max_lines - len(lines) - 1
     for change in changes:
         if not change.full or not change.added:
             continue
         if budget < 3:
-            lines.append(texts.truncated)
+            lines.append("… (потолок конверта)")
             return
         shown = change.added[: min(per_file, budget - 2)]
-        lines.append(texts.file_header.format(path=change.path))
+        lines.append(f"--- {change.path}, только добавленное ---")
         lines.extend(f"+ {line}" for line in shown)
         budget -= 1 + len(shown)
         if len(change.added) > len(shown):
-            lines.append(texts.more_lines.format(count=len(change.added) - len(shown)))
+            lines.append(f"+ … ещё {len(change.added) - len(shown)}")
             budget -= 1
 
 

src/beaver_gateway/core/events.py

@@ -1,7 +1,11 @@
-"""Streaming-event protocol shared by every backend adapter.
+"""Unified streaming-event protocol shared by every backend adapter.
 
-Aliases over :mod:`anthropic.types` plus ``build_*`` helpers that keep
-verbose constructors out of adapters.
+We piggyback on :mod:`anthropic.types` rather than reinvent the wire format:
+``AnthropicMessagesFrontend`` ultimately serializes whatever a backend yields
+straight to SSE via :py:meth:`pydantic.BaseModel.model_dump_json`, so events
+must be valid Anthropic ``message_stream`` records. The aliases below give
+the rest of the codebase one import path; the ``build_*`` helpers keep the
+verbose constructors out of every adapter.
 """
 
 from __future__ import annotations

src/beaver_gateway/core/gateway_tools.py

@@ -1,7 +1,8 @@
-"""In-process MCP server with the gateway's own tools.
+"""In-process MCP server with the gateway's own tools (§3.1, §3.2).
 
 One server per live session so every tool knows which conversation is
-calling. Which names a session gets comes from ``ClaudeAgent.gateway_tools``.
+calling; ``alwaysLoad`` keeps the tools out of tool search. Which names a
+session gets comes from ``ClaudeAgent.gateway_tools``.
 """
 
 from __future__ import annotations
@@ -12,10 +13,9 @@ from typing import TYPE_CHECKING, Any, cast
 
 from claude_agent_sdk import create_sdk_mcp_server, tool
 
-from beaver_gateway.conversations.injects import URGENCY
-from beaver_gateway.conversations.kinds import as_kind
-from beaver_gateway.frontends.files import AUTO, METHODS, SendFileError
-from beaver_gateway.security.redact import redact_data
+from beaver_gateway.core.injects import URGENCY
+from beaver_gateway.core.kinds import as_kind
+from beaver_gateway.core.redact import redact_data
 
 URGENCY_HELP = (
     "normal waits for the hourly batch or rides with the next turn, wake "
@@ -31,11 +31,11 @@ if TYPE_CHECKING:
 
     from claude_agent_sdk import McpSdkServerConfig, SdkMcpTool
 
-    from beaver_gateway.conversations.service import Conversations
+    from beaver_gateway.core.conversations import Conversations
 
 __all__ = ["SERVER_NAME", "TOOL_NAMES", "build_tool_server"]
 
-_log = logging.getLogger("beaver_gateway.conversations.tools")
+_log = logging.getLogger("beaver_gateway.core.gateway_tools")
 
 SERVER_NAME = "gateway"
 SAY_IN_USER_TURN = (
@@ -43,23 +43,7 @@ SAY_IN_USER_TURN = (
     "reply text reaches them by itself - put what you wanted to say into the "
     "reply instead of repeating it here"
 )
-TOOL_NAMES = (
-    "read_conversation",
-    "spawn",
-    "say",
-    "schedule",
-    "inject",
-    "close_chat",
-    "send_file",
-)
-SEND_FILE_HELP = (
-    "Send a file from disk to the human, into this conversation's window (the "
-    "master's if it has none): {note}. `method` is how Telegram renders it; "
-    "`auto` reads the extension - image is a photo, mp3/flac/m4a audio, ogg "
-    "voice, mp4/mov video, anything else a document. Photo and video are "
-    "recompressed, so pass `document` when the bytes must arrive untouched. "
-    "`caption` takes the same markdown as your replies."
-)
+TOOL_NAMES = ("read_conversation", "spawn", "say", "schedule", "inject", "close_chat")
 
 
 def build_tool_server(
@@ -84,10 +68,11 @@ def build_tool_server(
 def _redacting(spec: SdkMcpTool[Any]) -> SdkMcpTool[Any]:
     """Put a tool's result through the same mask as every other MCP.
 
-    These tools are mounted in-process, bypassing the ``FastMCP`` middleware
-    in :mod:`beaver_gateway.mcp.redacting`, so the filter is attached here
-    instead. ``read_conversation`` earns it: a replayed transcript can still
-    hold a credential written before redaction existed.
+    These tools are mounted in-process by the SDK, so they bypass the
+    ``FastMCP`` middleware in :mod:`beaver_gateway.mcp.redacting` and
+    need the filter attached here instead. ``read_conversation`` is the
+    one that earns it: it replays a transcript, and a transcript written
+    before any of this existed can still hold a credential.
     """
     inner = spec.handler
 
@@ -245,37 +230,10 @@ def _tools(conversations: Conversations, key: str) -> list[SdkMcpTool[Any]]:
             conv,
             str(args["text"]),
             urgency=cast("Any", args.get("urgency") or "normal"),
-            origin="agent",
+            origin="агент",
         )
         return _text(f"queued #{item.id}")
 
-    @tool(
-        "send_file",
-        SEND_FILE_HELP.format(note=conversations.file_note()),
-        {
-            "type": "object",
-            "properties": {
-                "path": {"type": "string", "description": "absolute path on disk"},
-                "caption": {"type": "string"},
-                "method": {"type": "string", "enum": [AUTO, *METHODS], "default": AUTO},
-            },
-            "required": ["path"],
-        },
-    )
-    async def send_file(args: dict[str, Any]) -> dict[str, Any]:
-        conv = await current()
-        try:
-            return _text(
-                await conversations.send_file(
-                    conv,
-                    str(args["path"]),
-                    caption=str(args.get("caption") or ""),
-                    method=str(args.get("method") or AUTO),
-                )
-            )
-        except SendFileError as exc:
-            return _error(str(exc))
-
     @tool(
         "close_chat",
         "Close this deep chat once the current reply is finished: the "
@@ -292,7 +250,7 @@ def _tools(conversations: Conversations, key: str) -> list[SdkMcpTool[Any]]:
             return _error(str(exc))
         return _text("ok: the chat closes after this reply")
 
-    return [read_conversation, spawn, say, schedule, inject, send_file, close_chat]
+    return [read_conversation, spawn, say, schedule, inject, close_chat]
 
 
 def _text(text: str) -> dict[str, Any]:

src/beaver_gateway/core/injects.py

@@ -1,15 +1,20 @@
-"""Persisted per-conversation queue: ``urgent > user > wake > normal``.
+"""Persisted per-conversation queue, ``urgent > user > wake > normal`` (§3.4).
 
-conversations/service.py runs one worker per conversation over these rows.
-A row still ``running`` when the gateway starts was cut by a restart and is
-flagged ``interrupted``, never re-run.
+One ``ClaudeSDKClient`` runs one turn at a time, so ordering has to happen
+before the client: the rows here are the queue, ``core/conversations``
+runs one worker per conversation over them. ``urgent`` cuts a running
+turn, ``user`` is the human, ``wake`` starts a turn as soon as the
+conversation is idle and takes the queued normals with it, ``normal``
+waits for the batching window or rides with the next turn. A row that is
+still ``running`` when the gateway starts was cut by a restart; it is
+flagged ``interrupted`` and never re-run.
 """
 
 from __future__ import annotations
 
 from dataclasses import dataclass
 from datetime import UTC, datetime
-from typing import TYPE_CHECKING, Any, Literal, cast
+from typing import TYPE_CHECKING, Literal, cast
 
 from sqlalchemy import func
 from sqlmodel import col, select
@@ -37,8 +42,8 @@ URGENCY: tuple[Priority, ...] = ("normal", "wake", "urgent")
 """What the API and the tools accept for ``urgency``: every priority but ``user``."""
 
 INTERRUPTED_TURN = (
-    "[this inject cut the previous turn: the «Request interrupted» above is "
-    "an interruption, not a refused tool call]"
+    "[этот инжект прервал предыдущий тёрн: «Request interrupted» выше - "
+    "прерывание, не отказ от тулзы]"
 )
 
 
@@ -53,7 +58,9 @@ class InjectContext:
 
 def inject_header(ctx: InjectContext) -> str:
     """Default framing; a setup overrides it via ``ConversationTexts.inject_header``."""
-    head = f"[inject: {ctx.origin} - not the user, no reply needed]"
+    head = (
+        f"[инжект: {ctx.origin} - это не Бобёр, отвечать не нужно, голос не обязателен]"
+    )
     return f"{head}\n{INTERRUPTED_TURN}" if ctx.interrupted_turn else head
 
 
@@ -70,23 +77,13 @@ class InjectQueue:
         self._db = db
 
     async def push(
-        self,
-        *,
-        conversation_id: int,
-        priority: Priority,
-        origin: str,
-        text: str,
-        attachments: list[dict[str, Any]] | None = None,
+        self, *, conversation_id: int, priority: Priority, origin: str, text: str
     ) -> InjectQueueItem:
         if priority not in PRIORITY_RANK:
             msg = f"unknown priority {priority!r}"
             raise ValueError(msg)
         row = InjectQueueItem(
-            conversation_id=conversation_id,
-            priority=priority,
-            origin=origin,
-            text=text,
-            attachments=attachments or None,
+            conversation_id=conversation_id, priority=priority, origin=origin, text=text
         )
         async with self._db.session() as session:
             session.add(row)

src/beaver_gateway/core/kinds.py

@@ -1,4 +1,4 @@
-"""Conversation kinds as one closed type for agents, frontends, service."""
+"""Conversation kinds (§3.1) as one closed type for agents, frontends, service."""
 
 from __future__ import annotations
 

src/beaver_gateway/core/policy.py

@@ -1,8 +1,18 @@
-"""PreToolUse policy - the boundary without permission prompts.
+"""PreToolUse policy - the boundary without permission prompts (§3.7).
 
-A rule is a callable ``(ToolCall) -> Deny | None`` declared per agent
-(``ClaudeAgent.policy``); the first :class:`Deny` becomes the tool result
-the model reads back. A rule that raises is a deny too: fails closed.
+``bypassPermissions`` everywhere; what a model may do is decided by
+mounts, ``disallowed_tools`` and the rules here. A rule is a callable
+``(ToolCall) -> Deny | None`` declared per agent (``ClaudeAgent.policy``);
+the SDK backend registers one in-process ``PreToolUse`` hook that runs
+the rules in order and turns the first :class:`Deny` into a hook deny
+whose reason the model reads as the tool result. Rules never see
+secrets and never prompt - they only say no, with a reason.
+
+A rule that raises is a deny too: the boundary fails closed, the
+traceback lands in the log.
+
+Every tool call - allowed or denied - is reported to the audit sink the
+backend was given, so the admin audit page shows what the model touched.
 """
 
 from __future__ import annotations
@@ -80,14 +90,11 @@ class ToolCall:
         return None
 
     def resolve(self, raw: str) -> Path:
-        """Absolute path for ``raw``, normalised lexically.
-
-        Not via ``Path.resolve``, which would follow symlinks on the host,
-        not the model's view.
-        """
         path = Path(raw).expanduser()
         if not path.is_absolute():
             path = self.cwd / path
+        # ``resolve`` would follow symlinks on the gateway host, which may
+        # not be the model's view; normalise lexically instead.
         return Path(*_normalize(path.parts))
 
 
@@ -129,7 +136,7 @@ async def evaluate(
             verdict = rule(call)
             if inspect.isawaitable(verdict):
                 verdict = await verdict
-        except Exception:  # noqa: BLE001
+        except Exception:  # noqa: BLE001 - a broken rule must fail closed
             name = getattr(rule, "__name__", repr(rule))
             _log.exception("policy rule %s failed on %s", name, call.tool)
             return Deny(reason=f"policy rule {name} failed; the call is refused")

src/beaver_gateway/core/prompt.py

@@ -1,7 +1,12 @@
 """System prompt assembly from a list of source files.
 
-:func:`assemble` concatenates an agent's granules in order. A source is a
-path, or a ``(tag, path)`` pair whose content is wrapped in ``<tag>...</tag>``.
+The gateway holds no prompt text: an agent names its granules (paths from
+``config.py``) and :func:`assemble` concatenates them in that order, so the
+result is byte-for-byte identical for every session of the same agent as
+long as the files are. A source is a path, or a ``(tag, path)`` pair whose
+content is wrapped in ``<tag>...</tag>`` - the markup lives here, the vault
+keeps plain markdown. Each granule's hash is logged at assembly so a
+drifted prompt can be traced to the file that changed.
 """
 
 from __future__ import annotations
@@ -16,13 +21,12 @@ if TYPE_CHECKING:
 
 __all__ = ["PromptSource", "assemble"]
 
-_log = logging.getLogger("beaver_gateway.agents.prompts")
+_log = logging.getLogger("beaver_gateway.core.prompt")
 
 PromptSource = str | Path | tuple[str, str | Path]
 
 
 def assemble(sources: Iterable[PromptSource]) -> str:
-    """Concatenate ``sources`` in order into one system prompt."""
     parts: list[str] = []
     for source in sources:
         tag, raw = source if isinstance(source, tuple) else (None, source)

src/beaver_gateway/core/redact.py

@@ -1,7 +1,33 @@
 """Recognise credentials in free text and mask them.
 
-Used to filter log output and to redact what MCP tools hand back to
-the model (see :mod:`beaver_gateway.mcp.redacting`).
+``docker logs`` is not a vault: the stack's stdout lands in an unrotated
+json file on the host, and the model itself can page through it (the
+komodo tool has ``logs`` / ``search_logs``). Nobody has to write a
+careless log call for a secret to end up there — two habits do it on
+their own:
+
+* ``httpx`` logs ``HTTP Request: POST <full url>`` at ``INFO``, so every
+  request to an upstream that keeps its credential *in* the URL prints
+  the credential once per call;
+* a transport error carries that same URL through the traceback, which
+  ``_log.exception`` writes out in full.
+
+Two of our upstreams are exactly that shape: a Google Calendar
+``.../private-<token>/basic.ics`` feed and a USOS ``?key=<token>`` ical
+feed, both handed to the calendar MCP as query parameters.
+
+So: silence the HTTP clients' per-request chatter (nothing here reads
+it), and run one redaction pass over every formatted record as a second
+line of defence.
+
+The same pass guards the wider channel — what an MCP tool hands back to
+the model (see :mod:`beaver_gateway.mcp.redacting`). :func:`redact`
+recognises three things: literal values of the secret-looking
+environment variables this process was started with, credentials that
+live in URLs and headers, and ``NAME: value`` assignments whose *name*
+says the value is a credential. The last one is what covers a config
+dump of a stack whose secrets this process never held — and it keeps
+the name, masking only the value, so the dump still says what is set.
 """
 
 from __future__ import annotations
@@ -17,29 +43,39 @@ if TYPE_CHECKING:
 
 MASK = "<…>"
 
+# Loggers that print request URLs at INFO. We keep their warnings.
 CHATTY: tuple[str, ...] = ("httpx", "httpcore", "aiohttp.client", "urllib3")
-"""Loggers that print request URLs at ``INFO``; only their warnings are kept."""
 
+# Loggers that bring their own handlers (uvicorn re-runs ``dictConfig``
+# when a server starts), so wrapping the root formatter misses them.
+# ``dictConfig`` drops a logger's handlers but keeps its filters.
 OWN_HANDLERS: tuple[str, ...] = ("uvicorn", "uvicorn.access", "uvicorn.error")
-"""Loggers whose own handlers survive ``dictConfig``, unreached by the
-root formatter."""
 
+# Names whose *value* is a credential — matched as a substring, so
+# ``FIREFLY_PAT`` and ``T3_MAC_TOKEN`` both qualify. Used twice: to pick
+# which environment variables contribute literal values, and to decide
+# whether a ``NAME: value`` line in a config dump should keep its value.
+# ``MCP`` is in here because every MCP URL in this stack carries its
+# credential in the query string.
 _SECRET_NAME = re.compile(
     r"TOKEN|SECRET|KEY|PASSWORD|PASS\b|PAT\b|BEARER|CREDENTIAL"
     r"|MCPS?\b|AUTH|PRIVATE|SESSION|DSN",
     re.IGNORECASE,
 )
-"""Substring match for names whose *value* is a credential."""
-
 _MIN_SECRET = 8
 
+# A query parameter may sit behind a plain ``?``/``&`` or behind their
+# percent-encoded twins when a whole URL is nested in another one.
 _LEFT = r"(?:(?<![A-Za-z0-9])|(?<=%26)|(?<=%3F))"
+# ...and runs until the next separator, encoded ``&`` included.
 _VALUE = r"(?:(?!%26)[^&\s\"'<>,;)\]}])+"
-"""Query-parameter boundaries: a value may follow a plain or
-percent-encoded ``?``/``&`` and runs to the next separator."""
 
+
+# Names worth masking where they sit in a dump next to their password,
+# but not worth hunting for as literals across every log line: a
+# username is short and ordinary, and ``CLAUDE_RUNNER_USER=beaver-runner``
+# turned into a global literal would mask half the entrypoint script.
 _SECRET_IN_DUMP = re.compile(r"USER\b|LOGIN\b", re.IGNORECASE)
-"""Masked only when paired with a password in a dump, not hunted as a global literal."""
 
 
 def is_secret_name(name: str) -> bool:
@@ -66,10 +102,13 @@ def _mask_named_value(match: re.Match[str]) -> str:
 type _Repl = str | Callable[[re.Match[str]], str]
 
 _RULES: tuple[tuple[re.Pattern[str], _Repl], ...] = (
+    # Google Calendar's secret path segment; the ``/basic.ics`` after it
+    # survives, encoded or not, so the line still says what it fetched.
     (
         re.compile(r"(private-)(?:(?!%2F)[A-Za-z0-9_%-]){8,}", re.IGNORECASE),
         r"\1" + MASK,
     ),
+    # ``key=…``, ``api_key=…``, ``token=…``, … in a query string.
     (
         re.compile(
             _LEFT
@@ -81,13 +120,22 @@ _RULES: tuple[tuple[re.Pattern[str], _Repl], ...] = (
         ),
         r"\1\2" + MASK,
     ),
+    # ``Authorization: Bearer …`` in a header dump; scheme goes too.
     (
         re.compile(
             r"(authorization[\"']?\s*[:=]\s*[\"']?)(?:\S+\s+)?\S+", re.IGNORECASE
         ),
         r"\1" + MASK,
     ),
+    # A bare ``Bearer <token>`` anywhere else.
     (re.compile(r"\b(bearer)\s+[A-Za-z0-9._~+/=-]{8,}", re.IGNORECASE), r"\1 " + MASK),
+    # ``NAME: value`` / ``NAME=value`` at the start of a line — the shape
+    # of a compose ``environment:`` block, an ``.env`` file, a printed
+    # settings object. Anchored to the line so that an ``actor=token:foo``
+    # in the middle of a log line doesn't swallow the rest of it, and
+    # ``=`` must be tight (``NAME=value``, the way env files write it) so
+    # that ``SESSION_SECRET = os.environ[…]`` in source the dispatcher is
+    # reading through t3code stays readable.
     (
         re.compile(
             r"""(?m)^(?P<lead>[ \t]*(?:-[ \t]+)?["']?)
@@ -98,6 +146,9 @@ _RULES: tuple[tuple[re.Pattern[str], _Repl], ...] = (
         ),
         _mask_named_value,
     ),
+    # The same at the start of a line in lower case, but only for names
+    # that are unambiguously a credential — ``_SECRET_NAME`` is too broad
+    # here, it would read the ``mcp:`` prefix of a log line as one.
     (
         re.compile(
             r"""(?m)^(?P<lead>[ \t]*(?:-[ \t]+)?["']?)
@@ -109,7 +160,12 @@ _RULES: tuple[tuple[re.Pattern[str], _Repl], ...] = (
         ),
         _mask_named_value,
     ),
+    # ``scheme://user:password@host`` — the credential a connection
+    # string carries. Host and database stay, so the line still locates
+    # the service it failed to reach.
     (re.compile(r"(://[^/\s:@]+:)[^@\s/]+(@)"), r"\1" + MASK + r"\2"),
+    # ``"name": "value"`` anywhere — the same idea for one-line JSON,
+    # where the quotes make the end of the value unambiguous.
     (
         re.compile(
             r'(?P<lead>")(?P<name>[^"\n]{1,64})(?P<sep>"\s*:\s*")'
@@ -118,10 +174,6 @@ _RULES: tuple[tuple[re.Pattern[str], _Repl], ...] = (
         _mask_named_value,
     ),
 )
-"""Redaction rules applied in order: calendar tokens, query-string secrets,
-``Authorization`` headers, bare bearer tokens, ``NAME: value``/``NAME=value``
-lines (upper- then lower-case names), connection-string passwords, and
-one-line JSON ``"name": "value"`` pairs."""
 
 _ENV_SECRETS: list[str] = []
 """Literal secret values to mask, longest first. Filled by :func:`install`."""
@@ -203,10 +255,9 @@ class RedactFilter(logging.Filter):
     """
 
     def filter(self, record: logging.LogRecord) -> bool:
-        """Let a record through unmasked if its template is malformed."""
         try:
             message = record.getMessage()
-        except (TypeError, ValueError):
+        except (TypeError, ValueError):  # a broken template is not ours to fix
             return True
         masked = redact(message)
         if masked != message:

src/beaver_gateway/core/registry.py

@@ -0,0 +1,117 @@
+"""Agent / MCP registries + the user-facing ``Gateway`` collector.
+
+The user's ``/config/config.py`` ends with::
+
+    gateway = Gateway(agents=[...], mcps=[...], frontends=[...])
+
+``cli.main`` picks that object up and builds the registries.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from collections.abc import Awaitable, Callable, Iterable, Iterator
+
+    from beaver_gateway.agents.base import BaseAgent
+    from beaver_gateway.core.conversations import ConversationTexts, UserSaid
+    from beaver_gateway.core.distill import Distiller
+    from beaver_gateway.core.envelope import RecallContext
+    from beaver_gateway.core.rotation import RotationPolicy
+    from beaver_gateway.core.scheduler import Budget, Job
+    from beaver_gateway.core.watch import VaultWatch
+    from beaver_gateway.frontends.base import Frontend
+    from beaver_gateway.mcp.types import McpServerT
+
+
+class AgentRegistry:
+    """Name → agent lookup with duplicate detection."""
+
+    def __init__(self, agents: Iterable[BaseAgent]) -> None:
+        self._agents: dict[str, BaseAgent] = {}
+        for a in agents:
+            if a.name in self._agents:
+                msg = f"duplicate agent name: {a.name!r}"
+                raise ValueError(msg)
+            self._agents[a.name] = a
+
+    def __getitem__(self, name: str) -> BaseAgent:
+        return self._agents[name]
+
+    def get(self, name: str) -> BaseAgent | None:
+        return self._agents.get(name)
+
+    def __iter__(self) -> Iterator[BaseAgent]:
+        return iter(self._agents.values())
+
+    def __len__(self) -> int:
+        return len(self._agents)
+
+    def __contains__(self, name: object) -> bool:
+        return name in self._agents
+
+
+class McpRegistry:
+    """Name → MCP server lookup with duplicate detection."""
+
+    def __init__(self, mcps: Iterable[McpServerT]) -> None:
+        self._mcps: dict[str, McpServerT] = {}
+        for m in mcps:
+            if m.name in self._mcps:
+                msg = f"duplicate mcp name: {m.name!r}"
+                raise ValueError(msg)
+            self._mcps[m.name] = m
+
+    def __getitem__(self, name: str) -> McpServerT:
+        return self._mcps[name]
+
+    def get(self, name: str) -> McpServerT | None:
+        return self._mcps.get(name)
+
+    def __iter__(self) -> Iterator[McpServerT]:
+        return iter(self._mcps.values())
+
+    def __len__(self) -> int:
+        return len(self._mcps)
+
+    def __contains__(self, name: object) -> bool:
+        return name in self._mcps
+
+
+@dataclass(slots=True)
+class Gateway:
+    """Top-level object the user assembles in ``/config/config.py``."""
+
+    agents: list[BaseAgent] = field(default_factory=list)
+    mcps: list[McpServerT] = field(default_factory=list)
+    frontends: list[Frontend] = field(default_factory=list)
+    texts: ConversationTexts | None = None
+    """Merge prompt and seed bodies for ``core/conversations`` (§8.2-8.3)."""
+    jobs: list[Job] = field(default_factory=list)
+    """Cron / webhook / event jobs for ``core/scheduler`` (§3.6, §4.5)."""
+    rotation: RotationPolicy | None = None
+    """When a master is rotated (§4.5); ``None`` keeps the defaults."""
+    watch: VaultWatch | None = None
+    """Vault watcher feeding the envelope (§3.5, §4.6); ``None`` = no vault block."""
+    recall: Callable[[RecallContext], str | None] | None = None
+    """Envelope lookup on the user's text: pointers into the vault (cards,
+    the agent's notes, due dates) the gateway knows no paths for (§3.3)."""
+    user_sink: Callable[[UserSaid], Awaitable[None] | None] | None = None
+    """Sees every user message as it enters a master or branch turn - the
+    setup's own grep-able log of what the user said, outside the transcript."""
+    budget: Budget | None = None
+    """Subscription window past which non-critical jobs wait (§4.5)."""
+    distiller: Distiller | None = None
+    """Who closes deep chats and where the digests and the index live (§8.4)."""
+    tz: str = "UTC"
+    """Local zone for the envelope clock and the rotation hour."""
+    host: str = "0.0.0.0"  # noqa: S104
+    port: int = 8000
+    """The one listener; every HTTP frontend is mounted under its ``path``."""
+    public_url: str | None = None
+    """Origin the reverse proxy shows the world (``https://b.example.com``).
+
+    Advertised endpoints and MCP discovery are built on it; ``None``
+    derives the origin from each request."""

src/beaver_gateway/core/rotation.py

@@ -1,7 +1,11 @@
-"""Master rotation: one logical master thread, many physical sessions.
+"""Master rotation (§4.5, §8.1, §8.3).
 
-When the policy says so, a new master takes over the window; the old one
-hands out, closes, and its branches and queued normal injects move over.
+One logical master thread, many physical sessions: when the policy says
+so, a new master is spawned and takes over the window atomically, the old
+one writes its handout as its last turn, closes, its finished branches get
+marked in their windows, its queued normal injects move over, and the new
+one receives "new day". Silence is measured by the user's messages only -
+injects never extend a day.
 """
 
 from __future__ import annotations
@@ -13,12 +17,12 @@ from typing import TYPE_CHECKING
 from zoneinfo import ZoneInfo
 
 if TYPE_CHECKING:
-    from beaver_gateway.conversations.service import Conversations
+    from beaver_gateway.core.conversations import Conversations
     from beaver_gateway.storage.models import Conversation
 
 __all__ = ["HandoutContext", "Rotation", "RotationPolicy"]
 
-_log = logging.getLogger("beaver_gateway.conversations.rotation")
+_log = logging.getLogger("beaver_gateway.core.rotation")
 
 
 @dataclass(frozen=True, slots=True)
@@ -42,11 +46,11 @@ class RotationPolicy:
         if now.astimezone(zone) < boundary:
             boundary -= timedelta(days=1)
         if created < boundary and silence > self.night_silence:
-            return "night"
+            return "ночь"
         if now - created > self.max_age and silence > self.short_silence:
-            return "age"
+            return "возраст"
         if context_tokens > self.max_context_tokens and silence > self.short_silence:
-            return "context"
+            return "транскрипт"
         return None
 
     def day_of(self, master: Conversation) -> date:

src/beaver_gateway/core/scheduler.py

@@ -1,7 +1,13 @@
-"""Jobs and deferred injects on pgqueuer.
+"""Jobs and deferred injects on pgqueuer (§3.6, §4.5).
 
-A job is a name, a handler and its triggers (cron, webhook, bus events); a
-handler only queues work, and the turn itself runs in the conversation's worker.
+A job is a name, a handler and its triggers: a cron expression, the
+webhook ``/hooks/<name>``, gateway bus events. The executor is pgqueuer on
+the gateway's own Postgres (a dedicated autocommit connection for
+LISTEN/NOTIFY), so cron ticks, webhook deliveries and the ``schedule``
+tool's one-off injects all live in one table and survive a restart. A
+handler only queues work for a conversation and returns; the turn itself
+runs in the conversation's worker. Non-critical jobs step aside while the
+subscription window is past its threshold.
 """
 
 from __future__ import annotations
@@ -27,8 +33,7 @@ from starlette.applications import Starlette
 from starlette.responses import JSONResponse
 from starlette.routing import Route
 
-from beaver_gateway.conversations.service import parse_at
-from beaver_gateway.jobs.job import Budget, Job, JobRun
+from beaver_gateway.core.conversations import parse_at
 from beaver_gateway.storage.models import JobRunRecord
 
 if TYPE_CHECKING:
@@ -39,14 +44,15 @@ if TYPE_CHECKING:
     from pgqueuer.ports.driver import Driver
     from starlette.requests import Request
 
-    from beaver_gateway.conversations.injects import Priority
-    from beaver_gateway.conversations.rotation import Rotation
-    from beaver_gateway.conversations.service import Conversations
+    from beaver_gateway.core.conversations import Conversations, DistillResult
+    from beaver_gateway.core.distill import LineCap
+    from beaver_gateway.core.injects import Priority
+    from beaver_gateway.core.rotation import Rotation
     from beaver_gateway.storage.models import Conversation
 
 __all__ = ["INJECT", "Budget", "Job", "JobRun", "LocalCron", "Scheduler", "next_run"]
 
-_log = logging.getLogger("beaver_gateway.jobs.scheduler")
+_log = logging.getLogger("beaver_gateway.core.scheduler")
 
 INJECT = "inject"
 RETRY = timedelta(minutes=15)
@@ -67,6 +73,111 @@ class LocalCron(ScheduleExecutor):
         return next_run(self.parameters.expression, self.tz)
 
 
+@dataclass(frozen=True, slots=True)
+class Job:
+    name: str
+    run: Callable[[JobRun], Awaitable[None]]
+    cron: str | None = None
+    webhook: bool = False
+    events: tuple[str, ...] = ()
+    critical: bool = True
+    dedupe: bool = True
+
+    @property
+    def entrypoint(self) -> str:
+        return f"job:{self.name}"
+
+
+@dataclass(frozen=True, slots=True)
+class Budget:
+    threshold: float = 0.7
+    tokens: int | None = None
+    window: timedelta = timedelta(hours=5)
+    limit_window: str = "five_hour"
+
+
+@dataclass(slots=True)
+class JobRun:
+    job: Job
+    trigger: str
+    payload: dict[str, Any]
+    scheduler: Scheduler
+
+    @property
+    def conversations(self) -> Conversations:
+        return self.scheduler.conversations
+
+    async def master(self) -> Conversation | None:
+        masters = await self.conversations.find(kind="master", status="open", limit=1)
+        return masters[0] if masters else None
+
+    async def inject_master(
+        self, text: str, *, urgency: Priority = "normal", origin: str | None = None
+    ) -> bool:
+        master = await self.master()
+        if master is None:
+            _log.error(
+                "job %s: no open master, inject lost: %s", self.job.name, text[:200]
+            )
+            return False
+        await self.conversations.inject(
+            master, text, urgency=urgency, origin=origin or self.job.name
+        )
+        return True
+
+    async def spawn_job(
+        self,
+        *,
+        agent: str,
+        text: str,
+        title: str | None = None,
+        line_cap: LineCap | None = None,
+    ) -> Conversation:
+        """A headless job turn; ``line_cap`` bounces a rewrite past the cap."""
+        return await self.conversations.spawn(
+            kind="job",
+            agent=agent,
+            seed="brief",
+            text=text,
+            title=title,
+            origin="job",
+            flags={"line_cap": line_cap.as_flags()} if line_cap else None,
+        )
+
+    async def close_idle(
+        self,
+        *,
+        kind: str = "deep",
+        days: int = 2,
+        limit: int = 3,
+        since: datetime | None = None,
+    ) -> list[DistillResult]:
+        """§4.5: close chats quiet for ``days``, at most ``limit`` per run."""
+        out: list[DistillResult] = []
+        for conv in await self.conversations.idle(
+            kind=kind, days=days, since=since, limit=limit
+        ):
+            try:
+                out.append(
+                    await self.conversations.distill(conv, reason=f"idle {days}d")
+                )
+            except Exception:  # noqa: BLE001
+                _log.exception("closing idle %s failed", conv.external_id)
+        return out
+
+    async def retry_in(self, delay: timedelta) -> None:
+        await self.scheduler.trigger(
+            self.job, self.payload, delay=delay, trigger=self.trigger
+        )
+
+    async def rotate(self) -> list[Conversation]:
+        rotation = self.scheduler.rotation
+        return await rotation.tick() if rotation is not None else []
+
+    def background(self, coro: Coroutine[Any, Any, Any]) -> None:
+        self.scheduler.background(coro)
+
+
 class Scheduler:
     def __init__(
         self,
@@ -101,6 +212,8 @@ class Scheduler:
     def job(self, name: str) -> Job | None:
         return self._jobs.get(name)
 
+    # ---- lifecycle -----------------------------------------------------
+
     async def start(self) -> None:
         if self._driver is not None:
             queries = Queries(self._driver)
@@ -161,6 +274,8 @@ class Scheduler:
                         self._dispatch(job, trigger="event", payload=dict(event))
                     )
 
+    # ---- dispatch ------------------------------------------------------
+
     async def _dispatch(
         self, job: Job, *, trigger: str, payload: dict[str, Any]
     ) -> None:
@@ -235,6 +350,8 @@ class Scheduler:
         )
         return int(ids[0]) if ids and ids[0] is not None else None
 
+    # ---- deferred injects ----------------------------------------------
+
     async def schedule(
         self,
         conv: Conversation,
@@ -307,6 +424,8 @@ class Scheduler:
             origin="schedule",
         )
 
+    # ---- budget --------------------------------------------------------
+
     async def utilization(self) -> float | None:
         now = datetime.now(UTC)
         values: list[float] = []
@@ -327,6 +446,8 @@ class Scheduler:
         utilization = await self.utilization()
         return utilization is not None and utilization > self.budget.threshold
 
+    # ---- introspection -------------------------------------------------
+
     async def snapshot(self) -> dict[str, Any]:
         crons: dict[str, PgSchedule] = {}
         queue: list[dict[str, Any]] = []
@@ -413,6 +534,8 @@ class Scheduler:
             "payload": row.payload,
         }
 
+    # ---- http ----------------------------------------------------------
+
     def app(self, authorize: Callable[[Request], Awaitable[Any]]) -> Starlette:
         async def hook(request: Request) -> JSONResponse:
             await authorize(request)
@@ -426,6 +549,8 @@ class Scheduler:
 
         return Starlette(routes=[Route("/{name}", hook, methods=["POST"])])
 
+    # ---- internals -----------------------------------------------------
+
     def background(self, coro: Coroutine[Any, Any, Any]) -> None:
         self._spawn(coro)
 

src/beaver_gateway/core/sessions.py

@@ -1,9 +1,13 @@
-"""Pool of live Agent SDK sessions across every Claude agent.
+"""Pool of live Agent SDK sessions across every Claude agent (§3.2).
 
-One :class:`Session` is one ``ClaudeSDKClient`` (one claude subprocess). The
-pool decides when a session is closed for idleness (TTL by conversation
-kind) and which one goes when memory runs out (RSS of the subprocess tree
-against the cgroup limit, or ``max_live`` with no limit).
+One :class:`Session` is one ``ClaudeSDKClient`` (one claude subprocess).
+The pool owns the two decisions the adapters used to make on their own:
+when a session is closed for idleness (TTL by conversation kind) and
+which one goes when memory runs out (measured RSS of the subprocess tree
+against the cgroup limit, ``max_live`` where there is no limit). Eviction
+only ever picks ``idle && !running_turn && !pending_question`` sessions
+that are neither pinned (the master) nor ``dirty`` (mirror gap not yet
+repaired); forks and jobs go first.
 """
 
 from __future__ import annotations
@@ -20,17 +24,11 @@ from typing import TYPE_CHECKING, Any, Protocol
 import psutil
 
 if TYPE_CHECKING:
-    from collections.abc import (
-        AsyncIterable,
-        AsyncIterator,
-        Callable,
-        Iterator,
-        Mapping,
-    )
+    from collections.abc import AsyncIterator, Iterator, Mapping
 
 __all__ = ["DEFAULT_TTL", "Session", "SessionClient", "SessionPool", "cgroup_limit"]
 
-_log = logging.getLogger("beaver_gateway.backends.sessions")
+_log = logging.getLogger("beaver_gateway.core.sessions")
 
 DEFAULT_TTL: Mapping[str, float | None] = {
     "master": None,
@@ -47,8 +45,8 @@ _RSS_HEADROOM = 0.8
 
 class SessionClient(Protocol):
     async def connect(self) -> None: ...
-    async def query(self, prompt: str | AsyncIterable[dict[str, Any]]) -> None: ...
-    def receive_messages(self) -> AsyncIterator[Any]: ...
+    async def query(self, prompt: str) -> None: ...
+    def receive_response(self) -> AsyncIterator[Any]: ...
     async def interrupt(self) -> None: ...
     async def disconnect(self) -> None: ...
 
@@ -71,21 +69,7 @@ class Session:
     created_at: float = field(default_factory=time.monotonic)
     turns: int = 0
     state: dict[str, Any] = field(default_factory=dict)
-    """Scratch for policy rules (``agents/policy.py``); dies with the process."""
-
-    reader: asyncio.Task[None] | None = None
-    """Reads the client for the session's whole life, not only during a turn."""
-    inbox: asyncio.Queue[Any] = field(default_factory=asyncio.Queue)
-    """Where the reader puts messages while a turn is attached."""
-    pending: list[Any] = field(default_factory=list)
-    """A turn the CLI started on its own (a background task reported back),
-    buffered until a consumer attaches."""
-    attached: bool = False
-    woken: bool = False
-    note: str | None = None
-    """Summary of the last task notification, the pending turn's prompt."""
-    observer: Callable[[Any], None] | None = None
-    """The last turn's observer; sees subagent traffic between turns."""
+    """Scratch for policy rules (``core/policy``); dies with the process."""
 
     @property
     def busy(self) -> bool:
@@ -93,10 +77,6 @@ class Session:
 
     @property
     def evictable(self) -> bool:
-        """Never evict a pinned (master) session or a ``dirty`` one.
-
-        ``dirty`` means a mirror gap not yet repaired.
-        """
         return not (self.pinned or self.dirty or self.busy or self.pending_question)
 
     @property
@@ -187,7 +167,6 @@ class SessionPool:
         return len(self._sessions) >= self._max_live
 
     def victims(self) -> list[Session]:
-        """Evictable sessions, forks/jobs first, then oldest idle."""
         candidates = [s for s in self._sessions.values() if s.evictable]
         candidates.sort(key=lambda s: (_EVICT_ORDER.get(s.kind, 1), s.last_used))
         return candidates
@@ -212,8 +191,6 @@ class SessionPool:
         if session is None:
             return
         _log.info("closing session %s (%s, %s)", session.session_id, session.kind, key)
-        if session.reader is not None and session.reader is not asyncio.current_task():
-            session.reader.cancel()
         try:
             await session.client.disconnect()
         except Exception:  # noqa: BLE001
@@ -260,7 +237,6 @@ class SessionPool:
                 "turns": s.turns,
                 "busy": s.busy,
                 "running_turn": s.running_turn,
-                "pending": len(s.pending),
                 "pending_question": s.pending_question,
                 "pinned": s.pinned,
                 "dirty": s.dirty,

src/beaver_gateway/core/transcript.py

@@ -1,7 +1,16 @@
 """Anthropic messages <-> Agent SDK transcript entries.
 
-:func:`build_entries` renders a message list as entries the Claude CLI
-itself writes; :func:`messages_from_entries` is the projection back.
+:func:`build_entries` renders a message list as the entries the Claude CLI
+itself writes (reference: ``t/spike_sdk/entries_reference.json``, CLI
+2.1.248 via ``import_session_to_store``): one ``user`` entry per prompt,
+one ``assistant`` entry per content block sharing a message id, one
+``user`` entry per ``tool_result`` parented on the matching ``tool_use``
+entry. Only ``user``/``assistant`` entries are produced - no attachments,
+titles or queue markers. Appending the result to a session store and
+resuming that session id seeds an external history into the SDK.
+
+:func:`messages_from_entries` is the projection back, used by tests and by
+anything that needs Anthropic-shape history out of a mirrored transcript.
 """
 
 from __future__ import annotations
@@ -52,14 +61,6 @@ def build_entries(
     permission_mode: str = "bypassPermissions",
     now: datetime | None = None,
 ) -> list[dict[str, Any]]:
-    """Render ``messages`` as entries a session store can resume from.
-
-    One ``user`` entry per prompt, one ``assistant`` entry per content
-    block sharing a message id, one ``user`` entry per ``tool_result``
-    parented on the matching ``tool_use``. Appending the result to a
-    session store and resuming that session id seeds external history
-    into the SDK.
-    """
     stamp = (now or datetime.now(UTC)).strftime("%Y-%m-%dT%H:%M:%S.") + (
         f"{(now or datetime.now(UTC)).microsecond // 1000:03d}Z"
     )
@@ -161,10 +162,7 @@ def build_entries(
     return entries
 
 
-def messages_from_entries(
-    entries: Iterable[Mapping[str, Any]], *, with_ts: bool = False
-) -> list[dict[str, Any]]:
-    """Messages as the model saw them; ``with_ts`` adds when each one was written."""
+def messages_from_entries(entries: Iterable[Mapping[str, Any]]) -> list[dict[str, Any]]:
     out: list[dict[str, Any]] = []
     last_message_id: str | None = None
     for entry in entries:
@@ -183,13 +181,7 @@ def messages_from_entries(
             ):
                 out[-1]["content"].extend(results)
             else:
-                message_out: dict[str, Any] = {
-                    "role": "user",
-                    "content": _user_content(content),
-                }
-                if with_ts:
-                    message_out["ts"] = entry.get("timestamp")
-                out.append(message_out)
+                out.append({"role": "user", "content": _user_content(content)})
             last_message_id = None
             continue
         blocks = _assistant_blocks(content)
@@ -202,10 +194,7 @@ def messages_from_entries(
         ):
             out[-1]["content"].extend(blocks)
         else:
-            message_out: dict[str, Any] = {"role": "assistant", "content": blocks}
-            if with_ts:
-                message_out["ts"] = entry.get("timestamp")
-            out.append(message_out)
+            out.append({"role": "assistant", "content": blocks})
         last_message_id = message_id
     return out
 
@@ -304,8 +293,10 @@ def _zero_usage() -> dict[str, Any]:
     }
 
 
+# ---- repair, windows, projections ---------------------------------------
+
 _PROMPT_TYPES = ("user", "assistant")
-_INTERRUPTED = "interrupted"
+_INTERRUPTED = "прервано"
 
 
 def open_tool_uses(

src/beaver_gateway/core/turn_capture.py

@@ -1,7 +1,10 @@
 """Side channel for what a backend learned during one turn.
 
 Frontends pass ``capture=TurnCapture()`` through ``Backend.complete``'s
-``**options``; only backends that keep state per conversation fill it in.
+``**options``. Backends that keep state per conversation (the Claude SDK
+adapter) fill it in after the stream closes; backends that don't
+(anthropic, raycast) drop the kwarg and the frontend falls back to a
+text-only history.
 """
 
 from __future__ import annotations

src/beaver_gateway/core/turn_record.py

@@ -0,0 +1,133 @@
+"""Cross-frontend turn record.
+
+Frontends that finish a turn (the Anthropic Messages frontend, the
+markdown frontend) emit a :class:`TurnRecord` to every handler in
+``GatewayRuntime.turn_log_handlers``. The markdown frontend uses this
+to persist chats from other frontends into the Obsidian vault — see
+``frontends/markdown/crossfront.py``.
+
+Kept tiny on purpose: it carries the structured-enough payload a logger
+needs (which agent ran, input history, the assembled assistant reply)
+and nothing else. The full event stream is gone by the time handlers
+run — if a future consumer needs deltas it would subscribe at a lower
+level, not here.
+"""
+
+from __future__ import annotations
+
+import re
+from dataclasses import dataclass
+from typing import TYPE_CHECKING, Any
+
+if TYPE_CHECKING:
+    from anthropic.types import Message, MessageParam
+
+
+__all__ = ["TurnRecord", "slugify"]
+
+
+# Filesystem-safe slug: collapse anything that isn't a word char or
+# space/hyphen to a hyphen, then squash runs of separators. Aimed at
+# letting users build filenames from ``record.first_user_text`` without
+# hand-rolling sanitization in every config.
+_SLUG_BAD_RE = re.compile(r"[^\w\s\-]+", flags=re.UNICODE)
+_SLUG_SEP_RE = re.compile(r"[\s\-]+", flags=re.UNICODE)
+
+
+def slugify(text: str, *, maxlen: int = 40) -> str:
+    """Sanitize ``text`` for use as a filename fragment.
+
+    Strips punctuation, collapses whitespace/hyphens into single ``-``,
+    and truncates to ``maxlen``. Returns ``"untitled"`` for empty input.
+    Unicode letters are preserved (Obsidian handles them fine; macOS
+    and modern Linux fs's too).
+    """
+    cleaned = _SLUG_BAD_RE.sub(" ", text).strip()
+    cleaned = _SLUG_SEP_RE.sub("-", cleaned).strip("-")
+    if not cleaned:
+        return "untitled"
+    if len(cleaned) > maxlen:
+        cleaned = cleaned[:maxlen].rstrip("-")
+    return cleaned or "untitled"
+
+
+@dataclass(frozen=True, slots=True)
+class TurnRecord:
+    """One completed turn, as seen by a frontend.
+
+    ``input_messages`` is the conversation history sent to the backend
+    (everything *before* the assistant reply). ``output_message`` is the
+    finalized assistant ``Message`` (post-accumulation, with all content
+    blocks attached). ``system`` is the per-request system prompt if any
+    — agents own their own ``system_prompt``, this is the override the
+    caller passed; most handlers can ignore it.
+
+    ``source`` names the frontend that produced the record so the
+    cross-frontend logger can avoid logging its own turns (markdown
+    frontend writing the file would otherwise also receive its own
+    broadcast and double-write).
+    """
+
+    agent_name: str
+    input_messages: list[MessageParam]
+    output_message: Message
+    system: str | None = None
+    source: str = "anthropic"
+
+    @property
+    def first_user_text(self) -> str:
+        """Plain text of the *earliest* user turn in this conversation.
+
+        Useful for naming new files by topic. Empty string if the input
+        history somehow has no user turn (shouldn't happen — turns are
+        broadcast only after a user → assistant cycle).
+        """
+        for msg in self.input_messages:
+            if msg.get("role") == "user":
+                return _text_of(msg.get("content", ""))
+        return ""
+
+    @property
+    def last_user_text(self) -> str:
+        """Plain text of the most recent user turn (the trigger)."""
+        for msg in reversed(self.input_messages):
+            if msg.get("role") == "user":
+                return _text_of(msg.get("content", ""))
+        return ""
+
+    @property
+    def assistant_text(self) -> str:
+        """Plain text of the assistant reply, with thinking/tool blocks dropped."""
+        chunks = [
+            getattr(block, "text", "") or ""
+            for block in getattr(self.output_message, "content", ())
+            if getattr(block, "type", None) == "text"
+        ]
+        return "\n\n".join(c for c in chunks if c)
+
+
+def _text_of(content: object) -> str:
+    """Flatten an Anthropic ``MessageParam.content`` field to a plain string.
+
+    Handles both shapes the SDK accepts: raw string, or a list of block
+    dicts (we pull ``text`` blocks only). Anything we don't recognize is
+    silently skipped — these helpers exist for naming files, not for
+    faithful content reconstruction.
+    """
+    if isinstance(content, str):
+        return content
+    if isinstance(content, list):
+        parts: list[str] = []
+        for blk in content:
+            if not isinstance(blk, dict):
+                continue
+            # ``MessageParam.content`` is typed as a union of typed-dicts
+            # per Anthropic SDK; we only care about plain ``text`` blocks
+            # and look them up via ``Any`` to dodge the keyed-typed-dict
+            # variance gymnastics (``ty`` won't let an open dict alias a
+            # closed-keyed one).
+            d: Any = blk
+            if d.get("type") == "text":
+                parts.append(str(d.get("text", "")))
+        return "\n\n".join(p for p in parts if p)
+    return ""

src/beaver_gateway/core/watch.py

@@ -1,8 +1,11 @@
-"""Vault watcher for the envelope.
+"""Vault watcher for the envelope (§3.5, §4.6).
 
-A content snapshot is taken at every envelope; a change is the diff of a
-file against that snapshot ("added lines only"). ``watchfiles`` debounces
-delivery, so a change carries the file's mtime, not when it landed.
+A content snapshot is taken at every envelope; a change is the diff of the
+file against that snapshot, "added lines only". ``watchfiles`` delivers
+paths with a settle debounce (Sync writes files in pieces), the change
+carries the file's mtime rather than the moment it landed. Which files
+show a full diff, which show up by name and which are ignored is a set of
+glob patterns the config hands in - the gateway itself knows no path.
 """
 
 from __future__ import annotations
@@ -26,7 +29,7 @@ if TYPE_CHECKING:
 
 __all__ = ["Change", "VaultWatch", "WatchRules"]
 
-_log = logging.getLogger("beaver_gateway.vault.watch")
+_log = logging.getLogger("beaver_gateway.core.watch")
 
 
 @dataclass(frozen=True, slots=True)

src/beaver_gateway/events/__init__.py

@@ -1 +0,0 @@
-"""The in-process event bus and the stream event protocol backends emit."""

src/beaver_gateway/frontends/_accumulate.py

@@ -1,7 +1,11 @@
 """Collapse an Anthropic event stream into one ``Message``.
 
-Mirrors the Anthropic SDK's own accumulator: folds content-block deltas
-into finalized blocks, indexed by their ``content_block`` index.
+Extracted from ``AnthropicMessagesFrontend`` so the markdown frontend
+can run the same accumulation logic when it wants the finalized turn
+rather than raw SSE chunks. Mirrors the Anthropic SDK's own accumulator:
+walks events, builds block dicts indexed by their ``content_block``
+index, folds text / thinking deltas in, buffers ``input_json_delta``
+chunks until the block closes (then JSON-parses them once).
 """
 
 from __future__ import annotations
@@ -29,7 +33,7 @@ from anthropic.types import (
 if TYPE_CHECKING:
     from collections.abc import AsyncIterator
 
-    from beaver_gateway.events.stream import MessageStreamEvent, StopReason
+    from beaver_gateway.core.events import MessageStreamEvent, StopReason
 
 
 __all__ = ["StreamAccumulator", "accumulate"]
@@ -38,7 +42,9 @@ __all__ = ["StreamAccumulator", "accumulate"]
 class StreamAccumulator:
     """Folds a stream of events into one ``Message``, incrementally.
 
-    Call :meth:`feed` for each event, then :meth:`finalize` once.
+    Use when you need to *both* forward events somewhere (SSE) *and*
+    keep a finalized ``Message`` for post-stream work (audit, logging
+    to disk). Call :meth:`feed` for each event, :meth:`finalize` once.
     """
 
     __slots__ = (
@@ -61,11 +67,10 @@ class StreamAccumulator:
         self._stop_sequence: str | None = None
 
     def feed(self, ev: MessageStreamEvent) -> None:
-        """Fold one stream event into the in-progress message state.
-
-        Uses ``isinstance`` rather than ``ev.type == ...`` so ty narrows
-        the event type from the class, not a string comparison.
-        """
+        # isinstance, not ``ev.type == "..."``: ty narrows on the
+        # discriminator only via the class, and the raw event union
+        # carries its own discriminators (``Raw*Event``) the SDK
+        # already promises.
         if isinstance(ev, RawMessageStartEvent):
             self._message_id = ev.message.id
             self._role = ev.message.role
@@ -104,11 +109,6 @@ class StreamAccumulator:
                 )
 
     def finalize(self, *, model: str) -> Message:
-        """Build the finalized ``Message`` from accumulated block state.
-
-        ``role`` is always ``"assistant"`` at the wire level; the cast
-        avoids a runtime check ty would otherwise require.
-        """
         content: list[Any] = []
         for idx in sorted(self._blocks):
             bd = self._blocks[idx]
@@ -120,6 +120,10 @@ class StreamAccumulator:
             elif btype == "thinking":
                 content.append(ThinkingBlock.model_validate(bd))
 
+        # ``role`` is always ``"assistant"`` at the wire level — we
+        # initialised the field to that and only overwrite from a
+        # ``RawMessageStartEvent`` which itself carries the same literal.
+        # The cast keeps both type-checkers happy without a runtime check.
         return Message(
             id=self._message_id or "msg_unknown",
             type="message",

src/beaver_gateway/frontends/_auth.py

@@ -1,7 +1,8 @@
 """Shared bearer-token verification for HTTP frontends.
 
-Reused by every bearer-protected frontend instead of duplicating the
-header-parsing logic.
+Extracted from ``AnthropicMessagesFrontend`` so the markdown frontend
+(and any future bearer-protected frontend) can reuse one canonical
+verifier instead of copy-pasting the header-parsing dance.
 """
 
 from __future__ import annotations
@@ -22,11 +23,14 @@ __all__ = ["require_token"]
 async def require_token(
     request: Request, runtime: GatewayRuntime, *, scope: str
 ) -> str:
-    """Verify the request's bearer token and scope; return its audit name.
+    """Verify the request's bearer + scope, return the token's audit name.
 
-    Checks ``X-Api-Key``, then ``Authorization: Bearer``, then ``?token=``.
-    401 on a missing/unknown token, 403 if the token's scope doesn't cover
-    ``scope``. Bootstrap tokens carry ``"*"`` and pass every scope check.
+    Accepts ``X-Api-Key: <token>`` (Anthropic SDK / LibreChat),
+    ``Authorization: Bearer <token>`` (curl, Cursor) and, when neither
+    header is present, ``?token=<token>`` (Komodo alerters). 401 on missing /
+    unknown token; 403 on a known token whose scope doesn't cover
+    ``scope``. Bootstrap tokens implicitly carry ``"*"`` and pass every
+    scope check.
     """
     api_key = request.headers.get("x-api-key")
     authorization = request.headers.get("authorization")
@@ -35,6 +39,8 @@ async def require_token(
     elif authorization:
         identity = await runtime.token_store.verify_bearer(authorization)
     else:
+        # Webhook senders that cannot set headers (Komodo alerters) put the
+        # token in the query string; the URL is not logged with it.
         qs_token = request.query_params.get("token")
         identity = await runtime.token_store.verify(qs_token) if qs_token else None
     if identity is None:

src/beaver_gateway/frontends/_sse.py

@@ -29,8 +29,10 @@ async def events_with_heartbeat(
 ) -> AsyncIterator[Any]:
     """Pass ``events`` through, yielding ``None`` after ``interval`` seconds of silence.
 
-    Only one consumer may iterate the result at a time; cancelling the
-    outer scope cancels the in-flight upstream fetch instead of leaking it.
+    One in-flight ``__anext__`` task is reused across timeouts: a second
+    consumer on the same async generator raises ``RuntimeError``.
+    Cancellation of the outer scope cancels that task instead of leaving
+    it dangling.
     """
     src = events.__aiter__()
     next_task: asyncio.Task[Any] | None = None

src/beaver_gateway/frontends/admin/frontend.py

@@ -1,7 +1,15 @@
 """Admin console: serves the ``ui/`` SPA and signs the operator in.
 
-Login/logout/session live under ``/admin/auth``; a process-lifetime
-admin bearer (scope ``*``) is minted at startup and rotates on restart.
+Everything the console shows comes from ``/api/*`` (``ApiFrontend``)
+with a bearer. This app, mounted at ``/admin``, owns three JSON routes
+under ``/admin/auth`` - login (``ADMIN_USER`` / ``ADMIN_PASS`` from env,
+session cookie signed with ``SESSION_SECRET``, 8 h), logout, and
+``session``, which hands a signed-in browser the process-lifetime admin
+bearer - and the static build under ``/admin/``.
+
+The bearer is minted at startup, registered in the token store with
+scope ``*`` and never persisted; a gateway restart rotates it, and the
+SPA refetches ``session`` on a 401.
 """
 
 from __future__ import annotations
@@ -19,8 +27,8 @@ import itsdangerous
 from fastapi import FastAPI, HTTPException, Request, status
 from fastapi.responses import FileResponse, JSONResponse, Response
 
+from beaver_gateway.core import audit
 from beaver_gateway.frontends.base import Frontend
-from beaver_gateway.security import audit
 
 if TYPE_CHECKING:
     from beaver_gateway.frontends.base import GatewayRuntime

src/beaver_gateway/frontends/anthropic.py

@@ -1,8 +1,17 @@
 """``POST /v1/messages`` frontend.
 
 Exposes the gateway as an Anthropic-compatible Messages endpoint, so any
-client that already speaks Anthropic can reach a configured agent by
-passing its name as ``model``.
+client that already speaks Anthropic (Cursor, Cline, the official SDK,
+``curl``) can hit a configured agent by passing its name as ``model``.
+
+A Claude agent behind this endpoint is a ``deep`` conversation: the client
+knows nothing about our ids, so the text fingerprint of the history it
+sends is the ``(anthropic, fingerprint)`` binding of the conversation,
+rebound after every turn to the fingerprint the next request will carry.
+A history nobody has seen becomes a new conversation, materialized by the
+home frontend of ``deep`` (the vault file), and every reply is published
+on the bus so that file follows the chat. Other agents (Raycast) stay
+stateless and are only archived through ``turn_log_handlers``.
 """
 
 from __future__ import annotations
@@ -16,21 +25,21 @@ from fastapi import FastAPI, HTTPException, Request, status
 from fastapi.responses import JSONResponse, StreamingResponse
 
 from beaver_gateway.agents.claude import ClaudeAgent
-from beaver_gateway.backends.capture import TurnCapture
-from beaver_gateway.backends.transcript import fingerprint, text_of
-from beaver_gateway.frontends.accumulate import StreamAccumulator
+from beaver_gateway.core import audit
+from beaver_gateway.core.transcript import fingerprint, text_of
+from beaver_gateway.core.turn_capture import TurnCapture
+from beaver_gateway.core.turn_record import TurnRecord
+from beaver_gateway.frontends._accumulate import StreamAccumulator
+from beaver_gateway.frontends._auth import require_token
 from beaver_gateway.frontends.base import Frontend
-from beaver_gateway.frontends.bearer import require_token
-from beaver_gateway.frontends.turn_record import TurnRecord
-from beaver_gateway.security import audit
 
 if TYPE_CHECKING:
     from collections.abc import AsyncIterator, Awaitable, Callable
 
     from anthropic.types import Message, MessageParam
 
-    from beaver_gateway.conversations.service import Conversations
-    from beaver_gateway.events.stream import MessageStreamEvent
+    from beaver_gateway.core.conversations import Conversations
+    from beaver_gateway.core.events import MessageStreamEvent
     from beaver_gateway.frontends.base import GatewayRuntime
     from beaver_gateway.storage.models import Conversation
 
@@ -266,8 +275,10 @@ async def _sse(
 ) -> AsyncIterator[bytes]:
     r"""Serialize an event stream to SSE, then hand the assembled ``Message`` on.
 
-    Wire format is ``event: <type>\ndata: <json>\n\n``; mid-stream errors
-    become a synthetic ``error`` event instead of a hung connection.
+    Each event becomes ``event: <type>\ndata: <json>\n\n`` - the shape
+    the Anthropic SDK's SSE decoder expects. Errors mid-stream are
+    swallowed into a synthetic ``error`` event so the client sees the
+    failure rather than a hung connection.
     """
     acc = StreamAccumulator()
     try:

src/beaver_gateway/frontends/api/__init__.py

@@ -1,4 +1,4 @@
-"""``ApiFrontend`` - the conversations API and event stream."""
+"""``ApiFrontend`` - the conversations API and event stream (§3.9)."""
 
 from beaver_gateway.frontends.api.frontend import ApiFrontend
 

src/beaver_gateway/frontends/api/frontend.py

@@ -1,7 +1,18 @@
-"""``ApiFrontend`` - ``/api/*``: conversations, SSE, sessions, usage, limits.
+"""``ApiFrontend`` - ``/api/*``: conversations, SSE, sessions, usage, limits (§3.9).
 
 Bearer scope ``api``; token and audit management need ``admin``. Every
-write goes through conversations/service.py; the frontend only shapes JSON.
+write goes through ``core/conversations``; the frontend only shapes JSON.
+``/api/events`` and ``/api/conversations/{id}/events`` replay the gateway
+bus as SSE with the same keepalive the markdown frontend uses, so a proxy
+never sees a silent socket.
+
+Usage figures come from the ``usage`` table (one row per turn; API-price
+``cost_usd`` and per-model ``model_usage`` are per-turn deltas of the SDK's
+cumulative ``ResultMessage`` counters, see ``storage.append_usage``);
+subscription quotas come from ``rate_limits``
+(``RateLimitEvent``). The quota covers the whole subscription, so
+``/api/limits`` puts the gateway's own spend for the window next to it
+for calibration by eye.
 """
 
 from __future__ import annotations
@@ -18,24 +29,20 @@ from fastapi.middleware.cors import CORSMiddleware
 from fastapi.responses import JSONResponse, StreamingResponse
 from sqlmodel import col, select
 
-from beaver_gateway.agents.claude import ClaudeAgent
-from beaver_gateway.agents.prompts import assemble
-from beaver_gateway.backends.transcript import prompt_count
-from beaver_gateway.conversations.context import compose
-from beaver_gateway.conversations.injects import URGENCY
-from beaver_gateway.conversations.kinds import Kind, as_kind
-from beaver_gateway.conversations.service import SEEDS, implied_title
-from beaver_gateway.frontends.base import Frontend
-from beaver_gateway.frontends.bearer import require_token
-from beaver_gateway.frontends.sse import (
+from beaver_gateway.core import audit
+from beaver_gateway.core.auth import VALID_SCOPES, hash_token
+from beaver_gateway.core.conversations import SEEDS, implied_title
+from beaver_gateway.core.injects import URGENCY
+from beaver_gateway.core.kinds import Kind, as_kind
+from beaver_gateway.frontends._auth import require_token
+from beaver_gateway.frontends._sse import (
     KEEPALIVE,
     SSE_HEADERS,
     events_with_heartbeat,
     sse_pack,
 )
-from beaver_gateway.frontends.urls import frontend_url
-from beaver_gateway.security import audit
-from beaver_gateway.security.auth import hash_token
+from beaver_gateway.frontends._urls import frontend_url
+from beaver_gateway.frontends.base import Frontend
 from beaver_gateway.storage import (
     create_token,
     list_audit_records,
@@ -49,15 +56,14 @@ from beaver_gateway.storage.models import (
     Token,
     Usage,
 )
-from beaver_gateway.vault.links import LinkIndex
 
 if TYPE_CHECKING:
     from collections.abc import AsyncIterator, Iterable, Sequence
     from pathlib import Path
 
-    from beaver_gateway.conversations.service import Conversations
+    from beaver_gateway.core.conversations import Conversations
+    from beaver_gateway.core.scheduler import Job, Scheduler
     from beaver_gateway.frontends.base import GatewayRuntime
-    from beaver_gateway.jobs.scheduler import Job, Scheduler
 
 _log = logging.getLogger("beaver_gateway.frontends.api")
 
@@ -75,10 +81,6 @@ WINDOWS: dict[str, timedelta] = {
 MEMORY_MAX_DEPTH = 12
 MEMORY_MAX_FILE = 2_000_000
 MEMORY_MAX_ENTRIES = 5000
-SEARCH_MIN = 2
-GRAPH_LIMIT = 120
-SEARCH_LIMIT = 20
-SNIPPET_MAX = 160
 
 
 class ApiFrontend(Frontend):
@@ -94,14 +96,12 @@ class ApiFrontend(Frontend):
         deep_agent: str | None = None,
         job_agent: str | None = None,
         memory_root: Path | None = None,
-        vault_root: Path | None = None,
     ) -> None:
         self.master_agent = master_agent
         self.branch_agent = branch_agent
         self.deep_agent = deep_agent
         self.job_agent = job_agent
         self.memory_root = memory_root.resolve() if memory_root is not None else None
-        self.vault_root = vault_root.resolve() if vault_root is not None else None
         self._app: FastAPI | None = None
 
     def agent_for(self, kind: Kind) -> str | None:
@@ -116,22 +116,14 @@ class ApiFrontend(Frontend):
         if runtime.conversations is None or runtime.bus is None:
             msg = "ApiFrontend needs runtime.conversations and runtime.bus"
             raise RuntimeError(msg)
-        self._app = build_app(
-            runtime, memory_root=self.memory_root, vault_root=self.vault_root
-        )
+        self._app = build_app(runtime, memory_root=self.memory_root)
 
     def app(self) -> FastAPI | None:
         return self._app
 
 
-def build_app(  # noqa: PLR0915
-    runtime: GatewayRuntime,
-    *,
-    memory_root: Path | None = None,
-    vault_root: Path | None = None,
-) -> FastAPI:
+def build_app(runtime: GatewayRuntime, *, memory_root: Path | None = None) -> FastAPI:  # noqa: PLR0915
     app = FastAPI(title="beaver-gateway / API")
-    links = LinkIndex(vault_root) if vault_root is not None else None
     app.add_middleware(
         CORSMiddleware,
         allow_origins=["*"],
@@ -217,7 +209,6 @@ def build_app(  # noqa: PLR0915
                         k: fe.agent_for(k) for k in fe.kinds if fe.agent_for(k)
                     },
                     "path": fe.path,
-                    "scope": fe.scope,
                     "url": frontend_url(request, runtime, fe),
                 }
                 for fe in runtime.frontends
@@ -238,49 +229,17 @@ def build_app(  # noqa: PLR0915
         firsts = await conversations.first_user_texts(
             cast("int", r.id) for r in rows if not r.title
         )
-        contexts = await conversations.context_tokens_by_id(r.external_id for r in rows)
-        public_ids = {r.id: r.external_id for r in rows}
         return {
             "conversations": [
                 {
                     **conversations.public(r),
                     "title": r.title or implied_title(firsts.get(cast("int", r.id))),
                     "last_item": _queue_item(latest.get(cast("int", r.id))),
-                    "context_tokens": contexts.get(r.external_id, 0),
-                    "parent": public_ids.get(r.parent_id) if r.parent_id else None,
                 }
                 for r in rows
             ]
         }
 
-    @app.get("/search")
-    async def search(request: Request) -> dict[str, Any]:
-        await require_token(request, runtime, scope=SCOPE)
-        query = (request.query_params.get("q") or "").strip()
-        if len(query) < SEARCH_MIN:
-            raise HTTPException(
-                status.HTTP_400_BAD_REQUEST, f"q must be at least {SEARCH_MIN} chars"
-            )
-        limit = min(query_int(request, "limit", SEARCH_LIMIT), SEARCH_LIMIT)
-        rows = await conversations.search(query, limit=limit)
-        firsts = await conversations.first_user_texts(
-            cast("int", r.id) for r in rows if not r.title
-        )
-        contexts = await conversations.context_tokens_by_id(r.external_id for r in rows)
-        files = _grep_memory(memory_root, query, limit=limit) if memory_root else []
-        return {
-            "query": query,
-            "conversations": [
-                {
-                    **conversations.public(r),
-                    "title": r.title or implied_title(firsts.get(cast("int", r.id))),
-                    "context_tokens": contexts.get(r.external_id, 0),
-                }
-                for r in rows
-            ],
-            "files": files,
-        }
-
     @app.post("/conversations", status_code=status.HTTP_201_CREATED)
     async def create_conversation(request: Request) -> dict[str, Any]:
         token = await require_token(request, runtime, scope=SCOPE)
@@ -353,45 +312,6 @@ def build_app(  # noqa: PLR0915
         conv = await conv_of(public_id)
         return {"id": conv.external_id, "messages": await conversations.history(conv)}
 
-    @app.get("/conversations/{public_id}/context")
-    async def get_context(public_id: str, request: Request) -> dict[str, Any]:
-        await require_token(request, runtime, scope=SCOPE)
-        conv = await conv_of(public_id)
-        agent = runtime.agents.get(conv.agent_name)
-        if not isinstance(agent, ClaudeAgent):
-            raise HTTPException(
-                status.HTTP_404_NOT_FOUND, f"no claude agent behind {public_id}"
-            )
-        entries = list(await conversations.entries(conv))
-        for sub in await conversations.subpaths(conv):
-            entries.extend(await conversations.entries(conv, subpath=sub))
-        out = compose(
-            agent,
-            as_kind(conv.kind),
-            entries,
-            context_tokens=await conversations.context_tokens(conv),
-        )
-        out["id"] = conv.external_id
-        out["turns"] = prompt_count(entries)
-        if request.query_params.get("prompt"):
-            sources = agent.prompt_for(as_kind(conv.kind))
-            out["prompt_text"] = (
-                assemble(sources) if sources is not None else agent.system_prompt
-            )
-        return out
-
-    @app.get("/vault/graph")
-    async def vault_graph(request: Request) -> dict[str, Any]:
-        await require_token(request, runtime, scope=SCOPE)
-        if links is None:
-            raise HTTPException(
-                status.HTTP_404_NOT_FOUND,
-                "no vault root: set ApiFrontend(vault_root=...)",
-            )
-        paths = [p for p in request.query_params.getlist("path") if p]
-        limit = min(query_int(request, "limit", GRAPH_LIMIT), GRAPH_LIMIT)
-        return {"root": str(links.root), **links.neighbours(paths, limit=limit)}
-
     @app.get("/conversations/{public_id}/entries")
     async def get_entries(public_id: str, request: Request) -> dict[str, Any]:
         await require_token(request, runtime, scope=SCOPE)
@@ -738,51 +658,22 @@ def build_app(  # noqa: PLR0915
         await require_token(request, runtime, scope=SCOPE)
         rows = await conversations.rate_limits(limit=200)
         latest: dict[str, RateLimit] = {}
-        unified: dict[str, tuple[RateLimit, dict[str, Any]]] = {}
         for row in rows:
             latest.setdefault(row.window, row)
-            for name, info in _unified_windows(row).items():
-                unified.setdefault(name, (row, info))
         now = datetime.now(UTC)
         windows = []
-        for window in dict.fromkeys([*latest, *unified]):
-            row = latest.get(window)
-            seen = unified.get(window)
-            public = _limit_public(row) if row else _limit_blank(window, seen)
-            resets = row.resets_at if row else None
-            if seen and _newer_figure(seen, row):
-                carrier, info = seen
-                public["utilization"] = info.get("utilization")
-                public["reported_at"] = _iso(carrier.ts)
-                resets = _unix_dt(info.get("resetsAt")) or resets
-                public["resets_at"] = _iso(resets)
-            length = _window_length(window)
-            anchor = row if row is not None else (seen[0] if seen else None)
-            if length is not None and resets is not None:
-                since = _aware(resets) - length
-            else:
-                since = _aware(anchor.ts) if anchor else now
+        for window, row in latest.items():
+            length = WINDOWS.get(window)
+            since = (
+                _aware(row.resets_at) - length
+                if length is not None and row.resets_at is not None
+                else _aware(row.ts)
+            )
             gateway = _sum_usage(await _usage_rows(runtime, since, now))
             gateway["since"] = since.isoformat(timespec="seconds")
-            known = next(
-                (
-                    r
-                    for r in rows
-                    if r.window == window
-                    and r.utilization is not None
-                    and _aware(r.ts) >= since
-                ),
-                None,
-            )
-            windows.append(
-                {
-                    **public,
-                    "gateway": gateway,
-                    "last_known": _limit_public(known) if known else None,
-                }
-            )
+            windows.append({**_limit_public(row), "gateway": gateway})
         windows.sort(
-            key=lambda w: (_window_length(w["window"]) or timedelta.max, w["window"])
+            key=lambda w: (WINDOWS.get(w["window"], timedelta.max), w["window"])
         )
         return {"windows": windows, "history": [_limit_public(r) for r in rows[:50]]}
 
@@ -815,19 +706,6 @@ def build_app(  # noqa: PLR0915
             "content": content,
         }
 
-    @app.get("/scopes")
-    async def scopes(request: Request) -> dict[str, Any]:
-        await require_token(request, runtime, scope=ADMIN_SCOPE)
-        by_frontend = {
-            fe.scope: fe.name or fe.path or "" for fe in runtime.frontends if fe.scope
-        }
-        return {
-            "scopes": [
-                {"scope": s, "frontend": by_frontend.get(s)}
-                for s in sorted(runtime.scopes)
-            ]
-        }
-
     @app.get("/tokens")
     async def tokens(request: Request) -> dict[str, Any]:
         await require_token(request, runtime, scope=ADMIN_SCOPE)
@@ -842,7 +720,7 @@ def build_app(  # noqa: PLR0915
         data = await body_of(request)
         name = text_of(data, "name").strip()
         scope = str(data.get("scope") or "*")
-        if scope not in runtime.scopes:
+        if scope not in VALID_SCOPES:
             raise HTTPException(status.HTTP_400_BAD_REQUEST, f"invalid scope {scope!r}")
         plaintext = secrets.token_urlsafe(32)
         async with runtime.db.session() as session:
@@ -1077,59 +955,6 @@ def _queue_item(item: InjectQueueItem | None) -> dict[str, Any] | None:
     }
 
 
-def _window_length(name: str) -> timedelta | None:
-    if name.startswith("five_hour"):
-        return timedelta(hours=5)
-    if "seven_day" in name:
-        return timedelta(days=7)
-    return WINDOWS.get(name)
-
-
-def _unix_dt(value: Any) -> datetime | None:
-    if isinstance(value, int | float) and value > 0:
-        return datetime.fromtimestamp(value, tz=UTC)
-    return None
-
-
-def _unified_windows(row: RateLimit) -> dict[str, dict[str, Any]]:
-    """Every window the event carried a figure for, not just the one it was about."""
-    raw = row.raw if isinstance(row.raw, dict) else {}
-    found = raw.get("unifiedWindows")
-    if not isinstance(found, dict):
-        return {}
-    return {
-        str(name): info
-        for name, info in found.items()
-        if isinstance(info, dict) and info.get("utilization") is not None
-    }
-
-
-def _newer_figure(
-    seen: tuple[RateLimit, dict[str, Any]], row: RateLimit | None
-) -> bool:
-    if row is None or row.utilization is None:
-        return True
-    return seen[0].id != row.id and _aware(seen[0].ts) > _aware(row.ts)
-
-
-def _limit_blank(
-    window: str, seen: tuple[RateLimit, dict[str, Any]] | None
-) -> dict[str, Any]:
-    carrier = seen[0] if seen else None
-    return {
-        "id": None,
-        "ts": _iso(carrier.ts) if carrier else None,
-        "window": window,
-        "status": "allowed",
-        "utilization": None,
-        "resets_at": None,
-        "overage_status": None,
-        "overage_resets_at": None,
-        "agent": carrier.agent_name if carrier else None,
-        "session_id": carrier.session_id if carrier else None,
-    }
-
-
 def _limit_public(row: RateLimit) -> dict[str, Any]:
     return {
         "id": row.id,
@@ -1179,35 +1004,6 @@ def _memory_path(root: Path, raw: str) -> Path:
     return target
 
 
-def _grep_memory(root: Path, query: str, *, limit: int) -> list[dict[str, Any]]:
-    needle = query.casefold()
-    hits: list[dict[str, Any]] = []
-    for path in sorted(root.rglob("*")):
-        if len(hits) >= limit:
-            break
-        if not path.is_file() or any(part.startswith(".") for part in path.parts):
-            continue
-        if path.suffix not in (".md", ".txt", ".json", ".yaml", ".yml"):
-            continue
-        try:
-            if path.stat().st_size > MEMORY_MAX_FILE:
-                continue
-            text = path.read_text(encoding="utf-8")
-        except (OSError, UnicodeDecodeError):
-            continue
-        rel = str(path.relative_to(root))
-        if needle in rel.casefold():
-            hits.append({"path": rel, "line": 0, "snippet": ""})
-            continue
-        for number, line in enumerate(text.splitlines(), 1):
-            if needle in line.casefold():
-                hits.append(
-                    {"path": rel, "line": number, "snippet": line.strip()[:SNIPPET_MAX]}
-                )
-                break
-    return hits
-
-
 def _tree(root: Path, directory: Path, *, depth: int) -> list[dict[str, Any]]:
     if depth > MEMORY_MAX_DEPTH:
         return []

src/beaver_gateway/frontends/base.py

@@ -1,7 +1,13 @@
-"""Frontend ABC and the runtime context handed to ``configure``.
+"""Frontend ABC + the runtime context handed to ``configure``.
 
-A frontend routes inbound traffic into the gateway (an HTTP mount, a
-poller, or both); ``GatewayRuntime`` carries the built state each needs.
+A frontend is anything that routes inbound traffic into the gateway: an
+HTTP surface mounted under its ``path`` on the single gateway port
+(``frontends/root.py``), a poller (Telegram), or both. ``GatewayRuntime``
+carries everything a frontend may need that isn't user-config: built
+registries, per-agent backends, and the in-memory token store. The
+user's ``/config/config.py`` defines a ``Gateway`` (lists); ``cli.main``
+turns that into a ``GatewayRuntime`` and hands it to each frontend's
+``configure``.
 """
 
 from __future__ import annotations
@@ -10,20 +16,16 @@ from abc import ABC, abstractmethod
 from dataclasses import dataclass, field
 from typing import TYPE_CHECKING, Any
 
-from beaver_gateway.frontends.files import SendFileError
-from beaver_gateway.security.auth import BUILTIN_SCOPES
-
 if TYPE_CHECKING:
     from collections.abc import Awaitable, Callable, Mapping, Sequence
-    from pathlib import Path
 
     from starlette.types import ASGIApp
 
-    from beaver_gateway.app import AgentRegistry, McpRegistry
     from beaver_gateway.backends.base import Backend
-    from beaver_gateway.conversations.kinds import Kind
-    from beaver_gateway.frontends.turn_record import TurnRecord
-    from beaver_gateway.security.auth import TokenStore
+    from beaver_gateway.core.auth import TokenStore
+    from beaver_gateway.core.kinds import Kind
+    from beaver_gateway.core.registry import AgentRegistry, McpRegistry
+    from beaver_gateway.core.turn_record import TurnRecord
     from beaver_gateway.storage import Database
     from beaver_gateway.storage.models import Conversation, ConversationBinding
 
@@ -34,8 +36,20 @@ if TYPE_CHECKING:
 class GatewayRuntime:
     """Post-build state of the gateway, shared with every frontend.
 
-    Backends are keyed by agent name, not type: one backend instance may
-    serve several agents, so the indirection lives at lookup time.
+    Backends are keyed by **agent name**, not type — one ``RaycastBackend``
+    instance can serve many ``RaycastAgent`` instances, but the lookup
+    site (an inbound request with ``model=<agent.name>``) already has
+    the name in hand, so the indirection lives one step earlier.
+
+    ``mcp_internal_urls`` is filled in Phase 2.1: one loopback URL per
+    declared ``McpServer`` so ``ClaudeSdkBackend``
+    can pass them to ``BackendOptions.mcp_servers`` without re-running
+    discovery.
+
+    ``db`` (Phase 4.1) is the shared :class:`Database` handle. Phase 4.2
+    will switch ``TokenStore`` to read from it; Phase 4.3 admin/audit
+    write through it. Phase 4.1 only attaches it — existing frontends
+    ignore it.
     """
 
     agents: AgentRegistry
@@ -44,53 +58,68 @@ class GatewayRuntime:
     token_store: TokenStore
     db: Database
     mcp_internal_urls: Mapping[str, str] = field(default_factory=dict)
+    # Phase 4.3 — AdminFrontend reads creds + cookie-signing key from
+    # the runtime so the user's ``config.py`` doesn't have to know
+    # anything about env wiring. Defaulted to empty so existing tests /
+    # call sites that don't touch the admin path keep building; the
+    # admin frontend ``configure()`` itself rejects empty values.
     admin_user: str = ""
-    """Operator login for the admin console, checked by ``AdminFrontend``."""
     admin_pass: str = ""
     session_secret: str = ""
+    # The full sibling-frontends list, in declaration order. AdminFrontend
+    # uses it to advertise concrete bearer-endpoint URLs (host/port) on
+    # the dashboard so the operator can copy ready-to-use links / curl
+    # snippets. Other frontends ignore it.
     frontends: Sequence[Frontend] = field(default_factory=tuple)
-    """Every frontend in declaration order, for advertising their URLs."""
+    # Frontends that finish a turn (Anthropic Messages, Markdown) iterate
+    # this list and ``await`` each handler with a ``TurnRecord``. Handlers
+    # are appended during ``configure()`` by frontends that want a
+    # cross-frontend chat archive — currently the markdown frontend's
+    # ``log_all_chats`` mode. Handler exceptions are caught at the call
+    # site; they never block the user-visible response.
+    #
+    # The field is typed as ``list[Any]`` rather than the precise
+    # ``list[TurnLogHandler]`` because the alias lives under TYPE_CHECKING
+    # to keep ``anthropic.types`` out of the runtime import graph for
+    # this base module.
     turn_log_handlers: list[TurnLogHandler] = field(default_factory=list)
-    """Called with a ``TurnRecord`` after each turn; failures never reach the user."""
+    # M1b: conversations service, event bus and the shared session pool.
+    # ``Any`` for the same import-graph reason as above; ``None`` only in
+    # tests that build a runtime without them.
     conversations: Any = None
     bus: Any = None
     pool: Any = None
     scheduler: Any = None
+    # External origin the reverse proxy puts in front of the gateway
+    # (``Gateway.public_url``); ``None`` means "derive from the request".
     public_url: str | None = None
-    """``Gateway.public_url``; ``None`` derives the origin from the request."""
-    scopes: frozenset[str] = BUILTIN_SCOPES
-    """Every token scope this gateway knows: the builtins plus each frontend's."""
-    send_file_roots: tuple[Path, ...] = ()
-    """Directories ``send_file`` may read, from ``SEND_FILE_ROOTS``."""
 
 
 class Frontend(ABC):
     """Routes inbound traffic into the gateway.
 
     HTTP frontends set ``path`` and return their ASGI app from ``app()``;
-    these are mounted under that path on the one gateway port. ``serve()``
-    is for non-HTTP work (polling, vault mirrors) and defaults to nothing.
-    ``landing`` marks the app that ``/`` redirects to.
-
-    ``scope`` is the token scope this frontend's routes are gated by; a
-    frontend that names its own scope gets keys nobody else's token opens,
-    and ``POST /api/tokens`` will mint them (see ``security/auth.py``).
+    ``cli`` mounts every such app under that path on the one gateway
+    port, so ``/anthropic/v1/messages`` reaches the Anthropic frontend's
+    ``/v1/messages``. ``serve()`` is for work outside HTTP - polling,
+    vault mirrors - and defaults to nothing. ``landing`` marks the app
+    that ``/`` redirects to (the admin console).
 
     A frontend that shows conversations declares ``name`` (the binding
-    key) and ``kinds`` (which conversation kinds it shows). The first
-    frontend whose ``materialize`` returns a binding is the *home* of
-    that kind, used by ``spawn`` for new conversations; ``agent_for``
-    names the default agent for a kind. Stateless frontends (MCP, admin)
-    leave these at their defaults.
+    key) and ``kinds`` (which conversation kinds it shows);
+    ``core/conversations`` refuses to bind a conversation to a frontend
+    outside its declaration. The first frontend in declaration order
+    whose ``materialize`` returns a binding is the *home* of that kind:
+    ``spawn`` calls it so a new conversation gets a window (a vault file,
+    a Telegram topic). ``agent_for`` names the default agent for a kind
+    so callers may omit ``agent``. Stateless frontends (MCP, admin) keep
+    the defaults and stay outside the routing.
     """
 
     name: str = ""
     kinds: tuple[Kind, ...] = ()
     path: str | None = None
     landing: bool = False
-    scope: str | None = None
-    sends_files: bool = False
-    """Whether ``send_file`` reaches a human through this frontend."""
 
     @abstractmethod
     def configure(self, runtime: GatewayRuntime) -> None: ...
@@ -110,19 +139,3 @@ class Frontend(ABC):
     async def mark_closed(self, conv: Conversation) -> bool:  # noqa: ARG002
         """Show in the window that the conversation is over (a renamed topic)."""
         return False
-
-    async def send_file(
-        self,
-        conv: Conversation,  # noqa: ARG002
-        path: str,  # noqa: ARG002
-        *,
-        caption: str = "",  # noqa: ARG002
-        method: str = "auto",  # noqa: ARG002
-    ) -> str:
-        """Put a file in this conversation's window; the answer goes to the model."""
-        msg = f"frontend {self.name!r} cannot send files"
-        raise SendFileError(msg)
-
-    def file_note(self) -> str:
-        """Roots and ceiling, for the tool description the model reads."""
-        return ""

src/beaver_gateway/frontends/files.py

@@ -1,95 +0,0 @@
-"""What an agent may hand to a human: an allowed path, a ceiling, a method.
-
-The ceiling and the method names belong to the frontend's API; the check
-lives here so every frontend refuses the same way, in words the model reads.
-"""
-
-from __future__ import annotations
-
-import mimetypes
-from dataclasses import dataclass, replace
-from pathlib import Path
-
-__all__ = ["AUTO", "DEFAULTS", "METHODS", "SendFileError", "SendFiles", "method_for"]
-
-AUTO = "auto"
-METHODS = ("document", "photo", "audio", "voice", "video", "video_note", "animation")
-
-_BY_SUFFIX = {
-    ".jpg": "photo",
-    ".jpeg": "photo",
-    ".png": "photo",
-    ".webp": "photo",
-    ".gif": "animation",
-    ".mp3": "audio",
-    ".flac": "audio",
-    ".m4a": "audio",
-    ".wav": "audio",
-    ".aac": "audio",
-    ".ogg": "voice",
-    ".oga": "voice",
-    ".opus": "voice",
-    ".mp4": "video",
-    ".m4v": "video",
-    ".mov": "video",
-}
-_BY_TYPE = {"image": "photo", "audio": "audio", "video": "video"}
-
-
-class SendFileError(Exception):
-    """Why a file was not sent, worded for the model that asked."""
-
-
-@dataclass(frozen=True, slots=True)
-class SendFiles:
-    """Which files a frontend accepts from an agent and how big they may be."""
-
-    roots: tuple[Path, ...] = ()
-    """Directories a path must resolve inside; nothing outside is sendable."""
-    max_bytes: int = 50 * 1024 * 1024
-    """Upload ceiling of the API behind the frontend; the Bot API gives 50 MB."""
-
-    def with_roots(self, *roots: Path) -> SendFiles:
-        return replace(self, roots=tuple(dict.fromkeys((*self.roots, *roots))))
-
-    @property
-    def note(self) -> str:
-        """Roots and ceiling as one line for the tool description."""
-        where = ", ".join(str(r) for r in self.roots) or "none, so nothing is sendable"
-        return f"paths under {where}, up to {mb(self.max_bytes)} MB"
-
-    def resolve(self, raw: str) -> Path:
-        path = Path(raw.strip()).expanduser().resolve()
-        if not any(path.is_relative_to(r.expanduser().resolve()) for r in self.roots):
-            msg = f"{path} is outside the allowed roots ({self.note}); copy it there"
-            raise SendFileError(msg)
-        if not path.is_file():
-            msg = f"{path} is not a file"
-            raise SendFileError(msg)
-        size = path.stat().st_size
-        if size > self.max_bytes:
-            msg = (
-                f"{path.name} is {mb(size)} MB, over the {mb(self.max_bytes)} MB limit"
-            )
-            raise SendFileError(msg)
-        return path
-
-
-DEFAULTS = SendFiles()
-
-
-def method_for(path: Path, method: str = AUTO) -> str:
-    if method and method != AUTO:
-        if method not in METHODS:
-            msg = f"unknown method {method!r}; one of: {', '.join((AUTO, *METHODS))}"
-            raise SendFileError(msg)
-        return method
-    guessed = _BY_SUFFIX.get(path.suffix.lower())
-    if guessed:
-        return guessed
-    mime, _ = mimetypes.guess_type(path.name)
-    return _BY_TYPE.get(mime.split("/")[0], "document") if mime else "document"
-
-
-def mb(size: int) -> str:
-    return f"{round(size / 1024 / 1024, 2):g}"

src/beaver_gateway/frontends/markdown/__init__.py

@@ -1,7 +1,13 @@
-"""Markdown frontend — chat archive backed by ``.md`` files in an Obsidian vault.
+"""Markdown frontend — turn-by-turn chat archive backed by ``.md`` files.
 
-An Obsidian plugin POSTs ``{filename, content?}`` to ``/chat``; the
-frontend runs the agent on a trailing user turn and appends the reply.
+The user maintains chats as plain markdown files in an Obsidian vault.
+A plugin in Obsidian POSTs ``{filename, content?}`` to ``/chat`` and the
+frontend parses the file, finds the last turn, and runs the agent if
+the last turn is ``user``. The full response is appended back to the
+file as an ``### Assistant:`` turn. With ``log_all_chats=True`` the
+frontend also subscribes to every other frontend's turns and writes
+them into ``{vault_path}/{logged_subdir}/`` so the vault is the single
+chronological archive of all conversations.
 """
 
 from beaver_gateway.frontends.markdown.frontend import MarkdownFrontend

src/beaver_gateway/frontends/markdown/crossfront.py

@@ -1,7 +1,17 @@
 """Cross-frontend chat logger.
 
-Mirrors turns completed by other frontends into the vault as ``.md``
-files, matching a conversation's continuation by content-hash fingerprint.
+When ``MarkdownFrontend(log_all_chats=True)`` is configured, every turn
+completed by any other frontend (currently the Anthropic Messages
+frontend) is mirrored into the vault as a ``.md`` file. Subsequent
+turns of the same conversation append to the same file — matched by a
+content-hash fingerprint stored in YAML frontmatter.
+
+The fingerprint hashes the message history *before* the new assistant
+reply. So the next request's input history (which now includes the
+prior assistant reply) hashes to the value we just persisted —
+``hash(prev_input + [assistant_reply])`` — and the lookup hits the
+same file. New conversations (no prior fingerprint match) get a fresh
+file under ``{vault_path}/{logged_subdir}/{agent_name}/``.
 """
 
 from __future__ import annotations
@@ -22,7 +32,7 @@ if TYPE_CHECKING:
 
     from anthropic.types import MessageParam
 
-    from beaver_gateway.frontends.turn_record import TurnRecord
+    from beaver_gateway.core.turn_record import TurnRecord
 
 
 _log = logging.getLogger("beaver_gateway.frontends.markdown.crossfront")
@@ -43,8 +53,12 @@ vault. ``None`` keeps ``{vault}/{logged_subdir}/{agent}/{YYYY-MM-DD}_{slug}.md``
 def fingerprint_messages(messages: Iterable[MessageParam]) -> str:
     """Stable, short hex hash of a conversation prefix.
 
-    Hashes ``(role, text-only content)`` pairs so differently-shaped message
-    histories that carry the same text still fingerprint identically.
+    Built from ``(role, normalized_content)`` pairs only — so the
+    Markdown frontend's parser-shaped messages (text-only) and the
+    Anthropic frontend's raw ``messages`` payload (which may also be
+    string-only at v1) hash compatibly when they represent the same
+    conversation. Tool blocks / images would diverge, but those aren't
+    in the v1 ingest path.
     """
     h = hashlib.sha1(usedforsecurity=False)
     for msg in messages:
@@ -71,8 +85,12 @@ def fingerprint_messages(messages: Iterable[MessageParam]) -> str:
 class CrossFrontendLogger:
     """Maintains the fingerprint→file map and writes turns to disk.
 
-    The map is in-process, rebuilt by ``warm_index`` on startup; all disk
-    writes funnel through one lock to sidestep races.
+    The map is in-process; on startup ``warm_index`` rebuilds it from
+    YAML frontmatter of every file under ``logged_subdir``. A miss
+    creates a new file, a hit appends to the existing one. All disk
+    work funnels through one ``asyncio.Lock`` because the writes are
+    cheap and serializing them sidesteps a class of races we don't need
+    to think about.
     """
 
     def __init__(
@@ -87,13 +105,20 @@ class CrossFrontendLogger:
         self._index: dict[str, Path] = {}
         self._lock = asyncio.Lock()
         self._chat_path_fn = chat_path
+        # When the user supplies a custom path function, files can land
+        # anywhere in the vault — so we have to scan the whole vault on
+        # startup to rebuild the fingerprint→path map. With the default
+        # layout we can bound the scan to ``_logs/``.
         self._scan_root = vault_path if chat_path is not None else self._root
 
     def warm_index(self) -> None:
         """Scan logged files synchronously, populating the fingerprint map.
 
-        Called before any cross-frontend turn arrives; a custom ``chat_path``
-        forces a full-vault walk instead of scanning just ``logged_subdir``.
+        Called from ``MarkdownFrontend.configure`` so the map is ready
+        before any cross-frontend turn arrives. ``frontmatter.load``
+        reads only enough of the file to parse the YAML head, so the
+        scan is cheap even on large vaults — but a custom ``log_path``
+        forces a full-vault walk; mention that in the constructor doc.
         """
         if not self._scan_root.exists():
             return
@@ -115,19 +140,29 @@ class CrossFrontendLogger:
     async def handle(self, record: TurnRecord) -> None:
         """Append or create a logged file for ``record``.
 
-        Skips ``source == "markdown"`` records (already on disk); matches
-        the target file by fingerprinting ``input_messages`` sans the new turn.
+        Records that the markdown frontend itself produced
+        (``source=="markdown"``) are skipped — those already live in the
+        user's hand-written file and shouldn't be duplicated into the
+        ``_logs`` shadow tree.
         """
         if record.source == "markdown":
             return
 
         async with self._lock:
+            # ``input_messages`` is the *full* history sent to the backend
+            # (last entry is the new user turn). Match against the prefix
+            # that excludes the new user turn — that's what the previous
+            # write stored as its fingerprint. Empty prefix is the
+            # well-known "brand new chat" sentinel.
             prefix = record.input_messages[:-1]
             prev_fp = fingerprint_messages(prefix) if prefix else None
             target = self._index.get(prev_fp) if prev_fp else None
             if target is None:
                 target = self._new_file_path(record)
 
+            # Build the full history including the assistant reply; the
+            # new fingerprint matches *that* prefix, so the next user
+            # turn (history grows by one user msg) will hit this file.
             assistant_msg: MessageParam = {
                 "role": "assistant",
                 "content": _flatten_text(record.output_message),
@@ -139,6 +174,9 @@ class CrossFrontendLogger:
                 existing = target.read_text(encoding="utf-8")
                 parsed = frontmatter.loads(existing)
                 body = strip_trailing_user_scaffold(parsed.content)
+                # We append only the *new* user turn (the last one in
+                # input_messages, since prior turns are already on disk)
+                # plus the assistant reply.
                 new_user = record.input_messages[-1]
                 new_block = renderer.render_user_param(new_user)
                 new_block = renderer.append_to_body(
@@ -147,6 +185,7 @@ class CrossFrontendLogger:
                 new_body = renderer.append_to_body(body, new_block)
                 metadata = dict(parsed.metadata)
             else:
+                # Materialize the whole conversation from scratch.
                 new_body = _render_full_history(
                     record.input_messages, record.output_message
                 )
@@ -158,15 +197,21 @@ class CrossFrontendLogger:
             metadata["fingerprint"] = new_fp
             metadata["source"] = record.source
             self._write(target, metadata, new_body)
+            # Maintain the index: drop the old fp (it's stale once we
+            # write the new turn), add the new one.
             if prev_fp:
                 self._index.pop(prev_fp, None)
             self._index[new_fp] = target
 
+    # ---- internals -----------------------------------------------------
+
     def _new_file_path(self, record: TurnRecord) -> Path:
         """Pick a fresh filename for a brand-new conversation.
 
-        Delegates to ``chat_path`` if set; otherwise
-        ``{logged_subdir}/{agent}/{date}_{hex8}.md``.
+        With a user-supplied ``chat_path`` we delegate to it (joining a
+        relative result with the vault root). Without one, we fall back
+        to ``{logged_subdir}/{agent}/{date}_{hex8}.md`` and ensure the
+        ``.md`` suffix in case the user picks a non-md extension by hand.
         """
         if self._chat_path_fn is not None:
             result = self._chat_path_fn(
@@ -179,6 +224,8 @@ class CrossFrontendLogger:
             result.parent.mkdir(parents=True, exist_ok=True)
             return result
         day = datetime.now(UTC).strftime("%Y-%m-%d")
+        # Short hex from the input hash so two same-day chats sort
+        # stably and don't collide.
         salt = fingerprint_messages(record.input_messages)[:8]
         agent_dir = self._root / record.agent_name
         agent_dir.mkdir(parents=True, exist_ok=True)
@@ -191,6 +238,9 @@ class CrossFrontendLogger:
             text = frontmatter.dumps(post) + "\n"
         else:
             text = body if body.endswith("\n") else body + "\n"
+        # Sync write inside the lock — keeps the implementation tiny;
+        # individual logged turns are small enough that the blocking
+        # write doesn't matter at human conversation rates.
         path.write_text(text, encoding="utf-8")
 
 
@@ -227,13 +277,18 @@ def _render_full_history(messages: list[MessageParam], assistant: Any) -> str:
 def strip_trailing_user_scaffold(body: str) -> str:
     """Drop a trailing empty ``### User:`` block if present.
 
-    Avoids leaving two ``### User:`` headers in a row when appending a
-    turn that wasn't typed into the file by hand.
+    Cross-frontend turns aren't typed into the file by the human — they
+    arrive whole from another frontend. If we leave the previous run's
+    scaffold in place, we'd write the new user turn right after an
+    empty marker (visual noise, two ``### User:`` headers in a row).
+    Trim it and let the append flow add a fresh scaffold at the end.
     """
     stripped = body.rstrip()
     marker = "### User:"
     if not stripped.endswith(marker):
         return body
+    # Walk back: the scaffold is the marker preceded by either start-of-file
+    # or an HR/blank line. Find the last newline before the marker, cut.
     head = stripped[: -len(marker)].rstrip()
     if head.endswith("---"):
         head = head[: -len("---")].rstrip()

src/beaver_gateway/frontends/markdown/frontend.py

@@ -1,7 +1,34 @@
 """``MarkdownFrontend`` — chat-via-markdown-files frontend.
 
-``POST /chat`` (and SSE ``/chat/stream``) parses the vault file, runs
-the agent on the trailing user turn, and appends the reply.
+Wires:
+
+* ``POST /chat {filename, content?, agent?}`` — bearer-authenticated
+  trigger. The plugin in Obsidian fires this after the user edits a
+  ``.md`` and the file gets synced (or with ``content`` to short-circuit
+  the sync delay). We parse the file, check the last turn — assistant
+  → no-op, user → run the agent and append.
+* ``GET /healthz`` — liveness.
+
+Concurrency model: an in-memory ``set[Path]`` of files currently in
+flight. Two concurrent requests for the same file → the second gets
+409. The set is single-process (one gateway instance) — that's by
+design; the markdown frontend is the only writer in its vault from
+the gateway side. The turn itself runs through ``core/conversations``
+(one turn per conversation, ``running_turn`` in the DB), so a message
+posted to the same conversation via ``/api`` waits its turn.
+
+A chat file is a ``deep`` conversation bound as
+``(markdown, <vault-relative path>)``; frontmatter carries only ``agent``
+and ``conversation_id`` (§3.10), tool calls are never rendered. The
+frontend is the home of ``deep``: ``materialize`` gives a conversation
+spawned elsewhere its file, and :class:`.mirror.ChatMirror` keeps that
+file in step with replies produced outside ``/chat``.
+
+Cross-frontend logging: when ``log_all_chats=True``, ``configure()``
+registers a handler on ``runtime.turn_log_handlers`` so every other
+frontend's completed turns also land in the vault. The handler logic
+lives in :mod:`.crossfront` so this module stays focused on the HTTP
+shape.
 """
 
 from __future__ import annotations
@@ -18,10 +45,23 @@ from fastapi import FastAPI, HTTPException, Request, status
 from fastapi.middleware.cors import CORSMiddleware
 from fastapi.responses import JSONResponse, StreamingResponse
 
-from beaver_gateway.backends.capture import TurnCapture
-from beaver_gateway.frontends.accumulate import StreamAccumulator
+from beaver_gateway.core import audit
+from beaver_gateway.core.conversation_store import (
+    diff_and_fork,
+    load_messages,
+    rewrite_messages,
+)
+from beaver_gateway.core.turn_capture import TurnCapture
+from beaver_gateway.core.turn_record import TurnRecord
+from beaver_gateway.frontends._accumulate import StreamAccumulator
+from beaver_gateway.frontends._auth import require_token
+from beaver_gateway.frontends._sse import (
+    KEEPALIVE,
+    SSE_HEADERS,
+    events_with_heartbeat,
+    sse_pack,
+)
 from beaver_gateway.frontends.base import Frontend
-from beaver_gateway.frontends.bearer import require_token
 from beaver_gateway.frontends.markdown import parser, renderer
 from beaver_gateway.frontends.markdown.crossfront import CrossFrontendLogger
 from beaver_gateway.frontends.markdown.files import (
@@ -29,25 +69,12 @@ from beaver_gateway.frontends.markdown.files import (
     reattach_frontmatter,
     write_atomic,
 )
-from beaver_gateway.frontends.markdown.history import (
-    diff_and_fork,
-    load_messages,
-    rewrite_messages,
-)
 from beaver_gateway.frontends.markdown.mirror import FRONTEND, ChatMirror
-from beaver_gateway.frontends.sse import (
-    KEEPALIVE,
-    SSE_HEADERS,
-    events_with_heartbeat,
-    sse_pack,
-)
-from beaver_gateway.frontends.turn_record import TurnRecord
-from beaver_gateway.security import audit
 
 if TYPE_CHECKING:
     from collections.abc import AsyncIterator, Callable
 
-    from beaver_gateway.conversations.kinds import Kind
+    from beaver_gateway.core.kinds import Kind
     from beaver_gateway.frontends.base import GatewayRuntime
     from beaver_gateway.storage.models import Conversation, ConversationBinding
 
@@ -58,17 +85,23 @@ _log = logging.getLogger("beaver_gateway.frontends.markdown")
 __all__ = ["MarkdownFrontend"]
 
 
+# How often we re-render the assistant turn into the .md file while the
+# backend stream is still open. Trades responsiveness (faster updates to
+# Obsidian sync / Raycast tailers) against write amplification. Each
+# ``RawContentBlockStopEvent`` also forces a flush regardless of the
+# timer, so block boundaries always land in the file.
 _STREAM_FLUSH_DEBOUNCE = 0.4
 
+# Debounce for the SSE ``/chat/stream`` path. Network IO is cheaper than
+# atomic file rewrites, so we send updates more frequently — the client
+# wants the lowest possible latency and we control the renderer on the
+# other end (the Obsidian plugin splices deltas into the editor, no
+# disk round-trip).
 _SSE_FLUSH_DEBOUNCE = 0.1
 
 
 class MarkdownFrontend(Frontend):
-    """FastAPI app behind ``POST /chat`` driven by Obsidian-vault files.
-
-    ``_busy`` tracks in-flight files; check-and-add must stay atomic (no
-    ``await`` between them) so concurrent requests reliably lose to 409.
-    """
+    """FastAPI app behind ``POST /chat`` driven by Obsidian-vault files."""
 
     name = FRONTEND
     kinds = ("deep",)
@@ -83,11 +116,6 @@ class MarkdownFrontend(Frontend):
         logged_subdir: str = "_logs",
         chat_path: Callable[[str, str, Path], Path] | None = None,
     ) -> None:
-        """Configure the vault-backed frontend.
-
-        ``chat_path``, if given, overrides where new chat files are created;
-        ``logged_subdir`` holds cross-frontend logs when ``log_all_chats`` is set.
-        """
         self.vault_path = Path(vault_path).expanduser().resolve()
         self.default_agent = default_agent
         self.log_all_chats = log_all_chats
@@ -95,6 +123,10 @@ class MarkdownFrontend(Frontend):
         self.chat_path = chat_path
         self._runtime: GatewayRuntime | None = None
         self._app: FastAPI | None = None
+        # Files currently being processed by an in-flight ``POST /chat``.
+        # Checked-and-added atomically in the request handler (no
+        # ``await`` between the check and the insert) so a concurrent
+        # request reliably loses the race to 409.
         self._busy: set[Path] = set()
         self._crossfront: CrossFrontendLogger | None = None
         self._mirror: ChatMirror | None = None
@@ -117,6 +149,9 @@ class MarkdownFrontend(Frontend):
                 logged_subdir=self.logged_subdir,
                 chat_path=self.chat_path,
             )
+            # Scan the existing logged files synchronously here so the
+            # fingerprint→path map is populated before the first
+            # cross-frontend turn arrives. Cheap: frontmatter-only read.
             self._crossfront.warm_index()
             runtime.turn_log_handlers.append(self._crossfront.handle)
         self._app = self._build_app(runtime)
@@ -140,10 +175,16 @@ class MarkdownFrontend(Frontend):
     async def serve(self) -> None:
         await self.mirror.run()
 
+    # ---- app builder ---------------------------------------------------
+
     def _build_app(self, runtime: GatewayRuntime) -> FastAPI:
-        """CORS is wide open here since auth is bearer-token, not cookie-based."""
         app = FastAPI(title="beaver-gateway / Markdown")
 
+        # ``/chat/stream`` is consumed via ``fetch`` from the Obsidian
+        # plugin (``requestUrl`` can't read a body incrementally), and
+        # ``fetch`` is subject to CORS. Auth is bearer-token so we don't
+        # need credentialed mode; allow any origin and the standard
+        # methods/headers. The other endpoints are happy to ride along.
         app.add_middleware(
             CORSMiddleware,
             allow_origins=["*"],
@@ -185,6 +226,8 @@ class MarkdownFrontend(Frontend):
 
             file_path = self._resolve_path(filename)
 
+            # Atomic check-and-claim: both ops run between awaits, so a
+            # second request can't slip into the same file slot.
             if file_path in self._busy:
                 return JSONResponse(
                     status_code=status.HTTP_409_CONFLICT,
@@ -205,10 +248,12 @@ class MarkdownFrontend(Frontend):
 
         @app.post("/chat/stream")
         async def chat_stream(request: Request) -> Any:
-            """SSE variant of ``/chat``: ``delta`` events, one disk write at the end.
-
-            The 409-conflict response is still plain JSON — the stream hasn't started.
-            """
+            # Same contract as ``/chat`` (bearer auth, identical body),
+            # but the response is ``text/event-stream`` and intermediate
+            # rendered states are pushed as ``delta`` events. The
+            # gateway-side disk write only happens once, at end of turn,
+            # so streaming consumers (Obsidian plugin) and Obsidian Sync
+            # don't fight over the same file mid-stream.
             token_name = await require_token(request, runtime, scope="messages")
             try:
                 body = await request.json()
@@ -231,6 +276,8 @@ class MarkdownFrontend(Frontend):
 
             file_path = self._resolve_path(filename)
 
+            # 409 path stays JSON — the stream hasn't started yet, so
+            # the caller can read it the same way as on ``/chat``.
             if file_path in self._busy:
                 return JSONResponse(
                     status_code=status.HTTP_409_CONFLICT,
@@ -258,6 +305,8 @@ class MarkdownFrontend(Frontend):
 
         return app
 
+    # ---- dispatch ------------------------------------------------------
+
     async def _handle_chat(
         self,
         *,
@@ -268,7 +317,6 @@ class MarkdownFrontend(Frontend):
         content_override: Any,
         agent_override: str | None,
     ) -> Any:
-        """Non-streaming ``/chat`` handler: parse, align, run, persist."""
         write_disk = content_override is None
         if isinstance(content_override, str):
             file_text = content_override
@@ -292,6 +340,8 @@ class MarkdownFrontend(Frontend):
                 "or configure `default_agent`",
             )
 
+            # When the parser produced no messages (file empty / only
+            # frontmatter), there's nothing to dispatch.
         if not parsed.messages:
             return {
                 "status": "nothing_to_do",
@@ -334,6 +384,9 @@ class MarkdownFrontend(Frontend):
             msgs=len(parsed.messages),
         )
 
+        # Resolve / mint the conversation row, align incoming against
+        # stored history, and feed the aligned messages to the backend
+        # - see ``core/conversation_store.py`` for the full rationale.
         conv, conv_external_id, stored_msgs = await self._resolve_conversation(
             runtime=runtime,
             metadata=parsed.metadata,
@@ -384,6 +437,9 @@ class MarkdownFrontend(Frontend):
             message=message,
         )
 
+        # Broadcast our own turn so other handlers (none today, but the
+        # symmetry is worth keeping) see what happened. ``source`` marks
+        # the origin so ``CrossFrontendLogger`` can skip its own files.
         record = TurnRecord(
             agent_name=agent.name,
             input_messages=list(parsed.messages),
@@ -404,7 +460,9 @@ class MarkdownFrontend(Frontend):
             "new_content": new_content,
         }
 
-    async def _handle_chat_streaming(  # noqa: PLR0915
+    # ---- streaming dispatch (SSE) --------------------------------------
+
+    async def _handle_chat_streaming(  # noqa: PLR0915 — mirrors _handle_chat, splitting only doubles read cost
         self,
         *,
         runtime: GatewayRuntime,
@@ -416,9 +474,21 @@ class MarkdownFrontend(Frontend):
     ) -> AsyncIterator[bytes]:
         """SSE counterpart of :meth:`_handle_chat`.
 
-        Errors surface as ``error`` frames (HTTP is already 200 by then); disk
-        is only written once at the end, avoiding a write race with Obsidian Sync.
+        Mirrors the same pipeline (resolve file → parse → resolve agent →
+        run backend → persist), but emits ``event: delta`` frames as the
+        rendered turn grows and a single terminal ``event: done`` /
+        ``event: error``. Errors that ``_handle_chat`` would surface as
+        ``HTTPException`` go out as ``error`` frames here (the HTTP
+        envelope is already 200 by the time the stream starts).
+
+        Intermediate disk writes are deliberately skipped — only the
+        post-stream :meth:`_write_assistant_reply` lands on disk, so the
+        gateway-side vault and the plugin-side editor are the only
+        writers in their respective halves of Obsidian Sync. Final
+        content is identical on both sides, so Sync no-ops.
         """
+        # With ``content`` the plugin is the only writer of the file
+        # (§3.10): the gateway never touches disk in that case.
         write_disk = content_override is None
         if isinstance(content_override, str):
             file_text = content_override
@@ -583,12 +653,18 @@ class MarkdownFrontend(Frontend):
                     or (now - last_flush) >= _SSE_FLUSH_DEBOUNCE
                 ):
                     payload = snapshot()
+                    # Skip duplicate snapshots — e.g. tool_use blocks
+                    # render to the same prefix as before they closed
+                    # (we don't surface the tool-call args in markdown).
                     if payload is not None and payload != last_payload:
                         yield sse_pack("delta", {"new_content": payload})
                         last_payload = payload
                     last_flush = now
-        except Exception as exc:  # noqa: BLE001
+        except Exception as exc:  # noqa: BLE001 — wire any backend failure as an SSE error frame
             _log.exception("backend failed for %s", filename)
+            # Mirror the legacy path: write the last partial + an error
+            # callout to disk so other consumers (logs, file watchers)
+            # see what arrived. The client gets a clean SSE ``error``.
             partial = acc.finalize(model=model)
             new_body = parsed.body
             if partial.content:
@@ -652,6 +728,8 @@ class MarkdownFrontend(Frontend):
             },
         )
 
+    # ---- helpers -------------------------------------------------------
+
     async def _stream_to_file(
         self,
         *,
@@ -664,8 +742,17 @@ class MarkdownFrontend(Frontend):
     ) -> Any:
         """Drain ``events`` into a ``Message``, flushing partials to disk.
 
-        Flushes on each block boundary and on a debounce timer; on backend
-        failure it still flushes a partial + error callout, then re-raises.
+        Flushes happen on each ``RawContentBlockStopEvent`` (natural
+        block boundary, content is markdown-consistent) and on the
+        ``_STREAM_FLUSH_DEBOUNCE`` timer between events. The partial
+        write keeps the as-parsed frontmatter; the post-stream final
+        write in ``_write_assistant_reply`` is what stamps the refreshed
+        fingerprint / agent / conversation_id.
+
+        On backend exception we still flush the last partial and append
+        an error callout, so the human sees both what arrived and why it
+        stopped. The exception propagates so ``_handle_chat`` can map it
+        to a 500.
         """
         acc = StreamAccumulator()
 
@@ -742,8 +829,11 @@ class MarkdownFrontend(Frontend):
     ) -> tuple[Conversation, str, list[dict[str, Any]]]:
         """Resolve the ``deep`` conversation for this file + its stored messages.
 
-        Precedence: frontmatter ``conversation_id`` > existing path binding >
-        an adopted unbound conversation with the same first prompt > a new one.
+        Frontmatter ``conversation_id`` wins; a file that lost it is found
+        by its visible ``(markdown, path)`` binding, then by adopting the
+        one unbound pre-SDK conversation that starts with the same prompt;
+        otherwise a new conversation is created. The binding follows the
+        file: a moved chat re-binds to its new path on the next turn.
         """
         conversations = runtime.conversations
         rel = file_path.relative_to(self.vault_path).as_posix()
@@ -792,8 +882,9 @@ class MarkdownFrontend(Frontend):
     ) -> None:
         """Stamp the DB with the post-turn canonical Anthropic-shape history.
 
-        Combines prior state + new user prompt + the backend's synthesized
-        cycle, falling back to text-only if ``capture`` is empty.
+        Combines the matched/spliced prior state, the new user prompt,
+        and the synthesized assistant/tool cycle from the backend (or a
+        text-only fallback for backends that left ``capture`` empty).
         """
         new_user_msg = {"role": "user", "content": new_user_text}
         synthesized = capture.synthesized_messages or _fallback_synthesized(message)
@@ -815,11 +906,12 @@ class MarkdownFrontend(Frontend):
         )
 
     def _resolve_path(self, filename: str) -> Path:
-        """Resolve ``filename`` under the vault; reject escapes.
-
-        Leading slashes are stripped first — ``Path.__truediv__`` would
-        otherwise discard ``vault_path`` for an absolute ``filename``.
-        """
+        """Resolve ``filename`` under the vault; reject escapes."""
+        # ``filename`` may be relative or absolute; we always anchor
+        # under ``vault_path`` so absolute paths from outside the vault
+        # don't sneak through. ``Path("/foo/bar")`` combined with a
+        # vault path keeps the absolute side; we strip leading slashes
+        # to coerce the rooted form into a relative path before joining.
         rel = filename.lstrip("/")
         if not rel.endswith(".md"):
             rel = rel + ".md"
@@ -836,8 +928,12 @@ class MarkdownFrontend(Frontend):
 def _fallback_synthesized(message: Any) -> list[dict[str, Any]]:
     """Build a single-assistant ``synthesized_messages`` list from a raw ``Message``.
 
-    Fallback for backends that don't populate :class:`TurnCapture` (anthropic
-    HTTP, raycast, …); the reply lands as one canonical-block message.
+    For backends that don't populate a :class:`TurnCapture` (anthropic
+    HTTP, raycast, …) we don't have access to per-tool-cycle
+    granularity, so the assistant reply lands in the DB as one
+    canonical-block message. Tool memory across cache misses would
+    degrade in that case, but those backends don't have the cache-miss
+    re-seed problem to begin with — they manage history client-side.
     """
     content: list[dict[str, Any]] = []
     for block in getattr(message, "content", ()):

src/beaver_gateway/frontends/markdown/mirror.py

@@ -1,8 +1,13 @@
-"""Vault files for ``deep`` conversations that were not typed into a file.
+"""Vault files for ``deep`` conversations that were not typed into a file (§3.10).
 
-``materialize`` creates the file for a conversation spawned elsewhere;
-``run`` appends replies produced by other origins and keeps the canonical
-history in sync so a continuation typed in Obsidian resumes the same session.
+``materialize`` is the markdown frontend's answer to ``spawn(kind=deep)``:
+a new file in the vault with ``agent`` + ``conversation_id`` frontmatter
+and the ``(markdown, path)`` binding. ``run`` tails the gateway bus and
+appends every ``reply`` of a markdown-bound conversation to its file -
+the seed turn of a spawn, a message posted through ``/api``, a turn
+that came in over ``/v1/messages`` - and stamps the same exchange into
+the canonical history, so a continuation typed in Obsidian aligns
+against the store and resumes the same SDK session instead of reseeding.
 """
 
 from __future__ import annotations
@@ -14,6 +19,8 @@ from typing import TYPE_CHECKING, Any
 
 import frontmatter
 
+from beaver_gateway.core.conversation_store import load_messages, rewrite_messages
+from beaver_gateway.core.turn_record import slugify
 from beaver_gateway.frontends.markdown import renderer
 from beaver_gateway.frontends.markdown.crossfront import strip_trailing_user_scaffold
 from beaver_gateway.frontends.markdown.files import (
@@ -21,14 +28,12 @@ from beaver_gateway.frontends.markdown.files import (
     reattach_frontmatter,
     write_atomic,
 )
-from beaver_gateway.frontends.markdown.history import load_messages, rewrite_messages
-from beaver_gateway.frontends.turn_record import slugify
 
 if TYPE_CHECKING:
     from collections.abc import Callable, Sequence
     from pathlib import Path
 
-    from beaver_gateway.events.bus import Event
+    from beaver_gateway.core.bus import Event
     from beaver_gateway.frontends.base import GatewayRuntime
     from beaver_gateway.storage.models import Conversation, ConversationBinding
 

src/beaver_gateway/frontends/markdown/parser.py

@@ -1,7 +1,18 @@
 """Parse a markdown chat file into Anthropic ``MessageParam`` history.
 
-Turn markers are ``### User:`` / ``### Assistant:`` H3 headers; backend
-input drops thinking/tool callouts, keeping assistant turns text-only.
+The file format is documented in ``frontends/markdown/__init__.py``:
+``### User:`` / ``### Assistant:`` H3 headers split turns, optional
+``---`` HRs between turns are visual-only, ``> [!thinking]-`` and
+``> [!tool]- <name>`` callouts mark structured assistant content.
+
+For backend consumption we strip thinking and tool_use callouts —
+assistant turns become text-only. Rationale: history replay through
+claude-code's JSONL injection only needs the *narrated* answer (the
+thinking signatures expire and the original tool_results aren't
+captured in the renderer's output, so a faithful tool_use round-trip
+isn't possible today). The renderer keeps callouts in the file because
+they're informational for the human reader; the parser drops them when
+shaping the backend's input.
 """
 
 from __future__ import annotations
@@ -40,7 +51,11 @@ class TextSegment:
 class ToolSegment:
     """A ``> [!tool]- <name>`` callout placeholder.
 
-    Only the tool name is captured; the summary suffix is decorative.
+    Only the tool ``name`` is captured — the " · summary" suffix on the
+    callout title and the JSON body inside the quote block are
+    decorative for the human reader; the canonical tool_use block lives
+    in the DB and is keyed by *position+name* against the structure
+    parsed here.
     """
 
     name: str
@@ -49,11 +64,26 @@ class ToolSegment:
 AssistantSegment = TextSegment | ToolSegment
 
 
+# Turn marker — must be exactly ``### User:`` or ``### Assistant:`` on
+# its own line. Trailing whitespace tolerated; nothing after the colon
+# on the same line (any inline content would mean the user typed
+# something that just happens to look like a header, and we'd rather
+# misparse than silently fold inline content into a turn).
 _TURN_RE = re.compile(r"^###\s+(User|Assistant):\s*$", re.MULTILINE)
 
+# Callout-start lines we strip from assistant turns when extracting
+# text. We don't try to parse the contents — for backend input we just
+# need to drop the whole quoted block.
 _CALLOUT_START_RE = re.compile(r"^>\s+\[!(thinking|tool)\]")
 
+# Tool-callout title line: ``> [!tool]- <name>`` or ``> [!tool]- <name> · <summary>``.
+# We only need the ``<name>`` part for skeleton matching; the summary is
+# decorative (built by ``renderer.summarize_tool_input`` from inputs the
+# user can edit visually without semantic consequence).
 _TOOL_TITLE_RE = re.compile(r"^>\s+\[!tool\]-\s*(.*?)\s*$")
+# Renderer joins name + summary with " · " (U+00B7) — see
+# ``renderer.summarize_tool_input``. We split on it to recover the
+# bare tool name.
 _TOOL_TITLE_SEP = " · "
 
 
@@ -61,8 +91,14 @@ _TOOL_TITLE_SEP = " · "
 class ParsedTurn:
     """One turn extracted from the chat file.
 
-    ``text`` is spoken content only; ``structure`` (assistant turns only)
-    carries the ordered text/tool segments used to align with stored history.
+    ``role`` is ``"user"`` or ``"assistant"``. ``text`` is the spoken
+    content with callouts stripped and HRs dropped — used both as the
+    backend's ``MessageParam.content`` (back-compat with the existing
+    parser shape) and as the diff key against stored turns.
+    ``structure`` is non-empty only for assistant turns: an ordered
+    list of ``TextSegment`` / ``ToolSegment`` reflecting the visible
+    layout of the assistant block, used by the conversation store to
+    align with the canonical tool_use blocks held in DB.
     """
 
     role: str
@@ -74,8 +110,16 @@ class ParsedTurn:
 class ParsedFile:
     """Result of parsing a single chat ``.md``.
 
-    ``messages`` is text-only history for the backend; ``turns`` carries
-    per-turn structure; ``body`` is the raw content after frontmatter.
+    ``metadata`` is the YAML frontmatter as a plain dict (empty if the
+    file has none). ``messages`` is the conversation history shaped for
+    ``Backend.complete`` — assistant turns are text-only. ``turns`` is
+    1:1 with ``messages`` and carries the per-turn structure (for
+    assistant turns) that the conversation store needs to detect
+    text-only edits vs. structural forks. ``body`` is the raw markdown
+    content *after* the frontmatter is stripped; the renderer needs it
+    when it appends a new assistant turn so it can preserve whatever
+    the human typed verbatim (including any callouts or HRs they
+    added).
     """
 
     metadata: dict[str, Any]
@@ -87,9 +131,15 @@ class ParsedFile:
 def parse(text: str) -> ParsedFile:
     """Parse a chat ``.md`` into ``(metadata, body, messages, turns)``.
 
-    A bare file (no turn markers) is treated as a single user turn; a
-    tool-only assistant turn gets placeholder ``" "`` content since the
-    backend rejects an empty string.
+    A file with no turn markers but non-empty body is treated as a
+    single user turn — the friendly path for "user types into a new
+    file and hits send" before any turn markers exist.
+
+    Assistant turns that have *only* tool callouts (no spoken text) are
+    preserved here even though their ``MessageParam.content`` is empty
+    — the structure carries tool-segment information the conversation
+    store needs for skeleton matching. The renderer in practice always
+    emits at least a trailing text block, so this branch is defensive.
     """
     parsed = frontmatter.loads(text)
     metadata = dict(parsed.metadata)
@@ -125,6 +175,14 @@ def parse(text: str) -> ParsedFile:
                     )
                 )
             elif has_tools:
+                # Tool-only assistant turn: nothing to feed the backend
+                # as ``content`` (it'd reject an empty string), but the
+                # structure must survive so the store can align it
+                # against stored tool_use blocks. We synthesize a
+                # single-space text content for backend round-trip; the
+                # conversation store will replace this payload with the
+                # canonical stored blocks before the backend ever sees
+                # it on a continuation.
                 messages.append({"role": "assistant", "content": " "})
                 parsed_turns.append(
                     ParsedTurn(role="assistant", text="", structure=tuple(structure))
@@ -138,8 +196,18 @@ def parse(text: str) -> ParsedFile:
 def parse_assistant_structure(raw: str) -> list[TextSegment | ToolSegment]:
     """Walk an assistant turn body, return its ordered text/tool segments.
 
-    Tool callouts become :class:`ToolSegment` (name only); thinking callouts
-    and HR lines are stripped; boundary-empty text segments are dropped.
+    Tool callouts become :class:`ToolSegment` with just the tool name —
+    the title's optional ``" · summary"`` suffix and the JSON body
+    inside the quote block are decorative; the canonical tool_use
+    block is held in the conversation store. Thinking callouts are
+    stripped entirely (they were never round-trippable through the
+    file — signatures expire). HR separator lines drop out.
+
+    Empty / whitespace-only text segments at the boundaries (start,
+    end, between adjacent tool callouts) are dropped so the skeleton
+    is robust against renderer whitespace choices; a non-empty text
+    segment with surrounding whitespace is trimmed on both ends but
+    preserved.
     """
     segments: list[TextSegment | ToolSegment] = []
     pending_text: list[str] = []
@@ -148,6 +216,9 @@ def parse_assistant_structure(raw: str) -> list[TextSegment | ToolSegment]:
         if not pending_text:
             return
         joined = "\n".join(pending_text)
+        # Collapse runs of >2 blank lines (created when we stripped a
+        # mid-block callout) into one so the diff against a re-render
+        # is stable.
         cleaned = re.sub(r"\n{3,}", "\n\n", joined).strip()
         pending_text.clear()
         if cleaned:
@@ -160,6 +231,7 @@ def parse_assistant_structure(raw: str) -> list[TextSegment | ToolSegment]:
         callout_match = _CALLOUT_START_RE.match(line)
         if callout_match:
             kind = callout_match.group(1)
+            # Capture tool name *before* advancing past the block.
             if kind == "tool":
                 title_match = _TOOL_TITLE_RE.match(line)
                 title = title_match.group(1) if title_match else ""
@@ -167,7 +239,9 @@ def parse_assistant_structure(raw: str) -> list[TextSegment | ToolSegment]:
                 _flush_text()
                 segments.append(ToolSegment(name=name))
             else:
+                # Thinking callout — drop the whole block, emit nothing.
                 _flush_text()
+            # Skip the rest of the quote block.
             while i < len(lines) and lines[i].lstrip().startswith(">"):
                 i += 1
             continue
@@ -183,7 +257,11 @@ def parse_assistant_structure(raw: str) -> list[TextSegment | ToolSegment]:
 def _segments_to_spoken_text(segments: list[TextSegment | ToolSegment]) -> str:
     r"""Reduce a structure list to the spoken-text view the backend sees.
 
-    Concatenates text segments with ``\n\n``, dropping tool segments.
+    Concatenates :class:`TextSegment` contents with ``\n\n`` between
+    them, dropping :class:`ToolSegment` entries. Equivalent to what
+    the pre-Conversation-store parser did — we keep that behavior so
+    existing fingerprints (frontmatter ``fingerprint`` field) stay
+    valid.
     """
     chunks = [s.text for s in segments if isinstance(s, TextSegment)]
     return "\n\n".join(c for c in chunks if c).strip()
@@ -212,11 +290,15 @@ def resolve_agent(
     return default
 
 
+# ---- internals ---------------------------------------------------------
+
+
 def _split_turns(body: str) -> list[tuple[str, str]]:
     """Walk turn markers, return ``[(role_lc, raw_body), ...]``.
 
-    A marker is an exact ``### User:`` / ``### Assistant:`` line; trailing
-    content after the colon means it's not a marker, not a turn boundary.
+    Body for each turn is everything between this marker and the next
+    (or EOF). Leading marker line itself is dropped. We don't trim
+    whitespace here — that's per-role.
     """
     matches = list(_TURN_RE.finditer(body))
     if not matches:
@@ -233,8 +315,9 @@ def _split_turns(body: str) -> list[tuple[str, str]]:
 def _strip_hrs(raw: str) -> str:
     """Drop decorative ``---`` separator lines (whole-line HRs only).
 
-    Only lines that are exactly ``---`` (with optional surrounding
-    whitespace) are removed; a ``---`` mid-paragraph stays.
+    A ``---`` mid-paragraph (rare, but possible) stays. Only lines that
+    are *exactly* the HR after optional surrounding whitespace are
+    removed — those are the ones the renderer emits between turns.
     """
     lines = raw.splitlines()
     kept = [ln for ln in lines if ln.strip() != "---"]

src/beaver_gateway/frontends/markdown/renderer.py

@@ -1,7 +1,10 @@
-"""Render Anthropic ``Message`` (and user turns) into markdown.
+"""Render Anthropic ``Message`` (and individual user turns) into markdown.
 
-One-way: produces the human-facing artifact only. The parser strips
-tool/thinking callouts separately when reshaping history for backend replay.
+The renderer is one-way: it produces the human-facing artifact in the
+vault. The parser strips tool/thinking callouts when reshaping the file
+for backend replay — so what we write here is purely for the human
+reader (and for the cross-frontend logger, which materializes turns
+from other frontends).
 """
 
 from __future__ import annotations
@@ -27,11 +30,23 @@ __all__ = [
 ]
 
 
+# Empty ``### User:`` block appended after each assistant reply so the
+# human has an obvious place to type the next turn. Parser drops empty
+# user blocks, so this doesn't re-trigger dispatch on its own.
+# Blank line after the header, like every rendered turn - the file stays
+# symmetric whether the human or the gateway wrote the marker.
 USER_SCAFFOLD = "### User:\n\n"
 
 
+# Default 4-backtick fence so tool results that contain literal ```` ``` ````
+# don't collide. JSON inputs use 3 backticks because they almost never
+# contain ``` and we get language syntax highlighting in Obsidian for free.
 FENCE = "````"
 
+# Input keys we dangle after the tool name in the callout title, best
+# first — purely cosmetic. ``description`` wins because when a tool
+# offers one it's a human-written summary of the call, which beats a
+# truncated shell command or path.
 _TITLE_KEYS = ("description", "path", "file", "filename", "url", "command", "query")
 
 
@@ -46,21 +61,25 @@ def render_assistant_text(text: str) -> str:
 
 
 def render_assistant_message(message: Message) -> str:
-    """Render an assistant ``Message`` into a turn block.
+    """Render an assistant ``Message`` (with content blocks) into a turn block.
 
-    Tool-use blocks render to nothing; of the thinking blocks between two
-    texts only the first one reaches the file.
+    Blocks render in their original order:
+
+    * ``ThinkingBlock`` → ``> [!thinking]-`` collapsed callout
+    * ``TextBlock`` → plain text (the spoken answer)
+    * ``ToolUseBlock`` → nothing (§3.10: tool calls never reach the file;
+      "what the agent is doing" is the activity panel fed by SSE)
+
+    Blank lines separate adjacent blocks; trailing newline guarantees
+    the next ``---`` / ``### User:`` marker lands on its own line.
     """
     parts: list[str] = ["### Assistant:", ""]
-    after_thinking = False
     for block in message.content:
-        thinking = isinstance(block, ThinkingBlock)
-        if thinking and after_thinking:
-            continue
         lines = list(_render_block(block))
         if not lines:
+            # Tool calls render to nothing - no separator for them either,
+            # or every tool leaves a blank line behind.
             continue
-        after_thinking = thinking
         parts.extend(lines)
         parts.append("")
     return "\n".join(parts).rstrip() + "\n"
@@ -69,12 +88,19 @@ def render_assistant_message(message: Message) -> str:
 def render_user_param(param: MessageParam) -> str:
     """Render a ``MessageParam`` user message into a ``### User:`` block.
 
-    Tool_result blocks in the content list are dropped silently.
+    Used by the cross-frontend logger when materializing turns from
+    other frontends. Tool_result blocks in the content list are dropped
+    silently — the markdown view doesn't track them (see ``parser.py``).
     """
     content = param.get("content", "")
     if isinstance(content, str):
         text = content
     else:
+        # The Anthropic SDK types ``content`` as a union of typed-dict
+        # *Param classes plus pydantic block models — both shapes appear
+        # in practice (raw incoming JSON yields dicts, SDK-built params
+        # yield BaseModels). Treat each entry as a dict-like and pull
+        # ``text`` opportunistically.
         chunks = [
             str(blk.get("text", ""))
             for blk in content
@@ -85,9 +111,11 @@ def render_user_param(param: MessageParam) -> str:
 
 
 def append_to_body(existing: str, new_block: str) -> str:
-    """Append ``new_block`` to ``existing`` with a decorative ``---`` separator.
+    """Append ``new_block`` to ``existing`` with a decorative HR separator.
 
-    The separator is visual only; the parser ignores it.
+    Preserves the original body verbatim (whitespace, callouts, any
+    formatting the human added). The HR is purely visual: parser ignores
+    it.
     """
     head = existing.rstrip()
     if head:
@@ -96,9 +124,18 @@ def append_to_body(existing: str, new_block: str) -> str:
 
 
 def summarize_tool_input(name: str, tool_input: object) -> str:
-    """Build the ``[!tool]- <summary>`` title, picking one salient input field."""
+    """Build the ``[!tool]- <summary>`` title string.
+
+    Tries to pick a single salient field (``description``, ``path``,
+    ``command``, etc.) from the input dict so the collapsed callout shows
+    something meaningful in Obsidian. Falls back to just the tool name.
+    """
     if not isinstance(tool_input, dict):
         return name
+    # Anthropic ``ToolUseBlock.input`` is typed as ``object`` — the
+    # SDK's runtime value is always a JSON dict (str→Any), so a local
+    # cast keeps the rest of the function readable without sprinkling
+    # per-line type narrowing on every ``.get`` call.
     d = cast("dict[str, Any]", tool_input)
     for key in _TITLE_KEYS:
         value = d.get(key)
@@ -108,6 +145,9 @@ def summarize_tool_input(name: str, tool_input: object) -> str:
     return name
 
 
+# ---- internals ---------------------------------------------------------
+
+
 def _render_block(block: object) -> Iterable[str]:
     if isinstance(block, TextBlock):
         text = (block.text or "").strip()
@@ -117,6 +157,7 @@ def _render_block(block: object) -> Iterable[str]:
     if isinstance(block, ThinkingBlock):
         yield from _render_thinking(block.thinking or "")
         return
+    # Tool-use blocks and unknown block types never reach the file.
 
 
 def _render_thinking(text: str) -> Iterable[str]:
@@ -126,7 +167,12 @@ def _render_thinking(text: str) -> Iterable[str]:
 
 
 def adaptive_fence(content: str) -> str:
-    """Return a backtick fence longer than any backtick run in ``content``."""
+    """Return a backtick fence at least one longer than the longest run in ``content``.
+
+    Currently unused (tool *results* aren't persisted yet) — kept here
+    so when result capture lands the rendering side already has the
+    primitive.
+    """
     longest = 0
     for match in re.finditer(r"`+", content):
         longest = max(longest, len(match.group(0)))

src/beaver_gateway/frontends/mcp_server.py

@@ -1,7 +1,36 @@
-"""External MCP frontend.
+"""External MCP frontend (Phase 3.1).
 
-Streamable-HTTP gateway that reverse-proxies the internal MCP aggregator,
-adding bearer auth, an audit log line per request, and a discovery page.
+A streamable-HTTP gateway in front of the internal MCP aggregator
+(``beaver_gateway.mcp.internal_app``). The aggregator hosts every
+declared ``McpServer`` (``python_tool``, stdio proxy, HTTP proxy)
+under ``/mcp/<name>`` plus a flat ``/mcp/all`` bundle on
+``127.0.0.1:INTERNAL_MCP_PORT`` — that's the *internal* shape.
+
+This frontend re-exposes those namespaces on its own port directly at
+``/<name>/`` (no ``/mcp/`` prefix in the external routes — the port
+itself already disambiguates). Caddy / nginx / Cloudflare in front
+typically strips a prefix back on: ``domain.com/mcp/* → :8001/*``,
+controlled by the operator's reverse-proxy config and surfaced to the
+admin dashboard via ``public_base_url``. Three additions on top of the
+raw aggregator:
+
+* **Bearer auth** — ``Authorization: Bearer <token>``, ``X-Api-Key``,
+  or ``?token=<…>`` query string. All three forms verify against the
+  same :class:`TokenStore` as ``AnthropicMessagesFrontend``.
+* **Audit log** — one line per request (token name, namespace,
+  request method/path, response status). The DB-backed audit log lives
+  in Phase 4; for now we just emit a structured log line.
+* **Discovery page** at ``GET /`` (auth-gated) — HTML rendered with a
+  tiny inline Jinja2 template listing every namespace plus copy-pastable
+  config snippets for Cursor / claude.ai / Claude Desktop.
+
+Why a reverse proxy and not a second mount? FastMCP's session managers
+are tied to the lifespan they were created in; running the same
+aggregator under two uvicorn servers double-initializes state. Building
+two parallel aggregators would double upstream connections (two
+subprocesses for every stdio MCP, two HTTP clients for every remote).
+A loopback proxy keeps one source of truth — the internal aggregator —
+and lets us layer policy on the outside.
 """
 
 from __future__ import annotations
@@ -16,10 +45,10 @@ from starlette.applications import Starlette
 from starlette.responses import HTMLResponse, JSONResponse, StreamingResponse
 from starlette.routing import Route
 
+from beaver_gateway.core import audit
+from beaver_gateway.frontends._urls import external_base
 from beaver_gateway.frontends.base import Frontend
-from beaver_gateway.frontends.urls import external_base
 from beaver_gateway.mcp.internal_app import ALL_NAMESPACE
-from beaver_gateway.security import audit
 
 if TYPE_CHECKING:
     from collections.abc import AsyncIterator, Mapping
@@ -32,6 +61,10 @@ if TYPE_CHECKING:
 _log = logging.getLogger("beaver_gateway.frontends.mcp_server")
 
 
+# Hop-by-hop headers that must NOT be forwarded across an HTTP proxy
+# (RFC 7230 §6.1). Bypassing this filter would break chunked transfer
+# encoding when ``Content-Length`` arrives, or upstream-aware proxies
+# would refuse the second hop's connection-pool reuse.
 _HOP_BY_HOP_HEADERS = frozenset(
     {
         "connection",
@@ -47,6 +80,11 @@ _HOP_BY_HOP_HEADERS = frozenset(
     }
 )
 
+# Standard auth-bearing headers we *do not* forward to the internal app —
+# the internal app is on loopback with no auth of its own, and forwarding
+# the inbound bearer would only confuse it. Each method-specific MCP
+# request from the upstream Cursor/etc. carries a fresh ``mcp-session-id``
+# that we *must* forward.
 _AUTH_HEADERS = frozenset({"authorization", "x-api-key"})
 
 
@@ -81,16 +119,17 @@ class McpServerFrontend(Frontend):
             self._http = None
 
     def _build_app(self, runtime: GatewayRuntime) -> Starlette:  # noqa: ARG002
-        """Build the Starlette app.
-
-        Literal routes (``/``, ``/healthz``) are listed before the
-        namespace wildcard so they win the match instead of being
-        swallowed by it; two routes per namespace cover both ``/x`` and
-        ``/x/y`` since Starlette won't fold them into one.
-        """
         routes = [
             Route("/", self._discovery, methods=["GET"]),
             Route("/healthz", self._healthz, methods=["GET"]),
+            # Namespaces mount at the root of this port — the port
+            # itself already disambiguates this from any other gateway
+            # surface. Two routes per namespace so both the
+            # trailing-slash and sub-path forms work (``/time`` AND
+            # ``/time/foo``); Starlette doesn't fold them into one
+            # route automatically. The literal routes above (``/``,
+            # ``/healthz``) are listed first and win the match, so
+            # they're not eaten by ``/{namespace}``.
             Route(
                 "/{namespace}",
                 self._proxy_endpoint,
@@ -112,7 +151,7 @@ class McpServerFrontend(Frontend):
         token_name, err = await _verify_request(request, runtime)
         if err is not None:
             return err
-        assert token_name is not None  # noqa: S101
+        assert token_name is not None  # noqa: S101 — narrow for ty
         base = external_base(request, runtime)
         html = _render_discovery_page(
             base_url=base, namespaces=list(runtime.mcps), actor=token_name
@@ -126,7 +165,7 @@ class McpServerFrontend(Frontend):
         token_name, err = await _verify_request(request, runtime)
         if err is not None:
             return err
-        assert token_name is not None  # noqa: S101
+        assert token_name is not None  # noqa: S101 — narrow for ty
 
         namespace = request.path_params["namespace"]
         subpath = request.path_params.get("path", "")
@@ -162,12 +201,11 @@ class McpServerFrontend(Frontend):
         )
 
     def _upstream_url(self, namespace: str, subpath: str) -> str | None:
-        """Resolve ``namespace`` to its internal loopback URL.
-
-        ``ALL_NAMESPACE`` isn't in the URL map, so its URL is synthesized
-        from any per-domain entry's authority.
-        """
         runtime = self._require_runtime()
+        # ``all`` is built by the aggregator unconditionally when at least
+        # one MCP is configured; the URL map only contains per-domain
+        # entries (see ``build_internal_app``), so we synthesize ``all``'s
+        # loopback URL from any per-domain URL's authority.
         if namespace == ALL_NAMESPACE:
             sample = next(iter(runtime.mcp_internal_urls.values()), None)
             if sample is None:
@@ -194,8 +232,13 @@ async def _verify_request(
 ) -> tuple[str | None, JSONResponse | None]:
     """Accept ``Authorization: Bearer``, ``X-Api-Key``, or ``?token=``.
 
-    Returns ``(actor_name, None)`` on success, else ``(None, error_response)``
-    — 401 for a missing/unknown token, 403 for one whose scope doesn't cover this call.
+    The third form is the escape hatch for clients that can only put
+    secrets in the URL (claude.ai's MCP config historically did this).
+    All three roads end at the same :class:`TokenStore`. Returns
+    ``(actor_name, None)`` on success, ``(None, 401|403)`` otherwise
+    — the caller forwards the response as-is. Splitting auth vs scope
+    failures matters: 401 says "send me a token", 403 says "this token
+    is real but not for this endpoint".
     """
     api_key = request.headers.get("x-api-key")
     if api_key:
@@ -233,7 +276,9 @@ def _forbidden(scope: str, required: str) -> JSONResponse:
 def _join_subpath(base_url: str, subpath: str) -> str:
     """Concatenate the loopback URL with the proxied sub-path.
 
-    ``base_url`` always ends in ``/`` to avoid Starlette's redirect dance.
+    ``base_url`` always ends in ``/`` (the aggregator publishes URLs
+    that way to avoid Starlette's 307 redirect dance); the sub-path is
+    appended verbatim, with the query string handled by the caller.
     """
     if subpath:
         return base_url + subpath.lstrip("/")
@@ -249,14 +294,18 @@ async def _reverse_proxy(
     actor: str,
     runtime: GatewayRuntime,
 ) -> StreamingResponse | JSONResponse:
-    """Bidirectionally stream an MCP request between client and internal aggregator.
+    """Bidirectionally stream an MCP request between client ↔ internal aggregator.
 
-    The audit row is written right after the upstream response headers
-    arrive, before relaying its body, so a client-truncated stream is
-    still audited.
+    Streamable-HTTP MCP responses can be a long-running SSE stream
+    (tools that emit partial progress) or a one-shot JSON body; we
+    don't peek — just relay chunks as they arrive in either direction
+    until both sides close.
     """
     qs = request.url.query
     if qs:
+        # Drop ``?token=`` from the forwarded URL — internal app doesn't
+        # need it, and propagating creds further than necessary widens
+        # the leak surface (logs, metrics, traces all see query strings).
         scrubbed = _scrub_query(qs, drop={"token"})
         if scrubbed:
             upstream_url = f"{upstream_url}?{scrubbed}"
@@ -302,6 +351,9 @@ async def _reverse_proxy(
         request.url.path,
         upstream_resp.status,
     )
+    # Audit at upstream-response time: status reflects the MCP call's
+    # outcome (200 / tool-error / 4xx). Streaming relay below may be
+    # cut short by the client, but the row is already in by then.
     await audit.log(
         runtime,
         actor=f"token:{actor}",
@@ -318,6 +370,7 @@ async def _reverse_proxy(
             async for chunk in upstream_resp.content.iter_any():
                 yield chunk
         except (aiohttp.ClientError, asyncio.CancelledError):
+            # Caller hung up or upstream dropped — just stop relaying.
             return
         finally:
             upstream_resp.release()
@@ -337,11 +390,6 @@ async def _request_body_iter(request: Request) -> AsyncIterator[bytes]:
 
 
 def _forward_headers(request: Request) -> dict[str, str]:
-    """Drop hop-by-hop headers (RFC 7230 §6.1) and inbound auth headers.
-
-    The internal aggregator is loopback-only with no auth of its own,
-    so forwarding the caller's bearer would only confuse it.
-    """
     out: dict[str, str] = {}
     for key, value in request.headers.items():
         lowered = key.lower()
@@ -374,7 +422,11 @@ def _scrub_query(query: str, *, drop: frozenset[str] | set[str]) -> str:
 
 
 def _render_discovery_page(*, base_url: str, namespaces: list[Any], actor: str) -> str:
-    """Render the auth-gated namespace + config-snippet page."""
+    """Render the auth-gated namespace + config-snippet page.
+
+    Inline HTML (no Jinja file) — keeps Phase 3 free of template-dir
+    plumbing that Phase 4's AdminFrontend will own.
+    """
     name_list = [getattr(ns, "name", str(ns)) for ns in namespaces]
     rows = (
         "\n".join(
@@ -408,7 +460,7 @@ def _escape(value: str) -> str:
     )
 
 
-_DISCOVERY_TEMPLATE = "\n".join(  # noqa: FLY002
+_DISCOVERY_TEMPLATE = "\n".join(  # noqa: FLY002 — readability beats one-string-blob
     [
         "<!doctype html>",
         '<html lang="en">',

src/beaver_gateway/frontends/root.py

@@ -46,7 +46,7 @@ def build_root_app(
     ]
     for fe in mounted:
         app = fe.app()
-        assert app is not None  # noqa: S101
+        assert app is not None  # noqa: S101 - filtered above; narrows for ty
         assert fe.path is not None  # noqa: S101
         routes.append(Mount(fe.path, app=app, name=fe.name or fe.path.strip("/")))
     for path, app in (extra or {}).items():

src/beaver_gateway/frontends/telegram/__init__.py

@@ -1,6 +1,5 @@
-"""Telegram frontend: General = master, topic = branch, drafts, inbox/outbox."""
+"""Telegram frontend (§3.8): General = master, topic = branch, drafts, inbox/outbox."""
 
 from beaver_gateway.frontends.telegram.frontend import Attachments, TelegramFrontend
-from beaver_gateway.frontends.telegram.texts import TelegramTexts
 
-__all__ = ["Attachments", "TelegramFrontend", "TelegramTexts"]
+__all__ = ["Attachments", "TelegramFrontend"]

src/beaver_gateway/frontends/telegram/drafts.py

@@ -1,8 +1,13 @@
-"""One ``sendMessageDraft`` stream per running turn.
+"""One ``sendMessageDraft`` stream per running turn (§3.8).
 
-Telegram folds a draft into the following message only when their texts are
-identical, so the last push is the final text verbatim. Drafts live 30s and
-throttle to about one edit/s, so this refreshes on a timer, not every delta.
+The client folds a draft into the message that follows only when their
+texts are identical - so the last push is the final text itself, rendered
+exactly as the outbox will send it, with no status line.
+
+A draft is ephemeral and lives 30 s, Telegram throttles edits to about one
+per second per chat, and thinking or a tool call would otherwise look like a
+hang - so the draft opens with a status line straight away, is refreshed on
+a timer rather than on every delta, and is kept alive while nothing changes.
 """
 
 from __future__ import annotations
@@ -38,7 +43,7 @@ class Draft:
         thread_id: int | None,
         turn_id: str,
         interval: float = 0.7,
-        status: str = "⏳ thinking",
+        status: str = "⏳ думаю",
     ) -> None:
         self._bot = bot
         self._chat_id = chat_id
@@ -47,17 +52,12 @@ class Draft:
         self._interval = interval
         self.status = status
         self.text = ""
-        self._paused = False
         self._dirty = True
         self._broken = False
         self._last_sent = 0.0
         self._task: asyncio.Task[None] | None = None
         self._inflight: asyncio.Future[None] | None = None
 
-    @property
-    def draft_id(self) -> int:
-        return self._draft_id
-
     def start(self) -> None:
         if self._task is None:
             self._task = asyncio.create_task(self._run())
@@ -66,22 +66,11 @@ class Draft:
         if status != self.status:
             self.status = status
             self._dirty = True
-            self._paused = bool(self.text.strip())
 
     def append(self, text: str) -> None:
-        """Add streamed text; a tool call in between starts a new paragraph.
-
-        The model ends a sentence, calls a tool, then goes on without a
-        break - the same text the final message shows as paragraphs.
-        """
-        if not text:
-            return
-        if self._paused and not self.text.endswith("\n"):
-            self.text = self.text.rstrip() + "\n\n" + text.lstrip()
-        else:
+        if text:
             self.text += text
-        self._paused = False
-        self._dirty = True
+            self._dirty = True
 
     async def stop(self) -> None:
         """Stop pushing and wait for the push in flight.
@@ -142,7 +131,6 @@ class Draft:
             message_thread_id=self._thread_id,
             text=text,
             parse_mode=parse_mode,
-            can_stop=True,
         )
 
     def _render(self) -> tuple[str, str]:

src/beaver_gateway/frontends/telegram/frontend.py

@@ -1,8 +1,13 @@
-"""``TelegramFrontend`` - the private chat with the bot as the window.
+"""``TelegramFrontend`` - the private chat with the bot as the window (§3.8).
 
-A private chat with topics has no General, so the gateway makes and rebinds
-one topic for the master; any other topic is a branch, and a message into a
-new one spawns it. Replies stream as drafts and land through the outbox.
+A private chat with topics has no General: the gateway makes one topic for
+the master (``master_topic``) and rebinds it to every new master; any other
+topic is a branch. The user makes a topic and the
+first message in it spawns the branch (``seed=morning``); a message into a
+topic whose branch is merged or closed spawns a new branch on the same
+topic. Replies stream as drafts and land through the outbox; turns that
+came from other windows are mirrored with a marker; ``origin=system`` is
+never shown. ``AskUserQuestion`` becomes inline buttons (§3.7).
 """
 
 from __future__ import annotations
@@ -21,13 +26,10 @@ from zoneinfo import ZoneInfo
 
 from aiogram import Bot
 from aiogram.client.default import DefaultBotProperties
-from aiogram.client.session.aiohttp import AiohttpSession
-from aiogram.client.telegram import TelegramAPIServer
-from aiogram.exceptions import TelegramAPIError, TelegramNetworkError
+from aiogram.exceptions import TelegramAPIError
 from aiogram.types import (
     BotCommand,
     CallbackQuery,
-    FSInputFile,
     InlineKeyboardButton,
     InlineKeyboardMarkup,
     Message,
@@ -36,23 +38,15 @@ from aiogram.types import (
 )
 
 from beaver_gateway.frontends.base import Frontend
-from beaver_gateway.frontends.files import (
-    DEFAULTS,
-    SendFileError,
-    SendFiles,
-    mb,
-    method_for,
-)
 from beaver_gateway.frontends.telegram.drafts import Draft
 from beaver_gateway.frontends.telegram.inbox import Inbox
 from beaver_gateway.frontends.telegram.outbox import Outbox
-from beaver_gateway.frontends.telegram.render import chunks, status_label, to_html
-from beaver_gateway.frontends.telegram.texts import TelegramTexts
+from beaver_gateway.frontends.telegram.render import chunks, status_label
 
 if TYPE_CHECKING:
-    from beaver_gateway.conversations.kinds import Kind
-    from beaver_gateway.conversations.service import Conversations
-    from beaver_gateway.events.bus import Event, EventBus
+    from beaver_gateway.core.bus import Event, EventBus
+    from beaver_gateway.core.conversations import Conversations
+    from beaver_gateway.core.kinds import Kind
     from beaver_gateway.frontends.base import GatewayRuntime
     from beaver_gateway.storage.models import Conversation, ConversationBinding
 
@@ -62,17 +56,28 @@ _log = logging.getLogger("beaver_gateway.frontends.telegram")
 
 FRONTEND = "telegram"
 _DONE = "done"
-_COMMANDS = ("merge", "new", "chat", "status", "help", "start")
-_CAPTION = 1024
-_NO_CAPTION = ("video_note",)
+_COMMAND_HELP = (
+    ("merge", "слить ветку в мастер"),
+    ("new", "новая ветка: в General - новый топик, в топике - заново на нём"),
+    ("chat", "открыть глубокий чат в vault: /chat тема"),
+    ("status", "что с этим разговором"),  # noqa: RUF001
+    ("help", "команды"),
+)
+_COMMANDS = (*(c for c, _ in _COMMAND_HELP), "start")
+_HELP = "General - мастер, любой другой топик - ветка.\n" + "\n".join(
+    f"/{c} - {d}" for c, d in _COMMAND_HELP
+)
 
 
 @dataclass(frozen=True, slots=True)
 class Attachments:
-    """Where files land.
+    """Where files from Telegram go.
 
-    ``ephemeral`` uses the gateway's data dir, ``vault`` an inbox under
-    ``dir``; ``keep_days`` sweeps older files, ``None`` never sweeps.
+    Files land in ``<root>/YYYY-MM-DD/<unixts>-<name>`` (the day in ``tz``)
+    and whatever is older than ``keep_days`` is swept; ``None`` never
+    sweeps. ``ephemeral`` - under the gateway's data dir; ``vault`` -
+    ``dir`` is an inbox inside the agent's zone: the agent moves keepers
+    next to the note, the rest is swept after ``keep_days``.
     """
 
     mode: Literal["ephemeral", "vault"] = "ephemeral"
@@ -107,46 +112,10 @@ class _Ask:
     picked: dict[int, list[str]] = field(default_factory=dict)
     done: set[int] = field(default_factory=set)
 
-    def to_flags(self, question_id: str) -> dict[str, Any]:
-        return {
-            "id": question_id,
-            "conversation_id": self.conversation_id,
-            "chat_id": self.chat_id,
-            "thread_id": self.thread_id,
-            "questions": self.questions,
-            "messages": self.messages,
-            "picked": {str(k): v for k, v in self.picked.items()},
-            "done": sorted(self.done),
-        }
-
-    @classmethod
-    def from_flags(cls, data: dict[str, Any]) -> _Ask:
-        return cls(
-            conversation_id=str(data["conversation_id"]),
-            chat_id=int(data["chat_id"]),
-            thread_id=data.get("thread_id"),
-            questions=list(data.get("questions") or []),
-            messages=list(data.get("messages") or []),
-            picked={int(k): list(v) for k, v in (data.get("picked") or {}).items()},
-            done=set(data.get("done") or []),
-        )
-
-
-@dataclass
-class _Album:
-    message: Message
-    thread_id: int | None
-    is_master: bool
-    texts: list[str] = field(default_factory=list)
-    notes: list[str] = field(default_factory=list)
-    attachments: list[dict[str, Any]] = field(default_factory=list)
-    task: asyncio.Task[None] | None = None
-
 
 class TelegramFrontend(Frontend):
     name = FRONTEND
     kinds = ("master", "branch")
-    sends_files = True
 
     def __init__(
         self,
@@ -157,25 +126,18 @@ class TelegramFrontend(Frontend):
         branch_agent: str | None = None,
         chat_id: int | None = None,
         attachments: Attachments = EPHEMERAL,
-        send_files: SendFiles = DEFAULTS,
-        api_base_url: str | None = None,
         draft_interval: float = 0.7,
         master_topic: str = "🦫 General",
         queued_reaction: str = "👀",
         poll_timeout: int = 30,
         outbox_backoff: float = 2.0,
-        texts: TelegramTexts | None = None,
-        album_delay: float = 1.0,
     ) -> None:
         self._token = token
-        self.texts = texts or TelegramTexts()
         self.user_id = user_id
         self.chat_id = chat_id if chat_id is not None else user_id
         self.master_agent = master_agent
         self.branch_agent = branch_agent
         self.attachments = attachments
-        self.send_files = send_files
-        self.api_base_url = api_base_url
         self.master_topic = master_topic
         self.draft_interval = draft_interval
         self.queued_reaction = queued_reaction
@@ -189,12 +151,12 @@ class TelegramFrontend(Frontend):
         self._master_window: str | None = None
         self._topic_names: dict[int, str] = {}
         self._drafts: dict[str, Draft] = {}
-        self.album_delay = album_delay
         self._asks: dict[str, _Ask] = {}
-        self._albums: dict[str, _Album] = {}
         self._reactions: dict[int, tuple[int, int]] = {}
         self._tasks: set[asyncio.Task[None]] = set()
 
+    # ---- Frontend --------------------------------------------------------
+
     def agent_for(self, kind: Kind) -> str | None:
         return {"master": self.master_agent, "branch": self.branch_agent}.get(kind)
 
@@ -204,14 +166,7 @@ class TelegramFrontend(Frontend):
             raise RuntimeError(msg)
         self._runtime = runtime
         if self._bot is None:
-            self._bot = Bot(
-                self._token,
-                session=self._session(),
-                default=DefaultBotProperties(parse_mode=None),
-            )
-        self.send_files = self.send_files.with_roots(
-            *runtime.send_file_roots, self.attachments.root
-        )
+            self._bot = Bot(self._token, default=DefaultBotProperties(parse_mode=None))
         self._inbox = Inbox(
             runtime.db, self._bot, handler=self._handle, poll_timeout=self.poll_timeout
         )
@@ -219,16 +174,6 @@ class TelegramFrontend(Frontend):
             runtime.db, self._bot, bus=runtime.bus, backoff=self.outbox_backoff
         )
 
-    def _session(self) -> AiohttpSession | None:
-        """A local Bot API server instead of api.telegram.org (2 GB uploads)."""
-        if not self.api_base_url:
-            return None
-        return AiohttpSession(
-            api=TelegramAPIServer.from_base(
-                self.api_base_url.rstrip("/"), is_local=True
-            )
-        )
-
     async def serve(self) -> None:
         me = await self.bot.get_me()
         _log.info(
@@ -239,7 +184,7 @@ class TelegramFrontend(Frontend):
             getattr(me, "has_topics_enabled", None),
         )
         await self.bot.set_my_commands(
-            [BotCommand(command=c, description=d) for c, d in self.texts.commands]
+            [BotCommand(command=c, description=d) for c, d in _COMMAND_HELP]
         )
         try:
             async with asyncio.TaskGroup() as tg:
@@ -260,7 +205,7 @@ class TelegramFrontend(Frontend):
         if conv.kind != "branch":
             return None
         topic = await self.bot.create_forum_topic(
-            self.chat_id, name=(conv.title or self.texts.branch)[:128]
+            self.chat_id, name=(conv.title or "ветка")[:128]
         )
         self._topic_names[topic.message_thread_id] = topic.name
         return await self.conversations.bind(
@@ -270,13 +215,12 @@ class TelegramFrontend(Frontend):
     async def mark_closed(self, conv: Conversation) -> bool:
         return await self.mark_topic(conv)
 
-    async def mark_topic(self, conv: Conversation, prefix: str | None = None) -> bool:
+    async def mark_topic(self, conv: Conversation, prefix: str = "✅ ") -> bool:
         """Rename the topic; closeForumTopic does not exist in private chats."""
-        prefix = self.texts.closed_prefix if prefix is None else prefix
         target = await self._target_of(conv)
         if target is None or target[1] is None:
             return False
-        name = self._topic_names.get(target[1]) or conv.title or self.texts.branch
+        name = self._topic_names.get(target[1]) or conv.title or "ветка"
         if name.startswith(prefix):
             return True
         await self.bot.edit_forum_topic(
@@ -285,43 +229,7 @@ class TelegramFrontend(Frontend):
         self._topic_names[target[1]] = f"{prefix}{name}"
         return True
 
-    def file_note(self) -> str:
-        return self.send_files.note
-
-    async def send_file(
-        self, conv: Conversation, path: str, *, caption: str = "", method: str = "auto"
-    ) -> str:
-        """Upload the file into this conversation's topic, caption and all."""
-        target = await self._target_of(conv)
-        if target is None:
-            msg = f"conversation {conv.external_id} has no Telegram window"
-            raise SendFileError(msg)
-        file = self.send_files.resolve(path)
-        how = method_for(file, method)
-        rest = caption.strip()
-        payload: dict[str, Any] = {
-            "message_thread_id": target[1],
-            how: FSInputFile(file),
-        }
-        if how == "audio":
-            payload["title"] = file.stem
-        if rest and how not in _NO_CAPTION and len(rest) <= _CAPTION:
-            payload |= {"caption": to_html(rest), "parse_mode": "HTML"}
-            rest = ""
-        try:
-            await getattr(self.bot, f"send_{how}")(target[0], **payload)
-        except TelegramNetworkError as exc:
-            msg = f"Telegram did not answer while sending {file.name}: {exc}"
-            raise SendFileError(msg) from exc
-        except TelegramAPIError as exc:
-            msg = f"Telegram refused {file.name} as {how}: {exc}"
-            raise SendFileError(msg) from exc
-        except (OSError, TimeoutError) as exc:
-            msg = f"sending {file.name} failed: {exc}"
-            raise SendFileError(msg) from exc
-        if rest:
-            await self._deliver(conv, rest)
-        return f"sent {file.name} as {how}, {mb(file.stat().st_size)} MB"
+    # ---- plumbing --------------------------------------------------------
 
     @property
     def bot(self) -> Bot:
@@ -412,9 +320,7 @@ class TelegramFrontend(Frontend):
             self._master_window = self._ext(topic.message_thread_id)
         return self._master_window
 
-    async def _master(
-        self, text: str | None = None, attachments: list[dict[str, Any]] | None = None
-    ) -> tuple[Conversation, bool]:
+    async def _master(self, text: str | None = None) -> tuple[Conversation, bool]:
         """The open master behind its window, spawning one when there is none.
 
         ``text`` rides with the seed of a fresh master; the second value says
@@ -436,39 +342,30 @@ class TelegramFrontend(Frontend):
             text=text,
             origin=FRONTEND,
             binding=(FRONTEND, ext),
-            attachments=attachments,
         )
         return conv, text is not None
 
     async def _branch(
-        self,
-        thread_id: int,
-        *,
-        title: str | None,
-        text: str | None,
-        attachments: list[dict[str, Any]] | None = None,
+        self, thread_id: int, *, title: str | None, text: str | None
     ) -> Conversation:
         master, _ = await self._master()
         return await self.conversations.spawn(
             kind="branch",
             seed="morning",
             parent=master,
-            title=(title or self._topic_names.get(thread_id) or self.texts.branch)[
-                :128
-            ],
+            title=(title or self._topic_names.get(thread_id) or "ветка")[:128],
             text=text,
             origin=FRONTEND,
             binding=(FRONTEND, self._ext(thread_id)),
-            attachments=attachments,
         )
 
+    # ---- inbox -----------------------------------------------------------
+
     async def _handle(self, update: Update) -> None:
         if update.message is not None:
             await self._on_message(update.message)
         elif update.callback_query is not None:
             await self._on_callback(update.callback_query)
-        elif update.stopped_message_generation is not None:
-            await self._on_stopped(update.stopped_message_generation.draft_id)
 
     async def _on_message(self, message: Message) -> None:
         created = message.forum_topic_created
@@ -503,79 +400,9 @@ class TelegramFrontend(Frontend):
                 )
             return
         text = (message.text or message.caption or "").strip()
-        note, attachment = await self._save_attachment(message)
-        if message.media_group_id:
-            self._collect_album(
-                message,
-                thread_id,
-                is_master=is_master,
-                text=text,
-                note=note,
-                attachment=attachment,
-            )
-            return
-        await self._dispatch(
-            message,
-            thread_id,
-            is_master=is_master,
-            text=text,
-            notes=[note] if note else [],
-            attachments=[attachment] if attachment else [],
-        )
-
-    def _collect_album(
-        self,
-        message: Message,
-        thread_id: int | None,
-        *,
-        is_master: bool,
-        text: str,
-        note: str | None,
-        attachment: dict[str, Any] | None,
-    ) -> None:
-        """An album arrives as separate messages; it becomes one turn."""
-        group = str(message.media_group_id)
-        album = self._albums.get(group)
-        if album is None:
-            album = _Album(message=message, thread_id=thread_id, is_master=is_master)
-            self._albums[group] = album
-        if text:
-            album.texts.append(text)
-        if note:
-            album.notes.append(note)
+        attachment = await self._save_attachment(message)
         if attachment:
-            album.attachments.append(attachment)
-        if album.task is not None:
-            album.task.cancel()
-        album.task = asyncio.create_task(self._flush_album(group))
-        self._tasks.add(album.task)
-        album.task.add_done_callback(self._tasks.discard)
-
-    async def _flush_album(self, group: str) -> None:
-        await asyncio.sleep(self.album_delay)
-        album = self._albums.pop(group, None)
-        if album is None:
-            return
-        await self._dispatch(
-            album.message,
-            album.thread_id,
-            is_master=album.is_master,
-            text="\n\n".join(album.texts),
-            notes=album.notes,
-            attachments=album.attachments,
-        )
-
-    async def _dispatch(
-        self,
-        message: Message,
-        thread_id: int | None,
-        *,
-        is_master: bool,
-        text: str,
-        notes: list[str],
-        attachments: list[dict[str, Any]],
-    ) -> None:
-        text = "\n\n".join(part for part in (text, *notes) if part).strip()
+            text = f"{text}\n\n{attachment}".strip()
         if not text:
             return
         if message.text and message.text.startswith("/"):
@@ -585,27 +412,20 @@ class TelegramFrontend(Frontend):
                 await self._command(command, args.strip(), message, thread_id)
                 return
         if is_master:
-            conv, consumed = await self._master(text, attachments or None)
+            conv, consumed = await self._master(text)
             if consumed:
                 return
         else:
-            topic = cast("int", thread_id)
-            conv = await self._live(topic)
+            conv = await self._live(thread_id)
             if conv is None:
-                title = self._topic_names.get(topic) or self._title_from(text)
-                await self._branch(
-                    topic, title=title, text=text, attachments=attachments or None
-                )
+                title = self._topic_names.get(thread_id) or self._title_from(text)
+                await self._branch(thread_id, title=title, text=text)
                 return
         pending = self.conversations.pending_question(conv.external_id)
         if pending is not None and self.conversations.answer(pending[0], text):
-            await self._close_ask(
-                pending[0], self.texts.question_typed.format(text=text)
-            )
+            await self._close_ask(pending[0], f"✍️ {text}")
             return
-        item = await self.conversations.post(
-            conv, text, origin=FRONTEND, attachments=attachments or None
-        )
+        item = await self.conversations.post(conv, text, origin=FRONTEND)
         if conv.running_turn or conv.pending_question:
             await self._react(message, item.id)
 
@@ -649,25 +469,57 @@ class TelegramFrontend(Frontend):
         with contextlib.suppress(TelegramAPIError):
             await self.bot.set_message_reaction(target[0], target[1], reaction=[])
 
-    async def _save_attachment(
-        self, message: Message
-    ) -> tuple[str | None, dict[str, Any] | None]:
-        """The note for the model and, for images, the file it gets to see inline."""
-        texts = self.texts
+    async def _save_attachment(self, message: Message) -> str | None:
+        file_id: str | None = None
+        name: str | None = None
+        kind = ""
+        size: int | None = None
         if message.sticker:
-            return texts.attachment_sticker, None
+            return "[вложение: стикер - не поддерживается]"
         if message.animation:
-            return texts.attachment_animation, None
-        file_id, kind, size, name, media_type = _media(message)
+            return "[вложение: анимация - не поддерживается]"
+        if message.photo:
+            photo = message.photo[-1]
+            file_id, kind, size = photo.file_id, "фото", photo.file_size
+            name = f"{photo.file_unique_id}.jpg"
+        elif message.document:
+            doc = message.document
+            file_id, kind, size = doc.file_id, "файл", doc.file_size
+            name = doc.file_name or f"{doc.file_unique_id}.bin"
+        elif message.voice:
+            file_id, kind, size = (
+                message.voice.file_id,
+                "голосовое",
+                message.voice.file_size,
+            )
+            name = f"{message.voice.file_unique_id}.ogg"
+        elif message.audio:
+            file_id, kind, size = (
+                message.audio.file_id,
+                "аудио",
+                message.audio.file_size,
+            )
+            name = message.audio.file_name or f"{message.audio.file_unique_id}.mp3"
+        elif message.video:
+            file_id, kind, size = (
+                message.video.file_id,
+                "видео",
+                message.video.file_size,
+            )
+            name = message.video.file_name or f"{message.video.file_unique_id}.mp4"
+        elif message.video_note:
+            file_id, kind, size = (
+                message.video_note.file_id,
+                "кружок",
+                message.video_note.file_size,
+            )
+            name = f"{message.video_note.file_unique_id}.mp4"
         if file_id is None or name is None:
-            return None, None
-        kind = texts.attachment_kinds.get(kind, kind)
+            return None
         if size and size > 20 * 1024 * 1024:
             return (
-                texts.attachment_too_big.format(
-                    kind=kind, name=name, mb=size // 1024 // 1024
-                ),
-                None,
+                f"[вложение: {kind} {name}, {size // 1024 // 1024} МБ - "
+                "больше 20 МБ, Telegram не отдаёт ботам]"
             )
         now = time.time()
         folder = self.attachments.root / self.attachments.day(now)
@@ -681,19 +533,14 @@ class TelegramFrontend(Frontend):
             path.chmod(0o644)
         except (TelegramAPIError, OSError) as exc:
             _log.warning("attachment download failed: %s", exc)
-            return texts.attachment_failed.format(kind=kind, name=name, error=exc), None
-        shown = texts.attachment_size.format(kb=size // 1024) if size else ""
+            return f"[вложение: {kind} {name} - не скачалось: {exc}]"
+        shown = f", {size // 1024} КБ" if size else ""
         keep = self.attachments.keep_days
-        seen = (
-            {"path": str(path), "media_type": media_type}
-            if media_type and media_type.startswith("image/")
-            else None
-        )
         if keep is None:
-            return texts.attachment.format(kind=kind, path=path, size=shown), seen
+            return f"[вложение: {kind} {path}{shown}]"
         return (
-            texts.attachment_kept.format(kind=kind, path=path, size=shown, days=keep),
-            seen,
+            f"[вложение: {kind} {path}{shown}; хранится {keep} дн., "
+            "перенеси в vault, если нужно]"
         )
 
     async def _sweep_loop(self, interval: float = 6 * 3600) -> None:
@@ -716,6 +563,8 @@ class TelegramFrontend(Frontend):
                 if folder.is_dir() and not any(folder.iterdir()):
                     folder.rmdir()
 
+    # ---- commands --------------------------------------------------------
+
     async def _command(
         self, command: str, args: str, message: Message, thread_id: int | None
     ) -> None:
@@ -726,9 +575,8 @@ class TelegramFrontend(Frontend):
             )
 
     async def _run_command(self, command: str, args: str, thread_id: int | None) -> str:
-        texts = self.texts
         if command in ("start", "help"):
-            return texts.help
+            return _HELP
         is_master = (
             thread_id is None or self._ext(thread_id) == await self._master_ext()
         )
@@ -737,9 +585,9 @@ class TelegramFrontend(Frontend):
             return await self._status(conv)
         if command == "merge":
             if conv is None or conv.kind != "branch":
-                return texts.merge_nothing
+                return "сливать нечего: это не открытая ветка"
             self._spawn_task(self._merge(conv))
-            return texts.merging
+            return "🔀 сливаю в мастер"
         if command == "new":
             if is_master or thread_id is None:
                 child = await self.conversations.spawn(
@@ -749,22 +597,20 @@ class TelegramFrontend(Frontend):
                     title=args or None,
                     origin=FRONTEND,
                 )
-                return texts.new_branch_topic.format(
-                    title=child.title or child.external_id
-                )
+                return f"🌿 ветка «{child.title or child.external_id}» - в новом топике"
             if conv is not None:
                 await self.conversations.set_status(conv, "closed")
             await self._branch(thread_id, title=args or None, text=None)
-            return texts.new_branch_here
+            return "🌿 новая ветка на этом топике"
         if command == "chat":
             if not args:
-                return texts.chat_usage
+                return "/chat <тема>"
             try:
                 deep = await self.conversations.spawn(
                     kind="deep", seed="clean", title=args, origin=FRONTEND
                 )
             except (ValueError, LookupError) as exc:
-                return texts.chat_failed.format(error=exc)
+                return f"не вышло: {exc}"
             where = next(
                 (
                     b.external_id
@@ -773,13 +619,12 @@ class TelegramFrontend(Frontend):
                 ),
                 deep.external_id,
             )
-            return texts.chat_opened.format(where=where)
-        return texts.help
+            return f"💬 глубокий чат: {where}"
+        return _HELP
 
     async def _status(self, conv: Conversation | None) -> str:
-        texts = self.texts
         if conv is None:
-            return texts.status_unbound
+            return "этот топик ни к чему не привязан - напиши, и откроется ветка"
         info = await self.conversations.describe(conv)
         queued = sum(
             1
@@ -790,22 +635,16 @@ class TelegramFrontend(Frontend):
         )
         lines = [
             f"{conv.kind} · {conv.status} · {conv.agent_name}",
-            texts.status_line.format(
-                live=texts.status_live if info["live"] else texts.status_dead,
-                turn=texts.status_running if conv.running_turn else texts.status_idle,
-                queued=queued,
-            ),
+            f"сессия: {'живая' if info['live'] else 'нет'}"
+            f" · тёрн: {'идёт' if conv.running_turn else 'нет'}"
+            f" · в очереди: {queued}",
         ]
         if conv.pending_question:
-            lines.append(texts.status_question)
+            lines.append("❓ ждёт ответа на вопрос")
         if conv.kind == "master":
             pool = self.conversations.pool
-            lines.append(
-                texts.status_pool.format(sessions=len(pool), mb=pool.rss() // (1 << 20))
-            )
-            lines.append(
-                texts.status_outbox.format(pending=await self.outbox.pending())
-            )
+            lines.append(f"пул: {len(pool)} сессий, rss {pool.rss() // (1 << 20)} МБ")
+            lines.append(f"outbox: {await self.outbox.pending()} в очереди")
         lines.append(f"id: {conv.external_id}")
         return "\n".join(lines)
 
@@ -814,13 +653,15 @@ class TelegramFrontend(Frontend):
             await self.conversations.merge(conv)
         except Exception as exc:  # noqa: BLE001
             _log.exception("merge of %s failed", conv.external_id)
-            await self._deliver(conv, self.texts.merge_failed.format(error=exc))
+            await self._deliver(conv, f"⚠️ слив не удался: {exc}")
 
     def _spawn_task(self, coro: Any) -> None:
         task = asyncio.create_task(coro)
         self._tasks.add(task)
         task.add_done_callback(self._tasks.discard)
 
+    # ---- bus -------------------------------------------------------------
+
     async def _events(self) -> None:
         async for event in self.bus.stream():
             try:
@@ -849,7 +690,7 @@ class TelegramFrontend(Frontend):
             return
         match kind:
             case "turn.start":
-                if event.get("origin") in ("user", "task"):
+                if event.get("origin") == "user":
                     await self._open_draft(key, event, target)
             case "stream":
                 self._feed_draft(key, event)
@@ -857,7 +698,7 @@ class TelegramFrontend(Frontend):
                 draft = self._drafts.get(key)
                 if draft is not None and event.get("parent_tool_use_id") is None:
                     draft.set_status(
-                        self._tool_status(event["name"], event.get("input"))
+                        f"⏳ {status_label(str(event['name']), event.get('input'))}"
                     )
             case "turn.end":
                 draft = self._drafts.get(key)
@@ -867,9 +708,9 @@ class TelegramFrontend(Frontend):
                     origin = event.get("origin")
                     await self._deliver(
                         conv,
-                        self.texts.turn_failed
+                        "⚠️ тёрн упал, смотри логи gateway"
                         if origin == "user"
-                        else self.texts.background_failed.format(origin=origin),
+                        else f"⚠️ фоновый тёрн ({origin}) упал, смотри логи gateway",
                         turn_id=event.get("turn_id"),
                         key=f"{event.get('turn_id')}:error",
                     )
@@ -883,25 +724,14 @@ class TelegramFrontend(Frontend):
                 await self._ask(conv, event, target)
             case "question.answered":
                 await self._close_ask(
-                    str(event["question_id"]),
-                    self.texts.question_answered.format(
-                        answer=event.get("answer") or ""
-                    ),
+                    str(event["question_id"]), f"✅ {event.get('answer') or ''}"
                 )
             case "question.timeout":
                 await self._close_ask(
-                    str(event["question_id"]), self.texts.question_timeout
+                    str(event["question_id"]), "⌛ время вышло - ответь текстом"
                 )
             case "conversation.merged":
-                await self._deliver(conv, self.texts.merged, key=f"{key}:merged")
-
-    async def _on_stopped(self, draft_id: int) -> None:
-        key = next((k for k, d in self._drafts.items() if d.draft_id == draft_id), None)
-        if key is None:
-            return
-        conv = await self.conversations.get(key)
-        if conv is not None:
-            await self.conversations.interrupt(conv)
+                await self._deliver(conv, "✅ слито в мастер", key=f"{key}:merged")
 
     async def _on_delivery_failed(self, event: Event) -> None:
         row = event.get("conversation_row")
@@ -943,17 +773,19 @@ class TelegramFrontend(Frontend):
         draft = self._drafts.pop(conv.external_id, None)
         if draft is not None:
             await draft.finish(chunks(text)[-1] if text.strip() else "")
-        if origin != FRONTEND and not origin.startswith("seed"):
+        if origin != FRONTEND and not origin.startswith("сид"):
             user_text = str(event.get("user_text") or "")
             if user_text:
                 await self._deliver(
                     conv,
-                    self.texts.mirrored.format(text=user_text),
+                    f"📝 из панели:\n{user_text}",
                     turn_id=turn_id,
                     key=f"{turn_id}:mirror",
                 )
         await self._deliver(conv, text, turn_id=turn_id, key=f"{turn_id}:reply")
 
+    # ---- drafts ------------------------------------------------------------
+
     async def _open_draft(
         self, key: str, event: Event, target: tuple[int, int | None]
     ) -> None:
@@ -966,7 +798,6 @@ class TelegramFrontend(Frontend):
             thread_id=target[1],
             turn_id=str(event.get("turn_id") or key),
             interval=self.draft_interval,
-            status=self.texts.waiting,
         )
         self._drafts[key] = draft
         draft.start()
@@ -980,29 +811,24 @@ class TelegramFrontend(Frontend):
         if kind == "content_block_delta":
             delta = raw.get("delta") or {}
             if delta.get("type") == "text_delta":
-                draft.set_status(self.texts.writing)
+                draft.set_status("✍️ пишу")
                 draft.append(str(delta.get("text") or ""))
             elif delta.get("type") == "thinking_delta":
-                draft.set_status(self.texts.thinking)
+                draft.set_status("🤔 думаю")
         elif kind == "content_block_start":
             block = raw.get("content_block") or {}
             if block.get("type") == "tool_use":
-                draft.set_status(self._tool_status(block.get("name") or "", None))
-
-    def _tool_status(self, name: Any, tool_input: Any) -> str:
-        label = status_label(str(name), tool_input, self.texts.tool_labels)
-        return self.texts.tool.format(label=label)
-
-    def _keyboard(
-        self, question_id: str, qi: int, question: dict[str, Any], picked: list[str]
-    ) -> InlineKeyboardMarkup:
-        return _keyboard(question_id, qi, question, picked, done=self.texts.done_button)
+                draft.set_status(
+                    f"⏳ {status_label(str(block.get('name') or ''), None)}"
+                )
 
     async def _close_draft(self, key: str) -> None:
         draft = self._drafts.pop(key, None)
         if draft is not None:
             await draft.stop()
 
+    # ---- questions (§3.7) ---------------------------------------------------
+
     async def _ask(
         self, conv: Conversation, event: Event, target: tuple[int, int | None]
     ) -> None:
@@ -1024,33 +850,12 @@ class TelegramFrontend(Frontend):
                     _question_html(question),
                     message_thread_id=target[1],
                     parse_mode="HTML",
-                    reply_markup=self._keyboard(question_id, qi, question, []),
+                    reply_markup=_keyboard(question_id, qi, question, []),
                 )
             except TelegramAPIError:
                 _log.exception("question %s could not be sent", question_id)
                 continue
             ask.messages.append(sent.message_id)
-        await self._persist_ask(question_id, ask)
-
-    async def _persist_ask(self, question_id: str, ask: _Ask | None) -> None:
-        """The open question lives in the conversation's flags across restarts."""
-        conv = await self.conversations.get(
-            ask.conversation_id if ask is not None else question_id
-        )
-        if conv is None:
-            return
-        await self.conversations.set_flags(
-            conv, {"ask": ask.to_flags(question_id) if ask is not None else None}
-        )
-
-    async def _restore_ask(self, question_id: str) -> _Ask | None:
-        for conv in await self.conversations.find(status="open", limit=500):
-            data = conv.flags.get("ask")
-            if isinstance(data, dict) and data.get("id") == question_id:
-                ask = _Ask.from_flags(data)
-                self._asks[question_id] = ask
-                return ask
-        return None
 
     async def _on_callback(self, query: CallbackQuery) -> None:
         if query.from_user.id != self.user_id or not query.data:
@@ -1059,9 +864,9 @@ class TelegramFrontend(Frontend):
         if len(parts) != 4 or parts[0] != "q":
             return
         _, question_id, qi_raw, choice = parts
-        ask = self._asks.get(question_id) or await self._restore_ask(question_id)
+        ask = self._asks.get(question_id)
         if ask is None or not qi_raw.isdigit():
-            await self._callback_reply(query, self.texts.question_closed)
+            await self._callback_reply(query, "вопрос уже закрыт")
             await self._strip_keyboard(query)
             return
         qi = int(qi_raw)
@@ -1096,23 +901,14 @@ class TelegramFrontend(Frontend):
                     await self.bot.edit_message_reply_markup(
                         chat_id=message.chat.id,
                         message_id=message.message_id,
-                        reply_markup=self._keyboard(question_id, qi, question, picked),
+                        reply_markup=_keyboard(question_id, qi, question, picked),
                     )
         await self._callback_reply(query, None)
-        if len(ask.done) < len(ask.questions):
-            await self._persist_ask(question_id, ask)
-            return
-        answer = _answer_text(ask)
-        self._asks.pop(question_id, None)
-        conv = await self.conversations.get(ask.conversation_id)
-        if conv is not None:
-            await self.conversations.set_flags(conv, {"ask": None})
-        if self.conversations.answer(question_id, answer):
-            return
-        if conv is not None and conv.status == "open":
-            await self.conversations.post(conv, answer, origin=FRONTEND)
-        else:
-            await self._edit_asks(ask, self.texts.question_timeout)
+        if len(ask.done) == len(ask.questions):
+            answer = _answer_text(ask)
+            self._asks.pop(question_id, None)
+            if not self.conversations.answer(question_id, answer):
+                await self._edit_asks(ask, "⌛ время вышло - ответь текстом")
 
     async def _strip_keyboard(self, query: CallbackQuery) -> None:
         message = query.message if isinstance(query.message, Message) else None
@@ -1132,12 +928,11 @@ class TelegramFrontend(Frontend):
     async def _close_ask(self, question_id: str, note: str) -> None:
         ask = self._asks.pop(question_id, None)
         if ask is not None:
-            await self._persist_ask(question_id, None)
             await self._edit_asks(ask, note)
 
     async def _edit_asks(self, ask: _Ask, note: str) -> None:
         for qi, message_id in enumerate(ask.messages):
-            if qi in ask.done and note != self.texts.question_timeout:
+            if qi in ask.done and not note.startswith("⌛"):
                 continue
             with contextlib.suppress(TelegramAPIError):
                 await self.bot.edit_message_text(
@@ -1148,50 +943,6 @@ class TelegramFrontend(Frontend):
                 )
 
 
-def _media(
-    message: Message,
-) -> tuple[str | None, str, int | None, str | None, str | None]:
-    """``(file_id, kind, size, name, media_type)`` of the message's media, if any."""
-    if message.photo:
-        photo = message.photo[-1]
-        return (
-            photo.file_id,
-            "photo",
-            photo.file_size,
-            f"{photo.file_unique_id}.jpg",
-            "image/jpeg",
-        )
-    if message.document:
-        doc = message.document
-        name = doc.file_name or f"{doc.file_unique_id}.bin"
-        return doc.file_id, "document", doc.file_size, name, doc.mime_type
-    if message.voice:
-        v = message.voice
-        return v.file_id, "voice", v.file_size, f"{v.file_unique_id}.ogg", None
-    if message.audio:
-        a = message.audio
-        return (
-            a.file_id,
-            "audio",
-            a.file_size,
-            a.file_name or f"{a.file_unique_id}.mp3",
-            None,
-        )
-    if message.video:
-        v = message.video
-        return (
-            v.file_id,
-            "video",
-            v.file_size,
-            v.file_name or f"{v.file_unique_id}.mp4",
-            None,
-        )
-    if message.video_note:
-        n = message.video_note
-        return n.file_id, "video_note", n.file_size, f"{n.file_unique_id}.mp4", None
-    return None, "", None, None, None
-
-
 def _question_html(question: dict[str, Any]) -> str:
     header = html.escape(str(question.get("header") or "").strip())
     body = html.escape(str(question.get("question") or "").strip())
@@ -1204,7 +955,7 @@ def _question_html(question: dict[str, Any]) -> str:
 
 
 def _keyboard(
-    question_id: str, qi: int, question: dict[str, Any], picked: list[str], *, done: str
+    question_id: str, qi: int, question: dict[str, Any], picked: list[str]
 ) -> InlineKeyboardMarkup:
     rows = [
         [
@@ -1220,7 +971,7 @@ def _keyboard(
         rows.append(
             [
                 InlineKeyboardButton(
-                    text=done, callback_data=f"q:{question_id}:{qi}:{_DONE}"
+                    text="✅ готово", callback_data=f"q:{question_id}:{qi}:{_DONE}"
                 )
             ]
         )

src/beaver_gateway/frontends/telegram/inbox.py

@@ -1,7 +1,8 @@
-"""Long-polling inbox.
+"""Long-polling inbox (§3.8).
 
 Every update lands in ``telegram_updates`` before the offset moves past it;
-a worker handles rows from the table, oldest first, resuming after a restart.
+a worker handles rows from the table, oldest first, and finishes whatever a
+previous process left unprocessed at startup.
 """
 
 from __future__ import annotations
@@ -31,7 +32,7 @@ __all__ = ["Inbox"]
 
 _log = logging.getLogger("beaver_gateway.frontends.telegram.inbox")
 
-ALLOWED_UPDATES = ("message", "callback_query", "stopped_message_generation")
+ALLOWED_UPDATES = ("message", "callback_query")
 
 
 def _unset(_value: object) -> None:

src/beaver_gateway/frontends/telegram/outbox.py

@@ -1,7 +1,8 @@
-"""Outbox: a reply is a ``deliveries`` row first, a message second.
+"""Outbox (§3.8): a reply is a ``deliveries`` row first, a message second.
 
-Rows are sent oldest first, retried with backoff on network errors, resent
-as plain text when Telegram rejects our HTML, and given up once it's gone.
+Rows are sent oldest first, retried with backoff on network errors and
+flood limits, resent as plain text when Telegram rejects our HTML, and
+given up only when Telegram says the window is gone.
 """
 
 from __future__ import annotations
@@ -31,7 +32,7 @@ from beaver_gateway.storage.models import Delivery
 if TYPE_CHECKING:
     from aiogram import Bot
 
-    from beaver_gateway.events.bus import EventBus
+    from beaver_gateway.core.bus import EventBus
     from beaver_gateway.storage.db import Database
 
 __all__ = ["Outbox"]

src/beaver_gateway/frontends/telegram/render.py

@@ -7,10 +7,8 @@ import re
 from dataclasses import dataclass
 from typing import TYPE_CHECKING, Any, Literal
 
-from beaver_gateway.frontends.telegram.texts import TOOL_LABELS
-
 if TYPE_CHECKING:
-    from collections.abc import Callable, Mapping
+    from collections.abc import Callable
 
 __all__ = ["LIMIT", "chunks", "status_label", "to_html", "to_html_tail"]
 
@@ -39,6 +37,27 @@ _ITALIC_U = re.compile(r"(?<![\w_])_(?!\s)(.+?)(?<!\s)_(?![\w_])")
 _STRIKE = re.compile(r"~~(.+?)~~")
 _SPOILER = re.compile(r"\|\|(.+?)\|\|")
 
+_LABELS: dict[str, str] = {
+    "Read": "читаю vault",
+    "Glob": "ищу файлы",
+    "Grep": "ищу в vault",
+    "Edit": "правлю файл",
+    "Write": "пишу файл",
+    "MultiEdit": "правлю файлы",
+    "Bash": "выполняю команду",
+    "WebSearch": "ищу в сети",
+    "WebFetch": "читаю страницу",
+    "Task": "запустил сабагента",
+    "Agent": "запустил сабагента",
+    "AskUserQuestion": "спрашиваю",
+    "TodoWrite": "планирую",
+    "Skill": "открываю скилл",
+    "mcp__gateway__spawn": "открываю разговор",
+    "mcp__gateway__read_conversation": "читаю разговор",
+    "mcp__gateway__say": "говорю",
+    "mcp__gateway__schedule": "ставлю напоминание",
+    "mcp__gateway__inject": "передаю в другой разговор",
+}
 
 _Kind = Literal["fence", "quote", "table", "text"]
 
@@ -81,12 +100,8 @@ def chunks(text: str, limit: int = LIMIT) -> list[str]:
     return out
 
 
-def status_label(
-    name: str,
-    tool_input: dict[str, Any] | None = None,
-    labels: Mapping[str, str] | None = None,
-) -> str:
-    label = (labels if labels is not None else TOOL_LABELS).get(name)
+def status_label(name: str, tool_input: dict[str, Any] | None = None) -> str:
+    label = _LABELS.get(name)
     if label is not None:
         return label
     if name.startswith("mcp__"):

src/beaver_gateway/frontends/telegram/texts.py

@@ -1,106 +0,0 @@
-"""Every string the Telegram frontend shows the user; English defaults, overridable."""
-
-from __future__ import annotations
-
-from dataclasses import dataclass, field
-
-__all__ = ["TOOL_LABELS", "TelegramTexts"]
-
-TOOL_LABELS: dict[str, str] = {
-    "Read": "reading",
-    "Glob": "looking for files",
-    "Grep": "searching",
-    "Edit": "editing a file",
-    "Write": "writing a file",
-    "MultiEdit": "editing files",
-    "Bash": "running a command",
-    "WebSearch": "searching the web",
-    "WebFetch": "reading a page",
-    "Task": "started a subagent",
-    "Agent": "started a subagent",
-    "AskUserQuestion": "asking",
-    "TodoWrite": "planning",
-    "Skill": "opening a skill",
-    "mcp__gateway__spawn": "opening a conversation",
-    "mcp__gateway__read_conversation": "reading a conversation",
-    "mcp__gateway__say": "speaking",
-    "mcp__gateway__schedule": "setting a reminder",
-    "mcp__gateway__inject": "passing to another conversation",
-    "mcp__gateway__send_file": "sending a file",
-}
-
-
-@dataclass(frozen=True, slots=True)
-class TelegramTexts:
-    commands: tuple[tuple[str, str], ...] = (
-        ("merge", "merge this branch into the master"),
-        ("new", "new branch: in General a new topic, in a topic a fresh one here"),
-        ("chat", "open a deep chat in the vault: /chat topic"),
-        ("status", "this conversation"),
-        ("help", "commands"),
-    )
-    help_head: str = "General is the master, any other topic is a branch."
-    branch: str = "branch"
-    closed_prefix: str = "✅ "
-    waiting: str = "⏳ thinking"
-    thinking: str = "🤔 thinking"
-    writing: str = "✍️ writing"
-    tool: str = "⏳ {label}"
-    tool_labels: dict[str, str] = field(default_factory=lambda: dict(TOOL_LABELS))
-    attachment_sticker: str = "[attachment: sticker - not supported]"
-    attachment_animation: str = "[attachment: animation - not supported]"
-    attachment_kinds: dict[str, str] = field(
-        default_factory=lambda: {
-            "photo": "photo",
-            "document": "file",
-            "voice": "voice message",
-            "audio": "audio",
-            "video": "video",
-            "video_note": "video note",
-        }
-    )
-    attachment_too_big: str = (
-        "[attachment: {kind} {name}, {mb} MB - over 20 MB, Telegram does not "
-        "serve it to bots]"
-    )
-    attachment_failed: str = "[attachment: {kind} {name} - download failed: {error}]"
-    attachment_size: str = ", {kb} KB"
-    attachment: str = "[attachment: {kind} {path}{size}]"
-    attachment_kept: str = (
-        "[attachment: {kind} {path}{size}; kept {days} days, move it into the "
-        "vault if it matters]"
-    )
-    merge_nothing: str = "nothing to merge: this is not an open branch"
-    merging: str = "🔀 merging into the master"
-    merged: str = "✅ merged into the master"
-    merge_failed: str = "⚠️ merge failed: {error}"
-    new_branch_topic: str = "🌿 branch «{title}» - in a new topic"
-    new_branch_here: str = "🌿 new branch on this topic"
-    chat_usage: str = "/chat <topic>"
-    chat_failed: str = "could not open: {error}"
-    chat_opened: str = "💬 deep chat: {where}"
-    status_unbound: str = "this topic is bound to nothing - write, and a branch opens"
-    status_line: str = "session: {live} · turn: {turn} · queued: {queued}"
-    status_live: str = "live"
-    status_dead: str = "none"
-    status_running: str = "running"
-    status_idle: str = "idle"
-    status_question: str = "❓ waiting for an answer"
-    status_pool: str = "pool: {sessions} sessions, rss {mb} MB"
-    status_outbox: str = "outbox: {pending} queued"
-    turn_failed: str = "⚠️ the turn failed, see the gateway logs"
-    background_failed: str = (
-        "⚠️ a background turn ({origin}) failed, see the gateway logs"
-    )
-    question_closed: str = "the question is already closed"
-    question_timeout: str = "⌛ time is up - answer as text"
-    question_answered: str = "✅ {answer}"
-    question_typed: str = "✍️ {text}"
-    done_button: str = "✅ done"
-    mirrored: str = "📝 from the panel:\n{text}"
-
-    @property
-    def help(self) -> str:
-        return (
-            self.help_head + "\n" + "\n".join(f"/{c} - {d}" for c, d in self.commands)
-        )

src/beaver_gateway/frontends/turn_record.py

@@ -1,95 +0,0 @@
-"""Cross-frontend turn record.
-
-Frontends emit a :class:`TurnRecord` to every handler in
-``GatewayRuntime.turn_log_handlers`` after a turn finishes.
-"""
-
-from __future__ import annotations
-
-import re
-from dataclasses import dataclass
-from typing import TYPE_CHECKING, Any
-
-if TYPE_CHECKING:
-    from anthropic.types import Message, MessageParam
-
-
-__all__ = ["TurnRecord", "slugify"]
-
-
-_SLUG_BAD_RE = re.compile(r"[^\w\s\-]+", flags=re.UNICODE)
-_SLUG_SEP_RE = re.compile(r"[\s\-]+", flags=re.UNICODE)
-
-
-def slugify(text: str, *, maxlen: int = 40) -> str:
-    """Sanitize ``text`` for use as a filename fragment.
-
-    Truncates to ``maxlen`` and returns ``"untitled"`` for empty input.
-    """
-    cleaned = _SLUG_BAD_RE.sub(" ", text).strip()
-    cleaned = _SLUG_SEP_RE.sub("-", cleaned).strip("-")
-    if not cleaned:
-        return "untitled"
-    if len(cleaned) > maxlen:
-        cleaned = cleaned[:maxlen].rstrip("-")
-    return cleaned or "untitled"
-
-
-@dataclass(frozen=True, slots=True)
-class TurnRecord:
-    """One completed turn, as seen by a frontend.
-
-    ``input_messages`` excludes the assistant reply; ``source`` names the
-    producing frontend so a cross-frontend logger can skip its own turns.
-    """
-
-    agent_name: str
-    input_messages: list[MessageParam]
-    output_message: Message
-    system: str | None = None
-    source: str = "anthropic"
-
-    @property
-    def first_user_text(self) -> str:
-        """Empty string if the input history has no user turn."""
-        for msg in self.input_messages:
-            if msg.get("role") == "user":
-                return _text_of(msg.get("content", ""))
-        return ""
-
-    @property
-    def last_user_text(self) -> str:
-        for msg in reversed(self.input_messages):
-            if msg.get("role") == "user":
-                return _text_of(msg.get("content", ""))
-        return ""
-
-    @property
-    def assistant_text(self) -> str:
-        """Plain text of the assistant reply, with thinking/tool blocks dropped."""
-        chunks = [
-            getattr(block, "text", "") or ""
-            for block in getattr(self.output_message, "content", ())
-            if getattr(block, "type", None) == "text"
-        ]
-        return "\n\n".join(c for c in chunks if c)
-
-
-def _text_of(content: object) -> str:
-    """Flatten an Anthropic ``MessageParam.content`` field to a plain string.
-
-    Handles both raw-string and block-list shapes; unrecognized content is
-    silently skipped since this exists for naming, not faithful reconstruction.
-    """
-    if isinstance(content, str):
-        return content
-    if isinstance(content, list):
-        parts: list[str] = []
-        for blk in content:
-            if not isinstance(blk, dict):
-                continue
-            d: Any = blk
-            if d.get("type") == "text":
-                parts.append(str(d.get("text", "")))
-        return "\n\n".join(p for p in parts if p)
-    return ""

src/beaver_gateway/jobs/__init__.py

@@ -1 +0,0 @@
-"""Cron, webhook and event jobs, deferred injects, the subscription budget."""

src/beaver_gateway/jobs/job.py

@@ -1,126 +0,0 @@
-"""What a job is: its triggers, the budget it respects, and what a run may do."""
-
-from __future__ import annotations
-
-import logging
-from dataclasses import dataclass
-from datetime import datetime, timedelta
-from typing import TYPE_CHECKING, Any
-
-if TYPE_CHECKING:
-    from collections.abc import Awaitable, Callable, Coroutine
-
-    from beaver_gateway.conversations.distill import LineCap
-    from beaver_gateway.conversations.injects import Priority
-    from beaver_gateway.conversations.service import Conversations, DistillResult
-    from beaver_gateway.jobs.scheduler import Scheduler
-    from beaver_gateway.storage.models import Conversation
-
-__all__ = ["Budget", "Job", "JobRun"]
-
-_log = logging.getLogger(__name__)
-
-
-@dataclass(frozen=True, slots=True)
-class Job:
-    name: str
-    run: Callable[[JobRun], Awaitable[None]]
-    cron: str | None = None
-    webhook: bool = False
-    events: tuple[str, ...] = ()
-    critical: bool = True
-    dedupe: bool = True
-
-    @property
-    def entrypoint(self) -> str:
-        return f"job:{self.name}"
-
-
-@dataclass(frozen=True, slots=True)
-class Budget:
-    threshold: float = 0.7
-    tokens: int | None = None
-    window: timedelta = timedelta(hours=5)
-    limit_window: str = "five_hour"
-
-
-@dataclass(slots=True)
-class JobRun:
-    job: Job
-    trigger: str
-    payload: dict[str, Any]
-    scheduler: Scheduler
-
-    @property
-    def conversations(self) -> Conversations:
-        return self.scheduler.conversations
-
-    async def master(self) -> Conversation | None:
-        masters = await self.conversations.find(kind="master", status="open", limit=1)
-        return masters[0] if masters else None
-
-    async def inject_master(
-        self, text: str, *, urgency: Priority = "normal", origin: str | None = None
-    ) -> bool:
-        master = await self.master()
-        if master is None:
-            _log.error(
-                "job %s: no open master, inject lost: %s", self.job.name, text[:200]
-            )
-            return False
-        await self.conversations.inject(
-            master, text, urgency=urgency, origin=origin or self.job.name
-        )
-        return True
-
-    async def spawn_job(
-        self,
-        *,
-        agent: str,
-        text: str,
-        title: str | None = None,
-        line_cap: LineCap | None = None,
-    ) -> Conversation:
-        """A headless job turn; ``line_cap`` bounces a rewrite past the cap."""
-        return await self.conversations.spawn(
-            kind="job",
-            agent=agent,
-            seed="brief",
-            text=text,
-            title=title,
-            origin="job",
-            flags={"line_cap": line_cap.as_flags()} if line_cap else None,
-        )
-
-    async def close_idle(
-        self,
-        *,
-        kind: str = "deep",
-        days: int = 2,
-        limit: int = 3,
-        since: datetime | None = None,
-    ) -> list[DistillResult]:
-        """Close chats quiet for ``days``, at most ``limit`` per run."""
-        out: list[DistillResult] = []
-        for conv in await self.conversations.idle(
-            kind=kind, days=days, since=since, limit=limit
-        ):
-            try:
-                out.append(
-                    await self.conversations.distill(conv, reason=f"idle {days}d")
-                )
-            except Exception:  # noqa: BLE001
-                _log.exception("closing idle %s failed", conv.external_id)
-        return out
-
-    async def retry_in(self, delay: timedelta) -> None:
-        await self.scheduler.trigger(
-            self.job, self.payload, delay=delay, trigger=self.trigger
-        )
-
-    async def rotate(self) -> list[Conversation]:
-        rotation = self.scheduler.rotation
-        return await rotation.tick() if rotation is not None else []
-
-    def background(self, coro: Coroutine[Any, Any, Any]) -> None:
-        self.scheduler.background(coro)

src/beaver_gateway/mcp/__init__.py

@@ -1,4 +1,4 @@
-"""MCP server definitions and the internal aggregator app."""
+"""MCP server definitions and (later) the internal aggregator app."""
 
 from __future__ import annotations
 

src/beaver_gateway/mcp/client_pool.py

@@ -1,7 +1,11 @@
 """Proxy ``FastMCP`` servers for user-declared external MCPs (``stdio``/``http``).
 
-Built via ``fastmcp.server.create_proxy``; the proxy lazily opens its
-transport on first request, so unused MCPs cost nothing until routed to.
+Both flavours end up as a ``FastMCPProxy`` instance, built via
+``fastmcp.server.create_proxy``. The proxy lazily opens the underlying
+client transport when the first MCP request arrives, so we don't pay
+for connections that nothing routes to. From the aggregator app's
+point of view a proxy is indistinguishable from a regular ``FastMCP``
+namespace — same ``http_app`` surface, same mount semantics.
 """
 
 from __future__ import annotations

src/beaver_gateway/mcp/internal_app.py

@@ -1,7 +1,26 @@
 """Internal MCP aggregator — one ASGI app, N FastMCP namespaces + ``all``.
 
-Each ``McpServer`` in the user's config is mounted under ``/mcp/<name>``;
-``/mcp/all/`` unions every namespace's tools for single-MCP clients.
+Each ``McpServer`` declared in the user's config becomes its own
+``FastMCP`` instance (regular for ``python_tool``, ``FastMCPProxy`` for
+``stdio``/``http``) and is mounted under ``/mcp/<name>`` on a single
+Starlette app. This app runs on ``127.0.0.1:INTERNAL_MCP_PORT`` (not
+EXPOSE'd in Docker) so the ClaudeCode subprocess can reach each
+namespace via loopback as a distinct MCP server URL — preserving
+per-domain framing while costing only one process worth of RAM
+(PRD §6).
+
+Phase 3 adds ``/mcp/all/``: a single FastMCP whose tools are the union
+of every namespace's tools, prefixed by FastMCP's ``namespace_<tool>``
+convention (e.g. ``time_current_time``). It's the escape-hatch for
+clients that can only configure one MCP server — discouraged for tool-
+heavy setups (PRD §6 cites the ~95%→~71% tool-selection drop on flat
+namespaces) but real and reachable.
+
+The aggregator returns both the app and a ``{name: url}`` map; Phase
+``ClaudeSdkBackend`` plugs the map directly into
+``BackendOptions.mcp_servers``. ``/mcp/all/`` is NOT included in that
+map — claude-code-agents always get per-domain framing; only the
+external MCP frontend (Phase 3.1) reverse-proxies the flat endpoint.
 """
 
 from __future__ import annotations
@@ -33,13 +52,11 @@ def build_internal_app(
 ) -> tuple[Starlette, dict[str, str], dict[str, FastMCP]]:
     """Build the aggregator ``Starlette`` app, per-namespace URL map, and server map.
 
-    ``host``/``port`` only flavour the URL strings handed back; the
-    caller (``cli.main``) is what actually listens on them. Published
-    URLs carry a trailing slash so ``Mount`` doesn't 307-redirect the
-    first request. Each
-    namespace gets its own :class:`RedactingMiddleware` so a direct
-    ``call_tool`` on a child (e.g. the Raycast backend does that) is
-    filtered too, not just requests through the mounted app.
+    ``host``/``port`` only flavour the URL strings handed back — actually
+    listening on them is the caller's job (``cli.main`` runs a uvicorn
+    server in a TaskGroup). We accept the address here so callers don't
+    have to format the URLs themselves and risk drifting from the
+    ``/mcp/<name>`` convention.
 
     Returns:
       * Starlette app to serve via uvicorn.
@@ -53,6 +70,10 @@ def build_internal_app(
         MCP tools into the Raycast wire).
     """
     servers: dict[str, FastMCP] = {spec.name: _build_server(spec) for spec in mcps}
+    # The one place every tool result passes through on its way to the
+    # model. Attached per namespace rather than once at the top so that
+    # a direct ``call_tool`` on a child (the Raycast backend does that)
+    # is filtered too.
     for server in servers.values():
         server.add_middleware(RedactingMiddleware())
 
@@ -61,6 +82,8 @@ def build_internal_app(
     }
     routes = [Mount(f"/mcp/{name}", app=app) for name, app in child_apps.items()]
 
+    # /mcp/all — flat-namespace bundle. Skip when there's nothing to
+    # bundle so we don't pay for an empty session manager lifecycle.
     all_app = None
     if servers:
         all_server = _build_all_server(servers)
@@ -70,11 +93,11 @@ def build_internal_app(
 
     @asynccontextmanager
     async def lifespan(_parent: Starlette) -> AsyncIterator[None]:
-        """Enter every child app's lifespan.
-
-        Without it the streamable-HTTP layer 500s on every request;
-        children unwind in reverse order.
-        """
+        # Each FastMCP http_app stores its session manager init in its
+        # own lifespan. Without entering them the streamable-HTTP layer
+        # 500s on every request. AsyncExitStack composes them so all
+        # children come up together and unwind in reverse order on
+        # shutdown.
         async with AsyncExitStack() as stack:
             for child in child_apps.values():
                 await stack.enter_async_context(child.router.lifespan_context(child))
@@ -85,22 +108,23 @@ def build_internal_app(
             yield
 
     app = Starlette(routes=routes, lifespan=lifespan)
+    # Trailing slash on the published URL skips Starlette's
+    # 307 redirect from ``/mcp/<name>`` to ``/mcp/<name>/`` that
+    # ``Mount`` produces when a child route lives at ``/``.
     urls = {name: f"http://{host}:{port}/mcp/{name}/" for name in servers}
     return app, urls, servers
 
 
 def _build_server(spec: McpServerT) -> FastMCP:
-    """Dispatch on the discriminated union to the matching builder.
-
-    The final branch is unreachable while ``McpServerT`` stays closed;
-    it exists to keep type-narrowing honest if a variant is ever added.
-    """
+    """Dispatch on the discriminated union to the matching builder."""
     if isinstance(spec, PythonToolMcp):
         return build_python_tool_server(spec)
     if isinstance(spec, StdioMcp):
         return build_stdio_proxy(spec)
     if isinstance(spec, HttpMcp):
         return build_http_proxy(spec)
+    # `McpServerT` is a closed union; this is unreachable but keeps
+    # type-narrowing honest if a new variant lands without updates here.
     msg = f"unsupported McpServer variant: {type(spec).__name__}"
     raise TypeError(msg)
 

src/beaver_gateway/mcp/lenient.py

@@ -1,8 +1,22 @@
 """Tolerant transports for upstream MCPs that don't strictly speak JSON-RPC.
 
-Some MCP servers print non-JSON chatter to stdout (banners, dependency-load
-messages) that the reference client forwards as exceptions, which bleed into
-client UIs as warnings. This transport drops those lines instead.
+Some real-world MCP servers print non-JSON chatter to stdout before / between
+their actual JSON-RPC frames (``Processing...``, banners, dependency-load
+messages, etc.). The reference ``mcp.client.stdio.stdio_client`` parses every
+stdout line as JSON-RPC and ships any parse failure downstream as an
+exception, which the MCP ``ClientSession`` then logs as a warning that bleeds
+into client UIs (Cursor, Cline) when they connect through us.
+
+``LenientStdioTransport`` re-implements the stdio-client wiring with one
+behavioural change: lines that don't parse as JSON-RPC are *silently
+dropped* (one ``DEBUG`` log entry, no exception forwarded). Downstream
+consumers see only valid messages. We keep the rest of the contract identical
+to the reference client, including the spec-mandated graceful shutdown
+sequence (close stdin → wait → SIGTERM → SIGKILL).
+
+The transport plugs into ``fastmcp.server.create_proxy`` just like
+``StdioTransport`` does, so the rest of the aggregator doesn't need to
+know which flavour it got.
 """
 
 from __future__ import annotations
@@ -56,15 +70,13 @@ class LenientStdioTransport(ClientTransport):
         cwd: str | None = None,
         log_file: TextIO | None = None,
     ) -> None:
-        """``log_file`` takes an already-open ``TextIO``.
-
-        Unlike the upstream transport, this one does not open a ``Path``
-        for you.
-        """
         self.command = command
         self.args = args
         self.env = env
         self.cwd = cwd
+        # TextIO only — pre-open Path callers themselves. The upstream
+        # ``StdioTransport`` opens Path for you, but we keep this thin so
+        # the type contract stays narrow and easy to validate.
         self.log_file = log_file
 
     @contextlib.asynccontextmanager
@@ -88,7 +100,7 @@ class LenientStdioTransport(ClientTransport):
 
 
 @contextlib.asynccontextmanager
-async def _lenient_stdio_client(  # noqa: PLR0915
+async def _lenient_stdio_client(  # noqa: PLR0915 — mirrors mcp.client.stdio.stdio_client
     server: StdioServerParameters, errlog: TextIO = sys.stderr
 ) -> AsyncIterator[
     tuple[
@@ -100,8 +112,8 @@ async def _lenient_stdio_client(  # noqa: PLR0915
 
     All differences from upstream live in ``stdout_reader``: lines that fail
     ``JSONRPCMessage.model_validate_json`` are logged at DEBUG and skipped,
-    never forwarded as exceptions. Shutdown still follows the MCP spec
-    sequence: close stdin, wait, SIGTERM, then SIGKILL.
+    never forwarded as exceptions. This is what makes warning-noisy MCPs
+    quiet from the consumer's point of view.
     """
     read_stream_writer, read_stream = anyio.create_memory_object_stream[
         SessionMessage | Exception
@@ -147,7 +159,7 @@ async def _lenient_stdio_client(  # noqa: PLR0915
                             continue
                         try:
                             message = types.JSONRPCMessage.model_validate_json(stripped)
-                        except Exception:  # noqa: BLE001
+                        except Exception:  # noqa: BLE001 — by design, see module doc
                             _log.debug(
                                 "lenient stdio: dropped non-JSON line: %r",
                                 stripped[:200],
@@ -180,6 +192,7 @@ async def _lenient_stdio_client(  # noqa: PLR0915
         try:
             yield read_stream, write_stream
         finally:
+            # MCP spec stdio shutdown: close stdin → wait → SIGTERM → SIGKILL.
             if process.stdin:
                 with contextlib.suppress(Exception):
                     await process.stdin.aclose()

src/beaver_gateway/mcp/redacting.py

@@ -1,7 +1,26 @@
-"""Redact credentials in MCP tool output before it reaches the model.
+"""Redact credentials on the way out of an MCP tool.
 
-Covers every MCP route; in-process tools like ``Bash``/``Read`` are
-guarded separately by ``agents.policy``.
+A tool result is a wider channel than the log. It goes straight into the
+model's context, from there into the turn record in Postgres, and from
+there into the markdown transcript in ``💬 чаты`` — which Obsidian Sync
+carries off the machine. One ``komodo`` deploy returns the resolved
+compose file, ``environment:`` block and all, so a single call can put
+every secret of a stack into all four places at once.
+
+One filter, not a list of exceptions: every MCP the model can reach is
+built into a ``FastMCP`` by :mod:`beaver_gateway.mcp.internal_app` —
+``python_tool`` bundles like komodo, stdio subprocesses, remote HTTP
+servers — and every route into one of them runs its middleware chain.
+That covers the per-namespace ``/mcp/<name>/`` mounts claude-code talks
+to, the ``/mcp/all`` bundle, the external MCP frontend reverse-proxying
+into both, and the Raycast backend's direct ``call_tool``. A new MCP in
+``config.py`` is covered the day it is added, without anyone
+remembering to list it here.
+
+What this does not reach: tools that never touch a FastMCP server — the
+gateway's own ``gateway`` tools, and everything claude-code runs inside
+its own process (``Bash``, ``Read``). Those are guarded by
+:mod:`beaver_gateway.core.policy` and the vault mounts instead.
 """
 
 from __future__ import annotations
@@ -12,7 +31,7 @@ import mcp.types as mt
 from fastmcp.server.middleware import Middleware
 from fastmcp.tools.base import ToolResult
 
-from beaver_gateway.security.redact import redact, redact_data
+from beaver_gateway.core.redact import redact, redact_data
 
 if TYPE_CHECKING:
     from fastmcp.server.middleware import CallNext, MiddlewareContext
@@ -50,12 +69,6 @@ class RedactingMiddleware(Middleware):
         context: MiddlewareContext[mt.CallToolRequestParams],
         call_next: CallNext[mt.CallToolRequestParams, ToolResult],
     ) -> ToolResult:
-        """Mask the result.
-
-        A non-``None`` ``meta`` on the masked copy takes the
-        ``CallToolResult`` path that skips output-schema validation,
-        since masked structured content may no longer match it.
-        """
         result = await call_next(context)
         content = [_redact_block(block) for block in result.content]
         structured = redact_data(result.structured_content)
@@ -64,5 +77,9 @@ class RedactingMiddleware(Middleware):
         return ToolResult(
             content=content,
             structured_content=structured,
+            # Masked structured content no longer has to satisfy the
+            # tool's output schema; a non-None meta takes the
+            # ``CallToolResult`` path that skips that validation, the
+            # same trick ``ResponseLimitingMiddleware`` uses.
             meta=result.meta if result.meta is not None else {},
         )

src/beaver_gateway/mcp/types.py

@@ -6,7 +6,7 @@ discriminated-union members so downstream code can ``match`` on ``kind``.
 
 from __future__ import annotations
 
-from collections.abc import Callable  # noqa: TC003
+from collections.abc import Callable  # noqa: TC003 — runtime use by pydantic
 from pathlib import Path
 from typing import TYPE_CHECKING, Annotated, Literal
 
@@ -20,35 +20,39 @@ class _BaseMcp(BaseModel):
     model_config = ConfigDict(frozen=True, arbitrary_types_allowed=True)
 
     name: str
-    """The namespace this MCP server is mounted under."""
 
 
 class StdioMcp(_BaseMcp):
-    """Subprocess MCP server spawned and connected to over stdio."""
+    """Subprocess MCP server we spawn and connect to over stdio.
+
+    ``lenient`` switches the upstream stdio reader to a tolerant variant
+    that silently drops non-JSON-RPC lines from the subprocess's stdout
+    (``Processing...``-style chatter, banners, dependency-load messages).
+    The reference ``mcp.client.stdio.stdio_client`` forwards those parse
+    failures as exceptions, which bleed into Cursor/Cline UIs as warnings
+    when they connect through us. Default ``False`` keeps the strict
+    contract — flip it on per-namespace for known-noisy upstreams.
+    """
 
     kind: Literal["stdio"] = "stdio"
     command: tuple[str, ...]
-    """Argv used to spawn the subprocess."""
     env: dict[str, str] | None = None
-    """Extra environment variables for the subprocess."""
     cwd: Path | None = None
-    """Working directory to spawn the subprocess in."""
     lenient: bool = False
-    """Tolerate non-JSON-RPC stdout lines instead of raising (the strict
-    reader's parse errors surface as warnings in Cursor/Cline). Off by default."""
 
 
 class HttpMcp(_BaseMcp):
-    """Remote MCP server reached over streamable HTTP."""
+    """Remote MCP server reached over streamable HTTP.
+
+    ``headers`` are forwarded on every request to the upstream MCP — handy
+    for upstreams that authenticate via custom header rather than a Bearer
+    token (``auth``).
+    """
 
     kind: Literal["http"] = "http"
     url: str
-    """Streamable-HTTP endpoint URL."""
     auth: str | None = None
-    """Bearer token sent as the upstream's ``Authorization`` header."""
     headers: dict[str, str] | None = None
-    """Extra headers forwarded on every request; for upstreams that
-    authenticate via a custom header instead of ``auth``."""
 
 
 class PythonToolMcp(_BaseMcp):
@@ -56,14 +60,13 @@ class PythonToolMcp(_BaseMcp):
 
     kind: Literal["python_tool"] = "python_tool"
     tools: tuple[Callable[..., object], ...]
-    """The callables to expose as MCP tools."""
 
 
 McpServerT = Annotated[StdioMcp | HttpMcp | PythonToolMcp, Field(discriminator="kind")]
 
 
 class McpServer:
-    """Factory facade for declaring MCP servers in config."""
+    """Factory facade matching the PRD-documented config surface."""
 
     @classmethod
     def stdio(
@@ -75,7 +78,6 @@ class McpServer:
         cwd: Path | str | None = None,
         lenient: bool = False,
     ) -> StdioMcp:
-        """Declare a subprocess MCP server spawned over stdio."""
         return StdioMcp(
             name=name,
             command=tuple(command),
@@ -93,12 +95,10 @@ class McpServer:
         auth: str | None = None,
         headers: dict[str, str] | None = None,
     ) -> HttpMcp:
-        """Declare a remote MCP server reached over streamable HTTP."""
         return HttpMcp(name=name, url=url, auth=auth, headers=headers)
 
     @classmethod
     def python_tool(
         cls, *, name: str, tools: Iterable[Callable[..., object]]
     ) -> PythonToolMcp:
-        """Declare a namespace of Python callables exposed as MCP tools."""
         return PythonToolMcp(name=name, tools=tuple(tools))

src/beaver_gateway/mcp/wrap.py

@@ -1,7 +1,8 @@
 """Wrap a ``PythonToolMcp`` spec into a mountable ``FastMCP`` instance.
 
-Each ``python_tool`` McpServer becomes its own ``FastMCP`` namespace so
-models keep the per-domain tool framing they were trained on.
+Each ``python_tool`` McpServer in the user's config becomes a separate
+``FastMCP`` namespace — one domain, one server URL — so models keep the
+per-domain framing they were trained on (see PRD §6).
 """
 
 from __future__ import annotations

src/beaver_gateway/security/__init__.py

@@ -1 +0,0 @@
-"""Bearer tokens, the audit log and credential redaction."""

src/beaver_gateway/security/audit.py

@@ -1,66 +0,0 @@
-"""Single entry point for writing :class:`AuditLog` rows.
-
-``log()`` is fire-and-forget: it awaits the DB write so callers get
-ordering, but never raises — failures are logged and swallowed.
-"""
-
-from __future__ import annotations
-
-import logging
-from typing import TYPE_CHECKING, Any
-
-from beaver_gateway.storage import append_audit
-
-if TYPE_CHECKING:
-    from beaver_gateway.frontends.base import GatewayRuntime
-
-
-_log = logging.getLogger("beaver_gateway.audit")
-
-
-KNOWN_KINDS: frozenset[str] = frozenset(
-    {
-        "messages",
-        "mcp_call",
-        "tool_call",
-        "login_ok",
-        "login_failed",
-        "logout",
-        "token_create",
-        "token_revoke",
-    }
-)
-"""Kinds the gateway currently emits; ``AuditLog.kind`` stays free-form."""
-
-
-async def log(
-    runtime: GatewayRuntime,
-    *,
-    actor: str,
-    kind: str,
-    agent_name: str | None = None,
-    **detail: Any,
-) -> None:
-    """Best-effort audit insert. Never raises; DB failures are logged.
-
-    ``actor`` is free-form (``"token:<name>"``, ``"admin:<user>"``,
-    ``"anon"``); ``kind`` is a short tag (see :data:`KNOWN_KINDS`); and
-    ``**detail`` is JSON-serialised into ``AuditLog.detail_json`` — keep
-    it small, not full request bodies.
-    """
-    try:
-        async with runtime.db.session() as session:
-            await append_audit(
-                session,
-                actor=actor,
-                kind=kind,
-                agent_name=agent_name,
-                detail=detail or None,
-            )
-    except Exception:  # noqa: BLE001
-        _log.exception(
-            "audit write failed: actor=%s kind=%s agent=%s", actor, kind, agent_name
-        )
-
-
-__all__ = ["KNOWN_KINDS", "log"]

src/beaver_gateway/settings.py

@@ -1,8 +1,8 @@
 """Process-wide configuration loaded from environment (`.env`).
 
-Everything that varies between deployments (creds, paths, ports) lives
-here. User-facing agent/MCP/frontend definitions live in ``config_path``
-and are loaded by :mod:`beaver_gateway.config`.
+Everything that varies between deployments (creds, paths, ports) lives here.
+User-facing agent/MCP/frontend definitions live in ``/config/config.py``
+and are loaded by ``config_loader`` (Phase 0.3).
 """
 
 from __future__ import annotations
@@ -20,22 +20,12 @@ class Settings(BaseSettings):
     )
 
     database_url: str
-    """SQLAlchemy async URL for the gateway's database."""
-
     admin_user: str
-    """Username for the admin UI login."""
-
     admin_pass: str
-    """Password for the admin UI login."""
-
     session_secret: str
-    """Secret used to sign admin UI session cookies."""
 
     internal_mcp_port: int = 8765
-    """Port the gateway's own internal MCP server listens on."""
-
     config_path: Path = Path("/config/config.py")
-    """Where the user-facing agent/MCP/frontend config module lives."""
 
     claude_runner_user: str | None = None
     """Unix user the claude subprocess runs as. Needs the gateway to be root."""
@@ -44,23 +34,23 @@ class Settings(BaseSettings):
     """``HOME`` for the claude subprocess (its ``~/.claude`` lives there)."""
 
     raycast_bearer: str | None = None
-    """Bearer token the gateway presents to the Raycast API."""
-
     raycast_config_path: Path = Path("/config/raycast.json")
-    """Where cached Raycast API config (fetched on first use) is stored."""
-
     raycast_device_id: str | None = None
-    """Stable per-install id (``secrets.token_hex(32)``); required when any
-    ``RaycastAgent`` is configured, so restarts don't look like a new device."""
+    """64-hex-char stable per-install id. Required when any ``RaycastAgent``
+    is configured — generate once via ``secrets.token_hex(32)`` and keep
+    in ``.env`` so Raycast doesn't see every restart as a new device."""
 
     raycast_locale: str = "en-US"
-    """``Accept-Language`` sent to the Raycast API and the default locale for
-    auto ``UserPreferences``. One value per gateway (one shared Client)."""
-
-    send_file_roots: str = "/mnt/hole/shared"
-    """Directories the ``send_file`` tool may read, separated by ``:``. A
-    frontend always adds the inbox its own attachments land in."""
+    """``Accept-Language`` header sent by the shared ``raycast_api.Client``
+    and the default locale used to render auto ``UserPreferences``. One
+    value per gateway since we open one Client total."""
 
     bootstrap_tokens: str = ""
-    """Out-of-band token seed: ``name1:value1,name2:value2``. Layers
-    alongside DB-issued tokens; used for first-run setup and recovery."""
+    """Out-of-band token seed: ``name1:value1,name2:value2``. Empty is
+    fine — DB-issued tokens (admin UI) carry steady-state auth.
+
+    This env channel layers alongside the DB store and is kept for
+    first-run setup, disaster recovery (admin password lost), and
+    ``examples/`` smoke tests. Bootstrap tokens implicitly carry
+    scope ``"*"`` and never get a ``last_used_at`` stamp (no DB row).
+    """

src/beaver_gateway/storage/backfill_usage.py

@@ -1,7 +1,13 @@
-"""Rewrite ``usage.cost_usd``/``model_usage`` as per-turn deltas.
+"""Rewrite ``usage.cost_usd`` / ``model_usage`` as per-turn deltas.
 
-Older rows stored a cumulative total, which overstates daily sums. Safe
-to rerun; reads ``DATABASE_URL`` like the gateway.
+Until 2026-09-01 the gateway stored ``ResultMessage.total_cost_usd`` verbatim,
+which is cumulative for the claude process - summing the column overstated a
+day by an order of magnitude. Run once after deploying the delta-aware
+:func:`beaver_gateway.storage.append_usage`::
+
+    python -m beaver_gateway.storage.backfill_usage
+
+Reads ``DATABASE_URL`` like the gateway (``.env`` included); safe to rerun.
 """
 
 from __future__ import annotations

src/beaver_gateway/storage/db.py

@@ -1,8 +1,15 @@
 """Async ``Database`` wrapper + the bare-minimum CRUD helpers.
 
-psycopg3 gives native async support for Postgres; SQLite goes through
-``aiosqlite``. Helpers take an ``AsyncSession`` so callers can batch
-operations into one transaction.
+Async to match the rest of the stack (aiohttp, uvicorn, claude-code-api).
+psycopg3 has native async support — ``postgresql+psycopg://...`` works
+with ``create_async_engine`` directly. SQLite goes through ``aiosqlite``
+(``sqlite+aiosqlite://...``); user-facing config still uses the plain
+``sqlite:///`` form and we normalise the URL here, so nothing leaks into
+``.env`` / docker-compose.
+
+No repository layer (PLAN §4.1 explicitly waives it). Helpers take an
+``AsyncSession`` so callers can batch operations into one transaction
+(e.g. touch ``last_used_at`` + write an audit line on the same request).
 """
 
 from __future__ import annotations
@@ -123,8 +130,11 @@ def _column_default(column: Any) -> str | None:
     return None
 
 
+# ---- Token CRUD ---------------------------------------------------------
+
+
 async def list_active_tokens(session: AsyncSession) -> Sequence[Token]:
-    """Return every non-revoked token."""
+    """Return every non-revoked token (Phase 4.2 seeds the cache from this)."""
     stmt = select(Token).where(Token.revoked_at.is_(None))  # ty: ignore[unresolved-attribute]
     result = await session.exec(stmt)
     return result.all()
@@ -133,7 +143,7 @@ async def list_active_tokens(session: AsyncSession) -> Sequence[Token]:
 async def list_tokens(
     session: AsyncSession, *, include_revoked: bool = False
 ) -> Sequence[Token]:
-    """Return tokens ordered newest-first."""
+    """Return tokens ordered newest-first (Phase 4.3 admin table)."""
     stmt = select(Token).order_by(Token.created_at.desc())  # ty: ignore[unresolved-attribute]
     if not include_revoked:
         stmt = stmt.where(Token.revoked_at.is_(None))  # ty: ignore[unresolved-attribute]
@@ -164,7 +174,7 @@ async def revoke_token(session: AsyncSession, *, token_id: int) -> bool:
 
 
 async def touch_token(session: AsyncSession, *, token_id: int) -> None:
-    """Bump ``last_used_at`` for one token."""
+    """Bump ``last_used_at``. Phase 4.2 batches these — not per-request."""
     row = await session.get(Token, token_id)
     if row is None:
         return
@@ -173,6 +183,9 @@ async def touch_token(session: AsyncSession, *, token_id: int) -> None:
     await session.commit()
 
 
+# ---- Audit --------------------------------------------------------------
+
+
 async def append_audit(
     session: AsyncSession,
     *,
@@ -210,6 +223,9 @@ async def list_audit_records(
     return result.all()
 
 
+# ---- Usage --------------------------------------------------------------
+
+
 async def append_usage(session: AsyncSession, row: Usage) -> None:
     """Persist a turn's usage with ``cost_usd`` / ``model_usage`` as per-turn deltas.
 
@@ -236,6 +252,8 @@ async def _last_usage(session: AsyncSession, session_id: str | None) -> Usage |
     return (await session.exec(stmt)).first()
 
 
+# Counters inside ``ResultMessage.model_usage[model]``; everything else there
+# (provider, costBasis, contextWindow, maxOutputTokens, ...) is a constant.
 _MODEL_USAGE_COUNTERS = (
     "inputTokens",
     "outputTokens",
@@ -244,8 +262,6 @@ _MODEL_USAGE_COUNTERS = (
     "webSearchRequests",
     "costUSD",
 )
-"""Counters inside ``ResultMessage.model_usage[model]``; other keys there
-are constants."""
 
 
 def usage_deltas(row: Usage, prev: Usage | None) -> None:

src/beaver_gateway/storage/models.py

@@ -1,8 +1,23 @@
 """SQLModel tables.
 
-Flat, no FK relationships modelled — ``actor``/``agent_name`` are plain
-strings. Datetimes are stored UTC via ``default_factory`` so SQLite and
-Postgres behave identically.
+Flat tables, no FK relationships modelled (``actor`` and
+``agent_name`` are stored as strings — joining audit→token by name is
+fine at this volume; we'll introduce FKs when the admin UI actually
+demands them).
+
+The ``Conversation`` + ``ConversationMessage`` pair persists chat
+history per frontend so we can survive cache misses without losing
+tool-call memory. The gateway is now stateful about conversation
+content (we keep the raw Anthropic-shape message list including
+``tool_use`` / ``tool_result`` blocks); the live ``claude-code-api``
+session pool stays the source of truth for *fingerprints*, and the DB
+mirrors what we'd want to re-seed if a session evicts. See
+``core/conversation_store.py`` for the diff-and-fork logic and
+``frontends/markdown/frontend.py`` for the integration point.
+
+Datetimes are stored UTC; we set ``default_factory`` rather than relying
+on DB defaults so SQLite + Postgres behave identically. Every row that
+needs an id uses ``Optional[int]`` so SQLAlchemy can autoincrement.
 """
 
 from __future__ import annotations
@@ -22,8 +37,10 @@ def _utcnow() -> datetime:
 class Token(SQLModel, table=True):
     """Bearer token issued to an external caller.
 
-    ``hashed_value`` holds the Argon2 hash. Plaintext is shown to the
-    user **once** at creation and then discarded.
+    ``hashed_value`` holds the Argon2 hash (Phase 4.2 — until then,
+    rows are written by tests / the admin UI, not by ``TokenStore``).
+    Plaintext is shown to the user **once** at creation and then
+    discarded.
     """
 
     id: int | None = Field(default=None, primary_key=True)
@@ -57,7 +74,7 @@ class AuditLog(SQLModel, table=True):
 
 
 class Conversation(SQLModel, table=True):
-    """One conversation: a master thread, a branch, a deep chat or a job.
+    """One conversation (§3.1): a master thread, a branch, a deep chat or a job.
 
     ``external_id`` is the public id (uuid) every frontend, the API and the
     usage table refer to; ``frontend`` names the frontend that created the
@@ -66,7 +83,7 @@ class Conversation(SQLModel, table=True):
 
     ``running_turn`` survives a restart: a non-null value at startup means
     the gateway died mid-turn and the transcript needs its open ``tool_use``
-    closed before the session is resumed.
+    closed before the session is resumed (``core/conversations``).
     """
 
     __tablename__ = "conversations"
@@ -102,10 +119,10 @@ class Conversation(SQLModel, table=True):
 class ConversationBinding(SQLModel, table=True):
     """Where a conversation shows up: a Telegram topic id, a vault-relative path.
 
-    Invariant: at most one *visible* binding per frontend per conversation -
-    enforced by the partial unique index. The same external id may point at
-    several conversations over time (a renamed topic gets a new branch),
-    only one of them visible.
+    Invariant (§3.1): at most one *visible* binding per frontend per
+    conversation - enforced by the partial unique index. The same external
+    id may point at several conversations over time (a renamed topic gets a
+    new branch), only one of them visible.
     """
 
     __tablename__ = "conversation_bindings"
@@ -130,7 +147,7 @@ class ConversationBinding(SQLModel, table=True):
 
 
 class InjectQueueItem(SQLModel, table=True):
-    """Persisted per-conversation queue, priority ``urgent > user > wake > normal``.
+    """Persisted per-conversation queue (§3.4), ``urgent > user > wake > normal``.
 
     ``status`` walks ``queued -> running -> done``; a row still ``running``
     at startup was cut by a restart and becomes ``interrupted`` - it is
@@ -144,11 +161,6 @@ class InjectQueueItem(SQLModel, table=True):
     priority: str = Field(index=True)
     origin: str = Field(default="system")
     text: str
-    attachments: list[dict[str, Any]] | None = Field(
-        default=None,
-        sa_column=Column(JSON().with_variant(JSONB(), "postgresql"), nullable=True),
-    )
-    """Files that ride with a user message: ``{"path", "media_type"}`` each."""
     status: str = Field(default="queued", index=True)
     turn_id: str | None = Field(default=None)
     interrupted_turn: bool | None = Field(default=None)
@@ -159,7 +171,7 @@ class InjectQueueItem(SQLModel, table=True):
 
 
 class TelegramUpdate(SQLModel, table=True):
-    """Inbox of the Telegram frontend.
+    """Inbox of the Telegram frontend (§3.8).
 
     The update is stored before the poll offset moves past it and handled
     from here, so a restart neither loses nor duplicates a message.
@@ -177,7 +189,7 @@ class TelegramUpdate(SQLModel, table=True):
 
 
 class Delivery(SQLModel, table=True):
-    """Outbox: a reply is a row first and a Telegram message second.
+    """Outbox (§3.8): a reply is a row first and a Telegram message second.
 
     ``status`` walks ``queued -> sent`` with retries on the way, ``failed``
     only when Telegram rejects the row for good (unknown thread, blocked
@@ -349,7 +361,7 @@ class RateLimit(SQLModel, table=True):
 
 
 class JobRunRecord(SQLModel, table=True):
-    """One finished run of a scheduler job: what fired it, how it ended.
+    """One finished run of a scheduler job (§3.6): what fired it, how it ended.
 
     ``error`` keeps the head of the traceback of a failed run; ``payload``
     is the trigger's data (``{}`` for cron) and should stay small.

src/beaver_gateway/storage/session_store.py

@@ -1,7 +1,9 @@
 """``SessionStore`` adapter for the Claude Agent SDK on top of :class:`Database`.
 
-Entries are JSON rows ordered by a per-key ``seq``; ``append`` is
-idempotent on ``entry["uuid"]`` since the SDK may re-deliver a batch.
+Entries are stored verbatim as JSON (``jsonb`` on Postgres), ordered by a
+per-key ``seq``. ``append`` is idempotent on ``entry["uuid"]`` because the
+SDK re-delivers a batch on retry; entries without a uuid are appended as
+they come. Runs on SQLite too - the conformance suite uses that in tests.
 """
 
 from __future__ import annotations

src/beaver_gateway/vault/__init__.py

@@ -1 +0,0 @@
-"""Watching a directory of notes for the envelope."""

src/beaver_gateway/vault/links.py

@@ -1,136 +0,0 @@
-"""Wikilink index over a vault: which note links to which, refreshed by mtime."""
-
-from __future__ import annotations
-
-import os
-import re
-import time
-from dataclasses import dataclass, field
-from pathlib import Path
-from typing import Any
-
-__all__ = ["LinkIndex"]
-
-_WIKILINK = re.compile(r"\[\[([^\]|#\n]+)(?:#[^\]|\n]*)?(?:\|[^\]\n]*)?\]\]")
-_MD = ".md"
-
-
-@dataclass(slots=True)
-class _Note:
-    mtime: float
-    links: set[str] = field(default_factory=set)
-
-
-class LinkIndex:
-    """Forward and backward links between the vault's markdown notes."""
-
-    def __init__(self, root: Path, *, max_age: float = 20.0) -> None:
-        self.root = root.resolve()
-        self.max_age = max_age
-        self._notes: dict[str, _Note] = {}
-        self._raw: dict[str, set[str]] = {}
-        self._scanned_at = 0.0
-
-    def refresh(self, *, force: bool = False) -> None:
-        if not force and time.monotonic() - self._scanned_at < self.max_age:
-            return
-        seen: set[str] = set()
-        for dirpath, dirnames, filenames in os.walk(self.root):
-            dirnames[:] = [d for d in dirnames if not d.startswith(".")]
-            for name in filenames:
-                if not name.endswith(_MD):
-                    continue
-                path = Path(dirpath) / name
-                rel = str(path.relative_to(self.root))
-                seen.add(rel)
-                try:
-                    mtime = path.stat().st_mtime
-                except OSError:
-                    continue
-                note = self._notes.get(rel)
-                if note is None or note.mtime != mtime:
-                    self._notes[rel] = _Note(mtime=mtime)
-                    self._raw[rel] = self._targets(path)
-        for rel in list(self._notes):
-            if rel not in seen:
-                self._notes.pop(rel, None)
-                self._raw.pop(rel, None)
-        self._resolve()
-        self._scanned_at = time.monotonic()
-
-    @staticmethod
-    def _targets(path: Path) -> set[str]:
-        try:
-            text = path.read_text(encoding="utf-8")
-        except (OSError, UnicodeDecodeError):
-            return set()
-        return {m.group(1).strip() for m in _WIKILINK.finditer(text)}
-
-    def _resolve(self) -> None:
-        by_stem: dict[str, str] = {}
-        for rel in sorted(self._notes):
-            by_stem.setdefault(Path(rel).stem.casefold(), rel)
-        by_path = {rel.casefold(): rel for rel in self._notes}
-        for rel, targets in self._raw.items():
-            links: set[str] = set()
-            for target in targets:
-                candidate = target if target.endswith(_MD) else f"{target}{_MD}"
-                found = by_path.get(candidate.casefold()) or by_stem.get(
-                    Path(target).stem.casefold()
-                )
-                if found and found != rel:
-                    links.add(found)
-            self._notes[rel].links = links
-
-    def resolve(self, raw: str) -> str:
-        """A path as a tool named it (absolute or relative) as an index key."""
-        path = Path(raw)
-        if path.is_absolute():
-            try:
-                return str(path.resolve().relative_to(self.root))
-            except (ValueError, OSError):
-                return raw
-        return raw
-
-    def neighbours(self, paths: list[str], *, limit: int = 80) -> dict[str, Any]:
-        """Nodes for ``paths`` plus every note one link away, and their edges."""
-        self.refresh()
-        touched = [self.resolve(p) for p in paths]
-        backlinks: dict[str, set[str]] = {}
-        for rel, note in self._notes.items():
-            for target in note.links:
-                backlinks.setdefault(target, set()).add(rel)
-        nodes: dict[str, dict[str, Any]] = {}
-        for rel in touched:
-            nodes[rel] = {
-                "path": rel,
-                "title": Path(rel).stem,
-                "touched": True,
-                "exists": rel in self._notes,
-            }
-        for rel in touched:
-            note = self._notes.get(rel)
-            around = (note.links if note else set()) | backlinks.get(rel, set())
-            for other in sorted(around):
-                if len(nodes) >= limit:
-                    break
-                nodes.setdefault(
-                    other,
-                    {
-                        "path": other,
-                        "title": Path(other).stem,
-                        "touched": False,
-                        "exists": True,
-                    },
-                )
-        edges: list[dict[str, str]] = []
-        for rel in nodes:
-            note = self._notes.get(rel)
-            if note is None:
-                continue
-            edges.extend(
-                {"from": rel, "to": target}
-                for target in sorted(note.links)
-                if target in nodes
-            )
-        return {"nodes": list(nodes.values()), "edges": edges}

tests/test_api.py

@@ -18,18 +18,18 @@ from claude_agent_sdk import (
 from httpx import ASGITransport, AsyncClient
 from test_conversations import ScriptedClient, World
 
-from beaver_gateway.frontends.markdown.history import rewrite_messages
-from beaver_gateway.security.auth import TokenStore
-from beaver_gateway.app import McpRegistry
-from beaver_gateway.jobs.scheduler import Job, JobRun, Scheduler
-from beaver_gateway.backends.transcript import build_entries
+from beaver_gateway.core.conversation_store import rewrite_messages
+from beaver_gateway.core.auth import TokenStore
+from beaver_gateway.core.registry import McpRegistry
+from beaver_gateway.core.scheduler import Job, JobRun, Scheduler
+from beaver_gateway.core.transcript import build_entries
 from beaver_gateway.frontends.admin import AdminFrontend
 from beaver_gateway.frontends.admin.frontend import build_app as build_admin
 from beaver_gateway.frontends.api import ApiFrontend
 from beaver_gateway.frontends.api.frontend import build_app as build_api
 from beaver_gateway.frontends.base import Frontend, GatewayRuntime
 from beaver_gateway.frontends.root import build_root_app
-from beaver_gateway.storage.models import RateLimit, TranscriptEntry, Usage
+from beaver_gateway.storage.models import RateLimit, Usage
 
 TOKEN = "tok"
 HEADERS = {"Authorization": f"Bearer {TOKEN}"}
@@ -247,7 +247,6 @@ async def test_usage_groups_by_agent_day_and_model(world: World) -> None:
 async def test_limits_report_latest_window_with_gateway_spend(world: World) -> None:
     api = Api(world)
     now = datetime.now(UTC)
-    resets = int((now + timedelta(hours=4)).timestamp())
     async with world.db.session() as session:
         session.add(
             RateLimit(
@@ -265,20 +264,7 @@ async def test_limits_report_latest_window_with_gateway_spend(world: World) -> N
                 resets_at=now + timedelta(hours=4),
             )
         )
-        session.add(
-            RateLimit(
-                window="seven_day",
-                status="allowed",
-                utilization=0.3,
-                raw={
-                    "unifiedWindows": {
-                        "five_hour": {"resetsAt": resets, "utilization": 0.72},
-                        "seven_day": {"resetsAt": resets, "utilization": 0.25},
-                        "seven_day_fable": {"resetsAt": resets, "utilization": 0.1},
-                    }
-                },
-            )
-        )
+        session.add(RateLimit(window="seven_day", status="allowed", utilization=0.3))
         await session.commit()
     await seed_usage(
         world,
@@ -294,19 +280,12 @@ async def test_limits_report_latest_window_with_gateway_spend(world: World) -> N
     )
     out = await api.get("/limits")
     windows = {w["window"]: w for w in out["windows"]}
-    assert windows["five_hour"]["utilization"] == 0.72
-    assert windows["five_hour"]["reported_at"] is not None
+    assert windows["five_hour"]["utilization"] == 0.9
     assert windows["five_hour"]["status"] == "allowed_warning"
     assert windows["five_hour"]["gateway"]["output"] == 7
     assert windows["five_hour"]["gateway"]["cost_usd"] == 0.25
     assert windows["seven_day"]["utilization"] == 0.3
-    assert windows["seven_day_fable"]["utilization"] == 0.1
-    assert windows["seven_day_fable"]["status"] == "allowed"
-    assert [w["window"] for w in out["windows"]] == [
-        "five_hour",
-        "seven_day",
-        "seven_day_fable",
-    ]
+    assert [w["window"] for w in out["windows"]] == ["five_hour", "seven_day"]
     assert len(out["history"]) == 3
 
 
@@ -494,7 +473,7 @@ async def test_close_distills_a_deep_chat(world: World) -> None:
     assert body["digest"] == str(config.dir / "2026-08-29 - тема.md")
     assert body["text"].count("\n") == 2
     assert (await world.conversations.get(chat.external_id)).status == "closed"
-    assert (await world.conversations.queue.recent(master.id))[0].origin == "digest"
+    assert (await world.conversations.queue.recent(master.id))[0].origin == "выжимка"
     again = await api.http.post(
         f"/conversations/{chat.external_id}/close", headers=HEADERS
     )
@@ -546,91 +525,3 @@ async def test_bind_without_a_window_materializes_one(world: World) -> None:
         headers=HEADERS,
     )
     assert unknown.status_code == 400
-
-
-async def test_search_finds_conversations_and_memory_files(world: World) -> None:
-    root = Path(tempfile.mkdtemp(prefix="beaver-mem-"))
-    (root / "notes").mkdir()
-    (root / "notes" / "studio.md").write_text("# studio\nthe kiln is warm\n")
-    api = Api(world, memory_root=root)
-    conv = await world.conversations.create(kind="master", agent="a", origin="test")
-    await world.conversations.set_title(conv, "kiln schedule")
-    other = await world.conversations.create(kind="deep", agent="d", origin="test")
-    await world.conversations.post(other, "go")
-    await world.settle(other, 1)
-    other = await world.conversations.get(other.external_id)
-    assert other is not None and other.session_id is not None
-    async with world.db.session() as db:
-        db.add(
-            TranscriptEntry(
-                project_key="p",
-                session_id=other.session_id,
-                seq=0,
-                entry={"type": "assistant", "message": {"content": "warm kiln"}},
-            )
-        )
-        await db.commit()
-    found = await api.get("/search", {"q": "kiln"})
-    assert {c["id"] for c in found["conversations"]} == {
-        conv.external_id,
-        other.external_id,
-    }
-    assert found["files"] == [
-        {"path": "notes/studio.md", "line": 2, "snippet": "the kiln is warm"}
-    ]
-    short = await api.http.get("/search", params={"q": "k"}, headers=HEADERS)
-    assert short.status_code == 400
-
-
-async def test_context_endpoint_reports_prompt_and_files(world: World) -> None:
-    api = Api(world)
-    conv = await world.conversations.create(kind="master", agent="a", origin="test")
-    await world.conversations.post(conv, "go")
-    await world.settle(conv, 1)
-    conv = await world.conversations.get(conv.external_id)
-    assert conv is not None and conv.session_id is not None
-    await world.store.append(
-        world.key(conv.session_id),
-        build_entries(
-            [
-                {"role": "user", "content": "go"},
-                {
-                    "role": "assistant",
-                    "content": [
-                        {
-                            "type": "tool_use",
-                            "id": "t1",
-                            "name": "Read",
-                            "input": {"file_path": str(world.root / "note.md")},
-                        }
-                    ],
-                },
-                {
-                    "role": "user",
-                    "content": [
-                        {"type": "tool_result", "tool_use_id": "t1", "content": "x"}
-                    ],
-                },
-            ],
-            session_id=conv.session_id,
-            cwd=str(world.root),
-            model="m",
-        ),
-    )
-    ctx = await api.get(f"/conversations/{conv.external_id}/context", {"prompt": 1})
-    assert ctx["agent"]["name"] == "a"
-    assert ctx["prompt"]["granules"][0]["bytes"] == len("hi")
-    assert ctx["prompt_text"] == "hi"
-    assert ctx["files"] == [
-        {
-            "path": "note.md",
-            "reads": 1,
-            "writes": 0,
-            "other": 0,
-            "last_at": ctx["files"][0]["last_at"],
-        }
-    ]
-    assert ctx["tool_counts"] == {"Read": 1}
-    assert ctx["turns"] == 1
-    graph = await api.http.get("/vault/graph", headers=HEADERS)
-    assert graph.status_code == 404

tests/test_auth_query_token.py

@@ -7,8 +7,8 @@ import pytest
 from fastapi import HTTPException
 from starlette.requests import Request
 
-from beaver_gateway.security.auth import TokenStore
-from beaver_gateway.frontends.bearer import require_token
+from beaver_gateway.core.auth import TokenStore
+from beaver_gateway.frontends._auth import require_token
 
 
 def _request(query: str = "", headers: dict[str, str] | None = None) -> Request:
@@ -61,7 +61,7 @@ async def test_bootstrap_entry_can_carry_a_scope() -> None:
 
 
 def test_access_log_filter_masks_query_tokens() -> None:
-    from beaver_gateway.security.redact import RedactFilter
+    from beaver_gateway.core.redact import RedactFilter
 
     record = logging.LogRecord(
         "uvicorn.access",