beaver-agent/README.md

# beaver-agent

My real [beaver-gateway](https://git.kotikot.com/beaver/beaver-gateway) setup: the Beavering Manager, a personal agent that lives on a home server, reads an Obsidian vault, talks through Telegram and deep chats in the vault, remembers people and promises, and has hands (Komodo, Home Assistant, T3 Code threads, calendars, Firefly, Telegram history through beavergram). The narrative and the docs are on [the site](https://beaver.kotikot.com/docs); this README is the operator's part: how to run it.

Uses Claude Code and Raycast subscriptions for agents, has some MCPs set up. You can use this as-is or modify to match your needs: `config.py` assembles the gateway from the `beaver_agent/` package. It is opinion, not core - every piece is meant to be rewritten for your own vault.

Seeding memory from an existing vault (old chats into digests, diary days and thoughts into notes about people) is one run of `scripts/seed_memory.py`; `--dry-run` first.

## Requirements

- Docker + Docker Compose
- Obsidian Sync (for `obsidian-headless`)
- claude.ai sub
- raycast sub
- Firefly III and its PAT

## First launch

### 1. Create `.env` and `raycast.json`

```bash
cp .env.example .env
# every empty value is documented inline; secrets: openssl rand -hex 32
nvim .env

# build raycast config on your mac (beta by default, consider checking raycast-api for instructions)
raycast-api init

# copy config.json to your deployment server

docker compose up -d
```

### 2. Set up Obsidian Sync

```bash
docker exec -it beaver-obsidian ob login
docker exec -it beaver-obsidian ob sync-setup --vault "yourvault"
docker restart beaver-obsidian
```

Check:

```bash
docker exec beaver-obsidian ls /vault
```

Only markdown is synced by default. To sync everything:

```bash
docker exec beaver-obsidian ob sync-config \
  --file-types image,audio,video,pdf,unsupported
docker restart beaver-obsidian
```

### 3. Set up claude

On your mac run `claude setup-token` and put the result into `.env` as
`CLAUDE_CODE_OAUTH_TOKEN`. That is the whole auth: no `/login` inside the
container, no dialogs. The token draws from your subscription limits like
regular Claude Code; rotate it by running `claude setup-token` again.

Claude agents run on the Claude Agent SDK. Prompts are granules under
`мета/бобер/промпты/` in the vault - plain markdown, the `<tag>` wrappers
and the order live in `beaver_agent/prompts.py`; skills are the folders under
`мета/бобер/скиллы/` (each becomes a plugin, keep `name:` in SKILL.md
latin), which window loads which set is `beaver_agent/skills.py`. The vault
must be synced before the gateway can start. The model process runs as
`beaver-runner` with a whitelisted environment; the vault is mounted
read-write (it is the dispatcher's home), and `beaver_agent/policy.py` keeps
it out of `.obsidian`, `мета` outside `мета/бобер`, and from deleting
anything but its own files.

### 4. Mint a token

Open admin at `http://localhost:62990/` (or `https://<DOMAIN>/` if Caddy), sign in with `ADMIN_USER` / `ADMIN_PASS`, go to **Tokens → Create**, scope `*` for first run.

### 5. Smoke test

```bash
docker exec beaver-gateway ls /vault | head
docker logs beaver-gateway | grep -i "agent registered"

curl http://localhost:62990/anthropic/v1/models \
  -H "Authorization: Bearer <YOUR_TOKEN>"
```

## Where to plug things in

The admin dashboard renders ready-to-copy URLs and snippets for each of these

Everything is on one port under its own path; Caddy forwards the whole domain as is.

- any Anthropic client: `http://localhost:62990/anthropic` or `https://<DOMAIN>/anthropic`, model = agent name (`beaver-opus-high` etc)
- MCP clients (Claude Desktop, Raycast extension): `/mcp/<name>/`, discovery page at `/mcp/`
- Obsidian companion plugin: `/md` as the plugin's "Base URL", `/api` for the panel
- **Admin UI:** `http://localhost:62990/` or `https://<DOMAIN>/`, login from `ADMIN_USER` / `ADMIN_PASS`

## Exposing to the internet

`caddy/site.caddy` holds the routes (one `reverse_proxy`, domain from `BEAVER_DOMAIN`); on a shared server it is bind-mounted into that server's Caddy as `projects.d/beaver-agent/`, on its own box `caddy/docker-compose.yml` runs a Caddy around it.

```bash
cd caddy
cp Caddyfile.example Caddyfile
cp .env.example .env   # BEAVER_DOMAIN, BEAVER_UPSTREAM, Cloudflare token if any
docker compose up -d
```

Set `PUBLIC_BASE_URL=https://<DOMAIN>` in beaver-agent's `.env` so advertised endpoints use the outside origin.

## Deploy: `main` is work, `stable` is production

Komodo deploys only the `stable` ref; pushing `main` deploys nothing. Commit
during the day, then ship when the master thread is idle:

```bash
make deploy                  # git push origin main:stable
```

The gateway image is built from `beaver-gateway#${GATEWAY_REF}` at deploy
time, so push the gateway first. `docker-compose.yml` has a healthcheck on
`/healthz`; Komodo waits for it before it considers the deploy done.

To drain first (архитектура §8.6), call the deploy hook instead of pushing
by hand: it waits until no master turn is running, then asks Komodo:

```bash
curl -X POST https://<DOMAIN>/hooks/deploy -H "Authorization: Bearer <TOKEN>"
```

Needs `KOMODO_URL` / `KOMODO_KEY` / `KOMODO_SECRET` in `.env` (and
`KOMODO_STACK`, default `beaver-agent`). Komodo alerts can be pointed at
`POST /hooks/komodo` the same way - they land in the master as an urgent
inject. Jobs, the queue and the subscription window are on the admin **Jobs** page.

The same key backs the `komodo` tool the dispatcher and deep chats get
(`beaver_agent/hands/komodo.py`): one MCP tool with an enumerated
`action` - `status`, `stacks`, `containers`, `logs`, `search_logs`, `updates`,
`update`, `deploy`, `restart`, `exec`. `exec` is `docker exec` into any
container of the fleet except infrastructure ones (`exec_deny`, periphery by
default); prune, destroy and a host terminal do not exist in the enumeration,
so they cannot be asked for. The key never reaches the model process.

## Layout: `config.py` + `beaver_agent/`

`config.py` is a page: it imports the pieces and calls `Gateway(...)`. The
pieces live in `beaver_agent/`, one concern per module:

- `vault.py` - paths and the agent's zone, `TZ`
- `prompts.py` - how each role's system prompt is assembled from granules
- `skills.py` - which window loads which skill set
- `policy.py` - `PreToolUse` rules and vault zones (`ZONES`, `VAULT_POLICY`)
- `hands/` - every MCP the setup has and which agent gets which one; the
  connectors (komodo, home assistant, vibegram) live here too
- `agents.py` - the roles (dispatcher, deep, distiller, curator, triage,
  raycast) and the list of instances with models and effort
- `frontends.py` - the windows: Telegram for master and branches, vault
  files for deep chats, `/api`, `/admin`, `/anthropic`, `/mcp`
- `texts.py` - everything the gateway says to the model or to the user,
  in Russian (the gateway's own defaults are English)
- `memory/` - the recall block under a message, the reply log, handouts
  and seeds, the vault watcher, the curator briefing
- `jobs/` - crons and webhooks, one file per job

## Leisure: the hourly tick (`beaver_agent/jobs/leisure.py`)

The master gets an inject with no task in it every hour: what time it is,
what kind of hour this is, and what is left of the day's plan. The active
window (12:00-03:59) ticks every hour; the quiet one (04:00-11:59, the hours
the owner has never once written in) ticks only in the four hours the plan
names - twenty ticks a day.

The day runs noon to noon and the first tick of it is the planning one: the
model lays the remaining hours out into *mine* (leisure only, the owner's
business stays out) and *mixed*, and writes them to `мета/бобер/досуг.md`:

```markdown
досуг: вкл
сутки: 2026-09-08
тихие часы: 04, 06, 08, 10
мои часы: 13, 15, 18, 21, 00, 02, 06
```

**The model writes that file, the job only reads it**, so they cannot
overwrite each other; how many leisure slots are spent follows from the plan
and the clock, so there is no state to keep. No plan for the day (a restart,
a fresh day, a broken file) is not an error: the job falls back to defaults
and says so in the inject. Fewer than seven leisure hours in the plan are
topped up to seven by the job - the guarantee is the mechanism's, not the
model's.

A tick lands as `wake` and starts a turn once the master is idle; a busy
master or a message less than five minutes old moves it five minutes on,
up to three times, so it never cuts into a live exchange. If a `schedule`
inject of the model's own is due later in the same hour, the tick goes in
as `normal` instead and rides into that turn with it - one turn, both
reasons, neither swallowed. The cron sits at `:07`, not `:00`, because
rotation is on the hour and a tick waiting in the queue counts as a busy
master.

Nothing of this reaches Telegram by itself: a turn started by an inject
publishes no reply, so the only voice in it is the `say` tool.
Switches: `досуг: выкл` as the first line of the file for every day, and
`LEISURE=off` in `.env` to drop the job entirely.

## The watchdog ring (`beaver_agent/jobs/heartbeat.py`)

An agreement with a foreign agent on someone else's home server: both print
one line a day into a vibegram room of their own, and both watch the other's
silence. The two rates are deliberately different: the beat is daily because
that is the period we promise the neighbour, while the check runs every six
hours, because a watchdog stepping once a day against a 36 h threshold would
only notice silence some 60 h in - it would be measuring the wrong thing. If the neighbour goes quiet past the threshold, *their* human is the
one who needs to hear about it, so the alarm goes to the master thread as a
spawned triage job and the dispatcher decides whether to wake the owner.

The room is a **second, independent** `Vibegram` client (`VIBEGRAM_HB`), not
the one the dispatcher's `vibegram` tool and the triage job share: one client
for both rooms would mean the triage job eats the heartbeats off the shared
`/api/pending` cursor, and the heartbeats drown in conversation. The model
gets no tool for this room - the watchdog is a mechanism.

The point of a watchdog is that it does not lie: an instrument that quietly
fails to watch is worse than no instrument, because it looks healthy. So there
are no defaults anywhere in it.

- `отметка` (17:12 local) prints `hb <nick> at=… boot=… every=… src=…`. The
  period is not a constant: it is derived from that job's own registered cron
  (`run.job.cron`). A schedule that does not reduce to "every N hours" gets
  `every=unknown src=none` - lying with a number is worse than refusing.
- `сторож` (`:42` every six hours) reads a window - **the last 25 messages of
  that room** (`Vibegram.window`) - finds the newest `hb claude-kai …`, and
  compares its age against
  `every * 1.5` where `every` comes **from the neighbour's own line**. The
  clock it trusts is `seen` - when *we* saw the line - not the `at=` inside it,
  which is the neighbour's statement about himself.
- Every run prints `ack claude-kai last=… seen=… res=… win=… armed=… thr=…`
  into the room. Anything the run could not measure says so out loud there:
  `every=unknown` in his line falls back to the last known period and marks the
  threshold `thr=36h-fallback`; a period never read once is `thr=unknown` and
  no alarm is taken.
- Never having seen a beat is not an alarm, it is `armed=no` - there is nothing
  to watch yet. Seven days of `armed=no` in a row tells the human once, so an
  unfinished ring does not quietly become permanent.
- The window is **not** `/api/pending`. That endpoint is a cursor: it hands a
  line over once and never again, so the neighbour's beat showed up in one run
  and was gone from the next one's window on a perfectly live room - the room
  held it the whole time. The tail comes from `/api/state?view=…` instead,
  which is already scoped to the room (hub event ids run across all rooms) and
  pages **forward** in chunks of 200 from `after=<id>`; the watchdog walks the
  history and keeps the last 25 *messages*, joins and leaves not counted.
- **An unread window is not a reading.** If the history could not be walked to
  its end (the page budget ran out), the missing part is exactly the newest,
  so `win=25/partial` goes into the `ack`, the log says `окно не дочитано`, and
  no alarm is taken in that run: "nobody in the window" and "the window was
  never finished" must not look the same.
- If the window's highest event id went **backwards** since the last run, the
  window is looking at the past: `win=stale`, and an alarm in that run is
  forbidden. Same for a hub that did not answer at all.
- The alarm names all three possible causes - the neighbour's machine, his
  cron, or the channel (the room itself) - and if our own write into the room
  went through in that same run while neither `hb` nor `ack` came back from
  him, it says to suspect the channel before the neighbour's death. It repeats
  at most once a day, not once a run.

State between runs is `мета/бобер/.состояние/сторож.json`; a lost file means
the ring disarms itself and starts the seven-day count again.

Setup is two variables in `.env` and one manual step:

```bash
# join the watchdog room with the invite code the neighbour sends
python3 scripts/vibegram_join.py <join-code> --nick <nick>
# put the token into .env as VIBEGRAM_HB_TOKEN, then
docker compose restart gateway
```

`VIBEGRAM_HB_NICK` is optional - without it the nick is `VIBEGRAM_NICK`. With
no `VIBEGRAM_HB_TOKEN` both jobs do nothing and the log says once at startup
that the watchdog is not configured. The neighbour's nick is the `PEER`
constant in `beaver_agent/hands/heartbeat.py`.

## The job-run watchdog (`beaver_agent/jobs/runs.py`)

The scheduler has always written one row per run into `job_runs` - name,
status, the head of the traceback in `error`. Nobody ever read it. The channel
to the neighbour agent was dead for 38 hours straight, the scheduler recorded
that 231 times, and nothing reached a human: the outage surfaced ten days later,
by hand, out of curiosity. The record existed, the reader did not. This job is
the reader.

- `прогоны` (`:27` every hour) reads `job_runs` for the last 7 days through
  the same connection the scheduler itself writes with
  (`run.conversations.db.session()` and the `JobRunRecord` model) - no second
  truth about where the journal lives.
- An alarm is taken only off the **tail**: three or more `failed` in a row
  counting back from a job's latest run, i.e. the job is *lying down* rather
  than having blinked once. A single failure is a network hiccup, and waking
  a human for it teaches them not to look.
- The alarm says the job's name, how many runs in a row died, when the series
  started, and the **first line** of the error - never the traceback. Replayed
  over the real outage it reads: «джоб `вайбграм` лежит 23 часа и дольше,
  144 прогона подряд с 12.09 22:00 UTC: HubError: vibegram 530 bad_json». It repeats at most once a day per job, and
  when the job recovers one short `отбой` line goes out and the counter resets.
  The window is a week, not a day, so a daily job (`отметка`, `закрытие`,
  `голосовой`) gets its three failures in a row in three days.
- **A job that stopped running is an alarm too, with its own text**: «джоб
  `закрытие` не запускался 2 суток при периоде 24 часа». For every job with a
  cron, the period comes from the registered cron (`run.scheduler.jobs`) via
  the same `every_from_cron` the neighbour watchdog uses, and the last run is
  taken from the whole journal, not the window. The threshold is **two
  periods**: one missed fire happens honestly (a redeploy landed on the cron
  minute, and the scheduler does not catch up), two in a row do not. It is
  never finer than two of the watchdog's own periods (2 hours), because an
  hourly watcher cannot tell a shorter gap from its own blindness between
  runs. Time when the scheduler was not running non-critical jobs at all is
  not counted as a miss: before the process started, and while the
  watchdog's own hourly chain is broken (the gateway was down, or the budget
  was throttled and non-critical crons were deferred without a record). Same
  discipline as failures: once a day per job, and an `отбой` when a run shows
  up.
- What silence does not cover, on purpose: jobs without a cron (webhooks,
  events) are not measured at all; a job that has never appeared in the
  journal waits silently for its first row; and a cron that does not reduce to
  a regular period (`куратор`, `0 2,8,12,16,20 * * *`) is skipped and named in
  the log as unguarded every run, because any number put in its place would
  be made up. `куратор` still falls under the failure count.
- **An empty answer is a statement about the query, not about health.** Zero
  rows in the last 24 hours (other than this job's own) means no job ran at all, and
  that is its own alarm with its own text - the scheduler is what to look at,
  not any one job. A failed query is no reading whatsoever: the run logs it and
  stays silent, because a database that did not answer is not evidence of
  health or of illness.
- Everything that goes out is masked (`mask`): the gateway's shared `redact`
  plus two local rules for what it cannot catch by construction - a
  `NAME=value` pair in the middle of a line (a traceback puts them there) and
  the query string of any URL, which is dropped whole because the parameter
  carrying the key can be named anything.

State between runs is `мета/бобер/.состояние/прогоны.json` (who was told and
when, separately for failures and for silence). A lost file means a lying job is reported again and an `отбой` for
something reported before the loss never comes - the evidence for it is gone.

One blind spot, deliberate: **the watchdog cannot report its own death.** Its
own runs are excluded from both the failure count and the silence check
(`run.job.name`), because a job that is down will not file a report about
itself - an observer inside the object does not report the object's death. If
`прогоны` itself starts failing or stops running, nothing here will say so;
that needs an outside eye, like the watchdog ring above.

## Policy (`beaver_agent/policy.py`)

`bypassPermissions` everywhere; the boundary is `PreToolUse` rules declared
per agent (`ClaudeAgent.policy`): the vault is the dispatcher's home, so it
may create, edit and move anywhere except `.obsidian` and `мета` outside
`мета/бобер`, and may delete only its own files; the distiller and curator
stay in `мета/бобер`; `rm`/`mv`/`cp`/`tee`/`sed -i`/redirects into the vault
outside those zones are refused with a reason; `mcp__firefly__store_*` and
`update_*` need the `firefly` skill opened first in the same session, and the
master cannot open the `vault/` skills at all (a branch can). Every tool call
lands in the admin **Audit** page as `tool_call`. Run `make check`.

## Editing the setup on the mac

`make sync` installs `../beaver-gateway` editable (extra `local`), so
imports resolve against the checkout you are working on; `uv sync --extra
prod` pulls the gateway from git instead. Neither is used by the image - the
container builds the gateway itself. To assemble the config against your
local vault: `BEAVER_VAULT=~/path/to/vault uv run python -c
'from beaver_gateway import config; from pathlib import Path;
config.load(Path("config.py"))'` with a `.env` next to it.

## Useful commands

```bash
# shell into the gateway container (claude CLI, bun, python with venv all live here)
docker exec -it beaver-gateway bash

# what obsidian-sync has pulled
docker exec beaver-obsidian ob status

# force a one-off sync (don't wait for the continuous tick)
docker exec beaver-obsidian ob sync

# per-service logs
docker compose logs -f gateway
docker compose logs -f obsidian-headless

# pull the latest beaver-gateway (when GATEWAY_REF=main)
docker compose build gateway && docker compose up -d gateway

# reload config.py / beaver_agent without a full rebuild
docker compose restart gateway
```

## Gotchas

- **claude in the container doesn't see the vault** - `VAULT` in `beaver_agent/vault.py` is `/vault` *inside* the container, not on the host. Don't change it; `BEAVER_VAULT` is for assembling the config on the mac only.
- **a claude turn fails immediately** - check `docker logs beaver-gateway` for the `claude[<agent>]:` stderr lines. Usually auth: redo step 3.
- **gateway restarts in a loop right after first `up`** - `beaver_agent/prompts.py` reads prompt granules from `/vault/мета/бобер/промпты`; until Obsidian Sync has pulled the vault they are missing. Finish step 2, it settles.
- **the model cannot write into `мета/бобер`** - the entrypoint grants `beaver-runner` an ACL on the two rw sub-mounts once at start; files that Sync creates later inherit it through the default ACL. If `setfacl` is unsupported on the volume it falls back to `chown`, and then files Sync writes afterwards as root stay read-only for the model until the next restart.