Compare commits

..
56 Commits
Author SHA1 Message Date
hh c9c2e94954 refactor: no comments left - one-line module docstrings, contracts on public fields only; jobs/job.py; example config and README 2026-09-02 00:33:04 +02:00
hh 90d0fb0f00 refactor(telegram): every user-facing string lives in TelegramTexts with English defaults 2026-09-02 00:16:38 +02:00
hh cae2ed4161 refactor: split flat core into capability packages, layer the conversations service, English defaults for every model-facing text 2026-09-02 00:13:20 +02:00
hh b96714338f fix(telegram,conversations): attachments by day with sweep in both modes, merge renames the topic, gone topics unbind, failures visible 2026-09-01 23:13:22 +02:00
hh 7853d61b94 feat(telegram): block-level markdown to html, wikilinks, tables, quotes, fence-safe chunking 2026-09-01 23:13:21 +02:00
hh f6a7ea8f29 feat(agents,claude_sdk): skill sets per conversation kind 2026-09-01 23:13:21 +02:00
hh 1954a6e9f2 feat(scheduler,conversations,api): cron and schedule in the gateway tz, job runs persisted 2026-09-01 23:13:17 +02:00
hh 9b1127acff chore(deps): claude-agent-sdk 0.2.150 bundles cli 2.1.257 2026-09-01 22:00:41 +02:00
hh 1e08dfb9da fix(mcp,redact,gateway_tools): mask credentials in tool results, names kept and values gone 2026-09-01 14:12:49 +00:00
hh 8e7db6baa4 fix(redact,cli): mask credentials in every log line, silence httpx request urls 2026-09-01 14:12:49 +00:00
hh acc79c2e42 feat(envelope,conversations): recall hook under the user text, user_sink for the setup's reply log 2026-09-01 16:08:53 +02:00
hh 63014a487d fix(storage,api): usage cost as per-turn delta of the sdk running total, backfill 2026-09-01 15:37:53 +02:00
hh 3406e2e178 feat(injects,conversations,scheduler,gateway_tools,api): wake priority, master and parent aliases, scheduled injects start a turn 2026-08-30 20:34:12 +02:00
hh 878f7d6473 fix(cli): mask query tokens in uvicorn access log 2026-08-30 18:30:26 +02:00
hh c438d4cb92 feat(auth): bootstrap token entries carry an optional scope 2026-08-30 18:25:48 +02:00
hh ba9d63667e feat(auth): query-string token for headerless webhook senders 2026-08-30 18:10:00 +02:00
hh d57f660a30 fix(gateway_tools,conversations): say refused in user turns, the reply reaches the human itself 2026-08-30 02:26:35 +02:00
hh cc32307921 feat(scheduler): job dedupe flag for webhooks that must not collapse 2026-08-29 22:15:15 +02:00
hh 811035ba4f fix(ui): chat keeps the bottom pinned while async markdown lands 2026-08-29 19:26:45 +02:00
hh 9efff6e814 feat(ui,api,telegram): shared conversation panel for obsidian and /admin/panel, bind materializes a window, inbox survives link previews 2026-08-29 18:30:33 +02:00
hh a989d2909d feat(injects,conversations): configurable inject header, urgent inject marks the turn it cut 2026-08-29 17:07:43 +02:00
hh 8d252f9867 fix(claude_sdk,conversations,rotation): context size from the last api call, new-day text by rotation reason 2026-08-29 16:54:29 +02:00
hh 2895e677c0 feat(policy,claude_sdk,cli,config_loader): pretooluse policy rules per agent, tool call audit, sibling imports for config 2026-08-29 15:27:26 +02:00
hh 23cfb16228 feat(distill,conversations,scheduler,api,ui): close deep chats with a distiller fork, digest and index, idle and weekly state jobs 2026-08-29 03:15:39 +02:00
hh ab9ce50b82 refactor(envelope): drop the reply counter and the injects line, keep time and vault changes 2026-08-29 02:38:39 +02:00
hh 662bdcdfda feat(scheduler,rotation,envelope,api,ui): pgqueuer jobs and deferred injects, master rotation with handout, vault envelope, jobs page 2026-08-29 02:29:05 +02:00
hh 0479d7cdda feat(conversations): pre-SDK rows read from canonical messages, implied titles, markdown adopts them by first prompt 2026-08-29 01:44:05 +02:00
hh 0cc2c5291f fix(markdown): a rewritten assistant reply reseeds the session instead of resuming it 2026-08-29 01:37:52 +02:00
hh 5cd18fa2d7 fix(ui): sidebar footer in one row, no user name 2026-08-29 01:19:58 +02:00
hh e45e9960ed fix(ui): thread header counts only queued items; label the live dot 2026-08-29 01:08:44 +02:00
hh 16b6bbddda feat: one gateway port with path-mounted frontends, markdown chat, collapsible sidebars 2026-08-29 00:49:06 +02:00
hh 540327efa1 feat(ui,api): chat view with live tail and autoscroll, conversation rail with last message previews 2026-08-29 00:18:08 +02:00
hh 645df6f2b4 feat(ui): endpoint catalog with copyable urls on the now page 2026-08-28 22:30:11 +02:00
hh b577034c83 chore(ui): beaver favicon from beaver-land and impeccable design sidecar 2026-08-28 22:20:36 +02:00
hh 00618ab7ad feat(ui,api,admin): sveltekit admin spa, usage by day/agent/conversation/model, rate limits, memory tree, turn snapshot 2026-08-28 22:14:15 +02:00
hh aaaf459e6d fix(backends): remember the session id from the first frame so a cut turn stays resumable 2026-08-28 19:38:47 +02:00
hh 4ad014726b fix(core): a turn cut by shutdown keeps running_turn for recover 2026-08-28 19:33:26 +02:00
hh 8c9c6eee69 fix(telegram): brief-seeded branches reply and stream in their topic 2026-08-28 19:21:03 +02:00
hh 21766fe2d7 feat(core): seeds without text wait for the first turn 2026-08-28 19:03:08 +02:00
hh 1f8dae57ac fix(telegram,core): no draft for seed turns 2026-08-28 18:59:00 +02:00
hh 66325c2879 fix(telegram): last draft push is the final text so the client folds it into the message 2026-08-28 18:51:16 +02:00
hh 5a44a16ba3 fix(telegram,core): bare seeds stay silent in telegram 2026-08-28 18:42:35 +02:00
hh c711dcab13 fix(telegram): final message waits for the draft push in flight 2026-08-28 18:32:45 +02:00
hh 3995741cd5 fix(telegram,core): first message seeds the branch, registered commands, no ellipses, html drafts 2026-08-28 18:25:46 +02:00
hh b0f280b3ca feat(telegram,core): master topic instead of general in private chats 2026-08-28 18:12:58 +02:00
hh 45a5eb4a94 feat(telegram,core,backends,storage): telegram frontend with inbox, outbox, drafts and question buttons 2026-08-28 17:57:09 +02:00
hh 7ae87aeeb8 fix(backends,markdown): stream turn events live, no blank lines for tool blocks 2026-08-28 17:27:35 +02:00
hh fd8cbacd7b feat(core): publish tool.result on the bus for the activity panel 2026-08-28 17:01:00 +02:00
hh bd2d2368e1 refactor(agents,core,frontends): typed kinds, prompts per kind, frontend agents as parameters 2026-08-28 16:36:44 +02:00
hh 1d8d65b69a feat(core,frontends,agents): agent kinds, frontend routing, anthropic on conversations, vault mirror 2026-08-28 03:50:33 +02:00
hh 33ccc78fec feat(core,backends,frontends,storage): conversations, inject queue, session pool, gateway mcp tools, api frontend 2026-08-28 03:08:30 +02:00
hh 50b7057fa4 fix(backends,storage,core): setuid in exec wrapper, missing-column migration, runner config dir for mirror, tagged prompt sources 2026-08-28 02:10:19 +02:00
hh e1f242a87a feat(backends,storage,core,markdown,infra): claude agent sdk backend, session store, transcript seeding, runner isolation 2026-08-28 01:56:39 +02:00
hhandClaude Fable 5 ee2dc918ee docs(env): .env.example for local development
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-27 23:45:53 +02:00
hhandClaude Fable 5 33cc769ca2 chore: make ruff format and ty clean
Format two mcp modules with the current ruff, and narrow the text-block
check in conversation_store through a typed helper so ty stops flagging
dict.get on an unnarrowed block.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-27 23:22:23 +02:00
hhandClaude Fable 5 e4282b38cd feat(ui,infra): SvelteKit ui scaffold, Makefile, pre-commit, claude hooks
ui/: SvelteKit 2 + Svelte 5 + Tailwind 4 + shadcn-svelte (luma, lucide)
on bun with ultracite; adapter-static with SPA fallback; theme tokens
taken from msos. Dockerfile gains a bun stage that builds ui/build.
Makefile and pre-commit cover ruff, ty, ultracite and svelte-check;
.claude settings run the matching fixer after edits. docs/PRODUCT.md
captures product truth for the UI work.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-27 23:22:18 +02:00
152 changed files with 2906 additions and 13287 deletions

No files matched your search

-4
View File
@@ -28,7 +28,3 @@ RAYCAST_BEARER=
# токен от @BotFather, свой user id - @userinfobot
TELEGRAM_BOT_TOKEN=
TELEGRAM_USER_ID=
# Откуда тулза send_file имеет право брать файлы (через ':').
# Папка входящих вложений разрешена всегда, её дописывает сам фронтенд.
SEND_FILE_ROOTS=/mnt/hole/shared
+3 -26
View File
@@ -45,36 +45,17 @@ RUN apt-get update \
&& rm -rf /var/lib/apt/lists/*
# Bun native binary (glibc): `bunx` runs the stdio MCP servers declared in
# the user's config.
# the user's config. The claude CLI itself ships inside the
# claude-agent-sdk wheel (`_bundled/claude`), nothing to install here.
COPY --from=oven/bun:1-slim /usr/local/bin/bun /usr/local/bin/bun
RUN ln -s /usr/local/bin/bun /usr/local/bin/bunx
# The claude-agent-sdk wheel bundles a CLI, but it trails the npm release:
# 0.2.157 carries 2.1.277, which does not know claude-opus-5-5 and writes an
# [claude-code:unrecognized_model] warning per turn. Install the npm build and
# point BEAVER_CLAUDE_CLI at it; the adapter prefers it over the bundled one.
#
# Only the glibc platform package is pulled: the meta package also drags in the
# musl build, and this image is glibc for the reason stated at the top. The
# tree lands under /opt world-readable because the model runs as beaver-runner,
# not as root.
ARG CLAUDE_CODE_VERSION=2.1.280
RUN mkdir -p /opt/claude-code \
&& cd /opt/claude-code \
&& echo '{"name":"claude-cli-holder","private":true}' > package.json \
&& bun add @anthropic-ai/claude-code-linux-x64@${CLAUDE_CODE_VERSION} \
&& ln -sf /opt/claude-code/node_modules/@anthropic-ai/claude-code-linux-x64/claude \
/usr/local/bin/claude-cli \
&& chmod -R a+rX /opt/claude-code \
&& /usr/local/bin/claude-cli --version
ENV BEAVER_CLAUDE_CLI=/usr/local/bin/claude-cli
# The model process runs as `beaver-runner`, not as the gateway: the
# adapter spawns claude under this uid with a whitelisted environment.
# `acl` lets the entrypoint grant it write access to the vault sub-mounts
# without chowning files that Obsidian Sync keeps rewriting as root.
RUN apt-get update \
&& apt-get install -y --no-install-recommends acl sudo \
&& apt-get install -y --no-install-recommends acl \
&& rm -rf /var/lib/apt/lists/* \
&& useradd --system --uid 1001 --create-home --shell /usr/sbin/nologin beaver-runner
@@ -83,10 +64,6 @@ ENV PATH=/app/.venv/bin:$PATH
COPY --from=builder /app/.venv /app/.venv
COPY --from=builder /app /app
COPY --from=ui /ui/build /app/ui/build
COPY --chmod=0755 docker/secret /usr/local/bin/secret
COPY --chmod=0755 docker/beaver-secret /usr/local/libexec/beaver-secret
COPY --chmod=0440 docker/sudoers /etc/sudoers.d/beaver-secret
RUN visudo -cf /etc/sudoers.d/beaver-secret
WORKDIR /app
ENTRYPOINT ["python", "-m"]
+9 -28
View File
@@ -7,8 +7,6 @@ boundary - without an opinion about what the agent is for. A setup is one
holds the primitives, [beaver-agent](https://git.kotikot.com/beaver/beaver-agent)
is a full setup built on them.
![The admin: what is in motion, what waits for an answer, the context and the quotas, the graph around the master](preview/shots/admin-now.webp)
## What it gives a setup
- **agents/** - what an agent is: a Claude agent with a prompt per
@@ -22,50 +20,33 @@ is a full setup built on them.
the envelope, the in-process gateway tools.
- **jobs/** - cron, webhook and event jobs on pgqueuer, deferred injects,
the subscription budget.
- **frontends/** - the windows: Telegram (master = General, topic = branch,
files out of the agent through `send_file`),
- **frontends/** - the windows: Telegram (master = General, topic = branch),
markdown files in a vault, `/api` + the admin SPA, an Anthropic-compatible
`/anthropic/v1/messages`, MCP re-exposure at `/mcp/<name>`, and
`WebhookFrontend` for a window a setup declares itself - its own request
and response schema, its own agent, its own token scope
(`docs/FRONTEND-PLUGINS.md`).
`/anthropic/v1/messages`, MCP re-exposure at `/mcp/<name>`.
- **mcp/** - MCP servers a setup declares (stdio, http, python tools),
aggregated in-process and handed to agents by name.
- **vault/** - watching a directory of notes for the envelope.
- **security/** - bearer tokens, the audit log, credential redaction.
- **keystore/** - secrets the model runs commands with: values and files
sealed with `SECRETS_KEY` (AES-256-GCM) in the gateway's database, set
write-only from the admin, handed out only through `secret run NAME --
cmd` (one `sudo` rule for `beaver-runner`, a root-only socket, files in
`/dev/shm`, values masked in the command's output).
- **events/** - the in-process bus every frontend taps, and the stream
event protocol backends emit.
- **storage/** - SQLModel tables and the Postgres session store.
`config.py` defines `Gateway` and loads a setup; `app.py` builds the runtime
from it; `cli.py` is the entrypoint. The admin SPA in `ui/` is the browser
window on all of it: conversations by day, the thread with the tree of tool
calls, the context of the last turn, memory, jobs, usage, tokens, secrets and the audit.
![A conversation in the admin: the thread, the queue, the details](preview/shots/admin-thread.webp)
![The Context tab: what the model held on the last turn - harness, prompt granules, skills, the notes the thread reached](preview/shots/admin-context.webp) Every string the gateway puts in front of
from it; `cli.py` is the entrypoint. Every string the gateway puts in front of
a model or a user has an English default and is overridable
(`ConversationTexts`, `TelegramTexts`).
## Run
```sh
cd examples
CLAUDE_CODE_OAUTH_TOKEN=$(claude setup-token) docker compose up --build
uv sync # python
cd ui && bun install && cd .. # admin + panel SPA, built into ui/build
cp examples/config.py /config/config.py
DATABASE_URL=postgresql+psycopg://... CONFIG_PATH=/config/config.py uv run beaver-gateway
```
`examples/config.py` is the smallest setup that starts: one agent, one
python tool, one cron job, the built-in frontends on port 8000; the compose
file next to it adds Postgres. Without Docker: `uv sync`, build the SPA with
`cd ui && bun install && bun run build`, then run `uv run beaver-gateway` with
`DATABASE_URL` and `CONFIG_PATH` set. A setup's own secrets come from `.env`
next to its config (`load_dotenv` runs before the config is executed).
`examples/config.py` is the smallest setup that starts. The setup's own
secrets come from `.env` (`load_dotenv` runs before the config is executed).
## Develop
-2
View File
@@ -1,2 +0,0 @@
#!/bin/sh
exec /usr/local/bin/python3 -I /app/src/beaver_gateway/keystore/cli.py --root "$@"
-2
View File
@@ -1,2 +0,0 @@
#!/bin/sh
exec /usr/local/bin/python3 -I /app/src/beaver_gateway/keystore/cli.py "$@"
-2
View File
@@ -1,2 +0,0 @@
Defaults:beaver-runner !syslog
beaver-runner ALL=(root) NOPASSWD: /usr/local/libexec/beaver-secret
+22 -27
View File
@@ -1,36 +1,31 @@
services:
postgres:
image: postgres:16-alpine
environment:
POSTGRES_USER: beaver
POSTGRES_PASSWORD: beaver
POSTGRES_DB: beaver
volumes:
- postgres-data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U beaver -d beaver"]
interval: 5s
timeout: 3s
retries: 20
# Minimal compose for the Phase 0 DoD: build the image, mount a
# single-file user config, watch the process print its summary line
# and exit cleanly.
#
# Real deployments add a postgres service, raycast-config.json bind,
# ~/.config/claude bind, etc. — Phase 5 lives in a separate example
# repo (see PLAN §5.1).
services:
gateway:
build:
context: ..
depends_on:
postgres:
condition: service_healthy
dockerfile: Dockerfile
environment:
CONFIG_PATH: /config/config.py
DATABASE_URL: postgresql+psycopg://beaver:beaver@postgres:5432/beaver
ADMIN_USER: admin
ADMIN_PASS: change-me
SESSION_SECRET: change-me-too
BOOTSTRAP_TOKENS: dev:change-me:*
CLAUDE_CODE_OAUTH_TOKEN: ${CLAUDE_CODE_OAUTH_TOKEN}
DATABASE_URL: "sqlite:///./gateway.db"
ADMIN_USER: "admin"
ADMIN_PASS: "change-me"
SESSION_SECRET: "dev-secret-change-me"
# Phase 1.4 — comma-separated `name:value` pairs. The frontend
# 401s every request until at least one token is set.
BOOTSTRAP_TOKENS: "cursor:dev-token-change-me"
# RAYCAST_BEARER / RAYCAST_DEVICE_ID + a raycast.json bind go
# here once you wire up an actual RaycastAgent. The example
# config.py declares one, so set these (or remove the agent)
# before exposing port 8000.
ports:
# The one gateway port: /anthropic, /mcp, /admin, /md under it
# (change ADMIN_USER/ADMIN_PASS/SESSION_SECRET before exposing).
- "8000:8000"
volumes:
- ./config.py:/config/config.py:ro
volumes:
postgres-data:
Binary file not shown.

Before

Width:  |  Height:  |  Size: 138 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 81 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 122 KiB

+1 -2
View File
@@ -14,9 +14,8 @@ dependencies = [
"anthropic>=0.103.0",
"anyio>=4.13.0",
"argon2-cffi>=25.1.0",
"claude-agent-sdk>=0.2.157",
"claude-agent-sdk>=0.2.150",
"croniter>=6.2.4",
"cryptography>=48.0.0",
"fastapi>=0.136.1",
"fastmcp>=3.3.1",
"greenlet>=3.5.0",
-5
View File
@@ -36,8 +36,3 @@ class BaseAgent(BaseModel):
system_prompt: str
expose_mcps: tuple[ExposedMcp, ...] = ()
accept_client_tools: bool = False
@property
def thinking_is_progress(self) -> bool:
"""Whether non-empty thinking is text for the human rather than reasoning."""
return False
+1 -14
View File
@@ -17,19 +17,13 @@ from beaver_gateway.agents.policy import PolicyRule # noqa: TC001
from beaver_gateway.agents.prompts import PromptSource # noqa: TC001
from beaver_gateway.conversations.kinds import KINDS, Kind
__all__ = ["PROGRESS_MODELS", "ClaudeAgent", "ClaudeOptions", "Prompts", "SkillSets"]
PROGRESS_MODELS = frozenset({"claude-opus-5-5", "claude-fable-5-1"})
"""Models whose non-empty thinking is the text written between tool calls."""
__all__ = ["ClaudeAgent", "ClaudeOptions", "Prompts", "SkillSets"]
class ClaudeOptions(BaseModel):
model_config = ConfigDict(frozen=True, arbitrary_types_allowed=True)
effort: str | None = None
thinking: Mapping[str, str] | None = None
"""Thinking config for the SDK; ``None`` keeps the CLI default."""
include_partial_messages: bool = True
"""Token-level deltas on the wire; off means one delta per finished block."""
@@ -46,7 +40,6 @@ class ClaudeOptions(BaseModel):
"""Extra inherited variable names to let through the env whitelist."""
max_turns: int | None = None
max_buffer_size: int = 16 * 1024 * 1024
session_store_flush: str = "eager"
"""``eager`` mirrors every transcript frame as it lands, so a gateway
killed mid-turn loses at most the frame in flight."""
@@ -125,12 +118,6 @@ class ClaudeAgent(BaseAgent):
raise ValueError(msg)
return self
@property
def thinking_is_progress(self) -> bool:
"""A progress model, unless ``display: summarized`` brings reasoning."""
summarized = (self.options.thinking or {}).get("display") == "summarized"
return self.model in PROGRESS_MODELS and not summarized
def prompt_for(self, kind: Kind) -> tuple[PromptSource, ...] | None:
return self.prompts.for_kind(kind)
+2 -46
View File
@@ -6,7 +6,6 @@ import asyncio
import functools
import logging
from contextlib import AsyncExitStack
from pathlib import Path
from typing import TYPE_CHECKING, Any
import psycopg
@@ -33,10 +32,8 @@ from beaver_gateway.frontends.base import GatewayRuntime
from beaver_gateway.frontends.bearer import require_token
from beaver_gateway.frontends.root import build_root_app
from beaver_gateway.jobs.scheduler import Scheduler
from beaver_gateway.keystore import SecretBox, SecretStore
from beaver_gateway.keystore import serve as serve_secrets
from beaver_gateway.mcp.internal_app import build_internal_app
from beaver_gateway.security.auth import TokenStore, scopes_with
from beaver_gateway.security.auth import TokenStore
from beaver_gateway.storage import (
Database,
PostgresSessionStore,
@@ -46,7 +43,7 @@ from beaver_gateway.storage import (
)
if TYPE_CHECKING:
from collections.abc import Iterable, Iterator, Sequence
from collections.abc import Iterable, Iterator
from claude_agent_sdk import McpSdkServerConfig
from fastmcp import FastMCP
@@ -122,7 +119,6 @@ async def run(gateway: Gateway, settings: Settings) -> None:
mcps = McpRegistry(gateway.mcps)
db = Database(settings.database_url)
await db.create_all()
secrets = SecretStore(db, _secret_box(settings))
token_store = TokenStore(
db,
bootstrap=TokenStore.parse_bootstrap(settings.bootstrap_tokens),
@@ -198,9 +194,6 @@ async def run(gateway: Gateway, settings: Settings) -> None:
pool=pool,
scheduler=scheduler,
public_url=gateway.public_url.rstrip("/") if gateway.public_url else None,
scopes=scopes_with(fe.scope for fe in gateway.frontends),
send_file_roots=_roots(settings.send_file_roots),
secrets=secrets,
)
for fe in gateway.frontends:
fe.configure(runtime)
@@ -228,43 +221,10 @@ async def run(gateway: Gateway, settings: Settings) -> None:
tg.create_task(_serve_root(gateway, extra={"/hooks": hooks}))
if gateway.watch is not None:
tg.create_task(gateway.watch.run())
if secrets.enabled:
tg.create_task(_serve_secrets(secrets, settings=settings, db=db))
for fe in gateway.frontends:
tg.create_task(fe.serve())
def _secret_box(settings: Settings) -> SecretBox | None:
if not settings.secrets_key:
return None
try:
return SecretBox.from_hex(settings.secrets_key)
except ValueError as exc:
_log.error("secrets are off: %s", exc)
return None
async def _serve_secrets(
store: SecretStore, *, settings: Settings, db: Database
) -> None:
async def used(names: Sequence[str]) -> None:
try:
async with db.session() as session:
await append_audit(
session,
actor="runner",
kind="secret_use",
detail={"names": list(names)},
)
except Exception: # noqa: BLE001
_log.exception("secret audit write failed")
try:
await serve_secrets(store, settings.secrets_socket, on_use=used)
except OSError:
_log.exception("secrets socket %s did not open", settings.secrets_socket)
async def _authorize_hook(
request: Request, *, runtime: GatewayRuntime, scope: str
) -> str:
@@ -336,10 +296,6 @@ async def _serve_internal_mcp(app: Starlette, *, settings: Settings) -> None:
await uvicorn.Server(config).serve()
def _roots(value: str) -> tuple[Path, ...]:
return tuple(Path(p) for p in value.split(":") if p.strip())
class _LateConversations:
conversations: Conversations | None = None
+67 -297
View File
@@ -20,7 +20,6 @@ import tempfile
import time
import uuid
import warnings
from collections import deque
from collections.abc import Mapping
from dataclasses import dataclass, field
from pathlib import Path
@@ -38,7 +37,6 @@ from claude_agent_sdk import (
PermissionResultDeny,
ResultMessage,
StreamEvent,
TaskNotificationMessage,
TextBlock,
ThinkingBlock,
ToolResultBlock,
@@ -50,12 +48,7 @@ from claude_agent_sdk import (
from beaver_gateway.agents import policy as policy_mod
from beaver_gateway.agents import prompts as prompt_assembly
from beaver_gateway.backends.capture import TurnCapture, TurnUsage
from beaver_gateway.backends.sessions import (
STDERR_LINES,
Session,
SessionClient,
SessionPool,
)
from beaver_gateway.backends.sessions import Session, SessionClient, SessionPool
from beaver_gateway.backends.transcript import (
build_entries,
close_open_tool_uses,
@@ -77,17 +70,9 @@ from beaver_gateway.events.stream import (
build_thinking_delta,
build_tool_use_block_start,
)
from beaver_gateway.security.redact import redact
if TYPE_CHECKING:
from collections.abc import (
AsyncIterator,
Awaitable,
Callable,
Coroutine,
Iterable,
Sequence,
)
from collections.abc import AsyncIterator, Awaitable, Callable, Iterable, Sequence
from anthropic.types import MessageParam
from claude_agent_sdk import (
@@ -111,7 +96,6 @@ warnings.filterwarnings("ignore", category=CanUseToolShadowedWarning)
ASK_TOOL = "AskUserQuestion"
__all__ = [
"ORIGIN_HEADER",
"AuditSink",
"ClaudeSdkBackend",
"RunnerConfig",
@@ -121,9 +105,6 @@ __all__ = [
"fingerprint",
]
ORIGIN_HEADER = "X-Conversation-Origin"
"""Rides on every exposed MCP request; the gateway proxy passes it upstream."""
ENV_KEEP: tuple[str, ...] = (
"PATH",
"HOME",
@@ -220,16 +201,8 @@ class ClaudeSdkBackend:
tool_server: Callable[[str, str], McpSdkServerConfig | None] | None = None,
asker: Callable[[str, dict[str, Any]], Awaitable[str]] | None = None,
audit_sink: Callable[[policy_mod.ToolAudit], Awaitable[None]] | None = None,
waker: Callable[[str, str], Coroutine[Any, Any, None]] | None = None,
) -> None:
"""One backend per Claude agent.
``waker(key, note)`` is called when a session starts a turn by itself
(a background task reported back); the caller then attaches.
"""
self._agent = agent
self.waker = waker
self._tasks: set[asyncio.Task[None]] = set()
self._audit_sink = audit_sink
self._store = session_store
self._runner = runner or RunnerConfig()
@@ -273,56 +246,16 @@ class ClaudeSdkBackend:
await self._pool.close(key)
async def interrupt(self, key: str) -> bool:
return await self._pool.interrupt(key)
live = self._pool.get(key)
if live is None or not live.busy:
return False
live.interrupt_requested = True
await live.client.interrupt()
return True
def live(self, key: str) -> Session | None:
return self._pool.get(key)
def has_pending(self, key: str) -> bool:
"""A turn the session started on its own is waiting for ``attach``."""
live = self._pool.get(key)
return live is not None and bool(live.pending)
def pending_note(self, key: str) -> str | None:
live = self._pool.get(key)
return live.note if live is not None else None
async def attach(
self,
*,
key: str,
capture: TurnCapture | None = None,
observer: Callable[[Any], None] | None = None,
turn_id: str | None = None,
) -> AsyncIterator[MessageStreamEvent]:
"""Consume the turn a live session started by itself as a normal turn.
The CLI runs one when a background task (a subagent) reports back:
no prompt goes out, the events are what the model said in reply to
the report, the turn ends on that result.
"""
live = self._pool.get(key)
if live is None or not live.pending:
return
message_id = f"msg_{uuid.uuid4().hex}"
yield build_message_start(message_id=message_id, model=self._agent.model)
turn = _Turn()
async with live.lock:
live.running_turn = turn_id or message_id
live.last_used = time.monotonic()
async for event in self._run_turn(live, None, turn, observer, capture):
yield event
live.turns += 1
live.last_used = time.monotonic()
live.running_turn = None
usage, _ = await self._after_turn(
live, turn, conversation_id=key, history=[], capture=capture
)
yield build_message_delta(
stop_reason=turn.stop_reason, usage=_wire_usage(usage)
)
yield build_message_stop()
async def repair_session(
self, session_id: str, *, text: str = "interrupted"
) -> int:
@@ -380,7 +313,7 @@ class ClaudeSdkBackend:
if not history or history[-1].get("role") != "user":
msg = "the last message must be a user turn"
raise ValueError(msg)
prompt = _prompt(history[-1].get("content"))
prompt = _prompt_text(history[-1].get("content"))
prior = history[:-1]
key = conversation_id or fingerprint(prior)
spec = _SessionSpec(kind=kind, pinned=pinned, tools=tools)
@@ -471,71 +404,70 @@ class ClaudeSdkBackend:
async def _run_turn(
self,
live: Session,
prompt: str | list[dict[str, Any]] | None,
prompt: str,
turn: _Turn,
observer: Callable[[Any], None] | None = None,
capture: TurnCapture | None = None,
) -> AsyncIterator[MessageStreamEvent]:
"""Run one prompt, or attach to a self-started one, yielding events.
"""Run one prompt, yielding wire events as they arrive.
``turn`` is filled in place (result, synthesized history, count of
events already yielded) so the caller can finish bookkeeping - and
decide whether a retry is still possible - after a failure. The
session id lands in ``capture`` with the first frame, so a turn cut
by a restart still leaves a resumable session behind.
A result whose ``origin`` is not the human's ends a turn the CLI ran
on its own; inside a prompted turn it is passed over, the events it
came with stay in this turn rather than getting lost.
"""
streaming = self._agent.options.include_partial_messages
raw: list[Any] = []
cursor = _Cursor()
live.observer = observer
live.attached = True
for queued in live.pending:
live.inbox.put_nowait(queued)
live.pending.clear()
try:
if prompt is not None:
await live.client.query(
prompt if isinstance(prompt, str) else _stream(prompt)
next_index = 0
offset = 0
await live.client.query(prompt)
async for message in live.client.receive_response():
if observer is not None:
observer(message)
session_id = getattr(message, "session_id", None)
if isinstance(session_id, str) and session_id and live.session_id is None:
live.session_id = session_id
if capture is not None:
capture.session_id = session_id
if isinstance(message, MirrorErrorMessage):
live.dirty = True
_log.error(
"session %s: mirror error, marked dirty: %s",
live.session_id,
message.error,
)
while True:
message = await live.inbox.get()
if message is _END:
if prompt is not None:
msg = f"claude exited before answering ({live.session_id})"
raise RuntimeError(msg)
break
if isinstance(message, BaseException):
raise message
if observer is not None:
observer(message)
if not self._note(live, message, capture):
continue
if getattr(message, "parent_tool_use_id", None) is not None:
continue
if isinstance(message, StreamEvent):
event = message.event
if event.get("type") == "message_start":
offset = next_index
turn.context_tokens = _context_of(event.get("message"))
continue
if isinstance(message, ResultMessage):
if prompt is not None and _self_started(message):
_log.info(
"session %s: a self-started turn ended inside a "
"prompted one, folded in",
live.session_id,
)
continue
turn.result = message
turn.stop_reason = _STOP_REASONS.get(
message.stop_reason or "", "end_turn"
)
break
for out in self._emit(message, turn, cursor, raw):
turn.events += 1
yield out
if turn.events == 0:
for out in _emit_whole(raw, cursor):
turn.events += 1
yield out
finally:
live.attached = False
live.woken = False
index = event.get("index")
if isinstance(index, int):
next_index = max(next_index, offset + index + 1)
if streaming:
for out in _emit_stream_event(event, offset + index):
turn.events += 1
yield out
elif isinstance(message, AssistantMessage):
raw.append(message)
if not streaming:
for block in message.content:
for out in _emit_block(block, next_index):
turn.events += 1
yield out
next_index += 1
elif isinstance(message, UserMessage):
raw.append(message)
elif isinstance(message, ResultMessage):
turn.result = message
turn.stop_reason = _STOP_REASONS.get(
message.stop_reason or "", "end_turn"
)
turn.synthesized = synthesize_turn_messages(raw)
_log.info(
"turn: agent=%s session=%s events=%d synthesized=%d stop=%s",
@@ -546,88 +478,6 @@ class ClaudeSdkBackend:
turn.stop_reason,
)
@staticmethod
def _note(live: Session, message: Any, capture: TurnCapture | None) -> bool:
"""Session bookkeeping for one message; False if not the main thread's."""
session_id = getattr(message, "session_id", None)
if isinstance(session_id, str) and session_id and live.session_id is None:
live.session_id = session_id
if capture is not None:
capture.session_id = session_id
if isinstance(message, MirrorErrorMessage):
live.dirty = True
_log.error(
"session %s: mirror error, marked dirty: %s",
live.session_id,
message.error,
)
return False
if isinstance(message, TaskNotificationMessage):
live.note = _note_of(message)
return False
return getattr(message, "parent_tool_use_id", None) is None
def _emit(
self, message: Any, turn: _Turn, cursor: _Cursor, raw: list[Any]
) -> Iterable[MessageStreamEvent]:
streaming = self._agent.options.include_partial_messages
if isinstance(message, StreamEvent):
event = message.event
if event.get("type") == "message_start":
cursor.offset = cursor.next_index
turn.context_tokens = _context_of(event.get("message"))
return
index = event.get("index")
if isinstance(index, int):
cursor.next_index = max(cursor.next_index, cursor.offset + index + 1)
if streaming:
yield from _emit_stream_event(event, cursor.offset + index)
elif isinstance(message, AssistantMessage):
raw.append(message)
if not streaming:
for block in message.content:
yield from _emit_block(block, cursor.next_index)
cursor.next_index += 1
elif isinstance(message, UserMessage):
raw.append(message)
async def _read(self, live: Session) -> None:
"""Read the client for the session's whole life.
During a turn every message goes to the turn's inbox. Between turns
a subagent still running is shown to the last observer, and the
first frame of the main thread means the CLI started a turn by
itself: it is buffered in ``pending`` and the waker is told once.
"""
try:
async for message in live.client.receive_messages():
if live.attached:
live.inbox.put_nowait(message)
continue
if isinstance(message, TaskNotificationMessage):
live.note = _note_of(message)
if _main_thread(message):
live.pending.append(message)
if not live.woken:
live.woken = True
self._wake(live)
elif live.observer is not None:
live.observer(message)
live.inbox.put_nowait(_END)
except asyncio.CancelledError:
raise
except Exception as exc: # noqa: BLE001
_log.error("session %s: reader died: %s", live.session_id, exc)
live.dirty = True
live.inbox.put_nowait(exc)
def _wake(self, live: Session) -> None:
if self.waker is None:
return
task = asyncio.create_task(self.waker(live.key, live.note or ""))
self._tasks.add(task)
task.add_done_callback(self._tasks.discard)
async def _acquire(
self,
key: str,
@@ -680,8 +530,7 @@ class ClaudeSdkBackend:
async def _spawn(
self, resume: str | None, *, key: str, spec: _SessionSpec
) -> Session:
stderr: deque[str] = deque(maxlen=STDERR_LINES)
options = self._build_options(resume, key=key, spec=spec, stderr=stderr)
options = self._build_options(resume, key=key, spec=spec)
client = self._factory(options)
await client.connect()
_log.info(
@@ -692,7 +541,7 @@ class ClaudeSdkBackend:
spec.tools,
self._runner.user,
)
live = Session(
return Session(
key=key,
agent=self._agent.name,
kind=spec.kind,
@@ -700,21 +549,13 @@ class ClaudeSdkBackend:
session_id=resume,
resumed=resume is not None,
pinned=spec.pinned,
stderr=stderr,
)
live.reader = asyncio.create_task(self._read(live))
return live
def _default_factory(self, options: ClaudeAgentOptions) -> SessionClient:
return _RunnerClient(options, uid=self._uid)
def _build_options(
self,
resume: str | None,
*,
key: str,
spec: _SessionSpec,
stderr: deque[str] | None = None,
self, resume: str | None, *, key: str, spec: _SessionSpec
) -> ClaudeAgentOptions:
agent = self._agent
opt = agent.options
@@ -728,14 +569,7 @@ class ClaudeSdkBackend:
system_prompt = (
prompt_assembly.assemble(sources) if sources else agent.system_prompt
)
servers: dict[str, Any] = (
{
name: {**server, "headers": {ORIGIN_HEADER: key}}
for name, server in self._servers.items()
}
if spec.tools
else {}
)
servers: dict[str, Any] = dict(self._servers) if spec.tools else {}
gateway = (
self._tool_server(key, spec.kind)
if spec.tools and self._tool_server is not None and agent.gateway_tools
@@ -746,7 +580,6 @@ class ClaudeSdkBackend:
return ClaudeAgentOptions(
model=agent.model or None,
effort=cast("Any", opt.effort),
thinking=cast("Any", dict(opt.thinking) if opt.thinking else None),
system_prompt=system_prompt,
setting_sources=[],
strict_mcp_config=True,
@@ -767,15 +600,11 @@ class ClaudeSdkBackend:
plugins=cast("Any", plugins),
skills="all" if plugins else None,
max_turns=opt.max_turns,
max_buffer_size=opt.max_buffer_size,
stderr=lambda line: self._stderr(line.rstrip(), stderr),
stderr=lambda line: _log.warning(
"claude[%s]: %s", agent.name, line.rstrip()
),
)
def _stderr(self, line: str, tail: deque[str] | None) -> None:
_log.warning("claude[%s]: %s", self._agent.name, line)
if tail is not None:
tail.append(redact(line))
def _can_use_tool(
self, key: str
) -> Callable[
@@ -910,15 +739,6 @@ class _SessionSpec:
tools: bool
_END = object()
@dataclass
class _Cursor:
next_index: int = 0
offset: int = 0
@dataclass
class _Turn:
events: int = 0
@@ -958,17 +778,7 @@ async def _disconnect(live: Session) -> None:
_log.exception("disconnect failed for session %s", live.session_id)
CLI_ENV = "BEAVER_CLAUDE_CLI"
"""Path to a ``claude`` CLI that wins over the one bundled in the wheel."""
def _claude_binary() -> str:
override = os.environ.get(CLI_ENV)
if override:
if not Path(override).is_file():
msg = f"{CLI_ENV} points at {override!r}, which is not a file"
raise FileNotFoundError(msg)
return override
bundled = Path(claude_agent_sdk.__file__).parent / "_bundled" / "claude"
if bundled.is_file():
return str(bundled)
@@ -1035,54 +845,14 @@ def _mcp_disallowed(
return out
def _prompt(content: Any) -> str | list[dict[str, Any]]:
"""Plain text when the turn is text only; the block list when images ride along."""
def _prompt_text(content: Any) -> str:
text = text_of(content)
if not text:
msg = "user message has no text content"
raise ValueError(msg)
if isinstance(content, list) and any(
isinstance(b, dict) and b.get("type") != "text" for b in content
):
return [dict(b) for b in content if isinstance(b, dict)]
return text
async def _stream(content: list[dict[str, Any]]) -> AsyncIterator[dict[str, Any]]:
yield {
"type": "user",
"message": {"role": "user", "content": content},
"parent_tool_use_id": None,
}
def _emit_whole(raw: Iterable[Any], cursor: _Cursor) -> Iterable[MessageStreamEvent]:
"""Blocks of a turn that streamed no deltas, e.g. one reported whole."""
for message in raw:
if isinstance(message, AssistantMessage):
for block in message.content:
yield from _emit_block(block, cursor.next_index)
cursor.next_index += 1
def _self_started(result: ResultMessage) -> bool:
origin = result.origin
return origin is not None and origin.get("kind") != "human"
def _main_thread(message: Any) -> bool:
if getattr(message, "parent_tool_use_id", None) is not None:
return False
return isinstance(
message, (StreamEvent, AssistantMessage, UserMessage, ResultMessage)
)
def _note_of(message: TaskNotificationMessage) -> str:
summary = message.summary or message.data.get("description") or message.task_id
return f"{summary} ({message.status})"
def synthesize_turn_messages(raw: Iterable[Any]) -> list[dict[str, Any]]:
out: list[dict[str, Any]] = []
for message in raw:
-118
View File
@@ -1,118 +0,0 @@
"""What runs under the gateway: the process tree with memory and CPU, and the kill."""
from __future__ import annotations
import contextlib
import os
import time
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
import psutil
from beaver_gateway.security.redact import redact
__all__ = ["Processes", "subtree"]
_VALUE_HIDDEN = frozenset(
{
"--system-prompt",
"--append-system-prompt",
"--mcp-config",
"--settings",
"--agents",
}
)
_ARG_CLIP = 80
_COMMAND_CLIP = 240
@dataclass(slots=True)
class Processes:
root: int = field(default_factory=os.getpid)
_cpu_seen: dict[int, tuple[float, float]] = field(default_factory=dict)
def tree_parents_first(self) -> list[dict[str, Any]]:
try:
found = psutil.Process(self.root).children(recursive=True)
except psutil.Error:
return []
now = time.monotonic()
rows: dict[int, dict[str, Any]] = {}
for process in found:
with contextlib.suppress(psutil.Error), process.oneshot():
spent = process.cpu_times()
rows[process.pid] = {
"pid": process.pid,
"parent": process.ppid(),
"name": process.name(),
"command": _command(process.cmdline()),
"rss": process.memory_info().rss,
"cpu": self._rate(process.pid, spent.user + spent.system, now),
"age_seconds": round(time.time() - process.create_time(), 1),
}
self._cpu_seen = {p: seen for p, seen in self._cpu_seen.items() if p in rows}
children: dict[int, list[int]] = {}
for pid, row in rows.items():
parent = row["parent"] if row["parent"] in rows else self.root
children.setdefault(parent, []).append(pid)
ordered: list[dict[str, Any]] = []
def walk(parent: int, depth: int) -> None:
for pid in sorted(children.get(parent, ())):
ordered.append({**rows[pid], "depth": depth})
walk(pid, depth + 1)
walk(self.root, 0)
return ordered
def kill_own_descendant(self, pid: int) -> bool:
try:
owned = psutil.Process(self.root).children(recursive=True)
target = next((p for p in owned if p.pid == pid), None)
if target is None:
return False
doomed = [*target.children(recursive=True), target]
except psutil.Error:
return False
for process in doomed:
with contextlib.suppress(psutil.Error):
process.kill()
return True
def _rate(self, pid: int, spent: float, now: float) -> float | None:
before = self._cpu_seen.get(pid)
self._cpu_seen[pid] = (spent, now)
if before is None or now <= before[1]:
return None
return round(100 * max(spent - before[0], 0.0) / (now - before[1]), 1)
def subtree(rows: list[dict[str, Any]], pid: int | None) -> list[dict[str, Any]]:
start = next((i for i, row in enumerate(rows) if row["pid"] == pid), None)
if start is None:
return []
base = rows[start]["depth"]
out = [{**rows[start], "depth": 0}]
for row in rows[start + 1 :]:
if row["depth"] <= base:
break
out.append({**row, "depth": row["depth"] - base})
return out
def _command(argv: list[str]) -> str:
if not argv:
return ""
parts = [Path(argv[0]).name]
hide = False
for arg in argv[1:]:
if hide:
parts.append("…")
hide = False
continue
hide = arg in _VALUE_HIDDEN
parts.append(arg if len(arg) <= _ARG_CLIP else f"{arg[:_ARG_CLIP]}…")
command = redact(" ".join(parts))
return command if len(command) <= _COMMAND_CLIP else f"{command[:_COMMAND_CLIP]}…"
+35 -91
View File
@@ -13,23 +13,14 @@ import contextlib
import logging
import os
import time
from collections import deque
from dataclasses import dataclass, field
from pathlib import Path
from typing import TYPE_CHECKING, Any, Protocol
import psutil
from beaver_gateway.backends.processes import Processes, subtree
if TYPE_CHECKING:
from collections.abc import (
AsyncIterable,
AsyncIterator,
Callable,
Iterator,
Mapping,
)
from collections.abc import AsyncIterator, Iterator, Mapping
__all__ = ["DEFAULT_TTL", "Session", "SessionClient", "SessionPool", "cgroup_limit"]
@@ -46,13 +37,12 @@ DEFAULT_TTL: Mapping[str, float | None] = {
_EVICT_ORDER = {"fork": 0, "job": 0, "deep": 1, "branch": 2, "master": 3}
_RSS_HEADROOM = 0.8
STDERR_LINES = 200
class SessionClient(Protocol):
async def connect(self) -> None: ...
async def query(self, prompt: str | AsyncIterable[dict[str, Any]]) -> None: ...
def receive_messages(self) -> AsyncIterator[Any]: ...
async def query(self, prompt: str) -> None: ...
def receive_response(self) -> AsyncIterator[Any]: ...
async def interrupt(self) -> None: ...
async def disconnect(self) -> None: ...
@@ -77,21 +67,6 @@ class Session:
state: dict[str, Any] = field(default_factory=dict)
"""Scratch for policy rules (``agents/policy.py``); dies with the process."""
reader: asyncio.Task[None] | None = None
"""Reads the client for the session's whole life, not only during a turn."""
inbox: asyncio.Queue[Any] = field(default_factory=asyncio.Queue)
"""Where the reader puts messages while a turn is attached."""
pending: list[Any] = field(default_factory=list)
"""A turn the CLI started on its own (a background task reported back),
buffered until a consumer attaches."""
attached: bool = False
woken: bool = False
note: str | None = None
"""Summary of the last task notification, the pending turn's prompt."""
observer: Callable[[Any], None] | None = None
"""The last turn's observer; sees subagent traffic between turns."""
stderr: deque[str] = field(default_factory=lambda: deque(maxlen=STDERR_LINES))
@property
def busy(self) -> bool:
return self.lock.locked() or self.running_turn is not None
@@ -126,7 +101,6 @@ class SessionPool:
self._max_live = max_live
self._ttl = dict(ttl)
self._reap_interval = reap_interval
self._processes = Processes()
def __len__(self) -> int:
return len(self._sessions)
@@ -174,6 +148,19 @@ class SessionPool:
total += child.memory_info().rss
return total
@staticmethod
def rss_of(session: Session) -> int | None:
pid = session.pid
if pid is None:
return None
try:
process = psutil.Process(pid)
return process.memory_info().rss + sum(
c.memory_info().rss for c in process.children(recursive=True)
)
except psutil.Error:
return None
def over_limit(self) -> bool:
if self._rss_limit is not None:
return self.rss() > self._rss_limit * _RSS_HEADROOM
@@ -205,23 +192,11 @@ class SessionPool:
if session is None:
return
_log.info("closing session %s (%s, %s)", session.session_id, session.kind, key)
if session.reader is not None and session.reader is not asyncio.current_task():
session.reader.cancel()
if session.attached:
session.inbox.put_nowait(RuntimeError(f"session {key} was closed"))
try:
await session.client.disconnect()
except Exception: # noqa: BLE001
_log.exception("disconnect failed for session %s", session.session_id)
async def interrupt(self, key: str) -> bool:
session = self._sessions.get(key)
if session is None or not session.busy:
return False
session.interrupt_requested = True
await session.client.interrupt()
return True
async def close_all(self, *, agent: str | None = None) -> None:
for session in list(self._sessions.values()):
if agent is None or session.agent == agent:
@@ -249,57 +224,26 @@ class SessionPool:
_log.exception("session reaper failed")
def snapshot(self) -> list[dict[str, Any]]:
return self.census()["sessions"]
def census(self) -> dict[str, Any]:
now = time.monotonic()
rows = self._processes.tree_parents_first()
owned: set[int] = set()
sessions: list[dict[str, Any]] = []
for s in self._sessions.values():
tree = subtree(rows, s.pid)
owned.update(row["pid"] for row in tree)
sessions.append(
{
"key": s.key,
"agent": s.agent,
"kind": s.kind,
"session_id": s.session_id,
"pid": s.pid,
"rss": sum(row["rss"] for row in tree) if tree else None,
"cpu": _cpu(tree),
"idle_seconds": round(now - s.last_used, 1),
"age_seconds": round(now - s.created_at, 1),
"turns": s.turns,
"busy": s.busy,
"running_turn": s.running_turn,
"pending": len(s.pending),
"pending_question": s.pending_question,
"pinned": s.pinned,
"dirty": s.dirty,
"processes": tree,
}
)
return {
"rss": sum(row["rss"] for row in rows),
"rss_limit": self._rss_limit,
"cpu": _cpu(rows),
"sessions": sessions,
"others": [row for row in rows if row["pid"] not in owned],
}
async def kill(self, pid: int) -> bool:
if not self._processes.kill_own_descendant(pid):
return False
for session in list(self._sessions.values()):
if session.pid == pid:
await self.close(session.key)
return True
def _cpu(rows: list[dict[str, Any]]) -> float | None:
rates = [row["cpu"] for row in rows if row["cpu"] is not None]
return round(sum(rates), 1) if rates else None
return [
{
"key": s.key,
"agent": s.agent,
"kind": s.kind,
"session_id": s.session_id,
"pid": s.pid,
"rss": self.rss_of(s),
"idle_seconds": round(now - s.last_used, 1),
"age_seconds": round(now - s.created_at, 1),
"turns": s.turns,
"busy": s.busy,
"running_turn": s.running_turn,
"pending_question": s.pending_question,
"pinned": s.pinned,
"dirty": s.dirty,
}
for s in self._sessions.values()
]
def cgroup_limit() -> int | None:
+9 -87
View File
@@ -10,7 +10,7 @@ import hashlib
import uuid as _uuid
from collections.abc import Mapping
from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any, cast
from typing import TYPE_CHECKING, Any
try:
from claude_agent_sdk._cli_version import __cli_version__
@@ -20,23 +20,19 @@ except ImportError: # pragma: no cover
_cli_version = "2.1.248"
if TYPE_CHECKING:
from collections.abc import Iterable, Sequence
from collections.abc import Iterable
__all__ = [
"CLI_VERSION",
"build_entries",
"close_open_tool_uses",
"fingerprint",
"last_message_before",
"messages_from_entries",
"open_tool_uses",
"prompt_count",
"render_messages",
"reply_text",
"spoken_text",
"strip_tool_entries",
"text_of",
"turn_starts",
"window_entries",
]
@@ -165,10 +161,7 @@ def build_entries(
return entries
def messages_from_entries(
entries: Iterable[Mapping[str, Any]], *, stamped: bool = False
) -> list[dict[str, Any]]:
"""Messages as the model saw them; ``stamped`` adds each one's entry id and time."""
def messages_from_entries(entries: Iterable[Mapping[str, Any]]) -> list[dict[str, Any]]:
out: list[dict[str, Any]] = []
last_message_id: str | None = None
for entry in entries:
@@ -187,14 +180,7 @@ def messages_from_entries(
):
out[-1]["content"].extend(results)
else:
message_out: dict[str, Any] = {
"role": "user",
"content": _user_content(content),
}
if stamped:
message_out["id"] = entry.get("uuid")
message_out["ts"] = entry.get("timestamp")
out.append(message_out)
out.append({"role": "user", "content": _user_content(content)})
last_message_id = None
continue
blocks = _assistant_blocks(content)
@@ -207,11 +193,7 @@ def messages_from_entries(
):
out[-1]["content"].extend(blocks)
else:
message_out: dict[str, Any] = {"role": "assistant", "content": blocks}
if stamped:
message_out["id"] = entry.get("uuid")
message_out["ts"] = entry.get("timestamp")
out.append(message_out)
out.append({"role": "assistant", "content": blocks})
last_message_id = message_id
return out
@@ -440,72 +422,13 @@ def prompt_count(entries: Iterable[Mapping[str, Any]]) -> int:
return sum(1 for e in entries if _is_prompt_entry(e))
def turn_starts(entries: Iterable[Mapping[str, Any]]) -> list[int]:
"""Where each turn opens: the index of every prompt the SDK was handed.
The CLI stamps only those with ``promptSource``; what it writes as a
user on its own (an interruption note, a skill body) stays inside the
turn it belongs to.
"""
return [
index
for index, entry in enumerate(entries)
if entry.get("promptSource") and _is_prompt_entry(entry)
]
def last_message_before(entries: Sequence[Mapping[str, Any]], index: int) -> str | None:
"""Uuid of the last ``user``/``assistant`` entry ahead of ``index``."""
for entry in reversed(entries[:index]):
uuid = entry.get("uuid")
if (
entry.get("type") in _PROMPT_TYPES
and isinstance(uuid, str)
and not entry.get("isSidechain")
):
return uuid
return None
def spoken_text(block: Any, *, thinking_is_progress: bool) -> str:
"""What a content block (dict or model) says to the human.
Text always; thinking only when the agent's thinking is progress
written for the human, never when it is reasoning.
"""
if isinstance(block, Mapping):
fields = cast("Mapping[str, Any]", block)
kind, text, thinking = (
fields.get("type"),
fields.get("text"),
fields.get("thinking"),
)
else:
kind = getattr(block, "type", None)
text, thinking = getattr(block, "text", None), getattr(block, "thinking", None)
if kind == "text":
return str(text or "").strip()
if kind == "thinking" and thinking_is_progress:
return str(thinking or "").strip()
return ""
def reply_text(content: Iterable[Any], *, thinking_is_progress: bool) -> str:
"""The reply as the human reads it: spoken blocks in order, a blank line apart."""
parts = (spoken_text(b, thinking_is_progress=thinking_is_progress) for b in content)
return "\n\n".join(p for p in parts if p)
def render_messages(
messages: Iterable[Mapping[str, Any]],
*,
thinking_is_progress: bool,
window: int | None = None,
messages: Iterable[Mapping[str, Any]], *, window: int | None = None
) -> str:
"""Plain-text projection for ``read_conversation``: ``user:``/``assistant:`` turns.
Tool calls collapse to a one-line summary per assistant turn; tool
results and reasoning are dropped, progress thinking reads as text.
results and thinking are dropped.
"""
turns: list[str] = []
tools: list[str] = []
@@ -531,9 +454,8 @@ def render_messages(
turns.append("user:\n" + _text_of_content(content))
continue
for block in _assistant_blocks(content):
text = spoken_text(block, thinking_is_progress=thinking_is_progress)
if text:
current.append(text)
if block.get("type") == "text" and block.get("text"):
current.append(str(block["text"]))
elif block.get("type") == "tool_use":
tools.append(str(block.get("name", "")))
flush()
-9
View File
@@ -2,7 +2,6 @@
from __future__ import annotations
import re
import sys
from dataclasses import dataclass, field
from typing import TYPE_CHECKING, Any
@@ -31,8 +30,6 @@ if TYPE_CHECKING:
__all__ = ["ConfigError", "Gateway", "load"]
_SCOPE = re.compile(r"[a-z][a-z0-9_-]*")
@dataclass(slots=True)
class Gateway:
@@ -138,12 +135,6 @@ def _validate(gw: Gateway, path: Path) -> None:
f"got {type(f).__name__}"
)
raise ConfigError(msg)
if f.scope is not None and not _SCOPE.fullmatch(f.scope):
msg = (
f"{path}: gateway.frontends[{i}] has scope {f.scope!r}; "
f"a frontend scope is a lowercase name like 'voice', never '*'"
)
raise ConfigError(msg)
names: set[str] = set()
for i, j in enumerate(gw.jobs):
if not isinstance(j, Job):
-246
View File
@@ -1,246 +0,0 @@
"""What a conversation's model holds: prompt granules, skills, tools, files touched."""
from __future__ import annotations
import re
from collections import Counter
from pathlib import Path
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from collections.abc import Iterable, Mapping
from beaver_gateway.agents.claude import ClaudeAgent
from beaver_gateway.conversations.kinds import Kind
__all__ = ["bash_paths", "compose", "files_touched", "granules", "skill_sets"]
CHARS_PER_TOKEN = 3.2
READ_TOOLS = frozenset({"Read", "NotebookRead"})
WRITE_TOOLS = frozenset({"Write", "Edit", "MultiEdit", "NotebookEdit"})
SEARCH_TOOLS = frozenset({"Glob", "Grep"})
PATH_KEYS = ("file_path", "notebook_path", "path", "file", "filename")
_FRONTMATTER = re.compile(r"^---\s*\n(.*?)\n---", re.DOTALL)
# A shell command names files as quoted strings or bare words; only notes
# and plain text files count, code and binaries are the agent's own business.
_SHELL_TOKEN = re.compile(r'"([^"\n]+)"|\'([^\'\n]+)\'|([^\s"\'`;|&()<>]+)')
_NOTE_SUFFIXES = (".md", ".canvas", ".txt", ".csv", ".json", ".yaml", ".yml")
_SHELL_CD = re.compile(r'(?:^|&&|;|\|\|)\s*cd\s+("([^"]+)"|\'([^\']+)\'|(\S+))')
_SHELL_WRITES = re.compile(
r"write_text|write_bytes|>>?\s*[\"\']?[^\s&|]|\btee\b|\bsed\s+-i|\bmv\b|\brm\b"
r"|\btouch\b|\bcp\b|open\([^)]*[\"\'][wa]"
)
def bash_paths(command: str, cwd: Path | None) -> tuple[list[str], Path | None, bool]:
"""Files a shell command names, where it leaves the shell, whether it writes.
Relative names resolve against ``cwd``, or against the directory of a
leading ``cd`` in the same command. Absolute names stay as they are.
"""
here = cwd
for match in _SHELL_CD.finditer(command):
target = match.group(2) or match.group(3) or match.group(4) or ""
target = target.strip()
if not target:
continue
here = Path(target) if target.startswith("/") else (here or Path()) / target
found: list[str] = []
for match in _SHELL_TOKEN.finditer(command):
raw = (match.group(1) or match.group(2) or match.group(3) or "").strip()
if not raw.lower().endswith(_NOTE_SUFFIXES) or "*" in raw:
continue
path = Path(raw) if raw.startswith("/") else (here or Path()) / raw
text = str(path)
if text not in found:
found.append(text)
return found, here, bool(_SHELL_WRITES.search(command))
def _tokens(chars: int) -> int:
return round(chars / CHARS_PER_TOKEN)
def _relative(path: str, cwd: Path | None) -> str:
if cwd is None:
return path
try:
return str(Path(path).relative_to(cwd))
except ValueError:
return path
def _tool_uses(entries: Iterable[Mapping[str, Any]]) -> Iterable[tuple[str, Any, str]]:
for entry in entries:
message = entry.get("message")
if entry.get("type") != "assistant" or not isinstance(message, dict):
continue
content = message.get("content")
if not isinstance(content, list):
continue
stamp = str(entry.get("timestamp") or "")
for block in content:
if isinstance(block, dict) and block.get("type") == "tool_use":
yield str(block.get("name") or "?"), block.get("input"), stamp
def files_touched(
entries: Iterable[Mapping[str, Any]], *, cwd: Path | None = None
) -> tuple[list[dict[str, Any]], dict[str, int]]:
"""Files named in tool inputs, with how they were touched, plus tool counts."""
files: dict[str, dict[str, Any]] = {}
counts: Counter[str] = Counter()
# The shell keeps its directory between calls; follow it.
shell_cwd = cwd
def touch(raw: str, how: str, stamp: str) -> None:
key = _relative(raw, cwd)
row = files.setdefault(
key, {"path": key, "reads": 0, "writes": 0, "other": 0, "last_at": stamp}
)
row[how] += 1
row["last_at"] = max(row["last_at"], stamp)
for name, tool_input, stamp in _tool_uses(entries):
counts[name] += 1
if not isinstance(tool_input, dict):
continue
if name == "Bash" and isinstance(tool_input.get("command"), str):
paths, shell_cwd, writes = bash_paths(tool_input["command"], shell_cwd)
for raw in paths:
touch(raw, "writes" if writes else "reads", stamp)
continue
raw = next(
(v for k in PATH_KEYS if isinstance(v := tool_input.get(k), str) and v),
None,
)
if raw is None or (name in SEARCH_TOOLS and "file_path" not in tool_input):
continue
if name in READ_TOOLS:
touch(raw, "reads", stamp)
elif name in WRITE_TOOLS:
touch(raw, "writes", stamp)
else:
touch(raw, "other", stamp)
ordered = sorted(files.values(), key=lambda r: r["last_at"], reverse=True)
return ordered, dict(counts)
def granules(agent: ClaudeAgent, kind: Kind) -> dict[str, Any]:
"""The system prompt as its sources: tag, path, size, token estimate."""
sources = agent.prompt_for(kind) if agent.serves(kind) else None
rows: list[dict[str, Any]] = []
if sources is None:
chars = len(agent.system_prompt)
if chars:
rows.append(
{
"tag": None,
"path": None,
"bytes": chars,
"tokens_est": _tokens(chars),
}
)
else:
for source in sources:
tag, raw = source if isinstance(source, tuple) else (None, source)
path = Path(raw)
try:
chars = len(path.read_text(encoding="utf-8"))
except OSError:
chars = 0
rows.append(
{
"tag": tag,
"path": _relative(str(path), agent.cwd),
"bytes": chars,
"tokens_est": _tokens(chars),
}
)
total = sum(r["bytes"] for r in rows)
return {"bytes": total, "tokens_est": _tokens(total), "granules": rows}
def _skill_meta(skill_md: Path) -> dict[str, str]:
try:
text = skill_md.read_text(encoding="utf-8")
except OSError:
return {}
match = _FRONTMATTER.match(text)
meta: dict[str, str] = {}
if match:
for line in match.group(1).splitlines():
key, sep, value = line.partition(":")
if sep:
meta[key.strip()] = value.strip()
return meta
def skill_sets(agent: ClaudeAgent, kind: Kind) -> list[dict[str, Any]]:
"""Skill-set directories the kind loads, with each skill's name and description."""
out: list[dict[str, Any]] = []
for directory in agent.skills_for(kind):
skills = []
try:
children = sorted(p for p in directory.iterdir() if p.is_dir())
except OSError:
children = []
for child in children:
skill_md = child / "SKILL.md"
if skill_md.is_file():
meta = _skill_meta(skill_md)
skills.append(
{
"name": meta.get("name") or child.name,
"description": meta.get("description", ""),
"path": _relative(str(child), agent.cwd),
}
)
out.append(
{
"set": directory.name,
"path": _relative(str(directory), agent.cwd),
"skills": skills,
}
)
return out
def compose(
agent: ClaudeAgent,
kind: Kind,
entries: Iterable[Mapping[str, Any]],
*,
context_tokens: int,
) -> dict[str, Any]:
files, counts = files_touched(entries, cwd=agent.cwd)
prompt = granules(agent, kind)
skills = skill_sets(agent, kind)
skills_chars = sum(
len(s["name"]) + len(s["description"])
for group in skills
for s in group["skills"]
)
return {
"agent": {
"name": agent.name,
"model": agent.model,
"effort": agent.options.effort,
},
"kind": kind,
"context_tokens": context_tokens,
"prompt": prompt,
"skills": skills,
"skills_tokens_est": _tokens(skills_chars),
"tools": {
"allowed": list(agent.options.tools) if agent.options.tools else None,
"disallowed": list(agent.options.disallowed_tools),
"gateway": list(agent.gateway_tools),
"mcps": [m.name for m in agent.expose_mcps],
},
"files": files,
"tool_counts": counts,
"history_tokens_est": max(
0, context_tokens - prompt["tokens_est"] - _tokens(skills_chars)
),
}
+6 -19
View File
@@ -25,8 +25,6 @@ class RecallContext:
text: str
kind: str
now: datetime
agent: str = ""
"""Agent whose turn it is; a setup with several tells them apart by it."""
@dataclass(slots=True)
@@ -42,12 +40,7 @@ class Envelope:
texts: EnvelopeTexts = field(default_factory=EnvelopeTexts)
def build(
self,
*,
now: datetime | None = None,
text: str = "",
kind: str = "master",
agent: str = "",
self, *, now: datetime | None = None, text: str = "", kind: str = "master"
) -> str:
now = now or datetime.now(UTC)
changes = self.watch.take() if self.watch is not None else []
@@ -66,26 +59,20 @@ class Envelope:
texts=self.texts,
)
self.last_at = now
block = self.recall_block(text=text, kind=kind, now=now, agent=agent)
block = self.recall_block(text=text, kind=kind, now=now)
return f"{out}\n{block}" if block else out
def recall_only(
self, *, text: str, kind: str, now: datetime | None = None, agent: str = ""
self, *, text: str, kind: str, now: datetime | None = None
) -> str | None:
block = self.recall_block(
text=text, kind=kind, now=now or datetime.now(UTC), agent=agent
)
block = self.recall_block(text=text, kind=kind, now=now or datetime.now(UTC))
return f"{self.texts.header}\n{block}" if block else None
def recall_block(
self, *, text: str, kind: str, now: datetime, agent: str = ""
) -> str | None:
def recall_block(self, *, text: str, kind: str, now: datetime) -> str | None:
if self.recall is None or not text.strip():
return None
try:
block = self.recall(
RecallContext(text=text, kind=kind, now=now, agent=agent)
)
block = self.recall(RecallContext(text=text, kind=kind, now=now))
except Exception: # noqa: BLE001
_log.exception("recall hook failed")
return None
+3 -13
View File
@@ -9,7 +9,7 @@ from __future__ import annotations
from dataclasses import dataclass
from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any, Literal, cast
from typing import TYPE_CHECKING, Literal, cast
from sqlalchemy import func
from sqlmodel import col, select
@@ -70,23 +70,13 @@ class InjectQueue:
self._db = db
async def push(
self,
*,
conversation_id: int,
priority: Priority,
origin: str,
text: str,
attachments: list[dict[str, Any]] | None = None,
self, *, conversation_id: int, priority: Priority, origin: str, text: str
) -> InjectQueueItem:
if priority not in PRIORITY_RANK:
msg = f"unknown priority {priority!r}"
raise ValueError(msg)
row = InjectQueueItem(
conversation_id=conversation_id,
priority=priority,
origin=origin,
text=text,
attachments=attachments or None,
conversation_id=conversation_id, priority=priority, origin=origin, text=text
)
async with self._db.session() as session:
session.add(row)
+2 -42
View File
@@ -1,4 +1,4 @@
"""Putting words into a conversation: a message, an inject, ``say``, a file."""
"""Putting words into a conversation: a message, an inject, ``say``, ``schedule``."""
from __future__ import annotations
@@ -6,7 +6,6 @@ import logging
from typing import TYPE_CHECKING, Any, cast
from beaver_gateway.conversations.turns import Turns
from beaver_gateway.frontends.files import SendFileError
if TYPE_CHECKING:
from datetime import datetime
@@ -21,19 +20,13 @@ _log = logging.getLogger(__name__)
class Messaging(Turns):
async def post(
self,
conv: Conversation,
text: str,
*,
origin: str = "user",
attachments: list[dict[str, Any]] | None = None,
self, conv: Conversation, text: str, *, origin: str = "user"
) -> InjectQueueItem:
item = await self._queue.push(
conversation_id=cast("int", conv.id),
priority="user",
origin=origin,
text=text,
attachments=attachments,
)
await self.touch_user(conv)
self._bus.publish(
@@ -93,14 +86,6 @@ class Messaging(Turns):
self._ensure_worker(cast("int", conv.id))
return item
async def interrupt(self, conv: Conversation) -> bool:
"""Cut the running turn; the reply so far still lands."""
try:
backend = self._backend(conv.agent_name)
except LookupError:
return False
return await backend.interrupt(conv.external_id)
async def say(self, conv: Conversation, text: str) -> dict[str, Any]:
runner = self._runners.get(cast("int", conv.id))
_log.info("say[%s]: %s", conv.external_id, text[:200])
@@ -111,31 +96,6 @@ class Messaging(Turns):
turn_id=runner.turn_id if runner is not None else None,
)
async def send_file(
self, conv: Conversation, path: str, *, caption: str = "", method: str = "auto"
) -> str:
"""Hand a file to the human: this conversation's window, else the master's."""
for target in (conv, await self.open_master()):
if target is None:
continue
for fe in self._frontends:
if target.kind in fe.kinds and fe.sends_files:
note = await fe.send_file(
target, path, caption=caption, method=method
)
_log.info("send_file[%s]: %s", target.external_id, note)
return note
msg = (
"nowhere to send: no frontend that sends files shows this "
"conversation, and there is no open master"
)
raise SendFileError(msg)
def file_note(self) -> str:
"""What the ``send_file`` tool tells the model about roots and size."""
sender = next((f for f in self._frontends if f.sends_files), None)
return sender.file_note() if sender else "no frontend here sends files"
async def schedule(
self,
conv: Conversation,
@@ -7,7 +7,7 @@ import contextlib
from typing import TYPE_CHECKING, Any, cast
from uuid import uuid4
from beaver_gateway.conversations.rewinding import Rewinding
from beaver_gateway.conversations.spawning import Spawning
from beaver_gateway.conversations.state import Question
if TYPE_CHECKING:
@@ -16,7 +16,7 @@ if TYPE_CHECKING:
__all__ = ["Questions"]
class Questions(Rewinding):
class Questions(Spawning):
async def ask(self, key: str, payload: dict[str, Any]) -> str | None:
"""Show the question and wait; ``None`` when nobody answered in time."""
conv = await self.get(key)
@@ -1,114 +0,0 @@
"""Taking turns back: the thread moves to a fork that ends before them."""
from __future__ import annotations
import logging
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any, cast
from claude_agent_sdk import fork_session_via_store
from beaver_gateway.backends.transcript import (
last_message_before,
messages_from_entries,
reply_text,
text_of,
turn_starts,
)
from beaver_gateway.conversations.spawning import Spawning
if TYPE_CHECKING:
from collections.abc import Sequence
from beaver_gateway.storage.models import Conversation
__all__ = ["Rewinding", "Rewound"]
_log = logging.getLogger(__name__)
PREVIEW = 280
@dataclass(frozen=True, slots=True)
class Rewound:
conversation: Conversation
turns: int
previous_session: str
class Rewinding(Spawning):
async def turns(
self, conv: Conversation, *, limit: int = 10
) -> list[dict[str, Any]]:
"""The latest turns that can be taken back, oldest first; the first stays."""
entries = await self.entries(conv)
starts = turn_starts(entries)
ends = [*starts[1:], len(entries)]
spans = list(zip(starts, ends, strict=True))[1:]
agent = self._agents.get(conv.agent_name)
progress = agent is not None and agent.thinking_is_progress
return [
self._turn(entries[start:end], thinking_is_progress=progress)
for start, end in spans[-limit:]
]
def _turn(
self, entries: Sequence[Any], *, thinking_is_progress: bool
) -> dict[str, Any]:
prompt, *rest = messages_from_entries(entries)
replies = (
reply_text(m["content"], thinking_is_progress=thinking_is_progress)
for m in reversed(rest)
if m["role"] == "assistant"
)
return {
"id": entries[0].get("uuid"),
"ts": entries[0].get("timestamp"),
"text": text_of(self._unwrapped(prompt)["content"])[:PREVIEW],
"reply": next((text for text in replies if text), "")[:PREVIEW],
"failed": any(e.get("isApiErrorMessage") for e in entries),
}
async def rewind(self, conv: Conversation, turn: str) -> Rewound:
"""Take ``turn`` and every turn after it out of what the model sees.
Nothing is deleted: the conversation moves to a fork of its session
that ends on the message before ``turn``, the cut session stays in
the store under its old id.
"""
row_id = cast("int", conv.id)
runner = self._runner(row_id)
if runner.lock.locked():
msg = f"conversation {conv.external_id} is mid-turn; interrupt it first"
raise RuntimeError(msg)
async with runner.lock:
row = await self.get_row(row_id) or conv
entries = await self.entries(row)
starts = turn_starts(entries)
cut = next((i for i in starts[1:] if entries[i].get("uuid") == turn), None)
keep = last_message_before(entries, cut) if cut is not None else None
if cut is None or keep is None:
msg = f"conversation {row.external_id} has no turn {turn} to take back"
raise LookupError(msg)
previous = cast("str", row.session_id)
forked = await fork_session_via_store(
self._store,
previous,
directory=str(self._claude_agent(row.agent_name).cwd),
up_to_message_id=keep,
)
await self._backend(row.agent_name).close(row.external_id)
async def apply(moved: Conversation) -> None:
moved.session_id = forked.session_id
row = await self._update(row, apply)
turns = sum(1 for start in starts if start >= cut)
_log.info(
"rewound %s by %d turns: session %s -> %s",
row.external_id,
turns,
previous,
forked.session_id,
)
return Rewound(conversation=row, turns=turns, previous_session=previous)
+2 -12
View File
@@ -77,19 +77,9 @@ class Rotation:
out.append((master, reason))
return out
async def tick(
self, now: datetime | None = None, *, force: str | None = None
) -> list[Conversation]:
"""Rotate the masters that are due; ``force`` rotates every open master."""
async def tick(self, now: datetime | None = None) -> list[Conversation]:
rotated: list[Conversation] = []
if force:
due = [
(m, force)
for m in await self._conversations.find(kind="master", status="open")
]
else:
due = await self.due(now)
for master, reason in due:
for master, reason in await self.due(now):
new = await self.rotate(master, reason)
if new is not None:
rotated.append(new)
+2 -82
View File
@@ -8,8 +8,6 @@ from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any, cast
from uuid import uuid4
from sqlalchemy import String, func
from sqlalchemy import cast as sa_cast
from sqlmodel import col, select
from beaver_gateway.backends.transcript import (
@@ -19,14 +17,12 @@ from beaver_gateway.backends.transcript import (
)
from beaver_gateway.conversations.kinds import KINDS, Kind
from beaver_gateway.conversations.state import State, iso
from beaver_gateway.conversations.texts import split_service
from beaver_gateway.frontends.markdown.history import load_messages
from beaver_gateway.storage.models import (
Conversation,
ConversationBinding,
ConversationMessage,
RateLimit,
TranscriptEntry,
Usage,
)
@@ -136,30 +132,6 @@ class Rows(State):
async with self._db.session() as session:
return list((await session.exec(stmt)).all())
async def search(self, query: str, *, limit: int = 20) -> list[Conversation]:
needle = f"%{query}%"
by_title = select(Conversation.id).where(col(Conversation.title).ilike(needle))
by_first = select(ConversationMessage.conversation_id).where(
ConversationMessage.seq == 0,
col(ConversationMessage.content_json).ilike(needle),
)
by_entry = (
select(Conversation.id)
.join(
TranscriptEntry,
col(TranscriptEntry.session_id) == col(Conversation.session_id),
)
.where(sa_cast(col(TranscriptEntry.entry), String).ilike(needle))
)
stmt = (
select(Conversation)
.where(col(Conversation.id).in_(by_title.union(by_first, by_entry)))
.order_by(col(Conversation.id).desc())
.limit(limit)
)
async with self._db.session() as session:
return list((await session.exec(stmt)).all())
async def bindings(self, conv: Conversation) -> list[ConversationBinding]:
async with self._db.session() as session:
result = await session.exec(
@@ -345,7 +317,6 @@ class Rows(State):
async def describe(self, conv: Conversation) -> dict[str, Any]:
out = self.public(conv)
out["title"] = await self.implied_title(conv)
out["context_tokens"] = await self.context_tokens(conv)
parent = await self.get_row(conv.parent_id) if conv.parent_id else None
out["parent"] = parent.external_id if parent is not None else None
out["bindings"] = [
@@ -353,8 +324,6 @@ class Rows(State):
for b in await self.bindings(conv)
]
live = self._pool.get(conv.external_id)
agent = self._agents.get(conv.agent_name)
out["thinking_is_progress"] = agent is not None and agent.thinking_is_progress
out["live"] = live is not None
out["busy"] = live.busy if live is not None else False
runner = self._runners.get(cast("int", conv.id))
@@ -390,25 +359,6 @@ class Rows(State):
).first()
return context_of(row)
async def context_tokens_by_id(self, ids: Iterable[str]) -> dict[str, int]:
wanted = list(ids)
if not wanted:
return {}
latest = (
select(func.max(Usage.id))
.where(col(Usage.conversation_id).in_(wanted))
.group_by(Usage.conversation_id)
)
async with self._db.session() as session:
rows = (
await session.exec(select(Usage).where(col(Usage.id).in_(latest)))
).all()
return {
cast("str", row.conversation_id): context_of(row)
for row in rows
if row.conversation_id
}
async def usage_tokens(self, since: datetime) -> int:
async with self._db.session() as session:
rows = (
@@ -469,13 +419,7 @@ class Rows(State):
return marked
async def read(self, conv: Conversation, *, window: int | None = None) -> str:
return render_messages(
await self.history(conv),
thinking_is_progress=self._claude_agent(
conv.agent_name
).thinking_is_progress,
window=window,
)
return render_messages(await self.history(conv), window=window)
async def history(self, conv: Conversation) -> list[dict[str, Any]]:
if conv.session_id is None:
@@ -483,31 +427,7 @@ class Rows(State):
return await load_messages(
session, conversation_id=cast("int", conv.id)
)
messages = messages_from_entries(
cast("Any", await self.entries(conv)), stamped=True
)
return [self._unwrapped(message) for message in messages]
def _unwrapped(self, message: dict[str, Any]) -> dict[str, Any]:
if message["role"] != "user":
return message
content = message["content"]
if isinstance(content, str):
text, service = split_service(content, self._texts)
return (
message
if service is None
else {**message, "content": text, "service": service}
)
for index, block in enumerate(content):
if block.get("type") != "text":
continue
text, service = split_service(str(block.get("text") or ""), self._texts)
if service is not None:
blocks = [*content]
blocks[index] = {**block, "text": text}
return {**message, "content": blocks, "service": service}
return message
return messages_from_entries(cast("Any", await self.entries(conv)))
async def entries(self, conv: Conversation, *, subpath: str = "") -> list[Any]:
if conv.session_id is None:
+1 -1
View File
@@ -1,7 +1,7 @@
"""``Conversations`` - the service every frontend, job and gateway tool talks to.
Built as layers, one file each: rows → seeds → turns → messaging →
spawning → rewinding → questions → closing; this file adds start, stop and restart
spawning → questions → closing; this file adds start, stop and restart
recovery. A turn started by a user message streams back to whoever asked;
a turn started by an inject streams nowhere and can only speak via ``say``.
"""
+2 -3
View File
@@ -44,7 +44,6 @@ class Spawning(Messaging):
origin: str = "api",
binding: tuple[str, str] | None = None,
flags: dict[str, Any] | None = None,
attachments: list[dict[str, Any]] | None = None,
) -> Conversation:
"""Create a conversation in a window and queue its seed.
@@ -96,7 +95,6 @@ class Spawning(Messaging):
priority="user",
origin=f"seed:{seed}" if seed == "brief" else origin,
text=await self.seed_text(ctx, window=window),
attachments=attachments,
)
self._ensure_worker(cast("int", conv.id))
return conv
@@ -148,7 +146,8 @@ class Spawning(Messaging):
)
if result.text.strip():
await self.inject(parent, result.text, urgency="normal", origin="merge")
await self.mark_closed(await self.set_status(conv, "merged"))
await self.set_status(conv, "merged")
await self.mark_closed(conv)
self._bus.publish(
"conversation.merged",
conversation_id=conv.external_id,
@@ -127,12 +127,6 @@ class State:
self._tasks = set()
self._idle_task = None
self.scheduler = None
for backend in backends.values():
if hasattr(backend, "waker"):
backend.waker = self._woken
async def _woken(self, key: str, note: str) -> None:
raise NotImplementedError
@property
def db(self) -> Database:
-15
View File
@@ -2,7 +2,6 @@
from __future__ import annotations
import re
from dataclasses import dataclass, field
from typing import TYPE_CHECKING
@@ -23,7 +22,6 @@ __all__ = [
"NewDayContext",
"SeedContext",
"UserSaid",
"split_service",
]
@@ -53,8 +51,6 @@ class UserSaid:
title: str | None
text: str
at: datetime
agent: str = ""
"""Agent whose turn it is; a setup with several tells them apart by it."""
@dataclass(frozen=True, slots=True)
@@ -81,8 +77,6 @@ class ConversationTexts:
)
inject_header: Callable[[injects.InjectContext], str] = injects.inject_header
bundle_header: str = "[injects accumulated since {since}; not the user]"
task_prompt: str = "[subagent reported back: {note}]"
"""Stands in for the user's words in mirrors of a turn a subagent started."""
interrupted: str = "interrupted"
answered: str = "The user answered: {answer}"
unanswered: str = (
@@ -129,12 +123,3 @@ class ConversationTexts:
"; {fixed} open tool calls received tool_result «{interrupted}»"
)
envelope: EnvelopeTexts = field(default_factory=EnvelopeTexts)
def split_service(text: str, texts: ConversationTexts) -> tuple[str, str | None]:
bundle = re.escape(texts.bundle_header).replace(r"\{since\}", ".+")
heads = f"{re.escape(texts.envelope.header)}|{bundle}"
match = re.search(rf"(?:\A|\n\n)(?=(?:{heads})$)", text, re.MULTILINE)
if match is None:
return text, None
return text[: match.start()], text[match.end() :]
+2 -46
View File
@@ -14,7 +14,6 @@ from claude_agent_sdk import create_sdk_mcp_server, tool
from beaver_gateway.conversations.injects import URGENCY
from beaver_gateway.conversations.kinds import as_kind
from beaver_gateway.frontends.files import AUTO, METHODS, SendFileError
from beaver_gateway.security.redact import redact_data
URGENCY_HELP = (
@@ -43,23 +42,7 @@ SAY_IN_USER_TURN = (
"reply text reaches them by itself - put what you wanted to say into the "
"reply instead of repeating it here"
)
TOOL_NAMES = (
"read_conversation",
"spawn",
"say",
"schedule",
"inject",
"close_chat",
"send_file",
)
SEND_FILE_HELP = (
"Send a file from disk to the human, into this conversation's window (the "
"master's if it has none): {note}. `method` is how Telegram renders it; "
"`auto` reads the extension - image is a photo, mp3/flac/m4a audio, ogg "
"voice, mp4/mov video, anything else a document. Photo and video are "
"recompressed, so pass `document` when the bytes must arrive untouched. "
"`caption` takes the same markdown as your replies."
)
TOOL_NAMES = ("read_conversation", "spawn", "say", "schedule", "inject", "close_chat")
def build_tool_server(
@@ -249,33 +232,6 @@ def _tools(conversations: Conversations, key: str) -> list[SdkMcpTool[Any]]:
)
return _text(f"queued #{item.id}")
@tool(
"send_file",
SEND_FILE_HELP.format(note=conversations.file_note()),
{
"type": "object",
"properties": {
"path": {"type": "string", "description": "absolute path on disk"},
"caption": {"type": "string"},
"method": {"type": "string", "enum": [AUTO, *METHODS], "default": AUTO},
},
"required": ["path"],
},
)
async def send_file(args: dict[str, Any]) -> dict[str, Any]:
conv = await current()
try:
return _text(
await conversations.send_file(
conv,
str(args["path"]),
caption=str(args.get("caption") or ""),
method=str(args.get("method") or AUTO),
)
)
except SendFileError as exc:
return _error(str(exc))
@tool(
"close_chat",
"Close this deep chat once the current reply is finished: the "
@@ -292,7 +248,7 @@ def _tools(conversations: Conversations, key: str) -> list[SdkMcpTool[Any]]:
return _error(str(exc))
return _text("ok: the chat closes after this reply")
return [read_conversation, spawn, say, schedule, inject, send_file, close_chat]
return [read_conversation, spawn, say, schedule, inject, close_chat]
def _text(text: str) -> dict[str, Any]:
+30 -138
View File
@@ -3,12 +3,10 @@
from __future__ import annotations
import asyncio
import base64
import contextlib
import inspect
import logging
from datetime import UTC, datetime
from pathlib import Path
from typing import TYPE_CHECKING, Any, cast
from uuid import uuid4
@@ -23,7 +21,7 @@ from claude_agent_sdk import (
)
from beaver_gateway.backends.capture import TurnCapture
from beaver_gateway.backends.transcript import reply_text, text_of
from beaver_gateway.backends.transcript import text_of
from beaver_gateway.conversations import injects
from beaver_gateway.conversations.seeds import Seeds
from beaver_gateway.conversations.state import Runner, aware, iso
@@ -36,7 +34,7 @@ if TYPE_CHECKING:
from beaver_gateway.events.stream import MessageStreamEvent
__all__ = ["Turns", "content_of", "image_block"]
__all__ = ["Turns"]
_log = logging.getLogger(__name__)
@@ -54,28 +52,21 @@ class Turns(Seeds):
tools: bool = True,
turn_id: str | None = None,
item_origin: str | None = None,
shown: str | None = None,
attach: bool = False,
) -> AsyncIterator[MessageStreamEvent]:
"""Run one turn under the conversation's lock; the only path to the backend.
``shown`` is what frontends display as the turn's prompt (the user's
words, not the envelope). ``attach`` sends nothing: the turn is one
the live session started by itself, a subagent reporting back.
"""
"""Run one turn under the conversation's lock; the only path to the backend."""
row_id = cast("int", conv.id)
runner = self._runner(row_id)
backend = self._backend(conv.agent_name)
turn_id = turn_id or f"turn_{uuid4().hex[:12]}"
capture = capture or TurnCapture()
resume = session_id if session_id is not None else conv.session_id
async with runner.lock:
runner.turn_id = turn_id
runner.origin = origin
runner.text = shown if shown is not None else _prompt_preview(messages)
runner.text = _prompt_preview(messages)
runner.started_at = datetime.now(UTC)
runner.tools = {}
row = await self._mark_running(conv, turn_id)
resume = session_id if session_id is not None else row.session_id
await self._mark_running(conv, turn_id)
before = await self.before_turn(conv)
self._bus.publish(
"turn.start",
@@ -88,28 +79,18 @@ class Turns(Seeds):
stop = "error"
cut = False
try:
observer = self._observer(conv, runner, turn_id, origin)
events = (
backend.attach(
key=conv.external_id,
capture=capture,
observer=observer,
turn_id=turn_id,
)
if attach
else backend.complete(
agent=self._claude_agent(conv.agent_name),
messages=messages,
conversation_id=conv.external_id,
session_id=resume if use_session else None,
reseed=not use_session,
capture=capture,
kind=conv.kind,
pinned=conv.kind == "master",
tools=tools,
observer=observer,
turn_id=turn_id,
)
events = backend.complete(
agent=self._claude_agent(conv.agent_name),
messages=messages,
conversation_id=conv.external_id,
session_id=resume if use_session else None,
reseed=not use_session,
capture=capture,
kind=conv.kind,
pinned=conv.kind == "master",
tools=tools,
observer=self._observer(conv, runner, turn_id, origin),
turn_id=turn_id,
)
async for event in events:
yield event
@@ -144,29 +125,27 @@ class Turns(Seeds):
tools: bool = True,
turn_id: str | None = None,
item_origin: str | None = None,
attachments: Sequence[dict[str, Any]] | None = None,
shown: str | None = None,
attach: bool = False,
) -> tuple[str, TurnCapture]:
capture = TurnCapture()
acc = StreamAccumulator()
agent = self._claude_agent(conv.agent_name)
async for event in self.turn(
conv,
messages=[{"role": "user", "content": content_of(text, attachments)}],
messages=[{"role": "user", "content": text}],
origin=origin,
capture=capture,
tools=tools,
turn_id=turn_id,
item_origin=item_origin,
shown=shown,
attach=attach,
):
acc.feed(event)
message = acc.finalize(model=agent.model)
return reply_text(
message.content, thinking_is_progress=agent.thinking_is_progress
), capture
reply = "\n\n".join(
getattr(b, "text", "")
for b in message.content
if getattr(b, "type", "") == "text"
).strip()
return reply, capture
async def before_turn(self, conv: Conversation) -> str | None: # noqa: ARG002
return None
@@ -178,47 +157,6 @@ class Turns(Seeds):
runner = self._runners.get(cast("int", conv.id))
return runner.origin if runner is not None and runner.turn_id else None
async def _woken(self, key: str, note: str) -> None:
"""The live session of ``key`` started a turn by itself; run it next."""
conv = await self.get(key)
if conv is None:
_log.warning("self-started turn on unknown conversation %s: %s", key, note)
return
_log.info("conversation %s woke by itself: %s", key, note)
self._ensure_worker(cast("int", conv.id))
async def _run_pending(self, conv: Conversation) -> None:
turn_id = f"turn_{uuid4().hex[:12]}"
note = self._backend(conv.agent_name).pending_note(conv.external_id) or ""
try:
text, _ = await self.run_text_turn(
conv,
note,
origin="task",
turn_id=turn_id,
item_origin="task",
shown=note,
attach=True,
)
except Exception: # noqa: BLE001
_log.exception(
"self-started turn %s on %s failed", turn_id, conv.external_id
)
return
if not text.strip():
return
self._bus.publish(
"reply",
conversation_id=conv.external_id,
turn_id=turn_id,
item=None,
item_origin="task",
source="task",
prompt=self._texts.task_prompt.format(note=note),
user_text=None,
text=text,
)
def _ensure_worker(self, row_id: int) -> None:
runner = self._runner(row_id)
runner.wake.set()
@@ -229,10 +167,6 @@ class Turns(Seeds):
async def _worker(self, row_id: int) -> None:
runner = self._runner(row_id)
while True:
conv = await self.get_row(row_id)
if conv is not None and self._has_pending(conv):
await self._run_pending(conv)
continue
items = await self._queue.pending(row_id)
batch, wait = self._pick(items)
if batch is None:
@@ -290,13 +224,7 @@ class Turns(Seeds):
prompt = f"{seed}\n\n{prompt}"
try:
text, capture = await self.run_text_turn(
conv,
prompt,
origin=origin,
turn_id=turn_id,
item_origin=head.origin,
attachments=head.attachments if origin == "user" else None,
shown=head.text if origin == "user" else None,
conv, prompt, origin=origin, turn_id=turn_id, item_origin=head.origin
)
except Exception: # noqa: BLE001
_log.exception("turn %s on %s failed", turn_id, conv.external_id)
@@ -318,12 +246,6 @@ class Turns(Seeds):
text=text,
)
def _has_pending(self, conv: Conversation) -> bool:
try:
return self._backend(conv.agent_name).has_pending(conv.external_id)
except LookupError:
return False
def _bundle(self, items: Sequence[InjectQueueItem]) -> str:
lines = [self._texts.bundle_header.format(since=iso(items[0].created_at))]
lines.extend(f"- [{i.origin}] {i.text}" for i in items)
@@ -333,11 +255,9 @@ class Turns(Seeds):
if self._envelope is None:
return None
if conv.kind == "master":
return self._envelope.build(text=text, kind="master", agent=conv.agent_name)
return self._envelope.build(text=text, kind="master")
if conv.kind == "branch":
return self._envelope.recall_only(
text=text, kind="branch", agent=conv.agent_name
)
return self._envelope.recall_only(text=text, kind="branch")
return None
async def _note_user(self, conv: Conversation, text: str) -> None:
@@ -349,7 +269,6 @@ class Turns(Seeds):
title=conv.title,
text=text,
at=datetime.now(UTC),
agent=conv.agent_name,
)
try:
result = self._user_sink(message)
@@ -463,12 +382,12 @@ class Turns(Seeds):
except Exception: # noqa: BLE001
_log.exception("rate limit write failed")
async def _mark_running(self, conv: Conversation, turn_id: str) -> Conversation:
async def _mark_running(self, conv: Conversation, turn_id: str) -> None:
async def apply(row: Conversation) -> None:
row.running_turn = turn_id
row.last_activity_at = datetime.now(UTC)
return await self._update(conv, apply)
await self._update(conv, apply)
async def _mark_done(
self, conv: Conversation, capture: TurnCapture, *, cut: bool = False
@@ -483,33 +402,6 @@ class Turns(Seeds):
await self._update(conv, apply)
IMAGE_MAX_BYTES = 5 * 1024 * 1024
def content_of(
text: str, attachments: Sequence[dict[str, Any]] | None
) -> str | list[dict[str, Any]]:
"""Plain text, or text plus the images that ride with it."""
blocks = [b for a in attachments or () if (b := image_block(a)) is not None]
if not blocks:
return text
return [{"type": "text", "text": text}, *blocks]
def image_block(attachment: dict[str, Any]) -> dict[str, Any] | None:
media_type = str(attachment.get("media_type") or "")
path = Path(str(attachment.get("path") or ""))
if not media_type.startswith("image/") or not path.is_file():
return None
if path.stat().st_size > IMAGE_MAX_BYTES:
return None
data = base64.b64encode(path.read_bytes()).decode("ascii")
return {
"type": "image",
"source": {"type": "base64", "media_type": media_type, "data": data},
}
def _prompt_preview(messages: Sequence[Any], limit: int = 400) -> str | None:
if not messages:
return None
+1 -7
View File
@@ -57,16 +57,10 @@ class EventBus:
event = await queue.get()
if (
conversation_id is None
or _conversation_of(event) == conversation_id
or event.get("conversation_id") == conversation_id
):
yield event
@property
def subscribers(self) -> int:
return len(self._subscribers)
def _conversation_of(event: Event) -> Any:
if event["type"].startswith("conversation."):
return event.get("id")
return event.get("conversation_id")
+9 -9
View File
@@ -17,7 +17,7 @@ from fastapi.responses import JSONResponse, StreamingResponse
from beaver_gateway.agents.claude import ClaudeAgent
from beaver_gateway.backends.capture import TurnCapture
from beaver_gateway.backends.transcript import fingerprint, reply_text, text_of
from beaver_gateway.backends.transcript import fingerprint, text_of
from beaver_gateway.frontends.accumulate import StreamAccumulator
from beaver_gateway.frontends.base import Frontend
from beaver_gateway.frontends.bearer import require_token
@@ -29,7 +29,6 @@ if TYPE_CHECKING:
from anthropic.types import Message, MessageParam
from beaver_gateway.agents.base import BaseAgent
from beaver_gateway.conversations.service import Conversations
from beaver_gateway.events.stream import MessageStreamEvent
from beaver_gateway.frontends.base import GatewayRuntime
@@ -159,7 +158,6 @@ class AnthropicMessagesFrontend(Frontend):
capture=capture,
turn_id=turn_id,
history=history,
thinking_is_progress=agent.thinking_is_progress,
)
else:
options: dict[str, Any] = {}
@@ -172,7 +170,7 @@ class AnthropicMessagesFrontend(Frontend):
async def after(message: Message) -> None:
await _broadcast_turn(
runtime,
agent=agent,
agent_name=agent.name,
input_messages=messages,
output_message=message,
system=system_str,
@@ -237,9 +235,12 @@ async def _finish(
capture: TurnCapture,
turn_id: str,
history: list[dict[str, str]] | None,
thinking_is_progress: bool,
) -> None:
reply = reply_text(message.content, thinking_is_progress=thinking_is_progress)
reply = "\n\n".join(
getattr(b, "text", "")
for b in message.content
if getattr(b, "type", "") == "text"
).strip()
tail = capture.synthesized_messages or [
{"role": "assistant", "content": [{"type": "text", "text": reply}]}
]
@@ -290,7 +291,7 @@ async def _sse(
async def _broadcast_turn(
runtime: GatewayRuntime,
*,
agent: BaseAgent,
agent_name: str,
input_messages: list[MessageParam],
output_message: Message,
system: str | None,
@@ -298,10 +299,9 @@ async def _broadcast_turn(
if not runtime.turn_log_handlers:
return
record = TurnRecord(
agent_name=agent.name,
agent_name=agent_name,
input_messages=list(input_messages),
output_message=output_message,
thinking_is_progress=agent.thinking_is_progress,
system=system,
source=FRONTEND,
)
+20 -429
View File
@@ -1,13 +1,11 @@
"""``ApiFrontend`` - ``/api/*``: conversations, SSE, sessions, usage, limits.
Bearer scope ``api``; tokens, secrets and audit need ``admin``. Every
Bearer scope ``api``; token and audit management need ``admin``. Every
write goes through conversations/service.py; the frontend only shapes JSON.
"""
from __future__ import annotations
import base64
import binascii
import json
import logging
import secrets
@@ -20,10 +18,6 @@ from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse, StreamingResponse
from sqlmodel import col, select
from beaver_gateway.agents.claude import ClaudeAgent
from beaver_gateway.agents.prompts import assemble
from beaver_gateway.backends.transcript import prompt_count
from beaver_gateway.conversations.context import compose
from beaver_gateway.conversations.injects import URGENCY
from beaver_gateway.conversations.kinds import Kind, as_kind
from beaver_gateway.conversations.service import SEEDS, implied_title
@@ -36,9 +30,8 @@ from beaver_gateway.frontends.sse import (
sse_pack,
)
from beaver_gateway.frontends.urls import frontend_url
from beaver_gateway.keystore import SecretsDisabledError
from beaver_gateway.security import audit
from beaver_gateway.security.auth import hash_token
from beaver_gateway.security.auth import VALID_SCOPES, hash_token
from beaver_gateway.storage import (
create_token,
list_audit_records,
@@ -49,17 +42,14 @@ from beaver_gateway.storage.models import (
Conversation,
InjectQueueItem,
RateLimit,
Secret,
Token,
Usage,
)
from beaver_gateway.vault.links import LinkIndex
if TYPE_CHECKING:
from collections.abc import AsyncIterator, Iterable, Sequence
from pathlib import Path
from beaver_gateway.backends.sessions import Session, SessionPool
from beaver_gateway.conversations.service import Conversations
from beaver_gateway.frontends.base import GatewayRuntime
from beaver_gateway.jobs.scheduler import Job, Scheduler
@@ -80,12 +70,6 @@ WINDOWS: dict[str, timedelta] = {
MEMORY_MAX_DEPTH = 12
MEMORY_MAX_FILE = 2_000_000
MEMORY_MAX_ENTRIES = 5000
SEARCH_MIN = 2
GRAPH_LIMIT = 120
SEARCH_LIMIT = 20
TURNS_LIMIT = 10
TURNS_MAX = 50
SNIPPET_MAX = 160
class ApiFrontend(Frontend):
@@ -101,14 +85,12 @@ class ApiFrontend(Frontend):
deep_agent: str | None = None,
job_agent: str | None = None,
memory_root: Path | None = None,
vault_root: Path | None = None,
) -> None:
self.master_agent = master_agent
self.branch_agent = branch_agent
self.deep_agent = deep_agent
self.job_agent = job_agent
self.memory_root = memory_root.resolve() if memory_root is not None else None
self.vault_root = vault_root.resolve() if vault_root is not None else None
self._app: FastAPI | None = None
def agent_for(self, kind: Kind) -> str | None:
@@ -123,22 +105,14 @@ class ApiFrontend(Frontend):
if runtime.conversations is None or runtime.bus is None:
msg = "ApiFrontend needs runtime.conversations and runtime.bus"
raise RuntimeError(msg)
self._app = build_app(
runtime, memory_root=self.memory_root, vault_root=self.vault_root
)
self._app = build_app(runtime, memory_root=self.memory_root)
def app(self) -> FastAPI | None:
return self._app
def build_app( # noqa: PLR0915
runtime: GatewayRuntime,
*,
memory_root: Path | None = None,
vault_root: Path | None = None,
) -> FastAPI:
def build_app(runtime: GatewayRuntime, *, memory_root: Path | None = None) -> FastAPI: # noqa: PLR0915
app = FastAPI(title="beaver-gateway / API")
links = LinkIndex(vault_root) if vault_root is not None else None
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
@@ -224,7 +198,6 @@ def build_app( # noqa: PLR0915
k: fe.agent_for(k) for k in fe.kinds if fe.agent_for(k)
},
"path": fe.path,
"scope": fe.scope,
"url": frontend_url(request, runtime, fe),
}
for fe in runtime.frontends
@@ -245,49 +218,17 @@ def build_app( # noqa: PLR0915
firsts = await conversations.first_user_texts(
cast("int", r.id) for r in rows if not r.title
)
contexts = await conversations.context_tokens_by_id(r.external_id for r in rows)
public_ids = {r.id: r.external_id for r in rows}
return {
"conversations": [
{
**conversations.public(r),
"title": r.title or implied_title(firsts.get(cast("int", r.id))),
"last_item": _queue_item(latest.get(cast("int", r.id))),
"context_tokens": contexts.get(r.external_id, 0),
"parent": public_ids.get(r.parent_id) if r.parent_id else None,
}
for r in rows
]
}
@app.get("/search")
async def search(request: Request) -> dict[str, Any]:
await require_token(request, runtime, scope=SCOPE)
query = (request.query_params.get("q") or "").strip()
if len(query) < SEARCH_MIN:
raise HTTPException(
status.HTTP_400_BAD_REQUEST, f"q must be at least {SEARCH_MIN} chars"
)
limit = min(query_int(request, "limit", SEARCH_LIMIT), SEARCH_LIMIT)
rows = await conversations.search(query, limit=limit)
firsts = await conversations.first_user_texts(
cast("int", r.id) for r in rows if not r.title
)
contexts = await conversations.context_tokens_by_id(r.external_id for r in rows)
files = _grep_memory(memory_root, query, limit=limit) if memory_root else []
return {
"query": query,
"conversations": [
{
**conversations.public(r),
"title": r.title or implied_title(firsts.get(cast("int", r.id))),
"context_tokens": contexts.get(r.external_id, 0),
}
for r in rows
],
"files": files,
}
@app.post("/conversations", status_code=status.HTTP_201_CREATED)
async def create_conversation(request: Request) -> dict[str, Any]:
token = await require_token(request, runtime, scope=SCOPE)
@@ -360,45 +301,6 @@ def build_app( # noqa: PLR0915
conv = await conv_of(public_id)
return {"id": conv.external_id, "messages": await conversations.history(conv)}
@app.get("/conversations/{public_id}/context")
async def get_context(public_id: str, request: Request) -> dict[str, Any]:
await require_token(request, runtime, scope=SCOPE)
conv = await conv_of(public_id)
agent = runtime.agents.get(conv.agent_name)
if not isinstance(agent, ClaudeAgent):
raise HTTPException(
status.HTTP_404_NOT_FOUND, f"no claude agent behind {public_id}"
)
entries = list(await conversations.entries(conv))
for sub in await conversations.subpaths(conv):
entries.extend(await conversations.entries(conv, subpath=sub))
out = compose(
agent,
as_kind(conv.kind),
entries,
context_tokens=await conversations.context_tokens(conv),
)
out["id"] = conv.external_id
out["turns"] = prompt_count(entries)
if request.query_params.get("prompt"):
sources = agent.prompt_for(as_kind(conv.kind))
out["prompt_text"] = (
assemble(sources) if sources is not None else agent.system_prompt
)
return out
@app.get("/vault/graph")
async def vault_graph(request: Request) -> dict[str, Any]:
await require_token(request, runtime, scope=SCOPE)
if links is None:
raise HTTPException(
status.HTTP_404_NOT_FOUND,
"no vault root: set ApiFrontend(vault_root=...)",
)
paths = [p for p in request.query_params.getlist("path") if p]
limit = min(query_int(request, "limit", GRAPH_LIMIT), GRAPH_LIMIT)
return {"root": str(links.root), **links.neighbours(paths, limit=limit)}
@app.get("/conversations/{public_id}/entries")
async def get_entries(public_id: str, request: Request) -> dict[str, Any]:
await require_token(request, runtime, scope=SCOPE)
@@ -584,42 +486,6 @@ def build_app( # noqa: PLR0915
"text": result.text,
}
@app.get("/conversations/{public_id}/turns")
async def get_turns(public_id: str, request: Request) -> dict[str, Any]:
await require_token(request, runtime, scope=SCOPE)
conv = await conv_of(public_id)
limit = min(query_int(request, "limit", TURNS_LIMIT), TURNS_MAX)
return {
"id": conv.external_id,
"turns": await conversations.turns(conv, limit=limit),
}
@app.post("/conversations/{public_id}/rewind")
async def post_rewind(public_id: str, request: Request) -> dict[str, Any]:
token = await require_token(request, runtime, scope=ADMIN_SCOPE)
conv = await conv_of(public_id)
turn = text_of(await body_of(request), "turn")
try:
result = await conversations.rewind(conv, turn)
except LookupError as exc:
raise HTTPException(status.HTTP_404_NOT_FOUND, str(exc)) from exc
except RuntimeError as exc:
raise HTTPException(status.HTTP_409_CONFLICT, str(exc)) from exc
out = {
"turns": result.turns,
"session_id": result.conversation.session_id,
"previous_session": result.previous_session,
}
await audit.log(
runtime,
actor=f"token:{token}",
kind="api_rewind",
agent_name=conv.agent_name,
conversation=conv.external_id,
**out,
)
return {"id": conv.external_id, **out}
@app.post("/conversations/{public_id}/bind")
async def post_bind(public_id: str, request: Request) -> dict[str, Any]:
await require_token(request, runtime, scope=SCOPE)
@@ -676,74 +542,15 @@ def build_app( # noqa: PLR0915
await require_token(request, runtime, scope=SCOPE)
return _sse(runtime, conversation_id=None)
def pool_of() -> SessionPool:
if runtime.pool is None:
raise HTTPException(status.HTTP_404_NOT_FOUND, "no session pool")
return runtime.pool
def session_of(key: str) -> Session:
session = pool_of().get(key)
if session is None:
raise HTTPException(status.HTTP_404_NOT_FOUND, f"no live session {key}")
return session
@app.get("/sessions")
async def sessions(request: Request) -> dict[str, Any]:
await require_token(request, runtime, scope=SCOPE)
if runtime.pool is None:
return {
"rss": None,
"rss_limit": None,
"cpu": None,
"sessions": [],
"others": [],
}
return runtime.pool.census()
@app.get("/sessions/{key}/stderr")
async def session_stderr(key: str, request: Request) -> dict[str, Any]:
await require_token(request, runtime, scope=ADMIN_SCOPE)
return {"key": key, "lines": list(session_of(key).stderr)}
@app.post("/sessions/{key}/interrupt")
async def session_interrupt(key: str, request: Request) -> dict[str, Any]:
token = await require_token(request, runtime, scope=SCOPE)
session = session_of(key)
interrupted = await pool_of().interrupt(key)
await audit.log(
runtime,
actor=f"token:{token}",
kind="session_interrupt",
agent_name=session.agent,
session=key,
interrupted=interrupted,
)
return {"key": key, "interrupted": interrupted}
@app.post("/sessions/{key}/close")
async def session_close(key: str, request: Request) -> dict[str, Any]:
token = await require_token(request, runtime, scope=ADMIN_SCOPE)
session = session_of(key)
await pool_of().close(key)
await audit.log(
runtime,
actor=f"token:{token}",
kind="session_close",
agent_name=session.agent,
session=key,
busy=session.busy,
)
return {"key": key, "closed": True}
@app.post("/processes/{pid}/kill")
async def process_kill(pid: int, request: Request) -> dict[str, Any]:
token = await require_token(request, runtime, scope=ADMIN_SCOPE)
if not await pool_of().kill(pid):
raise HTTPException(
status.HTTP_404_NOT_FOUND, f"no process {pid} under the gateway"
)
await audit.log(runtime, actor=f"token:{token}", kind="process_kill", pid=pid)
return {"pid": pid, "killed": True}
pool = runtime.pool
return {
"rss": pool.rss() if pool is not None else None,
"rss_limit": pool.rss_limit if pool is not None else None,
"sessions": pool.snapshot() if pool is not None else [],
}
@app.get("/schedules")
async def schedules(request: Request) -> dict[str, Any]:
@@ -840,51 +647,22 @@ def build_app( # noqa: PLR0915
await require_token(request, runtime, scope=SCOPE)
rows = await conversations.rate_limits(limit=200)
latest: dict[str, RateLimit] = {}
unified: dict[str, tuple[RateLimit, dict[str, Any]]] = {}
for row in rows:
latest.setdefault(row.window, row)
for name, info in _unified_windows(row).items():
unified.setdefault(name, (row, info))
now = datetime.now(UTC)
windows = []
for window in dict.fromkeys([*latest, *unified]):
row = latest.get(window)
seen = unified.get(window)
public = _limit_public(row) if row else _limit_blank(window, seen)
resets = row.resets_at if row else None
if seen and _newer_figure(seen, row):
carrier, info = seen
public["utilization"] = info.get("utilization")
public["reported_at"] = _iso(carrier.ts)
resets = _unix_dt(info.get("resetsAt")) or resets
public["resets_at"] = _iso(resets)
length = _window_length(window)
anchor = row if row is not None else (seen[0] if seen else None)
if length is not None and resets is not None:
since = _aware(resets) - length
else:
since = _aware(anchor.ts) if anchor else now
for window, row in latest.items():
length = WINDOWS.get(window)
since = (
_aware(row.resets_at) - length
if length is not None and row.resets_at is not None
else _aware(row.ts)
)
gateway = _sum_usage(await _usage_rows(runtime, since, now))
gateway["since"] = since.isoformat(timespec="seconds")
known = next(
(
r
for r in rows
if r.window == window
and r.utilization is not None
and _aware(r.ts) >= since
),
None,
)
windows.append(
{
**public,
"gateway": gateway,
"last_known": _limit_public(known) if known else None,
}
)
windows.append({**_limit_public(row), "gateway": gateway})
windows.sort(
key=lambda w: (_window_length(w["window"]) or timedelta.max, w["window"])
key=lambda w: (WINDOWS.get(w["window"], timedelta.max), w["window"])
)
return {"windows": windows, "history": [_limit_public(r) for r in rows[:50]]}
@@ -917,19 +695,6 @@ def build_app( # noqa: PLR0915
"content": content,
}
@app.get("/scopes")
async def scopes(request: Request) -> dict[str, Any]:
await require_token(request, runtime, scope=ADMIN_SCOPE)
by_frontend = {
fe.scope: fe.name or fe.path or "" for fe in runtime.frontends if fe.scope
}
return {
"scopes": [
{"scope": s, "frontend": by_frontend.get(s)}
for s in sorted(runtime.scopes)
]
}
@app.get("/tokens")
async def tokens(request: Request) -> dict[str, Any]:
await require_token(request, runtime, scope=ADMIN_SCOPE)
@@ -944,7 +709,7 @@ def build_app( # noqa: PLR0915
data = await body_of(request)
name = text_of(data, "name").strip()
scope = str(data.get("scope") or "*")
if scope not in runtime.scopes:
if scope not in VALID_SCOPES:
raise HTTPException(status.HTTP_400_BAD_REQUEST, f"invalid scope {scope!r}")
plaintext = secrets.token_urlsafe(32)
async with runtime.db.session() as session:
@@ -982,58 +747,6 @@ def build_app( # noqa: PLR0915
)
return {"id": token_id, "revoked": True}
@app.get("/secrets")
async def secrets_list(request: Request) -> dict[str, Any]:
await require_token(request, runtime, scope=ADMIN_SCOPE)
store = runtime.secrets
rows = await store.rows() if store is not None else []
return {
"enabled": store is not None and store.enabled,
"secrets": [_secret_public(r) for r in rows],
}
@app.put("/secrets/{name}")
async def secret_put(name: str, request: Request) -> dict[str, Any]:
actor = await require_token(request, runtime, scope=ADMIN_SCOPE)
store = runtime.secrets
if store is None or not store.enabled:
raise HTTPException(
status.HTTP_503_SERVICE_UNAVAILABLE, "SECRETS_KEY is not set"
)
data = await body_of(request)
description = data.get("description")
payload, filename = _secret_payload(data)
try:
row = await store.put(
name,
description=str(description) if description is not None else None,
data=payload,
filename=filename,
)
except SecretsDisabledError as exc:
raise HTTPException(status.HTTP_503_SERVICE_UNAVAILABLE, str(exc)) from exc
except ValueError as exc:
raise HTTPException(status.HTTP_400_BAD_REQUEST, str(exc)) from exc
await audit.log(
runtime,
actor=f"token:{actor}",
kind="secret_set",
name=name,
replaced=payload is not None,
)
return {"secret": _secret_public(row)}
@app.delete("/secrets/{name}")
async def secret_delete(name: str, request: Request) -> dict[str, Any]:
actor = await require_token(request, runtime, scope=ADMIN_SCOPE)
store = runtime.secrets
if store is None or not await store.delete(name):
raise HTTPException(status.HTTP_404_NOT_FOUND, f"no secret {name}")
await audit.log(
runtime, actor=f"token:{actor}", kind="secret_delete", name=name
)
return {"name": name, "deleted": True}
@app.get("/audit")
async def audit_list(request: Request) -> dict[str, Any]:
await require_token(request, runtime, scope=ADMIN_SCOPE)
@@ -1231,59 +944,6 @@ def _queue_item(item: InjectQueueItem | None) -> dict[str, Any] | None:
}
def _window_length(name: str) -> timedelta | None:
if name.startswith("five_hour"):
return timedelta(hours=5)
if "seven_day" in name:
return timedelta(days=7)
return WINDOWS.get(name)
def _unix_dt(value: Any) -> datetime | None:
if isinstance(value, int | float) and value > 0:
return datetime.fromtimestamp(value, tz=UTC)
return None
def _unified_windows(row: RateLimit) -> dict[str, dict[str, Any]]:
"""Every window the event carried a figure for, not just the one it was about."""
raw = row.raw if isinstance(row.raw, dict) else {}
found = raw.get("unifiedWindows")
if not isinstance(found, dict):
return {}
return {
str(name): info
for name, info in found.items()
if isinstance(info, dict) and info.get("utilization") is not None
}
def _newer_figure(
seen: tuple[RateLimit, dict[str, Any]], row: RateLimit | None
) -> bool:
if row is None or row.utilization is None:
return True
return seen[0].id != row.id and _aware(seen[0].ts) > _aware(row.ts)
def _limit_blank(
window: str, seen: tuple[RateLimit, dict[str, Any]] | None
) -> dict[str, Any]:
carrier = seen[0] if seen else None
return {
"id": None,
"ts": _iso(carrier.ts) if carrier else None,
"window": window,
"status": "allowed",
"utilization": None,
"resets_at": None,
"overage_status": None,
"overage_resets_at": None,
"agent": carrier.agent_name if carrier else None,
"session_id": carrier.session_id if carrier else None,
}
def _limit_public(row: RateLimit) -> dict[str, Any]:
return {
"id": row.id,
@@ -1310,46 +970,6 @@ def _token_public(row: Token) -> dict[str, Any]:
}
def _secret_public(row: Secret) -> dict[str, Any]:
return {
"name": row.name,
"description": row.description,
"kind": "file" if row.filename else "value",
"filename": row.filename,
"size": row.size,
"created_at": _iso(row.created_at),
"updated_at": _iso(row.updated_at),
"used_at": _iso(row.used_at),
}
def _secret_payload(data: dict[str, Any]) -> tuple[bytes | None, str | None]:
value, upload = data.get("value"), data.get("file")
if value is not None and upload is not None:
raise HTTPException(
status.HTTP_400_BAD_REQUEST, "send `value` or `file`, not both"
)
if isinstance(value, str):
if not value:
raise HTTPException(status.HTTP_400_BAD_REQUEST, "`value` is empty")
return value.encode(), None
if isinstance(upload, dict):
filename, content = upload.get("name"), upload.get("content")
if not isinstance(filename, str) or not isinstance(content, str):
raise HTTPException(
status.HTTP_400_BAD_REQUEST, "`file` needs `name` and base64 `content`"
)
try:
return base64.b64decode(content, validate=True), filename
except binascii.Error as exc:
raise HTTPException(
status.HTTP_400_BAD_REQUEST, "`file.content` is not base64"
) from exc
if value is not None or upload is not None:
raise HTTPException(status.HTTP_400_BAD_REQUEST, "`value` must be a string")
return None, None
def _detail(raw: str) -> Any:
try:
return json.loads(raw) if raw else {}
@@ -1373,35 +993,6 @@ def _memory_path(root: Path, raw: str) -> Path:
return target
def _grep_memory(root: Path, query: str, *, limit: int) -> list[dict[str, Any]]:
needle = query.casefold()
hits: list[dict[str, Any]] = []
for path in sorted(root.rglob("*")):
if len(hits) >= limit:
break
if not path.is_file() or any(part.startswith(".") for part in path.parts):
continue
if path.suffix not in (".md", ".txt", ".json", ".yaml", ".yml"):
continue
try:
if path.stat().st_size > MEMORY_MAX_FILE:
continue
text = path.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
continue
rel = str(path.relative_to(root))
if needle in rel.casefold():
hits.append({"path": rel, "line": 0, "snippet": ""})
continue
for number, line in enumerate(text.splitlines(), 1):
if needle in line.casefold():
hits.append(
{"path": rel, "line": number, "snippet": line.strip()[:SNIPPET_MAX]}
)
break
return hits
def _tree(root: Path, directory: Path, *, depth: int) -> list[dict[str, Any]]:
if depth > MEMORY_MAX_DEPTH:
return []
-34
View File
@@ -10,12 +10,8 @@ from abc import ABC, abstractmethod
from dataclasses import dataclass, field
from typing import TYPE_CHECKING, Any
from beaver_gateway.frontends.files import SendFileError
from beaver_gateway.security.auth import BUILTIN_SCOPES
if TYPE_CHECKING:
from collections.abc import Awaitable, Callable, Mapping, Sequence
from pathlib import Path
from starlette.types import ASGIApp
@@ -23,7 +19,6 @@ if TYPE_CHECKING:
from beaver_gateway.backends.base import Backend
from beaver_gateway.conversations.kinds import Kind
from beaver_gateway.frontends.turn_record import TurnRecord
from beaver_gateway.keystore import SecretStore
from beaver_gateway.security.auth import TokenStore
from beaver_gateway.storage import Database
from beaver_gateway.storage.models import Conversation, ConversationBinding
@@ -59,12 +54,6 @@ class GatewayRuntime:
scheduler: Any = None
public_url: str | None = None
"""``Gateway.public_url``; ``None`` derives the origin from the request."""
scopes: frozenset[str] = BUILTIN_SCOPES
"""Every token scope this gateway knows: the builtins plus each frontend's."""
send_file_roots: tuple[Path, ...] = ()
"""Directories ``send_file`` may read, from ``SEND_FILE_ROOTS``."""
secrets: SecretStore | None = None
"""Sealed secrets for ``secret run``; ``None`` in setups without them."""
class Frontend(ABC):
@@ -75,10 +64,6 @@ class Frontend(ABC):
is for non-HTTP work (polling, vault mirrors) and defaults to nothing.
``landing`` marks the app that ``/`` redirects to.
``scope`` is the token scope this frontend's routes are gated by; a
frontend that names its own scope gets keys nobody else's token opens,
and ``POST /api/tokens`` will mint them (see ``security/auth.py``).
A frontend that shows conversations declares ``name`` (the binding
key) and ``kinds`` (which conversation kinds it shows). The first
frontend whose ``materialize`` returns a binding is the *home* of
@@ -91,9 +76,6 @@ class Frontend(ABC):
kinds: tuple[Kind, ...] = ()
path: str | None = None
landing: bool = False
scope: str | None = None
sends_files: bool = False
"""Whether ``send_file`` reaches a human through this frontend."""
@abstractmethod
def configure(self, runtime: GatewayRuntime) -> None: ...
@@ -113,19 +95,3 @@ class Frontend(ABC):
async def mark_closed(self, conv: Conversation) -> bool: # noqa: ARG002
"""Show in the window that the conversation is over (a renamed topic)."""
return False
async def send_file(
self,
conv: Conversation, # noqa: ARG002
path: str, # noqa: ARG002
*,
caption: str = "", # noqa: ARG002
method: str = "auto", # noqa: ARG002
) -> str:
"""Put a file in this conversation's window; the answer goes to the model."""
msg = f"frontend {self.name!r} cannot send files"
raise SendFileError(msg)
def file_note(self) -> str:
"""Roots and ceiling, for the tool description the model reads."""
return ""
-95
View File
@@ -1,95 +0,0 @@
"""What an agent may hand to a human: an allowed path, a ceiling, a method.
The ceiling and the method names belong to the frontend's API; the check
lives here so every frontend refuses the same way, in words the model reads.
"""
from __future__ import annotations
import mimetypes
from dataclasses import dataclass, replace
from pathlib import Path
__all__ = ["AUTO", "DEFAULTS", "METHODS", "SendFileError", "SendFiles", "method_for"]
AUTO = "auto"
METHODS = ("document", "photo", "audio", "voice", "video", "video_note", "animation")
_BY_SUFFIX = {
".jpg": "photo",
".jpeg": "photo",
".png": "photo",
".webp": "photo",
".gif": "animation",
".mp3": "audio",
".flac": "audio",
".m4a": "audio",
".wav": "audio",
".aac": "audio",
".ogg": "voice",
".oga": "voice",
".opus": "voice",
".mp4": "video",
".m4v": "video",
".mov": "video",
}
_BY_TYPE = {"image": "photo", "audio": "audio", "video": "video"}
class SendFileError(Exception):
"""Why a file was not sent, worded for the model that asked."""
@dataclass(frozen=True, slots=True)
class SendFiles:
"""Which files a frontend accepts from an agent and how big they may be."""
roots: tuple[Path, ...] = ()
"""Directories a path must resolve inside; nothing outside is sendable."""
max_bytes: int = 50 * 1024 * 1024
"""Upload ceiling of the API behind the frontend; the Bot API gives 50 MB."""
def with_roots(self, *roots: Path) -> SendFiles:
return replace(self, roots=tuple(dict.fromkeys((*self.roots, *roots))))
@property
def note(self) -> str:
"""Roots and ceiling as one line for the tool description."""
where = ", ".join(str(r) for r in self.roots) or "none, so nothing is sendable"
return f"paths under {where}, up to {mb(self.max_bytes)} MB"
def resolve(self, raw: str) -> Path:
path = Path(raw.strip()).expanduser().resolve()
if not any(path.is_relative_to(r.expanduser().resolve()) for r in self.roots):
msg = f"{path} is outside the allowed roots ({self.note}); copy it there"
raise SendFileError(msg)
if not path.is_file():
msg = f"{path} is not a file"
raise SendFileError(msg)
size = path.stat().st_size
if size > self.max_bytes:
msg = (
f"{path.name} is {mb(size)} MB, over the {mb(self.max_bytes)} MB limit"
)
raise SendFileError(msg)
return path
DEFAULTS = SendFiles()
def method_for(path: Path, method: str = AUTO) -> str:
if method and method != AUTO:
if method not in METHODS:
msg = f"unknown method {method!r}; one of: {', '.join((AUTO, *METHODS))}"
raise SendFileError(msg)
return method
guessed = _BY_SUFFIX.get(path.suffix.lower())
if guessed:
return guessed
mime, _ = mimetypes.guess_type(path.name)
return _BY_TYPE.get(mime.split("/")[0], "document") if mime else "document"
def mb(size: int) -> str:
return f"{round(size / 1024 / 1024, 2):g}"
@@ -142,16 +142,14 @@ class CrossFrontendLogger:
new_user = record.input_messages[-1]
new_block = renderer.render_user_param(new_user)
new_block = renderer.append_to_body(
new_block,
renderer.render_assistant_message(
record.output_message,
thinking_is_progress=record.thinking_is_progress,
),
new_block, renderer.render_assistant_message(record.output_message)
)
new_body = renderer.append_to_body(body, new_block)
metadata = dict(parsed.metadata)
else:
new_body = _render_full_history(record)
new_body = _render_full_history(
record.input_messages, record.output_message
)
metadata = {}
new_body = renderer.append_to_body(new_body, renderer.USER_SCAFFOLD)
@@ -206,10 +204,10 @@ def _flatten_text(message: Any) -> str:
return "\n\n".join(c for c in chunks if c)
def _render_full_history(record: TurnRecord) -> str:
def _render_full_history(messages: list[MessageParam], assistant: Any) -> str:
"""Render an entire conversation (used when materializing a new logged file)."""
blocks: list[str] = []
for msg in record.input_messages:
for msg in messages:
role = msg.get("role")
if role == "user":
blocks.append(renderer.render_user_param(msg))
@@ -217,11 +215,7 @@ def _render_full_history(record: TurnRecord) -> str:
content = msg.get("content", "")
text = content if isinstance(content, str) else _content_to_text(content)
blocks.append(f"### Assistant:\n\n{text.strip()}\n" if text.strip() else "")
blocks.append(
renderer.render_assistant_message(
record.output_message, thinking_is_progress=record.thinking_is_progress
)
)
blocks.append(renderer.render_assistant_message(assistant))
body = ""
for block in blocks:
if not block:
@@ -341,11 +341,7 @@ class MarkdownFrontend(Frontend):
file_path=file_path,
first_user_text=parsed.turns[0].text,
)
outcome = diff_and_fork(
stored=stored_msgs,
incoming=parsed.turns,
thinking_is_progress=agent.thinking_is_progress,
)
outcome = diff_and_fork(stored=stored_msgs, incoming=parsed.turns)
capture = TurnCapture()
events = runtime.conversations.turn(
conv,
@@ -360,7 +356,6 @@ class MarkdownFrontend(Frontend):
file_path=file_path,
parsed=parsed,
model=agent.model or agent.name,
thinking_is_progress=agent.thinking_is_progress,
filename=filename,
write_disk=write_disk,
)
@@ -376,7 +371,6 @@ class MarkdownFrontend(Frontend):
parsed=parsed,
message=message,
agent_name=agent.name,
thinking_is_progress=agent.thinking_is_progress,
conv_external_id=conv_external_id,
write_disk=write_disk,
)
@@ -394,7 +388,6 @@ class MarkdownFrontend(Frontend):
agent_name=agent.name,
input_messages=list(parsed.messages),
output_message=message,
thinking_is_progress=agent.thinking_is_progress,
system=None,
source="markdown",
)
@@ -541,11 +534,7 @@ class MarkdownFrontend(Frontend):
len(stored_msgs),
len(parsed.turns),
)
outcome = diff_and_fork(
stored=stored_msgs,
incoming=parsed.turns,
thinking_is_progress=agent.thinking_is_progress,
)
outcome = diff_and_fork(stored=stored_msgs, incoming=parsed.turns)
_log.info(
"chat/stream: file=%s diff_and_fork divergence=%s "
"backend_msgs=%d persist_msgs=%d",
@@ -578,9 +567,7 @@ class MarkdownFrontend(Frontend):
partial = acc.finalize(model=model)
if not partial.content:
return None
rendered = renderer.render_assistant_message(
partial, thinking_is_progress=agent.thinking_is_progress
)
rendered = renderer.render_assistant_message(partial)
new_body = renderer.append_to_body(parsed.body, rendered)
return reattach_frontmatter(parsed.metadata, new_body)
@@ -606,10 +593,7 @@ class MarkdownFrontend(Frontend):
new_body = parsed.body
if partial.content:
new_body = renderer.append_to_body(
new_body,
renderer.render_assistant_message(
partial, thinking_is_progress=agent.thinking_is_progress
),
new_body, renderer.render_assistant_message(partial)
)
new_body = renderer.append_to_body(new_body, _render_error_block(exc))
if write_disk:
@@ -632,7 +616,6 @@ class MarkdownFrontend(Frontend):
parsed=parsed,
message=message,
agent_name=agent.name,
thinking_is_progress=agent.thinking_is_progress,
conv_external_id=conv_external_id,
write_disk=write_disk,
)
@@ -650,7 +633,6 @@ class MarkdownFrontend(Frontend):
agent_name=agent.name,
input_messages=list(parsed.messages),
output_message=message,
thinking_is_progress=agent.thinking_is_progress,
system=None,
source="markdown",
)
@@ -677,7 +659,6 @@ class MarkdownFrontend(Frontend):
file_path: Path,
parsed: parser.ParsedFile,
model: str,
thinking_is_progress: bool,
filename: str,
write_disk: bool = True,
) -> Any:
@@ -694,9 +675,7 @@ class MarkdownFrontend(Frontend):
partial = acc.finalize(model=model)
if not partial.content:
return
rendered = renderer.render_assistant_message(
partial, thinking_is_progress=thinking_is_progress
)
rendered = renderer.render_assistant_message(partial)
new_body = renderer.append_to_body(parsed.body, rendered)
await write_atomic(
file_path, reattach_frontmatter(parsed.metadata, new_body)
@@ -719,10 +698,7 @@ class MarkdownFrontend(Frontend):
new_body = parsed.body
if partial.content:
new_body = renderer.append_to_body(
new_body,
renderer.render_assistant_message(
partial, thinking_is_progress=thinking_is_progress
),
new_body, renderer.render_assistant_message(partial)
)
new_body = renderer.append_to_body(new_body, _render_error_block(exc))
if write_disk:
@@ -739,14 +715,11 @@ class MarkdownFrontend(Frontend):
parsed: parser.ParsedFile,
message: Any,
agent_name: str,
thinking_is_progress: bool,
conv_external_id: str,
write_disk: bool = True,
) -> str:
"""Render the assistant turn, refresh frontmatter, write if we own the file."""
rendered = renderer.render_assistant_message(
message, thinking_is_progress=thinking_is_progress
)
rendered = renderer.render_assistant_message(message)
new_body = renderer.append_to_body(parsed.body, rendered)
new_body = renderer.append_to_body(new_body, renderer.USER_SCAFFOLD)
updated_metadata = dict(parsed.metadata)
@@ -14,7 +14,6 @@ from typing import TYPE_CHECKING, Any, cast
from sqlalchemy import delete
from sqlmodel import select
from beaver_gateway.backends.transcript import spoken_text
from beaver_gateway.storage.models import Conversation, ConversationMessage
if TYPE_CHECKING:
@@ -181,9 +180,7 @@ class _StoredDisplayTurn:
text_segment_count: int
def _group_display_turns(
stored: list[dict[str, Any]], *, thinking_is_progress: bool
) -> list[_StoredDisplayTurn]:
def _group_display_turns(stored: list[dict[str, Any]]) -> list[_StoredDisplayTurn]:
"""Walk raw stored messages, group them into Obsidian-visible turns.
A user-prompt message (no ``tool_result`` blocks) opens a new turn;
@@ -213,9 +210,7 @@ def _group_display_turns(
break
group.append(m)
i += 1
spoken, skeleton, text_count = _summarize_assistant_group(
group, thinking_is_progress=thinking_is_progress
)
spoken, skeleton, text_count = _summarize_assistant_group(group)
out.append(
_StoredDisplayTurn(
role="assistant",
@@ -256,13 +251,12 @@ def _user_prompt_text(content: Any) -> str:
def _summarize_assistant_group(
group: list[dict[str, Any]], *, thinking_is_progress: bool
group: list[dict[str, Any]],
) -> tuple[str, list[str], int]:
"""Compute (spoken_text, tool_skeleton, text_segment_count) for a display group.
Must mirror what the renderer writes and the parser reads back: text and
progress thinking collapse into one segment, tool_use becomes a skeleton
entry, tool_result, reasoning and empty thinking are invisible.
Must mirror ``parser.parse_assistant_structure``: text blocks collapse into
one segment, tool_use becomes a skeleton entry, tool_result/thinking are invisible.
"""
from beaver_gateway.frontends.markdown.parser import TextSegment, ToolSegment
@@ -288,10 +282,12 @@ def _summarize_assistant_group(
for blk in content:
if not isinstance(blk, dict):
continue
text = spoken_text(blk, thinking_is_progress=thinking_is_progress)
if text:
pending.append(text)
elif blk.get("type") == "tool_use":
btype = blk.get("type")
if btype == "text":
text = str(blk.get("text", "")).strip()
if text:
pending.append(text)
elif btype == "tool_use":
_flush()
segments.append(ToolSegment(name=str(blk.get("name", ""))))
_flush()
@@ -303,10 +299,7 @@ def _summarize_assistant_group(
def diff_and_fork(
*,
stored: list[dict[str, Any]],
incoming: list[ParsedTurn],
thinking_is_progress: bool,
*, stored: list[dict[str, Any]], incoming: list[ParsedTurn]
) -> ForkOutcome:
"""Align the incoming parsed file against stored history.
@@ -321,15 +314,11 @@ def diff_and_fork(
)
raise ValueError(msg)
stored_groups = _group_display_turns(
stored, thinking_is_progress=thinking_is_progress
)
stored_groups = _group_display_turns(stored)
new_user_turn = incoming[-1]
prior_incoming = incoming[:-1]
spliced_groups, divergence, edited = _walk_prefix(
prior_incoming, stored_groups, thinking_is_progress=thinking_is_progress
)
spliced_groups, divergence, edited = _walk_prefix(prior_incoming, stored_groups)
if divergence is None and len(prior_incoming) < len(stored_groups):
if _file_lags_store(stored_groups, len(prior_incoming), new_user_turn):
@@ -369,10 +358,7 @@ def _file_lags_store(
def _walk_prefix(
prior_incoming: list[ParsedTurn],
stored_groups: list[_StoredDisplayTurn],
*,
thinking_is_progress: bool,
prior_incoming: list[ParsedTurn], stored_groups: list[_StoredDisplayTurn]
) -> tuple[list[list[dict[str, Any]]], int | None, bool]:
"""Walk incoming vs stored side-by-side until first divergence.
@@ -405,9 +391,7 @@ def _walk_prefix(
continue
if inc_skeleton and inc_text_count != st.text_segment_count:
return spliced_groups, i, edited
spliced = _splice_assistant_group(
stored_group=st, incoming=inc, thinking_is_progress=thinking_is_progress
)
spliced = _splice_assistant_group(stored_group=st, incoming=inc)
if spliced is None:
return spliced_groups, i, edited
spliced_groups.append(spliced)
@@ -442,10 +426,7 @@ def _assemble_tail(
def _splice_assistant_group(
*,
stored_group: _StoredDisplayTurn,
incoming: ParsedTurn,
thinking_is_progress: bool,
*, stored_group: _StoredDisplayTurn, incoming: ParsedTurn
) -> list[dict[str, Any]] | None:
"""Apply the file's prose to a stored assistant turn.
@@ -453,28 +434,27 @@ def _splice_assistant_group(
markdown can't express. Falls back to rebuilding from the file's
structure when the text blocks don't line up one-to-one.
"""
preserved = _splice_in_place(
stored_group=stored_group,
incoming=incoming,
thinking_is_progress=thinking_is_progress,
)
preserved = _splice_in_place(stored_group=stored_group, incoming=incoming)
if preserved is not None:
return preserved
return _splice_by_rebuild(stored_group=stored_group, incoming=incoming)
def _is_text_block(blk: object) -> bool:
if not isinstance(blk, dict):
return False
block = cast("dict[str, Any]", blk)
return block.get("type") == "text" and bool(str(block.get("text", "")).strip())
def _splice_in_place(
*,
stored_group: _StoredDisplayTurn,
incoming: ParsedTurn,
thinking_is_progress: bool,
*, stored_group: _StoredDisplayTurn, incoming: ParsedTurn
) -> list[dict[str, Any]] | None:
"""Copy the stored messages, swapping only their text block contents.
Preserves message/thinking-block boundaries that a rebuild would lose
(and that a cache-hit continuation depends on); returns ``None`` if
text blocks aren't 1:1 with incoming segments or the prose runs through
signed thinking blocks, so the caller rebuilds instead.
text blocks aren't 1:1 with incoming segments, so the caller rebuilds instead.
"""
from beaver_gateway.frontends.markdown.parser import TextSegment
@@ -486,12 +466,9 @@ def _splice_in_place(
content = msg.get("content")
if msg["role"] != "assistant" or not isinstance(content, list):
continue
for bi, blk in enumerate(cast("list[dict[str, Any]]", content)):
if not spoken_text(blk, thinking_is_progress=thinking_is_progress):
continue
if blk["type"] == "thinking":
return None
positions.append((mi, bi))
positions.extend(
(mi, bi) for bi, blk in enumerate(content) if _is_text_block(blk)
)
if len(positions) != len(new_texts):
return None
@@ -12,6 +12,8 @@ from typing import TYPE_CHECKING, Any, cast
from anthropic.types import Message, TextBlock, ThinkingBlock
if TYPE_CHECKING:
from collections.abc import Iterable
from anthropic.types import MessageParam
@@ -43,25 +45,19 @@ def render_assistant_text(text: str) -> str:
return f"### Assistant:\n\n{text.strip()}\n"
def render_assistant_message(message: Message, *, thinking_is_progress: bool) -> str:
def render_assistant_message(message: Message) -> str:
"""Render an assistant ``Message`` into a turn block.
Tool-use blocks render to nothing. Progress thinking renders as text in
place and vanishes when empty; reasoning renders as a collapsed callout,
of which only the first between two texts reaches the file.
Tool-use blocks render to nothing; only text and thinking content reach
the file (see :mod:`.parser` for how the reverse strip works).
"""
parts: list[str] = ["### Assistant:", ""]
after_reasoning = False
for block in message.content:
if isinstance(block, ThinkingBlock) and not thinking_is_progress:
if not after_reasoning:
parts.extend((*_render_reasoning(block.thinking or ""), ""))
after_reasoning = True
lines = list(_render_block(block))
if not lines:
continue
text = _spoken(block)
if text:
parts.extend((text, ""))
after_reasoning = False
parts.extend(lines)
parts.append("")
return "\n".join(parts).rstrip() + "\n"
@@ -107,17 +103,21 @@ def summarize_tool_input(name: str, tool_input: object) -> str:
return name
def _render_reasoning(text: str) -> tuple[str, ...]:
lines = text.strip().splitlines() or [""]
return ("> [!thinking]-", *(f"> {line}" if line else ">" for line in lines))
def _spoken(block: object) -> str:
def _render_block(block: object) -> Iterable[str]:
if isinstance(block, TextBlock):
return (block.text or "").strip()
text = (block.text or "").strip()
if text:
yield text
return
if isinstance(block, ThinkingBlock):
return (block.thinking or "").strip()
return ""
yield from _render_thinking(block.thinking or "")
return
def _render_thinking(text: str) -> Iterable[str]:
yield "> [!thinking]-"
for line in text.strip().splitlines() or [""]:
yield f"> {line}" if line else ">"
def adaptive_fence(content: str) -> str:
@@ -9,7 +9,6 @@ from __future__ import annotations
import asyncio
import contextlib
import html
import logging
import time
import zlib
@@ -27,7 +26,6 @@ __all__ = ["Draft"]
_log = logging.getLogger("beaver_gateway.frontends.telegram.drafts")
_TAIL = 3500
_REASONING_TAIL = 400
_KEEPALIVE = 20.0
@@ -49,18 +47,12 @@ class Draft:
self._interval = interval
self.status = status
self.text = ""
self.reasoning = ""
self._paused = False
self._dirty = True
self._broken = False
self._last_sent = 0.0
self._task: asyncio.Task[None] | None = None
self._inflight: asyncio.Future[None] | None = None
@property
def draft_id(self) -> int:
return self._draft_id
def start(self) -> None:
if self._task is None:
self._task = asyncio.create_task(self._run())
@@ -69,32 +61,10 @@ class Draft:
if status != self.status:
self.status = status
self._dirty = True
self._paused = bool(self.text.strip())
def append(self, text: str) -> None:
"""Add streamed text; a tool call in between starts a new paragraph.
The model ends a sentence, calls a tool, then goes on without a
break - the same text the final message shows as paragraphs.
"""
if not text:
return
if self._paused and not self.text.endswith("\n"):
self.text = self.text.rstrip() + "\n\n" + text.lstrip()
else:
self.text += text
self.reasoning = ""
self._paused = False
self._dirty = True
def think(self, text: str) -> None:
if text:
self.reasoning += text
self._dirty = True
def rethink(self) -> None:
if self.reasoning:
self.reasoning = ""
self.text += text
self._dirty = True
async def stop(self) -> None:
@@ -156,25 +126,11 @@ class Draft:
message_thread_id=self._thread_id,
text=text,
parse_mode=parse_mode,
can_stop=True,
)
def _render(self) -> tuple[str, str]:
rendered, plain = [to_html(self.status)], [self.status]
thought = _tail(self.reasoning.strip(), _REASONING_TAIL)
if thought:
rendered.append(f"<blockquote>{html.escape(thought)}</blockquote>")
plain.append(thought)
tail = self.text[-_TAIL:]
if tail.strip():
rendered.append(to_html_tail(self.text, _TAIL))
plain.append(tail)
return "\n\n".join(rendered), "\n\n".join(plain)
def _tail(text: str, limit: int) -> str:
if len(text) <= limit:
return text
cut = text[-limit:]
_, _, rest = cut.partition(" ")
return f"… {rest or cut}"
if not tail.strip():
return to_html(self.status), self.status
rendered = f"{to_html(self.status)}\n\n{to_html_tail(self.text, _TAIL)}"
return rendered, f"{self.status}\n\n{tail}"
+73 -372
View File
@@ -21,13 +21,10 @@ from zoneinfo import ZoneInfo
from aiogram import Bot
from aiogram.client.default import DefaultBotProperties
from aiogram.client.session.aiohttp import AiohttpSession
from aiogram.client.telegram import TelegramAPIServer
from aiogram.exceptions import TelegramAPIError, TelegramNetworkError
from aiogram.exceptions import TelegramAPIError
from aiogram.types import (
BotCommand,
CallbackQuery,
FSInputFile,
InlineKeyboardButton,
InlineKeyboardMarkup,
Message,
@@ -36,17 +33,10 @@ from aiogram.types import (
)
from beaver_gateway.frontends.base import Frontend
from beaver_gateway.frontends.files import (
DEFAULTS,
SendFileError,
SendFiles,
mb,
method_for,
)
from beaver_gateway.frontends.telegram.drafts import Draft
from beaver_gateway.frontends.telegram.inbox import Inbox
from beaver_gateway.frontends.telegram.outbox import Outbox
from beaver_gateway.frontends.telegram.render import chunks, status_label, to_html
from beaver_gateway.frontends.telegram.render import chunks, status_label
from beaver_gateway.frontends.telegram.texts import TelegramTexts
if TYPE_CHECKING:
@@ -63,8 +53,6 @@ _log = logging.getLogger("beaver_gateway.frontends.telegram")
FRONTEND = "telegram"
_DONE = "done"
_COMMANDS = ("merge", "new", "chat", "status", "help", "start")
_CAPTION = 1024
_NO_CAPTION = ("video_note",)
@dataclass(frozen=True, slots=True)
@@ -107,46 +95,10 @@ class _Ask:
picked: dict[int, list[str]] = field(default_factory=dict)
done: set[int] = field(default_factory=set)
def to_flags(self, question_id: str) -> dict[str, Any]:
return {
"id": question_id,
"conversation_id": self.conversation_id,
"chat_id": self.chat_id,
"thread_id": self.thread_id,
"questions": self.questions,
"messages": self.messages,
"picked": {str(k): v for k, v in self.picked.items()},
"done": sorted(self.done),
}
@classmethod
def from_flags(cls, data: dict[str, Any]) -> _Ask:
return cls(
conversation_id=str(data["conversation_id"]),
chat_id=int(data["chat_id"]),
thread_id=data.get("thread_id"),
questions=list(data.get("questions") or []),
messages=list(data.get("messages") or []),
picked={int(k): list(v) for k, v in (data.get("picked") or {}).items()},
done=set(data.get("done") or []),
)
@dataclass
class _Album:
message: Message
thread_id: int | None
is_master: bool
texts: list[str] = field(default_factory=list)
notes: list[str] = field(default_factory=list)
attachments: list[dict[str, Any]] = field(default_factory=list)
task: asyncio.Task[None] | None = None
class TelegramFrontend(Frontend):
name = FRONTEND
kinds = ("master", "branch")
sends_files = True
def __init__(
self,
@@ -157,15 +109,12 @@ class TelegramFrontend(Frontend):
branch_agent: str | None = None,
chat_id: int | None = None,
attachments: Attachments = EPHEMERAL,
send_files: SendFiles = DEFAULTS,
api_base_url: str | None = None,
draft_interval: float = 0.7,
master_topic: str = "🦫 General",
queued_reaction: str = "👀",
poll_timeout: int = 30,
outbox_backoff: float = 2.0,
texts: TelegramTexts | None = None,
album_delay: float = 1.0,
) -> None:
self._token = token
self.texts = texts or TelegramTexts()
@@ -174,8 +123,6 @@ class TelegramFrontend(Frontend):
self.master_agent = master_agent
self.branch_agent = branch_agent
self.attachments = attachments
self.send_files = send_files
self.api_base_url = api_base_url
self.master_topic = master_topic
self.draft_interval = draft_interval
self.queued_reaction = queued_reaction
@@ -189,9 +136,7 @@ class TelegramFrontend(Frontend):
self._master_window: str | None = None
self._topic_names: dict[int, str] = {}
self._drafts: dict[str, Draft] = {}
self.album_delay = album_delay
self._asks: dict[str, _Ask] = {}
self._albums: dict[str, _Album] = {}
self._reactions: dict[int, tuple[int, int]] = {}
self._tasks: set[asyncio.Task[None]] = set()
@@ -204,14 +149,7 @@ class TelegramFrontend(Frontend):
raise RuntimeError(msg)
self._runtime = runtime
if self._bot is None:
self._bot = Bot(
self._token,
session=self._session(),
default=DefaultBotProperties(parse_mode=None),
)
self.send_files = self.send_files.with_roots(
*runtime.send_file_roots, self.attachments.root
)
self._bot = Bot(self._token, default=DefaultBotProperties(parse_mode=None))
self._inbox = Inbox(
runtime.db, self._bot, handler=self._handle, poll_timeout=self.poll_timeout
)
@@ -219,16 +157,6 @@ class TelegramFrontend(Frontend):
runtime.db, self._bot, bus=runtime.bus, backoff=self.outbox_backoff
)
def _session(self) -> AiohttpSession | None:
"""A local Bot API server instead of api.telegram.org (2 GB uploads)."""
if not self.api_base_url:
return None
return AiohttpSession(
api=TelegramAPIServer.from_base(
self.api_base_url.rstrip("/"), is_local=True
)
)
async def serve(self) -> None:
me = await self.bot.get_me()
_log.info(
@@ -268,10 +196,7 @@ class TelegramFrontend(Frontend):
)
async def mark_closed(self, conv: Conversation) -> bool:
merged = conv.status == "merged"
return await self.mark_topic(
conv, self.texts.closed_prefix if merged else self.texts.idle_prefix
)
return await self.mark_topic(conv)
async def mark_topic(self, conv: Conversation, prefix: str | None = None) -> bool:
"""Rename the topic; closeForumTopic does not exist in private chats."""
@@ -282,47 +207,12 @@ class TelegramFrontend(Frontend):
name = self._topic_names.get(target[1]) or conv.title or self.texts.branch
if name.startswith(prefix):
return True
await self._rename(target[1], f"{prefix}{self._unmarked(name)}")
await self.bot.edit_forum_topic(
target[0], target[1], name=f"{prefix}{name}"[:128]
)
self._topic_names[target[1]] = f"{prefix}{name}"
return True
def file_note(self) -> str:
return self.send_files.note
async def send_file(
self, conv: Conversation, path: str, *, caption: str = "", method: str = "auto"
) -> str:
"""Upload the file into this conversation's topic, caption and all."""
target = await self._target_of(conv)
if target is None:
msg = f"conversation {conv.external_id} has no Telegram window"
raise SendFileError(msg)
file = self.send_files.resolve(path)
how = method_for(file, method)
rest = caption.strip()
payload: dict[str, Any] = {
"message_thread_id": target[1],
how: FSInputFile(file),
}
if how == "audio":
payload["title"] = file.stem
if rest and how not in _NO_CAPTION and len(rest) <= _CAPTION:
payload |= {"caption": to_html(rest), "parse_mode": "HTML"}
rest = ""
try:
await getattr(self.bot, f"send_{how}")(target[0], **payload)
except TelegramNetworkError as exc:
msg = f"Telegram did not answer while sending {file.name}: {exc}"
raise SendFileError(msg) from exc
except TelegramAPIError as exc:
msg = f"Telegram refused {file.name} as {how}: {exc}"
raise SendFileError(msg) from exc
except (OSError, TimeoutError) as exc:
msg = f"sending {file.name} failed: {exc}"
raise SendFileError(msg) from exc
if rest:
await self._deliver(conv, rest)
return f"sent {file.name} as {how}, {mb(file.stat().st_size)} MB"
@property
def bot(self) -> Bot:
if self._bot is None:
@@ -412,9 +302,7 @@ class TelegramFrontend(Frontend):
self._master_window = self._ext(topic.message_thread_id)
return self._master_window
async def _master(
self, text: str | None = None, attachments: list[dict[str, Any]] | None = None
) -> tuple[Conversation, bool]:
async def _master(self, text: str | None = None) -> tuple[Conversation, bool]:
"""The open master behind its window, spawning one when there is none.
``text`` rides with the seed of a fresh master; the second value says
@@ -436,17 +324,11 @@ class TelegramFrontend(Frontend):
text=text,
origin=FRONTEND,
binding=(FRONTEND, ext),
attachments=attachments,
)
return conv, text is not None
async def _branch(
self,
thread_id: int,
*,
title: str | None,
text: str | None,
attachments: list[dict[str, Any]] | None = None,
self, thread_id: int, *, title: str | None, text: str | None
) -> Conversation:
master, _ = await self._master()
return await self.conversations.spawn(
@@ -459,46 +341,13 @@ class TelegramFrontend(Frontend):
text=text,
origin=FRONTEND,
binding=(FRONTEND, self._ext(thread_id)),
attachments=attachments,
)
async def _branch_in_topic(
self, topic: int, *, text: str, attachments: list[dict[str, Any]] | None
) -> None:
"""A fresh topic is named by the first line, a closed one loses its mark."""
current = self._topic_names.get(topic)
previous = None
if current is None:
previous = await self.conversations.find_bound(
frontend=FRONTEND, external_id=self._ext(topic)
)
known = current or (previous.title if previous is not None else None)
title = (self._unmarked(known) if known else "") or self._title_from(text)
await self._branch(topic, title=title, text=text, attachments=attachments)
if current == title:
return
try:
await self._rename(topic, title)
except TelegramAPIError as exc:
_log.warning("topic %s not renamed to %r: %s", topic, title, exc)
async def _rename(self, topic: int, name: str) -> None:
name = name[:128]
await self.bot.edit_forum_topic(self.chat_id, topic, name=name)
self._topic_names[topic] = name
def _unmarked(self, name: str) -> str:
for prefix in (self.texts.closed_prefix, self.texts.idle_prefix):
name = name.removeprefix(prefix)
return name
async def _handle(self, update: Update) -> None:
if update.message is not None:
await self._on_message(update.message)
elif update.callback_query is not None:
await self._on_callback(update.callback_query)
elif update.stopped_message_generation is not None:
await self._on_stopped(update.stopped_message_generation.draft_id)
async def _on_message(self, message: Message) -> None:
created = message.forum_topic_created
@@ -533,79 +382,9 @@ class TelegramFrontend(Frontend):
)
return
text = (message.text or message.caption or "").strip()
note, attachment = await self._save_attachment(message)
if message.media_group_id:
self._collect_album(
message,
thread_id,
is_master=is_master,
text=text,
note=note,
attachment=attachment,
)
return
await self._dispatch(
message,
thread_id,
is_master=is_master,
text=text,
notes=[note] if note else [],
attachments=[attachment] if attachment else [],
)
def _collect_album(
self,
message: Message,
thread_id: int | None,
*,
is_master: bool,
text: str,
note: str | None,
attachment: dict[str, Any] | None,
) -> None:
"""An album arrives as separate messages; it becomes one turn."""
group = str(message.media_group_id)
album = self._albums.get(group)
if album is None:
album = _Album(message=message, thread_id=thread_id, is_master=is_master)
self._albums[group] = album
if text:
album.texts.append(text)
if note:
album.notes.append(note)
attachment = await self._save_attachment(message)
if attachment:
album.attachments.append(attachment)
if album.task is not None:
album.task.cancel()
album.task = asyncio.create_task(self._flush_album(group))
self._tasks.add(album.task)
album.task.add_done_callback(self._tasks.discard)
async def _flush_album(self, group: str) -> None:
await asyncio.sleep(self.album_delay)
album = self._albums.pop(group, None)
if album is None:
return
await self._dispatch(
album.message,
album.thread_id,
is_master=album.is_master,
text="\n\n".join(album.texts),
notes=album.notes,
attachments=album.attachments,
)
async def _dispatch(
self,
message: Message,
thread_id: int | None,
*,
is_master: bool,
text: str,
notes: list[str],
attachments: list[dict[str, Any]],
) -> None:
text = "\n\n".join(part for part in (text, *notes) if part).strip()
text = f"{text}\n\n{attachment}".strip()
if not text:
return
if message.text and message.text.startswith("/"):
@@ -615,16 +394,14 @@ class TelegramFrontend(Frontend):
await self._command(command, args.strip(), message, thread_id)
return
if is_master:
conv, consumed = await self._master(text, attachments or None)
conv, consumed = await self._master(text)
if consumed:
return
else:
topic = cast("int", thread_id)
conv = await self._live(topic)
conv = await self._live(thread_id)
if conv is None:
await self._branch_in_topic(
topic, text=text, attachments=attachments or None
)
title = self._topic_names.get(thread_id) or self._title_from(text)
await self._branch(thread_id, title=title, text=text)
return
pending = self.conversations.pending_question(conv.external_id)
if pending is not None and self.conversations.answer(pending[0], text):
@@ -632,9 +409,7 @@ class TelegramFrontend(Frontend):
pending[0], self.texts.question_typed.format(text=text)
)
return
item = await self.conversations.post(
conv, text, origin=FRONTEND, attachments=attachments or None
)
item = await self.conversations.post(conv, text, origin=FRONTEND)
if conv.running_turn or conv.pending_question:
await self._react(message, item.id)
@@ -678,25 +453,55 @@ class TelegramFrontend(Frontend):
with contextlib.suppress(TelegramAPIError):
await self.bot.set_message_reaction(target[0], target[1], reaction=[])
async def _save_attachment(
self, message: Message
) -> tuple[str | None, dict[str, Any] | None]:
"""The note for the model and, for images, the file it gets to see inline."""
async def _save_attachment(self, message: Message) -> str | None:
file_id: str | None = None
name: str | None = None
kind = ""
size: int | None = None
texts = self.texts
if message.sticker:
return texts.attachment_sticker, None
return texts.attachment_sticker
if message.animation:
return texts.attachment_animation, None
file_id, kind, size, name, media_type = _media(message)
return texts.attachment_animation
if message.photo:
photo = message.photo[-1]
file_id, kind, size = photo.file_id, "photo", photo.file_size
name = f"{photo.file_unique_id}.jpg"
elif message.document:
doc = message.document
file_id, kind, size = doc.file_id, "document", doc.file_size
name = doc.file_name or f"{doc.file_unique_id}.bin"
elif message.voice:
file_id, kind, size = (
message.voice.file_id,
"voice",
message.voice.file_size,
)
name = f"{message.voice.file_unique_id}.ogg"
elif message.audio:
file_id, kind, size = (
message.audio.file_id,
"audio",
message.audio.file_size,
)
name = message.audio.file_name or f"{message.audio.file_unique_id}.mp3"
elif message.video:
file_id, kind, size = (
message.video.file_id,
"video",
message.video.file_size,
)
name = message.video.file_name or f"{message.video.file_unique_id}.mp4"
elif message.video_note:
note = message.video_note
file_id, kind, size = note.file_id, "video_note", note.file_size
name = f"{note.file_unique_id}.mp4"
if file_id is None or name is None:
return None, None
return None
kind = texts.attachment_kinds.get(kind, kind)
if size and size > 20 * 1024 * 1024:
return (
texts.attachment_too_big.format(
kind=kind, name=name, mb=size // 1024 // 1024
),
None,
return texts.attachment_too_big.format(
kind=kind, name=name, mb=size // 1024 // 1024
)
now = time.time()
folder = self.attachments.root / self.attachments.day(now)
@@ -710,20 +515,12 @@ class TelegramFrontend(Frontend):
path.chmod(0o644)
except (TelegramAPIError, OSError) as exc:
_log.warning("attachment download failed: %s", exc)
return texts.attachment_failed.format(kind=kind, name=name, error=exc), None
return texts.attachment_failed.format(kind=kind, name=name, error=exc)
shown = texts.attachment_size.format(kb=size // 1024) if size else ""
keep = self.attachments.keep_days
seen = (
{"path": str(path), "media_type": media_type}
if media_type and media_type.startswith("image/")
else None
)
if keep is None:
return texts.attachment.format(kind=kind, path=path, size=shown), seen
return (
texts.attachment_kept.format(kind=kind, path=path, size=shown, days=keep),
seen,
)
return texts.attachment.format(kind=kind, path=path, size=shown)
return texts.attachment_kept.format(kind=kind, path=path, size=shown, days=keep)
async def _sweep_loop(self, interval: float = 6 * 3600) -> None:
while True:
@@ -878,13 +675,10 @@ class TelegramFrontend(Frontend):
return
match kind:
case "turn.start":
if event.get("origin") in ("user", "task"):
if event.get("origin") == "user":
await self._open_draft(key, event, target)
case "stream":
agents = cast("GatewayRuntime", self._runtime).agents
agent = agents.get(conv.agent_name)
progress = agent is not None and agent.thinking_is_progress
self._feed_draft(key, event, thinking_is_progress=progress)
self._feed_draft(key, event)
case "tool":
draft = self._drafts.get(key)
if draft is not None and event.get("parent_tool_use_id") is None:
@@ -927,14 +721,6 @@ class TelegramFrontend(Frontend):
case "conversation.merged":
await self._deliver(conv, self.texts.merged, key=f"{key}:merged")
async def _on_stopped(self, draft_id: int) -> None:
key = next((k for k, d in self._drafts.items() if d.draft_id == draft_id), None)
if key is None:
return
conv = await self.conversations.get(key)
if conv is not None:
await self.conversations.interrupt(conv)
async def _on_delivery_failed(self, event: Event) -> None:
row = event.get("conversation_row")
_log.error(
@@ -1003,9 +789,7 @@ class TelegramFrontend(Frontend):
self._drafts[key] = draft
draft.start()
def _feed_draft(
self, key: str, event: Event, *, thinking_is_progress: bool
) -> None:
def _feed_draft(self, key: str, event: Event) -> None:
draft = self._drafts.get(key)
if draft is None or event.get("parent_tool_use_id") is not None:
return
@@ -1017,19 +801,11 @@ class TelegramFrontend(Frontend):
draft.set_status(self.texts.writing)
draft.append(str(delta.get("text") or ""))
elif delta.get("type") == "thinking_delta":
progress = str(delta.get("thinking") or "")
if progress and thinking_is_progress:
draft.set_status(self.texts.writing)
draft.append(progress)
else:
draft.set_status(self.texts.thinking)
draft.think(progress)
draft.set_status(self.texts.thinking)
elif kind == "content_block_start":
block = raw.get("content_block") or {}
if block.get("type") == "tool_use":
draft.set_status(self._tool_status(block.get("name") or "", None))
elif block.get("type") == "thinking":
draft.rethink()
def _tool_status(self, name: Any, tool_input: Any) -> str:
label = status_label(str(name), tool_input, self.texts.tool_labels)
@@ -1072,27 +848,6 @@ class TelegramFrontend(Frontend):
_log.exception("question %s could not be sent", question_id)
continue
ask.messages.append(sent.message_id)
await self._persist_ask(question_id, ask)
async def _persist_ask(self, question_id: str, ask: _Ask | None) -> None:
"""The open question lives in the conversation's flags across restarts."""
conv = await self.conversations.get(
ask.conversation_id if ask is not None else question_id
)
if conv is None:
return
await self.conversations.set_flags(
conv, {"ask": ask.to_flags(question_id) if ask is not None else None}
)
async def _restore_ask(self, question_id: str) -> _Ask | None:
for conv in await self.conversations.find(status="open", limit=500):
data = conv.flags.get("ask")
if isinstance(data, dict) and data.get("id") == question_id:
ask = _Ask.from_flags(data)
self._asks[question_id] = ask
return ask
return None
async def _on_callback(self, query: CallbackQuery) -> None:
if query.from_user.id != self.user_id or not query.data:
@@ -1101,7 +856,7 @@ class TelegramFrontend(Frontend):
if len(parts) != 4 or parts[0] != "q":
return
_, question_id, qi_raw, choice = parts
ask = self._asks.get(question_id) or await self._restore_ask(question_id)
ask = self._asks.get(question_id)
if ask is None or not qi_raw.isdigit():
await self._callback_reply(query, self.texts.question_closed)
await self._strip_keyboard(query)
@@ -1141,20 +896,11 @@ class TelegramFrontend(Frontend):
reply_markup=self._keyboard(question_id, qi, question, picked),
)
await self._callback_reply(query, None)
if len(ask.done) < len(ask.questions):
await self._persist_ask(question_id, ask)
return
answer = _answer_text(ask)
self._asks.pop(question_id, None)
conv = await self.conversations.get(ask.conversation_id)
if conv is not None:
await self.conversations.set_flags(conv, {"ask": None})
if self.conversations.answer(question_id, answer):
return
if conv is not None and conv.status == "open":
await self.conversations.post(conv, answer, origin=FRONTEND)
else:
await self._edit_asks(ask, self.texts.question_timeout)
if len(ask.done) == len(ask.questions):
answer = _answer_text(ask)
self._asks.pop(question_id, None)
if not self.conversations.answer(question_id, answer):
await self._edit_asks(ask, self.texts.question_timeout)
async def _strip_keyboard(self, query: CallbackQuery) -> None:
message = query.message if isinstance(query.message, Message) else None
@@ -1174,7 +920,6 @@ class TelegramFrontend(Frontend):
async def _close_ask(self, question_id: str, note: str) -> None:
ask = self._asks.pop(question_id, None)
if ask is not None:
await self._persist_ask(question_id, None)
await self._edit_asks(ask, note)
async def _edit_asks(self, ask: _Ask, note: str) -> None:
@@ -1190,50 +935,6 @@ class TelegramFrontend(Frontend):
)
def _media(
message: Message,
) -> tuple[str | None, str, int | None, str | None, str | None]:
"""``(file_id, kind, size, name, media_type)`` of the message's media, if any."""
if message.photo:
photo = message.photo[-1]
return (
photo.file_id,
"photo",
photo.file_size,
f"{photo.file_unique_id}.jpg",
"image/jpeg",
)
if message.document:
doc = message.document
name = doc.file_name or f"{doc.file_unique_id}.bin"
return doc.file_id, "document", doc.file_size, name, doc.mime_type
if message.voice:
v = message.voice
return v.file_id, "voice", v.file_size, f"{v.file_unique_id}.ogg", None
if message.audio:
a = message.audio
return (
a.file_id,
"audio",
a.file_size,
a.file_name or f"{a.file_unique_id}.mp3",
None,
)
if message.video:
v = message.video
return (
v.file_id,
"video",
v.file_size,
v.file_name or f"{v.file_unique_id}.mp4",
None,
)
if message.video_note:
n = message.video_note
return n.file_id, "video_note", n.file_size, f"{n.file_unique_id}.mp4", None
return None, "", None, None, None
def _question_html(question: dict[str, Any]) -> str:
header = html.escape(str(question.get("header") or "").strip())
body = html.escape(str(question.get("question") or "").strip())
@@ -31,7 +31,7 @@ __all__ = ["Inbox"]
_log = logging.getLogger("beaver_gateway.frontends.telegram.inbox")
ALLOWED_UPDATES = ("message", "callback_query", "stopped_message_generation")
ALLOWED_UPDATES = ("message", "callback_query")
def _unset(_value: object) -> None:
@@ -26,7 +26,6 @@ TOOL_LABELS: dict[str, str] = {
"mcp__gateway__say": "speaking",
"mcp__gateway__schedule": "setting a reminder",
"mcp__gateway__inject": "passing to another conversation",
"mcp__gateway__send_file": "sending a file",
}
@@ -42,7 +41,6 @@ class TelegramTexts:
help_head: str = "General is the master, any other topic is a branch."
branch: str = "branch"
closed_prefix: str = "✅ "
idle_prefix: str = "💤 "
waiting: str = "⏳ thinking"
thinking: str = "🤔 thinking"
writing: str = "✍️ writing"
@@ -46,7 +46,6 @@ class TurnRecord:
agent_name: str
input_messages: list[MessageParam]
output_message: Message
thinking_is_progress: bool
system: str | None = None
source: str = "anthropic"
+3 -35
View File
@@ -5,7 +5,7 @@ from __future__ import annotations
import logging
from dataclasses import dataclass
from datetime import datetime, timedelta
from typing import TYPE_CHECKING, Any, cast
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from collections.abc import Awaitable, Callable, Coroutine
@@ -73,23 +73,6 @@ class JobRun:
)
return True
async def inject_into(
self,
public_id: str | None,
text: str,
*,
urgency: Priority = "normal",
origin: str | None = None,
) -> bool:
"""Inject into that conversation, or the master once it is gone or over."""
conv = await self.conversations.get(public_id) if public_id else None
if conv is None or conv.status != "open":
return await self.inject_master(text, urgency=urgency, origin=origin)
await self.conversations.inject(
conv, text, urgency=urgency, origin=origin or self.job.name
)
return True
async def spawn_job(
self,
*,
@@ -130,29 +113,14 @@ class JobRun:
_log.exception("closing idle %s failed", conv.external_id)
return out
async def close_quiet(
self, *, kind: str = "branch", days: int = 3, limit: int | None = None
) -> list[Conversation]:
"""Close chats quiet for ``days`` without a digest, their windows marked."""
out: list[Conversation] = []
for conv in await self.conversations.idle(kind=kind, days=days, limit=limit):
if await self.conversations.busy(
conv
) or await self.conversations.queue.pending(cast("int", conv.id)):
continue
row = await self.conversations.close(conv)
await self.conversations.mark_closed(row)
out.append(row)
return out
async def retry_in(self, delay: timedelta) -> None:
await self.scheduler.trigger(
self.job, self.payload, delay=delay, trigger=self.trigger
)
async def rotate(self, *, force: str | None = None) -> list[Conversation]:
async def rotate(self) -> list[Conversation]:
rotation = self.scheduler.rotation
return await rotation.tick(force=force) if rotation is not None else []
return await rotation.tick() if rotation is not None else []
def background(self, coro: Coroutine[Any, Any, Any]) -> None:
self.scheduler.background(coro)
-19
View File
@@ -1,19 +0,0 @@
"""Secrets the model runs commands with: sealed in the database, handed out to root."""
from beaver_gateway.keystore.server import serve
from beaver_gateway.keystore.store import (
NAME,
SecretBox,
SecretsDisabledError,
SecretStore,
UnknownSecretError,
)
__all__ = [
"NAME",
"SecretBox",
"SecretStore",
"SecretsDisabledError",
"UnknownSecretError",
"serve",
]
-289
View File
@@ -1,289 +0,0 @@
"""``secret ls`` and ``secret run NAME... -- command``: stdlib only, run by path.
The model's uid calls this file; it asks the root half (the same file
through ``sudo``) for the values, which reads them from the gateway's
root-only socket. The command then runs as the model's uid with each
value in its own variable, files in a private tmpfs directory, and the
values masked in what it prints.
"""
from __future__ import annotations
import base64
import contextlib
import json
import os
import re
import shutil
import signal
import socket
import subprocess
import sys
import tempfile
import threading
from pathlib import Path
from typing import IO, TYPE_CHECKING, Any
from urllib.parse import quote
if TYPE_CHECKING:
from io import BufferedReader
SOCKET = Path("/run/beaver-secrets/gateway.sock")
ROOT_HALF = ("sudo", "-n", "/usr/local/libexec/beaver-secret")
SHM = Path("/dev/shm") # noqa: S108
PREFIX = "beaver-secret-"
NAME = re.compile(r"[A-Z][A-Z0-9_]{0,63}")
MIN_MASKED = 6
HOLD = 1 << 16
CHUNK = 1 << 16
DRAIN = 2.0
USAGE = """usage:
secret ls names and descriptions
secret run NAME [NAME...] -- CMD run CMD with each secret in $NAME
A value secret arrives as its text; a file secret as the path to a
private copy that is removed when CMD exits. Values are masked in
CMD's output as <secret:NAME>.
"""
class SecretError(Exception):
pass
def request(payload: dict[str, Any], path: Path = SOCKET) -> dict[str, Any]:
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as conn:
try:
conn.connect(str(path))
except OSError as exc:
msg = f"the gateway's secrets socket is not reachable ({exc.strerror})"
raise SecretError(msg) from exc
conn.sendall(json.dumps(payload).encode() + b"\n")
chunks = []
while chunk := conn.recv(CHUNK):
chunks.append(chunk)
reply = json.loads(b"".join(chunks) or b"{}")
if not reply.get("ok"):
raise SecretError(reply.get("error") or "the gateway gave no answer")
return reply
def root_half(argv: list[str]) -> int:
if argv[:1] == ["ls"] and len(argv) == 1:
payload: dict[str, Any] = {"op": "ls"}
elif (
argv[:1] == ["get"]
and len(argv) > 1
and all(NAME.fullmatch(a) for a in argv[1:])
):
payload = {"op": "get", "names": argv[1:]}
else:
print("beaver-secret: ls | get NAME...", file=sys.stderr)
return 2
try:
reply = request(payload)
except SecretError as exc:
reply = {"ok": False, "error": str(exc)}
sys.stdout.write(json.dumps(reply))
return 0 if reply.get("ok") else 1
def ask(argv: list[str]) -> dict[str, Any]:
if os.geteuid() == 0:
return request(
{"op": "ls"} if argv == ["ls"] else {"op": "get", "names": argv[1:]}
)
proc = subprocess.run( # noqa: S603
[*ROOT_HALF, *argv], capture_output=True, check=False
)
try:
reply = json.loads(proc.stdout)
except json.JSONDecodeError:
err = proc.stderr.decode(errors="replace").strip()
msg = f"the root helper failed: {err or f'exit {proc.returncode}'}"
raise SecretError(msg) from None
if not reply.get("ok"):
raise SecretError(reply.get("error") or "the root helper gave no answer")
return reply
def list_secrets() -> int:
rows = ask(["ls"])["secrets"]
if not rows:
print("no secrets yet; a human adds them in the admin, System -> Secrets")
return 0
width = max(len(r["name"]) for r in rows)
for r in rows:
kind = f"file {r['filename']}" if r.get("filename") else "value"
line = f"{r['name']:<{width}} {kind}"
if r.get("description"):
line += f" - {r['description']}"
print(line)
return 0
class Masker:
"""Streams bytes through, replacing every needle; never splits one."""
def __init__(self, needles: dict[bytes, bytes]) -> None:
self._needles = needles
self._longest = max((len(n) for n in needles), default=0)
ordered = sorted(needles, key=lambda n: -len(n))
self._pattern = (
re.compile(b"|".join(re.escape(n) for n in ordered)) if ordered else None
)
self._pending = b""
def feed(self, data: bytes) -> bytes:
if self._pattern is None:
return data
self._pending += data
cut = self._pending.rfind(b"\n") + 1
if len(self._pending) - cut > HOLD:
cut = max(cut, len(self._pending) - self._longest + 1)
for match in self._pattern.finditer(self._pending):
if match.start() < cut < match.end():
cut = match.end()
head, self._pending = self._pending[:cut], self._pending[cut:]
return self._mask(head)
def flush(self) -> bytes:
tail, self._pending = self._pending, b""
return self._mask(tail)
def _mask(self, data: bytes) -> bytes:
if self._pattern is None or not data:
return data
return self._pattern.sub(lambda m: self._needles[m[0]], data)
def needles_for(values: dict[str, bytes]) -> dict[bytes, bytes]:
out: dict[bytes, bytes] = {}
for name, data in values.items():
mask = f"<secret:{name}>".encode()
try:
text = data.decode()
except UnicodeDecodeError:
continue
for part in text.splitlines() or [text]:
for form in {part.strip(), quote(part.strip(), safe="")}:
if len(form) >= MIN_MASKED:
out[form.encode()] = mask
return out
def sweep(root: Path) -> None:
for stale in root.glob(f"{PREFIX}*"):
pid = stale.name.removeprefix(PREFIX).split("-", 1)[0]
if not pid.isdigit() or _alive(int(pid)):
continue
shutil.rmtree(stale, ignore_errors=True)
def _alive(pid: int) -> bool:
try:
os.kill(pid, 0)
except ProcessLookupError:
return False
except PermissionError:
return True
return True
def _pump(src: BufferedReader, dst: IO[bytes], masker: Masker) -> None:
while chunk := src.read1(CHUNK):
dst.write(masker.feed(chunk))
dst.flush()
dst.write(masker.flush())
dst.flush()
def run(names: list[str], command: list[str], *, root: Path | None = None) -> int:
secrets = ask(["get", *names])["secrets"]
base = root or (SHM if SHM.is_dir() else Path(tempfile.gettempdir()))
sweep(base)
workdir = Path(tempfile.mkdtemp(prefix=f"{PREFIX}{os.getpid()}-", dir=base))
env = dict(os.environ)
values: dict[str, bytes] = {}
try:
for name in names:
item = secrets[name]
data = base64.b64decode(item["data"])
if item.get("filename"):
path = workdir / name / item["filename"]
path.parent.mkdir(mode=0o700)
fd = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
with os.fdopen(fd, "wb") as fh:
fh.write(data)
env[name] = str(path)
else:
env[name] = data.decode(errors="surrogateescape")
values[name] = data
return _spawn(command, env, needles_for(values))
finally:
shutil.rmtree(workdir, ignore_errors=True)
def _spawn(command: list[str], env: dict[str, str], needles: dict[bytes, bytes]) -> int:
try:
child = subprocess.Popen( # noqa: S603
command, env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE
)
except (OSError, ValueError) as exc:
reason = exc.strerror if isinstance(exc, OSError) else str(exc)
print(f"secret: cannot run {command[0]}: {reason}", file=sys.stderr)
return 127
forwarded = (
(signal.SIGINT, signal.SIGTERM, signal.SIGHUP)
if threading.current_thread() is threading.main_thread()
else ()
)
previous = {
sig: signal.signal(sig, lambda s, _f: child.send_signal(s)) for sig in forwarded
}
pumps = [
threading.Thread(target=_pump, args=(src, dst, Masker(needles)), daemon=True)
for src, dst in (
(child.stdout, sys.stdout.buffer),
(child.stderr, sys.stderr.buffer),
)
]
for pump in pumps:
pump.start()
try:
code = child.wait()
for pump in pumps:
pump.join(DRAIN)
finally:
for sig, handler in previous.items():
signal.signal(sig, handler)
return 128 - code if code < 0 else code
def main(argv: list[str] | None = None) -> int:
args = sys.argv[1:] if argv is None else argv
if args[:1] == ["--root"]:
return root_half(args[1:])
try:
if args == ["ls"]:
return list_secrets()
if args[:1] == ["run"] and "--" in args:
split = args.index("--")
names, command = args[1:split], args[split + 1 :]
bad = [n for n in names if not NAME.fullmatch(n)]
if names and command and not bad:
return run(names, command)
if bad:
print(f"secret: invalid name {bad[0]!r}", file=sys.stderr)
return 2
except SecretError as exc:
print(f"secret: {exc}", file=sys.stderr)
return 1
print(USAGE, end="", file=sys.stderr)
return 2
if __name__ == "__main__":
with contextlib.suppress(KeyboardInterrupt):
sys.exit(main())
-108
View File
@@ -1,108 +0,0 @@
"""Root-only unix socket that hands sealed secrets to the ``secret`` helper."""
from __future__ import annotations
import asyncio
import base64
import contextlib
import json
import logging
from typing import TYPE_CHECKING, Any
from beaver_gateway.keystore.store import (
SecretsDisabledError,
UnknownSecretError,
check_name,
)
if TYPE_CHECKING:
from collections.abc import Awaitable, Callable, Sequence
from pathlib import Path
from beaver_gateway.keystore.store import SecretStore
_log = logging.getLogger("beaver_gateway.keystore")
REQUEST_LIMIT = 1 << 16
async def serve(
store: SecretStore,
path: Path,
*,
on_use: Callable[[Sequence[str]], Awaitable[None]] | None = None,
) -> None:
"""Listen on ``path`` (mode 0600 in a 0700 directory) until cancelled."""
_clear(path)
async def handle(
reader: asyncio.StreamReader, writer: asyncio.StreamWriter
) -> None:
try:
line = await reader.readline()
reply = await answer(store, line, on_use=on_use)
writer.write(json.dumps(reply).encode() + b"\n")
await writer.drain()
except Exception: # noqa: BLE001
_log.exception("secret request failed")
finally:
writer.close()
with contextlib.suppress(Exception):
await writer.wait_closed()
server = await asyncio.start_unix_server(
handle, path=str(path), limit=REQUEST_LIMIT
)
_lock(path)
_log.info("secrets socket on %s", path)
async with server:
await server.serve_forever()
async def answer(
store: SecretStore,
line: bytes,
*,
on_use: Callable[[Sequence[str]], Awaitable[None]] | None = None,
) -> dict[str, Any]:
try:
request = json.loads(line)
op = request.get("op") if isinstance(request, dict) else None
if op == "ls":
return {"ok": True, "secrets": [_listed(r) for r in await store.rows()]}
if op == "get":
names = [check_name(str(n)) for n in request.get("names") or ()]
if not names:
return {"ok": False, "error": "no secret names given"}
revealed = await store.reveal(names)
if on_use is not None:
await on_use(names)
return {
"ok": True,
"secrets": {
r.name: {
"data": base64.b64encode(r.data).decode(),
"filename": r.filename,
}
for r in revealed
},
}
except UnknownSecretError as exc:
return {"ok": False, "error": f"unknown secret: {exc}"}
except (SecretsDisabledError, ValueError) as exc:
return {"ok": False, "error": str(exc)}
return {"ok": False, "error": "op must be `ls` or `get`"}
def _clear(path: Path) -> None:
path.parent.mkdir(mode=0o700, parents=True, exist_ok=True)
path.parent.chmod(0o700)
path.unlink(missing_ok=True)
def _lock(path: Path) -> None:
path.chmod(0o600)
def _listed(row: Any) -> dict[str, Any]:
return {"name": row.name, "description": row.description, "filename": row.filename}
-193
View File
@@ -1,193 +0,0 @@
"""Seal, list, replace and reveal secrets; the key never leaves this module."""
from __future__ import annotations
import os
import re
from dataclasses import dataclass
from datetime import UTC, datetime
from typing import TYPE_CHECKING
from cryptography.exceptions import InvalidTag
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
from sqlmodel import col, select
from beaver_gateway.storage.models import Secret
if TYPE_CHECKING:
from collections.abc import Iterable, Sequence
from beaver_gateway.storage import Database
NAME = re.compile(r"[A-Z][A-Z0-9_]{0,63}")
"""A secret's name is the environment variable a command receives it under."""
KEY_BYTES = 32
NONCE_BYTES = 12
MAX_SIZE = 1 << 20
"""Largest value or file the store accepts, in bytes."""
class SecretsDisabledError(RuntimeError):
"""``SECRETS_KEY`` is unset or unusable, so nothing can be sealed or opened."""
class UnknownSecretError(LookupError):
"""A name that is not in the store."""
class SecretBox:
"""AES-256-GCM with the secret's name as associated data."""
__slots__ = ("_aead",)
def __init__(self, key: bytes) -> None:
if len(key) != KEY_BYTES:
msg = f"SECRETS_KEY must be {KEY_BYTES} bytes, got {len(key)}"
raise ValueError(msg)
self._aead = AESGCM(key)
@classmethod
def from_hex(cls, text: str) -> SecretBox:
try:
key = bytes.fromhex(text.strip())
except ValueError as exc:
msg = "SECRETS_KEY must be hex, e.g. `openssl rand -hex 32`"
raise ValueError(msg) from exc
return cls(key)
def seal(self, name: str, data: bytes) -> bytes:
nonce = os.urandom(NONCE_BYTES)
return nonce + self._aead.encrypt(nonce, data, name.encode())
def open(self, name: str, sealed: bytes) -> bytes:
nonce, body = sealed[:NONCE_BYTES], sealed[NONCE_BYTES:]
return self._aead.decrypt(nonce, body, name.encode())
@dataclass(frozen=True, slots=True)
class Revealed:
name: str
data: bytes
filename: str | None
def _now() -> datetime:
return datetime.now(UTC)
def check_name(name: str) -> str:
if not NAME.fullmatch(name):
msg = (
f"invalid secret name {name!r}: use A-Z, 0-9 and _, starting with a letter"
)
raise ValueError(msg)
return name
def clean_filename(filename: str) -> str:
base = os.path.basename(filename.replace("\\", "/")).strip() # noqa: PTH119
if base in ("", ".", ".."):
msg = f"invalid file name {filename!r}"
raise ValueError(msg)
return base
class SecretStore:
"""Rows of :class:`Secret`; ``box`` is ``None`` when no key is configured."""
__slots__ = ("_box", "_db")
def __init__(self, db: Database, box: SecretBox | None) -> None:
self._db = db
self._box = box
@property
def enabled(self) -> bool:
return self._box is not None
def _require_box(self) -> SecretBox:
if self._box is None:
msg = "SECRETS_KEY is not set"
raise SecretsDisabledError(msg)
return self._box
async def rows(self) -> Sequence[Secret]:
async with self._db.session() as session:
result = await session.exec(select(Secret).order_by(col(Secret.name)))
return result.all()
async def put(
self,
name: str,
*,
description: str | None = None,
data: bytes | None = None,
filename: str | None = None,
) -> Secret:
"""Create or replace; ``data=None`` keeps the value, sets the description."""
check_name(name)
if data is not None:
if len(data) > MAX_SIZE:
msg = f"secret {name} is {len(data)} bytes, the limit is {MAX_SIZE}"
raise ValueError(msg)
sealed = self._require_box().seal(name, data)
async with self._db.session() as session:
row = await session.get(Secret, name)
if row is None:
if data is None:
msg = f"secret {name} needs a value or a file"
raise ValueError(msg)
row = Secret(name=name, sealed=sealed)
if data is not None:
row.sealed = sealed
row.size = len(data)
row.filename = clean_filename(filename) if filename else None
if description is not None:
row.description = description.strip()
row.updated_at = _now()
session.add(row)
await session.commit()
await session.refresh(row)
return row
async def delete(self, name: str) -> bool:
async with self._db.session() as session:
row = await session.get(Secret, name)
if row is None:
return False
await session.delete(row)
await session.commit()
return True
async def reveal(self, names: Iterable[str]) -> list[Revealed]:
box = self._require_box()
wanted = list(dict.fromkeys(names))
async with self._db.session() as session:
rows = {
r.name: r
for r in (
await session.exec(
select(Secret).where(col(Secret.name).in_(wanted))
)
).all()
}
missing = [n for n in wanted if n not in rows]
if missing:
raise UnknownSecretError(", ".join(missing))
out = [_open(box, rows[n]) for n in wanted]
now = _now()
for row in rows.values():
row.used_at = now
session.add(row)
await session.commit()
return out
def _open(box: SecretBox, row: Secret) -> Revealed:
try:
data = box.open(row.name, row.sealed)
except InvalidTag as exc:
msg = f"secret {row.name} does not open: SECRETS_KEY changed since it was set"
raise SecretsDisabledError(msg) from exc
return Revealed(name=row.name, data=data, filename=row.filename)
-3
View File
@@ -28,9 +28,6 @@ KNOWN_KINDS: frozenset[str] = frozenset(
"logout",
"token_create",
"token_revoke",
"secret_set",
"secret_delete",
"secret_use",
}
)
"""Kinds the gateway currently emits; ``AuditLog.kind`` stays free-form."""
+7 -16
View File
@@ -21,7 +21,7 @@ from argon2.exceptions import InvalidHashError, VerifyMismatchError
from beaver_gateway.storage import list_active_tokens, touch_token
if TYPE_CHECKING:
from collections.abc import Iterable, Mapping
from collections.abc import Mapping
from beaver_gateway.storage import Database
@@ -35,8 +35,8 @@ class TokenStoreError(ValueError):
"""Malformed ``BOOTSTRAP_TOKENS`` value or duplicate token."""
BUILTIN_SCOPES: frozenset[str] = frozenset({"*", "messages", "mcp", "admin", "api"})
"""The scopes the gateway's own frontends define.
VALID_SCOPES: frozenset[str] = frozenset({"*", "messages", "mcp", "admin", "api"})
"""The scopes a ``Token.scope`` may hold.
* ``*`` — wildcard, may use any frontend
* ``messages`` — Anthropic Messages frontend only
@@ -45,24 +45,16 @@ BUILTIN_SCOPES: frozenset[str] = frozenset({"*", "messages", "mcp", "admin", "ap
* ``admin`` — reserved for programmatic admin access; the AdminFrontend
itself authenticates via session cookies, not bearer tokens, so this
scope is unused today.
A setup's own frontend adds its scope on top; see :func:`scopes_with` and
``GatewayRuntime.scopes``, which is what ``POST /api/tokens`` validates.
"""
def scopes_with(extra: Iterable[str | None]) -> frozenset[str]:
"""The builtin scopes plus every frontend's own; ``None`` entries drop out."""
return BUILTIN_SCOPES | {s for s in extra if s}
@dataclass(frozen=True, slots=True)
class TokenIdentity:
"""What :meth:`TokenStore.verify` resolves to on success.
``token_id`` is the DB row id for persisted tokens, or ``None`` for
an env-bootstrap match (those have no DB row to touch). ``scope``
gates which frontend the token may hit (see :data:`BUILTIN_SCOPES`);
gates which frontend the token may hit (see :data:`VALID_SCOPES`);
bootstrap tokens implicitly get ``"*"``.
"""
@@ -75,8 +67,8 @@ class TokenIdentity:
``"*"`` is the wildcard; an exact match satisfies a single scope.
Unknown ``required`` values intentionally fall through to a
strict equality check — callers should pass a scope the runtime
knows (:data:`BUILTIN_SCOPES` plus every frontend's own).
strict equality check — callers should pass one of
:data:`VALID_SCOPES`.
"""
return self.scope in ("*", required)
@@ -333,12 +325,11 @@ class TokenStore:
__all__ = [
"BUILTIN_SCOPES",
"VALID_SCOPES",
"TokenIdentity",
"TokenStore",
"TokenStoreError",
"hash_token",
"scopes_with",
]
-11
View File
@@ -57,17 +57,6 @@ class Settings(BaseSettings):
"""``Accept-Language`` sent to the Raycast API and the default locale for
auto ``UserPreferences``. One value per gateway (one shared Client)."""
send_file_roots: str = "/mnt/hole/shared"
"""Directories the ``send_file`` tool may read, separated by ``:``. A
frontend always adds the inbox its own attachments land in."""
secrets_key: str | None = None
"""Hex AES-256 key (``openssl rand -hex 32``) sealing the secrets table;
unset leaves secrets off. Never reaches the model's environment."""
secrets_socket: Path = Path("/run/beaver-secrets/gateway.sock")
"""Root-only socket the ``secret`` helper reads values from."""
bootstrap_tokens: str = ""
"""Out-of-band token seed: ``name1:value1,name2:value2``. Layers
alongside DB-issued tokens; used for first-run setup and recovery."""
+1 -26
View File
@@ -10,7 +10,7 @@ from __future__ import annotations
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import JSON, Column, Index, LargeBinary, UniqueConstraint, text
from sqlalchemy import JSON, Column, Index, UniqueConstraint, text
from sqlalchemy.dialects.postgresql import JSONB
from sqlmodel import Field, SQLModel
@@ -144,11 +144,6 @@ class InjectQueueItem(SQLModel, table=True):
priority: str = Field(index=True)
origin: str = Field(default="system")
text: str
attachments: list[dict[str, Any]] | None = Field(
default=None,
sa_column=Column(JSON().with_variant(JSONB(), "postgresql"), nullable=True),
)
"""Files that ride with a user message: ``{"path", "media_type"}`` each."""
status: str = Field(default="queued", index=True)
turn_id: str | None = Field(default=None)
interrupted_turn: bool | None = Field(default=None)
@@ -374,25 +369,6 @@ class JobRunRecord(SQLModel, table=True):
)
class Secret(SQLModel, table=True):
"""A value or file the model may run a command with, sealed by ``SECRETS_KEY``.
``sealed`` is nonce + AES-GCM ciphertext bound to ``name``; nothing
here decrypts without the key, so a dump of the table stays closed.
"""
__tablename__ = "secrets"
name: str = Field(primary_key=True)
description: str = Field(default="")
filename: str | None = Field(default=None)
size: int = Field(default=0)
sealed: bytes = Field(sa_column=Column(LargeBinary, nullable=False))
created_at: datetime = Field(default_factory=_utcnow)
updated_at: datetime = Field(default_factory=_utcnow)
used_at: datetime | None = Field(default=None)
__all__ = [
"AuditLog",
"Conversation",
@@ -402,7 +378,6 @@ __all__ = [
"InjectQueueItem",
"JobRunRecord",
"RateLimit",
"Secret",
"TelegramUpdate",
"Token",
"TranscriptEntry",
-136
View File
@@ -1,136 +0,0 @@
"""Wikilink index over a vault: which note links to which, refreshed by mtime."""
from __future__ import annotations
import os
import re
import time
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
__all__ = ["LinkIndex"]
_WIKILINK = re.compile(r"\[\[([^\]|#\n]+)(?:#[^\]|\n]*)?(?:\|[^\]\n]*)?\]\]")
_MD = ".md"
@dataclass(slots=True)
class _Note:
mtime: float
links: set[str] = field(default_factory=set)
class LinkIndex:
"""Forward and backward links between the vault's markdown notes."""
def __init__(self, root: Path, *, max_age: float = 20.0) -> None:
self.root = root.resolve()
self.max_age = max_age
self._notes: dict[str, _Note] = {}
self._raw: dict[str, set[str]] = {}
self._scanned_at = 0.0
def refresh(self, *, force: bool = False) -> None:
if not force and time.monotonic() - self._scanned_at < self.max_age:
return
seen: set[str] = set()
for dirpath, dirnames, filenames in os.walk(self.root):
dirnames[:] = [d for d in dirnames if not d.startswith(".")]
for name in filenames:
if not name.endswith(_MD):
continue
path = Path(dirpath) / name
rel = str(path.relative_to(self.root))
seen.add(rel)
try:
mtime = path.stat().st_mtime
except OSError:
continue
note = self._notes.get(rel)
if note is None or note.mtime != mtime:
self._notes[rel] = _Note(mtime=mtime)
self._raw[rel] = self._targets(path)
for rel in list(self._notes):
if rel not in seen:
self._notes.pop(rel, None)
self._raw.pop(rel, None)
self._resolve()
self._scanned_at = time.monotonic()
@staticmethod
def _targets(path: Path) -> set[str]:
try:
text = path.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
return set()
return {m.group(1).strip() for m in _WIKILINK.finditer(text)}
def _resolve(self) -> None:
by_stem: dict[str, str] = {}
for rel in sorted(self._notes):
by_stem.setdefault(Path(rel).stem.casefold(), rel)
by_path = {rel.casefold(): rel for rel in self._notes}
for rel, targets in self._raw.items():
links: set[str] = set()
for target in targets:
candidate = target if target.endswith(_MD) else f"{target}{_MD}"
found = by_path.get(candidate.casefold()) or by_stem.get(
Path(target).stem.casefold()
)
if found and found != rel:
links.add(found)
self._notes[rel].links = links
def resolve(self, raw: str) -> str:
"""A path as a tool named it (absolute or relative) as an index key."""
path = Path(raw)
if path.is_absolute():
try:
return str(path.resolve().relative_to(self.root))
except (ValueError, OSError):
return raw
return raw
def neighbours(self, paths: list[str], *, limit: int = 80) -> dict[str, Any]:
"""Nodes for ``paths`` plus every note one link away, and their edges."""
self.refresh()
touched = [self.resolve(p) for p in paths]
backlinks: dict[str, set[str]] = {}
for rel, note in self._notes.items():
for target in note.links:
backlinks.setdefault(target, set()).add(rel)
nodes: dict[str, dict[str, Any]] = {}
for rel in touched:
nodes[rel] = {
"path": rel,
"title": Path(rel).stem,
"touched": True,
"exists": rel in self._notes,
}
for rel in touched:
note = self._notes.get(rel)
around = (note.links if note else set()) | backlinks.get(rel, set())
for other in sorted(around):
if len(nodes) >= limit:
break
nodes.setdefault(
other,
{
"path": other,
"title": Path(other).stem,
"touched": False,
"exists": True,
},
)
edges: list[dict[str, str]] = []
for rel in nodes:
note = self._notes.get(rel)
if note is None:
continue
edges.extend(
{"from": rel, "to": target}
for target in sorted(note.links)
if target in nodes
)
return {"nodes": list(nodes.values()), "edges": edges}
+5 -68
View File
@@ -13,26 +13,21 @@ import difflib
import fnmatch
import logging
import os
import re
from dataclasses import dataclass, field
from datetime import UTC, datetime
from pathlib import Path
from typing import TYPE_CHECKING
from zoneinfo import ZoneInfo
from watchfiles import DefaultFilter, awatch
from watchfiles import awatch
if TYPE_CHECKING:
from collections.abc import Iterable
from watchfiles import Change as FileChange
__all__ = ["Change", "VaultFilter", "VaultWatch", "WatchRules"]
__all__ = ["Change", "VaultWatch", "WatchRules"]
_log = logging.getLogger("beaver_gateway.vault.watch")
_FENCE = re.compile(r"^\s*(`{3,}|~{3,})\s*(.*?)\s*$")
@dataclass(frozen=True, slots=True)
class WatchRules:
@@ -47,8 +42,6 @@ class WatchRules:
names: tuple[str, ...] = ()
ignore: tuple[str, ...] = ()
suffixes: tuple[str, ...] = (".md",)
skip_fences: tuple[str, ...] = ("tasks", "dataview", "dataviewjs")
"""Fenced blocks of these languages are queries: ``full`` files drop them."""
def kind(self, rel: str, *, today: str) -> str | None:
"""The most specific matching pattern wins, whichever list it is in."""
@@ -81,26 +74,6 @@ class _Seen:
count: int
IGNORE_DIRS = (".obsidian", ".trash", ".git")
"""Housekeeping directories whose churn never means an edit of the vault."""
IGNORE_NAMES = (".sync.lock", "*.lock", "*.tmp", "*~")
"""Lock and scratch file names, matched against the entity name."""
class VaultFilter(DefaultFilter):
"""Drops housekeeping paths inside ``awatch``, before the loop is woken."""
def __init__(self) -> None:
super().__init__(ignore_dirs=(*DefaultFilter.ignore_dirs, *IGNORE_DIRS))
def __call__(self, change: FileChange, path: str) -> bool:
name = Path(path).name
if any(fnmatch.fnmatchcase(name, pattern) for pattern in IGNORE_NAMES):
return False
return super().__call__(change, path)
class VaultWatch:
def __init__(
self, root: Path, rules: WatchRules, *, tz: str = "UTC", debounce: float = 4.0
@@ -125,7 +98,6 @@ class VaultWatch:
_log.info("watching %s (%d files in snapshot)", self.root, len(self._seen))
async for changes in awatch(
self.root,
watch_filter=VaultFilter(),
debounce=int(self._debounce * 1000),
stop_event=self._stop,
ignore_permission_denied=True,
@@ -145,7 +117,7 @@ class VaultWatch:
if kind is None:
continue
with contextlib.suppress(OSError):
lines = self._lines(path, full=kind == "full")
lines = _read_lines(path)
self._seen[rel] = _Seen(
lines=lines if kind == "full" else None, count=len(lines)
)
@@ -163,7 +135,7 @@ class VaultWatch:
self._pending.pop(rel, None)
return None
stat = path.stat()
lines = self._lines(path, full=kind == "full")
lines = _read_lines(path)
seen = self._seen.get(rel)
change = Change(
path=rel,
@@ -191,16 +163,12 @@ class VaultWatch:
for change in changes:
path = self.root / change.path
with contextlib.suppress(OSError):
lines = self._lines(path, full=change.full)
lines = _read_lines(path)
self._seen[change.path] = _Seen(
lines=lines if change.full else None, count=len(lines)
)
return changes
def _lines(self, path: Path, *, full: bool) -> list[str]:
lines = _read_lines(path)
return _prose(lines, self.rules.skip_fences) if full else lines
def _rel(self, path: Path) -> str:
return path.resolve().relative_to(self.root.resolve()).as_posix()
@@ -225,37 +193,6 @@ def _read_lines(path: Path) -> list[str]:
return path.read_text(encoding="utf-8", errors="replace").splitlines()
def _prose(lines: list[str], skip_fences: tuple[str, ...]) -> list[str]:
start = 0
if lines and lines[0].strip() == "---":
closing = next(
(i for i, line in enumerate(lines[1:], 1) if line.strip() == "---"), None
)
start = closing + 1 if closing is not None else 0
out: list[str] = []
fence: str | None = None
skipping = False
for line in lines[start:]:
match = _FENCE.match(line)
if fence is None and match:
fence = match.group(1)
language = match.group(2).split(maxsplit=1)[0] if match.group(2) else ""
skipping = language.lower() in skip_fences
elif fence is not None and match and _closes(match, fence):
fence = None
if skipping:
skipping = False
continue
if not skipping and line.strip():
out.append(line)
return out
def _closes(match: re.Match[str], fence: str) -> bool:
mark = match.group(1)
return mark[0] == fence[0] and len(mark) >= len(fence) and not match.group(2)
def _added(old: list[str], new: list[str]) -> list[str]:
matcher = difflib.SequenceMatcher(a=old, b=new, autojunk=False)
added: list[str] = []
+5 -208
View File
@@ -16,7 +16,7 @@ from claude_agent_sdk import (
UserMessage,
)
from httpx import ASGITransport, AsyncClient
from test_conversations import ScriptedClient, World, _three_turns
from test_conversations import ScriptedClient, World
from beaver_gateway.frontends.markdown.history import rewrite_messages
from beaver_gateway.security.auth import TokenStore
@@ -29,7 +29,7 @@ from beaver_gateway.frontends.api import ApiFrontend
from beaver_gateway.frontends.api.frontend import build_app as build_api
from beaver_gateway.frontends.base import Frontend, GatewayRuntime
from beaver_gateway.frontends.root import build_root_app
from beaver_gateway.storage.models import RateLimit, TranscriptEntry, Usage
from beaver_gateway.storage.models import RateLimit, Usage
TOKEN = "tok"
HEADERS = {"Authorization": f"Bearer {TOKEN}"}
@@ -247,7 +247,6 @@ async def test_usage_groups_by_agent_day_and_model(world: World) -> None:
async def test_limits_report_latest_window_with_gateway_spend(world: World) -> None:
api = Api(world)
now = datetime.now(UTC)
resets = int((now + timedelta(hours=4)).timestamp())
async with world.db.session() as session:
session.add(
RateLimit(
@@ -265,20 +264,7 @@ async def test_limits_report_latest_window_with_gateway_spend(world: World) -> N
resets_at=now + timedelta(hours=4),
)
)
session.add(
RateLimit(
window="seven_day",
status="allowed",
utilization=0.3,
raw={
"unifiedWindows": {
"five_hour": {"resetsAt": resets, "utilization": 0.72},
"seven_day": {"resetsAt": resets, "utilization": 0.25},
"seven_day_fable": {"resetsAt": resets, "utilization": 0.1},
}
},
)
)
session.add(RateLimit(window="seven_day", status="allowed", utilization=0.3))
await session.commit()
await seed_usage(
world,
@@ -294,19 +280,12 @@ async def test_limits_report_latest_window_with_gateway_spend(world: World) -> N
)
out = await api.get("/limits")
windows = {w["window"]: w for w in out["windows"]}
assert windows["five_hour"]["utilization"] == 0.72
assert windows["five_hour"]["reported_at"] is not None
assert windows["five_hour"]["utilization"] == 0.9
assert windows["five_hour"]["status"] == "allowed_warning"
assert windows["five_hour"]["gateway"]["output"] == 7
assert windows["five_hour"]["gateway"]["cost_usd"] == 0.25
assert windows["seven_day"]["utilization"] == 0.3
assert windows["seven_day_fable"]["utilization"] == 0.1
assert windows["seven_day_fable"]["status"] == "allowed"
assert [w["window"] for w in out["windows"]] == [
"five_hour",
"seven_day",
"seven_day_fable",
]
assert [w["window"] for w in out["windows"]] == ["five_hour", "seven_day"]
assert len(out["history"]) == 3
@@ -396,100 +375,6 @@ async def test_describe_snapshots_open_tools_and_records_rate_limit(
assert page["offset"] == 0 and len(page["entries"]) == 1
async def test_sessions_show_their_processes_and_take_orders(world: World) -> None:
world.backend._factory = ToolClient # noqa: SLF001
api = Api(world)
api.store.grant("api-only", "weak", scope="api")
weak = {"Authorization": "Bearer weak"}
conv = await world.conversations.create(kind="master", agent="a", origin="test")
key = conv.external_id
ScriptedClient.hold = asyncio.Event()
await world.conversations.post(conv, "go")
await asyncio.sleep(0.3)
census = await api.get("/sessions")
(row,) = census["sessions"]
assert row["key"] == key and row["busy"]
assert row["processes"] == [] and row["cpu"] is None
assert isinstance(census["others"], list)
live = world.pool.get(key)
assert live is not None
live.stderr.append("warning: something")
stderr = await api.get(f"/sessions/{key}/stderr")
assert stderr["lines"] == ["warning: something"]
for path in (f"/sessions/{key}/close", "/processes/1/kill"):
assert (await api.http.post(path, headers=weak)).status_code == 403
assert (
await api.http.get(f"/sessions/{key}/stderr", headers=weak)
).status_code == 403
closed = await api.http.post(f"/sessions/{key}/close", headers=HEADERS)
assert closed.json() == {"key": key, "closed": True}
assert world.pool.get(key) is None
deadline = asyncio.get_running_loop().time() + 5
while ("user", "failed") not in await world.statuses(conv):
assert asyncio.get_running_loop().time() < deadline, "the cut turn hangs"
await asyncio.sleep(0.02)
again = await api.http.post(f"/sessions/{key}/close", headers=HEADERS)
assert again.status_code == 404
gone = await api.http.post("/processes/999999999/kill", headers=HEADERS)
assert gone.status_code == 404
ScriptedClient.hold = asyncio.Event()
await world.conversations.post(conv, "once more")
await asyncio.sleep(0.3)
interrupted = await api.http.post(f"/sessions/{key}/interrupt", headers=weak)
assert interrupted.json() == {"key": key, "interrupted": True}
assert ScriptedClient.instances[-1].interrupted
kinds = [r["kind"] for r in (await api.get("/audit"))["records"]]
assert kinds == ["session_interrupt", "session_close"]
async def test_rewind_lists_turns_and_needs_admin_scope(world: World) -> None:
api = Api(world)
api.store.grant("api-only", "weak", scope="api")
weak = {"Authorization": "Bearer weak"}
sid, entries = await _three_turns(world)
conv = await world.conversations.create(
kind="master", agent="a", origin="test", session_id=sid
)
path = f"/conversations/{conv.external_id}"
listed = (await api.http.get(f"{path}/turns", headers=weak)).json()
assert [t["text"] for t in listed["turns"]] == ["two", "three"]
assert (await api.get(f"{path}/turns", {"limit": 1}))["turns"] == [
listed["turns"][1]
]
body = {"turn": listed["turns"][0]["id"]}
denied = await api.http.post(f"{path}/rewind", json=body, headers=weak)
assert denied.status_code == 403
missing = await api.http.post(
f"{path}/rewind", json={"turn": entries[0]["uuid"]}, headers=HEADERS
)
assert missing.status_code == 404
done = await api.http.post(f"{path}/rewind", json=body, headers=HEADERS)
assert done.status_code == 200, done.text
out = done.json()
assert out["turns"] == 2 and out["previous_session"] == sid
assert (await api.get(path))["session_id"] == out["session_id"] != sid
assert (await api.get(f"{path}/turns"))["turns"] == []
messages = (await api.get(f"{path}/history"))["messages"]
assert [m["content"] for m in messages] == [
"one",
[{"type": "text", "text": "done one"}],
]
(record,) = (await api.get("/audit"))["records"]
assert record["kind"] == "api_rewind"
assert record["detail"]["previous_session"] == sid
ScriptedClient.hold = asyncio.Event()
await world.conversations.post(conv, "busy")
await asyncio.sleep(0.2)
busy = await api.http.post(f"{path}/rewind", json=body, headers=HEADERS)
assert busy.status_code == 409
ScriptedClient.hold.set()
async def test_tokens_and_audit_need_admin_scope(world: World) -> None:
api = Api(world)
api.store.grant("api-only", "weak", scope="api")
@@ -640,91 +525,3 @@ async def test_bind_without_a_window_materializes_one(world: World) -> None:
headers=HEADERS,
)
assert unknown.status_code == 400
async def test_search_finds_conversations_and_memory_files(world: World) -> None:
root = Path(tempfile.mkdtemp(prefix="beaver-mem-"))
(root / "notes").mkdir()
(root / "notes" / "studio.md").write_text("# studio\nthe kiln is warm\n")
api = Api(world, memory_root=root)
conv = await world.conversations.create(kind="master", agent="a", origin="test")
await world.conversations.set_title(conv, "kiln schedule")
other = await world.conversations.create(kind="deep", agent="d", origin="test")
await world.conversations.post(other, "go")
await world.settle(other, 1)
other = await world.conversations.get(other.external_id)
assert other is not None and other.session_id is not None
async with world.db.session() as db:
db.add(
TranscriptEntry(
project_key="p",
session_id=other.session_id,
seq=0,
entry={"type": "assistant", "message": {"content": "warm kiln"}},
)
)
await db.commit()
found = await api.get("/search", {"q": "kiln"})
assert {c["id"] for c in found["conversations"]} == {
conv.external_id,
other.external_id,
}
assert found["files"] == [
{"path": "notes/studio.md", "line": 2, "snippet": "the kiln is warm"}
]
short = await api.http.get("/search", params={"q": "k"}, headers=HEADERS)
assert short.status_code == 400
async def test_context_endpoint_reports_prompt_and_files(world: World) -> None:
api = Api(world)
conv = await world.conversations.create(kind="master", agent="a", origin="test")
await world.conversations.post(conv, "go")
await world.settle(conv, 1)
conv = await world.conversations.get(conv.external_id)
assert conv is not None and conv.session_id is not None
await world.store.append(
world.key(conv.session_id),
build_entries(
[
{"role": "user", "content": "go"},
{
"role": "assistant",
"content": [
{
"type": "tool_use",
"id": "t1",
"name": "Read",
"input": {"file_path": str(world.root / "note.md")},
}
],
},
{
"role": "user",
"content": [
{"type": "tool_result", "tool_use_id": "t1", "content": "x"}
],
},
],
session_id=conv.session_id,
cwd=str(world.root),
model="m",
),
)
ctx = await api.get(f"/conversations/{conv.external_id}/context", {"prompt": 1})
assert ctx["agent"]["name"] == "a"
assert ctx["prompt"]["granules"][0]["bytes"] == len("hi")
assert ctx["prompt_text"] == "hi"
assert ctx["files"] == [
{
"path": "note.md",
"reads": 1,
"writes": 0,
"other": 0,
"last_at": ctx["files"][0]["last_at"],
}
]
assert ctx["tool_counts"] == {"Read": 1}
assert ctx["turns"] == 1
graph = await api.http.get("/vault/graph", headers=HEADERS)
assert graph.status_code == 404
+1 -17
View File
@@ -86,7 +86,6 @@ class FakeClient:
self.prompts: list[str] = []
self.connected = False
self.session_id = options.resume or "fresh-session"
self.asked = asyncio.Event()
FakeClient.instances.append(self)
async def connect(self) -> None:
@@ -94,17 +93,6 @@ class FakeClient:
async def query(self, prompt: str) -> None:
self.prompts.append(prompt)
self.asked.set()
async def receive_messages(self):
served = 0
while True:
while len(self.prompts) <= served:
self.asked.clear()
await self.asked.wait()
served += 1
async for message in self.receive_response():
yield message
async def interrupt(self) -> None:
self.interrupted = True
@@ -250,11 +238,7 @@ async def test_stream_envelope_and_index_rebase(cwd: Path) -> None:
assert opts.strict_mcp_config is True
assert opts.permission_mode == "bypassPermissions"
assert opts.mcp_servers == {
"firefly": {
"type": "http",
"url": "http://127.0.0.1:1/mcp/firefly/",
"headers": {"X-Conversation-Origin": "conv-1"},
}
"firefly": {"type": "http", "url": "http://127.0.0.1:1/mcp/firefly/"}
}
assert opts.session_store is not None
assert opts.resume is None
-142
View File
@@ -1,142 +0,0 @@
import tempfile
from pathlib import Path
from beaver_gateway.agents.claude import ClaudeAgent, Prompts
from beaver_gateway.conversations.context import (
bash_paths,
compose,
files_touched,
granules,
)
from beaver_gateway.vault.links import LinkIndex
def entry(name: str, tool_input: dict, stamp: str = "2026-09-02T10:00:00Z") -> dict:
return {
"type": "assistant",
"timestamp": stamp,
"message": {
"content": [{"type": "tool_use", "name": name, "input": tool_input}]
},
}
def test_files_touched_groups_by_path_relative_to_cwd() -> None:
cwd = Path("/vault")
files, counts = files_touched(
[
entry("Read", {"file_path": "/vault/a.md"}, "2026-09-02T10:00:00Z"),
entry("Edit", {"file_path": "/vault/a.md"}, "2026-09-02T10:05:00Z"),
entry("Grep", {"pattern": "x", "path": "/vault"}),
entry("Bash", {"command": "ls"}),
entry("Read", {"file_path": "/elsewhere/b.md"}, "2026-09-02T09:00:00Z"),
],
cwd=cwd,
)
assert [f["path"] for f in files] == ["a.md", "/elsewhere/b.md"]
assert files[0] == {
"path": "a.md",
"reads": 1,
"writes": 1,
"other": 0,
"last_at": "2026-09-02T10:05:00Z",
}
assert counts == {"Read": 2, "Edit": 1, "Grep": 1, "Bash": 1}
def test_bash_paths_follow_cd_and_quotes() -> None:
paths, here, writes = bash_paths(
"cd /vault/мета && cat 'заметка.md' | head; sed -n 1,5p \"📆 доски/работа.md\"",
Path("/elsewhere"),
)
assert paths == ["/vault/мета/заметка.md", "/vault/мета/📆 доски/работа.md"]
assert here == Path("/vault/мета")
assert writes is False
_, _, writes = bash_paths(
'python3 - <<PY\np = pathlib.Path("поправки.md")\np.write_text(s)\nPY',
Path("/vault"),
)
assert writes is True
assert bash_paths("ls *.md; grep -c x notes.md", None)[0] == ["notes.md"]
def test_files_touched_reads_shell_commands_with_a_sticky_cwd() -> None:
cwd = Path("/vault")
files, counts = files_touched(
[
entry("Bash", {"command": "cd /vault/мета && cat a.md"}, "1"),
entry("Bash", {"command": "python3 - <<PY\nopen('b.md','w')\nPY"}, "2"),
entry("Bash", {"command": "ls"}, "3"),
],
cwd=cwd,
)
assert [(f["path"], f["reads"], f["writes"]) for f in files] == [
("мета/b.md", 0, 1),
("мета/a.md", 1, 0),
]
assert counts == {"Bash": 3}
def test_compose_reads_granules_and_skills() -> None:
root = Path(tempfile.mkdtemp(prefix="beaver-ctx-"))
(root / "voice.md").write_text("be brief " * 40)
(root / "env.md").write_text("you are in the master thread")
skills = root / "skills" / "common"
(skills / "ops").mkdir(parents=True)
(skills / "ops" / "SKILL.md").write_text(
"---\nname: ops\ndescription: servers\n---\n"
)
agent = ClaudeAgent(
name="a",
model="m",
cwd=root,
prompts=Prompts(master=(("voice", root / "voice.md"), root / "env.md")),
skill_sets=(skills,),
gateway_tools=("say",),
)
out = compose(
agent,
"master",
[entry("Read", {"file_path": str(root / "env.md")})],
context_tokens=50_000,
)
assert [g["path"] for g in out["prompt"]["granules"]] == ["voice.md", "env.md"]
assert out["prompt"]["granules"][0]["tag"] == "voice"
assert out["prompt"]["tokens_est"] > 0
assert out["skills"] == [
{
"set": "common",
"path": "skills/common",
"skills": [
{"name": "ops", "description": "servers", "path": "skills/common/ops"}
],
}
]
assert out["tools"]["gateway"] == ["say"]
assert out["files"][0]["path"] == "env.md"
assert out["history_tokens_est"] < 50_000
assert granules(agent, "deep")["granules"] == []
def test_link_index_resolves_stems_and_neighbours() -> None:
root = Path(tempfile.mkdtemp(prefix="beaver-links-"))
(root / "people").mkdir()
(root / "people" / "Marta.md").write_text("works at [[Studio]] with [[Ilya|him]]")
(root / "Studio.md").write_text("see [[people/Marta]] and [[nowhere]]")
(root / "Ilya.md").write_text("plain")
(root / ".obsidian").mkdir()
(root / ".obsidian" / "x.md").write_text("[[Studio]]")
index = LinkIndex(root)
graph = index.neighbours([str(root / "people" / "Marta.md")])
nodes = {n["path"]: n for n in graph["nodes"]}
assert nodes["people/Marta.md"]["touched"] is True
assert set(nodes) == {"people/Marta.md", "Studio.md", "Ilya.md"}
assert {(e["from"], e["to"]) for e in graph["edges"]} == {
("people/Marta.md", "Studio.md"),
("people/Marta.md", "Ilya.md"),
("Studio.md", "people/Marta.md"),
}
missing = index.neighbours(["gone.md"])
assert missing["nodes"] == [
{"path": "gone.md", "title": "gone", "touched": True, "exists": False}
]
+2 -323
View File
@@ -8,7 +8,6 @@ from zoneinfo import ZoneInfo
import pytest
from claude_agent_sdk import (
TaskNotificationMessage,
AssistantMessage,
InMemorySessionStore,
ResultMessage,
@@ -20,11 +19,7 @@ from claude_agent_sdk import (
project_key_for_directory,
)
from anthropic.types import TextBlock as AnthropicTextBlock
from anthropic.types import ThinkingBlock
from beaver_gateway.agents.claude import ClaudeAgent, ClaudeOptions
from beaver_gateway.agents.raycast import RaycastAgent
from beaver_gateway.backends.claude_sdk import ClaudeSdkBackend
from beaver_gateway.events.bus import EventBus
from beaver_gateway.conversations.service import (
@@ -40,7 +35,6 @@ from beaver_gateway.backends.transcript import (
close_open_tool_uses,
open_tool_uses,
render_messages,
reply_text,
strip_tool_entries,
window_entries,
)
@@ -94,46 +88,16 @@ class ScriptedClient:
def __init__(self, options: Any) -> None:
self.options = options
self.prompts: list[str] = []
self.contents: list[list[dict[str, Any]]] = []
self.session_id = options.resume or str(uuid.uuid4())
self.interrupted = False
self.connected = False
self.asked = asyncio.Event()
self.extra: list[Any] = []
ScriptedClient.instances.append(self)
async def connect(self) -> None:
self.connected = True
async def query(self, prompt) -> None:
if not isinstance(prompt, str):
messages = [m async for m in prompt]
content = messages[-1]["message"]["content"]
self.contents.append(content)
prompt = "\n".join(
b.get("text", "") for b in content if b.get("type") == "text"
)
async def query(self, prompt: str) -> None:
self.prompts.append(prompt)
self.asked.set()
async def receive_messages(self):
"""One response per prompt, forever; ``extra`` is what the CLI sends
on its own between prompts (a subagent reporting back)."""
served = 0
while True:
while len(self.prompts) <= served:
self.asked.clear()
if self.extra:
for message in self.extra:
if isinstance(message, asyncio.Event):
await message.wait()
continue
yield message
self.extra = []
await self.asked.wait()
served += 1
async for message in self.receive_response():
yield message
async def receive_response(self):
prompt = self.prompts[-1]
@@ -504,105 +468,6 @@ async def test_copy_seed_forks_parent_with_window(world: World) -> None:
assert (await world.conversations.get(child.external_id)).flags["seed"] is None
async def _three_turns(world: World) -> tuple[str, list[dict[str, Any]]]:
sid = str(uuid.uuid4())
history = [
{"role": "user", "content": "one"},
{"role": "assistant", "content": "done one"},
{"role": "user", "content": "two"},
{
"role": "assistant",
"content": [{"type": "tool_use", "id": "t1", "name": "Read", "input": {}}],
},
{
"role": "user",
"content": [{"type": "tool_result", "tool_use_id": "t1", "content": "x"}],
},
{"role": "assistant", "content": "done two"},
{"role": "user", "content": "three"},
{"role": "assistant", "content": "refused"},
]
entries = build_entries(history, session_id=sid, cwd=str(world.root), model="m")
entries[-1]["isApiErrorMessage"] = True
note = {
**entries[2],
"uuid": str(uuid.uuid4()),
"message": {
"role": "user",
"content": [{"type": "text", "text": "[Request interrupted by user]"}],
},
}
del note["promptSource"]
entries.insert(6, note)
await world.store.append(world.key(sid), entries)
return sid, entries
async def test_rewind_moves_the_thread_to_a_fork_that_ends_before_the_turn(
world: World,
) -> None:
sid, entries = await _three_turns(world)
conv = await world.conversations.create(
kind="master", agent="a", origin="test", session_id=sid
)
await world.conversations.post(conv, "live")
await world.settle(conv, 1)
assert world.pool.get(conv.external_id) is not None
turns = await world.conversations.turns(conv)
assert [(t["text"], t["reply"], t["failed"]) for t in turns] == [
("two", "done two", False),
("three", "refused", True),
]
assert [t["id"] for t in turns] == [entries[2]["uuid"], entries[7]["uuid"]]
result = await world.conversations.rewind(conv, turns[1]["id"])
assert result.turns == 1 and result.previous_session == sid
moved = result.conversation.session_id
assert moved != sid
assert (await world.conversations.get(conv.external_id)).session_id == moved
assert await world.store.load(world.key(sid)) == entries
kept = [
e["message"]["content"]
for e in await world.store.load(world.key(moved))
if e["type"] in ("user", "assistant")
]
assert len(kept) == 7 and kept[0] == "one"
assert kept[-1] == [{"type": "text", "text": "[Request interrupted by user]"}]
assert world.pool.get(conv.external_id) is None
await world.conversations.run_text_turn(conv, "after", origin="user")
assert conv.session_id == sid
assert ScriptedClient.instances[-1].options.resume == moved
(two,) = await world.conversations.turns(result.conversation)
assert two["text"] == "two"
again = await world.conversations.rewind(conv, two["id"])
assert again.turns == 1 and again.previous_session == moved
assert await world.conversations.turns(again.conversation) == []
first = (await world.store.load(world.key(again.conversation.session_id)))[0]
with pytest.raises(LookupError):
await world.conversations.rewind(conv, first["uuid"])
with pytest.raises(LookupError):
await world.conversations.rewind(conv, str(uuid.uuid4()))
async def test_rewind_waits_for_no_turn(world: World) -> None:
sid, entries = await _three_turns(world)
conv = await world.conversations.create(
kind="master", agent="a", origin="test", session_id=sid
)
ScriptedClient.hold = asyncio.Event()
await world.conversations.post(conv, "busy")
await asyncio.sleep(0.2)
with pytest.raises(RuntimeError, match="mid-turn"):
await world.conversations.rewind(conv, entries[7]["uuid"])
ScriptedClient.hold.set()
await world.settle(conv, 1)
assert (await world.conversations.get(conv.external_id)).session_id == sid
async def test_merge_injects_summary_into_parent(world: World) -> None:
sid = str(uuid.uuid4())
await world.store.append(
@@ -818,86 +683,12 @@ def test_transcript_helpers() -> None:
assert windowed[0]["message"]["content"] == "q2" and len(windowed) == 2
assert (
render_messages(
[{"role": "user", "content": "q"}, {"role": "assistant", "content": "a"}],
thinking_is_progress=True,
[{"role": "user", "content": "q"}, {"role": "assistant", "content": "a"}]
)
== "user:\nq\n\nassistant:\na"
)
def test_read_conversation_keeps_progress_and_drops_reasoning() -> None:
history = [
{"role": "user", "content": "status?"},
{
"role": "assistant",
"content": [
{"type": "text", "text": "Checking."},
{"type": "thinking", "thinking": "", "signature": "s"},
{"type": "tool_use", "id": "t1", "name": "komodo", "input": {}},
],
},
{
"role": "user",
"content": [{"type": "tool_result", "tool_use_id": "t1", "content": "ok"}],
},
{
"role": "assistant",
"content": [
{
"type": "thinking",
"thinking": "All up, reading logs.",
"signature": "s",
},
{"type": "tool_use", "id": "t2", "name": "komodo", "input": {}},
],
},
{
"role": "user",
"content": [{"type": "tool_result", "tool_use_id": "t2", "content": "ok"}],
},
{"role": "assistant", "content": [{"type": "text", "text": "Healthy."}]},
]
assert render_messages(history, thinking_is_progress=True) == (
"user:\nstatus?\n\nassistant:\nChecking.\n\nAll up, reading logs.\n\n"
"Healthy.\n(tools: komodo, komodo)"
)
assert render_messages(history, thinking_is_progress=False) == (
"user:\nstatus?\n\nassistant:\nChecking.\n\nHealthy.\n(tools: komodo, komodo)"
)
@pytest.mark.parametrize(
("model", "expected"),
[
("claude-opus-5-5", True),
("claude-fable-5-1", True),
("claude-opus-5", False),
("claude-sonnet-5", False),
("claude-haiku-4-5-20251001", False),
("claude-next-unknown", False),
],
)
def test_only_progress_models_read_their_thinking_as_progress(
tmp_path: Path, model: str, expected: bool
) -> None:
assert (
ClaudeAgent(name="c", model=model, cwd=tmp_path).thinking_is_progress
is expected
)
def test_summarized_display_and_raycast_are_never_progress(tmp_path: Path) -> None:
raycast = RaycastAgent(name="r", model="google-gemini-3.1-pro", system_prompt="")
summarized = ClaudeAgent(
name="s",
model="claude-opus-5-5",
cwd=tmp_path,
options=ClaudeOptions(thinking={"type": "adaptive", "display": "summarized"}),
)
assert not raycast.thinking_is_progress
assert not summarized.thinking_is_progress
async def test_agent_kinds_gate_create(world: World) -> None:
with pytest.raises(ValueError, match="does not serve kind 'deep'"):
await world.conversations.create(kind="deep", agent="a", origin="test")
@@ -1059,115 +850,3 @@ async def test_inject_into_a_closed_master_lands_in_the_open_one(world: World) -
assert item.conversation_id == new.id
assert await world.statuses(old) == []
assert await world.statuses(new) == [("normal", "queued")]
def _self_started(session_id: str, text: str) -> list[Any]:
"""What the CLI sends by itself once a background subagent reports back."""
return [
AssistantMessage(
content=[ToolUseBlock(id="tu_dig", name="Grep", input={"pattern": "au"})],
model="m",
parent_tool_use_id="tu_bg",
),
TaskNotificationMessage(
subtype="task_notification",
data={},
task_id="t1",
status="completed",
output_file="",
summary='Agent "dig" finished',
uuid="n1",
session_id=session_id,
),
AssistantMessage(content=[TextBlock(text=text)], model="m"),
ResultMessage(
subtype="success",
duration_ms=1,
duration_api_ms=1,
is_error=False,
num_turns=1,
session_id=session_id,
stop_reason="end_turn",
total_cost_usd=0.0,
usage={"input_tokens": 1, "output_tokens": 1},
origin={"kind": "task-notification"},
),
]
async def test_subagent_report_between_turns_is_its_own_turn(world: World) -> None:
conv = await world.conversations.create(kind="master", agent="a", origin="test")
seen: list[dict[str, Any]] = []
async def collect() -> None:
async for event in world.bus.stream(conversation_id=conv.external_id):
seen.append(event)
task = asyncio.create_task(collect())
await world.conversations.post(conv, "go")
await world.settle(conv, 1)
client = ScriptedClient.instances[0]
client.extra = _self_started(client.session_id, "the agent found gold")
client.asked.set()
await asyncio.sleep(0.3)
task.cancel()
replies = [e for e in seen if e["type"] == "reply"]
assert [r["text"] for r in replies] == ["ok:go", "the agent found gold"]
assert replies[1]["item_origin"] == "task"
starts = [e for e in seen if e["type"] == "turn.start"]
assert [s["origin"] for s in starts] == ["user", "task"]
assert starts[1]["text"] == 'Agent "dig" finished (completed)'
assert len(client.prompts) == 1
first_end = next(i for i, e in enumerate(seen) if e["type"] == "turn.end")
later = [e for e in seen[first_end + 1 :] if e["type"] in ("stream", "tool")]
assert later and all(e["turn_id"] == starts[0]["turn_id"] for e in later[:1])
assert all(e["turn_id"] == starts[1]["turn_id"] for e in later[1:])
assert (await world.conversations.get(conv.external_id)).running_turn is None
async def test_subagent_result_inside_a_turn_does_not_end_it(world: World) -> None:
conv = await world.conversations.create(kind="master", agent="a", origin="test")
await world.conversations.post(conv, "one")
await world.settle(conv, 1)
client = ScriptedClient.instances[0]
client.extra = _self_started(client.session_id, "late report")
await world.conversations.post(conv, "two")
await world.settle(conv, 2)
assert client.prompts[-1].startswith("two")
rows = await world.conversations.queue.recent(conv.id)
assert [r.status for r in rows] == ["done", "done"]
def test_reply_text_keeps_progress_updates_and_drops_hidden_reasoning() -> None:
blocks = [
ThinkingBlock(type="thinking", thinking="", signature="s"),
ThinkingBlock(type="thinking", thinking="looking it up", signature="s"),
AnthropicTextBlock(type="text", text="found it"),
]
assert reply_text(blocks, thinking_is_progress=True) == "looking it up\n\nfound it"
assert reply_text(blocks, thinking_is_progress=False) == "found it"
def test_reply_text_on_a_plain_answer_is_unchanged() -> None:
blocks = [AnthropicTextBlock(type="text", text=" just an answer ")]
assert reply_text(blocks, thinking_is_progress=True) == "just an answer"
async def test_a_conversation_stream_carries_updates_about_that_conversation() -> None:
bus = EventBus()
seen: list[tuple[str, str | None]] = []
async def collect() -> None:
async for event in bus.stream(conversation_id="c1"):
seen.append((event["type"], event.get("id")))
if len(seen) == 2:
return
task = asyncio.create_task(collect())
await asyncio.sleep(0)
bus.publish("conversation.updated", id="c2", running_turn=None)
bus.publish("conversation.updated", id="c1", running_turn=None)
bus.publish("stream", conversation_id="c2")
bus.publish("stream", conversation_id="c1")
await asyncio.wait_for(task, 1)
assert seen == [("conversation.updated", "c1"), ("stream", None)]
+1 -76
View File
@@ -7,11 +7,7 @@ from test_conversations import ScriptedClient, World, world
from beaver_gateway.conversations.service import UserSaid
from beaver_gateway.conversations.envelope import Envelope, RecallContext, render
from beaver_gateway.conversations.texts import (
ConversationTexts,
EnvelopeTexts,
split_service,
)
from beaver_gateway.conversations.texts import EnvelopeTexts
from beaver_gateway.vault.watch import Change, VaultWatch, WatchRules
__all__ = ["world"]
@@ -81,37 +77,6 @@ def test_watch_reports_only_added_lines_with_mtime() -> None:
assert only.added == ["- 14:00 снова"]
def test_a_templated_note_shows_its_text_without_queries_or_frontmatter() -> None:
root, watch = vault()
rel = f"дни/{watch.today()}.md"
diary = root / rel
watch._seen.pop(rel)
diary.write_text(
"---\ntags:\n - дни\n---\n## сводка\n- \n\n\n---\n\n# планы\nпросрочены:\n"
"```tasks\nnot done\ndue before 2026-10-13\n```\n"
"обязательно сегодня:\n~~~dataview\nlist\n~~~\n"
"```python\nprint(1)\n```\n06:04 - спать\n"
)
watch.note(diary)
(change,) = watch.take()
assert change.added == [
"## сводка",
"- ",
"---",
"# планы",
"просрочены:",
"обязательно сегодня:",
"```python",
"print(1)",
"```",
"06:04 - спать",
]
append(diary, "```tasks\ndone on 2026-10-13\n```\n07:10 - встал\n")
watch.note(diary)
(later,) = watch.take()
assert later.added == ["07:10 - встал"]
def test_envelope_respects_ceilings_and_names_only_window() -> None:
root, watch = vault()
today = watch.today()
@@ -164,24 +129,6 @@ def test_render_hits_total_ceiling() -> None:
assert lines[1] == "time: 2026-08-26 13:04 (Warsaw)"
def test_the_users_words_part_from_what_the_gateway_put_under_them() -> None:
texts = ConversationTexts()
envelope = f"{HEADER}\ntime: 2026-10-13 10:00 (UTC)"
bundle = texts.bundle_header.format(since="2026-10-13T10:00:00+00:00")
injects = f"{bundle}\n- [cron] tick"
assert split_service(f"hello\n\n{envelope}", texts) == ("hello", envelope)
assert split_service(f"hello\n\n{envelope}\n\n{injects}", texts) == (
"hello",
f"{envelope}\n\n{injects}",
)
assert split_service(f"hello\n\n{injects}", texts) == ("hello", injects)
assert split_service(f"\n\n{envelope}", texts) == ("", envelope)
assert split_service("hello\n\n[a note of mine]\nmore", texts) == (
"hello\n\n[a note of mine]\nmore",
None,
)
async def test_master_turn_gets_envelope_after_text_and_before_injects(
world: World,
) -> None:
@@ -272,25 +219,3 @@ async def test_failing_recall_or_sink_never_blocks_the_turn(world: World) -> Non
assert prompt.startswith("hello\n\n" + HEADER)
assert root.exists()
await asyncio.sleep(0)
async def test_the_recall_hook_and_the_sink_see_whose_turn_it_is(world: World) -> None:
root, watch = vault()
seen: list[RecallContext] = []
def recall(ctx: RecallContext) -> str | None:
seen.append(ctx)
return None if ctx.agent == "a" else "👤 указатель"
noted: list[UserSaid] = []
world.conversations._envelope = Envelope(watch=watch, tz="UTC", recall=recall) # noqa: SLF001
world.conversations._user_sink = noted.append # noqa: SLF001
master = await world.conversations.create(kind="master", agent="a", origin="test")
append(root / "люди" / "Прохор.md", "новое\n")
watch.note(root / "люди" / "Прохор.md")
await world.conversations.post(master, "что там у Прохор")
await world.settle(master, 1)
assert [ctx.agent for ctx in seen] == ["a"]
assert [m.agent for m in noted] == ["a"]
assert "👤" not in ScriptedClient.instances[0].prompts[0]
await asyncio.sleep(0)
-85
View File
@@ -1,85 +0,0 @@
from datetime import UTC, datetime, timedelta
from types import SimpleNamespace
from typing import Any, cast
from test_conversations import StubFrontend, World, world
from beaver_gateway.jobs.job import Job, JobRun
from beaver_gateway.storage.models import Conversation
__all__ = ["world"]
async def _noop(_: JobRun) -> None:
return None
def _run(world: World) -> JobRun:
return JobRun(
job=Job("test", _noop),
trigger="test",
payload={},
scheduler=cast("Any", SimpleNamespace(conversations=world.conversations)),
)
async def _quiet_since(world: World, conv: Conversation, ago: timedelta) -> None:
async def apply(row: Conversation) -> None:
row.last_activity_at = datetime.now(UTC) - ago
await world.conversations._update(conv, apply) # noqa: SLF001
async def test_close_quiet_closes_and_marks_only_idle_branches(world: World) -> None:
marked: list[tuple[str, str]] = []
class MarkingFrontend(StubFrontend):
async def mark_closed(self, conv: Conversation) -> bool:
marked.append((conv.external_id, conv.status))
return True
tg = MarkingFrontend("tg", ("master", "branch"))
tg.conversations = world.conversations
world.conversations._frontends = [tg] # noqa: SLF001
convs = world.conversations
master = await convs.create(kind="master", agent="a", session_id="m")
idle = await convs.create(kind="branch", agent="a", parent=master, session_id="1")
fresh = await convs.create(kind="branch", agent="a", parent=master, session_id="2")
waiting = await convs.create(
kind="branch", agent="a", parent=master, session_id="3"
)
await _quiet_since(world, idle, timedelta(days=4))
await _quiet_since(world, fresh, timedelta(days=1))
await _quiet_since(world, waiting, timedelta(days=4))
await convs.queue.push(
conversation_id=cast("int", waiting.id),
priority="normal",
origin="cron",
text="later",
)
closed = await _run(world).close_quiet(kind="branch", days=3)
assert [c.external_id for c in closed] == [idle.external_id]
assert marked == [(idle.external_id, "closed")]
assert (await convs.get(fresh.external_id)).status == "open"
assert (await convs.get(waiting.external_id)).status == "open"
async def test_inject_into_falls_back_to_the_master(world: World) -> None:
convs = world.conversations
master = await convs.create(kind="master", agent="a", session_id="m")
branch = await convs.create(kind="branch", agent="a", parent=master)
gone = await convs.create(kind="branch", agent="a", parent=master)
await convs.set_status(gone, "closed")
run = _run(world)
assert await run.inject_into(branch.external_id, "to branch", urgency="normal")
assert await run.inject_into(gone.external_id, "to closed", urgency="normal")
assert await run.inject_into(None, "to nobody", urgency="normal")
assert [i.text for i in await convs.queue.pending(branch.id)] == ["to branch"]
assert [i.text for i in await convs.queue.pending(master.id)] == [
"to closed",
"to nobody",
]
-299
View File
@@ -1,299 +0,0 @@
import asyncio
import base64
import os
import sys
import tempfile
from pathlib import Path
import pytest
from cryptography.exceptions import InvalidTag
from httpx import ASGITransport, AsyncClient
from sqlmodel import select
from beaver_gateway.app import AgentRegistry, McpRegistry
from beaver_gateway.frontends.api.frontend import build_app as build_api
from beaver_gateway.frontends.base import GatewayRuntime
from beaver_gateway.keystore import (
SecretBox,
SecretsDisabledError,
SecretStore,
UnknownSecretError,
cli,
serve,
)
from beaver_gateway.security.auth import TokenStore
from beaver_gateway.storage import Database
from beaver_gateway.storage.models import AuditLog, Secret
KEY = "11" * 32
HEADERS = {"Authorization": "Bearer tok"}
@pytest.fixture
async def db():
root = Path(tempfile.mkdtemp(prefix="beaver-keys-"))
database = Database(f"sqlite:///{root / 'k.db'}")
await database.create_all()
yield database
await database.dispose()
@pytest.fixture
def store(db: Database) -> SecretStore:
return SecretStore(db, SecretBox.from_hex(KEY))
def test_box_binds_the_value_to_its_name() -> None:
box = SecretBox.from_hex(KEY)
sealed = box.seal("A_KEY", b"hunter22")
assert b"hunter22" not in sealed
assert box.open("A_KEY", sealed) == b"hunter22"
with pytest.raises(InvalidTag):
box.open("OTHER", sealed)
with pytest.raises(ValueError, match="32 bytes"):
SecretBox.from_hex("abcd")
async def test_store_keeps_only_ciphertext_and_reveals_by_name(
store: SecretStore, db: Database
) -> None:
await store.put("NODEREAL_BSC", description="bsc rpc", data=b"nr-abcdef123")
await store.put("GCP_SA", data=b'{"k": 1}', filename="../sa.json")
async with db.session() as session:
rows = (await session.exec(select(Secret))).all()
assert all(b"nr-abcdef123" not in r.sealed for r in rows)
assert {r.name: r.filename for r in rows} == {
"GCP_SA": "sa.json",
"NODEREAL_BSC": None,
}
await store.put("NODEREAL_BSC", description=" rpc key ")
listed = {r.name: r for r in await store.rows()}
assert listed["NODEREAL_BSC"].description == "rpc key"
assert listed["NODEREAL_BSC"].used_at is None
revealed = await store.reveal(["NODEREAL_BSC", "GCP_SA"])
assert [(r.name, r.data) for r in revealed] == [
("NODEREAL_BSC", b"nr-abcdef123"),
("GCP_SA", b'{"k": 1}'),
]
assert all(r.used_at is not None for r in await store.rows())
with pytest.raises(UnknownSecretError):
await store.reveal(["NODEREAL_BSC", "MISSING"])
with pytest.raises(ValueError, match="invalid secret name"):
await store.put("lower", data=b"x")
with pytest.raises(ValueError, match="needs a value"):
await store.put("EMPTY", description="no value")
assert await store.delete("GCP_SA")
assert not await store.delete("GCP_SA")
async def test_another_key_or_no_key_opens_nothing(
store: SecretStore, db: Database
) -> None:
await store.put("A_KEY", data=b"value-123")
with pytest.raises(SecretsDisabledError, match="SECRETS_KEY changed"):
await SecretStore(db, SecretBox.from_hex("22" * 32)).reveal(["A_KEY"])
off = SecretStore(db, None)
assert not off.enabled
assert [r.name for r in await off.rows()] == ["A_KEY"]
with pytest.raises(SecretsDisabledError):
await off.reveal(["A_KEY"])
with pytest.raises(SecretsDisabledError):
await off.put("B_KEY", data=b"x")
@pytest.fixture
async def socket_path(store: SecretStore):
path = Path(tempfile.mkdtemp(prefix="bk-", dir="/tmp")) / "run" / "gw.sock" # noqa: S108
used: list[list[str]] = []
async def on_use(names):
used.append(list(names))
task = asyncio.create_task(serve(store, path, on_use=on_use))
for _ in range(100):
if path.exists():
break
await asyncio.sleep(0.01)
yield path, used
task.cancel()
async def test_socket_is_root_only_and_answers_ls_and_get(
store: SecretStore, socket_path
) -> None:
path, used = socket_path
assert path.stat().st_mode & 0o777 == 0o600
assert path.parent.stat().st_mode & 0o777 == 0o700
await store.put("API_TOKEN", description="the api", data=b"tok-123456")
ls = await asyncio.to_thread(cli.request, {"op": "ls"}, path)
assert ls["secrets"] == [
{"name": "API_TOKEN", "description": "the api", "filename": None}
]
got = await asyncio.to_thread(
cli.request, {"op": "get", "names": ["API_TOKEN"]}, path
)
assert base64.b64decode(got["secrets"]["API_TOKEN"]["data"]) == b"tok-123456"
assert used == [["API_TOKEN"]]
with pytest.raises(cli.SecretError, match="unknown secret: NOPE"):
await asyncio.to_thread(cli.request, {"op": "get", "names": ["NOPE"]}, path)
with pytest.raises(cli.SecretError, match="invalid secret name"):
await asyncio.to_thread(cli.request, {"op": "get", "names": ["../x"]}, path)
async def test_run_hands_values_and_files_and_masks_the_output(
store: SecretStore, socket_path, monkeypatch, capfd
) -> None:
path, _ = socket_path
await store.put("API_TOKEN", data=b"tok/123456")
await store.put("SA_JSON", data=b'{"private": "pem"}', filename="sa.json")
monkeypatch.setattr(
cli, "ask", lambda argv: cli.request({"op": "get", "names": argv[1:]}, path)
)
shm = Path(tempfile.mkdtemp(prefix="bk-shm-"))
stale = shm / f"{cli.PREFIX}999999-old"
stale.mkdir()
script = (
"import os, sys, pathlib;"
"p = pathlib.Path(os.environ['SA_JSON']);"
"print(p.read_text(), oct(p.stat().st_mode & 0o777), p.parent.parent.name.startswith('beaver-secret-'));"
"print('url', 'https://x/?key=' + os.environ['API_TOKEN'].replace('/', '%2F'));"
"sys.stderr.write('boom ' + os.environ['API_TOKEN'] + '\\n');"
"print(p, flush=True); sys.exit(3)"
)
code = await asyncio.to_thread(
cli.run, ["API_TOKEN", "SA_JSON"], [sys.executable, "-c", script], root=shm
)
out, err = capfd.readouterr()
assert code == 3
lines = out.splitlines()
assert lines[0] == '{"private": "pem"} 0o600 True'
assert lines[1] == "url https://x/?key=<secret:API_TOKEN>"
assert "tok/123456" not in out + err
assert err == "boom <secret:API_TOKEN>\n"
assert not Path(lines[2]).exists()
assert list(shm.iterdir()) == []
def test_masker_never_splits_a_value_across_chunks() -> None:
masker = cli.Masker(cli.needles_for({"K": b"abcdef123"}))
out = b"".join(
[
masker.feed(b"one abc"),
masker.feed(b"def1"),
masker.feed(b"23 two\nthr"),
masker.flush(),
]
)
assert out == b"one <secret:K> two\nthr"
assert cli.Masker({}).feed(b"as is") == b"as is"
blob = b"x" * (cli.HOLD + 10) + b"abcdef"
masker = cli.Masker(cli.needles_for({"K": b"abcdef123"}))
head = masker.feed(blob)
assert (
head + masker.feed(b"123") + masker.flush()
== b"x" * (cli.HOLD + 10) + b"<secret:K>"
)
assert cli.needles_for({"S": b"abc"}) == {}
def test_cli_rejects_bad_shapes(capsys) -> None:
assert cli.main(["run", "A_KEY"]) == 2
assert cli.main(["run", "bad-name", "--", "true"]) == 2
assert cli.main(["--root", "get", "../etc"]) == 2
assert cli.main(["--root", "rm"]) == 2
assert "usage" in capsys.readouterr().err
class _Runtime:
def __init__(self, db: Database, store: SecretStore | None) -> None:
self.tokens = TokenStore(db, bootstrap={"t": "tok"})
self.tokens.grant("weak", "weak", scope="api")
self.runtime = GatewayRuntime(
agents=AgentRegistry([]),
mcps=McpRegistry([]),
backends={},
token_store=self.tokens,
db=db,
conversations=object(),
bus=object(),
secrets=store,
)
self.http = AsyncClient(
transport=ASGITransport(app=build_api(self.runtime)), base_url="http://api"
)
async def test_api_writes_secrets_but_never_shows_a_value(
store: SecretStore, db: Database
) -> None:
api = _Runtime(db, store).http
assert (
await api.get("/secrets", headers={"Authorization": "Bearer weak"})
).status_code == 403
put = await api.put(
"/secrets/NODEREAL_BSC",
json={"description": "bsc", "value": "nr-secret-1"},
headers=HEADERS,
)
assert put.status_code == 200, put.text
upload = await api.put(
"/secrets/SA_JSON",
json={"file": {"name": "sa.json", "content": base64.b64encode(b"{}").decode()}},
headers=HEADERS,
)
assert upload.json()["secret"]["kind"] == "file"
rename = await api.put(
"/secrets/NODEREAL_BSC", json={"description": "rpc"}, headers=HEADERS
)
assert rename.json()["secret"]["size"] == len("nr-secret-1")
listed = await api.get("/secrets", headers=HEADERS)
assert listed.json()["enabled"] is True
assert "nr-secret-1" not in listed.text
assert [(s["name"], s["description"]) for s in listed.json()["secrets"]] == [
("NODEREAL_BSC", "rpc"),
("SA_JSON", ""),
]
assert (await store.reveal(["NODEREAL_BSC"]))[0].data == b"nr-secret-1"
bad = await api.put("/secrets/bad name", json={"value": "x"}, headers=HEADERS)
assert bad.status_code == 400
both = await api.put(
"/secrets/X_KEY",
json={"value": "x", "file": {"name": "a", "content": ""}},
headers=HEADERS,
)
assert both.status_code == 400
assert (await api.delete("/secrets/SA_JSON", headers=HEADERS)).status_code == 200
assert (await api.delete("/secrets/SA_JSON", headers=HEADERS)).status_code == 404
async with db.session() as session:
audit = (await session.exec(select(AuditLog))).all()
assert [a.kind for a in audit] == [
"secret_set",
"secret_set",
"secret_set",
"secret_delete",
]
assert all("nr-secret-1" not in a.detail_json for a in audit)
async def test_api_without_a_key_lists_but_refuses_writes(db: Database) -> None:
api = _Runtime(db, SecretStore(db, None)).http
assert (await api.get("/secrets", headers=HEADERS)).json() == {
"enabled": False,
"secrets": [],
}
put = await api.put("/secrets/A_KEY", json={"value": "x"}, headers=HEADERS)
assert put.status_code == 503
def test_cli_runs_with_isolated_stdlib_python() -> None:
source = Path(cli.__file__).read_text(encoding="utf-8")
imports = {
line.split()[1].split(".")[0]
for line in source.splitlines()
if line.startswith(("import ", "from "))
}
assert imports <= set(sys.stdlib_module_names) | {"__future__"}
assert os.access(cli.__file__, os.R_OK)
+8 -153
View File
@@ -1,11 +1,6 @@
from pathlib import Path
from anthropic.types import Message, TextBlock, ThinkingBlock, ToolUseBlock, Usage
from beaver_gateway.frontends.markdown import history, parser, renderer
from beaver_gateway.frontends.markdown.crossfront import CrossFrontendLogger
from beaver_gateway.frontends.markdown.parser import TextSegment, ToolSegment
from beaver_gateway.frontends.turn_record import TurnRecord
from beaver_gateway.frontends.markdown import renderer
def _message(*blocks: object) -> Message:
@@ -29,159 +24,19 @@ def test_tool_calls_leave_no_blank_lines() -> None:
ToolUseBlock(type="tool_use", id="t3", name="Grep", input={"pattern": "y"}),
TextBlock(type="text", text="second"),
)
for progress in (True, False):
assert renderer.render_assistant_message(
message, thinking_is_progress=progress
) == ("### Assistant:\n\nfirst\n\nsecond\n")
def _progress_turn() -> Message:
return _message(
TextBlock(type="text", text="Checking the containers."),
ThinkingBlock(type="thinking", thinking="", signature="s"),
ToolUseBlock(type="tool_use", id="t1", name="komodo", input={}),
ThinkingBlock(type="thinking", thinking="", signature="s"),
ThinkingBlock(type="thinking", thinking="All up, reading logs.", signature="s"),
ToolUseBlock(type="tool_use", id="t2", name="komodo", input={}),
ThinkingBlock(type="thinking", thinking="Found it.\nOne more.", signature="s"),
ToolUseBlock(type="tool_use", id="t3", name="komodo", input={}),
TextBlock(type="text", text="The stack is healthy."),
assert renderer.render_assistant_message(message) == (
"### Assistant:\n\nfirst\n\nsecond\n"
)
def _reasoning_turn() -> Message:
return _message(
ThinkingBlock(type="thinking", thinking="one", signature=""),
ToolUseBlock(type="tool_use", id="t1", name="web_search", input={}),
ThinkingBlock(type="thinking", thinking="two", signature=""),
ToolUseBlock(type="tool_use", id="t2", name="read_page", input={}),
ThinkingBlock(type="thinking", thinking="three", signature=""),
def test_thinking_renders_as_collapsed_callout() -> None:
message = _message(
ThinkingBlock(type="thinking", thinking="a\nb", signature="s"),
TextBlock(type="text", text="answer"),
ThinkingBlock(type="thinking", thinking="four\nfive", signature=""),
TextBlock(type="text", text="more"),
)
REASONING_TURN_AS_BEFORE = (
"### Assistant:\n\n> [!thinking]-\n> one\n\nanswer\n\n"
"> [!thinking]-\n> four\n> five\n\nmore\n"
)
def _full_circle(
message: Message, *, thinking_is_progress: bool
) -> history.ForkOutcome:
body = renderer.render_user_text("status?") + renderer.render_assistant_message(
message, thinking_is_progress=thinking_is_progress
assert renderer.render_assistant_message(message) == (
"### Assistant:\n\n> [!thinking]-\n> a\n> b\n\nanswer\n"
)
turns = parser.parse(body + renderer.render_user_text("thanks")).turns
stored = [
{"role": "user", "content": "status?"},
{"role": "assistant", "content": [b.model_dump() for b in message.content]},
]
outcome = history.diff_and_fork(
stored=stored, incoming=turns, thinking_is_progress=thinking_is_progress
)
assert outcome.divergence_index is None
assert not outcome.edited
assert outcome.messages[1] == stored[1]
return outcome
def test_progress_renders_as_text_in_place_and_empty_thinking_vanishes() -> None:
assert renderer.render_assistant_message(
_progress_turn(), thinking_is_progress=True
) == (
"### Assistant:\n\nChecking the containers.\n\nAll up, reading logs.\n\n"
"Found it.\nOne more.\n\nThe stack is healthy.\n"
)
def test_reasoning_renders_as_collapsed_callouts_as_before() -> None:
assert (
renderer.render_assistant_message(_reasoning_turn(), thinking_is_progress=False)
== REASONING_TURN_AS_BEFORE
)
def test_progress_round_trips_through_the_parser_and_the_stored_history() -> None:
message = _progress_turn()
body = renderer.render_assistant_message(message, thinking_is_progress=True)
assert parser.parse_assistant_structure(body.removeprefix("### Assistant:")) == [
TextSegment(
text="Checking the containers.\n\nAll up, reading logs.\n\n"
"Found it.\nOne more.\n\nThe stack is healthy."
)
]
_full_circle(message, thinking_is_progress=True)
def test_reasoning_round_trips_through_the_parser_and_the_stored_history() -> None:
message = _reasoning_turn()
body = renderer.render_assistant_message(message, thinking_is_progress=False)
assert parser.parse_assistant_structure(body.removeprefix("### Assistant:")) == [
TextSegment(text="answer"),
TextSegment(text="more"),
]
_full_circle(message, thinking_is_progress=False)
def test_an_edited_progress_turn_rebuilds_instead_of_rewriting_signed_blocks() -> None:
message = _progress_turn()
body = renderer.render_assistant_message(
message, thinking_is_progress=True
).replace("Found it", "Fixed it")
turns = parser.parse(
renderer.render_user_text("status?") + body + renderer.render_user_text("ok")
).turns
stored = [
{"role": "user", "content": "status?"},
{"role": "assistant", "content": [b.model_dump() for b in message.content]},
]
outcome = history.diff_and_fork(
stored=stored, incoming=turns, thinking_is_progress=True
)
assert outcome.edited
assert outcome.messages[1] == {
"role": "assistant",
"content": [{"type": "text", "text": turns[1].text}],
}
async def test_a_raycast_archive_with_reasoning_renders_as_before(
tmp_path: Path,
) -> None:
logger = CrossFrontendLogger(vault_path=tmp_path, logged_subdir="logged")
record = TurnRecord(
agent_name="beaver-gemini-pro-high",
input_messages=[{"role": "user", "content": "q"}],
output_message=_reasoning_turn(),
thinking_is_progress=False,
)
await logger.handle(record)
[written] = (tmp_path / "logged" / "beaver-gemini-pro-high").glob("*.md")
body = parser.parse(written.read_text()).body
assert f"---\n\n{REASONING_TURN_AS_BEFORE}\n---\n\n### User:" in body
def test_a_file_with_collapsed_thinking_callouts_parses_as_before() -> None:
body = (
"### User:\n\nq\n\n---\n\n### Assistant:\n\n"
"> [!thinking]-\n> weighing it\n> twice\n\nanswer\n\n"
"> [!tool]- Read · notes.md\n> input\n\n"
"> [!thinking]-\n>\n\nmore\n\n---\n\n### User:\n\nnext\n"
)
parsed = parser.parse(body)
assert parsed.turns[1].structure == (
TextSegment(text="answer"),
ToolSegment(name="Read"),
TextSegment(text="more"),
)
assert parsed.messages == [
{"role": "user", "content": "q"},
{"role": "assistant", "content": "answer\n\nmore"},
{"role": "user", "content": "next"},
]
def test_user_scaffold_matches_rendered_user_turn_shape() -> None:
-69
View File
@@ -1,69 +0,0 @@
from __future__ import annotations
import asyncio
import contextlib
import socket
from typing import TYPE_CHECKING, Any
import pytest
import uvicorn
from fastmcp import Client, FastMCP
from fastmcp.client.transports import StreamableHttpTransport
from fastmcp.server.dependencies import get_http_headers
from beaver_gateway.backends.claude_sdk import ORIGIN_HEADER
from beaver_gateway.mcp.internal_app import build_internal_app
from beaver_gateway.mcp.types import McpServer
if TYPE_CHECKING:
from collections.abc import AsyncIterator
def _free_port() -> int:
with socket.socket() as s:
s.bind(("127.0.0.1", 0))
return s.getsockname()[1]
@contextlib.asynccontextmanager
async def _serving(app: Any, port: int) -> AsyncIterator[None]:
server = uvicorn.Server(
uvicorn.Config(app, host="127.0.0.1", port=port, log_level="warning")
)
task = asyncio.create_task(server.serve())
while not server.started:
await asyncio.sleep(0.02)
try:
yield
finally:
server.should_exit = True
await task
@pytest.mark.anyio
async def test_the_origin_header_reaches_the_upstream_mcp() -> None:
upstream = FastMCP("upstream")
@upstream.tool
def caller() -> str:
return get_http_headers().get(ORIGIN_HEADER.lower(), "")
up, gateway = _free_port(), _free_port()
app, urls, _ = build_internal_app(
[McpServer.http(name="t3", url=f"http://127.0.0.1:{up}/mcp")],
host="127.0.0.1",
port=gateway,
)
seen = []
async with (
_serving(upstream.http_app(transport="http", path="/mcp"), up),
_serving(app, gateway),
):
for conversation in ("conv-a", "conv-b"):
transport = StreamableHttpTransport(
urls["t3"], headers={ORIGIN_HEADER: conversation}
)
async with Client(transport) as client:
result = await client.call_tool("caller", {})
seen.append(result.data)
assert seen == ["conv-a", "conv-b"]
-92
View File
@@ -1,92 +0,0 @@
import os
import subprocess
import sys
import time
from beaver_gateway.backends.processes import Processes, _command, subtree
SLEEP = "import time; time.sleep(60)"
NEST = (
"import subprocess, sys, time; "
f"subprocess.Popen([sys.executable, '-c', {SLEEP!r}]); time.sleep(60)"
)
def spawn(code: str) -> subprocess.Popen[bytes]:
return subprocess.Popen([sys.executable, "-c", code])
def rows_under(processes: Processes, pid: int, count: int) -> list[dict]:
deadline = time.monotonic() + 5
while time.monotonic() < deadline:
rows = subtree(processes.tree_parents_first(), pid)
if len(rows) >= count:
return rows
time.sleep(0.05)
msg = f"{pid} never grew to {count} processes"
raise AssertionError(msg)
def test_the_tree_shows_what_the_gateway_started_and_what_it_costs() -> None:
child = spawn(SLEEP)
try:
processes = Processes()
(row,) = rows_under(processes, child.pid, 1)
assert row["parent"] == os.getpid()
assert row["depth"] == 0
assert row["rss"] > 0
assert row["cpu"] is None
assert "time.sleep" in row["command"]
(again,) = rows_under(processes, child.pid, 1)
assert again["cpu"] is not None
finally:
child.kill()
child.wait()
def test_kill_takes_the_branch_and_nothing_the_gateway_did_not_start() -> None:
parent = spawn(NEST)
try:
processes = Processes()
top, nested = rows_under(processes, parent.pid, 2)
assert (top["depth"], nested["depth"]) == (0, 1)
assert not processes.kill_own_descendant(os.getpid())
assert not processes.kill_own_descendant(os.getppid())
assert processes.kill_own_descendant(parent.pid)
parent.wait(timeout=5)
alive = {row["pid"] for row in processes.tree_parents_first()}
assert not alive & {top["pid"], nested["pid"]}
finally:
parent.kill()
parent.wait()
def test_a_subtree_ends_at_the_next_sibling() -> None:
rows = [
{"pid": 1, "depth": 0},
{"pid": 2, "depth": 1},
{"pid": 3, "depth": 2},
{"pid": 4, "depth": 1},
{"pid": 5, "depth": 0},
]
assert [(r["pid"], r["depth"]) for r in subtree(rows, 2)] == [(2, 0), (3, 1)]
assert [r["pid"] for r in subtree(rows, 1)] == [1, 2, 3, 4]
assert subtree(rows, 9) == []
assert subtree(rows, None) == []
def test_a_command_line_keeps_prompts_and_configs_to_itself() -> None:
argv = [
"/opt/claude/claude",
"--model",
"m",
"--system-prompt",
"you are a beaver",
"--mcp-config",
'{"gateway": {"headers": {"Authorization": "Bearer abc"}}}',
"x" * 100,
]
assert _command(argv) == (
"claude --model m --system-prompt … --mcp-config … " + "x" * 80 + "…"
)
assert _command([]) == ""
-11
View File
@@ -84,17 +84,6 @@ async def test_rotation_does_not_touch_a_master_mid_turn(world: World) -> None:
await world.settle(conv, 1)
async def test_force_rotates_a_master_that_is_not_due(world: World) -> None:
conv = await world.conversations.create(kind="master", agent="a", origin="test")
rotation = Rotation(world.conversations, POLICY)
assert await rotation.tick() == []
rotated = await rotation.tick(force="manual")
assert len(rotated) == 1
assert rotated[0].external_id != conv.external_id
assert (await world.conversations.get(conv.external_id)).status == "closed"
assert len(await world.conversations.find(kind="master", status="open")) == 1
async def test_rotation_order_handout_close_marks_moves_and_new_day(
world: World,
) -> None:
-133
View File
@@ -1,133 +0,0 @@
import tempfile
from pathlib import Path
import httpx
import pytest
from fastapi import FastAPI, Request
from beaver_gateway.app import AgentRegistry, McpRegistry
from beaver_gateway.frontends.api.frontend import build_app as build_api
from beaver_gateway.frontends.base import Frontend, GatewayRuntime
from beaver_gateway.frontends.bearer import require_token
from beaver_gateway.security.auth import BUILTIN_SCOPES, TokenStore, scopes_with
from beaver_gateway.storage import Database
ROOT = {"Authorization": "Bearer root-key"}
VOICE = {"Authorization": "Bearer voice-key"}
OPS = {"Authorization": "Bearer ops-key"}
class Speaker(Frontend):
"""A frontend a setup declares, gated by a scope of its own."""
name = "voice"
path = "/voice"
scope = "voice"
def configure(self, runtime: GatewayRuntime) -> None:
self.runtime = runtime
def app(self) -> FastAPI:
app = FastAPI()
@app.post("/")
async def ask(request: Request) -> dict[str, str]:
return {"by": await require_token(request, self.runtime, scope=self.scope)}
return app
@pytest.fixture
async def runtime() -> GatewayRuntime:
root = Path(tempfile.mkdtemp(prefix="beaver-scopes-"))
db = Database(f"sqlite:///{root / 's.db'}")
await db.create_all()
speaker = Speaker()
rt = GatewayRuntime(
agents=AgentRegistry([]),
mcps=McpRegistry([]),
backends={},
token_store=TokenStore(
db,
bootstrap={"root": "root-key", "voice": "voice-key", "ops": "ops-key"},
bootstrap_scopes={"voice": "voice", "ops": "api"},
),
db=db,
frontends=(speaker,),
scopes=scopes_with(fe.scope for fe in (speaker,)),
)
speaker.configure(rt)
yield rt
await db.dispose()
def api(runtime: GatewayRuntime) -> httpx.AsyncClient:
return httpx.AsyncClient(
transport=httpx.ASGITransport(app=build_api(runtime)), base_url="http://t"
)
def frontend(runtime: GatewayRuntime) -> httpx.AsyncClient:
fe = runtime.frontends[0]
return httpx.AsyncClient(
transport=httpx.ASGITransport(app=fe.app()), base_url="http://t"
)
def test_a_frontend_scope_joins_the_builtin_vocabulary() -> None:
assert "voice" not in BUILTIN_SCOPES
assert scopes_with(["voice", None, "api"]) == BUILTIN_SCOPES | {"voice"}
async def test_the_scope_of_a_frontend_is_listed_with_its_name(
runtime: GatewayRuntime,
) -> None:
async with api(runtime) as client:
listed = await client.get("/scopes", headers=ROOT)
scopes = listed.json()["scopes"]
assert {"scope": "voice", "frontend": "voice"} in scopes
assert {"scope": "api", "frontend": None} in scopes
async def test_tokens_mint_for_a_declared_scope_and_not_for_an_unknown_one(
runtime: GatewayRuntime,
) -> None:
async with api(runtime) as client:
minted = await client.post(
"/tokens", json={"name": "esp32-hall", "scope": "voice"}, headers=ROOT
)
unknown = await client.post(
"/tokens", json={"name": "nope", "scope": "kitchen"}, headers=ROOT
)
assert minted.status_code == 201
assert minted.json()["token"]["scope"] == "voice"
assert unknown.status_code == 400
assert "kitchen" in unknown.json()["error"]
async def test_the_route_opens_for_its_own_scope_only(runtime: GatewayRuntime) -> None:
async with frontend(runtime) as client:
own = await client.post("/", headers=VOICE)
other = await client.post("/", headers=OPS)
wildcard = await client.post("/", headers=ROOT)
anonymous = await client.post("/")
assert own.status_code == 200
assert own.json()["by"] == "voice"
assert other.status_code == 403
assert wildcard.status_code == 200
assert anonymous.status_code == 401
async def test_a_minted_token_reaches_the_frontend_it_was_minted_for(
runtime: GatewayRuntime,
) -> None:
async with api(runtime) as client:
plaintext = (
await client.post(
"/tokens", json={"name": "esp32-hall", "scope": "voice"}, headers=ROOT
)
).json()["plaintext"]
async with frontend(runtime) as client:
opened = await client.post("/", headers={"X-Api-Key": plaintext})
assert opened.status_code == 200
assert opened.json()["by"] == "esp32-hall"
+3 -413
View File
@@ -1,11 +1,8 @@
import asyncio
import base64
import contextlib
import functools
import tempfile
from pathlib import Path
from types import SimpleNamespace
from typing import Any, cast
from typing import Any
import pytest
from aiogram.exceptions import TelegramBadRequest, TelegramNetworkError
@@ -15,12 +12,8 @@ from claude_agent_sdk import PermissionResultAllow, PermissionResultDeny
from beaver_gateway.app import McpRegistry
from beaver_gateway.backends.transcript import build_entries
from beaver_gateway.conversations.tools import _tools
from beaver_gateway.frontends.base import GatewayRuntime
from beaver_gateway.frontends.files import SendFileError, SendFiles, method_for
from beaver_gateway.frontends.telegram import Attachments, TelegramFrontend
from beaver_gateway.frontends.telegram.drafts import Draft
from beaver_gateway.frontends.telegram.texts import TelegramTexts
from beaver_gateway.frontends.telegram.render import (
LIMIT,
chunks,
@@ -40,8 +33,6 @@ class FakeBot:
def __init__(self) -> None:
self.updates: list[dict[str, Any]] = []
self.sent: list[dict[str, Any]] = []
self.files: list[dict[str, Any]] = []
self.reject_files = False
self.drafts: list[dict[str, Any]] = []
self.edits: list[dict[str, Any]] = []
self.topics: list[str] = []
@@ -138,21 +129,6 @@ class FakeBot:
async def download(self, file_id, destination=None) -> None:
Path(destination).write_bytes(b"data")
async def _upload(self, method: str, chat_id: int, **kwargs: Any) -> Any:
if self.reject_files:
raise TelegramBadRequest(
method=SendMessage(chat_id=0, text="x"),
message="Bad Request: PHOTO_INVALID_DIMENSIONS",
)
self._message_id += 1
self.files.append({"method": method, "chat_id": chat_id, **kwargs})
return SimpleNamespace(message_id=self._message_id)
def __getattr__(self, name: str) -> Any:
if name.startswith("send_"):
return functools.partial(self._upload, name.removeprefix("send_"))
raise AttributeError(name)
# helpers for tests
def push(self, payload: dict[str, Any]) -> None:
payload["update_id"] = 1000 + len(self.updates)
@@ -519,10 +495,10 @@ async def test_commands_status_merge_and_new(stack: Stack) -> None:
frontend="telegram", external_id=f"{USER}/11"
)
assert branch.status == "merged"
assert stack.bot.topics == ["🦫 General", "edit:11:work", "edit:11:✅ work"]
assert stack.bot.topics == ["🦫 General", "edit:11:✅ work"]
stack.bot.message("/new отчёт")
await stack.until(lambda: stack.sent_with("in a new topic"), what="new topic")
assert stack.bot.topics[-1] == "отчёт"
assert stack.bot.topics == ["🦫 General", "edit:11:✅ work", "отчёт"]
child = await stack.world.conversations.find_bound(
frontend="telegram", external_id=f"{USER}/902"
)
@@ -741,22 +717,6 @@ def test_chunks_split_blockquote_and_stay_under_limit() -> None:
assert " ".join(parts).split() == long_line.split()
def test_draft_breaks_a_paragraph_after_a_tool_call() -> None:
draft = Draft(cast("Any", None), chat_id=1, thread_id=None, turn_id="t")
draft.append("Checking the balance.")
draft.set_status("⏳ Bash")
draft.set_status("✍️ writing")
draft.append("It is 20 378.")
assert draft.text == "Checking the balance.\n\nIt is 20 378."
draft.set_status("⏳ Read")
draft.append("\n\nAnd a list:")
assert draft.text.endswith("It is 20 378.\n\nAnd a list:")
fresh = Draft(cast("Any", None), chat_id=1, thread_id=None, turn_id="u")
fresh.set_status("⏳ Bash")
fresh.append("First words")
assert fresh.text == "First words"
def test_to_html_tail_inside_fence() -> None:
full = (
"intro\n```py\n" + "\n".join(f"code {i} <" for i in range(200)) + "\n```\nafter"
@@ -959,373 +919,3 @@ async def test_gone_topic_unbinds_and_the_next_message_starts_afresh(
frontend="telegram", external_id=f"{USER}/7"
)
assert fresh is not None and fresh.id != branch.id and fresh.status == "open"
async def test_photo_reaches_the_model_as_an_image_block(stack: Stack) -> None:
stack.bot.media(_photo("что на фото"))
await stack.until(lambda: stack.sent_with("что на фото"), what="reply")
client = next(c for c in ScriptedClient.instances if c.contents)
content = client.contents[-1]
assert content[0]["type"] == "text" and "что на фото" in content[0]["text"]
image = content[1]
assert image["type"] == "image"
assert image["source"]["media_type"] == "image/jpeg"
assert base64.b64decode(image["source"]["data"]) == b"data"
async def test_album_becomes_one_turn(stack: Stack) -> None:
stack.tg.album_delay = 0.2
stack.bot.media({**_photo("альбом"), "media_group_id": "g1"})
second = _photo("")
second["photo"][0].update(file_id="f2", file_unique_id="u2")
del second["caption"]
stack.bot.media({**second, "media_group_id": "g1"})
await stack.until(lambda: stack.sent_with("альбом"), what="reply")
await asyncio.sleep(0.3)
prompts = [
p for c in ScriptedClient.instances for p in c.prompts if "attachment" in p
]
assert len(prompts) == 1
assert prompts[0].count("[attachment: photo") == 2
client = next(c for c in ScriptedClient.instances if c.contents)
assert [b["type"] for b in client.contents[-1]] == ["text", "image", "image"]
async def test_stop_button_interrupts_the_turn(stack: Stack) -> None:
stack.bot.message("hi")
await stack.until(lambda: stack.sent_with("hi"), what="reply")
ScriptedClient.hold = asyncio.Event()
stack.bot.message("думай долго")
await stack.until(lambda: stack.tg._drafts, what="draft") # noqa: SLF001
draft = next(iter(stack.tg._drafts.values())) # noqa: SLF001
await stack.until(
lambda: any(d.get("draft_id") == draft.draft_id for d in stack.bot.drafts),
what="draft pushed",
)
assert stack.bot.drafts[-1]["can_stop"] is True
stack.bot.push(
{
"stopped_message_generation": {
"chat": {"id": USER, "type": "private"},
"draft_id": draft.draft_id,
"message_thread_id": stack.bot.drafts[-1].get("message_thread_id"),
}
}
)
await stack.until(lambda: stack.sent_with("думай долго"), what="reply after stop")
master = await stack.world.conversations.find_bound(
frontend="telegram", external_id=GENERAL
)
items = await stack.world.conversations.queue.recent(master.id, limit=1)
assert items[0].status == "interrupted"
ScriptedClient.hold = None
async def test_question_survives_a_gateway_restart(stack: Stack) -> None:
stack.bot.message("hi")
await stack.until(lambda: stack.sent_with("hi"), what="reply")
master = await stack.world.conversations.find_bound(
frontend="telegram", external_id=GENERAL
)
payload = {
"questions": [
{
"header": "Цвет",
"question": "Какой цвет?",
"options": [{"label": "Синий"}, {"label": "Красный"}],
"multiSelect": False,
}
]
}
asking = asyncio.create_task(
stack.world.conversations.ask(master.external_id, payload)
)
question = await stack.until(
lambda: stack.sent_with("Какой цвет?"), what="question"
)
pending = stack.world.conversations.pending_question(master.external_id)
assert pending is not None
row = await stack.world.conversations.get(master.external_id)
assert row.flags["ask"]["id"] == pending[0]
asking.cancel()
with contextlib.suppress(asyncio.CancelledError):
await asking
stack.tg._asks.clear() # noqa: SLF001
stack.world.conversations._questions.clear() # noqa: SLF001
stack.bot.callback(f"q:{pending[0]}:0:1", question["message_id"])
await stack.until(lambda: stack.sent_with("ok:Красный"), what="answer as message")
row = await stack.world.conversations.get(master.external_id)
assert row.flags.get("ask") is None
def _sendable(name: str, body: bytes = b"x") -> tuple[Path, Path]:
root = Path(tempfile.mkdtemp(prefix="beaver-send-")).resolve()
file = root / name
file.write_bytes(body)
return root, file
def test_send_files_takes_only_paths_inside_its_roots() -> None:
root, note = _sendable("note.txt")
outside = Path(tempfile.mkdtemp(prefix="beaver-out-")).resolve()
(outside / "secret.txt").write_bytes(b"y")
(root / "escape.txt").symlink_to(outside / "secret.txt")
files = SendFiles(roots=(root,))
assert files.resolve(f"{root}/../{root.name}/./note.txt") == note
for bad in (
str(outside / "secret.txt"),
f"{root}/../{outside.name}/secret.txt",
str(root / "escape.txt"),
):
with pytest.raises(SendFileError, match="outside the allowed roots"):
files.resolve(bad)
with pytest.raises(SendFileError, match="is not a file"):
files.resolve(str(root / "gone.txt"))
with pytest.raises(SendFileError, match="outside the allowed roots"):
SendFiles().resolve(str(note))
def test_send_files_stops_a_file_over_the_limit() -> None:
root, big = _sendable("big.bin", b"0" * (2 * 1024 * 1024))
files = SendFiles(roots=(root,), max_bytes=1024 * 1024)
with pytest.raises(SendFileError, match="big.bin is 2 MB, over the 1 MB limit"):
files.resolve(str(big))
assert SendFiles(roots=(root,), max_bytes=4 * 1024 * 1024).resolve(str(big)) == big
assert files.note == f"paths under {root}, up to 1 MB"
def test_method_follows_the_extension_unless_it_is_given() -> None:
picked = {
name: method_for(Path(name))
for name in ("a.JPG", "b.gif", "c.mp3", "d.ogg", "e.mov", "f.zip", "g.svg")
}
assert picked == {
"a.JPG": "photo",
"b.gif": "animation",
"c.mp3": "audio",
"d.ogg": "voice",
"e.mov": "video",
"f.zip": "document",
"g.svg": "photo",
}
assert method_for(Path("a.jpg"), "document") == "document"
with pytest.raises(SendFileError, match="unknown method 'sticker'"):
method_for(Path("a.jpg"), "sticker")
async def _send(stack: Stack, conv, **args: Any) -> dict[str, Any]:
tool = next(
t
for t in _tools(stack.world.conversations, conv.external_id)
if t.name == "send_file"
)
return await tool.handler(args)
async def test_send_file_lands_in_the_topic_of_the_conversation(stack: Stack) -> None:
stack.bot.message("work", thread=11)
await stack.until(lambda: stack.sent_with("work"), what="branch reply")
branch = await stack.world.conversations.find_bound(
frontend="telegram", external_id=f"{USER}/11"
)
root, shot = _sendable("график.png", b"png")
stack.tg.send_files = SendFiles(roots=(root,))
result = await _send(stack, branch, path=str(shot), caption="**вот**")
assert result["content"][0]["text"].startswith("sent график.png as photo")
(sent,) = stack.bot.files
assert sent["method"] == "photo"
assert sent["message_thread_id"] == 11
assert sent["photo"].path == shot
assert sent["caption"] == "<b>вот</b>" and sent["parse_mode"] == "HTML"
async def test_long_caption_follows_the_file_as_a_message(stack: Stack) -> None:
stack.bot.message("hi")
await stack.until(lambda: stack.sent_with("hi"), what="reply")
master = await stack.world.conversations.find_bound(
frontend="telegram", external_id=GENERAL
)
root, doc = _sendable("отчёт.pdf")
stack.tg.send_files = SendFiles(roots=(root,))
await _send(stack, master, path=str(doc), caption="долго " * 300)
(sent,) = stack.bot.files
assert sent["method"] == "document" and "caption" not in sent
await stack.until(lambda: stack.sent_with("долго"), what="caption message")
async def test_send_file_without_a_window_falls_back_to_the_master(
stack: Stack,
) -> None:
stack.bot.message("hi")
await stack.until(lambda: stack.sent_with("hi"), what="reply")
job = await stack.world.conversations.spawn(
kind="job", agent="a", seed="brief", text="job", parent=None, origin="test"
)
root, song = _sendable("Пикник.mp3")
stack.tg.send_files = SendFiles(roots=(root,))
await _send(stack, job, path=str(song))
(sent,) = stack.bot.files
assert sent["method"] == "audio" and sent["title"] == "Пикник"
assert (sent["chat_id"], sent["message_thread_id"]) == (USER, 901)
async def test_a_refused_upload_comes_back_to_the_agent(stack: Stack) -> None:
stack.bot.message("hi")
await stack.until(lambda: stack.sent_with("hi"), what="reply")
master = await stack.world.conversations.find_bound(
frontend="telegram", external_id=GENERAL
)
root, shot = _sendable("bad.png")
stack.tg.send_files = SendFiles(roots=(root,))
stack.bot.reject_files = True
result = await _send(stack, master, path=str(shot))
assert result["is_error"]
assert "Telegram refused bad.png as photo" in result["content"][0]["text"]
outside = await _send(stack, master, path="/etc/passwd")
assert outside["is_error"]
assert "outside the allowed roots" in outside["content"][0]["text"]
def _progress_draft() -> Draft:
return Draft(cast("Any", None), chat_id=USER, thread_id=None, turn_id="t")
def _thinking_delta(text: str) -> Any:
return {
"event": {
"type": "content_block_delta",
"index": 0,
"delta": {"type": "thinking_delta", "thinking": text},
}
}
def test_progress_updates_reach_the_draft_but_hidden_reasoning_does_not() -> None:
draft = _progress_draft()
texts = TelegramTexts()
stub = SimpleNamespace(_drafts={"k": draft}, texts=texts)
feed = functools.partial(
TelegramFrontend._feed_draft, cast("Any", stub), "k", thinking_is_progress=True
)
feed(_thinking_delta(""))
assert draft.text == ""
assert draft.status == texts.thinking
feed(_thinking_delta("checking the vault"))
assert draft.text == "checking the vault"
assert draft.status == texts.writing
def test_reasoning_rides_under_the_status_and_never_reaches_the_text() -> None:
draft = _progress_draft()
texts = TelegramTexts()
stub = SimpleNamespace(_drafts={"k": draft}, texts=texts)
feed = functools.partial(
TelegramFrontend._feed_draft, cast("Any", stub), "k", thinking_is_progress=False
)
feed(_thinking_delta("weighing <it> "))
feed(_thinking_delta("against the note"))
assert draft.text == ""
assert draft.status == texts.thinking
assert draft._render() == (
f"{texts.thinking}\n\n"
"<blockquote>weighing &lt;it&gt; against the note</blockquote>",
f"{texts.thinking}\n\nweighing <it> against the note",
)
feed(
{
"event": {
"type": "content_block_start",
"index": 2,
"content_block": {"type": "thinking"},
}
}
)
assert draft.reasoning == ""
feed(_thinking_delta("word " * 200))
shown = draft._render()[1].removeprefix(f"{texts.thinking}\n\n")
assert shown.startswith("… word")
assert len(shown) <= 402
feed(
{
"event": {
"type": "content_block_delta",
"index": 3,
"delta": {"type": "text_delta", "text": "Done."},
}
}
)
assert draft.reasoning == ""
assert draft._render()[1] == f"{texts.writing}\n\nDone."
def test_a_tool_call_between_progress_updates_starts_a_new_paragraph() -> None:
draft = _progress_draft()
texts = TelegramTexts()
stub = SimpleNamespace(
_drafts={"k": draft},
texts=texts,
_tool_status=lambda name, _input: texts.tool.format(label=name),
)
feed = functools.partial(
TelegramFrontend._feed_draft, cast("Any", stub), "k", thinking_is_progress=True
)
feed(_thinking_delta("reading the note"))
feed(
{
"event": {
"type": "content_block_start",
"index": 1,
"content_block": {"type": "tool_use", "name": "Read"},
}
}
)
feed(_thinking_delta("now writing it back"))
assert draft.text == "reading the note\n\nnow writing it back"
async def test_a_closed_topic_is_marked_asleep_and_wakes_with_a_new_branch(
stack: Stack,
) -> None:
stack.bot.message("кухня на неделю\nи ещё строка", thread=12)
await stack.until(lambda: stack.sent_with("кухня на неделю"), what="reply")
assert stack.bot.topics[-1] == "edit:12:кухня на неделю"
branch = await stack.world.conversations.find_bound(
frontend="telegram", external_id=f"{USER}/12"
)
closed = await stack.world.conversations.close(branch)
assert await stack.world.conversations.mark_closed(closed)
assert stack.bot.topics[-1] == "edit:12:💤 кухня на неделю"
stack.tg._topic_names.clear() # noqa: SLF001
stack.bot.message("а в четверг?", thread=12)
await stack.until(lambda: stack.sent_with("а в четверг?"), what="new branch")
assert stack.bot.topics[-1] == "edit:12:кухня на неделю"
fresh = await stack.world.conversations.find_bound(
frontend="telegram", external_id=f"{USER}/12"
)
assert fresh.id != branch.id
assert fresh.status == "open"
assert fresh.title == "кухня на неделю"
async def test_a_named_topic_keeps_its_name(stack: Stack) -> None:
stack.bot.topic_created("план", 13)
stack.bot.message("hello", thread=13)
await stack.until(lambda: stack.sent_with("hello"), what="reply")
assert not any(t.startswith("edit:13:") for t in stack.bot.topics)
-63
View File
@@ -1,63 +0,0 @@
import tempfile
from collections.abc import AsyncIterator
from pathlib import Path
import pytest
from watchfiles import Change as FileChange
from beaver_gateway.vault import watch as watch_module
from beaver_gateway.vault.watch import VaultFilter, VaultWatch, WatchRules
VAULT = Path("/vault")
FILTER = VaultFilter()
def wakes(rel: str) -> bool:
return FILTER(FileChange.modified, str(VAULT / rel))
def test_obsidian_never_wakes_the_watcher() -> None:
assert not wakes(".obsidian/.sync.lock")
assert not wakes(".obsidian/.sync.lock/data")
assert not wakes(".obsidian")
assert not wakes(".obsidian/workspace.json")
assert not wakes(".obsidian/plugins/obsidian-git/data.json")
def test_other_service_paths_never_wake_the_watcher() -> None:
assert not wakes(".trash/📅 дни/2026-09-01.md")
assert not wakes(".git/index")
def test_notes_still_wake_the_watcher() -> None:
assert wakes("📅 дни/2026-09-09.md")
assert wakes("мета/бобер/состояние.md")
assert wakes("заметка.md")
assert wakes("💻 проекты/бобер/план.md")
def test_lock_beside_a_note_is_quiet_but_the_note_is_not() -> None:
assert not wakes("📅 дни/.2026-09-09.md.lock")
assert not wakes("📅 дни/2026-09-09.md.tmp")
assert not wakes("📅 дни/2026-09-09.md~")
assert wakes("📅 дни/2026-09-09.md")
async def test_the_filter_is_applied_by_awatch_itself(
monkeypatch: pytest.MonkeyPatch,
) -> None:
seen: dict[str, object] = {}
async def fake_awatch(
*paths: object, **kwargs: object
) -> AsyncIterator[set[object]]:
seen.update(kwargs)
seen["paths"] = paths
for change in ():
yield change
monkeypatch.setattr(watch_module, "awatch", fake_awatch)
root = Path(tempfile.mkdtemp(prefix="beaver-vault-"))
watch = VaultWatch(root, WatchRules(names=("**",)), tz="UTC")
await watch.run()
assert isinstance(seen["watch_filter"], VaultFilter)
-12
View File
@@ -5,7 +5,6 @@
"": {
"name": "ui",
"dependencies": {
"d3-force": "^3.0.0",
"dompurify": "^3.4.14",
"marked": "^18.0.11",
},
@@ -20,7 +19,6 @@
"@tailwindcss/forms": "^0.5.11",
"@tailwindcss/typography": "^0.5.19",
"@tailwindcss/vite": "^4.3.0",
"@types/d3-force": "^3.0.10",
"@types/node": "^26.4.0",
"bits-ui": "^2.19.0",
"clsx": "^2.1.1",
@@ -188,8 +186,6 @@
"@types/cookie": ["@types/cookie@0.6.0", "", {}, "sha512-4Kh9a6B2bQciAhf7FSuMRRkUWecJgJu9nPnx3yzpsfXX/c50REIqpHY4C82bXP90qrLtXtkDxTZosYO3UpOwlA=="],
"@types/d3-force": ["@types/d3-force@3.0.10", "", {}, "sha512-ZYeSaCF3p73RdOKcjj+swRlZfnYpK1EbaDiYICEEp5Q6sUiqFaFQ9qgoshp5CzIyyb/yD09kD9o2zEltCexlgw=="],
"@types/estree": ["@types/estree@1.0.9", "", {}, "sha512-GhdPgy1el4/ImP05X05Uw4cw2/M93BCUmnEvWZNStlCzEKME4Fkk+YpoA5OiHNQmoS7Cafb8Xa3Pya8m1Qrzeg=="],
"@types/node": ["@types/node@26.4.0", "", { "dependencies": { "undici-types": "~8.3.0" } }, "sha512-faiGnoIrLH/V8cibOMEAZ8pMw6oXqSukl29ra4mN8GdaB2ZewzeaLj+INpV5N+Z1eKWzY+IzaIZH2EIR6YZRNQ=="],
@@ -230,14 +226,6 @@
"cssesc": ["cssesc@3.0.0", "", { "bin": { "cssesc": "bin/cssesc" } }, "sha512-/Tb/JcjK111nNScGob5MNtsntNM1aCNUDipB/TkwZFhyDrrE47SOx/18wF2bbjgc3ZzCSKW1T5nt5EbFoAz/Vg=="],
"d3-dispatch": ["d3-dispatch@3.0.1", "", {}, "sha512-rzUyPU/S7rwUflMyLc1ETDeBj0NRuHKKAcvukozwhshr6g6c5d8zh4c2gQjY2bZ0dXeGLWc1PF174P2tVvKhfg=="],
"d3-force": ["d3-force@3.0.0", "", { "dependencies": { "d3-dispatch": "1 - 3", "d3-quadtree": "1 - 3", "d3-timer": "1 - 3" } }, "sha512-zxV/SsA+U4yte8051P4ECydjD/S+qeYtnaIyAs9tgHCqfguma/aAQDjo85A9Z6EKhBirHRJHXIgJUlffT4wdLg=="],
"d3-quadtree": ["d3-quadtree@3.0.1", "", {}, "sha512-04xDrxQTDTCFwP5H6hRhsRcb9xxv2RzkcsygFzmkSIOJy3PeRJP7sNk3VRIbKXcog561P9oU0/rVH6vDROAgUw=="],
"d3-timer": ["d3-timer@3.0.1", "", {}, "sha512-ndfJ/JxxMd3nw31uyKoY2naivF+r29V+Lc0svZxe1JvvIRmi8hUsrMvdOwgS1o6uBHmiz91geQ0ylPP0aj1VUA=="],
"deepmerge": ["deepmerge@4.3.1", "", {}, "sha512-3sUqbMEc77XqpdNO7FRyRog+eW3ph+GYCbj+rK+uYyRMuwsVy0rMiVtPn+QJlKFvWP/1PYpapqYn0Me2knFn+A=="],
"dequal": ["dequal@2.0.3", "", {}, "sha512-0je+qPKHEMohvfRTCEo3CrPG6cAzAYgmzKyxRiYSSDkS6eGJdyVJm7WaYA5ECaAD9wLB2T4EEeymA5aFVcYXCA=="],
-2
View File
@@ -14,7 +14,6 @@
"preview": "vite preview"
},
"dependencies": {
"d3-force": "^3.0.0",
"dompurify": "^3.4.14",
"marked": "^18.0.11"
},
@@ -29,7 +28,6 @@
"@tailwindcss/forms": "^0.5.11",
"@tailwindcss/typography": "^0.5.19",
"@tailwindcss/vite": "^4.3.0",
"@types/d3-force": "^3.0.10",
"@types/node": "^26.4.0",
"bits-ui": "^2.19.0",
"clsx": "^2.1.1",
+12 -15
View File
@@ -10,21 +10,18 @@
</head>
<body data-sveltekit-preload-data="hover">
<!--
impeccable direction contract (seed 071a8c77, reroll 1 bolder, mode operate, user-steered fusion)
THESIS: the dispatcher's strip board under a live island. What is alive is a strip in a bay
(waiting for you, in motion, quiet); the island at the top of every screen carries the live
state and opens into the board. Refuses the sidebar-and-cards admin dashboard.
OWN-WORLD: plum signal on a warm rack ground; strips as white/aubergine objects with a state
edge (plum pulse = running, amber = waiting), fixed columns, hairline rules, tabular numerals,
system sans; amber only for what waits for the operator; no cards inside cards, no badges as
pills, no decorative motion.
STORY: the operator lands on the board, answers a question on its strip, watches a running
strip, opens a thread where only the thread lives, reads context as a counter, finds anything
with ⌘K, and reaches the text-heavy rooms (memory, usage, system) as wide documents.
FIRST VIEWPORT: top bar with four section words left, the island centered (pulse, questions,
context, quota), System right. Below: the instrument row (context / 5 h / 7 d / spend), then
the bays with strips, the living graph of the master's neighbourhood on the right.
FORM: strip board × live activity; user-steered over the dealt hand; seed key 071a8c77.
impeccable direction contract (seed 20df1617, mode operate, brief-pinned world)
THESIS: activity first. The console opens on what the agents are doing right now, as a live
ledger of turns and tool calls, and refuses the hero-metric dashboard grid of cards.
OWN-WORLD: msos mauve/pink tokens on Inter; two neutral layers (cooler sidebar, content surface);
one accent for selection and the live state; kind hues from the msos status palette; hairline
borders, tabular numerals, rows and rails instead of cards; no cards inside cards.
STORY: the operator lands, sees what runs and how much quota is left, opens a thread, watches
tools and subagents stream, answers a question, checks spend, manages tokens and memory.
FIRST VIEWPORT: sidebar left with the gateway connection dot; main opens with the "now" ledger
(running turns as live rows), then quota bars and the 5 h / 7 d spend row, then live sessions.
Primary action: open a running conversation.
FORM: dense operator console, brief-pinned (roll assigned index 7, superseded by the pin).
FINISH: unreviewed and undocumented is unfinished; this build ends with the finish review,
the verdict, and DESIGN.md
-->
+2 -80
View File
@@ -3,7 +3,6 @@ import type {
AgentsResponse,
AuditPage,
BusEvent,
ContextResponse,
ConversationInfo,
ConversationSummary,
EntriesPage,
@@ -12,17 +11,10 @@ import type {
LimitsResponse,
MemoryFile,
MemoryTree,
ScopeRow,
SearchResponse,
SecretRow,
SecretsResponse,
SecretWrite,
SessionsResponse,
TokenRow,
TurnSummary,
UsageGroup,
UsageResponse,
VaultGraph,
} from "./types";
const TRAILING_SLASHES = /\/+$/;
@@ -150,19 +142,10 @@ export class ApiClient {
return this.request<T>("PATCH", path, body ?? {});
}
put<T>(path: string, body?: unknown): Promise<T> {
return this.request<T>("PUT", path, body ?? {});
}
delete<T>(path: string): Promise<T> {
return this.request<T>("DELETE", path);
}
async stream(
path: string,
onEvent: (event: BusEvent) => void,
signal: AbortSignal,
onActivity?: () => void
signal: AbortSignal
): Promise<void> {
const response = await fetch(this.url(path), {
headers: this.headers({ Accept: "text/event-stream" }),
@@ -176,7 +159,7 @@ export class ApiClient {
payload
);
}
await readSse(response, onEvent, signal, onActivity);
await readSse(response, onEvent, signal);
}
agents(): Promise<AgentsResponse> {
@@ -199,31 +182,6 @@ export class ApiClient {
return this.get(`/api/conversations/${id}/history`);
}
turns(id: string, limit: number): Promise<{ turns: TurnSummary[] }> {
return this.get(`/api/conversations/${id}/turns`, { limit });
}
rewind(id: string, turn: string): Promise<{ turns: number }> {
return this.post(`/api/conversations/${id}/rewind`, { turn });
}
context(id: string, prompt = false): Promise<ContextResponse> {
return this.get(`/api/conversations/${id}/context`, {
prompt: prompt ? 1 : undefined,
});
}
vaultGraph(paths: string[], limit?: number): Promise<VaultGraph> {
const search = new URLSearchParams();
for (const path of paths) {
search.append("path", path);
}
if (limit) {
search.set("limit", String(limit));
}
return this.get(`/api/vault/graph?${search.toString()}`);
}
entries(
id: string,
params?: { subpath?: string; offset?: number; limit?: number }
@@ -325,30 +283,10 @@ export class ApiClient {
return this.post("/api/conversations", body);
}
search(q: string, limit?: number): Promise<SearchResponse> {
return this.get("/api/search", { limit, q });
}
sessions(): Promise<SessionsResponse> {
return this.get("/api/sessions");
}
sessionStderr(key: string): Promise<{ key: string; lines: string[] }> {
return this.get(`/api/sessions/${encodeURIComponent(key)}/stderr`);
}
interruptSession(key: string): Promise<{ interrupted: boolean }> {
return this.post(`/api/sessions/${encodeURIComponent(key)}/interrupt`);
}
closeSession(key: string): Promise<{ closed: boolean }> {
return this.post(`/api/sessions/${encodeURIComponent(key)}/close`);
}
killProcess(pid: number): Promise<{ killed: boolean }> {
return this.post(`/api/processes/${pid}/kill`);
}
jobs(): Promise<JobsResponse> {
return this.get("/api/jobs");
}
@@ -388,10 +326,6 @@ export class ApiClient {
});
}
scopes(): Promise<{ scopes: ScopeRow[] }> {
return this.get("/api/scopes");
}
createToken(
name: string,
scope: string
@@ -403,18 +337,6 @@ export class ApiClient {
return this.post(`/api/tokens/${id}/revoke`);
}
secrets(): Promise<SecretsResponse> {
return this.get("/api/secrets");
}
putSecret(name: string, body: SecretWrite): Promise<{ secret: SecretRow }> {
return this.put(`/api/secrets/${encodeURIComponent(name)}`, body);
}
deleteSecret(name: string): Promise<{ name: string; deleted: boolean }> {
return this.delete(`/api/secrets/${encodeURIComponent(name)}`);
}
audit(params?: { before?: number; limit?: number }): Promise<AuditPage> {
return this.get("/api/audit", params);
}
+7 -101
View File
@@ -6,11 +6,6 @@ export type LiveState = "off" | "connecting" | "open" | "retrying" | "failed";
const BACKOFF_MIN_MS = 1000;
const BACKOFF_MAX_MS = 30_000;
const JITTER_MS = 250;
// The server sends a keepalive comment every 15s, so twice that of silence
// means the socket is dead even though nothing errored - which is exactly
// what a phone brings back from sleep: a stream that reads as open forever.
const STALE_AFTER_MS = 40_000;
const WATCH_EVERY_MS = 5000;
const FATAL = new Set([401, 403, 404]);
function sleep(ms: number, signal: AbortSignal): Promise<void> {
@@ -46,13 +41,9 @@ export class LiveStream {
state = $state<LiveState>("off");
detail = $state<string | null>(null);
private controller: AbortController | null = null;
private inner: AbortController | null = null;
private readonly client: () => ApiClient | null;
private readonly path: string;
private readonly options: LiveOptions;
private seen = 0;
private watcher: ReturnType<typeof setInterval> | null = null;
private wake: (() => void) | null = null;
constructor(
client: () => ApiClient | null,
@@ -74,7 +65,6 @@ export class LiveStream {
}
const controller = new AbortController();
this.controller = controller;
this.watch();
this.loop(controller.signal).catch((error: unknown) => {
this.state = "failed";
this.detail = describe(error);
@@ -82,108 +72,31 @@ export class LiveStream {
}
stop(): void {
this.unwatch();
this.controller?.abort();
this.controller = null;
this.inner = null;
this.state = "off";
}
/**
* Reconnect now: the operator tapped the dot, or the app came back to the
* foreground. A failed stream has no loop left to nudge, so it restarts.
*/
refresh(): void {
if (!this.controller || this.state === "failed") {
this.stop();
this.start();
return;
}
this.state = "connecting";
this.detail = null;
this.inner?.abort();
this.wake?.();
}
/**
* A stream that stopped delivering without erroring - the phone slept, the
* network changed under it - reads as open forever. Silence past the
* server's keepalive is the only tell, so the socket is dropped and the
* loop reconnects.
*/
private watch(): void {
if (this.watcher || typeof setInterval === "undefined") {
return;
}
this.watcher = setInterval(() => {
if (this.state !== "open" || Date.now() - this.seen < STALE_AFTER_MS) {
return;
}
this.state = "retrying";
this.detail = "stream went quiet";
this.inner?.abort();
}, WATCH_EVERY_MS);
}
private unwatch(): void {
if (this.watcher) {
clearInterval(this.watcher);
this.watcher = null;
}
}
private async loop(signal: AbortSignal): Promise<void> {
let delay = BACKOFF_MIN_MS;
while (!signal.aborted) {
this.state = delay === BACKOFF_MIN_MS ? "connecting" : "retrying";
// Each attempt gets its own controller, so the watchdog (and a manual
// refresh) can drop one socket without ending the loop.
const inner = new AbortController();
this.inner = inner;
const relay = () => inner.abort();
signal.addEventListener("abort", relay, { once: true });
// biome-ignore lint/performance/noAwaitInLoops: one connection at a time, by design
const outcome = await this.connectOnce(inner.signal, () => {
const outcome = await this.connectOnce(signal, () => {
delay = BACKOFF_MIN_MS;
});
signal.removeEventListener("abort", relay);
this.inner = null;
if (signal.aborted || outcome === "fatal") {
return;
}
// A socket dropped on purpose reconnects at once; a broken one waits.
const wait = outcome === "now" ? 0 : delay + Math.random() * JITTER_MS;
await this.pause(wait, signal);
delay =
outcome === "now"
? BACKOFF_MIN_MS
: Math.min(delay * 2, BACKOFF_MAX_MS);
await sleep(delay + Math.random() * JITTER_MS, signal);
delay = Math.min(delay * 2, BACKOFF_MAX_MS);
}
}
/** Backoff that a manual refresh can cut short. */
private pause(ms: number, signal: AbortSignal): Promise<void> {
if (ms <= 0) {
return Promise.resolve();
}
return new Promise((resolve) => {
const done = () => {
this.wake = null;
resolve();
};
this.wake = done;
sleep(ms, signal).then(() => {
if (this.wake === done) {
done();
}
}, done);
});
}
private async connectOnce(
signal: AbortSignal,
onOpen: () => void
): Promise<"retry" | "fatal" | "now"> {
): Promise<"retry" | "fatal"> {
try {
const client = this.client();
if (!client) {
@@ -191,9 +104,8 @@ export class LiveStream {
}
await this.options.prepare?.();
if (signal.aborted) {
return this.controller ? "now" : "fatal";
return "fatal";
}
this.seen = Date.now();
await client.stream(
this.path,
(event) => {
@@ -205,25 +117,19 @@ export class LiveStream {
}
this.options.onEvent(event);
},
signal,
() => {
this.seen = Date.now();
}
signal
);
this.state = "retrying";
this.detail = "stream closed";
return "retry";
} catch (error) {
if (signal.aborted) {
// Aborted while the session lives on: this was the watchdog or a
// manual refresh, so reconnect without waiting out the backoff.
return this.controller ? "now" : "fatal";
return "fatal";
}
if (error instanceof ApiError && FATAL.has(error.status)) {
this.state = "failed";
this.detail = `${error.status}: ${error.message}`;
this.controller = null;
this.unwatch();
return "fatal";
}
this.state = "retrying";
+1 -5
View File
@@ -38,10 +38,7 @@ function drain(buffer: string, onEvent: (event: BusEvent) => void): string {
export async function readSse(
response: Response,
onEvent: (event: BusEvent) => void,
signal?: AbortSignal,
// Every chunk the socket delivers, keepalive comments included: proof the
// connection is still alive on a phone that just came back from sleep.
onActivity?: () => void
signal?: AbortSignal
): Promise<void> {
const { body } = response;
if (!body) {
@@ -61,7 +58,6 @@ export async function readSse(
if (done) {
break;
}
onActivity?.();
buffer = drain(buffer + decoder.decode(value, { stream: true }), onEvent);
}
} finally {
-136
View File
@@ -54,7 +54,6 @@ export interface QueueItem {
export interface ConversationSummary {
agent: string;
context_tokens?: number;
created_at: string | null;
flags: Record<string, unknown>;
id: string;
@@ -63,7 +62,6 @@ export interface ConversationSummary {
last_item?: QueueItem | null;
last_user_activity_at: string | null;
origin: string;
parent?: string | null;
parent_row: number | null;
pending_question: boolean;
running_turn: string | null;
@@ -79,7 +77,6 @@ export interface ConversationInfo extends ConversationSummary {
parent: string | null;
question: PendingQuestion | null;
queue: QueueItem[];
thinking_is_progress?: boolean;
turn: TurnSnapshot | null;
}
@@ -106,19 +103,7 @@ export interface ContentBlock {
export interface HistoryMessage {
content: string | ContentBlock[];
id?: string | null;
role: "user" | "assistant";
service?: string;
// When the first block of this message was written; absent for old rows.
ts?: string | null;
}
export interface TurnSummary {
failed: boolean;
id: string;
reply: string;
text: string;
ts: string | null;
}
export interface EntriesPage {
@@ -143,7 +128,6 @@ export interface FrontendInfo {
kinds: Kind[];
name: string;
path: string | null;
scope: string | null;
type: string;
url: string | null;
}
@@ -154,31 +138,17 @@ export interface AgentsResponse {
mcps: { name: string; kind: string }[];
}
export interface ProcessRow {
age_seconds: number;
command: string;
cpu: number | null;
depth: number;
name: string;
parent: number;
pid: number;
rss: number;
}
export interface SessionRow {
age_seconds: number;
agent: string;
busy: boolean;
cpu: number | null;
dirty: boolean;
idle_seconds: number;
key: string;
kind: string;
pending: number;
pending_question: boolean;
pid: number | null;
pinned: boolean;
processes: ProcessRow[];
rss: number | null;
running_turn: string | null;
session_id: string | null;
@@ -186,8 +156,6 @@ export interface SessionRow {
}
export interface SessionsResponse {
cpu: number | null;
others: ProcessRow[];
rss: number | null;
rss_limit: number | null;
sessions: SessionRow[];
@@ -241,10 +209,6 @@ export interface LimitRecord {
export interface LimitWindow extends LimitRecord {
gateway: UsageTotals & { since: string };
// The last figure reported inside this window, if any: a lower bound.
last_known: LimitRecord | null;
// When the figure came from another event's unified report.
reported_at?: string | null;
}
export interface LimitsResponse {
@@ -273,11 +237,6 @@ export interface MemoryFile {
size: number;
}
export interface ScopeRow {
frontend: string | null;
scope: string;
}
export interface TokenRow {
created_at: string;
id: number;
@@ -287,28 +246,6 @@ export interface TokenRow {
scope: string;
}
export interface SecretRow {
created_at: string;
description: string;
filename: string | null;
kind: "file" | "value";
name: string;
size: number;
updated_at: string;
used_at: string | null;
}
export interface SecretsResponse {
enabled: boolean;
secrets: SecretRow[];
}
export interface SecretWrite {
description?: string;
file?: { content: string; name: string };
value?: string;
}
export interface AuditRecord {
actor: string;
agent: string | null;
@@ -373,76 +310,3 @@ export interface TurnUsage {
input?: number;
output?: number;
}
export interface SearchFile {
line: number;
path: string;
snippet: string;
}
export interface SearchResponse {
conversations: ConversationSummary[];
files: SearchFile[];
query: string;
}
export interface Granule {
bytes: number;
path: string | null;
tag: string | null;
tokens_est: number;
}
export interface SkillInfo {
description: string;
name: string;
path: string;
}
export interface SkillSetInfo {
path: string;
set: string;
skills: SkillInfo[];
}
export interface TouchedFile {
last_at: string;
other: number;
path: string;
reads: number;
writes: number;
}
export interface ContextResponse {
agent: { effort: string | null; model: string; name: string };
context_tokens: number;
files: TouchedFile[];
history_tokens_est: number;
id: string;
kind: Kind;
prompt: { bytes: number; granules: Granule[]; tokens_est: number };
prompt_text?: string;
skills: SkillSetInfo[];
skills_tokens_est: number;
tool_counts: Record<string, number>;
tools: {
allowed: string[] | null;
disallowed: string[];
gateway: string[];
mcps: string[];
};
turns: number;
}
export interface VaultNode {
exists: boolean;
path: string;
title: string;
touched: boolean;
}
export interface VaultGraph {
edges: { from: string; to: string }[];
nodes: VaultNode[];
root: string;
}
+121
View File
@@ -0,0 +1,121 @@
<script lang="ts">
import LogOutIcon from "@lucide/svelte/icons/log-out";
import PanelLeftCloseIcon from "@lucide/svelte/icons/panel-left-close";
import PanelLeftOpenIcon from "@lucide/svelte/icons/panel-left-open";
import { goto } from "$app/navigation";
import { base } from "$app/paths";
import { page } from "$app/state";
import LiveDot from "$lib/components/live-dot.svelte";
import ThemeToggle from "$lib/components/theme-toggle.svelte";
import { Button } from "$lib/components/ui/button";
import { gateway } from "$lib/gateway.svelte";
import { isActive, NAV } from "$lib/nav";
import { session } from "$lib/session.svelte";
import { ui } from "$lib/ui.svelte";
import { cn } from "$lib/utils";
const running = $derived(gateway.running.length);
const open = $derived(ui.nav);
async function signOut() {
await session.logout();
await goto(`${base}/login`);
}
</script>
<aside
class={cn(
"hidden shrink-0 flex-col border-r bg-sidebar text-sidebar-foreground transition-[width] duration-150 sm:flex",
open ? "w-52" : "w-12"
)}
>
<a
class={cn(
"flex h-12 items-center gap-2 border-b font-semibold tracking-tight",
open ? "px-4" : "justify-center"
)}
href="{base}/"
title="Beaver"
>
<span class="size-2.5 shrink-0 rounded-sm bg-primary"></span>
{#if open}
Beaver
{/if}
</a>
<nav aria-label="Sections" class="flex flex-1 flex-col gap-0.5 p-2">
{#each NAV as item (item.href)}
{@const active = isActive(page.url.pathname, base, item.href)}
<a
aria-current={active ? "page" : undefined}
aria-label={item.label}
class={cn(
"relative flex h-8 items-center gap-2.5 rounded-md text-sm transition-colors",
open ? "px-2.5" : "justify-center",
active
? "bg-sidebar-accent font-medium text-sidebar-accent-foreground"
: "text-sidebar-foreground/80 hover:bg-sidebar-accent/60 hover:text-sidebar-foreground"
)}
href="{base}{item.href}"
title={open ? undefined : item.label}
>
<item.icon class="size-4 shrink-0 text-icon" />
{#if open}
<span class="flex-1">{item.label}</span>
{/if}
{#if item.href === "/" && running > 0}
{#if open}
<span
class="tabular rounded-full bg-signal/12 px-1.5 font-medium text-signal text-xs"
>
{running}
</span>
{:else}
<span
class="absolute top-1 right-1 size-1.5 rounded-full bg-signal"
></span>
{/if}
{/if}
</a>
{/each}
</nav>
<div
class={cn(
"flex items-center gap-1 border-t",
open ? "px-3 py-2" : "flex-col px-1 py-2"
)}
>
<span
class={cn(open && "mr-auto")}
title="event stream from the gateway (SSE)"
>
<LiveDot
detail={gateway.live.detail}
label={open}
state={gateway.live.state}
/>
</span>
<ThemeToggle />
<Button
aria-label="Sign out"
onclick={signOut}
size="icon-sm"
title="Sign out"
variant="ghost"
>
<LogOutIcon class="size-4" />
</Button>
<Button
aria-label={open ? "Collapse sidebar" : "Expand sidebar"}
onclick={() => ui.toggleNav()}
size="icon-sm"
title="{open ? 'Collapse' : 'Expand'} sidebar (⌘B)"
variant="ghost"
>
{#if open}
<PanelLeftCloseIcon class="size-4" />
{:else}
<PanelLeftOpenIcon class="size-4" />
{/if}
</Button>
</div>
</aside>
+60 -23
View File
@@ -1,42 +1,79 @@
<script lang="ts">
import ActivityIcon from "@lucide/svelte/icons/activity";
import BrainIcon from "@lucide/svelte/icons/brain";
import GaugeIcon from "@lucide/svelte/icons/gauge";
import MessagesSquareIcon from "@lucide/svelte/icons/messages-square";
import SlidersHorizontalIcon from "@lucide/svelte/icons/sliders-horizontal";
import type { Component } from "svelte";
import MoreHorizontalIcon from "@lucide/svelte/icons/more-horizontal";
import { goto } from "$app/navigation";
import { base } from "$app/paths";
import { page } from "$app/state";
import { isSectionActive, SECTIONS, SYSTEM } from "$lib/nav";
import LiveDot from "$lib/components/live-dot.svelte";
import ThemeToggle from "$lib/components/theme-toggle.svelte";
import { Button } from "$lib/components/ui/button";
import * as Sheet from "$lib/components/ui/sheet";
import { gateway } from "$lib/gateway.svelte";
import { isActive, NAV } from "$lib/nav";
import { session } from "$lib/session.svelte";
import { cn } from "$lib/utils";
const ICONS: Record<string, Component<{ class?: string }>> = {
"/": ActivityIcon,
"/conversations": MessagesSquareIcon,
"/memory": BrainIcon,
"/system": SlidersHorizontalIcon,
"/usage": GaugeIcon,
};
const items = [...SECTIONS, SYSTEM];
let moreOpen = $state(false);
const primary = NAV.filter((item) => item.mobile);
const secondary = NAV.filter((item) => !item.mobile);
async function signOut() {
moreOpen = false;
await session.logout();
await goto(`${base}/login`);
}
</script>
<nav
aria-label="Sections"
class="flex shrink-0 items-stretch border-t bg-rack/90 pb-[env(safe-area-inset-bottom)] backdrop-blur-md sm:hidden"
class="flex shrink-0 items-stretch border-t bg-sidebar pb-[env(safe-area-inset-bottom)] sm:hidden"
>
{#each items as item (item.href)}
{@const active = isSectionActive(page.url.pathname, base, item.href)}
{@const Icon = ICONS[item.href]}
{#each primary as item (item.href)}
{@const active = isActive(page.url.pathname, base, item.href)}
<a
aria-current={active ? "page" : undefined}
class={cn(
"flex h-13 flex-1 flex-col items-center justify-center gap-0.5 text-[10px] transition-colors",
active ? "text-primary" : "text-muted-foreground"
"flex h-14 flex-1 flex-col items-center justify-center gap-1 text-[11px]",
active ? "text-link" : "text-muted-foreground"
)}
href="{base}{item.href}"
>
<Icon class="size-5" />
{item.label.split(" ")[0]}
<item.icon class="size-5" />
{item.label}
</a>
{/each}
<button
class="flex h-14 flex-1 flex-col items-center justify-center gap-1 text-[11px] text-muted-foreground"
onclick={() => {
moreOpen = true;
}}
type="button"
>
<MoreHorizontalIcon class="size-5" />
More
</button>
</nav>
<Sheet.Root bind:open={moreOpen}>
<Sheet.Content class="flex flex-col gap-1 pt-10" side="bottom">
<Sheet.Title class="px-2 pb-2">More</Sheet.Title>
{#each secondary as item (item.href)}
<a
class="flex h-11 items-center gap-3 rounded-md px-3 text-sm hover:bg-muted"
href="{base}{item.href}"
onclick={() => {
moreOpen = false;
}}
>
<item.icon class="size-4 text-icon" />
{item.label}
</a>
{/each}
<div class="mt-2 flex items-center justify-between border-t px-3 pt-3">
<LiveDot detail={gateway.live.detail} state={gateway.live.state} />
<div class="flex items-center gap-1">
<ThemeToggle />
<Button onclick={signOut} size="sm" variant="ghost">Sign out</Button>
</div>
</div>
</Sheet.Content>
</Sheet.Root>
@@ -0,0 +1,323 @@
<script lang="ts">
import PanelLeftCloseIcon from "@lucide/svelte/icons/panel-left-close";
import PlusIcon from "@lucide/svelte/icons/plus";
import { toast } from "svelte-sonner";
import { goto } from "$app/navigation";
import { base } from "$app/paths";
import type { AgentInfo, ConversationSummary } from "$lib/api/types";
import EmptyState from "$lib/components/empty-state.svelte";
import ErrorNote from "$lib/components/error-note.svelte";
import KindBadge from "$lib/components/kind-badge.svelte";
import StatusPill from "$lib/components/status-pill.svelte";
import { Button } from "$lib/components/ui/button";
import * as Dialog from "$lib/components/ui/dialog";
import { Input } from "$lib/components/ui/input";
import { Label } from "$lib/components/ui/label";
import * as Select from "$lib/components/ui/select";
import { Skeleton } from "$lib/components/ui/skeleton";
import { clip, fmtRelative, shortId } from "$lib/format";
import { gateway } from "$lib/gateway.svelte";
import { session } from "$lib/session.svelte";
import { ui } from "$lib/ui.svelte";
import { cn } from "$lib/utils";
let { selected = null }: { selected?: string | null } = $props();
const KINDS = ["all", "master", "branch", "deep", "job", "fork"];
const STATUSES = ["open", "all", "merged", "closed", "archived"];
const TITLE_MAX = 60;
const PREVIEW_MAX = 90;
let kind = $state("all");
let statusFilter = $state("open");
let search = $state("");
let createOpen = $state(false);
let agents = $state<AgentInfo[]>([]);
let form = $state({
agent: "",
kind: "deep",
seed: "clean",
text: "",
title: "",
});
let busy = $state(false);
const rows = $derived.by(() => {
const needle = search.trim().toLowerCase();
return gateway.conversations
.filter((row) => kind === "all" || row.kind === kind)
.filter((row) => statusFilter === "all" || row.status === statusFilter)
.filter(
(row) =>
!needle ||
(row.title ?? "").toLowerCase().includes(needle) ||
row.id.startsWith(needle) ||
row.agent.toLowerCase().includes(needle) ||
(row.last_item?.text ?? "").toLowerCase().includes(needle)
)
.sort(byActivity);
});
function activityOf(row: ConversationSummary): string {
return row.last_activity_at ?? row.created_at ?? "";
}
function byActivity(a: ConversationSummary, b: ConversationSummary) {
if (Boolean(a.running_turn) !== Boolean(b.running_turn)) {
return a.running_turn ? -1 : 1;
}
return activityOf(b).localeCompare(activityOf(a));
}
function titleOf(row: ConversationSummary): string {
if (row.title) {
return clip(row.title, TITLE_MAX);
}
if (row.kind === "master") {
return `Master · ${shortId(row.id)}`;
}
return `${row.kind} · ${shortId(row.id)}`;
}
const agentsForKind = $derived(
agents.filter((a) =>
a.kinds.includes(form.kind as AgentInfo["kinds"][number])
)
);
async function openCreate() {
createOpen = true;
if (agents.length === 0 && session.client) {
({ agents } = await session.client.agents());
}
}
async function create() {
if (!session.client) {
return;
}
busy = true;
try {
const created = await session.client.spawn({
agent: form.agent || undefined,
kind: form.kind,
seed: form.seed,
text: form.text || undefined,
title: form.title || undefined,
});
createOpen = false;
await goto(`${base}/conversations/${created.id}`);
} catch (error) {
toast.error(error instanceof Error ? error.message : String(error));
} finally {
busy = false;
}
}
</script>
<div class="flex h-full min-h-0 flex-col">
<div class="flex flex-wrap items-center gap-2 border-b px-3 py-2">
<Select.Root
onValueChange={(value) => {
kind = value;
}}
type="single"
value={kind}
>
<Select.Trigger class="h-8 text-xs" size="sm">
{kind === "all" ? "any kind" : kind}
</Select.Trigger>
<Select.Content>
{#each KINDS as option (option)}
<Select.Item
label={option === "all" ? "any kind" : option}
value={option}
/>
{/each}
</Select.Content>
</Select.Root>
<Select.Root
onValueChange={(value) => {
statusFilter = value;
}}
type="single"
value={statusFilter}
>
<Select.Trigger class="h-8 text-xs" size="sm">
{statusFilter === "all" ? "any status" : statusFilter}
</Select.Trigger>
<Select.Content>
{#each STATUSES as option (option)}
<Select.Item
label={option === "all" ? "any status" : option}
value={option}
/>
{/each}
</Select.Content>
</Select.Root>
<Input
aria-label="Search conversations"
class="h-8 min-w-24 flex-1 text-xs"
placeholder="search"
bind:value={search}
/>
<Button aria-label="New conversation" onclick={openCreate} size="icon-sm">
<PlusIcon class="size-4" />
</Button>
<Button
aria-label="Hide conversations"
class="hidden lg:inline-flex"
onclick={() => ui.toggleRail()}
size="icon-sm"
title="Hide conversations"
variant="ghost"
>
<PanelLeftCloseIcon class="size-4" />
</Button>
</div>
<div class="min-h-0 flex-1 overflow-y-auto">
{#if gateway.live.state === "failed"}
<div class="p-3">
<ErrorNote
message={gateway.live.detail ?? "event stream failed"}
retry={() => gateway.start()}
/>
</div>
{:else if !gateway.loaded}
<div class="flex flex-col gap-2 p-3">
<Skeleton class="h-12 w-full" />
<Skeleton class="h-12 w-full" />
<Skeleton class="h-12 w-full" />
</div>
{:else if rows.length === 0}
<div class="p-3">
<EmptyState
hint="Change the filters, or start one - a deep chat, a branch off the master, a headless job."
title="No conversations match"
>
<Button onclick={openCreate} size="sm" variant="outline">
New conversation
</Button>
</EmptyState>
</div>
{:else}
<ul class="flex flex-col">
{#each rows as row (row.id)}
<li>
<a
aria-current={selected === row.id ? "page" : undefined}
class={cn(
"row-hover flex flex-col gap-1 border-b px-3 py-2 text-sm",
selected === row.id && "bg-sidebar-accent"
)}
href="{base}/conversations/{row.id}"
>
<span class="flex items-center gap-2">
<KindBadge kind={row.kind} />
<span class="min-w-0 flex-1 truncate font-medium">
{titleOf(row)}
</span>
<span class="tabular shrink-0 text-muted-foreground text-xs">
{fmtRelative(activityOf(row))}
</span>
</span>
<span class="flex items-center gap-2 text-xs">
<StatusPill
status={row.running_turn ? "running" : row.status}
/>
<span class="text-muted-foreground">{row.agent}</span>
{#if row.pending_question}
<span class="font-medium text-link">question</span>
{/if}
</span>
{#if row.last_item}
<span class="truncate text-muted-foreground text-xs">
<span class="text-foreground/70"
>{row.last_item.origin}:</span
>
{clip(row.last_item.text, PREVIEW_MAX)}
</span>
{/if}
</a>
</li>
{/each}
</ul>
{/if}
</div>
</div>
<Dialog.Root bind:open={createOpen}>
<Dialog.Content>
<Dialog.Header>
<Dialog.Title>New conversation</Dialog.Title>
<Dialog.Description>
It opens in the home window of its kind (a vault file, a Telegram topic)
and stays silent until someone speaks.
</Dialog.Description>
</Dialog.Header>
<div class="grid grid-cols-2 gap-3">
<div class="flex flex-col gap-1.5">
<Label>Kind</Label>
<Select.Root
onValueChange={(value) => {
form.kind = value;
form.agent = "";
}}
type="single"
value={form.kind}
>
<Select.Trigger class="w-full">{form.kind}</Select.Trigger>
<Select.Content>
{#each ["deep", "branch", "master", "job"] as option (option)}
<Select.Item label={option} value={option} />
{/each}
</Select.Content>
</Select.Root>
</div>
<div class="flex flex-col gap-1.5">
<Label>Agent</Label>
<Select.Root
onValueChange={(value) => {
form.agent = value === "default" ? "" : value;
}}
type="single"
value={form.agent || "default"}
>
<Select.Trigger class="w-full">
{form.agent || "frontend default"}
</Select.Trigger>
<Select.Content>
<Select.Item label="frontend default" value="default" />
{#each agentsForKind as agent (agent.name)}
<Select.Item label={agent.name} value={agent.name} />
{/each}
</Select.Content>
</Select.Root>
</div>
<div class="col-span-2 flex flex-col gap-1.5">
<Label for="new-title">Title</Label>
<Input id="new-title" placeholder="optional" bind:value={form.title} />
</div>
<div class="col-span-2 flex flex-col gap-1.5">
<Label for="new-text">First message</Label>
<textarea
class="min-h-20 rounded-md border bg-input/30 px-3 py-2 text-sm focus-visible:border-ring focus-visible:outline-none focus-visible:ring-3 focus-visible:ring-ring/30"
id="new-text"
placeholder="optional - without it the window waits"
bind:value={form.text}
></textarea>
</div>
</div>
<Dialog.Footer>
<Button
onclick={() => {
createOpen = false;
}}
variant="ghost"
>
Cancel
</Button>
<Button disabled={busy} onclick={create}>Create</Button>
</Dialog.Footer>
</Dialog.Content>
</Dialog.Root>
+1 -11
View File
@@ -117,17 +117,7 @@
};
default:
return {
endpoints: base
? [
{
hint: fe.scope
? `POST the frontend's own payload; bearer with scope ${fe.scope}`
: "",
label: "base",
url: base,
},
]
: [],
endpoints: base ? [{ hint: "", label: "base", url: base }] : [],
frontend: fe,
note: "",
};
+13 -5
View File
@@ -1,5 +1,7 @@
<script lang="ts">
import type { Snippet } from "svelte";
import LiveDot from "$lib/components/live-dot.svelte";
import { gateway } from "$lib/gateway.svelte";
let {
title,
@@ -15,7 +17,7 @@
</script>
<header
class="flex min-h-12 flex-wrap items-center gap-x-4 gap-y-2 px-4 pt-4 pb-1 sm:px-6"
class="flex min-h-12 flex-wrap items-center gap-x-4 gap-y-2 border-b px-4 py-2 sm:px-6"
>
<div class="flex min-w-0 items-baseline gap-2">
<h1 class="truncate font-semibold text-lg tracking-tight">{title}</h1>
@@ -28,9 +30,15 @@
{@render children()}
</div>
{/if}
{#if actions}
<div class="ml-auto flex items-center gap-2">
<div class="ml-auto flex items-center gap-2">
{#if actions}
{@render actions()}
</div>
{/if}
{/if}
<LiveDot
class="sm:hidden"
detail={gateway.live.detail}
label={false}
state={gateway.live.state}
/>
</div>
</header>
+10 -21
View File
@@ -68,12 +68,6 @@ export function fmtSeconds(seconds: number | null | undefined): string {
: fmtDuration(seconds * THOUSAND);
}
export function fmtCpu(percent: number | null | undefined): string {
return percent === null || percent === undefined
? "–"
: `${Math.round(percent)}%`;
}
export function fmtBytes(bytes: number | null | undefined): string {
if (bytes === null || bytes === undefined) {
return "–";
@@ -109,13 +103,6 @@ export function fmtTime(iso: string | null | undefined): string {
: "–";
}
export function fmtMinute(iso: string | null | undefined): string {
const date = parseDate(iso);
return date
? date.toLocaleTimeString(undefined, { hour: "2-digit", minute: "2-digit" })
: "–";
}
export function fmtDateTime(iso: string | null | undefined): string {
const date = parseDate(iso);
return date
@@ -136,17 +123,19 @@ export function fmtRelative(
if (!date) {
return "–";
}
const diff = Math.max(0, now - date.getTime());
if (diff < MINUTE_MS) {
return "just now";
const diff = now - date.getTime();
const abs = Math.abs(diff);
const suffix = diff >= 0 ? "ago" : "from now";
if (abs < MINUTE_MS) {
return diff >= 0 ? "just now" : "in <1m";
}
if (diff < HOUR_MS) {
return `${Math.round(diff / MINUTE_MS)}m ago`;
if (abs < HOUR_MS) {
return `${Math.round(abs / MINUTE_MS)}m ${suffix}`;
}
if (diff < DAY_MS) {
return `${Math.round(diff / HOUR_MS)}h ago`;
if (abs < DAY_MS) {
return `${Math.round(abs / HOUR_MS)}h ${suffix}`;
}
return `${Math.round(diff / DAY_MS)}d ago`;
return `${Math.round(abs / DAY_MS)}d ${suffix}`;
}
export function fmtCountdown(iso: string | null | undefined, now = Date.now()) {
+2 -26
View File
@@ -1,18 +1,9 @@
import type {
BusEvent,
JobsResponse,
LimitsResponse,
LimitWindow,
} from "./api/types";
import { panelCache } from "./panel/cache";
import { browserHost } from "./panel/host";
import type { BusEvent, LimitsResponse, LimitWindow } from "./api/types";
import { ConversationIndex } from "./panel/index.svelte";
import { session } from "./session.svelte";
const TAPE_SIZE = 120;
const QUIET = new Set(["stream", "hello"]);
const JOBS_KEY = "jobs";
const cache = panelCache("beaver.admin");
// Gateway-wide live state for the admin: the conversation index, the last
// non-chatty events as a tape, and the quota windows updated the moment
@@ -20,20 +11,9 @@ const cache = panelCache("beaver.admin");
class Gateway extends ConversationIndex {
tape = $state<BusEvent[]>([]);
limits = $state<LimitsResponse | null>(null);
// The scheduler snapshot: what was last seen, shown at once, then refreshed.
jobs = $state<JobsResponse | null>(cache.get<JobsResponse>(JOBS_KEY) ?? null);
constructor() {
super(() => session.client, browserHost().cache ?? null);
}
async refreshJobs(): Promise<void> {
const client = this.client();
if (!client) {
return;
}
this.jobs = await client.jobs();
cache.set(JOBS_KEY, this.jobs);
super(() => session.client);
}
override async load(): Promise<void> {
@@ -48,7 +28,6 @@ class Gateway extends ConversationIndex {
if (limits) {
this.limits = limits;
}
this.refreshJobs().catch(() => undefined);
}
async refreshLimits(): Promise<void> {
@@ -69,9 +48,6 @@ class Gateway extends ConversationIndex {
this.applyLimit(event);
return;
}
if (event.type.startsWith("job.") || event.type.startsWith("schedule.")) {
this.refreshJobs().catch(() => undefined);
}
super.apply(event);
}
+1 -22
View File
@@ -6,27 +6,6 @@ export const LIMIT_LABELS: Record<string, string> = {
seven_day_sonnet: "7-day · Sonnet",
};
const SEVEN_DAY = "seven_day_";
export function limitLabel(window: string): string {
const known = LIMIT_LABELS[window];
if (known) {
return known;
}
if (window.startsWith(SEVEN_DAY)) {
const model = window.slice(SEVEN_DAY.length);
return `7-day · ${model.charAt(0).toUpperCase()}${model.slice(1)}`;
}
return window.replaceAll("_", " ");
}
// The status as a word, for the windows the API never puts a figure on.
export const LIMIT_WORDS: Record<string, string> = {
allowed: "clear",
allowed_warning: "near the limit",
rejected: "rejected",
};
export function limitWord(status: string): string {
return LIMIT_WORDS[status] ?? status.replace("_", " ");
return LIMIT_LABELS[window] ?? window;
}
+25 -36
View File
@@ -1,46 +1,35 @@
import ActivityIcon from "@lucide/svelte/icons/activity";
import BrainIcon from "@lucide/svelte/icons/brain";
import ClockIcon from "@lucide/svelte/icons/clock";
import GaugeIcon from "@lucide/svelte/icons/gauge";
import KeyRoundIcon from "@lucide/svelte/icons/key-round";
import MessagesSquareIcon from "@lucide/svelte/icons/messages-square";
import ScrollTextIcon from "@lucide/svelte/icons/scroll-text";
import type { Component } from "svelte";
export interface NavItem {
// Lit only on its own path, not on its children (an index page).
exact?: boolean;
href: string;
key: string;
icon: Component<{ class?: string }>;
label: string;
mobile: boolean;
}
export const SECTIONS: NavItem[] = [
{ href: "/", key: "1", label: "Now" },
{ href: "/conversations", key: "2", label: "Conversations" },
{ href: "/memory", key: "3", label: "Memory" },
{ href: "/usage", key: "4", label: "Usage" },
export const NAV: NavItem[] = [
{ href: "/", icon: ActivityIcon, label: "Now", mobile: true },
{
href: "/conversations",
icon: MessagesSquareIcon,
label: "Conversations",
mobile: true,
},
{ href: "/usage", icon: GaugeIcon, label: "Usage", mobile: true },
{ href: "/memory", icon: BrainIcon, label: "Memory", mobile: true },
{ href: "/tokens", icon: KeyRoundIcon, label: "Tokens", mobile: false },
{ href: "/jobs", icon: ClockIcon, label: "Jobs", mobile: false },
{ href: "/audit", icon: ScrollTextIcon, label: "Audit", mobile: false },
];
export const SYSTEM: NavItem = { href: "/system", key: "5", label: "System" };
export const SYSTEM_PAGES: NavItem[] = [
{ exact: true, href: "/system", key: "", label: "Sessions" },
{ href: "/system/agents", key: "", label: "Agents & endpoints" },
{ href: "/system/jobs", key: "", label: "Jobs" },
{ href: "/system/tokens", key: "", label: "Tokens" },
{ href: "/system/secrets", key: "", label: "Secrets" },
{ href: "/system/audit", key: "", label: "Audit" },
];
export function isActive(
pathname: string,
base: string,
href: string,
exact = false
) {
const path = pathname.startsWith(base)
? pathname.slice(base.length)
: pathname;
const current = path || "/";
if (href === "/" || exact) {
return current === href;
}
return current === href || current.startsWith(`${href}/`);
}
export function isSectionActive(pathname: string, base: string, href: string) {
export function isActive(pathname: string, base: string, href: string) {
const path = pathname.startsWith(base)
? pathname.slice(base.length)
: pathname;
-8
View File
@@ -9,7 +9,6 @@ import FileTextIcon from "@lucide/svelte/icons/file-text";
import GitBranchIcon from "@lucide/svelte/icons/git-branch";
import GitMergeIcon from "@lucide/svelte/icons/git-merge";
import PencilIcon from "@lucide/svelte/icons/pencil";
import RewindIcon from "@lucide/svelte/icons/rewind";
import RotateCcwIcon from "@lucide/svelte/icons/rotate-ccw";
import SendIcon from "@lucide/svelte/icons/send";
import type { Component } from "svelte";
@@ -39,7 +38,6 @@ export interface ActionScope {
onChanged: () => void;
onOpen: (id: string) => void;
onRename: () => void;
onRewind: () => void;
}
const TELEGRAM = "telegram";
@@ -105,12 +103,6 @@ export function conversationActions(
label: "Branch…",
run: scope.onBranch,
});
actions.push({
disabled: !info.session_id,
icon: RewindIcon,
label: "Rewind…",
run: scope.onRewind,
});
actions.push({
checked: remembers,
gap: true,
+1 -1
View File
@@ -23,7 +23,7 @@
onMount(() => {
const timer = setInterval(() => {
if (model.live.length > 0) {
if (model.running) {
now = Date.now();
}
}, TICK_MS);
+14 -119
View File
@@ -27,11 +27,6 @@ export interface Turn {
itemOrigin: string | null;
nodes: Record<string, ToolNode>;
origin: string;
// A tool call came between two pieces of text; the next piece starts a
// paragraph, as the final message will show it.
paused: boolean;
// What the model thought, when its thinking is reasoning and not progress.
reasoning: string;
resultSubtype: string | null;
roots: string[];
says: string[];
@@ -56,20 +51,6 @@ function str(value: unknown): string | null {
return typeof value === "string" ? value : null;
}
export function joinText(head: string, tail: string, paused: boolean): string {
if (paused && head.trim() && !head.endsWith("\n")) {
return `${head.trimEnd()}\n\n${tail.trimStart()}`;
}
return head + tail;
}
export function isLive(turn: Turn): boolean {
return (
turn.status === "running" ||
Object.values(turn.nodes).some((node) => node.status === "running")
);
}
function stopStatus(turn: Turn, stop: string | null): TurnStatus {
if (stop === "end_turn") {
return turn.status === "error" ? "error" : "done";
@@ -84,9 +65,6 @@ export class ActivityModel {
turns = $state<Turn[]>([]);
question = $state<PendingQuestion | null>(null);
// A subagent still working after its turn ended: nothing on this page
// knows the turn, so its tool calls hang under the launching Agent node
// of a turn named after that node.
private readonly handlers: Record<string, (c: Cursor) => void> = {
"conversation.created": (c) => this.onConversation(c),
"conversation.updated": (c) => this.onConversation(c),
@@ -111,12 +89,6 @@ export class ActivityModel {
return this.turns.find((turn) => turn.status === "running") ?? null;
}
// A turn is live while it runs or while a subagent it launched still
// works: the CLI reports subagent tool calls after the turn's own end.
get live(): Turn[] {
return this.turns.filter((turn) => isLive(turn));
}
setConversation(info: ConversationInfo): void {
this.conversation = info;
this.question = info.question;
@@ -179,42 +151,6 @@ export class ActivityModel {
turn.startedAt = ts;
turn.itemOrigin = str(event.item_origin);
turn.userText = str(event.text) ?? turn.userText;
if (turn.origin === "task") {
this.settleAgents(ts);
}
}
// The report is in: whatever Agent node still runs has finished.
private settleAgents(ts: string): void {
for (const turn of this.turns) {
if (turn.origin === "agent" && turn.status === "running") {
turn.status = "done";
turn.endedAt = ts;
}
for (const node of Object.values(turn.nodes)) {
const agent = SUBAGENT_TOOLS.has(node.name) || node.name === "?";
if (node.status === "running" && node.parent === null && agent) {
node.status = "done";
node.endedAt = ts;
}
}
}
}
// Subagent traffic belongs to the turn that launched the subagent, whatever
// turn the gateway was in when it arrived.
private turnFor(turnId: string, ts: string, parent: string | null): Turn {
if (parent) {
const owner = this.turns.find((turn) => parent in turn.nodes);
if (owner) {
return owner;
}
const current = this.turns.find((turn) => turn.id === turnId);
if (current?.status !== "running") {
return this.ensureTurn(`agent:${parent}`, ts, "agent");
}
}
return this.ensureTurn(turnId, ts, null);
}
private onStream({ event, ts, turnId, parent }: Cursor): void {
@@ -222,48 +158,22 @@ export class ActivityModel {
if (!(turnId && raw && typeof raw === "object")) {
return;
}
const turn = this.turnFor(turnId, ts, parent);
const turn = this.ensureTurn(turnId, ts, null);
const sdk = raw as Record<string, unknown>;
if (sdk.type === "content_block_start") {
this.onBlockStart(turn, sdk.content_block, parent, ts);
} else if (sdk.type === "content_block_delta" && parent === null) {
this.onDelta(turn, sdk.delta as Record<string, unknown> | undefined);
}
}
private onBlockStart(
turn: Turn,
raw: unknown,
parent: string | null,
ts: string
): void {
const block = raw as Record<string, unknown> | undefined;
if (block?.type === "tool_use" && typeof block.id === "string") {
this.ensureNode(turn, block.id, str(block.name) ?? "?", parent, ts);
} else if (block?.type === "thinking" && parent === null) {
turn.thinking += 1;
if (turn.reasoning) {
turn.reasoning += "\n\n";
const block = sdk.content_block as Record<string, unknown> | undefined;
if (block?.type === "tool_use" && typeof block.id === "string") {
this.ensureNode(turn, block.id, str(block.name) ?? "?", parent, ts);
} else if (block?.type === "thinking" && parent === null) {
turn.thinking += 1;
}
}
}
private onDelta(
turn: Turn,
delta: Record<string, unknown> | undefined
): void {
const thought = delta?.type === "thinking_delta" ? delta.thinking : null;
if (
typeof thought === "string" &&
this.conversation?.thinking_is_progress !== true
) {
turn.reasoning += thought;
return;
}
const text = thought ?? delta?.text;
if (typeof text === "string" && text) {
turn.text = joinText(turn.text, text, turn.paused);
turn.paused = false;
if (sdk.type === "content_block_delta" && parent === null) {
const delta = sdk.delta as Record<string, unknown> | undefined;
if (delta?.type === "text_delta" && typeof delta.text === "string") {
turn.text += delta.text;
}
}
}
@@ -271,10 +181,7 @@ export class ActivityModel {
if (!(turnId && typeof event.tool_use_id === "string")) {
return;
}
const turn = this.turnFor(turnId, ts, parent);
if (parent === null) {
turn.paused = true;
}
const turn = this.ensureTurn(turnId, ts, null);
const node = this.ensureNode(
turn,
event.tool_use_id,
@@ -289,7 +196,7 @@ export class ActivityModel {
if (!(turnId && typeof event.tool_use_id === "string")) {
return;
}
const turn = this.turnFor(turnId, ts, parent);
const turn = this.ensureTurn(turnId, ts, null);
const node = this.ensureNode(turn, event.tool_use_id, "?", parent, ts);
node.status = event.is_error ? "error" : "done";
node.endedAt = ts;
@@ -359,13 +266,11 @@ export class ActivityModel {
};
}
// The turn's own tool calls die with it; a subagent's keep going until
// their own results arrive.
private closeTurn(turn: Turn, status: TurnStatus, ts: string): void {
turn.status = status;
turn.endedAt = ts;
for (const node of Object.values(turn.nodes)) {
if (node.status === "running" && node.parent === null) {
if (node.status === "running") {
node.status = "aborted";
node.endedAt = ts;
}
@@ -386,8 +291,6 @@ export class ActivityModel {
itemOrigin: null,
nodes: {},
origin: origin ?? "?",
paused: false,
reasoning: "",
resultSubtype: null,
roots: [],
says: [],
@@ -491,11 +394,3 @@ export function summarizeInput(name: string, input: unknown): string {
export function toolLabel(name: string): string {
return name.startsWith("mcp__") ? name.slice(5).replace("__", " · ") : name;
}
// How a turn was started, as a word for the header.
export const ORIGIN_LABELS: Record<string, string> = {
agent: "subagent at work",
inject: "inject",
task: "subagent report",
user: "user",
};
-49
View File
@@ -1,49 +0,0 @@
// A small keyed cache behind the panel: the conversation index and the
// last threads land here so the next open paints before the network
// answers. localStorage when it exists, memory otherwise.
export interface PanelCache {
get: <T>(key: string) => T | null;
set: (key: string, value: unknown) => void;
}
const memory = new Map<string, string>();
function storage(): Pick<Storage, "getItem" | "setItem" | "removeItem"> {
try {
if (typeof localStorage !== "undefined") {
return localStorage;
}
} catch {
/* sandboxed */
}
return {
getItem: (key) => memory.get(key) ?? null,
removeItem: (key) => {
memory.delete(key);
},
setItem: (key, value) => {
memory.set(key, value);
},
};
}
export function panelCache(prefix: string): PanelCache {
const store = storage();
return {
get<T>(key: string): T | null {
try {
const raw = store.getItem(`${prefix}:${key}`);
return raw ? (JSON.parse(raw) as T) : null;
} catch {
return null;
}
},
set(key: string, value: unknown): void {
try {
store.setItem(`${prefix}:${key}`, JSON.stringify(value));
} catch {
store.removeItem(`${prefix}:${key}`);
}
},
};
}
+86 -311
View File
@@ -1,21 +1,17 @@
<script lang="ts">
import { tick, untrack } from "svelte";
import { tick } from "svelte";
import type { ApiClient } from "$lib/api/client";
import type { ContentBlock, HistoryMessage } from "$lib/api/types";
import EmptyState from "$lib/components/empty-state.svelte";
import ErrorNote from "$lib/components/error-note.svelte";
import { clip, fmtDateTime, fmtTime } from "$lib/format";
import { Skeleton } from "$lib/components/ui/skeleton";
import { Switch } from "$lib/components/ui/switch";
import { clip } from "$lib/format";
import { cn } from "$lib/utils";
import type { ActivityModel, ToolNode } from "./activity.svelte";
import { isLive } from "./activity.svelte";
import { cacheableHistory, historyKey } from "./history-cache";
import { usePanelHost } from "./host";
import type { ActivityModel } from "./activity.svelte";
import { summarizeInput, toolLabel } from "./activity.svelte";
import Markdown from "./markdown.svelte";
import QuestionCard from "./question-card.svelte";
import Reasoning from "./reasoning.svelte";
import SayBubble from "./say-bubble.svelte";
import ServiceLine from "./service-line.svelte";
import ToolNodeView from "./tool-node.svelte";
import TurnCard from "./turn-card.svelte";
let {
@@ -30,190 +26,99 @@
refreshKey?: number;
} = $props();
interface Step {
block: ContentBlock;
eager: boolean;
key: string;
stampedAt: string | null;
}
type Row =
| { day: string; key: string; kind: "day" }
| { key: string; kind: "system"; text: string; ts: string | null }
| {
eager: boolean;
key: string;
kind: "user";
service: string | null;
texts: string[];
ts: string | null;
}
| { key: string; kind: "agent"; steps: Step[] };
const RESULT_CLIP = 20_000;
const RENDERED_AT_ONCE = 40;
const host = usePanelHost();
const AT_BOTTOM_PX = 8;
const RESULT_CLIP = 600;
const NEAR_BOTTOM_PX = 120;
const TICK_MS = 1000;
const NO_NODES: Record<string, ToolNode> = {};
let messages = $state<HistoryMessage[] | null>(null);
let failure = $state<string | null>(null);
let showResults = $state(false);
let loadedAt = $state(new Date(0).toISOString());
let scroller = $state<HTMLDivElement | null>(null);
let body = $state<HTMLDivElement | null>(null);
let now = $state(Date.now());
let pinned = true;
let readerRow: { node: Element; top: number } | null = null;
let ownScroll = false;
const tail = $derived(
model.turns
.filter((turn) => isLive(turn) || turn.startedAt > loadedAt)
.filter((turn) => turn.status === "running" || turn.startedAt > loadedAt)
.reverse()
);
const tailSize = $derived(
tail.reduce(
(n, turn) =>
n +
turn.text.length +
turn.reasoning.length +
Object.keys(turn.nodes).length,
(n, turn) => n + turn.text.length + Object.keys(turn.nodes).length,
0
)
);
async function load() {
failure = null;
if (messages === null) {
const cached = host.cache?.get<HistoryMessage[]>(
historyKey(conversationId)
);
if (cached) {
messages = cached;
}
}
try {
({ messages } = await client.history(conversationId));
loadedAt = new Date().toISOString();
host.cache?.set(historyKey(conversationId), cacheableHistory(messages));
} catch (cause) {
failure = cause instanceof Error ? cause.message : String(cause);
}
}
function toBottom() {
function nearBottom(): boolean {
if (!scroller) {
return true;
}
const left =
scroller.scrollHeight - scroller.scrollTop - scroller.clientHeight;
return left < NEAR_BOTTOM_PX;
}
async function scrollToBottom() {
await tick();
if (scroller) {
scroller.scrollTop = 0;
scroller.scrollTop = scroller.scrollHeight;
}
}
function topRow(): { node: Element; top: number } | null {
if (!(scroller && body)) {
return null;
}
const edge = scroller.getBoundingClientRect().top;
const rows = body.children;
let low = 0;
let high = rows.length - 1;
while (low < high) {
const middle = Math.floor((low + high) / 2);
if (rows[middle].getBoundingClientRect().bottom > edge) {
high = middle;
} else {
low = middle + 1;
}
}
const node = rows[low];
return node ? { node, top: node.getBoundingClientRect().top } : null;
}
function trackReader() {
if (ownScroll) {
ownScroll = false;
return;
}
pinned = !scroller || Math.abs(scroller.scrollTop) < AT_BOTTOM_PX;
readerRow = pinned ? null : topRow();
}
function keepReaderRowInPlace() {
if (pinned) {
toBottom();
return;
}
if (!(scroller && readerRow?.node.isConnected)) {
return;
}
const drift = readerRow.node.getBoundingClientRect().top - readerRow.top;
const before = scroller.scrollTop;
scroller.scrollTop += drift;
ownScroll = scroller.scrollTop !== before;
}
function dayOf(iso: string | null | undefined): string {
return iso ? iso.slice(0, 10) : "";
}
function minuteOf(iso: string | null | undefined): string {
return iso ? fmtTime(iso).replace(SECONDS, "") : "";
}
$effect(() => {
if (refreshKey >= 0) {
untrack(() => load());
pinned = true;
load();
}
});
$effect(() => {
const count = (messages?.length ?? 0) + tailSize;
if (pinned && count >= 0) {
tick().then(toBottom);
scrollToBottom();
}
});
// Markdown lands asynchronously (Obsidian renders after the DOM exists),
// so follow the content's height, not just the message count.
$effect(() => {
if (!(scroller && body)) {
return;
}
const observer = new ResizeObserver(keepReaderRowInPlace);
const observer = new ResizeObserver(() => {
if (pinned && scroller) {
scroller.scrollTop = scroller.scrollHeight;
}
});
observer.observe(body);
return () => observer.disconnect();
});
$effect(() => {
const timer = setInterval(() => {
if (model.live.length > 0) {
if (model.running) {
now = Date.now();
}
}, TICK_MS);
return () => clearInterval(timer);
});
const SYSTEM_HEAD = /^(\[[^\]\n]+\]|<task-notification>)/;
const SECONDS = /:\d{2}$/;
const DAY_TIME = /,?\s*\d{2}:\d{2}$/;
function keyOf(message: HistoryMessage, index: number, total: number) {
const fromEnd = `${index - total}:${message.role}:${message.ts ?? ""}`;
return message.id ?? fromEnd;
}
function blocks(message: HistoryMessage): ContentBlock[] {
if (typeof message.content === "string") {
return [{ text: message.content, type: "text" }];
}
if (!model.conversation?.thinking_is_progress) {
return message.content;
}
return message.content.flatMap((block) => {
if (block.type !== "thinking") {
return [block];
}
return block.thinking?.trim()
? [{ text: block.thinking, type: "text" }]
: [];
});
return typeof message.content === "string"
? [{ text: message.content, type: "text" }]
: message.content;
}
function resultText(block: ContentBlock): string {
@@ -232,205 +137,75 @@
}
return "";
}
const results = $derived.by(() => {
const found = new Map<string, { failed: boolean; text: string }>();
for (const message of messages ?? []) {
if (typeof message.content === "string") {
continue;
}
for (const block of message.content) {
if (block.type === "tool_result" && block.tool_use_id) {
found.set(block.tool_use_id, {
failed: block.is_error === true,
text: clip(resultText(block), RESULT_CLIP),
});
}
}
}
return found;
});
function nodeOf(block: ContentBlock): ToolNode {
const result = block.id ? results.get(block.id) : undefined;
let status: ToolNode["status"] = "aborted";
if (result) {
status = result.failed ? "error" : "done";
}
return {
children: [],
endedAt: null,
id: block.id ?? "",
input: block.input ?? null,
name: block.name ?? "?",
parent: null,
result: result?.text ?? "",
startedAt: "",
status,
};
}
function said(block: ContentBlock): string | null {
const text = (block.input as { text?: unknown } | null | undefined)?.text;
return block.name?.endsWith("__say") && typeof text === "string"
? text
: null;
}
function userRow(
message: HistoryMessage,
parts: ContentBlock[],
key: string,
eager: boolean
): Row {
const ts = message.ts ?? null;
const texts = parts.flatMap((b) =>
b.type === "text" && b.text ? [b.text] : []
);
if (!message.service && SYSTEM_HEAD.test(texts[0]?.trimStart() ?? "")) {
return { key, kind: "system", text: texts.join("\n"), ts };
}
const service = message.service ?? null;
return { eager, key, kind: "user", service, texts, ts };
}
function stepsOf(
message: HistoryMessage,
parts: ContentBlock[],
key: string,
eager: boolean,
minute: string
): Step[] {
const ts = message.ts ?? null;
const stamped =
minuteOf(ts) !== minute ||
parts.some((b) => b.type === "text" && Boolean(b.text));
return parts.map((block, at) => ({
block,
eager,
key: `${key}:${at}`,
stampedAt: stamped && at === 0 ? ts : null,
}));
}
const rows = $derived.by(() => {
const out: Row[] = [];
const list = messages ?? [];
let run: Step[] | null = null;
let day = "";
let minute = "";
for (const [index, message] of list.entries()) {
const parts = blocks(message);
if (parts.every((b) => b.type === "tool_result")) {
continue;
}
const key = keyOf(message, index, list.length);
const eager = index >= list.length - RENDERED_AT_ONCE;
if (message.ts && dayOf(message.ts) !== day) {
day = dayOf(message.ts);
const label = fmtDateTime(message.ts).replace(DAY_TIME, "");
out.push({ day: label, key, kind: "day" });
run = null;
}
if (message.role === "user") {
out.push(userRow(message, parts, key, eager));
run = null;
} else if (run === null) {
run = stepsOf(message, parts, key, eager, minute);
out.push({ key, kind: "agent", steps: run });
} else {
run.push(...stepsOf(message, parts, key, eager, minute));
}
minute = message.ts ? minuteOf(message.ts) : minute;
}
return out;
});
</script>
<div
class="flex min-h-0 flex-1 flex-col-reverse overflow-y-auto overflow-x-hidden px-3 py-3 [overflow-anchor:none] @md:px-6"
onscroll={trackReader}
class="min-h-0 flex-1 overflow-y-auto px-3 py-3 @md:px-6"
onscroll={() => {
pinned = nearBottom();
}}
bind:this={scroller}
>
<div class="flex min-w-0 max-w-full flex-col gap-3" bind:this={body}>
<div class="flex flex-col gap-3" bind:this={body}>
<span
class="flex items-center gap-2 self-end text-muted-foreground text-xs"
>
<Switch aria-label="Show tool results" bind:checked={showResults} />
show tool results
</span>
{#if failure}
<ErrorNote message={failure} retry={load} />
{:else if messages === null}
<p class="py-6 text-center text-muted-foreground text-xs">
Loading the thread…
</p>
<div class="flex flex-col gap-2">
<Skeleton class="h-10 w-2/3" />
<Skeleton class="h-16 w-full" />
<Skeleton class="h-10 w-1/2" />
</div>
{:else if messages.length === 0 && tail.length === 0}
<EmptyState
hint="Nothing has been said here yet. Write below, or wait for the first inject."
title="Empty conversation"
/>
{:else}
{#each rows as row (`${row.kind}:${row.key}`)}
{#if row.kind === "day"}
<p class="label-quiet self-center pt-2">{row.day}</p>
{:else if row.kind === "system"}
<div class="flex min-w-0 items-start justify-center gap-2">
{#if row.ts}
<span
class="tabular shrink-0 py-1 text-[11px] text-muted-foreground opacity-60"
title={fmtDateTime(row.ts)}
>
{minuteOf(row.ts)}
</span>
{/if}
<ServiceLine text={row.text} />
</div>
{:else if row.kind === "user"}
<div class="group flex min-w-0 max-w-full flex-col items-end gap-1.5">
{#if row.ts}
<span
class="tabular px-1 text-[11px] text-muted-foreground opacity-60 transition-opacity group-hover:opacity-100"
title={fmtDateTime(row.ts)}
>
{minuteOf(row.ts)}
</span>
{/if}
{#each row.texts as text, index (index)}
<Markdown
class="min-w-0 @md:max-w-[75ch] max-w-full rounded-2xl rounded-br-md bg-primary/10 px-3.5 py-2"
eager={row.eager}
inert
{text}
/>
{/each}
{#if row.service}
<ServiceLine text={row.service} />
{/if}
</div>
{:else}
<div class="group flex min-w-0 max-w-full flex-col">
{#each row.steps as step (step.key)}
{@const block = step.block}
{#if step.stampedAt}
<span
class="tabular px-1 pt-1.5 text-[11px] text-muted-foreground opacity-60 transition-opacity group-hover:opacity-100"
title={fmtDateTime(step.stampedAt)}
>
{minuteOf(step.stampedAt)}
</span>
{/if}
{#each messages as message, index (index)}
{@const parts = blocks(message)}
{@const isResultOnly = parts.every((b) => b.type === "tool_result")}
{#if !(isResultOnly && !showResults)}
<div
class={cn(
"flex flex-col gap-1.5",
message.role === "user" && !isResultOnly && "items-end"
)}
>
{#each parts as block, blockIndex (blockIndex)}
{#if block.type === "text" && block.text}
<Markdown
class="min-w-0 @md:max-w-[75ch] max-w-full px-1 py-1"
eager={step.eager}
class={cn(
"max-w-[75ch] rounded-lg px-3 py-2",
message.role === "user" ? "bg-primary/10" : "bg-muted/50"
)}
text={block.text}
/>
{:else if block.type === "tool_use"}
{@const spoken = said(block)}
{#if spoken}
<div class="py-1"><SayBubble text={spoken} /></div>
{:else}
<ToolNodeView node={nodeOf(block)} nodes={NO_NODES} now={0} />
{/if}
{:else if block.type === "thinking" && block.thinking?.trim()}
<Reasoning text={block.thinking} />
<p class="flex items-baseline gap-2 px-1 text-xs">
<span class="font-medium"
>{toolLabel(block.name ?? "?")}</span
>
<span class="truncate text-muted-foreground">
{clip(summarizeInput(block.name ?? "", block.input), 160)}
</span>
</p>
{:else if block.type === "tool_result" && showResults}
<pre
class={cn(
"max-h-48 max-w-full overflow-auto whitespace-pre-wrap break-all rounded-md p-2 text-xs",
block.is_error
? "bg-destructive/8 text-destructive"
: "bg-muted/40"
)}
>{clip(resultText(block), RESULT_CLIP)}</pre>
{:else if block.type === "thinking"}
<p class="px-1.5 py-1 text-kind-deep text-xs">thinking</p>
<p class="px-1 text-kind-deep text-xs">thinking</p>
{/if}
{/each}
</div>
+58 -64
View File
@@ -1,8 +1,9 @@
<script lang="ts">
import ArrowUpIcon from "@lucide/svelte/icons/arrow-up";
import SendHorizontalIcon from "@lucide/svelte/icons/send-horizontal";
import { toast } from "svelte-sonner";
import type { ApiClient } from "$lib/api/client";
import { cn } from "$lib/utils";
import { Button } from "$lib/components/ui/button";
import * as Select from "$lib/components/ui/select";
let {
client,
@@ -16,31 +17,28 @@
type Mode = "message" | "inject" | "urgent";
const MODES: { value: Mode; label: string; hint: string }[] = [
{ hint: "as you, mirrored to the window", label: "Say", value: "message" },
{
hint: "a system note for the next turn",
hint: "as the user, mirrored to Telegram",
label: "Message",
value: "message",
},
{
hint: "system note, rides with the next turn",
label: "Inject",
value: "inject",
},
{ hint: "interrupts the running turn", label: "Urgent", value: "urgent" },
{
hint: "interrupts the running turn",
label: "Urgent inject",
value: "urgent",
},
];
const MAX_ROWS = 10;
const LINE_PX = 22;
let mode = $state<Mode>("message");
let text = $state("");
let busy = $state(false);
let area = $state<HTMLTextAreaElement | null>(null);
const current = $derived(MODES.find((m) => m.value === mode) ?? MODES[0]);
function grow() {
if (!area) {
return;
}
area.style.height = "auto";
area.style.height = `${Math.min(area.scrollHeight, LINE_PX * MAX_ROWS)}px`;
}
async function send() {
const body = text.trim();
if (!body || busy) {
@@ -58,7 +56,6 @@
);
}
text = "";
queueMicrotask(grow);
} catch (error) {
toast.error(error instanceof Error ? error.message : String(error));
} finally {
@@ -75,60 +72,57 @@
</script>
<form
class="flex flex-col gap-1.5 px-3 pt-2 pb-2 @md:px-6"
class="flex flex-col gap-2 border-t bg-background px-3 py-2"
onsubmit={(event) => {
event.preventDefault();
send();
}}
>
<div
class={cn(
"flex items-end gap-2 rounded-xl bg-strip px-3 py-2 shadow-lift ring-1 ring-border focus-within:ring-ring",
disabled && "opacity-60"
)}
>
<textarea
aria-label="Message"
class="max-h-56 min-h-6 w-full resize-none bg-transparent py-0.5 text-sm leading-[22px] outline-none placeholder:text-muted-foreground"
{disabled}
oninput={grow}
onkeydown={onKeydown}
placeholder={disabled
? "This conversation is closed."
: `${current.label} something… (⌘↩)`}
rows="1"
bind:this={area}
bind:value={text}
></textarea>
<button
aria-label="Send"
class="inline-flex size-7 shrink-0 items-center justify-center rounded-full bg-primary text-primary-foreground transition-opacity disabled:opacity-30"
<textarea
aria-label="Message"
class="min-h-16 w-full resize-y rounded-md border bg-input/30 px-3 py-2 text-sm focus-visible:border-ring focus-visible:outline-none focus-visible:ring-3 focus-visible:ring-ring/30"
{disabled}
onkeydown={onKeydown}
placeholder={mode === "message"
? "Say something to the agent…"
: "Text of the inject…"}
rows="2"
bind:value={text}
></textarea>
<div class="flex items-center gap-2">
<Select.Root
onValueChange={(value) => {
mode = value as Mode;
}}
type="single"
value={mode}
>
<Select.Trigger class="h-8 text-xs" size="sm"
>{current.label}</Select.Trigger
>
<Select.Content>
{#each MODES as option (option.value)}
<Select.Item label={option.label} value={option.value}>
<span class="flex flex-col">
<span>{option.label}</span>
<span class="text-muted-foreground text-xs">{option.hint}</span>
</span>
</Select.Item>
{/each}
</Select.Content>
</Select.Root>
<span class="hidden text-muted-foreground text-xs @md:inline"
>{current.hint}</span
>
<Button
class="ml-auto"
disabled={disabled || busy || !text.trim()}
size="sm"
type="submit"
>
<ArrowUpIcon class="size-4" />
</button>
</div>
<div class="flex items-center gap-3 px-1">
{#each MODES as option (option.value)}
<button
class={cn(
"rule-word text-xs",
mode === option.value && "text-foreground",
option.value === "urgent" && mode === option.value && "text-destructive"
)}
data-active={mode === option.value}
onclick={() => {
mode = option.value;
}}
title={option.hint}
type="button"
>
{option.label}
</button>
{/each}
<span class="hidden truncate text-muted-foreground text-xs @md:inline">
{current.hint}
</span>
<SendHorizontalIcon class="size-4" />
Send
<kbd class="hidden text-[10px] opacity-70 @md:inline">⌘↩</kbd>
</Button>
</div>
</form>
-395
View File
@@ -1,395 +0,0 @@
<script lang="ts">
import { onMount } from "svelte";
import type { ApiClient } from "$lib/api/client";
import type { ContextResponse, VaultGraph } from "$lib/api/types";
import ErrorNote from "$lib/components/error-note.svelte";
import { fmtRelative, fmtTokens } from "$lib/format";
import Graph, {
type GraphEdge,
type GraphNode,
} from "$lib/shell/graph.svelte";
import { cn } from "$lib/utils";
import { usePanelHost } from "./host";
let {
client,
conversationId,
window = 1_000_000,
}: {
client: ApiClient;
conversationId: string;
window?: number;
} = $props();
const HARNESS_EST = 15_000;
const FILES_SHOWN = 12;
const MD_SUFFIX = /\.md$/;
const host = usePanelHost();
let data = $state<ContextResponse | null>(null);
let graph = $state<VaultGraph | null>(null);
let graphError = $state<string | null>(null);
let failure = $state<string | null>(null);
let promptOpen = $state(false);
let promptText = $state<string | null>(null);
let filesOpen = $state(false);
async function load() {
failure = null;
try {
data = await client.context(conversationId);
} catch (cause) {
failure = cause instanceof Error ? cause.message : String(cause);
return;
}
const paths = data.files.map((f) => f.path);
if (paths.length === 0) {
graph = null;
return;
}
try {
graph = await client.vaultGraph(paths.slice(0, 40));
graphError = null;
} catch (cause) {
graph = null;
graphError = cause instanceof Error ? cause.message : String(cause);
}
}
async function showPrompt() {
promptOpen = !promptOpen;
if (promptOpen && promptText === null) {
try {
const full = await client.context(conversationId, true);
promptText = full.prompt_text ?? "";
} catch (cause) {
promptText = cause instanceof Error ? cause.message : String(cause);
}
}
}
onMount(load);
interface Layer {
label: string;
tokens: number;
tone: string;
}
const layers = $derived.by((): Layer[] => {
if (!data) {
return [];
}
const prompt = data.prompt.tokens_est;
const skills = data.skills_tokens_est;
const harness = Math.min(
HARNESS_EST,
Math.max(0, data.context_tokens - prompt - skills)
);
const history = Math.max(
0,
data.context_tokens - prompt - skills - harness
);
return [
{ label: "harness", tokens: harness, tone: "bg-muted-foreground/40" },
{ label: "prompt", tokens: prompt, tone: "bg-primary" },
{ label: "skills", tokens: skills, tone: "bg-kind-deep" },
{ label: "history & tools", tokens: history, tone: "bg-kind-branch" },
];
});
const total = $derived(data?.context_tokens ?? 0);
const share = (tokens: number) =>
total > 0 ? `${Math.max(0.5, (tokens / total) * 100)}%` : "0%";
const graphNodes = $derived.by((): GraphNode[] => {
if (!graph) {
return [];
}
const single = graph.nodes.filter((n) => n.touched).length === 1;
return graph.nodes.map((n) => ({
href: n.exists ? n.path : undefined,
id: n.path,
kind: "file",
label: n.title,
ring: ringOf(n.touched, single),
state: stateOf(n.touched, n.exists),
}));
});
const graphEdges = $derived.by((): GraphEdge[] =>
graph ? graph.edges.map((e) => ({ from: e.from, to: e.to })) : []
);
const files = $derived(
filesOpen ? (data?.files ?? []) : (data?.files ?? []).slice(0, FILES_SHOWN)
);
function ringOf(touched: boolean, single: boolean): number {
if (!touched) {
return 2;
}
return single ? 0 : 1;
}
function stateOf(touched: boolean, exists: boolean): GraphNode["state"] {
if (!touched) {
return "ghost";
}
return exists ? "quiet" : "closed";
}
function openNote(path: string) {
host.openLink(path.replace(MD_SUFFIX, ""), "internal");
}
function touch(file: {
reads: number;
writes: number;
other: number;
}): string {
const parts: string[] = [];
if (file.reads) {
parts.push(`read ×${file.reads}`);
}
if (file.writes) {
parts.push(`wrote ×${file.writes}`);
}
if (file.other) {
parts.push(`touched ×${file.other}`);
}
return parts.join(" · ");
}
</script>
<div class="@container flex max-w-5xl flex-col gap-8">
{#if failure}
<ErrorNote message={failure} retry={load} />
{:else if !data}
<p class="text-muted-foreground text-sm">Reading the context…</p>
{:else}
<section class="flex flex-col gap-3">
<div class="flex items-baseline gap-3">
<span
class="tabular font-semibold text-3xl leading-none tracking-tight"
>
{fmtTokens(total)}
</span>
<span class="text-muted-foreground text-sm">
of {fmtTokens(window)} · {data.turns}
{data.turns === 1 ? "turn" : "turns"}
· {data.agent.model}
{data.agent.effort
? ` · ${data.agent.effort}`
: ""}
</span>
</div>
<div class="flex h-2 w-full overflow-hidden rounded-full bg-muted">
{#each layers as layer (layer.label)}
{#if layer.tokens > 0}
<span
class={cn("h-full", layer.tone)}
style="width: {share(layer.tokens)}"
title="{layer.label}: {fmtTokens(layer.tokens)}"
></span>
{/if}
{/each}
</div>
<div class="flex flex-wrap gap-x-5 gap-y-1 text-xs">
{#each layers as layer (layer.label)}
<span class="flex items-center gap-1.5">
<span class={cn("size-2 rounded-full", layer.tone)}></span>
<span class="text-muted-foreground">{layer.label}</span>
<span class="tabular">{fmtTokens(layer.tokens)}</span>
</span>
{/each}
<span class="text-muted-foreground/70"
>estimates, except the total</span
>
</div>
</section>
<div class="grid gap-8 @3xl:grid-cols-[minmax(0,1fr)_22rem]">
<div class="flex min-w-0 flex-col gap-8">
<section class="flex flex-col gap-2">
<div class="flex items-baseline gap-3">
<h2 class="font-medium text-sm">System prompt</h2>
<span class="tabular text-muted-foreground text-xs">
{fmtTokens(data.prompt.tokens_est)}
· {data.prompt.granules.length} granules
</span>
<button
class="rule-word ml-auto text-xs"
data-active={promptOpen}
onclick={showPrompt}
type="button"
>
{promptOpen ? "hide text" : "read it"}
</button>
</div>
<div class="bay-rack">
{#each data.prompt.granules as g, index (index)}
<div class="strip text-sm">
<span
class="size-2 justify-self-center rounded-full bg-primary"
></span>
<span class="flex min-w-0 flex-col leading-tight">
<button
class="truncate text-left font-medium hover:underline"
onclick={() => g.path && openNote(g.path)}
type="button"
>
{g.path ?? "verbatim prompt"}
</button>
{#if g.tag}
<span
class="truncate font-mono text-[10px] text-muted-foreground"
>
&lt;{g.tag}&gt;
</span>
{/if}
</span>
<span class="tabular text-muted-foreground text-xs">
{fmtTokens(g.tokens_est)}
</span>
</div>
{/each}
</div>
{#if promptOpen}
<pre
class="max-h-[32rem] overflow-auto rounded-lg border bg-strip p-3 text-xs whitespace-pre-wrap break-words"
>{promptText ?? "…"}</pre>
{/if}
</section>
<section class="flex flex-col gap-2">
<div class="flex items-baseline gap-3">
<h2 class="font-medium text-sm">Skills</h2>
<span class="tabular text-muted-foreground text-xs">
{data.skills.reduce((n, s) => n + s.skills.length, 0)}
loaded
</span>
</div>
{#if data.skills.length === 0}
<p class="text-muted-foreground text-sm">None for this kind.</p>
{:else}
<div class="bay-rack">
{#each data.skills as set (set.path)}
{#each set.skills as skill (skill.path)}
<div class="strip text-sm">
<span
class="size-2 justify-self-center rounded-full bg-kind-deep"
></span>
<span class="flex min-w-0 flex-col leading-tight">
<span class="truncate font-medium">{skill.name}</span>
{#if skill.description}
<span class="truncate text-muted-foreground text-xs">
{skill.description}
</span>
{/if}
</span>
<span class="text-muted-foreground text-xs">{set.set}</span>
</div>
{/each}
{/each}
</div>
{/if}
</section>
<section class="flex flex-col gap-2">
<h2 class="font-medium text-sm">Tools</h2>
<dl
class="grid grid-cols-[auto_minmax(0,1fr)] gap-x-3 gap-y-1 text-xs"
>
{#if Object.keys(data.tool_counts).length > 0}
<dt class="text-muted-foreground">used</dt>
<dd class="tabular">
{Object.entries(data.tool_counts)
.sort((a, b) => b[1] - a[1])
.map(([name, n]) => `${name.replace("mcp__", "")} ×${n}`)
.join(" ")}
</dd>
{/if}
{#if data.tools.gateway.length}
<dt class="text-muted-foreground">gateway</dt>
<dd>{data.tools.gateway.join(", ")}</dd>
{/if}
{#if data.tools.mcps.length}
<dt class="text-muted-foreground">mcp</dt>
<dd>{data.tools.mcps.join(", ")}</dd>
{/if}
{#if data.tools.allowed}
<dt class="text-muted-foreground">allowed</dt>
<dd>{data.tools.allowed.join(", ")}</dd>
{/if}
{#if data.tools.disallowed.length}
<dt class="text-muted-foreground">off</dt>
<dd class="text-muted-foreground">
{data.tools.disallowed.join(", ")}
</dd>
{/if}
</dl>
</section>
</div>
<div class="flex min-w-0 flex-col gap-8">
<section class="flex flex-col gap-2">
<div class="flex items-baseline gap-3">
<h2 class="font-medium text-sm">Notes it reached</h2>
<span class="tabular text-muted-foreground text-xs"
>{data.files.length}</span
>
</div>
{#if graph && graph.nodes.length > 0}
<div class="h-72 overflow-hidden rounded-lg border bg-strip/60">
<Graph edges={graphEdges} nodes={graphNodes} onOpen={openNote} />
</div>
<p class="text-muted-foreground text-xs">
Solid: read or written by this thread. Hollow: one link away, not
reached.
</p>
{:else if graphError}
<p class="text-muted-foreground text-xs">{graphError}</p>
{/if}
{#if data.files.length === 0}
<p class="text-muted-foreground text-sm">No files touched yet.</p>
{:else}
<div class="bay-rack">
{#each files as file (file.path)}
<button
class="strip w-full text-left text-sm"
onclick={() => openNote(file.path)}
type="button"
>
<span
class={cn(
"size-2 justify-self-center rounded-full",
file.writes ? "bg-primary" : "bg-kind-branch"
)}
></span>
<span class="flex min-w-0 flex-col leading-tight">
<span class="truncate font-medium">{file.path}</span>
<span class="truncate text-muted-foreground text-xs"
>{touch(file)}</span
>
</span>
<span class="tabular text-muted-foreground text-xs">
{fmtRelative(file.last_at)}
</span>
</button>
{/each}
</div>
{#if data.files.length > FILES_SHOWN}
<button
class="rule-word self-start text-xs"
onclick={() => {
filesOpen = !filesOpen;
}}
type="button"
>
{filesOpen ? "fewer" : `all ${data.files.length}`}
</button>
{/if}
{/if}
</section>
</div>
</div>
{/if}
</div>
+78 -175
View File
@@ -1,16 +1,15 @@
<script lang="ts">
import ChevronDownIcon from "@lucide/svelte/icons/chevron-down";
import GitBranchIcon from "@lucide/svelte/icons/git-branch";
import type { ApiClient } from "$lib/api/client";
import type { LiveState } from "$lib/api/live.svelte";
import type { ConversationInfo } from "$lib/api/types";
import { fmtRelative, fmtTokens, shortId } from "$lib/format";
import KindMark from "$lib/shell/kind-mark.svelte";
import { cn } from "$lib/utils";
import BindingsList from "./bindings-list.svelte";
import KindBadge from "$lib/components/kind-badge.svelte";
import LiveDot from "$lib/components/live-dot.svelte";
import StatusPill from "$lib/components/status-pill.svelte";
import { Button } from "$lib/components/ui/button";
import { clip, fmtRelative, shortId } from "$lib/format";
import BranchDialog from "./branch-dialog.svelte";
import ConversationMenu from "./conversation-menu.svelte";
import QueueList from "./queue-list.svelte";
let {
client,
@@ -21,119 +20,63 @@
onChanged,
onOpen,
showTitle = true,
view = $bindable("chat"),
onContext,
}: {
client: ApiClient;
info: ConversationInfo;
liveState: LiveState;
liveDetail?: string | null;
// Where another conversation lives as a link; without it, parents open in place.
href?: ((id: string) => string) | null;
onChanged: () => void;
onOpen: (id: string) => void;
showTitle?: boolean;
view?: string;
onContext?: () => void;
} = $props();
const VIEWS = [
{ label: "Thread", value: "chat" },
{ label: "Activity", value: "activity" },
{ label: "Raw", value: "raw" },
{ label: "Context", value: "context" },
];
let branchOpen = $state(false);
let detailsOpen = $state(false);
const mood = $derived.by(() => {
if (info.running_turn) {
return {
dot: "bg-signal animate-pulse-dot",
text: "in motion",
tone: "text-signal",
};
}
if (info.pending_question) {
return {
dot: "bg-attention",
text: "waiting for you",
tone: "text-attention-foreground",
};
}
if (info.status !== "open") {
return {
dot: "bg-muted-foreground/50",
text: info.status,
tone: "text-muted-foreground",
};
}
return {
dot: "bg-muted-foreground/50",
text: `quiet · ${fmtRelative(info.last_activity_at)}`,
tone: "text-muted-foreground",
};
});
const WINDOW_MAX = 56;
const windows = $derived(info.bindings.filter((b) => b.visible));
const queued = $derived(
info.queue.filter((item) => item.status === "queued").length
);
const connection = $derived(
liveState === "open" ? null : (liveDetail ?? liveState)
);
</script>
<header class="flex flex-col border-b bg-background/80 backdrop-blur-md">
<div class="flex items-center gap-2 px-3 py-2 @md:px-6">
<header class="flex flex-col gap-1.5 border-b px-3 py-2.5 @md:px-6">
<div class="flex flex-wrap items-center gap-x-3 gap-y-1">
{#if showTitle}
<KindMark kind={info.kind} />
<KindBadge kind={info.kind} />
<h1 class="min-w-0 truncate font-semibold text-base tracking-tight">
{info.title || `${info.kind} ${shortId(info.id)}`}
</h1>
{/if}
<span class={cn("flex shrink-0 items-center gap-1.5 text-xs", mood.tone)}>
<span class={cn("size-1.5 rounded-full", mood.dot)}></span>
<span class="hidden @sm:inline">{mood.text}</span>
</span>
{#if queued > 0}
<span
class="tabular text-note text-xs"
title="messages waiting for the next turn"
>
+{queued}
</span>
<StatusPill status={info.running_turn ? "running" : info.status} />
{#if info.pending_question}
<span class="font-medium text-link text-xs">question pending</span>
{/if}
{#if connection}
<span class="truncate text-warn text-xs" title={liveDetail ?? undefined}>
{connection}
</span>
{/if}
<div class="ml-auto flex shrink-0 items-center gap-1">
{#if info.context_tokens}
<button
class="rule-word tabular inline-flex h-7 items-center gap-1 rounded-md px-2 text-xs hover:bg-accent"
onclick={() => {
view = "context";
onContext?.();
}}
title="context of the last turn - what the model is holding"
type="button"
>
{fmtTokens(info.context_tokens)}
<span class="text-muted-foreground/70">/ 1M</span>
</button>
{/if}
<button
<div class="ml-auto flex items-center gap-1">
<LiveDot
class="hidden @md:inline-flex"
detail={liveDetail}
state={liveState}
/>
<LiveDot
class="@md:hidden"
detail={liveDetail}
label={false}
state={liveState}
/>
<Button
aria-label="Branch off this conversation"
class="rule-word inline-flex size-7 items-center justify-center rounded-md hover:bg-accent disabled:opacity-40"
disabled={info.status !== "open"}
onclick={() => {
branchOpen = true;
}}
title="Branch off"
type="button"
size="sm"
variant="outline"
>
<GitBranchIcon class="size-4" />
</button>
<span class="hidden @md:inline">Branch</span>
</Button>
<ConversationMenu
{client}
current
@@ -143,99 +86,59 @@
{onChanged}
{onOpen}
/>
<button
aria-expanded={detailsOpen}
aria-label="Details"
class={cn(
"rule-word inline-flex size-7 items-center justify-center rounded-md hover:bg-accent",
detailsOpen && "bg-accent text-foreground"
)}
onclick={() => {
detailsOpen = !detailsOpen;
}}
title="Details"
type="button"
>
<ChevronDownIcon
class={cn("size-4 transition-transform", detailsOpen && "rotate-180")}
/>
</button>
</div>
</div>
<div class="flex items-center gap-4 px-3 pb-1.5 @md:px-6">
{#each VIEWS as item (item.value)}
<button
class="rule-word text-xs"
data-active={view === item.value}
onclick={() => {
view = item.value;
}}
type="button"
<!-- Below @md the column is the live thread's; the rest of this lives in Meta. -->
<div
class="flex flex-wrap items-center gap-x-3 gap-y-0.5 text-muted-foreground text-xs"
>
<span class="hidden @md:inline">{info.agent}</span>
<span class="hidden @md:inline">via {info.origin}</span>
{#if info.parent}
{#if href}
<a class="text-link hover:underline" href={href(info.parent)}>
parent {shortId(info.parent)}
</a>
{:else}
<button
class="text-link hover:underline"
onclick={() => info.parent && onOpen(info.parent)}
type="button"
>
parent {shortId(info.parent)}
</button>
{/if}
{/if}
<span class="tabular hidden @md:inline" title={info.id}>
{shortId(info.id)}
</span>
<span class="tabular hidden @md:inline">
{info.live ? "session live" : "no live session"}
{info.busy ? " · busy" : ""}
</span>
<span class="tabular">
last activity {fmtRelative(info.last_activity_at)}
</span>
{#if queued > 0}
<span
class="tabular font-medium text-note"
title="messages waiting for the next turn"
>
{item.label}
</button>
{/each}
{queued}
queued
</span>
{/if}
</div>
{#if detailsOpen}
{#if windows.length > 0}
<div
class="animate-fade-in grid gap-x-6 gap-y-3 border-t px-3 py-3 text-xs @md:grid-cols-2 @md:px-6"
class="hidden flex-wrap items-center gap-x-3 gap-y-0.5 text-muted-foreground text-xs @md:flex"
>
<dl class="grid grid-cols-[auto_minmax(0,1fr)] gap-x-3 gap-y-1">
<dt class="text-muted-foreground">agent</dt>
<dd>{info.agent}</dd>
<dt class="text-muted-foreground">opened via</dt>
<dd>{info.origin}</dd>
{#if info.parent}
<dt class="text-muted-foreground">parent</dt>
<dd>
{#if href}
<a class="text-link hover:underline" href={href(info.parent)}>
{shortId(info.parent)}
</a>
{:else}
<button
class="text-link hover:underline"
onclick={() => info.parent && onOpen(info.parent)}
type="button"
>
{shortId(info.parent)}
</button>
{/if}
</dd>
{/if}
<dt class="text-muted-foreground">id</dt>
<dd class="tabular break-all">{info.id}</dd>
<dt class="text-muted-foreground">session</dt>
<dd class="tabular break-all">
{info.session_id ?? "none yet"}
{info.live ? " · process alive" : ""}
</dd>
{#if Object.keys(info.flags).length > 0}
<dt class="text-muted-foreground">flags</dt>
<dd class="tabular">
{Object.entries(info.flags)
.map(([k, v]) => `${k}=${JSON.stringify(v)}`)
.join(" ")}
</dd>
{/if}
</dl>
<div class="flex flex-col gap-3">
<div class="flex flex-col gap-1">
<span class="text-muted-foreground">windows</span>
<BindingsList
bindings={info.bindings}
{client}
conversationId={info.id}
{onChanged}
/>
</div>
{#if info.queue.length > 0}
<div class="flex flex-col gap-1">
<span class="text-muted-foreground">queue</span>
<QueueList items={info.queue} />
</div>
{/if}
</div>
{#each windows as window (window.frontend + window.external_id)}
<span class="truncate" title="{window.frontend}: {window.external_id}">
<span class="text-foreground/70">{window.frontend}</span>
{clip(window.external_id, WINDOW_MAX)}
</span>
{/each}
</div>
{/if}
</header>
+3 -21
View File
@@ -13,7 +13,6 @@
import { usePanelHost } from "./host";
import MenuItems, { type MenuKit } from "./menu-items.svelte";
import RenameDialog from "./rename-dialog.svelte";
import RewindDialog from "./rewind-dialog.svelte";
let {
client,
@@ -25,7 +24,6 @@
onChanged,
class: className = "",
children,
trigger,
}: {
client: ApiClient;
id: string;
@@ -37,9 +35,6 @@
onChanged: () => void;
class?: string;
children?: Snippet;
// Render the trigger yourself with these props spread on it - the strip
// stays a strip instead of sitting inside a wrapper div.
trigger?: Snippet<[Record<string, unknown>]>;
} = $props();
const host = usePanelHost();
@@ -48,7 +43,6 @@
let failure = $state<string | null>(null);
let branchOpen = $state(false);
let renameOpen = $state(false);
let rewindOpen = $state(false);
const loaded = $derived(info ?? fetched);
@@ -79,9 +73,6 @@
onRename: () => {
renameOpen = true;
},
onRewind: () => {
rewindOpen = true;
},
});
const actions = $derived(loaded ? conversationActions(loaded, scope) : []);
</script>
@@ -102,17 +93,9 @@
{#if mode === "context"}
<ContextMenu.Root bind:open>
{#if trigger}
<ContextMenu.Trigger>
{#snippet child({ props })}
{@render trigger(props)}
{/snippet}
</ContextMenu.Trigger>
{:else}
<ContextMenu.Trigger class={className}>
{@render children?.()}
</ContextMenu.Trigger>
{/if}
<ContextMenu.Trigger class={className}>
{@render children?.()}
</ContextMenu.Trigger>
<ContextMenu.Content class="min-w-52">
{@render body(ContextMenu)}
</ContextMenu.Content>
@@ -146,5 +129,4 @@
bind:open={branchOpen}
/>
<RenameDialog {client} info={loaded} {onChanged} bind:open={renameOpen} />
<RewindDialog {client} info={loaded} {onChanged} bind:open={rewindOpen} />
{/if}
+219
View File
@@ -0,0 +1,219 @@
<script lang="ts">
import CheckIcon from "@lucide/svelte/icons/check";
import Link2Icon from "@lucide/svelte/icons/link-2";
import Link2OffIcon from "@lucide/svelte/icons/link-2-off";
import type { ApiClient } from "$lib/api/client";
import type { ConversationSummary, Kind } from "$lib/api/types";
import EmptyState from "$lib/components/empty-state.svelte";
import ErrorNote from "$lib/components/error-note.svelte";
import { Button } from "$lib/components/ui/button";
import { Input } from "$lib/components/ui/input";
import { Skeleton } from "$lib/components/ui/skeleton";
import ConversationRow from "./conversation-row.svelte";
import type { ConversationIndex } from "./index.svelte";
let {
client,
index,
selected = null,
onOpen,
follow = $bindable(false),
showFollow = false,
autofocus = false,
}: {
client: ApiClient;
index: ConversationIndex;
selected?: string | null;
onOpen: (id: string) => void;
follow?: boolean;
showFollow?: boolean;
autofocus?: boolean;
} = $props();
const GROUPS: { kind: Kind; label: string }[] = [
{ kind: "master", label: "Master" },
{ kind: "branch", label: "Branches" },
{ kind: "deep", label: "Deep chats" },
{ kind: "job", label: "Jobs" },
{ kind: "fork", label: "Forks" },
];
let search = $state("");
let withClosed = $state(false);
let list = $state<HTMLDivElement | null>(null);
let searchInput = $state<HTMLInputElement | null>(null);
function activityOf(row: ConversationSummary): string {
return row.last_activity_at ?? row.created_at ?? "";
}
function byActivity(a: ConversationSummary, b: ConversationSummary) {
if (Boolean(a.running_turn) !== Boolean(b.running_turn)) {
return a.running_turn ? -1 : 1;
}
return activityOf(b).localeCompare(activityOf(a));
}
const rows = $derived.by(() => {
const needle = search.trim().toLowerCase();
return index.conversations
.filter(
(row) => withClosed || row.status === "open" || row.id === selected
)
.filter(
(row) =>
!needle ||
(row.title ?? "").toLowerCase().includes(needle) ||
row.id.startsWith(needle) ||
row.agent.toLowerCase().includes(needle) ||
(row.last_item?.text ?? "").toLowerCase().includes(needle)
)
.sort(byActivity);
});
const groups = $derived(
GROUPS.map((group) => ({
...group,
rows: rows.filter((row) => row.kind === group.kind),
})).filter((group) => group.rows.length > 0)
);
function refresh() {
index.load().catch(() => undefined);
}
// ↑/↓ walk the rows, / jumps to the search box, Enter is the button's own.
function onKeydown(event: KeyboardEvent) {
if (event.key === "/" && event.target !== searchInput) {
event.preventDefault();
searchInput?.focus();
return;
}
if (event.key !== "ArrowDown" && event.key !== "ArrowUp") {
return;
}
const buttons = [
...(list?.querySelectorAll<HTMLButtonElement>("[data-row]") ?? []),
];
if (buttons.length === 0) {
return;
}
event.preventDefault();
const at = buttons.indexOf(document.activeElement as HTMLButtonElement);
const step = event.key === "ArrowDown" ? 1 : -1;
const next = at < 0 ? 0 : (at + step + buttons.length) % buttons.length;
buttons[next].focus();
}
$effect(() => {
if (autofocus) {
searchInput?.focus();
}
});
</script>
<!-- svelte-ignore a11y_no_noninteractive_element_interactions -->
<div
aria-label="Conversations"
class="flex h-full min-h-0 flex-col"
onkeydown={onKeydown}
role="listbox"
tabindex="-1"
>
<div class="flex items-center gap-1.5 border-b px-2 py-1.5">
<Input
aria-label="Search conversations"
class="h-8 min-w-0 flex-1 text-xs"
placeholder="search"
bind:ref={searchInput}
bind:value={search}
/>
<Button
aria-pressed={withClosed}
class="text-xs"
onclick={() => {
withClosed = !withClosed;
}}
size="xs"
title={withClosed
? "Hide closed conversations"
: "Show closed conversations"}
variant={withClosed ? "secondary" : "ghost"}
>
{#if withClosed}
<CheckIcon class="size-3" />
{/if}
closed
</Button>
{#if showFollow}
<Button
aria-label="Follow the active note"
aria-pressed={follow}
onclick={() => {
follow = !follow;
}}
size="icon-sm"
title={follow
? "Following the active note - click to pin this conversation"
: "Pinned - click to follow the active note"}
variant={follow ? "secondary" : "ghost"}
>
{#if follow}
<Link2Icon class="size-4" />
{:else}
<Link2OffIcon class="size-4" />
{/if}
</Button>
{/if}
</div>
<div class="min-h-0 flex-1 overflow-y-auto" bind:this={list}>
{#if index.live.state === "failed"}
<div class="p-3">
<ErrorNote
message={index.live.detail ?? "event stream failed"}
retry={() => index.start()}
/>
</div>
{:else if !index.loaded}
<div class="flex flex-col gap-2 p-3">
<Skeleton class="h-12 w-full" />
<Skeleton class="h-12 w-full" />
<Skeleton class="h-12 w-full" />
</div>
{:else if rows.length === 0}
<div class="p-3">
<EmptyState
hint={search
? "Nothing matches. Try fewer letters, or include closed ones."
: "Nothing is open. Say something to the master in Telegram, or send a note from Obsidian."}
title="No conversations"
/>
</div>
{:else}
{#each groups as group (group.kind)}
<section class="flex flex-col">
<h2
class="sticky top-0 z-10 bg-background/95 px-3 pt-3 pb-1 font-medium text-muted-foreground text-xs uppercase tracking-wide backdrop-blur-sm"
>
{group.label}
<span class="tabular ml-1 normal-case tracking-normal">
{group.rows.length}
</span>
</h2>
<ul class="flex flex-col">
{#each group.rows as row (row.id)}
<li class="border-b last:border-b-0">
<ConversationRow
{client}
onChanged={refresh}
{onOpen}
{row}
selected={selected === row.id}
/>
</li>
{/each}
</ul>
</section>
{/each}
{/if}
</div>
</div>
+91
View File
@@ -0,0 +1,91 @@
<script lang="ts">
import type { ApiClient } from "$lib/api/client";
import type { ConversationSummary } from "$lib/api/types";
import KindBadge from "$lib/components/kind-badge.svelte";
import StatusPill from "$lib/components/status-pill.svelte";
import { clip, fmtRelative, shortId } from "$lib/format";
import { cn } from "$lib/utils";
import ConversationMenu from "./conversation-menu.svelte";
import { usePanelHost } from "./host";
let {
client,
row,
selected = false,
onOpen,
onChanged,
}: {
client: ApiClient;
row: ConversationSummary;
selected?: boolean;
onOpen: (id: string) => void;
onChanged: () => void;
} = $props();
const host = usePanelHost();
const TITLE_MAX = 60;
const PREVIEW_MAX = 90;
const title = $derived(
row.title
? clip(row.title, TITLE_MAX)
: `${row.kind === "master" ? "Master" : row.kind} · ${shortId(row.id)}`
);
const when = $derived(row.last_activity_at ?? row.created_at ?? null);
</script>
<ConversationMenu
class={cn("group/row block", selected && "bg-sidebar-accent")}
{client}
current={selected}
id={row.id}
mode="context"
{onChanged}
{onOpen}
>
<div class="relative">
<button
aria-current={selected ? "true" : undefined}
class="row-hover flex w-full flex-col items-stretch justify-start gap-1 px-3 py-2 text-left text-sm"
data-row={row.id}
onclick={() => onOpen(row.id)}
type="button"
>
<span class="flex w-full items-center gap-2">
<KindBadge kind={row.kind} />
<span class="min-w-0 flex-1 truncate font-medium">{title}</span>
<span class="tabular shrink-0 text-muted-foreground text-xs">
{fmtRelative(when)}
</span>
</span>
<span class="flex w-full items-center gap-2 pr-7 text-xs">
<StatusPill status={row.running_turn ? "running" : row.status} />
<span class="truncate text-muted-foreground">{row.agent}</span>
{#if row.pending_question}
<span class="shrink-0 font-medium text-link">question</span>
{/if}
</span>
{#if row.last_item}
<span
class="block w-full min-w-0 truncate text-muted-foreground text-xs"
>
<span class="text-foreground/70">{row.last_item.origin}:</span>
{clip(row.last_item.text, PREVIEW_MAX)}
</span>
{/if}
</button>
<ConversationMenu
class={cn(
"absolute right-1.5 bottom-1 size-7 opacity-0 transition-opacity focus-visible:opacity-100 group-hover/row:opacity-100 aria-expanded:opacity-100",
host.touch && "opacity-100",
row.last_item && "bottom-6"
)}
{client}
current={selected}
id={row.id}
mode="button"
{onChanged}
{onOpen}
/>
</div>
</ConversationMenu>
+100 -41
View File
@@ -2,13 +2,15 @@
import { onMount, untrack } from "svelte";
import type { ApiClient } from "$lib/api/client";
import ErrorNote from "$lib/components/error-note.svelte";
import { Skeleton } from "$lib/components/ui/skeleton";
import * as Tabs from "$lib/components/ui/tabs";
import ActivityFeed from "./activity-feed.svelte";
import BindingsList from "./bindings-list.svelte";
import ChatView from "./chat-view.svelte";
import Composer from "./composer.svelte";
import ContextView from "./context-view.svelte";
import { ConversationFeed } from "./conversation.svelte";
import ConversationHeader from "./conversation-header.svelte";
import { usePanelHost } from "./host";
import QueueList from "./queue-list.svelte";
import RawEntries from "./raw-entries.svelte";
let {
@@ -17,8 +19,6 @@
href = null,
onOpen,
showTitle = true,
initialView = "chat",
onViewChange,
}: {
client: ApiClient;
id: string;
@@ -26,19 +26,13 @@
onOpen: (id: string) => void;
// Off when a switcher above the thread already names it.
showTitle?: boolean;
// "activity" beside a deep chat's own note: the file is the thread.
initialView?: string;
// The page keeps the open tab in the URL so it survives a reload.
onViewChange?: (view: string) => void;
} = $props();
const host = usePanelHost();
const feed = new ConversationFeed(
() => client,
untrack(() => id),
host.cache ?? null
untrack(() => id)
);
let view = $state(untrack(() => initialView));
let tab = $state("chat");
let historyKey = $state(0);
onMount(() => {
@@ -53,10 +47,6 @@
historyKey += 1;
}
$effect(() => {
onViewChange?.(view);
});
$effect(() => {
const { running } = feed.model;
if (!running) {
@@ -71,10 +61,9 @@
{#if feed.error && !info}
<div class="p-4"><ErrorNote message={feed.error} retry={refresh} /></div>
{:else if !info}
<div class="flex flex-1 items-center justify-center p-4">
<span
class="size-2 animate-pulse-dot rounded-full bg-muted-foreground/40"
></span>
<div class="flex flex-col gap-3 p-4">
<Skeleton class="h-8 w-1/2" />
<Skeleton class="h-24 w-full" />
</div>
{:else}
<ConversationHeader
@@ -86,36 +75,49 @@
onChanged={refresh}
{onOpen}
{showTitle}
bind:view
/>
<div class="flex min-h-0 flex-1 flex-col">
{#if view === "chat"}
<ChatView
{client}
conversationId={id}
model={feed.model}
refreshKey={historyKey}
/>
{:else if view === "activity"}
<div class="min-h-0 flex-1 overflow-y-auto px-3 py-3 @md:px-6">
<Tabs.Root class="flex min-h-0 flex-1 flex-col gap-0" bind:value={tab}>
<div class="border-b px-3 py-2 @md:px-6">
<Tabs.List class="w-fit max-w-full">
<Tabs.Trigger value="chat">Chat</Tabs.Trigger>
<Tabs.Trigger value="activity">Activity</Tabs.Trigger>
<Tabs.Trigger value="raw">Raw</Tabs.Trigger>
<Tabs.Trigger value="meta">Meta</Tabs.Trigger>
</Tabs.List>
</div>
<Tabs.Content class="flex min-h-0 flex-1 flex-col" value="chat">
<ChatView
{client}
conversationId={id}
model={feed.model}
refreshKey={historyKey}
/>
</Tabs.Content>
<Tabs.Content
class="min-h-0 flex-1 overflow-y-auto px-3 py-3 @md:px-6"
value="activity"
>
<ActivityFeed
{client}
connected={feed.live.state === "open"}
conversationId={id}
model={feed.model}
/>
</div>
{:else if view === "context"}
<div class="min-h-0 flex-1 overflow-y-auto px-3 py-4 @md:px-6">
{#key historyKey}
<ContextView {client} conversationId={id} />
{/key}
</div>
{:else}
<div class="min-h-0 flex-1 overflow-y-auto px-3 py-3 @md:px-6">
</Tabs.Content>
<Tabs.Content
class="min-h-0 flex-1 overflow-y-auto px-3 py-3 @md:px-6"
value="raw"
>
<RawEntries {client} conversationId={id} />
</div>
{/if}
</Tabs.Content>
<Tabs.Content
class="min-h-0 flex-1 overflow-y-auto px-3 py-3 @md:px-6"
value="meta"
>
{@render meta()}
</Tabs.Content>
</Tabs.Root>
<Composer
{client}
conversationId={id}
@@ -124,3 +126,60 @@
</div>
{/if}
</div>
{#snippet meta()}
{#if info}
<div class="flex max-w-3xl flex-col gap-5">
<section class="flex flex-col gap-1.5">
<h2
class="font-medium text-muted-foreground text-xs uppercase tracking-wide"
>
Queue
</h2>
<QueueList items={info.queue} />
</section>
<section class="flex flex-col gap-1.5">
<h2
class="font-medium text-muted-foreground text-xs uppercase tracking-wide"
>
Windows
</h2>
<BindingsList
bindings={info.bindings}
{client}
conversationId={id}
onChanged={refresh}
/>
</section>
<section class="flex flex-col gap-1.5">
<h2
class="font-medium text-muted-foreground text-xs uppercase tracking-wide"
>
Flags
</h2>
{#if Object.keys(info.flags).length === 0}
<p class="text-muted-foreground text-xs">No flags set.</p>
{:else}
<dl
class="grid grid-cols-[auto_minmax(0,1fr)] gap-x-3 gap-y-1 text-xs"
>
{#each Object.entries(info.flags) as [key, value] (key)}
<dt class="text-muted-foreground">{key}</dt>
<dd class="truncate">{JSON.stringify(value)}</dd>
{/each}
</dl>
{/if}
</section>
<section class="flex flex-col gap-1.5">
<h2
class="font-medium text-muted-foreground text-xs uppercase tracking-wide"
>
Session
</h2>
<p class="tabular break-all text-muted-foreground text-xs">
{info.session_id ?? "no session yet"}
</p>
</section>
</div>
{/if}
{/snippet}
Loaded 100 of 152 files, more files were not shown because too many files have changed in this diff. Show more