Compare commits

...
49 Commits
Author SHA1 Message Date
hh 9b1127acff chore(deps): claude-agent-sdk 0.2.150 bundles cli 2.1.257 2026-09-01 22:00:41 +02:00
hh 1e08dfb9da fix(mcp,redact,gateway_tools): mask credentials in tool results, names kept and values gone 2026-09-01 14:12:49 +00:00
hh 8e7db6baa4 fix(redact,cli): mask credentials in every log line, silence httpx request urls 2026-09-01 14:12:49 +00:00
hh acc79c2e42 feat(envelope,conversations): recall hook under the user text, user_sink for the setup's reply log 2026-09-01 16:08:53 +02:00
hh 63014a487d fix(storage,api): usage cost as per-turn delta of the sdk running total, backfill 2026-09-01 15:37:53 +02:00
hh 3406e2e178 feat(injects,conversations,scheduler,gateway_tools,api): wake priority, master and parent aliases, scheduled injects start a turn 2026-08-30 20:34:12 +02:00
hh 878f7d6473 fix(cli): mask query tokens in uvicorn access log 2026-08-30 18:30:26 +02:00
hh c438d4cb92 feat(auth): bootstrap token entries carry an optional scope 2026-08-30 18:25:48 +02:00
hh ba9d63667e feat(auth): query-string token for headerless webhook senders 2026-08-30 18:10:00 +02:00
hh d57f660a30 fix(gateway_tools,conversations): say refused in user turns, the reply reaches the human itself 2026-08-30 02:26:35 +02:00
hh cc32307921 feat(scheduler): job dedupe flag for webhooks that must not collapse 2026-08-29 22:15:15 +02:00
hh 811035ba4f fix(ui): chat keeps the bottom pinned while async markdown lands 2026-08-29 19:26:45 +02:00
hh 9efff6e814 feat(ui,api,telegram): shared conversation panel for obsidian and /admin/panel, bind materializes a window, inbox survives link previews 2026-08-29 18:30:33 +02:00
hh a989d2909d feat(injects,conversations): configurable inject header, urgent inject marks the turn it cut 2026-08-29 17:07:43 +02:00
hh 8d252f9867 fix(claude_sdk,conversations,rotation): context size from the last api call, new-day text by rotation reason 2026-08-29 16:54:29 +02:00
hh 2895e677c0 feat(policy,claude_sdk,cli,config_loader): pretooluse policy rules per agent, tool call audit, sibling imports for config 2026-08-29 15:27:26 +02:00
hh 23cfb16228 feat(distill,conversations,scheduler,api,ui): close deep chats with a distiller fork, digest and index, idle and weekly state jobs 2026-08-29 03:15:39 +02:00
hh ab9ce50b82 refactor(envelope): drop the reply counter and the injects line, keep time and vault changes 2026-08-29 02:38:39 +02:00
hh 662bdcdfda feat(scheduler,rotation,envelope,api,ui): pgqueuer jobs and deferred injects, master rotation with handout, vault envelope, jobs page 2026-08-29 02:29:05 +02:00
hh 0479d7cdda feat(conversations): pre-SDK rows read from canonical messages, implied titles, markdown adopts them by first prompt 2026-08-29 01:44:05 +02:00
hh 0cc2c5291f fix(markdown): a rewritten assistant reply reseeds the session instead of resuming it 2026-08-29 01:37:52 +02:00
hh 5cd18fa2d7 fix(ui): sidebar footer in one row, no user name 2026-08-29 01:19:58 +02:00
hh e45e9960ed fix(ui): thread header counts only queued items; label the live dot 2026-08-29 01:08:44 +02:00
hh 16b6bbddda feat: one gateway port with path-mounted frontends, markdown chat, collapsible sidebars 2026-08-29 00:49:06 +02:00
hh 540327efa1 feat(ui,api): chat view with live tail and autoscroll, conversation rail with last message previews 2026-08-29 00:18:08 +02:00
hh 645df6f2b4 feat(ui): endpoint catalog with copyable urls on the now page 2026-08-28 22:30:11 +02:00
hh b577034c83 chore(ui): beaver favicon from beaver-land and impeccable design sidecar 2026-08-28 22:20:36 +02:00
hh 00618ab7ad feat(ui,api,admin): sveltekit admin spa, usage by day/agent/conversation/model, rate limits, memory tree, turn snapshot 2026-08-28 22:14:15 +02:00
hh aaaf459e6d fix(backends): remember the session id from the first frame so a cut turn stays resumable 2026-08-28 19:38:47 +02:00
hh 4ad014726b fix(core): a turn cut by shutdown keeps running_turn for recover 2026-08-28 19:33:26 +02:00
hh 8c9c6eee69 fix(telegram): brief-seeded branches reply and stream in their topic 2026-08-28 19:21:03 +02:00
hh 21766fe2d7 feat(core): seeds without text wait for the first turn 2026-08-28 19:03:08 +02:00
hh 1f8dae57ac fix(telegram,core): no draft for seed turns 2026-08-28 18:59:00 +02:00
hh 66325c2879 fix(telegram): last draft push is the final text so the client folds it into the message 2026-08-28 18:51:16 +02:00
hh 5a44a16ba3 fix(telegram,core): bare seeds stay silent in telegram 2026-08-28 18:42:35 +02:00
hh c711dcab13 fix(telegram): final message waits for the draft push in flight 2026-08-28 18:32:45 +02:00
hh 3995741cd5 fix(telegram,core): first message seeds the branch, registered commands, no ellipses, html drafts 2026-08-28 18:25:46 +02:00
hh b0f280b3ca feat(telegram,core): master topic instead of general in private chats 2026-08-28 18:12:58 +02:00
hh 45a5eb4a94 feat(telegram,core,backends,storage): telegram frontend with inbox, outbox, drafts and question buttons 2026-08-28 17:57:09 +02:00
hh 7ae87aeeb8 fix(backends,markdown): stream turn events live, no blank lines for tool blocks 2026-08-28 17:27:35 +02:00
hh fd8cbacd7b feat(core): publish tool.result on the bus for the activity panel 2026-08-28 17:01:00 +02:00
hh bd2d2368e1 refactor(agents,core,frontends): typed kinds, prompts per kind, frontend agents as parameters 2026-08-28 16:36:44 +02:00
hh 1d8d65b69a feat(core,frontends,agents): agent kinds, frontend routing, anthropic on conversations, vault mirror 2026-08-28 03:50:33 +02:00
hh 33ccc78fec feat(core,backends,frontends,storage): conversations, inject queue, session pool, gateway mcp tools, api frontend 2026-08-28 03:08:30 +02:00
hh 50b7057fa4 fix(backends,storage,core): setuid in exec wrapper, missing-column migration, runner config dir for mirror, tagged prompt sources 2026-08-28 02:10:19 +02:00
hh e1f242a87a feat(backends,storage,core,markdown,infra): claude agent sdk backend, session store, transcript seeding, runner isolation 2026-08-28 01:56:39 +02:00
hhandClaude Fable 5 ee2dc918ee docs(env): .env.example for local development
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-27 23:45:53 +02:00
hhandClaude Fable 5 33cc769ca2 chore: make ruff format and ty clean
Format two mcp modules with the current ruff, and narrow the text-block
check in conversation_store through a typed helper so ty stops flagging
dict.get on an unnarrowed block.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-27 23:22:23 +02:00
hhandClaude Fable 5 e4282b38cd feat(ui,infra): SvelteKit ui scaffold, Makefile, pre-commit, claude hooks
ui/: SvelteKit 2 + Svelte 5 + Tailwind 4 + shadcn-svelte (luma, lucide)
on bun with ultracite; adapter-static with SPA fallback; theme tokens
taken from msos. Dockerfile gains a bun stage that builds ui/build.
Makefile and pre-commit cover ruff, ty, ultracite and svelte-check;
.claude settings run the matching fixer after edits. docs/PRODUCT.md
captures product truth for the UI work.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-27 23:22:18 +02:00
301 changed files with 27912 additions and 3980 deletions

No files matched your search

+22
View File
@@ -0,0 +1,22 @@
## Code Principles
- **Simplicity**: Write simple, straightforward code
- **Readability**: Make code easy to understand
- **Maintainability**: Write code that's easy to update
- **Less Code = Less Debt**: Minimize code footprint
- **NEVER write comments** - code should be self-documenting
## Layout
- `src/beaver_gateway` - python, unopinionated primitives (no vault paths, no agent names)
- `ui` - SvelteKit SPA (admin + Obsidian panel), served by the gateway from `ui/build`
- `docs/MISSION.md`, `docs/PRD.md`, `docs/PLAN.md`, `docs/PROGRESS.md` - keep PROGRESS.md current
## Checking commands
After writing code, always run (or `make check`):
```shell
ruff format # python formatter
ruff check --fix # python linter
ty check # python type-checker
cd ui && bun fix # frontend linter+formatter (ultracite)
cd ui && bun check # svelte-check
```
Dependencies: `uv add` / `cd ui && bun add`, never edit lock files by hand.
+23
View File
@@ -0,0 +1,23 @@
You are able to use the Svelte MCP server, where you have access to comprehensive Svelte 5 and SvelteKit documentation. Here's how to use the available tools effectively:
## Available Svelte MCP Tools:
### 1. list-sections
Use this FIRST to discover all available documentation sections. Returns a structured list with titles, use_cases, and paths.
When asked about Svelte or SvelteKit topics, ALWAYS use this tool at the start of the chat to find relevant sections.
### 2. get-documentation
Retrieves full documentation content for specific sections. Accepts single or multiple sections.
After calling the list-sections tool, you MUST analyze the returned documentation sections (especially the use_cases field) and then use the get-documentation tool to fetch ALL documentation sections that are relevant for the user's task.
### 3. svelte-autofixer
Analyzes Svelte code and returns issues and suggestions.
You MUST use this tool whenever writing Svelte code before sending it to the user. Keep calling it until no issues or suggestions are returned.
### 4. playground-link
Generates a Svelte Playground link with the provided code.
After completing the code, ask the user if they want a playground link. Only call this tool after user confirmation and NEVER if code was written to files in their project.
+123
View File
@@ -0,0 +1,123 @@
# Ultracite Code Standards
This project uses **Ultracite**, a zero-config preset that enforces strict code quality standards through automated formatting and linting.
## Quick Reference
- **Format code**: `bun x ultracite fix`
- **Check for issues**: `bun x ultracite check`
- **Diagnose setup**: `bun x ultracite doctor`
Biome (the underlying engine) provides robust linting and formatting. Most issues are automatically fixable.
---
## Core Principles
Write code that is **accessible, performant, type-safe, and maintainable**. Focus on clarity and explicit intent over brevity.
### Type Safety & Explicitness
- Use explicit types for function parameters and return values when they enhance clarity
- Prefer `unknown` over `any` when the type is genuinely unknown
- Use const assertions (`as const`) for immutable values and literal types
- Leverage TypeScript's type narrowing instead of type assertions
- Use meaningful variable names instead of magic numbers - extract constants with descriptive names
### Modern JavaScript/TypeScript
- Use arrow functions for callbacks and short functions
- Prefer `for...of` loops over `.forEach()` and indexed `for` loops
- Use optional chaining (`?.`) and nullish coalescing (`??`) for safer property access
- Prefer template literals over string concatenation
- Use destructuring for object and array assignments
- Use `const` by default, `let` only when reassignment is needed, never `var`
### Async & Promises
- Always `await` promises in async functions - don't forget to use the return value
- Use `async/await` syntax instead of promise chains for better readability
- Handle errors appropriately in async code with try-catch blocks
- Don't use async functions as Promise executors
### React & JSX
- Use function components over class components
- Call hooks at the top level only, never conditionally
- Specify all dependencies in hook dependency arrays correctly
- Use the `key` prop for elements in iterables (prefer unique IDs over array indices)
- Nest children between opening and closing tags instead of passing as props
- Don't define components inside other components
- Use semantic HTML and ARIA attributes for accessibility:
- Provide meaningful alt text for images
- Use proper heading hierarchy
- Add labels for form inputs
- Include keyboard event handlers alongside mouse events
- Use semantic elements (`<button>`, `<nav>`, etc.) instead of divs with roles
### Error Handling & Debugging
- Remove `console.log`, `debugger`, and `alert` statements from production code
- Throw `Error` objects with descriptive messages, not strings or other values
- Use `try-catch` blocks meaningfully - don't catch errors just to rethrow them
- Prefer early returns over nested conditionals for error cases
### Code Organization
- Keep functions focused and under reasonable cognitive complexity limits
- Extract complex conditions into well-named boolean variables
- Use early returns to reduce nesting
- Prefer simple conditionals over nested ternary operators
- Group related code together and separate concerns
### Security
- Add `rel="noopener"` when using `target="_blank"` on links
- Avoid `dangerouslySetInnerHTML` unless absolutely necessary
- Don't use `eval()` or assign directly to `document.cookie`
- Validate and sanitize user input
### Performance
- Avoid spread syntax in accumulators within loops
- Use top-level regex literals instead of creating them in loops
- Prefer specific imports over namespace imports
- Avoid barrel files (index files that re-export everything)
- Use proper image components (e.g., Next.js `<Image>`) over `<img>` tags
### Framework-Specific Guidance
**Next.js:**
- Use Next.js `<Image>` component for images
- Use `next/head` or App Router metadata API for head elements
- Use Server Components for async data fetching instead of async Client Components
**React 19+:**
- Use ref as a prop instead of `React.forwardRef`
**Solid/Svelte/Vue/Qwik:**
- Use `class` and `for` attributes (not `className` or `htmlFor`)
---
## Testing
- Write assertions inside `it()` or `test()` blocks
- Avoid done callbacks in async tests - use async/await instead
- Don't use `.only` or `.skip` in committed code
- Keep test suites reasonably flat - avoid excessive `describe` nesting
## When Biome Can't Help
Biome's linter will catch most issues automatically. Focus your attention on:
1. **Business logic correctness** - Biome can't validate your algorithms
2. **Meaningful naming** - Use descriptive names for functions, variables, and types
3. **Architecture decisions** - Component structure, data flow, and API design
4. **Edge cases** - Handle boundary conditions and error states
5. **User experience** - Accessibility, performance, and usability considerations
6. **Documentation** - Add comments for complex logic, but prefer self-documenting code
---
Most formatting and common issues are automatically fixed by Biome. Run `bun x ultracite fix` before committing to ensure compliance.
+15
View File
@@ -0,0 +1,15 @@
{
"hooks": {
"PostToolUse": [
{
"matcher": "Write|Edit",
"hooks": [
{
"type": "command",
"command": "f=$(python3 -c 'import json,sys;print(json.load(sys.stdin).get(\"tool_input\",{}).get(\"file_path\",\"\"))'); case \"$f\" in */ui/*) cd ui && bun fix --skip=correctness/noUnusedImports ;; *.py) uv run ruff format \"$f\" && uv run ruff check --fix \"$f\" ;; esac"
}
]
}
]
}
}
+7
View File
@@ -14,3 +14,10 @@ examples
**/.ruff_cache **/.ruff_cache
**/.ty_cache **/.ty_cache
README.md README.md
ui/node_modules
ui/.svelte-kit
ui/build
ui/.vscode
.claude
.mcp.json
Makefile
+30
View File
@@ -0,0 +1,30 @@
# beaver-gateway - локальный dev. В проде переменные приходят из .env
# стека beaver-agent (env_file), этот файл там не нужен.
# Postgres из `make db` (профиль db в ../beaver-agent) или sqlite для быстрого старта
DATABASE_URL=postgresql+psycopg://beaver:beaver@localhost:5432/beaver
#DATABASE_URL=sqlite+aiosqlite:///./db.db
# Admin UI
ADMIN_USER=admin
ADMIN_PASS=admin
SESSION_SECRET=dev-only-change-me
# Пользовательский config.py (агенты, MCP, фронтенды)
CONFIG_PATH=../beaver-agent/config.py
# Токены для Anthropic/MCP фронтендов без admin UI: name:value,name:value
BOOTSTRAP_TOKENS=dev:dev-token-change-me
# Claude: `claude setup-token` на маке
CLAUDE_CODE_OAUTH_TOKEN=
# Raycast (нужно только если в config.py есть RaycastAgent)
RAYCAST_CONFIG_PATH=../raycast-api/config.json
RAYCAST_DEVICE_ID=
RAYCAST_BEARER=
# Telegram (нужно только если в config.py есть TelegramFrontend):
# токен от @BotFather, свой user id - @userinfobot
TELEGRAM_BOT_TOKEN=
TELEGRAM_USER_ID=
+4
View File
@@ -11,6 +11,7 @@ wheels/
# Sloppy # Sloppy
docs/ docs/
t/
# IDE # IDE
.idea .idea
@@ -18,3 +19,6 @@ docs/
# Local env # Local env
.env .env
db.db db.db
# Impeccable review screenshots
.impeccable/review/
+628
View File
@@ -0,0 +1,628 @@
{
"schemaVersion": 2,
"generatedAt": "2026-08-29T00:00:00Z",
"title": "Design System: Beaver admin",
"extensions": {
"colorMeta": {
"primary": {
"role": "primary",
"displayName": "Mulberry",
"canonical": "oklch(0.47 0.13 340)",
"dark": "#7c3871",
"tonalRamp": [
"oklch(0.15 0.13 340)",
"oklch(0.26 0.13 340)",
"oklch(0.37 0.13 340)",
"oklch(0.47 0.13 340)",
"oklch(0.58 0.13 340)",
"oklch(0.70 0.13 340)",
"oklch(0.83 0.13 340)",
"oklch(0.95 0.13 340)"
]
},
"signal": {
"role": "primary",
"displayName": "Orchid Signal",
"canonical": "oklch(0.48 0.21 342)",
"dark": "#ff82f3",
"tonalRamp": [
"oklch(0.15 0.21 342)",
"oklch(0.26 0.21 342)",
"oklch(0.37 0.21 342)",
"oklch(0.48 0.21 342)",
"oklch(0.60 0.21 342)",
"oklch(0.72 0.21 342)",
"oklch(0.84 0.21 342)",
"oklch(0.95 0.21 342)"
]
},
"ring": {
"role": "primary",
"displayName": "Focus Ring",
"canonical": "oklch(0.55 0.2 342)",
"dark": "#ff82f3",
"tonalRamp": [
"oklch(0.15 0.2 342)",
"oklch(0.26 0.2 342)",
"oklch(0.38 0.2 342)",
"oklch(0.55 0.2 342)",
"oklch(0.64 0.2 342)",
"oklch(0.74 0.2 342)",
"oklch(0.85 0.2 342)",
"oklch(0.95 0.2 342)"
]
},
"status-reply": {
"role": "secondary",
"displayName": "Branch Blue",
"canonical": "oklch(0.55 0.14 235)",
"dark": "oklch(0.72 0.13 235)",
"tonalRamp": [
"oklch(0.15 0.14 235)",
"oklch(0.26 0.14 235)",
"oklch(0.40 0.14 235)",
"oklch(0.55 0.14 235)",
"oklch(0.65 0.14 235)",
"oklch(0.75 0.14 235)",
"oklch(0.85 0.14 235)",
"oklch(0.95 0.14 235)"
]
},
"status-meeting": {
"role": "secondary",
"displayName": "Deep Violet",
"canonical": "oklch(0.5 0.2 300)",
"dark": "oklch(0.72 0.16 300)",
"tonalRamp": [
"oklch(0.15 0.2 300)",
"oklch(0.26 0.2 300)",
"oklch(0.38 0.2 300)",
"oklch(0.50 0.2 300)",
"oklch(0.62 0.2 300)",
"oklch(0.74 0.2 300)",
"oklch(0.85 0.2 300)",
"oklch(0.95 0.2 300)"
]
},
"status-work": {
"role": "secondary",
"displayName": "Job Teal",
"canonical": "oklch(0.55 0.13 195)",
"dark": "oklch(0.75 0.12 195)",
"tonalRamp": [
"oklch(0.15 0.13 195)",
"oklch(0.26 0.13 195)",
"oklch(0.40 0.13 195)",
"oklch(0.55 0.13 195)",
"oklch(0.65 0.13 195)",
"oklch(0.75 0.13 195)",
"oklch(0.85 0.13 195)",
"oklch(0.95 0.13 195)"
]
},
"status-skip": {
"role": "secondary",
"displayName": "Fork Grey",
"canonical": "oklch(0.5 0.04 340)",
"dark": "#a989a3",
"tonalRamp": [
"oklch(0.15 0.04 340)",
"oklch(0.26 0.04 340)",
"oklch(0.38 0.04 340)",
"oklch(0.50 0.04 340)",
"oklch(0.62 0.04 340)",
"oklch(0.74 0.04 340)",
"oklch(0.85 0.04 340)",
"oklch(0.95 0.04 340)"
]
},
"status-done": {
"role": "tertiary",
"displayName": "OK Green",
"canonical": "oklch(0.55 0.14 155)",
"dark": "oklch(0.74 0.14 155)",
"tonalRamp": [
"oklch(0.15 0.14 155)",
"oklch(0.26 0.14 155)",
"oklch(0.40 0.14 155)",
"oklch(0.55 0.14 155)",
"oklch(0.65 0.14 155)",
"oklch(0.75 0.14 155)",
"oklch(0.85 0.14 155)",
"oklch(0.95 0.14 155)"
]
},
"warn": {
"role": "tertiary",
"displayName": "Warn Amber",
"canonical": "oklch(0.55 0.13 75)",
"dark": "oklch(0.8 0.13 75)",
"tonalRamp": [
"oklch(0.15 0.13 75)",
"oklch(0.26 0.13 75)",
"oklch(0.40 0.13 75)",
"oklch(0.55 0.13 75)",
"oklch(0.65 0.13 75)",
"oklch(0.75 0.13 75)",
"oklch(0.85 0.13 75)",
"oklch(0.95 0.13 75)"
]
},
"note": {
"role": "tertiary",
"displayName": "Inject Amber",
"canonical": "oklch(0.52 0.13 78)",
"dark": "oklch(0.84 0.14 88)",
"tonalRamp": [
"oklch(0.15 0.13 78)",
"oklch(0.26 0.13 78)",
"oklch(0.40 0.13 78)",
"oklch(0.52 0.13 78)",
"oklch(0.64 0.13 78)",
"oklch(0.75 0.13 78)",
"oklch(0.85 0.13 78)",
"oklch(0.95 0.13 78)"
]
},
"destructive": {
"role": "tertiary",
"displayName": "Error Red",
"canonical": "oklch(0.58 0.22 25)",
"dark": "oklch(0.62 0.2 25)",
"tonalRamp": [
"oklch(0.15 0.22 25)",
"oklch(0.26 0.22 25)",
"oklch(0.40 0.22 25)",
"oklch(0.58 0.22 25)",
"oklch(0.66 0.22 25)",
"oklch(0.75 0.22 25)",
"oklch(0.85 0.22 25)",
"oklch(0.95 0.22 25)"
]
},
"background": {
"role": "neutral",
"displayName": "Blush Paper",
"canonical": "oklch(0.99 0.004 340)",
"dark": "#22111e",
"tonalRamp": [
"oklch(0.15 0.004 340)",
"oklch(0.26 0.004 340)",
"oklch(0.40 0.004 340)",
"oklch(0.55 0.004 340)",
"oklch(0.70 0.004 340)",
"oklch(0.85 0.004 340)",
"oklch(0.95 0.004 340)",
"oklch(0.99 0.004 340)"
]
},
"foreground": {
"role": "neutral",
"displayName": "Plum Ink",
"canonical": "oklch(0.24 0.03 340)",
"dark": "#f5f5f5",
"tonalRamp": [
"oklch(0.15 0.03 340)",
"oklch(0.24 0.03 340)",
"oklch(0.38 0.03 340)",
"oklch(0.50 0.03 340)",
"oklch(0.62 0.03 340)",
"oklch(0.75 0.03 340)",
"oklch(0.87 0.03 340)",
"oklch(0.95 0.03 340)"
]
},
"sidebar": {
"role": "neutral",
"displayName": "Sidebar Layer",
"canonical": "oklch(0.975 0.01 340)",
"dark": "#2c1b29",
"tonalRamp": [
"oklch(0.15 0.01 340)",
"oklch(0.26 0.01 340)",
"oklch(0.40 0.01 340)",
"oklch(0.55 0.01 340)",
"oklch(0.70 0.01 340)",
"oklch(0.85 0.01 340)",
"oklch(0.94 0.01 340)",
"oklch(0.975 0.01 340)"
]
},
"sidebar-accent": {
"role": "neutral",
"displayName": "Sidebar Accent",
"canonical": "oklch(0.94 0.016 340)",
"dark": "#341d2f",
"tonalRamp": [
"oklch(0.15 0.016 340)",
"oklch(0.26 0.016 340)",
"oklch(0.40 0.016 340)",
"oklch(0.55 0.016 340)",
"oklch(0.70 0.016 340)",
"oklch(0.85 0.016 340)",
"oklch(0.94 0.016 340)",
"oklch(0.98 0.016 340)"
]
},
"muted": {
"role": "neutral",
"displayName": "Muted",
"canonical": "oklch(0.96 0.012 340)",
"dark": "#40283a",
"tonalRamp": [
"oklch(0.15 0.012 340)",
"oklch(0.26 0.012 340)",
"oklch(0.40 0.012 340)",
"oklch(0.55 0.012 340)",
"oklch(0.70 0.012 340)",
"oklch(0.85 0.012 340)",
"oklch(0.96 0.012 340)",
"oklch(0.99 0.012 340)"
]
},
"muted-foreground": {
"role": "neutral",
"displayName": "Muted Ink",
"canonical": "oklch(0.5 0.04 340)",
"dark": "#a989a3",
"tonalRamp": [
"oklch(0.15 0.04 340)",
"oklch(0.26 0.04 340)",
"oklch(0.38 0.04 340)",
"oklch(0.50 0.04 340)",
"oklch(0.62 0.04 340)",
"oklch(0.74 0.04 340)",
"oklch(0.85 0.04 340)",
"oklch(0.95 0.04 340)"
]
},
"icon": {
"role": "neutral",
"displayName": "Icon",
"canonical": "oklch(0.55 0.03 340)",
"dark": "#877384",
"tonalRamp": [
"oklch(0.15 0.03 340)",
"oklch(0.26 0.03 340)",
"oklch(0.40 0.03 340)",
"oklch(0.55 0.03 340)",
"oklch(0.65 0.03 340)",
"oklch(0.75 0.03 340)",
"oklch(0.85 0.03 340)",
"oklch(0.95 0.03 340)"
]
},
"border": {
"role": "neutral",
"displayName": "Hairline",
"canonical": "oklch(0.9 0.02 340)",
"dark": "oklch(0.92 0.04 340 / 10%)",
"tonalRamp": [
"oklch(0.15 0.02 340)",
"oklch(0.26 0.02 340)",
"oklch(0.40 0.02 340)",
"oklch(0.55 0.02 340)",
"oklch(0.70 0.02 340)",
"oklch(0.80 0.02 340)",
"oklch(0.90 0.02 340)",
"oklch(0.97 0.02 340)"
]
}
},
"typographyMeta": {
"headline": {
"displayName": "Headline",
"purpose": "Page titles in page-header.svelte; the sidebar wordmark shares weight and tracking."
},
"title": {
"displayName": "Title",
"purpose": "Thread title, stat values (tabular), memory file path."
},
"body": {
"displayName": "Body",
"purpose": "Rows, cells, bubbles, dialogs; primary row text at 500."
},
"label": {
"displayName": "Label",
"purpose": "Uppercase section headings; same size without uppercase for table headers, badges, pills, meta lines."
},
"mono": {
"displayName": "Mono",
"purpose": "Raw JSON, tool input/result blocks, event tape, ids, the plaintext token."
}
},
"shadows": [
{
"name": "login-lift",
"value": "0 8px 24px -12px rgb(0 0 0 / 0.25)",
"purpose": "The sign-in card only (routes/login/+page.svelte). The single application-layer shadow."
},
{
"name": "popover",
"value": "shadow-xl plus ring-1 ring-foreground/5 (10% dark)",
"purpose": "Luma dialogs, sheets, menus, selects. Overlay surfaces only."
}
],
"motion": [
{
"name": "pulse-dot",
"value": "pulse-dot 1.6s cubic-bezier(0.4, 0, 0.6, 1) infinite",
"purpose": "Signal-colored halo (0 to 5px, 55% to 0% alpha) on live and running dots."
},
{
"name": "rise",
"value": "rise 220ms cubic-bezier(0.16, 1, 0.3, 1) both",
"purpose": "Opacity 0 to 1 and translateY 3px to 0 when a turn card lands in the activity feed."
},
{
"name": "row-hover",
"value": "background-color 150ms ease-out",
"purpose": "Ink wash at 4% on interactive rows."
},
{
"name": "chevron",
"value": "transform 150ms",
"purpose": "90 degree rotation of expand chevrons in tool nodes, raw entries, memory tree."
},
{
"name": "bar-fill",
"value": "width 300ms",
"purpose": "Utilization bar fill in limit-bar.svelte."
},
{
"name": "dialog",
"value": "fade-in and zoom-in-95 over 100ms",
"purpose": "Luma dialog open and close."
},
{
"name": "reduced-motion",
"value": "transition-duration and animation-duration 0.01ms under prefers-reduced-motion",
"purpose": "Global kill switch in layout.css."
}
],
"breakpoints": [
{
"name": "sm",
"value": "640px",
"purpose": "Sidebar replaces bottom nav; header padding 24px; hidden row columns return."
},
{
"name": "md",
"value": "768px",
"purpose": "Memory tree and file split side by side."
},
{
"name": "lg",
"value": "1024px",
"purpose": "Thread meta rail (18rem) and two-column dashboard sections."
},
{
"name": "panel-rail",
"value": "768px",
"purpose": "Panel container width, not viewport: below it the switcher bar names the thread, from it the 18rem conversation rail shows."
}
],
"obsidian": {
"root": ".beaver-root",
"source": "beaver-plugin-obsidian/src/tailwind.css",
"aliases": {
"background": "--background-primary",
"card": "--background-primary-alt",
"popover": "--background-secondary",
"sidebar": "--background-secondary",
"foreground": "--text-normal",
"muted-foreground": "--text-muted",
"icon": "--text-muted",
"status-skip": "--text-faint",
"border": "--background-modifier-border",
"input": "--background-modifier-border",
"muted": "--background-modifier-hover",
"secondary": "--background-modifier-hover",
"accent": "--background-modifier-hover",
"sidebar-accent": "--background-modifier-hover",
"primary": "--color-accent",
"signal": "--color-accent",
"ring": "--color-accent",
"status-new": "--color-accent",
"primary-foreground": "--text-on-accent",
"link": "--link-color, --text-accent",
"destructive": "--text-error",
"font-sans": "--font-interface",
"font-mono": "--font-monospace"
},
"radii": {
"button, select-trigger": "--button-radius",
"input, tabs-trigger, menu items": "--input-radius",
"tabs-list, skeleton, dialog, popover, menus": "--radius-m (8px)",
"rows and notes": "--radius 0.45rem"
},
"overlays": {
"border": "1px var(--border)",
"shadow": "var(--shadow-s)",
"menuBackground": "var(--popover), opaque, no backdrop",
"portal": ".beaver-root.beaver-portal on body at --layer-menu (65)"
},
"tabs": {
"list": "color-mix(in oklab, var(--text-normal) 6%, transparent)",
"active": "color-mix(in oklab, var(--text-normal) 14%, transparent)"
}
}
},
"components": [
{
"name": "Sidebar nav item",
"kind": "nav",
"refersTo": "sidebar-nav-item",
"description": "Section link in the 13rem sidebar; the active item takes the sidebar accent fill, the Now item carries the running count in signal.",
"html": "<nav class=\"ds-nav\"><a class=\"ds-nav-item ds-nav-item--active\" href=\"#\" aria-current=\"page\"><svg width=\"16\" height=\"16\" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\"><path d=\"M22 12h-2.48a2 2 0 0 0-1.93 1.46l-2.35 8.36a.25.25 0 0 1-.48 0L9.24 2.18a.25.25 0 0 0-.48 0l-2.35 8.36A2 2 0 0 1 4.49 12H2\"/></svg><span class=\"ds-nav-label\">Now</span><span class=\"ds-nav-count\">1</span></a><a class=\"ds-nav-item\" href=\"#\"><svg width=\"16\" height=\"16\" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\"><path d=\"M14 9a2 2 0 0 1-2 2H6l-4 4V4c0-1.1.9-2 2-2h8a2 2 0 0 1 2 2z\"/><path d=\"M18 9h2a2 2 0 0 1 2 2v11l-4-4h-6a2 2 0 0 1-2-2v-1\"/></svg><span class=\"ds-nav-label\">Conversations</span></a></nav>",
"css": ".ds-nav { display: flex; flex-direction: column; gap: 2px; padding: 8px; width: 208px; background: var(--sidebar); color: var(--sidebar-foreground); font-family: 'Inter Variable', sans-serif; } .ds-nav-item { display: flex; align-items: center; gap: 10px; height: 32px; padding: 0 10px; border-radius: 0.36rem; font-size: 0.8125rem; color: color-mix(in oklab, var(--sidebar-foreground) 80%, transparent); text-decoration: none; transition: background-color 150ms, color 150ms; } .ds-nav-item svg { color: var(--icon); flex-shrink: 0; } .ds-nav-item:hover { background: color-mix(in oklab, var(--sidebar-accent) 60%, transparent); color: var(--sidebar-foreground); } .ds-nav-item--active { background: var(--sidebar-accent); color: var(--sidebar-foreground); font-weight: 500; } .ds-nav-item:focus-visible { outline: 2px solid var(--ring); outline-offset: 2px; } .ds-nav-label { flex: 1; } .ds-nav-count { font-variant-numeric: tabular-nums; font-size: 0.75rem; font-weight: 500; padding: 0 6px; border-radius: 999px; color: var(--signal); background: color-mix(in oklab, var(--signal) 12%, transparent); }"
},
{
"name": "Section heading",
"kind": "custom",
"refersTo": "section-heading",
"description": "The 12px uppercase muted h2 that opens every ledger section, optionally followed by a tabular summary.",
"html": "<div class=\"ds-section-head\"><h2 class=\"ds-section-heading\">Live sessions</h2><span class=\"ds-section-summary\">1 processes · 263 MB</span></div>",
"css": ".ds-section-head { display: flex; align-items: baseline; gap: 12px; font-family: 'Inter Variable', sans-serif; } .ds-section-heading { margin: 0; font-size: 0.75rem; font-weight: 500; letter-spacing: 0.025em; text-transform: uppercase; color: var(--muted-foreground); } .ds-section-summary { font-size: 0.75rem; color: var(--muted-foreground); font-variant-numeric: tabular-nums; }"
},
{
"name": "Ledger row (running)",
"kind": "card",
"refersTo": "ledger-row",
"description": "A running conversation on the Now page: pulsing signal dot, kind badge, two-line title, tabular elapsed time, hairline below, ink-wash hover.",
"html": "<a class=\"ds-row\" href=\"#\"><span class=\"ds-row-dot\"></span><span class=\"ds-badge ds-badge--master\">master</span><span class=\"ds-row-title\"><span class=\"ds-row-name\">master e56eabe0</span><span class=\"ds-row-sub\">beaver-dispatcher</span></span><span class=\"ds-row-time\">20.1s</span></a>",
"css": ".ds-row { display: grid; grid-template-columns: auto auto minmax(0, 1fr) auto; column-gap: 12px; align-items: center; padding: 8px 0; border-bottom: 1px solid var(--border); font-family: 'Inter Variable', sans-serif; font-size: 0.8125rem; color: var(--foreground); text-decoration: none; transition: background-color 150ms ease-out; } .ds-row:hover { background: color-mix(in oklab, var(--foreground) 4%, transparent); } .ds-row:focus-visible { outline: 2px solid var(--ring); outline-offset: 2px; } .ds-row-dot { width: 8px; height: 8px; border-radius: 999px; background: var(--signal); animation: ds-pulse 1.6s cubic-bezier(0.4, 0, 0.6, 1) infinite; } @keyframes ds-pulse { 0%, 100% { box-shadow: 0 0 0 0 color-mix(in oklab, var(--signal) 55%, transparent); } 50% { box-shadow: 0 0 0 5px color-mix(in oklab, var(--signal) 0%, transparent); } } .ds-row-title { display: flex; flex-direction: column; min-width: 0; } .ds-row-name { font-weight: 500; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } .ds-row-sub { font-size: 0.75rem; color: var(--muted-foreground); } .ds-row-time { font-size: 0.75rem; color: var(--muted-foreground); font-variant-numeric: tabular-nums; } .ds-badge { display: inline-flex; align-items: center; height: 20px; padding: 0 6px; border-radius: 0.36rem; font-size: 0.75rem; font-weight: 500; line-height: 1; } .ds-badge--master { color: var(--signal); background: color-mix(in oklab, var(--signal) 12%, transparent); }"
},
{
"name": "Kind badge",
"kind": "chip",
"refersTo": "kind-badge",
"description": "Conversation kind as its hue at 12% behind hue text; master, branch, deep, job, fork.",
"html": "<span class=\"ds-kinds\"><span class=\"ds-badge ds-badge--master\">master</span><span class=\"ds-badge ds-badge--branch\">branch</span><span class=\"ds-badge ds-badge--deep\">deep</span><span class=\"ds-badge ds-badge--job\">job</span><span class=\"ds-badge ds-badge--fork\">fork</span></span>",
"css": ".ds-kinds { display: inline-flex; gap: 4px; font-family: 'Inter Variable', sans-serif; } .ds-badge { display: inline-flex; align-items: center; height: 20px; padding: 0 6px; border-radius: 0.36rem; font-size: 0.75rem; font-weight: 500; line-height: 1; } .ds-badge--master { color: var(--signal); background: color-mix(in oklab, var(--signal) 12%, transparent); } .ds-badge--branch { color: var(--status-reply); background: color-mix(in oklab, var(--status-reply) 12%, transparent); } .ds-badge--deep { color: var(--status-meeting); background: color-mix(in oklab, var(--status-meeting) 12%, transparent); } .ds-badge--job { color: var(--status-work); background: color-mix(in oklab, var(--status-work) 12%, transparent); } .ds-badge--fork { color: var(--status-skip); background: color-mix(in oklab, var(--status-skip) 12%, transparent); }"
},
{
"name": "Status pill",
"kind": "chip",
"refersTo": "status-pill",
"description": "Outcome as a 6px current-color dot plus 12px text; running pulses in signal.",
"html": "<span class=\"ds-pills\"><span class=\"ds-pill ds-pill--running\"><span class=\"ds-pill-dot ds-pill-dot--pulse\"></span>running</span><span class=\"ds-pill ds-pill--done\"><span class=\"ds-pill-dot\"></span>done</span><span class=\"ds-pill ds-pill--warn\"><span class=\"ds-pill-dot\"></span>interrupted</span><span class=\"ds-pill ds-pill--error\"><span class=\"ds-pill-dot\"></span>failed</span><span class=\"ds-pill ds-pill--muted\"><span class=\"ds-pill-dot\"></span>closed</span></span>",
"css": ".ds-pills { display: inline-flex; gap: 12px; font-family: 'Inter Variable', sans-serif; } .ds-pill { display: inline-flex; align-items: center; gap: 6px; font-size: 0.75rem; font-weight: 500; } .ds-pill-dot { width: 6px; height: 6px; border-radius: 999px; background: currentColor; } .ds-pill-dot--pulse { animation: ds-pulse 1.6s cubic-bezier(0.4, 0, 0.6, 1) infinite; } @keyframes ds-pulse { 0%, 100% { box-shadow: 0 0 0 0 color-mix(in oklab, var(--signal) 55%, transparent); } 50% { box-shadow: 0 0 0 5px color-mix(in oklab, var(--signal) 0%, transparent); } } .ds-pill--running { color: var(--signal); } .ds-pill--done { color: var(--status-done); } .ds-pill--warn { color: var(--warn); } .ds-pill--error { color: var(--destructive); } .ds-pill--muted { color: var(--muted-foreground); }"
},
{
"name": "Tool node",
"kind": "custom",
"refersTo": "ledger-row",
"description": "One row of the tool tree: chevron, per-node status dot, tool label (deep-tinted for subagents) with input summary, tabular duration; children nest under a left rail.",
"html": "<div class=\"ds-tool\"><button class=\"ds-tool-row\" type=\"button\" aria-expanded=\"false\"><svg class=\"ds-tool-chev\" width=\"14\" height=\"14\" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\"><path d=\"m9 18 6-6-6-6\"/></svg><span class=\"ds-tool-dot ds-tool-dot--running\"></span><span class=\"ds-tool-main\"><span class=\"ds-tool-name\">Bash</span><span class=\"ds-tool-sum\">python3 -c \"import time; time.sleep(30)\"</span></span><span class=\"ds-tool-time\">13.5s</span></button><div class=\"ds-tool ds-tool--child\"><button class=\"ds-tool-row\" type=\"button\" aria-expanded=\"false\"><svg class=\"ds-tool-chev\" width=\"14\" height=\"14\" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\"><path d=\"m9 18 6-6-6-6\"/></svg><span class=\"ds-tool-dot ds-tool-dot--done\"></span><span class=\"ds-tool-main\"><span class=\"ds-tool-name ds-tool-name--subagent\">Agent</span><span class=\"ds-tool-sum\">Read role.md and summarize</span></span><span class=\"ds-tool-time\">12.1s</span></button></div></div>",
"css": ".ds-tool { display: flex; flex-direction: column; font-family: 'Inter Variable', sans-serif; } .ds-tool--child { margin-left: 12px; padding-left: 8px; border-left: 1px solid var(--border); } .ds-tool-row { display: grid; grid-template-columns: auto auto minmax(0, 1fr) auto; column-gap: 12px; align-items: center; padding: 4px 6px; border: 0; border-radius: 0.36rem; background: transparent; color: var(--foreground); font: inherit; font-size: 0.8125rem; text-align: left; cursor: pointer; transition: background-color 150ms ease-out; } .ds-tool-row:hover { background: color-mix(in oklab, var(--foreground) 4%, transparent); } .ds-tool-row:focus-visible { outline: 2px solid var(--ring); outline-offset: 2px; } .ds-tool-chev { color: var(--icon); transition: transform 150ms; } .ds-tool-row[aria-expanded=\"true\"] .ds-tool-chev { transform: rotate(90deg); } .ds-tool-dot { width: 8px; height: 8px; border-radius: 999px; } .ds-tool-dot--running { background: var(--signal); animation: ds-pulse 1.6s cubic-bezier(0.4, 0, 0.6, 1) infinite; } .ds-tool-dot--done { background: var(--status-done); } @keyframes ds-pulse { 0%, 100% { box-shadow: 0 0 0 0 color-mix(in oklab, var(--signal) 55%, transparent); } 50% { box-shadow: 0 0 0 5px color-mix(in oklab, var(--signal) 0%, transparent); } } .ds-tool-main { display: flex; align-items: baseline; gap: 8px; min-width: 0; } .ds-tool-name { font-weight: 500; flex-shrink: 0; } .ds-tool-name--subagent { color: var(--status-meeting); } .ds-tool-sum { color: var(--muted-foreground); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } .ds-tool-time { font-size: 0.75rem; color: var(--muted-foreground); font-variant-numeric: tabular-nums; }"
},
{
"name": "Primary button",
"kind": "button",
"refersTo": "button-primary",
"description": "Luma pill button in mulberry; the one filled action per surface (Send, New, Create, Sign in).",
"html": "<button class=\"ds-btn ds-btn--primary\" type=\"button\"><svg width=\"16\" height=\"16\" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\"><path d=\"M5 12h14\"/><path d=\"M12 5v14\"/></svg>New token</button>",
"css": ".ds-btn { display: inline-flex; align-items: center; justify-content: center; gap: 4px; height: 32px; padding: 0 12px; border: 1px solid transparent; border-radius: 2rem; font-family: 'Inter Variable', sans-serif; font-size: 0.8125rem; font-weight: 500; white-space: nowrap; cursor: pointer; user-select: none; transition: all 150ms; } .ds-btn:active { transform: translateY(1px); } .ds-btn:disabled { opacity: 0.5; pointer-events: none; } .ds-btn:focus-visible { border-color: var(--ring); box-shadow: 0 0 0 3px color-mix(in oklab, var(--ring) 30%, transparent); outline: none; } .ds-btn--primary { background: var(--primary); color: var(--primary-foreground); } .ds-btn--primary:hover { background: color-mix(in oklab, var(--primary) 80%, transparent); }"
},
{
"name": "Outline button",
"kind": "button",
"refersTo": "button-outline",
"description": "Secondary action everywhere (Branch, Retry, Copy, answer options): hairline pill on the background surface.",
"html": "<button class=\"ds-btn ds-btn--outline\" type=\"button\"><svg width=\"16\" height=\"16\" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\"><line x1=\"6\" x2=\"6\" y1=\"3\" y2=\"15\"/><circle cx=\"18\" cy=\"6\" r=\"3\"/><circle cx=\"6\" cy=\"18\" r=\"3\"/><path d=\"M18 9a9 9 0 0 1-9 9\"/></svg>Branch</button>",
"css": ".ds-btn { display: inline-flex; align-items: center; justify-content: center; gap: 4px; height: 32px; padding: 0 12px; border: 1px solid transparent; border-radius: 2rem; font-family: 'Inter Variable', sans-serif; font-size: 0.8125rem; font-weight: 500; white-space: nowrap; cursor: pointer; user-select: none; transition: all 150ms; } .ds-btn:active { transform: translateY(1px); } .ds-btn:focus-visible { border-color: var(--ring); box-shadow: 0 0 0 3px color-mix(in oklab, var(--ring) 30%, transparent); outline: none; } .ds-btn--outline { border-color: var(--border); background: var(--background); color: var(--foreground); } .ds-btn--outline:hover { background: var(--muted); }"
},
{
"name": "Ghost button",
"kind": "button",
"refersTo": "button-ghost",
"description": "Quiet action (Cancel, Revoke, Done, icon buttons): transparent until hover.",
"html": "<button class=\"ds-btn ds-btn--ghost\" type=\"button\">Cancel</button>",
"css": ".ds-btn { display: inline-flex; align-items: center; justify-content: center; gap: 4px; height: 32px; padding: 0 12px; border: 1px solid transparent; border-radius: 2rem; font-family: 'Inter Variable', sans-serif; font-size: 0.8125rem; font-weight: 500; white-space: nowrap; cursor: pointer; user-select: none; transition: all 150ms; } .ds-btn:active { transform: translateY(1px); } .ds-btn:focus-visible { border-color: var(--ring); box-shadow: 0 0 0 3px color-mix(in oklab, var(--ring) 30%, transparent); outline: none; } .ds-btn--ghost { background: transparent; color: var(--foreground); } .ds-btn--ghost:hover { background: var(--muted); }"
},
{
"name": "Text input",
"kind": "input",
"refersTo": "input",
"description": "Luma pill input on a 50% input tint; focus is a ring-colored border plus a 30% halo.",
"html": "<label class=\"ds-field\"><span class=\"ds-label\">Name</span><input class=\"ds-input\" type=\"text\" placeholder=\"cursor-laptop\"></label>",
"css": ".ds-field { display: flex; flex-direction: column; gap: 6px; font-family: 'Inter Variable', sans-serif; max-width: 320px; } .ds-label { font-size: 0.8125rem; font-weight: 500; color: var(--foreground); } .ds-input { height: 36px; width: 100%; padding: 4px 12px; border: 1px solid transparent; border-radius: 1.5rem; background: color-mix(in oklab, var(--input) 50%, transparent); color: var(--foreground); font: inherit; font-size: 0.8125rem; caret-color: var(--primary); outline: none; transition: color 150ms, box-shadow 150ms, background-color 150ms; } .ds-input::placeholder { color: var(--muted-foreground); } .ds-input:focus-visible { border-color: var(--ring); box-shadow: 0 0 0 3px color-mix(in oklab, var(--ring) 30%, transparent); } .ds-input:disabled { opacity: 0.5; cursor: not-allowed; }"
},
{
"name": "Empty state",
"kind": "card",
"refersTo": "empty-state",
"description": "The only dashed border in the system: a bordered note with a 500 title, a muted hint and an optional outline action.",
"html": "<div class=\"ds-empty\"><p class=\"ds-empty-title\">The agents are idle</p><p class=\"ds-empty-hint\">Nothing is in a turn. The ledger fills in the moment a message or an inject lands.</p></div>",
"css": ".ds-empty { display: flex; flex-direction: column; align-items: flex-start; gap: 4px; padding: 20px 16px; border: 1px dashed var(--border); border-radius: 0.45rem; font-family: 'Inter Variable', sans-serif; font-size: 0.8125rem; color: var(--foreground); } .ds-empty p { margin: 0; } .ds-empty-title { font-weight: 500; } .ds-empty-hint { color: var(--muted-foreground); }"
}
],
"narrative": {
"northStar": "The Live Ledger",
"overview": "The console is a ledger, not a dashboard. It opens on what the agents are doing right now, as rows of running turns and tool calls, and refuses the grid of metric cards (direction contract, ui/src/app.html). Everything is a row on a rail: hairline borders separate rows, tabular numerals line up columns, and small uppercase section headings name each ledger. Cards exist only as bordered notes for exceptions (an empty state, an error, a pending question, the login form), never as containers for other cards.\n\nThe palette is the msos mauve/pink theme on Inter, inherited verbatim from Kit's msos project (ui/src/routes/layout.css, :root and .dark). Two neutral layers do the spatial work: a slightly cooler, tinted sidebar layer for navigation and rails, and the content surface. One saturated accent (signal) is reserved for two meanings only: something is live right now, or something is selected. Every other hue is a status or kind hue from the msos status palette, used at low alpha as a tint and at full strength as text.\n\nDensity is that of an operator console: 13px body, 12px labels, 32px nav items, 8px row padding, 48px page header. Motion is limited to a soft pulse on live dots, a 220ms rise when a turn lands, and 150ms hover fades; everything collapses under prefers-reduced-motion.",
"keyCharacteristics": [
"Rows and rails with hairline borders; no cards inside cards",
"Tabular numerals on every number; right-aligned numeric columns",
"Uppercase 12px section headings in muted ink",
"Pink (signal) means live or selected, nothing else",
"Kind hues (master, branch, deep, job, fork) carried by low-alpha badges and text tints",
"Pill-shaped luma primitives (buttons, inputs, tabs) on top of small-radius application rows",
"Light and dark modes, both tinted toward hue 340; dark borders are 10% alpha overlays"
],
"rules": [
{
"name": "The Pink Ration Rule",
"body": "signal marks live state (pulsing dots, running rows, busy sessions, the running count) and selection (active mobile nav, master kind). It never decorates a heading, a button, or a static surface. Hover is a 4% ink wash, not pink.",
"section": "colors"
},
{
"name": "The Twelve Percent Rule",
"body": "Every colored chip is its hue at 12% alpha behind full-strength text of the same hue (kind-badge.svelte, sidebar running count). Solid colored chips do not exist.",
"section": "colors"
},
{
"name": "The Two Layers Rule",
"body": "Navigation and rails sit on sidebar (or sidebar/50); content sits on background. There is no third surface color; depth beyond that is a hairline.",
"section": "colors"
},
{
"name": "The Tabular Rule",
"body": "Every number - durations, token counts, money, percentages, timestamps, ids, byte sizes - carries the tabular utility (font-variant-numeric: tabular-nums, layout.css). Numeric table columns are right-aligned.",
"section": "typography"
},
{
"name": "The One Heading Rule",
"body": "A page has one headline (the header) and any number of uppercase 12px section labels; there is no intermediate heading size. Section labels are h2 elements, not decorative kickers.",
"section": "typography"
},
{
"name": "The Flat Ledger Rule",
"body": "Nothing inside the main column casts a shadow. Overlays (dialog, sheet, menu) and the login card are the only lifted surfaces.",
"section": "elevation"
},
{
"name": "The Host Accent Rule",
"body": "Inside Obsidian signal and primary are one variable. Live dots, running rows, the running count, the selected row, the primary button and the focus ring all take the user's accent; the Pink Ration Rule becomes an accent ration with the same two meanings, live and selected.",
"section": "obsidian"
},
{
"name": "The Borrowed Corner Rule",
"body": "No control chooses its own radius. Rows and notes use --radius; luma primitives are re-rounded to Obsidian's --button-radius, --input-radius and --radius-m so the panel sits beside the app's own settings and menus without a seam.",
"section": "obsidian"
},
{
"name": "The Alias Only Rule",
"body": "Panel components use token aliases (bg-primary/5, text-link, border-b) and the panel.css utilities. A literal colour, font or radius in ui/src/lib/panel/ breaks the Obsidian mapping.",
"section": "obsidian"
}
],
"dos": [
"Do build lists as ledger-grid rows with border-b and a per-row column template; use a native table when there are more than four numeric columns.",
"Do put tabular on every number and right-align numeric columns.",
"Do start each section with the 12px uppercase muted h2 and stack sections with gap-8.",
"Do use signal only for live state (pulsing dots, running rows, running counts) and selection.",
"Do color kinds and statuses through kind-badge.svelte and status-pill.svelte; extend their maps rather than inventing new hue pairs.",
"Do give hover to interactive rows only, via row-hover; keep non-interactive rows still.",
"Do put secondary surfaces (rails, trees) on bg-sidebar/50 with a hairline, not on a card.",
"Do show every error as error-note.svelte with a retry and every empty list as empty-state.svelte with a one-line hint.",
"Do respect prefers-reduced-motion; new motion must be a CSS transition or one of animate-pulse-dot and animate-rise.",
"Do keep the luma primitives as shipped; compose them with size=\"sm\" and variant=\"outline\" or \"ghost\" for in-page actions.",
"Do write panel components (ui/src/lib/panel/) against token aliases and panel.css utilities only, and check them in both the admin and the plugin.",
"Do decide panel layout from the panel's own width (@container, bind:clientWidth); 768px is where the rail replaces the switcher.",
"Do add conversation actions to actions.svelte.ts so the header menu and the row menu stay one list."
],
"donts": [
"Don't put a card inside a card, or wrap ledger rows in a bordered container.",
"Don't add drop shadows to in-page surfaces; only overlays and the login card are lifted.",
"Don't use signal pink for headings, buttons, or static decoration; primary mulberry is the button color.",
"Don't introduce a heading size between the page headline (1rem) and the section label (0.75rem uppercase).",
"Don't render solid colored chips; chips are hue/12 behind hue text.",
"Don't use hue-chip, hairline, touch-shown or scrollbar-none from layout.css as if they were system rules - they are defined and currently unused by the admin.",
"Don't apply this palette inside Obsidian; the panel takes Obsidian's variables (PRODUCT.md brand commitment).",
"Don't expect pills inside Obsidian; every radius there is the host's (--button-radius, --input-radius, --radius-m).",
"Don't mark an active tab with bg-background in panel code; the plugin overrides it to ink at 14%.",
"Don't let a menu or dialog render inside the sidedock; overlays mount in .beaver-root.beaver-portal at --layer-menu."
]
}
}
@@ -0,0 +1,13 @@
---
version: 1
slug: "beaver-gateway-ui-src-lib-panel"
primary_target: "beaver-gateway/ui/src/lib/panel"
related_targets: ["beaver-gateway/ui/src/routes/panel","beaver-plugin-obsidian/src/ui"]
---
Scope: the Beaver panel as an Obsidian view (sidedock column, tab, mobile) and its browser twin at /admin/panel. Mode: Operate.
Audience and job: the operator (Kit) watching and steering the agent next to their notes; the task is to see what runs, answer, branch, send a branch back to Telegram, close a deep chat.
Constraints: shared components in ui/src/lib/panel, bundled into the plugin by esbuild ($lib alias, one Svelte runtime, svelte-sonner shimmed to Notice); container queries, never viewport breakpoints, because the same tree lives in a 300px column and a 1400px tab; Obsidian's variables only through .beaver-root (beaver-calendar is the reference); nothing pink inside Obsidian.
Direction: the agent's terminal, thread always on screen; below 48rem the switcher popover in the header, above it the grouped rail (master, branches, deep chats, jobs) on the left; one action list (actions.svelte.ts) behind the header menu and the row context menu; markdown through Obsidian's renderer with [[links]] opening notes.
Memorable moment: the panel follows the note you open and the running turn's tool tree grows in place.
Unresolved: a "new conversation" affordance in the panel; keyboard navigation beyond ↑/↓ and /; the pinned one-line summary when a branch returns to Telegram (§8.2) is not produced by the gateway yet.
+8
View File
@@ -0,0 +1,8 @@
{
"mcpServers": {
"svelte": {
"type": "http",
"url": "https://mcp.svelte.dev/mcp"
}
}
}
+21 -7
View File
@@ -1,18 +1,32 @@
repos: repos:
- repo: https://github.com/astral-sh/ruff-pre-commit - repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.15.13 rev: v0.15.20
hooks: hooks:
- id: ruff-check - id: ruff-check
types_or: [python, pyi] types_or: [ python, pyi ]
args: [--fix] args: [ --fix ]
- id: ruff-format - id: ruff-format
types_or: [python, pyi] types_or: [ python, pyi ]
- repo: local - repo: local
hooks: hooks:
- id: ty - id: ty
name: ty check name: ty check
entry: uvx ty check entry: uv run ty check
language: python language: system
types_or: [python, pyi] types_or: [ python, pyi ]
pass_filenames: false
- id: ultracite
name: ultracite
entry: bash -c 'cd ui && bun x ultracite fix'
language: system
files: ^ui/
pass_filenames: false
- id: svelte-check
name: svelte check
entry: bash -c 'cd ui && bun check'
language: system
files: ^ui/
pass_filenames: false pass_filenames: false
+22 -17
View File
@@ -30,35 +30,40 @@ COPY . /app
RUN --mount=type=cache,target=/root/.cache/uv \ RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --frozen --no-dev --extra prod uv sync --frozen --no-dev --extra prod
# ---- UI: SvelteKit SPA built with bun ----
FROM oven/bun:1-slim AS ui
WORKDIR /ui
COPY ui/package.json ui/bun.lock ./
RUN bun install --frozen-lockfile
COPY ui ./
RUN bun --bun svelte-kit sync && bun --bun run build
# ---- Runtime ---- # ---- Runtime ----
FROM python:3.13-slim AS runtime FROM python:3.13-slim AS runtime
RUN apt-get update \ RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates git \ && apt-get install -y --no-install-recommends ca-certificates git \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
# Bun native binary (glibc) — fast `npm install` replacement, only used # Bun native binary (glibc): `bunx` runs the stdio MCP servers declared in
# to drop the claude CLI into the image. Not invoked at runtime. # the user's config. The claude CLI itself ships inside the
# claude-agent-sdk wheel (`_bundled/claude`), nothing to install here.
COPY --from=oven/bun:1-slim /usr/local/bin/bun /usr/local/bin/bun COPY --from=oven/bun:1-slim /usr/local/bin/bun /usr/local/bin/bun
RUN ln -s /usr/local/bin/bun /usr/local/bin/bunx RUN ln -s /usr/local/bin/bun /usr/local/bin/bunx
# `--trust` is required: without it bun skips the postinstall step that # The model process runs as `beaver-runner`, not as the gateway: the
# fetches claude's native binary (anthropics/claude-code#50203). The # adapter spawns claude under this uid with a whitelisted environment.
# postinstall itself is bun's smoke check — if it fails the layer # `acl` lets the entrypoint grant it write access to the vault sub-mounts
# fails. We deliberately DO NOT run `claude --version` here: claude # without chowning files that Obsidian Sync keeps rewriting as root.
# touches `$HOME` on every invocation (creates `/root/.claude/`, RUN apt-get update \
# `/root/.claude.json`, sometimes `/root/.config/claude/`), and those && apt-get install -y --no-install-recommends acl \
# build-time artifacts seed the runtime named-volume `claude-home` && rm -rf /var/lib/apt/lists/* \
# with stale "haven't onboarded" state, so the user gets re-prompted && useradd --system --uid 1001 --create-home --shell /usr/sbin/nologin beaver-runner
# for trust/bypass dialogs on every rebuild and the subscription auth
# can land on a tainted credential file. ENV PATH=/app/.venv/bin:$PATH
ENV BUN_INSTALL=/usr/local/bun-global \
PATH=/usr/local/bun-global/bin:/app/.venv/bin:$PATH
RUN bun install -g --trust @anthropic-ai/claude-code \
&& test -x "$(command -v claude)" \
&& rm -rf /root/.claude /root/.claude.json /root/.config/claude
COPY --from=builder /app/.venv /app/.venv COPY --from=builder /app/.venv /app/.venv
COPY --from=builder /app /app COPY --from=builder /app /app
COPY --from=ui /ui/build /app/ui/build
WORKDIR /app WORKDIR /app
ENTRYPOINT ["python", "-m"] ENTRYPOINT ["python", "-m"]
+31
View File
@@ -0,0 +1,31 @@
.PHONY: sync check fix run ui ui-build db db-down
sync:
uv sync --extra local
cd ui && bun install
check:
uv run ruff format --check
uv run ruff check
uv run ty check
cd ui && bun lint && bun check
fix:
uv run ruff format
uv run ruff check --fix
cd ui && bun fix
run:
uv run python -m beaver_gateway
ui:
cd ui && bun run dev
ui-build:
cd ui && bun run build
db:
docker compose -f ../beaver-agent/docker-compose.yml --project-directory ../beaver-agent --profile db up -d postgres
db-down:
docker compose -f ../beaver-agent/docker-compose.yml --project-directory ../beaver-agent --profile db down
+31 -51
View File
@@ -12,7 +12,7 @@ from beaver_gateway.agents.base import ExposedMcp
from beaver_gateway.agents.claude import ClaudeAgent from beaver_gateway.agents.claude import ClaudeAgent
from beaver_gateway.agents.raycast import RaycastAgent, RemoteTool, UserPreferences from beaver_gateway.agents.raycast import RaycastAgent, RemoteTool, UserPreferences
from beaver_gateway.core.registry import Gateway from beaver_gateway.core.registry import Gateway
from beaver_gateway.core.turn_record import TurnRecord, slugify from beaver_gateway.core.turn_record import slugify
from beaver_gateway.frontends.admin import AdminFrontend from beaver_gateway.frontends.admin import AdminFrontend
from beaver_gateway.frontends.anthropic import AnthropicMessagesFrontend from beaver_gateway.frontends.anthropic import AnthropicMessagesFrontend
from beaver_gateway.frontends.markdown import MarkdownFrontend from beaver_gateway.frontends.markdown import MarkdownFrontend
@@ -20,20 +20,19 @@ from beaver_gateway.frontends.mcp_server import McpServerFrontend
from beaver_gateway.mcp.types import McpServer from beaver_gateway.mcp.types import McpServer
def chat_log_path(record: TurnRecord, vault: Path) -> Path: def chat_path(title: str, agent: str, vault: Path) -> Path: # noqa: ARG001
"""Decide where a logged chat from another frontend lands in the vault. """Where a new chat file lands in the vault.
Called by ``MarkdownFrontend`` (with ``log_all_chats=True``) the first Called by ``MarkdownFrontend`` for every conversation that needs a
time a conversation needs a file — continuation turns are matched by file: a ``deep`` chat spawned by the dispatcher, a ``/v1/messages``
fingerprint and stick to the file picked here. Return value can be chat (``title`` = first user message), or with ``log_all_chats=True``
the archive of a stateless agent's turns. Return value can be
absolute or relative; relative paths are anchored under ``vault``. absolute or relative; relative paths are anchored under ``vault``.
Layout below: ``<vault>/<YYYY-MM>/<YYYY-MM-DD>_<topic>.md`` where Layout below: ``<vault>/<YYYY-MM>/<YYYY-MM-DD>_<topic>.md``.
``topic`` is a slug of the very first user message in the chat.
""" """
today = date.today() today = date.today()
topic = slugify(record.first_user_text, maxlen=40) return vault / f"{today:%Y-%m}" / f"{today:%Y-%m-%d}_{slugify(title, maxlen=40)}.md"
return vault / f"{today:%Y-%m}" / f"{today:%Y-%m-%d}_{topic}.md"
def current_time() -> str: def current_time() -> str:
@@ -145,41 +144,27 @@ gateway = Gateway(
# Anthropic-compatible Messages endpoint. Auth comes from # Anthropic-compatible Messages endpoint. Auth comes from
# `BOOTSTRAP_TOKENS` in the env (`name1:value1,name2:value2`). # `BOOTSTRAP_TOKENS` in the env (`name1:value1,name2:value2`).
# #
# Behind a reverse proxy (Caddy / nginx / Cloudflare) pass # Every HTTP frontend is mounted under its own path on the one
# `public_base_url=` so the admin dashboard advertises the # gateway port (`Gateway.port`, 8000 here): `/anthropic/v1/messages`.
# outside URL instead of `host:port`. Caddy strips its own # Behind a reverse proxy set `Gateway(public_url="https://domain.com")`
# prefix and the frontend's internal paths (`/v1/messages`, # so advertised endpoints use the outside origin; the proxy just
# `/v1/models`) get appended: # forwards everything to the gateway, no prefix stripping.
# Caddy: handle_path /ai/* { reverse_proxy localhost:8000 } AnthropicMessagesFrontend(),
# Config: AnthropicMessagesFrontend(
# port=8000,
# public_base_url="https://domain.com/ai")
# Result: https://domain.com/ai/v1/messages
AnthropicMessagesFrontend(host="0.0.0.0", port=8000),
# Phase 3 — re-exposes every declared `McpServer` outside the # Phase 3 — re-exposes every declared `McpServer` outside the
# gateway with bearer auth + audit log. Each namespace lives # gateway with bearer auth + audit log. Each namespace lives at
# at `/<name>/` on this port (the port itself disambiguates # `/mcp/<name>/`; a flat bundle is published at `/mcp/all/`.
# MCP traffic — no extra `/mcp` segment in the route); a flat # Discovery page (HTML, auth-gated) at `/mcp/` with copy-pastable
# bundle is published at `/all/`. Discovery page (HTML, # Cursor / Claude Desktop snippets. Auth re-uses `BOOTSTRAP_TOKENS`.
# auth-gated) at `/` with copy-pastable Cursor / Claude McpServerFrontend(),
# Desktop snippets. Auth re-uses `BOOTSTRAP_TOKENS`.
#
# Same `public_base_url=` knob as above. Caddy strips its
# prefix; the frontend's `/<name>/` segment gets appended:
# Caddy: handle_path /mcp/* { reverse_proxy localhost:8001 }
# Config: McpServerFrontend(
# port=8001,
# public_base_url="https://domain.com/mcp")
# Result: https://domain.com/mcp/<name>/ (and /mcp/all/)
McpServerFrontend(host="0.0.0.0", port=8001),
# Phase 4.3 — browser admin UI. Creds come from # Phase 4.3 — browser admin UI. Creds come from
# `ADMIN_USER`/`ADMIN_PASS`; the session cookie is signed with # `ADMIN_USER`/`ADMIN_PASS`; the session cookie is signed with
# `SESSION_SECRET`. Use it to mint tokens (Argon2-hashed in # `SESSION_SECRET`. Use it to mint tokens (Argon2-hashed in
# the DB), revoke them, and watch the audit log. Scope is # the DB), revoke them, and watch the audit log. Scope is
# enforced on the bearer frontends: tokens minted with scope # enforced on the bearer frontends: tokens minted with scope
# `messages` only work on `/v1/messages`; `mcp` only on # `messages` only work on `/v1/messages`; `mcp` only on
# `/mcp/<name>`; `*` works everywhere. # `/mcp/<name>`; `*` works everywhere. Served at `/admin/`; `/`
AdminFrontend(host="0.0.0.0", port=8002), # redirects there.
AdminFrontend(),
# Obsidian-vault chat frontend. Each `.md` is one conversation # Obsidian-vault chat frontend. Each `.md` is one conversation
# (User/Assistant turn pairs). The Obsidian companion plugin # (User/Assistant turn pairs). The Obsidian companion plugin
# POSTs `{filename, content?}` to `/chat` — the frontend reads # POSTs `{filename, content?}` to `/chat` — the frontend reads
@@ -192,9 +177,8 @@ gateway = Gateway(
# example boots cleanly; in real deployments mount the # example boots cleanly; in real deployments mount the
# Obsidian-sync container's vault volume to a stable path and # Obsidian-sync container's vault volume to a stable path and
# pass that instead. # pass that instead.
# Mounted at `/md` (`/md/chat`, `/md/chat/stream`).
MarkdownFrontend( MarkdownFrontend(
host="0.0.0.0",
port=8003,
# Point at the dedicated chats subdir of your real Obsidian # Point at the dedicated chats subdir of your real Obsidian
# vault — the gateway has no idea (and no need) about other # vault — the gateway has no idea (and no need) about other
# notes outside it. Path resolution / vault-escape checks # notes outside it. Path resolution / vault-escape checks
@@ -208,17 +192,13 @@ gateway = Gateway(
vault_path=Path(tempfile.mkdtemp(prefix="beaver-vault-")).resolve(), vault_path=Path(tempfile.mkdtemp(prefix="beaver-vault-")).resolve(),
default_agent="research", default_agent="research",
log_all_chats=True, log_all_chats=True,
# ``log_path`` (optional) overrides the default # ``chat_path`` (optional) overrides the default
# ``{vault}/_logs/<agent>/<date>_<hex>.md`` layout for chats # ``{vault}/_logs/<agent>/<date>_<slug>.md`` layout. Heads up:
# logged from OTHER frontends (Anthropic Messages, admin # any custom path forces ``warm_index`` to scan the entire
# in-browser chat). Defined as a top-of-file function so the # vault on startup so the fingerprint→file map of archived
# types are explicit and the IDE can hover them; a lambda # stateless chats survives a restart no matter where you put
# works too, but a real ``def`` keeps the signature visible # files.
# and lets you docstring it. Heads up: any custom path chat_path=chat_path,
# forces ``warm_index`` to scan the entire vault on startup
# so the fingerprint→file map survives a restart no matter
# where you put files.
log_path=chat_log_path,
), ),
], ],
) )
+2 -5
View File
@@ -24,11 +24,8 @@ services:
# config.py declares one, so set these (or remove the agent) # config.py declares one, so set these (or remove the agent)
# before exposing port 8000. # before exposing port 8000.
ports: ports:
# /v1/messages frontend # The one gateway port: /anthropic, /mcp, /admin, /md under it
# (change ADMIN_USER/ADMIN_PASS/SESSION_SECRET before exposing).
- "8000:8000" - "8000:8000"
# MCP server frontend
- "8001:8001"
# Admin UI (Phase 4.3) — change ADMIN_USER/ADMIN_PASS/SESSION_SECRET
- "8002:8002"
volumes: volumes:
- ./config.py:/config/config.py:ro - ./config.py:/config/config.py:ro
+10 -7
View File
@@ -8,16 +8,20 @@ authors = [
requires-python = ">=3.13" requires-python = ">=3.13"
dependencies = [ dependencies = [
"aiofile>=3.11.1", "aiofile>=3.11.1",
"aiogram>=3.31.0",
"aiohttp>=3.13.5", "aiohttp>=3.13.5",
"aiosqlite>=0.22.1", "aiosqlite>=0.22.1",
"anthropic>=0.103.0", "anthropic>=0.103.0",
"anyio>=4.13.0", "anyio>=4.13.0",
"argon2-cffi>=25.1.0", "argon2-cffi>=25.1.0",
"claude-agent-sdk>=0.2.150",
"croniter>=6.2.4",
"fastapi>=0.136.1", "fastapi>=0.136.1",
"fastmcp>=3.3.1", "fastmcp>=3.3.1",
"greenlet>=3.5.0", "greenlet>=3.5.0",
"itsdangerous>=2.2.0", "itsdangerous>=2.2.0",
"jinja2>=3.1.6", "pgqueuer>=1.3.2",
"psutil>=7.2.2",
"psycopg[binary]>=3.3.4", "psycopg[binary]>=3.3.4",
"pydantic>=2.13.4", "pydantic>=2.13.4",
"pydantic-settings>=2.14.1", "pydantic-settings>=2.14.1",
@@ -25,16 +29,15 @@ dependencies = [
"sqlmodel>=0.0.38", "sqlmodel>=0.0.38",
"uvicorn[standard]>=0.47.0", "uvicorn[standard]>=0.47.0",
"uvloop>=0.22.1", "uvloop>=0.22.1",
"watchfiles>=1.2.0",
] ]
[project.optional-dependencies] [project.optional-dependencies]
local = [ local = [
"raycast-api", "raycast-api",
"claude-code-api",
] ]
prod = [ prod = [
"raycast-api", "raycast-api",
"claude-code-api",
] ]
[tool.uv] [tool.uv]
@@ -47,10 +50,6 @@ raycast-api = [
{ path = "../raycast-api", editable = true, extra = "local" }, { path = "../raycast-api", editable = true, extra = "local" },
{ git = "https://git.kotikot.com/beaver/raycast-api.git", extra = "prod" }, { git = "https://git.kotikot.com/beaver/raycast-api.git", extra = "prod" },
] ]
claude-code-api = [
{ path = "../claude-code-api", editable = true, extra = "local" },
{ git = "https://git.kotikot.com/beaver/claude-code-api.git", extra = "prod" },
]
[project.scripts] [project.scripts]
beaver-gateway = "beaver_gateway:main" beaver-gateway = "beaver_gateway:main"
@@ -67,3 +66,7 @@ dev = [
"ruff>=0.15.13", "ruff>=0.15.13",
"ty>=0.0.37", "ty>=0.0.37",
] ]
[tool.pytest.ini_options]
asyncio_mode = "auto"
testpaths = ["tests"]
+8 -2
View File
@@ -2,7 +2,7 @@
``BaseAgent`` is the structural contract every backend-specific agent ``BaseAgent`` is the structural contract every backend-specific agent
honours. Subclasses add **capabilities as fields** (``ClaudeAgent.cwd``, honours. Subclasses add **capabilities as fields** (``ClaudeAgent.cwd``,
``RaycastAgent.streaming``) — backends dispatch on type, not on a runtime ``RaycastAgent.streaming``) - backends dispatch on type, not on a runtime
matrix. matrix.
""" """
@@ -15,10 +15,16 @@ from pydantic import BaseModel, ConfigDict
@dataclass(frozen=True, slots=True) @dataclass(frozen=True, slots=True)
class ExposedMcp: class ExposedMcp:
"""Reference to an ``McpServer`` (by name) exposed to a single agent.""" """Reference to an ``McpServer`` (by name) exposed to a single agent.
``tools`` is an allowlist of tool names (``None`` = every tool);
``deny`` is a tuple of ``fnmatch`` patterns removed on top of that,
e.g. ``("delete_*",)``.
"""
name: str name: str
tools: tuple[str, ...] | None = None tools: tuple[str, ...] | None = None
deny: tuple[str, ...] = ()
class BaseAgent(BaseModel): class BaseAgent(BaseModel):
+81 -229
View File
@@ -1,258 +1,110 @@
"""Claude Code agent definition. """Claude agent definition, backed by the Claude Agent SDK.
There is no ``streaming`` field, but there is The system prompt is ``system_prompt`` verbatim or, per conversation kind,
:attr:`ClaudeCodeOptions.include_partial_messages`, and the difference the granules named in ``prompts`` assembled at every session spawn (see
matters. Token-level deltas are not a property of the agent, they are a ``core/prompt.py``). ``skill_sets`` are directories of ``<skill>/SKILL.md``
property of the transport: the PTY transport reads a JSONL that only folders; each becomes a local SDK plugin. Nothing from disk is loaded
ever contains finished blocks, so it cannot stream no matter what the otherwise: the adapter runs with ``setting_sources=[]``.
agent asks for, while ``transport="stream_json"`` can. The flag lives
next to the transport that grants it rather than pretending to be a
free-standing capability.
``BaseAgent.system_prompt`` maps onto the claude CLI's
``--system-prompt`` — i.e. it really *is* the agent's system prompt,
not "added on top of claude-code's giant built-in". The additive slot
``--append-system-prompt`` is exposed via
:attr:`ClaudeCodeOptions.append_system_prompt` for the rare case
when the user wants claude-code's planning conventions / dynamic
sections *and* a delta on top. Difference, measured empirically:
* tools (names, JSON schemas, embedded guidance like Bash's "prefer
Read over cat/head/tail") survive both flags — they ride the
Anthropic API ``tools=[]`` field, not the system prompt text;
* ``--system-prompt`` drops ~8.6k tokens of claude-code's *textual*
baseline — agent persona, multi-step-work conventions ("use
TaskCreate proactively", etc.), and the dynamic per-machine sections
(cwd, env info, git status, memory paths — see the
``--exclude-dynamic-system-prompt-sections`` flag note in
``claude --help``: "Only applies with the default system prompt
(ignored with --system-prompt)").
We pick override as the default because it preserves the principle of
least surprise: the ``system_prompt=...`` you wrote on the agent is
what claude actually receives. If you need claude-code's full
planning/dynamic-context behaviour on top of your prompt, opt in via
``options=ClaudeCodeOptions(append_system_prompt=...)`` and leave
``system_prompt`` for your agent's identity (or vice versa — set
``system_prompt=""``-ish and put everything in ``append_*``).
Per-agent passthrough of ``claude_code_api.BackendOptions`` lives in
:class:`ClaudeCodeOptions`, attached to :class:`ClaudeAgent` as
``options``. Every tunable knob ``BackendOptions`` supports — except
the ones derived from the agent's own primary surface
(``cwd`` / ``model`` / ``system_prompt`` / ``available_native_tools`` →
``allowed_tools`` / ``expose_mcps`` → ``mcp_servers``) — is exposed
there. Defaults match upstream except where correctness demands
otherwise (``wait_for_turn_duration=True``,
``dangerously_skip_permissions=True``).
""" """
from __future__ import annotations from __future__ import annotations
from collections.abc import Mapping # noqa: TC003 — pydantic runtime from collections.abc import Mapping # noqa: TC003 - pydantic runtime
from pathlib import Path # noqa: TC003 — pydantic runtime from pathlib import Path # noqa: TC003 - pydantic runtime
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field from pydantic import BaseModel, ConfigDict, Field, model_validator
from beaver_gateway.agents.base import BaseAgent from beaver_gateway.agents.base import BaseAgent
from beaver_gateway.core.kinds import KINDS, Kind
from beaver_gateway.core.policy import PolicyRule # noqa: TC001 - pydantic runtime
from beaver_gateway.core.prompt import PromptSource # noqa: TC001 - pydantic runtime
HistoryInjectionMode = Literal["native_jsonl", "concat_message"] __all__ = ["ClaudeAgent", "ClaudeOptions", "Prompts"]
"""Mirrors ``claude_code_api.HistoryInjectionMode`` to avoid an import
cycle on the user-facing path (configs may be loaded without
``claude-code-api`` installed, e.g. ``--extra prod`` minus claude)."""
Transport = Literal["pty", "stream_json"]
"""Mirrors ``claude_code_api.Transport``, for the same reason."""
class ClaudeCodeOptions(BaseModel): class ClaudeOptions(BaseModel):
"""Per-agent passthrough for ``claude_code_api.BackendOptions``.
Mirrors every field of ``BackendOptions`` except those that come
from the agent's primary surface
(``cwd`` / ``model`` / ``system_prompt`` /
``available_native_tools`` / ``expose_mcps``). Defaults match
upstream, with two exceptions for correctness:
* ``wait_for_turn_duration=True`` — without it, extended-thinking
turns drop the text response because ``TurnManager`` returns on
the first terminal assistant record (the thinking snapshot) and
never reads the second one (the actual text). Phase 2.2 PROGRESS
describes the incident in detail.
* ``dangerously_skip_permissions=True`` — Beaver Gateway is a
single-user trusted-config product; the user already wrote the
``config.py`` that spawns claude. Permission prompts on a
headless backend mean stuck turns.
Any field on :class:`claude_code_api.BackendOptions` we forward
here lives on this model with the same name and type, so a future
``BackendOptions`` field addition is a one-liner here plus a
one-liner in :mod:`backends.claude_code`.
"""
model_config = ConfigDict(frozen=True, arbitrary_types_allowed=True) model_config = ConfigDict(frozen=True, arbitrary_types_allowed=True)
transport: Transport = "pty" effort: str | None = None
"""Which claude process backs this agent. include_partial_messages: bool = True
"""Token-level deltas on the wire; off means one delta per finished block."""
``stream_json`` runs ``claude -p`` with stream-json on stdin and
stdout: prompts go into a pipe, events come back as they happen, and
a native ``result`` record closes each turn. ``pty`` (the default,
kept for backwards compatibility) drives the interactive TUI and
tails the session JSONL, which costs a multi-second readiness wait
per spawn — the thing ``startup_delay`` bounds — and brings the
swallowed-paste failure mode with it.
Switching also makes the PTY-only knobs below inert
(``startup_delay`` / ``file_wait_timeout`` /
``turn_duration_timeout`` / ``wait_for_turn_duration``) and drops
interactive-only ``extra_args`` such as ``--remote-control``, which
would take claude out of print mode. Everything else — model,
prompts, tools, MCPs, history seeding — behaves identically."""
warmup_turn: bool | None = None
"""Spend a throwaway turn when a session spawns, so MCP tools are
live on the agent's first reply.
``None`` (the default) decides per agent: on when it exposes any
MCP, off when it doesn't. Set it explicitly to override.
It exists because the CLI begins connecting MCP servers when a turn
starts and doesn't wait for them — measured on the pi, turn one saw
29 tools with both servers ``pending``, turn two saw 90 with both
``connected``. For an agent whose whole job is those tools, a first
reply without them is worse than a couple of extra seconds. Nothing
cheaper works: sleeping longer changes nothing, and neither a
``/status`` command nor the SDK's ``initialize`` handshake touches
the MCP client.
``stream_json`` only — the PTY transport's readiness wait covers
this incidentally."""
include_partial_messages: bool = False
"""Stream token-level deltas to the client instead of one delta per
finished block. Requires ``transport="stream_json"``; inert on the
PTY transport, whose JSONL contains only finished blocks. Off by
default because it multiplies the SSE event count — worth it for
interactive chat, pointless for a client that waits for the whole
message anyway."""
append_system_prompt: str | None = None
"""Maps to claude CLI's ``--append-system-prompt``. Opting in
re-attaches claude-code's full built-in prompt (persona, planning
conventions like "use TaskCreate proactively", and the dynamic
per-machine sections — cwd, env info, git status, memory paths)
plus this string on top. By default we ship only
:attr:`BaseAgent.system_prompt` via ``--system-prompt`` (~8.6k
tokens lighter); use this when the agent should behave like a
real claude-code coding session and your text is a delta on top
of those built-ins. Tool schemas survive either way — they ride
the Anthropic ``tools=[]`` channel, not the prompt text."""
disallowed_tools: tuple[str, ...] = ()
"""Tool names that claude must refuse to call. Combines with the
agent's ``available_native_tools`` allowlist — disallow wins."""
permission_mode: str = "bypassPermissions" permission_mode: str = "bypassPermissions"
"""``claude --permission-mode`` value. Only meaningful when tools: tuple[str, ...] | None = None
``dangerously_skip_permissions=False``; otherwise the CLI bypasses """Base set of built-in tools; ``None`` keeps the CLI default set."""
the prompt path entirely."""
dangerously_skip_permissions: bool = True disallowed_tools: tuple[str, ...] = ()
"""Pass ``--dangerously-skip-permissions`` to claude. ``True`` by add_dirs: tuple[str, ...] = ()
default — see the class docstring for why.""" env: Mapping[str, str] = Field(default_factory=dict)
"""Extra environment for the claude subprocess (always passed through)."""
effort: str | None = None env_keep: tuple[str, ...] = ()
"""``claude --effort`` value (typically ``low`` / ``medium`` / """Extra inherited variable names to let through the env whitelist."""
``high``). Tunes reasoning effort budget for newer models."""
add_dir: tuple[str, ...] = () max_turns: int | None = None
"""Extra directories claude is allowed to read/edit beyond session_store_flush: str = "eager"
``cwd``. Maps to repeated ``--add-dir`` flags.""" """``eager`` mirrors every transcript frame as it lands, so a gateway
killed mid-turn loses at most the frame in flight."""
settings: str | None = None
"""Path to a ``--settings`` JSON file claude should load (hooks,
MCP servers, etc. that don't belong in our internal aggregator)."""
extra_args: tuple[str, ...] = () class Prompts(BaseModel):
"""Raw extra argv to append to the claude command. Escape hatch """Prompt assembly per conversation kind (§3.12): the granules, in order.
for flags we haven't surfaced as first-class options."""
extra_env: Mapping[str, str] = Field(default_factory=dict) A kind left ``None`` is not served by the agent; ``ClaudeAgent.kinds``
"""Additional environment variables for the spawned claude follows from the kinds set here.
process. Layered on top of the gateway's own env after """
``preserve_provider_env`` is applied."""
preserve_provider_env: bool = False model_config = ConfigDict(frozen=True)
"""When ``False`` (default), the spawn env strips
``ANTHROPIC_API_KEY`` / ``ANTHROPIC_AUTH_TOKEN`` /
``ANTHROPIC_BASE_URL`` so claude uses subscription auth instead of
leaking through whatever the gateway process inherited."""
history_injection_mode: HistoryInjectionMode = "native_jsonl" master: tuple[PromptSource, ...] | None = None
"""How prior turns are seeded into a fresh session when no live branch: tuple[PromptSource, ...] | None = None
session matches: ``native_jsonl`` writes a hand-crafted transcript deep: tuple[PromptSource, ...] | None = None
and ``--resume``s; ``concat_message`` instead folds history into job: tuple[PromptSource, ...] | None = None
the first user prompt. ``native_jsonl`` is more faithful.""" fork: tuple[PromptSource, ...] | None = None
wait_for_turn_duration: bool = True def for_kind(self, kind: Kind) -> tuple[PromptSource, ...] | None:
"""Keep reading JSONL until the ``turn_duration`` heartbeat return getattr(self, kind)
arrives, instead of returning on the first terminal assistant
record. ``True`` by default — see class docstring."""
include_meta_user: bool = False @property
"""Surface claude's ``isMeta=True`` user records (local-command def kinds(self) -> tuple[Kind, ...]:
caveats) as ``UserMessage`` events. Off by default — they're not return tuple(k for k in KINDS if self.for_kind(k) is not None)
part of the real conversation."""
startup_delay: float = 60.0
"""Upper bound (seconds) on waiting for claude's TUI to enable
bracketed-paste mode after spawn. claude-code-api polls PTY output
for the DECSET 2004 marker and returns as soon as it arrives, so a
generous cap costs nothing on a warm host — it only bounds the case
where the marker never shows.
It has to be generous, because timing out here means writing the
prompt into a terminal that hasn't enabled bracketed paste, where our
``ESC[200~`` framing is parsed as junk and the message is silently
dropped. Observed on a cold Raspberry Pi: 10s elapsed with *zero*
bytes of PTY output, the prompt went into the void, and the turn hung
until a human typed into the session by hand."""
file_wait_timeout: float = 30.0
"""How long to wait for the session JSONL to appear after spawn.
Failure here usually means a CLI auth / config problem."""
turn_duration_timeout: float = 5.0
"""How long to wait for the ``turn_duration`` heartbeat once a
terminal assistant has been seen. Bound on extra latency when
``wait_for_turn_duration=True``."""
idle_session_ttl: float = 1800.0
"""Seconds a pooled claude session may sit unused before it is
terminated. The backend pools live PTYs by conversation-history
fingerprint; when a conversation forks, the session behind the old
fingerprint becomes unreachable but stays resident — one live
``claude`` (hundreds of MB) each. 30 minutes keeps an active chat
warm across a coffee break while bounding what a fork storm can
strand. There is no count cap: concurrent conversations should all
stay warm. ``0`` disables reaping."""
class ClaudeAgent(BaseAgent): class ClaudeAgent(BaseAgent):
"""Agent backed by ``claude-code-api``.
``cwd`` is the working directory the claude CLI is spawned in;
typed as ``Path`` (Pydantic coerces strings) since claude-code-api
accepts ``str | os.PathLike[str]``. ``available_native_tools`` stays
``tuple[str, ...]`` because Claude Code's tool set is a moving target
(Bash/Read/Edit/Write/WebSearch/etc. — versions add and rename), so
pinning it as a ``Literal`` would rot the type each release.
"""
cwd: Path cwd: Path
available_native_tools: tuple[str, ...] = () system_prompt: str = ""
options: ClaudeCodeOptions = Field(default_factory=ClaudeCodeOptions) """Verbatim prompt for agents without ``prompts`` (tests, one-offs)."""
"""Per-agent passthrough for the underlying claude-code-api
``BackendOptions``. See :class:`ClaudeCodeOptions`.""" prompts: Prompts = Field(default_factory=Prompts)
kinds: tuple[Kind, ...] = ()
"""Conversation kinds this agent serves; ``create``/``spawn`` reject the
rest. Defaults to the kinds ``prompts`` covers, or ``("deep",)`` for a
verbatim ``system_prompt``."""
skill_sets: tuple[Path, ...] = ()
gateway_tools: tuple[str, ...] = ()
"""Gateway tools exposed in-process (``read_conversation``, ``spawn``,
``say``, ``schedule``, ``inject``); empty = no gateway MCP server."""
options: ClaudeOptions = Field(default_factory=ClaudeOptions)
policy: tuple[PolicyRule, ...] = ()
"""``PreToolUse`` rules (§3.7), run in order on every tool call; the
first ``Deny`` is what the model reads back. See ``core/policy``."""
@model_validator(mode="after")
def _kinds_follow_prompts(self) -> ClaudeAgent:
covered = self.prompts.kinds
if not self.kinds:
object.__setattr__(self, "kinds", covered or ("deep",))
elif covered:
missing = [k for k in self.kinds if k not in covered]
if missing:
msg = f"agent {self.name!r} serves {missing} without a prompt"
raise ValueError(msg)
return self
def prompt_for(self, kind: Kind) -> tuple[PromptSource, ...] | None:
return self.prompts.for_kind(kind)
def serves(self, kind: str) -> bool:
return kind in self.kinds
+1 -1
View File
@@ -1,6 +1,6 @@
"""Backend adapters. """Backend adapters.
Each backend wraps a provider-specific SDK (``raycast-api``, ``claude-code-api``) Each backend wraps a provider SDK (``raycast-api``, ``claude-agent-sdk``)
and yields the unified :class:`~beaver_gateway.core.events.MessageStreamEvent` and yields the unified :class:`~beaver_gateway.core.events.MessageStreamEvent`
family. The Anthropic-style frontend serialises events straight to SSE. family. The Anthropic-style frontend serialises events straight to SSE.
""" """
+9 -2
View File
@@ -5,8 +5,15 @@ into a stream of :class:`~beaver_gateway.core.events.MessageStreamEvent`
records. The frontend serializes whatever comes out straight to SSE, so records. The frontend serializes whatever comes out straight to SSE, so
backends are the only place where provider quirks are translated. backends are the only place where provider quirks are translated.
Implementations are plain :class:`typing.Protocol` conformers — no ABC Implementations are plain :class:`typing.Protocol` conformers - no ABC
subclassing — to keep them swappable in tests with bare async generators. subclassing - to keep them swappable in tests with bare async generators.
``**options`` is the one extension point. Known keys, all optional and
ignored by backends that don't keep state: ``conversation_id`` (stable id
the backend may pin a live session to), ``session_id`` (backend session to
resume when nothing is live), ``capture`` (a
:class:`~beaver_gateway.core.turn_capture.TurnCapture` the backend fills
after the stream closes).
""" """
from __future__ import annotations from __future__ import annotations
-502
View File
@@ -1,502 +0,0 @@
"""Claude Code backend adapter.
One :class:`ClaudeCodeBackendAdapter` per :class:`ClaudeAgent`. The
underlying :class:`claude_code_api.ClaudeCodeBackend` bakes ``cwd`` /
``model`` / ``system_prompt`` / MCP wiring into a single
:class:`~claude_code_api.BackendOptions` at construction time, so a
single backend instance is conceptually bound to one agent (different
agents would mean different cwds / system prompts / exposed MCPs and
thus different live-session pools).
Per :meth:`complete` we:
* hand the full Anthropic-style ``messages`` list to
``ClaudeCodeBackend.complete`` — it does its own fingerprint-based
session lookup, so we never need to track sessions ourselves;
* turn its events into one ``message_start`` … ``message_stop``
envelope per ``complete`` call, with content-block indices increasing
monotonically across the whole turn;
* close the envelope on the ``ResultMessage``.
How the blocks inside that envelope are produced depends on the agent's
transport. By default each ``AssistantMessage`` is re-emitted as
``content_block_start`` + one delta + ``content_block_stop`` per block —
a "stream" of finished blocks, because the PTY transport has nothing
finer to offer. With ``transport="stream_json"`` *and*
``include_partial_messages``, the backend instead feeds us the real
token-level ``StreamEvent``s and we emit from those; the whole-block
records still arrive and are still kept for :class:`TurnCapture`, but
emitting from both would duplicate every block on the wire.
The per-request ``system`` parameter is intentionally **ignored** —
``BackendOptions.system_prompt`` is fixed at session-spawn time, and the
agent's ``system_prompt`` is the canonical identity of the agent.
"""
from __future__ import annotations
import json
import logging
import uuid
from collections.abc import Mapping # runtime import: isinstance in _emit_stream_event
from dataclasses import dataclass, field
from typing import TYPE_CHECKING, Any, Self
from claude_code_api import (
AssistantMessage,
BackendOptions,
ClaudeCodeBackend,
ResultMessage,
StreamEvent,
TextBlock,
ThinkingBlock,
ToolUseBlock,
synthesize_turn_messages,
)
from beaver_gateway.agents.claude import ClaudeAgent
from beaver_gateway.core.events import (
StopReason,
build_content_block_stop,
build_input_json_delta,
build_message_delta,
build_message_start,
build_message_stop,
build_signature_delta,
build_text_block_start,
build_text_delta,
build_thinking_block_start,
build_thinking_delta,
build_tool_use_block_start,
)
if TYPE_CHECKING:
from collections.abc import AsyncIterator, Iterable
from anthropic.types import MessageParam
from beaver_gateway.agents.base import BaseAgent
from beaver_gateway.core.events import MessageStreamEvent
_log = logging.getLogger("beaver_gateway.backends.claude_code")
__all__ = ["ClaudeCodeBackendAdapter", "TurnCapture"]
@dataclass
class TurnCapture:
"""Side-channel sink for per-turn metadata.
Pass an instance via ``ClaudeCodeBackendAdapter.complete(capture=...)``.
After the stream finishes, :attr:`synthesized_messages` holds the
full assistant↔tool-result cycle (from
:func:`claude_code_api.synthesize_turn_messages`) — i.e. the exact
list of canonical Anthropic-shape messages claude-code-api stashed
the live session under. The markdown frontend uses this to write the
conversation history to its DB so a subsequent turn's prefix
fingerprint hits the same session.
Other backends (anthropic, raycast) ignore the kwarg — it lands in
their ``**options`` and is silently dropped.
"""
synthesized_messages: list[dict[str, Any]] = field(default_factory=list)
_CLAUDE_TO_ANTHROPIC_STOP: dict[str, StopReason] = {
"end_turn": "end_turn",
"tool_use": "tool_use",
"max_tokens": "max_tokens",
"stop_sequence": "stop_sequence",
"refusal": "refusal",
}
def _map_stop_reason(raw: str | None) -> StopReason:
"""Map claude-code's stop reason into Anthropic's vocabulary.
Unknown / missing values collapse to ``end_turn`` so the client sees
a clean finish rather than a wire-format error.
"""
if raw is None:
return "end_turn"
return _CLAUDE_TO_ANTHROPIC_STOP.get(raw, "end_turn")
def _build_mcp_servers(
agent: ClaudeAgent, mcp_internal_urls: Mapping[str, str]
) -> dict[str, dict[str, Any]] | None:
"""Render ``agent.expose_mcps`` into ``BackendOptions.mcp_servers``.
Each exposed MCP is a streamable-HTTP pointer at the gateway's
internal aggregator (built by :mod:`beaver_gateway.mcp.internal_app`).
``None`` keeps claude-code from materializing an ``--mcp-config``
file when the agent exposes nothing.
"""
if not agent.expose_mcps:
return None
servers: dict[str, dict[str, Any]] = {}
for em in agent.expose_mcps:
url = mcp_internal_urls.get(em.name)
if url is None:
msg = (
f"agent {agent.name!r} exposes MCP {em.name!r} "
"but no internal URL is registered for it"
)
raise ValueError(msg)
servers[em.name] = {"type": "http", "url": url}
return servers
def _build_backend_options(
agent: ClaudeAgent, mcp_internal_urls: Mapping[str, str]
) -> BackendOptions:
"""Compose the per-agent :class:`BackendOptions`.
Agent-primary fields:
* ``cwd`` / ``model`` come from the agent directly;
* ``system_prompt`` carries :attr:`BaseAgent.system_prompt`
verbatim — i.e. wire-level ``--system-prompt`` (~8.6k tokens
lighter than ``--append-system-prompt`` because claude-code's
persona/planning conventions and dynamic sections drop out;
tool schemas survive via the API ``tools=[]`` channel);
* ``append_system_prompt`` carries
:attr:`ClaudeCodeOptions.append_system_prompt`, normally
``None``. Setting it re-attaches claude-code's built-in prompt
*and* this delta — opt-in for "claude as a real coding session";
* ``allowed_tools`` follows the PLAN: when the user lists native
tools we restrict to those *plus* a per-MCP wildcard so MCP tools
stay reachable; when no native list is declared we leave
``allowed_tools`` empty (= all tools allowed by claude-code's
default);
* ``mcp_servers`` comes from :func:`_build_mcp_servers`.
Every other tunable knob is passed through from
:attr:`ClaudeAgent.options`. Our default overrides
(``wait_for_turn_duration=True``,
``dangerously_skip_permissions=True``) live on
:class:`ClaudeCodeOptions`, not here, so a user who builds
``ClaudeCodeOptions(...)`` explicitly inherits the same defaults
instead of getting whatever claude-code-api ships.
"""
allowed_tools: tuple[str, ...] = ()
if agent.available_native_tools:
mcp_wildcards = tuple(f"mcp__{em.name}" for em in agent.expose_mcps)
allowed_tools = tuple(agent.available_native_tools) + mcp_wildcards
opt = agent.options
return BackendOptions(
cwd=agent.cwd,
model=agent.model or None,
system_prompt=agent.system_prompt,
append_system_prompt=opt.append_system_prompt,
allowed_tools=allowed_tools,
mcp_servers=_build_mcp_servers(agent, mcp_internal_urls),
transport=opt.transport,
include_partial_messages=opt.include_partial_messages,
# ``None`` means "decide from the agent": an agent that exposes
# MCPs needs them on its first reply, one that doesn't shouldn't
# pay for a warm-up it gains nothing from.
warmup_turn=(
bool(agent.expose_mcps) if opt.warmup_turn is None else opt.warmup_turn
),
disallowed_tools=opt.disallowed_tools,
permission_mode=opt.permission_mode,
dangerously_skip_permissions=opt.dangerously_skip_permissions,
effort=opt.effort,
add_dir=opt.add_dir,
settings=opt.settings,
extra_args=opt.extra_args,
extra_env=opt.extra_env,
preserve_provider_env=opt.preserve_provider_env,
history_injection_mode=opt.history_injection_mode,
wait_for_turn_duration=opt.wait_for_turn_duration,
include_meta_user=opt.include_meta_user,
startup_delay=opt.startup_delay,
file_wait_timeout=opt.file_wait_timeout,
turn_duration_timeout=opt.turn_duration_timeout,
idle_session_ttl=opt.idle_session_ttl,
)
class ClaudeCodeBackendAdapter:
"""One ``claude-code-api`` backend bound to a single :class:`ClaudeAgent`.
Owns the underlying :class:`ClaudeCodeBackend`'s lifecycle through
the async-context-manager protocol so :mod:`beaver_gateway.cli` can
park it in its ``AsyncExitStack``.
"""
def __init__(
self, *, agent: ClaudeAgent, mcp_internal_urls: Mapping[str, str]
) -> None:
self._agent = agent
options = _build_backend_options(agent, mcp_internal_urls)
self._backend = ClaudeCodeBackend(options)
self._streaming = (
options.transport == "stream_json" and options.include_partial_messages
)
"""Whether the backend feeds us token-level ``StreamEvent``s.
When it does, blocks are emitted from those and the whole-block
``AssistantMessage`` records are used only for bookkeeping —
emitting from both would duplicate every block on the wire."""
@property
def agent(self) -> ClaudeAgent:
return self._agent
@property
def live_session_count(self) -> int:
return self._backend.live_session_count
@property
def live_sessions(self) -> dict[str, Any]:
"""Live claude processes keyed by claude session_id.
Pass-through to the underlying ``ClaudeCodeBackend``. The value
is a ``PtyClaudeProcess`` or a ``StreamClaudeProcess`` depending
on the agent's transport; both implement the
``captured_output()`` / ``add_output_listener`` / ``write``
surface the admin terminal consumes, so it works either way —
with the caveat that the stream transport's "terminal" is a
read-only view of the JSON event stream rather than a TUI you
can type into. Typed as ``Any`` to avoid leaking the lower
layer's type into the gateway's public surface.
"""
return self._backend.live_sessions
async def __aenter__(self) -> Self:
await self._backend.__aenter__()
return self
async def __aexit__(self, exc_type: object, exc: object, tb: object) -> None:
await self._backend.__aexit__(exc_type, exc, tb)
async def aclose(self) -> None:
await self._backend.aclose()
async def complete(
self,
*,
agent: BaseAgent,
messages: Iterable[MessageParam],
system: str | None = None, # noqa: ARG002 — see module docstring
capture: TurnCapture | None = None,
**options: Any, # noqa: ARG002 — no per-request knobs for claude-code yet
) -> AsyncIterator[MessageStreamEvent]:
if not isinstance(agent, ClaudeAgent):
msg = (
"ClaudeCodeBackendAdapter requires ClaudeAgent, "
f"got {type(agent).__name__}"
)
raise TypeError(msg)
if agent.name != self._agent.name:
# Adapter is per-agent; routing a different agent through it
# would mean a different cwd / system_prompt / MCP set than
# the live-session pool was spawned with.
msg = (
f"ClaudeCodeBackendAdapter bound to {self._agent.name!r} "
f"got request for {agent.name!r}"
)
raise ValueError(msg)
msgs_list: list[Mapping[str, Any]] = list(messages)
_log.info(
"complete: agent=%s n_messages=%d capture=%s live_sessions=%d",
agent.name,
len(msgs_list),
capture is not None,
self._backend.live_session_count,
)
message_id = f"msg_{uuid.uuid4().hex}"
yield build_message_start(message_id=message_id, model=agent.model)
next_index = 0
stop_reason: str | None = None
usage: Mapping[str, Any] | None = None
n_text = 0
n_thinking = 0
n_tool_use = 0
# We keep raw events so we can hand them to
# ``synthesize_turn_messages`` after the stream closes — the
# markdown frontend stores the result in its conversation
# history so the next turn's prefix matches the backend's
# session-pool fingerprint. UserMessage (tool_result) events
# are silently discarded from the wire but kept here.
raw_events: list[Any] = []
async for event in self._backend.complete(msgs_list):
raw_events.append(event)
if isinstance(event, StreamEvent):
# Only ever arrives when ``self._streaming`` — the
# backend doesn't emit these otherwise. Indices are
# already rebased onto the whole turn by claude-code-api.
for ev in _emit_stream_event(event.event):
yield ev
elif isinstance(event, AssistantMessage):
for block in event.content:
if isinstance(block, TextBlock):
n_text += 1
elif isinstance(block, ThinkingBlock):
n_thinking += 1
elif isinstance(block, ToolUseBlock):
n_tool_use += 1
if not self._streaming:
for ev in _emit_block(block, next_index):
yield ev
next_index += 1
elif isinstance(event, ResultMessage):
# ResultMessage is the terminal event from TurnManager
# — we capture its stop_reason / usage for the envelope
# below. We DO NOT break here: an early break would
# raise GeneratorExit inside claude-code-api's
# ``complete`` coroutine before it gets a chance to
# stash the live session under the post-turn
# fingerprint, so every continuation would miss the
# cache and reseed. Let the inner generator exit
# naturally instead.
stop_reason = event.stop_reason
usage = event.usage
# UserMessage (tool_result records) and SystemMessage
# (turn_duration heartbeats) carry no content for the
# /v1/messages caller — skip silently on the wire, but they
# ARE retained in ``raw_events`` for synthesis below.
if capture is not None:
capture.synthesized_messages = synthesize_turn_messages(raw_events)
_log.info(
"complete: agent=%s DONE text=%d thinking=%d tool_use=%d stop=%s synth=%d",
agent.name,
n_text,
n_thinking,
n_tool_use,
stop_reason,
len(capture.synthesized_messages) if capture is not None else 0,
)
yield build_message_delta(
stop_reason=_map_stop_reason(stop_reason), usage=_normalize_usage(usage)
)
yield build_message_stop()
def _emit_stream_event(event: Mapping[str, Any]) -> Iterable[MessageStreamEvent]:
"""Render one raw Anthropic streaming event through our own builders.
The payload is already Anthropic-shaped, so forwarding it verbatim
is tempting — but it arrives as a plain dict and the frontend
serializes pydantic models, so it would have to be validated against
the SDK's ``RawMessageStreamEvent`` union anyway. Rebuilding it from
the fields we recognize is the same amount of work and fails the
same way :func:`_emit_block` already does: a block or delta type
this gateway doesn't know is dropped rather than raising mid-stream
on a claude release that added one.
Only ``content_block_*`` events reach here — claude-code-api strips
the inner per-request message envelopes, because the caller owns the
single envelope spanning the whole turn.
"""
etype = event.get("type")
index = event.get("index")
if not isinstance(index, int):
return ()
if etype == "content_block_start":
block = event.get("content_block")
if not isinstance(block, Mapping):
return ()
btype = block.get("type")
if btype == "text":
return (build_text_block_start(index),)
if btype == "thinking":
return (build_thinking_block_start(index),)
if btype == "tool_use":
return (
build_tool_use_block_start(
index,
tool_use_id=str(block.get("id", "")),
name=str(block.get("name", "")),
),
)
return ()
if etype == "content_block_delta":
delta = event.get("delta")
if not isinstance(delta, Mapping):
return ()
dtype = delta.get("type")
if dtype == "text_delta":
return (build_text_delta(index, str(delta.get("text", ""))),)
if dtype == "thinking_delta":
return (build_thinking_delta(index, str(delta.get("thinking", ""))),)
if dtype == "signature_delta":
return (build_signature_delta(index, str(delta.get("signature", ""))),)
if dtype == "input_json_delta":
return (build_input_json_delta(index, str(delta.get("partial_json", ""))),)
return ()
if etype == "content_block_stop":
return (build_content_block_stop(index),)
return ()
def _emit_block(
block: TextBlock | ThinkingBlock | ToolUseBlock | Any, index: int
) -> Iterable[MessageStreamEvent]:
"""Render one ``claude-code`` content block as Anthropic stream events.
``ToolResultBlock`` would arrive only on user-role records — we
don't emit it here because :meth:`complete` skips ``UserMessage``.
"""
if isinstance(block, TextBlock):
return (
build_text_block_start(index),
build_text_delta(index, block.text),
build_content_block_stop(index),
)
if isinstance(block, ThinkingBlock):
return (
build_thinking_block_start(index),
build_thinking_delta(index, block.thinking),
build_signature_delta(index, block.signature),
build_content_block_stop(index),
)
if isinstance(block, ToolUseBlock):
partial = json.dumps(block.input, separators=(",", ":"), ensure_ascii=False)
return (
build_tool_use_block_start(index, tool_use_id=block.id, name=block.name),
build_input_json_delta(index, partial),
build_content_block_stop(index),
)
return ()
def _normalize_usage(usage: Mapping[str, Any] | None) -> dict[str, int] | None:
"""Coerce claude-code's ``usage`` dict to Anthropic ``MessageDeltaUsage`` shape.
claude-code copies whatever the JSONL ``usage`` record carried —
fields can be missing, strings, or ints. We pass through only the
fields ``MessageDeltaUsage`` knows about and discard the rest so an
odd ``cache_creation`` object structure doesn't fail pydantic
validation downstream.
"""
if not usage:
return None
out: dict[str, int] = {}
for key in (
"input_tokens",
"output_tokens",
"cache_creation_input_tokens",
"cache_read_input_tokens",
):
value = usage.get(key)
if isinstance(value, int):
out[key] = value
return out or None
File diff suppressed because it is too large. Load diff
+4 -1
View File
@@ -26,11 +26,12 @@ tools we route it back to the underlying MCP in-process, append the
result as a ``tool`` message, and re-issue the stream — all inside one result as a ``tool`` message, and re-issue the stream — all inside one
Anthropic envelope (one ``message_start`` … one ``message_stop``). The Anthropic envelope (one ``message_start`` … one ``message_stop``). The
tool_use blocks DO surface to the caller (mirrors what tool_use blocks DO surface to the caller (mirrors what
``ClaudeCodeBackendAdapter`` does), but tool_results stay internal. ``ClaudeSdkBackend`` does), but tool_results stay internal.
""" """
from __future__ import annotations from __future__ import annotations
import fnmatch
import json import json
import logging import logging
import uuid import uuid
@@ -297,6 +298,8 @@ def _build_agent_catalog(
for mt in mcp_tools.get(em.name, []): for mt in mcp_tools.get(em.name, []):
if em.tools is not None and mt.name not in em.tools: if em.tools is not None and mt.name not in em.tools:
continue continue
if any(fnmatch.fnmatchcase(mt.name, pat) for pat in em.deny):
continue
wire_name = f"{em.name}__{mt.name}" wire_name = f"{em.name}__{mt.name}"
routing[wire_name] = (em.name, mt.name) routing[wire_name] = (em.name, mt.name)
local_tools.append( local_tools.append(
+216 -16
View File
@@ -22,35 +22,63 @@ from __future__ import annotations
import asyncio import asyncio
import contextlib import contextlib
import functools
import logging import logging
import signal import signal
from contextlib import AsyncExitStack from contextlib import AsyncExitStack
from typing import TYPE_CHECKING from typing import TYPE_CHECKING, Any
import psycopg
import uvicorn import uvicorn
import uvloop import uvloop
from dotenv import load_dotenv from dotenv import load_dotenv
from pgqueuer import PsycopgDriver
from raycast_api import Client as RaycastClient from raycast_api import Client as RaycastClient
from raycast_api.config import Config as RaycastConfig from raycast_api.config import Config as RaycastConfig
from beaver_gateway import config_loader from beaver_gateway import config_loader
from beaver_gateway.agents.claude import ClaudeAgent from beaver_gateway.agents.claude import ClaudeAgent
from beaver_gateway.agents.raycast import RaycastAgent from beaver_gateway.agents.raycast import RaycastAgent
from beaver_gateway.backends.claude_code import ClaudeCodeBackendAdapter from beaver_gateway.backends.claude_sdk import (
ClaudeSdkBackend,
RunnerConfig,
UsageEvent,
)
from beaver_gateway.backends.raycast import RaycastBackend from beaver_gateway.backends.raycast import RaycastBackend
from beaver_gateway.core.auth import TokenStore from beaver_gateway.core.auth import TokenStore
from beaver_gateway.core.registry import AgentRegistry, McpRegistry from beaver_gateway.core.bus import EventBus
from beaver_gateway.core.conversations import Conversations
from beaver_gateway.core.envelope import Envelope
from beaver_gateway.core.gateway_tools import build_tool_server
from beaver_gateway.core.redact import install as install_redaction
from beaver_gateway.core.redact import load_secrets as load_secrets_to_mask
from beaver_gateway.core.registry import AgentRegistry, Gateway, McpRegistry
from beaver_gateway.core.rotation import Rotation, RotationPolicy
from beaver_gateway.core.scheduler import Scheduler
from beaver_gateway.core.sessions import SessionPool
from beaver_gateway.frontends._auth import require_token
from beaver_gateway.frontends.base import GatewayRuntime from beaver_gateway.frontends.base import GatewayRuntime
from beaver_gateway.frontends.root import build_root_app
from beaver_gateway.mcp.internal_app import build_internal_app from beaver_gateway.mcp.internal_app import build_internal_app
from beaver_gateway.settings import Settings from beaver_gateway.settings import Settings
from beaver_gateway.storage import Database from beaver_gateway.storage import (
Database,
PostgresSessionStore,
Usage,
append_audit,
append_usage,
)
if TYPE_CHECKING: if TYPE_CHECKING:
from claude_agent_sdk import McpSdkServerConfig
from fastmcp import FastMCP from fastmcp import FastMCP
from fastmcp.tools.base import Tool as FastMCPTool from fastmcp.tools.base import Tool as FastMCPTool
from starlette.applications import Starlette from starlette.applications import Starlette
from starlette.requests import Request
from starlette.types import ASGIApp
from beaver_gateway.backends.base import Backend from beaver_gateway.backends.base import Backend
from beaver_gateway.core.policy import ToolAudit
from beaver_gateway.mcp.types import McpServerT from beaver_gateway.mcp.types import McpServerT
@@ -62,6 +90,7 @@ def main() -> None:
logging.basicConfig( logging.basicConfig(
level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s" level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s"
) )
install_redaction()
_install_sigterm_handler() _install_sigterm_handler()
asyncio.run(_async_main(), loop_factory=uvloop.new_event_loop) asyncio.run(_async_main(), loop_factory=uvloop.new_event_loop)
@@ -97,6 +126,9 @@ async def _async_main() -> None:
# populates Settings fields, not the process environment. # populates Settings fields, not the process environment.
# ``override=False``: real env vars (Docker, systemd) win over .env. # ``override=False``: real env vars (Docker, systemd) win over .env.
load_dotenv(override=False) load_dotenv(override=False)
# Only now does the process environment hold the credentials the
# redactor masks literally (in Docker they arrive via ``env_file``).
load_secrets_to_mask()
settings = Settings() # ty: ignore[missing-argument] settings = Settings() # ty: ignore[missing-argument]
gateway = config_loader.load(settings.config_path) gateway = config_loader.load(settings.config_path)
@@ -114,7 +146,9 @@ async def _async_main() -> None:
# background task). BOOTSTRAP_TOKENS layers on top so first-run / # background task). BOOTSTRAP_TOKENS layers on top so first-run /
# examples still work without DB writes. # examples still work without DB writes.
token_store = TokenStore( token_store = TokenStore(
db, bootstrap=TokenStore.parse_bootstrap(settings.bootstrap_tokens) db,
bootstrap=TokenStore.parse_bootstrap(settings.bootstrap_tokens),
bootstrap_scopes=TokenStore.parse_bootstrap_scopes(settings.bootstrap_tokens),
) )
async with AsyncExitStack() as stack: async with AsyncExitStack() as stack:
@@ -122,8 +156,8 @@ async def _async_main() -> None:
await token_store.start() await token_store.start()
stack.push_async_callback(token_store.stop) stack.push_async_callback(token_store.stop)
# Internal MCP URLs must exist before we construct any # Internal MCP URLs must exist before we construct any
# ClaudeCodeBackendAdapter — adapters bake the URLs into their # ClaudeSdkBackend - adapters bake the URLs into their
# ``BackendOptions.mcp_servers`` at construction time. The # ``mcp_servers`` at construction time. The
# ``mcp_servers`` map is used by the Raycast backend, which # ``mcp_servers`` map is used by the Raycast backend, which
# needs in-process ``list_tools`` / ``call_tool`` access (the # needs in-process ``list_tools`` / ``call_tool`` access (the
# Raycast wire has no native MCP concept). # Raycast wire has no native MCP concept).
@@ -136,14 +170,48 @@ async def _async_main() -> None:
# surfaces at startup instead of mid-conversation. # surfaces at startup instead of mid-conversation.
mcp_tools = await _prefetch_mcp_tools(mcp_servers) mcp_tools = await _prefetch_mcp_tools(mcp_servers)
pool = SessionPool()
bus = EventBus()
late = _LateConversations()
session_store = PostgresSessionStore(db)
backends: dict[str, Backend] = await _build_backends( backends: dict[str, Backend] = await _build_backends(
settings=settings, settings=settings,
agents=agents, agents=agents,
stack=stack, stack=stack,
db=db,
session_store=session_store,
mcp_internal_urls=internal_urls, mcp_internal_urls=internal_urls,
mcp_servers=mcp_servers, mcp_servers=mcp_servers,
mcp_tools=mcp_tools, mcp_tools=mcp_tools,
pool=pool,
late=late,
) )
conversations = Conversations(
db=db,
agents=agents,
backends=backends,
bus=bus,
pool=pool,
store=session_store,
texts=gateway.texts,
frontends=gateway.frontends,
envelope=Envelope(
watch=gateway.watch, tz=gateway.tz, recall=gateway.recall
),
distiller=gateway.distiller,
user_sink=gateway.user_sink,
)
late.conversations = conversations
scheduler = Scheduler(
conversations=conversations,
jobs=gateway.jobs,
driver=await _pgqueuer_driver(settings.database_url, stack),
budget=gateway.budget,
rotation=Rotation(
conversations, gateway.rotation or RotationPolicy(tz=gateway.tz)
),
)
conversations.scheduler = scheduler
runtime = GatewayRuntime( runtime = GatewayRuntime(
agents=agents, agents=agents,
@@ -156,6 +224,11 @@ async def _async_main() -> None:
admin_pass=settings.admin_pass, admin_pass=settings.admin_pass,
session_secret=settings.session_secret, session_secret=settings.session_secret,
frontends=tuple(gateway.frontends), frontends=tuple(gateway.frontends),
conversations=conversations,
bus=bus,
pool=pool,
scheduler=scheduler,
public_url=gateway.public_url.rstrip("/") if gateway.public_url else None,
) )
for fe in gateway.frontends: for fe in gateway.frontends:
@@ -181,13 +254,60 @@ async def _async_main() -> None:
# it in this path and exit cleanly (Phase 0 DoD). # it in this path and exit cleanly (Phase 0 DoD).
return return
await conversations.start()
stack.push_async_callback(conversations.stop)
await scheduler.start()
stack.push_async_callback(scheduler.stop)
hooks = scheduler.app(
functools.partial(_authorize_hook, runtime=runtime, scope="api")
)
async with asyncio.TaskGroup() as tg: async with asyncio.TaskGroup() as tg:
tg.create_task(pool.reap_loop())
if internal_app is not None: if internal_app is not None:
tg.create_task(_serve_internal_mcp(internal_app, settings=settings)) tg.create_task(_serve_internal_mcp(internal_app, settings=settings))
tg.create_task(_serve_root(gateway, extra={"/hooks": hooks}))
if gateway.watch is not None:
tg.create_task(gateway.watch.run())
for fe in gateway.frontends: for fe in gateway.frontends:
tg.create_task(fe.serve()) tg.create_task(fe.serve())
async def _authorize_hook(
request: Request, *, runtime: GatewayRuntime, scope: str
) -> str:
return await require_token(request, runtime, scope=scope)
async def _pgqueuer_driver(url: str, stack: AsyncExitStack) -> PsycopgDriver | None:
"""A dedicated autocommit connection for pgqueuer's LISTEN/NOTIFY."""
plain = _plain_postgres_url(url)
if plain is None:
return None
conn = await psycopg.AsyncConnection.connect(plain, autocommit=True)
stack.push_async_callback(conn.close)
return PsycopgDriver(conn)
def _plain_postgres_url(url: str) -> str | None:
for prefix in ("postgresql+psycopg://", "postgresql://", "postgres://"):
if url.startswith(prefix):
return "postgresql://" + url[len(prefix) :]
return None
async def _serve_root(gateway: Gateway, *, extra: dict[str, ASGIApp]) -> None:
app = build_root_app(gateway.frontends, extra=extra)
config = uvicorn.Config(app, host=gateway.host, port=gateway.port, log_level="info")
_log.info(
"gateway on http://%s:%d - %s",
gateway.host,
gateway.port,
", ".join([*(fe.path for fe in gateway.frontends if fe.path), *extra])
or "no http frontends",
)
await uvicorn.Server(config).serve()
def _build_internal_mcp( def _build_internal_mcp(
mcps: list[McpServerT], *, settings: Settings mcps: list[McpServerT], *, settings: Settings
) -> tuple[Starlette | None, dict[str, str], dict[str, FastMCP]]: ) -> tuple[Starlette | None, dict[str, str], dict[str, FastMCP]]:
@@ -247,27 +367,49 @@ async def _serve_internal_mcp(app: Starlette, *, settings: Settings) -> None:
await server.serve() await server.serve()
class _LateConversations:
"""Backends need a tool-server factory before the service that backs it exists."""
conversations: Conversations | None = None
def server(
self, key: str, _kind: str, names: tuple[str, ...]
) -> McpSdkServerConfig | None:
if self.conversations is None or not names:
return None
return build_tool_server(self.conversations, conversation_key=key, names=names)
async def ask(self, key: str, payload: dict[str, Any]) -> str:
if self.conversations is None:
msg = "conversations service is not up yet"
raise RuntimeError(msg)
answer = await self.conversations.ask(key, payload)
return self.conversations.answer_text(answer)
async def _build_backends( async def _build_backends(
*, *,
settings: Settings, settings: Settings,
agents: AgentRegistry, agents: AgentRegistry,
stack: AsyncExitStack, stack: AsyncExitStack,
db: Database,
session_store: PostgresSessionStore,
mcp_internal_urls: dict[str, str], mcp_internal_urls: dict[str, str],
mcp_servers: dict[str, FastMCP], mcp_servers: dict[str, FastMCP],
mcp_tools: dict[str, list[FastMCPTool]], mcp_tools: dict[str, list[FastMCPTool]],
pool: SessionPool,
late: _LateConversations,
) -> dict[str, Backend]: ) -> dict[str, Backend]:
"""Construct one backend per agent name. """Construct one backend per agent name.
The Raycast ``Client`` is shared across every ``RaycastAgent`` The Raycast ``Client`` is shared across every ``RaycastAgent``
(bearer + device-id are process-wide), so we open it lazily — only (bearer + device-id are process-wide), so we open it lazily - only
when at least one ``RaycastAgent`` is present — and close it via when at least one ``RaycastAgent`` is present - and close it via
the caller's exit stack. the caller's exit stack.
Each :class:`ClaudeAgent` gets its own Each :class:`ClaudeAgent` gets its own :class:`ClaudeSdkBackend`
:class:`ClaudeCodeBackendAdapter`: ``BackendOptions`` pins (own live-session pool, own prompt and MCP set); all of them share
``cwd`` / ``model`` / ``system_prompt`` / ``mcp_servers`` for the the session store and the usage sink.
lifetime of the underlying ``ClaudeCodeBackend``, so different
agents can't share one.
""" """
backends: dict[str, Backend] = {} backends: dict[str, Backend] = {}
@@ -281,10 +423,68 @@ async def _build_backends(
for a in raycast_agents: for a in raycast_agents:
backends[a.name] = raycast_backend backends[a.name] = raycast_backend
runner = RunnerConfig(user=settings.claude_runner_user, home=settings.claude_home)
mcp_tool_names = {
name: [t.name for t in tools] for name, tools in mcp_tools.items()
}
async def record_usage(event: UsageEvent) -> None:
row = Usage(
agent_name=event.agent_name,
conversation_id=event.conversation_id,
session_id=event.session_id,
model=event.model,
effort=event.effort,
input_tokens=event.usage.input_tokens,
output_tokens=event.usage.output_tokens,
cache_read_tokens=event.usage.cache_read_tokens,
cache_creation_tokens=event.usage.cache_creation_tokens,
context_tokens=event.usage.context_tokens,
cost_usd=event.usage.cost_usd,
duration_ms=event.usage.duration_ms,
num_turns=event.usage.num_turns,
model_usage=event.usage.model_usage,
)
try:
async with db.session() as session:
await append_usage(session, row)
except Exception: # noqa: BLE001
_log.exception("usage write failed for %s", event.agent_name)
async def record_tool(event: ToolAudit) -> None:
detail = {
"conversation": event.conversation,
"kind": event.kind,
"tool": event.tool,
"decision": event.decision,
"reason": event.reason,
"brief": event.brief,
}
try:
async with db.session() as session:
await append_audit(
session,
actor=f"agent:{event.agent}",
kind="tool_call",
agent_name=event.agent,
detail=detail,
)
except Exception: # noqa: BLE001
_log.exception("tool audit write failed for %s", event.agent)
for a in agents: for a in agents:
if isinstance(a, ClaudeAgent): if isinstance(a, ClaudeAgent):
adapter = ClaudeCodeBackendAdapter( adapter = ClaudeSdkBackend(
agent=a, mcp_internal_urls=mcp_internal_urls agent=a,
mcp_internal_urls=mcp_internal_urls,
session_store=session_store,
mcp_tool_names=mcp_tool_names,
runner=runner,
usage_sink=record_usage,
pool=pool,
tool_server=functools.partial(late.server, names=a.gateway_tools),
asker=late.ask,
audit_sink=record_tool,
) )
await stack.enter_async_context(adapter) await stack.enter_async_context(adapter)
backends[a.name] = adapter backends[a.name] = adapter
+22
View File
@@ -14,12 +14,17 @@ contents here before handing it back.
from __future__ import annotations from __future__ import annotations
import sys
from typing import TYPE_CHECKING, Any from typing import TYPE_CHECKING, Any
from croniter import croniter
from beaver_gateway.agents.base import BaseAgent, ExposedMcp from beaver_gateway.agents.base import BaseAgent, ExposedMcp
from beaver_gateway.agents.claude import ClaudeAgent from beaver_gateway.agents.claude import ClaudeAgent
from beaver_gateway.agents.raycast import RaycastAgent from beaver_gateway.agents.raycast import RaycastAgent
from beaver_gateway.core.conversations import ConversationTexts
from beaver_gateway.core.registry import Gateway from beaver_gateway.core.registry import Gateway
from beaver_gateway.core.scheduler import Job
from beaver_gateway.frontends.base import Frontend from beaver_gateway.frontends.base import Frontend
from beaver_gateway.mcp.types import HttpMcp, McpServer, PythonToolMcp, StdioMcp from beaver_gateway.mcp.types import HttpMcp, McpServer, PythonToolMcp, StdioMcp
@@ -37,6 +42,7 @@ _PUBLIC_NAMES: dict[str, Any] = {
"McpServer": McpServer, "McpServer": McpServer,
"ExposedMcp": ExposedMcp, "ExposedMcp": ExposedMcp,
"Gateway": Gateway, "Gateway": Gateway,
"ConversationTexts": ConversationTexts,
} }
_McpInstance = StdioMcp | HttpMcp | PythonToolMcp _McpInstance = StdioMcp | HttpMcp | PythonToolMcp
@@ -54,6 +60,10 @@ def load(path: Path) -> Gateway:
raise ConfigError(msg) from exc raise ConfigError(msg) from exc
code = compile(source, str(path), "exec") code = compile(source, str(path), "exec")
# Siblings of the config (``policy.py``, ``mcps/``) import by name.
parent = str(path.resolve().parent)
if parent not in sys.path:
sys.path.insert(0, parent)
namespace: dict[str, Any] = {"__file__": str(path), **_PUBLIC_NAMES} namespace: dict[str, Any] = {"__file__": str(path), **_PUBLIC_NAMES}
exec(code, namespace) # noqa: S102 - exec'ing user config is the feature exec(code, namespace) # noqa: S102 - exec'ing user config is the feature
@@ -97,3 +107,15 @@ def _validate(gw: Gateway, path: Path) -> None:
f"got {type(f).__name__}" f"got {type(f).__name__}"
) )
raise ConfigError(msg) raise ConfigError(msg)
names: set[str] = set()
for i, j in enumerate(gw.jobs):
if not isinstance(j, Job):
msg = f"{path}: gateway.jobs[{i}] must be a Job, got {type(j).__name__}"
raise ConfigError(msg)
if j.name in names:
msg = f"{path}: duplicate job name {j.name!r}"
raise ConfigError(msg)
names.add(j.name)
if j.cron is not None and not croniter.is_valid(j.cron):
msg = f"{path}: job {j.name!r} has an invalid cron {j.cron!r}"
raise ConfigError(msg)
+1
View File
@@ -51,6 +51,7 @@ KNOWN_KINDS: frozenset[str] = frozenset(
{ {
"messages", # POST /v1/messages accepted "messages", # POST /v1/messages accepted
"mcp_call", # /mcp/<ns>/... proxied "mcp_call", # /mcp/<ns>/... proxied
"tool_call", # a model's tool call seen by the PreToolUse hook
"login_ok", "login_ok",
"login_failed", "login_failed",
"logout", "logout",
+45 -20
View File
@@ -60,12 +60,13 @@ class TokenStoreError(ValueError):
"""Malformed ``BOOTSTRAP_TOKENS`` value or duplicate token.""" """Malformed ``BOOTSTRAP_TOKENS`` value or duplicate token."""
VALID_SCOPES: frozenset[str] = frozenset({"*", "messages", "mcp", "admin"}) VALID_SCOPES: frozenset[str] = frozenset({"*", "messages", "mcp", "admin", "api"})
"""The scopes a ``Token.scope`` may hold (Phase 4.3 admin UI enforces). """The scopes a ``Token.scope`` may hold (Phase 4.3 admin UI enforces).
* ``*`` — wildcard, may use any frontend * ``*`` — wildcard, may use any frontend
* ``messages`` — Anthropic Messages frontend only * ``messages`` — Anthropic Messages frontend only
* ``mcp`` — MCP server frontend only * ``mcp`` — MCP server frontend only
* ``api`` — conversations API (``/api``) and its SSE
* ``admin`` — reserved for programmatic admin access; the AdminFrontend * ``admin`` — reserved for programmatic admin access; the AdminFrontend
itself authenticates via session cookies, not bearer tokens, so this itself authenticates via session cookies, not bearer tokens, so this
scope is unused today and kept for forward compatibility. scope is unused today and kept for forward compatibility.
@@ -154,6 +155,7 @@ class TokenStore:
db: Database | None = None, db: Database | None = None,
*, *,
bootstrap: Mapping[str, str] | None = None, bootstrap: Mapping[str, str] | None = None,
bootstrap_scopes: Mapping[str, str] | None = None,
ttl_seconds: float = 30.0, ttl_seconds: float = 30.0,
flush_interval: float = 5.0, flush_interval: float = 5.0,
) -> None: ) -> None:
@@ -172,9 +174,13 @@ class TokenStore:
raise TokenStoreError(msg) raise TokenStoreError(msg)
by_value[value] = name by_value[value] = name
self._bootstrap_by_value: dict[str, str] = by_value self._bootstrap_by_value: dict[str, str] = by_value
self._bootstrap_scopes: dict[str, str] = dict.fromkeys( # Bootstrap entries are ``"*"`` unless the env narrows them
by_value.values(), _BOOTSTRAP_SCOPE # (``name:value:scope``) - a webhook token in a URL should not be
) # an admin token.
self._bootstrap_scopes: dict[str, str] = {
name: (bootstrap_scopes or {}).get(name, _BOOTSTRAP_SCOPE)
for name in by_value.values()
}
self._db = db self._db = db
self._ttl = ttl_seconds self._ttl = ttl_seconds
@@ -190,22 +196,17 @@ class TokenStore:
@staticmethod @staticmethod
def parse_bootstrap(raw: str) -> dict[str, str]: def parse_bootstrap(raw: str) -> dict[str, str]:
"""Parse ``name1:value1,name2:value2`` (the ``BOOTSTRAP_TOKENS`` form).""" """Parse ``name1:value1,name2:value2[:scope]`` (``BOOTSTRAP_TOKENS``)."""
tokens: dict[str, str] = {} return {name: value for name, (value, _scope) in _parse_entries(raw).items()}
for chunk in raw.split(","):
entry = chunk.strip() @staticmethod
if not entry: def parse_bootstrap_scopes(raw: str) -> dict[str, str]:
continue """Scopes given as a third field in ``BOOTSTRAP_TOKENS``; the rest get ``*``."""
name, sep, value = entry.partition(":") return {
if not sep: name: scope
msg = f"token entry missing ':' separator: {entry!r}" for name, (_value, scope) in _parse_entries(raw).items()
raise TokenStoreError(msg) if scope is not None
name, value = name.strip(), value.strip() }
if name in tokens:
msg = f"duplicate token name: {name!r}"
raise TokenStoreError(msg)
tokens[name] = value
return tokens
@classmethod @classmethod
def from_env(cls, raw: str, db: Database | None = None) -> TokenStore: def from_env(cls, raw: str, db: Database | None = None) -> TokenStore:
@@ -235,6 +236,11 @@ class TokenStore:
self._flusher_task = None self._flusher_task = None
await self._flush_now() await self._flush_now()
def grant(self, name: str, value: str, *, scope: str = _BOOTSTRAP_SCOPE) -> None:
"""Add an in-memory token for the process lifetime (the admin UI's bearer)."""
self._bootstrap_by_value[value] = name
self._bootstrap_scopes[name] = scope
async def invalidate(self) -> None: async def invalidate(self) -> None:
"""Force the next verify to re-read from DB (Phase 4.3 admin hook).""" """Force the next verify to re-read from DB (Phase 4.3 admin hook)."""
self._loaded_at = 0.0 self._loaded_at = 0.0
@@ -372,3 +378,22 @@ __all__ = [
"TokenStoreError", "TokenStoreError",
"hash_token", "hash_token",
] ]
def _parse_entries(raw: str) -> dict[str, tuple[str, str | None]]:
entries: dict[str, tuple[str, str | None]] = {}
for chunk in raw.split(","):
entry = chunk.strip()
if not entry:
continue
name, sep, rest = entry.partition(":")
if not sep:
msg = f"token entry missing ':' separator: {entry!r}"
raise TokenStoreError(msg)
value, sep, scope = rest.partition(":")
name, value, scope = name.strip(), value.strip(), scope.strip()
if name in entries:
msg = f"duplicate token name: {name!r}"
raise TokenStoreError(msg)
entries[name] = (value, scope if sep and scope else None)
return entries
+69
View File
@@ -0,0 +1,69 @@
"""In-process event bus: what the gateway does, as a stream frontends can tap.
Events are plain dicts ``{"type", "seq", "ts", ...}``. Publishers never
block; a subscriber that falls behind loses its oldest events rather than
stalling the turn that produced them. ``/api/events`` serialises the
stream as SSE, the panel builds its subagent tree from ``stream`` events
carrying ``parent_tool_use_id``.
"""
from __future__ import annotations
import asyncio
import contextlib
import itertools
from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from collections.abc import AsyncIterator
__all__ = ["Event", "EventBus"]
Event = dict[str, Any]
class EventBus:
def __init__(self, *, maxsize: int = 2000) -> None:
self._maxsize = maxsize
self._subscribers: set[asyncio.Queue[Event]] = set()
self._seq = itertools.count(1)
def publish(self, type_: str, **data: Any) -> Event:
event: Event = {
"type": type_,
"seq": next(self._seq),
"ts": datetime.now(UTC).isoformat(timespec="milliseconds"),
**data,
}
for queue in list(self._subscribers):
if queue.full():
with contextlib.suppress(asyncio.QueueEmpty):
queue.get_nowait()
queue.put_nowait(event)
return event
@contextlib.asynccontextmanager
async def subscribe(self) -> AsyncIterator[asyncio.Queue[Event]]:
queue: asyncio.Queue[Event] = asyncio.Queue(maxsize=self._maxsize)
self._subscribers.add(queue)
try:
yield queue
finally:
self._subscribers.discard(queue)
async def stream(
self, *, conversation_id: str | None = None
) -> AsyncIterator[Event]:
async with self.subscribe() as queue:
while True:
event = await queue.get()
if (
conversation_id is None
or event.get("conversation_id") == conversation_id
):
yield event
@property
def subscribers(self) -> int:
return len(self._subscribers)
+46 -18
View File
@@ -63,6 +63,7 @@ __all__ = [
"load_messages", "load_messages",
"mint_conversation", "mint_conversation",
"rewrite_messages", "rewrite_messages",
"set_session_id",
] ]
@@ -86,6 +87,12 @@ class ForkOutcome:
messages: list[MessageParam] messages: list[MessageParam]
persist_messages: list[dict[str, Any]] persist_messages: list[dict[str, Any]]
divergence_index: int | None divergence_index: int | None
edited: bool = False
"""An earlier assistant turn's prose was rewritten in the file."""
@property
def reuse_session(self) -> bool:
return self.divergence_index is None and not self.edited
# ---- public store API --------------------------------------------------- # ---- public store API ---------------------------------------------------
@@ -121,6 +128,17 @@ async def mint_conversation(
return row return row
async def set_session_id(
session: AsyncSession, *, conversation_id: int, session_id: str | None
) -> None:
conv = await session.get(Conversation, conversation_id)
if conv is None or conv.session_id == session_id:
return
conv.session_id = session_id
session.add(conv)
await session.commit()
async def load_messages( async def load_messages(
session: AsyncSession, *, conversation_id: int session: AsyncSession, *, conversation_id: int
) -> list[dict[str, Any]]: ) -> list[dict[str, Any]]:
@@ -395,7 +413,7 @@ def diff_and_fork(
new_user_turn = incoming[-1] new_user_turn = incoming[-1]
prior_incoming = incoming[:-1] prior_incoming = incoming[:-1]
spliced_groups, divergence = _walk_prefix(prior_incoming, stored_groups) spliced_groups, divergence, edited = _walk_prefix(prior_incoming, stored_groups)
if divergence is None and len(prior_incoming) < len(stored_groups): if divergence is None and len(prior_incoming) < len(stored_groups):
if _file_lags_store(stored_groups, len(prior_incoming), new_user_turn): if _file_lags_store(stored_groups, len(prior_incoming), new_user_turn):
@@ -421,6 +439,7 @@ def diff_and_fork(
messages=backend_msgs, messages=backend_msgs,
persist_messages=persist_msgs, persist_messages=persist_msgs,
divergence_index=divergence, divergence_index=divergence,
edited=edited,
) )
@@ -458,44 +477,50 @@ def _file_lags_store(
def _walk_prefix( def _walk_prefix(
prior_incoming: list[ParsedTurn], stored_groups: list[_StoredDisplayTurn] prior_incoming: list[ParsedTurn], stored_groups: list[_StoredDisplayTurn]
) -> tuple[list[list[dict[str, Any]]], int | None]: ) -> tuple[list[list[dict[str, Any]]], int | None, bool]:
"""Walk incoming vs stored side-by-side until first divergence. """Walk incoming vs stored side-by-side until first divergence.
Returns the spliced/matched group list (one entry per matched Returns the spliced/matched group list (one entry per matched
display turn, each carrying the raw messages we'll feed to the display turn, each carrying the raw messages we'll feed to the
backend for that turn) and the divergence index (``None`` if all backend for that turn), the divergence index (``None`` if all
of ``prior_incoming`` matched). of ``prior_incoming`` matched) and whether any assistant prose
was spliced in from the file - a rewritten reply keeps the
structure but must not resume the session that said otherwise.
""" """
from beaver_gateway.frontends.markdown.parser import TextSegment, ToolSegment from beaver_gateway.frontends.markdown.parser import TextSegment, ToolSegment
spliced_groups: list[list[dict[str, Any]]] = [] spliced_groups: list[list[dict[str, Any]]] = []
edited = False
for i, inc in enumerate(prior_incoming): for i, inc in enumerate(prior_incoming):
if i >= len(stored_groups): if i >= len(stored_groups):
return spliced_groups, i return spliced_groups, i, edited
st = stored_groups[i] st = stored_groups[i]
if inc.role != st.role: if inc.role != st.role:
return spliced_groups, i return spliced_groups, i, edited
if inc.role == "user": if inc.role == "user":
if inc.text != st.spoken_text: if inc.text != st.spoken_text:
return spliced_groups, i return spliced_groups, i, edited
spliced_groups.append(list(st.messages)) spliced_groups.append(list(st.messages))
continue continue
inc_skeleton = tuple( inc_skeleton = tuple(
s.name for s in inc.structure if isinstance(s, ToolSegment) s.name for s in inc.structure if isinstance(s, ToolSegment)
) )
inc_text_count = sum(1 for s in inc.structure if isinstance(s, TextSegment)) inc_text_count = sum(1 for s in inc.structure if isinstance(s, TextSegment))
if inc_skeleton != st.skeleton: # Files rendered without tool callouts (§3.10) carry no skeleton:
return spliced_groups, i # prose alone decides whether the turn matched.
if inc_skeleton and inc_skeleton != st.skeleton:
return spliced_groups, i, edited
if inc.text == st.spoken_text: if inc.text == st.spoken_text:
spliced_groups.append(list(st.messages)) spliced_groups.append(list(st.messages))
continue continue
if inc_text_count != st.text_segment_count: if inc_skeleton and inc_text_count != st.text_segment_count:
return spliced_groups, i return spliced_groups, i, edited
spliced = _splice_assistant_group(stored_group=st, incoming=inc) spliced = _splice_assistant_group(stored_group=st, incoming=inc)
if spliced is None: if spliced is None:
return spliced_groups, i return spliced_groups, i, edited
spliced_groups.append(spliced) spliced_groups.append(spliced)
return spliced_groups, None edited = True
return spliced_groups, None, edited
def _assemble_tail( def _assemble_tail(
@@ -539,6 +564,13 @@ def _splice_assistant_group(
return _splice_by_rebuild(stored_group=stored_group, incoming=incoming) return _splice_by_rebuild(stored_group=stored_group, incoming=incoming)
def _is_text_block(blk: object) -> bool:
if not isinstance(blk, dict):
return False
block = cast("dict[str, Any]", blk)
return block.get("type") == "text" and bool(str(block.get("text", "")).strip())
def _splice_in_place( def _splice_in_place(
*, stored_group: _StoredDisplayTurn, incoming: ParsedTurn *, stored_group: _StoredDisplayTurn, incoming: ParsedTurn
) -> list[dict[str, Any]] | None: ) -> list[dict[str, Any]] | None:
@@ -570,11 +602,7 @@ def _splice_in_place(
if msg["role"] != "assistant" or not isinstance(content, list): if msg["role"] != "assistant" or not isinstance(content, list):
continue continue
positions.extend( positions.extend(
(mi, bi) (mi, bi) for bi, blk in enumerate(content) if _is_text_block(blk)
for bi, blk in enumerate(content)
if isinstance(blk, dict)
and blk.get("type") == "text"
and str(blk.get("text", "")).strip()
) )
if len(positions) != len(new_texts): if len(positions) != len(new_texts):
return None return None
File diff suppressed because it is too large. Load diff
+197
View File
@@ -0,0 +1,197 @@
"""Closing a deep chat: the digest, the index, the file line cap (§6.4, §8.4).
The gateway knows no path by itself (§0.8): ``Distiller`` from ``config.py``
names the agent, says where digests land and where the index lives, and
the distiller writes the file on its own. What the gateway does is check that a file
with a valid frontmatter appeared under ``Distiller.dir`` during the fork
turn, put one line into the index, and cap the merge text at
``SUMMARY_LINES``. ``LineCap`` is the same idea for a file a job rewrites
(``состояние.md``): a result longer than the cap is bounced - the file
goes back to what it was and the job is told to shorten.
"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import date, datetime
from pathlib import Path
from typing import TYPE_CHECKING, Any
import frontmatter
if TYPE_CHECKING:
from collections.abc import Iterable, Sequence
from beaver_gateway.storage.models import Conversation
__all__ = [
"SUMMARY_LINES",
"Digest",
"DistillContext",
"Distiller",
"LineCap",
"append_index",
"check_digest",
"find_digest",
"index_line",
"trim_summary",
"written_paths",
]
SUMMARY_LINES = 5
"""A merge into the master is at most this many lines (§6.4)."""
@dataclass(frozen=True, slots=True)
class Distiller:
"""Who closes deep chats, where digests land, where the index lives."""
agent: str
dir: Path
index: Path
type: str = "выжимка"
"""Value the ``type`` frontmatter key must carry."""
index_header: str = "# индекс\n\nстрока на выжимку: чат → его выжимка.\n" # noqa: RUF001
@dataclass(frozen=True, slots=True)
class DistillContext:
"""What the setup's distill prompt is built from."""
conversation: Conversation
title: str | None
source: str | None
"""The window the chat lives in (a vault-relative path for markdown)."""
chat_name: str
"""What a ``[[wikilink]]`` to the chat is called."""
memory: bool
reason: str
day: date
@dataclass(frozen=True, slots=True)
class Digest:
path: Path
source: str
date: date
@dataclass(frozen=True, slots=True)
class LineCap:
"""A file a job rewrites may not exceed ``max_lines`` after its turn."""
path: Path
max_lines: int
def as_flags(self) -> dict[str, Any]:
return {"path": str(self.path), "max_lines": self.max_lines}
@classmethod
def from_flags(cls, value: Any) -> LineCap | None:
if not isinstance(value, dict):
return None
path, max_lines = value.get("path"), value.get("max_lines")
if not isinstance(path, str) or not isinstance(max_lines, int):
return None
return cls(path=Path(path), max_lines=max_lines)
def written_paths(messages: Iterable[dict[str, Any]]) -> list[Path]:
"""Paths the turn's ``Write``/``Edit`` calls targeted, in order."""
out: list[Path] = []
for message in messages:
content = message.get("content")
if not isinstance(content, list):
continue
for block in content:
if not isinstance(block, dict) or block.get("type") != "tool_use":
continue
if block.get("name") not in ("Write", "Edit", "MultiEdit"):
continue
target = (block.get("input") or {}).get("file_path")
if isinstance(target, str) and target:
out.append(Path(target))
return out
def find_digest(
digests: Distiller, *, since: datetime, written: Sequence[Path] = ()
) -> Path | None:
"""The digest the fork turn produced.
A written path under ``dir`` first, otherwise the newest file there
modified since ``since``.
"""
root = digests.dir.resolve()
for path in reversed(list(written)):
candidate = path if path.is_absolute() else digests.dir / path
try:
inside = candidate.resolve().relative_to(root)
except (OSError, ValueError):
continue
if (digests.dir / inside).is_file():
return digests.dir / inside
if not digests.dir.is_dir():
return None
stamp = since.timestamp() - 1
fresh = [
p
for p in digests.dir.glob("*.md")
if p.is_file() and p.stat().st_mtime >= stamp
]
return max(fresh, key=lambda p: p.stat().st_mtime) if fresh else None
def check_digest(path: Path, digests: Distiller) -> Digest | str:
"""The digest's frontmatter parsed, or why it is not a digest."""
try:
post = frontmatter.load(str(path))
except (OSError, ValueError) as exc:
return f"не читается: {exc}"
meta = post.metadata
if meta.get("type") != digests.type:
return f"`type` должен быть `{digests.type}`, не {meta.get('type')!r}"
source = meta.get("source")
if not isinstance(source, str) or not source.strip():
return "`source` пустой"
when = meta.get("date")
if isinstance(when, datetime):
when = when.date()
elif isinstance(when, str):
try:
when = date.fromisoformat(when.strip())
except ValueError:
return f"`date` не дата: {when!r}"
if not isinstance(when, date):
return "`date` отсутствует"
if not post.content.strip():
return "тело пустое"
return Digest(path=path, source=source.strip(), date=when)
def index_line(digest: Digest, chat_name: str) -> str:
return f"- {digest.date.isoformat()} [[{chat_name}]] → [[{digest.path.stem}]]"
def append_index(digests: Distiller, line: str) -> None:
index = digests.index
text = index.read_text(encoding="utf-8") if index.exists() else ""
if line in text.splitlines():
return
if not text:
text = digests.index_header
if not text.endswith("\n"):
text += "\n"
index.parent.mkdir(parents=True, exist_ok=True)
index.write_text(text + line + "\n", encoding="utf-8")
def trim_summary(text: str, limit: int = SUMMARY_LINES) -> tuple[str, bool]:
"""The merge text cut to ``limit`` non-empty lines; ``True`` if it was."""
lines = [line.rstrip() for line in text.strip().splitlines() if line.strip()]
if len(lines) <= limit:
return "\n".join(lines), False
return "\n".join(lines[:limit]), True
+141
View File
@@ -0,0 +1,141 @@
"""The envelope (§3.3): a background block after the user's text.
Assembled when the turn starts, never when the message is queued: the
time and what changed in the vault since the last envelope (added lines
for the ``full`` files, names and counts for the rest). Ceilings keep it
a signal, not a document; the injects that ride along are bundled below
it by the queue, with their own header.
"""
from __future__ import annotations
import logging
from dataclasses import dataclass
from datetime import UTC, datetime
from typing import TYPE_CHECKING
from zoneinfo import ZoneInfo
if TYPE_CHECKING:
from collections.abc import Callable, Sequence
from beaver_gateway.core.watch import Change, VaultWatch
__all__ = ["Envelope", "RecallContext", "render"]
_log = logging.getLogger(__name__)
HEADER = (
"[конверт - фоновый сигнал, не обращение; "
"реагируй, только если относится к вопросу]"
)
@dataclass(frozen=True, slots=True)
class RecallContext:
"""What the setup's ``recall`` hook sees: the user's text and where it landed."""
text: str
kind: str
now: datetime
@dataclass(slots=True)
class Envelope:
watch: VaultWatch | None = None
tz: str = "UTC"
max_lines: int = 120
per_file: int = 30
names_only_within: float = 600.0
last_at: datetime | None = None
recall: Callable[[RecallContext], str | None] | None = None
"""Setup-side lookup run on the user's text at turn start: pointers into
the vault (a person's card, the agent's own notes, due dates) that the
gateway cannot know the paths of. Its lines go under the vault block;
a failure is logged and the envelope goes out without them."""
def build(
self, *, now: datetime | None = None, text: str = "", kind: str = "master"
) -> str:
now = now or datetime.now(UTC)
changes = self.watch.take() if self.watch is not None else []
names_only = (
self.last_at is not None
and (now - self.last_at).total_seconds() < self.names_only_within
)
out = render(
now=now,
tz=self.tz,
changes=changes,
since=self.last_at,
names_only=names_only,
max_lines=self.max_lines,
per_file=self.per_file,
)
self.last_at = now
block = self.recall_block(text=text, kind=kind, now=now)
return f"{out}\n{block}" if block else out
def recall_only(
self, *, text: str, kind: str, now: datetime | None = None
) -> str | None:
"""The recall lines under the header, without the vault diff (branches)."""
block = self.recall_block(text=text, kind=kind, now=now or datetime.now(UTC))
return f"{HEADER}\n{block}" if block else None
def recall_block(self, *, text: str, kind: str, now: datetime) -> str | None:
if self.recall is None or not text.strip():
return None
try:
block = self.recall(RecallContext(text=text, kind=kind, now=now))
except Exception: # noqa: BLE001
_log.exception("recall hook failed")
return None
return block.strip() or None if block else None
def render(
*,
now: datetime,
tz: str,
changes: Sequence[Change],
since: datetime | None,
names_only: bool,
max_lines: int = 120,
per_file: int = 30,
) -> str:
zone = ZoneInfo(tz)
stamp = now.astimezone(zone)
lines = [HEADER, f"время: {stamp:%Y-%m-%d %H:%M} ({_zone_label(tz)})"]
ordered = sorted(changes, key=lambda c: (not c.full, c.path))
since_label = (
f"с {since.astimezone(zone):%H:%M}" if since is not None else "со старта" # noqa: RUF001
)
if ordered:
names = ", ".join(f"{c.path} (+{c.added_count})" for c in ordered)
lines.append(f"vault, изменено {since_label}: {names}")
if not names_only:
_append_diffs(lines, ordered, max_lines=max_lines, per_file=per_file)
return "\n".join(lines[:max_lines])
def _append_diffs(
lines: list[str], changes: Sequence[Change], *, max_lines: int, per_file: int
) -> None:
budget = max_lines - len(lines) - 1
for change in changes:
if not change.full or not change.added:
continue
if budget < 3:
lines.append("… (потолок конверта)")
return
shown = change.added[: min(per_file, budget - 2)]
lines.append(f"--- {change.path}, только добавленное ---")
lines.extend(f"+ {line}" for line in shown)
budget -= 1 + len(shown)
if len(change.added) > len(shown):
lines.append(f"+ … ещё {len(change.added) - len(shown)}")
budget -= 1
def _zone_label(tz: str) -> str:
return tz.rsplit("/", 1)[-1].replace("_", " ")
+261
View File
@@ -0,0 +1,261 @@
"""In-process MCP server with the gateway's own tools (§3.1, §3.2).
One server per live session so every tool knows which conversation is
calling; ``alwaysLoad`` keeps the tools out of tool search. Which names a
session gets comes from ``ClaudeAgent.gateway_tools``.
"""
from __future__ import annotations
import logging
from dataclasses import replace
from typing import TYPE_CHECKING, Any, cast
from claude_agent_sdk import create_sdk_mcp_server, tool
from beaver_gateway.core.injects import URGENCY
from beaver_gateway.core.kinds import as_kind
from beaver_gateway.core.redact import redact_data
URGENCY_HELP = (
"normal waits for the hourly batch or rides with the next turn, wake "
"starts a turn as soon as the target is idle, urgent cuts a running turn"
)
TARGET_HELP = (
"conversation id, or `master` for the open master, `parent` for the "
"parent of this conversation"
)
if TYPE_CHECKING:
from collections.abc import Iterable
from claude_agent_sdk import McpSdkServerConfig, SdkMcpTool
from beaver_gateway.core.conversations import Conversations
__all__ = ["SERVER_NAME", "TOOL_NAMES", "build_tool_server"]
_log = logging.getLogger("beaver_gateway.core.gateway_tools")
SERVER_NAME = "gateway"
SAY_IN_USER_TURN = (
"not delivered: this turn was started by the human's message, so your "
"reply text reaches them by itself - put what you wanted to say into the "
"reply instead of repeating it here"
)
TOOL_NAMES = ("read_conversation", "spawn", "say", "schedule", "inject", "close_chat")
def build_tool_server(
conversations: Conversations, *, conversation_key: str, names: Iterable[str]
) -> McpSdkServerConfig | None:
wanted = set(names)
unknown = wanted - set(TOOL_NAMES)
if unknown:
msg = f"unknown gateway tools: {sorted(unknown)}"
raise ValueError(msg)
tools = [
_redacting(t)
for t in _tools(conversations, conversation_key)
if t.name in wanted
]
if not tools:
return None
server = create_sdk_mcp_server(SERVER_NAME, tools=tools)
return cast("McpSdkServerConfig", {**server, "alwaysLoad": True})
def _redacting(spec: SdkMcpTool[Any]) -> SdkMcpTool[Any]:
"""Put a tool's result through the same mask as every other MCP.
These tools are mounted in-process by the SDK, so they bypass the
``FastMCP`` middleware in :mod:`beaver_gateway.mcp.redacting` and
need the filter attached here instead. ``read_conversation`` is the
one that earns it: it replays a transcript, and a transcript written
before any of this existed can still hold a credential.
"""
inner = spec.handler
async def handler(args: Any) -> dict[str, Any]:
return cast("dict[str, Any]", redact_data(await inner(args)))
return replace(spec, handler=handler)
def _tools(conversations: Conversations, key: str) -> list[SdkMcpTool[Any]]:
async def current() -> Any:
conv = await conversations.get(key)
if conv is None:
msg = f"conversation {key} not found"
raise LookupError(msg)
return conv
async def target(raw: Any) -> Any:
conv = await conversations.resolve(str(raw), origin=await current())
if conv is None:
_log.warning("gateway tool in %s: conversation %r not found", key, raw)
return conv
@tool(
"read_conversation",
"Read another conversation (a branch, the master, a deep chat) as plain "
"text. `window` limits it to the last N user turns.",
{
"type": "object",
"properties": {
"id": {"type": "string", "description": TARGET_HELP},
"window": {"type": "integer", "minimum": 1},
},
"required": ["id"],
},
)
async def read_conversation(args: dict[str, Any]) -> dict[str, Any]:
conv = await target(args["id"])
if conv is None:
return _error(f"conversation {args['id']} not found ({TARGET_HELP})")
text = await conversations.read(conv, window=args.get("window"))
return _text(text or "(empty)")
@tool(
"spawn",
"Open a new conversation of the given kind (branch = your own thread, "
"deep = a long research chat, job = a headless task). `seed` is how it "
"starts: clean (nothing), morning (handout), copy (copy of this "
"conversation, last `window` turns), brief (your `text`). A branch "
"keeps your agent, other kinds get their frontend's default unless "
"`agent` says otherwise. Returns the id.",
{
"type": "object",
"properties": {
"kind": {"type": "string", "enum": ["branch", "deep", "job"]},
"agent": {"type": "string", "description": "agent name"},
"seed": {
"type": "string",
"enum": ["clean", "morning", "copy", "brief"],
"default": "clean",
},
"text": {"type": "string", "description": "brief for seed=brief"},
"title": {"type": "string"},
"window": {"type": "integer", "minimum": 1},
},
"required": ["kind"],
},
)
async def spawn(args: dict[str, Any]) -> dict[str, Any]:
parent = await current()
try:
child = await conversations.spawn(
kind=as_kind(str(args["kind"])),
agent=args.get("agent"),
seed=str(args.get("seed") or "clean"),
parent=parent,
text=args.get("text"),
title=args.get("title"),
window=args.get("window"),
origin="mcp",
)
except (ValueError, LookupError) as exc:
return _error(str(exc))
return _text(f"spawned {child.kind} {child.external_id}")
@tool(
"say",
"Say something to the human in the frontend this conversation is bound "
"to. The only way an inject-started turn can speak; silence is simply "
"not calling it. Refused in a turn started by the human's own message: "
"there your reply text reaches them by itself, so just write the reply.",
{"text": str},
)
async def say(args: dict[str, Any]) -> dict[str, Any]:
conv = await current()
if conversations.turn_origin(conv) == "user":
return _error(SAY_IN_USER_TURN)
await conversations.say(conv, str(args["text"]))
return _text("ok")
@tool(
"schedule",
"Promise yourself an inject later: `at` is `+15m`, `+2h`, `+1d` or an "
"ISO datetime; `text` arrives in this conversation at that time and "
"starts a turn (urgency wake). " + URGENCY_HELP + ".",
{
"type": "object",
"properties": {
"at": {"type": "string"},
"text": {"type": "string"},
"urgency": {"type": "string", "enum": list(URGENCY), "default": "wake"},
},
"required": ["at", "text"],
},
)
async def schedule(args: dict[str, Any]) -> dict[str, Any]:
conv = await current()
try:
job_id, when = await conversations.schedule(
conv,
str(args["at"]),
str(args["text"]),
urgency=cast("Any", args.get("urgency") or "wake"),
)
except (RuntimeError, ValueError) as exc:
return _error(str(exc))
return _text(f"scheduled #{job_id} at {when.isoformat(timespec='minutes')}")
@tool(
"inject",
"Put a system-origin message into another conversation's queue. "
+ URGENCY_HELP
+ ". An error here means nothing was delivered.",
{
"type": "object",
"properties": {
"conversation": {"type": "string", "description": TARGET_HELP},
"text": {"type": "string"},
"urgency": {
"type": "string",
"enum": list(URGENCY),
"default": "normal",
},
},
"required": ["conversation", "text"],
},
)
async def inject(args: dict[str, Any]) -> dict[str, Any]:
conv = await target(args["conversation"])
if conv is None:
return _error(
f"conversation {args['conversation']} not found ({TARGET_HELP})"
)
item = await conversations.inject(
conv,
str(args["text"]),
urgency=cast("Any", args.get("urgency") or "normal"),
origin="агент",
)
return _text(f"queued #{item.id}")
@tool(
"close_chat",
"Close this deep chat once the current reply is finished: the "
"distiller forks it, writes the digest and hands a short merge to the "
"master. Call it when the human says the discussion is over; the chat "
"file stays as it is.",
{"type": "object", "properties": {}},
)
async def close_chat(_args: dict[str, Any]) -> dict[str, Any]:
conv = await current()
try:
await conversations.request_close(conv)
except ValueError as exc:
return _error(str(exc))
return _text("ok: the chat closes after this reply")
return [read_conversation, spawn, say, schedule, inject, close_chat]
def _text(text: str) -> dict[str, Any]:
return {"content": [{"type": "text", "text": text}]}
def _error(text: str) -> dict[str, Any]:
return {"content": [{"type": "text", "text": text}], "is_error": True}
+216
View File
@@ -0,0 +1,216 @@
"""Persisted per-conversation queue, ``urgent > user > wake > normal`` (§3.4).
One ``ClaudeSDKClient`` runs one turn at a time, so ordering has to happen
before the client: the rows here are the queue, ``core/conversations``
runs one worker per conversation over them. ``urgent`` cuts a running
turn, ``user`` is the human, ``wake`` starts a turn as soon as the
conversation is idle and takes the queued normals with it, ``normal``
waits for the batching window or rides with the next turn. A row that is
still ``running`` when the gateway starts was cut by a restart; it is
flagged ``interrupted`` and never re-run.
"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import UTC, datetime
from typing import TYPE_CHECKING, Literal, cast
from sqlalchemy import func
from sqlmodel import col, select
from beaver_gateway.storage.models import InjectQueueItem
if TYPE_CHECKING:
from collections.abc import Iterable, Sequence
from beaver_gateway.storage.db import Database
from beaver_gateway.storage.models import Conversation
__all__ = [
"PRIORITY_RANK",
"URGENCY",
"InjectContext",
"InjectQueue",
"Priority",
"inject_header",
]
Priority = Literal["urgent", "user", "wake", "normal"]
PRIORITY_RANK: dict[str, int] = {"urgent": 0, "user": 1, "wake": 2, "normal": 3}
URGENCY: tuple[Priority, ...] = ("normal", "wake", "urgent")
"""What the API and the tools accept for ``urgency``: every priority but ``user``."""
INTERRUPTED_TURN = (
"[этот инжект прервал предыдущий тёрн: «Request interrupted» выше - "
"прерывание, не отказ от тулзы]"
)
@dataclass(frozen=True, slots=True)
class InjectContext:
"""What a header renderer knows about one queued inject."""
origin: str
priority: str
interrupted_turn: bool
def inject_header(ctx: InjectContext) -> str:
"""Default framing; a setup overrides it via ``ConversationTexts.inject_header``."""
head = (
f"[инжект: {ctx.origin} - это не Бобёр, отвечать не нужно, голос не обязателен]"
)
return f"{head}\n{INTERRUPTED_TURN}" if ctx.interrupted_turn else head
def context_of(item: InjectQueueItem) -> InjectContext:
return InjectContext(
origin=item.origin,
priority=item.priority,
interrupted_turn=bool(item.interrupted_turn),
)
class InjectQueue:
def __init__(self, db: Database) -> None:
self._db = db
async def push(
self, *, conversation_id: int, priority: Priority, origin: str, text: str
) -> InjectQueueItem:
if priority not in PRIORITY_RANK:
msg = f"unknown priority {priority!r}"
raise ValueError(msg)
row = InjectQueueItem(
conversation_id=conversation_id, priority=priority, origin=origin, text=text
)
async with self._db.session() as session:
session.add(row)
await session.commit()
await session.refresh(row)
return row
async def pending(self, conversation_id: int) -> list[InjectQueueItem]:
async with self._db.session() as session:
result = await session.exec(
select(InjectQueueItem)
.where(
InjectQueueItem.conversation_id == conversation_id,
InjectQueueItem.status == "queued",
)
.order_by(col(InjectQueueItem.created_at), col(InjectQueueItem.id))
)
rows = list(result.all())
rows.sort(key=lambda r: (PRIORITY_RANK.get(r.priority, 9), r.created_at))
return rows
async def conversations_with_pending(self) -> list[int]:
async with self._db.session() as session:
result = await session.exec(
select(InjectQueueItem.conversation_id)
.where(InjectQueueItem.status == "queued")
.distinct()
)
return list(result.all())
async def start(self, items: Iterable[InjectQueueItem], turn_id: str) -> None:
await self._mark(items, status="running", turn_id=turn_id, delivered=True)
async def finish(
self, items: Iterable[InjectQueueItem], status: str = "done"
) -> None:
await self._mark(items, status=status)
async def mark_interrupting(self, item: InjectQueueItem) -> None:
item.interrupted_turn = True
async with self._db.session() as session:
row = await session.get(InjectQueueItem, item.id)
if row is not None:
row.interrupted_turn = True
session.add(row)
await session.commit()
async def interrupted(self) -> Sequence[InjectQueueItem]:
async with self._db.session() as session:
result = await session.exec(
select(InjectQueueItem).where(InjectQueueItem.status == "running")
)
rows = list(result.all())
for row in rows:
row.status = "interrupted"
session.add(row)
await session.commit()
return rows
async def move(
self, source: Conversation, target: Conversation, *, priority: Priority
) -> int:
"""Re-home queued items of one priority (rotation carries normal over)."""
async with self._db.session() as session:
result = await session.exec(
select(InjectQueueItem).where(
InjectQueueItem.conversation_id == source.id,
InjectQueueItem.status == "queued",
InjectQueueItem.priority == priority,
)
)
rows = list(result.all())
for row in rows:
row.conversation_id = cast("int", target.id)
session.add(row)
await session.commit()
return len(rows)
async def recent(
self, conversation_id: int, *, limit: int = 50
) -> list[InjectQueueItem]:
async with self._db.session() as session:
result = await session.exec(
select(InjectQueueItem)
.where(InjectQueueItem.conversation_id == conversation_id)
.order_by(col(InjectQueueItem.id).desc())
.limit(limit)
)
return list(result.all())
async def latest(
self, conversation_ids: Iterable[int]
) -> dict[int, InjectQueueItem]:
ids = list(conversation_ids)
if not ids:
return {}
newest = (
select(func.max(InjectQueueItem.id))
.where(col(InjectQueueItem.conversation_id).in_(ids))
.group_by(col(InjectQueueItem.conversation_id))
)
async with self._db.session() as session:
result = await session.exec(
select(InjectQueueItem).where(col(InjectQueueItem.id).in_(newest))
)
return {row.conversation_id: row for row in result.all()}
async def _mark(
self,
items: Iterable[InjectQueueItem],
*,
status: str,
turn_id: str | None = None,
delivered: bool = False,
) -> None:
ids = [i.id for i in items if i.id is not None]
if not ids:
return
async with self._db.session() as session:
result = await session.exec(
select(InjectQueueItem).where(col(InjectQueueItem.id).in_(ids))
)
for row in result.all():
row.status = status
if turn_id is not None:
row.turn_id = turn_id
if delivered:
row.delivered_at = datetime.now(UTC)
session.add(row)
await session.commit()
+18
View File
@@ -0,0 +1,18 @@
"""Conversation kinds (§3.1) as one closed type for agents, frontends, service."""
from __future__ import annotations
from typing import Literal, get_args
__all__ = ["KINDS", "Kind", "as_kind"]
Kind = Literal["master", "branch", "deep", "job", "fork"]
KINDS: tuple[Kind, ...] = get_args(Kind)
def as_kind(value: str) -> Kind:
for kind in KINDS:
if value == kind:
return kind
msg = f"unknown conversation kind {value!r}"
raise ValueError(msg)
+174
View File
@@ -0,0 +1,174 @@
"""PreToolUse policy - the boundary without permission prompts (§3.7).
``bypassPermissions`` everywhere; what a model may do is decided by
mounts, ``disallowed_tools`` and the rules here. A rule is a callable
``(ToolCall) -> Deny | None`` declared per agent (``ClaudeAgent.policy``);
the SDK backend registers one in-process ``PreToolUse`` hook that runs
the rules in order and turns the first :class:`Deny` into a hook deny
whose reason the model reads as the tool result. Rules never see
secrets and never prompt - they only say no, with a reason.
A rule that raises is a deny too: the boundary fails closed, the
traceback lands in the log.
Every tool call - allowed or denied - is reported to the audit sink the
backend was given, so the admin audit page shows what the model touched.
"""
from __future__ import annotations
import inspect
import json
import logging
from collections.abc import Awaitable, Callable
from dataclasses import dataclass, field
from pathlib import Path
from typing import TYPE_CHECKING, Any, Literal
if TYPE_CHECKING:
from collections.abc import Iterable, Mapping, MutableMapping
__all__ = [
"AuditSink",
"Decision",
"Deny",
"PolicyRule",
"ToolAudit",
"ToolCall",
"brief",
"evaluate",
"hook_output",
]
_log = logging.getLogger("beaver_gateway.policy")
PATH_KEYS: tuple[str, ...] = ("file_path", "notebook_path", "path")
"""Input keys built-in file tools use for their target."""
Decision = Literal["allow", "deny"]
@dataclass(frozen=True, slots=True)
class Deny:
"""Refusal with the reason the model reads as the tool result."""
reason: str
@dataclass(frozen=True, slots=True)
class ToolCall:
"""One tool invocation as the hook sees it.
``state`` is a mutable per-session dict rules may write to (which
skills were opened, what was already asked); it lives as long as the
claude process and is empty again after a resume.
"""
tool: str
input: Mapping[str, Any]
agent: str
kind: str
conversation: str
cwd: Path
state: MutableMapping[str, Any] = field(default_factory=dict)
@property
def mcp(self) -> tuple[str, str] | None:
"""``(server, tool)`` for ``mcp__<server>__<tool>`` names, else ``None``."""
if not self.tool.startswith("mcp__"):
return None
_, _, rest = self.tool.partition("__")
server, sep, name = rest.partition("__")
return (server, name) if sep else (server, "")
def path(self, *keys: str) -> Path | None:
"""Target of a file tool, absolute; relative paths resolve against ``cwd``."""
for key in keys or PATH_KEYS:
raw = self.input.get(key)
if isinstance(raw, str) and raw:
return self.resolve(raw)
return None
def resolve(self, raw: str) -> Path:
path = Path(raw).expanduser()
if not path.is_absolute():
path = self.cwd / path
# ``resolve`` would follow symlinks on the gateway host, which may
# not be the model's view; normalise lexically instead.
return Path(*_normalize(path.parts))
def _normalize(parts: Iterable[str]) -> list[str]:
out: list[str] = []
for part in parts:
if part == "..":
if len(out) > 1:
out.pop()
elif part != ".":
out.append(part)
return out
type PolicyRule = Callable[[ToolCall], Deny | None | Awaitable[Deny | None]]
"""A rule; sync or async. First ``Deny`` wins, ``None`` passes to the next."""
@dataclass(frozen=True, slots=True)
class ToolAudit:
agent: str
conversation: str
kind: str
tool: str
decision: Decision
reason: str | None
brief: str
AuditSink = "Callable[[ToolAudit], Awaitable[None]]"
async def evaluate(
rules: Iterable[Callable[[ToolCall], Any]], call: ToolCall
) -> Deny | None:
"""Run ``rules`` in order; the first deny wins, an exception is a deny."""
for rule in rules:
try:
verdict = rule(call)
if inspect.isawaitable(verdict):
verdict = await verdict
except Exception: # noqa: BLE001 - a broken rule must fail closed
name = getattr(rule, "__name__", repr(rule))
_log.exception("policy rule %s failed on %s", name, call.tool)
return Deny(reason=f"policy rule {name} failed; the call is refused")
if isinstance(verdict, Deny):
return verdict
return None
def hook_output(deny: Deny | None) -> dict[str, Any]:
"""``PreToolUse`` hook JSON for a verdict: empty means allow."""
if deny is None:
return {}
return {
"hookSpecificOutput": {
"hookEventName": "PreToolUse",
"permissionDecision": "deny",
"permissionDecisionReason": deny.reason,
}
}
def brief(tool_input: Mapping[str, Any], *, limit: int = 240) -> str:
"""Short, log-safe summary of a call: the path, the command, or the args."""
for key in (*PATH_KEYS, "command", "skill", "pattern", "query", "url"):
value = tool_input.get(key)
if isinstance(value, str) and value:
return _cut(value, limit)
try:
return _cut(json.dumps(tool_input, ensure_ascii=False, sort_keys=True), limit)
except (TypeError, ValueError):
return _cut(repr(tool_input), limit)
def _cut(text: str, limit: int) -> str:
return text if len(text) <= limit else text[: limit - 1] + "…"
+42
View File
@@ -0,0 +1,42 @@
"""System prompt assembly from a list of source files.
The gateway holds no prompt text: an agent names its granules (paths from
``config.py``) and :func:`assemble` concatenates them in that order, so the
result is byte-for-byte identical for every session of the same agent as
long as the files are. A source is a path, or a ``(tag, path)`` pair whose
content is wrapped in ``<tag>...</tag>`` - the markup lives here, the vault
keeps plain markdown. Each granule's hash is logged at assembly so a
drifted prompt can be traced to the file that changed.
"""
from __future__ import annotations
import hashlib
import logging
from pathlib import Path
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from collections.abc import Iterable
__all__ = ["PromptSource", "assemble"]
_log = logging.getLogger("beaver_gateway.core.prompt")
PromptSource = str | Path | tuple[str, str | Path]
def assemble(sources: Iterable[PromptSource]) -> str:
parts: list[str] = []
for source in sources:
tag, raw = source if isinstance(source, tuple) else (None, source)
path = Path(raw)
text = path.read_text(encoding="utf-8").strip()
digest = hashlib.sha256(text.encode("utf-8")).hexdigest()[:12]
_log.info(
"prompt granule %s tag=%s sha=%s bytes=%d", path, tag, digest, len(text)
)
if not text:
continue
parts.append(f"<{tag}>\n{text}\n</{tag}>" if tag else text)
return "\n\n".join(parts) + "\n"
+286
View File
@@ -0,0 +1,286 @@
"""Recognise credentials in free text and mask them.
``docker logs`` is not a vault: the stack's stdout lands in an unrotated
json file on the host, and the model itself can page through it (the
komodo tool has ``logs`` / ``search_logs``). Nobody has to write a
careless log call for a secret to end up there — two habits do it on
their own:
* ``httpx`` logs ``HTTP Request: POST <full url>`` at ``INFO``, so every
request to an upstream that keeps its credential *in* the URL prints
the credential once per call;
* a transport error carries that same URL through the traceback, which
``_log.exception`` writes out in full.
Two of our upstreams are exactly that shape: a Google Calendar
``.../private-<token>/basic.ics`` feed and a USOS ``?key=<token>`` ical
feed, both handed to the calendar MCP as query parameters.
So: silence the HTTP clients' per-request chatter (nothing here reads
it), and run one redaction pass over every formatted record as a second
line of defence.
The same pass guards the wider channel — what an MCP tool hands back to
the model (see :mod:`beaver_gateway.mcp.redacting`). :func:`redact`
recognises three things: literal values of the secret-looking
environment variables this process was started with, credentials that
live in URLs and headers, and ``NAME: value`` assignments whose *name*
says the value is a credential. The last one is what covers a config
dump of a stack whose secrets this process never held — and it keeps
the name, masking only the value, so the dump still says what is set.
"""
from __future__ import annotations
import logging
import os
import re
from typing import TYPE_CHECKING
from urllib.parse import quote, unquote
if TYPE_CHECKING:
from collections.abc import Callable, Mapping
MASK = "<…>"
# Loggers that print request URLs at INFO. We keep their warnings.
CHATTY: tuple[str, ...] = ("httpx", "httpcore", "aiohttp.client", "urllib3")
# Loggers that bring their own handlers (uvicorn re-runs ``dictConfig``
# when a server starts), so wrapping the root formatter misses them.
# ``dictConfig`` drops a logger's handlers but keeps its filters.
OWN_HANDLERS: tuple[str, ...] = ("uvicorn", "uvicorn.access", "uvicorn.error")
# Names whose *value* is a credential — matched as a substring, so
# ``FIREFLY_PAT`` and ``T3_MAC_TOKEN`` both qualify. Used twice: to pick
# which environment variables contribute literal values, and to decide
# whether a ``NAME: value`` line in a config dump should keep its value.
# ``MCP`` is in here because every MCP URL in this stack carries its
# credential in the query string.
_SECRET_NAME = re.compile(
r"TOKEN|SECRET|KEY|PASSWORD|PASS\b|PAT\b|BEARER|CREDENTIAL"
r"|MCPS?\b|AUTH|PRIVATE|SESSION|DSN",
re.IGNORECASE,
)
_MIN_SECRET = 8
# A query parameter may sit behind a plain ``?``/``&`` or behind their
# percent-encoded twins when a whole URL is nested in another one.
_LEFT = r"(?:(?<![A-Za-z0-9])|(?<=%26)|(?<=%3F))"
# ...and runs until the next separator, encoded ``&`` included.
_VALUE = r"(?:(?!%26)[^&\s\"'<>,;)\]}])+"
# Names worth masking where they sit in a dump next to their password,
# but not worth hunting for as literals across every log line: a
# username is short and ordinary, and ``CLAUDE_RUNNER_USER=beaver-runner``
# turned into a global literal would mask half the entrypoint script.
_SECRET_IN_DUMP = re.compile(r"USER\b|LOGIN\b", re.IGNORECASE)
def is_secret_name(name: str) -> bool:
"""Whether a variable/field of this name holds a credential."""
return bool(_SECRET_NAME.search(name) or _SECRET_IN_DUMP.search(name))
def _mask_named_value(match: re.Match[str]) -> str:
"""Mask ``value`` in a ``NAME: value`` match, keeping the name.
A dump is still worth reading when it says *which* variables are
set; it is the values that must not travel. An already-masked or
empty value is left alone so the pass stays idempotent, and ``lead``
/ ``tail`` carry back whatever quoting the surrounding format used.
"""
groups = match.groupdict()
name, value = groups["name"], groups["value"]
if not is_secret_name(name) or value.strip(" \t\"'") in ("", MASK):
return match[0]
lead, tail = groups.get("lead") or "", groups.get("tail") or ""
return f"{lead}{name}{groups['sep']}{MASK}{tail}"
type _Repl = str | Callable[[re.Match[str]], str]
_RULES: tuple[tuple[re.Pattern[str], _Repl], ...] = (
# Google Calendar's secret path segment; the ``/basic.ics`` after it
# survives, encoded or not, so the line still says what it fetched.
(
re.compile(r"(private-)(?:(?!%2F)[A-Za-z0-9_%-]){8,}", re.IGNORECASE),
r"\1" + MASK,
),
# ``key=…``, ``api_key=…``, ``token=…``, … in a query string.
(
re.compile(
_LEFT
+ r"(api[-_]?key|access[-_]?token|auth[-_]?token"
+ r"|token|key|secret|password|passwd|pat|signature|sig)"
+ r"(=|%3D)"
+ _VALUE,
re.IGNORECASE,
),
r"\1\2" + MASK,
),
# ``Authorization: Bearer …`` in a header dump; scheme goes too.
(
re.compile(
r"(authorization[\"']?\s*[:=]\s*[\"']?)(?:\S+\s+)?\S+", re.IGNORECASE
),
r"\1" + MASK,
),
# A bare ``Bearer <token>`` anywhere else.
(re.compile(r"\b(bearer)\s+[A-Za-z0-9._~+/=-]{8,}", re.IGNORECASE), r"\1 " + MASK),
# ``NAME: value`` / ``NAME=value`` at the start of a line — the shape
# of a compose ``environment:`` block, an ``.env`` file, a printed
# settings object. Anchored to the line so that an ``actor=token:foo``
# in the middle of a log line doesn't swallow the rest of it, and
# ``=`` must be tight (``NAME=value``, the way env files write it) so
# that ``SESSION_SECRET = os.environ[…]`` in source the dispatcher is
# reading through t3code stays readable.
(
re.compile(
r"""(?m)^(?P<lead>[ \t]*(?:-[ \t]+)?["']?)
(?P<name>[A-Z][A-Z0-9_]{2,})
(?P<sep>["']?(?::[ \t]*|=))
(?P<value>"[^"\n]*"|'[^'\n]*'|[^\n]*)""",
re.VERBOSE,
),
_mask_named_value,
),
# The same at the start of a line in lower case, but only for names
# that are unambiguously a credential — ``_SECRET_NAME`` is too broad
# here, it would read the ``mcp:`` prefix of a log line as one.
(
re.compile(
r"""(?m)^(?P<lead>[ \t]*(?:-[ \t]+)?["']?)
(?P<name>password|passwd|secret|token|api[-_]?key|apikey
|access[-_]?token|auth[-_]?token|private[-_]?key|credentials?)
(?P<sep>["']?(?::[ \t]*|=))
(?P<value>"[^"\n]*"|'[^'\n]*'|[^\n]*)""",
re.VERBOSE | re.IGNORECASE,
),
_mask_named_value,
),
# ``scheme://user:password@host`` — the credential a connection
# string carries. Host and database stay, so the line still locates
# the service it failed to reach.
(re.compile(r"(://[^/\s:@]+:)[^@\s/]+(@)"), r"\1" + MASK + r"\2"),
# ``"name": "value"`` anywhere — the same idea for one-line JSON,
# where the quotes make the end of the value unambiguous.
(
re.compile(
r'(?P<lead>")(?P<name>[^"\n]{1,64})(?P<sep>"\s*:\s*")'
r'(?P<value>[^"\n]*)(?P<tail>")'
),
_mask_named_value,
),
)
_ENV_SECRETS: list[str] = []
"""Literal secret values to mask, longest first. Filled by :func:`install`."""
def env_secrets(env: Mapping[str, str]) -> list[str]:
"""Literal values worth masking, longest first.
Each secret-looking variable contributes its value plus the
percent-encoded and percent-decoded spellings of it, because a
credential that travels inside another URL is logged encoded.
"""
found: set[str] = set()
for name, value in env.items():
if not _SECRET_NAME.search(name):
continue
for form in (value, quote(value, safe=""), unquote(value)):
if len(form.strip()) >= _MIN_SECRET:
found.add(form.strip())
ordered = list(found)
ordered.sort(key=len, reverse=True)
return ordered
def redact(text: str) -> str:
"""Mask every credential we can recognise in ``text``."""
for value in _ENV_SECRETS:
if value in text:
text = text.replace(value, MASK)
for pattern, repl in _RULES:
text = pattern.sub(repl, text)
return text
def redact_data(value: object) -> object:
"""Mask every string inside a decoded JSON tree.
A field name is the only context a bare value has: ``"0d1e…"`` on
its own looks like nothing, ``{"ADMIN_PASS": "0d1e…"}`` is a
password. So a secret-looking key masks its whole subtree — the
structured twin of the ``NAME: value`` rule above.
"""
if isinstance(value, str):
return redact(value)
if isinstance(value, dict):
return {
key: MASK if is_secret_name(str(key)) and item else redact_data(item)
for key, item in value.items()
}
if isinstance(value, list):
return [redact_data(item) for item in value]
return value
class RedactingFormatter(logging.Formatter):
"""Wrap another formatter and redact whatever it produced.
Formatting first is the point: it catches the message, its ``%``
arguments and the traceback of an ``exception()`` call in one pass,
which a record-level filter cannot do.
"""
def __init__(self, inner: logging.Formatter | None = None) -> None:
super().__init__()
self._inner = inner if inner is not None else logging.Formatter()
def format(self, record: logging.LogRecord) -> str:
return redact(self._inner.format(record))
class RedactFilter(logging.Filter):
"""Redact at record level, for loggers whose handlers aren't ours.
Interpolates first and replaces the record with the result, because
a secret is often split across the template and its arguments
(``"key=%s", token``) and neither half looks like a credential on
its own. The arguments are dropped only when something was actually
masked, so untouched records stay structured.
"""
def filter(self, record: logging.LogRecord) -> bool:
try:
message = record.getMessage()
except (TypeError, ValueError): # a broken template is not ours to fix
return True
masked = redact(message)
if masked != message:
record.msg = masked
record.args = ()
return True
def load_secrets(env: Mapping[str, str] | None = None) -> None:
"""(Re)read the literal values to mask.
Called once at startup and again after ``.env`` is loaded, because
in dev the process environment only fills up at that point.
"""
_ENV_SECRETS[:] = env_secrets(os.environ if env is None else env)
def install(env: Mapping[str, str] | None = None) -> None:
"""Wire redaction into the root logger. Call right after ``basicConfig``."""
load_secrets(env)
for handler in logging.getLogger().handlers:
handler.setFormatter(RedactingFormatter(handler.formatter))
for name in CHATTY:
logging.getLogger(name).setLevel(logging.WARNING)
for name in OWN_HANDLERS:
logging.getLogger(name).addFilter(RedactFilter())
+35 -1
View File
@@ -13,9 +13,15 @@ from dataclasses import dataclass, field
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
if TYPE_CHECKING: if TYPE_CHECKING:
from collections.abc import Iterable, Iterator from collections.abc import Awaitable, Callable, Iterable, Iterator
from beaver_gateway.agents.base import BaseAgent from beaver_gateway.agents.base import BaseAgent
from beaver_gateway.core.conversations import ConversationTexts, UserSaid
from beaver_gateway.core.distill import Distiller
from beaver_gateway.core.envelope import RecallContext
from beaver_gateway.core.rotation import RotationPolicy
from beaver_gateway.core.scheduler import Budget, Job
from beaver_gateway.core.watch import VaultWatch
from beaver_gateway.frontends.base import Frontend from beaver_gateway.frontends.base import Frontend
from beaver_gateway.mcp.types import McpServerT from beaver_gateway.mcp.types import McpServerT
@@ -81,3 +87,31 @@ class Gateway:
agents: list[BaseAgent] = field(default_factory=list) agents: list[BaseAgent] = field(default_factory=list)
mcps: list[McpServerT] = field(default_factory=list) mcps: list[McpServerT] = field(default_factory=list)
frontends: list[Frontend] = field(default_factory=list) frontends: list[Frontend] = field(default_factory=list)
texts: ConversationTexts | None = None
"""Merge prompt and seed bodies for ``core/conversations`` (§8.2-8.3)."""
jobs: list[Job] = field(default_factory=list)
"""Cron / webhook / event jobs for ``core/scheduler`` (§3.6, §4.5)."""
rotation: RotationPolicy | None = None
"""When a master is rotated (§4.5); ``None`` keeps the defaults."""
watch: VaultWatch | None = None
"""Vault watcher feeding the envelope (§3.5, §4.6); ``None`` = no vault block."""
recall: Callable[[RecallContext], str | None] | None = None
"""Envelope lookup on the user's text: pointers into the vault (cards,
the agent's notes, due dates) the gateway knows no paths for (§3.3)."""
user_sink: Callable[[UserSaid], Awaitable[None] | None] | None = None
"""Sees every user message as it enters a master or branch turn - the
setup's own grep-able log of what the user said, outside the transcript."""
budget: Budget | None = None
"""Subscription window past which non-critical jobs wait (§4.5)."""
distiller: Distiller | None = None
"""Who closes deep chats and where the digests and the index live (§8.4)."""
tz: str = "UTC"
"""Local zone for the envelope clock and the rotation hour."""
host: str = "0.0.0.0" # noqa: S104
port: int = 8000
"""The one listener; every HTTP frontend is mounted under its ``path``."""
public_url: str | None = None
"""Origin the reverse proxy shows the world (``https://b.example.com``).
Advertised endpoints and MCP discovery are built on it; ``None``
derives the origin from each request."""
+131
View File
@@ -0,0 +1,131 @@
"""Master rotation (§4.5, §8.1, §8.3).
One logical master thread, many physical sessions: when the policy says
so, a new master is spawned and takes over the window atomically, the old
one writes its handout as its last turn, closes, its finished branches get
marked in their windows, its queued normal injects move over, and the new
one receives "new day". Silence is measured by the user's messages only -
injects never extend a day.
"""
from __future__ import annotations
import logging
from dataclasses import dataclass
from datetime import UTC, date, datetime, timedelta
from typing import TYPE_CHECKING
from zoneinfo import ZoneInfo
if TYPE_CHECKING:
from beaver_gateway.core.conversations import Conversations
from beaver_gateway.storage.models import Conversation
__all__ = ["HandoutContext", "Rotation", "RotationPolicy"]
_log = logging.getLogger("beaver_gateway.core.rotation")
@dataclass(frozen=True, slots=True)
class RotationPolicy:
tz: str = "UTC"
hour: int = 4
night_silence: timedelta = timedelta(hours=3)
max_age: timedelta = timedelta(hours=36)
short_silence: timedelta = timedelta(minutes=30)
max_context_tokens: int = 80_000
def reason(
self, master: Conversation, *, now: datetime, context_tokens: int
) -> str | None:
zone = ZoneInfo(self.tz)
created = _aware(master.created_at)
silence = now - _aware(master.last_user_activity_at or master.created_at)
boundary = now.astimezone(zone).replace(
hour=self.hour, minute=0, second=0, microsecond=0
)
if now.astimezone(zone) < boundary:
boundary -= timedelta(days=1)
if created < boundary and silence > self.night_silence:
return "ночь"
if now - created > self.max_age and silence > self.short_silence:
return "возраст"
if context_tokens > self.max_context_tokens and silence > self.short_silence:
return "транскрипт"
return None
def day_of(self, master: Conversation) -> date:
return _aware(master.created_at).astimezone(ZoneInfo(self.tz)).date()
@dataclass(frozen=True, slots=True)
class HandoutContext:
day: date
master: Conversation
reason: str
class Rotation:
def __init__(
self, conversations: Conversations, policy: RotationPolicy | None = None
) -> None:
self._conversations = conversations
self.policy = policy or RotationPolicy()
async def due(self, now: datetime | None = None) -> list[tuple[Conversation, str]]:
now = now or datetime.now(UTC)
out: list[tuple[Conversation, str]] = []
for master in await self._conversations.find(kind="master", status="open"):
tokens = await self._conversations.context_tokens(master)
reason = self.policy.reason(master, now=now, context_tokens=tokens)
if reason is not None:
out.append((master, reason))
return out
async def tick(self, now: datetime | None = None) -> list[Conversation]:
rotated: list[Conversation] = []
for master, reason in await self.due(now):
new = await self.rotate(master, reason)
if new is not None:
rotated.append(new)
return rotated
async def rotate(self, old: Conversation, reason: str) -> Conversation | None:
conversations = self._conversations
if await conversations.busy(old):
_log.info("rotation of %s skipped: busy", old.external_id)
return None
new = await conversations.spawn(
kind="master", agent=old.agent_name, seed="morning", origin="rotation"
)
day = self.policy.day_of(old)
_log.info(
"rotation (%s): %s -> %s, handout for %s",
reason,
old.external_id,
new.external_id,
day,
)
await conversations.handout(
old, HandoutContext(day=day, master=old, reason=reason)
)
await conversations.close(old)
for branch in await conversations.find(parent=old, limit=1000):
if branch.status == "open":
await conversations.reparent(branch, new)
elif not branch.running_turn:
await conversations.mark_closed(branch)
moved = await conversations.queue.move(old, new, priority="normal")
await conversations.new_day(new, reason=reason, moved=moved)
conversations.bus.publish(
"conversation.rotated",
conversation_id=new.external_id,
closed=old.external_id,
reason=reason,
handout_day=day.isoformat(),
moved=moved,
)
return new
def _aware(value: datetime) -> datetime:
return value if value.tzinfo is not None else value.replace(tzinfo=UTC)
+526
View File
@@ -0,0 +1,526 @@
"""Jobs and deferred injects on pgqueuer (§3.6, §4.5).
A job is a name, a handler and its triggers: a cron expression, the
webhook ``/hooks/<name>``, gateway bus events. The executor is pgqueuer on
the gateway's own Postgres (a dedicated autocommit connection for
LISTEN/NOTIFY), so cron ticks, webhook deliveries and the ``schedule``
tool's one-off injects all live in one table and survive a restart. A
handler only queues work for a conversation and returns; the turn itself
runs in the conversation's worker. Non-critical jobs step aside while the
subscription window is past its threshold.
"""
from __future__ import annotations
import asyncio
import contextlib
import json
import logging
from dataclasses import dataclass
from datetime import UTC, datetime, timedelta
from typing import TYPE_CHECKING, Any, cast
from pgqueuer import PgQueuer, Queries
from pgqueuer.models import JobId
from starlette.applications import Starlette
from starlette.responses import JSONResponse
from starlette.routing import Route
from beaver_gateway.core.conversations import parse_at
if TYPE_CHECKING:
from collections.abc import Awaitable, Callable, Coroutine, Sequence
from pgqueuer.models import Job as PgJob
from pgqueuer.models import Schedule as PgSchedule
from pgqueuer.ports.driver import Driver
from starlette.requests import Request
from beaver_gateway.core.conversations import Conversations, DistillResult
from beaver_gateway.core.distill import LineCap
from beaver_gateway.core.injects import Priority
from beaver_gateway.core.rotation import Rotation
from beaver_gateway.storage.models import Conversation
__all__ = ["INJECT", "Budget", "Job", "JobRun", "Scheduler"]
_log = logging.getLogger("beaver_gateway.core.scheduler")
INJECT = "inject"
RETRY = timedelta(minutes=15)
@dataclass(frozen=True, slots=True)
class Job:
name: str
run: Callable[[JobRun], Awaitable[None]]
cron: str | None = None
webhook: bool = False
events: tuple[str, ...] = ()
critical: bool = True
dedupe: bool = True
@property
def entrypoint(self) -> str:
return f"job:{self.name}"
@dataclass(frozen=True, slots=True)
class Budget:
threshold: float = 0.7
tokens: int | None = None
window: timedelta = timedelta(hours=5)
limit_window: str = "five_hour"
@dataclass(slots=True)
class JobRun:
job: Job
trigger: str
payload: dict[str, Any]
scheduler: Scheduler
@property
def conversations(self) -> Conversations:
return self.scheduler.conversations
async def master(self) -> Conversation | None:
masters = await self.conversations.find(kind="master", status="open", limit=1)
return masters[0] if masters else None
async def inject_master(
self, text: str, *, urgency: Priority = "normal", origin: str | None = None
) -> bool:
master = await self.master()
if master is None:
_log.error(
"job %s: no open master, inject lost: %s", self.job.name, text[:200]
)
return False
await self.conversations.inject(
master, text, urgency=urgency, origin=origin or self.job.name
)
return True
async def spawn_job(
self,
*,
agent: str,
text: str,
title: str | None = None,
line_cap: LineCap | None = None,
) -> Conversation:
"""A headless job turn; ``line_cap`` bounces a rewrite past the cap."""
return await self.conversations.spawn(
kind="job",
agent=agent,
seed="brief",
text=text,
title=title,
origin="job",
flags={"line_cap": line_cap.as_flags()} if line_cap else None,
)
async def close_idle(
self,
*,
kind: str = "deep",
days: int = 2,
limit: int = 3,
since: datetime | None = None,
) -> list[DistillResult]:
"""§4.5: close chats quiet for ``days``, at most ``limit`` per run."""
out: list[DistillResult] = []
for conv in await self.conversations.idle(
kind=kind, days=days, since=since, limit=limit
):
try:
out.append(
await self.conversations.distill(conv, reason=f"idle {days}d")
)
except Exception: # noqa: BLE001
_log.exception("closing idle %s failed", conv.external_id)
return out
async def retry_in(self, delay: timedelta) -> None:
await self.scheduler.trigger(
self.job, self.payload, delay=delay, trigger=self.trigger
)
async def rotate(self) -> list[Conversation]:
rotation = self.scheduler.rotation
return await rotation.tick() if rotation is not None else []
def background(self, coro: Coroutine[Any, Any, Any]) -> None:
self.scheduler.background(coro)
class Scheduler:
def __init__(
self,
*,
conversations: Conversations,
jobs: Sequence[Job] = (),
driver: Driver | None = None,
budget: Budget | None = None,
rotation: Rotation | None = None,
heartbeat: timedelta = timedelta(seconds=30),
) -> None:
self.conversations = conversations
self.rotation = rotation
self.budget = budget or Budget()
self._jobs = {job.name: job for job in jobs}
self._driver = driver
self._queries: Queries | None = None
self._heartbeat = heartbeat
self._tasks: set[asyncio.Task[Any]] = set()
self._runs: dict[str, dict[str, Any]] = {}
@property
def enabled(self) -> bool:
return self._queries is not None
@property
def jobs(self) -> list[Job]:
return list(self._jobs.values())
def job(self, name: str) -> Job | None:
return self._jobs.get(name)
# ---- lifecycle -----------------------------------------------------
async def start(self) -> None:
if self._driver is not None:
queries = Queries(self._driver)
if not await queries.has_table("pgqueuer"):
await queries.install()
pgq = PgQueuer(self._driver, queries=queries)
self._register(pgq)
self._queries = queries
self._spawn(pgq.run(heartbeat_timeout=self._heartbeat))
else:
_log.warning("scheduler: no postgres driver - cron and webhooks are off")
self._spawn(self._events())
async def stop(self) -> None:
for task in list(self._tasks):
task.cancel()
for task in list(self._tasks):
with contextlib.suppress(BaseException):
await task
self._tasks.clear()
self._queries = None
def _register(self, pgq: PgQueuer) -> None:
@pgq.entrypoint(INJECT)
async def deliver(job: PgJob) -> None:
await self._deliver(job)
for job in self._jobs.values():
self._register_job(pgq, job)
def _register_job(self, pgq: PgQueuer, job: Job) -> None:
@pgq.entrypoint(job.entrypoint)
async def queued(pg_job: PgJob) -> None:
data = _decode(pg_job.payload)
await self._dispatch(
job, trigger=str(data.pop("trigger", "queue")), payload=data
)
if job.cron:
@pgq.schedule(job.name, job.cron, clean_old=True)
async def cron(_schedule: PgSchedule) -> None:
await self._dispatch(job, trigger="cron", payload={})
async def _events(self) -> None:
listeners = [job for job in self._jobs.values() if job.events]
if not listeners:
return
async for event in self.conversations.bus.stream():
for job in listeners:
if event.get("type") in job.events:
self._spawn(
self._dispatch(job, trigger="event", payload=dict(event))
)
# ---- dispatch ------------------------------------------------------
async def _dispatch(
self, job: Job, *, trigger: str, payload: dict[str, Any]
) -> None:
bus = self.conversations.bus
if not job.critical and await self.throttled():
bus.publish("job.deferred", job=job.name, trigger=trigger)
if trigger != "cron":
await self.trigger(job, payload, delay=RETRY, trigger=trigger)
return
started = datetime.now(UTC)
bus.publish("job.start", job=job.name, trigger=trigger)
status = "done"
try:
await job.run(JobRun(job, trigger, payload, self))
except Exception: # noqa: BLE001
status = "failed"
_log.exception("job %s (%s) failed", job.name, trigger)
self._runs[job.name] = {
"trigger": trigger,
"started_at": started.isoformat(timespec="seconds"),
"status": status,
}
bus.publish("job.end", job=job.name, trigger=trigger, status=status)
async def trigger(
self,
job: Job,
payload: dict[str, Any] | None = None,
*,
delay: timedelta | None = None,
trigger: str = "manual",
) -> int | None:
"""Queue one run of ``job``; runs inline when there is no executor."""
data = {**(payload or {}), "trigger": trigger}
if self._queries is None:
self._spawn(
self._dispatch(job, trigger=trigger, payload=payload or {}), delay=delay
)
return None
ids = await self._queries.enqueue(
job.entrypoint, _encode(data), execute_after=delay
)
return int(ids[0]) if ids and ids[0] is not None else None
async def hook(self, name: str, payload: dict[str, Any]) -> int | None:
job = self._jobs.get(name)
if job is None or not job.webhook:
msg = f"no webhook job {name!r}"
raise LookupError(msg)
self.conversations.bus.publish("hook", job=name)
if self._queries is None:
return await self.trigger(job, payload, trigger="webhook")
ids = await self._queries.enqueue(
job.entrypoint,
_encode({**payload, "trigger": "webhook"}),
dedupe_key=f"hook:{name}" if job.dedupe else None,
on_conflict="skip" if job.dedupe else "raise",
)
return int(ids[0]) if ids and ids[0] is not None else None
# ---- deferred injects ----------------------------------------------
async def schedule(
self,
conv: Conversation,
at: str,
text: str,
*,
urgency: Priority = "wake",
dedupe_key: str | None = None,
) -> tuple[int | None, datetime]:
"""A deferred inject.
``wake`` by default: it was promised for a time, so it starts a turn
then instead of waiting for the normal window.
"""
if self._queries is None:
msg = "scheduler needs postgres; `schedule` is unavailable"
raise RuntimeError(msg)
when = parse_at(at)
delay = max(when - datetime.now(UTC), timedelta(0))
payload = _encode(
{
"conversation": conv.external_id,
"text": text,
"urgency": urgency,
"at": when.isoformat(timespec="seconds"),
}
)
ids = await self._queries.enqueue(
INJECT,
payload,
execute_after=delay,
dedupe_key=dedupe_key,
on_conflict="skip" if dedupe_key else "raise",
)
job_id = int(ids[0]) if ids and ids[0] is not None else None
self.conversations.bus.publish(
"schedule.created",
conversation_id=conv.external_id,
job=job_id,
execute_at=when.isoformat(timespec="seconds"),
)
return job_id, when
async def cancel(self, job_id: int) -> bool:
if self._queries is None:
return False
row = await self._queries.queue_job_by_id(JobId(job_id))
if row is None:
return False
await self._queries.mark_job_as_cancelled([JobId(job_id)])
self.conversations.bus.publish("schedule.cancelled", job=job_id)
return True
async def _deliver(self, job: PgJob) -> None:
data = _decode(job.payload)
key = str(data.get("conversation", ""))
conv = await self.conversations.resolve(key)
if conv is None:
_log.error(
"scheduled inject #%s lost: conversation %r is gone: %s",
job.id,
key,
str(data.get("text", ""))[:200],
)
return
await self.conversations.inject(
conv,
str(data.get("text", "")),
urgency=cast("Priority", data.get("urgency") or "wake"),
origin="schedule",
)
# ---- budget --------------------------------------------------------
async def utilization(self) -> float | None:
now = datetime.now(UTC)
values: list[float] = []
for row in await self.conversations.rate_limits(limit=100):
if row.window != self.budget.limit_window or row.utilization is None:
continue
fresh = now - _aware(row.ts) < self.budget.window
live = row.resets_at is None or _aware(row.resets_at) > now
if fresh and live:
values.append(float(row.utilization))
break
if self.budget.tokens:
tokens = await self.conversations.usage_tokens(now - self.budget.window)
values.append(tokens / self.budget.tokens)
return max(values) if values else None
async def throttled(self) -> bool:
utilization = await self.utilization()
return utilization is not None and utilization > self.budget.threshold
# ---- introspection -------------------------------------------------
async def snapshot(self) -> dict[str, Any]:
crons: dict[str, PgSchedule] = {}
queue: list[dict[str, Any]] = []
if self._queries is not None:
for row in await self._queries.peek_schedule():
crons[str(row.entrypoint)] = row
queue = [
_job_public(row) for row in await self._queries.browse_queue(limit=200)
]
utilization = await self.utilization()
return {
"enabled": self.enabled,
"utilization": utilization,
"throttled": utilization is not None
and utilization > self.budget.threshold,
"threshold": self.budget.threshold,
"jobs": [self._job_public(job, crons.get(job.name)) for job in self.jobs],
"queue": queue,
}
async def scheduled(self, conv: Conversation | None = None) -> list[dict[str, Any]]:
if self._queries is None:
return []
rows = await self._queries.browse_queue(limit=500, entrypoints=[INJECT])
out = [_job_public(row) for row in rows]
if conv is not None:
out = [
j for j in out if j["payload"].get("conversation") == conv.external_id
]
return out
def _job_public(self, job: Job, cron: PgSchedule | None) -> dict[str, Any]:
return {
"name": job.name,
"cron": job.cron,
"webhook": job.webhook,
"events": list(job.events),
"critical": job.critical,
"next_run": _iso(cron.next_run) if cron is not None else None,
"last_run": _iso(cron.last_run) if cron is not None else None,
"status": str(cron.status) if cron is not None else None,
"run": self._runs.get(job.name),
}
# ---- http ----------------------------------------------------------
def app(self, authorize: Callable[[Request], Awaitable[Any]]) -> Starlette:
async def hook(request: Request) -> JSONResponse:
await authorize(request)
name = request.path_params["name"]
payload = await _payload(request)
try:
job_id = await self.hook(name, payload)
except LookupError as exc:
return JSONResponse({"error": str(exc)}, status_code=404)
return JSONResponse({"job": job_id, "name": name}, status_code=202)
return Starlette(routes=[Route("/{name}", hook, methods=["POST"])])
# ---- internals -----------------------------------------------------
def background(self, coro: Coroutine[Any, Any, Any]) -> None:
self._spawn(coro)
def _spawn(
self, coro: Coroutine[Any, Any, Any], *, delay: timedelta | None = None
) -> None:
async def later() -> None:
if delay is not None:
await asyncio.sleep(delay.total_seconds())
await coro
task = asyncio.create_task(later() if delay is not None else coro)
self._tasks.add(task)
task.add_done_callback(self._tasks.discard)
async def _payload(request: Request) -> dict[str, Any]:
body = await request.body()
if not body:
return dict(request.query_params)
try:
data = json.loads(body)
except ValueError:
return {"raw": body.decode("utf-8", errors="replace")}
return data if isinstance(data, dict) else {"raw": data}
def _encode(data: dict[str, Any]) -> bytes:
return json.dumps(data, ensure_ascii=False, default=str).encode("utf-8")
def _decode(payload: bytes | None) -> dict[str, Any]:
if not payload:
return {}
try:
data = json.loads(payload)
except ValueError:
return {"raw": payload.decode("utf-8", errors="replace")}
return data if isinstance(data, dict) else {"raw": data}
def _job_public(row: PgJob) -> dict[str, Any]:
return {
"id": int(row.id),
"entrypoint": row.entrypoint,
"status": str(row.status),
"execute_after": _iso(row.execute_after),
"created": _iso(row.created),
"attempts": row.attempts,
"payload": _decode(row.payload),
}
def _aware(value: datetime) -> datetime:
return value if value.tzinfo is not None else value.replace(tzinfo=UTC)
def _iso(value: datetime | None) -> str | None:
return _aware(value).isoformat(timespec="seconds") if value is not None else None
+259
View File
@@ -0,0 +1,259 @@
"""Pool of live Agent SDK sessions across every Claude agent (§3.2).
One :class:`Session` is one ``ClaudeSDKClient`` (one claude subprocess).
The pool owns the two decisions the adapters used to make on their own:
when a session is closed for idleness (TTL by conversation kind) and
which one goes when memory runs out (measured RSS of the subprocess tree
against the cgroup limit, ``max_live`` where there is no limit). Eviction
only ever picks ``idle && !running_turn && !pending_question`` sessions
that are neither pinned (the master) nor ``dirty`` (mirror gap not yet
repaired); forks and jobs go first.
"""
from __future__ import annotations
import asyncio
import contextlib
import logging
import os
import time
from dataclasses import dataclass, field
from pathlib import Path
from typing import TYPE_CHECKING, Any, Protocol
import psutil
if TYPE_CHECKING:
from collections.abc import AsyncIterator, Iterator, Mapping
__all__ = ["DEFAULT_TTL", "Session", "SessionClient", "SessionPool", "cgroup_limit"]
_log = logging.getLogger("beaver_gateway.core.sessions")
DEFAULT_TTL: Mapping[str, float | None] = {
"master": None,
"branch": 7200.0,
"deep": 1800.0,
"job": 0.0,
"fork": 0.0,
}
"""Idle seconds before a session is closed; ``None`` = never (pinned kinds)."""
_EVICT_ORDER = {"fork": 0, "job": 0, "deep": 1, "branch": 2, "master": 3}
_RSS_HEADROOM = 0.8
class SessionClient(Protocol):
async def connect(self) -> None: ...
async def query(self, prompt: str) -> None: ...
def receive_response(self) -> AsyncIterator[Any]: ...
async def interrupt(self) -> None: ...
async def disconnect(self) -> None: ...
@dataclass
class Session:
key: str
agent: str
kind: str
client: SessionClient
session_id: str | None
resumed: bool
pinned: bool = False
dirty: bool = False
running_turn: str | None = None
pending_question: bool = False
interrupt_requested: bool = False
lock: asyncio.Lock = field(default_factory=asyncio.Lock)
last_used: float = field(default_factory=time.monotonic)
created_at: float = field(default_factory=time.monotonic)
turns: int = 0
state: dict[str, Any] = field(default_factory=dict)
"""Scratch for policy rules (``core/policy``); dies with the process."""
@property
def busy(self) -> bool:
return self.lock.locked() or self.running_turn is not None
@property
def evictable(self) -> bool:
return not (self.pinned or self.dirty or self.busy or self.pending_question)
@property
def pid(self) -> int | None:
transport = getattr(self.client, "_transport", None)
process = getattr(transport, "_process", None)
pid = getattr(process, "pid", None)
return pid if isinstance(pid, int) else None
class SessionPool:
def __init__(
self,
*,
rss_limit: int | None = None,
max_live: int = 8,
ttl: Mapping[str, float | None] = DEFAULT_TTL,
reap_interval: float = 60.0,
) -> None:
self._sessions: dict[str, Session] = {}
self._rss_limit = rss_limit if rss_limit is not None else cgroup_limit()
self._max_live = max_live
self._ttl = dict(ttl)
self._reap_interval = reap_interval
def __len__(self) -> int:
return len(self._sessions)
def __iter__(self) -> Iterator[Session]:
return iter(list(self._sessions.values()))
def __contains__(self, key: object) -> bool:
return key in self._sessions
@property
def rss_limit(self) -> int | None:
return self._rss_limit
def get(self, key: str) -> Session | None:
return self._sessions.get(key)
def add(self, session: Session) -> Session:
self._sessions[session.key] = session
return session
def pop(self, key: str) -> Session | None:
return self._sessions.pop(key, None)
def rekey(self, old: str, new: str) -> Session | None:
session = self._sessions.pop(old, None)
if session is None:
return None
stale = self._sessions.pop(new, None)
session.key = new
self._sessions[new] = session
return stale if stale is not session else None
def ttl_for(self, kind: str) -> float | None:
return self._ttl.get(kind, self._ttl.get("deep"))
def rss(self) -> int:
try:
children = psutil.Process(os.getpid()).children(recursive=True)
except psutil.Error:
return 0
total = 0
for child in children:
with contextlib.suppress(psutil.Error):
total += child.memory_info().rss
return total
@staticmethod
def rss_of(session: Session) -> int | None:
pid = session.pid
if pid is None:
return None
try:
process = psutil.Process(pid)
return process.memory_info().rss + sum(
c.memory_info().rss for c in process.children(recursive=True)
)
except psutil.Error:
return None
def over_limit(self) -> bool:
if self._rss_limit is not None:
return self.rss() > self._rss_limit * _RSS_HEADROOM
return len(self._sessions) >= self._max_live
def victims(self) -> list[Session]:
candidates = [s for s in self._sessions.values() if s.evictable]
candidates.sort(key=lambda s: (_EVICT_ORDER.get(s.kind, 1), s.last_used))
return candidates
async def make_room(self) -> int:
closed = 0
while self.over_limit():
victims = self.victims()
if not victims:
_log.warning(
"session pool over limit (%d live, rss=%d) but nothing evictable",
len(self._sessions),
self.rss(),
)
break
await self.close(victims[0].key)
closed += 1
return closed
async def close(self, key: str) -> None:
session = self._sessions.pop(key, None)
if session is None:
return
_log.info("closing session %s (%s, %s)", session.session_id, session.kind, key)
try:
await session.client.disconnect()
except Exception: # noqa: BLE001
_log.exception("disconnect failed for session %s", session.session_id)
async def close_all(self, *, agent: str | None = None) -> None:
for session in list(self._sessions.values()):
if agent is None or session.agent == agent:
await self.close(session.key)
async def reap_once(self) -> int:
now = time.monotonic()
closed = 0
for session in list(self._sessions.values()):
ttl = self.ttl_for(session.kind)
if ttl is None or not session.evictable:
continue
if now - session.last_used > ttl:
await self.close(session.key)
closed += 1
return closed
async def reap_loop(self) -> None:
while True:
await asyncio.sleep(self._reap_interval)
try:
await self.reap_once()
await self.make_room()
except Exception: # noqa: BLE001
_log.exception("session reaper failed")
def snapshot(self) -> list[dict[str, Any]]:
now = time.monotonic()
return [
{
"key": s.key,
"agent": s.agent,
"kind": s.kind,
"session_id": s.session_id,
"pid": s.pid,
"rss": self.rss_of(s),
"idle_seconds": round(now - s.last_used, 1),
"age_seconds": round(now - s.created_at, 1),
"turns": s.turns,
"busy": s.busy,
"running_turn": s.running_turn,
"pending_question": s.pending_question,
"pinned": s.pinned,
"dirty": s.dirty,
}
for s in self._sessions.values()
]
def cgroup_limit() -> int | None:
for path in (
"/sys/fs/cgroup/memory.max",
"/sys/fs/cgroup/memory/memory.limit_in_bytes",
):
try:
raw = Path(path).read_text(encoding="ascii").strip()
except OSError:
continue
if raw.isdigit() and int(raw) < 1 << 60:
return int(raw)
return None
+543
View File
@@ -0,0 +1,543 @@
"""Anthropic messages <-> Agent SDK transcript entries.
:func:`build_entries` renders a message list as the entries the Claude CLI
itself writes (reference: ``t/spike_sdk/entries_reference.json``, CLI
2.1.248 via ``import_session_to_store``): one ``user`` entry per prompt,
one ``assistant`` entry per content block sharing a message id, one
``user`` entry per ``tool_result`` parented on the matching ``tool_use``
entry. Only ``user``/``assistant`` entries are produced - no attachments,
titles or queue markers. Appending the result to a session store and
resuming that session id seeds an external history into the SDK.
:func:`messages_from_entries` is the projection back, used by tests and by
anything that needs Anthropic-shape history out of a mirrored transcript.
"""
from __future__ import annotations
import hashlib
import uuid as _uuid
from collections.abc import Mapping
from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any
try:
from claude_agent_sdk._cli_version import __cli_version__ as _cli_version
except ImportError: # pragma: no cover
_cli_version = "2.1.248"
if TYPE_CHECKING:
from collections.abc import Iterable
__all__ = [
"CLI_VERSION",
"build_entries",
"close_open_tool_uses",
"fingerprint",
"messages_from_entries",
"open_tool_uses",
"prompt_count",
"render_messages",
"strip_tool_entries",
"text_of",
"window_entries",
]
CLI_VERSION = _cli_version
_ENTRYPOINT = "sdk-py"
_USER_TYPE = "external"
_PROMPT_SOURCE = "sdk"
_GIT_BRANCH = "HEAD"
def build_entries(
messages: Iterable[Mapping[str, Any]],
*,
session_id: str,
cwd: str,
model: str,
permission_mode: str = "bypassPermissions",
now: datetime | None = None,
) -> list[dict[str, Any]]:
stamp = (now or datetime.now(UTC)).strftime("%Y-%m-%dT%H:%M:%S.") + (
f"{(now or datetime.now(UTC)).microsecond // 1000:03d}Z"
)
common = {
"isSidechain": False,
"userType": _USER_TYPE,
"entrypoint": _ENTRYPOINT,
"cwd": cwd,
"sessionId": session_id,
"version": CLI_VERSION,
"gitBranch": _GIT_BRANCH,
}
entries: list[dict[str, Any]] = []
parent: str | None = None
prompt_id: str | None = None
tool_use_owner: dict[str, str] = {}
for message in messages:
role = message.get("role")
content = message.get("content")
if role == "user":
results = _tool_results(content)
if results:
for block in results:
owner = tool_use_owner.get(
str(block.get("tool_use_id", "")), parent
)
uid = _new_uuid()
entries.append(
{
"parentUuid": owner,
"promptId": prompt_id,
"type": "user",
"message": {"role": "user", "content": [block]},
"uuid": uid,
"timestamp": stamp,
"toolUseResult": _result_text(block),
"sourceToolAssistantUUID": owner,
**common,
}
)
parent = uid
continue
prompt_id = _new_uuid()
uid = _new_uuid()
entries.append(
{
"parentUuid": parent,
"promptId": prompt_id,
"type": "user",
"message": {"role": "user", "content": _user_content(content)},
"uuid": uid,
"timestamp": stamp,
"permissionMode": permission_mode,
"promptSource": _PROMPT_SOURCE,
**common,
}
)
parent = uid
elif role == "assistant":
blocks = _assistant_blocks(content)
message_id = f"msg_{_uuid.uuid4().hex[:24]}"
request_id = f"req_{_uuid.uuid4().hex[:24]}"
stop_reason = (
"tool_use"
if any(b.get("type") == "tool_use" for b in blocks)
else "end_turn"
)
for block in blocks:
uid = _new_uuid()
entries.append(
{
"parentUuid": parent,
"message": {
"model": model,
"id": message_id,
"type": "message",
"role": "assistant",
"content": [block],
"stop_reason": stop_reason,
"stop_sequence": None,
"stop_details": None,
"usage": _zero_usage(),
"diagnostics": None,
},
"requestId": request_id,
"type": "assistant",
"uuid": uid,
"timestamp": stamp,
**common,
}
)
if block.get("type") == "tool_use" and block.get("id"):
tool_use_owner[str(block["id"])] = uid
parent = uid
else:
msg = f"message role must be user or assistant, got {role!r}"
raise ValueError(msg)
return entries
def messages_from_entries(entries: Iterable[Mapping[str, Any]]) -> list[dict[str, Any]]:
out: list[dict[str, Any]] = []
last_message_id: str | None = None
for entry in entries:
kind = entry.get("type")
message = entry.get("message")
if kind not in ("user", "assistant") or not isinstance(message, dict):
continue
content = message.get("content")
if kind == "user":
results = _tool_results(content)
if (
results
and out
and out[-1]["role"] == "user"
and _tool_results(out[-1]["content"])
):
out[-1]["content"].extend(results)
else:
out.append({"role": "user", "content": _user_content(content)})
last_message_id = None
continue
blocks = _assistant_blocks(content)
message_id = message.get("id")
if (
out
and out[-1]["role"] == "assistant"
and message_id is not None
and message_id == last_message_id
):
out[-1]["content"].extend(blocks)
else:
out.append({"role": "assistant", "content": blocks})
last_message_id = message_id
return out
def _new_uuid() -> str:
return str(_uuid.uuid4())
def _tool_results(content: Any) -> list[dict[str, Any]]:
if not isinstance(content, list):
return []
return [
_clean_block(b)
for b in content
if isinstance(b, dict) and b.get("type") == "tool_result"
]
def _user_content(content: Any) -> Any:
if isinstance(content, str):
return content
if isinstance(content, list):
return [_clean_block(b) for b in content if isinstance(b, dict)]
msg = f"user content must be str or list, got {type(content).__name__}"
raise TypeError(msg)
def _assistant_blocks(content: Any) -> list[dict[str, Any]]:
if isinstance(content, str):
return [{"type": "text", "text": content}]
if isinstance(content, list):
blocks = [_clean_block(b) for b in content if isinstance(b, dict)]
if blocks:
return blocks
msg = "assistant content must be a non-empty list of blocks"
raise ValueError(msg)
def _clean_block(block: dict[str, Any]) -> dict[str, Any]:
kind = block.get("type")
if kind == "tool_use":
return {
"type": "tool_use",
"id": block.get("id", ""),
"name": block.get("name", ""),
"input": block.get("input", {}),
}
if kind == "tool_result":
out: dict[str, Any] = {
"type": "tool_result",
"tool_use_id": block.get("tool_use_id", ""),
"content": block.get("content", ""),
}
if block.get("is_error") is not None:
out["is_error"] = bool(block["is_error"])
return out
if kind == "thinking":
return {
"type": "thinking",
"thinking": block.get("thinking", ""),
"signature": block.get("signature", ""),
}
if kind == "text":
return {"type": "text", "text": str(block.get("text", ""))}
return dict(block)
def _result_text(block: dict[str, Any]) -> str:
content = block.get("content", "")
if isinstance(content, str):
return content
if isinstance(content, list):
return "\n".join(
str(b.get("text", ""))
for b in content
if isinstance(b, dict) and b.get("type") == "text"
)
return str(content)
def _zero_usage() -> dict[str, Any]:
return {
"input_tokens": 0,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 0,
"service_tier": "standard",
"server_tool_use": {"web_search_requests": 0, "web_fetch_requests": 0},
"cache_creation": {
"ephemeral_1h_input_tokens": 0,
"ephemeral_5m_input_tokens": 0,
},
"inference_geo": "not_available",
"iterations": [],
"speed": "standard",
}
# ---- repair, windows, projections ---------------------------------------
_PROMPT_TYPES = ("user", "assistant")
_INTERRUPTED = "прервано"
def open_tool_uses(
entries: Iterable[Mapping[str, Any]],
) -> list[tuple[Mapping[str, Any], dict[str, Any]]]:
"""``(assistant entry, tool_use block)`` pairs that never got a ``tool_result``."""
closed: set[str] = set()
uses: list[tuple[Mapping[str, Any], dict[str, Any]]] = []
for entry in entries:
content = _entry_content(entry)
if entry.get("type") == "user":
closed.update(
str(b.get("tool_use_id", ""))
for b in content
if b.get("type") == "tool_result"
)
elif entry.get("type") == "assistant":
uses.extend((entry, b) for b in content if b.get("type") == "tool_use")
return [
(owner, block)
for owner, block in uses
if str(block.get("id", "")) not in closed
]
def close_open_tool_uses(
entries: list[Mapping[str, Any]], *, text: str = _INTERRUPTED
) -> list[dict[str, Any]]:
"""Synthetic ``tool_result`` entries on the leaf, one per open ``tool_use``.
Appending the result to the session store gives the next ``resume`` a
transcript the CLI accepts: an assistant message ending in ``tool_use``
without its result is rejected by the API on the next call.
"""
pending = open_tool_uses(entries)
if not pending:
return []
leaf = next(
(
e
for e in reversed(entries)
if e.get("type") in _PROMPT_TYPES and e.get("uuid")
),
None,
)
parent = str(leaf["uuid"]) if leaf is not None else None
stamp = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S.000Z")
out: list[dict[str, Any]] = []
for owner, block in pending:
uid = _new_uuid()
result = {
"type": "tool_result",
"tool_use_id": block.get("id", ""),
"content": text,
"is_error": True,
}
out.append(
{
"parentUuid": parent,
"promptId": owner.get("promptId"),
"type": "user",
"message": {"role": "user", "content": [result]},
"uuid": uid,
"timestamp": stamp,
"toolUseResult": text,
"sourceToolAssistantUUID": owner.get("uuid"),
**{k: owner[k] for k in _COMMON_KEYS if k in owner},
}
)
parent = uid
return out
_COMMON_KEYS = (
"isSidechain",
"userType",
"entrypoint",
"cwd",
"sessionId",
"version",
"gitBranch",
)
def window_entries(
entries: Iterable[Mapping[str, Any]], *, window: int | None = None
) -> list[dict[str, Any]]:
"""``user``/``assistant`` entries only, last ``window`` prompts, relinked."""
kept = [
dict(e)
for e in entries
if e.get("type") in _PROMPT_TYPES and isinstance(e.get("uuid"), str)
]
if window is not None and window > 0:
starts = [i for i, e in enumerate(kept) if _is_prompt_entry(e)]
if len(starts) > window:
kept = kept[starts[-window] :]
return _relink(kept)
def strip_tool_entries(entries: Iterable[Mapping[str, Any]]) -> list[dict[str, Any]]:
"""Drop ``tool_result`` entries and ``tool_use``/``thinking`` blocks; relink."""
out: list[dict[str, Any]] = []
for raw in entries:
entry = dict(raw)
message = entry.get("message")
if not isinstance(message, dict):
out.append(entry)
continue
content = message.get("content")
if isinstance(content, list):
blocks = [
b
for b in content
if isinstance(b, dict)
and b.get("type") not in ("tool_use", "tool_result", "thinking")
]
if not blocks:
continue
entry["message"] = {**message, "content": blocks}
out.append(entry)
return _relink(out)
def prompt_count(entries: Iterable[Mapping[str, Any]]) -> int:
return sum(1 for e in entries if _is_prompt_entry(e))
def render_messages(
messages: Iterable[Mapping[str, Any]], *, window: int | None = None
) -> str:
"""Plain-text projection for ``read_conversation``: ``user:``/``assistant:`` turns.
Tool calls collapse to a one-line summary per assistant turn; tool
results and thinking are dropped.
"""
turns: list[str] = []
tools: list[str] = []
current: list[str] = []
def flush() -> None:
if not current and not tools:
return
body = "\n\n".join(current).strip()
if tools:
body = (body + "\n" if body else "") + "(tools: " + ", ".join(tools) + ")"
turns.append("assistant:\n" + body)
current.clear()
tools.clear()
for message in messages:
role = message.get("role")
content = message.get("content")
if role == "user":
if _tool_results(content):
continue
flush()
turns.append("user:\n" + _text_of_content(content))
continue
for block in _assistant_blocks(content):
if block.get("type") == "text" and block.get("text"):
current.append(str(block["text"]))
elif block.get("type") == "tool_use":
tools.append(str(block.get("name", "")))
flush()
if window is not None and window > 0:
starts = [i for i, t in enumerate(turns) if t.startswith("user:")]
if len(starts) > window:
turns = turns[starts[-window] :]
return "\n\n".join(turns)
def _entry_content(entry: Mapping[str, Any]) -> list[dict[str, Any]]:
message = entry.get("message")
if not isinstance(message, dict):
return []
content = message.get("content")
if not isinstance(content, list):
return []
return [b for b in content if isinstance(b, dict)]
def _is_prompt_entry(entry: Mapping[str, Any]) -> bool:
if entry.get("type") != "user":
return False
message = entry.get("message")
if not isinstance(message, dict):
return False
content = message.get("content")
if isinstance(content, str):
return True
return not _tool_results(content)
def _relink(entries: list[dict[str, Any]]) -> list[dict[str, Any]]:
parent: str | None = None
for entry in entries:
entry["parentUuid"] = parent
parent = entry.get("uuid")
return entries
def _text_of_content(content: Any) -> str:
if isinstance(content, str):
return content
if isinstance(content, list):
return "\n\n".join(
str(b.get("text", ""))
for b in content
if isinstance(b, dict) and b.get("type") == "text" and b.get("text")
)
return ""
def fingerprint(messages: Iterable[Mapping[str, Any]]) -> str:
"""Text-only hash of a history; stateless callers are keyed by it."""
turns: list[tuple[str, str]] = []
for message in messages:
text = text_of(message.get("content"))
if not text:
continue
role = str(message.get("role", ""))
if turns and turns[-1][0] == role:
turns[-1] = (role, turns[-1][1] + "\n" + text)
else:
turns.append((role, text))
digest = hashlib.sha1(usedforsecurity=False)
for role, text in turns:
digest.update(role.encode("utf-8"))
digest.update(b"\x00")
digest.update(text.strip().encode("utf-8"))
digest.update(b"\x01")
return digest.hexdigest()
def text_of(content: Any) -> str:
if isinstance(content, str):
return content
if isinstance(content, list):
parts = [
str(b.get("text", ""))
for b in content
if isinstance(b, Mapping) and b.get("type") == "text"
]
return "\n".join(p for p in parts if p)
return ""
+47
View File
@@ -0,0 +1,47 @@
"""Side channel for what a backend learned during one turn.
Frontends pass ``capture=TurnCapture()`` through ``Backend.complete``'s
``**options``. Backends that keep state per conversation (the Claude SDK
adapter) fill it in after the stream closes; backends that don't
(anthropic, raycast) drop the kwarg and the frontend falls back to a
text-only history.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any
__all__ = ["TurnCapture", "TurnUsage"]
@dataclass(frozen=True, slots=True)
class TurnUsage:
input_tokens: int = 0
output_tokens: int = 0
cache_read_tokens: int = 0
cache_creation_tokens: int = 0
cost_usd: float | None = None
duration_ms: int | None = None
num_turns: int | None = None
model_usage: dict[str, Any] | None = None
"""``ResultMessage.model_usage`` verbatim: per-model tokens, cost, web searches."""
context_tokens: int = 0
"""Input of the last API call in the turn (fresh + cached + written to
cache) - the context size the model actually ran with, what Claude Code
shows as the context. The token fields above are sums over every API
call of the turn and grow with the number of tool calls."""
@dataclass
class TurnCapture:
synthesized_messages: list[dict[str, Any]] = field(default_factory=list)
"""The assistant/tool_result cycle of the turn as Anthropic messages."""
session_id: str | None = None
"""Backend session that ran the turn; persist and pass back as ``session_id``."""
usage: TurnUsage | None = None
interrupted: bool = False
"""The turn was cut by ``interrupt()`` (urgent inject), not by an error."""
+205
View File
@@ -0,0 +1,205 @@
"""Vault watcher for the envelope (§3.5, §4.6).
A content snapshot is taken at every envelope; a change is the diff of the
file against that snapshot, "added lines only". ``watchfiles`` delivers
paths with a settle debounce (Sync writes files in pieces), the change
carries the file's mtime rather than the moment it landed. Which files
show a full diff, which show up by name and which are ignored is a set of
glob patterns the config hands in - the gateway itself knows no path.
"""
from __future__ import annotations
import asyncio
import contextlib
import difflib
import fnmatch
import logging
import os
from dataclasses import dataclass, field
from datetime import UTC, datetime
from pathlib import Path
from typing import TYPE_CHECKING
from zoneinfo import ZoneInfo
from watchfiles import awatch
if TYPE_CHECKING:
from collections.abc import Iterable
__all__ = ["Change", "VaultWatch", "WatchRules"]
_log = logging.getLogger("beaver_gateway.core.watch")
@dataclass(frozen=True, slots=True)
class WatchRules:
"""Globs relative to the vault root; ``{today}`` expands to the local date.
``full`` files show their added lines, ``names`` files only their name
and how many lines they gained, ``ignore`` hides them; when several
patterns match, the most specific one decides.
"""
full: tuple[str, ...] = ()
names: tuple[str, ...] = ()
ignore: tuple[str, ...] = ()
suffixes: tuple[str, ...] = (".md",)
def kind(self, rel: str, *, today: str) -> str | None:
"""The most specific matching pattern wins, whichever list it is in."""
if not rel.endswith(self.suffixes):
return None
best: tuple[int, str | None] = (-1, None)
for kind, patterns in (
("full", self.full),
("names", self.names),
(None, self.ignore),
):
for pattern in patterns:
if _match(rel, pattern, today):
best = max(best, (_specificity(pattern), kind))
return best[1]
@dataclass(slots=True)
class Change:
path: str
mtime: datetime
added: list[str] = field(default_factory=list)
added_count: int = 0
full: bool = False
@dataclass(slots=True)
class _Seen:
lines: list[str] | None
count: int
class VaultWatch:
def __init__(
self, root: Path, rules: WatchRules, *, tz: str = "UTC", debounce: float = 4.0
) -> None:
self.root = root
self.rules = rules
self._tz = ZoneInfo(tz)
self._debounce = debounce
self._seen: dict[str, _Seen] = {}
self._pending: dict[str, Change] = {}
self._stop = asyncio.Event()
@property
def pending(self) -> list[Change]:
return list(self._pending.values())
def today(self, now: datetime | None = None) -> str:
return (now or datetime.now(UTC)).astimezone(self._tz).date().isoformat()
async def run(self) -> None:
await asyncio.to_thread(self.snapshot)
_log.info("watching %s (%d files in snapshot)", self.root, len(self._seen))
async for changes in awatch(
self.root,
debounce=int(self._debounce * 1000),
stop_event=self._stop,
ignore_permission_denied=True,
):
for _kind, raw in changes:
with contextlib.suppress(OSError):
self.note(Path(raw))
def stop(self) -> None:
self._stop.set()
def snapshot(self) -> None:
today = self.today()
for path in self._walk():
rel = self._rel(path)
kind = self.rules.kind(rel, today=today)
if kind is None:
continue
with contextlib.suppress(OSError):
lines = _read_lines(path)
self._seen[rel] = _Seen(
lines=lines if kind == "full" else None, count=len(lines)
)
def note(self, path: Path) -> Change | None:
"""Record a change of ``path`` against the last envelope's snapshot."""
try:
rel = self._rel(path)
except ValueError:
return None
kind = self.rules.kind(rel, today=self.today())
if kind is None:
return None
if not path.is_file():
self._pending.pop(rel, None)
return None
stat = path.stat()
lines = _read_lines(path)
seen = self._seen.get(rel)
change = Change(
path=rel,
mtime=datetime.fromtimestamp(stat.st_mtime, tz=UTC),
full=kind == "full",
)
if seen is not None and seen.lines is not None:
change.added = _added(seen.lines, lines)
change.added_count = len(change.added)
elif seen is not None:
change.added_count = max(len(lines) - seen.count, 0)
else:
change.added = lines if kind == "full" else []
change.added_count = len(lines)
if change.added_count == 0 and seen is not None:
self._pending.pop(rel, None)
return None
self._pending[rel] = change
return change
def take(self) -> list[Change]:
"""Hand out what changed since the last envelope and reset the snapshot."""
changes = list(self._pending.values())
self._pending.clear()
for change in changes:
path = self.root / change.path
with contextlib.suppress(OSError):
lines = _read_lines(path)
self._seen[change.path] = _Seen(
lines=lines if change.full else None, count=len(lines)
)
return changes
def _rel(self, path: Path) -> str:
return path.resolve().relative_to(self.root.resolve()).as_posix()
def _walk(self) -> Iterable[Path]:
for dirpath, dirnames, filenames in os.walk(self.root):
dirnames[:] = [d for d in dirnames if not d.startswith(".")]
for name in filenames:
yield Path(dirpath) / name
def _match(rel: str, pattern: str, today: str) -> bool:
return fnmatch.fnmatchcase(
rel, pattern.replace("{today}", today).replace("**", "*")
)
def _specificity(pattern: str) -> int:
return sum(1 for ch in pattern if ch not in "*?[]{}")
def _read_lines(path: Path) -> list[str]:
return path.read_text(encoding="utf-8", errors="replace").splitlines()
def _added(old: list[str], new: list[str]) -> list[str]:
matcher = difflib.SequenceMatcher(a=old, b=new, autojunk=False)
added: list[str] = []
for tag, _i1, _i2, j1, j2 in matcher.get_opcodes():
if tag in ("insert", "replace"):
added.extend(line for line in new[j1:j2] if line.strip())
return added
+13 -9
View File
@@ -25,20 +25,24 @@ async def require_token(
) -> str: ) -> str:
"""Verify the request's bearer + scope, return the token's audit name. """Verify the request's bearer + scope, return the token's audit name.
Accepts both ``X-Api-Key: <token>`` (Anthropic SDK / LibreChat) and Accepts ``X-Api-Key: <token>`` (Anthropic SDK / LibreChat),
``Authorization: Bearer <token>`` (curl, Cursor). 401 on missing / ``Authorization: Bearer <token>`` (curl, Cursor) and, when neither
header is present, ``?token=<token>`` (Komodo alerters). 401 on missing /
unknown token; 403 on a known token whose scope doesn't cover unknown token; 403 on a known token whose scope doesn't cover
``scope``. Bootstrap tokens implicitly carry ``"*"`` and pass every ``scope``. Bootstrap tokens implicitly carry ``"*"`` and pass every
scope check. scope check.
""" """
api_key = request.headers.get("x-api-key") api_key = request.headers.get("x-api-key")
identity = ( authorization = request.headers.get("authorization")
await runtime.token_store.verify(api_key) if api_key:
if api_key identity = await runtime.token_store.verify(api_key)
else await runtime.token_store.verify_bearer( elif authorization:
request.headers.get("authorization") identity = await runtime.token_store.verify_bearer(authorization)
) else:
) # Webhook senders that cannot set headers (Komodo alerters) put the
# token in the query string; the URL is not logged with it.
qs_token = request.query_params.get("token")
identity = await runtime.token_store.verify(qs_token) if qs_token else None
if identity is None: if identity is None:
raise HTTPException( raise HTTPException(
status.HTTP_401_UNAUTHORIZED, status.HTTP_401_UNAUTHORIZED,
+63
View File
@@ -0,0 +1,63 @@
"""Server-sent events helpers shared by the markdown and API frontends."""
from __future__ import annotations
import asyncio
import contextlib
import json
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from collections.abc import AsyncIterator
__all__ = ["HEARTBEAT_INTERVAL", "SSE_HEADERS", "events_with_heartbeat", "sse_pack"]
HEARTBEAT_INTERVAL = 15.0
"""Seconds of backend silence before a comment frame keeps the socket warm."""
SSE_HEADERS = {
"Cache-Control": "no-cache, no-transform",
"Connection": "keep-alive",
"X-Accel-Buffering": "no",
}
KEEPALIVE = b": keepalive\n\n"
async def events_with_heartbeat(
events: AsyncIterator[Any], interval: float = HEARTBEAT_INTERVAL
) -> AsyncIterator[Any]:
"""Pass ``events`` through, yielding ``None`` after ``interval`` seconds of silence.
One in-flight ``__anext__`` task is reused across timeouts: a second
consumer on the same async generator raises ``RuntimeError``.
Cancellation of the outer scope cancels that task instead of leaving
it dangling.
"""
src = events.__aiter__()
next_task: asyncio.Task[Any] | None = None
try:
while True:
if next_task is None:
next_task = asyncio.ensure_future(src.__anext__())
done, _pending = await asyncio.wait({next_task}, timeout=interval)
if not done:
yield None
continue
task = next_task
next_task = None
try:
result = task.result()
except StopAsyncIteration:
return
yield result
finally:
if next_task is not None and not next_task.done():
next_task.cancel()
with contextlib.suppress(BaseException):
await next_task
def sse_pack(event: str, data: dict[str, Any]) -> bytes:
body = json.dumps(data, ensure_ascii=False)
return f"event: {event}\ndata: {body}\n\n".encode()
+30
View File
@@ -0,0 +1,30 @@
"""Where a frontend is reachable from outside, as seen by one request."""
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from starlette.requests import Request
from beaver_gateway.frontends.base import Frontend, GatewayRuntime
def request_origin(request: Request, runtime: GatewayRuntime) -> str:
"""``Gateway.public_url`` if configured, else the request's own origin."""
if runtime.public_url:
return runtime.public_url
host = request.headers.get("host")
if not host:
return f"http://{request.url.hostname}:{request.url.port}"
scheme = request.headers.get("x-forwarded-proto") or request.url.scheme
return f"{scheme}://{host}"
def external_base(request: Request, runtime: GatewayRuntime) -> str:
"""Origin plus the mount path of the app handling ``request``."""
return request_origin(request, runtime) + str(request.scope.get("root_path", ""))
def frontend_url(request: Request, runtime: GatewayRuntime, fe: Frontend) -> str | None:
return request_origin(request, runtime) + fe.path if fe.path else None
@@ -1,10 +1,4 @@
"""Admin UI (Phase 4.3). """Admin console: the ``ui/`` SPA served from the admin port plus its login."""
Browser-facing console: login, dashboard, token CRUD, audit viewer.
Templates live in ``./templates``; the package loader picks them up via
``importlib.resources`` so the layout works editable and inside the
Docker image without copy hacks.
"""
from beaver_gateway.frontends.admin.frontend import AdminFrontend from beaver_gateway.frontends.admin.frontend import AdminFrontend
File diff suppressed because it is too large. Load diff
@@ -1,206 +0,0 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>{% block title %}beaver-gateway · admin{% endblock %}</title>
<script src="https://unpkg.com/htmx.org@2.0.4" defer></script>
<style>
:root {
--bg: #fbfbfd;
--fg: #1d1d1f;
--muted: #6e6e73;
--line: #e5e5ea;
--accent: #0071e3;
--danger: #d70015;
--surface: #ffffff;
--code-bg: #f5f5f7;
}
* { box-sizing: border-box; }
html, body { margin: 0; padding: 0; background: var(--bg); color: var(--fg); }
body {
font-family: -apple-system, "SF Pro Display", "SF Pro Text",
BlinkMacSystemFont, system-ui, sans-serif;
font-size: 15px;
line-height: 1.55;
letter-spacing: -0.005em;
}
a { color: var(--accent); text-decoration: none; }
a:hover { text-decoration: underline; }
code, pre, kbd, samp {
font-family: ui-monospace, "SF Mono", Menlo, Consolas, monospace;
font-size: 0.92em;
}
pre {
background: var(--code-bg);
padding: 1rem 1.25rem;
border-radius: 12px;
overflow-x: auto;
}
header.top {
border-bottom: 1px solid var(--line);
background: var(--surface);
}
header.top .inner {
max-width: 1080px;
margin: 0 auto;
padding: 1rem 1.5rem;
display: flex;
align-items: center;
gap: 1.25rem;
}
header.top h1 {
font-size: 1.05rem;
font-weight: 600;
letter-spacing: -0.01em;
margin: 0;
}
nav.tabs { display: flex; gap: 1.1rem; flex: 1; }
nav.tabs a {
color: var(--fg);
padding: 0.25rem 0.1rem;
border-bottom: 2px solid transparent;
}
nav.tabs a.active {
border-bottom-color: var(--fg);
}
header.top .actor {
color: var(--muted);
font-size: 0.9em;
}
main {
max-width: 1080px;
margin: 0 auto;
padding: 2.25rem 1.5rem 4rem;
}
h2 {
font-weight: 600;
letter-spacing: -0.015em;
margin: 2rem 0 0.75rem;
}
h2:first-of-type { margin-top: 0; }
.muted { color: var(--muted); }
.card {
background: var(--surface);
border: 1px solid var(--line);
border-radius: 14px;
padding: 1.25rem 1.5rem;
margin-bottom: 1.25rem;
}
table { width: 100%; border-collapse: collapse; }
th, td {
text-align: left;
padding: 0.6rem 0.85rem;
border-bottom: 1px solid var(--line);
vertical-align: top;
}
th {
font-weight: 500;
color: var(--muted);
font-size: 0.85em;
text-transform: uppercase;
letter-spacing: 0.04em;
}
tr:last-child td { border-bottom: none; }
.revoked td { color: var(--muted); }
button, .btn {
font-family: inherit;
font-size: 0.92em;
padding: 0.45rem 0.95rem;
border-radius: 8px;
border: 1px solid var(--line);
background: var(--surface);
color: var(--fg);
cursor: pointer;
transition: background 120ms ease;
}
button:hover, .btn:hover { background: var(--code-bg); }
button.primary, .btn.primary {
background: var(--accent);
border-color: var(--accent);
color: white;
}
button.primary:hover, .btn.primary:hover {
background: #005bb5;
}
button.danger { color: var(--danger); border-color: #f0c5c9; }
button.danger:hover { background: #fdeff1; }
form.inline { display: inline; margin: 0; }
.form-grid {
display: grid;
grid-template-columns: minmax(180px, 1fr) 160px auto;
gap: 0.75rem;
align-items: end;
}
.form-grid label {
display: block;
font-size: 0.8em;
color: var(--muted);
text-transform: uppercase;
letter-spacing: 0.05em;
margin-bottom: 0.3rem;
}
input[type="text"], input[type="password"], select {
width: 100%;
padding: 0.55rem 0.7rem;
border: 1px solid var(--line);
border-radius: 8px;
font-family: inherit;
font-size: 0.95em;
background: var(--surface);
color: var(--fg);
}
input:focus, select:focus { outline: 2px solid var(--accent); outline-offset: 1px; }
.banner {
padding: 1rem 1.25rem;
border-radius: 12px;
background: #f0f9ff;
border: 1px solid #cfe8ff;
margin: 0 0 1.25rem;
}
.banner.error {
background: #fff0f0;
border-color: #f5c2c5;
color: var(--danger);
}
.banner.warn {
background: #fff7e6;
border-color: #fadf9c;
}
.pill {
display: inline-block;
padding: 0.1rem 0.55rem;
border-radius: 999px;
background: var(--code-bg);
font-size: 0.78em;
color: var(--muted);
}
.pill.scope-wild { background: #ecfdf3; color: #027a48; }
.pill.scope-admin { background: #fff4e6; color: #b54708; }
</style>
</head>
<body>
{% block header %}
<header class="top">
<div class="inner">
<h1>beaver-gateway</h1>
<nav class="tabs">
<a href="{{ p }}/" class="{% if active == 'dashboard' %}active{% endif %}">Dashboard</a>
<a href="{{ p }}/chat" class="{% if active == 'chat' %}active{% endif %}">Chat</a>
<a href="{{ p }}/pty" class="{% if active == 'pty' %}active{% endif %}">PTY</a>
<a href="{{ p }}/tokens" class="{% if active == 'tokens' %}active{% endif %}">Tokens</a>
<a href="{{ p }}/audit" class="{% if active == 'audit' %}active{% endif %}">Audit</a>
</nav>
<span class="actor">Signed in as <strong>{{ user }}</strong></span>
<form class="inline" method="post" action="{{ p }}/logout">
<input type="hidden" name="csrf_token" value="{{ csrf }}">
<button type="submit">Log out</button>
</form>
</div>
</header>
{% endblock %}
<main>
{% block content %}{% endblock %}
</main>
</body>
</html>
@@ -1,12 +0,0 @@
{# Renders into #token-create-result (hx-target on the form). The
two OOB swaps reach the rest of the page: prepend the new row,
and erase the empty-state placeholder if it's still in the DOM. #}
<div class="banner warn">
<strong>Token created: {{ token.name }}</strong>
<p class="muted" style="margin:0.25rem 0 0.5rem;">Copy it now — this is the only time it will be shown.</p>
<pre style="margin:0;">{{ plaintext }}</pre>
</div>
<tbody id="tokens-rows" hx-swap-oob="afterbegin">
{% include "_token_row.html" %}
</tbody>
<tr id="tokens-empty" hx-swap-oob="delete"></tr>
@@ -1 +0,0 @@
<div class="banner error">{{ message }}</div>
@@ -1,25 +0,0 @@
<tr id="token-row-{{ token.id }}" {% if token.revoked_at %}class="revoked"{% endif %}>
<td><strong>{{ token.name }}</strong></td>
<td>
<span class="pill {% if token.scope == '*' %}scope-wild{% elif token.scope == 'admin' %}scope-admin{% endif %}">{{ token.scope }}</span>
</td>
<td><code>{{ token.created_at | fmt_dt }}</code></td>
<td><code>{{ token.last_used_at | fmt_dt }}</code></td>
<td><code>{{ token.revoked_at | fmt_dt }}</code></td>
<td style="text-align:right;">
{% if not token.revoked_at %}
<form
class="inline"
hx-post="{{ p }}/tokens/{{ token.id }}/revoke"
hx-target="#token-row-{{ token.id }}"
hx-swap="outerHTML"
hx-confirm="Revoke token {{ token.name }}? This cannot be undone."
>
<input type="hidden" name="csrf_token" value="{{ csrf }}">
<button class="danger" type="submit">Revoke</button>
</form>
{% else %}
<span class="muted">revoked</span>
{% endif %}
</td>
</tr>
@@ -1,37 +0,0 @@
{% extends "_layout.html" %}
{% set active = "audit" %}
{% block title %}beaver-gateway · audit{% endblock %}
{% block content %}
<h2>Audit log</h2>
<div class="card">
{% if audit %}
<table>
<thead>
<tr>
<th>Time</th><th>Actor</th><th>Kind</th><th>Agent</th><th>Detail</th>
</tr>
</thead>
<tbody>
{% for row in audit %}
<tr>
<td><code>{{ row.ts | fmt_dt }}</code></td>
<td>{{ row.actor }}</td>
<td><span class="pill">{{ row.kind }}</span></td>
<td>{{ row.agent_name or "—" }}</td>
<td><code>{{ row.detail_json | fmt_detail }}</code></td>
</tr>
{% endfor %}
</tbody>
</table>
<p class="muted" style="margin-top:1rem;">
{% if next_before %}
<a href="{{ p }}/audit?before={{ next_before }}">Older entries →</a>
{% else %}
End of log.
{% endif %}
</p>
{% else %}
<p class="muted">Nothing logged yet.</p>
{% endif %}
</div>
{% endblock %}
@@ -1,355 +0,0 @@
{% extends "_layout.html" %}
{% set active = "chat" %}
{% block title %}beaver-gateway · chat{% endblock %}
{% block content %}
<div class="chat-wrap">
<div class="chat-toolbar">
<label>
<span>Agent</span>
<select id="agent-select">
{% for a in agents %}
<option value="{{ a.name }}">{{ a.name }} · {{ a.model }}</option>
{% else %}
<option disabled>no agents with a backend</option>
{% endfor %}
</select>
</label>
<div class="spacer"></div>
<button id="new-chat-btn" type="button">New chat</button>
</div>
<div id="messages" class="chat-messages" aria-live="polite"></div>
<form id="chat-form" class="chat-input">
<textarea id="chat-text" rows="3" placeholder="Message… (⌘/Ctrl+Enter to send)" required></textarea>
<button type="submit" class="primary" id="send-btn">Send</button>
</form>
</div>
<style>
.chat-wrap {
display: flex; flex-direction: column;
height: calc(100vh - 200px); min-height: 480px;
}
.chat-toolbar {
display: flex; gap: 0.85rem; align-items: end;
margin-bottom: 1rem;
}
.chat-toolbar .spacer { flex: 1; }
.chat-toolbar label {
display: flex; flex-direction: column; gap: 0.3rem;
min-width: 260px;
}
.chat-toolbar label > span {
font-size: 0.8em; color: var(--muted);
text-transform: uppercase; letter-spacing: 0.05em;
}
.chat-messages {
flex: 1; overflow-y: auto;
background: var(--surface); border: 1px solid var(--line);
border-radius: 14px; padding: 1.25rem;
display: flex; flex-direction: column; gap: 1rem;
}
.chat-empty { color: var(--muted); margin: auto; }
.msg { display: flex; }
.msg.user { justify-content: flex-end; }
.msg.user .bubble {
max-width: 78%; padding: 0.7rem 0.95rem; border-radius: 14px;
background: var(--accent); color: white;
white-space: pre-wrap; word-wrap: break-word;
}
.msg.assistant .blocks {
display: flex; flex-direction: column; gap: 0.5rem;
max-width: 88%;
}
.block-text {
background: var(--code-bg); padding: 0.7rem 0.95rem;
border-radius: 14px;
white-space: pre-wrap; word-wrap: break-word;
}
details.tool-call, details.thinking-block {
border: 1px solid var(--line); border-radius: 10px;
background: var(--surface); padding: 0 0.85rem;
font-size: 0.9em;
}
details.tool-call > summary,
details.thinking-block > summary {
cursor: pointer; padding: 0.55rem 0;
list-style: none;
display: flex; align-items: center; gap: 0.55rem;
}
details > summary::-webkit-details-marker { display: none; }
details.tool-call > summary::before,
details.thinking-block > summary::before {
content: "▸"; color: var(--muted); font-size: 0.78em;
}
details.tool-call[open] > summary::before,
details.thinking-block[open] > summary::before { content: "▾"; }
.tool-name {
font-family: ui-monospace, "SF Mono", Menlo, monospace;
font-weight: 500;
}
.tool-id, .tool-label {
color: var(--muted); font-size: 0.78em;
font-family: ui-monospace, "SF Mono", Menlo, monospace;
}
details.tool-call pre, details.thinking-block pre {
margin: 0 0 0.6rem; padding: 0.7rem 0.85rem;
background: var(--code-bg); border-radius: 8px;
font-size: 0.85em; max-height: 360px; overflow: auto;
}
.tool-label { display: block; margin: 0.15rem 0 0.25rem; }
.chat-input { display: flex; gap: 0.75rem; margin-top: 1rem; }
.chat-input textarea {
flex: 1; padding: 0.7rem 0.9rem;
border: 1px solid var(--line); border-radius: 12px;
font-family: inherit; font-size: 0.95em;
background: var(--surface); color: var(--fg); resize: vertical;
}
.chat-input textarea:focus { outline: 2px solid var(--accent); outline-offset: 1px; }
.chat-input button { align-self: stretch; padding-left: 1.4rem; padding-right: 1.4rem; }
.chat-error {
color: var(--danger); font-size: 0.85em;
padding: 0.55rem 0.75rem;
background: #fff0f0; border-radius: 10px;
border: 1px solid #f5c2c5;
}
.chat-input button:disabled { opacity: 0.55; cursor: progress; }
</style>
<script>
(() => {
const CSRF = {{ csrf | tojson }};
const URL_PREFIX = {{ p | tojson }};
const agentSelect = document.getElementById("agent-select");
const messagesEl = document.getElementById("messages");
const form = document.getElementById("chat-form");
const textEl = document.getElementById("chat-text");
const sendBtn = document.getElementById("send-btn");
const newBtn = document.getElementById("new-chat-btn");
// Anthropic-style history sent to the backend. We keep assistant
// content text-only — tool_use blocks can't round-trip without
// matching tool_result, and backends (claude-code) run tools
// internally anyway.
let apiMessages = [];
function renderEmpty() {
messagesEl.innerHTML = '<div class="chat-empty">No messages yet — say something.</div>';
}
renderEmpty();
newBtn.addEventListener("click", () => {
apiMessages = [];
renderEmpty();
});
function clearEmpty() {
const e = messagesEl.querySelector(".chat-empty");
if (e) e.remove();
}
function scrollDown() { messagesEl.scrollTop = messagesEl.scrollHeight; }
function appendUser(text) {
clearEmpty();
const row = document.createElement("div");
row.className = "msg user";
const bub = document.createElement("div");
bub.className = "bubble";
bub.textContent = text;
row.appendChild(bub);
messagesEl.appendChild(row);
scrollDown();
}
function appendAssistant() {
clearEmpty();
const row = document.createElement("div");
row.className = "msg assistant";
const blocks = document.createElement("div");
blocks.className = "blocks";
row.appendChild(blocks);
messagesEl.appendChild(row);
scrollDown();
return blocks;
}
function appendError(text) {
clearEmpty();
const e = document.createElement("div");
e.className = "chat-error";
e.textContent = text;
messagesEl.appendChild(e);
scrollDown();
}
function ensureBlock(state, index, type, extra) {
if (state.blocks[index]) return state.blocks[index];
const block = { type };
if (type === "text") {
const el = document.createElement("div");
el.className = "block-text";
state.container.appendChild(el);
block.el = el;
} else if (type === "thinking") {
const det = document.createElement("details");
det.className = "thinking-block";
const sum = document.createElement("summary");
sum.textContent = "Thinking";
det.appendChild(sum);
const pre = document.createElement("pre");
det.appendChild(pre);
state.container.appendChild(det);
block.el = pre;
} else if (type === "tool_use") {
const det = document.createElement("details");
det.className = "tool-call";
const sum = document.createElement("summary");
const name = document.createElement("span");
name.className = "tool-name";
name.textContent = "🔧 " + (extra.name || "tool");
sum.appendChild(name);
if (extra.id) {
const idEl = document.createElement("span");
idEl.className = "tool-id";
idEl.textContent = extra.id;
sum.appendChild(idEl);
}
det.appendChild(sum);
const label = document.createElement("span");
label.className = "tool-label";
label.textContent = "input";
det.appendChild(label);
const pre = document.createElement("pre");
det.appendChild(pre);
state.container.appendChild(det);
block.el = pre;
block.jsonBuf = "";
block.seedInput = extra.input;
}
state.blocks[index] = block;
return block;
}
function applyEvent(state, ev) {
const t = ev.type;
if (t === "content_block_start") {
const cb = ev.content_block || {};
if (cb.type === "text") {
ensureBlock(state, ev.index, "text", {});
} else if (cb.type === "thinking") {
ensureBlock(state, ev.index, "thinking", {});
} else if (cb.type === "tool_use") {
ensureBlock(state, ev.index, "tool_use",
{ name: cb.name, id: cb.id, input: cb.input });
}
} else if (t === "content_block_delta") {
const d = ev.delta || {};
const b = state.blocks[ev.index];
if (!b) return;
if (d.type === "text_delta") {
b.el.textContent += d.text || "";
state.assistantText += d.text || "";
scrollDown();
} else if (d.type === "thinking_delta") {
b.el.textContent += d.thinking || "";
scrollDown();
} else if (d.type === "input_json_delta") {
b.jsonBuf += d.partial_json || "";
}
} else if (t === "content_block_stop") {
const b = state.blocks[ev.index];
if (!b) return;
if (b.type === "tool_use") {
let input = null;
if (b.jsonBuf && b.jsonBuf.trim()) {
try { input = JSON.parse(b.jsonBuf); }
catch { input = b.jsonBuf; }
} else if (b.seedInput !== undefined && b.seedInput !== null) {
input = b.seedInput;
}
b.el.textContent = input === null
? "(no input)"
: (typeof input === "string"
? input
: JSON.stringify(input, null, 2));
}
} else if (t === "error") {
const msg = (ev.error && ev.error.message) || "stream error";
appendError(msg);
}
}
async function send() {
const text = textEl.value.trim();
if (!text) return;
const model = agentSelect.value;
if (!model) { appendError("no agent selected"); return; }
apiMessages.push({ role: "user", content: text });
appendUser(text);
textEl.value = "";
sendBtn.disabled = true;
const container = appendAssistant();
const state = { container, blocks: {}, assistantText: "" };
let resp;
try {
resp = await fetch(URL_PREFIX + "/chat/send", {
method: "POST",
headers: {
"Content-Type": "application/json",
"X-CSRF-Token": CSRF,
},
body: JSON.stringify({ model, messages: apiMessages }),
});
} catch (e) {
appendError("network error: " + e.message);
sendBtn.disabled = false;
return;
}
if (!resp.ok || !resp.body) {
let msg = resp.status + " " + resp.statusText;
try { const body = await resp.text(); if (body) msg = body; } catch {}
appendError(msg);
sendBtn.disabled = false;
return;
}
const reader = resp.body.getReader();
const dec = new TextDecoder();
let buf = "";
while (true) {
const { value, done } = await reader.read();
if (done) break;
buf += dec.decode(value, { stream: true });
let i;
while ((i = buf.indexOf("\n\n")) >= 0) {
const raw = buf.slice(0, i);
buf = buf.slice(i + 2);
if (!raw) continue;
let dataLine = "";
for (const line of raw.split("\n")) {
if (line.startsWith("data:")) dataLine += line.slice(5).trimStart();
}
if (!dataLine) continue;
let payload;
try { payload = JSON.parse(dataLine); } catch { continue; }
applyEvent(state, payload);
}
}
if (state.assistantText) {
apiMessages.push({ role: "assistant", content: state.assistantText });
}
sendBtn.disabled = false;
textEl.focus();
}
form.addEventListener("submit", (e) => { e.preventDefault(); send(); });
textEl.addEventListener("keydown", (e) => {
if ((e.metaKey || e.ctrlKey) && e.key === "Enter") {
e.preventDefault();
send();
}
});
})();
</script>
{% endblock %}
@@ -1,490 +0,0 @@
{% extends "_layout.html" %}
{% set active = "dashboard" %}
{% block title %}beaver-gateway · dashboard{% endblock %}
{% block content %}
<h2>Agents</h2>
<div class="card">
{% if agents %}
<table>
<thead><tr><th>Name</th><th>Type</th><th>Model</th><th>Exposed MCPs</th></tr></thead>
<tbody>
{% for a in agents %}
<tr>
<td><code>{{ a.name }}</code></td>
<td><span class="pill">{{ a.__class__.__name__ }}</span></td>
<td><code>{{ a.model }}</code></td>
<td>
{% for em in a.expose_mcps %}<code>{{ em.name }}</code>{% if not loop.last %}, {% endif %}{% else %}<span class="muted">—</span>{% endfor %}
</td>
</tr>
{% endfor %}
</tbody>
</table>
{% else %}
<p class="muted">No agents configured.</p>
{% endif %}
</div>
<h2>MCP namespaces</h2>
<div class="card">
{% if mcps %}
<table>
<thead><tr><th>Name</th><th>Kind</th></tr></thead>
<tbody>
{% for m in mcps %}
<tr>
<td><code>{{ m.name }}</code></td>
<td><span class="pill">{{ m.kind }}</span></td>
</tr>
{% endfor %}
</tbody>
</table>
{% else %}
<p class="muted">No MCP servers configured.</p>
{% endif %}
</div>
<h2>Endpoints</h2>
<div class="card endpoints">
{% if endpoints.agents or endpoints.mcps %}
<div class="ep-controls">
<label class="ep-token">
<span>Token (hint)</span>
<select id="ep-token-select">
<option value="" data-scope="">— select a known token —</option>
{% for t in tokens %}
<option value="{{ t.name }}" data-scope="{{ t.scope }}">{{ t.name }} · scope {{ t.scope }}</option>
{% endfor %}
</select>
</label>
<label class="ep-secret">
<span>Bearer secret</span>
<input type="password" id="ep-token-secret" autocomplete="off" spellcheck="false"
placeholder="paste plaintext (we only store the Argon2 hash)">
</label>
<label class="ep-show">
<input type="checkbox" id="ep-show-secret"> <span>show</span>
</label>
</div>
<p class="muted ep-note">
Beaver stores only an Argon2 hash of each token, so the plaintext can't be reconstructed.
Paste the value you saved at creation; if you've lost it, <a href="{{ p }}/tokens">mint a new one</a>.
Below: pick a token to see which endpoints its scope covers, paste the secret to fill it
into URL / curl, then click Copy.
</p>
{% if endpoints.agents %}
<h3 class="ep-h3">Agents — <code>POST /v1/messages</code></h3>
<table class="ep-table" data-required-scope="messages">
<thead><tr><th>Agent</th><th>Model</th><th>URL</th><th class="ep-actions-th"></th></tr></thead>
<tbody>
{% for ep in endpoints.agents %}
<tr class="ep-row" data-kind="messages" data-agent="{{ ep.agent }}" data-url="{{ ep.url }}">
<td>
<code>{{ ep.agent }}</code>
<span class="pill">{{ ep.agent_type }}</span>
<span class="ep-scope-warn" hidden>scope mismatch</span>
</td>
<td><code>{{ ep.model }}</code></td>
<td><code class="ep-url">{{ ep.url }}</code></td>
<td class="ep-actions">
<button type="button" data-action="copy-url">Copy URL</button>
<button type="button" data-action="copy-curl">Copy curl</button>
<button type="button" data-action="toggle-curl" aria-expanded="false">▸ curl</button>
</td>
</tr>
<tr class="ep-curl-row" hidden><td colspan="4"><pre class="ep-curl"></pre></td></tr>
{% endfor %}
</tbody>
</table>
{% elif endpoints.anthropic_base is none and agents %}
<p class="muted">
Agents are declared but no <code>AnthropicMessagesFrontend</code> is configured —
add one to <code>Gateway(frontends=[...])</code> to expose them over HTTP.
</p>
{% endif %}
{% if endpoints.mcps %}
<h3 class="ep-h3">MCP — streamable HTTP</h3>
<table class="ep-table" data-required-scope="mcp">
<thead><tr><th>Namespace</th><th>Kind</th><th>URL</th><th class="ep-actions-th"></th></tr></thead>
<tbody>
{% for ep in endpoints.mcps %}
<tr class="ep-row" data-kind="mcp" data-namespace="{{ ep.namespace }}" data-url="{{ ep.url }}">
<td>
<code>{{ ep.namespace }}</code>
<span class="ep-scope-warn" hidden>scope mismatch</span>
</td>
<td><span class="pill">{{ ep.kind }}</span></td>
<td><code class="ep-url">{{ ep.url }}</code></td>
<td class="ep-actions">
<button type="button" data-action="copy-url">Copy URL</button>
<button type="button" data-action="copy-url-token">Copy URL+token</button>
<button type="button" data-action="copy-curl">Copy curl</button>
<button type="button" data-action="toggle-curl" aria-expanded="false">▸ curl</button>
</td>
</tr>
<tr class="ep-curl-row" hidden><td colspan="4"><pre class="ep-curl"></pre></td></tr>
{% endfor %}
</tbody>
</table>
{% elif endpoints.mcp_base is none and mcps %}
<p class="muted">
MCP servers are declared but no <code>McpServerFrontend</code> is configured —
add one to <code>Gateway(frontends=[...])</code> to expose them over HTTP.
</p>
{% endif %}
{% if endpoints.markdown %}
<h3 class="ep-h3">Markdown — <code>POST /chat</code></h3>
<table class="ep-table" data-required-scope="messages">
<thead><tr><th>Vault</th><th>Default agent</th><th>URL</th><th class="ep-actions-th"></th></tr></thead>
<tbody>
<tr class="ep-row"
data-kind="markdown"
data-url="{{ endpoints.markdown.url }}"
data-base="{{ endpoints.markdown.base }}"
data-sample-agent="{{ endpoints.markdown.sample_agent }}">
<td>
<code>{{ endpoints.markdown.vault_path }}</code>
<span class="ep-scope-warn" hidden>scope mismatch</span>
{% if endpoints.markdown.log_all_chats %}
<span class="pill">log_all_chats · {{ endpoints.markdown.logged_subdir }}/</span>
{% endif %}
</td>
<td>
{% if endpoints.markdown.default_agent %}
<code>{{ endpoints.markdown.default_agent }}</code>
{% else %}
<span class="muted">— (request must set <code>agent</code> or frontmatter)</span>
{% endif %}
</td>
<td>
<code class="ep-url">{{ endpoints.markdown.url }}</code>
<div class="muted" style="font-size:0.85em; margin-top:0.25rem;">
plugin base: <code>{{ endpoints.markdown.base }}</code>
</div>
</td>
<td class="ep-actions">
<button type="button" data-action="copy-url">Copy URL</button>
<button type="button" data-action="copy-base">Copy plugin base</button>
<button type="button" data-action="copy-curl">Copy curl</button>
<button type="button" data-action="toggle-curl" aria-expanded="false">▸ curl</button>
</td>
</tr>
<tr class="ep-curl-row" hidden><td colspan="4"><pre class="ep-curl"></pre></td></tr>
</tbody>
</table>
{% endif %}
{% else %}
<p class="muted">
Nothing exposed yet — declare agents / MCPs and the matching frontends
(<code>AnthropicMessagesFrontend</code>, <code>McpServerFrontend</code>) in your config.
</p>
{% endif %}
</div>
<h2>Recent activity</h2>
<div class="card">
{% if audit %}
<table>
<thead><tr><th>Time</th><th>Actor</th><th>Kind</th><th>Agent</th><th>Detail</th></tr></thead>
<tbody>
{% for row in audit %}
<tr>
<td><code>{{ row.ts | fmt_dt }}</code></td>
<td>{{ row.actor }}</td>
<td><span class="pill">{{ row.kind }}</span></td>
<td>{{ row.agent_name or "—" }}</td>
<td><code>{{ row.detail_json | fmt_detail }}</code></td>
</tr>
{% endfor %}
</tbody>
</table>
<p class="muted" style="margin-top:0.85rem;"><a href="{{ p }}/audit">Full log →</a></p>
{% else %}
<p class="muted">Nothing logged yet.</p>
{% endif %}
</div>
<style>
.endpoints .ep-controls {
display: grid;
grid-template-columns: minmax(220px, 1fr) 2fr auto;
gap: 0.85rem;
align-items: end;
margin-bottom: 0.5rem;
}
.endpoints .ep-controls label > span {
display: block;
font-size: 0.8em; color: var(--muted);
text-transform: uppercase; letter-spacing: 0.05em;
margin-bottom: 0.3rem;
}
.endpoints .ep-show {
display: flex; align-items: center; gap: 0.4rem;
padding-bottom: 0.6rem; color: var(--muted); font-size: 0.9em;
}
.endpoints .ep-note {
font-size: 0.85em; margin: 0.25rem 0 1.2rem;
}
.endpoints .ep-h3 {
font-size: 0.95rem; font-weight: 600;
margin: 1.5rem 0 0.55rem;
}
.endpoints .ep-table { table-layout: auto; }
.endpoints .ep-table th.ep-actions-th { width: 1%; }
.endpoints .ep-url {
word-break: break-all;
}
.endpoints .ep-actions {
white-space: nowrap;
text-align: right;
}
.endpoints .ep-actions button {
padding: 0.32rem 0.7rem;
font-size: 0.85em;
margin-left: 0.3rem;
}
.endpoints .ep-scope-warn {
display: inline-block;
margin-left: 0.5rem;
padding: 0.05rem 0.45rem;
background: #fff7e6;
border: 1px solid #fadf9c;
color: #8a5a00;
border-radius: 999px;
font-size: 0.72em;
}
.endpoints .ep-row.scope-mismatch td:not(.ep-actions) { opacity: 0.55; }
.endpoints .ep-row.scope-mismatch .ep-actions button { opacity: 0.7; }
.endpoints .ep-curl-row td { padding-top: 0; padding-bottom: 0; }
.endpoints .ep-curl {
margin: 0.4rem 0 1rem;
font-size: 0.82em;
max-height: 280px;
}
.endpoints button[data-copied="1"] {
background: #ecfdf3 !important;
border-color: #b6e6cb !important;
color: #027a48;
}
@media (max-width: 760px) {
.endpoints .ep-controls { grid-template-columns: 1fr; }
.endpoints .ep-actions { text-align: left; padding-top: 0.4rem; }
}
</style>
<script>
(() => {
const card = document.querySelector(".endpoints");
if (!card) return;
const sel = card.querySelector("#ep-token-select");
const secret = card.querySelector("#ep-token-secret");
const showCb = card.querySelector("#ep-show-secret");
if (!sel || !secret || !showCb) return;
const PLACEHOLDER = "<YOUR_TOKEN>";
function currentToken() {
const v = secret.value;
return v && v.length > 0 ? v : null;
}
function selectedScope() {
const o = sel.options[sel.selectedIndex];
return o ? (o.getAttribute("data-scope") || "") : "";
}
function scopeCovers(have, need) {
if (!have) return true; // no token selected — don't grey anything out
if (have === "*") return true;
return have === need;
}
function escSh(s) {
// single-quote shell escape: close, escape, reopen.
return "'" + String(s).replace(/'/g, "'\\''") + "'";
}
function buildCurl(row) {
const kind = row.getAttribute("data-kind");
const url = row.getAttribute("data-url");
const tok = currentToken() || PLACEHOLDER;
if (kind === "messages") {
const agent = row.getAttribute("data-agent");
const body = JSON.stringify({
model: agent,
messages: [{ role: "user", content: "hello" }],
stream: true,
});
return [
"curl -N \\",
" -H " + escSh("x-api-key: " + tok) + " \\",
" -H 'content-type: application/json' \\",
" -d " + escSh(body) + " \\",
" " + url,
].join("\n");
}
if (kind === "markdown") {
const agent = row.getAttribute("data-sample-agent") || "";
const body = JSON.stringify({
filename: "example.md",
content: "### User:\n\nhello\n",
agent: agent,
});
return [
"curl \\",
" -H " + escSh("Authorization: Bearer " + tok) + " \\",
" -H 'content-type: application/json' \\",
" -d " + escSh(body) + " \\",
" " + url,
].join("\n");
}
if (kind === "mcp") {
// Streamable-HTTP MCP wants `initialize` first — the response
// carries the `Mcp-Session-Id` header you must echo back on
// every subsequent call (tools/list, tools/call, ...). We use
// `-i` so the session-id is visible in the response, and ship
// the proper handshake body so a fresh paste actually works.
const body = JSON.stringify({
jsonrpc: "2.0",
id: 1,
method: "initialize",
params: {
protocolVersion: "2024-11-05",
capabilities: {},
clientInfo: { name: "beaver-admin-curl", version: "0" },
},
});
return [
"# 1) initialize — grab Mcp-Session-Id from the response headers",
"curl -N -i \\",
" -H " + escSh("Authorization: Bearer " + tok) + " \\",
" -H 'content-type: application/json' \\",
" -H 'accept: application/json, text/event-stream' \\",
" -d " + escSh(body) + " \\",
" " + url,
"",
"# 2) reuse that id on follow-up calls, e.g. tools/list:",
"# curl -N \\",
"# -H " + escSh("Authorization: Bearer " + tok) + " \\",
"# -H 'Mcp-Session-Id: <paste-from-step-1>' \\",
"# -H 'content-type: application/json' \\",
"# -H 'accept: application/json, text/event-stream' \\",
"# -d '{\"jsonrpc\":\"2.0\",\"id\":2,\"method\":\"tools/list\"}' \\",
"# " + url,
].join("\n");
}
return "";
}
function refreshScopeMarks() {
const have = selectedScope();
card.querySelectorAll(".ep-table").forEach((tbl) => {
const need = tbl.getAttribute("data-required-scope") || "";
const ok = scopeCovers(have, need);
tbl.querySelectorAll("tr.ep-row").forEach((row) => {
const warn = row.querySelector(".ep-scope-warn");
if (!ok) {
row.classList.add("scope-mismatch");
if (warn) warn.hidden = false;
} else {
row.classList.remove("scope-mismatch");
if (warn) warn.hidden = true;
}
});
});
}
function refreshOpenCurls() {
card.querySelectorAll("tr.ep-curl-row").forEach((cr) => {
if (cr.hidden) return;
const row = cr.previousElementSibling;
const pre = cr.querySelector(".ep-curl");
if (row && pre) pre.textContent = buildCurl(row);
});
}
function copyText(btn, text) {
const done = () => {
btn.setAttribute("data-copied", "1");
const prev = btn.textContent;
btn.textContent = "✓ copied";
setTimeout(() => {
btn.removeAttribute("data-copied");
btn.textContent = prev;
}, 1200);
};
if (navigator.clipboard && navigator.clipboard.writeText) {
navigator.clipboard.writeText(text).then(done, () => {
// Fallback for non-secure contexts.
fallbackCopy(text);
done();
});
} else {
fallbackCopy(text);
done();
}
}
function fallbackCopy(text) {
const ta = document.createElement("textarea");
ta.value = text;
ta.style.position = "fixed"; ta.style.opacity = "0";
document.body.appendChild(ta);
ta.select();
try { document.execCommand("copy"); } catch {}
document.body.removeChild(ta);
}
sel.addEventListener("change", () => {
const name = sel.value;
secret.placeholder = name
? "paste plaintext for '" + name + "' (we only store the hash)"
: "paste plaintext (we only store the Argon2 hash)";
refreshScopeMarks();
});
secret.addEventListener("input", refreshOpenCurls);
showCb.addEventListener("change", () => {
secret.type = showCb.checked ? "text" : "password";
});
card.addEventListener("click", (ev) => {
const btn = ev.target.closest("button[data-action]");
if (!btn) return;
const row = btn.closest("tr.ep-row");
if (!row) return;
const action = btn.getAttribute("data-action");
const url = row.getAttribute("data-url");
const tok = currentToken() || PLACEHOLDER;
if (action === "copy-url") {
copyText(btn, url);
} else if (action === "copy-base") {
const base = row.getAttribute("data-base");
if (base) copyText(btn, base);
} else if (action === "copy-url-token") {
const sep = url.indexOf("?") >= 0 ? "&" : "?";
copyText(btn, url + sep + "token=" + encodeURIComponent(tok));
} else if (action === "copy-curl") {
copyText(btn, buildCurl(row));
} else if (action === "toggle-curl") {
const curlRow = row.nextElementSibling;
if (!curlRow || !curlRow.classList.contains("ep-curl-row")) return;
const pre = curlRow.querySelector(".ep-curl");
if (curlRow.hidden) {
if (pre) pre.textContent = buildCurl(row);
curlRow.hidden = false;
btn.textContent = "▾ curl";
btn.setAttribute("aria-expanded", "true");
} else {
curlRow.hidden = true;
btn.textContent = "▸ curl";
btn.setAttribute("aria-expanded", "false");
}
}
});
refreshScopeMarks();
})();
</script>
{% endblock %}
@@ -1,83 +0,0 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>beaver-gateway · log in</title>
<style>
html, body { margin: 0; padding: 0; height: 100%; background: #fbfbfd; color: #1d1d1f; }
body {
font-family: -apple-system, "SF Pro Display", system-ui, sans-serif;
display: flex;
align-items: center;
justify-content: center;
min-height: 100vh;
}
.card {
background: #fff;
border: 1px solid #e5e5ea;
border-radius: 16px;
padding: 2rem 2.25rem;
width: min(360px, 92vw);
box-shadow: 0 10px 24px rgba(0,0,0,0.04);
}
h1 {
margin: 0 0 0.25rem;
font-size: 1.25rem;
font-weight: 600;
letter-spacing: -0.02em;
}
p.muted { color: #6e6e73; margin: 0 0 1.4rem; font-size: 0.92em; }
label { display: block; margin: 0.85rem 0 0.3rem; font-size: 0.82em; color: #6e6e73; }
input {
width: 100%;
padding: 0.6rem 0.75rem;
font-family: inherit;
font-size: 0.95em;
border: 1px solid #e5e5ea;
border-radius: 8px;
background: #fff;
box-sizing: border-box;
}
input:focus { outline: 2px solid #0071e3; outline-offset: 1px; }
button {
width: 100%;
margin-top: 1.25rem;
padding: 0.65rem;
font-family: inherit;
font-size: 0.95em;
background: #0071e3;
border: 1px solid #0071e3;
color: white;
border-radius: 8px;
cursor: pointer;
}
button:hover { background: #005bb5; }
.error {
background: #fff0f0;
border: 1px solid #f5c2c5;
color: #d70015;
padding: 0.65rem 0.85rem;
border-radius: 8px;
font-size: 0.9em;
margin-bottom: 1rem;
}
</style>
</head>
<body>
<div class="card">
<h1>beaver-gateway</h1>
<p class="muted">Sign in to manage tokens and view audit logs.</p>
{% if error %}
<div class="error">{{ error }}</div>
{% endif %}
<form method="post" action="{{ p }}/login">
<label for="username">Username</label>
<input id="username" name="username" type="text" autocomplete="username" required autofocus>
<label for="password">Password</label>
<input id="password" name="password" type="password" autocomplete="current-password" required>
<button type="submit">Sign in</button>
</form>
</div>
</body>
</html>
@@ -1,146 +0,0 @@
{% extends "_layout.html" %}
{% set active = "pty" %}
{% block title %}PTY {{ session_id }} · beaver-gateway{% endblock %}
{% block content %}
<style>
/* xterm.js styles inlined from the CDN bundle — keep them tight and
constrained to the terminal block so they don't leak into the rest
of the admin UI. */
.term-wrap {
background: #000;
border-radius: 12px;
padding: 12px;
border: 1px solid var(--line);
}
.term-host {
height: 70vh;
}
.term-host .xterm {
height: 100%;
}
.term-toolbar {
display: flex;
gap: 0.5rem;
align-items: center;
margin-bottom: 0.75rem;
flex-wrap: wrap;
}
.term-toolbar .status {
margin-left: auto;
font-size: 0.85em;
color: var(--muted);
}
.term-toolbar .status.connected { color: #027a48; }
.term-toolbar .status.dropped { color: var(--danger); }
</style>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@xterm/xterm@5.5.0/css/xterm.min.css">
<h2>PTY <code>{{ session_id }}</code></h2>
<p class="muted">
Live view of the claude subprocess TUI. Keystrokes you type here go
straight into its stdin — Enter, arrows, Ctrl-C, paste, all work.
<a href="{{ p }}/pty">← back to list</a>
</p>
<div class="term-toolbar">
<button id="reconnect-btn" type="button">Reconnect</button>
<button id="enter-btn" type="button" title="Send a single \r — handy if a paste is stuck in the input box">Send Enter</button>
<button id="ctrl-c-btn" type="button">Send Ctrl-C</button>
<span id="status" class="status">connecting…</span>
</div>
<div class="term-wrap">
<div id="term" class="term-host"></div>
</div>
<script src="https://cdn.jsdelivr.net/npm/@xterm/xterm@5.5.0/lib/xterm.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@xterm/addon-fit@0.10.0/lib/addon-fit.min.js"></script>
<script>
(function () {
const sessionId = {{ session_id|tojson }};
const wsBase =
(location.protocol === "https:" ? "wss://" : "ws://") +
location.host +
{{ p|tojson }} +
"/pty/" + encodeURIComponent(sessionId) + "/ws";
const term = new Terminal({
fontFamily: 'ui-monospace, "SF Mono", Menlo, Consolas, monospace',
fontSize: 13,
theme: {
background: "#000000",
foreground: "#d0d0d0",
},
convertEol: false,
cursorBlink: true,
scrollback: 5000,
});
const fit = new FitAddon.FitAddon();
term.loadAddon(fit);
term.open(document.getElementById("term"));
// Defer fit until after layout settles; xterm's measurement reads
// computed CSS that isn't stable until the page paints once.
requestAnimationFrame(() => fit.fit());
window.addEventListener("resize", () => fit.fit());
const statusEl = document.getElementById("status");
function setStatus(text, cls) {
statusEl.textContent = text;
statusEl.className = "status" + (cls ? " " + cls : "");
}
let ws = null;
let manuallyClosed = false;
function connect() {
manuallyClosed = false;
setStatus("connecting…", "");
ws = new WebSocket(wsBase);
ws.binaryType = "arraybuffer";
ws.onopen = () => setStatus("connected", "connected");
ws.onclose = () => {
if (!manuallyClosed) setStatus("disconnected", "dropped");
};
ws.onerror = () => setStatus("error", "dropped");
ws.onmessage = (ev) => {
if (typeof ev.data === "string") {
term.write(ev.data);
} else {
// ArrayBuffer — feed raw bytes to xterm. It expects either
// string or Uint8Array; the latter preserves byte boundaries
// exactly, which matters for ANSI sequences split across frames.
term.write(new Uint8Array(ev.data));
}
};
}
// Keystrokes -> server. xterm.js gives us the exact byte sequence
// the terminal would emit (e.g. Enter -> "\r", arrows -> "\x1b[A"
// etc.), so we just pass it through.
term.onData((data) => {
if (ws && ws.readyState === WebSocket.OPEN) {
ws.send(data);
}
});
document.getElementById("reconnect-btn").addEventListener("click", () => {
if (ws) {
manuallyClosed = true;
ws.close();
}
connect();
});
document.getElementById("enter-btn").addEventListener("click", () => {
if (ws && ws.readyState === WebSocket.OPEN) ws.send("\r");
term.focus();
});
document.getElementById("ctrl-c-btn").addEventListener("click", () => {
if (ws && ws.readyState === WebSocket.OPEN) ws.send("\x03");
term.focus();
});
connect();
})();
</script>
{% endblock %}
@@ -1,46 +0,0 @@
{% extends "_layout.html" %}
{% set active = "pty" %}
{% block title %}PTY · beaver-gateway · admin{% endblock %}
{% block content %}
<h2>Live PTY sessions</h2>
<p class="muted">
Each row is a running <code>claude</code> subprocess, most recently used first.
Open one to see what's currently rendered on its TUI and (if needed) type into
it directly. Sessions idle past the agent's TTL are terminated automatically.
</p>
{% if sessions %}
<div class="card" style="padding: 0;">
<table>
<thead>
<tr>
<th>Session ID</th>
<th>Agent</th>
<th>Idle</th>
<th>Age</th>
<th>PID</th>
<th>Buffer</th>
<th></th>
</tr>
</thead>
<tbody>
{% for s in sessions %}
<tr>
<td><code>{{ s.session_id }}</code></td>
<td>{{ s.agent }}</td>
<td>{{ s.idle }}</td>
<td class="muted">{{ s.age }}</td>
<td><span class="pill">{{ s.pid or "?" }}</span></td>
<td>{{ s.buffer_size }} bytes</td>
<td>
<a class="btn primary" href="{{ p }}/pty/{{ s.session_id }}">Open</a>
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% else %}
<div class="banner">No live PTY sessions right now. Start a turn and refresh.</div>
{% endif %}
{% endblock %}
@@ -1,64 +0,0 @@
{% extends "_layout.html" %}
{% set active = "tokens" %}
{% block title %}beaver-gateway · tokens{% endblock %}
{% block content %}
<h2>Create token</h2>
<div class="card">
<p class="muted">Plaintext is shown <strong>once</strong>, immediately after creation. Copy it before you navigate away — the database only ever holds the Argon2 hash.</p>
<div id="token-create-result"></div>
<form
hx-post="{{ p }}/tokens"
hx-target="#token-create-result"
hx-swap="innerHTML"
hx-on::after-request="if(event.detail.successful){this.reset();}"
>
<input type="hidden" name="csrf_token" value="{{ csrf }}">
<div class="form-grid">
<div>
<label for="name">Name</label>
<input id="name" name="name" type="text" placeholder="cursor / claude-desktop / mobile …" required>
</div>
<div>
<label for="scope">Scope</label>
<select id="scope" name="scope">
{% for s in scopes %}
<option value="{{ s }}" {% if s == "*" %}selected{% endif %}>{{ s }}</option>
{% endfor %}
</select>
</div>
<div>
<button class="primary" type="submit">Create</button>
</div>
</div>
</form>
</div>
<h2>
Tokens
<span class="muted" style="font-size:0.8em; font-weight:400; margin-left:0.5rem;">
{% if include_revoked %}
<a href="{{ p }}/tokens">Hide revoked</a>
{% else %}
<a href="{{ p }}/tokens?include_revoked=1">Show revoked</a>
{% endif %}
</span>
</h2>
<div class="card">
<table>
<thead>
<tr>
<th>Name</th><th>Scope</th><th>Created</th><th>Last used</th><th>Revoked</th><th></th>
</tr>
</thead>
{# Render the tbody unconditionally so the HTMX OOB swap on
create has a target even when the table starts empty. #}
<tbody id="tokens-rows">
{% for token in tokens %}
{% include "_token_row.html" %}
{% else %}
<tr id="tokens-empty"><td colspan="6" class="muted">No tokens yet. Create one above.</td></tr>
{% endfor %}
</tbody>
</table>
</div>
{% endblock %}
+149 -113
View File
@@ -4,39 +4,44 @@ Exposes the gateway as an Anthropic-compatible Messages endpoint, so any
client that already speaks Anthropic (Cursor, Cline, the official SDK, client that already speaks Anthropic (Cursor, Cline, the official SDK,
``curl``) can hit a configured agent by passing its name as ``model``. ``curl``) can hit a configured agent by passing its name as ``model``.
The frontend is intentionally thin: it authenticates the bearer token, A Claude agent behind this endpoint is a ``deep`` conversation: the client
resolves ``body.model`` to an agent + its backend, and then either knows nothing about our ids, so the text fingerprint of the history it
streams the backend's events straight to SSE or accumulates them into a sends is the ``(anthropic, fingerprint)`` binding of the conversation,
single ``Message`` for ``stream=false`` callers. All provider quirks rebound after every turn to the fingerprint the next request will carry.
already live in the backend adapters; we don't translate here. A history nobody has seen becomes a new conversation, materialized by the
home frontend of ``deep`` (the vault file), and every reply is published
Phase 1.4 wires only ``RaycastAgent`` through ``RaycastBackend``; on the bus so that file follows the chat. Other agents (Raycast) stay
``ClaudeAgent`` lands in Phase 2 and will plug into the same dispatch stateless and are only archived through ``turn_log_handlers``.
table without changes to this module.
""" """
from __future__ import annotations from __future__ import annotations
import json import json
import logging import logging
import uuid
from typing import TYPE_CHECKING, Any from typing import TYPE_CHECKING, Any
from fastapi import FastAPI, HTTPException, Request, status from fastapi import FastAPI, HTTPException, Request, status
from fastapi.responses import JSONResponse, StreamingResponse from fastapi.responses import JSONResponse, StreamingResponse
from beaver_gateway.agents.claude import ClaudeAgent
from beaver_gateway.core import audit from beaver_gateway.core import audit
from beaver_gateway.core.transcript import fingerprint, text_of
from beaver_gateway.core.turn_capture import TurnCapture
from beaver_gateway.core.turn_record import TurnRecord from beaver_gateway.core.turn_record import TurnRecord
from beaver_gateway.frontends._accumulate import StreamAccumulator, accumulate from beaver_gateway.frontends._accumulate import StreamAccumulator
from beaver_gateway.frontends._auth import require_token from beaver_gateway.frontends._auth import require_token
from beaver_gateway.frontends.base import Frontend from beaver_gateway.frontends.base import Frontend
if TYPE_CHECKING: if TYPE_CHECKING:
from collections.abc import AsyncIterator from collections.abc import AsyncIterator, Awaitable, Callable
from anthropic.types import Message, MessageParam from anthropic.types import Message, MessageParam
from beaver_gateway.core.conversations import Conversations
from beaver_gateway.core.events import MessageStreamEvent from beaver_gateway.core.events import MessageStreamEvent
from beaver_gateway.frontends.base import GatewayRuntime from beaver_gateway.frontends.base import GatewayRuntime
from beaver_gateway.storage.models import Conversation
_log = logging.getLogger("beaver_gateway.frontends.anthropic") _log = logging.getLogger("beaver_gateway.frontends.anthropic")
@@ -44,28 +49,18 @@ _log = logging.getLogger("beaver_gateway.frontends.anthropic")
__all__ = ["AnthropicMessagesFrontend"] __all__ = ["AnthropicMessagesFrontend"]
FRONTEND = "anthropic"
_TITLE_LEN = 80
class AnthropicMessagesFrontend(Frontend): class AnthropicMessagesFrontend(Frontend):
"""FastAPI app behind ``POST /v1/messages`` + ``GET /v1/models``.""" """FastAPI app behind ``POST /v1/messages`` + ``GET /v1/models``."""
def __init__( name = FRONTEND
self, kinds = ("deep",)
*, path = "/anthropic"
host: str = "0.0.0.0", # noqa: S104
port: int = 8000, def __init__(self) -> None:
public_base_url: str | None = None,
) -> None:
self.host = host
self.port = port
# External URL prefix the reverse proxy (Caddy/nginx/Cloudflare/…)
# uses to reach this frontend, e.g. ``https://api.example.com/ai``.
# The frontend's internal paths (``/v1/messages``, ``/v1/models``)
# are appended to it when the admin dashboard renders copy-pastable
# URLs. Trailing slash is stripped so the concatenation is
# idempotent. ``None`` means "advertise raw ``host:port``" (dev /
# no proxy) — the dashboard then derives the base from the
# browser's own request hostname.
self.public_base_url = public_base_url.rstrip("/") if public_base_url else None
self._runtime: GatewayRuntime | None = None self._runtime: GatewayRuntime | None = None
self._app: FastAPI | None = None self._app: FastAPI | None = None
@@ -73,21 +68,8 @@ class AnthropicMessagesFrontend(Frontend):
self._runtime = runtime self._runtime = runtime
self._app = self._build_app(runtime) self._app = self._build_app(runtime)
async def serve(self) -> None: def app(self) -> FastAPI | None:
# Local import: uvicorn pulls in a lot, no reason to load it when return self._app
# something else (a test, a script) imports this module just for
# the FastAPI factory.
import uvicorn
if self._app is None:
msg = "configure() must be called before serve()"
raise RuntimeError(msg)
config = uvicorn.Config(
self._app, host=self.host, port=self.port, log_level="info"
)
server = uvicorn.Server(config)
await server.serve()
def _build_app(self, runtime: GatewayRuntime) -> FastAPI: def _build_app(self, runtime: GatewayRuntime) -> FastAPI:
app = FastAPI(title="beaver-gateway / Anthropic Messages") app = FastAPI(title="beaver-gateway / Anthropic Messages")
@@ -134,9 +116,6 @@ class AnthropicMessagesFrontend(Frontend):
backend = runtime.backends.get(agent.name) backend = runtime.backends.get(agent.name)
if backend is None: if backend is None:
# Agent exists in config but its backend isn't wired in
# this phase (e.g. ClaudeAgent before Phase 2, or a
# RaycastAgent without RAYCAST_BEARER set at startup).
raise HTTPException( raise HTTPException(
status.HTTP_503_SERVICE_UNAVAILABLE, status.HTTP_503_SERVICE_UNAVAILABLE,
f"no backend configured for agent {agent.name!r}", f"no backend configured for agent {agent.name!r}",
@@ -144,7 +123,12 @@ class AnthropicMessagesFrontend(Frontend):
messages = body.get("messages") or [] messages = body.get("messages") or []
system = body.get("system") system = body.get("system")
system_str = system if isinstance(system, str) else None
stream_flag = bool(body.get("stream", False)) stream_flag = bool(body.get("stream", False))
if not messages or messages[-1].get("role") != "user":
raise HTTPException(
status.HTTP_400_BAD_REQUEST, "the last message must be a user turn"
)
_log.info( _log.info(
"messages: actor=%s agent=%s stream=%s msgs=%d", "messages: actor=%s agent=%s stream=%s msgs=%d",
@@ -153,10 +137,6 @@ class AnthropicMessagesFrontend(Frontend):
stream_flag, stream_flag,
len(messages), len(messages),
) )
# Record at request acceptance, not at stream completion:
# a long streaming response can be aborted mid-flight by
# the client, and we still want the row in the audit trail.
# Detail stays small — no message bodies, no system prompt.
await audit.log( await audit.log(
runtime, runtime,
actor=f"token:{token_name}", actor=f"token:{token_name}",
@@ -166,74 +146,139 @@ class AnthropicMessagesFrontend(Frontend):
msgs=len(messages), msgs=len(messages),
) )
# Forward per-request knobs the Anthropic body may carry — if isinstance(agent, ClaudeAgent) and runtime.conversations is not None:
# backend adapters layer these over per-agent defaults. Only conv, history = await _resolve(runtime.conversations, agent, messages)
# values explicitly present (not Anthropic-defaulted ones we turn_id = f"turn_{uuid.uuid4().hex[:12]}"
# never received) are forwarded, so the agent's default still capture = TurnCapture()
# wins when the caller omits the field. events = runtime.conversations.turn(
options: dict[str, Any] = {} conv,
if isinstance(body.get("temperature"), int | float): messages=messages,
options["temperature"] = body["temperature"] origin="user",
capture=capture,
turn_id=turn_id,
)
events = backend.complete( async def after(message: Message) -> None:
agent=agent, await _finish(
messages=messages, runtime,
system=system if isinstance(system, str) else None, conv,
**options, messages=messages,
) message=message,
capture=capture,
turn_id=turn_id,
history=history,
)
else:
options: dict[str, Any] = {}
if isinstance(body.get("temperature"), int | float):
options["temperature"] = body["temperature"]
events = backend.complete(
agent=agent, messages=messages, system=system_str, **options
)
system_str = system if isinstance(system, str) else None async def after(message: Message) -> None:
await _broadcast_turn(
if stream_flag: runtime,
# Side-accumulate while streaming so we can still emit a
# ``TurnRecord`` after the response closes. The buffered
# ``Message`` lives only in this coroutine's frame; SSE
# bytes still flow to the client unchanged.
return StreamingResponse(
_sse_and_broadcast(
events,
runtime=runtime,
agent_name=agent.name, agent_name=agent.name,
input_messages=messages, input_messages=messages,
output_message=message,
system=system_str, system=system_str,
model=model, )
),
if stream_flag:
return StreamingResponse(
_sse(events, model=model, after=after),
media_type="text/event-stream", media_type="text/event-stream",
) )
message = await accumulate(events, model=model) acc = StreamAccumulator()
await _broadcast_turn( async for ev in events:
runtime, acc.feed(ev)
agent_name=agent.name, message = acc.finalize(model=model)
input_messages=messages, await after(message)
output_message=message,
system=system_str,
)
return JSONResponse(content=message.model_dump(mode="json")) return JSONResponse(content=message.model_dump(mode="json"))
return app return app
async def _sse_and_broadcast( async def _resolve(
conversations: Conversations, agent: ClaudeAgent, messages: list[MessageParam]
) -> tuple[Conversation, list[dict[str, str]] | None]:
prior = [dict(m) for m in messages[:-1]]
if prior:
conv = await conversations.find_bound(
frontend=FRONTEND, external_id=fingerprint(prior)
)
if conv is not None:
if conv.agent_name != agent.name:
raise HTTPException(
status.HTTP_400_BAD_REQUEST,
f"this chat runs on {conv.agent_name!r}, not {agent.name!r}",
)
return conv, None
first = next(
(text_of(m.get("content")) for m in messages if m.get("role") == "user"), ""
)
try:
conv = await conversations.create(
kind="deep",
agent=agent.name,
origin=FRONTEND,
title=" ".join(first.split())[:_TITLE_LEN] or None,
)
except ValueError as exc:
raise HTTPException(status.HTTP_400_BAD_REQUEST, str(exc)) from exc
await conversations.materialize(conv)
history = [
{"role": str(m.get("role", "")), "text": text_of(m.get("content"))}
for m in prior
]
return conv, history
async def _finish(
runtime: GatewayRuntime,
conv: Conversation,
*,
messages: list[MessageParam],
message: Message,
capture: TurnCapture,
turn_id: str,
history: list[dict[str, str]] | None,
) -> None:
reply = "\n\n".join(
getattr(b, "text", "")
for b in message.content
if getattr(b, "type", "") == "text"
).strip()
tail = capture.synthesized_messages or [
{"role": "assistant", "content": [{"type": "text", "text": reply}]}
]
await runtime.conversations.bind(
conv, frontend=FRONTEND, external_id=fingerprint([*messages, *tail])
)
runtime.bus.publish(
"reply",
conversation_id=conv.external_id,
turn_id=turn_id,
source=FRONTEND,
prompt=text_of(messages[-1].get("content")),
text=reply,
history=history,
)
async def _sse(
events: AsyncIterator[MessageStreamEvent], events: AsyncIterator[MessageStreamEvent],
*, *,
runtime: GatewayRuntime,
agent_name: str,
input_messages: list[MessageParam],
system: str | None,
model: str, model: str,
after: Callable[[Message], Awaitable[None]],
) -> AsyncIterator[bytes]: ) -> AsyncIterator[bytes]:
r"""Serialize an event stream to SSE; broadcast a :class:`TurnRecord` after. r"""Serialize an event stream to SSE, then hand the assembled ``Message`` on.
Each event becomes ``event: <type>\ndata: <json>\n\n`` — the shape Each event becomes ``event: <type>\ndata: <json>\n\n`` - the shape
the Anthropic SDK's SSE decoder expects. Errors mid-stream are the Anthropic SDK's SSE decoder expects. Errors mid-stream are
swallowed into a synthetic ``error`` event so the client sees the swallowed into a synthetic ``error`` event so the client sees the
failure rather than a hung connection. failure rather than a hung connection.
The same events feed a :class:`StreamAccumulator` on the side so that
once the SSE response closes we can hand a fully-assembled
``Message`` to every ``runtime.turn_log_handlers`` entry (the
markdown frontend's archive logger lives in there). Broadcast
failures are caught — they must never bubble up to the client.
""" """
acc = StreamAccumulator() acc = StreamAccumulator()
try: try:
@@ -248,14 +293,10 @@ async def _sse_and_broadcast(
) )
yield f"event: error\ndata: {err}\n\n".encode() yield f"event: error\ndata: {err}\n\n".encode()
return return
message = acc.finalize(model=model) try:
await _broadcast_turn( await after(acc.finalize(model=model))
runtime, except Exception: # noqa: BLE001
agent_name=agent_name, _log.exception("post-turn hook failed")
input_messages=input_messages,
output_message=message,
system=system,
)
async def _broadcast_turn( async def _broadcast_turn(
@@ -266,11 +307,6 @@ async def _broadcast_turn(
output_message: Message, output_message: Message,
system: str | None, system: str | None,
) -> None: ) -> None:
"""Fire each ``turn_log_handlers`` entry with a fresh :class:`TurnRecord`.
Handler exceptions are caught and logged — they're observability
plumbing, not part of the user-visible request path.
"""
if not runtime.turn_log_handlers: if not runtime.turn_log_handlers:
return return
record = TurnRecord( record = TurnRecord(
@@ -278,7 +314,7 @@ async def _broadcast_turn(
input_messages=list(input_messages), input_messages=list(input_messages),
output_message=output_message, output_message=output_message,
system=system, system=system,
source="anthropic", source=FRONTEND,
) )
for handler in runtime.turn_log_handlers: for handler in runtime.turn_log_handlers:
try: try:
@@ -0,0 +1,5 @@
"""``ApiFrontend`` - the conversations API and event stream (§3.9)."""
from beaver_gateway.frontends.api.frontend import ApiFrontend
__all__ = ["ApiFrontend"]
File diff suppressed because it is too large. Load diff
+63 -11
View File
@@ -1,27 +1,33 @@
"""Frontend ABC + the runtime context handed to ``configure``. """Frontend ABC + the runtime context handed to ``configure``.
A frontend is anything that listens on a port and routes inbound traffic A frontend is anything that routes inbound traffic into the gateway: an
into the gateway. ``GatewayRuntime`` carries everything a frontend may HTTP surface mounted under its ``path`` on the single gateway port
need that isn't user-config: built registries, per-agent backends, and (``frontends/root.py``), a poller (Telegram), or both. ``GatewayRuntime``
the in-memory token store. The user's ``/config/config.py`` defines a carries everything a frontend may need that isn't user-config: built
``Gateway`` (lists); ``cli.main`` turns that into a ``GatewayRuntime`` registries, per-agent backends, and the in-memory token store. The
and hands it to each frontend's ``configure``. user's ``/config/config.py`` defines a ``Gateway`` (lists); ``cli.main``
turns that into a ``GatewayRuntime`` and hands it to each frontend's
``configure``.
""" """
from __future__ import annotations from __future__ import annotations
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from dataclasses import dataclass, field from dataclasses import dataclass, field
from typing import TYPE_CHECKING from typing import TYPE_CHECKING, Any
if TYPE_CHECKING: if TYPE_CHECKING:
from collections.abc import Awaitable, Callable, Mapping, Sequence from collections.abc import Awaitable, Callable, Mapping, Sequence
from starlette.types import ASGIApp
from beaver_gateway.backends.base import Backend from beaver_gateway.backends.base import Backend
from beaver_gateway.core.auth import TokenStore from beaver_gateway.core.auth import TokenStore
from beaver_gateway.core.kinds import Kind
from beaver_gateway.core.registry import AgentRegistry, McpRegistry from beaver_gateway.core.registry import AgentRegistry, McpRegistry
from beaver_gateway.core.turn_record import TurnRecord from beaver_gateway.core.turn_record import TurnRecord
from beaver_gateway.storage import Database from beaver_gateway.storage import Database
from beaver_gateway.storage.models import Conversation, ConversationBinding
TurnLogHandler = Callable[[TurnRecord], Awaitable[None]] TurnLogHandler = Callable[[TurnRecord], Awaitable[None]]
@@ -36,7 +42,7 @@ class GatewayRuntime:
the name in hand, so the indirection lives one step earlier. the name in hand, so the indirection lives one step earlier.
``mcp_internal_urls`` is filled in Phase 2.1: one loopback URL per ``mcp_internal_urls`` is filled in Phase 2.1: one loopback URL per
declared ``McpServer`` so ``ClaudeCodeBackendAdapter`` (Phase 2.2) declared ``McpServer`` so ``ClaudeSdkBackend``
can pass them to ``BackendOptions.mcp_servers`` without re-running can pass them to ``BackendOptions.mcp_servers`` without re-running
discovery. discovery.
@@ -77,13 +83,59 @@ class GatewayRuntime:
# to keep ``anthropic.types`` out of the runtime import graph for # to keep ``anthropic.types`` out of the runtime import graph for
# this base module. # this base module.
turn_log_handlers: list[TurnLogHandler] = field(default_factory=list) turn_log_handlers: list[TurnLogHandler] = field(default_factory=list)
# M1b: conversations service, event bus and the shared session pool.
# ``Any`` for the same import-graph reason as above; ``None`` only in
# tests that build a runtime without them.
conversations: Any = None
bus: Any = None
pool: Any = None
scheduler: Any = None
# External origin the reverse proxy puts in front of the gateway
# (``Gateway.public_url``); ``None`` means "derive from the request".
public_url: str | None = None
class Frontend(ABC): class Frontend(ABC):
"""Listens on a port, dispatches into the gateway.""" """Routes inbound traffic into the gateway.
HTTP frontends set ``path`` and return their ASGI app from ``app()``;
``cli`` mounts every such app under that path on the one gateway
port, so ``/anthropic/v1/messages`` reaches the Anthropic frontend's
``/v1/messages``. ``serve()`` is for work outside HTTP - polling,
vault mirrors - and defaults to nothing. ``landing`` marks the app
that ``/`` redirects to (the admin console).
A frontend that shows conversations declares ``name`` (the binding
key) and ``kinds`` (which conversation kinds it shows);
``core/conversations`` refuses to bind a conversation to a frontend
outside its declaration. The first frontend in declaration order
whose ``materialize`` returns a binding is the *home* of that kind:
``spawn`` calls it so a new conversation gets a window (a vault file,
a Telegram topic). ``agent_for`` names the default agent for a kind
so callers may omit ``agent``. Stateless frontends (MCP, admin) keep
the defaults and stay outside the routing.
"""
name: str = ""
kinds: tuple[Kind, ...] = ()
path: str | None = None
landing: bool = False
@abstractmethod @abstractmethod
def configure(self, runtime: GatewayRuntime) -> None: ... def configure(self, runtime: GatewayRuntime) -> None: ...
@abstractmethod def app(self) -> ASGIApp | None:
async def serve(self) -> None: ... return None
async def serve(self) -> None:
return
def agent_for(self, kind: Kind) -> str | None: # noqa: ARG002
return None
async def materialize(self, conv: Conversation) -> ConversationBinding | None: # noqa: ARG002
return None
async def mark_closed(self, conv: Conversation) -> bool: # noqa: ARG002
"""Show in the window that the conversation is over (a renamed topic)."""
return False
@@ -38,14 +38,16 @@ if TYPE_CHECKING:
_log = logging.getLogger("beaver_gateway.frontends.markdown.crossfront") _log = logging.getLogger("beaver_gateway.frontends.markdown.crossfront")
# User hook: take a turn + vault root, return where the new file should __all__ = [
# live. Returning a relative ``Path`` is treated as relative to the "ChatPathFn",
# vault. ``None`` (the default) keeps the built-in "CrossFrontendLogger",
# ``{vault}/{logged_subdir}/{agent}/{YYYY-MM-DD}_{hex8}.md`` layout. "fingerprint_messages",
LogPathFn = "Callable[[TurnRecord, Path], Path]" "strip_trailing_user_scaffold",
]
ChatPathFn = "Callable[[str, str, Path], Path]"
__all__ = ["CrossFrontendLogger", "LogPathFn", "fingerprint_messages"] """``(title, agent, vault) -> path`` of a new chat file; relative = under the
vault. ``None`` keeps ``{vault}/{logged_subdir}/{agent}/{YYYY-MM-DD}_{slug}.md``."""
def fingerprint_messages(messages: Iterable[MessageParam]) -> str: def fingerprint_messages(messages: Iterable[MessageParam]) -> str:
@@ -96,18 +98,18 @@ class CrossFrontendLogger:
*, *,
vault_path: Path, vault_path: Path,
logged_subdir: str, logged_subdir: str,
log_path: Callable[[TurnRecord, Path], Path] | None = None, chat_path: Callable[[str, str, Path], Path] | None = None,
) -> None: ) -> None:
self._vault = vault_path self._vault = vault_path
self._root = vault_path / logged_subdir self._root = vault_path / logged_subdir
self._index: dict[str, Path] = {} self._index: dict[str, Path] = {}
self._lock = asyncio.Lock() self._lock = asyncio.Lock()
self._log_path_fn = log_path self._chat_path_fn = chat_path
# When the user supplies a custom path function, files can land # When the user supplies a custom path function, files can land
# anywhere in the vault — so we have to scan the whole vault on # anywhere in the vault — so we have to scan the whole vault on
# startup to rebuild the fingerprint→path map. With the default # startup to rebuild the fingerprint→path map. With the default
# layout we can bound the scan to ``_logs/``. # layout we can bound the scan to ``_logs/``.
self._scan_root = vault_path if log_path is not None else self._root self._scan_root = vault_path if chat_path is not None else self._root
def warm_index(self) -> None: def warm_index(self) -> None:
"""Scan logged files synchronously, populating the fingerprint map. """Scan logged files synchronously, populating the fingerprint map.
@@ -171,7 +173,7 @@ class CrossFrontendLogger:
if target.exists(): if target.exists():
existing = target.read_text(encoding="utf-8") existing = target.read_text(encoding="utf-8")
parsed = frontmatter.loads(existing) parsed = frontmatter.loads(existing)
body = _strip_trailing_user_scaffold(parsed.content) body = strip_trailing_user_scaffold(parsed.content)
# We append only the *new* user turn (the last one in # We append only the *new* user turn (the last one in
# input_messages, since prior turns are already on disk) # input_messages, since prior turns are already on disk)
# plus the assistant reply. # plus the assistant reply.
@@ -206,13 +208,15 @@ class CrossFrontendLogger:
def _new_file_path(self, record: TurnRecord) -> Path: def _new_file_path(self, record: TurnRecord) -> Path:
"""Pick a fresh filename for a brand-new conversation. """Pick a fresh filename for a brand-new conversation.
With a user-supplied ``log_path`` we delegate to it (joining a With a user-supplied ``chat_path`` we delegate to it (joining a
relative result with the vault root). Without one, we fall back relative result with the vault root). Without one, we fall back
to ``{logged_subdir}/{agent}/{date}_{hex8}.md`` and ensure the to ``{logged_subdir}/{agent}/{date}_{hex8}.md`` and ensure the
``.md`` suffix in case the user picks a non-md extension by hand. ``.md`` suffix in case the user picks a non-md extension by hand.
""" """
if self._log_path_fn is not None: if self._chat_path_fn is not None:
result = self._log_path_fn(record, self._vault) result = self._chat_path_fn(
record.first_user_text, record.agent_name, self._vault
)
if not result.is_absolute(): if not result.is_absolute():
result = self._vault / result result = self._vault / result
if result.suffix != ".md": if result.suffix != ".md":
@@ -270,7 +274,7 @@ def _render_full_history(messages: list[MessageParam], assistant: Any) -> str:
return body return body
def _strip_trailing_user_scaffold(body: str) -> str: def strip_trailing_user_scaffold(body: str) -> str:
"""Drop a trailing empty ``### User:`` block if present. """Drop a trailing empty ``### User:`` block if present.
Cross-frontend turns aren't typed into the file by the human — they Cross-frontend turns aren't typed into the file by the human — they
@@ -0,0 +1,50 @@
"""Vault file IO shared by the markdown frontend and its mirror."""
from __future__ import annotations
import contextlib
import os
import tempfile
from typing import TYPE_CHECKING, Any
import aiofile
import frontmatter
if TYPE_CHECKING:
from pathlib import Path
__all__ = ["read_or_empty", "reattach_frontmatter", "write_atomic"]
async def read_or_empty(path: Path) -> str:
if not path.exists(): # noqa: ASYNC240
return ""
async with aiofile.async_open(path, "r", encoding="utf-8") as f:
return await f.read()
async def write_atomic(path: Path, content: str) -> None:
"""Write via tmp + ``os.replace`` in the same directory: no torn reads."""
path.parent.mkdir(parents=True, exist_ok=True)
fd, tmp_path = tempfile.mkstemp(
prefix=f".{path.name}.", suffix=".tmp", dir=str(path.parent)
)
try:
async with aiofile.async_open(tmp_path, "w", encoding="utf-8") as f:
await f.write(content)
os.close(fd)
os.replace(tmp_path, path) # noqa: PTH105
except BaseException:
with contextlib.suppress(OSError):
os.close(fd)
with contextlib.suppress(OSError):
os.unlink(tmp_path) # noqa: PTH108
raise
def reattach_frontmatter(metadata: dict[str, Any], body: str) -> str:
"""Re-emit a ``.md`` with YAML frontmatter; no block at all for empty metadata."""
if not metadata:
return body if body.endswith("\n") else body + "\n"
post = frontmatter.Post(content=body, **metadata)
return frontmatter.dumps(post) + "\n"
+186 -304
View File
@@ -13,7 +13,16 @@ Concurrency model: an in-memory ``set[Path]`` of files currently in
flight. Two concurrent requests for the same file → the second gets flight. Two concurrent requests for the same file → the second gets
409. The set is single-process (one gateway instance) — that's by 409. The set is single-process (one gateway instance) — that's by
design; the markdown frontend is the only writer in its vault from design; the markdown frontend is the only writer in its vault from
the gateway side. the gateway side. The turn itself runs through ``core/conversations``
(one turn per conversation, ``running_turn`` in the DB), so a message
posted to the same conversation via ``/api`` waits its turn.
A chat file is a ``deep`` conversation bound as
``(markdown, <vault-relative path>)``; frontmatter carries only ``agent``
and ``conversation_id`` (§3.10), tool calls are never rendered. The
frontend is the home of ``deep``: ``materialize`` gives a conversation
spawned elsewhere its file, and :class:`.mirror.ChatMirror` keeps that
file in step with replies produced outside ``/chat``.
Cross-frontend logging: when ``log_all_chats=True``, ``configure()`` Cross-frontend logging: when ``log_all_chats=True``, ``configure()``
registers a handler on ``runtime.turn_log_handlers`` so every other registers a handler on ``runtime.turn_log_handlers`` so every other
@@ -24,48 +33,50 @@ shape.
from __future__ import annotations from __future__ import annotations
import asyncio
import contextlib
import json import json
import logging import logging
import os
import tempfile
import time import time
from collections.abc import AsyncIterator from collections.abc import AsyncIterator
from pathlib import Path from pathlib import Path
from typing import TYPE_CHECKING, Any from typing import TYPE_CHECKING, Any, cast
import aiofile
from anthropic.types import RawContentBlockStopEvent from anthropic.types import RawContentBlockStopEvent
from fastapi import FastAPI, HTTPException, Request, status from fastapi import FastAPI, HTTPException, Request, status
from fastapi.middleware.cors import CORSMiddleware from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse, StreamingResponse from fastapi.responses import JSONResponse, StreamingResponse
from beaver_gateway.backends.claude_code import ClaudeCodeBackendAdapter, TurnCapture
from beaver_gateway.core import audit from beaver_gateway.core import audit
from beaver_gateway.core.conversation_store import ( from beaver_gateway.core.conversation_store import (
diff_and_fork, diff_and_fork,
load_conversation,
load_messages, load_messages,
mint_conversation,
rewrite_messages, rewrite_messages,
) )
from beaver_gateway.core.turn_capture import TurnCapture
from beaver_gateway.core.turn_record import TurnRecord from beaver_gateway.core.turn_record import TurnRecord
from beaver_gateway.frontends._accumulate import StreamAccumulator from beaver_gateway.frontends._accumulate import StreamAccumulator
from beaver_gateway.frontends._auth import require_token from beaver_gateway.frontends._auth import require_token
from beaver_gateway.frontends._sse import (
KEEPALIVE,
SSE_HEADERS,
events_with_heartbeat,
sse_pack,
)
from beaver_gateway.frontends.base import Frontend from beaver_gateway.frontends.base import Frontend
from beaver_gateway.frontends.markdown import parser, renderer from beaver_gateway.frontends.markdown import parser, renderer
from beaver_gateway.frontends.markdown.crossfront import ( from beaver_gateway.frontends.markdown.crossfront import CrossFrontendLogger
CrossFrontendLogger, from beaver_gateway.frontends.markdown.files import (
fingerprint_messages, read_or_empty,
reattach_frontmatter,
write_atomic,
) )
from beaver_gateway.frontends.markdown.mirror import FRONTEND, ChatMirror
if TYPE_CHECKING: if TYPE_CHECKING:
from collections.abc import AsyncIterator, Callable from collections.abc import AsyncIterator, Callable
from anthropic.types import MessageParam from beaver_gateway.core.kinds import Kind
from beaver_gateway.frontends.base import GatewayRuntime from beaver_gateway.frontends.base import GatewayRuntime
from beaver_gateway.storage.models import Conversation, ConversationBinding
_log = logging.getLogger("beaver_gateway.frontends.markdown") _log = logging.getLogger("beaver_gateway.frontends.markdown")
@@ -88,41 +99,28 @@ _STREAM_FLUSH_DEBOUNCE = 0.4
# disk round-trip). # disk round-trip).
_SSE_FLUSH_DEBOUNCE = 0.1 _SSE_FLUSH_DEBOUNCE = 0.1
# Interval between SSE comment-frames sent when the backend is silent
# (e.g. claude is mid-thinking on a large context). The Obsidian plugin
# and any intermediate proxies will hold the connection open as long as
# bytes keep flowing; a comment-frame is the cheapest legal SSE keepalive.
# Set well under typical proxy/client idle timeouts (60s).
_SSE_HEARTBEAT_INTERVAL = 15.0
class MarkdownFrontend(Frontend): class MarkdownFrontend(Frontend):
"""FastAPI app behind ``POST /chat`` driven by Obsidian-vault files.""" """FastAPI app behind ``POST /chat`` driven by Obsidian-vault files."""
name = FRONTEND
kinds = ("deep",)
path = "/md"
def __init__( def __init__(
self, self,
*, *,
vault_path: Path | str, vault_path: Path | str,
host: str = "0.0.0.0", # noqa: S104
port: int = 8003,
default_agent: str | None = None, default_agent: str | None = None,
log_all_chats: bool = False, log_all_chats: bool = False,
logged_subdir: str = "_logs", logged_subdir: str = "_logs",
log_path: Callable[[TurnRecord, Path], Path] | None = None, chat_path: Callable[[str, str, Path], Path] | None = None,
public_base_url: str | None = None,
) -> None: ) -> None:
self.vault_path = Path(vault_path).expanduser().resolve() self.vault_path = Path(vault_path).expanduser().resolve()
self.host = host
self.port = port
self.default_agent = default_agent self.default_agent = default_agent
self.log_all_chats = log_all_chats self.log_all_chats = log_all_chats
self.logged_subdir = logged_subdir self.logged_subdir = logged_subdir
self.log_path = log_path self.chat_path = chat_path
# External URL prefix when behind a reverse proxy — same role as
# on the other bearer frontends. Trailing slash trimmed for
# idempotent concatenation; ``None`` means "no proxy / advertise
# raw host:port".
self.public_base_url = public_base_url.rstrip("/") if public_base_url else None
self._runtime: GatewayRuntime | None = None self._runtime: GatewayRuntime | None = None
self._app: FastAPI | None = None self._app: FastAPI | None = None
# Files currently being processed by an in-flight ``POST /chat``. # Files currently being processed by an in-flight ``POST /chat``.
@@ -131,15 +129,25 @@ class MarkdownFrontend(Frontend):
# request reliably loses the race to 409. # request reliably loses the race to 409.
self._busy: set[Path] = set() self._busy: set[Path] = set()
self._crossfront: CrossFrontendLogger | None = None self._crossfront: CrossFrontendLogger | None = None
self._mirror: ChatMirror | None = None
def configure(self, runtime: GatewayRuntime) -> None: def configure(self, runtime: GatewayRuntime) -> None:
if runtime.conversations is None:
msg = "MarkdownFrontend needs runtime.conversations"
raise RuntimeError(msg)
self._runtime = runtime self._runtime = runtime
self.vault_path.mkdir(parents=True, exist_ok=True) self.vault_path.mkdir(parents=True, exist_ok=True)
self._mirror = ChatMirror(
vault_path=self.vault_path,
runtime=runtime,
logged_subdir=self.logged_subdir,
chat_path=self.chat_path,
)
if self.log_all_chats: if self.log_all_chats:
self._crossfront = CrossFrontendLogger( self._crossfront = CrossFrontendLogger(
vault_path=self.vault_path, vault_path=self.vault_path,
logged_subdir=self.logged_subdir, logged_subdir=self.logged_subdir,
log_path=self.log_path, chat_path=self.chat_path,
) )
# Scan the existing logged files synchronously here so the # Scan the existing logged files synchronously here so the
# fingerprint→path map is populated before the first # fingerprint→path map is populated before the first
@@ -148,17 +156,24 @@ class MarkdownFrontend(Frontend):
runtime.turn_log_handlers.append(self._crossfront.handle) runtime.turn_log_handlers.append(self._crossfront.handle)
self._app = self._build_app(runtime) self._app = self._build_app(runtime)
async def serve(self) -> None: @property
import uvicorn def mirror(self) -> ChatMirror:
if self._mirror is None:
if self._app is None: msg = "configure() must be called first"
msg = "configure() must be called before serve()"
raise RuntimeError(msg) raise RuntimeError(msg)
config = uvicorn.Config( return self._mirror
self._app, host=self.host, port=self.port, log_level="info"
) def agent_for(self, kind: Kind) -> str | None:
server = uvicorn.Server(config) return self.default_agent if kind == "deep" else None
await server.serve()
async def materialize(self, conv: Conversation) -> ConversationBinding | None:
return await self.mirror.materialize(conv)
def app(self) -> FastAPI | None:
return self._app
async def serve(self) -> None:
await self.mirror.run()
# ---- app builder --------------------------------------------------- # ---- app builder ---------------------------------------------------
@@ -285,17 +300,7 @@ class MarkdownFrontend(Frontend):
self._busy.discard(file_path) self._busy.discard(file_path)
return StreamingResponse( return StreamingResponse(
gen(), gen(), media_type="text/event-stream", headers=SSE_HEADERS
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache, no-transform",
"Connection": "keep-alive",
# nginx default-buffers SSE bodies; this header tells
# both nginx and uvicorn-behind-proxy to flush as we
# write. Harmless if the deployment has no reverse
# proxy in front.
"X-Accel-Buffering": "no",
},
) )
return app return app
@@ -312,11 +317,11 @@ class MarkdownFrontend(Frontend):
content_override: Any, content_override: Any,
agent_override: str | None, agent_override: str | None,
) -> Any: ) -> Any:
write_disk = content_override is None
if isinstance(content_override, str): if isinstance(content_override, str):
await _write_atomic(file_path, content_override)
file_text = content_override file_text = content_override
elif content_override is None: elif content_override is None:
file_text = await _read_or_empty(file_path) file_text = await read_or_empty(file_path)
else: else:
raise HTTPException( raise HTTPException(
status.HTTP_400_BAD_REQUEST, "`content` must be a string when present" status.HTTP_400_BAD_REQUEST, "`content` must be a string when present"
@@ -381,22 +386,22 @@ class MarkdownFrontend(Frontend):
# Resolve / mint the conversation row, align incoming against # Resolve / mint the conversation row, align incoming against
# stored history, and feed the aligned messages to the backend # stored history, and feed the aligned messages to the backend
# — see ``core/conversation_store.py`` for the full rationale. # - see ``core/conversation_store.py`` for the full rationale.
# If the backend isn't claude-code (no ``TurnCapture`` support)
# we fall through to the legacy parser-only path.
conv, conv_external_id, stored_msgs = await self._resolve_conversation( conv, conv_external_id, stored_msgs = await self._resolve_conversation(
runtime=runtime, metadata=parsed.metadata, agent_name=agent.name runtime=runtime,
metadata=parsed.metadata,
agent_name=agent.name,
file_path=file_path,
first_user_text=parsed.turns[0].text,
) )
outcome = diff_and_fork(stored=stored_msgs, incoming=parsed.turns) outcome = diff_and_fork(stored=stored_msgs, incoming=parsed.turns)
capture: TurnCapture | None = ( capture = TurnCapture()
TurnCapture() if isinstance(backend, ClaudeCodeBackendAdapter) else None events = runtime.conversations.turn(
) conv,
messages=outcome.messages,
kwargs: dict[str, Any] = {} origin="user",
if capture is not None: capture=capture,
kwargs["capture"] = capture use_session=outcome.reuse_session,
events = backend.complete(
agent=agent, messages=outcome.messages, system=None, **kwargs
) )
try: try:
message = await self._stream_to_file( message = await self._stream_to_file(
@@ -405,6 +410,7 @@ class MarkdownFrontend(Frontend):
parsed=parsed, parsed=parsed,
model=agent.model or agent.name, model=agent.model or agent.name,
filename=filename, filename=filename,
write_disk=write_disk,
) )
except HTTPException: except HTTPException:
raise raise
@@ -419,11 +425,12 @@ class MarkdownFrontend(Frontend):
message=message, message=message,
agent_name=agent.name, agent_name=agent.name,
conv_external_id=conv_external_id, conv_external_id=conv_external_id,
write_disk=write_disk,
) )
await self._persist_canonical_history( await self._persist_canonical_history(
runtime=runtime, runtime=runtime,
conversation_id=conv.id, conversation_id=cast("int", conv.id),
persist_messages=outcome.persist_messages, persist_messages=outcome.persist_messages,
new_user_text=parsed.turns[-1].text, new_user_text=parsed.turns[-1].text,
capture=capture, capture=capture,
@@ -480,17 +487,15 @@ class MarkdownFrontend(Frontend):
writers in their respective halves of Obsidian Sync. Final writers in their respective halves of Obsidian Sync. Final
content is identical on both sides, so Sync no-ops. content is identical on both sides, so Sync no-ops.
""" """
# File-text resolution + early bailouts. ``content_override`` is # With ``content`` the plugin is the only writer of the file
# still written to disk on the gateway side because that's the # (§3.10): the gateway never touches disk in that case.
# state the rest of the request consumes; it just doesn't keep write_disk = content_override is None
# ticking after that single write.
if isinstance(content_override, str): if isinstance(content_override, str):
await _write_atomic(file_path, content_override)
file_text = content_override file_text = content_override
elif content_override is None: elif content_override is None:
file_text = await _read_or_empty(file_path) file_text = await read_or_empty(file_path)
else: else:
yield _sse_pack( yield sse_pack(
"error", "error",
{ {
"status_code": status.HTTP_400_BAD_REQUEST, "status_code": status.HTTP_400_BAD_REQUEST,
@@ -506,7 +511,7 @@ class MarkdownFrontend(Frontend):
default=self.default_agent, default=self.default_agent,
) )
if not agent_name: if not agent_name:
yield _sse_pack( yield sse_pack(
"error", "error",
{ {
"status_code": status.HTTP_400_BAD_REQUEST, "status_code": status.HTTP_400_BAD_REQUEST,
@@ -519,7 +524,7 @@ class MarkdownFrontend(Frontend):
return return
if not parsed.messages: if not parsed.messages:
yield _sse_pack( yield sse_pack(
"done", "done",
{ {
"status": "nothing_to_do", "status": "nothing_to_do",
@@ -530,7 +535,7 @@ class MarkdownFrontend(Frontend):
return return
if parser.last_role(parsed.messages) == "assistant": if parser.last_role(parsed.messages) == "assistant":
yield _sse_pack( yield sse_pack(
"done", "done",
{ {
"status": "nothing_to_do", "status": "nothing_to_do",
@@ -542,7 +547,7 @@ class MarkdownFrontend(Frontend):
agent = runtime.agents.get(agent_name) agent = runtime.agents.get(agent_name)
if agent is None: if agent is None:
yield _sse_pack( yield sse_pack(
"error", "error",
{ {
"status_code": status.HTTP_404_NOT_FOUND, "status_code": status.HTTP_404_NOT_FOUND,
@@ -552,7 +557,7 @@ class MarkdownFrontend(Frontend):
return return
backend = runtime.backends.get(agent.name) backend = runtime.backends.get(agent.name)
if backend is None: if backend is None:
yield _sse_pack( yield sse_pack(
"error", "error",
{ {
"status_code": status.HTTP_503_SERVICE_UNAVAILABLE, "status_code": status.HTTP_503_SERVICE_UNAVAILABLE,
@@ -577,9 +582,19 @@ class MarkdownFrontend(Frontend):
msgs=len(parsed.messages), msgs=len(parsed.messages),
) )
conv, conv_external_id, stored_msgs = await self._resolve_conversation( try:
runtime=runtime, metadata=parsed.metadata, agent_name=agent.name conv, conv_external_id, stored_msgs = await self._resolve_conversation(
) runtime=runtime,
metadata=parsed.metadata,
agent_name=agent.name,
file_path=file_path,
first_user_text=parsed.turns[0].text,
)
except HTTPException as exc:
yield sse_pack(
"error", {"status_code": exc.status_code, "detail": exc.detail}
)
return
_log.info( _log.info(
"chat/stream: file=%s conv_external_id=%s conv_id=%d " "chat/stream: file=%s conv_external_id=%s conv_id=%d "
"stored_msgs=%d incoming_turns=%d", "stored_msgs=%d incoming_turns=%d",
@@ -598,21 +613,19 @@ class MarkdownFrontend(Frontend):
len(outcome.messages), len(outcome.messages),
len(outcome.persist_messages), len(outcome.persist_messages),
) )
capture: TurnCapture | None = ( capture = TurnCapture()
TurnCapture() if isinstance(backend, ClaudeCodeBackendAdapter) else None
)
kwargs: dict[str, Any] = {}
if capture is not None:
kwargs["capture"] = capture
_log.info( _log.info(
"chat/stream: file=%s calling backend.complete agent=%s capture=%s", "chat/stream: file=%s calling backend.complete agent=%s session=%s",
filename, filename,
agent.name, agent.name,
capture is not None, conv.session_id,
) )
events = backend.complete( events = runtime.conversations.turn(
agent=agent, messages=outcome.messages, system=None, **kwargs conv,
messages=outcome.messages,
origin="user",
capture=capture,
use_session=outcome.reuse_session,
) )
acc = StreamAccumulator() acc = StreamAccumulator()
@@ -626,16 +639,12 @@ class MarkdownFrontend(Frontend):
return None return None
rendered = renderer.render_assistant_message(partial) rendered = renderer.render_assistant_message(partial)
new_body = renderer.append_to_body(parsed.body, rendered) new_body = renderer.append_to_body(parsed.body, rendered)
return _reattach_frontmatter(parsed.metadata, new_body) return reattach_frontmatter(parsed.metadata, new_body)
try: try:
async for ev in _events_with_heartbeat(events): async for ev in events_with_heartbeat(events):
if ev is None: if ev is None:
# Backend is quiet (claude mid-thinking, MCP slow, yield KEEPALIVE
# whatever). SSE comment-frame keeps the TCP socket
# warm so the plugin / uvicorn / any reverse proxy
# doesn't time the request out before we finish.
yield b": keepalive\n\n"
continue continue
acc.feed(ev) acc.feed(ev)
now = time.monotonic() now = time.monotonic()
@@ -648,7 +657,7 @@ class MarkdownFrontend(Frontend):
# render to the same prefix as before they closed # render to the same prefix as before they closed
# (we don't surface the tool-call args in markdown). # (we don't surface the tool-call args in markdown).
if payload is not None and payload != last_payload: if payload is not None and payload != last_payload:
yield _sse_pack("delta", {"new_content": payload}) yield sse_pack("delta", {"new_content": payload})
last_payload = payload last_payload = payload
last_flush = now last_flush = now
except Exception as exc: # noqa: BLE001 — wire any backend failure as an SSE error frame except Exception as exc: # noqa: BLE001 — wire any backend failure as an SSE error frame
@@ -663,10 +672,11 @@ class MarkdownFrontend(Frontend):
new_body, renderer.render_assistant_message(partial) new_body, renderer.render_assistant_message(partial)
) )
new_body = renderer.append_to_body(new_body, _render_error_block(exc)) new_body = renderer.append_to_body(new_body, _render_error_block(exc))
await _write_atomic( if write_disk:
file_path, _reattach_frontmatter(parsed.metadata, new_body) await write_atomic(
) file_path, reattach_frontmatter(parsed.metadata, new_body)
yield _sse_pack( )
yield sse_pack(
"error", "error",
{ {
"status_code": status.HTTP_500_INTERNAL_SERVER_ERROR, "status_code": status.HTTP_500_INTERNAL_SERVER_ERROR,
@@ -683,11 +693,12 @@ class MarkdownFrontend(Frontend):
message=message, message=message,
agent_name=agent.name, agent_name=agent.name,
conv_external_id=conv_external_id, conv_external_id=conv_external_id,
write_disk=write_disk,
) )
await self._persist_canonical_history( await self._persist_canonical_history(
runtime=runtime, runtime=runtime,
conversation_id=conv.id, conversation_id=cast("int", conv.id),
persist_messages=outcome.persist_messages, persist_messages=outcome.persist_messages,
new_user_text=parsed.turns[-1].text, new_user_text=parsed.turns[-1].text,
capture=capture, capture=capture,
@@ -707,7 +718,7 @@ class MarkdownFrontend(Frontend):
except Exception: # noqa: BLE001 except Exception: # noqa: BLE001
_log.exception("turn_log_handler raised; continuing") _log.exception("turn_log_handler raised; continuing")
yield _sse_pack( yield sse_pack(
"done", "done",
{ {
"status": "ok", "status": "ok",
@@ -727,6 +738,7 @@ class MarkdownFrontend(Frontend):
parsed: parser.ParsedFile, parsed: parser.ParsedFile,
model: str, model: str,
filename: str, filename: str,
write_disk: bool = True,
) -> Any: ) -> Any:
"""Drain ``events`` into a ``Message``, flushing partials to disk. """Drain ``events`` into a ``Message``, flushing partials to disk.
@@ -745,13 +757,15 @@ class MarkdownFrontend(Frontend):
acc = StreamAccumulator() acc = StreamAccumulator()
async def flush_partial() -> None: async def flush_partial() -> None:
if not write_disk:
return
partial = acc.finalize(model=model) partial = acc.finalize(model=model)
if not partial.content: if not partial.content:
return return
rendered = renderer.render_assistant_message(partial) rendered = renderer.render_assistant_message(partial)
new_body = renderer.append_to_body(parsed.body, rendered) new_body = renderer.append_to_body(parsed.body, rendered)
await _write_atomic( await write_atomic(
file_path, _reattach_frontmatter(parsed.metadata, new_body) file_path, reattach_frontmatter(parsed.metadata, new_body)
) )
try: try:
@@ -774,9 +788,10 @@ class MarkdownFrontend(Frontend):
new_body, renderer.render_assistant_message(partial) new_body, renderer.render_assistant_message(partial)
) )
new_body = renderer.append_to_body(new_body, _render_error_block(exc)) new_body = renderer.append_to_body(new_body, _render_error_block(exc))
await _write_atomic( if write_disk:
file_path, _reattach_frontmatter(parsed.metadata, new_body) await write_atomic(
) file_path, reattach_frontmatter(parsed.metadata, new_body)
)
raise raise
return acc.finalize(model=model) return acc.finalize(model=model)
@@ -788,75 +803,70 @@ class MarkdownFrontend(Frontend):
message: Any, message: Any,
agent_name: str, agent_name: str,
conv_external_id: str, conv_external_id: str,
write_disk: bool = True,
) -> str: ) -> str:
"""Render the assistant turn, append to the file, refresh frontmatter.""" """Render the assistant turn, refresh frontmatter, write if we own the file."""
rendered = renderer.render_assistant_message(message) rendered = renderer.render_assistant_message(message)
new_body = renderer.append_to_body(parsed.body, rendered) new_body = renderer.append_to_body(parsed.body, rendered)
new_body = renderer.append_to_body(new_body, renderer.USER_SCAFFOLD) new_body = renderer.append_to_body(new_body, renderer.USER_SCAFFOLD)
# Recompute fingerprint so a future cross-frontend hit on this
# same conversation can find it. Stored as hex string in
# frontmatter — only the markdown frontend reads it.
assistant_param: MessageParam = {
"role": "assistant",
"content": _flatten_assistant_text(message),
}
updated_messages: list[MessageParam] = [*parsed.messages, assistant_param]
updated_metadata = dict(parsed.metadata) updated_metadata = dict(parsed.metadata)
updated_metadata.pop("fingerprint", None)
updated_metadata["agent"] = agent_name updated_metadata["agent"] = agent_name
updated_metadata["conversation_id"] = conv_external_id updated_metadata["conversation_id"] = conv_external_id
updated_metadata["fingerprint"] = fingerprint_messages(updated_messages) new_content = reattach_frontmatter(updated_metadata, new_body)
new_content = _reattach_frontmatter(updated_metadata, new_body) if write_disk:
await _write_atomic(file_path, new_content) await write_atomic(file_path, new_content)
return new_content return new_content
async def _resolve_conversation( async def _resolve_conversation(
self, *, runtime: GatewayRuntime, metadata: dict[str, Any], agent_name: str self,
) -> tuple[Any, str, list[dict[str, Any]]]: *,
"""Resolve the conversation row + stored messages for this request. runtime: GatewayRuntime,
metadata: dict[str, Any],
agent_name: str,
file_path: Path,
first_user_text: str = "",
) -> tuple[Conversation, str, list[dict[str, Any]]]:
"""Resolve the ``deep`` conversation for this file + its stored messages.
Looks up by frontmatter ``conversation_id``, mints a new row if Frontmatter ``conversation_id`` wins; a file that lost it is found
missing, and returns ``(conv, external_id, stored_messages)``. by its visible ``(markdown, path)`` binding, then by adopting the
``conv.id`` is guaranteed non-None because both one unbound pre-SDK conversation that starts with the same prompt;
``load_conversation`` (after refresh on a committed row) and otherwise a new conversation is created. The binding follows the
``mint_conversation`` (post-commit refresh) populate it. We file: a moved chat re-binds to its new path on the next turn.
coerce with a runtime check so the rest of the handler can
treat it as ``int``.
""" """
conversations = runtime.conversations
rel = file_path.relative_to(self.vault_path).as_posix()
raw = metadata.get("conversation_id") raw = metadata.get("conversation_id")
lookup_id = raw if isinstance(raw, str) and raw else None conv = await conversations.get(raw) if isinstance(raw, str) and raw else None
if conv is None:
conv = await conversations.find_bound(frontend=FRONTEND, external_id=rel)
if conv is None:
conv = await conversations.adopt(
kind="deep", first_user_text=first_user_text
)
if conv is not None:
_log.info("adopted conversation %s for %s", conv.external_id, rel)
if conv is None:
try:
conv = await conversations.create(
kind="deep", agent=agent_name, origin=FRONTEND, title=file_path.stem
)
except ValueError as exc:
raise HTTPException(status.HTTP_400_BAD_REQUEST, str(exc)) from exc
_log.info("minted conversation %s for %s", conv.external_id, rel)
bound = [
b
for b in await conversations.bindings(conv)
if b.frontend == FRONTEND and b.visible and b.external_id == rel
]
if not bound:
await conversations.bind(conv, frontend=FRONTEND, external_id=rel)
await conversations.touch_user(conv)
if conv.id is None:
msg = "conversation row missing primary key after commit"
raise RuntimeError(msg)
async with runtime.db.session() as session: async with runtime.db.session() as session:
conv = None
if lookup_id is not None:
conv = await load_conversation(
session, frontend="markdown", external_id=lookup_id
)
if conv is None:
_log.info(
"_resolve_conversation: frontmatter conv_id=%s "
"not found in DB, will mint new",
lookup_id,
)
else:
_log.info(
"_resolve_conversation: LOADED existing conv "
"id=%d external_id=%s",
conv.id or -1,
conv.external_id,
)
if conv is None:
conv = await mint_conversation(
session, frontend="markdown", agent_name=agent_name
)
_log.info(
"_resolve_conversation: MINTED new conv "
"id=%d external_id=%s agent=%s",
conv.id or -1,
conv.external_id,
agent_name,
)
if conv.id is None:
msg = "conversation row missing primary key after commit"
raise RuntimeError(msg)
stored = await load_messages(session, conversation_id=conv.id) stored = await load_messages(session, conversation_id=conv.id)
return conv, conv.external_id, stored return conv, conv.external_id, stored
@@ -867,21 +877,17 @@ class MarkdownFrontend(Frontend):
conversation_id: int, conversation_id: int,
persist_messages: list[dict[str, Any]], persist_messages: list[dict[str, Any]],
new_user_text: str, new_user_text: str,
capture: TurnCapture | None, capture: TurnCapture,
message: Any, message: Any,
) -> None: ) -> None:
"""Stamp the DB with the post-turn canonical Anthropic-shape history. """Stamp the DB with the post-turn canonical Anthropic-shape history.
Combines the matched/spliced prior state, the new user prompt, Combines the matched/spliced prior state, the new user prompt,
and the synthesized assistant↔tool cycle from the backend (or and the synthesized assistant/tool cycle from the backend (or a
a text-only fallback for backends without ``TurnCapture``). text-only fallback for backends that left ``capture`` empty).
""" """
new_user_msg = {"role": "user", "content": new_user_text} new_user_msg = {"role": "user", "content": new_user_text}
synthesized = ( synthesized = capture.synthesized_messages or _fallback_synthesized(message)
capture.synthesized_messages
if capture is not None
else _fallback_synthesized(message)
)
canonical = [*persist_messages, new_user_msg, *synthesized] canonical = [*persist_messages, new_user_msg, *synthesized]
_log.info( _log.info(
"_persist_canonical_history: conv_id=%d writing %d msgs " "_persist_canonical_history: conv_id=%d writing %d msgs "
@@ -919,115 +925,6 @@ class MarkdownFrontend(Frontend):
return candidate return candidate
# ---- module-level utilities ----------------------------------------------
async def _events_with_heartbeat(
events: AsyncIterator[Any], interval: float = _SSE_HEARTBEAT_INTERVAL
) -> AsyncIterator[Any]:
"""Wrap an async event stream with idle-time heartbeat markers.
Yields ``None`` every ``interval`` seconds during silence; real
events pass through unchanged. When the wrapped iterator is
exhausted, this generator returns. Cancellation propagates: if the
outer scope is cancelled we cancel the pending ``__anext__`` task
instead of leaving it dangling.
"""
src = events.__aiter__()
next_task: asyncio.Task[Any] | None = None
try:
while True:
# Reuse the in-flight task across timeouts. Spawning a fresh
# ``__anext__()`` while the previous one is still pending
# puts two consumers on the same async generator — that
# raises ``RuntimeError: anext(): asynchronous generator is
# already running``.
if next_task is None:
next_task = asyncio.ensure_future(src.__anext__())
done, _pending = await asyncio.wait({next_task}, timeout=interval)
if not done:
yield None
continue
task = next_task
next_task = None
try:
result = task.result()
except StopAsyncIteration:
return
yield result
finally:
if next_task is not None and not next_task.done():
next_task.cancel()
with contextlib.suppress(BaseException):
await next_task
def _sse_pack(event: str, data: dict[str, Any]) -> bytes:
r"""Format one Server-Sent Event frame.
Uses named events (``event: <name>``) so the plugin can dispatch on
type without parsing JSON discriminators. ``ensure_ascii=False`` so
multibyte content rides through verbatim instead of becoming
``\uXXXX`` blobs that bloat the wire.
"""
body = json.dumps(data, ensure_ascii=False)
return f"event: {event}\ndata: {body}\n\n".encode()
async def _read_or_empty(path: Path) -> str:
"""Return file contents, or empty string if the file doesn't exist."""
# ``path.exists()`` here is a metadata stat — microseconds — and
# gating an async read on whether the file is there is exactly the
# check we want. Switching to anyio.Path / aiofiles.os just to
# silence the async-pathlib lint would cost a dep edge for no
# practical win.
if not path.exists(): # noqa: ASYNC240
return ""
async with aiofile.async_open(path, "r", encoding="utf-8") as f:
return await f.read()
async def _write_atomic(path: Path, content: str) -> None:
"""Write ``content`` to ``path`` via tmp + ``os.replace`` (atomic)."""
path.parent.mkdir(parents=True, exist_ok=True)
# ``NamedTemporaryFile`` keeps the file open which complicates
# ``os.replace`` on some platforms. Build the tmp name manually,
# write+fsync, then rename. Same-directory so the rename is atomic.
tmp_name = tempfile.mkstemp(
prefix=f".{path.name}.", suffix=".tmp", dir=str(path.parent)
)
fd, tmp_path = tmp_name
try:
async with aiofile.async_open(tmp_path, "w", encoding="utf-8") as f:
await f.write(content)
os.close(fd)
# ``os.replace`` is the atomic primitive — ``Path.replace`` is a
# thin wrapper around the same syscall; either works, ``os.`` is
# the one Linux/POSIX docs reach for.
os.replace(tmp_path, path) # noqa: PTH105
except BaseException:
# Cleanup on failure: close fd, remove tmp.
with contextlib.suppress(OSError):
os.close(fd)
with contextlib.suppress(OSError):
os.unlink(tmp_path) # noqa: PTH108
raise
def _reattach_frontmatter(metadata: dict[str, Any], body: str) -> str:
r"""Re-emit a ``.md`` file with YAML frontmatter at the top.
Empty metadata → no frontmatter block (avoid littering every file
with a hollow ``---\n---``).
"""
if not metadata:
return body if body.endswith("\n") else body + "\n"
import frontmatter as _fm
post = _fm.Post(content=body, **metadata)
return _fm.dumps(post) + "\n"
def _fallback_synthesized(message: Any) -> list[dict[str, Any]]: def _fallback_synthesized(message: Any) -> list[dict[str, Any]]:
"""Build a single-assistant ``synthesized_messages`` list from a raw ``Message``. """Build a single-assistant ``synthesized_messages`` list from a raw ``Message``.
@@ -1065,21 +962,6 @@ def _fallback_synthesized(message: Any) -> list[dict[str, Any]]:
return [{"role": "assistant", "content": content}] return [{"role": "assistant", "content": content}]
def _flatten_assistant_text(message: Any) -> str:
"""Pull all text blocks from an assistant ``Message`` and join them.
Used when we need the assistant content as a plain string for
fingerprinting / equality with a parser-shaped history (parser
already drops thinking + tool_use from assistant turns).
"""
chunks = [
getattr(block, "text", "") or ""
for block in getattr(message, "content", ())
if getattr(block, "type", None) == "text"
]
return "\n\n".join(c for c in chunks if c)
def _render_error_block(exc: BaseException) -> str: def _render_error_block(exc: BaseException) -> str:
"""Render a backend failure as an Assistant turn with a ``[!error]-`` callout.""" """Render a backend failure as an Assistant turn with a ``[!error]-`` callout."""
msg = str(exc) or exc.__class__.__name__ msg = str(exc) or exc.__class__.__name__
@@ -0,0 +1,169 @@
"""Vault files for ``deep`` conversations that were not typed into a file (§3.10).
``materialize`` is the markdown frontend's answer to ``spawn(kind=deep)``:
a new file in the vault with ``agent`` + ``conversation_id`` frontmatter
and the ``(markdown, path)`` binding. ``run`` tails the gateway bus and
appends every ``reply`` of a markdown-bound conversation to its file -
the seed turn of a spawn, a message posted through ``/api``, a turn
that came in over ``/v1/messages`` - and stamps the same exchange into
the canonical history, so a continuation typed in Obsidian aligns
against the store and resumes the same SDK session instead of reseeding.
"""
from __future__ import annotations
import asyncio
import logging
from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any
import frontmatter
from beaver_gateway.core.conversation_store import load_messages, rewrite_messages
from beaver_gateway.core.turn_record import slugify
from beaver_gateway.frontends.markdown import renderer
from beaver_gateway.frontends.markdown.crossfront import strip_trailing_user_scaffold
from beaver_gateway.frontends.markdown.files import (
read_or_empty,
reattach_frontmatter,
write_atomic,
)
if TYPE_CHECKING:
from collections.abc import Callable, Sequence
from pathlib import Path
from beaver_gateway.core.bus import Event
from beaver_gateway.frontends.base import GatewayRuntime
from beaver_gateway.storage.models import Conversation, ConversationBinding
__all__ = ["FRONTEND", "ChatMirror"]
_log = logging.getLogger("beaver_gateway.frontends.markdown.mirror")
FRONTEND = "markdown"
class ChatMirror:
def __init__(
self,
*,
vault_path: Path,
runtime: GatewayRuntime,
logged_subdir: str,
chat_path: Callable[[str, str, Path], Path] | None = None,
) -> None:
self._vault = vault_path
self._runtime = runtime
self._root = vault_path / logged_subdir
self._chat_path_fn = chat_path
self._lock = asyncio.Lock()
def path_for(self, title: str, agent: str) -> Path:
if self._chat_path_fn is not None:
path = self._chat_path_fn(title, agent, self._vault)
if not path.is_absolute():
path = self._vault / path
else:
day = datetime.now(UTC).strftime("%Y-%m-%d")
path = self._root / agent / f"{day}_{slugify(title, maxlen=60)}.md"
if path.suffix != ".md":
path = path.with_suffix(".md")
path = path.resolve()
path.relative_to(self._vault)
candidate, n = path, 1
while candidate.exists():
n += 1
candidate = path.with_name(f"{path.stem} ({n}){path.suffix}")
return candidate
async def materialize(self, conv: Conversation) -> ConversationBinding:
path = self.path_for(conv.title or conv.external_id, conv.agent_name)
rel = path.relative_to(self._vault).as_posix()
async with self._lock:
await write_atomic(path, reattach_frontmatter(_frontmatter(conv), ""))
binding = await self._runtime.conversations.bind(
conv, frontend=FRONTEND, external_id=rel
)
_log.info("materialized %s as %s", conv.external_id, rel)
return binding
async def bound_path(self, conv: Conversation) -> Path | None:
for b in await self._runtime.conversations.bindings(conv):
if b.frontend == FRONTEND and b.visible:
return self._vault / b.external_id
return None
async def run(self) -> None:
async for event in self._runtime.bus.stream():
if event.get("type") != "reply":
continue
try:
await self._on_reply(event)
except Exception: # noqa: BLE001
_log.exception("mirror of reply %s failed", event.get("turn_id"))
async def _on_reply(self, event: Event) -> None:
conv = await self._runtime.conversations.get(str(event["conversation_id"]))
if conv is None:
return
await self.append(
conv,
prompt=str(event.get("prompt") or ""),
text=str(event.get("text") or ""),
history=event.get("history"),
)
async def append(
self,
conv: Conversation,
*,
prompt: str,
text: str,
history: Sequence[dict[str, Any]] | None = None,
) -> None:
path = await self.bound_path(conv)
if path is None:
return
prior = [
{"role": str(h["role"]), "content": str(h["text"])}
for h in history or ()
if h.get("text")
]
async with self._lock:
parsed = frontmatter.loads(await read_or_empty(path))
body = strip_trailing_user_scaffold(parsed.content)
if prior and "### " not in body:
for m in prior:
body = renderer.append_to_body(body, _render(m))
body = renderer.append_to_body(body, renderer.render_user_text(prompt))
body = renderer.append_to_body(body, renderer.render_assistant_text(text))
body = renderer.append_to_body(body, renderer.USER_SCAFFOLD)
metadata = {**parsed.metadata, **_frontmatter(conv)}
metadata.pop("fingerprint", None)
await write_atomic(path, reattach_frontmatter(metadata, body))
await self._persist(conv, prior=prior, prompt=prompt, text=text)
async def _persist(
self, conv: Conversation, *, prior: list[dict[str, Any]], prompt: str, text: str
) -> None:
if conv.id is None:
return
async with self._runtime.db.session() as session:
stored = await load_messages(session, conversation_id=conv.id)
canonical = [
*(stored or prior),
{"role": "user", "content": prompt},
{"role": "assistant", "content": [{"type": "text", "text": text}]},
]
await rewrite_messages(session, conversation_id=conv.id, messages=canonical)
def _frontmatter(conv: Conversation) -> dict[str, Any]:
return {"agent": conv.agent_name, "conversation_id": conv.external_id}
def _render(message: dict[str, Any]) -> str:
if message["role"] == "user":
return renderer.render_user_text(message["content"])
return renderer.render_assistant_text(message["content"])
@@ -9,11 +9,10 @@ from other frontends).
from __future__ import annotations from __future__ import annotations
import json
import re import re
from typing import TYPE_CHECKING, Any, cast from typing import TYPE_CHECKING, Any, cast
from anthropic.types import Message, TextBlock, ThinkingBlock, ToolUseBlock from anthropic.types import Message, TextBlock, ThinkingBlock
if TYPE_CHECKING: if TYPE_CHECKING:
from collections.abc import Iterable from collections.abc import Iterable
@@ -34,7 +33,9 @@ __all__ = [
# Empty ``### User:`` block appended after each assistant reply so the # Empty ``### User:`` block appended after each assistant reply so the
# human has an obvious place to type the next turn. Parser drops empty # human has an obvious place to type the next turn. Parser drops empty
# user blocks, so this doesn't re-trigger dispatch on its own. # user blocks, so this doesn't re-trigger dispatch on its own.
USER_SCAFFOLD = "### User:\n" # Blank line after the header, like every rendered turn - the file stays
# symmetric whether the human or the gateway wrote the marker.
USER_SCAFFOLD = "### User:\n\n"
# Default 4-backtick fence so tool results that contain literal ```` ``` ```` # Default 4-backtick fence so tool results that contain literal ```` ``` ````
@@ -54,6 +55,11 @@ def render_user_text(content: str) -> str:
return f"### User:\n\n{content.strip()}\n" return f"### User:\n\n{content.strip()}\n"
def render_assistant_text(text: str) -> str:
r"""Render a text-only assistant turn as ``### Assistant:\n\n<text>``."""
return f"### Assistant:\n\n{text.strip()}\n"
def render_assistant_message(message: Message) -> str: def render_assistant_message(message: Message) -> str:
"""Render an assistant ``Message`` (with content blocks) into a turn block. """Render an assistant ``Message`` (with content blocks) into a turn block.
@@ -61,16 +67,20 @@ def render_assistant_message(message: Message) -> str:
* ``ThinkingBlock`` → ``> [!thinking]-`` collapsed callout * ``ThinkingBlock`` → ``> [!thinking]-`` collapsed callout
* ``TextBlock`` → plain text (the spoken answer) * ``TextBlock`` → plain text (the spoken answer)
* ``ToolUseBlock`` → ``> [!tool]- <name>`` callout with the ``input`` * ``ToolUseBlock`` → nothing (§3.10: tool calls never reach the file;
JSON quoted inside. Tool *results* are not persisted — see "what the agent is doing" is the activity panel fed by SSE)
module docstring on ``parser.py`` for why.
Blank lines separate adjacent blocks; trailing newline guarantees Blank lines separate adjacent blocks; trailing newline guarantees
the next ``---`` / ``### User:`` marker lands on its own line. the next ``---`` / ``### User:`` marker lands on its own line.
""" """
parts: list[str] = ["### Assistant:", ""] parts: list[str] = ["### Assistant:", ""]
for block in message.content: for block in message.content:
parts.extend(_render_block(block)) lines = list(_render_block(block))
if not lines:
# Tool calls render to nothing - no separator for them either,
# or every tool leaves a blank line behind.
continue
parts.extend(lines)
parts.append("") parts.append("")
return "\n".join(parts).rstrip() + "\n" return "\n".join(parts).rstrip() + "\n"
@@ -147,10 +157,7 @@ def _render_block(block: object) -> Iterable[str]:
if isinstance(block, ThinkingBlock): if isinstance(block, ThinkingBlock):
yield from _render_thinking(block.thinking or "") yield from _render_thinking(block.thinking or "")
return return
if isinstance(block, ToolUseBlock): # Tool-use blocks and unknown block types never reach the file.
yield from _render_tool_use(block)
return
# Unknown block type — skip silently rather than corrupting the file.
def _render_thinking(text: str) -> Iterable[str]: def _render_thinking(text: str) -> Iterable[str]:
@@ -159,17 +166,6 @@ def _render_thinking(text: str) -> Iterable[str]:
yield f"> {line}" if line else ">" yield f"> {line}" if line else ">"
def _render_tool_use(block: ToolUseBlock) -> Iterable[str]:
title = summarize_tool_input(block.name, block.input)
yield f"> [!tool]- {title}"
yield "> **input:**"
yield "> ```json"
pretty = json.dumps(block.input, indent=2, ensure_ascii=False, sort_keys=True)
for line in pretty.splitlines():
yield f"> {line}" if line else ">"
yield "> ```"
def adaptive_fence(content: str) -> str: def adaptive_fence(content: str) -> str:
"""Return a backtick fence at least one longer than the longest run in ``content``. """Return a backtick fence at least one longer than the longest run in ``content``.
+16 -76
View File
@@ -37,7 +37,6 @@ from __future__ import annotations
import asyncio import asyncio
import logging import logging
from contextlib import asynccontextmanager
from typing import TYPE_CHECKING, Any from typing import TYPE_CHECKING, Any
from urllib.parse import urlencode from urllib.parse import urlencode
@@ -47,6 +46,7 @@ from starlette.responses import HTMLResponse, JSONResponse, StreamingResponse
from starlette.routing import Route from starlette.routing import Route
from beaver_gateway.core import audit from beaver_gateway.core import audit
from beaver_gateway.frontends._urls import external_base
from beaver_gateway.frontends.base import Frontend from beaver_gateway.frontends.base import Frontend
from beaver_gateway.mcp.internal_app import ALL_NAMESPACE from beaver_gateway.mcp.internal_app import ALL_NAMESPACE
@@ -94,76 +94,31 @@ __all__ = ["McpServerFrontend"]
class McpServerFrontend(Frontend): class McpServerFrontend(Frontend):
"""Auth + audit + reverse-proxy in front of the internal MCP aggregator.""" """Auth + audit + reverse-proxy in front of the internal MCP aggregator."""
def __init__( path = "/mcp"
self,
*, def __init__(self) -> None:
host: str = "0.0.0.0", # noqa: S104
port: int = 8001,
public_base_url: str | None = None,
) -> None:
self.host = host
self.port = port
# External URL prefix the reverse proxy uses to reach this
# frontend. Internal routes mount namespaces at the port root
# (``/<ns>/`` and ``/all/``) — Caddy / nginx / Cloudflare in
# front decides what external prefix they sit under. Typical
# symmetric setup:
#
# Caddy: handle_path /mcp/* { reverse_proxy localhost:8001 }
# config: public_base_url -> https://api.example.com/mcp
# dashboard advertises: https://api.example.com/mcp/<ns>/
#
# The frontend's ``/<ns>/`` segment gets appended verbatim. Set
# it to whatever matches your proxy. ``None`` means "advertise
# raw ``host:port`` derived from the inbound request" (dev /
# no proxy).
self.public_base_url = public_base_url.rstrip("/") if public_base_url else None
self._runtime: GatewayRuntime | None = None self._runtime: GatewayRuntime | None = None
self._app: Starlette | None = None self._app: Starlette | None = None
# Single shared aiohttp session, opened once when uvicorn starts
# the Starlette lifespan. Reused across every proxied request —
# MCP clients (esp. claude.ai) reconnect frequently, and a fresh
# ClientSession per call would cost a TCP handshake every time.
self._http: aiohttp.ClientSession | None = None self._http: aiohttp.ClientSession | None = None
def configure(self, runtime: GatewayRuntime) -> None: def configure(self, runtime: GatewayRuntime) -> None:
self._runtime = runtime self._runtime = runtime
self._app = self._build_app(runtime) self._app = self._build_app(runtime)
def app(self) -> Starlette | None:
return self._app
async def serve(self) -> None: async def serve(self) -> None:
import uvicorn self._http = aiohttp.ClientSession(
timeout=aiohttp.ClientTimeout(total=None, sock_read=600)
if self._app is None:
msg = "configure() must be called before serve()"
raise RuntimeError(msg)
config = uvicorn.Config(
self._app, host=self.host, port=self.port, log_level="info"
) )
server = uvicorn.Server(config) try:
await server.serve() await asyncio.Event().wait()
finally:
await self._http.close()
self._http = None
def _build_app(self, runtime: GatewayRuntime) -> Starlette: # noqa: ARG002 def _build_app(self, runtime: GatewayRuntime) -> Starlette: # noqa: ARG002
# The Starlette lifespan owns the shared aiohttp session: opened
# at startup, closed at shutdown so we don't leak sockets when
# uvicorn is restarted under the same process. Starlette wants a
# plain context manager, not an async-generator function — we
# decorate to match.
@asynccontextmanager
async def lifespan(_app: Starlette) -> AsyncIterator[None]:
self._http = aiohttp.ClientSession(
# Long sock_read because MCP tool calls can take a while
# (a claude tool over HTTP can easily stretch beyond 30s
# on a real tool).
timeout=aiohttp.ClientTimeout(total=None, sock_read=600)
)
try:
yield
finally:
if self._http is not None:
await self._http.close()
self._http = None
routes = [ routes = [
Route("/", self._discovery, methods=["GET"]), Route("/", self._discovery, methods=["GET"]),
Route("/healthz", self._healthz, methods=["GET"]), Route("/healthz", self._healthz, methods=["GET"]),
@@ -186,7 +141,7 @@ class McpServerFrontend(Frontend):
methods=["GET", "POST", "DELETE", "OPTIONS"], methods=["GET", "POST", "DELETE", "OPTIONS"],
), ),
] ]
return Starlette(routes=routes, lifespan=lifespan) return Starlette(routes=routes)
async def _healthz(self, _request: Request) -> JSONResponse: async def _healthz(self, _request: Request) -> JSONResponse:
return JSONResponse({"status": "ok"}) return JSONResponse({"status": "ok"})
@@ -197,13 +152,7 @@ class McpServerFrontend(Frontend):
if err is not None: if err is not None:
return err return err
assert token_name is not None # noqa: S101 — narrow for ty assert token_name is not None # noqa: S101 — narrow for ty
# ``public_base_url`` wins if configured — it's the operator's base = external_base(request, runtime)
# explicit statement of "this is the URL my reverse proxy puts
# in front of me". Otherwise: use the request's own scheme+host
# so snippets work behind generic reverse proxies / tunnels;
# and fall back to the configured host:port if the client
# didn't send Host (curl --raw).
base = self.public_base_url or _external_base_url(request, self.host, self.port)
html = _render_discovery_page( html = _render_discovery_page(
base_url=base, namespaces=list(runtime.mcps), actor=token_name base_url=base, namespaces=list(runtime.mcps), actor=token_name
) )
@@ -240,7 +189,6 @@ class McpServerFrontend(Frontend):
) )
if self._http is None: if self._http is None:
# Lifespan hasn't run yet (shouldn't happen with uvicorn).
return JSONResponse({"error": "frontend not ready"}, status_code=503) return JSONResponse({"error": "frontend not ready"}, status_code=503)
return await _reverse_proxy( return await _reverse_proxy(
@@ -337,14 +285,6 @@ def _join_subpath(base_url: str, subpath: str) -> str:
return base_url return base_url
def _external_base_url(request: Request, fallback_host: str, fallback_port: int) -> str:
host = request.headers.get("host")
if host:
scheme = request.headers.get("x-forwarded-proto", request.url.scheme)
return f"{scheme}://{host}"
return f"http://{fallback_host}:{fallback_port}"
async def _reverse_proxy( async def _reverse_proxy(
*, *,
client: aiohttp.ClientSession, client: aiohttp.ClientSession,
+54
View File
@@ -0,0 +1,54 @@
"""The one ASGI app the gateway listens with.
Every HTTP frontend is mounted under its ``path`` (``/anthropic``,
``/mcp``, ``/md``, ``/api``, ``/admin``); ``/healthz`` answers for the
whole process and ``/`` redirects to the landing frontend (the admin
console) when one is configured.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
from starlette.applications import Starlette
from starlette.responses import JSONResponse, RedirectResponse
from starlette.routing import Mount, Route
if TYPE_CHECKING:
from collections.abc import Iterable, Mapping
from starlette.requests import Request
from starlette.types import ASGIApp
from beaver_gateway.frontends.base import Frontend
__all__ = ["build_root_app"]
def build_root_app(
frontends: Iterable[Frontend], *, extra: Mapping[str, ASGIApp] | None = None
) -> Starlette:
mounted = [fe for fe in frontends if fe.path and fe.app() is not None]
landing = next((fe for fe in mounted if fe.landing), None)
paths = [*(fe.path for fe in mounted), *(extra or {})]
async def healthz(_request: Request) -> JSONResponse:
return JSONResponse({"status": "ok", "frontends": paths})
async def index(_request: Request) -> JSONResponse | RedirectResponse:
if landing is not None:
return RedirectResponse(f"{landing.path}/", status_code=307)
return JSONResponse({"frontends": paths})
routes: list[Route | Mount] = [
Route("/healthz", healthz, methods=["GET"]),
Route("/", index, methods=["GET"]),
]
for fe in mounted:
app = fe.app()
assert app is not None # noqa: S101 - filtered above; narrows for ty
assert fe.path is not None # noqa: S101
routes.append(Mount(fe.path, app=app, name=fe.name or fe.path.strip("/")))
for path, app in (extra or {}).items():
routes.append(Mount(path, app=app, name=path.strip("/")))
return Starlette(routes=routes)
@@ -0,0 +1,5 @@
"""Telegram frontend (§3.8): General = master, topic = branch, drafts, inbox/outbox."""
from beaver_gateway.frontends.telegram.frontend import Attachments, TelegramFrontend
__all__ = ["Attachments", "TelegramFrontend"]
@@ -0,0 +1,138 @@
"""One ``sendMessageDraft`` stream per running turn (§3.8).
The client folds a draft into the message that follows only when their
texts are identical - so the last push is the final text itself, rendered
exactly as the outbox will send it, with no status line.
A draft is ephemeral and lives 30 s, Telegram throttles edits to about one
per second per chat, and thinking or a tool call would otherwise look like a
hang - so the draft opens with a status line straight away, is refreshed on
a timer rather than on every delta, and is kept alive while nothing changes.
"""
from __future__ import annotations
import asyncio
import contextlib
import logging
import time
import zlib
from typing import TYPE_CHECKING
from aiogram.exceptions import TelegramAPIError, TelegramBadRequest
from beaver_gateway.frontends.telegram.render import to_html
if TYPE_CHECKING:
from aiogram import Bot
__all__ = ["Draft"]
_log = logging.getLogger("beaver_gateway.frontends.telegram.drafts")
_TAIL = 3500
_KEEPALIVE = 20.0
class Draft:
def __init__(
self,
bot: Bot,
*,
chat_id: int,
thread_id: int | None,
turn_id: str,
interval: float = 0.7,
status: str = "⏳ думаю",
) -> None:
self._bot = bot
self._chat_id = chat_id
self._thread_id = thread_id
self._draft_id = (zlib.crc32(turn_id.encode()) & 0x7FFFFFFF) or 1
self._interval = interval
self.status = status
self.text = ""
self._dirty = True
self._broken = False
self._last_sent = 0.0
self._task: asyncio.Task[None] | None = None
self._inflight: asyncio.Future[None] | None = None
def start(self) -> None:
if self._task is None:
self._task = asyncio.create_task(self._run())
def set_status(self, status: str) -> None:
if status != self.status:
self.status = status
self._dirty = True
def append(self, text: str) -> None:
if text:
self.text += text
self._dirty = True
async def stop(self) -> None:
"""Stop pushing and wait for the push in flight.
The final ``sendMessage`` must reach Telegram after the last draft,
or the draft lands on top of the message and lingers for its 30 s.
"""
if self._task is not None:
self._task.cancel()
await asyncio.gather(self._task, return_exceptions=True)
self._task = None
if self._inflight is not None:
with contextlib.suppress(Exception):
await self._inflight
self._inflight = None
async def finish(self, text: str) -> None:
"""Last push: the final text verbatim, so the message replaces the draft."""
await self.stop()
if self._broken or not text.strip():
return
try:
try:
await self._send(to_html(text), "HTML")
except TelegramBadRequest:
await self._send(text, None)
except TelegramAPIError as exc:
_log.warning(
"final draft to %s/%s failed: %s", self._chat_id, self._thread_id, exc
)
async def _run(self) -> None:
while not self._broken:
if self._dirty or time.monotonic() - self._last_sent > _KEEPALIVE:
self._inflight = asyncio.ensure_future(self._push())
await asyncio.shield(self._inflight)
await asyncio.sleep(self._interval)
async def _push(self) -> None:
self._dirty = False
self._last_sent = time.monotonic()
text = self._render()
try:
try:
await self._send(to_html(text), "HTML")
except TelegramBadRequest:
await self._send(text, None)
except TelegramAPIError as exc:
self._broken = True
_log.warning(
"draft to %s/%s stopped: %s", self._chat_id, self._thread_id, exc
)
async def _send(self, text: str, parse_mode: str | None) -> None:
await self._bot.send_message_draft(
chat_id=self._chat_id,
draft_id=self._draft_id,
message_thread_id=self._thread_id,
text=text,
parse_mode=parse_mode,
)
def _render(self) -> str:
tail = self.text[-_TAIL:]
return f"{self.status}\n\n{tail}" if tail.strip() else self.status
@@ -0,0 +1,898 @@
"""``TelegramFrontend`` - the private chat with the bot as the window (§3.8).
A private chat with topics has no General: the gateway makes one topic for
the master (``master_topic``) and rebinds it to every new master; any other
topic is a branch. The user makes a topic and the
first message in it spawns the branch (``seed=morning``); a message into a
topic whose branch is merged or closed spawns a new branch on the same
topic. Replies stream as drafts and land through the outbox; turns that
came from other windows are mirrored with a marker; ``origin=system`` is
never shown. ``AskUserQuestion`` becomes inline buttons (§3.7).
"""
from __future__ import annotations
import asyncio
import contextlib
import html
import logging
import tempfile
import time
from dataclasses import dataclass, field
from pathlib import Path
from typing import TYPE_CHECKING, Any, Literal, cast
from aiogram import Bot
from aiogram.client.default import DefaultBotProperties
from aiogram.exceptions import TelegramAPIError
from aiogram.types import (
BotCommand,
CallbackQuery,
InlineKeyboardButton,
InlineKeyboardMarkup,
Message,
ReactionTypeEmoji,
Update,
)
from beaver_gateway.frontends.base import Frontend
from beaver_gateway.frontends.telegram.drafts import Draft
from beaver_gateway.frontends.telegram.inbox import Inbox
from beaver_gateway.frontends.telegram.outbox import Outbox
from beaver_gateway.frontends.telegram.render import chunks, status_label
if TYPE_CHECKING:
from beaver_gateway.core.bus import Event, EventBus
from beaver_gateway.core.conversations import Conversations
from beaver_gateway.core.kinds import Kind
from beaver_gateway.frontends.base import GatewayRuntime
from beaver_gateway.storage.models import Conversation, ConversationBinding
__all__ = ["FRONTEND", "Attachments", "TelegramFrontend"]
_log = logging.getLogger("beaver_gateway.frontends.telegram")
FRONTEND = "telegram"
_DONE = "done"
_COMMAND_HELP = (
("merge", "слить ветку в мастер"),
("new", "новая ветка: в General - новый топик, в топике - заново на нём"),
("chat", "открыть глубокий чат в vault: /chat тема"),
("status", "что с этим разговором"), # noqa: RUF001
("help", "команды"),
)
_COMMANDS = (*(c for c, _ in _COMMAND_HELP), "start")
_HELP = "General - мастер, любой другой топик - ветка.\n" + "\n".join(
f"/{c} - {d}" for c, d in _COMMAND_HELP
)
@dataclass(frozen=True, slots=True)
class Attachments:
"""Where files from Telegram go.
``ephemeral`` - under the gateway's data dir, swept after ``keep_days``;
``vault`` - into a directory the agent owns. Pulling what is worth
keeping into the vault is the agent's job either way.
"""
mode: Literal["ephemeral", "vault"] = "ephemeral"
dir: Path | None = None
keep_days: int = 7
@property
def root(self) -> Path:
if self.dir is not None:
return self.dir
if self.mode == "vault":
msg = "Attachments(mode='vault') needs `dir`"
raise ValueError(msg)
return Path(tempfile.gettempdir()) / "beaver-attachments"
EPHEMERAL = Attachments()
@dataclass
class _Ask:
conversation_id: str
chat_id: int
thread_id: int | None
questions: list[dict[str, Any]]
messages: list[int] = field(default_factory=list)
picked: dict[int, list[str]] = field(default_factory=dict)
done: set[int] = field(default_factory=set)
class TelegramFrontend(Frontend):
name = FRONTEND
kinds = ("master", "branch")
def __init__(
self,
*,
token: str,
user_id: int,
master_agent: str | None = None,
branch_agent: str | None = None,
chat_id: int | None = None,
attachments: Attachments = EPHEMERAL,
draft_interval: float = 0.7,
master_topic: str = "🦫 General",
queued_reaction: str = "👀",
poll_timeout: int = 30,
outbox_backoff: float = 2.0,
) -> None:
self._token = token
self.user_id = user_id
self.chat_id = chat_id if chat_id is not None else user_id
self.master_agent = master_agent
self.branch_agent = branch_agent
self.attachments = attachments
self.master_topic = master_topic
self.draft_interval = draft_interval
self.queued_reaction = queued_reaction
self.poll_timeout = poll_timeout
self.outbox_backoff = outbox_backoff
self._runtime: GatewayRuntime | None = None
self._bot: Bot | None = None
self._inbox: Inbox | None = None
self._outbox: Outbox | None = None
self._targets: dict[str, tuple[int, int | None] | None] = {}
self._master_window: str | None = None
self._topic_names: dict[int, str] = {}
self._drafts: dict[str, Draft] = {}
self._asks: dict[str, _Ask] = {}
self._reactions: dict[int, tuple[int, int]] = {}
self._tasks: set[asyncio.Task[None]] = set()
# ---- Frontend --------------------------------------------------------
def agent_for(self, kind: Kind) -> str | None:
return {"master": self.master_agent, "branch": self.branch_agent}.get(kind)
def configure(self, runtime: GatewayRuntime) -> None:
if runtime.conversations is None or runtime.bus is None:
msg = "TelegramFrontend needs runtime.conversations and runtime.bus"
raise RuntimeError(msg)
self._runtime = runtime
if self._bot is None:
self._bot = Bot(self._token, default=DefaultBotProperties(parse_mode=None))
self._inbox = Inbox(
runtime.db, self._bot, handler=self._handle, poll_timeout=self.poll_timeout
)
self._outbox = Outbox(
runtime.db, self._bot, bus=runtime.bus, backoff=self.outbox_backoff
)
async def serve(self) -> None:
me = await self.bot.get_me()
_log.info(
"telegram: @%s, user %s, chat %s, topics=%s",
me.username,
self.user_id,
self.chat_id,
getattr(me, "has_topics_enabled", None),
)
self._sweep_attachments()
await self.bot.set_my_commands(
[BotCommand(command=c, description=d) for c, d in _COMMAND_HELP]
)
try:
async with asyncio.TaskGroup() as tg:
tg.create_task(self.inbox.run())
tg.create_task(self.outbox.run())
tg.create_task(self._events())
finally:
for draft in list(self._drafts.values()):
await draft.stop()
await self.bot.session.close()
async def materialize(self, conv: Conversation) -> ConversationBinding | None:
if conv.kind == "master":
return await self.conversations.bind(
conv, frontend=FRONTEND, external_id=await self._master_ext()
)
if conv.kind != "branch":
return None
topic = await self.bot.create_forum_topic(
self.chat_id, name=(conv.title or "ветка")[:128]
)
self._topic_names[topic.message_thread_id] = topic.name
return await self.conversations.bind(
conv, frontend=FRONTEND, external_id=self._ext(topic.message_thread_id)
)
async def mark_closed(self, conv: Conversation) -> bool:
return await self.mark_topic(conv)
async def mark_topic(self, conv: Conversation, prefix: str = "✅ ") -> bool:
"""Rename the topic; closeForumTopic does not exist in private chats."""
target = await self._target_of(conv)
if target is None or target[1] is None:
return False
name = self._topic_names.get(target[1]) or conv.title or "ветка"
if name.startswith(prefix):
return True
await self.bot.edit_forum_topic(
target[0], target[1], name=f"{prefix}{name}"[:128]
)
self._topic_names[target[1]] = f"{prefix}{name}"
return True
# ---- plumbing --------------------------------------------------------
@property
def bot(self) -> Bot:
if self._bot is None:
msg = "configure() must be called before use"
raise RuntimeError(msg)
return self._bot
@property
def inbox(self) -> Inbox:
return cast("Inbox", self._inbox)
@property
def outbox(self) -> Outbox:
return cast("Outbox", self._outbox)
@property
def conversations(self) -> Conversations:
return cast(
"Conversations", cast("GatewayRuntime", self._runtime).conversations
)
@property
def bus(self) -> EventBus:
return cast("EventBus", cast("GatewayRuntime", self._runtime).bus)
def _ext(self, thread_id: int | None) -> str:
return f"{self.chat_id}/{thread_id}" if thread_id else str(self.chat_id)
@staticmethod
def _parse_ext(ext: str) -> tuple[int, int | None]:
chat, _, thread = ext.partition("/")
return int(chat), int(thread) if thread else None
async def _target_of(self, conv: Conversation) -> tuple[int, int | None] | None:
key = conv.external_id
if key not in self._targets:
bound = next(
(
b
for b in await self.conversations.bindings(conv)
if b.frontend == FRONTEND and b.visible
),
None,
)
self._targets[key] = self._parse_ext(bound.external_id) if bound else None
return self._targets[key]
async def _deliver(
self,
conv: Conversation,
text: str,
*,
turn_id: str | None = None,
key: str | None = None,
) -> None:
target = await self._target_of(conv)
if target is None or not text.strip():
return
await self.outbox.enqueue(
chat_id=target[0],
thread_id=target[1],
text=text,
conversation_id=conv.id,
turn_id=turn_id,
dedupe_key=key,
)
async def _master_ext(self) -> str:
"""The master's window.
General in a forum group, our own topic in a private chat (Telegram
has no General there). Created once, then found through whatever
master used it last.
"""
if self._master_window is not None:
return self._master_window
last = await self.conversations.last_binding(frontend=FRONTEND, kind="master")
if last is not None:
self._master_window = last.external_id
elif self.chat_id < 0:
self._master_window = self._ext(None)
else:
topic = await self.bot.create_forum_topic(
self.chat_id, name=self.master_topic[:128]
)
self._topic_names[topic.message_thread_id] = topic.name
self._master_window = self._ext(topic.message_thread_id)
return self._master_window
async def _master(self, text: str | None = None) -> tuple[Conversation, bool]:
"""The open master behind its window, spawning one when there is none.
``text`` rides with the seed of a fresh master; the second value says
whether it was consumed that way.
"""
ext = await self._master_ext()
conv = await self.conversations.find_bound(frontend=FRONTEND, external_id=ext)
if conv is not None and conv.status == "open":
return conv, False
masters = await self.conversations.find(kind="master", status="open", limit=1)
if masters:
await self.conversations.bind(
masters[0], frontend=FRONTEND, external_id=ext
)
return masters[0], False
conv = await self.conversations.spawn(
kind="master",
seed="clean",
text=text,
origin=FRONTEND,
binding=(FRONTEND, ext),
)
return conv, text is not None
async def _branch(
self, thread_id: int, *, title: str | None, text: str | None
) -> Conversation:
master, _ = await self._master()
return await self.conversations.spawn(
kind="branch",
seed="morning",
parent=master,
title=(title or self._topic_names.get(thread_id) or "ветка")[:128],
text=text,
origin=FRONTEND,
binding=(FRONTEND, self._ext(thread_id)),
)
# ---- inbox -----------------------------------------------------------
async def _handle(self, update: Update) -> None:
if update.message is not None:
await self._on_message(update.message)
elif update.callback_query is not None:
await self._on_callback(update.callback_query)
async def _on_message(self, message: Message) -> None:
created = message.forum_topic_created
if (
created is not None
and message.message_thread_id
and not created.is_name_implicit
):
self._topic_names[message.message_thread_id] = created.name
if message.from_user is None or message.from_user.id != self.user_id:
_log.warning(
"ignoring message from %s", getattr(message.from_user, "id", None)
)
return
if message.chat.id != self.chat_id:
return
in_topic = message.is_topic_message or message.forum_topic_created is not None
thread_id = message.message_thread_id if in_topic else None
is_master = (
thread_id is None or self._ext(thread_id) == await self._master_ext()
)
if message.forum_topic_created is not None:
return
if message.forum_topic_edited is not None and thread_id is not None:
if message.forum_topic_edited.name:
self._topic_names[thread_id] = message.forum_topic_edited.name
return
text = (message.text or message.caption or "").strip()
attachment = await self._save_attachment(message, thread_id)
if attachment:
text = f"{text}\n\n{attachment}".strip()
if not text:
return
if message.text and message.text.startswith("/"):
command, _, args = message.text[1:].partition(" ")
command = command.partition("@")[0].lower()
if command in _COMMANDS:
await self._command(command, args.strip(), message, thread_id)
return
if is_master:
conv, consumed = await self._master(text)
if consumed:
return
else:
conv = await self._live(thread_id)
if conv is None:
title = self._topic_names.get(thread_id) or self._title_from(text)
await self._branch(thread_id, title=title, text=text)
return
pending = self.conversations.pending_question(conv.external_id)
if pending is not None and self.conversations.answer(pending[0], text):
await self._close_ask(pending[0], f"✍️ {text}")
return
item = await self.conversations.post(conv, text, origin=FRONTEND)
if conv.running_turn or conv.pending_question:
await self._react(message, item.id)
async def _live(self, thread_id: int) -> Conversation | None:
conv = await self.conversations.find_bound(
frontend=FRONTEND, external_id=self._ext(thread_id)
)
return conv if conv is not None and conv.status == "open" else None
@staticmethod
def _title_from(text: str) -> str:
line = text.strip().splitlines()[0]
return line if len(line) <= 60 else line[:57] + "…"
async def _react(self, message: Message, item_id: int | None) -> None:
if not self.queued_reaction or item_id is None:
return
try:
await self.bot.set_message_reaction(
message.chat.id,
message.message_id,
reaction=[ReactionTypeEmoji(emoji=self.queued_reaction)],
)
except TelegramAPIError as exc:
_log.debug("reaction failed: %s", exc)
return
self._reactions[item_id] = (message.chat.id, message.message_id)
async def _unreact(self, item_id: int | None) -> None:
target = self._reactions.pop(cast("int", item_id), None) if item_id else None
if target is None:
return
with contextlib.suppress(TelegramAPIError):
await self.bot.set_message_reaction(target[0], target[1], reaction=[])
async def _save_attachment(
self, message: Message, thread_id: int | None
) -> str | None:
file_id: str | None = None
name: str | None = None
kind = ""
size: int | None = None
if message.photo:
photo = message.photo[-1]
file_id, kind, size = photo.file_id, "фото", photo.file_size
name = f"{photo.file_unique_id}.jpg"
elif message.document:
doc = message.document
file_id, kind, size = doc.file_id, "файл", doc.file_size
name = doc.file_name or f"{doc.file_unique_id}.bin"
elif message.voice:
file_id, kind, size = (
message.voice.file_id,
"голосовое",
message.voice.file_size,
)
name = f"{message.voice.file_unique_id}.ogg"
elif message.audio:
file_id, kind, size = (
message.audio.file_id,
"аудио",
message.audio.file_size,
)
name = message.audio.file_name or f"{message.audio.file_unique_id}.mp3"
elif message.video:
file_id, kind, size = (
message.video.file_id,
"видео",
message.video.file_size,
)
name = message.video.file_name or f"{message.video.file_unique_id}.mp4"
elif message.video_note:
file_id, kind, size = (
message.video_note.file_id,
"кружок",
message.video_note.file_size,
)
name = f"{message.video_note.file_unique_id}.mp4"
if file_id is None or name is None:
return None
if size and size > 20 * 1024 * 1024:
return (
f"[вложение: {kind} {name}, {size // 1024 // 1024} МБ - "
"больше 20 МБ, Telegram не отдаёт ботам]"
)
folder = self.attachments.root / (self._ext(thread_id).replace("/", "_"))
folder.mkdir(parents=True, exist_ok=True)
with contextlib.suppress(OSError):
self.attachments.root.chmod(0o755)
folder.chmod(0o755)
path = folder / f"{int(time.time())}-{Path(name).name}"
try:
await self.bot.download(file_id, destination=path)
path.chmod(0o644)
except (TelegramAPIError, OSError) as exc:
_log.warning("attachment download failed: %s", exc)
return f"[вложение: {kind} {name} - не скачалось: {exc}]"
shown = f", {size // 1024} КБ" if size else ""
return f"[вложение: {kind} {path}{shown}]"
def _sweep_attachments(self) -> None:
if self.attachments.mode != "ephemeral":
return
root = self.attachments.root
if not root.exists():
return
cutoff = time.time() - self.attachments.keep_days * 86400
for path in root.rglob("*"):
with contextlib.suppress(OSError):
if path.is_file() and path.stat().st_mtime < cutoff:
path.unlink()
# ---- commands --------------------------------------------------------
async def _command(
self, command: str, args: str, message: Message, thread_id: int | None
) -> None:
reply = await self._run_command(command, args, thread_id)
if reply:
await self.outbox.enqueue(
chat_id=message.chat.id, thread_id=thread_id, text=reply
)
async def _run_command(self, command: str, args: str, thread_id: int | None) -> str:
if command in ("start", "help"):
return _HELP
is_master = (
thread_id is None or self._ext(thread_id) == await self._master_ext()
)
conv = (await self._master())[0] if is_master else await self._live(thread_id)
if command == "status":
return await self._status(conv)
if command == "merge":
if conv is None or conv.kind != "branch":
return "сливать нечего: это не открытая ветка"
self._spawn_task(self._merge(conv))
return "🔀 сливаю в мастер"
if command == "new":
if is_master or thread_id is None:
child = await self.conversations.spawn(
kind="branch",
seed="morning",
parent=conv,
title=args or None,
origin=FRONTEND,
)
return f"🌿 ветка «{child.title or child.external_id}» - в новом топике"
if conv is not None:
await self.conversations.set_status(conv, "closed")
await self._branch(thread_id, title=args or None, text=None)
return "🌿 новая ветка на этом топике"
if command == "chat":
if not args:
return "/chat <тема>"
try:
deep = await self.conversations.spawn(
kind="deep", seed="clean", title=args, origin=FRONTEND
)
except (ValueError, LookupError) as exc:
return f"не вышло: {exc}"
where = next(
(
b.external_id
for b in await self.conversations.bindings(deep)
if b.visible
),
deep.external_id,
)
return f"💬 глубокий чат: {where}"
return _HELP
async def _status(self, conv: Conversation | None) -> str:
if conv is None:
return "этот топик ни к чему не привязан - напиши, и откроется ветка"
info = await self.conversations.describe(conv)
queued = sum(
1
for i in await self.conversations.queue.recent(
cast("int", conv.id), limit=20
)
if i.status == "queued"
)
lines = [
f"{conv.kind} · {conv.status} · {conv.agent_name}",
f"сессия: {'живая' if info['live'] else 'нет'}"
f" · тёрн: {'идёт' if conv.running_turn else 'нет'}"
f" · в очереди: {queued}",
]
if conv.pending_question:
lines.append("❓ ждёт ответа на вопрос")
if conv.kind == "master":
pool = self.conversations.pool
lines.append(f"пул: {len(pool)} сессий, rss {pool.rss() // (1 << 20)} МБ")
lines.append(f"outbox: {await self.outbox.pending()} в очереди")
lines.append(f"id: {conv.external_id}")
return "\n".join(lines)
async def _merge(self, conv: Conversation) -> None:
try:
await self.conversations.merge(conv)
except Exception as exc: # noqa: BLE001
_log.exception("merge of %s failed", conv.external_id)
await self._deliver(conv, f"⚠️ слив не удался: {exc}")
def _spawn_task(self, coro: Any) -> None:
task = asyncio.create_task(coro)
self._tasks.add(task)
task.add_done_callback(self._tasks.discard)
# ---- bus -------------------------------------------------------------
async def _events(self) -> None:
async for event in self.bus.stream():
try:
await self._on_event(event)
except Exception: # noqa: BLE001
_log.exception("event %s failed", event.get("type"))
async def _on_event(self, event: Event) -> None:
kind = event["type"]
if kind == "conversation.bound":
self._targets.pop(str(event.get("conversation_id")), None)
return
if kind in ("delivery.sent", "delivery.failed", "conversation.created"):
return
key = event.get("conversation_id")
if not isinstance(key, str):
return
conv = await self.conversations.get(key)
if conv is None:
return
target = await self._target_of(conv)
if target is None:
return
match kind:
case "turn.start":
if event.get("origin") == "user":
await self._open_draft(key, event, target)
case "stream":
self._feed_draft(key, event)
case "tool":
draft = self._drafts.get(key)
if draft is not None and event.get("parent_tool_use_id") is None:
draft.set_status(
f"⏳ {status_label(str(event['name']), event.get('input'))}"
)
case "turn.end":
draft = self._drafts.get(key)
if draft is not None:
await draft.stop()
if event.get("stop") == "error" and event.get("origin") == "user":
await self._deliver(
conv,
"⚠️ тёрн упал, смотри логи gateway",
turn_id=event.get("turn_id"),
key=f"{event.get('turn_id')}:error",
)
case "reply":
await self._on_reply(conv, event)
case "say":
await self._deliver(
conv, str(event.get("text") or ""), turn_id=event.get("turn_id")
)
case "question":
await self._ask(conv, event, target)
case "question.answered":
await self._close_ask(
str(event["question_id"]), f"✅ {event.get('answer') or ''}"
)
case "question.timeout":
await self._close_ask(
str(event["question_id"]), "⌛ время вышло - ответь текстом"
)
case "conversation.merged":
await self._deliver(conv, "✅ слито в мастер", key=f"{key}:merged")
async def _on_reply(self, conv: Conversation, event: Event) -> None:
turn_id = str(event.get("turn_id") or "")
origin = str(event.get("item_origin") or "")
text = str(event.get("text") or "")
await self._unreact(event.get("item"))
draft = self._drafts.pop(conv.external_id, None)
if draft is not None:
await draft.finish(chunks(text)[-1] if text.strip() else "")
if origin != FRONTEND and not origin.startswith("сид"):
user_text = str(event.get("user_text") or "")
if user_text:
await self._deliver(
conv,
f"📝 из панели:\n{user_text}",
turn_id=turn_id,
key=f"{turn_id}:mirror",
)
await self._deliver(conv, text, turn_id=turn_id, key=f"{turn_id}:reply")
# ---- drafts ------------------------------------------------------------
async def _open_draft(
self, key: str, event: Event, target: tuple[int, int | None]
) -> None:
if target[0] < 0:
return
await self._close_draft(key)
draft = Draft(
self.bot,
chat_id=target[0],
thread_id=target[1],
turn_id=str(event.get("turn_id") or key),
interval=self.draft_interval,
)
self._drafts[key] = draft
draft.start()
def _feed_draft(self, key: str, event: Event) -> None:
draft = self._drafts.get(key)
if draft is None or event.get("parent_tool_use_id") is not None:
return
raw = event.get("event") or {}
kind = raw.get("type")
if kind == "content_block_delta":
delta = raw.get("delta") or {}
if delta.get("type") == "text_delta":
draft.set_status("✍️ пишу")
draft.append(str(delta.get("text") or ""))
elif delta.get("type") == "thinking_delta":
draft.set_status("🤔 думаю")
elif kind == "content_block_start":
block = raw.get("content_block") or {}
if block.get("type") == "tool_use":
draft.set_status(
f"⏳ {status_label(str(block.get('name') or ''), None)}"
)
async def _close_draft(self, key: str) -> None:
draft = self._drafts.pop(key, None)
if draft is not None:
await draft.stop()
# ---- questions (§3.7) ---------------------------------------------------
async def _ask(
self, conv: Conversation, event: Event, target: tuple[int, int | None]
) -> None:
question_id = str(event["question_id"])
questions = [q for q in event.get("questions") or [] if isinstance(q, dict)]
if not questions:
return
ask = _Ask(
conversation_id=conv.external_id,
chat_id=target[0],
thread_id=target[1],
questions=questions,
)
self._asks[question_id] = ask
for qi, question in enumerate(questions):
try:
sent = await self.bot.send_message(
target[0],
_question_html(question),
message_thread_id=target[1],
parse_mode="HTML",
reply_markup=_keyboard(question_id, qi, question, []),
)
except TelegramAPIError:
_log.exception("question %s could not be sent", question_id)
continue
ask.messages.append(sent.message_id)
async def _on_callback(self, query: CallbackQuery) -> None:
if query.from_user.id != self.user_id or not query.data:
return
parts = query.data.split(":")
if len(parts) != 4 or parts[0] != "q":
return
_, question_id, qi_raw, choice = parts
ask = self._asks.get(question_id)
if ask is None or not qi_raw.isdigit():
await self._callback_reply(query, "вопрос уже закрыт")
return
qi = int(qi_raw)
question = ask.questions[qi]
options = [str(o.get("label", "")) for o in question.get("options") or []]
picked = ask.picked.setdefault(qi, [])
multi = bool(question.get("multiSelect"))
if choice == _DONE:
ask.done.add(qi)
elif choice.isdigit() and int(choice) < len(options):
label = options[int(choice)]
if multi:
if label in picked:
picked.remove(label)
else:
picked.append(label)
else:
picked[:] = [label]
ask.done.add(qi)
message = query.message if isinstance(query.message, Message) else None
if message is not None:
with contextlib.suppress(TelegramAPIError):
if qi in ask.done:
await self.bot.edit_message_text(
f"{_question_html(question)}\n\n"
f"✅ {html.escape(', '.join(picked) or '-')}",
chat_id=message.chat.id,
message_id=message.message_id,
parse_mode="HTML",
)
else:
await self.bot.edit_message_reply_markup(
chat_id=message.chat.id,
message_id=message.message_id,
reply_markup=_keyboard(question_id, qi, question, picked),
)
await self._callback_reply(query, None)
if len(ask.done) == len(ask.questions):
answer = _answer_text(ask)
self._asks.pop(question_id, None)
if not self.conversations.answer(question_id, answer):
await self._edit_asks(ask, "⌛ время вышло - ответь текстом")
async def _callback_reply(self, query: CallbackQuery, text: str | None) -> None:
with contextlib.suppress(TelegramAPIError):
await self.bot.answer_callback_query(query.id, text=text)
async def _close_ask(self, question_id: str, note: str) -> None:
ask = self._asks.pop(question_id, None)
if ask is not None:
await self._edit_asks(ask, note)
async def _edit_asks(self, ask: _Ask, note: str) -> None:
for qi, message_id in enumerate(ask.messages):
if qi in ask.done and not note.startswith("⌛"):
continue
with contextlib.suppress(TelegramAPIError):
await self.bot.edit_message_text(
f"{_question_html(ask.questions[qi])}\n\n{html.escape(note)}",
chat_id=ask.chat_id,
message_id=message_id,
parse_mode="HTML",
)
def _question_html(question: dict[str, Any]) -> str:
header = html.escape(str(question.get("header") or "").strip())
body = html.escape(str(question.get("question") or "").strip())
lines = [f"❓ <b>{header}</b>" if header else "❓", body]
for option in question.get("options") or []:
label = html.escape(str(option.get("label", "")))
description = html.escape(str(option.get("description") or "").strip())
lines.append(f"• <b>{label}</b>" + (f" - {description}" if description else ""))
return "\n".join(line for line in lines if line)
def _keyboard(
question_id: str, qi: int, question: dict[str, Any], picked: list[str]
) -> InlineKeyboardMarkup:
rows = [
[
InlineKeyboardButton(
text=("☑ " if str(o.get("label", "")) in picked else "")
+ str(o.get("label", ""))[:60],
callback_data=f"q:{question_id}:{qi}:{oi}",
)
]
for oi, o in enumerate(question.get("options") or [])
]
if question.get("multiSelect"):
rows.append(
[
InlineKeyboardButton(
text="✅ готово", callback_data=f"q:{question_id}:{qi}:{_DONE}"
)
]
)
return InlineKeyboardMarkup(inline_keyboard=rows)
def _answer_text(ask: _Ask) -> str:
if len(ask.questions) == 1:
return ", ".join(ask.picked.get(0, [])) or "-"
return "; ".join(
f"{q.get('header') or q.get('question')}: "
f"{', '.join(ask.picked.get(i, [])) or '-'}"
for i, q in enumerate(ask.questions)
)
@@ -0,0 +1,144 @@
"""Long-polling inbox (§3.8).
Every update lands in ``telegram_updates`` before the offset moves past it;
a worker handles rows from the table, oldest first, and finishes whatever a
previous process left unprocessed at startup.
"""
from __future__ import annotations
import asyncio
import contextlib
import logging
from datetime import UTC, datetime
from typing import TYPE_CHECKING
from aiogram.exceptions import TelegramConflictError, TelegramNetworkError
from aiogram.types import Update
from sqlalchemy import func
from sqlalchemy.exc import IntegrityError
from sqlmodel import col, select
from beaver_gateway.storage.models import TelegramUpdate
if TYPE_CHECKING:
from collections.abc import Awaitable, Callable
from aiogram import Bot
from beaver_gateway.storage.db import Database
__all__ = ["Inbox"]
_log = logging.getLogger("beaver_gateway.frontends.telegram.inbox")
ALLOWED_UPDATES = ("message", "callback_query")
def _unset(_value: object) -> None:
"""Unset aiogram ``Default`` sentinels (link previews) are stored as null."""
class Inbox:
def __init__(
self,
db: Database,
bot: Bot,
*,
handler: Callable[[Update], Awaitable[None]],
poll_timeout: int = 30,
) -> None:
self._db = db
self._bot = bot
self._handler = handler
self._poll_timeout = poll_timeout
self._wake = asyncio.Event()
async def run(self) -> None:
async with asyncio.TaskGroup() as tg:
tg.create_task(self._poll())
tg.create_task(self._work())
async def _poll(self) -> None:
offset = await self._next_offset()
backoff = 1.0
while True:
try:
updates = await self._bot.get_updates(
offset=offset,
timeout=self._poll_timeout,
allowed_updates=list(ALLOWED_UPDATES),
request_timeout=self._poll_timeout + 10,
)
except TelegramConflictError:
_log.error("another poller holds this bot token; retrying in 10s")
await asyncio.sleep(10)
continue
except (TimeoutError, TelegramNetworkError, OSError) as exc:
_log.warning("getUpdates failed (%s); retrying in %.0fs", exc, backoff)
await asyncio.sleep(backoff)
backoff = min(backoff * 2, 60.0)
continue
backoff = 1.0
for update in updates:
await self._store(update)
offset = update.update_id + 1
if updates:
self._wake.set()
async def _store(self, update: Update) -> None:
row = TelegramUpdate(
update_id=update.update_id,
payload=update.model_dump(
mode="json", by_alias=True, exclude_none=True, fallback=_unset
),
)
async with self._db.session() as session:
session.add(row)
try:
await session.commit()
except IntegrityError:
await session.rollback()
async def _next_offset(self) -> int | None:
async with self._db.session() as session:
latest = (
await session.exec(select(func.max(col(TelegramUpdate.update_id))))
).one()
return int(latest) + 1 if latest is not None else None
async def _work(self) -> None:
while True:
rows = await self._pending()
if not rows:
self._wake.clear()
with contextlib.suppress(TimeoutError):
await asyncio.wait_for(self._wake.wait(), timeout=5.0)
continue
for row in rows:
await self._handle(row)
async def _pending(self, limit: int = 50) -> list[TelegramUpdate]:
async with self._db.session() as session:
result = await session.exec(
select(TelegramUpdate)
.where(col(TelegramUpdate.processed_at).is_(None))
.order_by(col(TelegramUpdate.update_id))
.limit(limit)
)
return list(result.all())
async def _handle(self, row: TelegramUpdate) -> None:
error: str | None = None
try:
await self._handler(Update.model_validate(row.payload))
except Exception as exc: # noqa: BLE001
error = f"{type(exc).__name__}: {exc}"[:500]
_log.exception("update %s failed", row.update_id)
async with self._db.session() as session:
stored = await session.get(TelegramUpdate, row.update_id)
if stored is not None:
stored.processed_at = datetime.now(UTC)
stored.error = error
session.add(stored)
await session.commit()
@@ -0,0 +1,235 @@
"""Outbox (§3.8): a reply is a ``deliveries`` row first, a message second.
Rows are sent oldest first, retried with backoff on network errors and
flood limits, resent as plain text when Telegram rejects our HTML, and
given up only when Telegram says the window is gone.
"""
from __future__ import annotations
import asyncio
import contextlib
import logging
from datetime import UTC, datetime, timedelta
from typing import TYPE_CHECKING
from aiogram.exceptions import (
TelegramBadRequest,
TelegramForbiddenError,
TelegramNetworkError,
TelegramNotFound,
TelegramRetryAfter,
TelegramServerError,
)
from aiogram.types import LinkPreviewOptions
from sqlalchemy import func
from sqlalchemy.exc import IntegrityError
from sqlmodel import col, select
from beaver_gateway.frontends.telegram.render import chunks, to_html
from beaver_gateway.storage.models import Delivery
if TYPE_CHECKING:
from aiogram import Bot
from beaver_gateway.core.bus import EventBus
from beaver_gateway.storage.db import Database
__all__ = ["Outbox"]
_log = logging.getLogger("beaver_gateway.frontends.telegram.outbox")
_MAX_BACKOFF = 300.0
_GONE = ("thread not found", "chat not found", "topic_deleted", "topic_closed")
_NO_PREVIEW = LinkPreviewOptions(is_disabled=True)
class Outbox:
def __init__(
self, db: Database, bot: Bot, *, bus: EventBus, backoff: float = 2.0
) -> None:
self._db = db
self._bot = bot
self._bus = bus
self._backoff = backoff
self._wake = asyncio.Event()
async def enqueue(
self,
*,
chat_id: int,
thread_id: int | None,
text: str,
conversation_id: int | None = None,
turn_id: str | None = None,
dedupe_key: str | None = None,
) -> list[Delivery]:
rows: list[Delivery] = []
for n, part in enumerate(chunks(text)):
row = Delivery(
conversation_id=conversation_id,
chat_id=chat_id,
thread_id=thread_id,
text=part,
turn_id=turn_id,
dedupe_key=f"{dedupe_key}:{n}" if dedupe_key else None,
)
async with self._db.session() as session:
session.add(row)
try:
await session.commit()
except IntegrityError:
await session.rollback()
continue
await session.refresh(row)
rows.append(row)
if rows:
self._wake.set()
return rows
async def run(self) -> None:
while True:
rows = await self._due()
if not rows:
self._wake.clear()
with contextlib.suppress(TimeoutError):
await asyncio.wait_for(
self._wake.wait(), timeout=await self._wait_for_next()
)
continue
for row in rows:
await self._send(row)
async def _wait_for_next(self, cap: float = 5.0) -> float:
async with self._db.session() as session:
earliest = (
await session.exec(
select(func.min(col(Delivery.next_attempt_at))).where(
Delivery.status == "queued"
)
)
).one()
if earliest is None:
return cap
now = datetime.now(UTC).replace(tzinfo=None)
return max(0.05, min(cap, (earliest - now).total_seconds()))
async def _due(self, limit: int = 50) -> list[Delivery]:
now = datetime.now(UTC).replace(tzinfo=None)
async with self._db.session() as session:
result = await session.exec(
select(Delivery)
.where(
Delivery.status == "queued", col(Delivery.next_attempt_at) <= now
)
.order_by(col(Delivery.id))
.limit(limit)
)
return list(result.all())
async def _send(self, row: Delivery) -> None:
try:
message = await self._bot.send_message(
row.chat_id,
row.text if row.plain else to_html(row.text),
message_thread_id=row.thread_id,
parse_mode=None if row.plain else "HTML",
link_preview_options=_NO_PREVIEW,
)
except TelegramRetryAfter as exc:
await self._retry(row, str(exc), delay=float(exc.retry_after))
except TelegramBadRequest as exc:
text = str(exc).lower()
if "parse" in text and not row.plain:
await self._retry(row, str(exc), delay=0.0, plain=True)
elif any(marker in text for marker in _GONE):
await self._fail(row, str(exc))
else:
await self._fail(row, str(exc))
except (TelegramNotFound, TelegramForbiddenError) as exc:
await self._fail(row, str(exc))
except (TelegramNetworkError, TelegramServerError, OSError) as exc:
await self._retry(
row,
str(exc),
delay=min(self._backoff ** (row.attempts + 1), _MAX_BACKOFF),
)
else:
await self._mark(row, status="sent", message_id=message.message_id)
self._bus.publish(
"delivery.sent",
delivery=row.id,
conversation_row=row.conversation_id,
chat_id=row.chat_id,
thread_id=row.thread_id,
message_id=message.message_id,
turn_id=row.turn_id,
)
async def _retry(
self, row: Delivery, error: str, *, delay: float, plain: bool = False
) -> None:
_log.warning(
"delivery #%s attempt %d failed: %s (retry in %.0fs)",
row.id,
row.attempts + 1,
error,
delay,
)
await self._mark(row, status="queued", error=error, delay=delay, plain=plain)
if delay < 5.0:
self._wake.set()
async def _fail(self, row: Delivery, error: str) -> None:
_log.error(
"delivery #%s to %s/%s given up: %s",
row.id,
row.chat_id,
row.thread_id,
error,
)
await self._mark(row, status="failed", error=error)
self._bus.publish(
"delivery.failed",
delivery=row.id,
conversation_row=row.conversation_id,
chat_id=row.chat_id,
thread_id=row.thread_id,
error=error,
)
async def _mark(
self,
row: Delivery,
*,
status: str,
error: str | None = None,
delay: float = 0.0,
plain: bool = False,
message_id: int | None = None,
) -> None:
async with self._db.session() as session:
stored = await session.get(Delivery, row.id)
if stored is None:
return
stored.status = status
stored.attempts += 1
stored.last_error = error[:500] if error else None
stored.next_attempt_at = (
datetime.now(UTC) + timedelta(seconds=delay)
).replace(tzinfo=None)
if plain:
stored.plain = True
if message_id is not None:
stored.message_id = message_id
if status == "sent":
stored.sent_at = datetime.now(UTC)
session.add(stored)
await session.commit()
async def pending(self) -> int:
async with self._db.session() as session:
result = await session.exec(
select(Delivery).where(Delivery.status == "queued")
)
return len(list(result.all()))
@@ -0,0 +1,111 @@
"""Model markdown → Telegram HTML, chunking, and the status line for drafts."""
from __future__ import annotations
import html
import re
from typing import Any
__all__ = ["LIMIT", "chunks", "status_label", "to_html"]
LIMIT = 4000
_FENCE = re.compile(r"```[^\n]*\n(.*?)(?:```|$)", re.DOTALL)
_INLINE_CODE = re.compile(r"(`[^`\n]+`)")
_HEADING = re.compile(r"^#{1,6}\s+(.+)$", re.MULTILINE)
_BOLD = re.compile(r"\*\*(.+?)\*\*|__(.+?)__", re.DOTALL)
_ITALIC = re.compile(r"(?<![\w*])\*(?!\s)(.+?)(?<!\s)\*(?![\w*])")
_ITALIC_U = re.compile(r"(?<![\w_])_(?!\s)(.+?)(?<!\s)_(?![\w_])")
_LINK = re.compile(r"\[([^\]]+)\]\((https?://[^)\s]+)\)")
_BULLET = re.compile(r"^(\s*)[-*]\s+", re.MULTILINE)
_STRIKE = re.compile(r"~~(.+?)~~")
_LABELS: dict[str, str] = {
"Read": "читаю vault",
"Glob": "ищу файлы",
"Grep": "ищу в vault",
"Edit": "правлю файл",
"Write": "пишу файл",
"MultiEdit": "правлю файлы",
"Bash": "выполняю команду",
"WebSearch": "ищу в сети",
"WebFetch": "читаю страницу",
"Task": "запустил сабагента",
"Agent": "запустил сабагента",
"AskUserQuestion": "спрашиваю",
"TodoWrite": "планирую",
"Skill": "открываю скилл",
"mcp__gateway__spawn": "открываю разговор",
"mcp__gateway__read_conversation": "читаю разговор",
"mcp__gateway__say": "говорю",
"mcp__gateway__schedule": "ставлю напоминание",
"mcp__gateway__inject": "передаю в другой разговор",
}
def to_html(text: str) -> str:
out: list[str] = []
pos = 0
for match in _FENCE.finditer(text):
out.append(_inline(text[pos : match.start()]))
out.append(f"<pre>{html.escape(match.group(1).rstrip())}</pre>")
pos = match.end()
out.append(_inline(text[pos:]))
return "".join(out).strip()
def _inline(text: str) -> str:
parts = _INLINE_CODE.split(text)
for i, part in enumerate(parts):
if i % 2:
parts[i] = f"<code>{html.escape(part[1:-1])}</code>"
continue
s = html.escape(part, quote=False)
s = _HEADING.sub(r"<b>\1</b>", s)
s = _BOLD.sub(lambda m: f"<b>{m.group(1) or m.group(2)}</b>", s)
s = _ITALIC.sub(r"<i>\1</i>", s)
s = _ITALIC_U.sub(r"<i>\1</i>", s)
s = _STRIKE.sub(r"<s>\1</s>", s)
s = _LINK.sub(r'<a href="\2">\1</a>', s)
parts[i] = _BULLET.sub(r"\1• ", s)
return "".join(parts)
def chunks(text: str, limit: int = LIMIT) -> list[str]:
text = text.strip()
if len(text) <= limit:
return [text] if text else []
out: list[str] = []
while len(text) > limit:
cut = _cut_point(text, limit)
out.append(text[:cut].rstrip())
text = text[cut:].lstrip()
if text:
out.append(text)
return out
def _cut_point(text: str, limit: int) -> int:
for sep in ("\n\n", "\n", ". ", " "):
cut = text.rfind(sep, limit // 2, limit)
if cut > 0:
return cut + len(sep)
return limit
def status_label(name: str, tool_input: dict[str, Any] | None = None) -> str:
label = _LABELS.get(name)
if label is not None:
return label
if name.startswith("mcp__"):
parts = name.split("__", 2)
server = parts[1]
tool = parts[2] if len(parts) == 3 else ""
return f"{server}: {tool}" if tool else server
hint = ""
if tool_input:
for key in ("description", "command", "file_path", "pattern", "query"):
value = tool_input.get(key)
if isinstance(value, str) and value.strip():
hint = value.strip().splitlines()[0][:60]
break
return f"{name} {hint}" if hint else name
+9 -1
View File
@@ -17,7 +17,7 @@ heavy setups (PRD §6 cites the ~95%→~71% tool-selection drop on flat
namespaces) but real and reachable. namespaces) but real and reachable.
The aggregator returns both the app and a ``{name: url}`` map; Phase The aggregator returns both the app and a ``{name: url}`` map; Phase
2.2's ``ClaudeCodeBackendAdapter`` plugs the map directly into ``ClaudeSdkBackend`` plugs the map directly into
``BackendOptions.mcp_servers``. ``/mcp/all/`` is NOT included in that ``BackendOptions.mcp_servers``. ``/mcp/all/`` is NOT included in that
map — claude-code-agents always get per-domain framing; only the map — claude-code-agents always get per-domain framing; only the
external MCP frontend (Phase 3.1) reverse-proxies the flat endpoint. external MCP frontend (Phase 3.1) reverse-proxies the flat endpoint.
@@ -33,6 +33,7 @@ from starlette.applications import Starlette
from starlette.routing import Mount from starlette.routing import Mount
from beaver_gateway.mcp.client_pool import build_http_proxy, build_stdio_proxy from beaver_gateway.mcp.client_pool import build_http_proxy, build_stdio_proxy
from beaver_gateway.mcp.redacting import RedactingMiddleware
from beaver_gateway.mcp.types import HttpMcp, PythonToolMcp, StdioMcp from beaver_gateway.mcp.types import HttpMcp, PythonToolMcp, StdioMcp
from beaver_gateway.mcp.wrap import build_python_tool_server from beaver_gateway.mcp.wrap import build_python_tool_server
@@ -69,6 +70,12 @@ def build_internal_app(
MCP tools into the Raycast wire). MCP tools into the Raycast wire).
""" """
servers: dict[str, FastMCP] = {spec.name: _build_server(spec) for spec in mcps} servers: dict[str, FastMCP] = {spec.name: _build_server(spec) for spec in mcps}
# The one place every tool result passes through on its way to the
# model. Attached per namespace rather than once at the top so that
# a direct ``call_tool`` on a child (the Raycast backend does that)
# is filtered too.
for server in servers.values():
server.add_middleware(RedactingMiddleware())
child_apps = { child_apps = {
name: s.http_app(transport="http", path="/") for name, s in servers.items() name: s.http_app(transport="http", path="/") for name, s in servers.items()
@@ -80,6 +87,7 @@ def build_internal_app(
all_app = None all_app = None
if servers: if servers:
all_server = _build_all_server(servers) all_server = _build_all_server(servers)
all_server.add_middleware(RedactingMiddleware())
all_app = all_server.http_app(transport="http", path="/") all_app = all_server.http_app(transport="http", path="/")
routes.append(Mount(f"/mcp/{ALL_NAMESPACE}", app=all_app)) routes.append(Mount(f"/mcp/{ALL_NAMESPACE}", app=all_app))
+13 -24
View File
@@ -84,23 +84,19 @@ class LenientStdioTransport(ClientTransport):
self, **session_kwargs: Unpack[SessionKwargs] self, **session_kwargs: Unpack[SessionKwargs]
) -> AsyncIterator[ClientSession]: ) -> AsyncIterator[ClientSession]:
errlog: TextIO = self.log_file if self.log_file is not None else sys.stderr errlog: TextIO = self.log_file if self.log_file is not None else sys.stderr
async with _lenient_stdio_client( async with (
StdioServerParameters( _lenient_stdio_client(
command=self.command, StdioServerParameters(
args=self.args, command=self.command, args=self.args, env=self.env, cwd=self.cwd
env=self.env, ),
cwd=self.cwd, errlog=errlog,
), ) as (read_stream, write_stream),
errlog=errlog, ClientSession(read_stream, write_stream, **session_kwargs) as session,
) as (read_stream, write_stream), ClientSession( ):
read_stream, write_stream, **session_kwargs
) as session:
yield session yield session
def __repr__(self) -> str: def __repr__(self) -> str:
return ( return f"<LenientStdioTransport(command={self.command!r}, args={self.args!r})>"
f"<LenientStdioTransport(command={self.command!r}, args={self.args!r})>"
)
@contextlib.asynccontextmanager @contextlib.asynccontextmanager
@@ -133,9 +129,7 @@ async def _lenient_stdio_client( # noqa: PLR0915 — mirrors mcp.client.stdio.s
command=command, command=command,
args=server.args, args=server.args,
env=( env=(
{**env_default, **server.env} {**env_default, **server.env} if server.env is not None else env_default
if server.env is not None
else env_default
), ),
errlog=errlog, errlog=errlog,
cwd=server.cwd, cwd=server.cwd,
@@ -164,9 +158,7 @@ async def _lenient_stdio_client( # noqa: PLR0915 — mirrors mcp.client.stdio.s
if not stripped: if not stripped:
continue continue
try: try:
message = types.JSONRPCMessage.model_validate_json( message = types.JSONRPCMessage.model_validate_json(stripped)
stripped
)
except Exception: # noqa: BLE001 — by design, see module doc except Exception: # noqa: BLE001 — by design, see module doc
_log.debug( _log.debug(
"lenient stdio: dropped non-JSON line: %r", "lenient stdio: dropped non-JSON line: %r",
@@ -194,10 +186,7 @@ async def _lenient_stdio_client( # noqa: PLR0915 — mirrors mcp.client.stdio.s
except anyio.ClosedResourceError: except anyio.ClosedResourceError:
await anyio.lowlevel.checkpoint() await anyio.lowlevel.checkpoint()
async with ( async with anyio.create_task_group() as tg, process:
anyio.create_task_group() as tg,
process,
):
tg.start_soon(stdout_reader) tg.start_soon(stdout_reader)
tg.start_soon(stdin_writer) tg.start_soon(stdin_writer)
try: try:
+85
View File
@@ -0,0 +1,85 @@
"""Redact credentials on the way out of an MCP tool.
A tool result is a wider channel than the log. It goes straight into the
model's context, from there into the turn record in Postgres, and from
there into the markdown transcript in ``💬 чаты`` — which Obsidian Sync
carries off the machine. One ``komodo`` deploy returns the resolved
compose file, ``environment:`` block and all, so a single call can put
every secret of a stack into all four places at once.
One filter, not a list of exceptions: every MCP the model can reach is
built into a ``FastMCP`` by :mod:`beaver_gateway.mcp.internal_app` —
``python_tool`` bundles like komodo, stdio subprocesses, remote HTTP
servers — and every route into one of them runs its middleware chain.
That covers the per-namespace ``/mcp/<name>/`` mounts claude-code talks
to, the ``/mcp/all`` bundle, the external MCP frontend reverse-proxying
into both, and the Raycast backend's direct ``call_tool``. A new MCP in
``config.py`` is covered the day it is added, without anyone
remembering to list it here.
What this does not reach: tools that never touch a FastMCP server — the
gateway's own ``gateway`` tools, and everything claude-code runs inside
its own process (``Bash``, ``Read``). Those are guarded by
:mod:`beaver_gateway.core.policy` and the vault mounts instead.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
import mcp.types as mt
from fastmcp.server.middleware import Middleware
from fastmcp.tools.base import ToolResult
from beaver_gateway.core.redact import redact, redact_data
if TYPE_CHECKING:
from fastmcp.server.middleware import CallNext, MiddlewareContext
__all__ = ["RedactingMiddleware"]
def _redact_block(block: mt.ContentBlock) -> mt.ContentBlock:
"""Mask a content block's text; binary blocks pass through."""
if isinstance(block, mt.TextContent):
masked = redact(block.text)
return (
block if masked == block.text else block.model_copy(update={"text": masked})
)
if isinstance(block, mt.EmbeddedResource) and isinstance(
block.resource, mt.TextResourceContents
):
masked = redact(block.resource.text)
if masked == block.resource.text:
return block
resource = block.resource.model_copy(update={"text": masked})
return block.model_copy(update={"resource": resource})
return block
class RedactingMiddleware(Middleware):
"""Mask credentials in whatever a tool call returns.
Idempotent, so mounting a server both on its own namespace and in
the ``/mcp/all`` bundle costs a second pass, not a double mask.
"""
async def on_call_tool(
self,
context: MiddlewareContext[mt.CallToolRequestParams],
call_next: CallNext[mt.CallToolRequestParams, ToolResult],
) -> ToolResult:
result = await call_next(context)
content = [_redact_block(block) for block in result.content]
structured = redact_data(result.structured_content)
if content == result.content and structured == result.structured_content:
return result
return ToolResult(
content=content,
structured_content=structured,
# Masked structured content no longer has to satisfy the
# tool's output schema; a non-None meta takes the
# ``CallToolResult`` path that skips that validation, the
# same trick ``ResponseLimitingMiddleware`` uses.
meta=result.meta if result.meta is not None else {},
)
+1 -3
View File
@@ -62,9 +62,7 @@ class PythonToolMcp(_BaseMcp):
tools: tuple[Callable[..., object], ...] tools: tuple[Callable[..., object], ...]
McpServerT = Annotated[ McpServerT = Annotated[StdioMcp | HttpMcp | PythonToolMcp, Field(discriminator="kind")]
StdioMcp | HttpMcp | PythonToolMcp, Field(discriminator="kind")
]
class McpServer: class McpServer:
+6
View File
@@ -27,6 +27,12 @@ class Settings(BaseSettings):
internal_mcp_port: int = 8765 internal_mcp_port: int = 8765
config_path: Path = Path("/config/config.py") config_path: Path = Path("/config/config.py")
claude_runner_user: str | None = None
"""Unix user the claude subprocess runs as. Needs the gateway to be root."""
claude_home: Path | None = None
"""``HOME`` for the claude subprocess (its ``~/.claude`` lives there)."""
raycast_bearer: str | None = None raycast_bearer: str | None = None
raycast_config_path: Path = Path("/config/raycast.json") raycast_config_path: Path = Path("/config/raycast.json")
raycast_device_id: str | None = None raycast_device_id: str | None = None
+31 -5
View File
@@ -1,32 +1,58 @@
"""SQLModel-backed persistence. """SQLModel-backed persistence.
Two tables — :class:`Token`, :class:`AuditLog` — plus a thin Tokens, audit, conversations, Agent SDK transcript entries and per-turn
:class:`Database` wrapper around an async SQLAlchemy engine. The usage, plus a thin :class:`Database` wrapper around an async SQLAlchemy
``GatewayRuntime`` carries the handle so auth (token verify / touch) engine. ``GatewayRuntime`` carries the handle.
and admin (token CRUD + audit listing) can reach it.
""" """
from beaver_gateway.storage.db import ( from beaver_gateway.storage.db import (
Database, Database,
append_audit, append_audit,
append_usage,
backfill_usage_deltas,
create_token, create_token,
list_active_tokens, list_active_tokens,
list_audit_records, list_audit_records,
list_tokens, list_tokens,
revoke_token, revoke_token,
touch_token, touch_token,
usage_deltas,
) )
from beaver_gateway.storage.models import AuditLog, Token from beaver_gateway.storage.models import (
AuditLog,
Conversation,
ConversationBinding,
Delivery,
InjectQueueItem,
RateLimit,
TelegramUpdate,
Token,
TranscriptEntry,
Usage,
)
from beaver_gateway.storage.session_store import PostgresSessionStore
__all__ = [ __all__ = [
"AuditLog", "AuditLog",
"Conversation",
"ConversationBinding",
"Database", "Database",
"Delivery",
"InjectQueueItem",
"PostgresSessionStore",
"RateLimit",
"TelegramUpdate",
"Token", "Token",
"TranscriptEntry",
"Usage",
"append_audit", "append_audit",
"append_usage",
"backfill_usage_deltas",
"create_token", "create_token",
"list_active_tokens", "list_active_tokens",
"list_audit_records", "list_audit_records",
"list_tokens", "list_tokens",
"revoke_token", "revoke_token",
"touch_token", "touch_token",
"usage_deltas",
] ]
@@ -0,0 +1,37 @@
"""Rewrite ``usage.cost_usd`` / ``model_usage`` as per-turn deltas.
Until 2026-09-01 the gateway stored ``ResultMessage.total_cost_usd`` verbatim,
which is cumulative for the claude process - summing the column overstated a
day by an order of magnitude. Run once after deploying the delta-aware
:func:`beaver_gateway.storage.append_usage`::
python -m beaver_gateway.storage.backfill_usage
Reads ``DATABASE_URL`` like the gateway (``.env`` included); safe to rerun.
"""
from __future__ import annotations
import asyncio
from dotenv import load_dotenv
from beaver_gateway.settings import Settings
from beaver_gateway.storage.db import Database, backfill_usage_deltas
async def _main() -> None:
load_dotenv(override=False)
settings = Settings() # ty: ignore[missing-argument]
db = Database(settings.database_url)
try:
await db.create_all()
async with db.session() as session:
changed = await backfill_usage_deltas(session)
finally:
await db.dispose()
print(f"usage rows rewritten: {changed}")
if __name__ == "__main__":
asyncio.run(_main())
+186 -3
View File
@@ -18,15 +18,17 @@ import json
from datetime import UTC, datetime from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any from typing import TYPE_CHECKING, Any
from sqlalchemy import inspect, text
from sqlalchemy.ext.asyncio import create_async_engine from sqlalchemy.ext.asyncio import create_async_engine
from sqlmodel import SQLModel, select from sqlmodel import SQLModel, col, select
from sqlmodel.ext.asyncio.session import AsyncSession from sqlmodel.ext.asyncio.session import AsyncSession
from beaver_gateway.storage.models import AuditLog, Token from beaver_gateway.storage.models import AuditLog, Token, Usage
if TYPE_CHECKING: if TYPE_CHECKING:
from collections.abc import Sequence from collections.abc import Sequence
from sqlalchemy import Connection
from sqlalchemy.ext.asyncio import AsyncEngine from sqlalchemy.ext.asyncio import AsyncEngine
@@ -72,9 +74,14 @@ class Database:
self._engine: AsyncEngine = create_async_engine(_to_async_url(url), echo=False) self._engine: AsyncEngine = create_async_engine(_to_async_url(url), echo=False)
async def create_all(self) -> None: async def create_all(self) -> None:
"""Idempotent ``CREATE TABLE IF NOT EXISTS`` for every model.""" """Create missing tables, then add columns models gained since.
Nullable columns only - that is the whole migration story until
Alembic is worth it.
"""
async with self._engine.begin() as conn: async with self._engine.begin() as conn:
await conn.run_sync(SQLModel.metadata.create_all) await conn.run_sync(SQLModel.metadata.create_all)
await conn.run_sync(_add_missing_columns)
def session(self) -> AsyncSession: def session(self) -> AsyncSession:
"""Open a fresh :class:`AsyncSession` (use as ``async with``). """Open a fresh :class:`AsyncSession` (use as ``async with``).
@@ -90,6 +97,39 @@ class Database:
await self._engine.dispose() await self._engine.dispose()
def _add_missing_columns(conn: Connection) -> None:
inspector = inspect(conn)
for table in SQLModel.metadata.sorted_tables:
existing = {c["name"] for c in inspector.get_columns(table.name)}
for column in table.columns:
if column.name in existing:
continue
kind = column.type.compile(conn.dialect)
ddl = f"ALTER TABLE {table.name} ADD COLUMN {column.name} {kind}"
default = _column_default(column)
if default is not None:
ddl += f" DEFAULT {default}"
conn.execute(text(ddl))
def _column_default(column: Any) -> str | None:
"""Literal for ``ADD COLUMN ... DEFAULT`` so old rows get the model default."""
server_default = getattr(column, "server_default", None)
if server_default is not None:
return str(server_default.arg.text)
default = getattr(column, "default", None)
if default is None or default.is_callable:
return None
value = default.arg
if isinstance(value, bool):
return "TRUE" if value else "FALSE"
if isinstance(value, int | float):
return repr(value)
if isinstance(value, str):
return "'" + value.replace("'", "''") + "'"
return None
# ---- Token CRUD --------------------------------------------------------- # ---- Token CRUD ---------------------------------------------------------
@@ -183,13 +223,156 @@ async def list_audit_records(
return result.all() return result.all()
# ---- Usage --------------------------------------------------------------
async def append_usage(session: AsyncSession, row: Usage) -> None:
"""Persist a turn's usage with ``cost_usd`` / ``model_usage`` as per-turn deltas.
``row`` arrives with the SDK's cumulative ``total_cost_usd`` and
``model_usage`` in ``cost_usd`` / ``model_usage``; they move to the
``*_total`` columns and the stored values become the difference against
the previous row of the same session (see :func:`usage_deltas`).
"""
prev = await _last_usage(session, row.session_id)
usage_deltas(row, prev)
session.add(row)
await session.commit()
async def _last_usage(session: AsyncSession, session_id: str | None) -> Usage | None:
if not session_id:
return None
stmt = (
select(Usage)
.where(col(Usage.session_id) == session_id)
.order_by(col(Usage.ts).desc(), col(Usage.id).desc())
.limit(1)
)
return (await session.exec(stmt)).first()
# Counters inside ``ResultMessage.model_usage[model]``; everything else there
# (provider, costBasis, contextWindow, maxOutputTokens, ...) is a constant.
_MODEL_USAGE_COUNTERS = (
"inputTokens",
"outputTokens",
"cacheReadInputTokens",
"cacheCreationInputTokens",
"webSearchRequests",
"costUSD",
)
def usage_deltas(row: Usage, prev: Usage | None) -> None:
"""Turn the SDK's cumulative cost counters on ``row`` into per-turn deltas.
``total_cost_usd`` and ``model_usage`` in a ``ResultMessage`` are running
totals of the claude process: they grow with every turn of the session and
start over when the process is respawned (rotation, resume after a
restart). A row fresh from the SDK (``cost_usd_total is None``) first moves
the raw values into ``cost_usd_total`` / ``model_usage_total``; then
``cost_usd`` / ``model_usage`` become ``current - previous`` against
``prev`` (the last row of the same session), or the raw value when there
is no previous row or the counter went backwards - a fresh process.
Idempotent on rows that already carry their totals, which is what the
backfill relies on.
"""
if row.cost_usd_total is None and row.model_usage_total is None:
row.cost_usd_total = row.cost_usd
row.model_usage_total = row.model_usage
prev_cost = prev.cost_usd_total if prev is not None else None
prev_models = prev.model_usage_total if prev is not None else None
row.cost_usd = _counter_delta(row.cost_usd_total, prev_cost)
row.model_usage = _model_usage_delta(row.model_usage_total, prev_models)
def _counter_delta(current: float | None, previous: float | None) -> float | None:
if current is None:
return None
if previous is None or current < previous:
return current
return round(current - previous, 6)
def _model_usage_delta(
current: dict[str, Any] | None, previous: dict[str, Any] | None
) -> dict[str, Any] | None:
if current is None:
return None
out: dict[str, Any] = {}
for model, fields in current.items():
if not isinstance(fields, dict):
out[model] = fields
continue
before = (previous or {}).get(model)
before = before if isinstance(before, dict) else {}
delta = dict(fields)
for key in _MODEL_USAGE_COUNTERS:
if key not in fields:
continue
cur, prev = fields.get(key), before.get(key)
if not isinstance(cur, int | float) or isinstance(cur, bool):
continue
prev_num = (
prev
if isinstance(prev, int | float) and not isinstance(prev, bool)
else None
)
value = _counter_delta(cur, prev_num)
delta[key] = (
round(value or 0, 6) if isinstance(cur, float) else int(value or 0)
)
out[model] = delta
return out
async def backfill_usage_deltas(session: AsyncSession) -> int:
"""Rewrite ``cost_usd`` / ``model_usage`` of every row as per-turn deltas.
Rows written before the ``*_total`` columns existed hold the SDK's
cumulative values in ``cost_usd`` / ``model_usage``; they are moved to the
totals first. Rows are replayed per session in time order. Idempotent:
a second run rewrites nothing. Returns the number of rows changed.
"""
stmt = select(Usage).order_by(col(Usage.session_id), col(Usage.ts), col(Usage.id))
rows = (await session.exec(stmt)).all()
last_by_session: dict[str, Usage] = {}
changed = 0
for row in rows:
before = (
row.cost_usd,
row.model_usage,
row.cost_usd_total,
row.model_usage_total,
)
prev = last_by_session.get(row.session_id) if row.session_id else None
usage_deltas(row, prev)
after = (
row.cost_usd,
row.model_usage,
row.cost_usd_total,
row.model_usage_total,
)
if after != before:
session.add(row)
changed += 1
if row.session_id:
last_by_session[row.session_id] = row
await session.commit()
return changed
__all__ = [ __all__ = [
"Database", "Database",
"append_audit", "append_audit",
"append_usage",
"backfill_usage_deltas",
"create_token", "create_token",
"list_active_tokens", "list_active_tokens",
"list_audit_records", "list_audit_records",
"list_tokens", "list_tokens",
"revoke_token", "revoke_token",
"touch_token", "touch_token",
"usage_deltas",
] ]
+258 -11
View File
@@ -1,6 +1,6 @@
"""SQLModel tables. """SQLModel tables.
Four tables, all flat, no FK relationships modelled (``actor`` and Flat tables, no FK relationships modelled (``actor`` and
``agent_name`` are stored as strings — joining audit→token by name is ``agent_name`` are stored as strings — joining audit→token by name is
fine at this volume; we'll introduce FKs when the admin UI actually fine at this volume; we'll introduce FKs when the admin UI actually
demands them). demands them).
@@ -23,8 +23,10 @@ needs an id uses ``Optional[int]`` so SQLAlchemy can autoincrement.
from __future__ import annotations from __future__ import annotations
from datetime import UTC, datetime from datetime import UTC, datetime
from typing import Any
from sqlalchemy import UniqueConstraint from sqlalchemy import JSON, Column, Index, UniqueConstraint, text
from sqlalchemy.dialects.postgresql import JSONB
from sqlmodel import Field, SQLModel from sqlmodel import Field, SQLModel
@@ -72,15 +74,16 @@ class AuditLog(SQLModel, table=True):
class Conversation(SQLModel, table=True): class Conversation(SQLModel, table=True):
"""One chat thread, scoped to a frontend. """One conversation (§3.1): a master thread, a branch, a deep chat or a job.
``external_id`` is the identifier the frontend uses to find this ``external_id`` is the public id (uuid) every frontend, the API and the
thread again on the next request — for the markdown frontend it's a usage table refer to; ``frontend`` names the frontend that created the
uuid we mint and persist into the file's frontmatter, for the row (``markdown``, ``api``, ``mcp``, ``system``). Where a conversation
anthropic frontend it'd be the same metadata.conversation_id the is *visible* lives in :class:`ConversationBinding`.
client passes. Unique per ``(frontend, external_id)`` because two
frontends sharing a uuid is fine; the same frontend reusing one is ``running_turn`` survives a restart: a non-null value at startup means
a bug. the gateway died mid-turn and the transcript needs its open ``tool_use``
closed before the session is resumed (``core/conversations``).
""" """
__tablename__ = "conversations" __tablename__ = "conversations"
@@ -92,8 +95,125 @@ class Conversation(SQLModel, table=True):
frontend: str = Field(index=True) frontend: str = Field(index=True)
external_id: str = Field(index=True) external_id: str = Field(index=True)
agent_name: str = Field(index=True) agent_name: str = Field(index=True)
session_id: str | None = Field(default=None, index=True)
kind: str = Field(default="deep", index=True)
parent_id: int | None = Field(default=None, index=True)
title: str | None = Field(default=None)
status: str = Field(default="open", index=True)
running_turn: str | None = Field(default=None)
pending_question: bool = Field(default=False)
flags: dict[str, Any] = Field(
default_factory=dict,
sa_column=Column(
JSON().with_variant(JSONB(), "postgresql"),
nullable=False,
server_default=text("'{}'"),
),
)
created_at: datetime = Field(default_factory=_utcnow) created_at: datetime = Field(default_factory=_utcnow)
updated_at: datetime = Field(default_factory=_utcnow) updated_at: datetime = Field(default_factory=_utcnow)
last_user_activity_at: datetime | None = Field(default=None)
last_activity_at: datetime | None = Field(default=None)
class ConversationBinding(SQLModel, table=True):
"""Where a conversation shows up: a Telegram topic id, a vault-relative path.
Invariant (§3.1): at most one *visible* binding per frontend per
conversation - enforced by the partial unique index. The same external
id may point at several conversations over time (a renamed topic gets a
new branch), only one of them visible.
"""
__tablename__ = "conversation_bindings"
__table_args__ = (
Index(
"uq_binding_visible",
"conversation_id",
"frontend",
unique=True,
postgresql_where=text("visible"),
sqlite_where=text("visible"),
),
Index("ix_binding_lookup", "frontend", "external_id"),
)
id: int | None = Field(default=None, primary_key=True)
conversation_id: int = Field(index=True)
frontend: str
external_id: str
visible: bool = Field(default=True)
created_at: datetime = Field(default_factory=_utcnow)
class InjectQueueItem(SQLModel, table=True):
"""Persisted per-conversation queue (§3.4), ``urgent > user > wake > normal``.
``status`` walks ``queued -> running -> done``; a row still ``running``
at startup was cut by a restart and becomes ``interrupted`` - it is
never re-run, the agent gets an inject saying so instead.
"""
__tablename__ = "inject_queue"
id: int | None = Field(default=None, primary_key=True)
conversation_id: int = Field(index=True)
priority: str = Field(index=True)
origin: str = Field(default="system")
text: str
status: str = Field(default="queued", index=True)
turn_id: str | None = Field(default=None)
interrupted_turn: bool | None = Field(default=None)
"""An urgent inject that cut a running turn; the header tells the model
the ``Request interrupted`` tool result above was this, not a refusal."""
created_at: datetime = Field(default_factory=_utcnow)
delivered_at: datetime | None = Field(default=None)
class TelegramUpdate(SQLModel, table=True):
"""Inbox of the Telegram frontend (§3.8).
The update is stored before the poll offset moves past it and handled
from here, so a restart neither loses nor duplicates a message.
"""
__tablename__ = "telegram_updates"
update_id: int = Field(primary_key=True, sa_column_kwargs={"autoincrement": False})
payload: dict[str, Any] = Field(
sa_column=Column(JSON().with_variant(JSONB(), "postgresql"), nullable=False)
)
received_at: datetime = Field(default_factory=_utcnow)
processed_at: datetime | None = Field(default=None, index=True)
error: str | None = Field(default=None)
class Delivery(SQLModel, table=True):
"""Outbox (§3.8): a reply is a row first and a Telegram message second.
``status`` walks ``queued -> sent`` with retries on the way, ``failed``
only when Telegram rejects the row for good (unknown thread, blocked
bot). ``dedupe_key`` keeps one row per turn chunk across restarts.
"""
__tablename__ = "deliveries"
id: int | None = Field(default=None, primary_key=True)
conversation_id: int | None = Field(default=None, index=True)
frontend: str = Field(default="telegram", index=True)
chat_id: int
thread_id: int | None = Field(default=None)
text: str
plain: bool = Field(default=False)
turn_id: str | None = Field(default=None)
dedupe_key: str | None = Field(default=None, unique=True)
status: str = Field(default="queued", index=True)
attempts: int = Field(default=0)
next_attempt_at: datetime = Field(default_factory=_utcnow, index=True)
last_error: str | None = Field(default=None)
message_id: int | None = Field(default=None)
created_at: datetime = Field(default_factory=_utcnow)
sent_at: datetime | None = Field(default=None)
class ConversationMessage(SQLModel, table=True): class ConversationMessage(SQLModel, table=True):
@@ -125,4 +245,131 @@ class ConversationMessage(SQLModel, table=True):
created_at: datetime = Field(default_factory=_utcnow) created_at: datetime = Field(default_factory=_utcnow)
__all__ = ["AuditLog", "Conversation", "ConversationMessage", "Token"] class TranscriptEntry(SQLModel, table=True):
"""One Agent SDK transcript line, mirrored from the session store protocol.
``(project_key, session_id, subpath)`` is the ``SessionKey``; ``subpath``
is ``""`` for the main transcript so the unique constraints stay simple.
``uuid`` is the SDK's idempotency key - the partial unique index rejects
a replayed batch, entries without a uuid are appended as-is.
"""
__tablename__ = "transcript_entries"
__table_args__ = (
UniqueConstraint(
"project_key", "session_id", "subpath", "seq", name="uq_transcript_seq"
),
Index(
"uq_transcript_uuid",
"project_key",
"session_id",
"subpath",
"uuid",
unique=True,
postgresql_where=text("uuid IS NOT NULL"),
sqlite_where=text("uuid IS NOT NULL"),
),
)
id: int | None = Field(default=None, primary_key=True)
project_key: str = Field(index=True)
session_id: str = Field(index=True)
subpath: str = Field(default="")
seq: int
uuid: str | None = Field(default=None)
entry: dict[str, Any] = Field(
sa_column=Column(JSON().with_variant(JSONB(), "postgresql"), nullable=False)
)
origin: str | None = Field(default=None)
source: str | None = Field(default=None)
turn_id: str | None = Field(default=None)
created_at: datetime = Field(default_factory=_utcnow)
class Usage(SQLModel, table=True):
"""Per-turn token accounting from ``ResultMessage``.
Token columns come from ``ResultMessage.usage`` and are sums over the
turn's API calls. ``cost_usd`` and ``model_usage`` are **per-turn deltas**:
the SDK reports ``total_cost_usd`` and ``model_usage`` cumulatively for
the claude process (growing across the turns of a session, reset when
the process is respawned), so :func:`append_usage` stores the raw values
in ``cost_usd_total`` / ``model_usage_total`` and the difference against
the previous row of the same session here. Summing ``cost_usd`` over rows
is therefore meaningful; summing ``cost_usd_total`` is not.
"""
__tablename__ = "usage"
id: int | None = Field(default=None, primary_key=True)
ts: datetime = Field(default_factory=_utcnow, index=True)
agent_name: str = Field(index=True)
conversation_id: str | None = Field(default=None, index=True)
session_id: str | None = Field(default=None, index=True)
model: str
effort: str | None = Field(default=None)
input_tokens: int = 0
output_tokens: int = 0
cache_read_tokens: int = 0
cache_creation_tokens: int = 0
context_tokens: int | None = Field(default=None)
"""Input of the turn's last API call - the real context size."""
cost_usd: float | None = Field(default=None)
duration_ms: int | None = Field(default=None)
num_turns: int | None = Field(default=None)
model_usage: dict[str, Any] | None = Field(
default=None,
sa_column=Column(JSON().with_variant(JSONB(), "postgresql"), nullable=True),
)
"""Per-model tokens, cost and web searches - per-turn delta (see class doc)."""
cost_usd_total: float | None = Field(default=None)
"""``ResultMessage.total_cost_usd`` verbatim: cumulative for the claude process."""
model_usage_total: dict[str, Any] | None = Field(
default=None,
sa_column=Column(JSON().with_variant(JSONB(), "postgresql"), nullable=True),
)
"""``ResultMessage.model_usage`` verbatim: cumulative for the claude process."""
class RateLimit(SQLModel, table=True):
"""One ``RateLimitEvent`` from the SDK; the newest row per ``window`` is the state.
The quota is per subscription, not per agent or conversation - the row
only remembers which session reported it.
"""
__tablename__ = "rate_limits"
id: int | None = Field(default=None, primary_key=True)
ts: datetime = Field(default_factory=_utcnow, index=True)
window: str = Field(index=True)
status: str
utilization: float | None = Field(default=None)
resets_at: datetime | None = Field(default=None)
overage_status: str | None = Field(default=None)
overage_resets_at: datetime | None = Field(default=None)
agent_name: str | None = Field(default=None)
session_id: str | None = Field(default=None)
raw: dict[str, Any] = Field(
default_factory=dict,
sa_column=Column(
JSON().with_variant(JSONB(), "postgresql"),
nullable=False,
server_default=text("'{}'"),
),
)
__all__ = [
"AuditLog",
"Conversation",
"ConversationBinding",
"ConversationMessage",
"Delivery",
"InjectQueueItem",
"RateLimit",
"TelegramUpdate",
"Token",
"TranscriptEntry",
"Usage",
]
+126
View File
@@ -0,0 +1,126 @@
"""``SessionStore`` adapter for the Claude Agent SDK on top of :class:`Database`.
Entries are stored verbatim as JSON (``jsonb`` on Postgres), ordered by a
per-key ``seq``. ``append`` is idempotent on ``entry["uuid"]`` because the
SDK re-delivers a batch on retry; entries without a uuid are appended as
they come. Runs on SQLite too - the conformance suite uses that in tests.
"""
from __future__ import annotations
from typing import TYPE_CHECKING, Any, cast
from claude_agent_sdk import SessionStore
from sqlalchemy import delete, func
from sqlmodel import col, select
from beaver_gateway.storage.models import TranscriptEntry
if TYPE_CHECKING:
from claude_agent_sdk.types import (
SessionKey,
SessionListSubkeysKey,
SessionStoreEntry,
)
from beaver_gateway.storage.db import Database
__all__ = ["PostgresSessionStore"]
class PostgresSessionStore(SessionStore):
def __init__(self, db: Database) -> None:
self._db = db
async def append(self, key: SessionKey, entries: list[SessionStoreEntry]) -> None:
if not entries:
return
project_key, session_id, subpath = _parts(key)
wanted = [
u for u in (e.get("uuid") for e in entries) if isinstance(u, str) and u
]
async with self._db.session() as session:
known: set[str] = set()
if wanted:
result = await session.exec(
select(TranscriptEntry.uuid).where(
TranscriptEntry.project_key == project_key,
TranscriptEntry.session_id == session_id,
TranscriptEntry.subpath == subpath,
col(TranscriptEntry.uuid).in_(wanted),
)
)
known = {u for u in result.all() if u is not None}
seq_result = await session.exec(
select(func.max(TranscriptEntry.seq)).where(
TranscriptEntry.project_key == project_key,
TranscriptEntry.session_id == session_id,
TranscriptEntry.subpath == subpath,
)
)
seq = seq_result.one() or 0
for entry in entries:
uuid = entry.get("uuid")
uuid_str = uuid if isinstance(uuid, str) and uuid else None
if uuid_str is not None:
if uuid_str in known:
continue
known.add(uuid_str)
seq += 1
session.add(
TranscriptEntry(
project_key=project_key,
session_id=session_id,
subpath=subpath,
seq=seq,
uuid=uuid_str,
entry=cast("dict[str, Any]", dict(entry)),
)
)
await session.commit()
async def load(self, key: SessionKey) -> list[SessionStoreEntry] | None:
project_key, session_id, subpath = _parts(key)
async with self._db.session() as session:
result = await session.exec(
select(TranscriptEntry.entry)
.where(
TranscriptEntry.project_key == project_key,
TranscriptEntry.session_id == session_id,
TranscriptEntry.subpath == subpath,
)
.order_by(col(TranscriptEntry.seq))
)
rows = result.all()
if not rows:
return None
return [cast("SessionStoreEntry", row) for row in rows]
async def list_subkeys(self, key: SessionListSubkeysKey) -> list[str]:
async with self._db.session() as session:
result = await session.exec(
select(TranscriptEntry.subpath)
.where(
TranscriptEntry.project_key == key["project_key"],
TranscriptEntry.session_id == key["session_id"],
TranscriptEntry.subpath != "",
)
.distinct()
)
return sorted(result.all())
async def delete(self, key: SessionKey) -> None:
project_key, session_id, subpath = _parts(key)
stmt = delete(TranscriptEntry).where(
col(TranscriptEntry.project_key) == project_key,
col(TranscriptEntry.session_id) == session_id,
)
if key.get("subpath"):
stmt = stmt.where(col(TranscriptEntry.subpath) == subpath)
async with self._db.session() as session:
await session.execute(stmt) # ty: ignore[deprecated]
await session.commit()
def _parts(key: SessionKey) -> tuple[str, str, str]:
return key["project_key"], key["session_id"], key.get("subpath") or ""
+9
View File
@@ -0,0 +1,9 @@
"""Print a TEST_DATABASE_URL for the compose postgres (`make db`) from beaver-agent/.env."""
from dotenv import dotenv_values
v = dotenv_values("../beaver-agent/.env")
user = v.get("POSTGRES_USER") or "beaver"
port = v.get("PORT_POSTGRES") or "5432"
db = v.get("POSTGRES_DB") or "beaver"
print(f"postgresql://{user}:{v['POSTGRES_PASSWORD']}@127.0.0.1:{port}/{db}")
+492
View File
@@ -0,0 +1,492 @@
import asyncio
import tempfile
from datetime import UTC, datetime, timedelta
from pathlib import Path
from typing import Any, cast
import pytest
from claude_agent_sdk import (
AssistantMessage,
RateLimitEvent,
RateLimitInfo,
ResultMessage,
TextBlock,
ToolResultBlock,
ToolUseBlock,
UserMessage,
)
from httpx import ASGITransport, AsyncClient
from test_conversations import ScriptedClient, World
from beaver_gateway.core.conversation_store import rewrite_messages
from beaver_gateway.core.auth import TokenStore
from beaver_gateway.core.registry import McpRegistry
from beaver_gateway.core.transcript import build_entries
from beaver_gateway.frontends.admin import AdminFrontend
from beaver_gateway.frontends.admin.frontend import build_app as build_admin
from beaver_gateway.frontends.api import ApiFrontend
from beaver_gateway.frontends.api.frontend import build_app as build_api
from beaver_gateway.frontends.base import Frontend, GatewayRuntime
from beaver_gateway.frontends.root import build_root_app
from beaver_gateway.storage.models import RateLimit, Usage
TOKEN = "tok"
HEADERS = {"Authorization": f"Bearer {TOKEN}"}
class ToolClient(ScriptedClient):
async def receive_response(self):
yield AssistantMessage(
content=[ToolUseBlock(id="tu_1", name="Bash", input={"command": "ls"})],
model="m",
)
yield AssistantMessage(
content=[ToolUseBlock(id="tu_2", name="Read", input={"file_path": "x"})],
model="m",
parent_tool_use_id="tu_1",
)
yield RateLimitEvent(
rate_limit_info=RateLimitInfo(
status="allowed_warning",
resets_at=int(datetime.now(UTC).timestamp()) + 3600,
rate_limit_type="five_hour",
utilization=0.8,
raw={"status": "allowed_warning"},
),
uuid="r",
session_id=self.session_id,
)
hold = ScriptedClient.hold
if hold is not None:
await hold.wait()
yield UserMessage(content=[ToolResultBlock(tool_use_id="tu_1", content="done")])
yield AssistantMessage(content=[TextBlock(text="ok")], model="m")
yield ResultMessage(
subtype="success",
duration_ms=1,
duration_api_ms=1,
is_error=False,
num_turns=1,
session_id=self.session_id,
stop_reason="end_turn",
total_cost_usd=0.5,
usage={"input_tokens": 10, "output_tokens": 5},
model_usage=cast(
"Any",
{
"claude-opus-5": {
"inputTokens": 8,
"outputTokens": 4,
"cacheReadInputTokens": 0,
"cacheCreationInputTokens": 0,
"webSearchRequests": 1,
"costUSD": 0.4,
},
"claude-haiku-4-5": {
"inputTokens": 2,
"outputTokens": 1,
"cacheReadInputTokens": 0,
"cacheCreationInputTokens": 0,
"webSearchRequests": 0,
"costUSD": 0.1,
},
},
),
)
class Api:
def __init__(self, world: World, memory_root: Path | None = None) -> None:
self.world = world
self.store = TokenStore(world.db, bootstrap={"t": TOKEN})
self.runtime = GatewayRuntime(
agents=world.conversations._agents, # noqa: SLF001
mcps=McpRegistry([]),
backends={"a": world.backend, "d": world.deep_backend},
token_store=self.store,
db=world.db,
admin_user="admin",
admin_pass="secret",
session_secret="s" * 32,
frontends=(world.api, world.markdown),
conversations=world.conversations,
bus=world.bus,
pool=world.pool,
)
self.app = build_api(self.runtime, memory_root=memory_root)
self.http = AsyncClient(
transport=ASGITransport(app=self.app), base_url="http://api"
)
def root(self, *frontends: Frontend) -> AsyncClient:
return AsyncClient(
transport=ASGITransport(app=build_root_app(frontends)), base_url="http://gw"
)
async def get(self, path: str, params: dict[str, Any] | None = None) -> Any:
res = await self.http.get(path, params=params, headers=HEADERS)
assert res.status_code == 200, res.text
return res.json()
@pytest.fixture
async def world() -> World:
root = Path(tempfile.mkdtemp(prefix="beaver-api-"))
w = await World(root).setup()
yield w
await w.conversations.stop()
await w.pool.close_all()
await w.db.dispose()
async def seed_usage(world: World, rows: list[dict[str, Any]]) -> None:
async with world.db.session() as session:
for row in rows:
session.add(Usage(**row))
await session.commit()
async def test_usage_groups_by_agent_day_and_model(world: World) -> None:
api = Api(world)
now = datetime.now(UTC).replace(tzinfo=None)
await seed_usage(
world,
[
{
"ts": now - timedelta(hours=1),
"agent_name": "a",
"conversation_id": "c1",
"model": "claude-opus-5",
"input_tokens": 10,
"output_tokens": 20,
"cache_read_tokens": 100,
"cache_creation_tokens": 5,
"cost_usd": 0.5,
"model_usage": {
"claude-opus-5": {
"inputTokens": 10,
"outputTokens": 20,
"cacheReadInputTokens": 100,
"cacheCreationInputTokens": 5,
"webSearchRequests": 2,
"costUSD": 0.5,
}
},
},
{
"ts": now - timedelta(days=2),
"agent_name": "d",
"conversation_id": "c2",
"model": "claude-sonnet-5",
"input_tokens": 1,
"output_tokens": 2,
"cost_usd": 0.1,
},
],
)
by_agent = await api.get("/usage", {"group_by": "agent"})
assert [r["agent"] for r in by_agent["rows"]] == ["a"]
assert by_agent["total"] == {
"turns": 1,
"input": 10,
"output": 20,
"cache_read": 100,
"cache_creation": 5,
"cost_usd": 0.5,
"web_searches": 2,
}
since = (datetime.now(UTC) - timedelta(days=3)).isoformat()
by_day = await api.get("/usage", {"group_by": "day", "since": since})
assert len(by_day["rows"]) == 2
assert by_day["rows"][0]["day"] < by_day["rows"][1]["day"]
by_model = await api.get("/usage", {"group_by": "model", "since": since})
models = {r["model"]: r for r in by_model["rows"]}
assert models["claude-opus-5"]["web_searches"] == 2
assert models["claude-sonnet-5"]["output"] == 2
by_conv = await api.get("/usage", {"group_by": "conversation", "since": since})
assert {r["conversation"] for r in by_conv["rows"]} == {"c1", "c2"}
bad = await api.http.get("/usage", params={"group_by": "x"}, headers=HEADERS)
assert bad.status_code == 400
async def test_limits_report_latest_window_with_gateway_spend(world: World) -> None:
api = Api(world)
now = datetime.now(UTC)
async with world.db.session() as session:
session.add(
RateLimit(
window="five_hour",
status="allowed",
utilization=0.2,
resets_at=now + timedelta(hours=4),
)
)
session.add(
RateLimit(
window="five_hour",
status="allowed_warning",
utilization=0.9,
resets_at=now + timedelta(hours=4),
)
)
session.add(RateLimit(window="seven_day", status="allowed", utilization=0.3))
await session.commit()
await seed_usage(
world,
[
{
"ts": (now - timedelta(minutes=30)).replace(tzinfo=None),
"agent_name": "a",
"model": "m",
"output_tokens": 7,
"cost_usd": 0.25,
}
],
)
out = await api.get("/limits")
windows = {w["window"]: w for w in out["windows"]}
assert windows["five_hour"]["utilization"] == 0.9
assert windows["five_hour"]["status"] == "allowed_warning"
assert windows["five_hour"]["gateway"]["output"] == 7
assert windows["five_hour"]["gateway"]["cost_usd"] == 0.25
assert windows["seven_day"]["utilization"] == 0.3
assert [w["window"] for w in out["windows"]] == ["five_hour", "seven_day"]
assert len(out["history"]) == 3
async def test_memory_tree_and_file(world: World) -> None:
root = world.root / "zone"
(root / "дни").mkdir(parents=True)
(root / "дни" / "2026-08-27.md").write_text("# day", encoding="utf-8")
(root / "состояние.md").write_text("# state", encoding="utf-8")
(root / ".hidden").write_text("x", encoding="utf-8")
api = Api(world, memory_root=root)
tree = await api.get("/memory")
names = [n["name"] for n in tree["tree"]]
assert names == ["дни", "состояние.md"]
assert tree["tree"][0]["children"][0]["path"] == "дни/2026-08-27.md"
file = await api.get("/memory/file", {"path": "дни/2026-08-27.md"})
assert file["content"] == "# day"
escape = await api.http.get(
"/memory/file", params={"path": "../w.db"}, headers=HEADERS
)
assert escape.status_code == 404
unset = Api(world)
res = await unset.http.get("/memory", headers=HEADERS)
assert res.status_code == 404
async def test_describe_snapshots_open_tools_and_records_rate_limit(
world: World,
) -> None:
world.backend._factory = ToolClient # noqa: SLF001
api = Api(world)
async def record(event: Any) -> None:
async with world.db.session() as session:
session.add(
Usage(
agent_name=event.agent_name,
conversation_id=event.conversation_id,
model=event.model,
cost_usd=event.usage.cost_usd,
model_usage=event.usage.model_usage,
)
)
await session.commit()
world.backend._usage_sink = record # noqa: SLF001
conv = await world.conversations.create(kind="master", agent="a", origin="test")
ScriptedClient.hold = asyncio.Event()
await world.conversations.post(conv, "go")
await asyncio.sleep(0.3)
described = await api.get(f"/conversations/{conv.external_id}")
turn = described["turn"]
assert turn is not None and turn["id"] == described["running_turn"]
tools = {t["tool_use_id"]: t for t in turn["tools"]}
assert tools["tu_1"]["name"] == "Bash" and tools["tu_1"]["ended_at"] is None
assert tools["tu_2"]["parent_tool_use_id"] == "tu_1"
ScriptedClient.hold.set()
await world.settle(conv, 1)
described = await api.get(f"/conversations/{conv.external_id}")
assert described["turn"] is None
limits = await api.get("/limits")
assert limits["windows"][0]["utilization"] == 0.8
assert limits["windows"][0]["gateway"]["cost_usd"] == 0.5
usage = await api.get("/usage", {"group_by": "model"})
assert {r["model"] for r in usage["rows"]} == {"claude-opus-5", "claude-haiku-4-5"}
assert usage["total"]["web_searches"] == 1
conv = await world.conversations.get(conv.external_id)
assert conv is not None and conv.session_id is not None
await world.store.append(
world.key(conv.session_id),
build_entries(
[
{"role": "user", "content": "go"},
{"role": "assistant", "content": [{"type": "text", "text": "ok"}]},
],
session_id=conv.session_id,
cwd=str(world.root),
model="m",
),
)
history = await api.get(f"/conversations/{conv.external_id}/history")
assert [m["role"] for m in history["messages"]] == ["user", "assistant"]
entries = await api.get(f"/conversations/{conv.external_id}/entries")
assert entries["total"] == len(entries["entries"]) == 2
page = await api.get(
f"/conversations/{conv.external_id}/entries", {"limit": 1, "offset": 0}
)
assert page["offset"] == 0 and len(page["entries"]) == 1
async def test_tokens_and_audit_need_admin_scope(world: World) -> None:
api = Api(world)
api.store.grant("api-only", "weak", scope="api")
weak = {"Authorization": "Bearer weak"}
assert (await api.http.get("/tokens", headers=weak)).status_code == 403
created = await api.http.post(
"/tokens", json={"name": "cursor", "scope": "mcp"}, headers=HEADERS
)
assert created.status_code == 201
plaintext = created.json()["plaintext"]
await api.store.invalidate()
identity = await api.store.verify(plaintext)
assert identity is not None and identity.scope == "mcp"
listed = await api.get("/tokens")
assert [t["name"] for t in listed["tokens"]] == ["cursor"]
token_id = listed["tokens"][0]["id"]
revoked = await api.http.post(f"/tokens/{token_id}/revoke", headers=HEADERS)
assert revoked.status_code == 200
assert (await api.get("/tokens"))["tokens"] == []
audit = await api.get("/audit")
assert [r["kind"] for r in audit["records"]] == ["token_revoke", "token_create"]
async def test_admin_login_hands_out_the_ui_bearer(world: World) -> None:
api = Api(world)
ui_dir = world.root / "build"
ui_dir.mkdir()
(ui_dir / "index.html").write_text("<html>ui</html>", encoding="utf-8")
admin = AdminFrontend(ui_dir=ui_dir)
admin._app = build_admin(api.runtime, token="ui-bearer", ui_dir=ui_dir) # noqa: SLF001
api.store.grant("admin-ui", "ui-bearer")
api_fe = ApiFrontend()
api_fe._app = api.app # noqa: SLF001
http = api.root(admin, api_fe)
assert (await http.get("/admin/auth/session")).status_code == 401
bad = await http.post(
"/admin/auth/login", json={"username": "admin", "password": "nope"}
)
assert bad.status_code == 401
ok = await http.post(
"/admin/auth/login", json={"username": "admin", "password": "secret"}
)
assert ok.status_code == 200
session = (await http.get("/admin/auth/session")).json()
assert session == {"user": "admin", "token": "ui-bearer"}
spa = await http.get("/admin/conversations/abc")
assert spa.status_code == 200 and spa.text == "<html>ui</html>"
root = await http.get("/")
assert root.status_code == 307 and root.headers["location"] == "/admin/"
assert (await http.get("/admin")).status_code == 307
health = await http.get("/healthz")
assert health.json()["frontends"] == ["/admin", "/api"]
escape = await http.get("/admin/%2e%2e/pyproject.toml")
assert escape.status_code == 200 and escape.text == "<html>ui</html>"
me = await api.http.get("/agents", headers={"Authorization": "Bearer ui-bearer"})
assert me.status_code == 200
assert (await http.post("/admin/auth/logout")).status_code == 204
assert (await http.get("/admin/auth/session")).status_code == 401
async def test_pre_sdk_rows_show_canonical_history_and_a_title(world: World) -> None:
api = Api(world)
conv = await world.conversations.create(kind="deep", agent="d", origin="markdown")
async with world.db.session() as session:
await rewrite_messages(
session,
conversation_id=cast("int", conv.id),
messages=[
{"role": "user", "content": "так смотри, план на май\nвторая строка"},
{"role": "assistant", "content": [{"type": "text", "text": "ок"}]},
],
)
rows = (await api.get("/conversations"))["conversations"]
assert [r["title"] for r in rows if r["id"] == conv.external_id] == [
"так смотри, план на май"
]
shown = await api.get(f"/conversations/{conv.external_id}")
assert shown["title"] == "так смотри, план на май" and shown["session_id"] is None
history = await api.get(f"/conversations/{conv.external_id}/history")
assert [m["role"] for m in history["messages"]] == ["user", "assistant"]
async def test_close_distills_a_deep_chat(world: World) -> None:
from test_distill import DistillerClient, deep_chat, distiller
config = distiller(world, DistillerClient)
api = Api(world)
master = await world.conversations.create(kind="master", agent="a", origin="test")
chat = await deep_chat(world)
res = await api.http.post(
f"/conversations/{chat.external_id}/close", headers=HEADERS
)
assert res.status_code == 200, res.text
body = res.json()
assert body["status"] == "closed" and body["error"] is None
assert body["digest"] == str(config.dir / "2026-08-29 - тема.md")
assert body["text"].count("\n") == 2
assert (await world.conversations.get(chat.external_id)).status == "closed"
assert (await world.conversations.queue.recent(master.id))[0].origin == "выжимка"
again = await api.http.post(
f"/conversations/{chat.external_id}/close", headers=HEADERS
)
assert again.status_code == 400
branch = await world.conversations.create(kind="branch", agent="a", origin="test")
plain = await api.http.post(
f"/conversations/{branch.external_id}/close", headers=HEADERS
)
assert plain.status_code == 200 and plain.json()["status"] == "closed"
async def test_bind_without_a_window_materializes_one(world: World) -> None:
api = Api(world)
chat = await world.conversations.create(kind="deep", agent="d", origin="test")
world.markdown.materialized.clear()
res = await api.http.post(
f"/conversations/{chat.external_id}/bind",
json={"frontend": "markdown"},
headers=HEADERS,
)
assert res.status_code == 200, res.text
assert world.markdown.materialized == [chat.external_id]
assert res.json()["bindings"] == [
{
"frontend": "markdown",
"external_id": f"markdown:{chat.external_id}",
"visible": True,
}
]
hidden = await api.http.post(
f"/conversations/{chat.external_id}/bind",
json={
"frontend": "markdown",
"external_id": f"markdown:{chat.external_id}",
"visible": False,
},
headers=HEADERS,
)
assert hidden.json()["bindings"][0]["visible"] is False
no_window = await api.http.post(
f"/conversations/{chat.external_id}/bind",
json={"frontend": "api"},
headers=HEADERS,
)
assert no_window.status_code == 400
unknown = await api.http.post(
f"/conversations/{chat.external_id}/bind",
json={"frontend": "nope"},
headers=HEADERS,
)
assert unknown.status_code == 400
+77
View File
@@ -0,0 +1,77 @@
"""``require_token`` accepts ``?token=`` only when no auth header is present."""
import logging
from types import SimpleNamespace
import pytest
from fastapi import HTTPException
from starlette.requests import Request
from beaver_gateway.core.auth import TokenStore
from beaver_gateway.frontends._auth import require_token
def _request(query: str = "", headers: dict[str, str] | None = None) -> Request:
scope = {
"type": "http",
"method": "POST",
"path": "/hooks/komodo",
"query_string": query.encode(),
"headers": [
(k.lower().encode(), v.encode()) for k, v in (headers or {}).items()
],
}
return Request(scope)
@pytest.fixture
def runtime() -> SimpleNamespace:
return SimpleNamespace(token_store=TokenStore(bootstrap={"komodo": "qs-secret"}))
async def test_query_token_authorizes_a_headerless_webhook(runtime) -> None:
name = await require_token(_request("token=qs-secret"), runtime, scope="api")
assert name == "komodo"
async def test_header_wins_over_query(runtime) -> None:
bad_header = _request("token=qs-secret", {"Authorization": "Bearer nope"})
with pytest.raises(HTTPException) as exc:
await require_token(bad_header, runtime, scope="api")
assert exc.value.status_code == 401
async def test_missing_everything_is_401(runtime) -> None:
with pytest.raises(HTTPException) as exc:
await require_token(_request(), runtime, scope="api")
assert exc.value.status_code == 401
async def test_bootstrap_entry_can_carry_a_scope() -> None:
raw = "admin:secret-a,komodo:hook-value:api"
store = TokenStore(
bootstrap=TokenStore.parse_bootstrap(raw),
bootstrap_scopes=TokenStore.parse_bootstrap_scopes(raw),
)
admin = await store.verify("secret-a")
hook = await store.verify("hook-value")
assert admin is not None and admin.scope == "*"
assert hook is not None and hook.scope == "api" and hook.name == "komodo"
assert not hook.allows("admin") and hook.allows("api")
def test_access_log_filter_masks_query_tokens() -> None:
from beaver_gateway.core.redact import RedactFilter
record = logging.LogRecord(
"uvicorn.access",
logging.INFO,
__file__,
1,
'%s - "%s %s HTTP/%s" %d',
("1.2.3.4:1", "POST", "/hooks/komodo?token=s3cret&x=1", "1.1", 202),
None,
)
assert RedactFilter().filter(record)
assert "s3cret" not in record.getMessage()
assert "/hooks/komodo?token=<…>&x=1" in record.getMessage()
+540
View File
@@ -0,0 +1,540 @@
import asyncio
import tempfile
from pathlib import Path
from typing import Any
import pytest
from anthropic.types import (
RawContentBlockDeltaEvent,
RawContentBlockStartEvent,
RawContentBlockStopEvent,
RawMessageDeltaEvent,
RawMessageStartEvent,
RawMessageStopEvent,
)
from claude_agent_sdk import (
AssistantMessage,
InMemorySessionStore,
ResultMessage,
StreamEvent,
TextBlock,
ToolResultBlock,
ToolUseBlock,
UserMessage,
project_key_for_directory,
)
from beaver_gateway.agents.base import ExposedMcp
from beaver_gateway.agents.claude import ClaudeAgent, ClaudeOptions, Prompts
from beaver_gateway.backends.claude_sdk import (
ClaudeSdkBackend,
RunnerConfig,
UsageEvent,
fingerprint,
)
from beaver_gateway.core.transcript import messages_from_entries
from beaver_gateway.core.turn_capture import TurnCapture
def _stream(index: int, text: str) -> list[StreamEvent]:
def ev(event: dict[str, Any]) -> StreamEvent:
return StreamEvent(uuid="u", session_id="s", event=event)
return [
ev(
{
"type": "content_block_start",
"index": index,
"content_block": {"type": "text", "text": ""},
}
),
ev(
{
"type": "content_block_delta",
"index": index,
"delta": {"type": "text_delta", "text": text},
}
),
ev({"type": "content_block_stop", "index": index}),
]
def _result(session_id: str) -> ResultMessage:
return ResultMessage(
subtype="success",
duration_ms=10,
duration_api_ms=5,
is_error=False,
num_turns=1,
session_id=session_id,
stop_reason="end_turn",
total_cost_usd=0.01,
usage={
"input_tokens": 3,
"output_tokens": 7,
"cache_read_input_tokens": 100,
"cache_creation_input_tokens": 20,
},
)
class FakeClient:
instances: list["FakeClient"] = []
def __init__(self, options: Any) -> None:
self.options = options
self.prompts: list[str] = []
self.connected = False
self.session_id = options.resume or "fresh-session"
FakeClient.instances.append(self)
async def connect(self) -> None:
self.connected = True
async def query(self, prompt: str) -> None:
self.prompts.append(prompt)
async def interrupt(self) -> None:
self.interrupted = True
async def receive_response(self):
start = {
"type": "message_start",
"message": {
"usage": {"input_tokens": 5, "cache_read_input_tokens": 20_000}
},
}
yield StreamEvent(uuid="u", session_id="s", event=start)
for e in _stream(0, "calling "):
yield e
tool = ToolUseBlock(id="toolu_1", name="Read", input={"path": "x"})
yield StreamEvent(
uuid="u",
session_id="s",
event={
"type": "content_block_start",
"index": 1,
"content_block": {"type": "tool_use", "id": "toolu_1", "name": "Read"},
},
)
yield StreamEvent(
uuid="u", session_id="s", event={"type": "content_block_stop", "index": 1}
)
yield AssistantMessage(content=[TextBlock(text="calling "), tool], model="m")
yield UserMessage(
content=[ToolResultBlock(tool_use_id="toolu_1", content="42")]
)
second = {
"type": "message_start",
"message": {
"usage": {
"input_tokens": 7,
"cache_read_input_tokens": 20_000,
"cache_creation_input_tokens": 3_000,
}
},
}
yield StreamEvent(uuid="u", session_id="s", event=second)
for e in _stream(0, "done"):
yield e
yield AssistantMessage(content=[TextBlock(text="done")], model="m")
yield StreamEvent(
uuid="sub",
session_id="s",
event={"type": "content_block_stop", "index": 5},
parent_tool_use_id="toolu_9",
)
yield _result(self.session_id)
async def disconnect(self) -> None:
self.connected = False
@pytest.fixture
def cwd() -> Path:
return Path(tempfile.mkdtemp(prefix="beaver-sdk-"))
def _backend(
cwd: Path, store: InMemorySessionStore, sink=None, **agent_kwargs
) -> ClaudeSdkBackend:
FakeClient.instances.clear()
agent = ClaudeAgent(
name="a",
model="claude-x",
system_prompt="hi",
cwd=cwd,
options=ClaudeOptions(effort="low"),
**agent_kwargs,
)
return ClaudeSdkBackend(
agent=agent,
mcp_internal_urls={"firefly": "http://127.0.0.1:1/mcp/firefly/"},
session_store=store,
mcp_tool_names={"firefly": ["list_account", "delete_account", "store_account"]},
usage_sink=sink,
client_factory=FakeClient,
work_dir=cwd / "work",
)
async def _drain(events) -> list[Any]:
return [e async for e in events]
async def test_stream_envelope_and_index_rebase(cwd: Path) -> None:
backend = _backend(
cwd, InMemorySessionStore(), expose_mcps=(ExposedMcp(name="firefly"),)
)
capture = TurnCapture()
events = await _drain(
backend.complete(
agent=backend.agent,
messages=[{"role": "user", "content": "hi"}],
conversation_id="conv-1",
capture=capture,
)
)
assert isinstance(events[0], RawMessageStartEvent)
assert isinstance(events[-1], RawMessageStopEvent)
assert isinstance(events[-2], RawMessageDeltaEvent)
assert events[-2].usage.output_tokens == 7
assert events[-2].usage.cache_read_input_tokens == 100
starts = [e.index for e in events if isinstance(e, RawContentBlockStartEvent)]
assert starts == [0, 1, 2]
stops = [e.index for e in events if isinstance(e, RawContentBlockStopEvent)]
assert stops == [0, 1, 2]
deltas = [e.delta.text for e in events if isinstance(e, RawContentBlockDeltaEvent)]
assert deltas == ["calling ", "done"]
assert capture.session_id == "fresh-session"
assert capture.usage is not None and capture.usage.cost_usd == 0.01
assert capture.usage.context_tokens == 23_007
assert capture.synthesized_messages == [
{
"role": "assistant",
"content": [
{"type": "text", "text": "calling "},
{
"type": "tool_use",
"id": "toolu_1",
"name": "Read",
"input": {"path": "x"},
},
],
},
{
"role": "user",
"content": [
{"type": "tool_result", "tool_use_id": "toolu_1", "content": "42"}
],
},
{"role": "assistant", "content": [{"type": "text", "text": "done"}]},
]
client = FakeClient.instances[0]
assert client.prompts == ["hi"]
opts = client.options
assert opts.setting_sources == []
assert opts.strict_mcp_config is True
assert opts.permission_mode == "bypassPermissions"
assert opts.mcp_servers == {
"firefly": {"type": "http", "url": "http://127.0.0.1:1/mcp/firefly/"}
}
assert opts.session_store is not None
assert opts.resume is None
assert Path(opts.cli_path).exists()
wrapper = Path(opts.cli_path).read_text()
assert "CLAUDE_" in wrapper and "DATABASE_URL" not in wrapper
async def test_conversation_reuses_live_session(cwd: Path) -> None:
backend = _backend(cwd, InMemorySessionStore())
for i in range(2):
await _drain(
backend.complete(
agent=backend.agent,
messages=[{"role": "user", "content": f"turn {i}"}],
conversation_id="conv-1",
)
)
assert len(FakeClient.instances) == 1
assert FakeClient.instances[0].prompts == ["turn 0", "turn 1"]
assert backend.sessions["conv-1"]["turns"] == 2
async def test_history_is_seeded_into_store_and_resumed(cwd: Path) -> None:
store = InMemorySessionStore()
backend = _backend(cwd, store)
history = [
{"role": "user", "content": "first"},
{"role": "assistant", "content": [{"type": "text", "text": "reply"}]},
{"role": "user", "content": "second"},
]
await _drain(
backend.complete(
agent=backend.agent, messages=history, conversation_id="conv-2"
)
)
client = FakeClient.instances[0]
assert client.options.resume is not None
key = {
"project_key": project_key_for_directory(str(cwd)),
"session_id": client.options.resume,
}
entries = store.get_entries(key)
assert messages_from_entries(entries) == history[:2]
assert client.prompts == ["second"]
async def test_known_session_id_is_resumed_without_seeding(cwd: Path) -> None:
store = InMemorySessionStore()
backend = _backend(cwd, store)
history = [
{"role": "user", "content": "first"},
{"role": "assistant", "content": "reply"},
{"role": "user", "content": "second"},
]
await _drain(
backend.complete(
agent=backend.agent,
messages=history,
conversation_id="conv-3",
session_id="known-sid",
)
)
assert FakeClient.instances[0].options.resume == "known-sid"
assert store.size == 0
async def test_stateless_caller_hits_same_session_next_turn(cwd: Path) -> None:
backend = _backend(cwd, InMemorySessionStore())
first = [{"role": "user", "content": "hi"}]
await _drain(backend.complete(agent=backend.agent, messages=first))
follow_up = [
*first,
{
"role": "assistant",
"content": [
{"type": "text", "text": "calling "},
{"type": "text", "text": "done"},
],
},
{"role": "user", "content": "more"},
]
assert list(backend.sessions) == [fingerprint(follow_up[:-1])]
await _drain(backend.complete(agent=backend.agent, messages=follow_up))
assert len(FakeClient.instances) == 1
assert FakeClient.instances[0].prompts == ["hi", "more"]
async def test_mcp_deny_and_usage_sink(cwd: Path) -> None:
seen: list[UsageEvent] = []
async def sink(event: UsageEvent) -> None:
seen.append(event)
backend = _backend(
cwd,
InMemorySessionStore(),
sink,
expose_mcps=(ExposedMcp(name="firefly", deny=("delete_*",)),),
)
await _drain(
backend.complete(
agent=backend.agent,
messages=[{"role": "user", "content": "x"}],
conversation_id="c",
)
)
assert FakeClient.instances[0].options.disallowed_tools == [
"mcp__firefly__delete_account"
]
assert len(seen) == 1
assert seen[0].usage.input_tokens == 3
assert seen[0].conversation_id == "c"
assert seen[0].session_id == "fresh-session"
async def test_skill_sets_become_sorted_plugins(cwd: Path) -> None:
sets = cwd / "skills"
for name in ("zeta", "общие"):
(sets / name / "demo").mkdir(parents=True)
(sets / name / "demo" / "SKILL.md").write_text("---\nname: demo\n---\n")
backend = _backend(
cwd, InMemorySessionStore(), skill_sets=(sets / "zeta", sets / "общие")
)
await _drain(
backend.complete(
agent=backend.agent,
messages=[{"role": "user", "content": "x"}],
conversation_id="c",
)
)
plugins = FakeClient.instances[0].options.plugins
assert [p["type"] for p in plugins] == ["local", "local"]
paths = [Path(p["path"]) for p in plugins]
assert [p.name for p in paths] == ["zeta", "общие"]
for path in paths:
assert (path / ".claude-plugin" / "plugin.json").exists()
assert (path / "skills" / "demo" / "SKILL.md").exists()
assert FakeClient.instances[0].options.skills == "all"
async def test_prompt_sources_are_assembled(cwd: Path) -> None:
(cwd / "a.md").write_text("alpha\n")
(cwd / "b.md").write_text("\nbeta\n\n")
backend = _backend(
cwd,
InMemorySessionStore(),
prompts=Prompts(deep=(("role", cwd / "a.md"), cwd / "b.md")),
)
await _drain(
backend.complete(
agent=backend.agent,
messages=[{"role": "user", "content": "x"}],
conversation_id="c",
)
)
assert (
FakeClient.instances[0].options.system_prompt
== "<role>\nalpha\n</role>\n\nbeta\n"
)
async def test_runner_user_lands_in_wrapper(cwd: Path) -> None:
import os
import pwd
me = pwd.getpwuid(os.getuid())
backend = _backend(cwd, InMemorySessionStore())
backend._runner = RunnerConfig(user=me.pw_name, home=cwd)
backend._uid, backend._gid = me.pw_uid, me.pw_gid
await _drain(
backend.complete(
agent=backend.agent,
messages=[{"role": "user", "content": "x"}],
conversation_id="c",
)
)
opts = FakeClient.instances[0].options
assert opts.env["HOME"] == str(cwd)
assert opts.env["CLAUDE_CONFIG_DIR"] == str(cwd / ".claude")
wrapper = Path(opts.cli_path).read_text()
assert f"UID = {me.pw_uid}" in wrapper
assert "os.setuid(UID)" in wrapper
async def test_close_disconnects(cwd: Path) -> None:
backend = _backend(cwd, InMemorySessionStore())
async with backend:
await _drain(
backend.complete(
agent=backend.agent,
messages=[{"role": "user", "content": "x"}],
conversation_id="c",
)
)
assert FakeClient.instances[0].connected is False
assert backend.sessions == {}
class GatedClient(FakeClient):
"""Streams one delta, then waits for ``gate`` before finishing the turn."""
gate: asyncio.Event
async def receive_response(self):
yield StreamEvent(
uuid="u", session_id="s", event={"type": "message_start", "message": {}}
)
for e in _stream(0, "first"):
yield e
await GatedClient.gate.wait()
yield AssistantMessage(content=[TextBlock(text="first")], model="m")
yield _result(self.session_id)
async def test_deltas_reach_the_caller_before_the_turn_ends(cwd: Path) -> None:
"""The turn is not buffered: a delta is observable while the CLI still runs."""
GatedClient.gate = asyncio.Event()
backend = _backend(cwd, InMemorySessionStore())
backend._factory = GatedClient
events = backend.complete(
agent=backend.agent,
messages=[{"role": "user", "content": "hi"}],
conversation_id="conv-live",
)
seen: list[Any] = []
async for ev in events:
seen.append(ev)
if isinstance(ev, RawContentBlockDeltaEvent):
break
assert seen[-1].delta.text == "first"
assert not GatedClient.gate.is_set()
GatedClient.gate.set()
rest = [e async for e in events]
assert isinstance(rest[-1], RawMessageStopEvent)
async def test_policy_hook_denies_and_audits(cwd: Path) -> None:
from beaver_gateway.core.policy import Deny, ToolCall
def no_days(c: ToolCall):
p = c.path()
if c.tool == "Write" and p is not None and "📅 дни" in p.parts:
return Deny(reason="дневник только читать")
c.state["seen"] = c.state.get("seen", 0) + 1
return None
audits = []
async def sink(a):
audits.append(a)
backend = _backend(cwd, InMemorySessionStore(), policy=(no_days,))
backend._audit_sink = sink
await _drain(
backend.complete(
agent=backend.agent,
messages=[{"role": "user", "content": "hi"}],
conversation_id="conv-1",
)
)
hook = FakeClient.instances[0].options.hooks["PreToolUse"][0].hooks[0]
denied = await hook(
{"tool_name": "Write", "tool_input": {"file_path": "/vault/📅 дни/x.md"}},
"t1",
{},
)
assert denied["hookSpecificOutput"]["permissionDecision"] == "deny"
assert (
denied["hookSpecificOutput"]["permissionDecisionReason"]
== "дневник только читать"
)
allowed = await hook(
{"tool_name": "Write", "tool_input": {"file_path": "/vault/мета/бобер/x.md"}},
"t2",
{},
)
assert allowed == {}
await hook(
{"tool_name": "Read", "tool_input": {"file_path": "/vault/a.md"}}, "t3", {}
)
assert backend.live("conv-1").state["seen"] == 2
assert [(a.tool, a.decision) for a in audits] == [
("Write", "deny"),
("Write", "allow"),
("Read", "allow"),
]
assert audits[0].brief == "/vault/📅 дни/x.md"
assert audits[0].kind == "deep" and audits[0].conversation == "conv-1"
async def test_no_policy_no_hooks(cwd: Path) -> None:
backend = _backend(cwd, InMemorySessionStore())
await _drain(
backend.complete(
agent=backend.agent, messages=[{"role": "user", "content": "hi"}]
)
)
assert FakeClient.instances[0].options.hooks is None
+16
View File
@@ -0,0 +1,16 @@
import tempfile
from pathlib import Path
from beaver_gateway import config_loader
def test_config_imports_sibling_modules():
root = Path(tempfile.mkdtemp(prefix="beaver-cfg-"))
(root / "policy_sibling.py").write_text("RULES = ('x',)\n")
(root / "config.py").write_text(
"from policy_sibling import RULES\n"
"assert RULES == ('x',)\n"
"gateway = Gateway()\n"
)
gw = config_loader.load(root / "config.py")
assert gw.agents == []
+844
View File
@@ -0,0 +1,844 @@
import asyncio
import tempfile
import uuid
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
import pytest
from claude_agent_sdk import (
AssistantMessage,
InMemorySessionStore,
ResultMessage,
StreamEvent,
TextBlock,
ToolResultBlock,
ToolUseBlock,
UserMessage,
project_key_for_directory,
)
from beaver_gateway.agents.claude import ClaudeAgent, ClaudeOptions
from beaver_gateway.backends.claude_sdk import ClaudeSdkBackend
from beaver_gateway.core.bus import EventBus
from beaver_gateway.core.conversations import Conversations, ConversationTexts, parse_at
from beaver_gateway.core.gateway_tools import SAY_IN_USER_TURN, _tools
from beaver_gateway.core.registry import AgentRegistry
from beaver_gateway.core.sessions import SessionPool
from beaver_gateway.core.transcript import (
build_entries,
close_open_tool_uses,
open_tool_uses,
render_messages,
strip_tool_entries,
window_entries,
)
from beaver_gateway.frontends.base import Frontend
from beaver_gateway.storage import Database
from beaver_gateway.storage.models import (
Conversation,
ConversationBinding,
InjectQueueItem,
)
class StubFrontend(Frontend):
def __init__(
self,
name: str,
kinds: tuple[str, ...],
agents: dict[str, str] | None = None,
*,
home: bool = False,
) -> None:
self.name = name
self.kinds = kinds
self.agents = agents or {}
self.home = home
self.materialized: list[str] = []
self.conversations = None
def configure(self, runtime) -> None:
pass
async def serve(self) -> None:
pass
def agent_for(self, kind: str) -> str | None:
return self.agents.get(kind)
async def materialize(self, conv: Conversation) -> ConversationBinding | None:
if not self.home:
return None
self.materialized.append(conv.external_id)
return await self.conversations.bind(
conv, frontend=self.name, external_id=f"{self.name}:{conv.external_id}"
)
class ScriptedClient:
instances: list["ScriptedClient"] = []
hold: asyncio.Event | None = None
def __init__(self, options: Any) -> None:
self.options = options
self.prompts: list[str] = []
self.session_id = options.resume or str(uuid.uuid4())
self.interrupted = False
self.connected = False
ScriptedClient.instances.append(self)
async def connect(self) -> None:
self.connected = True
async def query(self, prompt: str) -> None:
self.prompts.append(prompt)
async def receive_response(self):
prompt = self.prompts[-1]
yield StreamEvent(
uuid="u",
session_id=self.session_id,
event={"type": "message_start", "message": {}},
)
yield StreamEvent(
uuid="u",
session_id=self.session_id,
event={
"type": "content_block_start",
"index": 0,
"content_block": {"type": "text", "text": ""},
},
)
yield StreamEvent(
uuid="u",
session_id=self.session_id,
event={
"type": "content_block_delta",
"index": 0,
"delta": {"type": "text_delta", "text": f"ok:{prompt}"},
},
)
yield StreamEvent(
uuid="u",
session_id=self.session_id,
event={"type": "content_block_stop", "index": 0},
)
yield AssistantMessage(content=[TextBlock(text=f"ok:{prompt}")], model="m")
hold = ScriptedClient.hold
if hold is not None and not self.interrupted:
await hold.wait()
cut = self.interrupted
self.interrupted = False
yield ResultMessage(
subtype="error_during_execution" if cut else "success",
duration_ms=1,
duration_api_ms=1,
is_error=cut,
num_turns=1,
session_id=self.session_id,
stop_reason="end_turn",
total_cost_usd=0.0,
usage={"input_tokens": 1, "output_tokens": 1},
)
async def interrupt(self) -> None:
self.interrupted = True
if ScriptedClient.hold is not None:
ScriptedClient.hold.set()
async def disconnect(self) -> None:
self.connected = False
class World:
def __init__(self, root: Path) -> None:
ScriptedClient.instances.clear()
ScriptedClient.hold = None
self.root = root
self.db = Database(f"sqlite:///{root / 'w.db'}")
self.store = InMemorySessionStore()
self.agent = ClaudeAgent(
name="a",
model="m",
system_prompt="hi",
cwd=root,
kinds=("master", "branch", "job", "fork"),
gateway_tools=("say",),
options=ClaudeOptions(effort="low"),
)
self.deep_agent = ClaudeAgent(
name="d", model="m", system_prompt="deep", cwd=root
)
self.markdown = StubFrontend("markdown", ("deep",), {"deep": "d"}, home=True)
self.api = StubFrontend("api", ("master", "branch", "deep", "job"))
self.pool = SessionPool(rss_limit=1 << 40, max_live=100)
self.backend, self.deep_backend = (
ClaudeSdkBackend(
agent=agent,
mcp_internal_urls={},
session_store=self.store,
client_factory=ScriptedClient,
work_dir=root / "work",
pool=self.pool,
)
for agent in (self.agent, self.deep_agent)
)
self.bus = EventBus()
self.conversations = Conversations(
db=self.db,
agents=AgentRegistry([self.agent, self.deep_agent]),
backends={"a": self.backend, "d": self.deep_backend},
bus=self.bus,
pool=self.pool,
store=self.store,
texts=ConversationTexts(),
frontends=[self.api, self.markdown],
idle_interval=3600,
)
self.markdown.conversations = self.conversations
self.api.conversations = self.conversations
async def setup(self) -> "World":
await self.db.create_all()
return self
def key(self, session_id: str) -> dict[str, str]:
return {
"project_key": project_key_for_directory(str(self.root)),
"session_id": session_id,
}
async def statuses(self, conv) -> list[tuple[str, str]]:
rows = await self.conversations.queue.recent(conv.id)
return [(r.priority, r.status) for r in reversed(rows)]
async def settle(self, conv, expected: int, timeout: float = 5.0) -> None:
deadline = asyncio.get_running_loop().time() + timeout
while asyncio.get_running_loop().time() < deadline:
rows = await self.conversations.queue.recent(conv.id)
if sum(1 for r in rows if r.status == "done") >= expected:
return
await asyncio.sleep(0.02)
msg = f"queue did not settle: {await self.statuses(conv)}"
raise AssertionError(msg)
@pytest.fixture
async def world() -> World:
root = Path(tempfile.mkdtemp(prefix="beaver-conv-"))
w = await World(root).setup()
yield w
await w.conversations.stop()
await w.pool.close_all()
await w.db.dispose()
async def test_two_messages_run_one_at_a_time_in_order(world: World) -> None:
conv = await world.conversations.create(kind="master", agent="a", origin="test")
ScriptedClient.hold = asyncio.Event()
await world.conversations.post(conv, "first")
await asyncio.sleep(0.2)
await world.conversations.post(conv, "second")
await asyncio.sleep(0.2)
assert await world.statuses(conv) == [("user", "running"), ("user", "queued")]
live = world.pool.get(conv.external_id)
assert live is not None and live.busy
assert (await world.conversations.get(conv.external_id)).running_turn is not None
ScriptedClient.hold.set()
await world.settle(conv, 2)
assert len(ScriptedClient.instances) == 1
assert ScriptedClient.instances[0].prompts == ["first", "second"]
row = await world.conversations.get(conv.external_id)
assert row.running_turn is None
assert row.session_id == ScriptedClient.instances[0].session_id
async def test_say_is_refused_in_user_turns_and_works_in_inject_turns(
world: World,
) -> None:
conv = await world.conversations.create(kind="master", agent="a", origin="test")
say = next(
t for t in _tools(world.conversations, conv.external_id) if t.name == "say"
)
said: list[str] = []
async def watch() -> None:
async for event in world.bus.stream(conversation_id=conv.external_id):
if event["type"] == "say":
said.append(str(event["text"]))
watcher = asyncio.create_task(watch())
ScriptedClient.hold = asyncio.Event()
await world.conversations.post(conv, "hi")
await asyncio.sleep(0.2)
assert world.conversations.turn_origin(conv) == "user"
out = await say.handler({"text": "dup"})
assert out.get("is_error") and out["content"][0]["text"] == SAY_IN_USER_TURN
ScriptedClient.hold.set()
await world.settle(conv, 1)
assert world.conversations.turn_origin(conv) is None
ScriptedClient.hold = asyncio.Event()
await world.conversations.inject(conv, "tick", urgency="urgent", origin="крон")
await asyncio.sleep(0.2)
assert world.conversations.turn_origin(conv) == "inject"
out = await say.handler({"text": "psst"})
assert not out.get("is_error")
ScriptedClient.hold.set()
await world.settle(conv, 2)
await asyncio.sleep(0.1)
watcher.cancel()
assert said == ["psst"]
async def test_urgent_interrupts_and_goes_first(world: World) -> None:
conv = await world.conversations.create(kind="master", agent="a", origin="test")
ScriptedClient.hold = asyncio.Event()
await world.conversations.post(conv, "first")
await asyncio.sleep(0.2)
await world.conversations.post(conv, "second")
await world.conversations.inject(conv, "ALERT", urgency="urgent", origin="крон")
await world.settle(conv, 2)
client = ScriptedClient.instances[0]
assert await world.statuses(conv) == [
("user", "interrupted"),
("user", "done"),
("urgent", "done"),
]
assert await world.statuses(conv) == [
("user", "interrupted"),
("user", "done"),
("urgent", "done"),
]
assert client.prompts[0] == "first"
assert client.prompts[1].startswith("[инжект: крон")
assert "прервал предыдущий тёрн" in client.prompts[1]
assert client.prompts[1].endswith("ALERT")
assert client.prompts[2] == "second"
async def test_inject_header_is_configurable(world: World) -> None:
from beaver_gateway.core.conversations import ConversationTexts
world.conversations._texts = ConversationTexts( # noqa: SLF001
inject_header=lambda ctx: (
f"[от {ctx.origin}, {ctx.priority}, cut={ctx.interrupted_turn}]"
)
)
conv = await world.conversations.create(kind="master", agent="a", origin="test")
await world.conversations.inject(conv, "hi", urgency="urgent", origin="panel")
await world.settle(conv, 1)
assert ScriptedClient.instances[0].prompts[0] == "[от panel, urgent, cut=False]\nhi"
async def test_normal_injects_ride_with_the_next_user_message(world: World) -> None:
conv = await world.conversations.create(kind="master", agent="a", origin="test")
await world.conversations.inject(
conv, "vault changed", urgency="normal", origin="watch"
)
await asyncio.sleep(0.2)
assert await world.statuses(conv) == [("normal", "queued")]
await world.conversations.post(conv, "hello")
await world.settle(conv, 2)
prompts = ScriptedClient.instances[0].prompts
assert len(prompts) == 1
assert prompts[0].startswith("hello\n\n[инжекты")
assert "- [watch] vault changed" in prompts[0]
async def test_inject_turn_reply_is_not_routed(world: World) -> None:
conv = await world.conversations.create(kind="master", agent="a", origin="test")
seen: list[dict[str, Any]] = []
async def collect() -> None:
async for event in world.bus.stream(conversation_id=conv.external_id):
seen.append(event)
task = asyncio.create_task(collect())
world.conversations._normal_window = 0.05
await world.conversations.inject(conv, "tick", urgency="normal", origin="крон")
await world.settle(conv, 1)
await asyncio.sleep(0.05)
task.cancel()
types = [e["type"] for e in seen]
assert "turn.start" in types and "turn.end" in types
assert "reply" not in types
assert all(e.get("origin") == "inject" for e in seen if e["type"] == "turn.start")
async def test_spawn_seeds_first_user_message(world: World) -> None:
conv = await world.conversations.spawn(
kind="branch", agent="a", seed="brief", text="do X", title="t"
)
await world.settle(conv, 1)
prompt = ScriptedClient.instances[0].prompts[0]
assert prompt.startswith("[сид: brief] branch «t», ")
assert prompt.endswith("\n\ndo X")
assert ScriptedClient.instances[0].options.system_prompt == "hi"
async def test_fork_leaves_original_untouched(world: World) -> None:
sid = str(uuid.uuid4())
history = [
{"role": "user", "content": "one"},
{
"role": "assistant",
"content": [{"type": "tool_use", "id": "t1", "name": "Read", "input": {}}],
},
{
"role": "user",
"content": [{"type": "tool_result", "tool_use_id": "t1", "content": "x"}],
},
{"role": "assistant", "content": "done one"},
{"role": "user", "content": "two"},
{"role": "assistant", "content": "done two"},
{"role": "user", "content": "three"},
{"role": "assistant", "content": "done three"},
]
entries = build_entries(history, session_id=sid, cwd=str(world.root), model="m")
await world.store.append(world.key(sid), entries)
conv = await world.conversations.create(
kind="branch", agent="a", origin="test", session_id=sid
)
before = [dict(e) for e in await world.store.load(world.key(sid))]
result = await world.conversations.fork(
conv, "summarize", window=2, strip_tools=True
)
after = await world.store.load(world.key(sid))
assert after == before
assert result.text == "ok:summarize"
assert result.conversation.status == "closed"
assert result.conversation.parent_id == conv.id
fork_entries = await world.store.load(world.key(result.conversation.session_id))
assert not {e["uuid"] for e in fork_entries} & {e["uuid"] for e in before}
texts = [e["message"]["content"] for e in fork_entries]
assert texts[0] == "two"
assert len(fork_entries) == 4
assert fork_entries[0]["parentUuid"] is None
assert all(
b["type"] == "text"
for e in fork_entries[1:]
for b in e["message"]["content"]
if isinstance(e["message"]["content"], list)
)
client = ScriptedClient.instances[0]
assert client.options.resume == result.conversation.session_id
assert client.options.mcp_servers == {}
assert world.pool.get(result.conversation.external_id) is None
async def test_copy_seed_forks_parent_with_window(world: World) -> None:
sid = str(uuid.uuid4())
history = [
{"role": "user", "content": f"q{i}"}
if i % 2 == 0
else {"role": "assistant", "content": f"a{i}"}
for i in range(8)
]
await world.store.append(
world.key(sid),
build_entries(history, session_id=sid, cwd=str(world.root), model="m"),
)
parent = await world.conversations.create(
kind="master", agent="a", origin="test", session_id=sid
)
child = await world.conversations.spawn(
kind="branch", agent="a", seed="copy", parent=parent, window=1
)
assert child.session_id and child.session_id != sid
copied = await world.store.load(world.key(child.session_id))
assert [e["message"]["content"] for e in copied] == [
"q6",
[{"type": "text", "text": "a7"}],
]
assert (await world.conversations.get(child.external_id)).flags["seed"] == "copy"
await world.conversations.post(child, "go")
await world.settle(child, 1)
assert ScriptedClient.instances[0].options.resume == child.session_id
prompt = ScriptedClient.instances[0].prompts[0]
assert prompt.startswith("[сид: copy] branch, ")
assert "последние 1 тёрнов" in prompt and prompt.endswith("\n\ngo")
assert (await world.conversations.get(child.external_id)).flags["seed"] is None
async def test_merge_injects_summary_into_parent(world: World) -> None:
sid = str(uuid.uuid4())
await world.store.append(
world.key(sid),
build_entries(
[{"role": "user", "content": "q"}, {"role": "assistant", "content": "a"}],
session_id=sid,
cwd=str(world.root),
model="m",
),
)
master = await world.conversations.create(kind="master", agent="a", origin="test")
branch = await world.conversations.create(
kind="branch", agent="a", origin="test", parent=master, session_id=sid
)
result = await world.conversations.merge(branch)
assert result.text.startswith("ok:")
assert (await world.conversations.get(branch.external_id)).status == "merged"
assert await world.statuses(master) == [("normal", "queued")]
item = (await world.conversations.queue.recent(master.id))[0]
assert item.origin == "слив" and item.text == result.text
async def test_recover_closes_open_tool_use_and_injects_interrupted(
world: World,
) -> None:
sid = str(uuid.uuid4())
history = [
{"role": "user", "content": "go"},
{
"role": "assistant",
"content": [
{
"type": "tool_use",
"id": "t9",
"name": "Bash",
"input": {"command": "sleep"},
}
],
},
]
await world.store.append(
world.key(sid),
build_entries(history, session_id=sid, cwd=str(world.root), model="m"),
)
conv = await world.conversations.create(
kind="master", agent="a", origin="test", session_id=sid
)
async with world.db.session() as session:
row = await session.get(type(conv), conv.id)
row.running_turn = "turn_dead"
session.add(row)
session.add(
InjectQueueItem(
conversation_id=conv.id,
priority="user",
origin="user",
text="go",
status="running",
turn_id="turn_dead",
)
)
await session.commit()
cut = await world.conversations.recover()
assert [c.external_id for c in cut] == [conv.external_id]
entries = await world.store.load(world.key(sid))
assert not open_tool_uses(entries)
tail = entries[-1]
assert tail["type"] == "user"
assert tail["message"]["content"][0] == {
"type": "tool_result",
"tool_use_id": "t9",
"content": "прервано",
"is_error": True,
}
assert tail["parentUuid"] == entries[-2]["uuid"]
assert (await world.conversations.get(conv.external_id)).running_turn is None
assert await world.statuses(conv) == [("user", "interrupted"), ("normal", "queued")]
note = (await world.conversations.queue.recent(conv.id))[0]
assert (
"turn_dead" in note.text
and "оборван" in note.text
and "1 незакрытых" in note.text
)
await asyncio.sleep(0.2)
assert ScriptedClient.instances == []
async def test_graceful_stop_keeps_running_turn_for_recover(world: World) -> None:
conv = await world.conversations.create(kind="master", agent="a", origin="test")
ScriptedClient.hold = asyncio.Event()
await world.conversations.post(conv, "long")
await asyncio.sleep(0.2)
assert (await world.conversations.get(conv.external_id)).running_turn is not None
await world.conversations.stop()
row = await world.conversations.get(conv.external_id)
assert row.running_turn is not None
assert row.session_id == ScriptedClient.instances[0].session_id
cut = await world.conversations.recover()
assert [c.external_id for c in cut] == [conv.external_id]
assert (await world.conversations.get(conv.external_id)).running_turn is None
assert await world.statuses(conv) == [("user", "interrupted"), ("normal", "queued")]
ScriptedClient.hold = None
async def test_read_and_bindings(world: World) -> None:
sid = str(uuid.uuid4())
history = [
{"role": "user", "content": "q1"},
{
"role": "assistant",
"content": [
{"type": "text", "text": "a1"},
{"type": "tool_use", "id": "t", "name": "Read", "input": {}},
],
},
{
"role": "user",
"content": [{"type": "tool_result", "tool_use_id": "t", "content": "x"}],
},
{"role": "assistant", "content": "a1b"},
{"role": "user", "content": "q2"},
{"role": "assistant", "content": "a2"},
]
await world.store.append(
world.key(sid),
build_entries(history, session_id=sid, cwd=str(world.root), model="m"),
)
conv = await world.conversations.create(
kind="deep", agent="d", origin="test", session_id=sid
)
assert (
await world.conversations.read(conv)
== "user:\nq1\n\nassistant:\na1\n\na1b\n(tools: Read)\n\nuser:\nq2\n\nassistant:\na2"
)
assert (
await world.conversations.read(conv, window=1) == "user:\nq2\n\nassistant:\na2"
)
await world.conversations.bind(conv, frontend="markdown", external_id="a.md")
await world.conversations.bind(conv, frontend="markdown", external_id="b.md")
bindings = await world.conversations.bindings(conv)
assert [(b.external_id, b.visible) for b in bindings] == [
("a.md", False),
("b.md", True),
]
found = await world.conversations.find_bound(
frontend="markdown", external_id="b.md"
)
assert found is not None and found.id == conv.id
assert (
await world.conversations.find_bound(frontend="markdown", external_id="a.md")
is None
)
async def test_schedule_without_scheduler_and_parse_at(world: World) -> None:
conv = await world.conversations.create(kind="master", agent="a", origin="test")
with pytest.raises(RuntimeError, match="no scheduler"):
await world.conversations.schedule(conv, "+15m", "push X")
assert await world.conversations.schedules(conv) == []
assert parse_at("2026-09-01T10:00:00+02:00") == datetime(
2026, 9, 1, 8, 0, tzinfo=UTC
)
delta = (parse_at("+15m") - datetime.now(UTC)).total_seconds()
assert 14 * 60 < delta <= 15 * 60
with pytest.raises(ValueError, match="Invalid isoformat"):
parse_at("tomorrow")
def test_transcript_helpers() -> None:
history = [
{"role": "user", "content": "q1"},
{
"role": "assistant",
"content": [
{"type": "thinking", "thinking": "hm", "signature": "s"},
{"type": "tool_use", "id": "t", "name": "Read", "input": {}},
],
},
{
"role": "user",
"content": [{"type": "tool_result", "tool_use_id": "t", "content": "x"}],
},
{"role": "assistant", "content": "a1"},
{"role": "user", "content": "q2"},
{
"role": "assistant",
"content": [{"type": "tool_use", "id": "t2", "name": "Bash", "input": {}}],
},
]
entries = build_entries(history, session_id="s", cwd="/v", model="m")
assert [b["id"] for _, b in open_tool_uses(entries)] == ["t2"]
fixes = close_open_tool_uses(entries)
assert len(fixes) == 1 and fixes[0]["sessionId"] == "s" and fixes[0]["cwd"] == "/v"
assert not open_tool_uses([*entries, *fixes])
stripped = strip_tool_entries(entries)
assert [e["message"]["content"] for e in stripped] == [
"q1",
[{"type": "text", "text": "a1"}],
"q2",
]
assert (
stripped[0]["parentUuid"] is None
and stripped[1]["parentUuid"] == stripped[0]["uuid"]
)
windowed = window_entries(entries, window=1)
assert windowed[0]["message"]["content"] == "q2" and len(windowed) == 2
assert (
render_messages(
[{"role": "user", "content": "q"}, {"role": "assistant", "content": "a"}]
)
== "user:\nq\n\nassistant:\na"
)
async def test_agent_kinds_gate_create(world: World) -> None:
with pytest.raises(ValueError, match="does not serve kind 'deep'"):
await world.conversations.create(kind="deep", agent="a", origin="test")
with pytest.raises(ValueError, match="does not serve kind 'master'"):
await world.conversations.create(kind="master", agent="d", origin="test")
with pytest.raises(ValueError, match="'d' does not serve kind 'master'"):
await world.conversations.spawn(kind="master", seed="clean", agent="d")
conv = await world.conversations.create(kind="deep", agent="d", origin="test")
assert conv.agent_name == "d"
async def test_frontend_kinds_gate_bind(world: World) -> None:
master = await world.conversations.create(kind="master", agent="a", origin="test")
with pytest.raises(ValueError, match="'markdown' does not show kind 'master'"):
await world.conversations.bind(master, frontend="markdown", external_id="m.md")
with pytest.raises(ValueError, match="unknown frontend 'telegram'"):
await world.conversations.bind(master, frontend="telegram", external_id="1")
await world.conversations.bind(master, frontend="api", external_id="x")
async def test_spawn_defaults_agent_and_materializes(world: World) -> None:
master = await world.conversations.create(kind="master", agent="a", origin="test")
deep = await world.conversations.spawn(
kind="deep", seed="brief", text="dig", title="t", parent=master, origin="mcp"
)
assert deep.agent_name == "d"
assert world.markdown.materialized == [deep.external_id]
bindings = await world.conversations.bindings(deep)
assert [(b.frontend, b.external_id) for b in bindings] == [
("markdown", f"markdown:{deep.external_id}")
]
branch = await world.conversations.spawn(kind="branch", seed="clean", parent=master)
assert branch.agent_name == "a"
assert world.markdown.materialized == [deep.external_id]
with pytest.raises(ValueError, match="no default agent for kind 'job'"):
await world.conversations.spawn(kind="job", seed="clean")
await world.settle(deep, 1)
def test_agent_kinds_follow_prompts(tmp_path: Path) -> None:
from beaver_gateway.agents.claude import Prompts
plain = ClaudeAgent(name="p", model="m", system_prompt="hi", cwd=tmp_path)
assert plain.kinds == ("deep",)
dispatcher = ClaudeAgent(
name="x", model="m", cwd=tmp_path, prompts=Prompts(master=("a.md",), fork=())
)
assert dispatcher.kinds == ("master", "fork")
assert dispatcher.prompt_for("master") == ("a.md",)
assert dispatcher.prompt_for("deep") is None
with pytest.raises(ValueError, match=r"serves \['deep'\] without a prompt"):
ClaudeAgent(
name="y",
model="m",
cwd=tmp_path,
kinds=("master", "deep"),
prompts=Prompts(master=()),
)
async def test_observer_publishes_tool_results_for_the_panel(world: World) -> None:
seen: list[dict[str, Any]] = []
conv = await world.conversations.create(kind="master", agent="a", origin="test")
async def collect() -> None:
async for event in world.bus.stream(conversation_id=conv.external_id):
seen.append(event)
task = asyncio.create_task(collect())
await asyncio.sleep(0)
runner = world.conversations._runner(conv.id)
runner.turn_id = "t1"
observe = world.conversations._observer(conv, runner, "t1", "user")
observe(
AssistantMessage(
content=[ToolUseBlock(id="toolu_1", name="Bash", input={"command": "ls"})],
model="m",
parent_tool_use_id="toolu_0",
)
)
observe(
UserMessage(
content=[
ToolResultBlock(tool_use_id="toolu_1", content="a\nb", is_error=False)
],
parent_tool_use_id="toolu_0",
)
)
observe(UserMessage(content="plain text, no tool result"))
await asyncio.sleep(0.05)
task.cancel()
assert [e["type"] for e in seen] == ["tool", "tool.result"]
tool, result = seen
assert tool["tool_use_id"] == "toolu_1" and tool["parent_tool_use_id"] == "toolu_0"
snapshot = runner.snapshot()
assert snapshot is not None and snapshot["tools"][0]["content"] == "a\nb"
assert result["tool_use_id"] == "toolu_1"
assert result["parent_tool_use_id"] == "toolu_0"
assert result["turn_id"] == "t1" and result["is_error"] is False
assert result["content"] == "a\nb"
async def test_wake_injects_start_a_turn_and_take_normals_along(world: World) -> None:
conv = await world.conversations.create(kind="master", agent="a", origin="test")
await world.conversations.inject(conv, "digest", urgency="normal", origin="watch")
await asyncio.sleep(0.2)
assert await world.statuses(conv) == [("normal", "queued")]
await world.conversations.inject(
conv, "reminder", urgency="wake", origin="schedule"
)
await world.settle(conv, 2)
prompts = ScriptedClient.instances[0].prompts
assert len(prompts) == 1
assert prompts[0].startswith("[инжект: schedule")
assert "reminder" in prompts[0]
assert "[инжект: watch" in prompts[0]
assert "digest" in prompts[0]
assert await world.statuses(conv) == [("normal", "done"), ("wake", "done")]
async def test_wake_does_not_cut_a_running_turn(world: World) -> None:
conv = await world.conversations.create(kind="master", agent="a", origin="test")
ScriptedClient.hold = asyncio.Event()
await world.conversations.post(conv, "first")
await asyncio.sleep(0.2)
await world.conversations.inject(conv, "later", urgency="wake", origin="schedule")
await asyncio.sleep(0.2)
assert await world.statuses(conv) == [("user", "running"), ("wake", "queued")]
assert not ScriptedClient.instances[0].interrupted
assert await world.conversations.busy(conv)
ScriptedClient.hold.set()
await world.settle(conv, 2)
assert await world.statuses(conv) == [("user", "done"), ("wake", "done")]
assert ScriptedClient.instances[0].prompts[1].endswith("later")
async def test_resolve_aliases_master_and_parent(world: World) -> None:
master = await world.conversations.create(kind="master", agent="a", origin="test")
branch = await world.conversations.create(
kind="branch", agent="a", parent=master, origin="test"
)
resolve = world.conversations.resolve
assert (await resolve("master")).id == master.id
assert (await resolve(" master ")).id == master.id
assert (await resolve("parent", origin=branch)).id == master.id
assert await resolve("parent", origin=master) is None
assert await resolve("parent") is None
assert (await resolve(branch.external_id)).id == branch.id
assert await resolve("nope") is None
await world.conversations.set_status(master, "closed")
assert await resolve("master") is None
async def test_inject_into_a_closed_master_lands_in_the_open_one(world: World) -> None:
old = await world.conversations.create(kind="master", agent="a", origin="test")
old = await world.conversations.set_status(old, "closed")
new = await world.conversations.create(kind="master", agent="a", origin="test")
item = await world.conversations.inject(old, "late", urgency="normal", origin="x")
assert item.conversation_id == new.id
assert await world.statuses(old) == []
assert await world.statuses(new) == [("normal", "queued")]
+61
View File
@@ -0,0 +1,61 @@
import sqlite3
import tempfile
from pathlib import Path
from sqlmodel import select
from beaver_gateway.storage import Database
from beaver_gateway.storage.models import Conversation
async def test_create_all_adds_missing_columns() -> None:
path = Path(tempfile.mkdtemp(prefix="beaver-migrate-")) / "old.db"
raw = sqlite3.connect(path)
raw.execute(
"CREATE TABLE conversations (id INTEGER PRIMARY KEY, frontend VARCHAR NOT NULL, "
"external_id VARCHAR NOT NULL, agent_name VARCHAR NOT NULL, "
"created_at DATETIME NOT NULL, updated_at DATETIME NOT NULL)"
)
raw.execute(
"INSERT INTO conversations VALUES (1, 'markdown', 'x', 'a', '2026-01-01', '2026-01-01')"
)
raw.commit()
raw.close()
db = Database(f"sqlite:///{path}")
await db.create_all()
async with db.session() as session:
conv = (await session.exec(select(Conversation))).one()
assert conv.session_id is None
conv.session_id = "sid"
session.add(conv)
await session.commit()
await db.dispose()
async def test_create_all_backfills_defaults_for_old_rows() -> None:
path = Path(tempfile.mkdtemp(prefix="beaver-migrate-")) / "old.db"
raw = sqlite3.connect(path)
raw.execute(
"CREATE TABLE conversations (id INTEGER PRIMARY KEY, frontend VARCHAR NOT NULL, "
"external_id VARCHAR NOT NULL, agent_name VARCHAR NOT NULL, session_id VARCHAR, "
"created_at DATETIME NOT NULL, updated_at DATETIME NOT NULL)"
)
raw.execute(
"INSERT INTO conversations VALUES (1, 'markdown', 'x', 'a', NULL, '2026-01-01', '2026-01-01')"
)
raw.commit()
raw.close()
db = Database(f"sqlite:///{path}")
await db.create_all()
async with db.session() as session:
conv = (await session.exec(select(Conversation))).one()
assert (conv.kind, conv.status, conv.pending_question, conv.flags) == (
"deep",
"open",
False,
{},
)
assert conv.running_turn is None
await db.dispose()
+419
View File
@@ -0,0 +1,419 @@
import asyncio
import uuid
from datetime import UTC, datetime, timedelta
from pathlib import Path
from typing import Any
from claude_agent_sdk import (
AssistantMessage,
ResultMessage,
TextBlock,
ToolResultBlock,
ToolUseBlock,
UserMessage,
project_key_for_directory,
)
from test_conversations import ScriptedClient, World, world
from beaver_gateway.agents.claude import ClaudeAgent, ClaudeOptions
from beaver_gateway.backends.claude_sdk import ClaudeSdkBackend
from beaver_gateway.core.conversations import ConversationTexts
from beaver_gateway.core.distill import (
Distiller,
DistillContext,
LineCap,
check_digest,
trim_summary,
)
from beaver_gateway.core.gateway_tools import _tools, build_tool_server
from beaver_gateway.core.registry import AgentRegistry
from beaver_gateway.core.scheduler import Job, JobRun, Scheduler
from beaver_gateway.core.transcript import build_entries
from beaver_gateway.storage.models import Conversation
__all__ = ["world"]
DIGEST = """---
type: выжимка
source: "[[{chat}]]"
date: 2026-08-29
---
# тема
## контекст
зачем открывали
## решили
- одно
"""
class DistillerClient(ScriptedClient):
"""The distiller: writes a digest file when told to, answers in N lines."""
digest_dir: Path | None = None
lines = 3
write = True
frontmatter = DIGEST
async def receive_response(self):
prompt = self.prompts[-1]
written: list[ToolUseBlock] = []
if self.write and self.digest_dir is not None and "файл не пиши" not in prompt:
chat = prompt.split("«", 1)[1].split("»", 1)[0] if "«" in prompt else "чат"
path = self.digest_dir / "2026-08-29 - тема.md"
path.write_text(self.frontmatter.format(chat=chat), encoding="utf-8")
written.append(
ToolUseBlock(id="w1", name="Write", input={"file_path": str(path)})
)
for block in written:
yield AssistantMessage(content=[block], model="m")
yield UserMessage(content=[ToolResultBlock(tool_use_id=block.id)])
text = "\n".join(f"строка {i} про {prompt[:20]!r}" for i in range(self.lines))
yield AssistantMessage(content=[TextBlock(text=text)], model="m")
yield ResultMessage(
subtype="success",
duration_ms=1,
duration_api_ms=1,
is_error=False,
num_turns=1,
session_id=self.session_id,
stop_reason="end_turn",
total_cost_usd=0.0,
usage={"input_tokens": 1, "output_tokens": 1},
)
class ClosingClient(ScriptedClient):
"""A deep chat that calls ``close_chat`` inside its reply."""
conversations: Any = None
key = ""
async def receive_response(self):
prompt = self.prompts[-1]
if "обсудили" in prompt:
assert "gateway" in self.options.mcp_servers
tools = _tools(ClosingClient.conversations, ClosingClient.key)
close = next(t for t in tools if t.name == "close_chat")
result = await close.handler({})
assert "closes after this reply" in result["content"][0]["text"]
yield AssistantMessage(
content=[
ToolUseBlock(id="c1", name="mcp__gateway__close_chat", input={})
],
model="m",
)
yield AssistantMessage(content=[TextBlock(text=f"ok:{prompt}")], model="m")
yield ResultMessage(
subtype="success",
duration_ms=1,
duration_api_ms=1,
is_error=False,
num_turns=1,
session_id=self.session_id,
stop_reason="end_turn",
total_cost_usd=0.0,
usage={"input_tokens": 1, "output_tokens": 1},
)
class CapClient(ScriptedClient):
"""A job that rewrites the capped file with ``lines`` lines."""
target: Path | None = None
lines = 70
async def receive_response(self):
assert self.target is not None
self.target.write_text(
"\n".join(f"- строка {i}" for i in range(self.lines)), encoding="utf-8"
)
CapClient.lines = 10
async for event in super().receive_response():
yield event
def distiller(world: World, client: type[ScriptedClient]) -> Distiller:
vault = world.root / "vault"
digests = vault / "мета" / "бобер" / "выжимки"
digests.mkdir(parents=True)
agent = ClaudeAgent(
name="x",
model="m",
system_prompt="distill",
cwd=vault,
kinds=("fork", "job"),
options=ClaudeOptions(
effort="medium", tools=("Read", "Write"), include_partial_messages=False
),
)
backend = ClaudeSdkBackend(
agent=agent,
mcp_internal_urls={},
session_store=world.store,
client_factory=client,
work_dir=world.root / "work",
pool=world.pool,
)
world.conversations._agents = AgentRegistry( # noqa: SLF001
[world.agent, world.deep_agent, agent]
)
world.conversations._backends["x"] = backend # noqa: SLF001
config = Distiller(agent="x", dir=digests, index=digests.parent / "индекс.md")
world.conversations._distiller = config # noqa: SLF001
DistillerClient.digest_dir = digests
DistillerClient.lines = 3
DistillerClient.write = True
DistillerClient.frontmatter = DIGEST
return config
async def deep_chat(world: World, name: str = "2026-08-20 - тема чата") -> Conversation:
sid = str(uuid.uuid4())
await world.store.append(
world.key(sid),
build_entries(
[
{"role": "user", "content": "q"},
{
"role": "assistant",
"content": [
{"type": "tool_use", "id": "t1", "name": "Read", "input": {}}
],
},
{
"role": "user",
"content": [
{"type": "tool_result", "tool_use_id": "t1", "content": "x"}
],
},
{"role": "assistant", "content": "a"},
],
session_id=sid,
cwd=str(world.root),
model="m",
),
)
conv = await world.conversations.create(
kind="deep", agent="d", origin="markdown", session_id=sid
)
await world.conversations.bind(
conv, frontend="markdown", external_id=f"2026-08/{name}.md"
)
return conv
async def test_distill_writes_the_digest_indexes_it_and_merges_short(
world: World,
) -> None:
config = distiller(world, DistillerClient)
prompts: list[DistillContext] = []
def distill_prompt(ctx: DistillContext) -> str:
prompts.append(ctx)
return f"Чат «{ctx.chat_name}» закрыт ({ctx.reason})."
world.conversations._texts = ConversationTexts(distill=distill_prompt) # noqa: SLF001
master = await world.conversations.create(kind="master", agent="a", origin="test")
chat = await deep_chat(world)
result = await world.conversations.distill(chat, reason="api")
assert prompts[0].chat_name == "2026-08-20 - тема чата"
assert prompts[0].memory is True
assert result.digest is not None and result.error is None
assert result.digest.path == config.dir / "2026-08-29 - тема.md"
assert result.digest.source == "[[2026-08-20 - тема чата]]"
index = config.index.read_text(encoding="utf-8")
assert index.startswith("# индекс")
assert "- 2026-08-29 [[2026-08-20 - тема чата]] → [[2026-08-29 - тема]]" in index
assert result.text.count("\n") == 2 and not result.trimmed
closed = await world.conversations.get(chat.external_id)
assert closed.status == "closed"
assert closed.flags["digest"] == str(result.digest.path)
assert closed.flags["closed_reason"] == "api"
fork = await world.conversations.get(result.fork.external_id)
assert fork.kind == "fork" and fork.agent_name == "x" and fork.status == "closed"
items = await world.conversations.queue.recent(master.id)
assert items[0].origin == "выжимка"
assert items[0].text.startswith(
"Закрыт глубокий чат [[2026-08-20 - тема чата]], выжимка [[2026-08-29 - тема]]."
)
assert items[0].text.endswith(result.text)
forked = ScriptedClient.instances[-1]
assert forked.options.resume is not None
vault_key = {
"project_key": project_key_for_directory(str(config.dir.parents[2])),
"session_id": forked.options.resume,
}
entries = await world.store.load(vault_key)
assert entries
assert all(
not isinstance(e.get("message", {}).get("content"), list)
or all(b.get("type") == "text" for b in e["message"]["content"])
for e in entries
)
async def test_memory_off_merges_without_a_file(world: World) -> None:
config = distiller(world, DistillerClient)
master = await world.conversations.create(kind="master", agent="a", origin="test")
chat = await deep_chat(world)
await world.conversations.set_flags(chat, {"memory": False})
result = await world.conversations.distill(chat, reason="api")
assert result.digest is None and result.error is None
assert list(config.dir.iterdir()) == []
assert not config.index.exists()
assert (await world.conversations.get(chat.external_id)).status == "closed"
items = await world.conversations.queue.recent(master.id)
assert len(items) == 1 and items[0].text.endswith(result.text)
assert ", выжимка" not in items[0].text
assert "файл не пиши" in ScriptedClient.instances[-1].prompts[0]
async def test_bad_frontmatter_and_long_merge_are_reported(world: World) -> None:
distiller(world, DistillerClient)
DistillerClient.frontmatter = "---\ntype: заметка\n---\n# тема\n"
DistillerClient.lines = 9
await world.conversations.create(kind="master", agent="a", origin="test")
chat = await deep_chat(world)
result = await world.conversations.distill(chat, reason="idle 2d")
assert result.digest is None
assert result.error is not None and "`type`" in result.error
assert result.trimmed and result.text.count("\n") == 4
closed = await world.conversations.get(chat.external_id)
assert closed.status == "closed"
assert closed.flags["digest_error"] == result.error
def test_check_digest_rejects_what_is_not_a_digest(tmp_path: Path) -> None:
config = Distiller(agent="x", dir=tmp_path, index=tmp_path / "i.md")
path = tmp_path / "d.md"
path.write_text("---\ntype: выжимка\nsource: ''\ndate: 2026-08-29\n---\nx\n")
assert check_digest(path, config) == "`source` пустой"
path.write_text("---\ntype: выжимка\nsource: '[[a]]'\ndate: вчера\n---\nx\n")
assert "`date`" in check_digest(path, config)
path.write_text("---\ntype: выжимка\nsource: '[[a]]'\ndate: 2026-08-29\n---\n\n")
assert check_digest(path, config) == "тело пустое"
path.write_text("---\ntype: выжимка\nsource: '[[a]]'\ndate: 2026-08-29\n---\nx\n")
digest = check_digest(path, config)
assert not isinstance(digest, str) and digest.date.isoformat() == "2026-08-29"
assert trim_summary("a\n\nb\nc\nd\ne\nf") == ("a\nb\nc\nd\ne", True)
async def test_close_chat_tool_closes_after_the_reply(world: World) -> None:
distiller(world, DistillerClient)
world.deep_agent = ClaudeAgent(
name="d",
model="m",
system_prompt="deep",
cwd=world.root,
gateway_tools=("close_chat",),
)
world.conversations._agents = AgentRegistry( # noqa: SLF001
[world.agent, world.deep_agent, world.conversations._agents.get("x")] # noqa: SLF001
)
world.conversations._backends["d"] = ClaudeSdkBackend( # noqa: SLF001
agent=world.deep_agent,
mcp_internal_urls={},
session_store=world.store,
client_factory=ClosingClient,
work_dir=world.root / "work",
pool=world.pool,
tool_server=lambda key, _kind: build_tool_server(
world.conversations, conversation_key=key, names=("close_chat",)
),
)
master = await world.conversations.create(kind="master", agent="a", origin="test")
chat = await deep_chat(world)
ClosingClient.conversations = world.conversations
ClosingClient.key = chat.external_id
await world.conversations.post(chat, "ок, обсудили")
await world.settle(chat, 1)
for _ in range(100):
row = await world.conversations.get(chat.external_id)
if row.status == "closed":
break
await asyncio.sleep(0.05)
else:
raise AssertionError("chat did not close")
assert row.flags["closed_reason"] == "close_chat"
assert row.flags["digest"] is not None
items = await world.conversations.queue.recent(master.id)
assert items[0].origin == "выжимка"
async def test_idle_picks_quiet_chats_after_launch_at_most_limit(world: World) -> None:
distiller(world, DistillerClient)
now = datetime.now(UTC)
launch = now - timedelta(days=10)
async def aged(name: str, days: float) -> Conversation:
conv = await deep_chat(world, name)
async def apply(row: Conversation) -> None:
row.last_activity_at = now - timedelta(days=days)
return await world.conversations._update(conv, apply) # noqa: SLF001
old = await aged("old", 30)
a = await aged("a", 5)
b = await aged("b", 4)
c = await aged("c", 3)
d = await aged("d", 2.5)
fresh = await aged("fresh", 1)
no_session = await world.conversations.create(kind="deep", agent="d", origin="t")
idle = await world.conversations.idle(kind="deep", days=2, since=launch)
assert [x.external_id for x in idle] == [
a.external_id,
b.external_id,
c.external_id,
d.external_id,
]
assert old.external_id not in {x.external_id for x in idle}
assert fresh.external_id not in {x.external_id for x in idle}
assert no_session.external_id not in {x.external_id for x in idle}
scheduler = Scheduler(conversations=world.conversations)
run = JobRun(Job("закрытие", lambda _run: asyncio.sleep(0)), "cron", {}, scheduler)
closed = await run.close_idle(kind="deep", days=2, limit=3, since=launch)
assert [r.conversation.external_id for r in closed] == [
a.external_id,
b.external_id,
c.external_id,
]
assert (await world.conversations.get(d.external_id)).status == "open"
assert len(await world.conversations.idle(kind="deep", days=2, since=launch)) == 1
async def test_line_cap_bounces_a_long_rewrite_and_asks_to_shorten(
world: World,
) -> None:
distiller(world, CapClient)
state = world.root / "vault" / "состояние.md"
state.write_text("# состояние\n- было так\n", encoding="utf-8")
CapClient.target = state
CapClient.lines = 70
scheduler = Scheduler(conversations=world.conversations)
run = JobRun(Job("память", lambda _run: asyncio.sleep(0)), "cron", {}, scheduler)
job = await run.spawn_job(
agent="x", text="перепиши", line_cap=LineCap(state, max_lines=60)
)
await world.settle(job, 2)
client = ScriptedClient.instances[-1]
assert len(client.prompts) == 2
assert "70 строк при потолке 60" in client.prompts[1]
assert "[инжект: потолок" in client.prompts[1]
assert state.read_text(encoding="utf-8").count("\n") == 10 - 1
row = await world.conversations.get(job.external_id)
assert row.flags["line_cap_attempts"] == 1
+218
View File
@@ -0,0 +1,218 @@
import asyncio
import tempfile
from datetime import UTC, datetime, timedelta
from pathlib import Path
from test_conversations import ScriptedClient, World, world
from beaver_gateway.core.conversations import UserSaid
from beaver_gateway.core.envelope import HEADER, Envelope, RecallContext, render
from beaver_gateway.core.watch import Change, VaultWatch, WatchRules
__all__ = ["world"]
RULES = WatchRules(
full=("дни/{today}.md",),
names=("дни/*", "люди/*", "мета/бобер/*"),
ignore=("чаты/*", "мета/*"),
)
def vault() -> tuple[Path, VaultWatch]:
root = Path(tempfile.mkdtemp(prefix="beaver-vault-"))
for sub in ("дни", "люди", "чаты", "мета/бобер", "мета/чужое"):
(root / sub).mkdir(parents=True)
today = datetime.now(UTC).date().isoformat()
(root / "дни" / f"{today}.md").write_text("# день\n- 09:00 проснулся\n")
(root / "люди" / "Прохор.md").write_text("# Прохор\nстрока\n")
(root / "чаты" / "чат.md").write_text("чат\n")
(root / "мета" / "чужое" / "x.md").write_text("x\n")
watch = VaultWatch(root, RULES, tz="UTC")
watch.snapshot()
return root, watch
def append(path: Path, text: str) -> None:
with path.open("a", encoding="utf-8") as fh:
fh.write(text)
def test_rules_classify_paths() -> None:
today = "2026-08-29"
assert RULES.kind("дни/2026-08-29.md", today=today) == "full"
assert RULES.kind("дни/2026-08-28.md", today=today) == "names"
assert RULES.kind("люди/Прохор.md", today=today) == "names"
assert RULES.kind("мета/бобер/состояние.md", today=today) == "names"
assert RULES.kind("мета/чужое/x.md", today=today) is None
assert RULES.kind("чаты/чат.md", today=today) is None
assert RULES.kind("люди/фото.png", today=today) is None
def test_watch_reports_only_added_lines_with_mtime() -> None:
root, watch = vault()
today = watch.today()
diary = root / "дни" / f"{today}.md"
append(diary, "- 12:40 вышел\n- 13:00 кофе\n")
append(root / "люди" / "Прохор.md", "ещё\n")
append(root / "чаты" / "чат.md", "ignored\n")
for path in (diary, root / "люди" / "Прохор.md", root / "чаты" / "чат.md"):
watch.note(path)
changes = {c.path: c for c in watch.take()}
assert set(changes) == {f"дни/{today}.md", "люди/Прохор.md"}
assert changes[f"дни/{today}.md"].added == ["- 12:40 вышел", "- 13:00 кофе"]
assert changes[f"дни/{today}.md"].full
assert changes["люди/Прохор.md"].added_count == 1
assert changes["люди/Прохор.md"].added == []
assert abs(
changes[f"дни/{today}.md"].mtime
- datetime.fromtimestamp(diary.stat().st_mtime, tz=UTC)
) < timedelta(seconds=1)
assert watch.take() == []
append(diary, "- 14:00 снова\n")
watch.note(diary)
(only,) = watch.take()
assert only.added == ["- 14:00 снова"]
def test_envelope_respects_ceilings_and_names_only_window() -> None:
root, watch = vault()
today = watch.today()
diary = root / "дни" / f"{today}.md"
append(diary, "".join(f"- строка {i}\n" for i in range(200)))
for name in ("Петя", "Маша"):
(root / "люди" / f"{name}.md").write_text("новый\n" * 50)
watch.note(root / "люди" / f"{name}.md")
watch.note(diary)
envelope = Envelope(watch=watch, tz="Europe/Warsaw")
text = envelope.build()
lines = text.splitlines()
assert lines[0] == HEADER
assert "(Warsaw)" in lines[1]
assert lines[2].startswith("vault, изменено со старта: ")
assert f"дни/{today}.md (+200)" in lines[2]
assert "люди/Петя.md (+50)" in lines[2]
assert sum(1 for line in lines if line.startswith("+ ")) == 31
assert "+ … ещё 170" in lines
assert len(lines) <= 120
append(diary, "- ещё одна\n")
watch.note(diary)
second = envelope.build()
assert f"дни/{today}.md (+1)" in second
assert "+ - ещё одна" not in second
def test_render_hits_total_ceiling() -> None:
now = datetime(2026, 8, 26, 11, 4, tzinfo=UTC)
changes = [
Change(
path=f"дни/{i}.md",
mtime=now,
added=[f"l{j}" for j in range(30)],
added_count=30,
full=True,
)
for i in range(10)
]
text = render(
now=now,
tz="Europe/Warsaw",
changes=changes,
since=now - timedelta(hours=1),
names_only=False,
)
lines = text.splitlines()
assert len(lines) <= 120
assert "… (потолок конверта)" in lines
assert lines[1] == "время: 2026-08-26 13:04 (Warsaw)"
async def test_master_turn_gets_envelope_after_text_and_before_injects(
world: World,
) -> None:
root, watch = vault()
world.conversations._envelope = Envelope(watch=watch, tz="UTC") # noqa: SLF001
master = await world.conversations.create(kind="master", agent="a", origin="test")
append(root / "люди" / "Прохор.md", "новое\n")
watch.note(root / "люди" / "Прохор.md")
await world.conversations.inject(master, "tick", urgency="normal", origin="крон")
await world.conversations.post(master, "hello")
await world.settle(master, 2)
prompt = ScriptedClient.instances[0].prompts[0]
head, _, rest = prompt.partition("\n\n")
assert head == "hello"
assert rest.startswith(HEADER)
assert "люди/Прохор.md (+1)" in rest
assert rest.index("[инжекты") > rest.index(HEADER)
branch = await world.conversations.spawn(
kind="branch", parent=master, seed="brief", text="do X"
)
await world.settle(branch, 1)
assert "[конверт" not in ScriptedClient.instances[-1].prompts[0]
await asyncio.sleep(0)
async def test_recall_lines_follow_the_vault_block_and_reach_branches(
world: World,
) -> None:
root, watch = vault()
seen: list[tuple[str, str]] = []
def recall(ctx: RecallContext) -> str | None:
seen.append((ctx.kind, ctx.text))
return "👤 Прохор - карточка `люди/Прохор.md`" if "Прохор" in ctx.text else None
noted: list[UserSaid] = []
world.conversations._envelope = Envelope(watch=watch, tz="UTC", recall=recall) # noqa: SLF001
world.conversations._user_sink = noted.append # noqa: SLF001
master = await world.conversations.create(kind="master", agent="a", origin="test")
append(root / "люди" / "Прохор.md", "новое\n")
watch.note(root / "люди" / "Прохор.md")
await world.conversations.post(master, "что там у Прохор")
await world.settle(master, 1)
prompt = ScriptedClient.instances[0].prompts[0]
head, _, rest = prompt.partition("\n\n")
assert head == "что там у Прохор"
lines = rest.splitlines()
assert lines[0] == HEADER
assert "люди/Прохор.md (+1)" in rest
assert lines[-1] == "👤 Прохор - карточка `люди/Прохор.md`"
assert rest.index("люди/Прохор.md (+1)") < rest.index("👤 Прохор")
assert seen == [("master", "что там у Прохор")]
assert [(m.kind, m.text) for m in noted] == [("master", "что там у Прохор")]
branch = await world.conversations.spawn(
kind="branch", parent=master, seed="clean", text="про Прохор подробнее"
)
await world.settle(branch, 1)
branch_prompt = ScriptedClient.instances[-1].prompts[0]
assert "про Прохор подробнее\n\n" + HEADER in branch_prompt
assert branch_prompt.endswith(f"{HEADER}\n👤 Прохор - карточка `люди/Прохор.md`")
assert "vault, изменено" not in branch_prompt
# the first branch turn carries the seed line above the text
assert seen[-1][0] == "branch" and seen[-1][1].endswith("про Прохор подробнее")
assert noted[-1].kind == "branch"
await world.conversations.post(master, "ничего про людей")
await world.settle(master, 2)
second = ScriptedClient.instances[0].prompts[1]
assert "👤" not in second
assert second.partition("\n\n")[2].startswith(HEADER)
await asyncio.sleep(0)
async def test_failing_recall_or_sink_never_blocks_the_turn(world: World) -> None:
root, watch = vault()
def boom(_: object) -> str:
msg = "nope"
raise RuntimeError(msg)
world.conversations._envelope = Envelope(watch=watch, tz="UTC", recall=boom) # noqa: SLF001
world.conversations._user_sink = boom # noqa: SLF001
master = await world.conversations.create(kind="master", agent="a", origin="test")
await world.conversations.post(master, "hello")
await world.settle(master, 1)
prompt = ScriptedClient.instances[0].prompts[0]
assert prompt.startswith("hello\n\n" + HEADER)
assert root.exists()
await asyncio.sleep(0)
+45
View File
@@ -0,0 +1,45 @@
from anthropic.types import Message, TextBlock, ThinkingBlock, ToolUseBlock, Usage
from beaver_gateway.frontends.markdown import renderer
def _message(*blocks: object) -> Message:
return Message(
id="m",
type="message",
role="assistant",
model="x",
content=list(blocks), # type: ignore[arg-type]
stop_reason="end_turn",
stop_sequence=None,
usage=Usage(input_tokens=1, output_tokens=1),
)
def test_tool_calls_leave_no_blank_lines() -> None:
message = _message(
TextBlock(type="text", text="first"),
ToolUseBlock(type="tool_use", id="t1", name="Bash", input={"command": "ls"}),
ToolUseBlock(type="tool_use", id="t2", name="Read", input={"file_path": "x"}),
ToolUseBlock(type="tool_use", id="t3", name="Grep", input={"pattern": "y"}),
TextBlock(type="text", text="second"),
)
assert renderer.render_assistant_message(message) == (
"### Assistant:\n\nfirst\n\nsecond\n"
)
def test_thinking_renders_as_collapsed_callout() -> None:
message = _message(
ThinkingBlock(type="thinking", thinking="a\nb", signature="s"),
TextBlock(type="text", text="answer"),
)
assert renderer.render_assistant_message(message) == (
"### Assistant:\n\n> [!thinking]-\n> a\n> b\n\nanswer\n"
)
def test_user_scaffold_matches_rendered_user_turn_shape() -> None:
body = renderer.append_to_body("### Assistant:\n\nhi\n", renderer.USER_SCAFFOLD)
assert body.endswith("\n\n---\n\n### User:\n\n")
assert renderer.render_user_text("q").startswith("### User:\n\n")
+208
View File
@@ -0,0 +1,208 @@
from __future__ import annotations
import mcp.types as mt
import pytest
from fastmcp import FastMCP
from fastmcp.tools.base import ToolResult
from beaver_gateway.core import redact as redact_mod
from beaver_gateway.core.redact import redact
from beaver_gateway.mcp.internal_app import build_internal_app
from beaver_gateway.mcp.redacting import RedactingMiddleware
from beaver_gateway.mcp.types import McpServer
# A trimmed but otherwise verbatim ``komodo deploy`` result: this is the
# shape Komodo returns for the ``Compose Config`` stage of a
# DeployStackService, which the komodo tool passes on to the model as
# ``-- <stage> ok`` plus the stage's stdout. Values are substituted.
KOMODO_DEPLOY = """DeployStackService Complete ok
-- Service/s ok
gateway
-- Compose Config ok
name: beaver-agent
services:
gateway:
container_name: beaver-gateway
environment:
ADMIN_PASS: 0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f50
ADMIN_USER: admin
BEAVERGRAM_MCP: https://tg.example.dev/mcp?token=bg-0123456789abcdef
BOOTSTRAP_TOKENS: admin:bs-0123456789abcdef,komodo:hook-0123456789
CALENDAR_MCPS: personal=https://cal.example.com/api/mcp?icsUrl=https%3A%2F%2Fcalendar.google.com%2Fcalendar%2Fical%2Fx%2Fprivate-0123456789abcdef%2Fbasic.ics
CLAUDE_CODE_OAUTH_TOKEN: sk-ant-oat01-0123456789abcdef
CLAUDE_HOME: /home/beaver-runner
COMPOSE_PROFILES: db,gateway,obsidian,t3
DATABASE_URL: postgresql+psycopg://beaver:pg-0123456789abcdef@postgres:5432/beaver
KOMODO_URL: https://komod.example.dev
POSTGRES_DB: beaver
POSTGRES_PASSWORD: pg-0123456789abcdef
POSTGRES_USER: beaver
ports:
- 62990:62990
-- Compose Up ok
Container beaver-gateway Started
"""
SECRETS = (
"0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f50",
"bg-0123456789abcdef",
"bs-0123456789abcdef",
"hook-0123456789",
"private-0123456789abcdef",
"sk-ant-oat01-0123456789abcdef",
"pg-0123456789abcdef",
)
@pytest.fixture(autouse=True)
def _no_env_secrets() -> None:
"""Prove the patterns alone carry it — no literal env values loaded."""
redact_mod._ENV_SECRETS[:] = [] # noqa: SLF001
def test_compose_config_dump_loses_every_value_but_keeps_every_name() -> None:
out = redact(KOMODO_DEPLOY)
for secret in SECRETS:
assert secret not in out, secret
# Names stay: the dump still says what is configured.
for name in (
"ADMIN_PASS",
"BOOTSTRAP_TOKENS",
"CALENDAR_MCPS",
"POSTGRES_PASSWORD",
# a username is half of a login pair — in prod ADMIN_USER is a
# 64-hex string, not "admin"
"ADMIN_USER",
"POSTGRES_USER",
):
assert f"{name}: <…>" in out
# Non-secret values are untouched, so the result stays worth reading.
assert "POSTGRES_DB: beaver" in out
assert "CLAUDE_HOME: /home/beaver-runner" in out
assert "COMPOSE_PROFILES: db,gateway,obsidian,t3" in out
assert "KOMODO_URL: https://komod.example.dev" in out
assert "container_name: beaver-gateway" in out
assert "-- Compose Up ok" in out
assert " Container beaver-gateway Started" in out
def test_database_url_keeps_its_shape_and_loses_its_password() -> None:
# DATABASE_URL is not a secret-looking name, so the whole line is not
# masked; the password is caught as the userinfo of the connection
# string, which is what keeps the host and database readable.
out = redact(KOMODO_DEPLOY)
line = next(ln for ln in out.splitlines() if "DATABASE_URL" in ln)
assert "pg-0123456789abcdef" not in line
assert "postgres:5432/beaver" in line
def test_redaction_is_idempotent() -> None:
once = redact(KOMODO_DEPLOY)
assert redact(once) == once
def test_a_log_line_is_not_swallowed_by_the_assignment_rule() -> None:
# ``token:`` in mid-line must not eat the rest of an audit line.
line = "mcp: actor=token:komodo namespace=calendar POST /mcp/ -> 200"
assert redact(line) == line
@pytest.mark.anyio
async def test_middleware_masks_a_tool_result() -> None:
def deploy(stack: str) -> str:
"""Deploy a stack."""
return f"{stack}\n{KOMODO_DEPLOY}"
server = FastMCP(name="komodo", tools=[deploy])
server.add_middleware(RedactingMiddleware())
result = await server.call_tool("deploy", {"stack": "beaver-agent"})
text = "\n".join(b.text for b in result.content if isinstance(b, mt.TextContent))
for secret in SECRETS:
assert secret not in text, secret
assert "ADMIN_PASS: <…>" in text
@pytest.mark.anyio
async def test_middleware_masks_structured_content() -> None:
def config() -> dict[str, object]:
"""Return a config blob."""
return {
"env": {"ADMIN_PASS": "0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f50"},
"port": 62990,
}
server = FastMCP(name="cfg", tools=[config])
server.add_middleware(RedactingMiddleware())
result = await server.call_tool("config", {})
dumped = str(result.structured_content) + str(result.content)
assert "0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f50" not in dumped
assert "62990" in dumped
@pytest.mark.anyio
async def test_clean_result_passes_through_untouched() -> None:
def ping() -> str:
"""Say hello."""
return "pong"
server = FastMCP(name="p", tools=[ping])
server.add_middleware(RedactingMiddleware())
result = await server.call_tool("ping", {})
assert isinstance(result, ToolResult)
assert result.content[0].text == "pong" # ty: ignore[unresolved-attribute]
@pytest.mark.anyio
async def test_every_namespace_built_by_the_aggregator_is_filtered() -> None:
def deploy() -> str:
"""Deploy."""
return KOMODO_DEPLOY
_app, _urls, servers = build_internal_app(
[McpServer.python_tool(name="komodo", tools=[deploy])],
host="127.0.0.1",
port=8765,
)
for name, server in servers.items():
result = await server.call_tool(
"deploy" if name == "komodo" else "komodo_deploy", {}
)
text = "\n".join(
b.text for b in result.content if isinstance(b, mt.TextContent)
)
assert "0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f50" not in text, name
@pytest.mark.anyio
async def test_gateway_own_tools_are_filtered_too() -> None:
# These bypass the FastMCP middleware — the SDK mounts them
# in-process — so they carry their own wrapper.
from claude_agent_sdk import SdkMcpTool
from beaver_gateway.core.gateway_tools import _redacting
async def handler(_args: dict[str, object]) -> dict[str, object]:
return {"content": [{"type": "text", "text": KOMODO_DEPLOY}]}
spec = _redacting(
SdkMcpTool(
name="read_conversation", description="", input_schema={}, handler=handler
)
)
out = str(await spec.handler({}))
for secret in SECRETS:
assert secret not in out, secret
def test_source_code_the_dispatcher_reads_is_not_mangled() -> None:
# t3code hands back files and diffs. A SCREAMING_SNAKE constant is
# an assignment too, but ``NAME = expr`` is code, not a config dump.
source = (
'SESSION_SECRET = os.environ["SESSION_SECRET"]\nAPI_KEY_HEADER = "X-Api-Key"'
)
assert redact(source) == source
def test_env_file_lines_are_still_masked() -> None:
out = redact("ADMIN_PASS=0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f50\nPOSTGRES_DB=beaver")
assert out == "ADMIN_PASS=<…>\nPOSTGRES_DB=beaver"
+71
View File
@@ -0,0 +1,71 @@
from pathlib import Path
import pytest
from beaver_gateway.core.policy import Deny, ToolCall, brief, evaluate, hook_output
def call(tool: str, **tool_input) -> ToolCall:
return ToolCall(
tool=tool,
input=tool_input,
agent="a",
kind="master",
conversation="c1",
cwd=Path("/vault"),
)
def test_path_resolves_relative_against_cwd():
assert call("Write", file_path="мета/бобер/x.md").path() == Path(
"/vault/мета/бобер/x.md"
)
assert call("Write", file_path="/vault/a/../b.md").path() == Path("/vault/b.md")
assert call("Write", file_path="../../etc/passwd").path() == Path("/etc/passwd")
assert call("Bash", command="ls").path() is None
def test_mcp_split():
assert call("mcp__firefly__store_transaction").mcp == (
"firefly",
"store_transaction",
)
assert call("Write").mcp is None
async def test_first_deny_wins_and_async_rules_work():
seen = []
def allow(c):
seen.append("allow")
async def deny(c):
return Deny(reason="no")
def never(c):
raise AssertionError
verdict = await evaluate((allow, deny, never), call("Write"))
assert verdict == Deny(reason="no")
assert seen == ["allow"]
async def test_raising_rule_fails_closed():
def boom(c):
raise RuntimeError("x")
verdict = await evaluate((boom,), call("Write"))
assert verdict is not None and "boom" in verdict.reason
def test_hook_output_shapes():
assert hook_output(None) == {}
out = hook_output(Deny(reason="r"))
assert out["hookSpecificOutput"]["permissionDecision"] == "deny"
assert out["hookSpecificOutput"]["permissionDecisionReason"] == "r"
def test_brief_prefers_path_then_command_and_truncates():
assert brief({"file_path": "/v/x.md", "content": "a" * 999}) == "/v/x.md"
assert brief({"command": "ls -la"}) == "ls -la"
assert len(brief({"q": "a" * 999})) == 240
Loaded 100 of 301 files, more files were not shown because too many files have changed in this diff. Show more