Compare commits
49
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9b1127acff | ||
|
|
1e08dfb9da | ||
|
|
8e7db6baa4 | ||
|
|
acc79c2e42 | ||
|
|
63014a487d | ||
|
|
3406e2e178 | ||
|
|
878f7d6473 | ||
|
|
c438d4cb92 | ||
|
|
ba9d63667e | ||
|
|
d57f660a30 | ||
|
|
cc32307921 | ||
|
|
811035ba4f | ||
|
|
9efff6e814 | ||
|
|
a989d2909d | ||
|
|
8d252f9867 | ||
|
|
2895e677c0 | ||
|
|
23cfb16228 | ||
|
|
ab9ce50b82 | ||
|
|
662bdcdfda | ||
|
|
0479d7cdda | ||
|
|
0cc2c5291f | ||
|
|
5cd18fa2d7 | ||
|
|
e45e9960ed | ||
|
|
16b6bbddda | ||
|
|
540327efa1 | ||
|
|
645df6f2b4 | ||
|
|
b577034c83 | ||
|
|
00618ab7ad | ||
|
|
aaaf459e6d | ||
|
|
4ad014726b | ||
|
|
8c9c6eee69 | ||
|
|
21766fe2d7 | ||
|
|
1f8dae57ac | ||
|
|
66325c2879 | ||
|
|
5a44a16ba3 | ||
|
|
c711dcab13 | ||
|
|
3995741cd5 | ||
|
|
b0f280b3ca | ||
|
|
45a5eb4a94 | ||
|
|
7ae87aeeb8 | ||
|
|
fd8cbacd7b | ||
|
|
bd2d2368e1 | ||
|
|
1d8d65b69a | ||
|
|
33ccc78fec | ||
|
|
50b7057fa4 | ||
|
|
e1f242a87a | ||
|
|
ee2dc918ee | ||
|
|
33cc769ca2 | ||
|
|
e4282b38cd |
No files matched your search
@@ -0,0 +1,22 @@
|
|||||||
|
## Code Principles
|
||||||
|
- **Simplicity**: Write simple, straightforward code
|
||||||
|
- **Readability**: Make code easy to understand
|
||||||
|
- **Maintainability**: Write code that's easy to update
|
||||||
|
- **Less Code = Less Debt**: Minimize code footprint
|
||||||
|
- **NEVER write comments** - code should be self-documenting
|
||||||
|
|
||||||
|
## Layout
|
||||||
|
- `src/beaver_gateway` - python, unopinionated primitives (no vault paths, no agent names)
|
||||||
|
- `ui` - SvelteKit SPA (admin + Obsidian panel), served by the gateway from `ui/build`
|
||||||
|
- `docs/MISSION.md`, `docs/PRD.md`, `docs/PLAN.md`, `docs/PROGRESS.md` - keep PROGRESS.md current
|
||||||
|
|
||||||
|
## Checking commands
|
||||||
|
After writing code, always run (or `make check`):
|
||||||
|
```shell
|
||||||
|
ruff format # python formatter
|
||||||
|
ruff check --fix # python linter
|
||||||
|
ty check # python type-checker
|
||||||
|
cd ui && bun fix # frontend linter+formatter (ultracite)
|
||||||
|
cd ui && bun check # svelte-check
|
||||||
|
```
|
||||||
|
Dependencies: `uv add` / `cd ui && bun add`, never edit lock files by hand.
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
You are able to use the Svelte MCP server, where you have access to comprehensive Svelte 5 and SvelteKit documentation. Here's how to use the available tools effectively:
|
||||||
|
|
||||||
|
## Available Svelte MCP Tools:
|
||||||
|
|
||||||
|
### 1. list-sections
|
||||||
|
|
||||||
|
Use this FIRST to discover all available documentation sections. Returns a structured list with titles, use_cases, and paths.
|
||||||
|
When asked about Svelte or SvelteKit topics, ALWAYS use this tool at the start of the chat to find relevant sections.
|
||||||
|
|
||||||
|
### 2. get-documentation
|
||||||
|
|
||||||
|
Retrieves full documentation content for specific sections. Accepts single or multiple sections.
|
||||||
|
After calling the list-sections tool, you MUST analyze the returned documentation sections (especially the use_cases field) and then use the get-documentation tool to fetch ALL documentation sections that are relevant for the user's task.
|
||||||
|
|
||||||
|
### 3. svelte-autofixer
|
||||||
|
|
||||||
|
Analyzes Svelte code and returns issues and suggestions.
|
||||||
|
You MUST use this tool whenever writing Svelte code before sending it to the user. Keep calling it until no issues or suggestions are returned.
|
||||||
|
|
||||||
|
### 4. playground-link
|
||||||
|
|
||||||
|
Generates a Svelte Playground link with the provided code.
|
||||||
|
After completing the code, ask the user if they want a playground link. Only call this tool after user confirmation and NEVER if code was written to files in their project.
|
||||||
@@ -0,0 +1,123 @@
|
|||||||
|
# Ultracite Code Standards
|
||||||
|
|
||||||
|
This project uses **Ultracite**, a zero-config preset that enforces strict code quality standards through automated formatting and linting.
|
||||||
|
|
||||||
|
## Quick Reference
|
||||||
|
|
||||||
|
- **Format code**: `bun x ultracite fix`
|
||||||
|
- **Check for issues**: `bun x ultracite check`
|
||||||
|
- **Diagnose setup**: `bun x ultracite doctor`
|
||||||
|
|
||||||
|
Biome (the underlying engine) provides robust linting and formatting. Most issues are automatically fixable.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Core Principles
|
||||||
|
|
||||||
|
Write code that is **accessible, performant, type-safe, and maintainable**. Focus on clarity and explicit intent over brevity.
|
||||||
|
|
||||||
|
### Type Safety & Explicitness
|
||||||
|
|
||||||
|
- Use explicit types for function parameters and return values when they enhance clarity
|
||||||
|
- Prefer `unknown` over `any` when the type is genuinely unknown
|
||||||
|
- Use const assertions (`as const`) for immutable values and literal types
|
||||||
|
- Leverage TypeScript's type narrowing instead of type assertions
|
||||||
|
- Use meaningful variable names instead of magic numbers - extract constants with descriptive names
|
||||||
|
|
||||||
|
### Modern JavaScript/TypeScript
|
||||||
|
|
||||||
|
- Use arrow functions for callbacks and short functions
|
||||||
|
- Prefer `for...of` loops over `.forEach()` and indexed `for` loops
|
||||||
|
- Use optional chaining (`?.`) and nullish coalescing (`??`) for safer property access
|
||||||
|
- Prefer template literals over string concatenation
|
||||||
|
- Use destructuring for object and array assignments
|
||||||
|
- Use `const` by default, `let` only when reassignment is needed, never `var`
|
||||||
|
|
||||||
|
### Async & Promises
|
||||||
|
|
||||||
|
- Always `await` promises in async functions - don't forget to use the return value
|
||||||
|
- Use `async/await` syntax instead of promise chains for better readability
|
||||||
|
- Handle errors appropriately in async code with try-catch blocks
|
||||||
|
- Don't use async functions as Promise executors
|
||||||
|
|
||||||
|
### React & JSX
|
||||||
|
|
||||||
|
- Use function components over class components
|
||||||
|
- Call hooks at the top level only, never conditionally
|
||||||
|
- Specify all dependencies in hook dependency arrays correctly
|
||||||
|
- Use the `key` prop for elements in iterables (prefer unique IDs over array indices)
|
||||||
|
- Nest children between opening and closing tags instead of passing as props
|
||||||
|
- Don't define components inside other components
|
||||||
|
- Use semantic HTML and ARIA attributes for accessibility:
|
||||||
|
- Provide meaningful alt text for images
|
||||||
|
- Use proper heading hierarchy
|
||||||
|
- Add labels for form inputs
|
||||||
|
- Include keyboard event handlers alongside mouse events
|
||||||
|
- Use semantic elements (`<button>`, `<nav>`, etc.) instead of divs with roles
|
||||||
|
|
||||||
|
### Error Handling & Debugging
|
||||||
|
|
||||||
|
- Remove `console.log`, `debugger`, and `alert` statements from production code
|
||||||
|
- Throw `Error` objects with descriptive messages, not strings or other values
|
||||||
|
- Use `try-catch` blocks meaningfully - don't catch errors just to rethrow them
|
||||||
|
- Prefer early returns over nested conditionals for error cases
|
||||||
|
|
||||||
|
### Code Organization
|
||||||
|
|
||||||
|
- Keep functions focused and under reasonable cognitive complexity limits
|
||||||
|
- Extract complex conditions into well-named boolean variables
|
||||||
|
- Use early returns to reduce nesting
|
||||||
|
- Prefer simple conditionals over nested ternary operators
|
||||||
|
- Group related code together and separate concerns
|
||||||
|
|
||||||
|
### Security
|
||||||
|
|
||||||
|
- Add `rel="noopener"` when using `target="_blank"` on links
|
||||||
|
- Avoid `dangerouslySetInnerHTML` unless absolutely necessary
|
||||||
|
- Don't use `eval()` or assign directly to `document.cookie`
|
||||||
|
- Validate and sanitize user input
|
||||||
|
|
||||||
|
### Performance
|
||||||
|
|
||||||
|
- Avoid spread syntax in accumulators within loops
|
||||||
|
- Use top-level regex literals instead of creating them in loops
|
||||||
|
- Prefer specific imports over namespace imports
|
||||||
|
- Avoid barrel files (index files that re-export everything)
|
||||||
|
- Use proper image components (e.g., Next.js `<Image>`) over `<img>` tags
|
||||||
|
|
||||||
|
### Framework-Specific Guidance
|
||||||
|
|
||||||
|
**Next.js:**
|
||||||
|
- Use Next.js `<Image>` component for images
|
||||||
|
- Use `next/head` or App Router metadata API for head elements
|
||||||
|
- Use Server Components for async data fetching instead of async Client Components
|
||||||
|
|
||||||
|
**React 19+:**
|
||||||
|
- Use ref as a prop instead of `React.forwardRef`
|
||||||
|
|
||||||
|
**Solid/Svelte/Vue/Qwik:**
|
||||||
|
- Use `class` and `for` attributes (not `className` or `htmlFor`)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
- Write assertions inside `it()` or `test()` blocks
|
||||||
|
- Avoid done callbacks in async tests - use async/await instead
|
||||||
|
- Don't use `.only` or `.skip` in committed code
|
||||||
|
- Keep test suites reasonably flat - avoid excessive `describe` nesting
|
||||||
|
|
||||||
|
## When Biome Can't Help
|
||||||
|
|
||||||
|
Biome's linter will catch most issues automatically. Focus your attention on:
|
||||||
|
|
||||||
|
1. **Business logic correctness** - Biome can't validate your algorithms
|
||||||
|
2. **Meaningful naming** - Use descriptive names for functions, variables, and types
|
||||||
|
3. **Architecture decisions** - Component structure, data flow, and API design
|
||||||
|
4. **Edge cases** - Handle boundary conditions and error states
|
||||||
|
5. **User experience** - Accessibility, performance, and usability considerations
|
||||||
|
6. **Documentation** - Add comments for complex logic, but prefer self-documenting code
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
Most formatting and common issues are automatically fixed by Biome. Run `bun x ultracite fix` before committing to ensure compliance.
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
{
|
||||||
|
"hooks": {
|
||||||
|
"PostToolUse": [
|
||||||
|
{
|
||||||
|
"matcher": "Write|Edit",
|
||||||
|
"hooks": [
|
||||||
|
{
|
||||||
|
"type": "command",
|
||||||
|
"command": "f=$(python3 -c 'import json,sys;print(json.load(sys.stdin).get(\"tool_input\",{}).get(\"file_path\",\"\"))'); case \"$f\" in */ui/*) cd ui && bun fix --skip=correctness/noUnusedImports ;; *.py) uv run ruff format \"$f\" && uv run ruff check --fix \"$f\" ;; esac"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -14,3 +14,10 @@ examples
|
|||||||
**/.ruff_cache
|
**/.ruff_cache
|
||||||
**/.ty_cache
|
**/.ty_cache
|
||||||
README.md
|
README.md
|
||||||
|
ui/node_modules
|
||||||
|
ui/.svelte-kit
|
||||||
|
ui/build
|
||||||
|
ui/.vscode
|
||||||
|
.claude
|
||||||
|
.mcp.json
|
||||||
|
Makefile
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
# beaver-gateway - локальный dev. В проде переменные приходят из .env
|
||||||
|
# стека beaver-agent (env_file), этот файл там не нужен.
|
||||||
|
|
||||||
|
# Postgres из `make db` (профиль db в ../beaver-agent) или sqlite для быстрого старта
|
||||||
|
DATABASE_URL=postgresql+psycopg://beaver:beaver@localhost:5432/beaver
|
||||||
|
#DATABASE_URL=sqlite+aiosqlite:///./db.db
|
||||||
|
|
||||||
|
# Admin UI
|
||||||
|
ADMIN_USER=admin
|
||||||
|
ADMIN_PASS=admin
|
||||||
|
SESSION_SECRET=dev-only-change-me
|
||||||
|
|
||||||
|
# Пользовательский config.py (агенты, MCP, фронтенды)
|
||||||
|
CONFIG_PATH=../beaver-agent/config.py
|
||||||
|
|
||||||
|
# Токены для Anthropic/MCP фронтендов без admin UI: name:value,name:value
|
||||||
|
BOOTSTRAP_TOKENS=dev:dev-token-change-me
|
||||||
|
|
||||||
|
# Claude: `claude setup-token` на маке
|
||||||
|
CLAUDE_CODE_OAUTH_TOKEN=
|
||||||
|
|
||||||
|
# Raycast (нужно только если в config.py есть RaycastAgent)
|
||||||
|
RAYCAST_CONFIG_PATH=../raycast-api/config.json
|
||||||
|
RAYCAST_DEVICE_ID=
|
||||||
|
RAYCAST_BEARER=
|
||||||
|
|
||||||
|
# Telegram (нужно только если в config.py есть TelegramFrontend):
|
||||||
|
# токен от @BotFather, свой user id - @userinfobot
|
||||||
|
TELEGRAM_BOT_TOKEN=
|
||||||
|
TELEGRAM_USER_ID=
|
||||||
@@ -11,6 +11,7 @@ wheels/
|
|||||||
|
|
||||||
# Sloppy
|
# Sloppy
|
||||||
docs/
|
docs/
|
||||||
|
t/
|
||||||
|
|
||||||
# IDE
|
# IDE
|
||||||
.idea
|
.idea
|
||||||
@@ -18,3 +19,6 @@ docs/
|
|||||||
# Local env
|
# Local env
|
||||||
.env
|
.env
|
||||||
db.db
|
db.db
|
||||||
|
|
||||||
|
# Impeccable review screenshots
|
||||||
|
.impeccable/review/
|
||||||
@@ -0,0 +1,628 @@
|
|||||||
|
{
|
||||||
|
"schemaVersion": 2,
|
||||||
|
"generatedAt": "2026-08-29T00:00:00Z",
|
||||||
|
"title": "Design System: Beaver admin",
|
||||||
|
"extensions": {
|
||||||
|
"colorMeta": {
|
||||||
|
"primary": {
|
||||||
|
"role": "primary",
|
||||||
|
"displayName": "Mulberry",
|
||||||
|
"canonical": "oklch(0.47 0.13 340)",
|
||||||
|
"dark": "#7c3871",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.13 340)",
|
||||||
|
"oklch(0.26 0.13 340)",
|
||||||
|
"oklch(0.37 0.13 340)",
|
||||||
|
"oklch(0.47 0.13 340)",
|
||||||
|
"oklch(0.58 0.13 340)",
|
||||||
|
"oklch(0.70 0.13 340)",
|
||||||
|
"oklch(0.83 0.13 340)",
|
||||||
|
"oklch(0.95 0.13 340)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"signal": {
|
||||||
|
"role": "primary",
|
||||||
|
"displayName": "Orchid Signal",
|
||||||
|
"canonical": "oklch(0.48 0.21 342)",
|
||||||
|
"dark": "#ff82f3",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.21 342)",
|
||||||
|
"oklch(0.26 0.21 342)",
|
||||||
|
"oklch(0.37 0.21 342)",
|
||||||
|
"oklch(0.48 0.21 342)",
|
||||||
|
"oklch(0.60 0.21 342)",
|
||||||
|
"oklch(0.72 0.21 342)",
|
||||||
|
"oklch(0.84 0.21 342)",
|
||||||
|
"oklch(0.95 0.21 342)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"ring": {
|
||||||
|
"role": "primary",
|
||||||
|
"displayName": "Focus Ring",
|
||||||
|
"canonical": "oklch(0.55 0.2 342)",
|
||||||
|
"dark": "#ff82f3",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.2 342)",
|
||||||
|
"oklch(0.26 0.2 342)",
|
||||||
|
"oklch(0.38 0.2 342)",
|
||||||
|
"oklch(0.55 0.2 342)",
|
||||||
|
"oklch(0.64 0.2 342)",
|
||||||
|
"oklch(0.74 0.2 342)",
|
||||||
|
"oklch(0.85 0.2 342)",
|
||||||
|
"oklch(0.95 0.2 342)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"status-reply": {
|
||||||
|
"role": "secondary",
|
||||||
|
"displayName": "Branch Blue",
|
||||||
|
"canonical": "oklch(0.55 0.14 235)",
|
||||||
|
"dark": "oklch(0.72 0.13 235)",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.14 235)",
|
||||||
|
"oklch(0.26 0.14 235)",
|
||||||
|
"oklch(0.40 0.14 235)",
|
||||||
|
"oklch(0.55 0.14 235)",
|
||||||
|
"oklch(0.65 0.14 235)",
|
||||||
|
"oklch(0.75 0.14 235)",
|
||||||
|
"oklch(0.85 0.14 235)",
|
||||||
|
"oklch(0.95 0.14 235)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"status-meeting": {
|
||||||
|
"role": "secondary",
|
||||||
|
"displayName": "Deep Violet",
|
||||||
|
"canonical": "oklch(0.5 0.2 300)",
|
||||||
|
"dark": "oklch(0.72 0.16 300)",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.2 300)",
|
||||||
|
"oklch(0.26 0.2 300)",
|
||||||
|
"oklch(0.38 0.2 300)",
|
||||||
|
"oklch(0.50 0.2 300)",
|
||||||
|
"oklch(0.62 0.2 300)",
|
||||||
|
"oklch(0.74 0.2 300)",
|
||||||
|
"oklch(0.85 0.2 300)",
|
||||||
|
"oklch(0.95 0.2 300)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"status-work": {
|
||||||
|
"role": "secondary",
|
||||||
|
"displayName": "Job Teal",
|
||||||
|
"canonical": "oklch(0.55 0.13 195)",
|
||||||
|
"dark": "oklch(0.75 0.12 195)",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.13 195)",
|
||||||
|
"oklch(0.26 0.13 195)",
|
||||||
|
"oklch(0.40 0.13 195)",
|
||||||
|
"oklch(0.55 0.13 195)",
|
||||||
|
"oklch(0.65 0.13 195)",
|
||||||
|
"oklch(0.75 0.13 195)",
|
||||||
|
"oklch(0.85 0.13 195)",
|
||||||
|
"oklch(0.95 0.13 195)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"status-skip": {
|
||||||
|
"role": "secondary",
|
||||||
|
"displayName": "Fork Grey",
|
||||||
|
"canonical": "oklch(0.5 0.04 340)",
|
||||||
|
"dark": "#a989a3",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.04 340)",
|
||||||
|
"oklch(0.26 0.04 340)",
|
||||||
|
"oklch(0.38 0.04 340)",
|
||||||
|
"oklch(0.50 0.04 340)",
|
||||||
|
"oklch(0.62 0.04 340)",
|
||||||
|
"oklch(0.74 0.04 340)",
|
||||||
|
"oklch(0.85 0.04 340)",
|
||||||
|
"oklch(0.95 0.04 340)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"status-done": {
|
||||||
|
"role": "tertiary",
|
||||||
|
"displayName": "OK Green",
|
||||||
|
"canonical": "oklch(0.55 0.14 155)",
|
||||||
|
"dark": "oklch(0.74 0.14 155)",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.14 155)",
|
||||||
|
"oklch(0.26 0.14 155)",
|
||||||
|
"oklch(0.40 0.14 155)",
|
||||||
|
"oklch(0.55 0.14 155)",
|
||||||
|
"oklch(0.65 0.14 155)",
|
||||||
|
"oklch(0.75 0.14 155)",
|
||||||
|
"oklch(0.85 0.14 155)",
|
||||||
|
"oklch(0.95 0.14 155)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"warn": {
|
||||||
|
"role": "tertiary",
|
||||||
|
"displayName": "Warn Amber",
|
||||||
|
"canonical": "oklch(0.55 0.13 75)",
|
||||||
|
"dark": "oklch(0.8 0.13 75)",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.13 75)",
|
||||||
|
"oklch(0.26 0.13 75)",
|
||||||
|
"oklch(0.40 0.13 75)",
|
||||||
|
"oklch(0.55 0.13 75)",
|
||||||
|
"oklch(0.65 0.13 75)",
|
||||||
|
"oklch(0.75 0.13 75)",
|
||||||
|
"oklch(0.85 0.13 75)",
|
||||||
|
"oklch(0.95 0.13 75)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"note": {
|
||||||
|
"role": "tertiary",
|
||||||
|
"displayName": "Inject Amber",
|
||||||
|
"canonical": "oklch(0.52 0.13 78)",
|
||||||
|
"dark": "oklch(0.84 0.14 88)",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.13 78)",
|
||||||
|
"oklch(0.26 0.13 78)",
|
||||||
|
"oklch(0.40 0.13 78)",
|
||||||
|
"oklch(0.52 0.13 78)",
|
||||||
|
"oklch(0.64 0.13 78)",
|
||||||
|
"oklch(0.75 0.13 78)",
|
||||||
|
"oklch(0.85 0.13 78)",
|
||||||
|
"oklch(0.95 0.13 78)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"destructive": {
|
||||||
|
"role": "tertiary",
|
||||||
|
"displayName": "Error Red",
|
||||||
|
"canonical": "oklch(0.58 0.22 25)",
|
||||||
|
"dark": "oklch(0.62 0.2 25)",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.22 25)",
|
||||||
|
"oklch(0.26 0.22 25)",
|
||||||
|
"oklch(0.40 0.22 25)",
|
||||||
|
"oklch(0.58 0.22 25)",
|
||||||
|
"oklch(0.66 0.22 25)",
|
||||||
|
"oklch(0.75 0.22 25)",
|
||||||
|
"oklch(0.85 0.22 25)",
|
||||||
|
"oklch(0.95 0.22 25)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"background": {
|
||||||
|
"role": "neutral",
|
||||||
|
"displayName": "Blush Paper",
|
||||||
|
"canonical": "oklch(0.99 0.004 340)",
|
||||||
|
"dark": "#22111e",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.004 340)",
|
||||||
|
"oklch(0.26 0.004 340)",
|
||||||
|
"oklch(0.40 0.004 340)",
|
||||||
|
"oklch(0.55 0.004 340)",
|
||||||
|
"oklch(0.70 0.004 340)",
|
||||||
|
"oklch(0.85 0.004 340)",
|
||||||
|
"oklch(0.95 0.004 340)",
|
||||||
|
"oklch(0.99 0.004 340)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"foreground": {
|
||||||
|
"role": "neutral",
|
||||||
|
"displayName": "Plum Ink",
|
||||||
|
"canonical": "oklch(0.24 0.03 340)",
|
||||||
|
"dark": "#f5f5f5",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.03 340)",
|
||||||
|
"oklch(0.24 0.03 340)",
|
||||||
|
"oklch(0.38 0.03 340)",
|
||||||
|
"oklch(0.50 0.03 340)",
|
||||||
|
"oklch(0.62 0.03 340)",
|
||||||
|
"oklch(0.75 0.03 340)",
|
||||||
|
"oklch(0.87 0.03 340)",
|
||||||
|
"oklch(0.95 0.03 340)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"sidebar": {
|
||||||
|
"role": "neutral",
|
||||||
|
"displayName": "Sidebar Layer",
|
||||||
|
"canonical": "oklch(0.975 0.01 340)",
|
||||||
|
"dark": "#2c1b29",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.01 340)",
|
||||||
|
"oklch(0.26 0.01 340)",
|
||||||
|
"oklch(0.40 0.01 340)",
|
||||||
|
"oklch(0.55 0.01 340)",
|
||||||
|
"oklch(0.70 0.01 340)",
|
||||||
|
"oklch(0.85 0.01 340)",
|
||||||
|
"oklch(0.94 0.01 340)",
|
||||||
|
"oklch(0.975 0.01 340)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"sidebar-accent": {
|
||||||
|
"role": "neutral",
|
||||||
|
"displayName": "Sidebar Accent",
|
||||||
|
"canonical": "oklch(0.94 0.016 340)",
|
||||||
|
"dark": "#341d2f",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.016 340)",
|
||||||
|
"oklch(0.26 0.016 340)",
|
||||||
|
"oklch(0.40 0.016 340)",
|
||||||
|
"oklch(0.55 0.016 340)",
|
||||||
|
"oklch(0.70 0.016 340)",
|
||||||
|
"oklch(0.85 0.016 340)",
|
||||||
|
"oklch(0.94 0.016 340)",
|
||||||
|
"oklch(0.98 0.016 340)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"muted": {
|
||||||
|
"role": "neutral",
|
||||||
|
"displayName": "Muted",
|
||||||
|
"canonical": "oklch(0.96 0.012 340)",
|
||||||
|
"dark": "#40283a",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.012 340)",
|
||||||
|
"oklch(0.26 0.012 340)",
|
||||||
|
"oklch(0.40 0.012 340)",
|
||||||
|
"oklch(0.55 0.012 340)",
|
||||||
|
"oklch(0.70 0.012 340)",
|
||||||
|
"oklch(0.85 0.012 340)",
|
||||||
|
"oklch(0.96 0.012 340)",
|
||||||
|
"oklch(0.99 0.012 340)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"muted-foreground": {
|
||||||
|
"role": "neutral",
|
||||||
|
"displayName": "Muted Ink",
|
||||||
|
"canonical": "oklch(0.5 0.04 340)",
|
||||||
|
"dark": "#a989a3",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.04 340)",
|
||||||
|
"oklch(0.26 0.04 340)",
|
||||||
|
"oklch(0.38 0.04 340)",
|
||||||
|
"oklch(0.50 0.04 340)",
|
||||||
|
"oklch(0.62 0.04 340)",
|
||||||
|
"oklch(0.74 0.04 340)",
|
||||||
|
"oklch(0.85 0.04 340)",
|
||||||
|
"oklch(0.95 0.04 340)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"icon": {
|
||||||
|
"role": "neutral",
|
||||||
|
"displayName": "Icon",
|
||||||
|
"canonical": "oklch(0.55 0.03 340)",
|
||||||
|
"dark": "#877384",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.03 340)",
|
||||||
|
"oklch(0.26 0.03 340)",
|
||||||
|
"oklch(0.40 0.03 340)",
|
||||||
|
"oklch(0.55 0.03 340)",
|
||||||
|
"oklch(0.65 0.03 340)",
|
||||||
|
"oklch(0.75 0.03 340)",
|
||||||
|
"oklch(0.85 0.03 340)",
|
||||||
|
"oklch(0.95 0.03 340)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"border": {
|
||||||
|
"role": "neutral",
|
||||||
|
"displayName": "Hairline",
|
||||||
|
"canonical": "oklch(0.9 0.02 340)",
|
||||||
|
"dark": "oklch(0.92 0.04 340 / 10%)",
|
||||||
|
"tonalRamp": [
|
||||||
|
"oklch(0.15 0.02 340)",
|
||||||
|
"oklch(0.26 0.02 340)",
|
||||||
|
"oklch(0.40 0.02 340)",
|
||||||
|
"oklch(0.55 0.02 340)",
|
||||||
|
"oklch(0.70 0.02 340)",
|
||||||
|
"oklch(0.80 0.02 340)",
|
||||||
|
"oklch(0.90 0.02 340)",
|
||||||
|
"oklch(0.97 0.02 340)"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"typographyMeta": {
|
||||||
|
"headline": {
|
||||||
|
"displayName": "Headline",
|
||||||
|
"purpose": "Page titles in page-header.svelte; the sidebar wordmark shares weight and tracking."
|
||||||
|
},
|
||||||
|
"title": {
|
||||||
|
"displayName": "Title",
|
||||||
|
"purpose": "Thread title, stat values (tabular), memory file path."
|
||||||
|
},
|
||||||
|
"body": {
|
||||||
|
"displayName": "Body",
|
||||||
|
"purpose": "Rows, cells, bubbles, dialogs; primary row text at 500."
|
||||||
|
},
|
||||||
|
"label": {
|
||||||
|
"displayName": "Label",
|
||||||
|
"purpose": "Uppercase section headings; same size without uppercase for table headers, badges, pills, meta lines."
|
||||||
|
},
|
||||||
|
"mono": {
|
||||||
|
"displayName": "Mono",
|
||||||
|
"purpose": "Raw JSON, tool input/result blocks, event tape, ids, the plaintext token."
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"shadows": [
|
||||||
|
{
|
||||||
|
"name": "login-lift",
|
||||||
|
"value": "0 8px 24px -12px rgb(0 0 0 / 0.25)",
|
||||||
|
"purpose": "The sign-in card only (routes/login/+page.svelte). The single application-layer shadow."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "popover",
|
||||||
|
"value": "shadow-xl plus ring-1 ring-foreground/5 (10% dark)",
|
||||||
|
"purpose": "Luma dialogs, sheets, menus, selects. Overlay surfaces only."
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"motion": [
|
||||||
|
{
|
||||||
|
"name": "pulse-dot",
|
||||||
|
"value": "pulse-dot 1.6s cubic-bezier(0.4, 0, 0.6, 1) infinite",
|
||||||
|
"purpose": "Signal-colored halo (0 to 5px, 55% to 0% alpha) on live and running dots."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "rise",
|
||||||
|
"value": "rise 220ms cubic-bezier(0.16, 1, 0.3, 1) both",
|
||||||
|
"purpose": "Opacity 0 to 1 and translateY 3px to 0 when a turn card lands in the activity feed."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "row-hover",
|
||||||
|
"value": "background-color 150ms ease-out",
|
||||||
|
"purpose": "Ink wash at 4% on interactive rows."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "chevron",
|
||||||
|
"value": "transform 150ms",
|
||||||
|
"purpose": "90 degree rotation of expand chevrons in tool nodes, raw entries, memory tree."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "bar-fill",
|
||||||
|
"value": "width 300ms",
|
||||||
|
"purpose": "Utilization bar fill in limit-bar.svelte."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "dialog",
|
||||||
|
"value": "fade-in and zoom-in-95 over 100ms",
|
||||||
|
"purpose": "Luma dialog open and close."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "reduced-motion",
|
||||||
|
"value": "transition-duration and animation-duration 0.01ms under prefers-reduced-motion",
|
||||||
|
"purpose": "Global kill switch in layout.css."
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"breakpoints": [
|
||||||
|
{
|
||||||
|
"name": "sm",
|
||||||
|
"value": "640px",
|
||||||
|
"purpose": "Sidebar replaces bottom nav; header padding 24px; hidden row columns return."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "md",
|
||||||
|
"value": "768px",
|
||||||
|
"purpose": "Memory tree and file split side by side."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "lg",
|
||||||
|
"value": "1024px",
|
||||||
|
"purpose": "Thread meta rail (18rem) and two-column dashboard sections."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "panel-rail",
|
||||||
|
"value": "768px",
|
||||||
|
"purpose": "Panel container width, not viewport: below it the switcher bar names the thread, from it the 18rem conversation rail shows."
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"obsidian": {
|
||||||
|
"root": ".beaver-root",
|
||||||
|
"source": "beaver-plugin-obsidian/src/tailwind.css",
|
||||||
|
"aliases": {
|
||||||
|
"background": "--background-primary",
|
||||||
|
"card": "--background-primary-alt",
|
||||||
|
"popover": "--background-secondary",
|
||||||
|
"sidebar": "--background-secondary",
|
||||||
|
"foreground": "--text-normal",
|
||||||
|
"muted-foreground": "--text-muted",
|
||||||
|
"icon": "--text-muted",
|
||||||
|
"status-skip": "--text-faint",
|
||||||
|
"border": "--background-modifier-border",
|
||||||
|
"input": "--background-modifier-border",
|
||||||
|
"muted": "--background-modifier-hover",
|
||||||
|
"secondary": "--background-modifier-hover",
|
||||||
|
"accent": "--background-modifier-hover",
|
||||||
|
"sidebar-accent": "--background-modifier-hover",
|
||||||
|
"primary": "--color-accent",
|
||||||
|
"signal": "--color-accent",
|
||||||
|
"ring": "--color-accent",
|
||||||
|
"status-new": "--color-accent",
|
||||||
|
"primary-foreground": "--text-on-accent",
|
||||||
|
"link": "--link-color, --text-accent",
|
||||||
|
"destructive": "--text-error",
|
||||||
|
"font-sans": "--font-interface",
|
||||||
|
"font-mono": "--font-monospace"
|
||||||
|
},
|
||||||
|
"radii": {
|
||||||
|
"button, select-trigger": "--button-radius",
|
||||||
|
"input, tabs-trigger, menu items": "--input-radius",
|
||||||
|
"tabs-list, skeleton, dialog, popover, menus": "--radius-m (8px)",
|
||||||
|
"rows and notes": "--radius 0.45rem"
|
||||||
|
},
|
||||||
|
"overlays": {
|
||||||
|
"border": "1px var(--border)",
|
||||||
|
"shadow": "var(--shadow-s)",
|
||||||
|
"menuBackground": "var(--popover), opaque, no backdrop",
|
||||||
|
"portal": ".beaver-root.beaver-portal on body at --layer-menu (65)"
|
||||||
|
},
|
||||||
|
"tabs": {
|
||||||
|
"list": "color-mix(in oklab, var(--text-normal) 6%, transparent)",
|
||||||
|
"active": "color-mix(in oklab, var(--text-normal) 14%, transparent)"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"components": [
|
||||||
|
{
|
||||||
|
"name": "Sidebar nav item",
|
||||||
|
"kind": "nav",
|
||||||
|
"refersTo": "sidebar-nav-item",
|
||||||
|
"description": "Section link in the 13rem sidebar; the active item takes the sidebar accent fill, the Now item carries the running count in signal.",
|
||||||
|
"html": "<nav class=\"ds-nav\"><a class=\"ds-nav-item ds-nav-item--active\" href=\"#\" aria-current=\"page\"><svg width=\"16\" height=\"16\" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\"><path d=\"M22 12h-2.48a2 2 0 0 0-1.93 1.46l-2.35 8.36a.25.25 0 0 1-.48 0L9.24 2.18a.25.25 0 0 0-.48 0l-2.35 8.36A2 2 0 0 1 4.49 12H2\"/></svg><span class=\"ds-nav-label\">Now</span><span class=\"ds-nav-count\">1</span></a><a class=\"ds-nav-item\" href=\"#\"><svg width=\"16\" height=\"16\" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\"><path d=\"M14 9a2 2 0 0 1-2 2H6l-4 4V4c0-1.1.9-2 2-2h8a2 2 0 0 1 2 2z\"/><path d=\"M18 9h2a2 2 0 0 1 2 2v11l-4-4h-6a2 2 0 0 1-2-2v-1\"/></svg><span class=\"ds-nav-label\">Conversations</span></a></nav>",
|
||||||
|
"css": ".ds-nav { display: flex; flex-direction: column; gap: 2px; padding: 8px; width: 208px; background: var(--sidebar); color: var(--sidebar-foreground); font-family: 'Inter Variable', sans-serif; } .ds-nav-item { display: flex; align-items: center; gap: 10px; height: 32px; padding: 0 10px; border-radius: 0.36rem; font-size: 0.8125rem; color: color-mix(in oklab, var(--sidebar-foreground) 80%, transparent); text-decoration: none; transition: background-color 150ms, color 150ms; } .ds-nav-item svg { color: var(--icon); flex-shrink: 0; } .ds-nav-item:hover { background: color-mix(in oklab, var(--sidebar-accent) 60%, transparent); color: var(--sidebar-foreground); } .ds-nav-item--active { background: var(--sidebar-accent); color: var(--sidebar-foreground); font-weight: 500; } .ds-nav-item:focus-visible { outline: 2px solid var(--ring); outline-offset: 2px; } .ds-nav-label { flex: 1; } .ds-nav-count { font-variant-numeric: tabular-nums; font-size: 0.75rem; font-weight: 500; padding: 0 6px; border-radius: 999px; color: var(--signal); background: color-mix(in oklab, var(--signal) 12%, transparent); }"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Section heading",
|
||||||
|
"kind": "custom",
|
||||||
|
"refersTo": "section-heading",
|
||||||
|
"description": "The 12px uppercase muted h2 that opens every ledger section, optionally followed by a tabular summary.",
|
||||||
|
"html": "<div class=\"ds-section-head\"><h2 class=\"ds-section-heading\">Live sessions</h2><span class=\"ds-section-summary\">1 processes · 263 MB</span></div>",
|
||||||
|
"css": ".ds-section-head { display: flex; align-items: baseline; gap: 12px; font-family: 'Inter Variable', sans-serif; } .ds-section-heading { margin: 0; font-size: 0.75rem; font-weight: 500; letter-spacing: 0.025em; text-transform: uppercase; color: var(--muted-foreground); } .ds-section-summary { font-size: 0.75rem; color: var(--muted-foreground); font-variant-numeric: tabular-nums; }"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Ledger row (running)",
|
||||||
|
"kind": "card",
|
||||||
|
"refersTo": "ledger-row",
|
||||||
|
"description": "A running conversation on the Now page: pulsing signal dot, kind badge, two-line title, tabular elapsed time, hairline below, ink-wash hover.",
|
||||||
|
"html": "<a class=\"ds-row\" href=\"#\"><span class=\"ds-row-dot\"></span><span class=\"ds-badge ds-badge--master\">master</span><span class=\"ds-row-title\"><span class=\"ds-row-name\">master e56eabe0</span><span class=\"ds-row-sub\">beaver-dispatcher</span></span><span class=\"ds-row-time\">20.1s</span></a>",
|
||||||
|
"css": ".ds-row { display: grid; grid-template-columns: auto auto minmax(0, 1fr) auto; column-gap: 12px; align-items: center; padding: 8px 0; border-bottom: 1px solid var(--border); font-family: 'Inter Variable', sans-serif; font-size: 0.8125rem; color: var(--foreground); text-decoration: none; transition: background-color 150ms ease-out; } .ds-row:hover { background: color-mix(in oklab, var(--foreground) 4%, transparent); } .ds-row:focus-visible { outline: 2px solid var(--ring); outline-offset: 2px; } .ds-row-dot { width: 8px; height: 8px; border-radius: 999px; background: var(--signal); animation: ds-pulse 1.6s cubic-bezier(0.4, 0, 0.6, 1) infinite; } @keyframes ds-pulse { 0%, 100% { box-shadow: 0 0 0 0 color-mix(in oklab, var(--signal) 55%, transparent); } 50% { box-shadow: 0 0 0 5px color-mix(in oklab, var(--signal) 0%, transparent); } } .ds-row-title { display: flex; flex-direction: column; min-width: 0; } .ds-row-name { font-weight: 500; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } .ds-row-sub { font-size: 0.75rem; color: var(--muted-foreground); } .ds-row-time { font-size: 0.75rem; color: var(--muted-foreground); font-variant-numeric: tabular-nums; } .ds-badge { display: inline-flex; align-items: center; height: 20px; padding: 0 6px; border-radius: 0.36rem; font-size: 0.75rem; font-weight: 500; line-height: 1; } .ds-badge--master { color: var(--signal); background: color-mix(in oklab, var(--signal) 12%, transparent); }"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Kind badge",
|
||||||
|
"kind": "chip",
|
||||||
|
"refersTo": "kind-badge",
|
||||||
|
"description": "Conversation kind as its hue at 12% behind hue text; master, branch, deep, job, fork.",
|
||||||
|
"html": "<span class=\"ds-kinds\"><span class=\"ds-badge ds-badge--master\">master</span><span class=\"ds-badge ds-badge--branch\">branch</span><span class=\"ds-badge ds-badge--deep\">deep</span><span class=\"ds-badge ds-badge--job\">job</span><span class=\"ds-badge ds-badge--fork\">fork</span></span>",
|
||||||
|
"css": ".ds-kinds { display: inline-flex; gap: 4px; font-family: 'Inter Variable', sans-serif; } .ds-badge { display: inline-flex; align-items: center; height: 20px; padding: 0 6px; border-radius: 0.36rem; font-size: 0.75rem; font-weight: 500; line-height: 1; } .ds-badge--master { color: var(--signal); background: color-mix(in oklab, var(--signal) 12%, transparent); } .ds-badge--branch { color: var(--status-reply); background: color-mix(in oklab, var(--status-reply) 12%, transparent); } .ds-badge--deep { color: var(--status-meeting); background: color-mix(in oklab, var(--status-meeting) 12%, transparent); } .ds-badge--job { color: var(--status-work); background: color-mix(in oklab, var(--status-work) 12%, transparent); } .ds-badge--fork { color: var(--status-skip); background: color-mix(in oklab, var(--status-skip) 12%, transparent); }"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Status pill",
|
||||||
|
"kind": "chip",
|
||||||
|
"refersTo": "status-pill",
|
||||||
|
"description": "Outcome as a 6px current-color dot plus 12px text; running pulses in signal.",
|
||||||
|
"html": "<span class=\"ds-pills\"><span class=\"ds-pill ds-pill--running\"><span class=\"ds-pill-dot ds-pill-dot--pulse\"></span>running</span><span class=\"ds-pill ds-pill--done\"><span class=\"ds-pill-dot\"></span>done</span><span class=\"ds-pill ds-pill--warn\"><span class=\"ds-pill-dot\"></span>interrupted</span><span class=\"ds-pill ds-pill--error\"><span class=\"ds-pill-dot\"></span>failed</span><span class=\"ds-pill ds-pill--muted\"><span class=\"ds-pill-dot\"></span>closed</span></span>",
|
||||||
|
"css": ".ds-pills { display: inline-flex; gap: 12px; font-family: 'Inter Variable', sans-serif; } .ds-pill { display: inline-flex; align-items: center; gap: 6px; font-size: 0.75rem; font-weight: 500; } .ds-pill-dot { width: 6px; height: 6px; border-radius: 999px; background: currentColor; } .ds-pill-dot--pulse { animation: ds-pulse 1.6s cubic-bezier(0.4, 0, 0.6, 1) infinite; } @keyframes ds-pulse { 0%, 100% { box-shadow: 0 0 0 0 color-mix(in oklab, var(--signal) 55%, transparent); } 50% { box-shadow: 0 0 0 5px color-mix(in oklab, var(--signal) 0%, transparent); } } .ds-pill--running { color: var(--signal); } .ds-pill--done { color: var(--status-done); } .ds-pill--warn { color: var(--warn); } .ds-pill--error { color: var(--destructive); } .ds-pill--muted { color: var(--muted-foreground); }"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Tool node",
|
||||||
|
"kind": "custom",
|
||||||
|
"refersTo": "ledger-row",
|
||||||
|
"description": "One row of the tool tree: chevron, per-node status dot, tool label (deep-tinted for subagents) with input summary, tabular duration; children nest under a left rail.",
|
||||||
|
"html": "<div class=\"ds-tool\"><button class=\"ds-tool-row\" type=\"button\" aria-expanded=\"false\"><svg class=\"ds-tool-chev\" width=\"14\" height=\"14\" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\"><path d=\"m9 18 6-6-6-6\"/></svg><span class=\"ds-tool-dot ds-tool-dot--running\"></span><span class=\"ds-tool-main\"><span class=\"ds-tool-name\">Bash</span><span class=\"ds-tool-sum\">python3 -c \"import time; time.sleep(30)\"</span></span><span class=\"ds-tool-time\">13.5s</span></button><div class=\"ds-tool ds-tool--child\"><button class=\"ds-tool-row\" type=\"button\" aria-expanded=\"false\"><svg class=\"ds-tool-chev\" width=\"14\" height=\"14\" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\"><path d=\"m9 18 6-6-6-6\"/></svg><span class=\"ds-tool-dot ds-tool-dot--done\"></span><span class=\"ds-tool-main\"><span class=\"ds-tool-name ds-tool-name--subagent\">Agent</span><span class=\"ds-tool-sum\">Read role.md and summarize</span></span><span class=\"ds-tool-time\">12.1s</span></button></div></div>",
|
||||||
|
"css": ".ds-tool { display: flex; flex-direction: column; font-family: 'Inter Variable', sans-serif; } .ds-tool--child { margin-left: 12px; padding-left: 8px; border-left: 1px solid var(--border); } .ds-tool-row { display: grid; grid-template-columns: auto auto minmax(0, 1fr) auto; column-gap: 12px; align-items: center; padding: 4px 6px; border: 0; border-radius: 0.36rem; background: transparent; color: var(--foreground); font: inherit; font-size: 0.8125rem; text-align: left; cursor: pointer; transition: background-color 150ms ease-out; } .ds-tool-row:hover { background: color-mix(in oklab, var(--foreground) 4%, transparent); } .ds-tool-row:focus-visible { outline: 2px solid var(--ring); outline-offset: 2px; } .ds-tool-chev { color: var(--icon); transition: transform 150ms; } .ds-tool-row[aria-expanded=\"true\"] .ds-tool-chev { transform: rotate(90deg); } .ds-tool-dot { width: 8px; height: 8px; border-radius: 999px; } .ds-tool-dot--running { background: var(--signal); animation: ds-pulse 1.6s cubic-bezier(0.4, 0, 0.6, 1) infinite; } .ds-tool-dot--done { background: var(--status-done); } @keyframes ds-pulse { 0%, 100% { box-shadow: 0 0 0 0 color-mix(in oklab, var(--signal) 55%, transparent); } 50% { box-shadow: 0 0 0 5px color-mix(in oklab, var(--signal) 0%, transparent); } } .ds-tool-main { display: flex; align-items: baseline; gap: 8px; min-width: 0; } .ds-tool-name { font-weight: 500; flex-shrink: 0; } .ds-tool-name--subagent { color: var(--status-meeting); } .ds-tool-sum { color: var(--muted-foreground); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } .ds-tool-time { font-size: 0.75rem; color: var(--muted-foreground); font-variant-numeric: tabular-nums; }"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Primary button",
|
||||||
|
"kind": "button",
|
||||||
|
"refersTo": "button-primary",
|
||||||
|
"description": "Luma pill button in mulberry; the one filled action per surface (Send, New, Create, Sign in).",
|
||||||
|
"html": "<button class=\"ds-btn ds-btn--primary\" type=\"button\"><svg width=\"16\" height=\"16\" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\"><path d=\"M5 12h14\"/><path d=\"M12 5v14\"/></svg>New token</button>",
|
||||||
|
"css": ".ds-btn { display: inline-flex; align-items: center; justify-content: center; gap: 4px; height: 32px; padding: 0 12px; border: 1px solid transparent; border-radius: 2rem; font-family: 'Inter Variable', sans-serif; font-size: 0.8125rem; font-weight: 500; white-space: nowrap; cursor: pointer; user-select: none; transition: all 150ms; } .ds-btn:active { transform: translateY(1px); } .ds-btn:disabled { opacity: 0.5; pointer-events: none; } .ds-btn:focus-visible { border-color: var(--ring); box-shadow: 0 0 0 3px color-mix(in oklab, var(--ring) 30%, transparent); outline: none; } .ds-btn--primary { background: var(--primary); color: var(--primary-foreground); } .ds-btn--primary:hover { background: color-mix(in oklab, var(--primary) 80%, transparent); }"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Outline button",
|
||||||
|
"kind": "button",
|
||||||
|
"refersTo": "button-outline",
|
||||||
|
"description": "Secondary action everywhere (Branch, Retry, Copy, answer options): hairline pill on the background surface.",
|
||||||
|
"html": "<button class=\"ds-btn ds-btn--outline\" type=\"button\"><svg width=\"16\" height=\"16\" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\"><line x1=\"6\" x2=\"6\" y1=\"3\" y2=\"15\"/><circle cx=\"18\" cy=\"6\" r=\"3\"/><circle cx=\"6\" cy=\"18\" r=\"3\"/><path d=\"M18 9a9 9 0 0 1-9 9\"/></svg>Branch</button>",
|
||||||
|
"css": ".ds-btn { display: inline-flex; align-items: center; justify-content: center; gap: 4px; height: 32px; padding: 0 12px; border: 1px solid transparent; border-radius: 2rem; font-family: 'Inter Variable', sans-serif; font-size: 0.8125rem; font-weight: 500; white-space: nowrap; cursor: pointer; user-select: none; transition: all 150ms; } .ds-btn:active { transform: translateY(1px); } .ds-btn:focus-visible { border-color: var(--ring); box-shadow: 0 0 0 3px color-mix(in oklab, var(--ring) 30%, transparent); outline: none; } .ds-btn--outline { border-color: var(--border); background: var(--background); color: var(--foreground); } .ds-btn--outline:hover { background: var(--muted); }"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Ghost button",
|
||||||
|
"kind": "button",
|
||||||
|
"refersTo": "button-ghost",
|
||||||
|
"description": "Quiet action (Cancel, Revoke, Done, icon buttons): transparent until hover.",
|
||||||
|
"html": "<button class=\"ds-btn ds-btn--ghost\" type=\"button\">Cancel</button>",
|
||||||
|
"css": ".ds-btn { display: inline-flex; align-items: center; justify-content: center; gap: 4px; height: 32px; padding: 0 12px; border: 1px solid transparent; border-radius: 2rem; font-family: 'Inter Variable', sans-serif; font-size: 0.8125rem; font-weight: 500; white-space: nowrap; cursor: pointer; user-select: none; transition: all 150ms; } .ds-btn:active { transform: translateY(1px); } .ds-btn:focus-visible { border-color: var(--ring); box-shadow: 0 0 0 3px color-mix(in oklab, var(--ring) 30%, transparent); outline: none; } .ds-btn--ghost { background: transparent; color: var(--foreground); } .ds-btn--ghost:hover { background: var(--muted); }"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Text input",
|
||||||
|
"kind": "input",
|
||||||
|
"refersTo": "input",
|
||||||
|
"description": "Luma pill input on a 50% input tint; focus is a ring-colored border plus a 30% halo.",
|
||||||
|
"html": "<label class=\"ds-field\"><span class=\"ds-label\">Name</span><input class=\"ds-input\" type=\"text\" placeholder=\"cursor-laptop\"></label>",
|
||||||
|
"css": ".ds-field { display: flex; flex-direction: column; gap: 6px; font-family: 'Inter Variable', sans-serif; max-width: 320px; } .ds-label { font-size: 0.8125rem; font-weight: 500; color: var(--foreground); } .ds-input { height: 36px; width: 100%; padding: 4px 12px; border: 1px solid transparent; border-radius: 1.5rem; background: color-mix(in oklab, var(--input) 50%, transparent); color: var(--foreground); font: inherit; font-size: 0.8125rem; caret-color: var(--primary); outline: none; transition: color 150ms, box-shadow 150ms, background-color 150ms; } .ds-input::placeholder { color: var(--muted-foreground); } .ds-input:focus-visible { border-color: var(--ring); box-shadow: 0 0 0 3px color-mix(in oklab, var(--ring) 30%, transparent); } .ds-input:disabled { opacity: 0.5; cursor: not-allowed; }"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Empty state",
|
||||||
|
"kind": "card",
|
||||||
|
"refersTo": "empty-state",
|
||||||
|
"description": "The only dashed border in the system: a bordered note with a 500 title, a muted hint and an optional outline action.",
|
||||||
|
"html": "<div class=\"ds-empty\"><p class=\"ds-empty-title\">The agents are idle</p><p class=\"ds-empty-hint\">Nothing is in a turn. The ledger fills in the moment a message or an inject lands.</p></div>",
|
||||||
|
"css": ".ds-empty { display: flex; flex-direction: column; align-items: flex-start; gap: 4px; padding: 20px 16px; border: 1px dashed var(--border); border-radius: 0.45rem; font-family: 'Inter Variable', sans-serif; font-size: 0.8125rem; color: var(--foreground); } .ds-empty p { margin: 0; } .ds-empty-title { font-weight: 500; } .ds-empty-hint { color: var(--muted-foreground); }"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"narrative": {
|
||||||
|
"northStar": "The Live Ledger",
|
||||||
|
"overview": "The console is a ledger, not a dashboard. It opens on what the agents are doing right now, as rows of running turns and tool calls, and refuses the grid of metric cards (direction contract, ui/src/app.html). Everything is a row on a rail: hairline borders separate rows, tabular numerals line up columns, and small uppercase section headings name each ledger. Cards exist only as bordered notes for exceptions (an empty state, an error, a pending question, the login form), never as containers for other cards.\n\nThe palette is the msos mauve/pink theme on Inter, inherited verbatim from Kit's msos project (ui/src/routes/layout.css, :root and .dark). Two neutral layers do the spatial work: a slightly cooler, tinted sidebar layer for navigation and rails, and the content surface. One saturated accent (signal) is reserved for two meanings only: something is live right now, or something is selected. Every other hue is a status or kind hue from the msos status palette, used at low alpha as a tint and at full strength as text.\n\nDensity is that of an operator console: 13px body, 12px labels, 32px nav items, 8px row padding, 48px page header. Motion is limited to a soft pulse on live dots, a 220ms rise when a turn lands, and 150ms hover fades; everything collapses under prefers-reduced-motion.",
|
||||||
|
"keyCharacteristics": [
|
||||||
|
"Rows and rails with hairline borders; no cards inside cards",
|
||||||
|
"Tabular numerals on every number; right-aligned numeric columns",
|
||||||
|
"Uppercase 12px section headings in muted ink",
|
||||||
|
"Pink (signal) means live or selected, nothing else",
|
||||||
|
"Kind hues (master, branch, deep, job, fork) carried by low-alpha badges and text tints",
|
||||||
|
"Pill-shaped luma primitives (buttons, inputs, tabs) on top of small-radius application rows",
|
||||||
|
"Light and dark modes, both tinted toward hue 340; dark borders are 10% alpha overlays"
|
||||||
|
],
|
||||||
|
"rules": [
|
||||||
|
{
|
||||||
|
"name": "The Pink Ration Rule",
|
||||||
|
"body": "signal marks live state (pulsing dots, running rows, busy sessions, the running count) and selection (active mobile nav, master kind). It never decorates a heading, a button, or a static surface. Hover is a 4% ink wash, not pink.",
|
||||||
|
"section": "colors"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "The Twelve Percent Rule",
|
||||||
|
"body": "Every colored chip is its hue at 12% alpha behind full-strength text of the same hue (kind-badge.svelte, sidebar running count). Solid colored chips do not exist.",
|
||||||
|
"section": "colors"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "The Two Layers Rule",
|
||||||
|
"body": "Navigation and rails sit on sidebar (or sidebar/50); content sits on background. There is no third surface color; depth beyond that is a hairline.",
|
||||||
|
"section": "colors"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "The Tabular Rule",
|
||||||
|
"body": "Every number - durations, token counts, money, percentages, timestamps, ids, byte sizes - carries the tabular utility (font-variant-numeric: tabular-nums, layout.css). Numeric table columns are right-aligned.",
|
||||||
|
"section": "typography"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "The One Heading Rule",
|
||||||
|
"body": "A page has one headline (the header) and any number of uppercase 12px section labels; there is no intermediate heading size. Section labels are h2 elements, not decorative kickers.",
|
||||||
|
"section": "typography"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "The Flat Ledger Rule",
|
||||||
|
"body": "Nothing inside the main column casts a shadow. Overlays (dialog, sheet, menu) and the login card are the only lifted surfaces.",
|
||||||
|
"section": "elevation"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "The Host Accent Rule",
|
||||||
|
"body": "Inside Obsidian signal and primary are one variable. Live dots, running rows, the running count, the selected row, the primary button and the focus ring all take the user's accent; the Pink Ration Rule becomes an accent ration with the same two meanings, live and selected.",
|
||||||
|
"section": "obsidian"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "The Borrowed Corner Rule",
|
||||||
|
"body": "No control chooses its own radius. Rows and notes use --radius; luma primitives are re-rounded to Obsidian's --button-radius, --input-radius and --radius-m so the panel sits beside the app's own settings and menus without a seam.",
|
||||||
|
"section": "obsidian"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "The Alias Only Rule",
|
||||||
|
"body": "Panel components use token aliases (bg-primary/5, text-link, border-b) and the panel.css utilities. A literal colour, font or radius in ui/src/lib/panel/ breaks the Obsidian mapping.",
|
||||||
|
"section": "obsidian"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"dos": [
|
||||||
|
"Do build lists as ledger-grid rows with border-b and a per-row column template; use a native table when there are more than four numeric columns.",
|
||||||
|
"Do put tabular on every number and right-align numeric columns.",
|
||||||
|
"Do start each section with the 12px uppercase muted h2 and stack sections with gap-8.",
|
||||||
|
"Do use signal only for live state (pulsing dots, running rows, running counts) and selection.",
|
||||||
|
"Do color kinds and statuses through kind-badge.svelte and status-pill.svelte; extend their maps rather than inventing new hue pairs.",
|
||||||
|
"Do give hover to interactive rows only, via row-hover; keep non-interactive rows still.",
|
||||||
|
"Do put secondary surfaces (rails, trees) on bg-sidebar/50 with a hairline, not on a card.",
|
||||||
|
"Do show every error as error-note.svelte with a retry and every empty list as empty-state.svelte with a one-line hint.",
|
||||||
|
"Do respect prefers-reduced-motion; new motion must be a CSS transition or one of animate-pulse-dot and animate-rise.",
|
||||||
|
"Do keep the luma primitives as shipped; compose them with size=\"sm\" and variant=\"outline\" or \"ghost\" for in-page actions.",
|
||||||
|
"Do write panel components (ui/src/lib/panel/) against token aliases and panel.css utilities only, and check them in both the admin and the plugin.",
|
||||||
|
"Do decide panel layout from the panel's own width (@container, bind:clientWidth); 768px is where the rail replaces the switcher.",
|
||||||
|
"Do add conversation actions to actions.svelte.ts so the header menu and the row menu stay one list."
|
||||||
|
],
|
||||||
|
"donts": [
|
||||||
|
"Don't put a card inside a card, or wrap ledger rows in a bordered container.",
|
||||||
|
"Don't add drop shadows to in-page surfaces; only overlays and the login card are lifted.",
|
||||||
|
"Don't use signal pink for headings, buttons, or static decoration; primary mulberry is the button color.",
|
||||||
|
"Don't introduce a heading size between the page headline (1rem) and the section label (0.75rem uppercase).",
|
||||||
|
"Don't render solid colored chips; chips are hue/12 behind hue text.",
|
||||||
|
"Don't use hue-chip, hairline, touch-shown or scrollbar-none from layout.css as if they were system rules - they are defined and currently unused by the admin.",
|
||||||
|
"Don't apply this palette inside Obsidian; the panel takes Obsidian's variables (PRODUCT.md brand commitment).",
|
||||||
|
"Don't expect pills inside Obsidian; every radius there is the host's (--button-radius, --input-radius, --radius-m).",
|
||||||
|
"Don't mark an active tab with bg-background in panel code; the plugin overrides it to ink at 14%.",
|
||||||
|
"Don't let a menu or dialog render inside the sidedock; overlays mount in .beaver-root.beaver-portal at --layer-menu."
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
---
|
||||||
|
version: 1
|
||||||
|
slug: "beaver-gateway-ui-src-lib-panel"
|
||||||
|
primary_target: "beaver-gateway/ui/src/lib/panel"
|
||||||
|
related_targets: ["beaver-gateway/ui/src/routes/panel","beaver-plugin-obsidian/src/ui"]
|
||||||
|
---
|
||||||
|
|
||||||
|
Scope: the Beaver panel as an Obsidian view (sidedock column, tab, mobile) and its browser twin at /admin/panel. Mode: Operate.
|
||||||
|
Audience and job: the operator (Kit) watching and steering the agent next to their notes; the task is to see what runs, answer, branch, send a branch back to Telegram, close a deep chat.
|
||||||
|
Constraints: shared components in ui/src/lib/panel, bundled into the plugin by esbuild ($lib alias, one Svelte runtime, svelte-sonner shimmed to Notice); container queries, never viewport breakpoints, because the same tree lives in a 300px column and a 1400px tab; Obsidian's variables only through .beaver-root (beaver-calendar is the reference); nothing pink inside Obsidian.
|
||||||
|
Direction: the agent's terminal, thread always on screen; below 48rem the switcher popover in the header, above it the grouped rail (master, branches, deep chats, jobs) on the left; one action list (actions.svelte.ts) behind the header menu and the row context menu; markdown through Obsidian's renderer with [[links]] opening notes.
|
||||||
|
Memorable moment: the panel follows the note you open and the running turn's tool tree grows in place.
|
||||||
|
Unresolved: a "new conversation" affordance in the panel; keyboard navigation beyond ↑/↓ and /; the pinned one-line summary when a branch returns to Telegram (§8.2) is not produced by the gateway yet.
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
{
|
||||||
|
"mcpServers": {
|
||||||
|
"svelte": {
|
||||||
|
"type": "http",
|
||||||
|
"url": "https://mcp.svelte.dev/mcp"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
+21
-7
@@ -1,18 +1,32 @@
|
|||||||
repos:
|
repos:
|
||||||
- repo: https://github.com/astral-sh/ruff-pre-commit
|
- repo: https://github.com/astral-sh/ruff-pre-commit
|
||||||
rev: v0.15.13
|
rev: v0.15.20
|
||||||
hooks:
|
hooks:
|
||||||
- id: ruff-check
|
- id: ruff-check
|
||||||
types_or: [python, pyi]
|
types_or: [ python, pyi ]
|
||||||
args: [--fix]
|
args: [ --fix ]
|
||||||
- id: ruff-format
|
- id: ruff-format
|
||||||
types_or: [python, pyi]
|
types_or: [ python, pyi ]
|
||||||
|
|
||||||
- repo: local
|
- repo: local
|
||||||
hooks:
|
hooks:
|
||||||
- id: ty
|
- id: ty
|
||||||
name: ty check
|
name: ty check
|
||||||
entry: uvx ty check
|
entry: uv run ty check
|
||||||
language: python
|
language: system
|
||||||
types_or: [python, pyi]
|
types_or: [ python, pyi ]
|
||||||
|
pass_filenames: false
|
||||||
|
|
||||||
|
- id: ultracite
|
||||||
|
name: ultracite
|
||||||
|
entry: bash -c 'cd ui && bun x ultracite fix'
|
||||||
|
language: system
|
||||||
|
files: ^ui/
|
||||||
|
pass_filenames: false
|
||||||
|
|
||||||
|
- id: svelte-check
|
||||||
|
name: svelte check
|
||||||
|
entry: bash -c 'cd ui && bun check'
|
||||||
|
language: system
|
||||||
|
files: ^ui/
|
||||||
pass_filenames: false
|
pass_filenames: false
|
||||||
+22
-17
@@ -30,35 +30,40 @@ COPY . /app
|
|||||||
RUN --mount=type=cache,target=/root/.cache/uv \
|
RUN --mount=type=cache,target=/root/.cache/uv \
|
||||||
uv sync --frozen --no-dev --extra prod
|
uv sync --frozen --no-dev --extra prod
|
||||||
|
|
||||||
|
# ---- UI: SvelteKit SPA built with bun ----
|
||||||
|
FROM oven/bun:1-slim AS ui
|
||||||
|
WORKDIR /ui
|
||||||
|
COPY ui/package.json ui/bun.lock ./
|
||||||
|
RUN bun install --frozen-lockfile
|
||||||
|
COPY ui ./
|
||||||
|
RUN bun --bun svelte-kit sync && bun --bun run build
|
||||||
|
|
||||||
# ---- Runtime ----
|
# ---- Runtime ----
|
||||||
FROM python:3.13-slim AS runtime
|
FROM python:3.13-slim AS runtime
|
||||||
RUN apt-get update \
|
RUN apt-get update \
|
||||||
&& apt-get install -y --no-install-recommends ca-certificates git \
|
&& apt-get install -y --no-install-recommends ca-certificates git \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
# Bun native binary (glibc) — fast `npm install` replacement, only used
|
# Bun native binary (glibc): `bunx` runs the stdio MCP servers declared in
|
||||||
# to drop the claude CLI into the image. Not invoked at runtime.
|
# the user's config. The claude CLI itself ships inside the
|
||||||
|
# claude-agent-sdk wheel (`_bundled/claude`), nothing to install here.
|
||||||
COPY --from=oven/bun:1-slim /usr/local/bin/bun /usr/local/bin/bun
|
COPY --from=oven/bun:1-slim /usr/local/bin/bun /usr/local/bin/bun
|
||||||
RUN ln -s /usr/local/bin/bun /usr/local/bin/bunx
|
RUN ln -s /usr/local/bin/bun /usr/local/bin/bunx
|
||||||
|
|
||||||
# `--trust` is required: without it bun skips the postinstall step that
|
# The model process runs as `beaver-runner`, not as the gateway: the
|
||||||
# fetches claude's native binary (anthropics/claude-code#50203). The
|
# adapter spawns claude under this uid with a whitelisted environment.
|
||||||
# postinstall itself is bun's smoke check — if it fails the layer
|
# `acl` lets the entrypoint grant it write access to the vault sub-mounts
|
||||||
# fails. We deliberately DO NOT run `claude --version` here: claude
|
# without chowning files that Obsidian Sync keeps rewriting as root.
|
||||||
# touches `$HOME` on every invocation (creates `/root/.claude/`,
|
RUN apt-get update \
|
||||||
# `/root/.claude.json`, sometimes `/root/.config/claude/`), and those
|
&& apt-get install -y --no-install-recommends acl \
|
||||||
# build-time artifacts seed the runtime named-volume `claude-home`
|
&& rm -rf /var/lib/apt/lists/* \
|
||||||
# with stale "haven't onboarded" state, so the user gets re-prompted
|
&& useradd --system --uid 1001 --create-home --shell /usr/sbin/nologin beaver-runner
|
||||||
# for trust/bypass dialogs on every rebuild and the subscription auth
|
|
||||||
# can land on a tainted credential file.
|
ENV PATH=/app/.venv/bin:$PATH
|
||||||
ENV BUN_INSTALL=/usr/local/bun-global \
|
|
||||||
PATH=/usr/local/bun-global/bin:/app/.venv/bin:$PATH
|
|
||||||
RUN bun install -g --trust @anthropic-ai/claude-code \
|
|
||||||
&& test -x "$(command -v claude)" \
|
|
||||||
&& rm -rf /root/.claude /root/.claude.json /root/.config/claude
|
|
||||||
|
|
||||||
COPY --from=builder /app/.venv /app/.venv
|
COPY --from=builder /app/.venv /app/.venv
|
||||||
COPY --from=builder /app /app
|
COPY --from=builder /app /app
|
||||||
|
COPY --from=ui /ui/build /app/ui/build
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
|
|
||||||
ENTRYPOINT ["python", "-m"]
|
ENTRYPOINT ["python", "-m"]
|
||||||
|
|||||||
@@ -0,0 +1,31 @@
|
|||||||
|
.PHONY: sync check fix run ui ui-build db db-down
|
||||||
|
|
||||||
|
sync:
|
||||||
|
uv sync --extra local
|
||||||
|
cd ui && bun install
|
||||||
|
|
||||||
|
check:
|
||||||
|
uv run ruff format --check
|
||||||
|
uv run ruff check
|
||||||
|
uv run ty check
|
||||||
|
cd ui && bun lint && bun check
|
||||||
|
|
||||||
|
fix:
|
||||||
|
uv run ruff format
|
||||||
|
uv run ruff check --fix
|
||||||
|
cd ui && bun fix
|
||||||
|
|
||||||
|
run:
|
||||||
|
uv run python -m beaver_gateway
|
||||||
|
|
||||||
|
ui:
|
||||||
|
cd ui && bun run dev
|
||||||
|
|
||||||
|
ui-build:
|
||||||
|
cd ui && bun run build
|
||||||
|
|
||||||
|
db:
|
||||||
|
docker compose -f ../beaver-agent/docker-compose.yml --project-directory ../beaver-agent --profile db up -d postgres
|
||||||
|
|
||||||
|
db-down:
|
||||||
|
docker compose -f ../beaver-agent/docker-compose.yml --project-directory ../beaver-agent --profile db down
|
||||||
+31
-51
@@ -12,7 +12,7 @@ from beaver_gateway.agents.base import ExposedMcp
|
|||||||
from beaver_gateway.agents.claude import ClaudeAgent
|
from beaver_gateway.agents.claude import ClaudeAgent
|
||||||
from beaver_gateway.agents.raycast import RaycastAgent, RemoteTool, UserPreferences
|
from beaver_gateway.agents.raycast import RaycastAgent, RemoteTool, UserPreferences
|
||||||
from beaver_gateway.core.registry import Gateway
|
from beaver_gateway.core.registry import Gateway
|
||||||
from beaver_gateway.core.turn_record import TurnRecord, slugify
|
from beaver_gateway.core.turn_record import slugify
|
||||||
from beaver_gateway.frontends.admin import AdminFrontend
|
from beaver_gateway.frontends.admin import AdminFrontend
|
||||||
from beaver_gateway.frontends.anthropic import AnthropicMessagesFrontend
|
from beaver_gateway.frontends.anthropic import AnthropicMessagesFrontend
|
||||||
from beaver_gateway.frontends.markdown import MarkdownFrontend
|
from beaver_gateway.frontends.markdown import MarkdownFrontend
|
||||||
@@ -20,20 +20,19 @@ from beaver_gateway.frontends.mcp_server import McpServerFrontend
|
|||||||
from beaver_gateway.mcp.types import McpServer
|
from beaver_gateway.mcp.types import McpServer
|
||||||
|
|
||||||
|
|
||||||
def chat_log_path(record: TurnRecord, vault: Path) -> Path:
|
def chat_path(title: str, agent: str, vault: Path) -> Path: # noqa: ARG001
|
||||||
"""Decide where a logged chat from another frontend lands in the vault.
|
"""Where a new chat file lands in the vault.
|
||||||
|
|
||||||
Called by ``MarkdownFrontend`` (with ``log_all_chats=True``) the first
|
Called by ``MarkdownFrontend`` for every conversation that needs a
|
||||||
time a conversation needs a file — continuation turns are matched by
|
file: a ``deep`` chat spawned by the dispatcher, a ``/v1/messages``
|
||||||
fingerprint and stick to the file picked here. Return value can be
|
chat (``title`` = first user message), or with ``log_all_chats=True``
|
||||||
|
the archive of a stateless agent's turns. Return value can be
|
||||||
absolute or relative; relative paths are anchored under ``vault``.
|
absolute or relative; relative paths are anchored under ``vault``.
|
||||||
|
|
||||||
Layout below: ``<vault>/<YYYY-MM>/<YYYY-MM-DD>_<topic>.md`` where
|
Layout below: ``<vault>/<YYYY-MM>/<YYYY-MM-DD>_<topic>.md``.
|
||||||
``topic`` is a slug of the very first user message in the chat.
|
|
||||||
"""
|
"""
|
||||||
today = date.today()
|
today = date.today()
|
||||||
topic = slugify(record.first_user_text, maxlen=40)
|
return vault / f"{today:%Y-%m}" / f"{today:%Y-%m-%d}_{slugify(title, maxlen=40)}.md"
|
||||||
return vault / f"{today:%Y-%m}" / f"{today:%Y-%m-%d}_{topic}.md"
|
|
||||||
|
|
||||||
|
|
||||||
def current_time() -> str:
|
def current_time() -> str:
|
||||||
@@ -145,41 +144,27 @@ gateway = Gateway(
|
|||||||
# Anthropic-compatible Messages endpoint. Auth comes from
|
# Anthropic-compatible Messages endpoint. Auth comes from
|
||||||
# `BOOTSTRAP_TOKENS` in the env (`name1:value1,name2:value2`).
|
# `BOOTSTRAP_TOKENS` in the env (`name1:value1,name2:value2`).
|
||||||
#
|
#
|
||||||
# Behind a reverse proxy (Caddy / nginx / Cloudflare) pass
|
# Every HTTP frontend is mounted under its own path on the one
|
||||||
# `public_base_url=` so the admin dashboard advertises the
|
# gateway port (`Gateway.port`, 8000 here): `/anthropic/v1/messages`.
|
||||||
# outside URL instead of `host:port`. Caddy strips its own
|
# Behind a reverse proxy set `Gateway(public_url="https://domain.com")`
|
||||||
# prefix and the frontend's internal paths (`/v1/messages`,
|
# so advertised endpoints use the outside origin; the proxy just
|
||||||
# `/v1/models`) get appended:
|
# forwards everything to the gateway, no prefix stripping.
|
||||||
# Caddy: handle_path /ai/* { reverse_proxy localhost:8000 }
|
AnthropicMessagesFrontend(),
|
||||||
# Config: AnthropicMessagesFrontend(
|
|
||||||
# port=8000,
|
|
||||||
# public_base_url="https://domain.com/ai")
|
|
||||||
# Result: https://domain.com/ai/v1/messages
|
|
||||||
AnthropicMessagesFrontend(host="0.0.0.0", port=8000),
|
|
||||||
# Phase 3 — re-exposes every declared `McpServer` outside the
|
# Phase 3 — re-exposes every declared `McpServer` outside the
|
||||||
# gateway with bearer auth + audit log. Each namespace lives
|
# gateway with bearer auth + audit log. Each namespace lives at
|
||||||
# at `/<name>/` on this port (the port itself disambiguates
|
# `/mcp/<name>/`; a flat bundle is published at `/mcp/all/`.
|
||||||
# MCP traffic — no extra `/mcp` segment in the route); a flat
|
# Discovery page (HTML, auth-gated) at `/mcp/` with copy-pastable
|
||||||
# bundle is published at `/all/`. Discovery page (HTML,
|
# Cursor / Claude Desktop snippets. Auth re-uses `BOOTSTRAP_TOKENS`.
|
||||||
# auth-gated) at `/` with copy-pastable Cursor / Claude
|
McpServerFrontend(),
|
||||||
# Desktop snippets. Auth re-uses `BOOTSTRAP_TOKENS`.
|
|
||||||
#
|
|
||||||
# Same `public_base_url=` knob as above. Caddy strips its
|
|
||||||
# prefix; the frontend's `/<name>/` segment gets appended:
|
|
||||||
# Caddy: handle_path /mcp/* { reverse_proxy localhost:8001 }
|
|
||||||
# Config: McpServerFrontend(
|
|
||||||
# port=8001,
|
|
||||||
# public_base_url="https://domain.com/mcp")
|
|
||||||
# Result: https://domain.com/mcp/<name>/ (and /mcp/all/)
|
|
||||||
McpServerFrontend(host="0.0.0.0", port=8001),
|
|
||||||
# Phase 4.3 — browser admin UI. Creds come from
|
# Phase 4.3 — browser admin UI. Creds come from
|
||||||
# `ADMIN_USER`/`ADMIN_PASS`; the session cookie is signed with
|
# `ADMIN_USER`/`ADMIN_PASS`; the session cookie is signed with
|
||||||
# `SESSION_SECRET`. Use it to mint tokens (Argon2-hashed in
|
# `SESSION_SECRET`. Use it to mint tokens (Argon2-hashed in
|
||||||
# the DB), revoke them, and watch the audit log. Scope is
|
# the DB), revoke them, and watch the audit log. Scope is
|
||||||
# enforced on the bearer frontends: tokens minted with scope
|
# enforced on the bearer frontends: tokens minted with scope
|
||||||
# `messages` only work on `/v1/messages`; `mcp` only on
|
# `messages` only work on `/v1/messages`; `mcp` only on
|
||||||
# `/mcp/<name>`; `*` works everywhere.
|
# `/mcp/<name>`; `*` works everywhere. Served at `/admin/`; `/`
|
||||||
AdminFrontend(host="0.0.0.0", port=8002),
|
# redirects there.
|
||||||
|
AdminFrontend(),
|
||||||
# Obsidian-vault chat frontend. Each `.md` is one conversation
|
# Obsidian-vault chat frontend. Each `.md` is one conversation
|
||||||
# (User/Assistant turn pairs). The Obsidian companion plugin
|
# (User/Assistant turn pairs). The Obsidian companion plugin
|
||||||
# POSTs `{filename, content?}` to `/chat` — the frontend reads
|
# POSTs `{filename, content?}` to `/chat` — the frontend reads
|
||||||
@@ -192,9 +177,8 @@ gateway = Gateway(
|
|||||||
# example boots cleanly; in real deployments mount the
|
# example boots cleanly; in real deployments mount the
|
||||||
# Obsidian-sync container's vault volume to a stable path and
|
# Obsidian-sync container's vault volume to a stable path and
|
||||||
# pass that instead.
|
# pass that instead.
|
||||||
|
# Mounted at `/md` (`/md/chat`, `/md/chat/stream`).
|
||||||
MarkdownFrontend(
|
MarkdownFrontend(
|
||||||
host="0.0.0.0",
|
|
||||||
port=8003,
|
|
||||||
# Point at the dedicated chats subdir of your real Obsidian
|
# Point at the dedicated chats subdir of your real Obsidian
|
||||||
# vault — the gateway has no idea (and no need) about other
|
# vault — the gateway has no idea (and no need) about other
|
||||||
# notes outside it. Path resolution / vault-escape checks
|
# notes outside it. Path resolution / vault-escape checks
|
||||||
@@ -208,17 +192,13 @@ gateway = Gateway(
|
|||||||
vault_path=Path(tempfile.mkdtemp(prefix="beaver-vault-")).resolve(),
|
vault_path=Path(tempfile.mkdtemp(prefix="beaver-vault-")).resolve(),
|
||||||
default_agent="research",
|
default_agent="research",
|
||||||
log_all_chats=True,
|
log_all_chats=True,
|
||||||
# ``log_path`` (optional) overrides the default
|
# ``chat_path`` (optional) overrides the default
|
||||||
# ``{vault}/_logs/<agent>/<date>_<hex>.md`` layout for chats
|
# ``{vault}/_logs/<agent>/<date>_<slug>.md`` layout. Heads up:
|
||||||
# logged from OTHER frontends (Anthropic Messages, admin
|
# any custom path forces ``warm_index`` to scan the entire
|
||||||
# in-browser chat). Defined as a top-of-file function so the
|
# vault on startup so the fingerprint→file map of archived
|
||||||
# types are explicit and the IDE can hover them; a lambda
|
# stateless chats survives a restart no matter where you put
|
||||||
# works too, but a real ``def`` keeps the signature visible
|
# files.
|
||||||
# and lets you docstring it. Heads up: any custom path
|
chat_path=chat_path,
|
||||||
# forces ``warm_index`` to scan the entire vault on startup
|
|
||||||
# so the fingerprint→file map survives a restart no matter
|
|
||||||
# where you put files.
|
|
||||||
log_path=chat_log_path,
|
|
||||||
),
|
),
|
||||||
],
|
],
|
||||||
)
|
)
|
||||||
@@ -24,11 +24,8 @@ services:
|
|||||||
# config.py declares one, so set these (or remove the agent)
|
# config.py declares one, so set these (or remove the agent)
|
||||||
# before exposing port 8000.
|
# before exposing port 8000.
|
||||||
ports:
|
ports:
|
||||||
# /v1/messages frontend
|
# The one gateway port: /anthropic, /mcp, /admin, /md under it
|
||||||
|
# (change ADMIN_USER/ADMIN_PASS/SESSION_SECRET before exposing).
|
||||||
- "8000:8000"
|
- "8000:8000"
|
||||||
# MCP server frontend
|
|
||||||
- "8001:8001"
|
|
||||||
# Admin UI (Phase 4.3) — change ADMIN_USER/ADMIN_PASS/SESSION_SECRET
|
|
||||||
- "8002:8002"
|
|
||||||
volumes:
|
volumes:
|
||||||
- ./config.py:/config/config.py:ro
|
- ./config.py:/config/config.py:ro
|
||||||
+10
-7
@@ -8,16 +8,20 @@ authors = [
|
|||||||
requires-python = ">=3.13"
|
requires-python = ">=3.13"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"aiofile>=3.11.1",
|
"aiofile>=3.11.1",
|
||||||
|
"aiogram>=3.31.0",
|
||||||
"aiohttp>=3.13.5",
|
"aiohttp>=3.13.5",
|
||||||
"aiosqlite>=0.22.1",
|
"aiosqlite>=0.22.1",
|
||||||
"anthropic>=0.103.0",
|
"anthropic>=0.103.0",
|
||||||
"anyio>=4.13.0",
|
"anyio>=4.13.0",
|
||||||
"argon2-cffi>=25.1.0",
|
"argon2-cffi>=25.1.0",
|
||||||
|
"claude-agent-sdk>=0.2.150",
|
||||||
|
"croniter>=6.2.4",
|
||||||
"fastapi>=0.136.1",
|
"fastapi>=0.136.1",
|
||||||
"fastmcp>=3.3.1",
|
"fastmcp>=3.3.1",
|
||||||
"greenlet>=3.5.0",
|
"greenlet>=3.5.0",
|
||||||
"itsdangerous>=2.2.0",
|
"itsdangerous>=2.2.0",
|
||||||
"jinja2>=3.1.6",
|
"pgqueuer>=1.3.2",
|
||||||
|
"psutil>=7.2.2",
|
||||||
"psycopg[binary]>=3.3.4",
|
"psycopg[binary]>=3.3.4",
|
||||||
"pydantic>=2.13.4",
|
"pydantic>=2.13.4",
|
||||||
"pydantic-settings>=2.14.1",
|
"pydantic-settings>=2.14.1",
|
||||||
@@ -25,16 +29,15 @@ dependencies = [
|
|||||||
"sqlmodel>=0.0.38",
|
"sqlmodel>=0.0.38",
|
||||||
"uvicorn[standard]>=0.47.0",
|
"uvicorn[standard]>=0.47.0",
|
||||||
"uvloop>=0.22.1",
|
"uvloop>=0.22.1",
|
||||||
|
"watchfiles>=1.2.0",
|
||||||
]
|
]
|
||||||
|
|
||||||
[project.optional-dependencies]
|
[project.optional-dependencies]
|
||||||
local = [
|
local = [
|
||||||
"raycast-api",
|
"raycast-api",
|
||||||
"claude-code-api",
|
|
||||||
]
|
]
|
||||||
prod = [
|
prod = [
|
||||||
"raycast-api",
|
"raycast-api",
|
||||||
"claude-code-api",
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[tool.uv]
|
[tool.uv]
|
||||||
@@ -47,10 +50,6 @@ raycast-api = [
|
|||||||
{ path = "../raycast-api", editable = true, extra = "local" },
|
{ path = "../raycast-api", editable = true, extra = "local" },
|
||||||
{ git = "https://git.kotikot.com/beaver/raycast-api.git", extra = "prod" },
|
{ git = "https://git.kotikot.com/beaver/raycast-api.git", extra = "prod" },
|
||||||
]
|
]
|
||||||
claude-code-api = [
|
|
||||||
{ path = "../claude-code-api", editable = true, extra = "local" },
|
|
||||||
{ git = "https://git.kotikot.com/beaver/claude-code-api.git", extra = "prod" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[project.scripts]
|
[project.scripts]
|
||||||
beaver-gateway = "beaver_gateway:main"
|
beaver-gateway = "beaver_gateway:main"
|
||||||
@@ -67,3 +66,7 @@ dev = [
|
|||||||
"ruff>=0.15.13",
|
"ruff>=0.15.13",
|
||||||
"ty>=0.0.37",
|
"ty>=0.0.37",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[tool.pytest.ini_options]
|
||||||
|
asyncio_mode = "auto"
|
||||||
|
testpaths = ["tests"]
|
||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
``BaseAgent`` is the structural contract every backend-specific agent
|
``BaseAgent`` is the structural contract every backend-specific agent
|
||||||
honours. Subclasses add **capabilities as fields** (``ClaudeAgent.cwd``,
|
honours. Subclasses add **capabilities as fields** (``ClaudeAgent.cwd``,
|
||||||
``RaycastAgent.streaming``) — backends dispatch on type, not on a runtime
|
``RaycastAgent.streaming``) - backends dispatch on type, not on a runtime
|
||||||
matrix.
|
matrix.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
@@ -15,10 +15,16 @@ from pydantic import BaseModel, ConfigDict
|
|||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
@dataclass(frozen=True, slots=True)
|
||||||
class ExposedMcp:
|
class ExposedMcp:
|
||||||
"""Reference to an ``McpServer`` (by name) exposed to a single agent."""
|
"""Reference to an ``McpServer`` (by name) exposed to a single agent.
|
||||||
|
|
||||||
|
``tools`` is an allowlist of tool names (``None`` = every tool);
|
||||||
|
``deny`` is a tuple of ``fnmatch`` patterns removed on top of that,
|
||||||
|
e.g. ``("delete_*",)``.
|
||||||
|
"""
|
||||||
|
|
||||||
name: str
|
name: str
|
||||||
tools: tuple[str, ...] | None = None
|
tools: tuple[str, ...] | None = None
|
||||||
|
deny: tuple[str, ...] = ()
|
||||||
|
|
||||||
|
|
||||||
class BaseAgent(BaseModel):
|
class BaseAgent(BaseModel):
|
||||||
|
|||||||
@@ -1,258 +1,110 @@
|
|||||||
"""Claude Code agent definition.
|
"""Claude agent definition, backed by the Claude Agent SDK.
|
||||||
|
|
||||||
There is no ``streaming`` field, but there is
|
The system prompt is ``system_prompt`` verbatim or, per conversation kind,
|
||||||
:attr:`ClaudeCodeOptions.include_partial_messages`, and the difference
|
the granules named in ``prompts`` assembled at every session spawn (see
|
||||||
matters. Token-level deltas are not a property of the agent, they are a
|
``core/prompt.py``). ``skill_sets`` are directories of ``<skill>/SKILL.md``
|
||||||
property of the transport: the PTY transport reads a JSONL that only
|
folders; each becomes a local SDK plugin. Nothing from disk is loaded
|
||||||
ever contains finished blocks, so it cannot stream no matter what the
|
otherwise: the adapter runs with ``setting_sources=[]``.
|
||||||
agent asks for, while ``transport="stream_json"`` can. The flag lives
|
|
||||||
next to the transport that grants it rather than pretending to be a
|
|
||||||
free-standing capability.
|
|
||||||
|
|
||||||
``BaseAgent.system_prompt`` maps onto the claude CLI's
|
|
||||||
``--system-prompt`` — i.e. it really *is* the agent's system prompt,
|
|
||||||
not "added on top of claude-code's giant built-in". The additive slot
|
|
||||||
``--append-system-prompt`` is exposed via
|
|
||||||
:attr:`ClaudeCodeOptions.append_system_prompt` for the rare case
|
|
||||||
when the user wants claude-code's planning conventions / dynamic
|
|
||||||
sections *and* a delta on top. Difference, measured empirically:
|
|
||||||
|
|
||||||
* tools (names, JSON schemas, embedded guidance like Bash's "prefer
|
|
||||||
Read over cat/head/tail") survive both flags — they ride the
|
|
||||||
Anthropic API ``tools=[]`` field, not the system prompt text;
|
|
||||||
* ``--system-prompt`` drops ~8.6k tokens of claude-code's *textual*
|
|
||||||
baseline — agent persona, multi-step-work conventions ("use
|
|
||||||
TaskCreate proactively", etc.), and the dynamic per-machine sections
|
|
||||||
(cwd, env info, git status, memory paths — see the
|
|
||||||
``--exclude-dynamic-system-prompt-sections`` flag note in
|
|
||||||
``claude --help``: "Only applies with the default system prompt
|
|
||||||
(ignored with --system-prompt)").
|
|
||||||
|
|
||||||
We pick override as the default because it preserves the principle of
|
|
||||||
least surprise: the ``system_prompt=...`` you wrote on the agent is
|
|
||||||
what claude actually receives. If you need claude-code's full
|
|
||||||
planning/dynamic-context behaviour on top of your prompt, opt in via
|
|
||||||
``options=ClaudeCodeOptions(append_system_prompt=...)`` and leave
|
|
||||||
``system_prompt`` for your agent's identity (or vice versa — set
|
|
||||||
``system_prompt=""``-ish and put everything in ``append_*``).
|
|
||||||
|
|
||||||
Per-agent passthrough of ``claude_code_api.BackendOptions`` lives in
|
|
||||||
:class:`ClaudeCodeOptions`, attached to :class:`ClaudeAgent` as
|
|
||||||
``options``. Every tunable knob ``BackendOptions`` supports — except
|
|
||||||
the ones derived from the agent's own primary surface
|
|
||||||
(``cwd`` / ``model`` / ``system_prompt`` / ``available_native_tools`` →
|
|
||||||
``allowed_tools`` / ``expose_mcps`` → ``mcp_servers``) — is exposed
|
|
||||||
there. Defaults match upstream except where correctness demands
|
|
||||||
otherwise (``wait_for_turn_duration=True``,
|
|
||||||
``dangerously_skip_permissions=True``).
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from collections.abc import Mapping # noqa: TC003 — pydantic runtime
|
from collections.abc import Mapping # noqa: TC003 - pydantic runtime
|
||||||
from pathlib import Path # noqa: TC003 — pydantic runtime
|
from pathlib import Path # noqa: TC003 - pydantic runtime
|
||||||
from typing import Literal
|
|
||||||
|
|
||||||
from pydantic import BaseModel, ConfigDict, Field
|
from pydantic import BaseModel, ConfigDict, Field, model_validator
|
||||||
|
|
||||||
from beaver_gateway.agents.base import BaseAgent
|
from beaver_gateway.agents.base import BaseAgent
|
||||||
|
from beaver_gateway.core.kinds import KINDS, Kind
|
||||||
|
from beaver_gateway.core.policy import PolicyRule # noqa: TC001 - pydantic runtime
|
||||||
|
from beaver_gateway.core.prompt import PromptSource # noqa: TC001 - pydantic runtime
|
||||||
|
|
||||||
HistoryInjectionMode = Literal["native_jsonl", "concat_message"]
|
__all__ = ["ClaudeAgent", "ClaudeOptions", "Prompts"]
|
||||||
"""Mirrors ``claude_code_api.HistoryInjectionMode`` to avoid an import
|
|
||||||
cycle on the user-facing path (configs may be loaded without
|
|
||||||
``claude-code-api`` installed, e.g. ``--extra prod`` minus claude)."""
|
|
||||||
|
|
||||||
Transport = Literal["pty", "stream_json"]
|
|
||||||
"""Mirrors ``claude_code_api.Transport``, for the same reason."""
|
|
||||||
|
|
||||||
|
|
||||||
class ClaudeCodeOptions(BaseModel):
|
class ClaudeOptions(BaseModel):
|
||||||
"""Per-agent passthrough for ``claude_code_api.BackendOptions``.
|
|
||||||
|
|
||||||
Mirrors every field of ``BackendOptions`` except those that come
|
|
||||||
from the agent's primary surface
|
|
||||||
(``cwd`` / ``model`` / ``system_prompt`` /
|
|
||||||
``available_native_tools`` / ``expose_mcps``). Defaults match
|
|
||||||
upstream, with two exceptions for correctness:
|
|
||||||
|
|
||||||
* ``wait_for_turn_duration=True`` — without it, extended-thinking
|
|
||||||
turns drop the text response because ``TurnManager`` returns on
|
|
||||||
the first terminal assistant record (the thinking snapshot) and
|
|
||||||
never reads the second one (the actual text). Phase 2.2 PROGRESS
|
|
||||||
describes the incident in detail.
|
|
||||||
* ``dangerously_skip_permissions=True`` — Beaver Gateway is a
|
|
||||||
single-user trusted-config product; the user already wrote the
|
|
||||||
``config.py`` that spawns claude. Permission prompts on a
|
|
||||||
headless backend mean stuck turns.
|
|
||||||
|
|
||||||
Any field on :class:`claude_code_api.BackendOptions` we forward
|
|
||||||
here lives on this model with the same name and type, so a future
|
|
||||||
``BackendOptions`` field addition is a one-liner here plus a
|
|
||||||
one-liner in :mod:`backends.claude_code`.
|
|
||||||
"""
|
|
||||||
|
|
||||||
model_config = ConfigDict(frozen=True, arbitrary_types_allowed=True)
|
model_config = ConfigDict(frozen=True, arbitrary_types_allowed=True)
|
||||||
|
|
||||||
transport: Transport = "pty"
|
effort: str | None = None
|
||||||
"""Which claude process backs this agent.
|
include_partial_messages: bool = True
|
||||||
|
"""Token-level deltas on the wire; off means one delta per finished block."""
|
||||||
``stream_json`` runs ``claude -p`` with stream-json on stdin and
|
|
||||||
stdout: prompts go into a pipe, events come back as they happen, and
|
|
||||||
a native ``result`` record closes each turn. ``pty`` (the default,
|
|
||||||
kept for backwards compatibility) drives the interactive TUI and
|
|
||||||
tails the session JSONL, which costs a multi-second readiness wait
|
|
||||||
per spawn — the thing ``startup_delay`` bounds — and brings the
|
|
||||||
swallowed-paste failure mode with it.
|
|
||||||
|
|
||||||
Switching also makes the PTY-only knobs below inert
|
|
||||||
(``startup_delay`` / ``file_wait_timeout`` /
|
|
||||||
``turn_duration_timeout`` / ``wait_for_turn_duration``) and drops
|
|
||||||
interactive-only ``extra_args`` such as ``--remote-control``, which
|
|
||||||
would take claude out of print mode. Everything else — model,
|
|
||||||
prompts, tools, MCPs, history seeding — behaves identically."""
|
|
||||||
|
|
||||||
warmup_turn: bool | None = None
|
|
||||||
"""Spend a throwaway turn when a session spawns, so MCP tools are
|
|
||||||
live on the agent's first reply.
|
|
||||||
|
|
||||||
``None`` (the default) decides per agent: on when it exposes any
|
|
||||||
MCP, off when it doesn't. Set it explicitly to override.
|
|
||||||
|
|
||||||
It exists because the CLI begins connecting MCP servers when a turn
|
|
||||||
starts and doesn't wait for them — measured on the pi, turn one saw
|
|
||||||
29 tools with both servers ``pending``, turn two saw 90 with both
|
|
||||||
``connected``. For an agent whose whole job is those tools, a first
|
|
||||||
reply without them is worse than a couple of extra seconds. Nothing
|
|
||||||
cheaper works: sleeping longer changes nothing, and neither a
|
|
||||||
``/status`` command nor the SDK's ``initialize`` handshake touches
|
|
||||||
the MCP client.
|
|
||||||
|
|
||||||
``stream_json`` only — the PTY transport's readiness wait covers
|
|
||||||
this incidentally."""
|
|
||||||
|
|
||||||
include_partial_messages: bool = False
|
|
||||||
"""Stream token-level deltas to the client instead of one delta per
|
|
||||||
finished block. Requires ``transport="stream_json"``; inert on the
|
|
||||||
PTY transport, whose JSONL contains only finished blocks. Off by
|
|
||||||
default because it multiplies the SSE event count — worth it for
|
|
||||||
interactive chat, pointless for a client that waits for the whole
|
|
||||||
message anyway."""
|
|
||||||
|
|
||||||
append_system_prompt: str | None = None
|
|
||||||
"""Maps to claude CLI's ``--append-system-prompt``. Opting in
|
|
||||||
re-attaches claude-code's full built-in prompt (persona, planning
|
|
||||||
conventions like "use TaskCreate proactively", and the dynamic
|
|
||||||
per-machine sections — cwd, env info, git status, memory paths)
|
|
||||||
plus this string on top. By default we ship only
|
|
||||||
:attr:`BaseAgent.system_prompt` via ``--system-prompt`` (~8.6k
|
|
||||||
tokens lighter); use this when the agent should behave like a
|
|
||||||
real claude-code coding session and your text is a delta on top
|
|
||||||
of those built-ins. Tool schemas survive either way — they ride
|
|
||||||
the Anthropic ``tools=[]`` channel, not the prompt text."""
|
|
||||||
|
|
||||||
disallowed_tools: tuple[str, ...] = ()
|
|
||||||
"""Tool names that claude must refuse to call. Combines with the
|
|
||||||
agent's ``available_native_tools`` allowlist — disallow wins."""
|
|
||||||
|
|
||||||
permission_mode: str = "bypassPermissions"
|
permission_mode: str = "bypassPermissions"
|
||||||
"""``claude --permission-mode`` value. Only meaningful when
|
tools: tuple[str, ...] | None = None
|
||||||
``dangerously_skip_permissions=False``; otherwise the CLI bypasses
|
"""Base set of built-in tools; ``None`` keeps the CLI default set."""
|
||||||
the prompt path entirely."""
|
|
||||||
|
|
||||||
dangerously_skip_permissions: bool = True
|
disallowed_tools: tuple[str, ...] = ()
|
||||||
"""Pass ``--dangerously-skip-permissions`` to claude. ``True`` by
|
add_dirs: tuple[str, ...] = ()
|
||||||
default — see the class docstring for why."""
|
env: Mapping[str, str] = Field(default_factory=dict)
|
||||||
|
"""Extra environment for the claude subprocess (always passed through)."""
|
||||||
|
|
||||||
effort: str | None = None
|
env_keep: tuple[str, ...] = ()
|
||||||
"""``claude --effort`` value (typically ``low`` / ``medium`` /
|
"""Extra inherited variable names to let through the env whitelist."""
|
||||||
``high``). Tunes reasoning effort budget for newer models."""
|
|
||||||
|
|
||||||
add_dir: tuple[str, ...] = ()
|
max_turns: int | None = None
|
||||||
"""Extra directories claude is allowed to read/edit beyond
|
session_store_flush: str = "eager"
|
||||||
``cwd``. Maps to repeated ``--add-dir`` flags."""
|
"""``eager`` mirrors every transcript frame as it lands, so a gateway
|
||||||
|
killed mid-turn loses at most the frame in flight."""
|
||||||
|
|
||||||
settings: str | None = None
|
|
||||||
"""Path to a ``--settings`` JSON file claude should load (hooks,
|
|
||||||
MCP servers, etc. that don't belong in our internal aggregator)."""
|
|
||||||
|
|
||||||
extra_args: tuple[str, ...] = ()
|
class Prompts(BaseModel):
|
||||||
"""Raw extra argv to append to the claude command. Escape hatch
|
"""Prompt assembly per conversation kind (§3.12): the granules, in order.
|
||||||
for flags we haven't surfaced as first-class options."""
|
|
||||||
|
|
||||||
extra_env: Mapping[str, str] = Field(default_factory=dict)
|
A kind left ``None`` is not served by the agent; ``ClaudeAgent.kinds``
|
||||||
"""Additional environment variables for the spawned claude
|
follows from the kinds set here.
|
||||||
process. Layered on top of the gateway's own env after
|
"""
|
||||||
``preserve_provider_env`` is applied."""
|
|
||||||
|
|
||||||
preserve_provider_env: bool = False
|
model_config = ConfigDict(frozen=True)
|
||||||
"""When ``False`` (default), the spawn env strips
|
|
||||||
``ANTHROPIC_API_KEY`` / ``ANTHROPIC_AUTH_TOKEN`` /
|
|
||||||
``ANTHROPIC_BASE_URL`` so claude uses subscription auth instead of
|
|
||||||
leaking through whatever the gateway process inherited."""
|
|
||||||
|
|
||||||
history_injection_mode: HistoryInjectionMode = "native_jsonl"
|
master: tuple[PromptSource, ...] | None = None
|
||||||
"""How prior turns are seeded into a fresh session when no live
|
branch: tuple[PromptSource, ...] | None = None
|
||||||
session matches: ``native_jsonl`` writes a hand-crafted transcript
|
deep: tuple[PromptSource, ...] | None = None
|
||||||
and ``--resume``s; ``concat_message`` instead folds history into
|
job: tuple[PromptSource, ...] | None = None
|
||||||
the first user prompt. ``native_jsonl`` is more faithful."""
|
fork: tuple[PromptSource, ...] | None = None
|
||||||
|
|
||||||
wait_for_turn_duration: bool = True
|
def for_kind(self, kind: Kind) -> tuple[PromptSource, ...] | None:
|
||||||
"""Keep reading JSONL until the ``turn_duration`` heartbeat
|
return getattr(self, kind)
|
||||||
arrives, instead of returning on the first terminal assistant
|
|
||||||
record. ``True`` by default — see class docstring."""
|
|
||||||
|
|
||||||
include_meta_user: bool = False
|
@property
|
||||||
"""Surface claude's ``isMeta=True`` user records (local-command
|
def kinds(self) -> tuple[Kind, ...]:
|
||||||
caveats) as ``UserMessage`` events. Off by default — they're not
|
return tuple(k for k in KINDS if self.for_kind(k) is not None)
|
||||||
part of the real conversation."""
|
|
||||||
|
|
||||||
startup_delay: float = 60.0
|
|
||||||
"""Upper bound (seconds) on waiting for claude's TUI to enable
|
|
||||||
bracketed-paste mode after spawn. claude-code-api polls PTY output
|
|
||||||
for the DECSET 2004 marker and returns as soon as it arrives, so a
|
|
||||||
generous cap costs nothing on a warm host — it only bounds the case
|
|
||||||
where the marker never shows.
|
|
||||||
|
|
||||||
It has to be generous, because timing out here means writing the
|
|
||||||
prompt into a terminal that hasn't enabled bracketed paste, where our
|
|
||||||
``ESC[200~`` framing is parsed as junk and the message is silently
|
|
||||||
dropped. Observed on a cold Raspberry Pi: 10s elapsed with *zero*
|
|
||||||
bytes of PTY output, the prompt went into the void, and the turn hung
|
|
||||||
until a human typed into the session by hand."""
|
|
||||||
|
|
||||||
file_wait_timeout: float = 30.0
|
|
||||||
"""How long to wait for the session JSONL to appear after spawn.
|
|
||||||
Failure here usually means a CLI auth / config problem."""
|
|
||||||
|
|
||||||
turn_duration_timeout: float = 5.0
|
|
||||||
"""How long to wait for the ``turn_duration`` heartbeat once a
|
|
||||||
terminal assistant has been seen. Bound on extra latency when
|
|
||||||
``wait_for_turn_duration=True``."""
|
|
||||||
|
|
||||||
idle_session_ttl: float = 1800.0
|
|
||||||
"""Seconds a pooled claude session may sit unused before it is
|
|
||||||
terminated. The backend pools live PTYs by conversation-history
|
|
||||||
fingerprint; when a conversation forks, the session behind the old
|
|
||||||
fingerprint becomes unreachable but stays resident — one live
|
|
||||||
``claude`` (hundreds of MB) each. 30 minutes keeps an active chat
|
|
||||||
warm across a coffee break while bounding what a fork storm can
|
|
||||||
strand. There is no count cap: concurrent conversations should all
|
|
||||||
stay warm. ``0`` disables reaping."""
|
|
||||||
|
|
||||||
|
|
||||||
class ClaudeAgent(BaseAgent):
|
class ClaudeAgent(BaseAgent):
|
||||||
"""Agent backed by ``claude-code-api``.
|
|
||||||
|
|
||||||
``cwd`` is the working directory the claude CLI is spawned in;
|
|
||||||
typed as ``Path`` (Pydantic coerces strings) since claude-code-api
|
|
||||||
accepts ``str | os.PathLike[str]``. ``available_native_tools`` stays
|
|
||||||
``tuple[str, ...]`` because Claude Code's tool set is a moving target
|
|
||||||
(Bash/Read/Edit/Write/WebSearch/etc. — versions add and rename), so
|
|
||||||
pinning it as a ``Literal`` would rot the type each release.
|
|
||||||
"""
|
|
||||||
|
|
||||||
cwd: Path
|
cwd: Path
|
||||||
available_native_tools: tuple[str, ...] = ()
|
system_prompt: str = ""
|
||||||
options: ClaudeCodeOptions = Field(default_factory=ClaudeCodeOptions)
|
"""Verbatim prompt for agents without ``prompts`` (tests, one-offs)."""
|
||||||
"""Per-agent passthrough for the underlying claude-code-api
|
|
||||||
``BackendOptions``. See :class:`ClaudeCodeOptions`."""
|
prompts: Prompts = Field(default_factory=Prompts)
|
||||||
|
kinds: tuple[Kind, ...] = ()
|
||||||
|
"""Conversation kinds this agent serves; ``create``/``spawn`` reject the
|
||||||
|
rest. Defaults to the kinds ``prompts`` covers, or ``("deep",)`` for a
|
||||||
|
verbatim ``system_prompt``."""
|
||||||
|
|
||||||
|
skill_sets: tuple[Path, ...] = ()
|
||||||
|
gateway_tools: tuple[str, ...] = ()
|
||||||
|
"""Gateway tools exposed in-process (``read_conversation``, ``spawn``,
|
||||||
|
``say``, ``schedule``, ``inject``); empty = no gateway MCP server."""
|
||||||
|
|
||||||
|
options: ClaudeOptions = Field(default_factory=ClaudeOptions)
|
||||||
|
policy: tuple[PolicyRule, ...] = ()
|
||||||
|
"""``PreToolUse`` rules (§3.7), run in order on every tool call; the
|
||||||
|
first ``Deny`` is what the model reads back. See ``core/policy``."""
|
||||||
|
|
||||||
|
@model_validator(mode="after")
|
||||||
|
def _kinds_follow_prompts(self) -> ClaudeAgent:
|
||||||
|
covered = self.prompts.kinds
|
||||||
|
if not self.kinds:
|
||||||
|
object.__setattr__(self, "kinds", covered or ("deep",))
|
||||||
|
elif covered:
|
||||||
|
missing = [k for k in self.kinds if k not in covered]
|
||||||
|
if missing:
|
||||||
|
msg = f"agent {self.name!r} serves {missing} without a prompt"
|
||||||
|
raise ValueError(msg)
|
||||||
|
return self
|
||||||
|
|
||||||
|
def prompt_for(self, kind: Kind) -> tuple[PromptSource, ...] | None:
|
||||||
|
return self.prompts.for_kind(kind)
|
||||||
|
|
||||||
|
def serves(self, kind: str) -> bool:
|
||||||
|
return kind in self.kinds
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
"""Backend adapters.
|
"""Backend adapters.
|
||||||
|
|
||||||
Each backend wraps a provider-specific SDK (``raycast-api``, ``claude-code-api``)
|
Each backend wraps a provider SDK (``raycast-api``, ``claude-agent-sdk``)
|
||||||
and yields the unified :class:`~beaver_gateway.core.events.MessageStreamEvent`
|
and yields the unified :class:`~beaver_gateway.core.events.MessageStreamEvent`
|
||||||
family. The Anthropic-style frontend serialises events straight to SSE.
|
family. The Anthropic-style frontend serialises events straight to SSE.
|
||||||
"""
|
"""
|
||||||
@@ -5,8 +5,15 @@ into a stream of :class:`~beaver_gateway.core.events.MessageStreamEvent`
|
|||||||
records. The frontend serializes whatever comes out straight to SSE, so
|
records. The frontend serializes whatever comes out straight to SSE, so
|
||||||
backends are the only place where provider quirks are translated.
|
backends are the only place where provider quirks are translated.
|
||||||
|
|
||||||
Implementations are plain :class:`typing.Protocol` conformers — no ABC
|
Implementations are plain :class:`typing.Protocol` conformers - no ABC
|
||||||
subclassing — to keep them swappable in tests with bare async generators.
|
subclassing - to keep them swappable in tests with bare async generators.
|
||||||
|
|
||||||
|
``**options`` is the one extension point. Known keys, all optional and
|
||||||
|
ignored by backends that don't keep state: ``conversation_id`` (stable id
|
||||||
|
the backend may pin a live session to), ``session_id`` (backend session to
|
||||||
|
resume when nothing is live), ``capture`` (a
|
||||||
|
:class:`~beaver_gateway.core.turn_capture.TurnCapture` the backend fills
|
||||||
|
after the stream closes).
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|||||||
@@ -1,502 +0,0 @@
|
|||||||
"""Claude Code backend adapter.
|
|
||||||
|
|
||||||
One :class:`ClaudeCodeBackendAdapter` per :class:`ClaudeAgent`. The
|
|
||||||
underlying :class:`claude_code_api.ClaudeCodeBackend` bakes ``cwd`` /
|
|
||||||
``model`` / ``system_prompt`` / MCP wiring into a single
|
|
||||||
:class:`~claude_code_api.BackendOptions` at construction time, so a
|
|
||||||
single backend instance is conceptually bound to one agent (different
|
|
||||||
agents would mean different cwds / system prompts / exposed MCPs and
|
|
||||||
thus different live-session pools).
|
|
||||||
|
|
||||||
Per :meth:`complete` we:
|
|
||||||
|
|
||||||
* hand the full Anthropic-style ``messages`` list to
|
|
||||||
``ClaudeCodeBackend.complete`` — it does its own fingerprint-based
|
|
||||||
session lookup, so we never need to track sessions ourselves;
|
|
||||||
* turn its events into one ``message_start`` … ``message_stop``
|
|
||||||
envelope per ``complete`` call, with content-block indices increasing
|
|
||||||
monotonically across the whole turn;
|
|
||||||
* close the envelope on the ``ResultMessage``.
|
|
||||||
|
|
||||||
How the blocks inside that envelope are produced depends on the agent's
|
|
||||||
transport. By default each ``AssistantMessage`` is re-emitted as
|
|
||||||
``content_block_start`` + one delta + ``content_block_stop`` per block —
|
|
||||||
a "stream" of finished blocks, because the PTY transport has nothing
|
|
||||||
finer to offer. With ``transport="stream_json"`` *and*
|
|
||||||
``include_partial_messages``, the backend instead feeds us the real
|
|
||||||
token-level ``StreamEvent``s and we emit from those; the whole-block
|
|
||||||
records still arrive and are still kept for :class:`TurnCapture`, but
|
|
||||||
emitting from both would duplicate every block on the wire.
|
|
||||||
|
|
||||||
The per-request ``system`` parameter is intentionally **ignored** —
|
|
||||||
``BackendOptions.system_prompt`` is fixed at session-spawn time, and the
|
|
||||||
agent's ``system_prompt`` is the canonical identity of the agent.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import logging
|
|
||||||
import uuid
|
|
||||||
from collections.abc import Mapping # runtime import: isinstance in _emit_stream_event
|
|
||||||
from dataclasses import dataclass, field
|
|
||||||
from typing import TYPE_CHECKING, Any, Self
|
|
||||||
|
|
||||||
from claude_code_api import (
|
|
||||||
AssistantMessage,
|
|
||||||
BackendOptions,
|
|
||||||
ClaudeCodeBackend,
|
|
||||||
ResultMessage,
|
|
||||||
StreamEvent,
|
|
||||||
TextBlock,
|
|
||||||
ThinkingBlock,
|
|
||||||
ToolUseBlock,
|
|
||||||
synthesize_turn_messages,
|
|
||||||
)
|
|
||||||
|
|
||||||
from beaver_gateway.agents.claude import ClaudeAgent
|
|
||||||
from beaver_gateway.core.events import (
|
|
||||||
StopReason,
|
|
||||||
build_content_block_stop,
|
|
||||||
build_input_json_delta,
|
|
||||||
build_message_delta,
|
|
||||||
build_message_start,
|
|
||||||
build_message_stop,
|
|
||||||
build_signature_delta,
|
|
||||||
build_text_block_start,
|
|
||||||
build_text_delta,
|
|
||||||
build_thinking_block_start,
|
|
||||||
build_thinking_delta,
|
|
||||||
build_tool_use_block_start,
|
|
||||||
)
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from collections.abc import AsyncIterator, Iterable
|
|
||||||
|
|
||||||
from anthropic.types import MessageParam
|
|
||||||
|
|
||||||
from beaver_gateway.agents.base import BaseAgent
|
|
||||||
from beaver_gateway.core.events import MessageStreamEvent
|
|
||||||
|
|
||||||
|
|
||||||
_log = logging.getLogger("beaver_gateway.backends.claude_code")
|
|
||||||
|
|
||||||
|
|
||||||
__all__ = ["ClaudeCodeBackendAdapter", "TurnCapture"]
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class TurnCapture:
|
|
||||||
"""Side-channel sink for per-turn metadata.
|
|
||||||
|
|
||||||
Pass an instance via ``ClaudeCodeBackendAdapter.complete(capture=...)``.
|
|
||||||
After the stream finishes, :attr:`synthesized_messages` holds the
|
|
||||||
full assistant↔tool-result cycle (from
|
|
||||||
:func:`claude_code_api.synthesize_turn_messages`) — i.e. the exact
|
|
||||||
list of canonical Anthropic-shape messages claude-code-api stashed
|
|
||||||
the live session under. The markdown frontend uses this to write the
|
|
||||||
conversation history to its DB so a subsequent turn's prefix
|
|
||||||
fingerprint hits the same session.
|
|
||||||
|
|
||||||
Other backends (anthropic, raycast) ignore the kwarg — it lands in
|
|
||||||
their ``**options`` and is silently dropped.
|
|
||||||
"""
|
|
||||||
|
|
||||||
synthesized_messages: list[dict[str, Any]] = field(default_factory=list)
|
|
||||||
|
|
||||||
|
|
||||||
_CLAUDE_TO_ANTHROPIC_STOP: dict[str, StopReason] = {
|
|
||||||
"end_turn": "end_turn",
|
|
||||||
"tool_use": "tool_use",
|
|
||||||
"max_tokens": "max_tokens",
|
|
||||||
"stop_sequence": "stop_sequence",
|
|
||||||
"refusal": "refusal",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _map_stop_reason(raw: str | None) -> StopReason:
|
|
||||||
"""Map claude-code's stop reason into Anthropic's vocabulary.
|
|
||||||
|
|
||||||
Unknown / missing values collapse to ``end_turn`` so the client sees
|
|
||||||
a clean finish rather than a wire-format error.
|
|
||||||
"""
|
|
||||||
if raw is None:
|
|
||||||
return "end_turn"
|
|
||||||
return _CLAUDE_TO_ANTHROPIC_STOP.get(raw, "end_turn")
|
|
||||||
|
|
||||||
|
|
||||||
def _build_mcp_servers(
|
|
||||||
agent: ClaudeAgent, mcp_internal_urls: Mapping[str, str]
|
|
||||||
) -> dict[str, dict[str, Any]] | None:
|
|
||||||
"""Render ``agent.expose_mcps`` into ``BackendOptions.mcp_servers``.
|
|
||||||
|
|
||||||
Each exposed MCP is a streamable-HTTP pointer at the gateway's
|
|
||||||
internal aggregator (built by :mod:`beaver_gateway.mcp.internal_app`).
|
|
||||||
``None`` keeps claude-code from materializing an ``--mcp-config``
|
|
||||||
file when the agent exposes nothing.
|
|
||||||
"""
|
|
||||||
if not agent.expose_mcps:
|
|
||||||
return None
|
|
||||||
servers: dict[str, dict[str, Any]] = {}
|
|
||||||
for em in agent.expose_mcps:
|
|
||||||
url = mcp_internal_urls.get(em.name)
|
|
||||||
if url is None:
|
|
||||||
msg = (
|
|
||||||
f"agent {agent.name!r} exposes MCP {em.name!r} "
|
|
||||||
"but no internal URL is registered for it"
|
|
||||||
)
|
|
||||||
raise ValueError(msg)
|
|
||||||
servers[em.name] = {"type": "http", "url": url}
|
|
||||||
return servers
|
|
||||||
|
|
||||||
|
|
||||||
def _build_backend_options(
|
|
||||||
agent: ClaudeAgent, mcp_internal_urls: Mapping[str, str]
|
|
||||||
) -> BackendOptions:
|
|
||||||
"""Compose the per-agent :class:`BackendOptions`.
|
|
||||||
|
|
||||||
Agent-primary fields:
|
|
||||||
|
|
||||||
* ``cwd`` / ``model`` come from the agent directly;
|
|
||||||
* ``system_prompt`` carries :attr:`BaseAgent.system_prompt`
|
|
||||||
verbatim — i.e. wire-level ``--system-prompt`` (~8.6k tokens
|
|
||||||
lighter than ``--append-system-prompt`` because claude-code's
|
|
||||||
persona/planning conventions and dynamic sections drop out;
|
|
||||||
tool schemas survive via the API ``tools=[]`` channel);
|
|
||||||
* ``append_system_prompt`` carries
|
|
||||||
:attr:`ClaudeCodeOptions.append_system_prompt`, normally
|
|
||||||
``None``. Setting it re-attaches claude-code's built-in prompt
|
|
||||||
*and* this delta — opt-in for "claude as a real coding session";
|
|
||||||
* ``allowed_tools`` follows the PLAN: when the user lists native
|
|
||||||
tools we restrict to those *plus* a per-MCP wildcard so MCP tools
|
|
||||||
stay reachable; when no native list is declared we leave
|
|
||||||
``allowed_tools`` empty (= all tools allowed by claude-code's
|
|
||||||
default);
|
|
||||||
* ``mcp_servers`` comes from :func:`_build_mcp_servers`.
|
|
||||||
|
|
||||||
Every other tunable knob is passed through from
|
|
||||||
:attr:`ClaudeAgent.options`. Our default overrides
|
|
||||||
(``wait_for_turn_duration=True``,
|
|
||||||
``dangerously_skip_permissions=True``) live on
|
|
||||||
:class:`ClaudeCodeOptions`, not here, so a user who builds
|
|
||||||
``ClaudeCodeOptions(...)`` explicitly inherits the same defaults
|
|
||||||
instead of getting whatever claude-code-api ships.
|
|
||||||
"""
|
|
||||||
allowed_tools: tuple[str, ...] = ()
|
|
||||||
if agent.available_native_tools:
|
|
||||||
mcp_wildcards = tuple(f"mcp__{em.name}" for em in agent.expose_mcps)
|
|
||||||
allowed_tools = tuple(agent.available_native_tools) + mcp_wildcards
|
|
||||||
|
|
||||||
opt = agent.options
|
|
||||||
return BackendOptions(
|
|
||||||
cwd=agent.cwd,
|
|
||||||
model=agent.model or None,
|
|
||||||
system_prompt=agent.system_prompt,
|
|
||||||
append_system_prompt=opt.append_system_prompt,
|
|
||||||
allowed_tools=allowed_tools,
|
|
||||||
mcp_servers=_build_mcp_servers(agent, mcp_internal_urls),
|
|
||||||
transport=opt.transport,
|
|
||||||
include_partial_messages=opt.include_partial_messages,
|
|
||||||
# ``None`` means "decide from the agent": an agent that exposes
|
|
||||||
# MCPs needs them on its first reply, one that doesn't shouldn't
|
|
||||||
# pay for a warm-up it gains nothing from.
|
|
||||||
warmup_turn=(
|
|
||||||
bool(agent.expose_mcps) if opt.warmup_turn is None else opt.warmup_turn
|
|
||||||
),
|
|
||||||
disallowed_tools=opt.disallowed_tools,
|
|
||||||
permission_mode=opt.permission_mode,
|
|
||||||
dangerously_skip_permissions=opt.dangerously_skip_permissions,
|
|
||||||
effort=opt.effort,
|
|
||||||
add_dir=opt.add_dir,
|
|
||||||
settings=opt.settings,
|
|
||||||
extra_args=opt.extra_args,
|
|
||||||
extra_env=opt.extra_env,
|
|
||||||
preserve_provider_env=opt.preserve_provider_env,
|
|
||||||
history_injection_mode=opt.history_injection_mode,
|
|
||||||
wait_for_turn_duration=opt.wait_for_turn_duration,
|
|
||||||
include_meta_user=opt.include_meta_user,
|
|
||||||
startup_delay=opt.startup_delay,
|
|
||||||
file_wait_timeout=opt.file_wait_timeout,
|
|
||||||
turn_duration_timeout=opt.turn_duration_timeout,
|
|
||||||
idle_session_ttl=opt.idle_session_ttl,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class ClaudeCodeBackendAdapter:
|
|
||||||
"""One ``claude-code-api`` backend bound to a single :class:`ClaudeAgent`.
|
|
||||||
|
|
||||||
Owns the underlying :class:`ClaudeCodeBackend`'s lifecycle through
|
|
||||||
the async-context-manager protocol so :mod:`beaver_gateway.cli` can
|
|
||||||
park it in its ``AsyncExitStack``.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self, *, agent: ClaudeAgent, mcp_internal_urls: Mapping[str, str]
|
|
||||||
) -> None:
|
|
||||||
self._agent = agent
|
|
||||||
options = _build_backend_options(agent, mcp_internal_urls)
|
|
||||||
self._backend = ClaudeCodeBackend(options)
|
|
||||||
self._streaming = (
|
|
||||||
options.transport == "stream_json" and options.include_partial_messages
|
|
||||||
)
|
|
||||||
"""Whether the backend feeds us token-level ``StreamEvent``s.
|
|
||||||
|
|
||||||
When it does, blocks are emitted from those and the whole-block
|
|
||||||
``AssistantMessage`` records are used only for bookkeeping —
|
|
||||||
emitting from both would duplicate every block on the wire."""
|
|
||||||
|
|
||||||
@property
|
|
||||||
def agent(self) -> ClaudeAgent:
|
|
||||||
return self._agent
|
|
||||||
|
|
||||||
@property
|
|
||||||
def live_session_count(self) -> int:
|
|
||||||
return self._backend.live_session_count
|
|
||||||
|
|
||||||
@property
|
|
||||||
def live_sessions(self) -> dict[str, Any]:
|
|
||||||
"""Live claude processes keyed by claude session_id.
|
|
||||||
|
|
||||||
Pass-through to the underlying ``ClaudeCodeBackend``. The value
|
|
||||||
is a ``PtyClaudeProcess`` or a ``StreamClaudeProcess`` depending
|
|
||||||
on the agent's transport; both implement the
|
|
||||||
``captured_output()`` / ``add_output_listener`` / ``write``
|
|
||||||
surface the admin terminal consumes, so it works either way —
|
|
||||||
with the caveat that the stream transport's "terminal" is a
|
|
||||||
read-only view of the JSON event stream rather than a TUI you
|
|
||||||
can type into. Typed as ``Any`` to avoid leaking the lower
|
|
||||||
layer's type into the gateway's public surface.
|
|
||||||
"""
|
|
||||||
return self._backend.live_sessions
|
|
||||||
|
|
||||||
async def __aenter__(self) -> Self:
|
|
||||||
await self._backend.__aenter__()
|
|
||||||
return self
|
|
||||||
|
|
||||||
async def __aexit__(self, exc_type: object, exc: object, tb: object) -> None:
|
|
||||||
await self._backend.__aexit__(exc_type, exc, tb)
|
|
||||||
|
|
||||||
async def aclose(self) -> None:
|
|
||||||
await self._backend.aclose()
|
|
||||||
|
|
||||||
async def complete(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
agent: BaseAgent,
|
|
||||||
messages: Iterable[MessageParam],
|
|
||||||
system: str | None = None, # noqa: ARG002 — see module docstring
|
|
||||||
capture: TurnCapture | None = None,
|
|
||||||
**options: Any, # noqa: ARG002 — no per-request knobs for claude-code yet
|
|
||||||
) -> AsyncIterator[MessageStreamEvent]:
|
|
||||||
if not isinstance(agent, ClaudeAgent):
|
|
||||||
msg = (
|
|
||||||
"ClaudeCodeBackendAdapter requires ClaudeAgent, "
|
|
||||||
f"got {type(agent).__name__}"
|
|
||||||
)
|
|
||||||
raise TypeError(msg)
|
|
||||||
if agent.name != self._agent.name:
|
|
||||||
# Adapter is per-agent; routing a different agent through it
|
|
||||||
# would mean a different cwd / system_prompt / MCP set than
|
|
||||||
# the live-session pool was spawned with.
|
|
||||||
msg = (
|
|
||||||
f"ClaudeCodeBackendAdapter bound to {self._agent.name!r} "
|
|
||||||
f"got request for {agent.name!r}"
|
|
||||||
)
|
|
||||||
raise ValueError(msg)
|
|
||||||
|
|
||||||
msgs_list: list[Mapping[str, Any]] = list(messages)
|
|
||||||
_log.info(
|
|
||||||
"complete: agent=%s n_messages=%d capture=%s live_sessions=%d",
|
|
||||||
agent.name,
|
|
||||||
len(msgs_list),
|
|
||||||
capture is not None,
|
|
||||||
self._backend.live_session_count,
|
|
||||||
)
|
|
||||||
|
|
||||||
message_id = f"msg_{uuid.uuid4().hex}"
|
|
||||||
yield build_message_start(message_id=message_id, model=agent.model)
|
|
||||||
|
|
||||||
next_index = 0
|
|
||||||
stop_reason: str | None = None
|
|
||||||
usage: Mapping[str, Any] | None = None
|
|
||||||
n_text = 0
|
|
||||||
n_thinking = 0
|
|
||||||
n_tool_use = 0
|
|
||||||
# We keep raw events so we can hand them to
|
|
||||||
# ``synthesize_turn_messages`` after the stream closes — the
|
|
||||||
# markdown frontend stores the result in its conversation
|
|
||||||
# history so the next turn's prefix matches the backend's
|
|
||||||
# session-pool fingerprint. UserMessage (tool_result) events
|
|
||||||
# are silently discarded from the wire but kept here.
|
|
||||||
raw_events: list[Any] = []
|
|
||||||
|
|
||||||
async for event in self._backend.complete(msgs_list):
|
|
||||||
raw_events.append(event)
|
|
||||||
if isinstance(event, StreamEvent):
|
|
||||||
# Only ever arrives when ``self._streaming`` — the
|
|
||||||
# backend doesn't emit these otherwise. Indices are
|
|
||||||
# already rebased onto the whole turn by claude-code-api.
|
|
||||||
for ev in _emit_stream_event(event.event):
|
|
||||||
yield ev
|
|
||||||
elif isinstance(event, AssistantMessage):
|
|
||||||
for block in event.content:
|
|
||||||
if isinstance(block, TextBlock):
|
|
||||||
n_text += 1
|
|
||||||
elif isinstance(block, ThinkingBlock):
|
|
||||||
n_thinking += 1
|
|
||||||
elif isinstance(block, ToolUseBlock):
|
|
||||||
n_tool_use += 1
|
|
||||||
if not self._streaming:
|
|
||||||
for ev in _emit_block(block, next_index):
|
|
||||||
yield ev
|
|
||||||
next_index += 1
|
|
||||||
elif isinstance(event, ResultMessage):
|
|
||||||
# ResultMessage is the terminal event from TurnManager
|
|
||||||
# — we capture its stop_reason / usage for the envelope
|
|
||||||
# below. We DO NOT break here: an early break would
|
|
||||||
# raise GeneratorExit inside claude-code-api's
|
|
||||||
# ``complete`` coroutine before it gets a chance to
|
|
||||||
# stash the live session under the post-turn
|
|
||||||
# fingerprint, so every continuation would miss the
|
|
||||||
# cache and reseed. Let the inner generator exit
|
|
||||||
# naturally instead.
|
|
||||||
stop_reason = event.stop_reason
|
|
||||||
usage = event.usage
|
|
||||||
# UserMessage (tool_result records) and SystemMessage
|
|
||||||
# (turn_duration heartbeats) carry no content for the
|
|
||||||
# /v1/messages caller — skip silently on the wire, but they
|
|
||||||
# ARE retained in ``raw_events`` for synthesis below.
|
|
||||||
|
|
||||||
if capture is not None:
|
|
||||||
capture.synthesized_messages = synthesize_turn_messages(raw_events)
|
|
||||||
|
|
||||||
_log.info(
|
|
||||||
"complete: agent=%s DONE text=%d thinking=%d tool_use=%d stop=%s synth=%d",
|
|
||||||
agent.name,
|
|
||||||
n_text,
|
|
||||||
n_thinking,
|
|
||||||
n_tool_use,
|
|
||||||
stop_reason,
|
|
||||||
len(capture.synthesized_messages) if capture is not None else 0,
|
|
||||||
)
|
|
||||||
|
|
||||||
yield build_message_delta(
|
|
||||||
stop_reason=_map_stop_reason(stop_reason), usage=_normalize_usage(usage)
|
|
||||||
)
|
|
||||||
yield build_message_stop()
|
|
||||||
|
|
||||||
|
|
||||||
def _emit_stream_event(event: Mapping[str, Any]) -> Iterable[MessageStreamEvent]:
|
|
||||||
"""Render one raw Anthropic streaming event through our own builders.
|
|
||||||
|
|
||||||
The payload is already Anthropic-shaped, so forwarding it verbatim
|
|
||||||
is tempting — but it arrives as a plain dict and the frontend
|
|
||||||
serializes pydantic models, so it would have to be validated against
|
|
||||||
the SDK's ``RawMessageStreamEvent`` union anyway. Rebuilding it from
|
|
||||||
the fields we recognize is the same amount of work and fails the
|
|
||||||
same way :func:`_emit_block` already does: a block or delta type
|
|
||||||
this gateway doesn't know is dropped rather than raising mid-stream
|
|
||||||
on a claude release that added one.
|
|
||||||
|
|
||||||
Only ``content_block_*`` events reach here — claude-code-api strips
|
|
||||||
the inner per-request message envelopes, because the caller owns the
|
|
||||||
single envelope spanning the whole turn.
|
|
||||||
"""
|
|
||||||
etype = event.get("type")
|
|
||||||
index = event.get("index")
|
|
||||||
if not isinstance(index, int):
|
|
||||||
return ()
|
|
||||||
|
|
||||||
if etype == "content_block_start":
|
|
||||||
block = event.get("content_block")
|
|
||||||
if not isinstance(block, Mapping):
|
|
||||||
return ()
|
|
||||||
btype = block.get("type")
|
|
||||||
if btype == "text":
|
|
||||||
return (build_text_block_start(index),)
|
|
||||||
if btype == "thinking":
|
|
||||||
return (build_thinking_block_start(index),)
|
|
||||||
if btype == "tool_use":
|
|
||||||
return (
|
|
||||||
build_tool_use_block_start(
|
|
||||||
index,
|
|
||||||
tool_use_id=str(block.get("id", "")),
|
|
||||||
name=str(block.get("name", "")),
|
|
||||||
),
|
|
||||||
)
|
|
||||||
return ()
|
|
||||||
|
|
||||||
if etype == "content_block_delta":
|
|
||||||
delta = event.get("delta")
|
|
||||||
if not isinstance(delta, Mapping):
|
|
||||||
return ()
|
|
||||||
dtype = delta.get("type")
|
|
||||||
if dtype == "text_delta":
|
|
||||||
return (build_text_delta(index, str(delta.get("text", ""))),)
|
|
||||||
if dtype == "thinking_delta":
|
|
||||||
return (build_thinking_delta(index, str(delta.get("thinking", ""))),)
|
|
||||||
if dtype == "signature_delta":
|
|
||||||
return (build_signature_delta(index, str(delta.get("signature", ""))),)
|
|
||||||
if dtype == "input_json_delta":
|
|
||||||
return (build_input_json_delta(index, str(delta.get("partial_json", ""))),)
|
|
||||||
return ()
|
|
||||||
|
|
||||||
if etype == "content_block_stop":
|
|
||||||
return (build_content_block_stop(index),)
|
|
||||||
|
|
||||||
return ()
|
|
||||||
|
|
||||||
|
|
||||||
def _emit_block(
|
|
||||||
block: TextBlock | ThinkingBlock | ToolUseBlock | Any, index: int
|
|
||||||
) -> Iterable[MessageStreamEvent]:
|
|
||||||
"""Render one ``claude-code`` content block as Anthropic stream events.
|
|
||||||
|
|
||||||
``ToolResultBlock`` would arrive only on user-role records — we
|
|
||||||
don't emit it here because :meth:`complete` skips ``UserMessage``.
|
|
||||||
"""
|
|
||||||
if isinstance(block, TextBlock):
|
|
||||||
return (
|
|
||||||
build_text_block_start(index),
|
|
||||||
build_text_delta(index, block.text),
|
|
||||||
build_content_block_stop(index),
|
|
||||||
)
|
|
||||||
if isinstance(block, ThinkingBlock):
|
|
||||||
return (
|
|
||||||
build_thinking_block_start(index),
|
|
||||||
build_thinking_delta(index, block.thinking),
|
|
||||||
build_signature_delta(index, block.signature),
|
|
||||||
build_content_block_stop(index),
|
|
||||||
)
|
|
||||||
if isinstance(block, ToolUseBlock):
|
|
||||||
partial = json.dumps(block.input, separators=(",", ":"), ensure_ascii=False)
|
|
||||||
return (
|
|
||||||
build_tool_use_block_start(index, tool_use_id=block.id, name=block.name),
|
|
||||||
build_input_json_delta(index, partial),
|
|
||||||
build_content_block_stop(index),
|
|
||||||
)
|
|
||||||
return ()
|
|
||||||
|
|
||||||
|
|
||||||
def _normalize_usage(usage: Mapping[str, Any] | None) -> dict[str, int] | None:
|
|
||||||
"""Coerce claude-code's ``usage`` dict to Anthropic ``MessageDeltaUsage`` shape.
|
|
||||||
|
|
||||||
claude-code copies whatever the JSONL ``usage`` record carried —
|
|
||||||
fields can be missing, strings, or ints. We pass through only the
|
|
||||||
fields ``MessageDeltaUsage`` knows about and discard the rest so an
|
|
||||||
odd ``cache_creation`` object structure doesn't fail pydantic
|
|
||||||
validation downstream.
|
|
||||||
"""
|
|
||||||
if not usage:
|
|
||||||
return None
|
|
||||||
out: dict[str, int] = {}
|
|
||||||
for key in (
|
|
||||||
"input_tokens",
|
|
||||||
"output_tokens",
|
|
||||||
"cache_creation_input_tokens",
|
|
||||||
"cache_read_input_tokens",
|
|
||||||
):
|
|
||||||
value = usage.get(key)
|
|
||||||
if isinstance(value, int):
|
|
||||||
out[key] = value
|
|
||||||
return out or None
|
|
||||||
File diff suppressed because it is too large.
Load diff
@@ -26,11 +26,12 @@ tools we route it back to the underlying MCP in-process, append the
|
|||||||
result as a ``tool`` message, and re-issue the stream — all inside one
|
result as a ``tool`` message, and re-issue the stream — all inside one
|
||||||
Anthropic envelope (one ``message_start`` … one ``message_stop``). The
|
Anthropic envelope (one ``message_start`` … one ``message_stop``). The
|
||||||
tool_use blocks DO surface to the caller (mirrors what
|
tool_use blocks DO surface to the caller (mirrors what
|
||||||
``ClaudeCodeBackendAdapter`` does), but tool_results stay internal.
|
``ClaudeSdkBackend`` does), but tool_results stay internal.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import fnmatch
|
||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
import uuid
|
import uuid
|
||||||
@@ -297,6 +298,8 @@ def _build_agent_catalog(
|
|||||||
for mt in mcp_tools.get(em.name, []):
|
for mt in mcp_tools.get(em.name, []):
|
||||||
if em.tools is not None and mt.name not in em.tools:
|
if em.tools is not None and mt.name not in em.tools:
|
||||||
continue
|
continue
|
||||||
|
if any(fnmatch.fnmatchcase(mt.name, pat) for pat in em.deny):
|
||||||
|
continue
|
||||||
wire_name = f"{em.name}__{mt.name}"
|
wire_name = f"{em.name}__{mt.name}"
|
||||||
routing[wire_name] = (em.name, mt.name)
|
routing[wire_name] = (em.name, mt.name)
|
||||||
local_tools.append(
|
local_tools.append(
|
||||||
|
|||||||
+216
-16
@@ -22,35 +22,63 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
import contextlib
|
import contextlib
|
||||||
|
import functools
|
||||||
import logging
|
import logging
|
||||||
import signal
|
import signal
|
||||||
from contextlib import AsyncExitStack
|
from contextlib import AsyncExitStack
|
||||||
from typing import TYPE_CHECKING
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
|
import psycopg
|
||||||
import uvicorn
|
import uvicorn
|
||||||
import uvloop
|
import uvloop
|
||||||
from dotenv import load_dotenv
|
from dotenv import load_dotenv
|
||||||
|
from pgqueuer import PsycopgDriver
|
||||||
from raycast_api import Client as RaycastClient
|
from raycast_api import Client as RaycastClient
|
||||||
from raycast_api.config import Config as RaycastConfig
|
from raycast_api.config import Config as RaycastConfig
|
||||||
|
|
||||||
from beaver_gateway import config_loader
|
from beaver_gateway import config_loader
|
||||||
from beaver_gateway.agents.claude import ClaudeAgent
|
from beaver_gateway.agents.claude import ClaudeAgent
|
||||||
from beaver_gateway.agents.raycast import RaycastAgent
|
from beaver_gateway.agents.raycast import RaycastAgent
|
||||||
from beaver_gateway.backends.claude_code import ClaudeCodeBackendAdapter
|
from beaver_gateway.backends.claude_sdk import (
|
||||||
|
ClaudeSdkBackend,
|
||||||
|
RunnerConfig,
|
||||||
|
UsageEvent,
|
||||||
|
)
|
||||||
from beaver_gateway.backends.raycast import RaycastBackend
|
from beaver_gateway.backends.raycast import RaycastBackend
|
||||||
from beaver_gateway.core.auth import TokenStore
|
from beaver_gateway.core.auth import TokenStore
|
||||||
from beaver_gateway.core.registry import AgentRegistry, McpRegistry
|
from beaver_gateway.core.bus import EventBus
|
||||||
|
from beaver_gateway.core.conversations import Conversations
|
||||||
|
from beaver_gateway.core.envelope import Envelope
|
||||||
|
from beaver_gateway.core.gateway_tools import build_tool_server
|
||||||
|
from beaver_gateway.core.redact import install as install_redaction
|
||||||
|
from beaver_gateway.core.redact import load_secrets as load_secrets_to_mask
|
||||||
|
from beaver_gateway.core.registry import AgentRegistry, Gateway, McpRegistry
|
||||||
|
from beaver_gateway.core.rotation import Rotation, RotationPolicy
|
||||||
|
from beaver_gateway.core.scheduler import Scheduler
|
||||||
|
from beaver_gateway.core.sessions import SessionPool
|
||||||
|
from beaver_gateway.frontends._auth import require_token
|
||||||
from beaver_gateway.frontends.base import GatewayRuntime
|
from beaver_gateway.frontends.base import GatewayRuntime
|
||||||
|
from beaver_gateway.frontends.root import build_root_app
|
||||||
from beaver_gateway.mcp.internal_app import build_internal_app
|
from beaver_gateway.mcp.internal_app import build_internal_app
|
||||||
from beaver_gateway.settings import Settings
|
from beaver_gateway.settings import Settings
|
||||||
from beaver_gateway.storage import Database
|
from beaver_gateway.storage import (
|
||||||
|
Database,
|
||||||
|
PostgresSessionStore,
|
||||||
|
Usage,
|
||||||
|
append_audit,
|
||||||
|
append_usage,
|
||||||
|
)
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
|
from claude_agent_sdk import McpSdkServerConfig
|
||||||
from fastmcp import FastMCP
|
from fastmcp import FastMCP
|
||||||
from fastmcp.tools.base import Tool as FastMCPTool
|
from fastmcp.tools.base import Tool as FastMCPTool
|
||||||
from starlette.applications import Starlette
|
from starlette.applications import Starlette
|
||||||
|
from starlette.requests import Request
|
||||||
|
from starlette.types import ASGIApp
|
||||||
|
|
||||||
from beaver_gateway.backends.base import Backend
|
from beaver_gateway.backends.base import Backend
|
||||||
|
from beaver_gateway.core.policy import ToolAudit
|
||||||
from beaver_gateway.mcp.types import McpServerT
|
from beaver_gateway.mcp.types import McpServerT
|
||||||
|
|
||||||
|
|
||||||
@@ -62,6 +90,7 @@ def main() -> None:
|
|||||||
logging.basicConfig(
|
logging.basicConfig(
|
||||||
level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s"
|
level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s"
|
||||||
)
|
)
|
||||||
|
install_redaction()
|
||||||
_install_sigterm_handler()
|
_install_sigterm_handler()
|
||||||
asyncio.run(_async_main(), loop_factory=uvloop.new_event_loop)
|
asyncio.run(_async_main(), loop_factory=uvloop.new_event_loop)
|
||||||
|
|
||||||
@@ -97,6 +126,9 @@ async def _async_main() -> None:
|
|||||||
# populates Settings fields, not the process environment.
|
# populates Settings fields, not the process environment.
|
||||||
# ``override=False``: real env vars (Docker, systemd) win over .env.
|
# ``override=False``: real env vars (Docker, systemd) win over .env.
|
||||||
load_dotenv(override=False)
|
load_dotenv(override=False)
|
||||||
|
# Only now does the process environment hold the credentials the
|
||||||
|
# redactor masks literally (in Docker they arrive via ``env_file``).
|
||||||
|
load_secrets_to_mask()
|
||||||
|
|
||||||
settings = Settings() # ty: ignore[missing-argument]
|
settings = Settings() # ty: ignore[missing-argument]
|
||||||
gateway = config_loader.load(settings.config_path)
|
gateway = config_loader.load(settings.config_path)
|
||||||
@@ -114,7 +146,9 @@ async def _async_main() -> None:
|
|||||||
# background task). BOOTSTRAP_TOKENS layers on top so first-run /
|
# background task). BOOTSTRAP_TOKENS layers on top so first-run /
|
||||||
# examples still work without DB writes.
|
# examples still work without DB writes.
|
||||||
token_store = TokenStore(
|
token_store = TokenStore(
|
||||||
db, bootstrap=TokenStore.parse_bootstrap(settings.bootstrap_tokens)
|
db,
|
||||||
|
bootstrap=TokenStore.parse_bootstrap(settings.bootstrap_tokens),
|
||||||
|
bootstrap_scopes=TokenStore.parse_bootstrap_scopes(settings.bootstrap_tokens),
|
||||||
)
|
)
|
||||||
|
|
||||||
async with AsyncExitStack() as stack:
|
async with AsyncExitStack() as stack:
|
||||||
@@ -122,8 +156,8 @@ async def _async_main() -> None:
|
|||||||
await token_store.start()
|
await token_store.start()
|
||||||
stack.push_async_callback(token_store.stop)
|
stack.push_async_callback(token_store.stop)
|
||||||
# Internal MCP URLs must exist before we construct any
|
# Internal MCP URLs must exist before we construct any
|
||||||
# ClaudeCodeBackendAdapter — adapters bake the URLs into their
|
# ClaudeSdkBackend - adapters bake the URLs into their
|
||||||
# ``BackendOptions.mcp_servers`` at construction time. The
|
# ``mcp_servers`` at construction time. The
|
||||||
# ``mcp_servers`` map is used by the Raycast backend, which
|
# ``mcp_servers`` map is used by the Raycast backend, which
|
||||||
# needs in-process ``list_tools`` / ``call_tool`` access (the
|
# needs in-process ``list_tools`` / ``call_tool`` access (the
|
||||||
# Raycast wire has no native MCP concept).
|
# Raycast wire has no native MCP concept).
|
||||||
@@ -136,14 +170,48 @@ async def _async_main() -> None:
|
|||||||
# surfaces at startup instead of mid-conversation.
|
# surfaces at startup instead of mid-conversation.
|
||||||
mcp_tools = await _prefetch_mcp_tools(mcp_servers)
|
mcp_tools = await _prefetch_mcp_tools(mcp_servers)
|
||||||
|
|
||||||
|
pool = SessionPool()
|
||||||
|
bus = EventBus()
|
||||||
|
late = _LateConversations()
|
||||||
|
session_store = PostgresSessionStore(db)
|
||||||
backends: dict[str, Backend] = await _build_backends(
|
backends: dict[str, Backend] = await _build_backends(
|
||||||
settings=settings,
|
settings=settings,
|
||||||
agents=agents,
|
agents=agents,
|
||||||
stack=stack,
|
stack=stack,
|
||||||
|
db=db,
|
||||||
|
session_store=session_store,
|
||||||
mcp_internal_urls=internal_urls,
|
mcp_internal_urls=internal_urls,
|
||||||
mcp_servers=mcp_servers,
|
mcp_servers=mcp_servers,
|
||||||
mcp_tools=mcp_tools,
|
mcp_tools=mcp_tools,
|
||||||
|
pool=pool,
|
||||||
|
late=late,
|
||||||
)
|
)
|
||||||
|
conversations = Conversations(
|
||||||
|
db=db,
|
||||||
|
agents=agents,
|
||||||
|
backends=backends,
|
||||||
|
bus=bus,
|
||||||
|
pool=pool,
|
||||||
|
store=session_store,
|
||||||
|
texts=gateway.texts,
|
||||||
|
frontends=gateway.frontends,
|
||||||
|
envelope=Envelope(
|
||||||
|
watch=gateway.watch, tz=gateway.tz, recall=gateway.recall
|
||||||
|
),
|
||||||
|
distiller=gateway.distiller,
|
||||||
|
user_sink=gateway.user_sink,
|
||||||
|
)
|
||||||
|
late.conversations = conversations
|
||||||
|
scheduler = Scheduler(
|
||||||
|
conversations=conversations,
|
||||||
|
jobs=gateway.jobs,
|
||||||
|
driver=await _pgqueuer_driver(settings.database_url, stack),
|
||||||
|
budget=gateway.budget,
|
||||||
|
rotation=Rotation(
|
||||||
|
conversations, gateway.rotation or RotationPolicy(tz=gateway.tz)
|
||||||
|
),
|
||||||
|
)
|
||||||
|
conversations.scheduler = scheduler
|
||||||
|
|
||||||
runtime = GatewayRuntime(
|
runtime = GatewayRuntime(
|
||||||
agents=agents,
|
agents=agents,
|
||||||
@@ -156,6 +224,11 @@ async def _async_main() -> None:
|
|||||||
admin_pass=settings.admin_pass,
|
admin_pass=settings.admin_pass,
|
||||||
session_secret=settings.session_secret,
|
session_secret=settings.session_secret,
|
||||||
frontends=tuple(gateway.frontends),
|
frontends=tuple(gateway.frontends),
|
||||||
|
conversations=conversations,
|
||||||
|
bus=bus,
|
||||||
|
pool=pool,
|
||||||
|
scheduler=scheduler,
|
||||||
|
public_url=gateway.public_url.rstrip("/") if gateway.public_url else None,
|
||||||
)
|
)
|
||||||
|
|
||||||
for fe in gateway.frontends:
|
for fe in gateway.frontends:
|
||||||
@@ -181,13 +254,60 @@ async def _async_main() -> None:
|
|||||||
# it in this path and exit cleanly (Phase 0 DoD).
|
# it in this path and exit cleanly (Phase 0 DoD).
|
||||||
return
|
return
|
||||||
|
|
||||||
|
await conversations.start()
|
||||||
|
stack.push_async_callback(conversations.stop)
|
||||||
|
await scheduler.start()
|
||||||
|
stack.push_async_callback(scheduler.stop)
|
||||||
|
hooks = scheduler.app(
|
||||||
|
functools.partial(_authorize_hook, runtime=runtime, scope="api")
|
||||||
|
)
|
||||||
async with asyncio.TaskGroup() as tg:
|
async with asyncio.TaskGroup() as tg:
|
||||||
|
tg.create_task(pool.reap_loop())
|
||||||
if internal_app is not None:
|
if internal_app is not None:
|
||||||
tg.create_task(_serve_internal_mcp(internal_app, settings=settings))
|
tg.create_task(_serve_internal_mcp(internal_app, settings=settings))
|
||||||
|
tg.create_task(_serve_root(gateway, extra={"/hooks": hooks}))
|
||||||
|
if gateway.watch is not None:
|
||||||
|
tg.create_task(gateway.watch.run())
|
||||||
for fe in gateway.frontends:
|
for fe in gateway.frontends:
|
||||||
tg.create_task(fe.serve())
|
tg.create_task(fe.serve())
|
||||||
|
|
||||||
|
|
||||||
|
async def _authorize_hook(
|
||||||
|
request: Request, *, runtime: GatewayRuntime, scope: str
|
||||||
|
) -> str:
|
||||||
|
return await require_token(request, runtime, scope=scope)
|
||||||
|
|
||||||
|
|
||||||
|
async def _pgqueuer_driver(url: str, stack: AsyncExitStack) -> PsycopgDriver | None:
|
||||||
|
"""A dedicated autocommit connection for pgqueuer's LISTEN/NOTIFY."""
|
||||||
|
plain = _plain_postgres_url(url)
|
||||||
|
if plain is None:
|
||||||
|
return None
|
||||||
|
conn = await psycopg.AsyncConnection.connect(plain, autocommit=True)
|
||||||
|
stack.push_async_callback(conn.close)
|
||||||
|
return PsycopgDriver(conn)
|
||||||
|
|
||||||
|
|
||||||
|
def _plain_postgres_url(url: str) -> str | None:
|
||||||
|
for prefix in ("postgresql+psycopg://", "postgresql://", "postgres://"):
|
||||||
|
if url.startswith(prefix):
|
||||||
|
return "postgresql://" + url[len(prefix) :]
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
async def _serve_root(gateway: Gateway, *, extra: dict[str, ASGIApp]) -> None:
|
||||||
|
app = build_root_app(gateway.frontends, extra=extra)
|
||||||
|
config = uvicorn.Config(app, host=gateway.host, port=gateway.port, log_level="info")
|
||||||
|
_log.info(
|
||||||
|
"gateway on http://%s:%d - %s",
|
||||||
|
gateway.host,
|
||||||
|
gateway.port,
|
||||||
|
", ".join([*(fe.path for fe in gateway.frontends if fe.path), *extra])
|
||||||
|
or "no http frontends",
|
||||||
|
)
|
||||||
|
await uvicorn.Server(config).serve()
|
||||||
|
|
||||||
|
|
||||||
def _build_internal_mcp(
|
def _build_internal_mcp(
|
||||||
mcps: list[McpServerT], *, settings: Settings
|
mcps: list[McpServerT], *, settings: Settings
|
||||||
) -> tuple[Starlette | None, dict[str, str], dict[str, FastMCP]]:
|
) -> tuple[Starlette | None, dict[str, str], dict[str, FastMCP]]:
|
||||||
@@ -247,27 +367,49 @@ async def _serve_internal_mcp(app: Starlette, *, settings: Settings) -> None:
|
|||||||
await server.serve()
|
await server.serve()
|
||||||
|
|
||||||
|
|
||||||
|
class _LateConversations:
|
||||||
|
"""Backends need a tool-server factory before the service that backs it exists."""
|
||||||
|
|
||||||
|
conversations: Conversations | None = None
|
||||||
|
|
||||||
|
def server(
|
||||||
|
self, key: str, _kind: str, names: tuple[str, ...]
|
||||||
|
) -> McpSdkServerConfig | None:
|
||||||
|
if self.conversations is None or not names:
|
||||||
|
return None
|
||||||
|
return build_tool_server(self.conversations, conversation_key=key, names=names)
|
||||||
|
|
||||||
|
async def ask(self, key: str, payload: dict[str, Any]) -> str:
|
||||||
|
if self.conversations is None:
|
||||||
|
msg = "conversations service is not up yet"
|
||||||
|
raise RuntimeError(msg)
|
||||||
|
answer = await self.conversations.ask(key, payload)
|
||||||
|
return self.conversations.answer_text(answer)
|
||||||
|
|
||||||
|
|
||||||
async def _build_backends(
|
async def _build_backends(
|
||||||
*,
|
*,
|
||||||
settings: Settings,
|
settings: Settings,
|
||||||
agents: AgentRegistry,
|
agents: AgentRegistry,
|
||||||
stack: AsyncExitStack,
|
stack: AsyncExitStack,
|
||||||
|
db: Database,
|
||||||
|
session_store: PostgresSessionStore,
|
||||||
mcp_internal_urls: dict[str, str],
|
mcp_internal_urls: dict[str, str],
|
||||||
mcp_servers: dict[str, FastMCP],
|
mcp_servers: dict[str, FastMCP],
|
||||||
mcp_tools: dict[str, list[FastMCPTool]],
|
mcp_tools: dict[str, list[FastMCPTool]],
|
||||||
|
pool: SessionPool,
|
||||||
|
late: _LateConversations,
|
||||||
) -> dict[str, Backend]:
|
) -> dict[str, Backend]:
|
||||||
"""Construct one backend per agent name.
|
"""Construct one backend per agent name.
|
||||||
|
|
||||||
The Raycast ``Client`` is shared across every ``RaycastAgent``
|
The Raycast ``Client`` is shared across every ``RaycastAgent``
|
||||||
(bearer + device-id are process-wide), so we open it lazily — only
|
(bearer + device-id are process-wide), so we open it lazily - only
|
||||||
when at least one ``RaycastAgent`` is present — and close it via
|
when at least one ``RaycastAgent`` is present - and close it via
|
||||||
the caller's exit stack.
|
the caller's exit stack.
|
||||||
|
|
||||||
Each :class:`ClaudeAgent` gets its own
|
Each :class:`ClaudeAgent` gets its own :class:`ClaudeSdkBackend`
|
||||||
:class:`ClaudeCodeBackendAdapter`: ``BackendOptions`` pins
|
(own live-session pool, own prompt and MCP set); all of them share
|
||||||
``cwd`` / ``model`` / ``system_prompt`` / ``mcp_servers`` for the
|
the session store and the usage sink.
|
||||||
lifetime of the underlying ``ClaudeCodeBackend``, so different
|
|
||||||
agents can't share one.
|
|
||||||
"""
|
"""
|
||||||
backends: dict[str, Backend] = {}
|
backends: dict[str, Backend] = {}
|
||||||
|
|
||||||
@@ -281,10 +423,68 @@ async def _build_backends(
|
|||||||
for a in raycast_agents:
|
for a in raycast_agents:
|
||||||
backends[a.name] = raycast_backend
|
backends[a.name] = raycast_backend
|
||||||
|
|
||||||
|
runner = RunnerConfig(user=settings.claude_runner_user, home=settings.claude_home)
|
||||||
|
mcp_tool_names = {
|
||||||
|
name: [t.name for t in tools] for name, tools in mcp_tools.items()
|
||||||
|
}
|
||||||
|
|
||||||
|
async def record_usage(event: UsageEvent) -> None:
|
||||||
|
row = Usage(
|
||||||
|
agent_name=event.agent_name,
|
||||||
|
conversation_id=event.conversation_id,
|
||||||
|
session_id=event.session_id,
|
||||||
|
model=event.model,
|
||||||
|
effort=event.effort,
|
||||||
|
input_tokens=event.usage.input_tokens,
|
||||||
|
output_tokens=event.usage.output_tokens,
|
||||||
|
cache_read_tokens=event.usage.cache_read_tokens,
|
||||||
|
cache_creation_tokens=event.usage.cache_creation_tokens,
|
||||||
|
context_tokens=event.usage.context_tokens,
|
||||||
|
cost_usd=event.usage.cost_usd,
|
||||||
|
duration_ms=event.usage.duration_ms,
|
||||||
|
num_turns=event.usage.num_turns,
|
||||||
|
model_usage=event.usage.model_usage,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
async with db.session() as session:
|
||||||
|
await append_usage(session, row)
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
_log.exception("usage write failed for %s", event.agent_name)
|
||||||
|
|
||||||
|
async def record_tool(event: ToolAudit) -> None:
|
||||||
|
detail = {
|
||||||
|
"conversation": event.conversation,
|
||||||
|
"kind": event.kind,
|
||||||
|
"tool": event.tool,
|
||||||
|
"decision": event.decision,
|
||||||
|
"reason": event.reason,
|
||||||
|
"brief": event.brief,
|
||||||
|
}
|
||||||
|
try:
|
||||||
|
async with db.session() as session:
|
||||||
|
await append_audit(
|
||||||
|
session,
|
||||||
|
actor=f"agent:{event.agent}",
|
||||||
|
kind="tool_call",
|
||||||
|
agent_name=event.agent,
|
||||||
|
detail=detail,
|
||||||
|
)
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
_log.exception("tool audit write failed for %s", event.agent)
|
||||||
|
|
||||||
for a in agents:
|
for a in agents:
|
||||||
if isinstance(a, ClaudeAgent):
|
if isinstance(a, ClaudeAgent):
|
||||||
adapter = ClaudeCodeBackendAdapter(
|
adapter = ClaudeSdkBackend(
|
||||||
agent=a, mcp_internal_urls=mcp_internal_urls
|
agent=a,
|
||||||
|
mcp_internal_urls=mcp_internal_urls,
|
||||||
|
session_store=session_store,
|
||||||
|
mcp_tool_names=mcp_tool_names,
|
||||||
|
runner=runner,
|
||||||
|
usage_sink=record_usage,
|
||||||
|
pool=pool,
|
||||||
|
tool_server=functools.partial(late.server, names=a.gateway_tools),
|
||||||
|
asker=late.ask,
|
||||||
|
audit_sink=record_tool,
|
||||||
)
|
)
|
||||||
await stack.enter_async_context(adapter)
|
await stack.enter_async_context(adapter)
|
||||||
backends[a.name] = adapter
|
backends[a.name] = adapter
|
||||||
|
|||||||
@@ -14,12 +14,17 @@ contents here before handing it back.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import sys
|
||||||
from typing import TYPE_CHECKING, Any
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
|
from croniter import croniter
|
||||||
|
|
||||||
from beaver_gateway.agents.base import BaseAgent, ExposedMcp
|
from beaver_gateway.agents.base import BaseAgent, ExposedMcp
|
||||||
from beaver_gateway.agents.claude import ClaudeAgent
|
from beaver_gateway.agents.claude import ClaudeAgent
|
||||||
from beaver_gateway.agents.raycast import RaycastAgent
|
from beaver_gateway.agents.raycast import RaycastAgent
|
||||||
|
from beaver_gateway.core.conversations import ConversationTexts
|
||||||
from beaver_gateway.core.registry import Gateway
|
from beaver_gateway.core.registry import Gateway
|
||||||
|
from beaver_gateway.core.scheduler import Job
|
||||||
from beaver_gateway.frontends.base import Frontend
|
from beaver_gateway.frontends.base import Frontend
|
||||||
from beaver_gateway.mcp.types import HttpMcp, McpServer, PythonToolMcp, StdioMcp
|
from beaver_gateway.mcp.types import HttpMcp, McpServer, PythonToolMcp, StdioMcp
|
||||||
|
|
||||||
@@ -37,6 +42,7 @@ _PUBLIC_NAMES: dict[str, Any] = {
|
|||||||
"McpServer": McpServer,
|
"McpServer": McpServer,
|
||||||
"ExposedMcp": ExposedMcp,
|
"ExposedMcp": ExposedMcp,
|
||||||
"Gateway": Gateway,
|
"Gateway": Gateway,
|
||||||
|
"ConversationTexts": ConversationTexts,
|
||||||
}
|
}
|
||||||
|
|
||||||
_McpInstance = StdioMcp | HttpMcp | PythonToolMcp
|
_McpInstance = StdioMcp | HttpMcp | PythonToolMcp
|
||||||
@@ -54,6 +60,10 @@ def load(path: Path) -> Gateway:
|
|||||||
raise ConfigError(msg) from exc
|
raise ConfigError(msg) from exc
|
||||||
|
|
||||||
code = compile(source, str(path), "exec")
|
code = compile(source, str(path), "exec")
|
||||||
|
# Siblings of the config (``policy.py``, ``mcps/``) import by name.
|
||||||
|
parent = str(path.resolve().parent)
|
||||||
|
if parent not in sys.path:
|
||||||
|
sys.path.insert(0, parent)
|
||||||
namespace: dict[str, Any] = {"__file__": str(path), **_PUBLIC_NAMES}
|
namespace: dict[str, Any] = {"__file__": str(path), **_PUBLIC_NAMES}
|
||||||
exec(code, namespace) # noqa: S102 - exec'ing user config is the feature
|
exec(code, namespace) # noqa: S102 - exec'ing user config is the feature
|
||||||
|
|
||||||
@@ -97,3 +107,15 @@ def _validate(gw: Gateway, path: Path) -> None:
|
|||||||
f"got {type(f).__name__}"
|
f"got {type(f).__name__}"
|
||||||
)
|
)
|
||||||
raise ConfigError(msg)
|
raise ConfigError(msg)
|
||||||
|
names: set[str] = set()
|
||||||
|
for i, j in enumerate(gw.jobs):
|
||||||
|
if not isinstance(j, Job):
|
||||||
|
msg = f"{path}: gateway.jobs[{i}] must be a Job, got {type(j).__name__}"
|
||||||
|
raise ConfigError(msg)
|
||||||
|
if j.name in names:
|
||||||
|
msg = f"{path}: duplicate job name {j.name!r}"
|
||||||
|
raise ConfigError(msg)
|
||||||
|
names.add(j.name)
|
||||||
|
if j.cron is not None and not croniter.is_valid(j.cron):
|
||||||
|
msg = f"{path}: job {j.name!r} has an invalid cron {j.cron!r}"
|
||||||
|
raise ConfigError(msg)
|
||||||
@@ -51,6 +51,7 @@ KNOWN_KINDS: frozenset[str] = frozenset(
|
|||||||
{
|
{
|
||||||
"messages", # POST /v1/messages accepted
|
"messages", # POST /v1/messages accepted
|
||||||
"mcp_call", # /mcp/<ns>/... proxied
|
"mcp_call", # /mcp/<ns>/... proxied
|
||||||
|
"tool_call", # a model's tool call seen by the PreToolUse hook
|
||||||
"login_ok",
|
"login_ok",
|
||||||
"login_failed",
|
"login_failed",
|
||||||
"logout",
|
"logout",
|
||||||
|
|||||||
@@ -60,12 +60,13 @@ class TokenStoreError(ValueError):
|
|||||||
"""Malformed ``BOOTSTRAP_TOKENS`` value or duplicate token."""
|
"""Malformed ``BOOTSTRAP_TOKENS`` value or duplicate token."""
|
||||||
|
|
||||||
|
|
||||||
VALID_SCOPES: frozenset[str] = frozenset({"*", "messages", "mcp", "admin"})
|
VALID_SCOPES: frozenset[str] = frozenset({"*", "messages", "mcp", "admin", "api"})
|
||||||
"""The scopes a ``Token.scope`` may hold (Phase 4.3 admin UI enforces).
|
"""The scopes a ``Token.scope`` may hold (Phase 4.3 admin UI enforces).
|
||||||
|
|
||||||
* ``*`` — wildcard, may use any frontend
|
* ``*`` — wildcard, may use any frontend
|
||||||
* ``messages`` — Anthropic Messages frontend only
|
* ``messages`` — Anthropic Messages frontend only
|
||||||
* ``mcp`` — MCP server frontend only
|
* ``mcp`` — MCP server frontend only
|
||||||
|
* ``api`` — conversations API (``/api``) and its SSE
|
||||||
* ``admin`` — reserved for programmatic admin access; the AdminFrontend
|
* ``admin`` — reserved for programmatic admin access; the AdminFrontend
|
||||||
itself authenticates via session cookies, not bearer tokens, so this
|
itself authenticates via session cookies, not bearer tokens, so this
|
||||||
scope is unused today and kept for forward compatibility.
|
scope is unused today and kept for forward compatibility.
|
||||||
@@ -154,6 +155,7 @@ class TokenStore:
|
|||||||
db: Database | None = None,
|
db: Database | None = None,
|
||||||
*,
|
*,
|
||||||
bootstrap: Mapping[str, str] | None = None,
|
bootstrap: Mapping[str, str] | None = None,
|
||||||
|
bootstrap_scopes: Mapping[str, str] | None = None,
|
||||||
ttl_seconds: float = 30.0,
|
ttl_seconds: float = 30.0,
|
||||||
flush_interval: float = 5.0,
|
flush_interval: float = 5.0,
|
||||||
) -> None:
|
) -> None:
|
||||||
@@ -172,9 +174,13 @@ class TokenStore:
|
|||||||
raise TokenStoreError(msg)
|
raise TokenStoreError(msg)
|
||||||
by_value[value] = name
|
by_value[value] = name
|
||||||
self._bootstrap_by_value: dict[str, str] = by_value
|
self._bootstrap_by_value: dict[str, str] = by_value
|
||||||
self._bootstrap_scopes: dict[str, str] = dict.fromkeys(
|
# Bootstrap entries are ``"*"`` unless the env narrows them
|
||||||
by_value.values(), _BOOTSTRAP_SCOPE
|
# (``name:value:scope``) - a webhook token in a URL should not be
|
||||||
)
|
# an admin token.
|
||||||
|
self._bootstrap_scopes: dict[str, str] = {
|
||||||
|
name: (bootstrap_scopes or {}).get(name, _BOOTSTRAP_SCOPE)
|
||||||
|
for name in by_value.values()
|
||||||
|
}
|
||||||
|
|
||||||
self._db = db
|
self._db = db
|
||||||
self._ttl = ttl_seconds
|
self._ttl = ttl_seconds
|
||||||
@@ -190,22 +196,17 @@ class TokenStore:
|
|||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def parse_bootstrap(raw: str) -> dict[str, str]:
|
def parse_bootstrap(raw: str) -> dict[str, str]:
|
||||||
"""Parse ``name1:value1,name2:value2`` (the ``BOOTSTRAP_TOKENS`` form)."""
|
"""Parse ``name1:value1,name2:value2[:scope]`` (``BOOTSTRAP_TOKENS``)."""
|
||||||
tokens: dict[str, str] = {}
|
return {name: value for name, (value, _scope) in _parse_entries(raw).items()}
|
||||||
for chunk in raw.split(","):
|
|
||||||
entry = chunk.strip()
|
@staticmethod
|
||||||
if not entry:
|
def parse_bootstrap_scopes(raw: str) -> dict[str, str]:
|
||||||
continue
|
"""Scopes given as a third field in ``BOOTSTRAP_TOKENS``; the rest get ``*``."""
|
||||||
name, sep, value = entry.partition(":")
|
return {
|
||||||
if not sep:
|
name: scope
|
||||||
msg = f"token entry missing ':' separator: {entry!r}"
|
for name, (_value, scope) in _parse_entries(raw).items()
|
||||||
raise TokenStoreError(msg)
|
if scope is not None
|
||||||
name, value = name.strip(), value.strip()
|
}
|
||||||
if name in tokens:
|
|
||||||
msg = f"duplicate token name: {name!r}"
|
|
||||||
raise TokenStoreError(msg)
|
|
||||||
tokens[name] = value
|
|
||||||
return tokens
|
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def from_env(cls, raw: str, db: Database | None = None) -> TokenStore:
|
def from_env(cls, raw: str, db: Database | None = None) -> TokenStore:
|
||||||
@@ -235,6 +236,11 @@ class TokenStore:
|
|||||||
self._flusher_task = None
|
self._flusher_task = None
|
||||||
await self._flush_now()
|
await self._flush_now()
|
||||||
|
|
||||||
|
def grant(self, name: str, value: str, *, scope: str = _BOOTSTRAP_SCOPE) -> None:
|
||||||
|
"""Add an in-memory token for the process lifetime (the admin UI's bearer)."""
|
||||||
|
self._bootstrap_by_value[value] = name
|
||||||
|
self._bootstrap_scopes[name] = scope
|
||||||
|
|
||||||
async def invalidate(self) -> None:
|
async def invalidate(self) -> None:
|
||||||
"""Force the next verify to re-read from DB (Phase 4.3 admin hook)."""
|
"""Force the next verify to re-read from DB (Phase 4.3 admin hook)."""
|
||||||
self._loaded_at = 0.0
|
self._loaded_at = 0.0
|
||||||
@@ -372,3 +378,22 @@ __all__ = [
|
|||||||
"TokenStoreError",
|
"TokenStoreError",
|
||||||
"hash_token",
|
"hash_token",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_entries(raw: str) -> dict[str, tuple[str, str | None]]:
|
||||||
|
entries: dict[str, tuple[str, str | None]] = {}
|
||||||
|
for chunk in raw.split(","):
|
||||||
|
entry = chunk.strip()
|
||||||
|
if not entry:
|
||||||
|
continue
|
||||||
|
name, sep, rest = entry.partition(":")
|
||||||
|
if not sep:
|
||||||
|
msg = f"token entry missing ':' separator: {entry!r}"
|
||||||
|
raise TokenStoreError(msg)
|
||||||
|
value, sep, scope = rest.partition(":")
|
||||||
|
name, value, scope = name.strip(), value.strip(), scope.strip()
|
||||||
|
if name in entries:
|
||||||
|
msg = f"duplicate token name: {name!r}"
|
||||||
|
raise TokenStoreError(msg)
|
||||||
|
entries[name] = (value, scope if sep and scope else None)
|
||||||
|
return entries
|
||||||
@@ -0,0 +1,69 @@
|
|||||||
|
"""In-process event bus: what the gateway does, as a stream frontends can tap.
|
||||||
|
|
||||||
|
Events are plain dicts ``{"type", "seq", "ts", ...}``. Publishers never
|
||||||
|
block; a subscriber that falls behind loses its oldest events rather than
|
||||||
|
stalling the turn that produced them. ``/api/events`` serialises the
|
||||||
|
stream as SSE, the panel builds its subagent tree from ``stream`` events
|
||||||
|
carrying ``parent_tool_use_id``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import contextlib
|
||||||
|
import itertools
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import AsyncIterator
|
||||||
|
|
||||||
|
__all__ = ["Event", "EventBus"]
|
||||||
|
|
||||||
|
Event = dict[str, Any]
|
||||||
|
|
||||||
|
|
||||||
|
class EventBus:
|
||||||
|
def __init__(self, *, maxsize: int = 2000) -> None:
|
||||||
|
self._maxsize = maxsize
|
||||||
|
self._subscribers: set[asyncio.Queue[Event]] = set()
|
||||||
|
self._seq = itertools.count(1)
|
||||||
|
|
||||||
|
def publish(self, type_: str, **data: Any) -> Event:
|
||||||
|
event: Event = {
|
||||||
|
"type": type_,
|
||||||
|
"seq": next(self._seq),
|
||||||
|
"ts": datetime.now(UTC).isoformat(timespec="milliseconds"),
|
||||||
|
**data,
|
||||||
|
}
|
||||||
|
for queue in list(self._subscribers):
|
||||||
|
if queue.full():
|
||||||
|
with contextlib.suppress(asyncio.QueueEmpty):
|
||||||
|
queue.get_nowait()
|
||||||
|
queue.put_nowait(event)
|
||||||
|
return event
|
||||||
|
|
||||||
|
@contextlib.asynccontextmanager
|
||||||
|
async def subscribe(self) -> AsyncIterator[asyncio.Queue[Event]]:
|
||||||
|
queue: asyncio.Queue[Event] = asyncio.Queue(maxsize=self._maxsize)
|
||||||
|
self._subscribers.add(queue)
|
||||||
|
try:
|
||||||
|
yield queue
|
||||||
|
finally:
|
||||||
|
self._subscribers.discard(queue)
|
||||||
|
|
||||||
|
async def stream(
|
||||||
|
self, *, conversation_id: str | None = None
|
||||||
|
) -> AsyncIterator[Event]:
|
||||||
|
async with self.subscribe() as queue:
|
||||||
|
while True:
|
||||||
|
event = await queue.get()
|
||||||
|
if (
|
||||||
|
conversation_id is None
|
||||||
|
or event.get("conversation_id") == conversation_id
|
||||||
|
):
|
||||||
|
yield event
|
||||||
|
|
||||||
|
@property
|
||||||
|
def subscribers(self) -> int:
|
||||||
|
return len(self._subscribers)
|
||||||
@@ -63,6 +63,7 @@ __all__ = [
|
|||||||
"load_messages",
|
"load_messages",
|
||||||
"mint_conversation",
|
"mint_conversation",
|
||||||
"rewrite_messages",
|
"rewrite_messages",
|
||||||
|
"set_session_id",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
@@ -86,6 +87,12 @@ class ForkOutcome:
|
|||||||
messages: list[MessageParam]
|
messages: list[MessageParam]
|
||||||
persist_messages: list[dict[str, Any]]
|
persist_messages: list[dict[str, Any]]
|
||||||
divergence_index: int | None
|
divergence_index: int | None
|
||||||
|
edited: bool = False
|
||||||
|
"""An earlier assistant turn's prose was rewritten in the file."""
|
||||||
|
|
||||||
|
@property
|
||||||
|
def reuse_session(self) -> bool:
|
||||||
|
return self.divergence_index is None and not self.edited
|
||||||
|
|
||||||
|
|
||||||
# ---- public store API ---------------------------------------------------
|
# ---- public store API ---------------------------------------------------
|
||||||
@@ -121,6 +128,17 @@ async def mint_conversation(
|
|||||||
return row
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
async def set_session_id(
|
||||||
|
session: AsyncSession, *, conversation_id: int, session_id: str | None
|
||||||
|
) -> None:
|
||||||
|
conv = await session.get(Conversation, conversation_id)
|
||||||
|
if conv is None or conv.session_id == session_id:
|
||||||
|
return
|
||||||
|
conv.session_id = session_id
|
||||||
|
session.add(conv)
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
|
||||||
async def load_messages(
|
async def load_messages(
|
||||||
session: AsyncSession, *, conversation_id: int
|
session: AsyncSession, *, conversation_id: int
|
||||||
) -> list[dict[str, Any]]:
|
) -> list[dict[str, Any]]:
|
||||||
@@ -395,7 +413,7 @@ def diff_and_fork(
|
|||||||
new_user_turn = incoming[-1]
|
new_user_turn = incoming[-1]
|
||||||
prior_incoming = incoming[:-1]
|
prior_incoming = incoming[:-1]
|
||||||
|
|
||||||
spliced_groups, divergence = _walk_prefix(prior_incoming, stored_groups)
|
spliced_groups, divergence, edited = _walk_prefix(prior_incoming, stored_groups)
|
||||||
|
|
||||||
if divergence is None and len(prior_incoming) < len(stored_groups):
|
if divergence is None and len(prior_incoming) < len(stored_groups):
|
||||||
if _file_lags_store(stored_groups, len(prior_incoming), new_user_turn):
|
if _file_lags_store(stored_groups, len(prior_incoming), new_user_turn):
|
||||||
@@ -421,6 +439,7 @@ def diff_and_fork(
|
|||||||
messages=backend_msgs,
|
messages=backend_msgs,
|
||||||
persist_messages=persist_msgs,
|
persist_messages=persist_msgs,
|
||||||
divergence_index=divergence,
|
divergence_index=divergence,
|
||||||
|
edited=edited,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -458,44 +477,50 @@ def _file_lags_store(
|
|||||||
|
|
||||||
def _walk_prefix(
|
def _walk_prefix(
|
||||||
prior_incoming: list[ParsedTurn], stored_groups: list[_StoredDisplayTurn]
|
prior_incoming: list[ParsedTurn], stored_groups: list[_StoredDisplayTurn]
|
||||||
) -> tuple[list[list[dict[str, Any]]], int | None]:
|
) -> tuple[list[list[dict[str, Any]]], int | None, bool]:
|
||||||
"""Walk incoming vs stored side-by-side until first divergence.
|
"""Walk incoming vs stored side-by-side until first divergence.
|
||||||
|
|
||||||
Returns the spliced/matched group list (one entry per matched
|
Returns the spliced/matched group list (one entry per matched
|
||||||
display turn, each carrying the raw messages we'll feed to the
|
display turn, each carrying the raw messages we'll feed to the
|
||||||
backend for that turn) and the divergence index (``None`` if all
|
backend for that turn), the divergence index (``None`` if all
|
||||||
of ``prior_incoming`` matched).
|
of ``prior_incoming`` matched) and whether any assistant prose
|
||||||
|
was spliced in from the file - a rewritten reply keeps the
|
||||||
|
structure but must not resume the session that said otherwise.
|
||||||
"""
|
"""
|
||||||
from beaver_gateway.frontends.markdown.parser import TextSegment, ToolSegment
|
from beaver_gateway.frontends.markdown.parser import TextSegment, ToolSegment
|
||||||
|
|
||||||
spliced_groups: list[list[dict[str, Any]]] = []
|
spliced_groups: list[list[dict[str, Any]]] = []
|
||||||
|
edited = False
|
||||||
for i, inc in enumerate(prior_incoming):
|
for i, inc in enumerate(prior_incoming):
|
||||||
if i >= len(stored_groups):
|
if i >= len(stored_groups):
|
||||||
return spliced_groups, i
|
return spliced_groups, i, edited
|
||||||
st = stored_groups[i]
|
st = stored_groups[i]
|
||||||
if inc.role != st.role:
|
if inc.role != st.role:
|
||||||
return spliced_groups, i
|
return spliced_groups, i, edited
|
||||||
if inc.role == "user":
|
if inc.role == "user":
|
||||||
if inc.text != st.spoken_text:
|
if inc.text != st.spoken_text:
|
||||||
return spliced_groups, i
|
return spliced_groups, i, edited
|
||||||
spliced_groups.append(list(st.messages))
|
spliced_groups.append(list(st.messages))
|
||||||
continue
|
continue
|
||||||
inc_skeleton = tuple(
|
inc_skeleton = tuple(
|
||||||
s.name for s in inc.structure if isinstance(s, ToolSegment)
|
s.name for s in inc.structure if isinstance(s, ToolSegment)
|
||||||
)
|
)
|
||||||
inc_text_count = sum(1 for s in inc.structure if isinstance(s, TextSegment))
|
inc_text_count = sum(1 for s in inc.structure if isinstance(s, TextSegment))
|
||||||
if inc_skeleton != st.skeleton:
|
# Files rendered without tool callouts (§3.10) carry no skeleton:
|
||||||
return spliced_groups, i
|
# prose alone decides whether the turn matched.
|
||||||
|
if inc_skeleton and inc_skeleton != st.skeleton:
|
||||||
|
return spliced_groups, i, edited
|
||||||
if inc.text == st.spoken_text:
|
if inc.text == st.spoken_text:
|
||||||
spliced_groups.append(list(st.messages))
|
spliced_groups.append(list(st.messages))
|
||||||
continue
|
continue
|
||||||
if inc_text_count != st.text_segment_count:
|
if inc_skeleton and inc_text_count != st.text_segment_count:
|
||||||
return spliced_groups, i
|
return spliced_groups, i, edited
|
||||||
spliced = _splice_assistant_group(stored_group=st, incoming=inc)
|
spliced = _splice_assistant_group(stored_group=st, incoming=inc)
|
||||||
if spliced is None:
|
if spliced is None:
|
||||||
return spliced_groups, i
|
return spliced_groups, i, edited
|
||||||
spliced_groups.append(spliced)
|
spliced_groups.append(spliced)
|
||||||
return spliced_groups, None
|
edited = True
|
||||||
|
return spliced_groups, None, edited
|
||||||
|
|
||||||
|
|
||||||
def _assemble_tail(
|
def _assemble_tail(
|
||||||
@@ -539,6 +564,13 @@ def _splice_assistant_group(
|
|||||||
return _splice_by_rebuild(stored_group=stored_group, incoming=incoming)
|
return _splice_by_rebuild(stored_group=stored_group, incoming=incoming)
|
||||||
|
|
||||||
|
|
||||||
|
def _is_text_block(blk: object) -> bool:
|
||||||
|
if not isinstance(blk, dict):
|
||||||
|
return False
|
||||||
|
block = cast("dict[str, Any]", blk)
|
||||||
|
return block.get("type") == "text" and bool(str(block.get("text", "")).strip())
|
||||||
|
|
||||||
|
|
||||||
def _splice_in_place(
|
def _splice_in_place(
|
||||||
*, stored_group: _StoredDisplayTurn, incoming: ParsedTurn
|
*, stored_group: _StoredDisplayTurn, incoming: ParsedTurn
|
||||||
) -> list[dict[str, Any]] | None:
|
) -> list[dict[str, Any]] | None:
|
||||||
@@ -570,11 +602,7 @@ def _splice_in_place(
|
|||||||
if msg["role"] != "assistant" or not isinstance(content, list):
|
if msg["role"] != "assistant" or not isinstance(content, list):
|
||||||
continue
|
continue
|
||||||
positions.extend(
|
positions.extend(
|
||||||
(mi, bi)
|
(mi, bi) for bi, blk in enumerate(content) if _is_text_block(blk)
|
||||||
for bi, blk in enumerate(content)
|
|
||||||
if isinstance(blk, dict)
|
|
||||||
and blk.get("type") == "text"
|
|
||||||
and str(blk.get("text", "")).strip()
|
|
||||||
)
|
)
|
||||||
if len(positions) != len(new_texts):
|
if len(positions) != len(new_texts):
|
||||||
return None
|
return None
|
||||||
|
|||||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,197 @@
|
|||||||
|
"""Closing a deep chat: the digest, the index, the file line cap (§6.4, §8.4).
|
||||||
|
|
||||||
|
The gateway knows no path by itself (§0.8): ``Distiller`` from ``config.py``
|
||||||
|
names the agent, says where digests land and where the index lives, and
|
||||||
|
the distiller writes the file on its own. What the gateway does is check that a file
|
||||||
|
with a valid frontmatter appeared under ``Distiller.dir`` during the fork
|
||||||
|
turn, put one line into the index, and cap the merge text at
|
||||||
|
``SUMMARY_LINES``. ``LineCap`` is the same idea for a file a job rewrites
|
||||||
|
(``состояние.md``): a result longer than the cap is bounced - the file
|
||||||
|
goes back to what it was and the job is told to shorten.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import date, datetime
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
|
import frontmatter
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Iterable, Sequence
|
||||||
|
|
||||||
|
from beaver_gateway.storage.models import Conversation
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"SUMMARY_LINES",
|
||||||
|
"Digest",
|
||||||
|
"DistillContext",
|
||||||
|
"Distiller",
|
||||||
|
"LineCap",
|
||||||
|
"append_index",
|
||||||
|
"check_digest",
|
||||||
|
"find_digest",
|
||||||
|
"index_line",
|
||||||
|
"trim_summary",
|
||||||
|
"written_paths",
|
||||||
|
]
|
||||||
|
|
||||||
|
SUMMARY_LINES = 5
|
||||||
|
"""A merge into the master is at most this many lines (§6.4)."""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class Distiller:
|
||||||
|
"""Who closes deep chats, where digests land, where the index lives."""
|
||||||
|
|
||||||
|
agent: str
|
||||||
|
dir: Path
|
||||||
|
index: Path
|
||||||
|
type: str = "выжимка"
|
||||||
|
"""Value the ``type`` frontmatter key must carry."""
|
||||||
|
|
||||||
|
index_header: str = "# индекс\n\nстрока на выжимку: чат → его выжимка.\n" # noqa: RUF001
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class DistillContext:
|
||||||
|
"""What the setup's distill prompt is built from."""
|
||||||
|
|
||||||
|
conversation: Conversation
|
||||||
|
title: str | None
|
||||||
|
source: str | None
|
||||||
|
"""The window the chat lives in (a vault-relative path for markdown)."""
|
||||||
|
|
||||||
|
chat_name: str
|
||||||
|
"""What a ``[[wikilink]]`` to the chat is called."""
|
||||||
|
|
||||||
|
memory: bool
|
||||||
|
reason: str
|
||||||
|
day: date
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class Digest:
|
||||||
|
path: Path
|
||||||
|
source: str
|
||||||
|
date: date
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class LineCap:
|
||||||
|
"""A file a job rewrites may not exceed ``max_lines`` after its turn."""
|
||||||
|
|
||||||
|
path: Path
|
||||||
|
max_lines: int
|
||||||
|
|
||||||
|
def as_flags(self) -> dict[str, Any]:
|
||||||
|
return {"path": str(self.path), "max_lines": self.max_lines}
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_flags(cls, value: Any) -> LineCap | None:
|
||||||
|
if not isinstance(value, dict):
|
||||||
|
return None
|
||||||
|
path, max_lines = value.get("path"), value.get("max_lines")
|
||||||
|
if not isinstance(path, str) or not isinstance(max_lines, int):
|
||||||
|
return None
|
||||||
|
return cls(path=Path(path), max_lines=max_lines)
|
||||||
|
|
||||||
|
|
||||||
|
def written_paths(messages: Iterable[dict[str, Any]]) -> list[Path]:
|
||||||
|
"""Paths the turn's ``Write``/``Edit`` calls targeted, in order."""
|
||||||
|
out: list[Path] = []
|
||||||
|
for message in messages:
|
||||||
|
content = message.get("content")
|
||||||
|
if not isinstance(content, list):
|
||||||
|
continue
|
||||||
|
for block in content:
|
||||||
|
if not isinstance(block, dict) or block.get("type") != "tool_use":
|
||||||
|
continue
|
||||||
|
if block.get("name") not in ("Write", "Edit", "MultiEdit"):
|
||||||
|
continue
|
||||||
|
target = (block.get("input") or {}).get("file_path")
|
||||||
|
if isinstance(target, str) and target:
|
||||||
|
out.append(Path(target))
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def find_digest(
|
||||||
|
digests: Distiller, *, since: datetime, written: Sequence[Path] = ()
|
||||||
|
) -> Path | None:
|
||||||
|
"""The digest the fork turn produced.
|
||||||
|
|
||||||
|
A written path under ``dir`` first, otherwise the newest file there
|
||||||
|
modified since ``since``.
|
||||||
|
"""
|
||||||
|
root = digests.dir.resolve()
|
||||||
|
for path in reversed(list(written)):
|
||||||
|
candidate = path if path.is_absolute() else digests.dir / path
|
||||||
|
try:
|
||||||
|
inside = candidate.resolve().relative_to(root)
|
||||||
|
except (OSError, ValueError):
|
||||||
|
continue
|
||||||
|
if (digests.dir / inside).is_file():
|
||||||
|
return digests.dir / inside
|
||||||
|
if not digests.dir.is_dir():
|
||||||
|
return None
|
||||||
|
stamp = since.timestamp() - 1
|
||||||
|
fresh = [
|
||||||
|
p
|
||||||
|
for p in digests.dir.glob("*.md")
|
||||||
|
if p.is_file() and p.stat().st_mtime >= stamp
|
||||||
|
]
|
||||||
|
return max(fresh, key=lambda p: p.stat().st_mtime) if fresh else None
|
||||||
|
|
||||||
|
|
||||||
|
def check_digest(path: Path, digests: Distiller) -> Digest | str:
|
||||||
|
"""The digest's frontmatter parsed, or why it is not a digest."""
|
||||||
|
try:
|
||||||
|
post = frontmatter.load(str(path))
|
||||||
|
except (OSError, ValueError) as exc:
|
||||||
|
return f"не читается: {exc}"
|
||||||
|
meta = post.metadata
|
||||||
|
if meta.get("type") != digests.type:
|
||||||
|
return f"`type` должен быть `{digests.type}`, не {meta.get('type')!r}"
|
||||||
|
source = meta.get("source")
|
||||||
|
if not isinstance(source, str) or not source.strip():
|
||||||
|
return "`source` пустой"
|
||||||
|
when = meta.get("date")
|
||||||
|
if isinstance(when, datetime):
|
||||||
|
when = when.date()
|
||||||
|
elif isinstance(when, str):
|
||||||
|
try:
|
||||||
|
when = date.fromisoformat(when.strip())
|
||||||
|
except ValueError:
|
||||||
|
return f"`date` не дата: {when!r}"
|
||||||
|
if not isinstance(when, date):
|
||||||
|
return "`date` отсутствует"
|
||||||
|
if not post.content.strip():
|
||||||
|
return "тело пустое"
|
||||||
|
return Digest(path=path, source=source.strip(), date=when)
|
||||||
|
|
||||||
|
|
||||||
|
def index_line(digest: Digest, chat_name: str) -> str:
|
||||||
|
return f"- {digest.date.isoformat()} [[{chat_name}]] → [[{digest.path.stem}]]"
|
||||||
|
|
||||||
|
|
||||||
|
def append_index(digests: Distiller, line: str) -> None:
|
||||||
|
index = digests.index
|
||||||
|
text = index.read_text(encoding="utf-8") if index.exists() else ""
|
||||||
|
if line in text.splitlines():
|
||||||
|
return
|
||||||
|
if not text:
|
||||||
|
text = digests.index_header
|
||||||
|
if not text.endswith("\n"):
|
||||||
|
text += "\n"
|
||||||
|
index.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
index.write_text(text + line + "\n", encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def trim_summary(text: str, limit: int = SUMMARY_LINES) -> tuple[str, bool]:
|
||||||
|
"""The merge text cut to ``limit`` non-empty lines; ``True`` if it was."""
|
||||||
|
lines = [line.rstrip() for line in text.strip().splitlines() if line.strip()]
|
||||||
|
if len(lines) <= limit:
|
||||||
|
return "\n".join(lines), False
|
||||||
|
return "\n".join(lines[:limit]), True
|
||||||
@@ -0,0 +1,141 @@
|
|||||||
|
"""The envelope (§3.3): a background block after the user's text.
|
||||||
|
|
||||||
|
Assembled when the turn starts, never when the message is queued: the
|
||||||
|
time and what changed in the vault since the last envelope (added lines
|
||||||
|
for the ``full`` files, names and counts for the rest). Ceilings keep it
|
||||||
|
a signal, not a document; the injects that ride along are bundled below
|
||||||
|
it by the queue, with their own header.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
from zoneinfo import ZoneInfo
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Callable, Sequence
|
||||||
|
|
||||||
|
from beaver_gateway.core.watch import Change, VaultWatch
|
||||||
|
|
||||||
|
__all__ = ["Envelope", "RecallContext", "render"]
|
||||||
|
|
||||||
|
_log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
HEADER = (
|
||||||
|
"[конверт - фоновый сигнал, не обращение; "
|
||||||
|
"реагируй, только если относится к вопросу]"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class RecallContext:
|
||||||
|
"""What the setup's ``recall`` hook sees: the user's text and where it landed."""
|
||||||
|
|
||||||
|
text: str
|
||||||
|
kind: str
|
||||||
|
now: datetime
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(slots=True)
|
||||||
|
class Envelope:
|
||||||
|
watch: VaultWatch | None = None
|
||||||
|
tz: str = "UTC"
|
||||||
|
max_lines: int = 120
|
||||||
|
per_file: int = 30
|
||||||
|
names_only_within: float = 600.0
|
||||||
|
last_at: datetime | None = None
|
||||||
|
recall: Callable[[RecallContext], str | None] | None = None
|
||||||
|
"""Setup-side lookup run on the user's text at turn start: pointers into
|
||||||
|
the vault (a person's card, the agent's own notes, due dates) that the
|
||||||
|
gateway cannot know the paths of. Its lines go under the vault block;
|
||||||
|
a failure is logged and the envelope goes out without them."""
|
||||||
|
|
||||||
|
def build(
|
||||||
|
self, *, now: datetime | None = None, text: str = "", kind: str = "master"
|
||||||
|
) -> str:
|
||||||
|
now = now or datetime.now(UTC)
|
||||||
|
changes = self.watch.take() if self.watch is not None else []
|
||||||
|
names_only = (
|
||||||
|
self.last_at is not None
|
||||||
|
and (now - self.last_at).total_seconds() < self.names_only_within
|
||||||
|
)
|
||||||
|
out = render(
|
||||||
|
now=now,
|
||||||
|
tz=self.tz,
|
||||||
|
changes=changes,
|
||||||
|
since=self.last_at,
|
||||||
|
names_only=names_only,
|
||||||
|
max_lines=self.max_lines,
|
||||||
|
per_file=self.per_file,
|
||||||
|
)
|
||||||
|
self.last_at = now
|
||||||
|
block = self.recall_block(text=text, kind=kind, now=now)
|
||||||
|
return f"{out}\n{block}" if block else out
|
||||||
|
|
||||||
|
def recall_only(
|
||||||
|
self, *, text: str, kind: str, now: datetime | None = None
|
||||||
|
) -> str | None:
|
||||||
|
"""The recall lines under the header, without the vault diff (branches)."""
|
||||||
|
block = self.recall_block(text=text, kind=kind, now=now or datetime.now(UTC))
|
||||||
|
return f"{HEADER}\n{block}" if block else None
|
||||||
|
|
||||||
|
def recall_block(self, *, text: str, kind: str, now: datetime) -> str | None:
|
||||||
|
if self.recall is None or not text.strip():
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
block = self.recall(RecallContext(text=text, kind=kind, now=now))
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
_log.exception("recall hook failed")
|
||||||
|
return None
|
||||||
|
return block.strip() or None if block else None
|
||||||
|
|
||||||
|
|
||||||
|
def render(
|
||||||
|
*,
|
||||||
|
now: datetime,
|
||||||
|
tz: str,
|
||||||
|
changes: Sequence[Change],
|
||||||
|
since: datetime | None,
|
||||||
|
names_only: bool,
|
||||||
|
max_lines: int = 120,
|
||||||
|
per_file: int = 30,
|
||||||
|
) -> str:
|
||||||
|
zone = ZoneInfo(tz)
|
||||||
|
stamp = now.astimezone(zone)
|
||||||
|
lines = [HEADER, f"время: {stamp:%Y-%m-%d %H:%M} ({_zone_label(tz)})"]
|
||||||
|
ordered = sorted(changes, key=lambda c: (not c.full, c.path))
|
||||||
|
since_label = (
|
||||||
|
f"с {since.astimezone(zone):%H:%M}" if since is not None else "со старта" # noqa: RUF001
|
||||||
|
)
|
||||||
|
if ordered:
|
||||||
|
names = ", ".join(f"{c.path} (+{c.added_count})" for c in ordered)
|
||||||
|
lines.append(f"vault, изменено {since_label}: {names}")
|
||||||
|
if not names_only:
|
||||||
|
_append_diffs(lines, ordered, max_lines=max_lines, per_file=per_file)
|
||||||
|
return "\n".join(lines[:max_lines])
|
||||||
|
|
||||||
|
|
||||||
|
def _append_diffs(
|
||||||
|
lines: list[str], changes: Sequence[Change], *, max_lines: int, per_file: int
|
||||||
|
) -> None:
|
||||||
|
budget = max_lines - len(lines) - 1
|
||||||
|
for change in changes:
|
||||||
|
if not change.full or not change.added:
|
||||||
|
continue
|
||||||
|
if budget < 3:
|
||||||
|
lines.append("… (потолок конверта)")
|
||||||
|
return
|
||||||
|
shown = change.added[: min(per_file, budget - 2)]
|
||||||
|
lines.append(f"--- {change.path}, только добавленное ---")
|
||||||
|
lines.extend(f"+ {line}" for line in shown)
|
||||||
|
budget -= 1 + len(shown)
|
||||||
|
if len(change.added) > len(shown):
|
||||||
|
lines.append(f"+ … ещё {len(change.added) - len(shown)}")
|
||||||
|
budget -= 1
|
||||||
|
|
||||||
|
|
||||||
|
def _zone_label(tz: str) -> str:
|
||||||
|
return tz.rsplit("/", 1)[-1].replace("_", " ")
|
||||||
@@ -0,0 +1,261 @@
|
|||||||
|
"""In-process MCP server with the gateway's own tools (§3.1, §3.2).
|
||||||
|
|
||||||
|
One server per live session so every tool knows which conversation is
|
||||||
|
calling; ``alwaysLoad`` keeps the tools out of tool search. Which names a
|
||||||
|
session gets comes from ``ClaudeAgent.gateway_tools``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from dataclasses import replace
|
||||||
|
from typing import TYPE_CHECKING, Any, cast
|
||||||
|
|
||||||
|
from claude_agent_sdk import create_sdk_mcp_server, tool
|
||||||
|
|
||||||
|
from beaver_gateway.core.injects import URGENCY
|
||||||
|
from beaver_gateway.core.kinds import as_kind
|
||||||
|
from beaver_gateway.core.redact import redact_data
|
||||||
|
|
||||||
|
URGENCY_HELP = (
|
||||||
|
"normal waits for the hourly batch or rides with the next turn, wake "
|
||||||
|
"starts a turn as soon as the target is idle, urgent cuts a running turn"
|
||||||
|
)
|
||||||
|
TARGET_HELP = (
|
||||||
|
"conversation id, or `master` for the open master, `parent` for the "
|
||||||
|
"parent of this conversation"
|
||||||
|
)
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Iterable
|
||||||
|
|
||||||
|
from claude_agent_sdk import McpSdkServerConfig, SdkMcpTool
|
||||||
|
|
||||||
|
from beaver_gateway.core.conversations import Conversations
|
||||||
|
|
||||||
|
__all__ = ["SERVER_NAME", "TOOL_NAMES", "build_tool_server"]
|
||||||
|
|
||||||
|
_log = logging.getLogger("beaver_gateway.core.gateway_tools")
|
||||||
|
|
||||||
|
SERVER_NAME = "gateway"
|
||||||
|
SAY_IN_USER_TURN = (
|
||||||
|
"not delivered: this turn was started by the human's message, so your "
|
||||||
|
"reply text reaches them by itself - put what you wanted to say into the "
|
||||||
|
"reply instead of repeating it here"
|
||||||
|
)
|
||||||
|
TOOL_NAMES = ("read_conversation", "spawn", "say", "schedule", "inject", "close_chat")
|
||||||
|
|
||||||
|
|
||||||
|
def build_tool_server(
|
||||||
|
conversations: Conversations, *, conversation_key: str, names: Iterable[str]
|
||||||
|
) -> McpSdkServerConfig | None:
|
||||||
|
wanted = set(names)
|
||||||
|
unknown = wanted - set(TOOL_NAMES)
|
||||||
|
if unknown:
|
||||||
|
msg = f"unknown gateway tools: {sorted(unknown)}"
|
||||||
|
raise ValueError(msg)
|
||||||
|
tools = [
|
||||||
|
_redacting(t)
|
||||||
|
for t in _tools(conversations, conversation_key)
|
||||||
|
if t.name in wanted
|
||||||
|
]
|
||||||
|
if not tools:
|
||||||
|
return None
|
||||||
|
server = create_sdk_mcp_server(SERVER_NAME, tools=tools)
|
||||||
|
return cast("McpSdkServerConfig", {**server, "alwaysLoad": True})
|
||||||
|
|
||||||
|
|
||||||
|
def _redacting(spec: SdkMcpTool[Any]) -> SdkMcpTool[Any]:
|
||||||
|
"""Put a tool's result through the same mask as every other MCP.
|
||||||
|
|
||||||
|
These tools are mounted in-process by the SDK, so they bypass the
|
||||||
|
``FastMCP`` middleware in :mod:`beaver_gateway.mcp.redacting` and
|
||||||
|
need the filter attached here instead. ``read_conversation`` is the
|
||||||
|
one that earns it: it replays a transcript, and a transcript written
|
||||||
|
before any of this existed can still hold a credential.
|
||||||
|
"""
|
||||||
|
inner = spec.handler
|
||||||
|
|
||||||
|
async def handler(args: Any) -> dict[str, Any]:
|
||||||
|
return cast("dict[str, Any]", redact_data(await inner(args)))
|
||||||
|
|
||||||
|
return replace(spec, handler=handler)
|
||||||
|
|
||||||
|
|
||||||
|
def _tools(conversations: Conversations, key: str) -> list[SdkMcpTool[Any]]:
|
||||||
|
async def current() -> Any:
|
||||||
|
conv = await conversations.get(key)
|
||||||
|
if conv is None:
|
||||||
|
msg = f"conversation {key} not found"
|
||||||
|
raise LookupError(msg)
|
||||||
|
return conv
|
||||||
|
|
||||||
|
async def target(raw: Any) -> Any:
|
||||||
|
conv = await conversations.resolve(str(raw), origin=await current())
|
||||||
|
if conv is None:
|
||||||
|
_log.warning("gateway tool in %s: conversation %r not found", key, raw)
|
||||||
|
return conv
|
||||||
|
|
||||||
|
@tool(
|
||||||
|
"read_conversation",
|
||||||
|
"Read another conversation (a branch, the master, a deep chat) as plain "
|
||||||
|
"text. `window` limits it to the last N user turns.",
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"id": {"type": "string", "description": TARGET_HELP},
|
||||||
|
"window": {"type": "integer", "minimum": 1},
|
||||||
|
},
|
||||||
|
"required": ["id"],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
async def read_conversation(args: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
conv = await target(args["id"])
|
||||||
|
if conv is None:
|
||||||
|
return _error(f"conversation {args['id']} not found ({TARGET_HELP})")
|
||||||
|
text = await conversations.read(conv, window=args.get("window"))
|
||||||
|
return _text(text or "(empty)")
|
||||||
|
|
||||||
|
@tool(
|
||||||
|
"spawn",
|
||||||
|
"Open a new conversation of the given kind (branch = your own thread, "
|
||||||
|
"deep = a long research chat, job = a headless task). `seed` is how it "
|
||||||
|
"starts: clean (nothing), morning (handout), copy (copy of this "
|
||||||
|
"conversation, last `window` turns), brief (your `text`). A branch "
|
||||||
|
"keeps your agent, other kinds get their frontend's default unless "
|
||||||
|
"`agent` says otherwise. Returns the id.",
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"kind": {"type": "string", "enum": ["branch", "deep", "job"]},
|
||||||
|
"agent": {"type": "string", "description": "agent name"},
|
||||||
|
"seed": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": ["clean", "morning", "copy", "brief"],
|
||||||
|
"default": "clean",
|
||||||
|
},
|
||||||
|
"text": {"type": "string", "description": "brief for seed=brief"},
|
||||||
|
"title": {"type": "string"},
|
||||||
|
"window": {"type": "integer", "minimum": 1},
|
||||||
|
},
|
||||||
|
"required": ["kind"],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
async def spawn(args: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
parent = await current()
|
||||||
|
try:
|
||||||
|
child = await conversations.spawn(
|
||||||
|
kind=as_kind(str(args["kind"])),
|
||||||
|
agent=args.get("agent"),
|
||||||
|
seed=str(args.get("seed") or "clean"),
|
||||||
|
parent=parent,
|
||||||
|
text=args.get("text"),
|
||||||
|
title=args.get("title"),
|
||||||
|
window=args.get("window"),
|
||||||
|
origin="mcp",
|
||||||
|
)
|
||||||
|
except (ValueError, LookupError) as exc:
|
||||||
|
return _error(str(exc))
|
||||||
|
return _text(f"spawned {child.kind} {child.external_id}")
|
||||||
|
|
||||||
|
@tool(
|
||||||
|
"say",
|
||||||
|
"Say something to the human in the frontend this conversation is bound "
|
||||||
|
"to. The only way an inject-started turn can speak; silence is simply "
|
||||||
|
"not calling it. Refused in a turn started by the human's own message: "
|
||||||
|
"there your reply text reaches them by itself, so just write the reply.",
|
||||||
|
{"text": str},
|
||||||
|
)
|
||||||
|
async def say(args: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
conv = await current()
|
||||||
|
if conversations.turn_origin(conv) == "user":
|
||||||
|
return _error(SAY_IN_USER_TURN)
|
||||||
|
await conversations.say(conv, str(args["text"]))
|
||||||
|
return _text("ok")
|
||||||
|
|
||||||
|
@tool(
|
||||||
|
"schedule",
|
||||||
|
"Promise yourself an inject later: `at` is `+15m`, `+2h`, `+1d` or an "
|
||||||
|
"ISO datetime; `text` arrives in this conversation at that time and "
|
||||||
|
"starts a turn (urgency wake). " + URGENCY_HELP + ".",
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"at": {"type": "string"},
|
||||||
|
"text": {"type": "string"},
|
||||||
|
"urgency": {"type": "string", "enum": list(URGENCY), "default": "wake"},
|
||||||
|
},
|
||||||
|
"required": ["at", "text"],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
async def schedule(args: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
conv = await current()
|
||||||
|
try:
|
||||||
|
job_id, when = await conversations.schedule(
|
||||||
|
conv,
|
||||||
|
str(args["at"]),
|
||||||
|
str(args["text"]),
|
||||||
|
urgency=cast("Any", args.get("urgency") or "wake"),
|
||||||
|
)
|
||||||
|
except (RuntimeError, ValueError) as exc:
|
||||||
|
return _error(str(exc))
|
||||||
|
return _text(f"scheduled #{job_id} at {when.isoformat(timespec='minutes')}")
|
||||||
|
|
||||||
|
@tool(
|
||||||
|
"inject",
|
||||||
|
"Put a system-origin message into another conversation's queue. "
|
||||||
|
+ URGENCY_HELP
|
||||||
|
+ ". An error here means nothing was delivered.",
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"conversation": {"type": "string", "description": TARGET_HELP},
|
||||||
|
"text": {"type": "string"},
|
||||||
|
"urgency": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": list(URGENCY),
|
||||||
|
"default": "normal",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"required": ["conversation", "text"],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
async def inject(args: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
conv = await target(args["conversation"])
|
||||||
|
if conv is None:
|
||||||
|
return _error(
|
||||||
|
f"conversation {args['conversation']} not found ({TARGET_HELP})"
|
||||||
|
)
|
||||||
|
item = await conversations.inject(
|
||||||
|
conv,
|
||||||
|
str(args["text"]),
|
||||||
|
urgency=cast("Any", args.get("urgency") or "normal"),
|
||||||
|
origin="агент",
|
||||||
|
)
|
||||||
|
return _text(f"queued #{item.id}")
|
||||||
|
|
||||||
|
@tool(
|
||||||
|
"close_chat",
|
||||||
|
"Close this deep chat once the current reply is finished: the "
|
||||||
|
"distiller forks it, writes the digest and hands a short merge to the "
|
||||||
|
"master. Call it when the human says the discussion is over; the chat "
|
||||||
|
"file stays as it is.",
|
||||||
|
{"type": "object", "properties": {}},
|
||||||
|
)
|
||||||
|
async def close_chat(_args: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
conv = await current()
|
||||||
|
try:
|
||||||
|
await conversations.request_close(conv)
|
||||||
|
except ValueError as exc:
|
||||||
|
return _error(str(exc))
|
||||||
|
return _text("ok: the chat closes after this reply")
|
||||||
|
|
||||||
|
return [read_conversation, spawn, say, schedule, inject, close_chat]
|
||||||
|
|
||||||
|
|
||||||
|
def _text(text: str) -> dict[str, Any]:
|
||||||
|
return {"content": [{"type": "text", "text": text}]}
|
||||||
|
|
||||||
|
|
||||||
|
def _error(text: str) -> dict[str, Any]:
|
||||||
|
return {"content": [{"type": "text", "text": text}], "is_error": True}
|
||||||
@@ -0,0 +1,216 @@
|
|||||||
|
"""Persisted per-conversation queue, ``urgent > user > wake > normal`` (§3.4).
|
||||||
|
|
||||||
|
One ``ClaudeSDKClient`` runs one turn at a time, so ordering has to happen
|
||||||
|
before the client: the rows here are the queue, ``core/conversations``
|
||||||
|
runs one worker per conversation over them. ``urgent`` cuts a running
|
||||||
|
turn, ``user`` is the human, ``wake`` starts a turn as soon as the
|
||||||
|
conversation is idle and takes the queued normals with it, ``normal``
|
||||||
|
waits for the batching window or rides with the next turn. A row that is
|
||||||
|
still ``running`` when the gateway starts was cut by a restart; it is
|
||||||
|
flagged ``interrupted`` and never re-run.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from typing import TYPE_CHECKING, Literal, cast
|
||||||
|
|
||||||
|
from sqlalchemy import func
|
||||||
|
from sqlmodel import col, select
|
||||||
|
|
||||||
|
from beaver_gateway.storage.models import InjectQueueItem
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Iterable, Sequence
|
||||||
|
|
||||||
|
from beaver_gateway.storage.db import Database
|
||||||
|
from beaver_gateway.storage.models import Conversation
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"PRIORITY_RANK",
|
||||||
|
"URGENCY",
|
||||||
|
"InjectContext",
|
||||||
|
"InjectQueue",
|
||||||
|
"Priority",
|
||||||
|
"inject_header",
|
||||||
|
]
|
||||||
|
|
||||||
|
Priority = Literal["urgent", "user", "wake", "normal"]
|
||||||
|
PRIORITY_RANK: dict[str, int] = {"urgent": 0, "user": 1, "wake": 2, "normal": 3}
|
||||||
|
URGENCY: tuple[Priority, ...] = ("normal", "wake", "urgent")
|
||||||
|
"""What the API and the tools accept for ``urgency``: every priority but ``user``."""
|
||||||
|
|
||||||
|
INTERRUPTED_TURN = (
|
||||||
|
"[этот инжект прервал предыдущий тёрн: «Request interrupted» выше - "
|
||||||
|
"прерывание, не отказ от тулзы]"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class InjectContext:
|
||||||
|
"""What a header renderer knows about one queued inject."""
|
||||||
|
|
||||||
|
origin: str
|
||||||
|
priority: str
|
||||||
|
interrupted_turn: bool
|
||||||
|
|
||||||
|
|
||||||
|
def inject_header(ctx: InjectContext) -> str:
|
||||||
|
"""Default framing; a setup overrides it via ``ConversationTexts.inject_header``."""
|
||||||
|
head = (
|
||||||
|
f"[инжект: {ctx.origin} - это не Бобёр, отвечать не нужно, голос не обязателен]"
|
||||||
|
)
|
||||||
|
return f"{head}\n{INTERRUPTED_TURN}" if ctx.interrupted_turn else head
|
||||||
|
|
||||||
|
|
||||||
|
def context_of(item: InjectQueueItem) -> InjectContext:
|
||||||
|
return InjectContext(
|
||||||
|
origin=item.origin,
|
||||||
|
priority=item.priority,
|
||||||
|
interrupted_turn=bool(item.interrupted_turn),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class InjectQueue:
|
||||||
|
def __init__(self, db: Database) -> None:
|
||||||
|
self._db = db
|
||||||
|
|
||||||
|
async def push(
|
||||||
|
self, *, conversation_id: int, priority: Priority, origin: str, text: str
|
||||||
|
) -> InjectQueueItem:
|
||||||
|
if priority not in PRIORITY_RANK:
|
||||||
|
msg = f"unknown priority {priority!r}"
|
||||||
|
raise ValueError(msg)
|
||||||
|
row = InjectQueueItem(
|
||||||
|
conversation_id=conversation_id, priority=priority, origin=origin, text=text
|
||||||
|
)
|
||||||
|
async with self._db.session() as session:
|
||||||
|
session.add(row)
|
||||||
|
await session.commit()
|
||||||
|
await session.refresh(row)
|
||||||
|
return row
|
||||||
|
|
||||||
|
async def pending(self, conversation_id: int) -> list[InjectQueueItem]:
|
||||||
|
async with self._db.session() as session:
|
||||||
|
result = await session.exec(
|
||||||
|
select(InjectQueueItem)
|
||||||
|
.where(
|
||||||
|
InjectQueueItem.conversation_id == conversation_id,
|
||||||
|
InjectQueueItem.status == "queued",
|
||||||
|
)
|
||||||
|
.order_by(col(InjectQueueItem.created_at), col(InjectQueueItem.id))
|
||||||
|
)
|
||||||
|
rows = list(result.all())
|
||||||
|
rows.sort(key=lambda r: (PRIORITY_RANK.get(r.priority, 9), r.created_at))
|
||||||
|
return rows
|
||||||
|
|
||||||
|
async def conversations_with_pending(self) -> list[int]:
|
||||||
|
async with self._db.session() as session:
|
||||||
|
result = await session.exec(
|
||||||
|
select(InjectQueueItem.conversation_id)
|
||||||
|
.where(InjectQueueItem.status == "queued")
|
||||||
|
.distinct()
|
||||||
|
)
|
||||||
|
return list(result.all())
|
||||||
|
|
||||||
|
async def start(self, items: Iterable[InjectQueueItem], turn_id: str) -> None:
|
||||||
|
await self._mark(items, status="running", turn_id=turn_id, delivered=True)
|
||||||
|
|
||||||
|
async def finish(
|
||||||
|
self, items: Iterable[InjectQueueItem], status: str = "done"
|
||||||
|
) -> None:
|
||||||
|
await self._mark(items, status=status)
|
||||||
|
|
||||||
|
async def mark_interrupting(self, item: InjectQueueItem) -> None:
|
||||||
|
item.interrupted_turn = True
|
||||||
|
async with self._db.session() as session:
|
||||||
|
row = await session.get(InjectQueueItem, item.id)
|
||||||
|
if row is not None:
|
||||||
|
row.interrupted_turn = True
|
||||||
|
session.add(row)
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
async def interrupted(self) -> Sequence[InjectQueueItem]:
|
||||||
|
async with self._db.session() as session:
|
||||||
|
result = await session.exec(
|
||||||
|
select(InjectQueueItem).where(InjectQueueItem.status == "running")
|
||||||
|
)
|
||||||
|
rows = list(result.all())
|
||||||
|
for row in rows:
|
||||||
|
row.status = "interrupted"
|
||||||
|
session.add(row)
|
||||||
|
await session.commit()
|
||||||
|
return rows
|
||||||
|
|
||||||
|
async def move(
|
||||||
|
self, source: Conversation, target: Conversation, *, priority: Priority
|
||||||
|
) -> int:
|
||||||
|
"""Re-home queued items of one priority (rotation carries normal over)."""
|
||||||
|
async with self._db.session() as session:
|
||||||
|
result = await session.exec(
|
||||||
|
select(InjectQueueItem).where(
|
||||||
|
InjectQueueItem.conversation_id == source.id,
|
||||||
|
InjectQueueItem.status == "queued",
|
||||||
|
InjectQueueItem.priority == priority,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
rows = list(result.all())
|
||||||
|
for row in rows:
|
||||||
|
row.conversation_id = cast("int", target.id)
|
||||||
|
session.add(row)
|
||||||
|
await session.commit()
|
||||||
|
return len(rows)
|
||||||
|
|
||||||
|
async def recent(
|
||||||
|
self, conversation_id: int, *, limit: int = 50
|
||||||
|
) -> list[InjectQueueItem]:
|
||||||
|
async with self._db.session() as session:
|
||||||
|
result = await session.exec(
|
||||||
|
select(InjectQueueItem)
|
||||||
|
.where(InjectQueueItem.conversation_id == conversation_id)
|
||||||
|
.order_by(col(InjectQueueItem.id).desc())
|
||||||
|
.limit(limit)
|
||||||
|
)
|
||||||
|
return list(result.all())
|
||||||
|
|
||||||
|
async def latest(
|
||||||
|
self, conversation_ids: Iterable[int]
|
||||||
|
) -> dict[int, InjectQueueItem]:
|
||||||
|
ids = list(conversation_ids)
|
||||||
|
if not ids:
|
||||||
|
return {}
|
||||||
|
newest = (
|
||||||
|
select(func.max(InjectQueueItem.id))
|
||||||
|
.where(col(InjectQueueItem.conversation_id).in_(ids))
|
||||||
|
.group_by(col(InjectQueueItem.conversation_id))
|
||||||
|
)
|
||||||
|
async with self._db.session() as session:
|
||||||
|
result = await session.exec(
|
||||||
|
select(InjectQueueItem).where(col(InjectQueueItem.id).in_(newest))
|
||||||
|
)
|
||||||
|
return {row.conversation_id: row for row in result.all()}
|
||||||
|
|
||||||
|
async def _mark(
|
||||||
|
self,
|
||||||
|
items: Iterable[InjectQueueItem],
|
||||||
|
*,
|
||||||
|
status: str,
|
||||||
|
turn_id: str | None = None,
|
||||||
|
delivered: bool = False,
|
||||||
|
) -> None:
|
||||||
|
ids = [i.id for i in items if i.id is not None]
|
||||||
|
if not ids:
|
||||||
|
return
|
||||||
|
async with self._db.session() as session:
|
||||||
|
result = await session.exec(
|
||||||
|
select(InjectQueueItem).where(col(InjectQueueItem.id).in_(ids))
|
||||||
|
)
|
||||||
|
for row in result.all():
|
||||||
|
row.status = status
|
||||||
|
if turn_id is not None:
|
||||||
|
row.turn_id = turn_id
|
||||||
|
if delivered:
|
||||||
|
row.delivered_at = datetime.now(UTC)
|
||||||
|
session.add(row)
|
||||||
|
await session.commit()
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
"""Conversation kinds (§3.1) as one closed type for agents, frontends, service."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Literal, get_args
|
||||||
|
|
||||||
|
__all__ = ["KINDS", "Kind", "as_kind"]
|
||||||
|
|
||||||
|
Kind = Literal["master", "branch", "deep", "job", "fork"]
|
||||||
|
KINDS: tuple[Kind, ...] = get_args(Kind)
|
||||||
|
|
||||||
|
|
||||||
|
def as_kind(value: str) -> Kind:
|
||||||
|
for kind in KINDS:
|
||||||
|
if value == kind:
|
||||||
|
return kind
|
||||||
|
msg = f"unknown conversation kind {value!r}"
|
||||||
|
raise ValueError(msg)
|
||||||
@@ -0,0 +1,174 @@
|
|||||||
|
"""PreToolUse policy - the boundary without permission prompts (§3.7).
|
||||||
|
|
||||||
|
``bypassPermissions`` everywhere; what a model may do is decided by
|
||||||
|
mounts, ``disallowed_tools`` and the rules here. A rule is a callable
|
||||||
|
``(ToolCall) -> Deny | None`` declared per agent (``ClaudeAgent.policy``);
|
||||||
|
the SDK backend registers one in-process ``PreToolUse`` hook that runs
|
||||||
|
the rules in order and turns the first :class:`Deny` into a hook deny
|
||||||
|
whose reason the model reads as the tool result. Rules never see
|
||||||
|
secrets and never prompt - they only say no, with a reason.
|
||||||
|
|
||||||
|
A rule that raises is a deny too: the boundary fails closed, the
|
||||||
|
traceback lands in the log.
|
||||||
|
|
||||||
|
Every tool call - allowed or denied - is reported to the audit sink the
|
||||||
|
backend was given, so the admin audit page shows what the model touched.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import inspect
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
from collections.abc import Awaitable, Callable
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import TYPE_CHECKING, Any, Literal
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Iterable, Mapping, MutableMapping
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"AuditSink",
|
||||||
|
"Decision",
|
||||||
|
"Deny",
|
||||||
|
"PolicyRule",
|
||||||
|
"ToolAudit",
|
||||||
|
"ToolCall",
|
||||||
|
"brief",
|
||||||
|
"evaluate",
|
||||||
|
"hook_output",
|
||||||
|
]
|
||||||
|
|
||||||
|
_log = logging.getLogger("beaver_gateway.policy")
|
||||||
|
|
||||||
|
PATH_KEYS: tuple[str, ...] = ("file_path", "notebook_path", "path")
|
||||||
|
"""Input keys built-in file tools use for their target."""
|
||||||
|
|
||||||
|
Decision = Literal["allow", "deny"]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class Deny:
|
||||||
|
"""Refusal with the reason the model reads as the tool result."""
|
||||||
|
|
||||||
|
reason: str
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class ToolCall:
|
||||||
|
"""One tool invocation as the hook sees it.
|
||||||
|
|
||||||
|
``state`` is a mutable per-session dict rules may write to (which
|
||||||
|
skills were opened, what was already asked); it lives as long as the
|
||||||
|
claude process and is empty again after a resume.
|
||||||
|
"""
|
||||||
|
|
||||||
|
tool: str
|
||||||
|
input: Mapping[str, Any]
|
||||||
|
agent: str
|
||||||
|
kind: str
|
||||||
|
conversation: str
|
||||||
|
cwd: Path
|
||||||
|
state: MutableMapping[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def mcp(self) -> tuple[str, str] | None:
|
||||||
|
"""``(server, tool)`` for ``mcp__<server>__<tool>`` names, else ``None``."""
|
||||||
|
if not self.tool.startswith("mcp__"):
|
||||||
|
return None
|
||||||
|
_, _, rest = self.tool.partition("__")
|
||||||
|
server, sep, name = rest.partition("__")
|
||||||
|
return (server, name) if sep else (server, "")
|
||||||
|
|
||||||
|
def path(self, *keys: str) -> Path | None:
|
||||||
|
"""Target of a file tool, absolute; relative paths resolve against ``cwd``."""
|
||||||
|
for key in keys or PATH_KEYS:
|
||||||
|
raw = self.input.get(key)
|
||||||
|
if isinstance(raw, str) and raw:
|
||||||
|
return self.resolve(raw)
|
||||||
|
return None
|
||||||
|
|
||||||
|
def resolve(self, raw: str) -> Path:
|
||||||
|
path = Path(raw).expanduser()
|
||||||
|
if not path.is_absolute():
|
||||||
|
path = self.cwd / path
|
||||||
|
# ``resolve`` would follow symlinks on the gateway host, which may
|
||||||
|
# not be the model's view; normalise lexically instead.
|
||||||
|
return Path(*_normalize(path.parts))
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize(parts: Iterable[str]) -> list[str]:
|
||||||
|
out: list[str] = []
|
||||||
|
for part in parts:
|
||||||
|
if part == "..":
|
||||||
|
if len(out) > 1:
|
||||||
|
out.pop()
|
||||||
|
elif part != ".":
|
||||||
|
out.append(part)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
type PolicyRule = Callable[[ToolCall], Deny | None | Awaitable[Deny | None]]
|
||||||
|
"""A rule; sync or async. First ``Deny`` wins, ``None`` passes to the next."""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class ToolAudit:
|
||||||
|
agent: str
|
||||||
|
conversation: str
|
||||||
|
kind: str
|
||||||
|
tool: str
|
||||||
|
decision: Decision
|
||||||
|
reason: str | None
|
||||||
|
brief: str
|
||||||
|
|
||||||
|
|
||||||
|
AuditSink = "Callable[[ToolAudit], Awaitable[None]]"
|
||||||
|
|
||||||
|
|
||||||
|
async def evaluate(
|
||||||
|
rules: Iterable[Callable[[ToolCall], Any]], call: ToolCall
|
||||||
|
) -> Deny | None:
|
||||||
|
"""Run ``rules`` in order; the first deny wins, an exception is a deny."""
|
||||||
|
for rule in rules:
|
||||||
|
try:
|
||||||
|
verdict = rule(call)
|
||||||
|
if inspect.isawaitable(verdict):
|
||||||
|
verdict = await verdict
|
||||||
|
except Exception: # noqa: BLE001 - a broken rule must fail closed
|
||||||
|
name = getattr(rule, "__name__", repr(rule))
|
||||||
|
_log.exception("policy rule %s failed on %s", name, call.tool)
|
||||||
|
return Deny(reason=f"policy rule {name} failed; the call is refused")
|
||||||
|
if isinstance(verdict, Deny):
|
||||||
|
return verdict
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def hook_output(deny: Deny | None) -> dict[str, Any]:
|
||||||
|
"""``PreToolUse`` hook JSON for a verdict: empty means allow."""
|
||||||
|
if deny is None:
|
||||||
|
return {}
|
||||||
|
return {
|
||||||
|
"hookSpecificOutput": {
|
||||||
|
"hookEventName": "PreToolUse",
|
||||||
|
"permissionDecision": "deny",
|
||||||
|
"permissionDecisionReason": deny.reason,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def brief(tool_input: Mapping[str, Any], *, limit: int = 240) -> str:
|
||||||
|
"""Short, log-safe summary of a call: the path, the command, or the args."""
|
||||||
|
for key in (*PATH_KEYS, "command", "skill", "pattern", "query", "url"):
|
||||||
|
value = tool_input.get(key)
|
||||||
|
if isinstance(value, str) and value:
|
||||||
|
return _cut(value, limit)
|
||||||
|
try:
|
||||||
|
return _cut(json.dumps(tool_input, ensure_ascii=False, sort_keys=True), limit)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return _cut(repr(tool_input), limit)
|
||||||
|
|
||||||
|
|
||||||
|
def _cut(text: str, limit: int) -> str:
|
||||||
|
return text if len(text) <= limit else text[: limit - 1] + "…"
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
"""System prompt assembly from a list of source files.
|
||||||
|
|
||||||
|
The gateway holds no prompt text: an agent names its granules (paths from
|
||||||
|
``config.py``) and :func:`assemble` concatenates them in that order, so the
|
||||||
|
result is byte-for-byte identical for every session of the same agent as
|
||||||
|
long as the files are. A source is a path, or a ``(tag, path)`` pair whose
|
||||||
|
content is wrapped in ``<tag>...</tag>`` - the markup lives here, the vault
|
||||||
|
keeps plain markdown. Each granule's hash is logged at assembly so a
|
||||||
|
drifted prompt can be traced to the file that changed.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import logging
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Iterable
|
||||||
|
|
||||||
|
__all__ = ["PromptSource", "assemble"]
|
||||||
|
|
||||||
|
_log = logging.getLogger("beaver_gateway.core.prompt")
|
||||||
|
|
||||||
|
PromptSource = str | Path | tuple[str, str | Path]
|
||||||
|
|
||||||
|
|
||||||
|
def assemble(sources: Iterable[PromptSource]) -> str:
|
||||||
|
parts: list[str] = []
|
||||||
|
for source in sources:
|
||||||
|
tag, raw = source if isinstance(source, tuple) else (None, source)
|
||||||
|
path = Path(raw)
|
||||||
|
text = path.read_text(encoding="utf-8").strip()
|
||||||
|
digest = hashlib.sha256(text.encode("utf-8")).hexdigest()[:12]
|
||||||
|
_log.info(
|
||||||
|
"prompt granule %s tag=%s sha=%s bytes=%d", path, tag, digest, len(text)
|
||||||
|
)
|
||||||
|
if not text:
|
||||||
|
continue
|
||||||
|
parts.append(f"<{tag}>\n{text}\n</{tag}>" if tag else text)
|
||||||
|
return "\n\n".join(parts) + "\n"
|
||||||
@@ -0,0 +1,286 @@
|
|||||||
|
"""Recognise credentials in free text and mask them.
|
||||||
|
|
||||||
|
``docker logs`` is not a vault: the stack's stdout lands in an unrotated
|
||||||
|
json file on the host, and the model itself can page through it (the
|
||||||
|
komodo tool has ``logs`` / ``search_logs``). Nobody has to write a
|
||||||
|
careless log call for a secret to end up there — two habits do it on
|
||||||
|
their own:
|
||||||
|
|
||||||
|
* ``httpx`` logs ``HTTP Request: POST <full url>`` at ``INFO``, so every
|
||||||
|
request to an upstream that keeps its credential *in* the URL prints
|
||||||
|
the credential once per call;
|
||||||
|
* a transport error carries that same URL through the traceback, which
|
||||||
|
``_log.exception`` writes out in full.
|
||||||
|
|
||||||
|
Two of our upstreams are exactly that shape: a Google Calendar
|
||||||
|
``.../private-<token>/basic.ics`` feed and a USOS ``?key=<token>`` ical
|
||||||
|
feed, both handed to the calendar MCP as query parameters.
|
||||||
|
|
||||||
|
So: silence the HTTP clients' per-request chatter (nothing here reads
|
||||||
|
it), and run one redaction pass over every formatted record as a second
|
||||||
|
line of defence.
|
||||||
|
|
||||||
|
The same pass guards the wider channel — what an MCP tool hands back to
|
||||||
|
the model (see :mod:`beaver_gateway.mcp.redacting`). :func:`redact`
|
||||||
|
recognises three things: literal values of the secret-looking
|
||||||
|
environment variables this process was started with, credentials that
|
||||||
|
live in URLs and headers, and ``NAME: value`` assignments whose *name*
|
||||||
|
says the value is a credential. The last one is what covers a config
|
||||||
|
dump of a stack whose secrets this process never held — and it keeps
|
||||||
|
the name, masking only the value, so the dump still says what is set.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
from urllib.parse import quote, unquote
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Callable, Mapping
|
||||||
|
|
||||||
|
MASK = "<…>"
|
||||||
|
|
||||||
|
# Loggers that print request URLs at INFO. We keep their warnings.
|
||||||
|
CHATTY: tuple[str, ...] = ("httpx", "httpcore", "aiohttp.client", "urllib3")
|
||||||
|
|
||||||
|
# Loggers that bring their own handlers (uvicorn re-runs ``dictConfig``
|
||||||
|
# when a server starts), so wrapping the root formatter misses them.
|
||||||
|
# ``dictConfig`` drops a logger's handlers but keeps its filters.
|
||||||
|
OWN_HANDLERS: tuple[str, ...] = ("uvicorn", "uvicorn.access", "uvicorn.error")
|
||||||
|
|
||||||
|
# Names whose *value* is a credential — matched as a substring, so
|
||||||
|
# ``FIREFLY_PAT`` and ``T3_MAC_TOKEN`` both qualify. Used twice: to pick
|
||||||
|
# which environment variables contribute literal values, and to decide
|
||||||
|
# whether a ``NAME: value`` line in a config dump should keep its value.
|
||||||
|
# ``MCP`` is in here because every MCP URL in this stack carries its
|
||||||
|
# credential in the query string.
|
||||||
|
_SECRET_NAME = re.compile(
|
||||||
|
r"TOKEN|SECRET|KEY|PASSWORD|PASS\b|PAT\b|BEARER|CREDENTIAL"
|
||||||
|
r"|MCPS?\b|AUTH|PRIVATE|SESSION|DSN",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
_MIN_SECRET = 8
|
||||||
|
|
||||||
|
# A query parameter may sit behind a plain ``?``/``&`` or behind their
|
||||||
|
# percent-encoded twins when a whole URL is nested in another one.
|
||||||
|
_LEFT = r"(?:(?<![A-Za-z0-9])|(?<=%26)|(?<=%3F))"
|
||||||
|
# ...and runs until the next separator, encoded ``&`` included.
|
||||||
|
_VALUE = r"(?:(?!%26)[^&\s\"'<>,;)\]}])+"
|
||||||
|
|
||||||
|
|
||||||
|
# Names worth masking where they sit in a dump next to their password,
|
||||||
|
# but not worth hunting for as literals across every log line: a
|
||||||
|
# username is short and ordinary, and ``CLAUDE_RUNNER_USER=beaver-runner``
|
||||||
|
# turned into a global literal would mask half the entrypoint script.
|
||||||
|
_SECRET_IN_DUMP = re.compile(r"USER\b|LOGIN\b", re.IGNORECASE)
|
||||||
|
|
||||||
|
|
||||||
|
def is_secret_name(name: str) -> bool:
|
||||||
|
"""Whether a variable/field of this name holds a credential."""
|
||||||
|
return bool(_SECRET_NAME.search(name) or _SECRET_IN_DUMP.search(name))
|
||||||
|
|
||||||
|
|
||||||
|
def _mask_named_value(match: re.Match[str]) -> str:
|
||||||
|
"""Mask ``value`` in a ``NAME: value`` match, keeping the name.
|
||||||
|
|
||||||
|
A dump is still worth reading when it says *which* variables are
|
||||||
|
set; it is the values that must not travel. An already-masked or
|
||||||
|
empty value is left alone so the pass stays idempotent, and ``lead``
|
||||||
|
/ ``tail`` carry back whatever quoting the surrounding format used.
|
||||||
|
"""
|
||||||
|
groups = match.groupdict()
|
||||||
|
name, value = groups["name"], groups["value"]
|
||||||
|
if not is_secret_name(name) or value.strip(" \t\"'") in ("", MASK):
|
||||||
|
return match[0]
|
||||||
|
lead, tail = groups.get("lead") or "", groups.get("tail") or ""
|
||||||
|
return f"{lead}{name}{groups['sep']}{MASK}{tail}"
|
||||||
|
|
||||||
|
|
||||||
|
type _Repl = str | Callable[[re.Match[str]], str]
|
||||||
|
|
||||||
|
_RULES: tuple[tuple[re.Pattern[str], _Repl], ...] = (
|
||||||
|
# Google Calendar's secret path segment; the ``/basic.ics`` after it
|
||||||
|
# survives, encoded or not, so the line still says what it fetched.
|
||||||
|
(
|
||||||
|
re.compile(r"(private-)(?:(?!%2F)[A-Za-z0-9_%-]){8,}", re.IGNORECASE),
|
||||||
|
r"\1" + MASK,
|
||||||
|
),
|
||||||
|
# ``key=…``, ``api_key=…``, ``token=…``, … in a query string.
|
||||||
|
(
|
||||||
|
re.compile(
|
||||||
|
_LEFT
|
||||||
|
+ r"(api[-_]?key|access[-_]?token|auth[-_]?token"
|
||||||
|
+ r"|token|key|secret|password|passwd|pat|signature|sig)"
|
||||||
|
+ r"(=|%3D)"
|
||||||
|
+ _VALUE,
|
||||||
|
re.IGNORECASE,
|
||||||
|
),
|
||||||
|
r"\1\2" + MASK,
|
||||||
|
),
|
||||||
|
# ``Authorization: Bearer …`` in a header dump; scheme goes too.
|
||||||
|
(
|
||||||
|
re.compile(
|
||||||
|
r"(authorization[\"']?\s*[:=]\s*[\"']?)(?:\S+\s+)?\S+", re.IGNORECASE
|
||||||
|
),
|
||||||
|
r"\1" + MASK,
|
||||||
|
),
|
||||||
|
# A bare ``Bearer <token>`` anywhere else.
|
||||||
|
(re.compile(r"\b(bearer)\s+[A-Za-z0-9._~+/=-]{8,}", re.IGNORECASE), r"\1 " + MASK),
|
||||||
|
# ``NAME: value`` / ``NAME=value`` at the start of a line — the shape
|
||||||
|
# of a compose ``environment:`` block, an ``.env`` file, a printed
|
||||||
|
# settings object. Anchored to the line so that an ``actor=token:foo``
|
||||||
|
# in the middle of a log line doesn't swallow the rest of it, and
|
||||||
|
# ``=`` must be tight (``NAME=value``, the way env files write it) so
|
||||||
|
# that ``SESSION_SECRET = os.environ[…]`` in source the dispatcher is
|
||||||
|
# reading through t3code stays readable.
|
||||||
|
(
|
||||||
|
re.compile(
|
||||||
|
r"""(?m)^(?P<lead>[ \t]*(?:-[ \t]+)?["']?)
|
||||||
|
(?P<name>[A-Z][A-Z0-9_]{2,})
|
||||||
|
(?P<sep>["']?(?::[ \t]*|=))
|
||||||
|
(?P<value>"[^"\n]*"|'[^'\n]*'|[^\n]*)""",
|
||||||
|
re.VERBOSE,
|
||||||
|
),
|
||||||
|
_mask_named_value,
|
||||||
|
),
|
||||||
|
# The same at the start of a line in lower case, but only for names
|
||||||
|
# that are unambiguously a credential — ``_SECRET_NAME`` is too broad
|
||||||
|
# here, it would read the ``mcp:`` prefix of a log line as one.
|
||||||
|
(
|
||||||
|
re.compile(
|
||||||
|
r"""(?m)^(?P<lead>[ \t]*(?:-[ \t]+)?["']?)
|
||||||
|
(?P<name>password|passwd|secret|token|api[-_]?key|apikey
|
||||||
|
|access[-_]?token|auth[-_]?token|private[-_]?key|credentials?)
|
||||||
|
(?P<sep>["']?(?::[ \t]*|=))
|
||||||
|
(?P<value>"[^"\n]*"|'[^'\n]*'|[^\n]*)""",
|
||||||
|
re.VERBOSE | re.IGNORECASE,
|
||||||
|
),
|
||||||
|
_mask_named_value,
|
||||||
|
),
|
||||||
|
# ``scheme://user:password@host`` — the credential a connection
|
||||||
|
# string carries. Host and database stay, so the line still locates
|
||||||
|
# the service it failed to reach.
|
||||||
|
(re.compile(r"(://[^/\s:@]+:)[^@\s/]+(@)"), r"\1" + MASK + r"\2"),
|
||||||
|
# ``"name": "value"`` anywhere — the same idea for one-line JSON,
|
||||||
|
# where the quotes make the end of the value unambiguous.
|
||||||
|
(
|
||||||
|
re.compile(
|
||||||
|
r'(?P<lead>")(?P<name>[^"\n]{1,64})(?P<sep>"\s*:\s*")'
|
||||||
|
r'(?P<value>[^"\n]*)(?P<tail>")'
|
||||||
|
),
|
||||||
|
_mask_named_value,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
_ENV_SECRETS: list[str] = []
|
||||||
|
"""Literal secret values to mask, longest first. Filled by :func:`install`."""
|
||||||
|
|
||||||
|
|
||||||
|
def env_secrets(env: Mapping[str, str]) -> list[str]:
|
||||||
|
"""Literal values worth masking, longest first.
|
||||||
|
|
||||||
|
Each secret-looking variable contributes its value plus the
|
||||||
|
percent-encoded and percent-decoded spellings of it, because a
|
||||||
|
credential that travels inside another URL is logged encoded.
|
||||||
|
"""
|
||||||
|
found: set[str] = set()
|
||||||
|
for name, value in env.items():
|
||||||
|
if not _SECRET_NAME.search(name):
|
||||||
|
continue
|
||||||
|
for form in (value, quote(value, safe=""), unquote(value)):
|
||||||
|
if len(form.strip()) >= _MIN_SECRET:
|
||||||
|
found.add(form.strip())
|
||||||
|
ordered = list(found)
|
||||||
|
ordered.sort(key=len, reverse=True)
|
||||||
|
return ordered
|
||||||
|
|
||||||
|
|
||||||
|
def redact(text: str) -> str:
|
||||||
|
"""Mask every credential we can recognise in ``text``."""
|
||||||
|
for value in _ENV_SECRETS:
|
||||||
|
if value in text:
|
||||||
|
text = text.replace(value, MASK)
|
||||||
|
for pattern, repl in _RULES:
|
||||||
|
text = pattern.sub(repl, text)
|
||||||
|
return text
|
||||||
|
|
||||||
|
|
||||||
|
def redact_data(value: object) -> object:
|
||||||
|
"""Mask every string inside a decoded JSON tree.
|
||||||
|
|
||||||
|
A field name is the only context a bare value has: ``"0d1e…"`` on
|
||||||
|
its own looks like nothing, ``{"ADMIN_PASS": "0d1e…"}`` is a
|
||||||
|
password. So a secret-looking key masks its whole subtree — the
|
||||||
|
structured twin of the ``NAME: value`` rule above.
|
||||||
|
"""
|
||||||
|
if isinstance(value, str):
|
||||||
|
return redact(value)
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return {
|
||||||
|
key: MASK if is_secret_name(str(key)) and item else redact_data(item)
|
||||||
|
for key, item in value.items()
|
||||||
|
}
|
||||||
|
if isinstance(value, list):
|
||||||
|
return [redact_data(item) for item in value]
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
class RedactingFormatter(logging.Formatter):
|
||||||
|
"""Wrap another formatter and redact whatever it produced.
|
||||||
|
|
||||||
|
Formatting first is the point: it catches the message, its ``%``
|
||||||
|
arguments and the traceback of an ``exception()`` call in one pass,
|
||||||
|
which a record-level filter cannot do.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, inner: logging.Formatter | None = None) -> None:
|
||||||
|
super().__init__()
|
||||||
|
self._inner = inner if inner is not None else logging.Formatter()
|
||||||
|
|
||||||
|
def format(self, record: logging.LogRecord) -> str:
|
||||||
|
return redact(self._inner.format(record))
|
||||||
|
|
||||||
|
|
||||||
|
class RedactFilter(logging.Filter):
|
||||||
|
"""Redact at record level, for loggers whose handlers aren't ours.
|
||||||
|
|
||||||
|
Interpolates first and replaces the record with the result, because
|
||||||
|
a secret is often split across the template and its arguments
|
||||||
|
(``"key=%s", token``) and neither half looks like a credential on
|
||||||
|
its own. The arguments are dropped only when something was actually
|
||||||
|
masked, so untouched records stay structured.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def filter(self, record: logging.LogRecord) -> bool:
|
||||||
|
try:
|
||||||
|
message = record.getMessage()
|
||||||
|
except (TypeError, ValueError): # a broken template is not ours to fix
|
||||||
|
return True
|
||||||
|
masked = redact(message)
|
||||||
|
if masked != message:
|
||||||
|
record.msg = masked
|
||||||
|
record.args = ()
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def load_secrets(env: Mapping[str, str] | None = None) -> None:
|
||||||
|
"""(Re)read the literal values to mask.
|
||||||
|
|
||||||
|
Called once at startup and again after ``.env`` is loaded, because
|
||||||
|
in dev the process environment only fills up at that point.
|
||||||
|
"""
|
||||||
|
_ENV_SECRETS[:] = env_secrets(os.environ if env is None else env)
|
||||||
|
|
||||||
|
|
||||||
|
def install(env: Mapping[str, str] | None = None) -> None:
|
||||||
|
"""Wire redaction into the root logger. Call right after ``basicConfig``."""
|
||||||
|
load_secrets(env)
|
||||||
|
for handler in logging.getLogger().handlers:
|
||||||
|
handler.setFormatter(RedactingFormatter(handler.formatter))
|
||||||
|
for name in CHATTY:
|
||||||
|
logging.getLogger(name).setLevel(logging.WARNING)
|
||||||
|
for name in OWN_HANDLERS:
|
||||||
|
logging.getLogger(name).addFilter(RedactFilter())
|
||||||
@@ -13,9 +13,15 @@ from dataclasses import dataclass, field
|
|||||||
from typing import TYPE_CHECKING
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from collections.abc import Iterable, Iterator
|
from collections.abc import Awaitable, Callable, Iterable, Iterator
|
||||||
|
|
||||||
from beaver_gateway.agents.base import BaseAgent
|
from beaver_gateway.agents.base import BaseAgent
|
||||||
|
from beaver_gateway.core.conversations import ConversationTexts, UserSaid
|
||||||
|
from beaver_gateway.core.distill import Distiller
|
||||||
|
from beaver_gateway.core.envelope import RecallContext
|
||||||
|
from beaver_gateway.core.rotation import RotationPolicy
|
||||||
|
from beaver_gateway.core.scheduler import Budget, Job
|
||||||
|
from beaver_gateway.core.watch import VaultWatch
|
||||||
from beaver_gateway.frontends.base import Frontend
|
from beaver_gateway.frontends.base import Frontend
|
||||||
from beaver_gateway.mcp.types import McpServerT
|
from beaver_gateway.mcp.types import McpServerT
|
||||||
|
|
||||||
@@ -81,3 +87,31 @@ class Gateway:
|
|||||||
agents: list[BaseAgent] = field(default_factory=list)
|
agents: list[BaseAgent] = field(default_factory=list)
|
||||||
mcps: list[McpServerT] = field(default_factory=list)
|
mcps: list[McpServerT] = field(default_factory=list)
|
||||||
frontends: list[Frontend] = field(default_factory=list)
|
frontends: list[Frontend] = field(default_factory=list)
|
||||||
|
texts: ConversationTexts | None = None
|
||||||
|
"""Merge prompt and seed bodies for ``core/conversations`` (§8.2-8.3)."""
|
||||||
|
jobs: list[Job] = field(default_factory=list)
|
||||||
|
"""Cron / webhook / event jobs for ``core/scheduler`` (§3.6, §4.5)."""
|
||||||
|
rotation: RotationPolicy | None = None
|
||||||
|
"""When a master is rotated (§4.5); ``None`` keeps the defaults."""
|
||||||
|
watch: VaultWatch | None = None
|
||||||
|
"""Vault watcher feeding the envelope (§3.5, §4.6); ``None`` = no vault block."""
|
||||||
|
recall: Callable[[RecallContext], str | None] | None = None
|
||||||
|
"""Envelope lookup on the user's text: pointers into the vault (cards,
|
||||||
|
the agent's notes, due dates) the gateway knows no paths for (§3.3)."""
|
||||||
|
user_sink: Callable[[UserSaid], Awaitable[None] | None] | None = None
|
||||||
|
"""Sees every user message as it enters a master or branch turn - the
|
||||||
|
setup's own grep-able log of what the user said, outside the transcript."""
|
||||||
|
budget: Budget | None = None
|
||||||
|
"""Subscription window past which non-critical jobs wait (§4.5)."""
|
||||||
|
distiller: Distiller | None = None
|
||||||
|
"""Who closes deep chats and where the digests and the index live (§8.4)."""
|
||||||
|
tz: str = "UTC"
|
||||||
|
"""Local zone for the envelope clock and the rotation hour."""
|
||||||
|
host: str = "0.0.0.0" # noqa: S104
|
||||||
|
port: int = 8000
|
||||||
|
"""The one listener; every HTTP frontend is mounted under its ``path``."""
|
||||||
|
public_url: str | None = None
|
||||||
|
"""Origin the reverse proxy shows the world (``https://b.example.com``).
|
||||||
|
|
||||||
|
Advertised endpoints and MCP discovery are built on it; ``None``
|
||||||
|
derives the origin from each request."""
|
||||||
@@ -0,0 +1,131 @@
|
|||||||
|
"""Master rotation (§4.5, §8.1, §8.3).
|
||||||
|
|
||||||
|
One logical master thread, many physical sessions: when the policy says
|
||||||
|
so, a new master is spawned and takes over the window atomically, the old
|
||||||
|
one writes its handout as its last turn, closes, its finished branches get
|
||||||
|
marked in their windows, its queued normal injects move over, and the new
|
||||||
|
one receives "new day". Silence is measured by the user's messages only -
|
||||||
|
injects never extend a day.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import UTC, date, datetime, timedelta
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
from zoneinfo import ZoneInfo
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from beaver_gateway.core.conversations import Conversations
|
||||||
|
from beaver_gateway.storage.models import Conversation
|
||||||
|
|
||||||
|
__all__ = ["HandoutContext", "Rotation", "RotationPolicy"]
|
||||||
|
|
||||||
|
_log = logging.getLogger("beaver_gateway.core.rotation")
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class RotationPolicy:
|
||||||
|
tz: str = "UTC"
|
||||||
|
hour: int = 4
|
||||||
|
night_silence: timedelta = timedelta(hours=3)
|
||||||
|
max_age: timedelta = timedelta(hours=36)
|
||||||
|
short_silence: timedelta = timedelta(minutes=30)
|
||||||
|
max_context_tokens: int = 80_000
|
||||||
|
|
||||||
|
def reason(
|
||||||
|
self, master: Conversation, *, now: datetime, context_tokens: int
|
||||||
|
) -> str | None:
|
||||||
|
zone = ZoneInfo(self.tz)
|
||||||
|
created = _aware(master.created_at)
|
||||||
|
silence = now - _aware(master.last_user_activity_at or master.created_at)
|
||||||
|
boundary = now.astimezone(zone).replace(
|
||||||
|
hour=self.hour, minute=0, second=0, microsecond=0
|
||||||
|
)
|
||||||
|
if now.astimezone(zone) < boundary:
|
||||||
|
boundary -= timedelta(days=1)
|
||||||
|
if created < boundary and silence > self.night_silence:
|
||||||
|
return "ночь"
|
||||||
|
if now - created > self.max_age and silence > self.short_silence:
|
||||||
|
return "возраст"
|
||||||
|
if context_tokens > self.max_context_tokens and silence > self.short_silence:
|
||||||
|
return "транскрипт"
|
||||||
|
return None
|
||||||
|
|
||||||
|
def day_of(self, master: Conversation) -> date:
|
||||||
|
return _aware(master.created_at).astimezone(ZoneInfo(self.tz)).date()
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class HandoutContext:
|
||||||
|
day: date
|
||||||
|
master: Conversation
|
||||||
|
reason: str
|
||||||
|
|
||||||
|
|
||||||
|
class Rotation:
|
||||||
|
def __init__(
|
||||||
|
self, conversations: Conversations, policy: RotationPolicy | None = None
|
||||||
|
) -> None:
|
||||||
|
self._conversations = conversations
|
||||||
|
self.policy = policy or RotationPolicy()
|
||||||
|
|
||||||
|
async def due(self, now: datetime | None = None) -> list[tuple[Conversation, str]]:
|
||||||
|
now = now or datetime.now(UTC)
|
||||||
|
out: list[tuple[Conversation, str]] = []
|
||||||
|
for master in await self._conversations.find(kind="master", status="open"):
|
||||||
|
tokens = await self._conversations.context_tokens(master)
|
||||||
|
reason = self.policy.reason(master, now=now, context_tokens=tokens)
|
||||||
|
if reason is not None:
|
||||||
|
out.append((master, reason))
|
||||||
|
return out
|
||||||
|
|
||||||
|
async def tick(self, now: datetime | None = None) -> list[Conversation]:
|
||||||
|
rotated: list[Conversation] = []
|
||||||
|
for master, reason in await self.due(now):
|
||||||
|
new = await self.rotate(master, reason)
|
||||||
|
if new is not None:
|
||||||
|
rotated.append(new)
|
||||||
|
return rotated
|
||||||
|
|
||||||
|
async def rotate(self, old: Conversation, reason: str) -> Conversation | None:
|
||||||
|
conversations = self._conversations
|
||||||
|
if await conversations.busy(old):
|
||||||
|
_log.info("rotation of %s skipped: busy", old.external_id)
|
||||||
|
return None
|
||||||
|
new = await conversations.spawn(
|
||||||
|
kind="master", agent=old.agent_name, seed="morning", origin="rotation"
|
||||||
|
)
|
||||||
|
day = self.policy.day_of(old)
|
||||||
|
_log.info(
|
||||||
|
"rotation (%s): %s -> %s, handout for %s",
|
||||||
|
reason,
|
||||||
|
old.external_id,
|
||||||
|
new.external_id,
|
||||||
|
day,
|
||||||
|
)
|
||||||
|
await conversations.handout(
|
||||||
|
old, HandoutContext(day=day, master=old, reason=reason)
|
||||||
|
)
|
||||||
|
await conversations.close(old)
|
||||||
|
for branch in await conversations.find(parent=old, limit=1000):
|
||||||
|
if branch.status == "open":
|
||||||
|
await conversations.reparent(branch, new)
|
||||||
|
elif not branch.running_turn:
|
||||||
|
await conversations.mark_closed(branch)
|
||||||
|
moved = await conversations.queue.move(old, new, priority="normal")
|
||||||
|
await conversations.new_day(new, reason=reason, moved=moved)
|
||||||
|
conversations.bus.publish(
|
||||||
|
"conversation.rotated",
|
||||||
|
conversation_id=new.external_id,
|
||||||
|
closed=old.external_id,
|
||||||
|
reason=reason,
|
||||||
|
handout_day=day.isoformat(),
|
||||||
|
moved=moved,
|
||||||
|
)
|
||||||
|
return new
|
||||||
|
|
||||||
|
|
||||||
|
def _aware(value: datetime) -> datetime:
|
||||||
|
return value if value.tzinfo is not None else value.replace(tzinfo=UTC)
|
||||||
@@ -0,0 +1,526 @@
|
|||||||
|
"""Jobs and deferred injects on pgqueuer (§3.6, §4.5).
|
||||||
|
|
||||||
|
A job is a name, a handler and its triggers: a cron expression, the
|
||||||
|
webhook ``/hooks/<name>``, gateway bus events. The executor is pgqueuer on
|
||||||
|
the gateway's own Postgres (a dedicated autocommit connection for
|
||||||
|
LISTEN/NOTIFY), so cron ticks, webhook deliveries and the ``schedule``
|
||||||
|
tool's one-off injects all live in one table and survive a restart. A
|
||||||
|
handler only queues work for a conversation and returns; the turn itself
|
||||||
|
runs in the conversation's worker. Non-critical jobs step aside while the
|
||||||
|
subscription window is past its threshold.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import contextlib
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import UTC, datetime, timedelta
|
||||||
|
from typing import TYPE_CHECKING, Any, cast
|
||||||
|
|
||||||
|
from pgqueuer import PgQueuer, Queries
|
||||||
|
from pgqueuer.models import JobId
|
||||||
|
from starlette.applications import Starlette
|
||||||
|
from starlette.responses import JSONResponse
|
||||||
|
from starlette.routing import Route
|
||||||
|
|
||||||
|
from beaver_gateway.core.conversations import parse_at
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Awaitable, Callable, Coroutine, Sequence
|
||||||
|
|
||||||
|
from pgqueuer.models import Job as PgJob
|
||||||
|
from pgqueuer.models import Schedule as PgSchedule
|
||||||
|
from pgqueuer.ports.driver import Driver
|
||||||
|
from starlette.requests import Request
|
||||||
|
|
||||||
|
from beaver_gateway.core.conversations import Conversations, DistillResult
|
||||||
|
from beaver_gateway.core.distill import LineCap
|
||||||
|
from beaver_gateway.core.injects import Priority
|
||||||
|
from beaver_gateway.core.rotation import Rotation
|
||||||
|
from beaver_gateway.storage.models import Conversation
|
||||||
|
|
||||||
|
__all__ = ["INJECT", "Budget", "Job", "JobRun", "Scheduler"]
|
||||||
|
|
||||||
|
_log = logging.getLogger("beaver_gateway.core.scheduler")
|
||||||
|
|
||||||
|
INJECT = "inject"
|
||||||
|
RETRY = timedelta(minutes=15)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class Job:
|
||||||
|
name: str
|
||||||
|
run: Callable[[JobRun], Awaitable[None]]
|
||||||
|
cron: str | None = None
|
||||||
|
webhook: bool = False
|
||||||
|
events: tuple[str, ...] = ()
|
||||||
|
critical: bool = True
|
||||||
|
dedupe: bool = True
|
||||||
|
|
||||||
|
@property
|
||||||
|
def entrypoint(self) -> str:
|
||||||
|
return f"job:{self.name}"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class Budget:
|
||||||
|
threshold: float = 0.7
|
||||||
|
tokens: int | None = None
|
||||||
|
window: timedelta = timedelta(hours=5)
|
||||||
|
limit_window: str = "five_hour"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(slots=True)
|
||||||
|
class JobRun:
|
||||||
|
job: Job
|
||||||
|
trigger: str
|
||||||
|
payload: dict[str, Any]
|
||||||
|
scheduler: Scheduler
|
||||||
|
|
||||||
|
@property
|
||||||
|
def conversations(self) -> Conversations:
|
||||||
|
return self.scheduler.conversations
|
||||||
|
|
||||||
|
async def master(self) -> Conversation | None:
|
||||||
|
masters = await self.conversations.find(kind="master", status="open", limit=1)
|
||||||
|
return masters[0] if masters else None
|
||||||
|
|
||||||
|
async def inject_master(
|
||||||
|
self, text: str, *, urgency: Priority = "normal", origin: str | None = None
|
||||||
|
) -> bool:
|
||||||
|
master = await self.master()
|
||||||
|
if master is None:
|
||||||
|
_log.error(
|
||||||
|
"job %s: no open master, inject lost: %s", self.job.name, text[:200]
|
||||||
|
)
|
||||||
|
return False
|
||||||
|
await self.conversations.inject(
|
||||||
|
master, text, urgency=urgency, origin=origin or self.job.name
|
||||||
|
)
|
||||||
|
return True
|
||||||
|
|
||||||
|
async def spawn_job(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
agent: str,
|
||||||
|
text: str,
|
||||||
|
title: str | None = None,
|
||||||
|
line_cap: LineCap | None = None,
|
||||||
|
) -> Conversation:
|
||||||
|
"""A headless job turn; ``line_cap`` bounces a rewrite past the cap."""
|
||||||
|
return await self.conversations.spawn(
|
||||||
|
kind="job",
|
||||||
|
agent=agent,
|
||||||
|
seed="brief",
|
||||||
|
text=text,
|
||||||
|
title=title,
|
||||||
|
origin="job",
|
||||||
|
flags={"line_cap": line_cap.as_flags()} if line_cap else None,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def close_idle(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
kind: str = "deep",
|
||||||
|
days: int = 2,
|
||||||
|
limit: int = 3,
|
||||||
|
since: datetime | None = None,
|
||||||
|
) -> list[DistillResult]:
|
||||||
|
"""§4.5: close chats quiet for ``days``, at most ``limit`` per run."""
|
||||||
|
out: list[DistillResult] = []
|
||||||
|
for conv in await self.conversations.idle(
|
||||||
|
kind=kind, days=days, since=since, limit=limit
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
out.append(
|
||||||
|
await self.conversations.distill(conv, reason=f"idle {days}d")
|
||||||
|
)
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
_log.exception("closing idle %s failed", conv.external_id)
|
||||||
|
return out
|
||||||
|
|
||||||
|
async def retry_in(self, delay: timedelta) -> None:
|
||||||
|
await self.scheduler.trigger(
|
||||||
|
self.job, self.payload, delay=delay, trigger=self.trigger
|
||||||
|
)
|
||||||
|
|
||||||
|
async def rotate(self) -> list[Conversation]:
|
||||||
|
rotation = self.scheduler.rotation
|
||||||
|
return await rotation.tick() if rotation is not None else []
|
||||||
|
|
||||||
|
def background(self, coro: Coroutine[Any, Any, Any]) -> None:
|
||||||
|
self.scheduler.background(coro)
|
||||||
|
|
||||||
|
|
||||||
|
class Scheduler:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
conversations: Conversations,
|
||||||
|
jobs: Sequence[Job] = (),
|
||||||
|
driver: Driver | None = None,
|
||||||
|
budget: Budget | None = None,
|
||||||
|
rotation: Rotation | None = None,
|
||||||
|
heartbeat: timedelta = timedelta(seconds=30),
|
||||||
|
) -> None:
|
||||||
|
self.conversations = conversations
|
||||||
|
self.rotation = rotation
|
||||||
|
self.budget = budget or Budget()
|
||||||
|
self._jobs = {job.name: job for job in jobs}
|
||||||
|
self._driver = driver
|
||||||
|
self._queries: Queries | None = None
|
||||||
|
self._heartbeat = heartbeat
|
||||||
|
self._tasks: set[asyncio.Task[Any]] = set()
|
||||||
|
self._runs: dict[str, dict[str, Any]] = {}
|
||||||
|
|
||||||
|
@property
|
||||||
|
def enabled(self) -> bool:
|
||||||
|
return self._queries is not None
|
||||||
|
|
||||||
|
@property
|
||||||
|
def jobs(self) -> list[Job]:
|
||||||
|
return list(self._jobs.values())
|
||||||
|
|
||||||
|
def job(self, name: str) -> Job | None:
|
||||||
|
return self._jobs.get(name)
|
||||||
|
|
||||||
|
# ---- lifecycle -----------------------------------------------------
|
||||||
|
|
||||||
|
async def start(self) -> None:
|
||||||
|
if self._driver is not None:
|
||||||
|
queries = Queries(self._driver)
|
||||||
|
if not await queries.has_table("pgqueuer"):
|
||||||
|
await queries.install()
|
||||||
|
pgq = PgQueuer(self._driver, queries=queries)
|
||||||
|
self._register(pgq)
|
||||||
|
self._queries = queries
|
||||||
|
self._spawn(pgq.run(heartbeat_timeout=self._heartbeat))
|
||||||
|
else:
|
||||||
|
_log.warning("scheduler: no postgres driver - cron and webhooks are off")
|
||||||
|
self._spawn(self._events())
|
||||||
|
|
||||||
|
async def stop(self) -> None:
|
||||||
|
for task in list(self._tasks):
|
||||||
|
task.cancel()
|
||||||
|
for task in list(self._tasks):
|
||||||
|
with contextlib.suppress(BaseException):
|
||||||
|
await task
|
||||||
|
self._tasks.clear()
|
||||||
|
self._queries = None
|
||||||
|
|
||||||
|
def _register(self, pgq: PgQueuer) -> None:
|
||||||
|
@pgq.entrypoint(INJECT)
|
||||||
|
async def deliver(job: PgJob) -> None:
|
||||||
|
await self._deliver(job)
|
||||||
|
|
||||||
|
for job in self._jobs.values():
|
||||||
|
self._register_job(pgq, job)
|
||||||
|
|
||||||
|
def _register_job(self, pgq: PgQueuer, job: Job) -> None:
|
||||||
|
@pgq.entrypoint(job.entrypoint)
|
||||||
|
async def queued(pg_job: PgJob) -> None:
|
||||||
|
data = _decode(pg_job.payload)
|
||||||
|
await self._dispatch(
|
||||||
|
job, trigger=str(data.pop("trigger", "queue")), payload=data
|
||||||
|
)
|
||||||
|
|
||||||
|
if job.cron:
|
||||||
|
|
||||||
|
@pgq.schedule(job.name, job.cron, clean_old=True)
|
||||||
|
async def cron(_schedule: PgSchedule) -> None:
|
||||||
|
await self._dispatch(job, trigger="cron", payload={})
|
||||||
|
|
||||||
|
async def _events(self) -> None:
|
||||||
|
listeners = [job for job in self._jobs.values() if job.events]
|
||||||
|
if not listeners:
|
||||||
|
return
|
||||||
|
async for event in self.conversations.bus.stream():
|
||||||
|
for job in listeners:
|
||||||
|
if event.get("type") in job.events:
|
||||||
|
self._spawn(
|
||||||
|
self._dispatch(job, trigger="event", payload=dict(event))
|
||||||
|
)
|
||||||
|
|
||||||
|
# ---- dispatch ------------------------------------------------------
|
||||||
|
|
||||||
|
async def _dispatch(
|
||||||
|
self, job: Job, *, trigger: str, payload: dict[str, Any]
|
||||||
|
) -> None:
|
||||||
|
bus = self.conversations.bus
|
||||||
|
if not job.critical and await self.throttled():
|
||||||
|
bus.publish("job.deferred", job=job.name, trigger=trigger)
|
||||||
|
if trigger != "cron":
|
||||||
|
await self.trigger(job, payload, delay=RETRY, trigger=trigger)
|
||||||
|
return
|
||||||
|
started = datetime.now(UTC)
|
||||||
|
bus.publish("job.start", job=job.name, trigger=trigger)
|
||||||
|
status = "done"
|
||||||
|
try:
|
||||||
|
await job.run(JobRun(job, trigger, payload, self))
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
status = "failed"
|
||||||
|
_log.exception("job %s (%s) failed", job.name, trigger)
|
||||||
|
self._runs[job.name] = {
|
||||||
|
"trigger": trigger,
|
||||||
|
"started_at": started.isoformat(timespec="seconds"),
|
||||||
|
"status": status,
|
||||||
|
}
|
||||||
|
bus.publish("job.end", job=job.name, trigger=trigger, status=status)
|
||||||
|
|
||||||
|
async def trigger(
|
||||||
|
self,
|
||||||
|
job: Job,
|
||||||
|
payload: dict[str, Any] | None = None,
|
||||||
|
*,
|
||||||
|
delay: timedelta | None = None,
|
||||||
|
trigger: str = "manual",
|
||||||
|
) -> int | None:
|
||||||
|
"""Queue one run of ``job``; runs inline when there is no executor."""
|
||||||
|
data = {**(payload or {}), "trigger": trigger}
|
||||||
|
if self._queries is None:
|
||||||
|
self._spawn(
|
||||||
|
self._dispatch(job, trigger=trigger, payload=payload or {}), delay=delay
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
ids = await self._queries.enqueue(
|
||||||
|
job.entrypoint, _encode(data), execute_after=delay
|
||||||
|
)
|
||||||
|
return int(ids[0]) if ids and ids[0] is not None else None
|
||||||
|
|
||||||
|
async def hook(self, name: str, payload: dict[str, Any]) -> int | None:
|
||||||
|
job = self._jobs.get(name)
|
||||||
|
if job is None or not job.webhook:
|
||||||
|
msg = f"no webhook job {name!r}"
|
||||||
|
raise LookupError(msg)
|
||||||
|
self.conversations.bus.publish("hook", job=name)
|
||||||
|
if self._queries is None:
|
||||||
|
return await self.trigger(job, payload, trigger="webhook")
|
||||||
|
ids = await self._queries.enqueue(
|
||||||
|
job.entrypoint,
|
||||||
|
_encode({**payload, "trigger": "webhook"}),
|
||||||
|
dedupe_key=f"hook:{name}" if job.dedupe else None,
|
||||||
|
on_conflict="skip" if job.dedupe else "raise",
|
||||||
|
)
|
||||||
|
return int(ids[0]) if ids and ids[0] is not None else None
|
||||||
|
|
||||||
|
# ---- deferred injects ----------------------------------------------
|
||||||
|
|
||||||
|
async def schedule(
|
||||||
|
self,
|
||||||
|
conv: Conversation,
|
||||||
|
at: str,
|
||||||
|
text: str,
|
||||||
|
*,
|
||||||
|
urgency: Priority = "wake",
|
||||||
|
dedupe_key: str | None = None,
|
||||||
|
) -> tuple[int | None, datetime]:
|
||||||
|
"""A deferred inject.
|
||||||
|
|
||||||
|
``wake`` by default: it was promised for a time, so it starts a turn
|
||||||
|
then instead of waiting for the normal window.
|
||||||
|
"""
|
||||||
|
if self._queries is None:
|
||||||
|
msg = "scheduler needs postgres; `schedule` is unavailable"
|
||||||
|
raise RuntimeError(msg)
|
||||||
|
when = parse_at(at)
|
||||||
|
delay = max(when - datetime.now(UTC), timedelta(0))
|
||||||
|
payload = _encode(
|
||||||
|
{
|
||||||
|
"conversation": conv.external_id,
|
||||||
|
"text": text,
|
||||||
|
"urgency": urgency,
|
||||||
|
"at": when.isoformat(timespec="seconds"),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
ids = await self._queries.enqueue(
|
||||||
|
INJECT,
|
||||||
|
payload,
|
||||||
|
execute_after=delay,
|
||||||
|
dedupe_key=dedupe_key,
|
||||||
|
on_conflict="skip" if dedupe_key else "raise",
|
||||||
|
)
|
||||||
|
job_id = int(ids[0]) if ids and ids[0] is not None else None
|
||||||
|
self.conversations.bus.publish(
|
||||||
|
"schedule.created",
|
||||||
|
conversation_id=conv.external_id,
|
||||||
|
job=job_id,
|
||||||
|
execute_at=when.isoformat(timespec="seconds"),
|
||||||
|
)
|
||||||
|
return job_id, when
|
||||||
|
|
||||||
|
async def cancel(self, job_id: int) -> bool:
|
||||||
|
if self._queries is None:
|
||||||
|
return False
|
||||||
|
row = await self._queries.queue_job_by_id(JobId(job_id))
|
||||||
|
if row is None:
|
||||||
|
return False
|
||||||
|
await self._queries.mark_job_as_cancelled([JobId(job_id)])
|
||||||
|
self.conversations.bus.publish("schedule.cancelled", job=job_id)
|
||||||
|
return True
|
||||||
|
|
||||||
|
async def _deliver(self, job: PgJob) -> None:
|
||||||
|
data = _decode(job.payload)
|
||||||
|
key = str(data.get("conversation", ""))
|
||||||
|
conv = await self.conversations.resolve(key)
|
||||||
|
if conv is None:
|
||||||
|
_log.error(
|
||||||
|
"scheduled inject #%s lost: conversation %r is gone: %s",
|
||||||
|
job.id,
|
||||||
|
key,
|
||||||
|
str(data.get("text", ""))[:200],
|
||||||
|
)
|
||||||
|
return
|
||||||
|
await self.conversations.inject(
|
||||||
|
conv,
|
||||||
|
str(data.get("text", "")),
|
||||||
|
urgency=cast("Priority", data.get("urgency") or "wake"),
|
||||||
|
origin="schedule",
|
||||||
|
)
|
||||||
|
|
||||||
|
# ---- budget --------------------------------------------------------
|
||||||
|
|
||||||
|
async def utilization(self) -> float | None:
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
values: list[float] = []
|
||||||
|
for row in await self.conversations.rate_limits(limit=100):
|
||||||
|
if row.window != self.budget.limit_window or row.utilization is None:
|
||||||
|
continue
|
||||||
|
fresh = now - _aware(row.ts) < self.budget.window
|
||||||
|
live = row.resets_at is None or _aware(row.resets_at) > now
|
||||||
|
if fresh and live:
|
||||||
|
values.append(float(row.utilization))
|
||||||
|
break
|
||||||
|
if self.budget.tokens:
|
||||||
|
tokens = await self.conversations.usage_tokens(now - self.budget.window)
|
||||||
|
values.append(tokens / self.budget.tokens)
|
||||||
|
return max(values) if values else None
|
||||||
|
|
||||||
|
async def throttled(self) -> bool:
|
||||||
|
utilization = await self.utilization()
|
||||||
|
return utilization is not None and utilization > self.budget.threshold
|
||||||
|
|
||||||
|
# ---- introspection -------------------------------------------------
|
||||||
|
|
||||||
|
async def snapshot(self) -> dict[str, Any]:
|
||||||
|
crons: dict[str, PgSchedule] = {}
|
||||||
|
queue: list[dict[str, Any]] = []
|
||||||
|
if self._queries is not None:
|
||||||
|
for row in await self._queries.peek_schedule():
|
||||||
|
crons[str(row.entrypoint)] = row
|
||||||
|
queue = [
|
||||||
|
_job_public(row) for row in await self._queries.browse_queue(limit=200)
|
||||||
|
]
|
||||||
|
utilization = await self.utilization()
|
||||||
|
return {
|
||||||
|
"enabled": self.enabled,
|
||||||
|
"utilization": utilization,
|
||||||
|
"throttled": utilization is not None
|
||||||
|
and utilization > self.budget.threshold,
|
||||||
|
"threshold": self.budget.threshold,
|
||||||
|
"jobs": [self._job_public(job, crons.get(job.name)) for job in self.jobs],
|
||||||
|
"queue": queue,
|
||||||
|
}
|
||||||
|
|
||||||
|
async def scheduled(self, conv: Conversation | None = None) -> list[dict[str, Any]]:
|
||||||
|
if self._queries is None:
|
||||||
|
return []
|
||||||
|
rows = await self._queries.browse_queue(limit=500, entrypoints=[INJECT])
|
||||||
|
out = [_job_public(row) for row in rows]
|
||||||
|
if conv is not None:
|
||||||
|
out = [
|
||||||
|
j for j in out if j["payload"].get("conversation") == conv.external_id
|
||||||
|
]
|
||||||
|
return out
|
||||||
|
|
||||||
|
def _job_public(self, job: Job, cron: PgSchedule | None) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"name": job.name,
|
||||||
|
"cron": job.cron,
|
||||||
|
"webhook": job.webhook,
|
||||||
|
"events": list(job.events),
|
||||||
|
"critical": job.critical,
|
||||||
|
"next_run": _iso(cron.next_run) if cron is not None else None,
|
||||||
|
"last_run": _iso(cron.last_run) if cron is not None else None,
|
||||||
|
"status": str(cron.status) if cron is not None else None,
|
||||||
|
"run": self._runs.get(job.name),
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---- http ----------------------------------------------------------
|
||||||
|
|
||||||
|
def app(self, authorize: Callable[[Request], Awaitable[Any]]) -> Starlette:
|
||||||
|
async def hook(request: Request) -> JSONResponse:
|
||||||
|
await authorize(request)
|
||||||
|
name = request.path_params["name"]
|
||||||
|
payload = await _payload(request)
|
||||||
|
try:
|
||||||
|
job_id = await self.hook(name, payload)
|
||||||
|
except LookupError as exc:
|
||||||
|
return JSONResponse({"error": str(exc)}, status_code=404)
|
||||||
|
return JSONResponse({"job": job_id, "name": name}, status_code=202)
|
||||||
|
|
||||||
|
return Starlette(routes=[Route("/{name}", hook, methods=["POST"])])
|
||||||
|
|
||||||
|
# ---- internals -----------------------------------------------------
|
||||||
|
|
||||||
|
def background(self, coro: Coroutine[Any, Any, Any]) -> None:
|
||||||
|
self._spawn(coro)
|
||||||
|
|
||||||
|
def _spawn(
|
||||||
|
self, coro: Coroutine[Any, Any, Any], *, delay: timedelta | None = None
|
||||||
|
) -> None:
|
||||||
|
async def later() -> None:
|
||||||
|
if delay is not None:
|
||||||
|
await asyncio.sleep(delay.total_seconds())
|
||||||
|
await coro
|
||||||
|
|
||||||
|
task = asyncio.create_task(later() if delay is not None else coro)
|
||||||
|
self._tasks.add(task)
|
||||||
|
task.add_done_callback(self._tasks.discard)
|
||||||
|
|
||||||
|
|
||||||
|
async def _payload(request: Request) -> dict[str, Any]:
|
||||||
|
body = await request.body()
|
||||||
|
if not body:
|
||||||
|
return dict(request.query_params)
|
||||||
|
try:
|
||||||
|
data = json.loads(body)
|
||||||
|
except ValueError:
|
||||||
|
return {"raw": body.decode("utf-8", errors="replace")}
|
||||||
|
return data if isinstance(data, dict) else {"raw": data}
|
||||||
|
|
||||||
|
|
||||||
|
def _encode(data: dict[str, Any]) -> bytes:
|
||||||
|
return json.dumps(data, ensure_ascii=False, default=str).encode("utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def _decode(payload: bytes | None) -> dict[str, Any]:
|
||||||
|
if not payload:
|
||||||
|
return {}
|
||||||
|
try:
|
||||||
|
data = json.loads(payload)
|
||||||
|
except ValueError:
|
||||||
|
return {"raw": payload.decode("utf-8", errors="replace")}
|
||||||
|
return data if isinstance(data, dict) else {"raw": data}
|
||||||
|
|
||||||
|
|
||||||
|
def _job_public(row: PgJob) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"id": int(row.id),
|
||||||
|
"entrypoint": row.entrypoint,
|
||||||
|
"status": str(row.status),
|
||||||
|
"execute_after": _iso(row.execute_after),
|
||||||
|
"created": _iso(row.created),
|
||||||
|
"attempts": row.attempts,
|
||||||
|
"payload": _decode(row.payload),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _aware(value: datetime) -> datetime:
|
||||||
|
return value if value.tzinfo is not None else value.replace(tzinfo=UTC)
|
||||||
|
|
||||||
|
|
||||||
|
def _iso(value: datetime | None) -> str | None:
|
||||||
|
return _aware(value).isoformat(timespec="seconds") if value is not None else None
|
||||||
@@ -0,0 +1,259 @@
|
|||||||
|
"""Pool of live Agent SDK sessions across every Claude agent (§3.2).
|
||||||
|
|
||||||
|
One :class:`Session` is one ``ClaudeSDKClient`` (one claude subprocess).
|
||||||
|
The pool owns the two decisions the adapters used to make on their own:
|
||||||
|
when a session is closed for idleness (TTL by conversation kind) and
|
||||||
|
which one goes when memory runs out (measured RSS of the subprocess tree
|
||||||
|
against the cgroup limit, ``max_live`` where there is no limit). Eviction
|
||||||
|
only ever picks ``idle && !running_turn && !pending_question`` sessions
|
||||||
|
that are neither pinned (the master) nor ``dirty`` (mirror gap not yet
|
||||||
|
repaired); forks and jobs go first.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import contextlib
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import time
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import TYPE_CHECKING, Any, Protocol
|
||||||
|
|
||||||
|
import psutil
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import AsyncIterator, Iterator, Mapping
|
||||||
|
|
||||||
|
__all__ = ["DEFAULT_TTL", "Session", "SessionClient", "SessionPool", "cgroup_limit"]
|
||||||
|
|
||||||
|
_log = logging.getLogger("beaver_gateway.core.sessions")
|
||||||
|
|
||||||
|
DEFAULT_TTL: Mapping[str, float | None] = {
|
||||||
|
"master": None,
|
||||||
|
"branch": 7200.0,
|
||||||
|
"deep": 1800.0,
|
||||||
|
"job": 0.0,
|
||||||
|
"fork": 0.0,
|
||||||
|
}
|
||||||
|
"""Idle seconds before a session is closed; ``None`` = never (pinned kinds)."""
|
||||||
|
|
||||||
|
_EVICT_ORDER = {"fork": 0, "job": 0, "deep": 1, "branch": 2, "master": 3}
|
||||||
|
_RSS_HEADROOM = 0.8
|
||||||
|
|
||||||
|
|
||||||
|
class SessionClient(Protocol):
|
||||||
|
async def connect(self) -> None: ...
|
||||||
|
async def query(self, prompt: str) -> None: ...
|
||||||
|
def receive_response(self) -> AsyncIterator[Any]: ...
|
||||||
|
async def interrupt(self) -> None: ...
|
||||||
|
async def disconnect(self) -> None: ...
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Session:
|
||||||
|
key: str
|
||||||
|
agent: str
|
||||||
|
kind: str
|
||||||
|
client: SessionClient
|
||||||
|
session_id: str | None
|
||||||
|
resumed: bool
|
||||||
|
pinned: bool = False
|
||||||
|
dirty: bool = False
|
||||||
|
running_turn: str | None = None
|
||||||
|
pending_question: bool = False
|
||||||
|
interrupt_requested: bool = False
|
||||||
|
lock: asyncio.Lock = field(default_factory=asyncio.Lock)
|
||||||
|
last_used: float = field(default_factory=time.monotonic)
|
||||||
|
created_at: float = field(default_factory=time.monotonic)
|
||||||
|
turns: int = 0
|
||||||
|
state: dict[str, Any] = field(default_factory=dict)
|
||||||
|
"""Scratch for policy rules (``core/policy``); dies with the process."""
|
||||||
|
|
||||||
|
@property
|
||||||
|
def busy(self) -> bool:
|
||||||
|
return self.lock.locked() or self.running_turn is not None
|
||||||
|
|
||||||
|
@property
|
||||||
|
def evictable(self) -> bool:
|
||||||
|
return not (self.pinned or self.dirty or self.busy or self.pending_question)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def pid(self) -> int | None:
|
||||||
|
transport = getattr(self.client, "_transport", None)
|
||||||
|
process = getattr(transport, "_process", None)
|
||||||
|
pid = getattr(process, "pid", None)
|
||||||
|
return pid if isinstance(pid, int) else None
|
||||||
|
|
||||||
|
|
||||||
|
class SessionPool:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
rss_limit: int | None = None,
|
||||||
|
max_live: int = 8,
|
||||||
|
ttl: Mapping[str, float | None] = DEFAULT_TTL,
|
||||||
|
reap_interval: float = 60.0,
|
||||||
|
) -> None:
|
||||||
|
self._sessions: dict[str, Session] = {}
|
||||||
|
self._rss_limit = rss_limit if rss_limit is not None else cgroup_limit()
|
||||||
|
self._max_live = max_live
|
||||||
|
self._ttl = dict(ttl)
|
||||||
|
self._reap_interval = reap_interval
|
||||||
|
|
||||||
|
def __len__(self) -> int:
|
||||||
|
return len(self._sessions)
|
||||||
|
|
||||||
|
def __iter__(self) -> Iterator[Session]:
|
||||||
|
return iter(list(self._sessions.values()))
|
||||||
|
|
||||||
|
def __contains__(self, key: object) -> bool:
|
||||||
|
return key in self._sessions
|
||||||
|
|
||||||
|
@property
|
||||||
|
def rss_limit(self) -> int | None:
|
||||||
|
return self._rss_limit
|
||||||
|
|
||||||
|
def get(self, key: str) -> Session | None:
|
||||||
|
return self._sessions.get(key)
|
||||||
|
|
||||||
|
def add(self, session: Session) -> Session:
|
||||||
|
self._sessions[session.key] = session
|
||||||
|
return session
|
||||||
|
|
||||||
|
def pop(self, key: str) -> Session | None:
|
||||||
|
return self._sessions.pop(key, None)
|
||||||
|
|
||||||
|
def rekey(self, old: str, new: str) -> Session | None:
|
||||||
|
session = self._sessions.pop(old, None)
|
||||||
|
if session is None:
|
||||||
|
return None
|
||||||
|
stale = self._sessions.pop(new, None)
|
||||||
|
session.key = new
|
||||||
|
self._sessions[new] = session
|
||||||
|
return stale if stale is not session else None
|
||||||
|
|
||||||
|
def ttl_for(self, kind: str) -> float | None:
|
||||||
|
return self._ttl.get(kind, self._ttl.get("deep"))
|
||||||
|
|
||||||
|
def rss(self) -> int:
|
||||||
|
try:
|
||||||
|
children = psutil.Process(os.getpid()).children(recursive=True)
|
||||||
|
except psutil.Error:
|
||||||
|
return 0
|
||||||
|
total = 0
|
||||||
|
for child in children:
|
||||||
|
with contextlib.suppress(psutil.Error):
|
||||||
|
total += child.memory_info().rss
|
||||||
|
return total
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def rss_of(session: Session) -> int | None:
|
||||||
|
pid = session.pid
|
||||||
|
if pid is None:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
process = psutil.Process(pid)
|
||||||
|
return process.memory_info().rss + sum(
|
||||||
|
c.memory_info().rss for c in process.children(recursive=True)
|
||||||
|
)
|
||||||
|
except psutil.Error:
|
||||||
|
return None
|
||||||
|
|
||||||
|
def over_limit(self) -> bool:
|
||||||
|
if self._rss_limit is not None:
|
||||||
|
return self.rss() > self._rss_limit * _RSS_HEADROOM
|
||||||
|
return len(self._sessions) >= self._max_live
|
||||||
|
|
||||||
|
def victims(self) -> list[Session]:
|
||||||
|
candidates = [s for s in self._sessions.values() if s.evictable]
|
||||||
|
candidates.sort(key=lambda s: (_EVICT_ORDER.get(s.kind, 1), s.last_used))
|
||||||
|
return candidates
|
||||||
|
|
||||||
|
async def make_room(self) -> int:
|
||||||
|
closed = 0
|
||||||
|
while self.over_limit():
|
||||||
|
victims = self.victims()
|
||||||
|
if not victims:
|
||||||
|
_log.warning(
|
||||||
|
"session pool over limit (%d live, rss=%d) but nothing evictable",
|
||||||
|
len(self._sessions),
|
||||||
|
self.rss(),
|
||||||
|
)
|
||||||
|
break
|
||||||
|
await self.close(victims[0].key)
|
||||||
|
closed += 1
|
||||||
|
return closed
|
||||||
|
|
||||||
|
async def close(self, key: str) -> None:
|
||||||
|
session = self._sessions.pop(key, None)
|
||||||
|
if session is None:
|
||||||
|
return
|
||||||
|
_log.info("closing session %s (%s, %s)", session.session_id, session.kind, key)
|
||||||
|
try:
|
||||||
|
await session.client.disconnect()
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
_log.exception("disconnect failed for session %s", session.session_id)
|
||||||
|
|
||||||
|
async def close_all(self, *, agent: str | None = None) -> None:
|
||||||
|
for session in list(self._sessions.values()):
|
||||||
|
if agent is None or session.agent == agent:
|
||||||
|
await self.close(session.key)
|
||||||
|
|
||||||
|
async def reap_once(self) -> int:
|
||||||
|
now = time.monotonic()
|
||||||
|
closed = 0
|
||||||
|
for session in list(self._sessions.values()):
|
||||||
|
ttl = self.ttl_for(session.kind)
|
||||||
|
if ttl is None or not session.evictable:
|
||||||
|
continue
|
||||||
|
if now - session.last_used > ttl:
|
||||||
|
await self.close(session.key)
|
||||||
|
closed += 1
|
||||||
|
return closed
|
||||||
|
|
||||||
|
async def reap_loop(self) -> None:
|
||||||
|
while True:
|
||||||
|
await asyncio.sleep(self._reap_interval)
|
||||||
|
try:
|
||||||
|
await self.reap_once()
|
||||||
|
await self.make_room()
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
_log.exception("session reaper failed")
|
||||||
|
|
||||||
|
def snapshot(self) -> list[dict[str, Any]]:
|
||||||
|
now = time.monotonic()
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"key": s.key,
|
||||||
|
"agent": s.agent,
|
||||||
|
"kind": s.kind,
|
||||||
|
"session_id": s.session_id,
|
||||||
|
"pid": s.pid,
|
||||||
|
"rss": self.rss_of(s),
|
||||||
|
"idle_seconds": round(now - s.last_used, 1),
|
||||||
|
"age_seconds": round(now - s.created_at, 1),
|
||||||
|
"turns": s.turns,
|
||||||
|
"busy": s.busy,
|
||||||
|
"running_turn": s.running_turn,
|
||||||
|
"pending_question": s.pending_question,
|
||||||
|
"pinned": s.pinned,
|
||||||
|
"dirty": s.dirty,
|
||||||
|
}
|
||||||
|
for s in self._sessions.values()
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def cgroup_limit() -> int | None:
|
||||||
|
for path in (
|
||||||
|
"/sys/fs/cgroup/memory.max",
|
||||||
|
"/sys/fs/cgroup/memory/memory.limit_in_bytes",
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
raw = Path(path).read_text(encoding="ascii").strip()
|
||||||
|
except OSError:
|
||||||
|
continue
|
||||||
|
if raw.isdigit() and int(raw) < 1 << 60:
|
||||||
|
return int(raw)
|
||||||
|
return None
|
||||||
@@ -0,0 +1,543 @@
|
|||||||
|
"""Anthropic messages <-> Agent SDK transcript entries.
|
||||||
|
|
||||||
|
:func:`build_entries` renders a message list as the entries the Claude CLI
|
||||||
|
itself writes (reference: ``t/spike_sdk/entries_reference.json``, CLI
|
||||||
|
2.1.248 via ``import_session_to_store``): one ``user`` entry per prompt,
|
||||||
|
one ``assistant`` entry per content block sharing a message id, one
|
||||||
|
``user`` entry per ``tool_result`` parented on the matching ``tool_use``
|
||||||
|
entry. Only ``user``/``assistant`` entries are produced - no attachments,
|
||||||
|
titles or queue markers. Appending the result to a session store and
|
||||||
|
resuming that session id seeds an external history into the SDK.
|
||||||
|
|
||||||
|
:func:`messages_from_entries` is the projection back, used by tests and by
|
||||||
|
anything that needs Anthropic-shape history out of a mirrored transcript.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import uuid as _uuid
|
||||||
|
from collections.abc import Mapping
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
|
try:
|
||||||
|
from claude_agent_sdk._cli_version import __cli_version__ as _cli_version
|
||||||
|
except ImportError: # pragma: no cover
|
||||||
|
_cli_version = "2.1.248"
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Iterable
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"CLI_VERSION",
|
||||||
|
"build_entries",
|
||||||
|
"close_open_tool_uses",
|
||||||
|
"fingerprint",
|
||||||
|
"messages_from_entries",
|
||||||
|
"open_tool_uses",
|
||||||
|
"prompt_count",
|
||||||
|
"render_messages",
|
||||||
|
"strip_tool_entries",
|
||||||
|
"text_of",
|
||||||
|
"window_entries",
|
||||||
|
]
|
||||||
|
|
||||||
|
CLI_VERSION = _cli_version
|
||||||
|
_ENTRYPOINT = "sdk-py"
|
||||||
|
_USER_TYPE = "external"
|
||||||
|
_PROMPT_SOURCE = "sdk"
|
||||||
|
_GIT_BRANCH = "HEAD"
|
||||||
|
|
||||||
|
|
||||||
|
def build_entries(
|
||||||
|
messages: Iterable[Mapping[str, Any]],
|
||||||
|
*,
|
||||||
|
session_id: str,
|
||||||
|
cwd: str,
|
||||||
|
model: str,
|
||||||
|
permission_mode: str = "bypassPermissions",
|
||||||
|
now: datetime | None = None,
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
stamp = (now or datetime.now(UTC)).strftime("%Y-%m-%dT%H:%M:%S.") + (
|
||||||
|
f"{(now or datetime.now(UTC)).microsecond // 1000:03d}Z"
|
||||||
|
)
|
||||||
|
common = {
|
||||||
|
"isSidechain": False,
|
||||||
|
"userType": _USER_TYPE,
|
||||||
|
"entrypoint": _ENTRYPOINT,
|
||||||
|
"cwd": cwd,
|
||||||
|
"sessionId": session_id,
|
||||||
|
"version": CLI_VERSION,
|
||||||
|
"gitBranch": _GIT_BRANCH,
|
||||||
|
}
|
||||||
|
entries: list[dict[str, Any]] = []
|
||||||
|
parent: str | None = None
|
||||||
|
prompt_id: str | None = None
|
||||||
|
tool_use_owner: dict[str, str] = {}
|
||||||
|
|
||||||
|
for message in messages:
|
||||||
|
role = message.get("role")
|
||||||
|
content = message.get("content")
|
||||||
|
if role == "user":
|
||||||
|
results = _tool_results(content)
|
||||||
|
if results:
|
||||||
|
for block in results:
|
||||||
|
owner = tool_use_owner.get(
|
||||||
|
str(block.get("tool_use_id", "")), parent
|
||||||
|
)
|
||||||
|
uid = _new_uuid()
|
||||||
|
entries.append(
|
||||||
|
{
|
||||||
|
"parentUuid": owner,
|
||||||
|
"promptId": prompt_id,
|
||||||
|
"type": "user",
|
||||||
|
"message": {"role": "user", "content": [block]},
|
||||||
|
"uuid": uid,
|
||||||
|
"timestamp": stamp,
|
||||||
|
"toolUseResult": _result_text(block),
|
||||||
|
"sourceToolAssistantUUID": owner,
|
||||||
|
**common,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
parent = uid
|
||||||
|
continue
|
||||||
|
prompt_id = _new_uuid()
|
||||||
|
uid = _new_uuid()
|
||||||
|
entries.append(
|
||||||
|
{
|
||||||
|
"parentUuid": parent,
|
||||||
|
"promptId": prompt_id,
|
||||||
|
"type": "user",
|
||||||
|
"message": {"role": "user", "content": _user_content(content)},
|
||||||
|
"uuid": uid,
|
||||||
|
"timestamp": stamp,
|
||||||
|
"permissionMode": permission_mode,
|
||||||
|
"promptSource": _PROMPT_SOURCE,
|
||||||
|
**common,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
parent = uid
|
||||||
|
elif role == "assistant":
|
||||||
|
blocks = _assistant_blocks(content)
|
||||||
|
message_id = f"msg_{_uuid.uuid4().hex[:24]}"
|
||||||
|
request_id = f"req_{_uuid.uuid4().hex[:24]}"
|
||||||
|
stop_reason = (
|
||||||
|
"tool_use"
|
||||||
|
if any(b.get("type") == "tool_use" for b in blocks)
|
||||||
|
else "end_turn"
|
||||||
|
)
|
||||||
|
for block in blocks:
|
||||||
|
uid = _new_uuid()
|
||||||
|
entries.append(
|
||||||
|
{
|
||||||
|
"parentUuid": parent,
|
||||||
|
"message": {
|
||||||
|
"model": model,
|
||||||
|
"id": message_id,
|
||||||
|
"type": "message",
|
||||||
|
"role": "assistant",
|
||||||
|
"content": [block],
|
||||||
|
"stop_reason": stop_reason,
|
||||||
|
"stop_sequence": None,
|
||||||
|
"stop_details": None,
|
||||||
|
"usage": _zero_usage(),
|
||||||
|
"diagnostics": None,
|
||||||
|
},
|
||||||
|
"requestId": request_id,
|
||||||
|
"type": "assistant",
|
||||||
|
"uuid": uid,
|
||||||
|
"timestamp": stamp,
|
||||||
|
**common,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
if block.get("type") == "tool_use" and block.get("id"):
|
||||||
|
tool_use_owner[str(block["id"])] = uid
|
||||||
|
parent = uid
|
||||||
|
else:
|
||||||
|
msg = f"message role must be user or assistant, got {role!r}"
|
||||||
|
raise ValueError(msg)
|
||||||
|
return entries
|
||||||
|
|
||||||
|
|
||||||
|
def messages_from_entries(entries: Iterable[Mapping[str, Any]]) -> list[dict[str, Any]]:
|
||||||
|
out: list[dict[str, Any]] = []
|
||||||
|
last_message_id: str | None = None
|
||||||
|
for entry in entries:
|
||||||
|
kind = entry.get("type")
|
||||||
|
message = entry.get("message")
|
||||||
|
if kind not in ("user", "assistant") or not isinstance(message, dict):
|
||||||
|
continue
|
||||||
|
content = message.get("content")
|
||||||
|
if kind == "user":
|
||||||
|
results = _tool_results(content)
|
||||||
|
if (
|
||||||
|
results
|
||||||
|
and out
|
||||||
|
and out[-1]["role"] == "user"
|
||||||
|
and _tool_results(out[-1]["content"])
|
||||||
|
):
|
||||||
|
out[-1]["content"].extend(results)
|
||||||
|
else:
|
||||||
|
out.append({"role": "user", "content": _user_content(content)})
|
||||||
|
last_message_id = None
|
||||||
|
continue
|
||||||
|
blocks = _assistant_blocks(content)
|
||||||
|
message_id = message.get("id")
|
||||||
|
if (
|
||||||
|
out
|
||||||
|
and out[-1]["role"] == "assistant"
|
||||||
|
and message_id is not None
|
||||||
|
and message_id == last_message_id
|
||||||
|
):
|
||||||
|
out[-1]["content"].extend(blocks)
|
||||||
|
else:
|
||||||
|
out.append({"role": "assistant", "content": blocks})
|
||||||
|
last_message_id = message_id
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _new_uuid() -> str:
|
||||||
|
return str(_uuid.uuid4())
|
||||||
|
|
||||||
|
|
||||||
|
def _tool_results(content: Any) -> list[dict[str, Any]]:
|
||||||
|
if not isinstance(content, list):
|
||||||
|
return []
|
||||||
|
return [
|
||||||
|
_clean_block(b)
|
||||||
|
for b in content
|
||||||
|
if isinstance(b, dict) and b.get("type") == "tool_result"
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _user_content(content: Any) -> Any:
|
||||||
|
if isinstance(content, str):
|
||||||
|
return content
|
||||||
|
if isinstance(content, list):
|
||||||
|
return [_clean_block(b) for b in content if isinstance(b, dict)]
|
||||||
|
msg = f"user content must be str or list, got {type(content).__name__}"
|
||||||
|
raise TypeError(msg)
|
||||||
|
|
||||||
|
|
||||||
|
def _assistant_blocks(content: Any) -> list[dict[str, Any]]:
|
||||||
|
if isinstance(content, str):
|
||||||
|
return [{"type": "text", "text": content}]
|
||||||
|
if isinstance(content, list):
|
||||||
|
blocks = [_clean_block(b) for b in content if isinstance(b, dict)]
|
||||||
|
if blocks:
|
||||||
|
return blocks
|
||||||
|
msg = "assistant content must be a non-empty list of blocks"
|
||||||
|
raise ValueError(msg)
|
||||||
|
|
||||||
|
|
||||||
|
def _clean_block(block: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
kind = block.get("type")
|
||||||
|
if kind == "tool_use":
|
||||||
|
return {
|
||||||
|
"type": "tool_use",
|
||||||
|
"id": block.get("id", ""),
|
||||||
|
"name": block.get("name", ""),
|
||||||
|
"input": block.get("input", {}),
|
||||||
|
}
|
||||||
|
if kind == "tool_result":
|
||||||
|
out: dict[str, Any] = {
|
||||||
|
"type": "tool_result",
|
||||||
|
"tool_use_id": block.get("tool_use_id", ""),
|
||||||
|
"content": block.get("content", ""),
|
||||||
|
}
|
||||||
|
if block.get("is_error") is not None:
|
||||||
|
out["is_error"] = bool(block["is_error"])
|
||||||
|
return out
|
||||||
|
if kind == "thinking":
|
||||||
|
return {
|
||||||
|
"type": "thinking",
|
||||||
|
"thinking": block.get("thinking", ""),
|
||||||
|
"signature": block.get("signature", ""),
|
||||||
|
}
|
||||||
|
if kind == "text":
|
||||||
|
return {"type": "text", "text": str(block.get("text", ""))}
|
||||||
|
return dict(block)
|
||||||
|
|
||||||
|
|
||||||
|
def _result_text(block: dict[str, Any]) -> str:
|
||||||
|
content = block.get("content", "")
|
||||||
|
if isinstance(content, str):
|
||||||
|
return content
|
||||||
|
if isinstance(content, list):
|
||||||
|
return "\n".join(
|
||||||
|
str(b.get("text", ""))
|
||||||
|
for b in content
|
||||||
|
if isinstance(b, dict) and b.get("type") == "text"
|
||||||
|
)
|
||||||
|
return str(content)
|
||||||
|
|
||||||
|
|
||||||
|
def _zero_usage() -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"input_tokens": 0,
|
||||||
|
"cache_creation_input_tokens": 0,
|
||||||
|
"cache_read_input_tokens": 0,
|
||||||
|
"output_tokens": 0,
|
||||||
|
"service_tier": "standard",
|
||||||
|
"server_tool_use": {"web_search_requests": 0, "web_fetch_requests": 0},
|
||||||
|
"cache_creation": {
|
||||||
|
"ephemeral_1h_input_tokens": 0,
|
||||||
|
"ephemeral_5m_input_tokens": 0,
|
||||||
|
},
|
||||||
|
"inference_geo": "not_available",
|
||||||
|
"iterations": [],
|
||||||
|
"speed": "standard",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ---- repair, windows, projections ---------------------------------------
|
||||||
|
|
||||||
|
_PROMPT_TYPES = ("user", "assistant")
|
||||||
|
_INTERRUPTED = "прервано"
|
||||||
|
|
||||||
|
|
||||||
|
def open_tool_uses(
|
||||||
|
entries: Iterable[Mapping[str, Any]],
|
||||||
|
) -> list[tuple[Mapping[str, Any], dict[str, Any]]]:
|
||||||
|
"""``(assistant entry, tool_use block)`` pairs that never got a ``tool_result``."""
|
||||||
|
closed: set[str] = set()
|
||||||
|
uses: list[tuple[Mapping[str, Any], dict[str, Any]]] = []
|
||||||
|
for entry in entries:
|
||||||
|
content = _entry_content(entry)
|
||||||
|
if entry.get("type") == "user":
|
||||||
|
closed.update(
|
||||||
|
str(b.get("tool_use_id", ""))
|
||||||
|
for b in content
|
||||||
|
if b.get("type") == "tool_result"
|
||||||
|
)
|
||||||
|
elif entry.get("type") == "assistant":
|
||||||
|
uses.extend((entry, b) for b in content if b.get("type") == "tool_use")
|
||||||
|
return [
|
||||||
|
(owner, block)
|
||||||
|
for owner, block in uses
|
||||||
|
if str(block.get("id", "")) not in closed
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def close_open_tool_uses(
|
||||||
|
entries: list[Mapping[str, Any]], *, text: str = _INTERRUPTED
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
"""Synthetic ``tool_result`` entries on the leaf, one per open ``tool_use``.
|
||||||
|
|
||||||
|
Appending the result to the session store gives the next ``resume`` a
|
||||||
|
transcript the CLI accepts: an assistant message ending in ``tool_use``
|
||||||
|
without its result is rejected by the API on the next call.
|
||||||
|
"""
|
||||||
|
pending = open_tool_uses(entries)
|
||||||
|
if not pending:
|
||||||
|
return []
|
||||||
|
leaf = next(
|
||||||
|
(
|
||||||
|
e
|
||||||
|
for e in reversed(entries)
|
||||||
|
if e.get("type") in _PROMPT_TYPES and e.get("uuid")
|
||||||
|
),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
parent = str(leaf["uuid"]) if leaf is not None else None
|
||||||
|
stamp = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S.000Z")
|
||||||
|
out: list[dict[str, Any]] = []
|
||||||
|
for owner, block in pending:
|
||||||
|
uid = _new_uuid()
|
||||||
|
result = {
|
||||||
|
"type": "tool_result",
|
||||||
|
"tool_use_id": block.get("id", ""),
|
||||||
|
"content": text,
|
||||||
|
"is_error": True,
|
||||||
|
}
|
||||||
|
out.append(
|
||||||
|
{
|
||||||
|
"parentUuid": parent,
|
||||||
|
"promptId": owner.get("promptId"),
|
||||||
|
"type": "user",
|
||||||
|
"message": {"role": "user", "content": [result]},
|
||||||
|
"uuid": uid,
|
||||||
|
"timestamp": stamp,
|
||||||
|
"toolUseResult": text,
|
||||||
|
"sourceToolAssistantUUID": owner.get("uuid"),
|
||||||
|
**{k: owner[k] for k in _COMMON_KEYS if k in owner},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
parent = uid
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
_COMMON_KEYS = (
|
||||||
|
"isSidechain",
|
||||||
|
"userType",
|
||||||
|
"entrypoint",
|
||||||
|
"cwd",
|
||||||
|
"sessionId",
|
||||||
|
"version",
|
||||||
|
"gitBranch",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def window_entries(
|
||||||
|
entries: Iterable[Mapping[str, Any]], *, window: int | None = None
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
"""``user``/``assistant`` entries only, last ``window`` prompts, relinked."""
|
||||||
|
kept = [
|
||||||
|
dict(e)
|
||||||
|
for e in entries
|
||||||
|
if e.get("type") in _PROMPT_TYPES and isinstance(e.get("uuid"), str)
|
||||||
|
]
|
||||||
|
if window is not None and window > 0:
|
||||||
|
starts = [i for i, e in enumerate(kept) if _is_prompt_entry(e)]
|
||||||
|
if len(starts) > window:
|
||||||
|
kept = kept[starts[-window] :]
|
||||||
|
return _relink(kept)
|
||||||
|
|
||||||
|
|
||||||
|
def strip_tool_entries(entries: Iterable[Mapping[str, Any]]) -> list[dict[str, Any]]:
|
||||||
|
"""Drop ``tool_result`` entries and ``tool_use``/``thinking`` blocks; relink."""
|
||||||
|
out: list[dict[str, Any]] = []
|
||||||
|
for raw in entries:
|
||||||
|
entry = dict(raw)
|
||||||
|
message = entry.get("message")
|
||||||
|
if not isinstance(message, dict):
|
||||||
|
out.append(entry)
|
||||||
|
continue
|
||||||
|
content = message.get("content")
|
||||||
|
if isinstance(content, list):
|
||||||
|
blocks = [
|
||||||
|
b
|
||||||
|
for b in content
|
||||||
|
if isinstance(b, dict)
|
||||||
|
and b.get("type") not in ("tool_use", "tool_result", "thinking")
|
||||||
|
]
|
||||||
|
if not blocks:
|
||||||
|
continue
|
||||||
|
entry["message"] = {**message, "content": blocks}
|
||||||
|
out.append(entry)
|
||||||
|
return _relink(out)
|
||||||
|
|
||||||
|
|
||||||
|
def prompt_count(entries: Iterable[Mapping[str, Any]]) -> int:
|
||||||
|
return sum(1 for e in entries if _is_prompt_entry(e))
|
||||||
|
|
||||||
|
|
||||||
|
def render_messages(
|
||||||
|
messages: Iterable[Mapping[str, Any]], *, window: int | None = None
|
||||||
|
) -> str:
|
||||||
|
"""Plain-text projection for ``read_conversation``: ``user:``/``assistant:`` turns.
|
||||||
|
|
||||||
|
Tool calls collapse to a one-line summary per assistant turn; tool
|
||||||
|
results and thinking are dropped.
|
||||||
|
"""
|
||||||
|
turns: list[str] = []
|
||||||
|
tools: list[str] = []
|
||||||
|
current: list[str] = []
|
||||||
|
|
||||||
|
def flush() -> None:
|
||||||
|
if not current and not tools:
|
||||||
|
return
|
||||||
|
body = "\n\n".join(current).strip()
|
||||||
|
if tools:
|
||||||
|
body = (body + "\n" if body else "") + "(tools: " + ", ".join(tools) + ")"
|
||||||
|
turns.append("assistant:\n" + body)
|
||||||
|
current.clear()
|
||||||
|
tools.clear()
|
||||||
|
|
||||||
|
for message in messages:
|
||||||
|
role = message.get("role")
|
||||||
|
content = message.get("content")
|
||||||
|
if role == "user":
|
||||||
|
if _tool_results(content):
|
||||||
|
continue
|
||||||
|
flush()
|
||||||
|
turns.append("user:\n" + _text_of_content(content))
|
||||||
|
continue
|
||||||
|
for block in _assistant_blocks(content):
|
||||||
|
if block.get("type") == "text" and block.get("text"):
|
||||||
|
current.append(str(block["text"]))
|
||||||
|
elif block.get("type") == "tool_use":
|
||||||
|
tools.append(str(block.get("name", "")))
|
||||||
|
flush()
|
||||||
|
if window is not None and window > 0:
|
||||||
|
starts = [i for i, t in enumerate(turns) if t.startswith("user:")]
|
||||||
|
if len(starts) > window:
|
||||||
|
turns = turns[starts[-window] :]
|
||||||
|
return "\n\n".join(turns)
|
||||||
|
|
||||||
|
|
||||||
|
def _entry_content(entry: Mapping[str, Any]) -> list[dict[str, Any]]:
|
||||||
|
message = entry.get("message")
|
||||||
|
if not isinstance(message, dict):
|
||||||
|
return []
|
||||||
|
content = message.get("content")
|
||||||
|
if not isinstance(content, list):
|
||||||
|
return []
|
||||||
|
return [b for b in content if isinstance(b, dict)]
|
||||||
|
|
||||||
|
|
||||||
|
def _is_prompt_entry(entry: Mapping[str, Any]) -> bool:
|
||||||
|
if entry.get("type") != "user":
|
||||||
|
return False
|
||||||
|
message = entry.get("message")
|
||||||
|
if not isinstance(message, dict):
|
||||||
|
return False
|
||||||
|
content = message.get("content")
|
||||||
|
if isinstance(content, str):
|
||||||
|
return True
|
||||||
|
return not _tool_results(content)
|
||||||
|
|
||||||
|
|
||||||
|
def _relink(entries: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||||
|
parent: str | None = None
|
||||||
|
for entry in entries:
|
||||||
|
entry["parentUuid"] = parent
|
||||||
|
parent = entry.get("uuid")
|
||||||
|
return entries
|
||||||
|
|
||||||
|
|
||||||
|
def _text_of_content(content: Any) -> str:
|
||||||
|
if isinstance(content, str):
|
||||||
|
return content
|
||||||
|
if isinstance(content, list):
|
||||||
|
return "\n\n".join(
|
||||||
|
str(b.get("text", ""))
|
||||||
|
for b in content
|
||||||
|
if isinstance(b, dict) and b.get("type") == "text" and b.get("text")
|
||||||
|
)
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
def fingerprint(messages: Iterable[Mapping[str, Any]]) -> str:
|
||||||
|
"""Text-only hash of a history; stateless callers are keyed by it."""
|
||||||
|
turns: list[tuple[str, str]] = []
|
||||||
|
for message in messages:
|
||||||
|
text = text_of(message.get("content"))
|
||||||
|
if not text:
|
||||||
|
continue
|
||||||
|
role = str(message.get("role", ""))
|
||||||
|
if turns and turns[-1][0] == role:
|
||||||
|
turns[-1] = (role, turns[-1][1] + "\n" + text)
|
||||||
|
else:
|
||||||
|
turns.append((role, text))
|
||||||
|
digest = hashlib.sha1(usedforsecurity=False)
|
||||||
|
for role, text in turns:
|
||||||
|
digest.update(role.encode("utf-8"))
|
||||||
|
digest.update(b"\x00")
|
||||||
|
digest.update(text.strip().encode("utf-8"))
|
||||||
|
digest.update(b"\x01")
|
||||||
|
return digest.hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def text_of(content: Any) -> str:
|
||||||
|
if isinstance(content, str):
|
||||||
|
return content
|
||||||
|
if isinstance(content, list):
|
||||||
|
parts = [
|
||||||
|
str(b.get("text", ""))
|
||||||
|
for b in content
|
||||||
|
if isinstance(b, Mapping) and b.get("type") == "text"
|
||||||
|
]
|
||||||
|
return "\n".join(p for p in parts if p)
|
||||||
|
return ""
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
"""Side channel for what a backend learned during one turn.
|
||||||
|
|
||||||
|
Frontends pass ``capture=TurnCapture()`` through ``Backend.complete``'s
|
||||||
|
``**options``. Backends that keep state per conversation (the Claude SDK
|
||||||
|
adapter) fill it in after the stream closes; backends that don't
|
||||||
|
(anthropic, raycast) drop the kwarg and the frontend falls back to a
|
||||||
|
text-only history.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
__all__ = ["TurnCapture", "TurnUsage"]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class TurnUsage:
|
||||||
|
input_tokens: int = 0
|
||||||
|
output_tokens: int = 0
|
||||||
|
cache_read_tokens: int = 0
|
||||||
|
cache_creation_tokens: int = 0
|
||||||
|
cost_usd: float | None = None
|
||||||
|
duration_ms: int | None = None
|
||||||
|
num_turns: int | None = None
|
||||||
|
model_usage: dict[str, Any] | None = None
|
||||||
|
"""``ResultMessage.model_usage`` verbatim: per-model tokens, cost, web searches."""
|
||||||
|
|
||||||
|
context_tokens: int = 0
|
||||||
|
"""Input of the last API call in the turn (fresh + cached + written to
|
||||||
|
cache) - the context size the model actually ran with, what Claude Code
|
||||||
|
shows as the context. The token fields above are sums over every API
|
||||||
|
call of the turn and grow with the number of tool calls."""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class TurnCapture:
|
||||||
|
synthesized_messages: list[dict[str, Any]] = field(default_factory=list)
|
||||||
|
"""The assistant/tool_result cycle of the turn as Anthropic messages."""
|
||||||
|
|
||||||
|
session_id: str | None = None
|
||||||
|
"""Backend session that ran the turn; persist and pass back as ``session_id``."""
|
||||||
|
|
||||||
|
usage: TurnUsage | None = None
|
||||||
|
interrupted: bool = False
|
||||||
|
"""The turn was cut by ``interrupt()`` (urgent inject), not by an error."""
|
||||||
@@ -0,0 +1,205 @@
|
|||||||
|
"""Vault watcher for the envelope (§3.5, §4.6).
|
||||||
|
|
||||||
|
A content snapshot is taken at every envelope; a change is the diff of the
|
||||||
|
file against that snapshot, "added lines only". ``watchfiles`` delivers
|
||||||
|
paths with a settle debounce (Sync writes files in pieces), the change
|
||||||
|
carries the file's mtime rather than the moment it landed. Which files
|
||||||
|
show a full diff, which show up by name and which are ignored is a set of
|
||||||
|
glob patterns the config hands in - the gateway itself knows no path.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import contextlib
|
||||||
|
import difflib
|
||||||
|
import fnmatch
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
from zoneinfo import ZoneInfo
|
||||||
|
|
||||||
|
from watchfiles import awatch
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Iterable
|
||||||
|
|
||||||
|
__all__ = ["Change", "VaultWatch", "WatchRules"]
|
||||||
|
|
||||||
|
_log = logging.getLogger("beaver_gateway.core.watch")
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class WatchRules:
|
||||||
|
"""Globs relative to the vault root; ``{today}`` expands to the local date.
|
||||||
|
|
||||||
|
``full`` files show their added lines, ``names`` files only their name
|
||||||
|
and how many lines they gained, ``ignore`` hides them; when several
|
||||||
|
patterns match, the most specific one decides.
|
||||||
|
"""
|
||||||
|
|
||||||
|
full: tuple[str, ...] = ()
|
||||||
|
names: tuple[str, ...] = ()
|
||||||
|
ignore: tuple[str, ...] = ()
|
||||||
|
suffixes: tuple[str, ...] = (".md",)
|
||||||
|
|
||||||
|
def kind(self, rel: str, *, today: str) -> str | None:
|
||||||
|
"""The most specific matching pattern wins, whichever list it is in."""
|
||||||
|
if not rel.endswith(self.suffixes):
|
||||||
|
return None
|
||||||
|
best: tuple[int, str | None] = (-1, None)
|
||||||
|
for kind, patterns in (
|
||||||
|
("full", self.full),
|
||||||
|
("names", self.names),
|
||||||
|
(None, self.ignore),
|
||||||
|
):
|
||||||
|
for pattern in patterns:
|
||||||
|
if _match(rel, pattern, today):
|
||||||
|
best = max(best, (_specificity(pattern), kind))
|
||||||
|
return best[1]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(slots=True)
|
||||||
|
class Change:
|
||||||
|
path: str
|
||||||
|
mtime: datetime
|
||||||
|
added: list[str] = field(default_factory=list)
|
||||||
|
added_count: int = 0
|
||||||
|
full: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(slots=True)
|
||||||
|
class _Seen:
|
||||||
|
lines: list[str] | None
|
||||||
|
count: int
|
||||||
|
|
||||||
|
|
||||||
|
class VaultWatch:
|
||||||
|
def __init__(
|
||||||
|
self, root: Path, rules: WatchRules, *, tz: str = "UTC", debounce: float = 4.0
|
||||||
|
) -> None:
|
||||||
|
self.root = root
|
||||||
|
self.rules = rules
|
||||||
|
self._tz = ZoneInfo(tz)
|
||||||
|
self._debounce = debounce
|
||||||
|
self._seen: dict[str, _Seen] = {}
|
||||||
|
self._pending: dict[str, Change] = {}
|
||||||
|
self._stop = asyncio.Event()
|
||||||
|
|
||||||
|
@property
|
||||||
|
def pending(self) -> list[Change]:
|
||||||
|
return list(self._pending.values())
|
||||||
|
|
||||||
|
def today(self, now: datetime | None = None) -> str:
|
||||||
|
return (now or datetime.now(UTC)).astimezone(self._tz).date().isoformat()
|
||||||
|
|
||||||
|
async def run(self) -> None:
|
||||||
|
await asyncio.to_thread(self.snapshot)
|
||||||
|
_log.info("watching %s (%d files in snapshot)", self.root, len(self._seen))
|
||||||
|
async for changes in awatch(
|
||||||
|
self.root,
|
||||||
|
debounce=int(self._debounce * 1000),
|
||||||
|
stop_event=self._stop,
|
||||||
|
ignore_permission_denied=True,
|
||||||
|
):
|
||||||
|
for _kind, raw in changes:
|
||||||
|
with contextlib.suppress(OSError):
|
||||||
|
self.note(Path(raw))
|
||||||
|
|
||||||
|
def stop(self) -> None:
|
||||||
|
self._stop.set()
|
||||||
|
|
||||||
|
def snapshot(self) -> None:
|
||||||
|
today = self.today()
|
||||||
|
for path in self._walk():
|
||||||
|
rel = self._rel(path)
|
||||||
|
kind = self.rules.kind(rel, today=today)
|
||||||
|
if kind is None:
|
||||||
|
continue
|
||||||
|
with contextlib.suppress(OSError):
|
||||||
|
lines = _read_lines(path)
|
||||||
|
self._seen[rel] = _Seen(
|
||||||
|
lines=lines if kind == "full" else None, count=len(lines)
|
||||||
|
)
|
||||||
|
|
||||||
|
def note(self, path: Path) -> Change | None:
|
||||||
|
"""Record a change of ``path`` against the last envelope's snapshot."""
|
||||||
|
try:
|
||||||
|
rel = self._rel(path)
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
kind = self.rules.kind(rel, today=self.today())
|
||||||
|
if kind is None:
|
||||||
|
return None
|
||||||
|
if not path.is_file():
|
||||||
|
self._pending.pop(rel, None)
|
||||||
|
return None
|
||||||
|
stat = path.stat()
|
||||||
|
lines = _read_lines(path)
|
||||||
|
seen = self._seen.get(rel)
|
||||||
|
change = Change(
|
||||||
|
path=rel,
|
||||||
|
mtime=datetime.fromtimestamp(stat.st_mtime, tz=UTC),
|
||||||
|
full=kind == "full",
|
||||||
|
)
|
||||||
|
if seen is not None and seen.lines is not None:
|
||||||
|
change.added = _added(seen.lines, lines)
|
||||||
|
change.added_count = len(change.added)
|
||||||
|
elif seen is not None:
|
||||||
|
change.added_count = max(len(lines) - seen.count, 0)
|
||||||
|
else:
|
||||||
|
change.added = lines if kind == "full" else []
|
||||||
|
change.added_count = len(lines)
|
||||||
|
if change.added_count == 0 and seen is not None:
|
||||||
|
self._pending.pop(rel, None)
|
||||||
|
return None
|
||||||
|
self._pending[rel] = change
|
||||||
|
return change
|
||||||
|
|
||||||
|
def take(self) -> list[Change]:
|
||||||
|
"""Hand out what changed since the last envelope and reset the snapshot."""
|
||||||
|
changes = list(self._pending.values())
|
||||||
|
self._pending.clear()
|
||||||
|
for change in changes:
|
||||||
|
path = self.root / change.path
|
||||||
|
with contextlib.suppress(OSError):
|
||||||
|
lines = _read_lines(path)
|
||||||
|
self._seen[change.path] = _Seen(
|
||||||
|
lines=lines if change.full else None, count=len(lines)
|
||||||
|
)
|
||||||
|
return changes
|
||||||
|
|
||||||
|
def _rel(self, path: Path) -> str:
|
||||||
|
return path.resolve().relative_to(self.root.resolve()).as_posix()
|
||||||
|
|
||||||
|
def _walk(self) -> Iterable[Path]:
|
||||||
|
for dirpath, dirnames, filenames in os.walk(self.root):
|
||||||
|
dirnames[:] = [d for d in dirnames if not d.startswith(".")]
|
||||||
|
for name in filenames:
|
||||||
|
yield Path(dirpath) / name
|
||||||
|
|
||||||
|
|
||||||
|
def _match(rel: str, pattern: str, today: str) -> bool:
|
||||||
|
return fnmatch.fnmatchcase(
|
||||||
|
rel, pattern.replace("{today}", today).replace("**", "*")
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _specificity(pattern: str) -> int:
|
||||||
|
return sum(1 for ch in pattern if ch not in "*?[]{}")
|
||||||
|
|
||||||
|
|
||||||
|
def _read_lines(path: Path) -> list[str]:
|
||||||
|
return path.read_text(encoding="utf-8", errors="replace").splitlines()
|
||||||
|
|
||||||
|
|
||||||
|
def _added(old: list[str], new: list[str]) -> list[str]:
|
||||||
|
matcher = difflib.SequenceMatcher(a=old, b=new, autojunk=False)
|
||||||
|
added: list[str] = []
|
||||||
|
for tag, _i1, _i2, j1, j2 in matcher.get_opcodes():
|
||||||
|
if tag in ("insert", "replace"):
|
||||||
|
added.extend(line for line in new[j1:j2] if line.strip())
|
||||||
|
return added
|
||||||
@@ -25,20 +25,24 @@ async def require_token(
|
|||||||
) -> str:
|
) -> str:
|
||||||
"""Verify the request's bearer + scope, return the token's audit name.
|
"""Verify the request's bearer + scope, return the token's audit name.
|
||||||
|
|
||||||
Accepts both ``X-Api-Key: <token>`` (Anthropic SDK / LibreChat) and
|
Accepts ``X-Api-Key: <token>`` (Anthropic SDK / LibreChat),
|
||||||
``Authorization: Bearer <token>`` (curl, Cursor). 401 on missing /
|
``Authorization: Bearer <token>`` (curl, Cursor) and, when neither
|
||||||
|
header is present, ``?token=<token>`` (Komodo alerters). 401 on missing /
|
||||||
unknown token; 403 on a known token whose scope doesn't cover
|
unknown token; 403 on a known token whose scope doesn't cover
|
||||||
``scope``. Bootstrap tokens implicitly carry ``"*"`` and pass every
|
``scope``. Bootstrap tokens implicitly carry ``"*"`` and pass every
|
||||||
scope check.
|
scope check.
|
||||||
"""
|
"""
|
||||||
api_key = request.headers.get("x-api-key")
|
api_key = request.headers.get("x-api-key")
|
||||||
identity = (
|
authorization = request.headers.get("authorization")
|
||||||
await runtime.token_store.verify(api_key)
|
if api_key:
|
||||||
if api_key
|
identity = await runtime.token_store.verify(api_key)
|
||||||
else await runtime.token_store.verify_bearer(
|
elif authorization:
|
||||||
request.headers.get("authorization")
|
identity = await runtime.token_store.verify_bearer(authorization)
|
||||||
)
|
else:
|
||||||
)
|
# Webhook senders that cannot set headers (Komodo alerters) put the
|
||||||
|
# token in the query string; the URL is not logged with it.
|
||||||
|
qs_token = request.query_params.get("token")
|
||||||
|
identity = await runtime.token_store.verify(qs_token) if qs_token else None
|
||||||
if identity is None:
|
if identity is None:
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
status.HTTP_401_UNAUTHORIZED,
|
status.HTTP_401_UNAUTHORIZED,
|
||||||
|
|||||||
@@ -0,0 +1,63 @@
|
|||||||
|
"""Server-sent events helpers shared by the markdown and API frontends."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import contextlib
|
||||||
|
import json
|
||||||
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import AsyncIterator
|
||||||
|
|
||||||
|
__all__ = ["HEARTBEAT_INTERVAL", "SSE_HEADERS", "events_with_heartbeat", "sse_pack"]
|
||||||
|
|
||||||
|
HEARTBEAT_INTERVAL = 15.0
|
||||||
|
"""Seconds of backend silence before a comment frame keeps the socket warm."""
|
||||||
|
|
||||||
|
SSE_HEADERS = {
|
||||||
|
"Cache-Control": "no-cache, no-transform",
|
||||||
|
"Connection": "keep-alive",
|
||||||
|
"X-Accel-Buffering": "no",
|
||||||
|
}
|
||||||
|
|
||||||
|
KEEPALIVE = b": keepalive\n\n"
|
||||||
|
|
||||||
|
|
||||||
|
async def events_with_heartbeat(
|
||||||
|
events: AsyncIterator[Any], interval: float = HEARTBEAT_INTERVAL
|
||||||
|
) -> AsyncIterator[Any]:
|
||||||
|
"""Pass ``events`` through, yielding ``None`` after ``interval`` seconds of silence.
|
||||||
|
|
||||||
|
One in-flight ``__anext__`` task is reused across timeouts: a second
|
||||||
|
consumer on the same async generator raises ``RuntimeError``.
|
||||||
|
Cancellation of the outer scope cancels that task instead of leaving
|
||||||
|
it dangling.
|
||||||
|
"""
|
||||||
|
src = events.__aiter__()
|
||||||
|
next_task: asyncio.Task[Any] | None = None
|
||||||
|
try:
|
||||||
|
while True:
|
||||||
|
if next_task is None:
|
||||||
|
next_task = asyncio.ensure_future(src.__anext__())
|
||||||
|
done, _pending = await asyncio.wait({next_task}, timeout=interval)
|
||||||
|
if not done:
|
||||||
|
yield None
|
||||||
|
continue
|
||||||
|
task = next_task
|
||||||
|
next_task = None
|
||||||
|
try:
|
||||||
|
result = task.result()
|
||||||
|
except StopAsyncIteration:
|
||||||
|
return
|
||||||
|
yield result
|
||||||
|
finally:
|
||||||
|
if next_task is not None and not next_task.done():
|
||||||
|
next_task.cancel()
|
||||||
|
with contextlib.suppress(BaseException):
|
||||||
|
await next_task
|
||||||
|
|
||||||
|
|
||||||
|
def sse_pack(event: str, data: dict[str, Any]) -> bytes:
|
||||||
|
body = json.dumps(data, ensure_ascii=False)
|
||||||
|
return f"event: {event}\ndata: {body}\n\n".encode()
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
"""Where a frontend is reachable from outside, as seen by one request."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from starlette.requests import Request
|
||||||
|
|
||||||
|
from beaver_gateway.frontends.base import Frontend, GatewayRuntime
|
||||||
|
|
||||||
|
|
||||||
|
def request_origin(request: Request, runtime: GatewayRuntime) -> str:
|
||||||
|
"""``Gateway.public_url`` if configured, else the request's own origin."""
|
||||||
|
if runtime.public_url:
|
||||||
|
return runtime.public_url
|
||||||
|
host = request.headers.get("host")
|
||||||
|
if not host:
|
||||||
|
return f"http://{request.url.hostname}:{request.url.port}"
|
||||||
|
scheme = request.headers.get("x-forwarded-proto") or request.url.scheme
|
||||||
|
return f"{scheme}://{host}"
|
||||||
|
|
||||||
|
|
||||||
|
def external_base(request: Request, runtime: GatewayRuntime) -> str:
|
||||||
|
"""Origin plus the mount path of the app handling ``request``."""
|
||||||
|
return request_origin(request, runtime) + str(request.scope.get("root_path", ""))
|
||||||
|
|
||||||
|
|
||||||
|
def frontend_url(request: Request, runtime: GatewayRuntime, fe: Frontend) -> str | None:
|
||||||
|
return request_origin(request, runtime) + fe.path if fe.path else None
|
||||||
@@ -1,10 +1,4 @@
|
|||||||
"""Admin UI (Phase 4.3).
|
"""Admin console: the ``ui/`` SPA served from the admin port plus its login."""
|
||||||
|
|
||||||
Browser-facing console: login, dashboard, token CRUD, audit viewer.
|
|
||||||
Templates live in ``./templates``; the package loader picks them up via
|
|
||||||
``importlib.resources`` so the layout works editable and inside the
|
|
||||||
Docker image without copy hacks.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from beaver_gateway.frontends.admin.frontend import AdminFrontend
|
from beaver_gateway.frontends.admin.frontend import AdminFrontend
|
||||||
|
|
||||||
|
|||||||
File diff suppressed because it is too large.
Load diff
@@ -1,206 +0,0 @@
|
|||||||
<!doctype html>
|
|
||||||
<html lang="en">
|
|
||||||
<head>
|
|
||||||
<meta charset="utf-8">
|
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
||||||
<title>{% block title %}beaver-gateway · admin{% endblock %}</title>
|
|
||||||
<script src="https://unpkg.com/htmx.org@2.0.4" defer></script>
|
|
||||||
<style>
|
|
||||||
:root {
|
|
||||||
--bg: #fbfbfd;
|
|
||||||
--fg: #1d1d1f;
|
|
||||||
--muted: #6e6e73;
|
|
||||||
--line: #e5e5ea;
|
|
||||||
--accent: #0071e3;
|
|
||||||
--danger: #d70015;
|
|
||||||
--surface: #ffffff;
|
|
||||||
--code-bg: #f5f5f7;
|
|
||||||
}
|
|
||||||
* { box-sizing: border-box; }
|
|
||||||
html, body { margin: 0; padding: 0; background: var(--bg); color: var(--fg); }
|
|
||||||
body {
|
|
||||||
font-family: -apple-system, "SF Pro Display", "SF Pro Text",
|
|
||||||
BlinkMacSystemFont, system-ui, sans-serif;
|
|
||||||
font-size: 15px;
|
|
||||||
line-height: 1.55;
|
|
||||||
letter-spacing: -0.005em;
|
|
||||||
}
|
|
||||||
a { color: var(--accent); text-decoration: none; }
|
|
||||||
a:hover { text-decoration: underline; }
|
|
||||||
code, pre, kbd, samp {
|
|
||||||
font-family: ui-monospace, "SF Mono", Menlo, Consolas, monospace;
|
|
||||||
font-size: 0.92em;
|
|
||||||
}
|
|
||||||
pre {
|
|
||||||
background: var(--code-bg);
|
|
||||||
padding: 1rem 1.25rem;
|
|
||||||
border-radius: 12px;
|
|
||||||
overflow-x: auto;
|
|
||||||
}
|
|
||||||
header.top {
|
|
||||||
border-bottom: 1px solid var(--line);
|
|
||||||
background: var(--surface);
|
|
||||||
}
|
|
||||||
header.top .inner {
|
|
||||||
max-width: 1080px;
|
|
||||||
margin: 0 auto;
|
|
||||||
padding: 1rem 1.5rem;
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 1.25rem;
|
|
||||||
}
|
|
||||||
header.top h1 {
|
|
||||||
font-size: 1.05rem;
|
|
||||||
font-weight: 600;
|
|
||||||
letter-spacing: -0.01em;
|
|
||||||
margin: 0;
|
|
||||||
}
|
|
||||||
nav.tabs { display: flex; gap: 1.1rem; flex: 1; }
|
|
||||||
nav.tabs a {
|
|
||||||
color: var(--fg);
|
|
||||||
padding: 0.25rem 0.1rem;
|
|
||||||
border-bottom: 2px solid transparent;
|
|
||||||
}
|
|
||||||
nav.tabs a.active {
|
|
||||||
border-bottom-color: var(--fg);
|
|
||||||
}
|
|
||||||
header.top .actor {
|
|
||||||
color: var(--muted);
|
|
||||||
font-size: 0.9em;
|
|
||||||
}
|
|
||||||
main {
|
|
||||||
max-width: 1080px;
|
|
||||||
margin: 0 auto;
|
|
||||||
padding: 2.25rem 1.5rem 4rem;
|
|
||||||
}
|
|
||||||
h2 {
|
|
||||||
font-weight: 600;
|
|
||||||
letter-spacing: -0.015em;
|
|
||||||
margin: 2rem 0 0.75rem;
|
|
||||||
}
|
|
||||||
h2:first-of-type { margin-top: 0; }
|
|
||||||
.muted { color: var(--muted); }
|
|
||||||
.card {
|
|
||||||
background: var(--surface);
|
|
||||||
border: 1px solid var(--line);
|
|
||||||
border-radius: 14px;
|
|
||||||
padding: 1.25rem 1.5rem;
|
|
||||||
margin-bottom: 1.25rem;
|
|
||||||
}
|
|
||||||
table { width: 100%; border-collapse: collapse; }
|
|
||||||
th, td {
|
|
||||||
text-align: left;
|
|
||||||
padding: 0.6rem 0.85rem;
|
|
||||||
border-bottom: 1px solid var(--line);
|
|
||||||
vertical-align: top;
|
|
||||||
}
|
|
||||||
th {
|
|
||||||
font-weight: 500;
|
|
||||||
color: var(--muted);
|
|
||||||
font-size: 0.85em;
|
|
||||||
text-transform: uppercase;
|
|
||||||
letter-spacing: 0.04em;
|
|
||||||
}
|
|
||||||
tr:last-child td { border-bottom: none; }
|
|
||||||
.revoked td { color: var(--muted); }
|
|
||||||
button, .btn {
|
|
||||||
font-family: inherit;
|
|
||||||
font-size: 0.92em;
|
|
||||||
padding: 0.45rem 0.95rem;
|
|
||||||
border-radius: 8px;
|
|
||||||
border: 1px solid var(--line);
|
|
||||||
background: var(--surface);
|
|
||||||
color: var(--fg);
|
|
||||||
cursor: pointer;
|
|
||||||
transition: background 120ms ease;
|
|
||||||
}
|
|
||||||
button:hover, .btn:hover { background: var(--code-bg); }
|
|
||||||
button.primary, .btn.primary {
|
|
||||||
background: var(--accent);
|
|
||||||
border-color: var(--accent);
|
|
||||||
color: white;
|
|
||||||
}
|
|
||||||
button.primary:hover, .btn.primary:hover {
|
|
||||||
background: #005bb5;
|
|
||||||
}
|
|
||||||
button.danger { color: var(--danger); border-color: #f0c5c9; }
|
|
||||||
button.danger:hover { background: #fdeff1; }
|
|
||||||
form.inline { display: inline; margin: 0; }
|
|
||||||
.form-grid {
|
|
||||||
display: grid;
|
|
||||||
grid-template-columns: minmax(180px, 1fr) 160px auto;
|
|
||||||
gap: 0.75rem;
|
|
||||||
align-items: end;
|
|
||||||
}
|
|
||||||
.form-grid label {
|
|
||||||
display: block;
|
|
||||||
font-size: 0.8em;
|
|
||||||
color: var(--muted);
|
|
||||||
text-transform: uppercase;
|
|
||||||
letter-spacing: 0.05em;
|
|
||||||
margin-bottom: 0.3rem;
|
|
||||||
}
|
|
||||||
input[type="text"], input[type="password"], select {
|
|
||||||
width: 100%;
|
|
||||||
padding: 0.55rem 0.7rem;
|
|
||||||
border: 1px solid var(--line);
|
|
||||||
border-radius: 8px;
|
|
||||||
font-family: inherit;
|
|
||||||
font-size: 0.95em;
|
|
||||||
background: var(--surface);
|
|
||||||
color: var(--fg);
|
|
||||||
}
|
|
||||||
input:focus, select:focus { outline: 2px solid var(--accent); outline-offset: 1px; }
|
|
||||||
.banner {
|
|
||||||
padding: 1rem 1.25rem;
|
|
||||||
border-radius: 12px;
|
|
||||||
background: #f0f9ff;
|
|
||||||
border: 1px solid #cfe8ff;
|
|
||||||
margin: 0 0 1.25rem;
|
|
||||||
}
|
|
||||||
.banner.error {
|
|
||||||
background: #fff0f0;
|
|
||||||
border-color: #f5c2c5;
|
|
||||||
color: var(--danger);
|
|
||||||
}
|
|
||||||
.banner.warn {
|
|
||||||
background: #fff7e6;
|
|
||||||
border-color: #fadf9c;
|
|
||||||
}
|
|
||||||
.pill {
|
|
||||||
display: inline-block;
|
|
||||||
padding: 0.1rem 0.55rem;
|
|
||||||
border-radius: 999px;
|
|
||||||
background: var(--code-bg);
|
|
||||||
font-size: 0.78em;
|
|
||||||
color: var(--muted);
|
|
||||||
}
|
|
||||||
.pill.scope-wild { background: #ecfdf3; color: #027a48; }
|
|
||||||
.pill.scope-admin { background: #fff4e6; color: #b54708; }
|
|
||||||
</style>
|
|
||||||
</head>
|
|
||||||
<body>
|
|
||||||
{% block header %}
|
|
||||||
<header class="top">
|
|
||||||
<div class="inner">
|
|
||||||
<h1>beaver-gateway</h1>
|
|
||||||
<nav class="tabs">
|
|
||||||
<a href="{{ p }}/" class="{% if active == 'dashboard' %}active{% endif %}">Dashboard</a>
|
|
||||||
<a href="{{ p }}/chat" class="{% if active == 'chat' %}active{% endif %}">Chat</a>
|
|
||||||
<a href="{{ p }}/pty" class="{% if active == 'pty' %}active{% endif %}">PTY</a>
|
|
||||||
<a href="{{ p }}/tokens" class="{% if active == 'tokens' %}active{% endif %}">Tokens</a>
|
|
||||||
<a href="{{ p }}/audit" class="{% if active == 'audit' %}active{% endif %}">Audit</a>
|
|
||||||
</nav>
|
|
||||||
<span class="actor">Signed in as <strong>{{ user }}</strong></span>
|
|
||||||
<form class="inline" method="post" action="{{ p }}/logout">
|
|
||||||
<input type="hidden" name="csrf_token" value="{{ csrf }}">
|
|
||||||
<button type="submit">Log out</button>
|
|
||||||
</form>
|
|
||||||
</div>
|
|
||||||
</header>
|
|
||||||
{% endblock %}
|
|
||||||
<main>
|
|
||||||
{% block content %}{% endblock %}
|
|
||||||
</main>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
{# Renders into #token-create-result (hx-target on the form). The
|
|
||||||
two OOB swaps reach the rest of the page: prepend the new row,
|
|
||||||
and erase the empty-state placeholder if it's still in the DOM. #}
|
|
||||||
<div class="banner warn">
|
|
||||||
<strong>Token created: {{ token.name }}</strong>
|
|
||||||
<p class="muted" style="margin:0.25rem 0 0.5rem;">Copy it now — this is the only time it will be shown.</p>
|
|
||||||
<pre style="margin:0;">{{ plaintext }}</pre>
|
|
||||||
</div>
|
|
||||||
<tbody id="tokens-rows" hx-swap-oob="afterbegin">
|
|
||||||
{% include "_token_row.html" %}
|
|
||||||
</tbody>
|
|
||||||
<tr id="tokens-empty" hx-swap-oob="delete"></tr>
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
<div class="banner error">{{ message }}</div>
|
|
||||||
@@ -1,25 +0,0 @@
|
|||||||
<tr id="token-row-{{ token.id }}" {% if token.revoked_at %}class="revoked"{% endif %}>
|
|
||||||
<td><strong>{{ token.name }}</strong></td>
|
|
||||||
<td>
|
|
||||||
<span class="pill {% if token.scope == '*' %}scope-wild{% elif token.scope == 'admin' %}scope-admin{% endif %}">{{ token.scope }}</span>
|
|
||||||
</td>
|
|
||||||
<td><code>{{ token.created_at | fmt_dt }}</code></td>
|
|
||||||
<td><code>{{ token.last_used_at | fmt_dt }}</code></td>
|
|
||||||
<td><code>{{ token.revoked_at | fmt_dt }}</code></td>
|
|
||||||
<td style="text-align:right;">
|
|
||||||
{% if not token.revoked_at %}
|
|
||||||
<form
|
|
||||||
class="inline"
|
|
||||||
hx-post="{{ p }}/tokens/{{ token.id }}/revoke"
|
|
||||||
hx-target="#token-row-{{ token.id }}"
|
|
||||||
hx-swap="outerHTML"
|
|
||||||
hx-confirm="Revoke token {{ token.name }}? This cannot be undone."
|
|
||||||
>
|
|
||||||
<input type="hidden" name="csrf_token" value="{{ csrf }}">
|
|
||||||
<button class="danger" type="submit">Revoke</button>
|
|
||||||
</form>
|
|
||||||
{% else %}
|
|
||||||
<span class="muted">revoked</span>
|
|
||||||
{% endif %}
|
|
||||||
</td>
|
|
||||||
</tr>
|
|
||||||
@@ -1,37 +0,0 @@
|
|||||||
{% extends "_layout.html" %}
|
|
||||||
{% set active = "audit" %}
|
|
||||||
{% block title %}beaver-gateway · audit{% endblock %}
|
|
||||||
{% block content %}
|
|
||||||
<h2>Audit log</h2>
|
|
||||||
<div class="card">
|
|
||||||
{% if audit %}
|
|
||||||
<table>
|
|
||||||
<thead>
|
|
||||||
<tr>
|
|
||||||
<th>Time</th><th>Actor</th><th>Kind</th><th>Agent</th><th>Detail</th>
|
|
||||||
</tr>
|
|
||||||
</thead>
|
|
||||||
<tbody>
|
|
||||||
{% for row in audit %}
|
|
||||||
<tr>
|
|
||||||
<td><code>{{ row.ts | fmt_dt }}</code></td>
|
|
||||||
<td>{{ row.actor }}</td>
|
|
||||||
<td><span class="pill">{{ row.kind }}</span></td>
|
|
||||||
<td>{{ row.agent_name or "—" }}</td>
|
|
||||||
<td><code>{{ row.detail_json | fmt_detail }}</code></td>
|
|
||||||
</tr>
|
|
||||||
{% endfor %}
|
|
||||||
</tbody>
|
|
||||||
</table>
|
|
||||||
<p class="muted" style="margin-top:1rem;">
|
|
||||||
{% if next_before %}
|
|
||||||
<a href="{{ p }}/audit?before={{ next_before }}">Older entries →</a>
|
|
||||||
{% else %}
|
|
||||||
End of log.
|
|
||||||
{% endif %}
|
|
||||||
</p>
|
|
||||||
{% else %}
|
|
||||||
<p class="muted">Nothing logged yet.</p>
|
|
||||||
{% endif %}
|
|
||||||
</div>
|
|
||||||
{% endblock %}
|
|
||||||
@@ -1,355 +0,0 @@
|
|||||||
{% extends "_layout.html" %}
|
|
||||||
{% set active = "chat" %}
|
|
||||||
{% block title %}beaver-gateway · chat{% endblock %}
|
|
||||||
{% block content %}
|
|
||||||
<div class="chat-wrap">
|
|
||||||
<div class="chat-toolbar">
|
|
||||||
<label>
|
|
||||||
<span>Agent</span>
|
|
||||||
<select id="agent-select">
|
|
||||||
{% for a in agents %}
|
|
||||||
<option value="{{ a.name }}">{{ a.name }} · {{ a.model }}</option>
|
|
||||||
{% else %}
|
|
||||||
<option disabled>no agents with a backend</option>
|
|
||||||
{% endfor %}
|
|
||||||
</select>
|
|
||||||
</label>
|
|
||||||
<div class="spacer"></div>
|
|
||||||
<button id="new-chat-btn" type="button">New chat</button>
|
|
||||||
</div>
|
|
||||||
<div id="messages" class="chat-messages" aria-live="polite"></div>
|
|
||||||
<form id="chat-form" class="chat-input">
|
|
||||||
<textarea id="chat-text" rows="3" placeholder="Message… (⌘/Ctrl+Enter to send)" required></textarea>
|
|
||||||
<button type="submit" class="primary" id="send-btn">Send</button>
|
|
||||||
</form>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<style>
|
|
||||||
.chat-wrap {
|
|
||||||
display: flex; flex-direction: column;
|
|
||||||
height: calc(100vh - 200px); min-height: 480px;
|
|
||||||
}
|
|
||||||
.chat-toolbar {
|
|
||||||
display: flex; gap: 0.85rem; align-items: end;
|
|
||||||
margin-bottom: 1rem;
|
|
||||||
}
|
|
||||||
.chat-toolbar .spacer { flex: 1; }
|
|
||||||
.chat-toolbar label {
|
|
||||||
display: flex; flex-direction: column; gap: 0.3rem;
|
|
||||||
min-width: 260px;
|
|
||||||
}
|
|
||||||
.chat-toolbar label > span {
|
|
||||||
font-size: 0.8em; color: var(--muted);
|
|
||||||
text-transform: uppercase; letter-spacing: 0.05em;
|
|
||||||
}
|
|
||||||
.chat-messages {
|
|
||||||
flex: 1; overflow-y: auto;
|
|
||||||
background: var(--surface); border: 1px solid var(--line);
|
|
||||||
border-radius: 14px; padding: 1.25rem;
|
|
||||||
display: flex; flex-direction: column; gap: 1rem;
|
|
||||||
}
|
|
||||||
.chat-empty { color: var(--muted); margin: auto; }
|
|
||||||
.msg { display: flex; }
|
|
||||||
.msg.user { justify-content: flex-end; }
|
|
||||||
.msg.user .bubble {
|
|
||||||
max-width: 78%; padding: 0.7rem 0.95rem; border-radius: 14px;
|
|
||||||
background: var(--accent); color: white;
|
|
||||||
white-space: pre-wrap; word-wrap: break-word;
|
|
||||||
}
|
|
||||||
.msg.assistant .blocks {
|
|
||||||
display: flex; flex-direction: column; gap: 0.5rem;
|
|
||||||
max-width: 88%;
|
|
||||||
}
|
|
||||||
.block-text {
|
|
||||||
background: var(--code-bg); padding: 0.7rem 0.95rem;
|
|
||||||
border-radius: 14px;
|
|
||||||
white-space: pre-wrap; word-wrap: break-word;
|
|
||||||
}
|
|
||||||
details.tool-call, details.thinking-block {
|
|
||||||
border: 1px solid var(--line); border-radius: 10px;
|
|
||||||
background: var(--surface); padding: 0 0.85rem;
|
|
||||||
font-size: 0.9em;
|
|
||||||
}
|
|
||||||
details.tool-call > summary,
|
|
||||||
details.thinking-block > summary {
|
|
||||||
cursor: pointer; padding: 0.55rem 0;
|
|
||||||
list-style: none;
|
|
||||||
display: flex; align-items: center; gap: 0.55rem;
|
|
||||||
}
|
|
||||||
details > summary::-webkit-details-marker { display: none; }
|
|
||||||
details.tool-call > summary::before,
|
|
||||||
details.thinking-block > summary::before {
|
|
||||||
content: "▸"; color: var(--muted); font-size: 0.78em;
|
|
||||||
}
|
|
||||||
details.tool-call[open] > summary::before,
|
|
||||||
details.thinking-block[open] > summary::before { content: "▾"; }
|
|
||||||
.tool-name {
|
|
||||||
font-family: ui-monospace, "SF Mono", Menlo, monospace;
|
|
||||||
font-weight: 500;
|
|
||||||
}
|
|
||||||
.tool-id, .tool-label {
|
|
||||||
color: var(--muted); font-size: 0.78em;
|
|
||||||
font-family: ui-monospace, "SF Mono", Menlo, monospace;
|
|
||||||
}
|
|
||||||
details.tool-call pre, details.thinking-block pre {
|
|
||||||
margin: 0 0 0.6rem; padding: 0.7rem 0.85rem;
|
|
||||||
background: var(--code-bg); border-radius: 8px;
|
|
||||||
font-size: 0.85em; max-height: 360px; overflow: auto;
|
|
||||||
}
|
|
||||||
.tool-label { display: block; margin: 0.15rem 0 0.25rem; }
|
|
||||||
.chat-input { display: flex; gap: 0.75rem; margin-top: 1rem; }
|
|
||||||
.chat-input textarea {
|
|
||||||
flex: 1; padding: 0.7rem 0.9rem;
|
|
||||||
border: 1px solid var(--line); border-radius: 12px;
|
|
||||||
font-family: inherit; font-size: 0.95em;
|
|
||||||
background: var(--surface); color: var(--fg); resize: vertical;
|
|
||||||
}
|
|
||||||
.chat-input textarea:focus { outline: 2px solid var(--accent); outline-offset: 1px; }
|
|
||||||
.chat-input button { align-self: stretch; padding-left: 1.4rem; padding-right: 1.4rem; }
|
|
||||||
.chat-error {
|
|
||||||
color: var(--danger); font-size: 0.85em;
|
|
||||||
padding: 0.55rem 0.75rem;
|
|
||||||
background: #fff0f0; border-radius: 10px;
|
|
||||||
border: 1px solid #f5c2c5;
|
|
||||||
}
|
|
||||||
.chat-input button:disabled { opacity: 0.55; cursor: progress; }
|
|
||||||
</style>
|
|
||||||
|
|
||||||
<script>
|
|
||||||
(() => {
|
|
||||||
const CSRF = {{ csrf | tojson }};
|
|
||||||
const URL_PREFIX = {{ p | tojson }};
|
|
||||||
const agentSelect = document.getElementById("agent-select");
|
|
||||||
const messagesEl = document.getElementById("messages");
|
|
||||||
const form = document.getElementById("chat-form");
|
|
||||||
const textEl = document.getElementById("chat-text");
|
|
||||||
const sendBtn = document.getElementById("send-btn");
|
|
||||||
const newBtn = document.getElementById("new-chat-btn");
|
|
||||||
|
|
||||||
// Anthropic-style history sent to the backend. We keep assistant
|
|
||||||
// content text-only — tool_use blocks can't round-trip without
|
|
||||||
// matching tool_result, and backends (claude-code) run tools
|
|
||||||
// internally anyway.
|
|
||||||
let apiMessages = [];
|
|
||||||
|
|
||||||
function renderEmpty() {
|
|
||||||
messagesEl.innerHTML = '<div class="chat-empty">No messages yet — say something.</div>';
|
|
||||||
}
|
|
||||||
renderEmpty();
|
|
||||||
|
|
||||||
newBtn.addEventListener("click", () => {
|
|
||||||
apiMessages = [];
|
|
||||||
renderEmpty();
|
|
||||||
});
|
|
||||||
|
|
||||||
function clearEmpty() {
|
|
||||||
const e = messagesEl.querySelector(".chat-empty");
|
|
||||||
if (e) e.remove();
|
|
||||||
}
|
|
||||||
function scrollDown() { messagesEl.scrollTop = messagesEl.scrollHeight; }
|
|
||||||
|
|
||||||
function appendUser(text) {
|
|
||||||
clearEmpty();
|
|
||||||
const row = document.createElement("div");
|
|
||||||
row.className = "msg user";
|
|
||||||
const bub = document.createElement("div");
|
|
||||||
bub.className = "bubble";
|
|
||||||
bub.textContent = text;
|
|
||||||
row.appendChild(bub);
|
|
||||||
messagesEl.appendChild(row);
|
|
||||||
scrollDown();
|
|
||||||
}
|
|
||||||
function appendAssistant() {
|
|
||||||
clearEmpty();
|
|
||||||
const row = document.createElement("div");
|
|
||||||
row.className = "msg assistant";
|
|
||||||
const blocks = document.createElement("div");
|
|
||||||
blocks.className = "blocks";
|
|
||||||
row.appendChild(blocks);
|
|
||||||
messagesEl.appendChild(row);
|
|
||||||
scrollDown();
|
|
||||||
return blocks;
|
|
||||||
}
|
|
||||||
function appendError(text) {
|
|
||||||
clearEmpty();
|
|
||||||
const e = document.createElement("div");
|
|
||||||
e.className = "chat-error";
|
|
||||||
e.textContent = text;
|
|
||||||
messagesEl.appendChild(e);
|
|
||||||
scrollDown();
|
|
||||||
}
|
|
||||||
|
|
||||||
function ensureBlock(state, index, type, extra) {
|
|
||||||
if (state.blocks[index]) return state.blocks[index];
|
|
||||||
const block = { type };
|
|
||||||
if (type === "text") {
|
|
||||||
const el = document.createElement("div");
|
|
||||||
el.className = "block-text";
|
|
||||||
state.container.appendChild(el);
|
|
||||||
block.el = el;
|
|
||||||
} else if (type === "thinking") {
|
|
||||||
const det = document.createElement("details");
|
|
||||||
det.className = "thinking-block";
|
|
||||||
const sum = document.createElement("summary");
|
|
||||||
sum.textContent = "Thinking";
|
|
||||||
det.appendChild(sum);
|
|
||||||
const pre = document.createElement("pre");
|
|
||||||
det.appendChild(pre);
|
|
||||||
state.container.appendChild(det);
|
|
||||||
block.el = pre;
|
|
||||||
} else if (type === "tool_use") {
|
|
||||||
const det = document.createElement("details");
|
|
||||||
det.className = "tool-call";
|
|
||||||
const sum = document.createElement("summary");
|
|
||||||
const name = document.createElement("span");
|
|
||||||
name.className = "tool-name";
|
|
||||||
name.textContent = "🔧 " + (extra.name || "tool");
|
|
||||||
sum.appendChild(name);
|
|
||||||
if (extra.id) {
|
|
||||||
const idEl = document.createElement("span");
|
|
||||||
idEl.className = "tool-id";
|
|
||||||
idEl.textContent = extra.id;
|
|
||||||
sum.appendChild(idEl);
|
|
||||||
}
|
|
||||||
det.appendChild(sum);
|
|
||||||
const label = document.createElement("span");
|
|
||||||
label.className = "tool-label";
|
|
||||||
label.textContent = "input";
|
|
||||||
det.appendChild(label);
|
|
||||||
const pre = document.createElement("pre");
|
|
||||||
det.appendChild(pre);
|
|
||||||
state.container.appendChild(det);
|
|
||||||
block.el = pre;
|
|
||||||
block.jsonBuf = "";
|
|
||||||
block.seedInput = extra.input;
|
|
||||||
}
|
|
||||||
state.blocks[index] = block;
|
|
||||||
return block;
|
|
||||||
}
|
|
||||||
|
|
||||||
function applyEvent(state, ev) {
|
|
||||||
const t = ev.type;
|
|
||||||
if (t === "content_block_start") {
|
|
||||||
const cb = ev.content_block || {};
|
|
||||||
if (cb.type === "text") {
|
|
||||||
ensureBlock(state, ev.index, "text", {});
|
|
||||||
} else if (cb.type === "thinking") {
|
|
||||||
ensureBlock(state, ev.index, "thinking", {});
|
|
||||||
} else if (cb.type === "tool_use") {
|
|
||||||
ensureBlock(state, ev.index, "tool_use",
|
|
||||||
{ name: cb.name, id: cb.id, input: cb.input });
|
|
||||||
}
|
|
||||||
} else if (t === "content_block_delta") {
|
|
||||||
const d = ev.delta || {};
|
|
||||||
const b = state.blocks[ev.index];
|
|
||||||
if (!b) return;
|
|
||||||
if (d.type === "text_delta") {
|
|
||||||
b.el.textContent += d.text || "";
|
|
||||||
state.assistantText += d.text || "";
|
|
||||||
scrollDown();
|
|
||||||
} else if (d.type === "thinking_delta") {
|
|
||||||
b.el.textContent += d.thinking || "";
|
|
||||||
scrollDown();
|
|
||||||
} else if (d.type === "input_json_delta") {
|
|
||||||
b.jsonBuf += d.partial_json || "";
|
|
||||||
}
|
|
||||||
} else if (t === "content_block_stop") {
|
|
||||||
const b = state.blocks[ev.index];
|
|
||||||
if (!b) return;
|
|
||||||
if (b.type === "tool_use") {
|
|
||||||
let input = null;
|
|
||||||
if (b.jsonBuf && b.jsonBuf.trim()) {
|
|
||||||
try { input = JSON.parse(b.jsonBuf); }
|
|
||||||
catch { input = b.jsonBuf; }
|
|
||||||
} else if (b.seedInput !== undefined && b.seedInput !== null) {
|
|
||||||
input = b.seedInput;
|
|
||||||
}
|
|
||||||
b.el.textContent = input === null
|
|
||||||
? "(no input)"
|
|
||||||
: (typeof input === "string"
|
|
||||||
? input
|
|
||||||
: JSON.stringify(input, null, 2));
|
|
||||||
}
|
|
||||||
} else if (t === "error") {
|
|
||||||
const msg = (ev.error && ev.error.message) || "stream error";
|
|
||||||
appendError(msg);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async function send() {
|
|
||||||
const text = textEl.value.trim();
|
|
||||||
if (!text) return;
|
|
||||||
const model = agentSelect.value;
|
|
||||||
if (!model) { appendError("no agent selected"); return; }
|
|
||||||
|
|
||||||
apiMessages.push({ role: "user", content: text });
|
|
||||||
appendUser(text);
|
|
||||||
textEl.value = "";
|
|
||||||
sendBtn.disabled = true;
|
|
||||||
|
|
||||||
const container = appendAssistant();
|
|
||||||
const state = { container, blocks: {}, assistantText: "" };
|
|
||||||
|
|
||||||
let resp;
|
|
||||||
try {
|
|
||||||
resp = await fetch(URL_PREFIX + "/chat/send", {
|
|
||||||
method: "POST",
|
|
||||||
headers: {
|
|
||||||
"Content-Type": "application/json",
|
|
||||||
"X-CSRF-Token": CSRF,
|
|
||||||
},
|
|
||||||
body: JSON.stringify({ model, messages: apiMessages }),
|
|
||||||
});
|
|
||||||
} catch (e) {
|
|
||||||
appendError("network error: " + e.message);
|
|
||||||
sendBtn.disabled = false;
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
if (!resp.ok || !resp.body) {
|
|
||||||
let msg = resp.status + " " + resp.statusText;
|
|
||||||
try { const body = await resp.text(); if (body) msg = body; } catch {}
|
|
||||||
appendError(msg);
|
|
||||||
sendBtn.disabled = false;
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
const reader = resp.body.getReader();
|
|
||||||
const dec = new TextDecoder();
|
|
||||||
let buf = "";
|
|
||||||
while (true) {
|
|
||||||
const { value, done } = await reader.read();
|
|
||||||
if (done) break;
|
|
||||||
buf += dec.decode(value, { stream: true });
|
|
||||||
let i;
|
|
||||||
while ((i = buf.indexOf("\n\n")) >= 0) {
|
|
||||||
const raw = buf.slice(0, i);
|
|
||||||
buf = buf.slice(i + 2);
|
|
||||||
if (!raw) continue;
|
|
||||||
let dataLine = "";
|
|
||||||
for (const line of raw.split("\n")) {
|
|
||||||
if (line.startsWith("data:")) dataLine += line.slice(5).trimStart();
|
|
||||||
}
|
|
||||||
if (!dataLine) continue;
|
|
||||||
let payload;
|
|
||||||
try { payload = JSON.parse(dataLine); } catch { continue; }
|
|
||||||
applyEvent(state, payload);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if (state.assistantText) {
|
|
||||||
apiMessages.push({ role: "assistant", content: state.assistantText });
|
|
||||||
}
|
|
||||||
sendBtn.disabled = false;
|
|
||||||
textEl.focus();
|
|
||||||
}
|
|
||||||
|
|
||||||
form.addEventListener("submit", (e) => { e.preventDefault(); send(); });
|
|
||||||
textEl.addEventListener("keydown", (e) => {
|
|
||||||
if ((e.metaKey || e.ctrlKey) && e.key === "Enter") {
|
|
||||||
e.preventDefault();
|
|
||||||
send();
|
|
||||||
}
|
|
||||||
});
|
|
||||||
})();
|
|
||||||
</script>
|
|
||||||
{% endblock %}
|
|
||||||
@@ -1,490 +0,0 @@
|
|||||||
{% extends "_layout.html" %}
|
|
||||||
{% set active = "dashboard" %}
|
|
||||||
{% block title %}beaver-gateway · dashboard{% endblock %}
|
|
||||||
{% block content %}
|
|
||||||
<h2>Agents</h2>
|
|
||||||
<div class="card">
|
|
||||||
{% if agents %}
|
|
||||||
<table>
|
|
||||||
<thead><tr><th>Name</th><th>Type</th><th>Model</th><th>Exposed MCPs</th></tr></thead>
|
|
||||||
<tbody>
|
|
||||||
{% for a in agents %}
|
|
||||||
<tr>
|
|
||||||
<td><code>{{ a.name }}</code></td>
|
|
||||||
<td><span class="pill">{{ a.__class__.__name__ }}</span></td>
|
|
||||||
<td><code>{{ a.model }}</code></td>
|
|
||||||
<td>
|
|
||||||
{% for em in a.expose_mcps %}<code>{{ em.name }}</code>{% if not loop.last %}, {% endif %}{% else %}<span class="muted">—</span>{% endfor %}
|
|
||||||
</td>
|
|
||||||
</tr>
|
|
||||||
{% endfor %}
|
|
||||||
</tbody>
|
|
||||||
</table>
|
|
||||||
{% else %}
|
|
||||||
<p class="muted">No agents configured.</p>
|
|
||||||
{% endif %}
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<h2>MCP namespaces</h2>
|
|
||||||
<div class="card">
|
|
||||||
{% if mcps %}
|
|
||||||
<table>
|
|
||||||
<thead><tr><th>Name</th><th>Kind</th></tr></thead>
|
|
||||||
<tbody>
|
|
||||||
{% for m in mcps %}
|
|
||||||
<tr>
|
|
||||||
<td><code>{{ m.name }}</code></td>
|
|
||||||
<td><span class="pill">{{ m.kind }}</span></td>
|
|
||||||
</tr>
|
|
||||||
{% endfor %}
|
|
||||||
</tbody>
|
|
||||||
</table>
|
|
||||||
{% else %}
|
|
||||||
<p class="muted">No MCP servers configured.</p>
|
|
||||||
{% endif %}
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<h2>Endpoints</h2>
|
|
||||||
<div class="card endpoints">
|
|
||||||
{% if endpoints.agents or endpoints.mcps %}
|
|
||||||
<div class="ep-controls">
|
|
||||||
<label class="ep-token">
|
|
||||||
<span>Token (hint)</span>
|
|
||||||
<select id="ep-token-select">
|
|
||||||
<option value="" data-scope="">— select a known token —</option>
|
|
||||||
{% for t in tokens %}
|
|
||||||
<option value="{{ t.name }}" data-scope="{{ t.scope }}">{{ t.name }} · scope {{ t.scope }}</option>
|
|
||||||
{% endfor %}
|
|
||||||
</select>
|
|
||||||
</label>
|
|
||||||
<label class="ep-secret">
|
|
||||||
<span>Bearer secret</span>
|
|
||||||
<input type="password" id="ep-token-secret" autocomplete="off" spellcheck="false"
|
|
||||||
placeholder="paste plaintext (we only store the Argon2 hash)">
|
|
||||||
</label>
|
|
||||||
<label class="ep-show">
|
|
||||||
<input type="checkbox" id="ep-show-secret"> <span>show</span>
|
|
||||||
</label>
|
|
||||||
</div>
|
|
||||||
<p class="muted ep-note">
|
|
||||||
Beaver stores only an Argon2 hash of each token, so the plaintext can't be reconstructed.
|
|
||||||
Paste the value you saved at creation; if you've lost it, <a href="{{ p }}/tokens">mint a new one</a>.
|
|
||||||
Below: pick a token to see which endpoints its scope covers, paste the secret to fill it
|
|
||||||
into URL / curl, then click Copy.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
{% if endpoints.agents %}
|
|
||||||
<h3 class="ep-h3">Agents — <code>POST /v1/messages</code></h3>
|
|
||||||
<table class="ep-table" data-required-scope="messages">
|
|
||||||
<thead><tr><th>Agent</th><th>Model</th><th>URL</th><th class="ep-actions-th"></th></tr></thead>
|
|
||||||
<tbody>
|
|
||||||
{% for ep in endpoints.agents %}
|
|
||||||
<tr class="ep-row" data-kind="messages" data-agent="{{ ep.agent }}" data-url="{{ ep.url }}">
|
|
||||||
<td>
|
|
||||||
<code>{{ ep.agent }}</code>
|
|
||||||
<span class="pill">{{ ep.agent_type }}</span>
|
|
||||||
<span class="ep-scope-warn" hidden>scope mismatch</span>
|
|
||||||
</td>
|
|
||||||
<td><code>{{ ep.model }}</code></td>
|
|
||||||
<td><code class="ep-url">{{ ep.url }}</code></td>
|
|
||||||
<td class="ep-actions">
|
|
||||||
<button type="button" data-action="copy-url">Copy URL</button>
|
|
||||||
<button type="button" data-action="copy-curl">Copy curl</button>
|
|
||||||
<button type="button" data-action="toggle-curl" aria-expanded="false">▸ curl</button>
|
|
||||||
</td>
|
|
||||||
</tr>
|
|
||||||
<tr class="ep-curl-row" hidden><td colspan="4"><pre class="ep-curl"></pre></td></tr>
|
|
||||||
{% endfor %}
|
|
||||||
</tbody>
|
|
||||||
</table>
|
|
||||||
{% elif endpoints.anthropic_base is none and agents %}
|
|
||||||
<p class="muted">
|
|
||||||
Agents are declared but no <code>AnthropicMessagesFrontend</code> is configured —
|
|
||||||
add one to <code>Gateway(frontends=[...])</code> to expose them over HTTP.
|
|
||||||
</p>
|
|
||||||
{% endif %}
|
|
||||||
|
|
||||||
{% if endpoints.mcps %}
|
|
||||||
<h3 class="ep-h3">MCP — streamable HTTP</h3>
|
|
||||||
<table class="ep-table" data-required-scope="mcp">
|
|
||||||
<thead><tr><th>Namespace</th><th>Kind</th><th>URL</th><th class="ep-actions-th"></th></tr></thead>
|
|
||||||
<tbody>
|
|
||||||
{% for ep in endpoints.mcps %}
|
|
||||||
<tr class="ep-row" data-kind="mcp" data-namespace="{{ ep.namespace }}" data-url="{{ ep.url }}">
|
|
||||||
<td>
|
|
||||||
<code>{{ ep.namespace }}</code>
|
|
||||||
<span class="ep-scope-warn" hidden>scope mismatch</span>
|
|
||||||
</td>
|
|
||||||
<td><span class="pill">{{ ep.kind }}</span></td>
|
|
||||||
<td><code class="ep-url">{{ ep.url }}</code></td>
|
|
||||||
<td class="ep-actions">
|
|
||||||
<button type="button" data-action="copy-url">Copy URL</button>
|
|
||||||
<button type="button" data-action="copy-url-token">Copy URL+token</button>
|
|
||||||
<button type="button" data-action="copy-curl">Copy curl</button>
|
|
||||||
<button type="button" data-action="toggle-curl" aria-expanded="false">▸ curl</button>
|
|
||||||
</td>
|
|
||||||
</tr>
|
|
||||||
<tr class="ep-curl-row" hidden><td colspan="4"><pre class="ep-curl"></pre></td></tr>
|
|
||||||
{% endfor %}
|
|
||||||
</tbody>
|
|
||||||
</table>
|
|
||||||
{% elif endpoints.mcp_base is none and mcps %}
|
|
||||||
<p class="muted">
|
|
||||||
MCP servers are declared but no <code>McpServerFrontend</code> is configured —
|
|
||||||
add one to <code>Gateway(frontends=[...])</code> to expose them over HTTP.
|
|
||||||
</p>
|
|
||||||
{% endif %}
|
|
||||||
|
|
||||||
{% if endpoints.markdown %}
|
|
||||||
<h3 class="ep-h3">Markdown — <code>POST /chat</code></h3>
|
|
||||||
<table class="ep-table" data-required-scope="messages">
|
|
||||||
<thead><tr><th>Vault</th><th>Default agent</th><th>URL</th><th class="ep-actions-th"></th></tr></thead>
|
|
||||||
<tbody>
|
|
||||||
<tr class="ep-row"
|
|
||||||
data-kind="markdown"
|
|
||||||
data-url="{{ endpoints.markdown.url }}"
|
|
||||||
data-base="{{ endpoints.markdown.base }}"
|
|
||||||
data-sample-agent="{{ endpoints.markdown.sample_agent }}">
|
|
||||||
<td>
|
|
||||||
<code>{{ endpoints.markdown.vault_path }}</code>
|
|
||||||
<span class="ep-scope-warn" hidden>scope mismatch</span>
|
|
||||||
{% if endpoints.markdown.log_all_chats %}
|
|
||||||
<span class="pill">log_all_chats · {{ endpoints.markdown.logged_subdir }}/</span>
|
|
||||||
{% endif %}
|
|
||||||
</td>
|
|
||||||
<td>
|
|
||||||
{% if endpoints.markdown.default_agent %}
|
|
||||||
<code>{{ endpoints.markdown.default_agent }}</code>
|
|
||||||
{% else %}
|
|
||||||
<span class="muted">— (request must set <code>agent</code> or frontmatter)</span>
|
|
||||||
{% endif %}
|
|
||||||
</td>
|
|
||||||
<td>
|
|
||||||
<code class="ep-url">{{ endpoints.markdown.url }}</code>
|
|
||||||
<div class="muted" style="font-size:0.85em; margin-top:0.25rem;">
|
|
||||||
plugin base: <code>{{ endpoints.markdown.base }}</code>
|
|
||||||
</div>
|
|
||||||
</td>
|
|
||||||
<td class="ep-actions">
|
|
||||||
<button type="button" data-action="copy-url">Copy URL</button>
|
|
||||||
<button type="button" data-action="copy-base">Copy plugin base</button>
|
|
||||||
<button type="button" data-action="copy-curl">Copy curl</button>
|
|
||||||
<button type="button" data-action="toggle-curl" aria-expanded="false">▸ curl</button>
|
|
||||||
</td>
|
|
||||||
</tr>
|
|
||||||
<tr class="ep-curl-row" hidden><td colspan="4"><pre class="ep-curl"></pre></td></tr>
|
|
||||||
</tbody>
|
|
||||||
</table>
|
|
||||||
{% endif %}
|
|
||||||
{% else %}
|
|
||||||
<p class="muted">
|
|
||||||
Nothing exposed yet — declare agents / MCPs and the matching frontends
|
|
||||||
(<code>AnthropicMessagesFrontend</code>, <code>McpServerFrontend</code>) in your config.
|
|
||||||
</p>
|
|
||||||
{% endif %}
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<h2>Recent activity</h2>
|
|
||||||
<div class="card">
|
|
||||||
{% if audit %}
|
|
||||||
<table>
|
|
||||||
<thead><tr><th>Time</th><th>Actor</th><th>Kind</th><th>Agent</th><th>Detail</th></tr></thead>
|
|
||||||
<tbody>
|
|
||||||
{% for row in audit %}
|
|
||||||
<tr>
|
|
||||||
<td><code>{{ row.ts | fmt_dt }}</code></td>
|
|
||||||
<td>{{ row.actor }}</td>
|
|
||||||
<td><span class="pill">{{ row.kind }}</span></td>
|
|
||||||
<td>{{ row.agent_name or "—" }}</td>
|
|
||||||
<td><code>{{ row.detail_json | fmt_detail }}</code></td>
|
|
||||||
</tr>
|
|
||||||
{% endfor %}
|
|
||||||
</tbody>
|
|
||||||
</table>
|
|
||||||
<p class="muted" style="margin-top:0.85rem;"><a href="{{ p }}/audit">Full log →</a></p>
|
|
||||||
{% else %}
|
|
||||||
<p class="muted">Nothing logged yet.</p>
|
|
||||||
{% endif %}
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<style>
|
|
||||||
.endpoints .ep-controls {
|
|
||||||
display: grid;
|
|
||||||
grid-template-columns: minmax(220px, 1fr) 2fr auto;
|
|
||||||
gap: 0.85rem;
|
|
||||||
align-items: end;
|
|
||||||
margin-bottom: 0.5rem;
|
|
||||||
}
|
|
||||||
.endpoints .ep-controls label > span {
|
|
||||||
display: block;
|
|
||||||
font-size: 0.8em; color: var(--muted);
|
|
||||||
text-transform: uppercase; letter-spacing: 0.05em;
|
|
||||||
margin-bottom: 0.3rem;
|
|
||||||
}
|
|
||||||
.endpoints .ep-show {
|
|
||||||
display: flex; align-items: center; gap: 0.4rem;
|
|
||||||
padding-bottom: 0.6rem; color: var(--muted); font-size: 0.9em;
|
|
||||||
}
|
|
||||||
.endpoints .ep-note {
|
|
||||||
font-size: 0.85em; margin: 0.25rem 0 1.2rem;
|
|
||||||
}
|
|
||||||
.endpoints .ep-h3 {
|
|
||||||
font-size: 0.95rem; font-weight: 600;
|
|
||||||
margin: 1.5rem 0 0.55rem;
|
|
||||||
}
|
|
||||||
.endpoints .ep-table { table-layout: auto; }
|
|
||||||
.endpoints .ep-table th.ep-actions-th { width: 1%; }
|
|
||||||
.endpoints .ep-url {
|
|
||||||
word-break: break-all;
|
|
||||||
}
|
|
||||||
.endpoints .ep-actions {
|
|
||||||
white-space: nowrap;
|
|
||||||
text-align: right;
|
|
||||||
}
|
|
||||||
.endpoints .ep-actions button {
|
|
||||||
padding: 0.32rem 0.7rem;
|
|
||||||
font-size: 0.85em;
|
|
||||||
margin-left: 0.3rem;
|
|
||||||
}
|
|
||||||
.endpoints .ep-scope-warn {
|
|
||||||
display: inline-block;
|
|
||||||
margin-left: 0.5rem;
|
|
||||||
padding: 0.05rem 0.45rem;
|
|
||||||
background: #fff7e6;
|
|
||||||
border: 1px solid #fadf9c;
|
|
||||||
color: #8a5a00;
|
|
||||||
border-radius: 999px;
|
|
||||||
font-size: 0.72em;
|
|
||||||
}
|
|
||||||
.endpoints .ep-row.scope-mismatch td:not(.ep-actions) { opacity: 0.55; }
|
|
||||||
.endpoints .ep-row.scope-mismatch .ep-actions button { opacity: 0.7; }
|
|
||||||
.endpoints .ep-curl-row td { padding-top: 0; padding-bottom: 0; }
|
|
||||||
.endpoints .ep-curl {
|
|
||||||
margin: 0.4rem 0 1rem;
|
|
||||||
font-size: 0.82em;
|
|
||||||
max-height: 280px;
|
|
||||||
}
|
|
||||||
.endpoints button[data-copied="1"] {
|
|
||||||
background: #ecfdf3 !important;
|
|
||||||
border-color: #b6e6cb !important;
|
|
||||||
color: #027a48;
|
|
||||||
}
|
|
||||||
@media (max-width: 760px) {
|
|
||||||
.endpoints .ep-controls { grid-template-columns: 1fr; }
|
|
||||||
.endpoints .ep-actions { text-align: left; padding-top: 0.4rem; }
|
|
||||||
}
|
|
||||||
</style>
|
|
||||||
|
|
||||||
<script>
|
|
||||||
(() => {
|
|
||||||
const card = document.querySelector(".endpoints");
|
|
||||||
if (!card) return;
|
|
||||||
const sel = card.querySelector("#ep-token-select");
|
|
||||||
const secret = card.querySelector("#ep-token-secret");
|
|
||||||
const showCb = card.querySelector("#ep-show-secret");
|
|
||||||
if (!sel || !secret || !showCb) return;
|
|
||||||
|
|
||||||
const PLACEHOLDER = "<YOUR_TOKEN>";
|
|
||||||
|
|
||||||
function currentToken() {
|
|
||||||
const v = secret.value;
|
|
||||||
return v && v.length > 0 ? v : null;
|
|
||||||
}
|
|
||||||
|
|
||||||
function selectedScope() {
|
|
||||||
const o = sel.options[sel.selectedIndex];
|
|
||||||
return o ? (o.getAttribute("data-scope") || "") : "";
|
|
||||||
}
|
|
||||||
|
|
||||||
function scopeCovers(have, need) {
|
|
||||||
if (!have) return true; // no token selected — don't grey anything out
|
|
||||||
if (have === "*") return true;
|
|
||||||
return have === need;
|
|
||||||
}
|
|
||||||
|
|
||||||
function escSh(s) {
|
|
||||||
// single-quote shell escape: close, escape, reopen.
|
|
||||||
return "'" + String(s).replace(/'/g, "'\\''") + "'";
|
|
||||||
}
|
|
||||||
|
|
||||||
function buildCurl(row) {
|
|
||||||
const kind = row.getAttribute("data-kind");
|
|
||||||
const url = row.getAttribute("data-url");
|
|
||||||
const tok = currentToken() || PLACEHOLDER;
|
|
||||||
if (kind === "messages") {
|
|
||||||
const agent = row.getAttribute("data-agent");
|
|
||||||
const body = JSON.stringify({
|
|
||||||
model: agent,
|
|
||||||
messages: [{ role: "user", content: "hello" }],
|
|
||||||
stream: true,
|
|
||||||
});
|
|
||||||
return [
|
|
||||||
"curl -N \\",
|
|
||||||
" -H " + escSh("x-api-key: " + tok) + " \\",
|
|
||||||
" -H 'content-type: application/json' \\",
|
|
||||||
" -d " + escSh(body) + " \\",
|
|
||||||
" " + url,
|
|
||||||
].join("\n");
|
|
||||||
}
|
|
||||||
if (kind === "markdown") {
|
|
||||||
const agent = row.getAttribute("data-sample-agent") || "";
|
|
||||||
const body = JSON.stringify({
|
|
||||||
filename: "example.md",
|
|
||||||
content: "### User:\n\nhello\n",
|
|
||||||
agent: agent,
|
|
||||||
});
|
|
||||||
return [
|
|
||||||
"curl \\",
|
|
||||||
" -H " + escSh("Authorization: Bearer " + tok) + " \\",
|
|
||||||
" -H 'content-type: application/json' \\",
|
|
||||||
" -d " + escSh(body) + " \\",
|
|
||||||
" " + url,
|
|
||||||
].join("\n");
|
|
||||||
}
|
|
||||||
if (kind === "mcp") {
|
|
||||||
// Streamable-HTTP MCP wants `initialize` first — the response
|
|
||||||
// carries the `Mcp-Session-Id` header you must echo back on
|
|
||||||
// every subsequent call (tools/list, tools/call, ...). We use
|
|
||||||
// `-i` so the session-id is visible in the response, and ship
|
|
||||||
// the proper handshake body so a fresh paste actually works.
|
|
||||||
const body = JSON.stringify({
|
|
||||||
jsonrpc: "2.0",
|
|
||||||
id: 1,
|
|
||||||
method: "initialize",
|
|
||||||
params: {
|
|
||||||
protocolVersion: "2024-11-05",
|
|
||||||
capabilities: {},
|
|
||||||
clientInfo: { name: "beaver-admin-curl", version: "0" },
|
|
||||||
},
|
|
||||||
});
|
|
||||||
return [
|
|
||||||
"# 1) initialize — grab Mcp-Session-Id from the response headers",
|
|
||||||
"curl -N -i \\",
|
|
||||||
" -H " + escSh("Authorization: Bearer " + tok) + " \\",
|
|
||||||
" -H 'content-type: application/json' \\",
|
|
||||||
" -H 'accept: application/json, text/event-stream' \\",
|
|
||||||
" -d " + escSh(body) + " \\",
|
|
||||||
" " + url,
|
|
||||||
"",
|
|
||||||
"# 2) reuse that id on follow-up calls, e.g. tools/list:",
|
|
||||||
"# curl -N \\",
|
|
||||||
"# -H " + escSh("Authorization: Bearer " + tok) + " \\",
|
|
||||||
"# -H 'Mcp-Session-Id: <paste-from-step-1>' \\",
|
|
||||||
"# -H 'content-type: application/json' \\",
|
|
||||||
"# -H 'accept: application/json, text/event-stream' \\",
|
|
||||||
"# -d '{\"jsonrpc\":\"2.0\",\"id\":2,\"method\":\"tools/list\"}' \\",
|
|
||||||
"# " + url,
|
|
||||||
].join("\n");
|
|
||||||
}
|
|
||||||
return "";
|
|
||||||
}
|
|
||||||
|
|
||||||
function refreshScopeMarks() {
|
|
||||||
const have = selectedScope();
|
|
||||||
card.querySelectorAll(".ep-table").forEach((tbl) => {
|
|
||||||
const need = tbl.getAttribute("data-required-scope") || "";
|
|
||||||
const ok = scopeCovers(have, need);
|
|
||||||
tbl.querySelectorAll("tr.ep-row").forEach((row) => {
|
|
||||||
const warn = row.querySelector(".ep-scope-warn");
|
|
||||||
if (!ok) {
|
|
||||||
row.classList.add("scope-mismatch");
|
|
||||||
if (warn) warn.hidden = false;
|
|
||||||
} else {
|
|
||||||
row.classList.remove("scope-mismatch");
|
|
||||||
if (warn) warn.hidden = true;
|
|
||||||
}
|
|
||||||
});
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
function refreshOpenCurls() {
|
|
||||||
card.querySelectorAll("tr.ep-curl-row").forEach((cr) => {
|
|
||||||
if (cr.hidden) return;
|
|
||||||
const row = cr.previousElementSibling;
|
|
||||||
const pre = cr.querySelector(".ep-curl");
|
|
||||||
if (row && pre) pre.textContent = buildCurl(row);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
function copyText(btn, text) {
|
|
||||||
const done = () => {
|
|
||||||
btn.setAttribute("data-copied", "1");
|
|
||||||
const prev = btn.textContent;
|
|
||||||
btn.textContent = "✓ copied";
|
|
||||||
setTimeout(() => {
|
|
||||||
btn.removeAttribute("data-copied");
|
|
||||||
btn.textContent = prev;
|
|
||||||
}, 1200);
|
|
||||||
};
|
|
||||||
if (navigator.clipboard && navigator.clipboard.writeText) {
|
|
||||||
navigator.clipboard.writeText(text).then(done, () => {
|
|
||||||
// Fallback for non-secure contexts.
|
|
||||||
fallbackCopy(text);
|
|
||||||
done();
|
|
||||||
});
|
|
||||||
} else {
|
|
||||||
fallbackCopy(text);
|
|
||||||
done();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
function fallbackCopy(text) {
|
|
||||||
const ta = document.createElement("textarea");
|
|
||||||
ta.value = text;
|
|
||||||
ta.style.position = "fixed"; ta.style.opacity = "0";
|
|
||||||
document.body.appendChild(ta);
|
|
||||||
ta.select();
|
|
||||||
try { document.execCommand("copy"); } catch {}
|
|
||||||
document.body.removeChild(ta);
|
|
||||||
}
|
|
||||||
|
|
||||||
sel.addEventListener("change", () => {
|
|
||||||
const name = sel.value;
|
|
||||||
secret.placeholder = name
|
|
||||||
? "paste plaintext for '" + name + "' (we only store the hash)"
|
|
||||||
: "paste plaintext (we only store the Argon2 hash)";
|
|
||||||
refreshScopeMarks();
|
|
||||||
});
|
|
||||||
secret.addEventListener("input", refreshOpenCurls);
|
|
||||||
showCb.addEventListener("change", () => {
|
|
||||||
secret.type = showCb.checked ? "text" : "password";
|
|
||||||
});
|
|
||||||
|
|
||||||
card.addEventListener("click", (ev) => {
|
|
||||||
const btn = ev.target.closest("button[data-action]");
|
|
||||||
if (!btn) return;
|
|
||||||
const row = btn.closest("tr.ep-row");
|
|
||||||
if (!row) return;
|
|
||||||
const action = btn.getAttribute("data-action");
|
|
||||||
const url = row.getAttribute("data-url");
|
|
||||||
const tok = currentToken() || PLACEHOLDER;
|
|
||||||
if (action === "copy-url") {
|
|
||||||
copyText(btn, url);
|
|
||||||
} else if (action === "copy-base") {
|
|
||||||
const base = row.getAttribute("data-base");
|
|
||||||
if (base) copyText(btn, base);
|
|
||||||
} else if (action === "copy-url-token") {
|
|
||||||
const sep = url.indexOf("?") >= 0 ? "&" : "?";
|
|
||||||
copyText(btn, url + sep + "token=" + encodeURIComponent(tok));
|
|
||||||
} else if (action === "copy-curl") {
|
|
||||||
copyText(btn, buildCurl(row));
|
|
||||||
} else if (action === "toggle-curl") {
|
|
||||||
const curlRow = row.nextElementSibling;
|
|
||||||
if (!curlRow || !curlRow.classList.contains("ep-curl-row")) return;
|
|
||||||
const pre = curlRow.querySelector(".ep-curl");
|
|
||||||
if (curlRow.hidden) {
|
|
||||||
if (pre) pre.textContent = buildCurl(row);
|
|
||||||
curlRow.hidden = false;
|
|
||||||
btn.textContent = "▾ curl";
|
|
||||||
btn.setAttribute("aria-expanded", "true");
|
|
||||||
} else {
|
|
||||||
curlRow.hidden = true;
|
|
||||||
btn.textContent = "▸ curl";
|
|
||||||
btn.setAttribute("aria-expanded", "false");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
refreshScopeMarks();
|
|
||||||
})();
|
|
||||||
</script>
|
|
||||||
{% endblock %}
|
|
||||||
@@ -1,83 +0,0 @@
|
|||||||
<!doctype html>
|
|
||||||
<html lang="en">
|
|
||||||
<head>
|
|
||||||
<meta charset="utf-8">
|
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
||||||
<title>beaver-gateway · log in</title>
|
|
||||||
<style>
|
|
||||||
html, body { margin: 0; padding: 0; height: 100%; background: #fbfbfd; color: #1d1d1f; }
|
|
||||||
body {
|
|
||||||
font-family: -apple-system, "SF Pro Display", system-ui, sans-serif;
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
justify-content: center;
|
|
||||||
min-height: 100vh;
|
|
||||||
}
|
|
||||||
.card {
|
|
||||||
background: #fff;
|
|
||||||
border: 1px solid #e5e5ea;
|
|
||||||
border-radius: 16px;
|
|
||||||
padding: 2rem 2.25rem;
|
|
||||||
width: min(360px, 92vw);
|
|
||||||
box-shadow: 0 10px 24px rgba(0,0,0,0.04);
|
|
||||||
}
|
|
||||||
h1 {
|
|
||||||
margin: 0 0 0.25rem;
|
|
||||||
font-size: 1.25rem;
|
|
||||||
font-weight: 600;
|
|
||||||
letter-spacing: -0.02em;
|
|
||||||
}
|
|
||||||
p.muted { color: #6e6e73; margin: 0 0 1.4rem; font-size: 0.92em; }
|
|
||||||
label { display: block; margin: 0.85rem 0 0.3rem; font-size: 0.82em; color: #6e6e73; }
|
|
||||||
input {
|
|
||||||
width: 100%;
|
|
||||||
padding: 0.6rem 0.75rem;
|
|
||||||
font-family: inherit;
|
|
||||||
font-size: 0.95em;
|
|
||||||
border: 1px solid #e5e5ea;
|
|
||||||
border-radius: 8px;
|
|
||||||
background: #fff;
|
|
||||||
box-sizing: border-box;
|
|
||||||
}
|
|
||||||
input:focus { outline: 2px solid #0071e3; outline-offset: 1px; }
|
|
||||||
button {
|
|
||||||
width: 100%;
|
|
||||||
margin-top: 1.25rem;
|
|
||||||
padding: 0.65rem;
|
|
||||||
font-family: inherit;
|
|
||||||
font-size: 0.95em;
|
|
||||||
background: #0071e3;
|
|
||||||
border: 1px solid #0071e3;
|
|
||||||
color: white;
|
|
||||||
border-radius: 8px;
|
|
||||||
cursor: pointer;
|
|
||||||
}
|
|
||||||
button:hover { background: #005bb5; }
|
|
||||||
.error {
|
|
||||||
background: #fff0f0;
|
|
||||||
border: 1px solid #f5c2c5;
|
|
||||||
color: #d70015;
|
|
||||||
padding: 0.65rem 0.85rem;
|
|
||||||
border-radius: 8px;
|
|
||||||
font-size: 0.9em;
|
|
||||||
margin-bottom: 1rem;
|
|
||||||
}
|
|
||||||
</style>
|
|
||||||
</head>
|
|
||||||
<body>
|
|
||||||
<div class="card">
|
|
||||||
<h1>beaver-gateway</h1>
|
|
||||||
<p class="muted">Sign in to manage tokens and view audit logs.</p>
|
|
||||||
{% if error %}
|
|
||||||
<div class="error">{{ error }}</div>
|
|
||||||
{% endif %}
|
|
||||||
<form method="post" action="{{ p }}/login">
|
|
||||||
<label for="username">Username</label>
|
|
||||||
<input id="username" name="username" type="text" autocomplete="username" required autofocus>
|
|
||||||
<label for="password">Password</label>
|
|
||||||
<input id="password" name="password" type="password" autocomplete="current-password" required>
|
|
||||||
<button type="submit">Sign in</button>
|
|
||||||
</form>
|
|
||||||
</div>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
@@ -1,146 +0,0 @@
|
|||||||
{% extends "_layout.html" %}
|
|
||||||
{% set active = "pty" %}
|
|
||||||
{% block title %}PTY {{ session_id }} · beaver-gateway{% endblock %}
|
|
||||||
{% block content %}
|
|
||||||
<style>
|
|
||||||
/* xterm.js styles inlined from the CDN bundle — keep them tight and
|
|
||||||
constrained to the terminal block so they don't leak into the rest
|
|
||||||
of the admin UI. */
|
|
||||||
.term-wrap {
|
|
||||||
background: #000;
|
|
||||||
border-radius: 12px;
|
|
||||||
padding: 12px;
|
|
||||||
border: 1px solid var(--line);
|
|
||||||
}
|
|
||||||
.term-host {
|
|
||||||
height: 70vh;
|
|
||||||
}
|
|
||||||
.term-host .xterm {
|
|
||||||
height: 100%;
|
|
||||||
}
|
|
||||||
.term-toolbar {
|
|
||||||
display: flex;
|
|
||||||
gap: 0.5rem;
|
|
||||||
align-items: center;
|
|
||||||
margin-bottom: 0.75rem;
|
|
||||||
flex-wrap: wrap;
|
|
||||||
}
|
|
||||||
.term-toolbar .status {
|
|
||||||
margin-left: auto;
|
|
||||||
font-size: 0.85em;
|
|
||||||
color: var(--muted);
|
|
||||||
}
|
|
||||||
.term-toolbar .status.connected { color: #027a48; }
|
|
||||||
.term-toolbar .status.dropped { color: var(--danger); }
|
|
||||||
</style>
|
|
||||||
|
|
||||||
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@xterm/xterm@5.5.0/css/xterm.min.css">
|
|
||||||
|
|
||||||
<h2>PTY <code>{{ session_id }}</code></h2>
|
|
||||||
<p class="muted">
|
|
||||||
Live view of the claude subprocess TUI. Keystrokes you type here go
|
|
||||||
straight into its stdin — Enter, arrows, Ctrl-C, paste, all work.
|
|
||||||
<a href="{{ p }}/pty">← back to list</a>
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<div class="term-toolbar">
|
|
||||||
<button id="reconnect-btn" type="button">Reconnect</button>
|
|
||||||
<button id="enter-btn" type="button" title="Send a single \r — handy if a paste is stuck in the input box">Send Enter</button>
|
|
||||||
<button id="ctrl-c-btn" type="button">Send Ctrl-C</button>
|
|
||||||
<span id="status" class="status">connecting…</span>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="term-wrap">
|
|
||||||
<div id="term" class="term-host"></div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<script src="https://cdn.jsdelivr.net/npm/@xterm/xterm@5.5.0/lib/xterm.min.js"></script>
|
|
||||||
<script src="https://cdn.jsdelivr.net/npm/@xterm/addon-fit@0.10.0/lib/addon-fit.min.js"></script>
|
|
||||||
<script>
|
|
||||||
(function () {
|
|
||||||
const sessionId = {{ session_id|tojson }};
|
|
||||||
const wsBase =
|
|
||||||
(location.protocol === "https:" ? "wss://" : "ws://") +
|
|
||||||
location.host +
|
|
||||||
{{ p|tojson }} +
|
|
||||||
"/pty/" + encodeURIComponent(sessionId) + "/ws";
|
|
||||||
|
|
||||||
const term = new Terminal({
|
|
||||||
fontFamily: 'ui-monospace, "SF Mono", Menlo, Consolas, monospace',
|
|
||||||
fontSize: 13,
|
|
||||||
theme: {
|
|
||||||
background: "#000000",
|
|
||||||
foreground: "#d0d0d0",
|
|
||||||
},
|
|
||||||
convertEol: false,
|
|
||||||
cursorBlink: true,
|
|
||||||
scrollback: 5000,
|
|
||||||
});
|
|
||||||
const fit = new FitAddon.FitAddon();
|
|
||||||
term.loadAddon(fit);
|
|
||||||
term.open(document.getElementById("term"));
|
|
||||||
// Defer fit until after layout settles; xterm's measurement reads
|
|
||||||
// computed CSS that isn't stable until the page paints once.
|
|
||||||
requestAnimationFrame(() => fit.fit());
|
|
||||||
window.addEventListener("resize", () => fit.fit());
|
|
||||||
|
|
||||||
const statusEl = document.getElementById("status");
|
|
||||||
function setStatus(text, cls) {
|
|
||||||
statusEl.textContent = text;
|
|
||||||
statusEl.className = "status" + (cls ? " " + cls : "");
|
|
||||||
}
|
|
||||||
|
|
||||||
let ws = null;
|
|
||||||
let manuallyClosed = false;
|
|
||||||
|
|
||||||
function connect() {
|
|
||||||
manuallyClosed = false;
|
|
||||||
setStatus("connecting…", "");
|
|
||||||
ws = new WebSocket(wsBase);
|
|
||||||
ws.binaryType = "arraybuffer";
|
|
||||||
ws.onopen = () => setStatus("connected", "connected");
|
|
||||||
ws.onclose = () => {
|
|
||||||
if (!manuallyClosed) setStatus("disconnected", "dropped");
|
|
||||||
};
|
|
||||||
ws.onerror = () => setStatus("error", "dropped");
|
|
||||||
ws.onmessage = (ev) => {
|
|
||||||
if (typeof ev.data === "string") {
|
|
||||||
term.write(ev.data);
|
|
||||||
} else {
|
|
||||||
// ArrayBuffer — feed raw bytes to xterm. It expects either
|
|
||||||
// string or Uint8Array; the latter preserves byte boundaries
|
|
||||||
// exactly, which matters for ANSI sequences split across frames.
|
|
||||||
term.write(new Uint8Array(ev.data));
|
|
||||||
}
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
// Keystrokes -> server. xterm.js gives us the exact byte sequence
|
|
||||||
// the terminal would emit (e.g. Enter -> "\r", arrows -> "\x1b[A"
|
|
||||||
// etc.), so we just pass it through.
|
|
||||||
term.onData((data) => {
|
|
||||||
if (ws && ws.readyState === WebSocket.OPEN) {
|
|
||||||
ws.send(data);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
document.getElementById("reconnect-btn").addEventListener("click", () => {
|
|
||||||
if (ws) {
|
|
||||||
manuallyClosed = true;
|
|
||||||
ws.close();
|
|
||||||
}
|
|
||||||
connect();
|
|
||||||
});
|
|
||||||
document.getElementById("enter-btn").addEventListener("click", () => {
|
|
||||||
if (ws && ws.readyState === WebSocket.OPEN) ws.send("\r");
|
|
||||||
term.focus();
|
|
||||||
});
|
|
||||||
document.getElementById("ctrl-c-btn").addEventListener("click", () => {
|
|
||||||
if (ws && ws.readyState === WebSocket.OPEN) ws.send("\x03");
|
|
||||||
term.focus();
|
|
||||||
});
|
|
||||||
|
|
||||||
connect();
|
|
||||||
})();
|
|
||||||
</script>
|
|
||||||
{% endblock %}
|
|
||||||
@@ -1,46 +0,0 @@
|
|||||||
{% extends "_layout.html" %}
|
|
||||||
{% set active = "pty" %}
|
|
||||||
{% block title %}PTY · beaver-gateway · admin{% endblock %}
|
|
||||||
{% block content %}
|
|
||||||
<h2>Live PTY sessions</h2>
|
|
||||||
<p class="muted">
|
|
||||||
Each row is a running <code>claude</code> subprocess, most recently used first.
|
|
||||||
Open one to see what's currently rendered on its TUI and (if needed) type into
|
|
||||||
it directly. Sessions idle past the agent's TTL are terminated automatically.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
{% if sessions %}
|
|
||||||
<div class="card" style="padding: 0;">
|
|
||||||
<table>
|
|
||||||
<thead>
|
|
||||||
<tr>
|
|
||||||
<th>Session ID</th>
|
|
||||||
<th>Agent</th>
|
|
||||||
<th>Idle</th>
|
|
||||||
<th>Age</th>
|
|
||||||
<th>PID</th>
|
|
||||||
<th>Buffer</th>
|
|
||||||
<th></th>
|
|
||||||
</tr>
|
|
||||||
</thead>
|
|
||||||
<tbody>
|
|
||||||
{% for s in sessions %}
|
|
||||||
<tr>
|
|
||||||
<td><code>{{ s.session_id }}</code></td>
|
|
||||||
<td>{{ s.agent }}</td>
|
|
||||||
<td>{{ s.idle }}</td>
|
|
||||||
<td class="muted">{{ s.age }}</td>
|
|
||||||
<td><span class="pill">{{ s.pid or "?" }}</span></td>
|
|
||||||
<td>{{ s.buffer_size }} bytes</td>
|
|
||||||
<td>
|
|
||||||
<a class="btn primary" href="{{ p }}/pty/{{ s.session_id }}">Open</a>
|
|
||||||
</td>
|
|
||||||
</tr>
|
|
||||||
{% endfor %}
|
|
||||||
</tbody>
|
|
||||||
</table>
|
|
||||||
</div>
|
|
||||||
{% else %}
|
|
||||||
<div class="banner">No live PTY sessions right now. Start a turn and refresh.</div>
|
|
||||||
{% endif %}
|
|
||||||
{% endblock %}
|
|
||||||
@@ -1,64 +0,0 @@
|
|||||||
{% extends "_layout.html" %}
|
|
||||||
{% set active = "tokens" %}
|
|
||||||
{% block title %}beaver-gateway · tokens{% endblock %}
|
|
||||||
{% block content %}
|
|
||||||
<h2>Create token</h2>
|
|
||||||
<div class="card">
|
|
||||||
<p class="muted">Plaintext is shown <strong>once</strong>, immediately after creation. Copy it before you navigate away — the database only ever holds the Argon2 hash.</p>
|
|
||||||
<div id="token-create-result"></div>
|
|
||||||
<form
|
|
||||||
hx-post="{{ p }}/tokens"
|
|
||||||
hx-target="#token-create-result"
|
|
||||||
hx-swap="innerHTML"
|
|
||||||
hx-on::after-request="if(event.detail.successful){this.reset();}"
|
|
||||||
>
|
|
||||||
<input type="hidden" name="csrf_token" value="{{ csrf }}">
|
|
||||||
<div class="form-grid">
|
|
||||||
<div>
|
|
||||||
<label for="name">Name</label>
|
|
||||||
<input id="name" name="name" type="text" placeholder="cursor / claude-desktop / mobile …" required>
|
|
||||||
</div>
|
|
||||||
<div>
|
|
||||||
<label for="scope">Scope</label>
|
|
||||||
<select id="scope" name="scope">
|
|
||||||
{% for s in scopes %}
|
|
||||||
<option value="{{ s }}" {% if s == "*" %}selected{% endif %}>{{ s }}</option>
|
|
||||||
{% endfor %}
|
|
||||||
</select>
|
|
||||||
</div>
|
|
||||||
<div>
|
|
||||||
<button class="primary" type="submit">Create</button>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</form>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<h2>
|
|
||||||
Tokens
|
|
||||||
<span class="muted" style="font-size:0.8em; font-weight:400; margin-left:0.5rem;">
|
|
||||||
{% if include_revoked %}
|
|
||||||
<a href="{{ p }}/tokens">Hide revoked</a>
|
|
||||||
{% else %}
|
|
||||||
<a href="{{ p }}/tokens?include_revoked=1">Show revoked</a>
|
|
||||||
{% endif %}
|
|
||||||
</span>
|
|
||||||
</h2>
|
|
||||||
<div class="card">
|
|
||||||
<table>
|
|
||||||
<thead>
|
|
||||||
<tr>
|
|
||||||
<th>Name</th><th>Scope</th><th>Created</th><th>Last used</th><th>Revoked</th><th></th>
|
|
||||||
</tr>
|
|
||||||
</thead>
|
|
||||||
{# Render the tbody unconditionally so the HTMX OOB swap on
|
|
||||||
create has a target even when the table starts empty. #}
|
|
||||||
<tbody id="tokens-rows">
|
|
||||||
{% for token in tokens %}
|
|
||||||
{% include "_token_row.html" %}
|
|
||||||
{% else %}
|
|
||||||
<tr id="tokens-empty"><td colspan="6" class="muted">No tokens yet. Create one above.</td></tr>
|
|
||||||
{% endfor %}
|
|
||||||
</tbody>
|
|
||||||
</table>
|
|
||||||
</div>
|
|
||||||
{% endblock %}
|
|
||||||
@@ -4,39 +4,44 @@ Exposes the gateway as an Anthropic-compatible Messages endpoint, so any
|
|||||||
client that already speaks Anthropic (Cursor, Cline, the official SDK,
|
client that already speaks Anthropic (Cursor, Cline, the official SDK,
|
||||||
``curl``) can hit a configured agent by passing its name as ``model``.
|
``curl``) can hit a configured agent by passing its name as ``model``.
|
||||||
|
|
||||||
The frontend is intentionally thin: it authenticates the bearer token,
|
A Claude agent behind this endpoint is a ``deep`` conversation: the client
|
||||||
resolves ``body.model`` to an agent + its backend, and then either
|
knows nothing about our ids, so the text fingerprint of the history it
|
||||||
streams the backend's events straight to SSE or accumulates them into a
|
sends is the ``(anthropic, fingerprint)`` binding of the conversation,
|
||||||
single ``Message`` for ``stream=false`` callers. All provider quirks
|
rebound after every turn to the fingerprint the next request will carry.
|
||||||
already live in the backend adapters; we don't translate here.
|
A history nobody has seen becomes a new conversation, materialized by the
|
||||||
|
home frontend of ``deep`` (the vault file), and every reply is published
|
||||||
Phase 1.4 wires only ``RaycastAgent`` through ``RaycastBackend``;
|
on the bus so that file follows the chat. Other agents (Raycast) stay
|
||||||
``ClaudeAgent`` lands in Phase 2 and will plug into the same dispatch
|
stateless and are only archived through ``turn_log_handlers``.
|
||||||
table without changes to this module.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
|
import uuid
|
||||||
from typing import TYPE_CHECKING, Any
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
from fastapi import FastAPI, HTTPException, Request, status
|
from fastapi import FastAPI, HTTPException, Request, status
|
||||||
from fastapi.responses import JSONResponse, StreamingResponse
|
from fastapi.responses import JSONResponse, StreamingResponse
|
||||||
|
|
||||||
|
from beaver_gateway.agents.claude import ClaudeAgent
|
||||||
from beaver_gateway.core import audit
|
from beaver_gateway.core import audit
|
||||||
|
from beaver_gateway.core.transcript import fingerprint, text_of
|
||||||
|
from beaver_gateway.core.turn_capture import TurnCapture
|
||||||
from beaver_gateway.core.turn_record import TurnRecord
|
from beaver_gateway.core.turn_record import TurnRecord
|
||||||
from beaver_gateway.frontends._accumulate import StreamAccumulator, accumulate
|
from beaver_gateway.frontends._accumulate import StreamAccumulator
|
||||||
from beaver_gateway.frontends._auth import require_token
|
from beaver_gateway.frontends._auth import require_token
|
||||||
from beaver_gateway.frontends.base import Frontend
|
from beaver_gateway.frontends.base import Frontend
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from collections.abc import AsyncIterator
|
from collections.abc import AsyncIterator, Awaitable, Callable
|
||||||
|
|
||||||
from anthropic.types import Message, MessageParam
|
from anthropic.types import Message, MessageParam
|
||||||
|
|
||||||
|
from beaver_gateway.core.conversations import Conversations
|
||||||
from beaver_gateway.core.events import MessageStreamEvent
|
from beaver_gateway.core.events import MessageStreamEvent
|
||||||
from beaver_gateway.frontends.base import GatewayRuntime
|
from beaver_gateway.frontends.base import GatewayRuntime
|
||||||
|
from beaver_gateway.storage.models import Conversation
|
||||||
|
|
||||||
|
|
||||||
_log = logging.getLogger("beaver_gateway.frontends.anthropic")
|
_log = logging.getLogger("beaver_gateway.frontends.anthropic")
|
||||||
@@ -44,28 +49,18 @@ _log = logging.getLogger("beaver_gateway.frontends.anthropic")
|
|||||||
|
|
||||||
__all__ = ["AnthropicMessagesFrontend"]
|
__all__ = ["AnthropicMessagesFrontend"]
|
||||||
|
|
||||||
|
FRONTEND = "anthropic"
|
||||||
|
_TITLE_LEN = 80
|
||||||
|
|
||||||
|
|
||||||
class AnthropicMessagesFrontend(Frontend):
|
class AnthropicMessagesFrontend(Frontend):
|
||||||
"""FastAPI app behind ``POST /v1/messages`` + ``GET /v1/models``."""
|
"""FastAPI app behind ``POST /v1/messages`` + ``GET /v1/models``."""
|
||||||
|
|
||||||
def __init__(
|
name = FRONTEND
|
||||||
self,
|
kinds = ("deep",)
|
||||||
*,
|
path = "/anthropic"
|
||||||
host: str = "0.0.0.0", # noqa: S104
|
|
||||||
port: int = 8000,
|
def __init__(self) -> None:
|
||||||
public_base_url: str | None = None,
|
|
||||||
) -> None:
|
|
||||||
self.host = host
|
|
||||||
self.port = port
|
|
||||||
# External URL prefix the reverse proxy (Caddy/nginx/Cloudflare/…)
|
|
||||||
# uses to reach this frontend, e.g. ``https://api.example.com/ai``.
|
|
||||||
# The frontend's internal paths (``/v1/messages``, ``/v1/models``)
|
|
||||||
# are appended to it when the admin dashboard renders copy-pastable
|
|
||||||
# URLs. Trailing slash is stripped so the concatenation is
|
|
||||||
# idempotent. ``None`` means "advertise raw ``host:port``" (dev /
|
|
||||||
# no proxy) — the dashboard then derives the base from the
|
|
||||||
# browser's own request hostname.
|
|
||||||
self.public_base_url = public_base_url.rstrip("/") if public_base_url else None
|
|
||||||
self._runtime: GatewayRuntime | None = None
|
self._runtime: GatewayRuntime | None = None
|
||||||
self._app: FastAPI | None = None
|
self._app: FastAPI | None = None
|
||||||
|
|
||||||
@@ -73,21 +68,8 @@ class AnthropicMessagesFrontend(Frontend):
|
|||||||
self._runtime = runtime
|
self._runtime = runtime
|
||||||
self._app = self._build_app(runtime)
|
self._app = self._build_app(runtime)
|
||||||
|
|
||||||
async def serve(self) -> None:
|
def app(self) -> FastAPI | None:
|
||||||
# Local import: uvicorn pulls in a lot, no reason to load it when
|
return self._app
|
||||||
# something else (a test, a script) imports this module just for
|
|
||||||
# the FastAPI factory.
|
|
||||||
import uvicorn
|
|
||||||
|
|
||||||
if self._app is None:
|
|
||||||
msg = "configure() must be called before serve()"
|
|
||||||
raise RuntimeError(msg)
|
|
||||||
|
|
||||||
config = uvicorn.Config(
|
|
||||||
self._app, host=self.host, port=self.port, log_level="info"
|
|
||||||
)
|
|
||||||
server = uvicorn.Server(config)
|
|
||||||
await server.serve()
|
|
||||||
|
|
||||||
def _build_app(self, runtime: GatewayRuntime) -> FastAPI:
|
def _build_app(self, runtime: GatewayRuntime) -> FastAPI:
|
||||||
app = FastAPI(title="beaver-gateway / Anthropic Messages")
|
app = FastAPI(title="beaver-gateway / Anthropic Messages")
|
||||||
@@ -134,9 +116,6 @@ class AnthropicMessagesFrontend(Frontend):
|
|||||||
|
|
||||||
backend = runtime.backends.get(agent.name)
|
backend = runtime.backends.get(agent.name)
|
||||||
if backend is None:
|
if backend is None:
|
||||||
# Agent exists in config but its backend isn't wired in
|
|
||||||
# this phase (e.g. ClaudeAgent before Phase 2, or a
|
|
||||||
# RaycastAgent without RAYCAST_BEARER set at startup).
|
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
status.HTTP_503_SERVICE_UNAVAILABLE,
|
status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||||
f"no backend configured for agent {agent.name!r}",
|
f"no backend configured for agent {agent.name!r}",
|
||||||
@@ -144,7 +123,12 @@ class AnthropicMessagesFrontend(Frontend):
|
|||||||
|
|
||||||
messages = body.get("messages") or []
|
messages = body.get("messages") or []
|
||||||
system = body.get("system")
|
system = body.get("system")
|
||||||
|
system_str = system if isinstance(system, str) else None
|
||||||
stream_flag = bool(body.get("stream", False))
|
stream_flag = bool(body.get("stream", False))
|
||||||
|
if not messages or messages[-1].get("role") != "user":
|
||||||
|
raise HTTPException(
|
||||||
|
status.HTTP_400_BAD_REQUEST, "the last message must be a user turn"
|
||||||
|
)
|
||||||
|
|
||||||
_log.info(
|
_log.info(
|
||||||
"messages: actor=%s agent=%s stream=%s msgs=%d",
|
"messages: actor=%s agent=%s stream=%s msgs=%d",
|
||||||
@@ -153,10 +137,6 @@ class AnthropicMessagesFrontend(Frontend):
|
|||||||
stream_flag,
|
stream_flag,
|
||||||
len(messages),
|
len(messages),
|
||||||
)
|
)
|
||||||
# Record at request acceptance, not at stream completion:
|
|
||||||
# a long streaming response can be aborted mid-flight by
|
|
||||||
# the client, and we still want the row in the audit trail.
|
|
||||||
# Detail stays small — no message bodies, no system prompt.
|
|
||||||
await audit.log(
|
await audit.log(
|
||||||
runtime,
|
runtime,
|
||||||
actor=f"token:{token_name}",
|
actor=f"token:{token_name}",
|
||||||
@@ -166,74 +146,139 @@ class AnthropicMessagesFrontend(Frontend):
|
|||||||
msgs=len(messages),
|
msgs=len(messages),
|
||||||
)
|
)
|
||||||
|
|
||||||
# Forward per-request knobs the Anthropic body may carry —
|
if isinstance(agent, ClaudeAgent) and runtime.conversations is not None:
|
||||||
# backend adapters layer these over per-agent defaults. Only
|
conv, history = await _resolve(runtime.conversations, agent, messages)
|
||||||
# values explicitly present (not Anthropic-defaulted ones we
|
turn_id = f"turn_{uuid.uuid4().hex[:12]}"
|
||||||
# never received) are forwarded, so the agent's default still
|
capture = TurnCapture()
|
||||||
# wins when the caller omits the field.
|
events = runtime.conversations.turn(
|
||||||
options: dict[str, Any] = {}
|
conv,
|
||||||
if isinstance(body.get("temperature"), int | float):
|
messages=messages,
|
||||||
options["temperature"] = body["temperature"]
|
origin="user",
|
||||||
|
capture=capture,
|
||||||
|
turn_id=turn_id,
|
||||||
|
)
|
||||||
|
|
||||||
events = backend.complete(
|
async def after(message: Message) -> None:
|
||||||
agent=agent,
|
await _finish(
|
||||||
messages=messages,
|
runtime,
|
||||||
system=system if isinstance(system, str) else None,
|
conv,
|
||||||
**options,
|
messages=messages,
|
||||||
)
|
message=message,
|
||||||
|
capture=capture,
|
||||||
|
turn_id=turn_id,
|
||||||
|
history=history,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
options: dict[str, Any] = {}
|
||||||
|
if isinstance(body.get("temperature"), int | float):
|
||||||
|
options["temperature"] = body["temperature"]
|
||||||
|
events = backend.complete(
|
||||||
|
agent=agent, messages=messages, system=system_str, **options
|
||||||
|
)
|
||||||
|
|
||||||
system_str = system if isinstance(system, str) else None
|
async def after(message: Message) -> None:
|
||||||
|
await _broadcast_turn(
|
||||||
if stream_flag:
|
runtime,
|
||||||
# Side-accumulate while streaming so we can still emit a
|
|
||||||
# ``TurnRecord`` after the response closes. The buffered
|
|
||||||
# ``Message`` lives only in this coroutine's frame; SSE
|
|
||||||
# bytes still flow to the client unchanged.
|
|
||||||
return StreamingResponse(
|
|
||||||
_sse_and_broadcast(
|
|
||||||
events,
|
|
||||||
runtime=runtime,
|
|
||||||
agent_name=agent.name,
|
agent_name=agent.name,
|
||||||
input_messages=messages,
|
input_messages=messages,
|
||||||
|
output_message=message,
|
||||||
system=system_str,
|
system=system_str,
|
||||||
model=model,
|
)
|
||||||
),
|
|
||||||
|
if stream_flag:
|
||||||
|
return StreamingResponse(
|
||||||
|
_sse(events, model=model, after=after),
|
||||||
media_type="text/event-stream",
|
media_type="text/event-stream",
|
||||||
)
|
)
|
||||||
message = await accumulate(events, model=model)
|
acc = StreamAccumulator()
|
||||||
await _broadcast_turn(
|
async for ev in events:
|
||||||
runtime,
|
acc.feed(ev)
|
||||||
agent_name=agent.name,
|
message = acc.finalize(model=model)
|
||||||
input_messages=messages,
|
await after(message)
|
||||||
output_message=message,
|
|
||||||
system=system_str,
|
|
||||||
)
|
|
||||||
return JSONResponse(content=message.model_dump(mode="json"))
|
return JSONResponse(content=message.model_dump(mode="json"))
|
||||||
|
|
||||||
return app
|
return app
|
||||||
|
|
||||||
|
|
||||||
async def _sse_and_broadcast(
|
async def _resolve(
|
||||||
|
conversations: Conversations, agent: ClaudeAgent, messages: list[MessageParam]
|
||||||
|
) -> tuple[Conversation, list[dict[str, str]] | None]:
|
||||||
|
prior = [dict(m) for m in messages[:-1]]
|
||||||
|
if prior:
|
||||||
|
conv = await conversations.find_bound(
|
||||||
|
frontend=FRONTEND, external_id=fingerprint(prior)
|
||||||
|
)
|
||||||
|
if conv is not None:
|
||||||
|
if conv.agent_name != agent.name:
|
||||||
|
raise HTTPException(
|
||||||
|
status.HTTP_400_BAD_REQUEST,
|
||||||
|
f"this chat runs on {conv.agent_name!r}, not {agent.name!r}",
|
||||||
|
)
|
||||||
|
return conv, None
|
||||||
|
first = next(
|
||||||
|
(text_of(m.get("content")) for m in messages if m.get("role") == "user"), ""
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
conv = await conversations.create(
|
||||||
|
kind="deep",
|
||||||
|
agent=agent.name,
|
||||||
|
origin=FRONTEND,
|
||||||
|
title=" ".join(first.split())[:_TITLE_LEN] or None,
|
||||||
|
)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status.HTTP_400_BAD_REQUEST, str(exc)) from exc
|
||||||
|
await conversations.materialize(conv)
|
||||||
|
history = [
|
||||||
|
{"role": str(m.get("role", "")), "text": text_of(m.get("content"))}
|
||||||
|
for m in prior
|
||||||
|
]
|
||||||
|
return conv, history
|
||||||
|
|
||||||
|
|
||||||
|
async def _finish(
|
||||||
|
runtime: GatewayRuntime,
|
||||||
|
conv: Conversation,
|
||||||
|
*,
|
||||||
|
messages: list[MessageParam],
|
||||||
|
message: Message,
|
||||||
|
capture: TurnCapture,
|
||||||
|
turn_id: str,
|
||||||
|
history: list[dict[str, str]] | None,
|
||||||
|
) -> None:
|
||||||
|
reply = "\n\n".join(
|
||||||
|
getattr(b, "text", "")
|
||||||
|
for b in message.content
|
||||||
|
if getattr(b, "type", "") == "text"
|
||||||
|
).strip()
|
||||||
|
tail = capture.synthesized_messages or [
|
||||||
|
{"role": "assistant", "content": [{"type": "text", "text": reply}]}
|
||||||
|
]
|
||||||
|
await runtime.conversations.bind(
|
||||||
|
conv, frontend=FRONTEND, external_id=fingerprint([*messages, *tail])
|
||||||
|
)
|
||||||
|
runtime.bus.publish(
|
||||||
|
"reply",
|
||||||
|
conversation_id=conv.external_id,
|
||||||
|
turn_id=turn_id,
|
||||||
|
source=FRONTEND,
|
||||||
|
prompt=text_of(messages[-1].get("content")),
|
||||||
|
text=reply,
|
||||||
|
history=history,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def _sse(
|
||||||
events: AsyncIterator[MessageStreamEvent],
|
events: AsyncIterator[MessageStreamEvent],
|
||||||
*,
|
*,
|
||||||
runtime: GatewayRuntime,
|
|
||||||
agent_name: str,
|
|
||||||
input_messages: list[MessageParam],
|
|
||||||
system: str | None,
|
|
||||||
model: str,
|
model: str,
|
||||||
|
after: Callable[[Message], Awaitable[None]],
|
||||||
) -> AsyncIterator[bytes]:
|
) -> AsyncIterator[bytes]:
|
||||||
r"""Serialize an event stream to SSE; broadcast a :class:`TurnRecord` after.
|
r"""Serialize an event stream to SSE, then hand the assembled ``Message`` on.
|
||||||
|
|
||||||
Each event becomes ``event: <type>\ndata: <json>\n\n`` — the shape
|
Each event becomes ``event: <type>\ndata: <json>\n\n`` - the shape
|
||||||
the Anthropic SDK's SSE decoder expects. Errors mid-stream are
|
the Anthropic SDK's SSE decoder expects. Errors mid-stream are
|
||||||
swallowed into a synthetic ``error`` event so the client sees the
|
swallowed into a synthetic ``error`` event so the client sees the
|
||||||
failure rather than a hung connection.
|
failure rather than a hung connection.
|
||||||
|
|
||||||
The same events feed a :class:`StreamAccumulator` on the side so that
|
|
||||||
once the SSE response closes we can hand a fully-assembled
|
|
||||||
``Message`` to every ``runtime.turn_log_handlers`` entry (the
|
|
||||||
markdown frontend's archive logger lives in there). Broadcast
|
|
||||||
failures are caught — they must never bubble up to the client.
|
|
||||||
"""
|
"""
|
||||||
acc = StreamAccumulator()
|
acc = StreamAccumulator()
|
||||||
try:
|
try:
|
||||||
@@ -248,14 +293,10 @@ async def _sse_and_broadcast(
|
|||||||
)
|
)
|
||||||
yield f"event: error\ndata: {err}\n\n".encode()
|
yield f"event: error\ndata: {err}\n\n".encode()
|
||||||
return
|
return
|
||||||
message = acc.finalize(model=model)
|
try:
|
||||||
await _broadcast_turn(
|
await after(acc.finalize(model=model))
|
||||||
runtime,
|
except Exception: # noqa: BLE001
|
||||||
agent_name=agent_name,
|
_log.exception("post-turn hook failed")
|
||||||
input_messages=input_messages,
|
|
||||||
output_message=message,
|
|
||||||
system=system,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
async def _broadcast_turn(
|
async def _broadcast_turn(
|
||||||
@@ -266,11 +307,6 @@ async def _broadcast_turn(
|
|||||||
output_message: Message,
|
output_message: Message,
|
||||||
system: str | None,
|
system: str | None,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Fire each ``turn_log_handlers`` entry with a fresh :class:`TurnRecord`.
|
|
||||||
|
|
||||||
Handler exceptions are caught and logged — they're observability
|
|
||||||
plumbing, not part of the user-visible request path.
|
|
||||||
"""
|
|
||||||
if not runtime.turn_log_handlers:
|
if not runtime.turn_log_handlers:
|
||||||
return
|
return
|
||||||
record = TurnRecord(
|
record = TurnRecord(
|
||||||
@@ -278,7 +314,7 @@ async def _broadcast_turn(
|
|||||||
input_messages=list(input_messages),
|
input_messages=list(input_messages),
|
||||||
output_message=output_message,
|
output_message=output_message,
|
||||||
system=system,
|
system=system,
|
||||||
source="anthropic",
|
source=FRONTEND,
|
||||||
)
|
)
|
||||||
for handler in runtime.turn_log_handlers:
|
for handler in runtime.turn_log_handlers:
|
||||||
try:
|
try:
|
||||||
|
|||||||
@@ -0,0 +1,5 @@
|
|||||||
|
"""``ApiFrontend`` - the conversations API and event stream (§3.9)."""
|
||||||
|
|
||||||
|
from beaver_gateway.frontends.api.frontend import ApiFrontend
|
||||||
|
|
||||||
|
__all__ = ["ApiFrontend"]
|
||||||
File diff suppressed because it is too large.
Load diff
@@ -1,27 +1,33 @@
|
|||||||
"""Frontend ABC + the runtime context handed to ``configure``.
|
"""Frontend ABC + the runtime context handed to ``configure``.
|
||||||
|
|
||||||
A frontend is anything that listens on a port and routes inbound traffic
|
A frontend is anything that routes inbound traffic into the gateway: an
|
||||||
into the gateway. ``GatewayRuntime`` carries everything a frontend may
|
HTTP surface mounted under its ``path`` on the single gateway port
|
||||||
need that isn't user-config: built registries, per-agent backends, and
|
(``frontends/root.py``), a poller (Telegram), or both. ``GatewayRuntime``
|
||||||
the in-memory token store. The user's ``/config/config.py`` defines a
|
carries everything a frontend may need that isn't user-config: built
|
||||||
``Gateway`` (lists); ``cli.main`` turns that into a ``GatewayRuntime``
|
registries, per-agent backends, and the in-memory token store. The
|
||||||
and hands it to each frontend's ``configure``.
|
user's ``/config/config.py`` defines a ``Gateway`` (lists); ``cli.main``
|
||||||
|
turns that into a ``GatewayRuntime`` and hands it to each frontend's
|
||||||
|
``configure``.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from abc import ABC, abstractmethod
|
from abc import ABC, abstractmethod
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from typing import TYPE_CHECKING
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from collections.abc import Awaitable, Callable, Mapping, Sequence
|
from collections.abc import Awaitable, Callable, Mapping, Sequence
|
||||||
|
|
||||||
|
from starlette.types import ASGIApp
|
||||||
|
|
||||||
from beaver_gateway.backends.base import Backend
|
from beaver_gateway.backends.base import Backend
|
||||||
from beaver_gateway.core.auth import TokenStore
|
from beaver_gateway.core.auth import TokenStore
|
||||||
|
from beaver_gateway.core.kinds import Kind
|
||||||
from beaver_gateway.core.registry import AgentRegistry, McpRegistry
|
from beaver_gateway.core.registry import AgentRegistry, McpRegistry
|
||||||
from beaver_gateway.core.turn_record import TurnRecord
|
from beaver_gateway.core.turn_record import TurnRecord
|
||||||
from beaver_gateway.storage import Database
|
from beaver_gateway.storage import Database
|
||||||
|
from beaver_gateway.storage.models import Conversation, ConversationBinding
|
||||||
|
|
||||||
TurnLogHandler = Callable[[TurnRecord], Awaitable[None]]
|
TurnLogHandler = Callable[[TurnRecord], Awaitable[None]]
|
||||||
|
|
||||||
@@ -36,7 +42,7 @@ class GatewayRuntime:
|
|||||||
the name in hand, so the indirection lives one step earlier.
|
the name in hand, so the indirection lives one step earlier.
|
||||||
|
|
||||||
``mcp_internal_urls`` is filled in Phase 2.1: one loopback URL per
|
``mcp_internal_urls`` is filled in Phase 2.1: one loopback URL per
|
||||||
declared ``McpServer`` so ``ClaudeCodeBackendAdapter`` (Phase 2.2)
|
declared ``McpServer`` so ``ClaudeSdkBackend``
|
||||||
can pass them to ``BackendOptions.mcp_servers`` without re-running
|
can pass them to ``BackendOptions.mcp_servers`` without re-running
|
||||||
discovery.
|
discovery.
|
||||||
|
|
||||||
@@ -77,13 +83,59 @@ class GatewayRuntime:
|
|||||||
# to keep ``anthropic.types`` out of the runtime import graph for
|
# to keep ``anthropic.types`` out of the runtime import graph for
|
||||||
# this base module.
|
# this base module.
|
||||||
turn_log_handlers: list[TurnLogHandler] = field(default_factory=list)
|
turn_log_handlers: list[TurnLogHandler] = field(default_factory=list)
|
||||||
|
# M1b: conversations service, event bus and the shared session pool.
|
||||||
|
# ``Any`` for the same import-graph reason as above; ``None`` only in
|
||||||
|
# tests that build a runtime without them.
|
||||||
|
conversations: Any = None
|
||||||
|
bus: Any = None
|
||||||
|
pool: Any = None
|
||||||
|
scheduler: Any = None
|
||||||
|
# External origin the reverse proxy puts in front of the gateway
|
||||||
|
# (``Gateway.public_url``); ``None`` means "derive from the request".
|
||||||
|
public_url: str | None = None
|
||||||
|
|
||||||
|
|
||||||
class Frontend(ABC):
|
class Frontend(ABC):
|
||||||
"""Listens on a port, dispatches into the gateway."""
|
"""Routes inbound traffic into the gateway.
|
||||||
|
|
||||||
|
HTTP frontends set ``path`` and return their ASGI app from ``app()``;
|
||||||
|
``cli`` mounts every such app under that path on the one gateway
|
||||||
|
port, so ``/anthropic/v1/messages`` reaches the Anthropic frontend's
|
||||||
|
``/v1/messages``. ``serve()`` is for work outside HTTP - polling,
|
||||||
|
vault mirrors - and defaults to nothing. ``landing`` marks the app
|
||||||
|
that ``/`` redirects to (the admin console).
|
||||||
|
|
||||||
|
A frontend that shows conversations declares ``name`` (the binding
|
||||||
|
key) and ``kinds`` (which conversation kinds it shows);
|
||||||
|
``core/conversations`` refuses to bind a conversation to a frontend
|
||||||
|
outside its declaration. The first frontend in declaration order
|
||||||
|
whose ``materialize`` returns a binding is the *home* of that kind:
|
||||||
|
``spawn`` calls it so a new conversation gets a window (a vault file,
|
||||||
|
a Telegram topic). ``agent_for`` names the default agent for a kind
|
||||||
|
so callers may omit ``agent``. Stateless frontends (MCP, admin) keep
|
||||||
|
the defaults and stay outside the routing.
|
||||||
|
"""
|
||||||
|
|
||||||
|
name: str = ""
|
||||||
|
kinds: tuple[Kind, ...] = ()
|
||||||
|
path: str | None = None
|
||||||
|
landing: bool = False
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def configure(self, runtime: GatewayRuntime) -> None: ...
|
def configure(self, runtime: GatewayRuntime) -> None: ...
|
||||||
|
|
||||||
@abstractmethod
|
def app(self) -> ASGIApp | None:
|
||||||
async def serve(self) -> None: ...
|
return None
|
||||||
|
|
||||||
|
async def serve(self) -> None:
|
||||||
|
return
|
||||||
|
|
||||||
|
def agent_for(self, kind: Kind) -> str | None: # noqa: ARG002
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def materialize(self, conv: Conversation) -> ConversationBinding | None: # noqa: ARG002
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def mark_closed(self, conv: Conversation) -> bool: # noqa: ARG002
|
||||||
|
"""Show in the window that the conversation is over (a renamed topic)."""
|
||||||
|
return False
|
||||||
@@ -38,14 +38,16 @@ if TYPE_CHECKING:
|
|||||||
_log = logging.getLogger("beaver_gateway.frontends.markdown.crossfront")
|
_log = logging.getLogger("beaver_gateway.frontends.markdown.crossfront")
|
||||||
|
|
||||||
|
|
||||||
# User hook: take a turn + vault root, return where the new file should
|
__all__ = [
|
||||||
# live. Returning a relative ``Path`` is treated as relative to the
|
"ChatPathFn",
|
||||||
# vault. ``None`` (the default) keeps the built-in
|
"CrossFrontendLogger",
|
||||||
# ``{vault}/{logged_subdir}/{agent}/{YYYY-MM-DD}_{hex8}.md`` layout.
|
"fingerprint_messages",
|
||||||
LogPathFn = "Callable[[TurnRecord, Path], Path]"
|
"strip_trailing_user_scaffold",
|
||||||
|
]
|
||||||
|
|
||||||
|
ChatPathFn = "Callable[[str, str, Path], Path]"
|
||||||
__all__ = ["CrossFrontendLogger", "LogPathFn", "fingerprint_messages"]
|
"""``(title, agent, vault) -> path`` of a new chat file; relative = under the
|
||||||
|
vault. ``None`` keeps ``{vault}/{logged_subdir}/{agent}/{YYYY-MM-DD}_{slug}.md``."""
|
||||||
|
|
||||||
|
|
||||||
def fingerprint_messages(messages: Iterable[MessageParam]) -> str:
|
def fingerprint_messages(messages: Iterable[MessageParam]) -> str:
|
||||||
@@ -96,18 +98,18 @@ class CrossFrontendLogger:
|
|||||||
*,
|
*,
|
||||||
vault_path: Path,
|
vault_path: Path,
|
||||||
logged_subdir: str,
|
logged_subdir: str,
|
||||||
log_path: Callable[[TurnRecord, Path], Path] | None = None,
|
chat_path: Callable[[str, str, Path], Path] | None = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
self._vault = vault_path
|
self._vault = vault_path
|
||||||
self._root = vault_path / logged_subdir
|
self._root = vault_path / logged_subdir
|
||||||
self._index: dict[str, Path] = {}
|
self._index: dict[str, Path] = {}
|
||||||
self._lock = asyncio.Lock()
|
self._lock = asyncio.Lock()
|
||||||
self._log_path_fn = log_path
|
self._chat_path_fn = chat_path
|
||||||
# When the user supplies a custom path function, files can land
|
# When the user supplies a custom path function, files can land
|
||||||
# anywhere in the vault — so we have to scan the whole vault on
|
# anywhere in the vault — so we have to scan the whole vault on
|
||||||
# startup to rebuild the fingerprint→path map. With the default
|
# startup to rebuild the fingerprint→path map. With the default
|
||||||
# layout we can bound the scan to ``_logs/``.
|
# layout we can bound the scan to ``_logs/``.
|
||||||
self._scan_root = vault_path if log_path is not None else self._root
|
self._scan_root = vault_path if chat_path is not None else self._root
|
||||||
|
|
||||||
def warm_index(self) -> None:
|
def warm_index(self) -> None:
|
||||||
"""Scan logged files synchronously, populating the fingerprint map.
|
"""Scan logged files synchronously, populating the fingerprint map.
|
||||||
@@ -171,7 +173,7 @@ class CrossFrontendLogger:
|
|||||||
if target.exists():
|
if target.exists():
|
||||||
existing = target.read_text(encoding="utf-8")
|
existing = target.read_text(encoding="utf-8")
|
||||||
parsed = frontmatter.loads(existing)
|
parsed = frontmatter.loads(existing)
|
||||||
body = _strip_trailing_user_scaffold(parsed.content)
|
body = strip_trailing_user_scaffold(parsed.content)
|
||||||
# We append only the *new* user turn (the last one in
|
# We append only the *new* user turn (the last one in
|
||||||
# input_messages, since prior turns are already on disk)
|
# input_messages, since prior turns are already on disk)
|
||||||
# plus the assistant reply.
|
# plus the assistant reply.
|
||||||
@@ -206,13 +208,15 @@ class CrossFrontendLogger:
|
|||||||
def _new_file_path(self, record: TurnRecord) -> Path:
|
def _new_file_path(self, record: TurnRecord) -> Path:
|
||||||
"""Pick a fresh filename for a brand-new conversation.
|
"""Pick a fresh filename for a brand-new conversation.
|
||||||
|
|
||||||
With a user-supplied ``log_path`` we delegate to it (joining a
|
With a user-supplied ``chat_path`` we delegate to it (joining a
|
||||||
relative result with the vault root). Without one, we fall back
|
relative result with the vault root). Without one, we fall back
|
||||||
to ``{logged_subdir}/{agent}/{date}_{hex8}.md`` and ensure the
|
to ``{logged_subdir}/{agent}/{date}_{hex8}.md`` and ensure the
|
||||||
``.md`` suffix in case the user picks a non-md extension by hand.
|
``.md`` suffix in case the user picks a non-md extension by hand.
|
||||||
"""
|
"""
|
||||||
if self._log_path_fn is not None:
|
if self._chat_path_fn is not None:
|
||||||
result = self._log_path_fn(record, self._vault)
|
result = self._chat_path_fn(
|
||||||
|
record.first_user_text, record.agent_name, self._vault
|
||||||
|
)
|
||||||
if not result.is_absolute():
|
if not result.is_absolute():
|
||||||
result = self._vault / result
|
result = self._vault / result
|
||||||
if result.suffix != ".md":
|
if result.suffix != ".md":
|
||||||
@@ -270,7 +274,7 @@ def _render_full_history(messages: list[MessageParam], assistant: Any) -> str:
|
|||||||
return body
|
return body
|
||||||
|
|
||||||
|
|
||||||
def _strip_trailing_user_scaffold(body: str) -> str:
|
def strip_trailing_user_scaffold(body: str) -> str:
|
||||||
"""Drop a trailing empty ``### User:`` block if present.
|
"""Drop a trailing empty ``### User:`` block if present.
|
||||||
|
|
||||||
Cross-frontend turns aren't typed into the file by the human — they
|
Cross-frontend turns aren't typed into the file by the human — they
|
||||||
|
|||||||
@@ -0,0 +1,50 @@
|
|||||||
|
"""Vault file IO shared by the markdown frontend and its mirror."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import contextlib
|
||||||
|
import os
|
||||||
|
import tempfile
|
||||||
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
|
import aiofile
|
||||||
|
import frontmatter
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
__all__ = ["read_or_empty", "reattach_frontmatter", "write_atomic"]
|
||||||
|
|
||||||
|
|
||||||
|
async def read_or_empty(path: Path) -> str:
|
||||||
|
if not path.exists(): # noqa: ASYNC240
|
||||||
|
return ""
|
||||||
|
async with aiofile.async_open(path, "r", encoding="utf-8") as f:
|
||||||
|
return await f.read()
|
||||||
|
|
||||||
|
|
||||||
|
async def write_atomic(path: Path, content: str) -> None:
|
||||||
|
"""Write via tmp + ``os.replace`` in the same directory: no torn reads."""
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
fd, tmp_path = tempfile.mkstemp(
|
||||||
|
prefix=f".{path.name}.", suffix=".tmp", dir=str(path.parent)
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
async with aiofile.async_open(tmp_path, "w", encoding="utf-8") as f:
|
||||||
|
await f.write(content)
|
||||||
|
os.close(fd)
|
||||||
|
os.replace(tmp_path, path) # noqa: PTH105
|
||||||
|
except BaseException:
|
||||||
|
with contextlib.suppress(OSError):
|
||||||
|
os.close(fd)
|
||||||
|
with contextlib.suppress(OSError):
|
||||||
|
os.unlink(tmp_path) # noqa: PTH108
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
def reattach_frontmatter(metadata: dict[str, Any], body: str) -> str:
|
||||||
|
"""Re-emit a ``.md`` with YAML frontmatter; no block at all for empty metadata."""
|
||||||
|
if not metadata:
|
||||||
|
return body if body.endswith("\n") else body + "\n"
|
||||||
|
post = frontmatter.Post(content=body, **metadata)
|
||||||
|
return frontmatter.dumps(post) + "\n"
|
||||||
@@ -13,7 +13,16 @@ Concurrency model: an in-memory ``set[Path]`` of files currently in
|
|||||||
flight. Two concurrent requests for the same file → the second gets
|
flight. Two concurrent requests for the same file → the second gets
|
||||||
409. The set is single-process (one gateway instance) — that's by
|
409. The set is single-process (one gateway instance) — that's by
|
||||||
design; the markdown frontend is the only writer in its vault from
|
design; the markdown frontend is the only writer in its vault from
|
||||||
the gateway side.
|
the gateway side. The turn itself runs through ``core/conversations``
|
||||||
|
(one turn per conversation, ``running_turn`` in the DB), so a message
|
||||||
|
posted to the same conversation via ``/api`` waits its turn.
|
||||||
|
|
||||||
|
A chat file is a ``deep`` conversation bound as
|
||||||
|
``(markdown, <vault-relative path>)``; frontmatter carries only ``agent``
|
||||||
|
and ``conversation_id`` (§3.10), tool calls are never rendered. The
|
||||||
|
frontend is the home of ``deep``: ``materialize`` gives a conversation
|
||||||
|
spawned elsewhere its file, and :class:`.mirror.ChatMirror` keeps that
|
||||||
|
file in step with replies produced outside ``/chat``.
|
||||||
|
|
||||||
Cross-frontend logging: when ``log_all_chats=True``, ``configure()``
|
Cross-frontend logging: when ``log_all_chats=True``, ``configure()``
|
||||||
registers a handler on ``runtime.turn_log_handlers`` so every other
|
registers a handler on ``runtime.turn_log_handlers`` so every other
|
||||||
@@ -24,48 +33,50 @@ shape.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import contextlib
|
|
||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
import os
|
|
||||||
import tempfile
|
|
||||||
import time
|
import time
|
||||||
from collections.abc import AsyncIterator
|
from collections.abc import AsyncIterator
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import TYPE_CHECKING, Any
|
from typing import TYPE_CHECKING, Any, cast
|
||||||
|
|
||||||
import aiofile
|
|
||||||
from anthropic.types import RawContentBlockStopEvent
|
from anthropic.types import RawContentBlockStopEvent
|
||||||
from fastapi import FastAPI, HTTPException, Request, status
|
from fastapi import FastAPI, HTTPException, Request, status
|
||||||
from fastapi.middleware.cors import CORSMiddleware
|
from fastapi.middleware.cors import CORSMiddleware
|
||||||
from fastapi.responses import JSONResponse, StreamingResponse
|
from fastapi.responses import JSONResponse, StreamingResponse
|
||||||
|
|
||||||
from beaver_gateway.backends.claude_code import ClaudeCodeBackendAdapter, TurnCapture
|
|
||||||
from beaver_gateway.core import audit
|
from beaver_gateway.core import audit
|
||||||
from beaver_gateway.core.conversation_store import (
|
from beaver_gateway.core.conversation_store import (
|
||||||
diff_and_fork,
|
diff_and_fork,
|
||||||
load_conversation,
|
|
||||||
load_messages,
|
load_messages,
|
||||||
mint_conversation,
|
|
||||||
rewrite_messages,
|
rewrite_messages,
|
||||||
)
|
)
|
||||||
|
from beaver_gateway.core.turn_capture import TurnCapture
|
||||||
from beaver_gateway.core.turn_record import TurnRecord
|
from beaver_gateway.core.turn_record import TurnRecord
|
||||||
from beaver_gateway.frontends._accumulate import StreamAccumulator
|
from beaver_gateway.frontends._accumulate import StreamAccumulator
|
||||||
from beaver_gateway.frontends._auth import require_token
|
from beaver_gateway.frontends._auth import require_token
|
||||||
|
from beaver_gateway.frontends._sse import (
|
||||||
|
KEEPALIVE,
|
||||||
|
SSE_HEADERS,
|
||||||
|
events_with_heartbeat,
|
||||||
|
sse_pack,
|
||||||
|
)
|
||||||
from beaver_gateway.frontends.base import Frontend
|
from beaver_gateway.frontends.base import Frontend
|
||||||
from beaver_gateway.frontends.markdown import parser, renderer
|
from beaver_gateway.frontends.markdown import parser, renderer
|
||||||
from beaver_gateway.frontends.markdown.crossfront import (
|
from beaver_gateway.frontends.markdown.crossfront import CrossFrontendLogger
|
||||||
CrossFrontendLogger,
|
from beaver_gateway.frontends.markdown.files import (
|
||||||
fingerprint_messages,
|
read_or_empty,
|
||||||
|
reattach_frontmatter,
|
||||||
|
write_atomic,
|
||||||
)
|
)
|
||||||
|
from beaver_gateway.frontends.markdown.mirror import FRONTEND, ChatMirror
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from collections.abc import AsyncIterator, Callable
|
from collections.abc import AsyncIterator, Callable
|
||||||
|
|
||||||
from anthropic.types import MessageParam
|
from beaver_gateway.core.kinds import Kind
|
||||||
|
|
||||||
from beaver_gateway.frontends.base import GatewayRuntime
|
from beaver_gateway.frontends.base import GatewayRuntime
|
||||||
|
from beaver_gateway.storage.models import Conversation, ConversationBinding
|
||||||
|
|
||||||
|
|
||||||
_log = logging.getLogger("beaver_gateway.frontends.markdown")
|
_log = logging.getLogger("beaver_gateway.frontends.markdown")
|
||||||
@@ -88,41 +99,28 @@ _STREAM_FLUSH_DEBOUNCE = 0.4
|
|||||||
# disk round-trip).
|
# disk round-trip).
|
||||||
_SSE_FLUSH_DEBOUNCE = 0.1
|
_SSE_FLUSH_DEBOUNCE = 0.1
|
||||||
|
|
||||||
# Interval between SSE comment-frames sent when the backend is silent
|
|
||||||
# (e.g. claude is mid-thinking on a large context). The Obsidian plugin
|
|
||||||
# and any intermediate proxies will hold the connection open as long as
|
|
||||||
# bytes keep flowing; a comment-frame is the cheapest legal SSE keepalive.
|
|
||||||
# Set well under typical proxy/client idle timeouts (60s).
|
|
||||||
_SSE_HEARTBEAT_INTERVAL = 15.0
|
|
||||||
|
|
||||||
|
|
||||||
class MarkdownFrontend(Frontend):
|
class MarkdownFrontend(Frontend):
|
||||||
"""FastAPI app behind ``POST /chat`` driven by Obsidian-vault files."""
|
"""FastAPI app behind ``POST /chat`` driven by Obsidian-vault files."""
|
||||||
|
|
||||||
|
name = FRONTEND
|
||||||
|
kinds = ("deep",)
|
||||||
|
path = "/md"
|
||||||
|
|
||||||
def __init__(
|
def __init__(
|
||||||
self,
|
self,
|
||||||
*,
|
*,
|
||||||
vault_path: Path | str,
|
vault_path: Path | str,
|
||||||
host: str = "0.0.0.0", # noqa: S104
|
|
||||||
port: int = 8003,
|
|
||||||
default_agent: str | None = None,
|
default_agent: str | None = None,
|
||||||
log_all_chats: bool = False,
|
log_all_chats: bool = False,
|
||||||
logged_subdir: str = "_logs",
|
logged_subdir: str = "_logs",
|
||||||
log_path: Callable[[TurnRecord, Path], Path] | None = None,
|
chat_path: Callable[[str, str, Path], Path] | None = None,
|
||||||
public_base_url: str | None = None,
|
|
||||||
) -> None:
|
) -> None:
|
||||||
self.vault_path = Path(vault_path).expanduser().resolve()
|
self.vault_path = Path(vault_path).expanduser().resolve()
|
||||||
self.host = host
|
|
||||||
self.port = port
|
|
||||||
self.default_agent = default_agent
|
self.default_agent = default_agent
|
||||||
self.log_all_chats = log_all_chats
|
self.log_all_chats = log_all_chats
|
||||||
self.logged_subdir = logged_subdir
|
self.logged_subdir = logged_subdir
|
||||||
self.log_path = log_path
|
self.chat_path = chat_path
|
||||||
# External URL prefix when behind a reverse proxy — same role as
|
|
||||||
# on the other bearer frontends. Trailing slash trimmed for
|
|
||||||
# idempotent concatenation; ``None`` means "no proxy / advertise
|
|
||||||
# raw host:port".
|
|
||||||
self.public_base_url = public_base_url.rstrip("/") if public_base_url else None
|
|
||||||
self._runtime: GatewayRuntime | None = None
|
self._runtime: GatewayRuntime | None = None
|
||||||
self._app: FastAPI | None = None
|
self._app: FastAPI | None = None
|
||||||
# Files currently being processed by an in-flight ``POST /chat``.
|
# Files currently being processed by an in-flight ``POST /chat``.
|
||||||
@@ -131,15 +129,25 @@ class MarkdownFrontend(Frontend):
|
|||||||
# request reliably loses the race to 409.
|
# request reliably loses the race to 409.
|
||||||
self._busy: set[Path] = set()
|
self._busy: set[Path] = set()
|
||||||
self._crossfront: CrossFrontendLogger | None = None
|
self._crossfront: CrossFrontendLogger | None = None
|
||||||
|
self._mirror: ChatMirror | None = None
|
||||||
|
|
||||||
def configure(self, runtime: GatewayRuntime) -> None:
|
def configure(self, runtime: GatewayRuntime) -> None:
|
||||||
|
if runtime.conversations is None:
|
||||||
|
msg = "MarkdownFrontend needs runtime.conversations"
|
||||||
|
raise RuntimeError(msg)
|
||||||
self._runtime = runtime
|
self._runtime = runtime
|
||||||
self.vault_path.mkdir(parents=True, exist_ok=True)
|
self.vault_path.mkdir(parents=True, exist_ok=True)
|
||||||
|
self._mirror = ChatMirror(
|
||||||
|
vault_path=self.vault_path,
|
||||||
|
runtime=runtime,
|
||||||
|
logged_subdir=self.logged_subdir,
|
||||||
|
chat_path=self.chat_path,
|
||||||
|
)
|
||||||
if self.log_all_chats:
|
if self.log_all_chats:
|
||||||
self._crossfront = CrossFrontendLogger(
|
self._crossfront = CrossFrontendLogger(
|
||||||
vault_path=self.vault_path,
|
vault_path=self.vault_path,
|
||||||
logged_subdir=self.logged_subdir,
|
logged_subdir=self.logged_subdir,
|
||||||
log_path=self.log_path,
|
chat_path=self.chat_path,
|
||||||
)
|
)
|
||||||
# Scan the existing logged files synchronously here so the
|
# Scan the existing logged files synchronously here so the
|
||||||
# fingerprint→path map is populated before the first
|
# fingerprint→path map is populated before the first
|
||||||
@@ -148,17 +156,24 @@ class MarkdownFrontend(Frontend):
|
|||||||
runtime.turn_log_handlers.append(self._crossfront.handle)
|
runtime.turn_log_handlers.append(self._crossfront.handle)
|
||||||
self._app = self._build_app(runtime)
|
self._app = self._build_app(runtime)
|
||||||
|
|
||||||
async def serve(self) -> None:
|
@property
|
||||||
import uvicorn
|
def mirror(self) -> ChatMirror:
|
||||||
|
if self._mirror is None:
|
||||||
if self._app is None:
|
msg = "configure() must be called first"
|
||||||
msg = "configure() must be called before serve()"
|
|
||||||
raise RuntimeError(msg)
|
raise RuntimeError(msg)
|
||||||
config = uvicorn.Config(
|
return self._mirror
|
||||||
self._app, host=self.host, port=self.port, log_level="info"
|
|
||||||
)
|
def agent_for(self, kind: Kind) -> str | None:
|
||||||
server = uvicorn.Server(config)
|
return self.default_agent if kind == "deep" else None
|
||||||
await server.serve()
|
|
||||||
|
async def materialize(self, conv: Conversation) -> ConversationBinding | None:
|
||||||
|
return await self.mirror.materialize(conv)
|
||||||
|
|
||||||
|
def app(self) -> FastAPI | None:
|
||||||
|
return self._app
|
||||||
|
|
||||||
|
async def serve(self) -> None:
|
||||||
|
await self.mirror.run()
|
||||||
|
|
||||||
# ---- app builder ---------------------------------------------------
|
# ---- app builder ---------------------------------------------------
|
||||||
|
|
||||||
@@ -285,17 +300,7 @@ class MarkdownFrontend(Frontend):
|
|||||||
self._busy.discard(file_path)
|
self._busy.discard(file_path)
|
||||||
|
|
||||||
return StreamingResponse(
|
return StreamingResponse(
|
||||||
gen(),
|
gen(), media_type="text/event-stream", headers=SSE_HEADERS
|
||||||
media_type="text/event-stream",
|
|
||||||
headers={
|
|
||||||
"Cache-Control": "no-cache, no-transform",
|
|
||||||
"Connection": "keep-alive",
|
|
||||||
# nginx default-buffers SSE bodies; this header tells
|
|
||||||
# both nginx and uvicorn-behind-proxy to flush as we
|
|
||||||
# write. Harmless if the deployment has no reverse
|
|
||||||
# proxy in front.
|
|
||||||
"X-Accel-Buffering": "no",
|
|
||||||
},
|
|
||||||
)
|
)
|
||||||
|
|
||||||
return app
|
return app
|
||||||
@@ -312,11 +317,11 @@ class MarkdownFrontend(Frontend):
|
|||||||
content_override: Any,
|
content_override: Any,
|
||||||
agent_override: str | None,
|
agent_override: str | None,
|
||||||
) -> Any:
|
) -> Any:
|
||||||
|
write_disk = content_override is None
|
||||||
if isinstance(content_override, str):
|
if isinstance(content_override, str):
|
||||||
await _write_atomic(file_path, content_override)
|
|
||||||
file_text = content_override
|
file_text = content_override
|
||||||
elif content_override is None:
|
elif content_override is None:
|
||||||
file_text = await _read_or_empty(file_path)
|
file_text = await read_or_empty(file_path)
|
||||||
else:
|
else:
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
status.HTTP_400_BAD_REQUEST, "`content` must be a string when present"
|
status.HTTP_400_BAD_REQUEST, "`content` must be a string when present"
|
||||||
@@ -381,22 +386,22 @@ class MarkdownFrontend(Frontend):
|
|||||||
|
|
||||||
# Resolve / mint the conversation row, align incoming against
|
# Resolve / mint the conversation row, align incoming against
|
||||||
# stored history, and feed the aligned messages to the backend
|
# stored history, and feed the aligned messages to the backend
|
||||||
# — see ``core/conversation_store.py`` for the full rationale.
|
# - see ``core/conversation_store.py`` for the full rationale.
|
||||||
# If the backend isn't claude-code (no ``TurnCapture`` support)
|
|
||||||
# we fall through to the legacy parser-only path.
|
|
||||||
conv, conv_external_id, stored_msgs = await self._resolve_conversation(
|
conv, conv_external_id, stored_msgs = await self._resolve_conversation(
|
||||||
runtime=runtime, metadata=parsed.metadata, agent_name=agent.name
|
runtime=runtime,
|
||||||
|
metadata=parsed.metadata,
|
||||||
|
agent_name=agent.name,
|
||||||
|
file_path=file_path,
|
||||||
|
first_user_text=parsed.turns[0].text,
|
||||||
)
|
)
|
||||||
outcome = diff_and_fork(stored=stored_msgs, incoming=parsed.turns)
|
outcome = diff_and_fork(stored=stored_msgs, incoming=parsed.turns)
|
||||||
capture: TurnCapture | None = (
|
capture = TurnCapture()
|
||||||
TurnCapture() if isinstance(backend, ClaudeCodeBackendAdapter) else None
|
events = runtime.conversations.turn(
|
||||||
)
|
conv,
|
||||||
|
messages=outcome.messages,
|
||||||
kwargs: dict[str, Any] = {}
|
origin="user",
|
||||||
if capture is not None:
|
capture=capture,
|
||||||
kwargs["capture"] = capture
|
use_session=outcome.reuse_session,
|
||||||
events = backend.complete(
|
|
||||||
agent=agent, messages=outcome.messages, system=None, **kwargs
|
|
||||||
)
|
)
|
||||||
try:
|
try:
|
||||||
message = await self._stream_to_file(
|
message = await self._stream_to_file(
|
||||||
@@ -405,6 +410,7 @@ class MarkdownFrontend(Frontend):
|
|||||||
parsed=parsed,
|
parsed=parsed,
|
||||||
model=agent.model or agent.name,
|
model=agent.model or agent.name,
|
||||||
filename=filename,
|
filename=filename,
|
||||||
|
write_disk=write_disk,
|
||||||
)
|
)
|
||||||
except HTTPException:
|
except HTTPException:
|
||||||
raise
|
raise
|
||||||
@@ -419,11 +425,12 @@ class MarkdownFrontend(Frontend):
|
|||||||
message=message,
|
message=message,
|
||||||
agent_name=agent.name,
|
agent_name=agent.name,
|
||||||
conv_external_id=conv_external_id,
|
conv_external_id=conv_external_id,
|
||||||
|
write_disk=write_disk,
|
||||||
)
|
)
|
||||||
|
|
||||||
await self._persist_canonical_history(
|
await self._persist_canonical_history(
|
||||||
runtime=runtime,
|
runtime=runtime,
|
||||||
conversation_id=conv.id,
|
conversation_id=cast("int", conv.id),
|
||||||
persist_messages=outcome.persist_messages,
|
persist_messages=outcome.persist_messages,
|
||||||
new_user_text=parsed.turns[-1].text,
|
new_user_text=parsed.turns[-1].text,
|
||||||
capture=capture,
|
capture=capture,
|
||||||
@@ -480,17 +487,15 @@ class MarkdownFrontend(Frontend):
|
|||||||
writers in their respective halves of Obsidian Sync. Final
|
writers in their respective halves of Obsidian Sync. Final
|
||||||
content is identical on both sides, so Sync no-ops.
|
content is identical on both sides, so Sync no-ops.
|
||||||
"""
|
"""
|
||||||
# File-text resolution + early bailouts. ``content_override`` is
|
# With ``content`` the plugin is the only writer of the file
|
||||||
# still written to disk on the gateway side because that's the
|
# (§3.10): the gateway never touches disk in that case.
|
||||||
# state the rest of the request consumes; it just doesn't keep
|
write_disk = content_override is None
|
||||||
# ticking after that single write.
|
|
||||||
if isinstance(content_override, str):
|
if isinstance(content_override, str):
|
||||||
await _write_atomic(file_path, content_override)
|
|
||||||
file_text = content_override
|
file_text = content_override
|
||||||
elif content_override is None:
|
elif content_override is None:
|
||||||
file_text = await _read_or_empty(file_path)
|
file_text = await read_or_empty(file_path)
|
||||||
else:
|
else:
|
||||||
yield _sse_pack(
|
yield sse_pack(
|
||||||
"error",
|
"error",
|
||||||
{
|
{
|
||||||
"status_code": status.HTTP_400_BAD_REQUEST,
|
"status_code": status.HTTP_400_BAD_REQUEST,
|
||||||
@@ -506,7 +511,7 @@ class MarkdownFrontend(Frontend):
|
|||||||
default=self.default_agent,
|
default=self.default_agent,
|
||||||
)
|
)
|
||||||
if not agent_name:
|
if not agent_name:
|
||||||
yield _sse_pack(
|
yield sse_pack(
|
||||||
"error",
|
"error",
|
||||||
{
|
{
|
||||||
"status_code": status.HTTP_400_BAD_REQUEST,
|
"status_code": status.HTTP_400_BAD_REQUEST,
|
||||||
@@ -519,7 +524,7 @@ class MarkdownFrontend(Frontend):
|
|||||||
return
|
return
|
||||||
|
|
||||||
if not parsed.messages:
|
if not parsed.messages:
|
||||||
yield _sse_pack(
|
yield sse_pack(
|
||||||
"done",
|
"done",
|
||||||
{
|
{
|
||||||
"status": "nothing_to_do",
|
"status": "nothing_to_do",
|
||||||
@@ -530,7 +535,7 @@ class MarkdownFrontend(Frontend):
|
|||||||
return
|
return
|
||||||
|
|
||||||
if parser.last_role(parsed.messages) == "assistant":
|
if parser.last_role(parsed.messages) == "assistant":
|
||||||
yield _sse_pack(
|
yield sse_pack(
|
||||||
"done",
|
"done",
|
||||||
{
|
{
|
||||||
"status": "nothing_to_do",
|
"status": "nothing_to_do",
|
||||||
@@ -542,7 +547,7 @@ class MarkdownFrontend(Frontend):
|
|||||||
|
|
||||||
agent = runtime.agents.get(agent_name)
|
agent = runtime.agents.get(agent_name)
|
||||||
if agent is None:
|
if agent is None:
|
||||||
yield _sse_pack(
|
yield sse_pack(
|
||||||
"error",
|
"error",
|
||||||
{
|
{
|
||||||
"status_code": status.HTTP_404_NOT_FOUND,
|
"status_code": status.HTTP_404_NOT_FOUND,
|
||||||
@@ -552,7 +557,7 @@ class MarkdownFrontend(Frontend):
|
|||||||
return
|
return
|
||||||
backend = runtime.backends.get(agent.name)
|
backend = runtime.backends.get(agent.name)
|
||||||
if backend is None:
|
if backend is None:
|
||||||
yield _sse_pack(
|
yield sse_pack(
|
||||||
"error",
|
"error",
|
||||||
{
|
{
|
||||||
"status_code": status.HTTP_503_SERVICE_UNAVAILABLE,
|
"status_code": status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||||
@@ -577,9 +582,19 @@ class MarkdownFrontend(Frontend):
|
|||||||
msgs=len(parsed.messages),
|
msgs=len(parsed.messages),
|
||||||
)
|
)
|
||||||
|
|
||||||
conv, conv_external_id, stored_msgs = await self._resolve_conversation(
|
try:
|
||||||
runtime=runtime, metadata=parsed.metadata, agent_name=agent.name
|
conv, conv_external_id, stored_msgs = await self._resolve_conversation(
|
||||||
)
|
runtime=runtime,
|
||||||
|
metadata=parsed.metadata,
|
||||||
|
agent_name=agent.name,
|
||||||
|
file_path=file_path,
|
||||||
|
first_user_text=parsed.turns[0].text,
|
||||||
|
)
|
||||||
|
except HTTPException as exc:
|
||||||
|
yield sse_pack(
|
||||||
|
"error", {"status_code": exc.status_code, "detail": exc.detail}
|
||||||
|
)
|
||||||
|
return
|
||||||
_log.info(
|
_log.info(
|
||||||
"chat/stream: file=%s conv_external_id=%s conv_id=%d "
|
"chat/stream: file=%s conv_external_id=%s conv_id=%d "
|
||||||
"stored_msgs=%d incoming_turns=%d",
|
"stored_msgs=%d incoming_turns=%d",
|
||||||
@@ -598,21 +613,19 @@ class MarkdownFrontend(Frontend):
|
|||||||
len(outcome.messages),
|
len(outcome.messages),
|
||||||
len(outcome.persist_messages),
|
len(outcome.persist_messages),
|
||||||
)
|
)
|
||||||
capture: TurnCapture | None = (
|
capture = TurnCapture()
|
||||||
TurnCapture() if isinstance(backend, ClaudeCodeBackendAdapter) else None
|
|
||||||
)
|
|
||||||
|
|
||||||
kwargs: dict[str, Any] = {}
|
|
||||||
if capture is not None:
|
|
||||||
kwargs["capture"] = capture
|
|
||||||
_log.info(
|
_log.info(
|
||||||
"chat/stream: file=%s calling backend.complete agent=%s capture=%s",
|
"chat/stream: file=%s calling backend.complete agent=%s session=%s",
|
||||||
filename,
|
filename,
|
||||||
agent.name,
|
agent.name,
|
||||||
capture is not None,
|
conv.session_id,
|
||||||
)
|
)
|
||||||
events = backend.complete(
|
events = runtime.conversations.turn(
|
||||||
agent=agent, messages=outcome.messages, system=None, **kwargs
|
conv,
|
||||||
|
messages=outcome.messages,
|
||||||
|
origin="user",
|
||||||
|
capture=capture,
|
||||||
|
use_session=outcome.reuse_session,
|
||||||
)
|
)
|
||||||
|
|
||||||
acc = StreamAccumulator()
|
acc = StreamAccumulator()
|
||||||
@@ -626,16 +639,12 @@ class MarkdownFrontend(Frontend):
|
|||||||
return None
|
return None
|
||||||
rendered = renderer.render_assistant_message(partial)
|
rendered = renderer.render_assistant_message(partial)
|
||||||
new_body = renderer.append_to_body(parsed.body, rendered)
|
new_body = renderer.append_to_body(parsed.body, rendered)
|
||||||
return _reattach_frontmatter(parsed.metadata, new_body)
|
return reattach_frontmatter(parsed.metadata, new_body)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
async for ev in _events_with_heartbeat(events):
|
async for ev in events_with_heartbeat(events):
|
||||||
if ev is None:
|
if ev is None:
|
||||||
# Backend is quiet (claude mid-thinking, MCP slow,
|
yield KEEPALIVE
|
||||||
# whatever). SSE comment-frame keeps the TCP socket
|
|
||||||
# warm so the plugin / uvicorn / any reverse proxy
|
|
||||||
# doesn't time the request out before we finish.
|
|
||||||
yield b": keepalive\n\n"
|
|
||||||
continue
|
continue
|
||||||
acc.feed(ev)
|
acc.feed(ev)
|
||||||
now = time.monotonic()
|
now = time.monotonic()
|
||||||
@@ -648,7 +657,7 @@ class MarkdownFrontend(Frontend):
|
|||||||
# render to the same prefix as before they closed
|
# render to the same prefix as before they closed
|
||||||
# (we don't surface the tool-call args in markdown).
|
# (we don't surface the tool-call args in markdown).
|
||||||
if payload is not None and payload != last_payload:
|
if payload is not None and payload != last_payload:
|
||||||
yield _sse_pack("delta", {"new_content": payload})
|
yield sse_pack("delta", {"new_content": payload})
|
||||||
last_payload = payload
|
last_payload = payload
|
||||||
last_flush = now
|
last_flush = now
|
||||||
except Exception as exc: # noqa: BLE001 — wire any backend failure as an SSE error frame
|
except Exception as exc: # noqa: BLE001 — wire any backend failure as an SSE error frame
|
||||||
@@ -663,10 +672,11 @@ class MarkdownFrontend(Frontend):
|
|||||||
new_body, renderer.render_assistant_message(partial)
|
new_body, renderer.render_assistant_message(partial)
|
||||||
)
|
)
|
||||||
new_body = renderer.append_to_body(new_body, _render_error_block(exc))
|
new_body = renderer.append_to_body(new_body, _render_error_block(exc))
|
||||||
await _write_atomic(
|
if write_disk:
|
||||||
file_path, _reattach_frontmatter(parsed.metadata, new_body)
|
await write_atomic(
|
||||||
)
|
file_path, reattach_frontmatter(parsed.metadata, new_body)
|
||||||
yield _sse_pack(
|
)
|
||||||
|
yield sse_pack(
|
||||||
"error",
|
"error",
|
||||||
{
|
{
|
||||||
"status_code": status.HTTP_500_INTERNAL_SERVER_ERROR,
|
"status_code": status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||||
@@ -683,11 +693,12 @@ class MarkdownFrontend(Frontend):
|
|||||||
message=message,
|
message=message,
|
||||||
agent_name=agent.name,
|
agent_name=agent.name,
|
||||||
conv_external_id=conv_external_id,
|
conv_external_id=conv_external_id,
|
||||||
|
write_disk=write_disk,
|
||||||
)
|
)
|
||||||
|
|
||||||
await self._persist_canonical_history(
|
await self._persist_canonical_history(
|
||||||
runtime=runtime,
|
runtime=runtime,
|
||||||
conversation_id=conv.id,
|
conversation_id=cast("int", conv.id),
|
||||||
persist_messages=outcome.persist_messages,
|
persist_messages=outcome.persist_messages,
|
||||||
new_user_text=parsed.turns[-1].text,
|
new_user_text=parsed.turns[-1].text,
|
||||||
capture=capture,
|
capture=capture,
|
||||||
@@ -707,7 +718,7 @@ class MarkdownFrontend(Frontend):
|
|||||||
except Exception: # noqa: BLE001
|
except Exception: # noqa: BLE001
|
||||||
_log.exception("turn_log_handler raised; continuing")
|
_log.exception("turn_log_handler raised; continuing")
|
||||||
|
|
||||||
yield _sse_pack(
|
yield sse_pack(
|
||||||
"done",
|
"done",
|
||||||
{
|
{
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
@@ -727,6 +738,7 @@ class MarkdownFrontend(Frontend):
|
|||||||
parsed: parser.ParsedFile,
|
parsed: parser.ParsedFile,
|
||||||
model: str,
|
model: str,
|
||||||
filename: str,
|
filename: str,
|
||||||
|
write_disk: bool = True,
|
||||||
) -> Any:
|
) -> Any:
|
||||||
"""Drain ``events`` into a ``Message``, flushing partials to disk.
|
"""Drain ``events`` into a ``Message``, flushing partials to disk.
|
||||||
|
|
||||||
@@ -745,13 +757,15 @@ class MarkdownFrontend(Frontend):
|
|||||||
acc = StreamAccumulator()
|
acc = StreamAccumulator()
|
||||||
|
|
||||||
async def flush_partial() -> None:
|
async def flush_partial() -> None:
|
||||||
|
if not write_disk:
|
||||||
|
return
|
||||||
partial = acc.finalize(model=model)
|
partial = acc.finalize(model=model)
|
||||||
if not partial.content:
|
if not partial.content:
|
||||||
return
|
return
|
||||||
rendered = renderer.render_assistant_message(partial)
|
rendered = renderer.render_assistant_message(partial)
|
||||||
new_body = renderer.append_to_body(parsed.body, rendered)
|
new_body = renderer.append_to_body(parsed.body, rendered)
|
||||||
await _write_atomic(
|
await write_atomic(
|
||||||
file_path, _reattach_frontmatter(parsed.metadata, new_body)
|
file_path, reattach_frontmatter(parsed.metadata, new_body)
|
||||||
)
|
)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
@@ -774,9 +788,10 @@ class MarkdownFrontend(Frontend):
|
|||||||
new_body, renderer.render_assistant_message(partial)
|
new_body, renderer.render_assistant_message(partial)
|
||||||
)
|
)
|
||||||
new_body = renderer.append_to_body(new_body, _render_error_block(exc))
|
new_body = renderer.append_to_body(new_body, _render_error_block(exc))
|
||||||
await _write_atomic(
|
if write_disk:
|
||||||
file_path, _reattach_frontmatter(parsed.metadata, new_body)
|
await write_atomic(
|
||||||
)
|
file_path, reattach_frontmatter(parsed.metadata, new_body)
|
||||||
|
)
|
||||||
raise
|
raise
|
||||||
return acc.finalize(model=model)
|
return acc.finalize(model=model)
|
||||||
|
|
||||||
@@ -788,75 +803,70 @@ class MarkdownFrontend(Frontend):
|
|||||||
message: Any,
|
message: Any,
|
||||||
agent_name: str,
|
agent_name: str,
|
||||||
conv_external_id: str,
|
conv_external_id: str,
|
||||||
|
write_disk: bool = True,
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Render the assistant turn, append to the file, refresh frontmatter."""
|
"""Render the assistant turn, refresh frontmatter, write if we own the file."""
|
||||||
rendered = renderer.render_assistant_message(message)
|
rendered = renderer.render_assistant_message(message)
|
||||||
new_body = renderer.append_to_body(parsed.body, rendered)
|
new_body = renderer.append_to_body(parsed.body, rendered)
|
||||||
new_body = renderer.append_to_body(new_body, renderer.USER_SCAFFOLD)
|
new_body = renderer.append_to_body(new_body, renderer.USER_SCAFFOLD)
|
||||||
# Recompute fingerprint so a future cross-frontend hit on this
|
|
||||||
# same conversation can find it. Stored as hex string in
|
|
||||||
# frontmatter — only the markdown frontend reads it.
|
|
||||||
assistant_param: MessageParam = {
|
|
||||||
"role": "assistant",
|
|
||||||
"content": _flatten_assistant_text(message),
|
|
||||||
}
|
|
||||||
updated_messages: list[MessageParam] = [*parsed.messages, assistant_param]
|
|
||||||
updated_metadata = dict(parsed.metadata)
|
updated_metadata = dict(parsed.metadata)
|
||||||
|
updated_metadata.pop("fingerprint", None)
|
||||||
updated_metadata["agent"] = agent_name
|
updated_metadata["agent"] = agent_name
|
||||||
updated_metadata["conversation_id"] = conv_external_id
|
updated_metadata["conversation_id"] = conv_external_id
|
||||||
updated_metadata["fingerprint"] = fingerprint_messages(updated_messages)
|
new_content = reattach_frontmatter(updated_metadata, new_body)
|
||||||
new_content = _reattach_frontmatter(updated_metadata, new_body)
|
if write_disk:
|
||||||
await _write_atomic(file_path, new_content)
|
await write_atomic(file_path, new_content)
|
||||||
return new_content
|
return new_content
|
||||||
|
|
||||||
async def _resolve_conversation(
|
async def _resolve_conversation(
|
||||||
self, *, runtime: GatewayRuntime, metadata: dict[str, Any], agent_name: str
|
self,
|
||||||
) -> tuple[Any, str, list[dict[str, Any]]]:
|
*,
|
||||||
"""Resolve the conversation row + stored messages for this request.
|
runtime: GatewayRuntime,
|
||||||
|
metadata: dict[str, Any],
|
||||||
|
agent_name: str,
|
||||||
|
file_path: Path,
|
||||||
|
first_user_text: str = "",
|
||||||
|
) -> tuple[Conversation, str, list[dict[str, Any]]]:
|
||||||
|
"""Resolve the ``deep`` conversation for this file + its stored messages.
|
||||||
|
|
||||||
Looks up by frontmatter ``conversation_id``, mints a new row if
|
Frontmatter ``conversation_id`` wins; a file that lost it is found
|
||||||
missing, and returns ``(conv, external_id, stored_messages)``.
|
by its visible ``(markdown, path)`` binding, then by adopting the
|
||||||
``conv.id`` is guaranteed non-None because both
|
one unbound pre-SDK conversation that starts with the same prompt;
|
||||||
``load_conversation`` (after refresh on a committed row) and
|
otherwise a new conversation is created. The binding follows the
|
||||||
``mint_conversation`` (post-commit refresh) populate it. We
|
file: a moved chat re-binds to its new path on the next turn.
|
||||||
coerce with a runtime check so the rest of the handler can
|
|
||||||
treat it as ``int``.
|
|
||||||
"""
|
"""
|
||||||
|
conversations = runtime.conversations
|
||||||
|
rel = file_path.relative_to(self.vault_path).as_posix()
|
||||||
raw = metadata.get("conversation_id")
|
raw = metadata.get("conversation_id")
|
||||||
lookup_id = raw if isinstance(raw, str) and raw else None
|
conv = await conversations.get(raw) if isinstance(raw, str) and raw else None
|
||||||
|
if conv is None:
|
||||||
|
conv = await conversations.find_bound(frontend=FRONTEND, external_id=rel)
|
||||||
|
if conv is None:
|
||||||
|
conv = await conversations.adopt(
|
||||||
|
kind="deep", first_user_text=first_user_text
|
||||||
|
)
|
||||||
|
if conv is not None:
|
||||||
|
_log.info("adopted conversation %s for %s", conv.external_id, rel)
|
||||||
|
if conv is None:
|
||||||
|
try:
|
||||||
|
conv = await conversations.create(
|
||||||
|
kind="deep", agent=agent_name, origin=FRONTEND, title=file_path.stem
|
||||||
|
)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status.HTTP_400_BAD_REQUEST, str(exc)) from exc
|
||||||
|
_log.info("minted conversation %s for %s", conv.external_id, rel)
|
||||||
|
bound = [
|
||||||
|
b
|
||||||
|
for b in await conversations.bindings(conv)
|
||||||
|
if b.frontend == FRONTEND and b.visible and b.external_id == rel
|
||||||
|
]
|
||||||
|
if not bound:
|
||||||
|
await conversations.bind(conv, frontend=FRONTEND, external_id=rel)
|
||||||
|
await conversations.touch_user(conv)
|
||||||
|
if conv.id is None:
|
||||||
|
msg = "conversation row missing primary key after commit"
|
||||||
|
raise RuntimeError(msg)
|
||||||
async with runtime.db.session() as session:
|
async with runtime.db.session() as session:
|
||||||
conv = None
|
|
||||||
if lookup_id is not None:
|
|
||||||
conv = await load_conversation(
|
|
||||||
session, frontend="markdown", external_id=lookup_id
|
|
||||||
)
|
|
||||||
if conv is None:
|
|
||||||
_log.info(
|
|
||||||
"_resolve_conversation: frontmatter conv_id=%s "
|
|
||||||
"not found in DB, will mint new",
|
|
||||||
lookup_id,
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
_log.info(
|
|
||||||
"_resolve_conversation: LOADED existing conv "
|
|
||||||
"id=%d external_id=%s",
|
|
||||||
conv.id or -1,
|
|
||||||
conv.external_id,
|
|
||||||
)
|
|
||||||
if conv is None:
|
|
||||||
conv = await mint_conversation(
|
|
||||||
session, frontend="markdown", agent_name=agent_name
|
|
||||||
)
|
|
||||||
_log.info(
|
|
||||||
"_resolve_conversation: MINTED new conv "
|
|
||||||
"id=%d external_id=%s agent=%s",
|
|
||||||
conv.id or -1,
|
|
||||||
conv.external_id,
|
|
||||||
agent_name,
|
|
||||||
)
|
|
||||||
if conv.id is None:
|
|
||||||
msg = "conversation row missing primary key after commit"
|
|
||||||
raise RuntimeError(msg)
|
|
||||||
stored = await load_messages(session, conversation_id=conv.id)
|
stored = await load_messages(session, conversation_id=conv.id)
|
||||||
return conv, conv.external_id, stored
|
return conv, conv.external_id, stored
|
||||||
|
|
||||||
@@ -867,21 +877,17 @@ class MarkdownFrontend(Frontend):
|
|||||||
conversation_id: int,
|
conversation_id: int,
|
||||||
persist_messages: list[dict[str, Any]],
|
persist_messages: list[dict[str, Any]],
|
||||||
new_user_text: str,
|
new_user_text: str,
|
||||||
capture: TurnCapture | None,
|
capture: TurnCapture,
|
||||||
message: Any,
|
message: Any,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Stamp the DB with the post-turn canonical Anthropic-shape history.
|
"""Stamp the DB with the post-turn canonical Anthropic-shape history.
|
||||||
|
|
||||||
Combines the matched/spliced prior state, the new user prompt,
|
Combines the matched/spliced prior state, the new user prompt,
|
||||||
and the synthesized assistant↔tool cycle from the backend (or
|
and the synthesized assistant/tool cycle from the backend (or a
|
||||||
a text-only fallback for backends without ``TurnCapture``).
|
text-only fallback for backends that left ``capture`` empty).
|
||||||
"""
|
"""
|
||||||
new_user_msg = {"role": "user", "content": new_user_text}
|
new_user_msg = {"role": "user", "content": new_user_text}
|
||||||
synthesized = (
|
synthesized = capture.synthesized_messages or _fallback_synthesized(message)
|
||||||
capture.synthesized_messages
|
|
||||||
if capture is not None
|
|
||||||
else _fallback_synthesized(message)
|
|
||||||
)
|
|
||||||
canonical = [*persist_messages, new_user_msg, *synthesized]
|
canonical = [*persist_messages, new_user_msg, *synthesized]
|
||||||
_log.info(
|
_log.info(
|
||||||
"_persist_canonical_history: conv_id=%d writing %d msgs "
|
"_persist_canonical_history: conv_id=%d writing %d msgs "
|
||||||
@@ -919,115 +925,6 @@ class MarkdownFrontend(Frontend):
|
|||||||
return candidate
|
return candidate
|
||||||
|
|
||||||
|
|
||||||
# ---- module-level utilities ----------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
async def _events_with_heartbeat(
|
|
||||||
events: AsyncIterator[Any], interval: float = _SSE_HEARTBEAT_INTERVAL
|
|
||||||
) -> AsyncIterator[Any]:
|
|
||||||
"""Wrap an async event stream with idle-time heartbeat markers.
|
|
||||||
|
|
||||||
Yields ``None`` every ``interval`` seconds during silence; real
|
|
||||||
events pass through unchanged. When the wrapped iterator is
|
|
||||||
exhausted, this generator returns. Cancellation propagates: if the
|
|
||||||
outer scope is cancelled we cancel the pending ``__anext__`` task
|
|
||||||
instead of leaving it dangling.
|
|
||||||
"""
|
|
||||||
src = events.__aiter__()
|
|
||||||
next_task: asyncio.Task[Any] | None = None
|
|
||||||
try:
|
|
||||||
while True:
|
|
||||||
# Reuse the in-flight task across timeouts. Spawning a fresh
|
|
||||||
# ``__anext__()`` while the previous one is still pending
|
|
||||||
# puts two consumers on the same async generator — that
|
|
||||||
# raises ``RuntimeError: anext(): asynchronous generator is
|
|
||||||
# already running``.
|
|
||||||
if next_task is None:
|
|
||||||
next_task = asyncio.ensure_future(src.__anext__())
|
|
||||||
done, _pending = await asyncio.wait({next_task}, timeout=interval)
|
|
||||||
if not done:
|
|
||||||
yield None
|
|
||||||
continue
|
|
||||||
task = next_task
|
|
||||||
next_task = None
|
|
||||||
try:
|
|
||||||
result = task.result()
|
|
||||||
except StopAsyncIteration:
|
|
||||||
return
|
|
||||||
yield result
|
|
||||||
finally:
|
|
||||||
if next_task is not None and not next_task.done():
|
|
||||||
next_task.cancel()
|
|
||||||
with contextlib.suppress(BaseException):
|
|
||||||
await next_task
|
|
||||||
|
|
||||||
|
|
||||||
def _sse_pack(event: str, data: dict[str, Any]) -> bytes:
|
|
||||||
r"""Format one Server-Sent Event frame.
|
|
||||||
|
|
||||||
Uses named events (``event: <name>``) so the plugin can dispatch on
|
|
||||||
type without parsing JSON discriminators. ``ensure_ascii=False`` so
|
|
||||||
multibyte content rides through verbatim instead of becoming
|
|
||||||
``\uXXXX`` blobs that bloat the wire.
|
|
||||||
"""
|
|
||||||
body = json.dumps(data, ensure_ascii=False)
|
|
||||||
return f"event: {event}\ndata: {body}\n\n".encode()
|
|
||||||
|
|
||||||
|
|
||||||
async def _read_or_empty(path: Path) -> str:
|
|
||||||
"""Return file contents, or empty string if the file doesn't exist."""
|
|
||||||
# ``path.exists()`` here is a metadata stat — microseconds — and
|
|
||||||
# gating an async read on whether the file is there is exactly the
|
|
||||||
# check we want. Switching to anyio.Path / aiofiles.os just to
|
|
||||||
# silence the async-pathlib lint would cost a dep edge for no
|
|
||||||
# practical win.
|
|
||||||
if not path.exists(): # noqa: ASYNC240
|
|
||||||
return ""
|
|
||||||
async with aiofile.async_open(path, "r", encoding="utf-8") as f:
|
|
||||||
return await f.read()
|
|
||||||
|
|
||||||
|
|
||||||
async def _write_atomic(path: Path, content: str) -> None:
|
|
||||||
"""Write ``content`` to ``path`` via tmp + ``os.replace`` (atomic)."""
|
|
||||||
path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
# ``NamedTemporaryFile`` keeps the file open which complicates
|
|
||||||
# ``os.replace`` on some platforms. Build the tmp name manually,
|
|
||||||
# write+fsync, then rename. Same-directory so the rename is atomic.
|
|
||||||
tmp_name = tempfile.mkstemp(
|
|
||||||
prefix=f".{path.name}.", suffix=".tmp", dir=str(path.parent)
|
|
||||||
)
|
|
||||||
fd, tmp_path = tmp_name
|
|
||||||
try:
|
|
||||||
async with aiofile.async_open(tmp_path, "w", encoding="utf-8") as f:
|
|
||||||
await f.write(content)
|
|
||||||
os.close(fd)
|
|
||||||
# ``os.replace`` is the atomic primitive — ``Path.replace`` is a
|
|
||||||
# thin wrapper around the same syscall; either works, ``os.`` is
|
|
||||||
# the one Linux/POSIX docs reach for.
|
|
||||||
os.replace(tmp_path, path) # noqa: PTH105
|
|
||||||
except BaseException:
|
|
||||||
# Cleanup on failure: close fd, remove tmp.
|
|
||||||
with contextlib.suppress(OSError):
|
|
||||||
os.close(fd)
|
|
||||||
with contextlib.suppress(OSError):
|
|
||||||
os.unlink(tmp_path) # noqa: PTH108
|
|
||||||
raise
|
|
||||||
|
|
||||||
|
|
||||||
def _reattach_frontmatter(metadata: dict[str, Any], body: str) -> str:
|
|
||||||
r"""Re-emit a ``.md`` file with YAML frontmatter at the top.
|
|
||||||
|
|
||||||
Empty metadata → no frontmatter block (avoid littering every file
|
|
||||||
with a hollow ``---\n---``).
|
|
||||||
"""
|
|
||||||
if not metadata:
|
|
||||||
return body if body.endswith("\n") else body + "\n"
|
|
||||||
import frontmatter as _fm
|
|
||||||
|
|
||||||
post = _fm.Post(content=body, **metadata)
|
|
||||||
return _fm.dumps(post) + "\n"
|
|
||||||
|
|
||||||
|
|
||||||
def _fallback_synthesized(message: Any) -> list[dict[str, Any]]:
|
def _fallback_synthesized(message: Any) -> list[dict[str, Any]]:
|
||||||
"""Build a single-assistant ``synthesized_messages`` list from a raw ``Message``.
|
"""Build a single-assistant ``synthesized_messages`` list from a raw ``Message``.
|
||||||
|
|
||||||
@@ -1065,21 +962,6 @@ def _fallback_synthesized(message: Any) -> list[dict[str, Any]]:
|
|||||||
return [{"role": "assistant", "content": content}]
|
return [{"role": "assistant", "content": content}]
|
||||||
|
|
||||||
|
|
||||||
def _flatten_assistant_text(message: Any) -> str:
|
|
||||||
"""Pull all text blocks from an assistant ``Message`` and join them.
|
|
||||||
|
|
||||||
Used when we need the assistant content as a plain string for
|
|
||||||
fingerprinting / equality with a parser-shaped history (parser
|
|
||||||
already drops thinking + tool_use from assistant turns).
|
|
||||||
"""
|
|
||||||
chunks = [
|
|
||||||
getattr(block, "text", "") or ""
|
|
||||||
for block in getattr(message, "content", ())
|
|
||||||
if getattr(block, "type", None) == "text"
|
|
||||||
]
|
|
||||||
return "\n\n".join(c for c in chunks if c)
|
|
||||||
|
|
||||||
|
|
||||||
def _render_error_block(exc: BaseException) -> str:
|
def _render_error_block(exc: BaseException) -> str:
|
||||||
"""Render a backend failure as an Assistant turn with a ``[!error]-`` callout."""
|
"""Render a backend failure as an Assistant turn with a ``[!error]-`` callout."""
|
||||||
msg = str(exc) or exc.__class__.__name__
|
msg = str(exc) or exc.__class__.__name__
|
||||||
|
|||||||
@@ -0,0 +1,169 @@
|
|||||||
|
"""Vault files for ``deep`` conversations that were not typed into a file (§3.10).
|
||||||
|
|
||||||
|
``materialize`` is the markdown frontend's answer to ``spawn(kind=deep)``:
|
||||||
|
a new file in the vault with ``agent`` + ``conversation_id`` frontmatter
|
||||||
|
and the ``(markdown, path)`` binding. ``run`` tails the gateway bus and
|
||||||
|
appends every ``reply`` of a markdown-bound conversation to its file -
|
||||||
|
the seed turn of a spawn, a message posted through ``/api``, a turn
|
||||||
|
that came in over ``/v1/messages`` - and stamps the same exchange into
|
||||||
|
the canonical history, so a continuation typed in Obsidian aligns
|
||||||
|
against the store and resumes the same SDK session instead of reseeding.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import logging
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
|
import frontmatter
|
||||||
|
|
||||||
|
from beaver_gateway.core.conversation_store import load_messages, rewrite_messages
|
||||||
|
from beaver_gateway.core.turn_record import slugify
|
||||||
|
from beaver_gateway.frontends.markdown import renderer
|
||||||
|
from beaver_gateway.frontends.markdown.crossfront import strip_trailing_user_scaffold
|
||||||
|
from beaver_gateway.frontends.markdown.files import (
|
||||||
|
read_or_empty,
|
||||||
|
reattach_frontmatter,
|
||||||
|
write_atomic,
|
||||||
|
)
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Callable, Sequence
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from beaver_gateway.core.bus import Event
|
||||||
|
from beaver_gateway.frontends.base import GatewayRuntime
|
||||||
|
from beaver_gateway.storage.models import Conversation, ConversationBinding
|
||||||
|
|
||||||
|
__all__ = ["FRONTEND", "ChatMirror"]
|
||||||
|
|
||||||
|
_log = logging.getLogger("beaver_gateway.frontends.markdown.mirror")
|
||||||
|
|
||||||
|
FRONTEND = "markdown"
|
||||||
|
|
||||||
|
|
||||||
|
class ChatMirror:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
vault_path: Path,
|
||||||
|
runtime: GatewayRuntime,
|
||||||
|
logged_subdir: str,
|
||||||
|
chat_path: Callable[[str, str, Path], Path] | None = None,
|
||||||
|
) -> None:
|
||||||
|
self._vault = vault_path
|
||||||
|
self._runtime = runtime
|
||||||
|
self._root = vault_path / logged_subdir
|
||||||
|
self._chat_path_fn = chat_path
|
||||||
|
self._lock = asyncio.Lock()
|
||||||
|
|
||||||
|
def path_for(self, title: str, agent: str) -> Path:
|
||||||
|
if self._chat_path_fn is not None:
|
||||||
|
path = self._chat_path_fn(title, agent, self._vault)
|
||||||
|
if not path.is_absolute():
|
||||||
|
path = self._vault / path
|
||||||
|
else:
|
||||||
|
day = datetime.now(UTC).strftime("%Y-%m-%d")
|
||||||
|
path = self._root / agent / f"{day}_{slugify(title, maxlen=60)}.md"
|
||||||
|
if path.suffix != ".md":
|
||||||
|
path = path.with_suffix(".md")
|
||||||
|
path = path.resolve()
|
||||||
|
path.relative_to(self._vault)
|
||||||
|
candidate, n = path, 1
|
||||||
|
while candidate.exists():
|
||||||
|
n += 1
|
||||||
|
candidate = path.with_name(f"{path.stem} ({n}){path.suffix}")
|
||||||
|
return candidate
|
||||||
|
|
||||||
|
async def materialize(self, conv: Conversation) -> ConversationBinding:
|
||||||
|
path = self.path_for(conv.title or conv.external_id, conv.agent_name)
|
||||||
|
rel = path.relative_to(self._vault).as_posix()
|
||||||
|
async with self._lock:
|
||||||
|
await write_atomic(path, reattach_frontmatter(_frontmatter(conv), ""))
|
||||||
|
binding = await self._runtime.conversations.bind(
|
||||||
|
conv, frontend=FRONTEND, external_id=rel
|
||||||
|
)
|
||||||
|
_log.info("materialized %s as %s", conv.external_id, rel)
|
||||||
|
return binding
|
||||||
|
|
||||||
|
async def bound_path(self, conv: Conversation) -> Path | None:
|
||||||
|
for b in await self._runtime.conversations.bindings(conv):
|
||||||
|
if b.frontend == FRONTEND and b.visible:
|
||||||
|
return self._vault / b.external_id
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def run(self) -> None:
|
||||||
|
async for event in self._runtime.bus.stream():
|
||||||
|
if event.get("type") != "reply":
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
await self._on_reply(event)
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
_log.exception("mirror of reply %s failed", event.get("turn_id"))
|
||||||
|
|
||||||
|
async def _on_reply(self, event: Event) -> None:
|
||||||
|
conv = await self._runtime.conversations.get(str(event["conversation_id"]))
|
||||||
|
if conv is None:
|
||||||
|
return
|
||||||
|
await self.append(
|
||||||
|
conv,
|
||||||
|
prompt=str(event.get("prompt") or ""),
|
||||||
|
text=str(event.get("text") or ""),
|
||||||
|
history=event.get("history"),
|
||||||
|
)
|
||||||
|
|
||||||
|
async def append(
|
||||||
|
self,
|
||||||
|
conv: Conversation,
|
||||||
|
*,
|
||||||
|
prompt: str,
|
||||||
|
text: str,
|
||||||
|
history: Sequence[dict[str, Any]] | None = None,
|
||||||
|
) -> None:
|
||||||
|
path = await self.bound_path(conv)
|
||||||
|
if path is None:
|
||||||
|
return
|
||||||
|
prior = [
|
||||||
|
{"role": str(h["role"]), "content": str(h["text"])}
|
||||||
|
for h in history or ()
|
||||||
|
if h.get("text")
|
||||||
|
]
|
||||||
|
async with self._lock:
|
||||||
|
parsed = frontmatter.loads(await read_or_empty(path))
|
||||||
|
body = strip_trailing_user_scaffold(parsed.content)
|
||||||
|
if prior and "### " not in body:
|
||||||
|
for m in prior:
|
||||||
|
body = renderer.append_to_body(body, _render(m))
|
||||||
|
body = renderer.append_to_body(body, renderer.render_user_text(prompt))
|
||||||
|
body = renderer.append_to_body(body, renderer.render_assistant_text(text))
|
||||||
|
body = renderer.append_to_body(body, renderer.USER_SCAFFOLD)
|
||||||
|
metadata = {**parsed.metadata, **_frontmatter(conv)}
|
||||||
|
metadata.pop("fingerprint", None)
|
||||||
|
await write_atomic(path, reattach_frontmatter(metadata, body))
|
||||||
|
await self._persist(conv, prior=prior, prompt=prompt, text=text)
|
||||||
|
|
||||||
|
async def _persist(
|
||||||
|
self, conv: Conversation, *, prior: list[dict[str, Any]], prompt: str, text: str
|
||||||
|
) -> None:
|
||||||
|
if conv.id is None:
|
||||||
|
return
|
||||||
|
async with self._runtime.db.session() as session:
|
||||||
|
stored = await load_messages(session, conversation_id=conv.id)
|
||||||
|
canonical = [
|
||||||
|
*(stored or prior),
|
||||||
|
{"role": "user", "content": prompt},
|
||||||
|
{"role": "assistant", "content": [{"type": "text", "text": text}]},
|
||||||
|
]
|
||||||
|
await rewrite_messages(session, conversation_id=conv.id, messages=canonical)
|
||||||
|
|
||||||
|
|
||||||
|
def _frontmatter(conv: Conversation) -> dict[str, Any]:
|
||||||
|
return {"agent": conv.agent_name, "conversation_id": conv.external_id}
|
||||||
|
|
||||||
|
|
||||||
|
def _render(message: dict[str, Any]) -> str:
|
||||||
|
if message["role"] == "user":
|
||||||
|
return renderer.render_user_text(message["content"])
|
||||||
|
return renderer.render_assistant_text(message["content"])
|
||||||
@@ -9,11 +9,10 @@ from other frontends).
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import json
|
|
||||||
import re
|
import re
|
||||||
from typing import TYPE_CHECKING, Any, cast
|
from typing import TYPE_CHECKING, Any, cast
|
||||||
|
|
||||||
from anthropic.types import Message, TextBlock, ThinkingBlock, ToolUseBlock
|
from anthropic.types import Message, TextBlock, ThinkingBlock
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from collections.abc import Iterable
|
from collections.abc import Iterable
|
||||||
@@ -34,7 +33,9 @@ __all__ = [
|
|||||||
# Empty ``### User:`` block appended after each assistant reply so the
|
# Empty ``### User:`` block appended after each assistant reply so the
|
||||||
# human has an obvious place to type the next turn. Parser drops empty
|
# human has an obvious place to type the next turn. Parser drops empty
|
||||||
# user blocks, so this doesn't re-trigger dispatch on its own.
|
# user blocks, so this doesn't re-trigger dispatch on its own.
|
||||||
USER_SCAFFOLD = "### User:\n"
|
# Blank line after the header, like every rendered turn - the file stays
|
||||||
|
# symmetric whether the human or the gateway wrote the marker.
|
||||||
|
USER_SCAFFOLD = "### User:\n\n"
|
||||||
|
|
||||||
|
|
||||||
# Default 4-backtick fence so tool results that contain literal ```` ``` ````
|
# Default 4-backtick fence so tool results that contain literal ```` ``` ````
|
||||||
@@ -54,6 +55,11 @@ def render_user_text(content: str) -> str:
|
|||||||
return f"### User:\n\n{content.strip()}\n"
|
return f"### User:\n\n{content.strip()}\n"
|
||||||
|
|
||||||
|
|
||||||
|
def render_assistant_text(text: str) -> str:
|
||||||
|
r"""Render a text-only assistant turn as ``### Assistant:\n\n<text>``."""
|
||||||
|
return f"### Assistant:\n\n{text.strip()}\n"
|
||||||
|
|
||||||
|
|
||||||
def render_assistant_message(message: Message) -> str:
|
def render_assistant_message(message: Message) -> str:
|
||||||
"""Render an assistant ``Message`` (with content blocks) into a turn block.
|
"""Render an assistant ``Message`` (with content blocks) into a turn block.
|
||||||
|
|
||||||
@@ -61,16 +67,20 @@ def render_assistant_message(message: Message) -> str:
|
|||||||
|
|
||||||
* ``ThinkingBlock`` → ``> [!thinking]-`` collapsed callout
|
* ``ThinkingBlock`` → ``> [!thinking]-`` collapsed callout
|
||||||
* ``TextBlock`` → plain text (the spoken answer)
|
* ``TextBlock`` → plain text (the spoken answer)
|
||||||
* ``ToolUseBlock`` → ``> [!tool]- <name>`` callout with the ``input``
|
* ``ToolUseBlock`` → nothing (§3.10: tool calls never reach the file;
|
||||||
JSON quoted inside. Tool *results* are not persisted — see
|
"what the agent is doing" is the activity panel fed by SSE)
|
||||||
module docstring on ``parser.py`` for why.
|
|
||||||
|
|
||||||
Blank lines separate adjacent blocks; trailing newline guarantees
|
Blank lines separate adjacent blocks; trailing newline guarantees
|
||||||
the next ``---`` / ``### User:`` marker lands on its own line.
|
the next ``---`` / ``### User:`` marker lands on its own line.
|
||||||
"""
|
"""
|
||||||
parts: list[str] = ["### Assistant:", ""]
|
parts: list[str] = ["### Assistant:", ""]
|
||||||
for block in message.content:
|
for block in message.content:
|
||||||
parts.extend(_render_block(block))
|
lines = list(_render_block(block))
|
||||||
|
if not lines:
|
||||||
|
# Tool calls render to nothing - no separator for them either,
|
||||||
|
# or every tool leaves a blank line behind.
|
||||||
|
continue
|
||||||
|
parts.extend(lines)
|
||||||
parts.append("")
|
parts.append("")
|
||||||
return "\n".join(parts).rstrip() + "\n"
|
return "\n".join(parts).rstrip() + "\n"
|
||||||
|
|
||||||
@@ -147,10 +157,7 @@ def _render_block(block: object) -> Iterable[str]:
|
|||||||
if isinstance(block, ThinkingBlock):
|
if isinstance(block, ThinkingBlock):
|
||||||
yield from _render_thinking(block.thinking or "")
|
yield from _render_thinking(block.thinking or "")
|
||||||
return
|
return
|
||||||
if isinstance(block, ToolUseBlock):
|
# Tool-use blocks and unknown block types never reach the file.
|
||||||
yield from _render_tool_use(block)
|
|
||||||
return
|
|
||||||
# Unknown block type — skip silently rather than corrupting the file.
|
|
||||||
|
|
||||||
|
|
||||||
def _render_thinking(text: str) -> Iterable[str]:
|
def _render_thinking(text: str) -> Iterable[str]:
|
||||||
@@ -159,17 +166,6 @@ def _render_thinking(text: str) -> Iterable[str]:
|
|||||||
yield f"> {line}" if line else ">"
|
yield f"> {line}" if line else ">"
|
||||||
|
|
||||||
|
|
||||||
def _render_tool_use(block: ToolUseBlock) -> Iterable[str]:
|
|
||||||
title = summarize_tool_input(block.name, block.input)
|
|
||||||
yield f"> [!tool]- {title}"
|
|
||||||
yield "> **input:**"
|
|
||||||
yield "> ```json"
|
|
||||||
pretty = json.dumps(block.input, indent=2, ensure_ascii=False, sort_keys=True)
|
|
||||||
for line in pretty.splitlines():
|
|
||||||
yield f"> {line}" if line else ">"
|
|
||||||
yield "> ```"
|
|
||||||
|
|
||||||
|
|
||||||
def adaptive_fence(content: str) -> str:
|
def adaptive_fence(content: str) -> str:
|
||||||
"""Return a backtick fence at least one longer than the longest run in ``content``.
|
"""Return a backtick fence at least one longer than the longest run in ``content``.
|
||||||
|
|
||||||
|
|||||||
@@ -37,7 +37,6 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
import logging
|
import logging
|
||||||
from contextlib import asynccontextmanager
|
|
||||||
from typing import TYPE_CHECKING, Any
|
from typing import TYPE_CHECKING, Any
|
||||||
from urllib.parse import urlencode
|
from urllib.parse import urlencode
|
||||||
|
|
||||||
@@ -47,6 +46,7 @@ from starlette.responses import HTMLResponse, JSONResponse, StreamingResponse
|
|||||||
from starlette.routing import Route
|
from starlette.routing import Route
|
||||||
|
|
||||||
from beaver_gateway.core import audit
|
from beaver_gateway.core import audit
|
||||||
|
from beaver_gateway.frontends._urls import external_base
|
||||||
from beaver_gateway.frontends.base import Frontend
|
from beaver_gateway.frontends.base import Frontend
|
||||||
from beaver_gateway.mcp.internal_app import ALL_NAMESPACE
|
from beaver_gateway.mcp.internal_app import ALL_NAMESPACE
|
||||||
|
|
||||||
@@ -94,76 +94,31 @@ __all__ = ["McpServerFrontend"]
|
|||||||
class McpServerFrontend(Frontend):
|
class McpServerFrontend(Frontend):
|
||||||
"""Auth + audit + reverse-proxy in front of the internal MCP aggregator."""
|
"""Auth + audit + reverse-proxy in front of the internal MCP aggregator."""
|
||||||
|
|
||||||
def __init__(
|
path = "/mcp"
|
||||||
self,
|
|
||||||
*,
|
def __init__(self) -> None:
|
||||||
host: str = "0.0.0.0", # noqa: S104
|
|
||||||
port: int = 8001,
|
|
||||||
public_base_url: str | None = None,
|
|
||||||
) -> None:
|
|
||||||
self.host = host
|
|
||||||
self.port = port
|
|
||||||
# External URL prefix the reverse proxy uses to reach this
|
|
||||||
# frontend. Internal routes mount namespaces at the port root
|
|
||||||
# (``/<ns>/`` and ``/all/``) — Caddy / nginx / Cloudflare in
|
|
||||||
# front decides what external prefix they sit under. Typical
|
|
||||||
# symmetric setup:
|
|
||||||
#
|
|
||||||
# Caddy: handle_path /mcp/* { reverse_proxy localhost:8001 }
|
|
||||||
# config: public_base_url -> https://api.example.com/mcp
|
|
||||||
# dashboard advertises: https://api.example.com/mcp/<ns>/
|
|
||||||
#
|
|
||||||
# The frontend's ``/<ns>/`` segment gets appended verbatim. Set
|
|
||||||
# it to whatever matches your proxy. ``None`` means "advertise
|
|
||||||
# raw ``host:port`` derived from the inbound request" (dev /
|
|
||||||
# no proxy).
|
|
||||||
self.public_base_url = public_base_url.rstrip("/") if public_base_url else None
|
|
||||||
self._runtime: GatewayRuntime | None = None
|
self._runtime: GatewayRuntime | None = None
|
||||||
self._app: Starlette | None = None
|
self._app: Starlette | None = None
|
||||||
# Single shared aiohttp session, opened once when uvicorn starts
|
|
||||||
# the Starlette lifespan. Reused across every proxied request —
|
|
||||||
# MCP clients (esp. claude.ai) reconnect frequently, and a fresh
|
|
||||||
# ClientSession per call would cost a TCP handshake every time.
|
|
||||||
self._http: aiohttp.ClientSession | None = None
|
self._http: aiohttp.ClientSession | None = None
|
||||||
|
|
||||||
def configure(self, runtime: GatewayRuntime) -> None:
|
def configure(self, runtime: GatewayRuntime) -> None:
|
||||||
self._runtime = runtime
|
self._runtime = runtime
|
||||||
self._app = self._build_app(runtime)
|
self._app = self._build_app(runtime)
|
||||||
|
|
||||||
|
def app(self) -> Starlette | None:
|
||||||
|
return self._app
|
||||||
|
|
||||||
async def serve(self) -> None:
|
async def serve(self) -> None:
|
||||||
import uvicorn
|
self._http = aiohttp.ClientSession(
|
||||||
|
timeout=aiohttp.ClientTimeout(total=None, sock_read=600)
|
||||||
if self._app is None:
|
|
||||||
msg = "configure() must be called before serve()"
|
|
||||||
raise RuntimeError(msg)
|
|
||||||
|
|
||||||
config = uvicorn.Config(
|
|
||||||
self._app, host=self.host, port=self.port, log_level="info"
|
|
||||||
)
|
)
|
||||||
server = uvicorn.Server(config)
|
try:
|
||||||
await server.serve()
|
await asyncio.Event().wait()
|
||||||
|
finally:
|
||||||
|
await self._http.close()
|
||||||
|
self._http = None
|
||||||
|
|
||||||
def _build_app(self, runtime: GatewayRuntime) -> Starlette: # noqa: ARG002
|
def _build_app(self, runtime: GatewayRuntime) -> Starlette: # noqa: ARG002
|
||||||
# The Starlette lifespan owns the shared aiohttp session: opened
|
|
||||||
# at startup, closed at shutdown so we don't leak sockets when
|
|
||||||
# uvicorn is restarted under the same process. Starlette wants a
|
|
||||||
# plain context manager, not an async-generator function — we
|
|
||||||
# decorate to match.
|
|
||||||
@asynccontextmanager
|
|
||||||
async def lifespan(_app: Starlette) -> AsyncIterator[None]:
|
|
||||||
self._http = aiohttp.ClientSession(
|
|
||||||
# Long sock_read because MCP tool calls can take a while
|
|
||||||
# (a claude tool over HTTP can easily stretch beyond 30s
|
|
||||||
# on a real tool).
|
|
||||||
timeout=aiohttp.ClientTimeout(total=None, sock_read=600)
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
yield
|
|
||||||
finally:
|
|
||||||
if self._http is not None:
|
|
||||||
await self._http.close()
|
|
||||||
self._http = None
|
|
||||||
|
|
||||||
routes = [
|
routes = [
|
||||||
Route("/", self._discovery, methods=["GET"]),
|
Route("/", self._discovery, methods=["GET"]),
|
||||||
Route("/healthz", self._healthz, methods=["GET"]),
|
Route("/healthz", self._healthz, methods=["GET"]),
|
||||||
@@ -186,7 +141,7 @@ class McpServerFrontend(Frontend):
|
|||||||
methods=["GET", "POST", "DELETE", "OPTIONS"],
|
methods=["GET", "POST", "DELETE", "OPTIONS"],
|
||||||
),
|
),
|
||||||
]
|
]
|
||||||
return Starlette(routes=routes, lifespan=lifespan)
|
return Starlette(routes=routes)
|
||||||
|
|
||||||
async def _healthz(self, _request: Request) -> JSONResponse:
|
async def _healthz(self, _request: Request) -> JSONResponse:
|
||||||
return JSONResponse({"status": "ok"})
|
return JSONResponse({"status": "ok"})
|
||||||
@@ -197,13 +152,7 @@ class McpServerFrontend(Frontend):
|
|||||||
if err is not None:
|
if err is not None:
|
||||||
return err
|
return err
|
||||||
assert token_name is not None # noqa: S101 — narrow for ty
|
assert token_name is not None # noqa: S101 — narrow for ty
|
||||||
# ``public_base_url`` wins if configured — it's the operator's
|
base = external_base(request, runtime)
|
||||||
# explicit statement of "this is the URL my reverse proxy puts
|
|
||||||
# in front of me". Otherwise: use the request's own scheme+host
|
|
||||||
# so snippets work behind generic reverse proxies / tunnels;
|
|
||||||
# and fall back to the configured host:port if the client
|
|
||||||
# didn't send Host (curl --raw).
|
|
||||||
base = self.public_base_url or _external_base_url(request, self.host, self.port)
|
|
||||||
html = _render_discovery_page(
|
html = _render_discovery_page(
|
||||||
base_url=base, namespaces=list(runtime.mcps), actor=token_name
|
base_url=base, namespaces=list(runtime.mcps), actor=token_name
|
||||||
)
|
)
|
||||||
@@ -240,7 +189,6 @@ class McpServerFrontend(Frontend):
|
|||||||
)
|
)
|
||||||
|
|
||||||
if self._http is None:
|
if self._http is None:
|
||||||
# Lifespan hasn't run yet (shouldn't happen with uvicorn).
|
|
||||||
return JSONResponse({"error": "frontend not ready"}, status_code=503)
|
return JSONResponse({"error": "frontend not ready"}, status_code=503)
|
||||||
|
|
||||||
return await _reverse_proxy(
|
return await _reverse_proxy(
|
||||||
@@ -337,14 +285,6 @@ def _join_subpath(base_url: str, subpath: str) -> str:
|
|||||||
return base_url
|
return base_url
|
||||||
|
|
||||||
|
|
||||||
def _external_base_url(request: Request, fallback_host: str, fallback_port: int) -> str:
|
|
||||||
host = request.headers.get("host")
|
|
||||||
if host:
|
|
||||||
scheme = request.headers.get("x-forwarded-proto", request.url.scheme)
|
|
||||||
return f"{scheme}://{host}"
|
|
||||||
return f"http://{fallback_host}:{fallback_port}"
|
|
||||||
|
|
||||||
|
|
||||||
async def _reverse_proxy(
|
async def _reverse_proxy(
|
||||||
*,
|
*,
|
||||||
client: aiohttp.ClientSession,
|
client: aiohttp.ClientSession,
|
||||||
|
|||||||
@@ -0,0 +1,54 @@
|
|||||||
|
"""The one ASGI app the gateway listens with.
|
||||||
|
|
||||||
|
Every HTTP frontend is mounted under its ``path`` (``/anthropic``,
|
||||||
|
``/mcp``, ``/md``, ``/api``, ``/admin``); ``/healthz`` answers for the
|
||||||
|
whole process and ``/`` redirects to the landing frontend (the admin
|
||||||
|
console) when one is configured.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
from starlette.applications import Starlette
|
||||||
|
from starlette.responses import JSONResponse, RedirectResponse
|
||||||
|
from starlette.routing import Mount, Route
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Iterable, Mapping
|
||||||
|
|
||||||
|
from starlette.requests import Request
|
||||||
|
from starlette.types import ASGIApp
|
||||||
|
|
||||||
|
from beaver_gateway.frontends.base import Frontend
|
||||||
|
|
||||||
|
__all__ = ["build_root_app"]
|
||||||
|
|
||||||
|
|
||||||
|
def build_root_app(
|
||||||
|
frontends: Iterable[Frontend], *, extra: Mapping[str, ASGIApp] | None = None
|
||||||
|
) -> Starlette:
|
||||||
|
mounted = [fe for fe in frontends if fe.path and fe.app() is not None]
|
||||||
|
landing = next((fe for fe in mounted if fe.landing), None)
|
||||||
|
paths = [*(fe.path for fe in mounted), *(extra or {})]
|
||||||
|
|
||||||
|
async def healthz(_request: Request) -> JSONResponse:
|
||||||
|
return JSONResponse({"status": "ok", "frontends": paths})
|
||||||
|
|
||||||
|
async def index(_request: Request) -> JSONResponse | RedirectResponse:
|
||||||
|
if landing is not None:
|
||||||
|
return RedirectResponse(f"{landing.path}/", status_code=307)
|
||||||
|
return JSONResponse({"frontends": paths})
|
||||||
|
|
||||||
|
routes: list[Route | Mount] = [
|
||||||
|
Route("/healthz", healthz, methods=["GET"]),
|
||||||
|
Route("/", index, methods=["GET"]),
|
||||||
|
]
|
||||||
|
for fe in mounted:
|
||||||
|
app = fe.app()
|
||||||
|
assert app is not None # noqa: S101 - filtered above; narrows for ty
|
||||||
|
assert fe.path is not None # noqa: S101
|
||||||
|
routes.append(Mount(fe.path, app=app, name=fe.name or fe.path.strip("/")))
|
||||||
|
for path, app in (extra or {}).items():
|
||||||
|
routes.append(Mount(path, app=app, name=path.strip("/")))
|
||||||
|
return Starlette(routes=routes)
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
"""Telegram frontend (§3.8): General = master, topic = branch, drafts, inbox/outbox."""
|
||||||
|
|
||||||
|
from beaver_gateway.frontends.telegram.frontend import Attachments, TelegramFrontend
|
||||||
|
|
||||||
|
__all__ = ["Attachments", "TelegramFrontend"]
|
||||||
@@ -0,0 +1,138 @@
|
|||||||
|
"""One ``sendMessageDraft`` stream per running turn (§3.8).
|
||||||
|
|
||||||
|
The client folds a draft into the message that follows only when their
|
||||||
|
texts are identical - so the last push is the final text itself, rendered
|
||||||
|
exactly as the outbox will send it, with no status line.
|
||||||
|
|
||||||
|
A draft is ephemeral and lives 30 s, Telegram throttles edits to about one
|
||||||
|
per second per chat, and thinking or a tool call would otherwise look like a
|
||||||
|
hang - so the draft opens with a status line straight away, is refreshed on
|
||||||
|
a timer rather than on every delta, and is kept alive while nothing changes.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import contextlib
|
||||||
|
import logging
|
||||||
|
import time
|
||||||
|
import zlib
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
from aiogram.exceptions import TelegramAPIError, TelegramBadRequest
|
||||||
|
|
||||||
|
from beaver_gateway.frontends.telegram.render import to_html
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from aiogram import Bot
|
||||||
|
|
||||||
|
__all__ = ["Draft"]
|
||||||
|
|
||||||
|
_log = logging.getLogger("beaver_gateway.frontends.telegram.drafts")
|
||||||
|
|
||||||
|
_TAIL = 3500
|
||||||
|
_KEEPALIVE = 20.0
|
||||||
|
|
||||||
|
|
||||||
|
class Draft:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
bot: Bot,
|
||||||
|
*,
|
||||||
|
chat_id: int,
|
||||||
|
thread_id: int | None,
|
||||||
|
turn_id: str,
|
||||||
|
interval: float = 0.7,
|
||||||
|
status: str = "⏳ думаю",
|
||||||
|
) -> None:
|
||||||
|
self._bot = bot
|
||||||
|
self._chat_id = chat_id
|
||||||
|
self._thread_id = thread_id
|
||||||
|
self._draft_id = (zlib.crc32(turn_id.encode()) & 0x7FFFFFFF) or 1
|
||||||
|
self._interval = interval
|
||||||
|
self.status = status
|
||||||
|
self.text = ""
|
||||||
|
self._dirty = True
|
||||||
|
self._broken = False
|
||||||
|
self._last_sent = 0.0
|
||||||
|
self._task: asyncio.Task[None] | None = None
|
||||||
|
self._inflight: asyncio.Future[None] | None = None
|
||||||
|
|
||||||
|
def start(self) -> None:
|
||||||
|
if self._task is None:
|
||||||
|
self._task = asyncio.create_task(self._run())
|
||||||
|
|
||||||
|
def set_status(self, status: str) -> None:
|
||||||
|
if status != self.status:
|
||||||
|
self.status = status
|
||||||
|
self._dirty = True
|
||||||
|
|
||||||
|
def append(self, text: str) -> None:
|
||||||
|
if text:
|
||||||
|
self.text += text
|
||||||
|
self._dirty = True
|
||||||
|
|
||||||
|
async def stop(self) -> None:
|
||||||
|
"""Stop pushing and wait for the push in flight.
|
||||||
|
|
||||||
|
The final ``sendMessage`` must reach Telegram after the last draft,
|
||||||
|
or the draft lands on top of the message and lingers for its 30 s.
|
||||||
|
"""
|
||||||
|
if self._task is not None:
|
||||||
|
self._task.cancel()
|
||||||
|
await asyncio.gather(self._task, return_exceptions=True)
|
||||||
|
self._task = None
|
||||||
|
if self._inflight is not None:
|
||||||
|
with contextlib.suppress(Exception):
|
||||||
|
await self._inflight
|
||||||
|
self._inflight = None
|
||||||
|
|
||||||
|
async def finish(self, text: str) -> None:
|
||||||
|
"""Last push: the final text verbatim, so the message replaces the draft."""
|
||||||
|
await self.stop()
|
||||||
|
if self._broken or not text.strip():
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
try:
|
||||||
|
await self._send(to_html(text), "HTML")
|
||||||
|
except TelegramBadRequest:
|
||||||
|
await self._send(text, None)
|
||||||
|
except TelegramAPIError as exc:
|
||||||
|
_log.warning(
|
||||||
|
"final draft to %s/%s failed: %s", self._chat_id, self._thread_id, exc
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _run(self) -> None:
|
||||||
|
while not self._broken:
|
||||||
|
if self._dirty or time.monotonic() - self._last_sent > _KEEPALIVE:
|
||||||
|
self._inflight = asyncio.ensure_future(self._push())
|
||||||
|
await asyncio.shield(self._inflight)
|
||||||
|
await asyncio.sleep(self._interval)
|
||||||
|
|
||||||
|
async def _push(self) -> None:
|
||||||
|
self._dirty = False
|
||||||
|
self._last_sent = time.monotonic()
|
||||||
|
text = self._render()
|
||||||
|
try:
|
||||||
|
try:
|
||||||
|
await self._send(to_html(text), "HTML")
|
||||||
|
except TelegramBadRequest:
|
||||||
|
await self._send(text, None)
|
||||||
|
except TelegramAPIError as exc:
|
||||||
|
self._broken = True
|
||||||
|
_log.warning(
|
||||||
|
"draft to %s/%s stopped: %s", self._chat_id, self._thread_id, exc
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _send(self, text: str, parse_mode: str | None) -> None:
|
||||||
|
await self._bot.send_message_draft(
|
||||||
|
chat_id=self._chat_id,
|
||||||
|
draft_id=self._draft_id,
|
||||||
|
message_thread_id=self._thread_id,
|
||||||
|
text=text,
|
||||||
|
parse_mode=parse_mode,
|
||||||
|
)
|
||||||
|
|
||||||
|
def _render(self) -> str:
|
||||||
|
tail = self.text[-_TAIL:]
|
||||||
|
return f"{self.status}\n\n{tail}" if tail.strip() else self.status
|
||||||
@@ -0,0 +1,898 @@
|
|||||||
|
"""``TelegramFrontend`` - the private chat with the bot as the window (§3.8).
|
||||||
|
|
||||||
|
A private chat with topics has no General: the gateway makes one topic for
|
||||||
|
the master (``master_topic``) and rebinds it to every new master; any other
|
||||||
|
topic is a branch. The user makes a topic and the
|
||||||
|
first message in it spawns the branch (``seed=morning``); a message into a
|
||||||
|
topic whose branch is merged or closed spawns a new branch on the same
|
||||||
|
topic. Replies stream as drafts and land through the outbox; turns that
|
||||||
|
came from other windows are mirrored with a marker; ``origin=system`` is
|
||||||
|
never shown. ``AskUserQuestion`` becomes inline buttons (§3.7).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import contextlib
|
||||||
|
import html
|
||||||
|
import logging
|
||||||
|
import tempfile
|
||||||
|
import time
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import TYPE_CHECKING, Any, Literal, cast
|
||||||
|
|
||||||
|
from aiogram import Bot
|
||||||
|
from aiogram.client.default import DefaultBotProperties
|
||||||
|
from aiogram.exceptions import TelegramAPIError
|
||||||
|
from aiogram.types import (
|
||||||
|
BotCommand,
|
||||||
|
CallbackQuery,
|
||||||
|
InlineKeyboardButton,
|
||||||
|
InlineKeyboardMarkup,
|
||||||
|
Message,
|
||||||
|
ReactionTypeEmoji,
|
||||||
|
Update,
|
||||||
|
)
|
||||||
|
|
||||||
|
from beaver_gateway.frontends.base import Frontend
|
||||||
|
from beaver_gateway.frontends.telegram.drafts import Draft
|
||||||
|
from beaver_gateway.frontends.telegram.inbox import Inbox
|
||||||
|
from beaver_gateway.frontends.telegram.outbox import Outbox
|
||||||
|
from beaver_gateway.frontends.telegram.render import chunks, status_label
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from beaver_gateway.core.bus import Event, EventBus
|
||||||
|
from beaver_gateway.core.conversations import Conversations
|
||||||
|
from beaver_gateway.core.kinds import Kind
|
||||||
|
from beaver_gateway.frontends.base import GatewayRuntime
|
||||||
|
from beaver_gateway.storage.models import Conversation, ConversationBinding
|
||||||
|
|
||||||
|
__all__ = ["FRONTEND", "Attachments", "TelegramFrontend"]
|
||||||
|
|
||||||
|
_log = logging.getLogger("beaver_gateway.frontends.telegram")
|
||||||
|
|
||||||
|
FRONTEND = "telegram"
|
||||||
|
_DONE = "done"
|
||||||
|
_COMMAND_HELP = (
|
||||||
|
("merge", "слить ветку в мастер"),
|
||||||
|
("new", "новая ветка: в General - новый топик, в топике - заново на нём"),
|
||||||
|
("chat", "открыть глубокий чат в vault: /chat тема"),
|
||||||
|
("status", "что с этим разговором"), # noqa: RUF001
|
||||||
|
("help", "команды"),
|
||||||
|
)
|
||||||
|
_COMMANDS = (*(c for c, _ in _COMMAND_HELP), "start")
|
||||||
|
_HELP = "General - мастер, любой другой топик - ветка.\n" + "\n".join(
|
||||||
|
f"/{c} - {d}" for c, d in _COMMAND_HELP
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class Attachments:
|
||||||
|
"""Where files from Telegram go.
|
||||||
|
|
||||||
|
``ephemeral`` - under the gateway's data dir, swept after ``keep_days``;
|
||||||
|
``vault`` - into a directory the agent owns. Pulling what is worth
|
||||||
|
keeping into the vault is the agent's job either way.
|
||||||
|
"""
|
||||||
|
|
||||||
|
mode: Literal["ephemeral", "vault"] = "ephemeral"
|
||||||
|
dir: Path | None = None
|
||||||
|
keep_days: int = 7
|
||||||
|
|
||||||
|
@property
|
||||||
|
def root(self) -> Path:
|
||||||
|
if self.dir is not None:
|
||||||
|
return self.dir
|
||||||
|
if self.mode == "vault":
|
||||||
|
msg = "Attachments(mode='vault') needs `dir`"
|
||||||
|
raise ValueError(msg)
|
||||||
|
return Path(tempfile.gettempdir()) / "beaver-attachments"
|
||||||
|
|
||||||
|
|
||||||
|
EPHEMERAL = Attachments()
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class _Ask:
|
||||||
|
conversation_id: str
|
||||||
|
chat_id: int
|
||||||
|
thread_id: int | None
|
||||||
|
questions: list[dict[str, Any]]
|
||||||
|
messages: list[int] = field(default_factory=list)
|
||||||
|
picked: dict[int, list[str]] = field(default_factory=dict)
|
||||||
|
done: set[int] = field(default_factory=set)
|
||||||
|
|
||||||
|
|
||||||
|
class TelegramFrontend(Frontend):
|
||||||
|
name = FRONTEND
|
||||||
|
kinds = ("master", "branch")
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
token: str,
|
||||||
|
user_id: int,
|
||||||
|
master_agent: str | None = None,
|
||||||
|
branch_agent: str | None = None,
|
||||||
|
chat_id: int | None = None,
|
||||||
|
attachments: Attachments = EPHEMERAL,
|
||||||
|
draft_interval: float = 0.7,
|
||||||
|
master_topic: str = "🦫 General",
|
||||||
|
queued_reaction: str = "👀",
|
||||||
|
poll_timeout: int = 30,
|
||||||
|
outbox_backoff: float = 2.0,
|
||||||
|
) -> None:
|
||||||
|
self._token = token
|
||||||
|
self.user_id = user_id
|
||||||
|
self.chat_id = chat_id if chat_id is not None else user_id
|
||||||
|
self.master_agent = master_agent
|
||||||
|
self.branch_agent = branch_agent
|
||||||
|
self.attachments = attachments
|
||||||
|
self.master_topic = master_topic
|
||||||
|
self.draft_interval = draft_interval
|
||||||
|
self.queued_reaction = queued_reaction
|
||||||
|
self.poll_timeout = poll_timeout
|
||||||
|
self.outbox_backoff = outbox_backoff
|
||||||
|
self._runtime: GatewayRuntime | None = None
|
||||||
|
self._bot: Bot | None = None
|
||||||
|
self._inbox: Inbox | None = None
|
||||||
|
self._outbox: Outbox | None = None
|
||||||
|
self._targets: dict[str, tuple[int, int | None] | None] = {}
|
||||||
|
self._master_window: str | None = None
|
||||||
|
self._topic_names: dict[int, str] = {}
|
||||||
|
self._drafts: dict[str, Draft] = {}
|
||||||
|
self._asks: dict[str, _Ask] = {}
|
||||||
|
self._reactions: dict[int, tuple[int, int]] = {}
|
||||||
|
self._tasks: set[asyncio.Task[None]] = set()
|
||||||
|
|
||||||
|
# ---- Frontend --------------------------------------------------------
|
||||||
|
|
||||||
|
def agent_for(self, kind: Kind) -> str | None:
|
||||||
|
return {"master": self.master_agent, "branch": self.branch_agent}.get(kind)
|
||||||
|
|
||||||
|
def configure(self, runtime: GatewayRuntime) -> None:
|
||||||
|
if runtime.conversations is None or runtime.bus is None:
|
||||||
|
msg = "TelegramFrontend needs runtime.conversations and runtime.bus"
|
||||||
|
raise RuntimeError(msg)
|
||||||
|
self._runtime = runtime
|
||||||
|
if self._bot is None:
|
||||||
|
self._bot = Bot(self._token, default=DefaultBotProperties(parse_mode=None))
|
||||||
|
self._inbox = Inbox(
|
||||||
|
runtime.db, self._bot, handler=self._handle, poll_timeout=self.poll_timeout
|
||||||
|
)
|
||||||
|
self._outbox = Outbox(
|
||||||
|
runtime.db, self._bot, bus=runtime.bus, backoff=self.outbox_backoff
|
||||||
|
)
|
||||||
|
|
||||||
|
async def serve(self) -> None:
|
||||||
|
me = await self.bot.get_me()
|
||||||
|
_log.info(
|
||||||
|
"telegram: @%s, user %s, chat %s, topics=%s",
|
||||||
|
me.username,
|
||||||
|
self.user_id,
|
||||||
|
self.chat_id,
|
||||||
|
getattr(me, "has_topics_enabled", None),
|
||||||
|
)
|
||||||
|
self._sweep_attachments()
|
||||||
|
await self.bot.set_my_commands(
|
||||||
|
[BotCommand(command=c, description=d) for c, d in _COMMAND_HELP]
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
async with asyncio.TaskGroup() as tg:
|
||||||
|
tg.create_task(self.inbox.run())
|
||||||
|
tg.create_task(self.outbox.run())
|
||||||
|
tg.create_task(self._events())
|
||||||
|
finally:
|
||||||
|
for draft in list(self._drafts.values()):
|
||||||
|
await draft.stop()
|
||||||
|
await self.bot.session.close()
|
||||||
|
|
||||||
|
async def materialize(self, conv: Conversation) -> ConversationBinding | None:
|
||||||
|
if conv.kind == "master":
|
||||||
|
return await self.conversations.bind(
|
||||||
|
conv, frontend=FRONTEND, external_id=await self._master_ext()
|
||||||
|
)
|
||||||
|
if conv.kind != "branch":
|
||||||
|
return None
|
||||||
|
topic = await self.bot.create_forum_topic(
|
||||||
|
self.chat_id, name=(conv.title or "ветка")[:128]
|
||||||
|
)
|
||||||
|
self._topic_names[topic.message_thread_id] = topic.name
|
||||||
|
return await self.conversations.bind(
|
||||||
|
conv, frontend=FRONTEND, external_id=self._ext(topic.message_thread_id)
|
||||||
|
)
|
||||||
|
|
||||||
|
async def mark_closed(self, conv: Conversation) -> bool:
|
||||||
|
return await self.mark_topic(conv)
|
||||||
|
|
||||||
|
async def mark_topic(self, conv: Conversation, prefix: str = "✅ ") -> bool:
|
||||||
|
"""Rename the topic; closeForumTopic does not exist in private chats."""
|
||||||
|
target = await self._target_of(conv)
|
||||||
|
if target is None or target[1] is None:
|
||||||
|
return False
|
||||||
|
name = self._topic_names.get(target[1]) or conv.title or "ветка"
|
||||||
|
if name.startswith(prefix):
|
||||||
|
return True
|
||||||
|
await self.bot.edit_forum_topic(
|
||||||
|
target[0], target[1], name=f"{prefix}{name}"[:128]
|
||||||
|
)
|
||||||
|
self._topic_names[target[1]] = f"{prefix}{name}"
|
||||||
|
return True
|
||||||
|
|
||||||
|
# ---- plumbing --------------------------------------------------------
|
||||||
|
|
||||||
|
@property
|
||||||
|
def bot(self) -> Bot:
|
||||||
|
if self._bot is None:
|
||||||
|
msg = "configure() must be called before use"
|
||||||
|
raise RuntimeError(msg)
|
||||||
|
return self._bot
|
||||||
|
|
||||||
|
@property
|
||||||
|
def inbox(self) -> Inbox:
|
||||||
|
return cast("Inbox", self._inbox)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def outbox(self) -> Outbox:
|
||||||
|
return cast("Outbox", self._outbox)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def conversations(self) -> Conversations:
|
||||||
|
return cast(
|
||||||
|
"Conversations", cast("GatewayRuntime", self._runtime).conversations
|
||||||
|
)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def bus(self) -> EventBus:
|
||||||
|
return cast("EventBus", cast("GatewayRuntime", self._runtime).bus)
|
||||||
|
|
||||||
|
def _ext(self, thread_id: int | None) -> str:
|
||||||
|
return f"{self.chat_id}/{thread_id}" if thread_id else str(self.chat_id)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _parse_ext(ext: str) -> tuple[int, int | None]:
|
||||||
|
chat, _, thread = ext.partition("/")
|
||||||
|
return int(chat), int(thread) if thread else None
|
||||||
|
|
||||||
|
async def _target_of(self, conv: Conversation) -> tuple[int, int | None] | None:
|
||||||
|
key = conv.external_id
|
||||||
|
if key not in self._targets:
|
||||||
|
bound = next(
|
||||||
|
(
|
||||||
|
b
|
||||||
|
for b in await self.conversations.bindings(conv)
|
||||||
|
if b.frontend == FRONTEND and b.visible
|
||||||
|
),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
self._targets[key] = self._parse_ext(bound.external_id) if bound else None
|
||||||
|
return self._targets[key]
|
||||||
|
|
||||||
|
async def _deliver(
|
||||||
|
self,
|
||||||
|
conv: Conversation,
|
||||||
|
text: str,
|
||||||
|
*,
|
||||||
|
turn_id: str | None = None,
|
||||||
|
key: str | None = None,
|
||||||
|
) -> None:
|
||||||
|
target = await self._target_of(conv)
|
||||||
|
if target is None or not text.strip():
|
||||||
|
return
|
||||||
|
await self.outbox.enqueue(
|
||||||
|
chat_id=target[0],
|
||||||
|
thread_id=target[1],
|
||||||
|
text=text,
|
||||||
|
conversation_id=conv.id,
|
||||||
|
turn_id=turn_id,
|
||||||
|
dedupe_key=key,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _master_ext(self) -> str:
|
||||||
|
"""The master's window.
|
||||||
|
|
||||||
|
General in a forum group, our own topic in a private chat (Telegram
|
||||||
|
has no General there). Created once, then found through whatever
|
||||||
|
master used it last.
|
||||||
|
"""
|
||||||
|
if self._master_window is not None:
|
||||||
|
return self._master_window
|
||||||
|
last = await self.conversations.last_binding(frontend=FRONTEND, kind="master")
|
||||||
|
if last is not None:
|
||||||
|
self._master_window = last.external_id
|
||||||
|
elif self.chat_id < 0:
|
||||||
|
self._master_window = self._ext(None)
|
||||||
|
else:
|
||||||
|
topic = await self.bot.create_forum_topic(
|
||||||
|
self.chat_id, name=self.master_topic[:128]
|
||||||
|
)
|
||||||
|
self._topic_names[topic.message_thread_id] = topic.name
|
||||||
|
self._master_window = self._ext(topic.message_thread_id)
|
||||||
|
return self._master_window
|
||||||
|
|
||||||
|
async def _master(self, text: str | None = None) -> tuple[Conversation, bool]:
|
||||||
|
"""The open master behind its window, spawning one when there is none.
|
||||||
|
|
||||||
|
``text`` rides with the seed of a fresh master; the second value says
|
||||||
|
whether it was consumed that way.
|
||||||
|
"""
|
||||||
|
ext = await self._master_ext()
|
||||||
|
conv = await self.conversations.find_bound(frontend=FRONTEND, external_id=ext)
|
||||||
|
if conv is not None and conv.status == "open":
|
||||||
|
return conv, False
|
||||||
|
masters = await self.conversations.find(kind="master", status="open", limit=1)
|
||||||
|
if masters:
|
||||||
|
await self.conversations.bind(
|
||||||
|
masters[0], frontend=FRONTEND, external_id=ext
|
||||||
|
)
|
||||||
|
return masters[0], False
|
||||||
|
conv = await self.conversations.spawn(
|
||||||
|
kind="master",
|
||||||
|
seed="clean",
|
||||||
|
text=text,
|
||||||
|
origin=FRONTEND,
|
||||||
|
binding=(FRONTEND, ext),
|
||||||
|
)
|
||||||
|
return conv, text is not None
|
||||||
|
|
||||||
|
async def _branch(
|
||||||
|
self, thread_id: int, *, title: str | None, text: str | None
|
||||||
|
) -> Conversation:
|
||||||
|
master, _ = await self._master()
|
||||||
|
return await self.conversations.spawn(
|
||||||
|
kind="branch",
|
||||||
|
seed="morning",
|
||||||
|
parent=master,
|
||||||
|
title=(title or self._topic_names.get(thread_id) or "ветка")[:128],
|
||||||
|
text=text,
|
||||||
|
origin=FRONTEND,
|
||||||
|
binding=(FRONTEND, self._ext(thread_id)),
|
||||||
|
)
|
||||||
|
|
||||||
|
# ---- inbox -----------------------------------------------------------
|
||||||
|
|
||||||
|
async def _handle(self, update: Update) -> None:
|
||||||
|
if update.message is not None:
|
||||||
|
await self._on_message(update.message)
|
||||||
|
elif update.callback_query is not None:
|
||||||
|
await self._on_callback(update.callback_query)
|
||||||
|
|
||||||
|
async def _on_message(self, message: Message) -> None:
|
||||||
|
created = message.forum_topic_created
|
||||||
|
if (
|
||||||
|
created is not None
|
||||||
|
and message.message_thread_id
|
||||||
|
and not created.is_name_implicit
|
||||||
|
):
|
||||||
|
self._topic_names[message.message_thread_id] = created.name
|
||||||
|
if message.from_user is None or message.from_user.id != self.user_id:
|
||||||
|
_log.warning(
|
||||||
|
"ignoring message from %s", getattr(message.from_user, "id", None)
|
||||||
|
)
|
||||||
|
return
|
||||||
|
if message.chat.id != self.chat_id:
|
||||||
|
return
|
||||||
|
in_topic = message.is_topic_message or message.forum_topic_created is not None
|
||||||
|
thread_id = message.message_thread_id if in_topic else None
|
||||||
|
is_master = (
|
||||||
|
thread_id is None or self._ext(thread_id) == await self._master_ext()
|
||||||
|
)
|
||||||
|
if message.forum_topic_created is not None:
|
||||||
|
return
|
||||||
|
if message.forum_topic_edited is not None and thread_id is not None:
|
||||||
|
if message.forum_topic_edited.name:
|
||||||
|
self._topic_names[thread_id] = message.forum_topic_edited.name
|
||||||
|
return
|
||||||
|
text = (message.text or message.caption or "").strip()
|
||||||
|
attachment = await self._save_attachment(message, thread_id)
|
||||||
|
if attachment:
|
||||||
|
text = f"{text}\n\n{attachment}".strip()
|
||||||
|
if not text:
|
||||||
|
return
|
||||||
|
if message.text and message.text.startswith("/"):
|
||||||
|
command, _, args = message.text[1:].partition(" ")
|
||||||
|
command = command.partition("@")[0].lower()
|
||||||
|
if command in _COMMANDS:
|
||||||
|
await self._command(command, args.strip(), message, thread_id)
|
||||||
|
return
|
||||||
|
if is_master:
|
||||||
|
conv, consumed = await self._master(text)
|
||||||
|
if consumed:
|
||||||
|
return
|
||||||
|
else:
|
||||||
|
conv = await self._live(thread_id)
|
||||||
|
if conv is None:
|
||||||
|
title = self._topic_names.get(thread_id) or self._title_from(text)
|
||||||
|
await self._branch(thread_id, title=title, text=text)
|
||||||
|
return
|
||||||
|
pending = self.conversations.pending_question(conv.external_id)
|
||||||
|
if pending is not None and self.conversations.answer(pending[0], text):
|
||||||
|
await self._close_ask(pending[0], f"✍️ {text}")
|
||||||
|
return
|
||||||
|
item = await self.conversations.post(conv, text, origin=FRONTEND)
|
||||||
|
if conv.running_turn or conv.pending_question:
|
||||||
|
await self._react(message, item.id)
|
||||||
|
|
||||||
|
async def _live(self, thread_id: int) -> Conversation | None:
|
||||||
|
conv = await self.conversations.find_bound(
|
||||||
|
frontend=FRONTEND, external_id=self._ext(thread_id)
|
||||||
|
)
|
||||||
|
return conv if conv is not None and conv.status == "open" else None
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _title_from(text: str) -> str:
|
||||||
|
line = text.strip().splitlines()[0]
|
||||||
|
return line if len(line) <= 60 else line[:57] + "…"
|
||||||
|
|
||||||
|
async def _react(self, message: Message, item_id: int | None) -> None:
|
||||||
|
if not self.queued_reaction or item_id is None:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
await self.bot.set_message_reaction(
|
||||||
|
message.chat.id,
|
||||||
|
message.message_id,
|
||||||
|
reaction=[ReactionTypeEmoji(emoji=self.queued_reaction)],
|
||||||
|
)
|
||||||
|
except TelegramAPIError as exc:
|
||||||
|
_log.debug("reaction failed: %s", exc)
|
||||||
|
return
|
||||||
|
self._reactions[item_id] = (message.chat.id, message.message_id)
|
||||||
|
|
||||||
|
async def _unreact(self, item_id: int | None) -> None:
|
||||||
|
target = self._reactions.pop(cast("int", item_id), None) if item_id else None
|
||||||
|
if target is None:
|
||||||
|
return
|
||||||
|
with contextlib.suppress(TelegramAPIError):
|
||||||
|
await self.bot.set_message_reaction(target[0], target[1], reaction=[])
|
||||||
|
|
||||||
|
async def _save_attachment(
|
||||||
|
self, message: Message, thread_id: int | None
|
||||||
|
) -> str | None:
|
||||||
|
file_id: str | None = None
|
||||||
|
name: str | None = None
|
||||||
|
kind = ""
|
||||||
|
size: int | None = None
|
||||||
|
if message.photo:
|
||||||
|
photo = message.photo[-1]
|
||||||
|
file_id, kind, size = photo.file_id, "фото", photo.file_size
|
||||||
|
name = f"{photo.file_unique_id}.jpg"
|
||||||
|
elif message.document:
|
||||||
|
doc = message.document
|
||||||
|
file_id, kind, size = doc.file_id, "файл", doc.file_size
|
||||||
|
name = doc.file_name or f"{doc.file_unique_id}.bin"
|
||||||
|
elif message.voice:
|
||||||
|
file_id, kind, size = (
|
||||||
|
message.voice.file_id,
|
||||||
|
"голосовое",
|
||||||
|
message.voice.file_size,
|
||||||
|
)
|
||||||
|
name = f"{message.voice.file_unique_id}.ogg"
|
||||||
|
elif message.audio:
|
||||||
|
file_id, kind, size = (
|
||||||
|
message.audio.file_id,
|
||||||
|
"аудио",
|
||||||
|
message.audio.file_size,
|
||||||
|
)
|
||||||
|
name = message.audio.file_name or f"{message.audio.file_unique_id}.mp3"
|
||||||
|
elif message.video:
|
||||||
|
file_id, kind, size = (
|
||||||
|
message.video.file_id,
|
||||||
|
"видео",
|
||||||
|
message.video.file_size,
|
||||||
|
)
|
||||||
|
name = message.video.file_name or f"{message.video.file_unique_id}.mp4"
|
||||||
|
elif message.video_note:
|
||||||
|
file_id, kind, size = (
|
||||||
|
message.video_note.file_id,
|
||||||
|
"кружок",
|
||||||
|
message.video_note.file_size,
|
||||||
|
)
|
||||||
|
name = f"{message.video_note.file_unique_id}.mp4"
|
||||||
|
if file_id is None or name is None:
|
||||||
|
return None
|
||||||
|
if size and size > 20 * 1024 * 1024:
|
||||||
|
return (
|
||||||
|
f"[вложение: {kind} {name}, {size // 1024 // 1024} МБ - "
|
||||||
|
"больше 20 МБ, Telegram не отдаёт ботам]"
|
||||||
|
)
|
||||||
|
folder = self.attachments.root / (self._ext(thread_id).replace("/", "_"))
|
||||||
|
folder.mkdir(parents=True, exist_ok=True)
|
||||||
|
with contextlib.suppress(OSError):
|
||||||
|
self.attachments.root.chmod(0o755)
|
||||||
|
folder.chmod(0o755)
|
||||||
|
path = folder / f"{int(time.time())}-{Path(name).name}"
|
||||||
|
try:
|
||||||
|
await self.bot.download(file_id, destination=path)
|
||||||
|
path.chmod(0o644)
|
||||||
|
except (TelegramAPIError, OSError) as exc:
|
||||||
|
_log.warning("attachment download failed: %s", exc)
|
||||||
|
return f"[вложение: {kind} {name} - не скачалось: {exc}]"
|
||||||
|
shown = f", {size // 1024} КБ" if size else ""
|
||||||
|
return f"[вложение: {kind} {path}{shown}]"
|
||||||
|
|
||||||
|
def _sweep_attachments(self) -> None:
|
||||||
|
if self.attachments.mode != "ephemeral":
|
||||||
|
return
|
||||||
|
root = self.attachments.root
|
||||||
|
if not root.exists():
|
||||||
|
return
|
||||||
|
cutoff = time.time() - self.attachments.keep_days * 86400
|
||||||
|
for path in root.rglob("*"):
|
||||||
|
with contextlib.suppress(OSError):
|
||||||
|
if path.is_file() and path.stat().st_mtime < cutoff:
|
||||||
|
path.unlink()
|
||||||
|
|
||||||
|
# ---- commands --------------------------------------------------------
|
||||||
|
|
||||||
|
async def _command(
|
||||||
|
self, command: str, args: str, message: Message, thread_id: int | None
|
||||||
|
) -> None:
|
||||||
|
reply = await self._run_command(command, args, thread_id)
|
||||||
|
if reply:
|
||||||
|
await self.outbox.enqueue(
|
||||||
|
chat_id=message.chat.id, thread_id=thread_id, text=reply
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _run_command(self, command: str, args: str, thread_id: int | None) -> str:
|
||||||
|
if command in ("start", "help"):
|
||||||
|
return _HELP
|
||||||
|
is_master = (
|
||||||
|
thread_id is None or self._ext(thread_id) == await self._master_ext()
|
||||||
|
)
|
||||||
|
conv = (await self._master())[0] if is_master else await self._live(thread_id)
|
||||||
|
if command == "status":
|
||||||
|
return await self._status(conv)
|
||||||
|
if command == "merge":
|
||||||
|
if conv is None or conv.kind != "branch":
|
||||||
|
return "сливать нечего: это не открытая ветка"
|
||||||
|
self._spawn_task(self._merge(conv))
|
||||||
|
return "🔀 сливаю в мастер"
|
||||||
|
if command == "new":
|
||||||
|
if is_master or thread_id is None:
|
||||||
|
child = await self.conversations.spawn(
|
||||||
|
kind="branch",
|
||||||
|
seed="morning",
|
||||||
|
parent=conv,
|
||||||
|
title=args or None,
|
||||||
|
origin=FRONTEND,
|
||||||
|
)
|
||||||
|
return f"🌿 ветка «{child.title or child.external_id}» - в новом топике"
|
||||||
|
if conv is not None:
|
||||||
|
await self.conversations.set_status(conv, "closed")
|
||||||
|
await self._branch(thread_id, title=args or None, text=None)
|
||||||
|
return "🌿 новая ветка на этом топике"
|
||||||
|
if command == "chat":
|
||||||
|
if not args:
|
||||||
|
return "/chat <тема>"
|
||||||
|
try:
|
||||||
|
deep = await self.conversations.spawn(
|
||||||
|
kind="deep", seed="clean", title=args, origin=FRONTEND
|
||||||
|
)
|
||||||
|
except (ValueError, LookupError) as exc:
|
||||||
|
return f"не вышло: {exc}"
|
||||||
|
where = next(
|
||||||
|
(
|
||||||
|
b.external_id
|
||||||
|
for b in await self.conversations.bindings(deep)
|
||||||
|
if b.visible
|
||||||
|
),
|
||||||
|
deep.external_id,
|
||||||
|
)
|
||||||
|
return f"💬 глубокий чат: {where}"
|
||||||
|
return _HELP
|
||||||
|
|
||||||
|
async def _status(self, conv: Conversation | None) -> str:
|
||||||
|
if conv is None:
|
||||||
|
return "этот топик ни к чему не привязан - напиши, и откроется ветка"
|
||||||
|
info = await self.conversations.describe(conv)
|
||||||
|
queued = sum(
|
||||||
|
1
|
||||||
|
for i in await self.conversations.queue.recent(
|
||||||
|
cast("int", conv.id), limit=20
|
||||||
|
)
|
||||||
|
if i.status == "queued"
|
||||||
|
)
|
||||||
|
lines = [
|
||||||
|
f"{conv.kind} · {conv.status} · {conv.agent_name}",
|
||||||
|
f"сессия: {'живая' if info['live'] else 'нет'}"
|
||||||
|
f" · тёрн: {'идёт' if conv.running_turn else 'нет'}"
|
||||||
|
f" · в очереди: {queued}",
|
||||||
|
]
|
||||||
|
if conv.pending_question:
|
||||||
|
lines.append("❓ ждёт ответа на вопрос")
|
||||||
|
if conv.kind == "master":
|
||||||
|
pool = self.conversations.pool
|
||||||
|
lines.append(f"пул: {len(pool)} сессий, rss {pool.rss() // (1 << 20)} МБ")
|
||||||
|
lines.append(f"outbox: {await self.outbox.pending()} в очереди")
|
||||||
|
lines.append(f"id: {conv.external_id}")
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
async def _merge(self, conv: Conversation) -> None:
|
||||||
|
try:
|
||||||
|
await self.conversations.merge(conv)
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
_log.exception("merge of %s failed", conv.external_id)
|
||||||
|
await self._deliver(conv, f"⚠️ слив не удался: {exc}")
|
||||||
|
|
||||||
|
def _spawn_task(self, coro: Any) -> None:
|
||||||
|
task = asyncio.create_task(coro)
|
||||||
|
self._tasks.add(task)
|
||||||
|
task.add_done_callback(self._tasks.discard)
|
||||||
|
|
||||||
|
# ---- bus -------------------------------------------------------------
|
||||||
|
|
||||||
|
async def _events(self) -> None:
|
||||||
|
async for event in self.bus.stream():
|
||||||
|
try:
|
||||||
|
await self._on_event(event)
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
_log.exception("event %s failed", event.get("type"))
|
||||||
|
|
||||||
|
async def _on_event(self, event: Event) -> None:
|
||||||
|
kind = event["type"]
|
||||||
|
if kind == "conversation.bound":
|
||||||
|
self._targets.pop(str(event.get("conversation_id")), None)
|
||||||
|
return
|
||||||
|
if kind in ("delivery.sent", "delivery.failed", "conversation.created"):
|
||||||
|
return
|
||||||
|
key = event.get("conversation_id")
|
||||||
|
if not isinstance(key, str):
|
||||||
|
return
|
||||||
|
conv = await self.conversations.get(key)
|
||||||
|
if conv is None:
|
||||||
|
return
|
||||||
|
target = await self._target_of(conv)
|
||||||
|
if target is None:
|
||||||
|
return
|
||||||
|
match kind:
|
||||||
|
case "turn.start":
|
||||||
|
if event.get("origin") == "user":
|
||||||
|
await self._open_draft(key, event, target)
|
||||||
|
case "stream":
|
||||||
|
self._feed_draft(key, event)
|
||||||
|
case "tool":
|
||||||
|
draft = self._drafts.get(key)
|
||||||
|
if draft is not None and event.get("parent_tool_use_id") is None:
|
||||||
|
draft.set_status(
|
||||||
|
f"⏳ {status_label(str(event['name']), event.get('input'))}"
|
||||||
|
)
|
||||||
|
case "turn.end":
|
||||||
|
draft = self._drafts.get(key)
|
||||||
|
if draft is not None:
|
||||||
|
await draft.stop()
|
||||||
|
if event.get("stop") == "error" and event.get("origin") == "user":
|
||||||
|
await self._deliver(
|
||||||
|
conv,
|
||||||
|
"⚠️ тёрн упал, смотри логи gateway",
|
||||||
|
turn_id=event.get("turn_id"),
|
||||||
|
key=f"{event.get('turn_id')}:error",
|
||||||
|
)
|
||||||
|
case "reply":
|
||||||
|
await self._on_reply(conv, event)
|
||||||
|
case "say":
|
||||||
|
await self._deliver(
|
||||||
|
conv, str(event.get("text") or ""), turn_id=event.get("turn_id")
|
||||||
|
)
|
||||||
|
case "question":
|
||||||
|
await self._ask(conv, event, target)
|
||||||
|
case "question.answered":
|
||||||
|
await self._close_ask(
|
||||||
|
str(event["question_id"]), f"✅ {event.get('answer') or ''}"
|
||||||
|
)
|
||||||
|
case "question.timeout":
|
||||||
|
await self._close_ask(
|
||||||
|
str(event["question_id"]), "⌛ время вышло - ответь текстом"
|
||||||
|
)
|
||||||
|
case "conversation.merged":
|
||||||
|
await self._deliver(conv, "✅ слито в мастер", key=f"{key}:merged")
|
||||||
|
|
||||||
|
async def _on_reply(self, conv: Conversation, event: Event) -> None:
|
||||||
|
turn_id = str(event.get("turn_id") or "")
|
||||||
|
origin = str(event.get("item_origin") or "")
|
||||||
|
text = str(event.get("text") or "")
|
||||||
|
await self._unreact(event.get("item"))
|
||||||
|
draft = self._drafts.pop(conv.external_id, None)
|
||||||
|
if draft is not None:
|
||||||
|
await draft.finish(chunks(text)[-1] if text.strip() else "")
|
||||||
|
if origin != FRONTEND and not origin.startswith("сид"):
|
||||||
|
user_text = str(event.get("user_text") or "")
|
||||||
|
if user_text:
|
||||||
|
await self._deliver(
|
||||||
|
conv,
|
||||||
|
f"📝 из панели:\n{user_text}",
|
||||||
|
turn_id=turn_id,
|
||||||
|
key=f"{turn_id}:mirror",
|
||||||
|
)
|
||||||
|
await self._deliver(conv, text, turn_id=turn_id, key=f"{turn_id}:reply")
|
||||||
|
|
||||||
|
# ---- drafts ------------------------------------------------------------
|
||||||
|
|
||||||
|
async def _open_draft(
|
||||||
|
self, key: str, event: Event, target: tuple[int, int | None]
|
||||||
|
) -> None:
|
||||||
|
if target[0] < 0:
|
||||||
|
return
|
||||||
|
await self._close_draft(key)
|
||||||
|
draft = Draft(
|
||||||
|
self.bot,
|
||||||
|
chat_id=target[0],
|
||||||
|
thread_id=target[1],
|
||||||
|
turn_id=str(event.get("turn_id") or key),
|
||||||
|
interval=self.draft_interval,
|
||||||
|
)
|
||||||
|
self._drafts[key] = draft
|
||||||
|
draft.start()
|
||||||
|
|
||||||
|
def _feed_draft(self, key: str, event: Event) -> None:
|
||||||
|
draft = self._drafts.get(key)
|
||||||
|
if draft is None or event.get("parent_tool_use_id") is not None:
|
||||||
|
return
|
||||||
|
raw = event.get("event") or {}
|
||||||
|
kind = raw.get("type")
|
||||||
|
if kind == "content_block_delta":
|
||||||
|
delta = raw.get("delta") or {}
|
||||||
|
if delta.get("type") == "text_delta":
|
||||||
|
draft.set_status("✍️ пишу")
|
||||||
|
draft.append(str(delta.get("text") or ""))
|
||||||
|
elif delta.get("type") == "thinking_delta":
|
||||||
|
draft.set_status("🤔 думаю")
|
||||||
|
elif kind == "content_block_start":
|
||||||
|
block = raw.get("content_block") or {}
|
||||||
|
if block.get("type") == "tool_use":
|
||||||
|
draft.set_status(
|
||||||
|
f"⏳ {status_label(str(block.get('name') or ''), None)}"
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _close_draft(self, key: str) -> None:
|
||||||
|
draft = self._drafts.pop(key, None)
|
||||||
|
if draft is not None:
|
||||||
|
await draft.stop()
|
||||||
|
|
||||||
|
# ---- questions (§3.7) ---------------------------------------------------
|
||||||
|
|
||||||
|
async def _ask(
|
||||||
|
self, conv: Conversation, event: Event, target: tuple[int, int | None]
|
||||||
|
) -> None:
|
||||||
|
question_id = str(event["question_id"])
|
||||||
|
questions = [q for q in event.get("questions") or [] if isinstance(q, dict)]
|
||||||
|
if not questions:
|
||||||
|
return
|
||||||
|
ask = _Ask(
|
||||||
|
conversation_id=conv.external_id,
|
||||||
|
chat_id=target[0],
|
||||||
|
thread_id=target[1],
|
||||||
|
questions=questions,
|
||||||
|
)
|
||||||
|
self._asks[question_id] = ask
|
||||||
|
for qi, question in enumerate(questions):
|
||||||
|
try:
|
||||||
|
sent = await self.bot.send_message(
|
||||||
|
target[0],
|
||||||
|
_question_html(question),
|
||||||
|
message_thread_id=target[1],
|
||||||
|
parse_mode="HTML",
|
||||||
|
reply_markup=_keyboard(question_id, qi, question, []),
|
||||||
|
)
|
||||||
|
except TelegramAPIError:
|
||||||
|
_log.exception("question %s could not be sent", question_id)
|
||||||
|
continue
|
||||||
|
ask.messages.append(sent.message_id)
|
||||||
|
|
||||||
|
async def _on_callback(self, query: CallbackQuery) -> None:
|
||||||
|
if query.from_user.id != self.user_id or not query.data:
|
||||||
|
return
|
||||||
|
parts = query.data.split(":")
|
||||||
|
if len(parts) != 4 or parts[0] != "q":
|
||||||
|
return
|
||||||
|
_, question_id, qi_raw, choice = parts
|
||||||
|
ask = self._asks.get(question_id)
|
||||||
|
if ask is None or not qi_raw.isdigit():
|
||||||
|
await self._callback_reply(query, "вопрос уже закрыт")
|
||||||
|
return
|
||||||
|
qi = int(qi_raw)
|
||||||
|
question = ask.questions[qi]
|
||||||
|
options = [str(o.get("label", "")) for o in question.get("options") or []]
|
||||||
|
picked = ask.picked.setdefault(qi, [])
|
||||||
|
multi = bool(question.get("multiSelect"))
|
||||||
|
if choice == _DONE:
|
||||||
|
ask.done.add(qi)
|
||||||
|
elif choice.isdigit() and int(choice) < len(options):
|
||||||
|
label = options[int(choice)]
|
||||||
|
if multi:
|
||||||
|
if label in picked:
|
||||||
|
picked.remove(label)
|
||||||
|
else:
|
||||||
|
picked.append(label)
|
||||||
|
else:
|
||||||
|
picked[:] = [label]
|
||||||
|
ask.done.add(qi)
|
||||||
|
message = query.message if isinstance(query.message, Message) else None
|
||||||
|
if message is not None:
|
||||||
|
with contextlib.suppress(TelegramAPIError):
|
||||||
|
if qi in ask.done:
|
||||||
|
await self.bot.edit_message_text(
|
||||||
|
f"{_question_html(question)}\n\n"
|
||||||
|
f"✅ {html.escape(', '.join(picked) or '-')}",
|
||||||
|
chat_id=message.chat.id,
|
||||||
|
message_id=message.message_id,
|
||||||
|
parse_mode="HTML",
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
await self.bot.edit_message_reply_markup(
|
||||||
|
chat_id=message.chat.id,
|
||||||
|
message_id=message.message_id,
|
||||||
|
reply_markup=_keyboard(question_id, qi, question, picked),
|
||||||
|
)
|
||||||
|
await self._callback_reply(query, None)
|
||||||
|
if len(ask.done) == len(ask.questions):
|
||||||
|
answer = _answer_text(ask)
|
||||||
|
self._asks.pop(question_id, None)
|
||||||
|
if not self.conversations.answer(question_id, answer):
|
||||||
|
await self._edit_asks(ask, "⌛ время вышло - ответь текстом")
|
||||||
|
|
||||||
|
async def _callback_reply(self, query: CallbackQuery, text: str | None) -> None:
|
||||||
|
with contextlib.suppress(TelegramAPIError):
|
||||||
|
await self.bot.answer_callback_query(query.id, text=text)
|
||||||
|
|
||||||
|
async def _close_ask(self, question_id: str, note: str) -> None:
|
||||||
|
ask = self._asks.pop(question_id, None)
|
||||||
|
if ask is not None:
|
||||||
|
await self._edit_asks(ask, note)
|
||||||
|
|
||||||
|
async def _edit_asks(self, ask: _Ask, note: str) -> None:
|
||||||
|
for qi, message_id in enumerate(ask.messages):
|
||||||
|
if qi in ask.done and not note.startswith("⌛"):
|
||||||
|
continue
|
||||||
|
with contextlib.suppress(TelegramAPIError):
|
||||||
|
await self.bot.edit_message_text(
|
||||||
|
f"{_question_html(ask.questions[qi])}\n\n{html.escape(note)}",
|
||||||
|
chat_id=ask.chat_id,
|
||||||
|
message_id=message_id,
|
||||||
|
parse_mode="HTML",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _question_html(question: dict[str, Any]) -> str:
|
||||||
|
header = html.escape(str(question.get("header") or "").strip())
|
||||||
|
body = html.escape(str(question.get("question") or "").strip())
|
||||||
|
lines = [f"❓ <b>{header}</b>" if header else "❓", body]
|
||||||
|
for option in question.get("options") or []:
|
||||||
|
label = html.escape(str(option.get("label", "")))
|
||||||
|
description = html.escape(str(option.get("description") or "").strip())
|
||||||
|
lines.append(f"• <b>{label}</b>" + (f" - {description}" if description else ""))
|
||||||
|
return "\n".join(line for line in lines if line)
|
||||||
|
|
||||||
|
|
||||||
|
def _keyboard(
|
||||||
|
question_id: str, qi: int, question: dict[str, Any], picked: list[str]
|
||||||
|
) -> InlineKeyboardMarkup:
|
||||||
|
rows = [
|
||||||
|
[
|
||||||
|
InlineKeyboardButton(
|
||||||
|
text=("☑ " if str(o.get("label", "")) in picked else "")
|
||||||
|
+ str(o.get("label", ""))[:60],
|
||||||
|
callback_data=f"q:{question_id}:{qi}:{oi}",
|
||||||
|
)
|
||||||
|
]
|
||||||
|
for oi, o in enumerate(question.get("options") or [])
|
||||||
|
]
|
||||||
|
if question.get("multiSelect"):
|
||||||
|
rows.append(
|
||||||
|
[
|
||||||
|
InlineKeyboardButton(
|
||||||
|
text="✅ готово", callback_data=f"q:{question_id}:{qi}:{_DONE}"
|
||||||
|
)
|
||||||
|
]
|
||||||
|
)
|
||||||
|
return InlineKeyboardMarkup(inline_keyboard=rows)
|
||||||
|
|
||||||
|
|
||||||
|
def _answer_text(ask: _Ask) -> str:
|
||||||
|
if len(ask.questions) == 1:
|
||||||
|
return ", ".join(ask.picked.get(0, [])) or "-"
|
||||||
|
return "; ".join(
|
||||||
|
f"{q.get('header') or q.get('question')}: "
|
||||||
|
f"{', '.join(ask.picked.get(i, [])) or '-'}"
|
||||||
|
for i, q in enumerate(ask.questions)
|
||||||
|
)
|
||||||
@@ -0,0 +1,144 @@
|
|||||||
|
"""Long-polling inbox (§3.8).
|
||||||
|
|
||||||
|
Every update lands in ``telegram_updates`` before the offset moves past it;
|
||||||
|
a worker handles rows from the table, oldest first, and finishes whatever a
|
||||||
|
previous process left unprocessed at startup.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import contextlib
|
||||||
|
import logging
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
from aiogram.exceptions import TelegramConflictError, TelegramNetworkError
|
||||||
|
from aiogram.types import Update
|
||||||
|
from sqlalchemy import func
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
from sqlmodel import col, select
|
||||||
|
|
||||||
|
from beaver_gateway.storage.models import TelegramUpdate
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Awaitable, Callable
|
||||||
|
|
||||||
|
from aiogram import Bot
|
||||||
|
|
||||||
|
from beaver_gateway.storage.db import Database
|
||||||
|
|
||||||
|
__all__ = ["Inbox"]
|
||||||
|
|
||||||
|
_log = logging.getLogger("beaver_gateway.frontends.telegram.inbox")
|
||||||
|
|
||||||
|
ALLOWED_UPDATES = ("message", "callback_query")
|
||||||
|
|
||||||
|
|
||||||
|
def _unset(_value: object) -> None:
|
||||||
|
"""Unset aiogram ``Default`` sentinels (link previews) are stored as null."""
|
||||||
|
|
||||||
|
|
||||||
|
class Inbox:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
db: Database,
|
||||||
|
bot: Bot,
|
||||||
|
*,
|
||||||
|
handler: Callable[[Update], Awaitable[None]],
|
||||||
|
poll_timeout: int = 30,
|
||||||
|
) -> None:
|
||||||
|
self._db = db
|
||||||
|
self._bot = bot
|
||||||
|
self._handler = handler
|
||||||
|
self._poll_timeout = poll_timeout
|
||||||
|
self._wake = asyncio.Event()
|
||||||
|
|
||||||
|
async def run(self) -> None:
|
||||||
|
async with asyncio.TaskGroup() as tg:
|
||||||
|
tg.create_task(self._poll())
|
||||||
|
tg.create_task(self._work())
|
||||||
|
|
||||||
|
async def _poll(self) -> None:
|
||||||
|
offset = await self._next_offset()
|
||||||
|
backoff = 1.0
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
updates = await self._bot.get_updates(
|
||||||
|
offset=offset,
|
||||||
|
timeout=self._poll_timeout,
|
||||||
|
allowed_updates=list(ALLOWED_UPDATES),
|
||||||
|
request_timeout=self._poll_timeout + 10,
|
||||||
|
)
|
||||||
|
except TelegramConflictError:
|
||||||
|
_log.error("another poller holds this bot token; retrying in 10s")
|
||||||
|
await asyncio.sleep(10)
|
||||||
|
continue
|
||||||
|
except (TimeoutError, TelegramNetworkError, OSError) as exc:
|
||||||
|
_log.warning("getUpdates failed (%s); retrying in %.0fs", exc, backoff)
|
||||||
|
await asyncio.sleep(backoff)
|
||||||
|
backoff = min(backoff * 2, 60.0)
|
||||||
|
continue
|
||||||
|
backoff = 1.0
|
||||||
|
for update in updates:
|
||||||
|
await self._store(update)
|
||||||
|
offset = update.update_id + 1
|
||||||
|
if updates:
|
||||||
|
self._wake.set()
|
||||||
|
|
||||||
|
async def _store(self, update: Update) -> None:
|
||||||
|
row = TelegramUpdate(
|
||||||
|
update_id=update.update_id,
|
||||||
|
payload=update.model_dump(
|
||||||
|
mode="json", by_alias=True, exclude_none=True, fallback=_unset
|
||||||
|
),
|
||||||
|
)
|
||||||
|
async with self._db.session() as session:
|
||||||
|
session.add(row)
|
||||||
|
try:
|
||||||
|
await session.commit()
|
||||||
|
except IntegrityError:
|
||||||
|
await session.rollback()
|
||||||
|
|
||||||
|
async def _next_offset(self) -> int | None:
|
||||||
|
async with self._db.session() as session:
|
||||||
|
latest = (
|
||||||
|
await session.exec(select(func.max(col(TelegramUpdate.update_id))))
|
||||||
|
).one()
|
||||||
|
return int(latest) + 1 if latest is not None else None
|
||||||
|
|
||||||
|
async def _work(self) -> None:
|
||||||
|
while True:
|
||||||
|
rows = await self._pending()
|
||||||
|
if not rows:
|
||||||
|
self._wake.clear()
|
||||||
|
with contextlib.suppress(TimeoutError):
|
||||||
|
await asyncio.wait_for(self._wake.wait(), timeout=5.0)
|
||||||
|
continue
|
||||||
|
for row in rows:
|
||||||
|
await self._handle(row)
|
||||||
|
|
||||||
|
async def _pending(self, limit: int = 50) -> list[TelegramUpdate]:
|
||||||
|
async with self._db.session() as session:
|
||||||
|
result = await session.exec(
|
||||||
|
select(TelegramUpdate)
|
||||||
|
.where(col(TelegramUpdate.processed_at).is_(None))
|
||||||
|
.order_by(col(TelegramUpdate.update_id))
|
||||||
|
.limit(limit)
|
||||||
|
)
|
||||||
|
return list(result.all())
|
||||||
|
|
||||||
|
async def _handle(self, row: TelegramUpdate) -> None:
|
||||||
|
error: str | None = None
|
||||||
|
try:
|
||||||
|
await self._handler(Update.model_validate(row.payload))
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
error = f"{type(exc).__name__}: {exc}"[:500]
|
||||||
|
_log.exception("update %s failed", row.update_id)
|
||||||
|
async with self._db.session() as session:
|
||||||
|
stored = await session.get(TelegramUpdate, row.update_id)
|
||||||
|
if stored is not None:
|
||||||
|
stored.processed_at = datetime.now(UTC)
|
||||||
|
stored.error = error
|
||||||
|
session.add(stored)
|
||||||
|
await session.commit()
|
||||||
@@ -0,0 +1,235 @@
|
|||||||
|
"""Outbox (§3.8): a reply is a ``deliveries`` row first, a message second.
|
||||||
|
|
||||||
|
Rows are sent oldest first, retried with backoff on network errors and
|
||||||
|
flood limits, resent as plain text when Telegram rejects our HTML, and
|
||||||
|
given up only when Telegram says the window is gone.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import contextlib
|
||||||
|
import logging
|
||||||
|
from datetime import UTC, datetime, timedelta
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
from aiogram.exceptions import (
|
||||||
|
TelegramBadRequest,
|
||||||
|
TelegramForbiddenError,
|
||||||
|
TelegramNetworkError,
|
||||||
|
TelegramNotFound,
|
||||||
|
TelegramRetryAfter,
|
||||||
|
TelegramServerError,
|
||||||
|
)
|
||||||
|
from aiogram.types import LinkPreviewOptions
|
||||||
|
from sqlalchemy import func
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
from sqlmodel import col, select
|
||||||
|
|
||||||
|
from beaver_gateway.frontends.telegram.render import chunks, to_html
|
||||||
|
from beaver_gateway.storage.models import Delivery
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from aiogram import Bot
|
||||||
|
|
||||||
|
from beaver_gateway.core.bus import EventBus
|
||||||
|
from beaver_gateway.storage.db import Database
|
||||||
|
|
||||||
|
__all__ = ["Outbox"]
|
||||||
|
|
||||||
|
_log = logging.getLogger("beaver_gateway.frontends.telegram.outbox")
|
||||||
|
|
||||||
|
_MAX_BACKOFF = 300.0
|
||||||
|
_GONE = ("thread not found", "chat not found", "topic_deleted", "topic_closed")
|
||||||
|
_NO_PREVIEW = LinkPreviewOptions(is_disabled=True)
|
||||||
|
|
||||||
|
|
||||||
|
class Outbox:
|
||||||
|
def __init__(
|
||||||
|
self, db: Database, bot: Bot, *, bus: EventBus, backoff: float = 2.0
|
||||||
|
) -> None:
|
||||||
|
self._db = db
|
||||||
|
self._bot = bot
|
||||||
|
self._bus = bus
|
||||||
|
self._backoff = backoff
|
||||||
|
self._wake = asyncio.Event()
|
||||||
|
|
||||||
|
async def enqueue(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
chat_id: int,
|
||||||
|
thread_id: int | None,
|
||||||
|
text: str,
|
||||||
|
conversation_id: int | None = None,
|
||||||
|
turn_id: str | None = None,
|
||||||
|
dedupe_key: str | None = None,
|
||||||
|
) -> list[Delivery]:
|
||||||
|
rows: list[Delivery] = []
|
||||||
|
for n, part in enumerate(chunks(text)):
|
||||||
|
row = Delivery(
|
||||||
|
conversation_id=conversation_id,
|
||||||
|
chat_id=chat_id,
|
||||||
|
thread_id=thread_id,
|
||||||
|
text=part,
|
||||||
|
turn_id=turn_id,
|
||||||
|
dedupe_key=f"{dedupe_key}:{n}" if dedupe_key else None,
|
||||||
|
)
|
||||||
|
async with self._db.session() as session:
|
||||||
|
session.add(row)
|
||||||
|
try:
|
||||||
|
await session.commit()
|
||||||
|
except IntegrityError:
|
||||||
|
await session.rollback()
|
||||||
|
continue
|
||||||
|
await session.refresh(row)
|
||||||
|
rows.append(row)
|
||||||
|
if rows:
|
||||||
|
self._wake.set()
|
||||||
|
return rows
|
||||||
|
|
||||||
|
async def run(self) -> None:
|
||||||
|
while True:
|
||||||
|
rows = await self._due()
|
||||||
|
if not rows:
|
||||||
|
self._wake.clear()
|
||||||
|
with contextlib.suppress(TimeoutError):
|
||||||
|
await asyncio.wait_for(
|
||||||
|
self._wake.wait(), timeout=await self._wait_for_next()
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
for row in rows:
|
||||||
|
await self._send(row)
|
||||||
|
|
||||||
|
async def _wait_for_next(self, cap: float = 5.0) -> float:
|
||||||
|
async with self._db.session() as session:
|
||||||
|
earliest = (
|
||||||
|
await session.exec(
|
||||||
|
select(func.min(col(Delivery.next_attempt_at))).where(
|
||||||
|
Delivery.status == "queued"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).one()
|
||||||
|
if earliest is None:
|
||||||
|
return cap
|
||||||
|
now = datetime.now(UTC).replace(tzinfo=None)
|
||||||
|
return max(0.05, min(cap, (earliest - now).total_seconds()))
|
||||||
|
|
||||||
|
async def _due(self, limit: int = 50) -> list[Delivery]:
|
||||||
|
now = datetime.now(UTC).replace(tzinfo=None)
|
||||||
|
async with self._db.session() as session:
|
||||||
|
result = await session.exec(
|
||||||
|
select(Delivery)
|
||||||
|
.where(
|
||||||
|
Delivery.status == "queued", col(Delivery.next_attempt_at) <= now
|
||||||
|
)
|
||||||
|
.order_by(col(Delivery.id))
|
||||||
|
.limit(limit)
|
||||||
|
)
|
||||||
|
return list(result.all())
|
||||||
|
|
||||||
|
async def _send(self, row: Delivery) -> None:
|
||||||
|
try:
|
||||||
|
message = await self._bot.send_message(
|
||||||
|
row.chat_id,
|
||||||
|
row.text if row.plain else to_html(row.text),
|
||||||
|
message_thread_id=row.thread_id,
|
||||||
|
parse_mode=None if row.plain else "HTML",
|
||||||
|
link_preview_options=_NO_PREVIEW,
|
||||||
|
)
|
||||||
|
except TelegramRetryAfter as exc:
|
||||||
|
await self._retry(row, str(exc), delay=float(exc.retry_after))
|
||||||
|
except TelegramBadRequest as exc:
|
||||||
|
text = str(exc).lower()
|
||||||
|
if "parse" in text and not row.plain:
|
||||||
|
await self._retry(row, str(exc), delay=0.0, plain=True)
|
||||||
|
elif any(marker in text for marker in _GONE):
|
||||||
|
await self._fail(row, str(exc))
|
||||||
|
else:
|
||||||
|
await self._fail(row, str(exc))
|
||||||
|
except (TelegramNotFound, TelegramForbiddenError) as exc:
|
||||||
|
await self._fail(row, str(exc))
|
||||||
|
except (TelegramNetworkError, TelegramServerError, OSError) as exc:
|
||||||
|
await self._retry(
|
||||||
|
row,
|
||||||
|
str(exc),
|
||||||
|
delay=min(self._backoff ** (row.attempts + 1), _MAX_BACKOFF),
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
await self._mark(row, status="sent", message_id=message.message_id)
|
||||||
|
self._bus.publish(
|
||||||
|
"delivery.sent",
|
||||||
|
delivery=row.id,
|
||||||
|
conversation_row=row.conversation_id,
|
||||||
|
chat_id=row.chat_id,
|
||||||
|
thread_id=row.thread_id,
|
||||||
|
message_id=message.message_id,
|
||||||
|
turn_id=row.turn_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _retry(
|
||||||
|
self, row: Delivery, error: str, *, delay: float, plain: bool = False
|
||||||
|
) -> None:
|
||||||
|
_log.warning(
|
||||||
|
"delivery #%s attempt %d failed: %s (retry in %.0fs)",
|
||||||
|
row.id,
|
||||||
|
row.attempts + 1,
|
||||||
|
error,
|
||||||
|
delay,
|
||||||
|
)
|
||||||
|
await self._mark(row, status="queued", error=error, delay=delay, plain=plain)
|
||||||
|
if delay < 5.0:
|
||||||
|
self._wake.set()
|
||||||
|
|
||||||
|
async def _fail(self, row: Delivery, error: str) -> None:
|
||||||
|
_log.error(
|
||||||
|
"delivery #%s to %s/%s given up: %s",
|
||||||
|
row.id,
|
||||||
|
row.chat_id,
|
||||||
|
row.thread_id,
|
||||||
|
error,
|
||||||
|
)
|
||||||
|
await self._mark(row, status="failed", error=error)
|
||||||
|
self._bus.publish(
|
||||||
|
"delivery.failed",
|
||||||
|
delivery=row.id,
|
||||||
|
conversation_row=row.conversation_id,
|
||||||
|
chat_id=row.chat_id,
|
||||||
|
thread_id=row.thread_id,
|
||||||
|
error=error,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _mark(
|
||||||
|
self,
|
||||||
|
row: Delivery,
|
||||||
|
*,
|
||||||
|
status: str,
|
||||||
|
error: str | None = None,
|
||||||
|
delay: float = 0.0,
|
||||||
|
plain: bool = False,
|
||||||
|
message_id: int | None = None,
|
||||||
|
) -> None:
|
||||||
|
async with self._db.session() as session:
|
||||||
|
stored = await session.get(Delivery, row.id)
|
||||||
|
if stored is None:
|
||||||
|
return
|
||||||
|
stored.status = status
|
||||||
|
stored.attempts += 1
|
||||||
|
stored.last_error = error[:500] if error else None
|
||||||
|
stored.next_attempt_at = (
|
||||||
|
datetime.now(UTC) + timedelta(seconds=delay)
|
||||||
|
).replace(tzinfo=None)
|
||||||
|
if plain:
|
||||||
|
stored.plain = True
|
||||||
|
if message_id is not None:
|
||||||
|
stored.message_id = message_id
|
||||||
|
if status == "sent":
|
||||||
|
stored.sent_at = datetime.now(UTC)
|
||||||
|
session.add(stored)
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
async def pending(self) -> int:
|
||||||
|
async with self._db.session() as session:
|
||||||
|
result = await session.exec(
|
||||||
|
select(Delivery).where(Delivery.status == "queued")
|
||||||
|
)
|
||||||
|
return len(list(result.all()))
|
||||||
@@ -0,0 +1,111 @@
|
|||||||
|
"""Model markdown → Telegram HTML, chunking, and the status line for drafts."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import html
|
||||||
|
import re
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
__all__ = ["LIMIT", "chunks", "status_label", "to_html"]
|
||||||
|
|
||||||
|
LIMIT = 4000
|
||||||
|
_FENCE = re.compile(r"```[^\n]*\n(.*?)(?:```|$)", re.DOTALL)
|
||||||
|
_INLINE_CODE = re.compile(r"(`[^`\n]+`)")
|
||||||
|
_HEADING = re.compile(r"^#{1,6}\s+(.+)$", re.MULTILINE)
|
||||||
|
_BOLD = re.compile(r"\*\*(.+?)\*\*|__(.+?)__", re.DOTALL)
|
||||||
|
_ITALIC = re.compile(r"(?<![\w*])\*(?!\s)(.+?)(?<!\s)\*(?![\w*])")
|
||||||
|
_ITALIC_U = re.compile(r"(?<![\w_])_(?!\s)(.+?)(?<!\s)_(?![\w_])")
|
||||||
|
_LINK = re.compile(r"\[([^\]]+)\]\((https?://[^)\s]+)\)")
|
||||||
|
_BULLET = re.compile(r"^(\s*)[-*]\s+", re.MULTILINE)
|
||||||
|
_STRIKE = re.compile(r"~~(.+?)~~")
|
||||||
|
|
||||||
|
_LABELS: dict[str, str] = {
|
||||||
|
"Read": "читаю vault",
|
||||||
|
"Glob": "ищу файлы",
|
||||||
|
"Grep": "ищу в vault",
|
||||||
|
"Edit": "правлю файл",
|
||||||
|
"Write": "пишу файл",
|
||||||
|
"MultiEdit": "правлю файлы",
|
||||||
|
"Bash": "выполняю команду",
|
||||||
|
"WebSearch": "ищу в сети",
|
||||||
|
"WebFetch": "читаю страницу",
|
||||||
|
"Task": "запустил сабагента",
|
||||||
|
"Agent": "запустил сабагента",
|
||||||
|
"AskUserQuestion": "спрашиваю",
|
||||||
|
"TodoWrite": "планирую",
|
||||||
|
"Skill": "открываю скилл",
|
||||||
|
"mcp__gateway__spawn": "открываю разговор",
|
||||||
|
"mcp__gateway__read_conversation": "читаю разговор",
|
||||||
|
"mcp__gateway__say": "говорю",
|
||||||
|
"mcp__gateway__schedule": "ставлю напоминание",
|
||||||
|
"mcp__gateway__inject": "передаю в другой разговор",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def to_html(text: str) -> str:
|
||||||
|
out: list[str] = []
|
||||||
|
pos = 0
|
||||||
|
for match in _FENCE.finditer(text):
|
||||||
|
out.append(_inline(text[pos : match.start()]))
|
||||||
|
out.append(f"<pre>{html.escape(match.group(1).rstrip())}</pre>")
|
||||||
|
pos = match.end()
|
||||||
|
out.append(_inline(text[pos:]))
|
||||||
|
return "".join(out).strip()
|
||||||
|
|
||||||
|
|
||||||
|
def _inline(text: str) -> str:
|
||||||
|
parts = _INLINE_CODE.split(text)
|
||||||
|
for i, part in enumerate(parts):
|
||||||
|
if i % 2:
|
||||||
|
parts[i] = f"<code>{html.escape(part[1:-1])}</code>"
|
||||||
|
continue
|
||||||
|
s = html.escape(part, quote=False)
|
||||||
|
s = _HEADING.sub(r"<b>\1</b>", s)
|
||||||
|
s = _BOLD.sub(lambda m: f"<b>{m.group(1) or m.group(2)}</b>", s)
|
||||||
|
s = _ITALIC.sub(r"<i>\1</i>", s)
|
||||||
|
s = _ITALIC_U.sub(r"<i>\1</i>", s)
|
||||||
|
s = _STRIKE.sub(r"<s>\1</s>", s)
|
||||||
|
s = _LINK.sub(r'<a href="\2">\1</a>', s)
|
||||||
|
parts[i] = _BULLET.sub(r"\1• ", s)
|
||||||
|
return "".join(parts)
|
||||||
|
|
||||||
|
|
||||||
|
def chunks(text: str, limit: int = LIMIT) -> list[str]:
|
||||||
|
text = text.strip()
|
||||||
|
if len(text) <= limit:
|
||||||
|
return [text] if text else []
|
||||||
|
out: list[str] = []
|
||||||
|
while len(text) > limit:
|
||||||
|
cut = _cut_point(text, limit)
|
||||||
|
out.append(text[:cut].rstrip())
|
||||||
|
text = text[cut:].lstrip()
|
||||||
|
if text:
|
||||||
|
out.append(text)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _cut_point(text: str, limit: int) -> int:
|
||||||
|
for sep in ("\n\n", "\n", ". ", " "):
|
||||||
|
cut = text.rfind(sep, limit // 2, limit)
|
||||||
|
if cut > 0:
|
||||||
|
return cut + len(sep)
|
||||||
|
return limit
|
||||||
|
|
||||||
|
|
||||||
|
def status_label(name: str, tool_input: dict[str, Any] | None = None) -> str:
|
||||||
|
label = _LABELS.get(name)
|
||||||
|
if label is not None:
|
||||||
|
return label
|
||||||
|
if name.startswith("mcp__"):
|
||||||
|
parts = name.split("__", 2)
|
||||||
|
server = parts[1]
|
||||||
|
tool = parts[2] if len(parts) == 3 else ""
|
||||||
|
return f"{server}: {tool}" if tool else server
|
||||||
|
hint = ""
|
||||||
|
if tool_input:
|
||||||
|
for key in ("description", "command", "file_path", "pattern", "query"):
|
||||||
|
value = tool_input.get(key)
|
||||||
|
if isinstance(value, str) and value.strip():
|
||||||
|
hint = value.strip().splitlines()[0][:60]
|
||||||
|
break
|
||||||
|
return f"{name} {hint}" if hint else name
|
||||||
@@ -17,7 +17,7 @@ heavy setups (PRD §6 cites the ~95%→~71% tool-selection drop on flat
|
|||||||
namespaces) but real and reachable.
|
namespaces) but real and reachable.
|
||||||
|
|
||||||
The aggregator returns both the app and a ``{name: url}`` map; Phase
|
The aggregator returns both the app and a ``{name: url}`` map; Phase
|
||||||
2.2's ``ClaudeCodeBackendAdapter`` plugs the map directly into
|
``ClaudeSdkBackend`` plugs the map directly into
|
||||||
``BackendOptions.mcp_servers``. ``/mcp/all/`` is NOT included in that
|
``BackendOptions.mcp_servers``. ``/mcp/all/`` is NOT included in that
|
||||||
map — claude-code-agents always get per-domain framing; only the
|
map — claude-code-agents always get per-domain framing; only the
|
||||||
external MCP frontend (Phase 3.1) reverse-proxies the flat endpoint.
|
external MCP frontend (Phase 3.1) reverse-proxies the flat endpoint.
|
||||||
@@ -33,6 +33,7 @@ from starlette.applications import Starlette
|
|||||||
from starlette.routing import Mount
|
from starlette.routing import Mount
|
||||||
|
|
||||||
from beaver_gateway.mcp.client_pool import build_http_proxy, build_stdio_proxy
|
from beaver_gateway.mcp.client_pool import build_http_proxy, build_stdio_proxy
|
||||||
|
from beaver_gateway.mcp.redacting import RedactingMiddleware
|
||||||
from beaver_gateway.mcp.types import HttpMcp, PythonToolMcp, StdioMcp
|
from beaver_gateway.mcp.types import HttpMcp, PythonToolMcp, StdioMcp
|
||||||
from beaver_gateway.mcp.wrap import build_python_tool_server
|
from beaver_gateway.mcp.wrap import build_python_tool_server
|
||||||
|
|
||||||
@@ -69,6 +70,12 @@ def build_internal_app(
|
|||||||
MCP tools into the Raycast wire).
|
MCP tools into the Raycast wire).
|
||||||
"""
|
"""
|
||||||
servers: dict[str, FastMCP] = {spec.name: _build_server(spec) for spec in mcps}
|
servers: dict[str, FastMCP] = {spec.name: _build_server(spec) for spec in mcps}
|
||||||
|
# The one place every tool result passes through on its way to the
|
||||||
|
# model. Attached per namespace rather than once at the top so that
|
||||||
|
# a direct ``call_tool`` on a child (the Raycast backend does that)
|
||||||
|
# is filtered too.
|
||||||
|
for server in servers.values():
|
||||||
|
server.add_middleware(RedactingMiddleware())
|
||||||
|
|
||||||
child_apps = {
|
child_apps = {
|
||||||
name: s.http_app(transport="http", path="/") for name, s in servers.items()
|
name: s.http_app(transport="http", path="/") for name, s in servers.items()
|
||||||
@@ -80,6 +87,7 @@ def build_internal_app(
|
|||||||
all_app = None
|
all_app = None
|
||||||
if servers:
|
if servers:
|
||||||
all_server = _build_all_server(servers)
|
all_server = _build_all_server(servers)
|
||||||
|
all_server.add_middleware(RedactingMiddleware())
|
||||||
all_app = all_server.http_app(transport="http", path="/")
|
all_app = all_server.http_app(transport="http", path="/")
|
||||||
routes.append(Mount(f"/mcp/{ALL_NAMESPACE}", app=all_app))
|
routes.append(Mount(f"/mcp/{ALL_NAMESPACE}", app=all_app))
|
||||||
|
|
||||||
|
|||||||
@@ -84,23 +84,19 @@ class LenientStdioTransport(ClientTransport):
|
|||||||
self, **session_kwargs: Unpack[SessionKwargs]
|
self, **session_kwargs: Unpack[SessionKwargs]
|
||||||
) -> AsyncIterator[ClientSession]:
|
) -> AsyncIterator[ClientSession]:
|
||||||
errlog: TextIO = self.log_file if self.log_file is not None else sys.stderr
|
errlog: TextIO = self.log_file if self.log_file is not None else sys.stderr
|
||||||
async with _lenient_stdio_client(
|
async with (
|
||||||
StdioServerParameters(
|
_lenient_stdio_client(
|
||||||
command=self.command,
|
StdioServerParameters(
|
||||||
args=self.args,
|
command=self.command, args=self.args, env=self.env, cwd=self.cwd
|
||||||
env=self.env,
|
),
|
||||||
cwd=self.cwd,
|
errlog=errlog,
|
||||||
),
|
) as (read_stream, write_stream),
|
||||||
errlog=errlog,
|
ClientSession(read_stream, write_stream, **session_kwargs) as session,
|
||||||
) as (read_stream, write_stream), ClientSession(
|
):
|
||||||
read_stream, write_stream, **session_kwargs
|
|
||||||
) as session:
|
|
||||||
yield session
|
yield session
|
||||||
|
|
||||||
def __repr__(self) -> str:
|
def __repr__(self) -> str:
|
||||||
return (
|
return f"<LenientStdioTransport(command={self.command!r}, args={self.args!r})>"
|
||||||
f"<LenientStdioTransport(command={self.command!r}, args={self.args!r})>"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@contextlib.asynccontextmanager
|
@contextlib.asynccontextmanager
|
||||||
@@ -133,9 +129,7 @@ async def _lenient_stdio_client( # noqa: PLR0915 — mirrors mcp.client.stdio.s
|
|||||||
command=command,
|
command=command,
|
||||||
args=server.args,
|
args=server.args,
|
||||||
env=(
|
env=(
|
||||||
{**env_default, **server.env}
|
{**env_default, **server.env} if server.env is not None else env_default
|
||||||
if server.env is not None
|
|
||||||
else env_default
|
|
||||||
),
|
),
|
||||||
errlog=errlog,
|
errlog=errlog,
|
||||||
cwd=server.cwd,
|
cwd=server.cwd,
|
||||||
@@ -164,9 +158,7 @@ async def _lenient_stdio_client( # noqa: PLR0915 — mirrors mcp.client.stdio.s
|
|||||||
if not stripped:
|
if not stripped:
|
||||||
continue
|
continue
|
||||||
try:
|
try:
|
||||||
message = types.JSONRPCMessage.model_validate_json(
|
message = types.JSONRPCMessage.model_validate_json(stripped)
|
||||||
stripped
|
|
||||||
)
|
|
||||||
except Exception: # noqa: BLE001 — by design, see module doc
|
except Exception: # noqa: BLE001 — by design, see module doc
|
||||||
_log.debug(
|
_log.debug(
|
||||||
"lenient stdio: dropped non-JSON line: %r",
|
"lenient stdio: dropped non-JSON line: %r",
|
||||||
@@ -194,10 +186,7 @@ async def _lenient_stdio_client( # noqa: PLR0915 — mirrors mcp.client.stdio.s
|
|||||||
except anyio.ClosedResourceError:
|
except anyio.ClosedResourceError:
|
||||||
await anyio.lowlevel.checkpoint()
|
await anyio.lowlevel.checkpoint()
|
||||||
|
|
||||||
async with (
|
async with anyio.create_task_group() as tg, process:
|
||||||
anyio.create_task_group() as tg,
|
|
||||||
process,
|
|
||||||
):
|
|
||||||
tg.start_soon(stdout_reader)
|
tg.start_soon(stdout_reader)
|
||||||
tg.start_soon(stdin_writer)
|
tg.start_soon(stdin_writer)
|
||||||
try:
|
try:
|
||||||
|
|||||||
@@ -0,0 +1,85 @@
|
|||||||
|
"""Redact credentials on the way out of an MCP tool.
|
||||||
|
|
||||||
|
A tool result is a wider channel than the log. It goes straight into the
|
||||||
|
model's context, from there into the turn record in Postgres, and from
|
||||||
|
there into the markdown transcript in ``💬 чаты`` — which Obsidian Sync
|
||||||
|
carries off the machine. One ``komodo`` deploy returns the resolved
|
||||||
|
compose file, ``environment:`` block and all, so a single call can put
|
||||||
|
every secret of a stack into all four places at once.
|
||||||
|
|
||||||
|
One filter, not a list of exceptions: every MCP the model can reach is
|
||||||
|
built into a ``FastMCP`` by :mod:`beaver_gateway.mcp.internal_app` —
|
||||||
|
``python_tool`` bundles like komodo, stdio subprocesses, remote HTTP
|
||||||
|
servers — and every route into one of them runs its middleware chain.
|
||||||
|
That covers the per-namespace ``/mcp/<name>/`` mounts claude-code talks
|
||||||
|
to, the ``/mcp/all`` bundle, the external MCP frontend reverse-proxying
|
||||||
|
into both, and the Raycast backend's direct ``call_tool``. A new MCP in
|
||||||
|
``config.py`` is covered the day it is added, without anyone
|
||||||
|
remembering to list it here.
|
||||||
|
|
||||||
|
What this does not reach: tools that never touch a FastMCP server — the
|
||||||
|
gateway's own ``gateway`` tools, and everything claude-code runs inside
|
||||||
|
its own process (``Bash``, ``Read``). Those are guarded by
|
||||||
|
:mod:`beaver_gateway.core.policy` and the vault mounts instead.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
import mcp.types as mt
|
||||||
|
from fastmcp.server.middleware import Middleware
|
||||||
|
from fastmcp.tools.base import ToolResult
|
||||||
|
|
||||||
|
from beaver_gateway.core.redact import redact, redact_data
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from fastmcp.server.middleware import CallNext, MiddlewareContext
|
||||||
|
|
||||||
|
__all__ = ["RedactingMiddleware"]
|
||||||
|
|
||||||
|
|
||||||
|
def _redact_block(block: mt.ContentBlock) -> mt.ContentBlock:
|
||||||
|
"""Mask a content block's text; binary blocks pass through."""
|
||||||
|
if isinstance(block, mt.TextContent):
|
||||||
|
masked = redact(block.text)
|
||||||
|
return (
|
||||||
|
block if masked == block.text else block.model_copy(update={"text": masked})
|
||||||
|
)
|
||||||
|
if isinstance(block, mt.EmbeddedResource) and isinstance(
|
||||||
|
block.resource, mt.TextResourceContents
|
||||||
|
):
|
||||||
|
masked = redact(block.resource.text)
|
||||||
|
if masked == block.resource.text:
|
||||||
|
return block
|
||||||
|
resource = block.resource.model_copy(update={"text": masked})
|
||||||
|
return block.model_copy(update={"resource": resource})
|
||||||
|
return block
|
||||||
|
|
||||||
|
|
||||||
|
class RedactingMiddleware(Middleware):
|
||||||
|
"""Mask credentials in whatever a tool call returns.
|
||||||
|
|
||||||
|
Idempotent, so mounting a server both on its own namespace and in
|
||||||
|
the ``/mcp/all`` bundle costs a second pass, not a double mask.
|
||||||
|
"""
|
||||||
|
|
||||||
|
async def on_call_tool(
|
||||||
|
self,
|
||||||
|
context: MiddlewareContext[mt.CallToolRequestParams],
|
||||||
|
call_next: CallNext[mt.CallToolRequestParams, ToolResult],
|
||||||
|
) -> ToolResult:
|
||||||
|
result = await call_next(context)
|
||||||
|
content = [_redact_block(block) for block in result.content]
|
||||||
|
structured = redact_data(result.structured_content)
|
||||||
|
if content == result.content and structured == result.structured_content:
|
||||||
|
return result
|
||||||
|
return ToolResult(
|
||||||
|
content=content,
|
||||||
|
structured_content=structured,
|
||||||
|
# Masked structured content no longer has to satisfy the
|
||||||
|
# tool's output schema; a non-None meta takes the
|
||||||
|
# ``CallToolResult`` path that skips that validation, the
|
||||||
|
# same trick ``ResponseLimitingMiddleware`` uses.
|
||||||
|
meta=result.meta if result.meta is not None else {},
|
||||||
|
)
|
||||||
@@ -62,9 +62,7 @@ class PythonToolMcp(_BaseMcp):
|
|||||||
tools: tuple[Callable[..., object], ...]
|
tools: tuple[Callable[..., object], ...]
|
||||||
|
|
||||||
|
|
||||||
McpServerT = Annotated[
|
McpServerT = Annotated[StdioMcp | HttpMcp | PythonToolMcp, Field(discriminator="kind")]
|
||||||
StdioMcp | HttpMcp | PythonToolMcp, Field(discriminator="kind")
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
class McpServer:
|
class McpServer:
|
||||||
|
|||||||
@@ -27,6 +27,12 @@ class Settings(BaseSettings):
|
|||||||
internal_mcp_port: int = 8765
|
internal_mcp_port: int = 8765
|
||||||
config_path: Path = Path("/config/config.py")
|
config_path: Path = Path("/config/config.py")
|
||||||
|
|
||||||
|
claude_runner_user: str | None = None
|
||||||
|
"""Unix user the claude subprocess runs as. Needs the gateway to be root."""
|
||||||
|
|
||||||
|
claude_home: Path | None = None
|
||||||
|
"""``HOME`` for the claude subprocess (its ``~/.claude`` lives there)."""
|
||||||
|
|
||||||
raycast_bearer: str | None = None
|
raycast_bearer: str | None = None
|
||||||
raycast_config_path: Path = Path("/config/raycast.json")
|
raycast_config_path: Path = Path("/config/raycast.json")
|
||||||
raycast_device_id: str | None = None
|
raycast_device_id: str | None = None
|
||||||
|
|||||||
@@ -1,32 +1,58 @@
|
|||||||
"""SQLModel-backed persistence.
|
"""SQLModel-backed persistence.
|
||||||
|
|
||||||
Two tables — :class:`Token`, :class:`AuditLog` — plus a thin
|
Tokens, audit, conversations, Agent SDK transcript entries and per-turn
|
||||||
:class:`Database` wrapper around an async SQLAlchemy engine. The
|
usage, plus a thin :class:`Database` wrapper around an async SQLAlchemy
|
||||||
``GatewayRuntime`` carries the handle so auth (token verify / touch)
|
engine. ``GatewayRuntime`` carries the handle.
|
||||||
and admin (token CRUD + audit listing) can reach it.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from beaver_gateway.storage.db import (
|
from beaver_gateway.storage.db import (
|
||||||
Database,
|
Database,
|
||||||
append_audit,
|
append_audit,
|
||||||
|
append_usage,
|
||||||
|
backfill_usage_deltas,
|
||||||
create_token,
|
create_token,
|
||||||
list_active_tokens,
|
list_active_tokens,
|
||||||
list_audit_records,
|
list_audit_records,
|
||||||
list_tokens,
|
list_tokens,
|
||||||
revoke_token,
|
revoke_token,
|
||||||
touch_token,
|
touch_token,
|
||||||
|
usage_deltas,
|
||||||
)
|
)
|
||||||
from beaver_gateway.storage.models import AuditLog, Token
|
from beaver_gateway.storage.models import (
|
||||||
|
AuditLog,
|
||||||
|
Conversation,
|
||||||
|
ConversationBinding,
|
||||||
|
Delivery,
|
||||||
|
InjectQueueItem,
|
||||||
|
RateLimit,
|
||||||
|
TelegramUpdate,
|
||||||
|
Token,
|
||||||
|
TranscriptEntry,
|
||||||
|
Usage,
|
||||||
|
)
|
||||||
|
from beaver_gateway.storage.session_store import PostgresSessionStore
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
"AuditLog",
|
"AuditLog",
|
||||||
|
"Conversation",
|
||||||
|
"ConversationBinding",
|
||||||
"Database",
|
"Database",
|
||||||
|
"Delivery",
|
||||||
|
"InjectQueueItem",
|
||||||
|
"PostgresSessionStore",
|
||||||
|
"RateLimit",
|
||||||
|
"TelegramUpdate",
|
||||||
"Token",
|
"Token",
|
||||||
|
"TranscriptEntry",
|
||||||
|
"Usage",
|
||||||
"append_audit",
|
"append_audit",
|
||||||
|
"append_usage",
|
||||||
|
"backfill_usage_deltas",
|
||||||
"create_token",
|
"create_token",
|
||||||
"list_active_tokens",
|
"list_active_tokens",
|
||||||
"list_audit_records",
|
"list_audit_records",
|
||||||
"list_tokens",
|
"list_tokens",
|
||||||
"revoke_token",
|
"revoke_token",
|
||||||
"touch_token",
|
"touch_token",
|
||||||
|
"usage_deltas",
|
||||||
]
|
]
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
"""Rewrite ``usage.cost_usd`` / ``model_usage`` as per-turn deltas.
|
||||||
|
|
||||||
|
Until 2026-09-01 the gateway stored ``ResultMessage.total_cost_usd`` verbatim,
|
||||||
|
which is cumulative for the claude process - summing the column overstated a
|
||||||
|
day by an order of magnitude. Run once after deploying the delta-aware
|
||||||
|
:func:`beaver_gateway.storage.append_usage`::
|
||||||
|
|
||||||
|
python -m beaver_gateway.storage.backfill_usage
|
||||||
|
|
||||||
|
Reads ``DATABASE_URL`` like the gateway (``.env`` included); safe to rerun.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
from dotenv import load_dotenv
|
||||||
|
|
||||||
|
from beaver_gateway.settings import Settings
|
||||||
|
from beaver_gateway.storage.db import Database, backfill_usage_deltas
|
||||||
|
|
||||||
|
|
||||||
|
async def _main() -> None:
|
||||||
|
load_dotenv(override=False)
|
||||||
|
settings = Settings() # ty: ignore[missing-argument]
|
||||||
|
db = Database(settings.database_url)
|
||||||
|
try:
|
||||||
|
await db.create_all()
|
||||||
|
async with db.session() as session:
|
||||||
|
changed = await backfill_usage_deltas(session)
|
||||||
|
finally:
|
||||||
|
await db.dispose()
|
||||||
|
print(f"usage rows rewritten: {changed}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
asyncio.run(_main())
|
||||||
@@ -18,15 +18,17 @@ import json
|
|||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
from typing import TYPE_CHECKING, Any
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
|
from sqlalchemy import inspect, text
|
||||||
from sqlalchemy.ext.asyncio import create_async_engine
|
from sqlalchemy.ext.asyncio import create_async_engine
|
||||||
from sqlmodel import SQLModel, select
|
from sqlmodel import SQLModel, col, select
|
||||||
from sqlmodel.ext.asyncio.session import AsyncSession
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||||
|
|
||||||
from beaver_gateway.storage.models import AuditLog, Token
|
from beaver_gateway.storage.models import AuditLog, Token, Usage
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from collections.abc import Sequence
|
from collections.abc import Sequence
|
||||||
|
|
||||||
|
from sqlalchemy import Connection
|
||||||
from sqlalchemy.ext.asyncio import AsyncEngine
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||||
|
|
||||||
|
|
||||||
@@ -72,9 +74,14 @@ class Database:
|
|||||||
self._engine: AsyncEngine = create_async_engine(_to_async_url(url), echo=False)
|
self._engine: AsyncEngine = create_async_engine(_to_async_url(url), echo=False)
|
||||||
|
|
||||||
async def create_all(self) -> None:
|
async def create_all(self) -> None:
|
||||||
"""Idempotent ``CREATE TABLE IF NOT EXISTS`` for every model."""
|
"""Create missing tables, then add columns models gained since.
|
||||||
|
|
||||||
|
Nullable columns only - that is the whole migration story until
|
||||||
|
Alembic is worth it.
|
||||||
|
"""
|
||||||
async with self._engine.begin() as conn:
|
async with self._engine.begin() as conn:
|
||||||
await conn.run_sync(SQLModel.metadata.create_all)
|
await conn.run_sync(SQLModel.metadata.create_all)
|
||||||
|
await conn.run_sync(_add_missing_columns)
|
||||||
|
|
||||||
def session(self) -> AsyncSession:
|
def session(self) -> AsyncSession:
|
||||||
"""Open a fresh :class:`AsyncSession` (use as ``async with``).
|
"""Open a fresh :class:`AsyncSession` (use as ``async with``).
|
||||||
@@ -90,6 +97,39 @@ class Database:
|
|||||||
await self._engine.dispose()
|
await self._engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
def _add_missing_columns(conn: Connection) -> None:
|
||||||
|
inspector = inspect(conn)
|
||||||
|
for table in SQLModel.metadata.sorted_tables:
|
||||||
|
existing = {c["name"] for c in inspector.get_columns(table.name)}
|
||||||
|
for column in table.columns:
|
||||||
|
if column.name in existing:
|
||||||
|
continue
|
||||||
|
kind = column.type.compile(conn.dialect)
|
||||||
|
ddl = f"ALTER TABLE {table.name} ADD COLUMN {column.name} {kind}"
|
||||||
|
default = _column_default(column)
|
||||||
|
if default is not None:
|
||||||
|
ddl += f" DEFAULT {default}"
|
||||||
|
conn.execute(text(ddl))
|
||||||
|
|
||||||
|
|
||||||
|
def _column_default(column: Any) -> str | None:
|
||||||
|
"""Literal for ``ADD COLUMN ... DEFAULT`` so old rows get the model default."""
|
||||||
|
server_default = getattr(column, "server_default", None)
|
||||||
|
if server_default is not None:
|
||||||
|
return str(server_default.arg.text)
|
||||||
|
default = getattr(column, "default", None)
|
||||||
|
if default is None or default.is_callable:
|
||||||
|
return None
|
||||||
|
value = default.arg
|
||||||
|
if isinstance(value, bool):
|
||||||
|
return "TRUE" if value else "FALSE"
|
||||||
|
if isinstance(value, int | float):
|
||||||
|
return repr(value)
|
||||||
|
if isinstance(value, str):
|
||||||
|
return "'" + value.replace("'", "''") + "'"
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
# ---- Token CRUD ---------------------------------------------------------
|
# ---- Token CRUD ---------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
@@ -183,13 +223,156 @@ async def list_audit_records(
|
|||||||
return result.all()
|
return result.all()
|
||||||
|
|
||||||
|
|
||||||
|
# ---- Usage --------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def append_usage(session: AsyncSession, row: Usage) -> None:
|
||||||
|
"""Persist a turn's usage with ``cost_usd`` / ``model_usage`` as per-turn deltas.
|
||||||
|
|
||||||
|
``row`` arrives with the SDK's cumulative ``total_cost_usd`` and
|
||||||
|
``model_usage`` in ``cost_usd`` / ``model_usage``; they move to the
|
||||||
|
``*_total`` columns and the stored values become the difference against
|
||||||
|
the previous row of the same session (see :func:`usage_deltas`).
|
||||||
|
"""
|
||||||
|
prev = await _last_usage(session, row.session_id)
|
||||||
|
usage_deltas(row, prev)
|
||||||
|
session.add(row)
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
|
||||||
|
async def _last_usage(session: AsyncSession, session_id: str | None) -> Usage | None:
|
||||||
|
if not session_id:
|
||||||
|
return None
|
||||||
|
stmt = (
|
||||||
|
select(Usage)
|
||||||
|
.where(col(Usage.session_id) == session_id)
|
||||||
|
.order_by(col(Usage.ts).desc(), col(Usage.id).desc())
|
||||||
|
.limit(1)
|
||||||
|
)
|
||||||
|
return (await session.exec(stmt)).first()
|
||||||
|
|
||||||
|
|
||||||
|
# Counters inside ``ResultMessage.model_usage[model]``; everything else there
|
||||||
|
# (provider, costBasis, contextWindow, maxOutputTokens, ...) is a constant.
|
||||||
|
_MODEL_USAGE_COUNTERS = (
|
||||||
|
"inputTokens",
|
||||||
|
"outputTokens",
|
||||||
|
"cacheReadInputTokens",
|
||||||
|
"cacheCreationInputTokens",
|
||||||
|
"webSearchRequests",
|
||||||
|
"costUSD",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def usage_deltas(row: Usage, prev: Usage | None) -> None:
|
||||||
|
"""Turn the SDK's cumulative cost counters on ``row`` into per-turn deltas.
|
||||||
|
|
||||||
|
``total_cost_usd`` and ``model_usage`` in a ``ResultMessage`` are running
|
||||||
|
totals of the claude process: they grow with every turn of the session and
|
||||||
|
start over when the process is respawned (rotation, resume after a
|
||||||
|
restart). A row fresh from the SDK (``cost_usd_total is None``) first moves
|
||||||
|
the raw values into ``cost_usd_total`` / ``model_usage_total``; then
|
||||||
|
``cost_usd`` / ``model_usage`` become ``current - previous`` against
|
||||||
|
``prev`` (the last row of the same session), or the raw value when there
|
||||||
|
is no previous row or the counter went backwards - a fresh process.
|
||||||
|
Idempotent on rows that already carry their totals, which is what the
|
||||||
|
backfill relies on.
|
||||||
|
"""
|
||||||
|
if row.cost_usd_total is None and row.model_usage_total is None:
|
||||||
|
row.cost_usd_total = row.cost_usd
|
||||||
|
row.model_usage_total = row.model_usage
|
||||||
|
prev_cost = prev.cost_usd_total if prev is not None else None
|
||||||
|
prev_models = prev.model_usage_total if prev is not None else None
|
||||||
|
row.cost_usd = _counter_delta(row.cost_usd_total, prev_cost)
|
||||||
|
row.model_usage = _model_usage_delta(row.model_usage_total, prev_models)
|
||||||
|
|
||||||
|
|
||||||
|
def _counter_delta(current: float | None, previous: float | None) -> float | None:
|
||||||
|
if current is None:
|
||||||
|
return None
|
||||||
|
if previous is None or current < previous:
|
||||||
|
return current
|
||||||
|
return round(current - previous, 6)
|
||||||
|
|
||||||
|
|
||||||
|
def _model_usage_delta(
|
||||||
|
current: dict[str, Any] | None, previous: dict[str, Any] | None
|
||||||
|
) -> dict[str, Any] | None:
|
||||||
|
if current is None:
|
||||||
|
return None
|
||||||
|
out: dict[str, Any] = {}
|
||||||
|
for model, fields in current.items():
|
||||||
|
if not isinstance(fields, dict):
|
||||||
|
out[model] = fields
|
||||||
|
continue
|
||||||
|
before = (previous or {}).get(model)
|
||||||
|
before = before if isinstance(before, dict) else {}
|
||||||
|
delta = dict(fields)
|
||||||
|
for key in _MODEL_USAGE_COUNTERS:
|
||||||
|
if key not in fields:
|
||||||
|
continue
|
||||||
|
cur, prev = fields.get(key), before.get(key)
|
||||||
|
if not isinstance(cur, int | float) or isinstance(cur, bool):
|
||||||
|
continue
|
||||||
|
prev_num = (
|
||||||
|
prev
|
||||||
|
if isinstance(prev, int | float) and not isinstance(prev, bool)
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
value = _counter_delta(cur, prev_num)
|
||||||
|
delta[key] = (
|
||||||
|
round(value or 0, 6) if isinstance(cur, float) else int(value or 0)
|
||||||
|
)
|
||||||
|
out[model] = delta
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
async def backfill_usage_deltas(session: AsyncSession) -> int:
|
||||||
|
"""Rewrite ``cost_usd`` / ``model_usage`` of every row as per-turn deltas.
|
||||||
|
|
||||||
|
Rows written before the ``*_total`` columns existed hold the SDK's
|
||||||
|
cumulative values in ``cost_usd`` / ``model_usage``; they are moved to the
|
||||||
|
totals first. Rows are replayed per session in time order. Idempotent:
|
||||||
|
a second run rewrites nothing. Returns the number of rows changed.
|
||||||
|
"""
|
||||||
|
stmt = select(Usage).order_by(col(Usage.session_id), col(Usage.ts), col(Usage.id))
|
||||||
|
rows = (await session.exec(stmt)).all()
|
||||||
|
last_by_session: dict[str, Usage] = {}
|
||||||
|
changed = 0
|
||||||
|
for row in rows:
|
||||||
|
before = (
|
||||||
|
row.cost_usd,
|
||||||
|
row.model_usage,
|
||||||
|
row.cost_usd_total,
|
||||||
|
row.model_usage_total,
|
||||||
|
)
|
||||||
|
prev = last_by_session.get(row.session_id) if row.session_id else None
|
||||||
|
usage_deltas(row, prev)
|
||||||
|
after = (
|
||||||
|
row.cost_usd,
|
||||||
|
row.model_usage,
|
||||||
|
row.cost_usd_total,
|
||||||
|
row.model_usage_total,
|
||||||
|
)
|
||||||
|
if after != before:
|
||||||
|
session.add(row)
|
||||||
|
changed += 1
|
||||||
|
if row.session_id:
|
||||||
|
last_by_session[row.session_id] = row
|
||||||
|
await session.commit()
|
||||||
|
return changed
|
||||||
|
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
"Database",
|
"Database",
|
||||||
"append_audit",
|
"append_audit",
|
||||||
|
"append_usage",
|
||||||
|
"backfill_usage_deltas",
|
||||||
"create_token",
|
"create_token",
|
||||||
"list_active_tokens",
|
"list_active_tokens",
|
||||||
"list_audit_records",
|
"list_audit_records",
|
||||||
"list_tokens",
|
"list_tokens",
|
||||||
"revoke_token",
|
"revoke_token",
|
||||||
"touch_token",
|
"touch_token",
|
||||||
|
"usage_deltas",
|
||||||
]
|
]
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
"""SQLModel tables.
|
"""SQLModel tables.
|
||||||
|
|
||||||
Four tables, all flat, no FK relationships modelled (``actor`` and
|
Flat tables, no FK relationships modelled (``actor`` and
|
||||||
``agent_name`` are stored as strings — joining audit→token by name is
|
``agent_name`` are stored as strings — joining audit→token by name is
|
||||||
fine at this volume; we'll introduce FKs when the admin UI actually
|
fine at this volume; we'll introduce FKs when the admin UI actually
|
||||||
demands them).
|
demands them).
|
||||||
@@ -23,8 +23,10 @@ needs an id uses ``Optional[int]`` so SQLAlchemy can autoincrement.
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
from sqlalchemy import UniqueConstraint
|
from sqlalchemy import JSON, Column, Index, UniqueConstraint, text
|
||||||
|
from sqlalchemy.dialects.postgresql import JSONB
|
||||||
from sqlmodel import Field, SQLModel
|
from sqlmodel import Field, SQLModel
|
||||||
|
|
||||||
|
|
||||||
@@ -72,15 +74,16 @@ class AuditLog(SQLModel, table=True):
|
|||||||
|
|
||||||
|
|
||||||
class Conversation(SQLModel, table=True):
|
class Conversation(SQLModel, table=True):
|
||||||
"""One chat thread, scoped to a frontend.
|
"""One conversation (§3.1): a master thread, a branch, a deep chat or a job.
|
||||||
|
|
||||||
``external_id`` is the identifier the frontend uses to find this
|
``external_id`` is the public id (uuid) every frontend, the API and the
|
||||||
thread again on the next request — for the markdown frontend it's a
|
usage table refer to; ``frontend`` names the frontend that created the
|
||||||
uuid we mint and persist into the file's frontmatter, for the
|
row (``markdown``, ``api``, ``mcp``, ``system``). Where a conversation
|
||||||
anthropic frontend it'd be the same metadata.conversation_id the
|
is *visible* lives in :class:`ConversationBinding`.
|
||||||
client passes. Unique per ``(frontend, external_id)`` because two
|
|
||||||
frontends sharing a uuid is fine; the same frontend reusing one is
|
``running_turn`` survives a restart: a non-null value at startup means
|
||||||
a bug.
|
the gateway died mid-turn and the transcript needs its open ``tool_use``
|
||||||
|
closed before the session is resumed (``core/conversations``).
|
||||||
"""
|
"""
|
||||||
|
|
||||||
__tablename__ = "conversations"
|
__tablename__ = "conversations"
|
||||||
@@ -92,8 +95,125 @@ class Conversation(SQLModel, table=True):
|
|||||||
frontend: str = Field(index=True)
|
frontend: str = Field(index=True)
|
||||||
external_id: str = Field(index=True)
|
external_id: str = Field(index=True)
|
||||||
agent_name: str = Field(index=True)
|
agent_name: str = Field(index=True)
|
||||||
|
session_id: str | None = Field(default=None, index=True)
|
||||||
|
kind: str = Field(default="deep", index=True)
|
||||||
|
parent_id: int | None = Field(default=None, index=True)
|
||||||
|
title: str | None = Field(default=None)
|
||||||
|
status: str = Field(default="open", index=True)
|
||||||
|
running_turn: str | None = Field(default=None)
|
||||||
|
pending_question: bool = Field(default=False)
|
||||||
|
flags: dict[str, Any] = Field(
|
||||||
|
default_factory=dict,
|
||||||
|
sa_column=Column(
|
||||||
|
JSON().with_variant(JSONB(), "postgresql"),
|
||||||
|
nullable=False,
|
||||||
|
server_default=text("'{}'"),
|
||||||
|
),
|
||||||
|
)
|
||||||
created_at: datetime = Field(default_factory=_utcnow)
|
created_at: datetime = Field(default_factory=_utcnow)
|
||||||
updated_at: datetime = Field(default_factory=_utcnow)
|
updated_at: datetime = Field(default_factory=_utcnow)
|
||||||
|
last_user_activity_at: datetime | None = Field(default=None)
|
||||||
|
last_activity_at: datetime | None = Field(default=None)
|
||||||
|
|
||||||
|
|
||||||
|
class ConversationBinding(SQLModel, table=True):
|
||||||
|
"""Where a conversation shows up: a Telegram topic id, a vault-relative path.
|
||||||
|
|
||||||
|
Invariant (§3.1): at most one *visible* binding per frontend per
|
||||||
|
conversation - enforced by the partial unique index. The same external
|
||||||
|
id may point at several conversations over time (a renamed topic gets a
|
||||||
|
new branch), only one of them visible.
|
||||||
|
"""
|
||||||
|
|
||||||
|
__tablename__ = "conversation_bindings"
|
||||||
|
__table_args__ = (
|
||||||
|
Index(
|
||||||
|
"uq_binding_visible",
|
||||||
|
"conversation_id",
|
||||||
|
"frontend",
|
||||||
|
unique=True,
|
||||||
|
postgresql_where=text("visible"),
|
||||||
|
sqlite_where=text("visible"),
|
||||||
|
),
|
||||||
|
Index("ix_binding_lookup", "frontend", "external_id"),
|
||||||
|
)
|
||||||
|
|
||||||
|
id: int | None = Field(default=None, primary_key=True)
|
||||||
|
conversation_id: int = Field(index=True)
|
||||||
|
frontend: str
|
||||||
|
external_id: str
|
||||||
|
visible: bool = Field(default=True)
|
||||||
|
created_at: datetime = Field(default_factory=_utcnow)
|
||||||
|
|
||||||
|
|
||||||
|
class InjectQueueItem(SQLModel, table=True):
|
||||||
|
"""Persisted per-conversation queue (§3.4), ``urgent > user > wake > normal``.
|
||||||
|
|
||||||
|
``status`` walks ``queued -> running -> done``; a row still ``running``
|
||||||
|
at startup was cut by a restart and becomes ``interrupted`` - it is
|
||||||
|
never re-run, the agent gets an inject saying so instead.
|
||||||
|
"""
|
||||||
|
|
||||||
|
__tablename__ = "inject_queue"
|
||||||
|
|
||||||
|
id: int | None = Field(default=None, primary_key=True)
|
||||||
|
conversation_id: int = Field(index=True)
|
||||||
|
priority: str = Field(index=True)
|
||||||
|
origin: str = Field(default="system")
|
||||||
|
text: str
|
||||||
|
status: str = Field(default="queued", index=True)
|
||||||
|
turn_id: str | None = Field(default=None)
|
||||||
|
interrupted_turn: bool | None = Field(default=None)
|
||||||
|
"""An urgent inject that cut a running turn; the header tells the model
|
||||||
|
the ``Request interrupted`` tool result above was this, not a refusal."""
|
||||||
|
created_at: datetime = Field(default_factory=_utcnow)
|
||||||
|
delivered_at: datetime | None = Field(default=None)
|
||||||
|
|
||||||
|
|
||||||
|
class TelegramUpdate(SQLModel, table=True):
|
||||||
|
"""Inbox of the Telegram frontend (§3.8).
|
||||||
|
|
||||||
|
The update is stored before the poll offset moves past it and handled
|
||||||
|
from here, so a restart neither loses nor duplicates a message.
|
||||||
|
"""
|
||||||
|
|
||||||
|
__tablename__ = "telegram_updates"
|
||||||
|
|
||||||
|
update_id: int = Field(primary_key=True, sa_column_kwargs={"autoincrement": False})
|
||||||
|
payload: dict[str, Any] = Field(
|
||||||
|
sa_column=Column(JSON().with_variant(JSONB(), "postgresql"), nullable=False)
|
||||||
|
)
|
||||||
|
received_at: datetime = Field(default_factory=_utcnow)
|
||||||
|
processed_at: datetime | None = Field(default=None, index=True)
|
||||||
|
error: str | None = Field(default=None)
|
||||||
|
|
||||||
|
|
||||||
|
class Delivery(SQLModel, table=True):
|
||||||
|
"""Outbox (§3.8): a reply is a row first and a Telegram message second.
|
||||||
|
|
||||||
|
``status`` walks ``queued -> sent`` with retries on the way, ``failed``
|
||||||
|
only when Telegram rejects the row for good (unknown thread, blocked
|
||||||
|
bot). ``dedupe_key`` keeps one row per turn chunk across restarts.
|
||||||
|
"""
|
||||||
|
|
||||||
|
__tablename__ = "deliveries"
|
||||||
|
|
||||||
|
id: int | None = Field(default=None, primary_key=True)
|
||||||
|
conversation_id: int | None = Field(default=None, index=True)
|
||||||
|
frontend: str = Field(default="telegram", index=True)
|
||||||
|
chat_id: int
|
||||||
|
thread_id: int | None = Field(default=None)
|
||||||
|
text: str
|
||||||
|
plain: bool = Field(default=False)
|
||||||
|
turn_id: str | None = Field(default=None)
|
||||||
|
dedupe_key: str | None = Field(default=None, unique=True)
|
||||||
|
status: str = Field(default="queued", index=True)
|
||||||
|
attempts: int = Field(default=0)
|
||||||
|
next_attempt_at: datetime = Field(default_factory=_utcnow, index=True)
|
||||||
|
last_error: str | None = Field(default=None)
|
||||||
|
message_id: int | None = Field(default=None)
|
||||||
|
created_at: datetime = Field(default_factory=_utcnow)
|
||||||
|
sent_at: datetime | None = Field(default=None)
|
||||||
|
|
||||||
|
|
||||||
class ConversationMessage(SQLModel, table=True):
|
class ConversationMessage(SQLModel, table=True):
|
||||||
@@ -125,4 +245,131 @@ class ConversationMessage(SQLModel, table=True):
|
|||||||
created_at: datetime = Field(default_factory=_utcnow)
|
created_at: datetime = Field(default_factory=_utcnow)
|
||||||
|
|
||||||
|
|
||||||
__all__ = ["AuditLog", "Conversation", "ConversationMessage", "Token"]
|
class TranscriptEntry(SQLModel, table=True):
|
||||||
|
"""One Agent SDK transcript line, mirrored from the session store protocol.
|
||||||
|
|
||||||
|
``(project_key, session_id, subpath)`` is the ``SessionKey``; ``subpath``
|
||||||
|
is ``""`` for the main transcript so the unique constraints stay simple.
|
||||||
|
``uuid`` is the SDK's idempotency key - the partial unique index rejects
|
||||||
|
a replayed batch, entries without a uuid are appended as-is.
|
||||||
|
"""
|
||||||
|
|
||||||
|
__tablename__ = "transcript_entries"
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint(
|
||||||
|
"project_key", "session_id", "subpath", "seq", name="uq_transcript_seq"
|
||||||
|
),
|
||||||
|
Index(
|
||||||
|
"uq_transcript_uuid",
|
||||||
|
"project_key",
|
||||||
|
"session_id",
|
||||||
|
"subpath",
|
||||||
|
"uuid",
|
||||||
|
unique=True,
|
||||||
|
postgresql_where=text("uuid IS NOT NULL"),
|
||||||
|
sqlite_where=text("uuid IS NOT NULL"),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
id: int | None = Field(default=None, primary_key=True)
|
||||||
|
project_key: str = Field(index=True)
|
||||||
|
session_id: str = Field(index=True)
|
||||||
|
subpath: str = Field(default="")
|
||||||
|
seq: int
|
||||||
|
uuid: str | None = Field(default=None)
|
||||||
|
entry: dict[str, Any] = Field(
|
||||||
|
sa_column=Column(JSON().with_variant(JSONB(), "postgresql"), nullable=False)
|
||||||
|
)
|
||||||
|
origin: str | None = Field(default=None)
|
||||||
|
source: str | None = Field(default=None)
|
||||||
|
turn_id: str | None = Field(default=None)
|
||||||
|
created_at: datetime = Field(default_factory=_utcnow)
|
||||||
|
|
||||||
|
|
||||||
|
class Usage(SQLModel, table=True):
|
||||||
|
"""Per-turn token accounting from ``ResultMessage``.
|
||||||
|
|
||||||
|
Token columns come from ``ResultMessage.usage`` and are sums over the
|
||||||
|
turn's API calls. ``cost_usd`` and ``model_usage`` are **per-turn deltas**:
|
||||||
|
the SDK reports ``total_cost_usd`` and ``model_usage`` cumulatively for
|
||||||
|
the claude process (growing across the turns of a session, reset when
|
||||||
|
the process is respawned), so :func:`append_usage` stores the raw values
|
||||||
|
in ``cost_usd_total`` / ``model_usage_total`` and the difference against
|
||||||
|
the previous row of the same session here. Summing ``cost_usd`` over rows
|
||||||
|
is therefore meaningful; summing ``cost_usd_total`` is not.
|
||||||
|
"""
|
||||||
|
|
||||||
|
__tablename__ = "usage"
|
||||||
|
|
||||||
|
id: int | None = Field(default=None, primary_key=True)
|
||||||
|
ts: datetime = Field(default_factory=_utcnow, index=True)
|
||||||
|
agent_name: str = Field(index=True)
|
||||||
|
conversation_id: str | None = Field(default=None, index=True)
|
||||||
|
session_id: str | None = Field(default=None, index=True)
|
||||||
|
model: str
|
||||||
|
effort: str | None = Field(default=None)
|
||||||
|
input_tokens: int = 0
|
||||||
|
output_tokens: int = 0
|
||||||
|
cache_read_tokens: int = 0
|
||||||
|
cache_creation_tokens: int = 0
|
||||||
|
context_tokens: int | None = Field(default=None)
|
||||||
|
"""Input of the turn's last API call - the real context size."""
|
||||||
|
cost_usd: float | None = Field(default=None)
|
||||||
|
duration_ms: int | None = Field(default=None)
|
||||||
|
num_turns: int | None = Field(default=None)
|
||||||
|
model_usage: dict[str, Any] | None = Field(
|
||||||
|
default=None,
|
||||||
|
sa_column=Column(JSON().with_variant(JSONB(), "postgresql"), nullable=True),
|
||||||
|
)
|
||||||
|
"""Per-model tokens, cost and web searches - per-turn delta (see class doc)."""
|
||||||
|
cost_usd_total: float | None = Field(default=None)
|
||||||
|
"""``ResultMessage.total_cost_usd`` verbatim: cumulative for the claude process."""
|
||||||
|
model_usage_total: dict[str, Any] | None = Field(
|
||||||
|
default=None,
|
||||||
|
sa_column=Column(JSON().with_variant(JSONB(), "postgresql"), nullable=True),
|
||||||
|
)
|
||||||
|
"""``ResultMessage.model_usage`` verbatim: cumulative for the claude process."""
|
||||||
|
|
||||||
|
|
||||||
|
class RateLimit(SQLModel, table=True):
|
||||||
|
"""One ``RateLimitEvent`` from the SDK; the newest row per ``window`` is the state.
|
||||||
|
|
||||||
|
The quota is per subscription, not per agent or conversation - the row
|
||||||
|
only remembers which session reported it.
|
||||||
|
"""
|
||||||
|
|
||||||
|
__tablename__ = "rate_limits"
|
||||||
|
|
||||||
|
id: int | None = Field(default=None, primary_key=True)
|
||||||
|
ts: datetime = Field(default_factory=_utcnow, index=True)
|
||||||
|
window: str = Field(index=True)
|
||||||
|
status: str
|
||||||
|
utilization: float | None = Field(default=None)
|
||||||
|
resets_at: datetime | None = Field(default=None)
|
||||||
|
overage_status: str | None = Field(default=None)
|
||||||
|
overage_resets_at: datetime | None = Field(default=None)
|
||||||
|
agent_name: str | None = Field(default=None)
|
||||||
|
session_id: str | None = Field(default=None)
|
||||||
|
raw: dict[str, Any] = Field(
|
||||||
|
default_factory=dict,
|
||||||
|
sa_column=Column(
|
||||||
|
JSON().with_variant(JSONB(), "postgresql"),
|
||||||
|
nullable=False,
|
||||||
|
server_default=text("'{}'"),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"AuditLog",
|
||||||
|
"Conversation",
|
||||||
|
"ConversationBinding",
|
||||||
|
"ConversationMessage",
|
||||||
|
"Delivery",
|
||||||
|
"InjectQueueItem",
|
||||||
|
"RateLimit",
|
||||||
|
"TelegramUpdate",
|
||||||
|
"Token",
|
||||||
|
"TranscriptEntry",
|
||||||
|
"Usage",
|
||||||
|
]
|
||||||
@@ -0,0 +1,126 @@
|
|||||||
|
"""``SessionStore`` adapter for the Claude Agent SDK on top of :class:`Database`.
|
||||||
|
|
||||||
|
Entries are stored verbatim as JSON (``jsonb`` on Postgres), ordered by a
|
||||||
|
per-key ``seq``. ``append`` is idempotent on ``entry["uuid"]`` because the
|
||||||
|
SDK re-delivers a batch on retry; entries without a uuid are appended as
|
||||||
|
they come. Runs on SQLite too - the conformance suite uses that in tests.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import TYPE_CHECKING, Any, cast
|
||||||
|
|
||||||
|
from claude_agent_sdk import SessionStore
|
||||||
|
from sqlalchemy import delete, func
|
||||||
|
from sqlmodel import col, select
|
||||||
|
|
||||||
|
from beaver_gateway.storage.models import TranscriptEntry
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from claude_agent_sdk.types import (
|
||||||
|
SessionKey,
|
||||||
|
SessionListSubkeysKey,
|
||||||
|
SessionStoreEntry,
|
||||||
|
)
|
||||||
|
|
||||||
|
from beaver_gateway.storage.db import Database
|
||||||
|
|
||||||
|
__all__ = ["PostgresSessionStore"]
|
||||||
|
|
||||||
|
|
||||||
|
class PostgresSessionStore(SessionStore):
|
||||||
|
def __init__(self, db: Database) -> None:
|
||||||
|
self._db = db
|
||||||
|
|
||||||
|
async def append(self, key: SessionKey, entries: list[SessionStoreEntry]) -> None:
|
||||||
|
if not entries:
|
||||||
|
return
|
||||||
|
project_key, session_id, subpath = _parts(key)
|
||||||
|
wanted = [
|
||||||
|
u for u in (e.get("uuid") for e in entries) if isinstance(u, str) and u
|
||||||
|
]
|
||||||
|
async with self._db.session() as session:
|
||||||
|
known: set[str] = set()
|
||||||
|
if wanted:
|
||||||
|
result = await session.exec(
|
||||||
|
select(TranscriptEntry.uuid).where(
|
||||||
|
TranscriptEntry.project_key == project_key,
|
||||||
|
TranscriptEntry.session_id == session_id,
|
||||||
|
TranscriptEntry.subpath == subpath,
|
||||||
|
col(TranscriptEntry.uuid).in_(wanted),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
known = {u for u in result.all() if u is not None}
|
||||||
|
seq_result = await session.exec(
|
||||||
|
select(func.max(TranscriptEntry.seq)).where(
|
||||||
|
TranscriptEntry.project_key == project_key,
|
||||||
|
TranscriptEntry.session_id == session_id,
|
||||||
|
TranscriptEntry.subpath == subpath,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
seq = seq_result.one() or 0
|
||||||
|
for entry in entries:
|
||||||
|
uuid = entry.get("uuid")
|
||||||
|
uuid_str = uuid if isinstance(uuid, str) and uuid else None
|
||||||
|
if uuid_str is not None:
|
||||||
|
if uuid_str in known:
|
||||||
|
continue
|
||||||
|
known.add(uuid_str)
|
||||||
|
seq += 1
|
||||||
|
session.add(
|
||||||
|
TranscriptEntry(
|
||||||
|
project_key=project_key,
|
||||||
|
session_id=session_id,
|
||||||
|
subpath=subpath,
|
||||||
|
seq=seq,
|
||||||
|
uuid=uuid_str,
|
||||||
|
entry=cast("dict[str, Any]", dict(entry)),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
async def load(self, key: SessionKey) -> list[SessionStoreEntry] | None:
|
||||||
|
project_key, session_id, subpath = _parts(key)
|
||||||
|
async with self._db.session() as session:
|
||||||
|
result = await session.exec(
|
||||||
|
select(TranscriptEntry.entry)
|
||||||
|
.where(
|
||||||
|
TranscriptEntry.project_key == project_key,
|
||||||
|
TranscriptEntry.session_id == session_id,
|
||||||
|
TranscriptEntry.subpath == subpath,
|
||||||
|
)
|
||||||
|
.order_by(col(TranscriptEntry.seq))
|
||||||
|
)
|
||||||
|
rows = result.all()
|
||||||
|
if not rows:
|
||||||
|
return None
|
||||||
|
return [cast("SessionStoreEntry", row) for row in rows]
|
||||||
|
|
||||||
|
async def list_subkeys(self, key: SessionListSubkeysKey) -> list[str]:
|
||||||
|
async with self._db.session() as session:
|
||||||
|
result = await session.exec(
|
||||||
|
select(TranscriptEntry.subpath)
|
||||||
|
.where(
|
||||||
|
TranscriptEntry.project_key == key["project_key"],
|
||||||
|
TranscriptEntry.session_id == key["session_id"],
|
||||||
|
TranscriptEntry.subpath != "",
|
||||||
|
)
|
||||||
|
.distinct()
|
||||||
|
)
|
||||||
|
return sorted(result.all())
|
||||||
|
|
||||||
|
async def delete(self, key: SessionKey) -> None:
|
||||||
|
project_key, session_id, subpath = _parts(key)
|
||||||
|
stmt = delete(TranscriptEntry).where(
|
||||||
|
col(TranscriptEntry.project_key) == project_key,
|
||||||
|
col(TranscriptEntry.session_id) == session_id,
|
||||||
|
)
|
||||||
|
if key.get("subpath"):
|
||||||
|
stmt = stmt.where(col(TranscriptEntry.subpath) == subpath)
|
||||||
|
async with self._db.session() as session:
|
||||||
|
await session.execute(stmt) # ty: ignore[deprecated]
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
|
||||||
|
def _parts(key: SessionKey) -> tuple[str, str, str]:
|
||||||
|
return key["project_key"], key["session_id"], key.get("subpath") or ""
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
"""Print a TEST_DATABASE_URL for the compose postgres (`make db`) from beaver-agent/.env."""
|
||||||
|
|
||||||
|
from dotenv import dotenv_values
|
||||||
|
|
||||||
|
v = dotenv_values("../beaver-agent/.env")
|
||||||
|
user = v.get("POSTGRES_USER") or "beaver"
|
||||||
|
port = v.get("PORT_POSTGRES") or "5432"
|
||||||
|
db = v.get("POSTGRES_DB") or "beaver"
|
||||||
|
print(f"postgresql://{user}:{v['POSTGRES_PASSWORD']}@127.0.0.1:{port}/{db}")
|
||||||
@@ -0,0 +1,492 @@
|
|||||||
|
import asyncio
|
||||||
|
import tempfile
|
||||||
|
from datetime import UTC, datetime, timedelta
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, cast
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from claude_agent_sdk import (
|
||||||
|
AssistantMessage,
|
||||||
|
RateLimitEvent,
|
||||||
|
RateLimitInfo,
|
||||||
|
ResultMessage,
|
||||||
|
TextBlock,
|
||||||
|
ToolResultBlock,
|
||||||
|
ToolUseBlock,
|
||||||
|
UserMessage,
|
||||||
|
)
|
||||||
|
from httpx import ASGITransport, AsyncClient
|
||||||
|
from test_conversations import ScriptedClient, World
|
||||||
|
|
||||||
|
from beaver_gateway.core.conversation_store import rewrite_messages
|
||||||
|
from beaver_gateway.core.auth import TokenStore
|
||||||
|
from beaver_gateway.core.registry import McpRegistry
|
||||||
|
from beaver_gateway.core.transcript import build_entries
|
||||||
|
from beaver_gateway.frontends.admin import AdminFrontend
|
||||||
|
from beaver_gateway.frontends.admin.frontend import build_app as build_admin
|
||||||
|
from beaver_gateway.frontends.api import ApiFrontend
|
||||||
|
from beaver_gateway.frontends.api.frontend import build_app as build_api
|
||||||
|
from beaver_gateway.frontends.base import Frontend, GatewayRuntime
|
||||||
|
from beaver_gateway.frontends.root import build_root_app
|
||||||
|
from beaver_gateway.storage.models import RateLimit, Usage
|
||||||
|
|
||||||
|
TOKEN = "tok"
|
||||||
|
HEADERS = {"Authorization": f"Bearer {TOKEN}"}
|
||||||
|
|
||||||
|
|
||||||
|
class ToolClient(ScriptedClient):
|
||||||
|
async def receive_response(self):
|
||||||
|
yield AssistantMessage(
|
||||||
|
content=[ToolUseBlock(id="tu_1", name="Bash", input={"command": "ls"})],
|
||||||
|
model="m",
|
||||||
|
)
|
||||||
|
yield AssistantMessage(
|
||||||
|
content=[ToolUseBlock(id="tu_2", name="Read", input={"file_path": "x"})],
|
||||||
|
model="m",
|
||||||
|
parent_tool_use_id="tu_1",
|
||||||
|
)
|
||||||
|
yield RateLimitEvent(
|
||||||
|
rate_limit_info=RateLimitInfo(
|
||||||
|
status="allowed_warning",
|
||||||
|
resets_at=int(datetime.now(UTC).timestamp()) + 3600,
|
||||||
|
rate_limit_type="five_hour",
|
||||||
|
utilization=0.8,
|
||||||
|
raw={"status": "allowed_warning"},
|
||||||
|
),
|
||||||
|
uuid="r",
|
||||||
|
session_id=self.session_id,
|
||||||
|
)
|
||||||
|
hold = ScriptedClient.hold
|
||||||
|
if hold is not None:
|
||||||
|
await hold.wait()
|
||||||
|
yield UserMessage(content=[ToolResultBlock(tool_use_id="tu_1", content="done")])
|
||||||
|
yield AssistantMessage(content=[TextBlock(text="ok")], model="m")
|
||||||
|
yield ResultMessage(
|
||||||
|
subtype="success",
|
||||||
|
duration_ms=1,
|
||||||
|
duration_api_ms=1,
|
||||||
|
is_error=False,
|
||||||
|
num_turns=1,
|
||||||
|
session_id=self.session_id,
|
||||||
|
stop_reason="end_turn",
|
||||||
|
total_cost_usd=0.5,
|
||||||
|
usage={"input_tokens": 10, "output_tokens": 5},
|
||||||
|
model_usage=cast(
|
||||||
|
"Any",
|
||||||
|
{
|
||||||
|
"claude-opus-5": {
|
||||||
|
"inputTokens": 8,
|
||||||
|
"outputTokens": 4,
|
||||||
|
"cacheReadInputTokens": 0,
|
||||||
|
"cacheCreationInputTokens": 0,
|
||||||
|
"webSearchRequests": 1,
|
||||||
|
"costUSD": 0.4,
|
||||||
|
},
|
||||||
|
"claude-haiku-4-5": {
|
||||||
|
"inputTokens": 2,
|
||||||
|
"outputTokens": 1,
|
||||||
|
"cacheReadInputTokens": 0,
|
||||||
|
"cacheCreationInputTokens": 0,
|
||||||
|
"webSearchRequests": 0,
|
||||||
|
"costUSD": 0.1,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class Api:
|
||||||
|
def __init__(self, world: World, memory_root: Path | None = None) -> None:
|
||||||
|
self.world = world
|
||||||
|
self.store = TokenStore(world.db, bootstrap={"t": TOKEN})
|
||||||
|
self.runtime = GatewayRuntime(
|
||||||
|
agents=world.conversations._agents, # noqa: SLF001
|
||||||
|
mcps=McpRegistry([]),
|
||||||
|
backends={"a": world.backend, "d": world.deep_backend},
|
||||||
|
token_store=self.store,
|
||||||
|
db=world.db,
|
||||||
|
admin_user="admin",
|
||||||
|
admin_pass="secret",
|
||||||
|
session_secret="s" * 32,
|
||||||
|
frontends=(world.api, world.markdown),
|
||||||
|
conversations=world.conversations,
|
||||||
|
bus=world.bus,
|
||||||
|
pool=world.pool,
|
||||||
|
)
|
||||||
|
self.app = build_api(self.runtime, memory_root=memory_root)
|
||||||
|
self.http = AsyncClient(
|
||||||
|
transport=ASGITransport(app=self.app), base_url="http://api"
|
||||||
|
)
|
||||||
|
|
||||||
|
def root(self, *frontends: Frontend) -> AsyncClient:
|
||||||
|
return AsyncClient(
|
||||||
|
transport=ASGITransport(app=build_root_app(frontends)), base_url="http://gw"
|
||||||
|
)
|
||||||
|
|
||||||
|
async def get(self, path: str, params: dict[str, Any] | None = None) -> Any:
|
||||||
|
res = await self.http.get(path, params=params, headers=HEADERS)
|
||||||
|
assert res.status_code == 200, res.text
|
||||||
|
return res.json()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
async def world() -> World:
|
||||||
|
root = Path(tempfile.mkdtemp(prefix="beaver-api-"))
|
||||||
|
w = await World(root).setup()
|
||||||
|
yield w
|
||||||
|
await w.conversations.stop()
|
||||||
|
await w.pool.close_all()
|
||||||
|
await w.db.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
async def seed_usage(world: World, rows: list[dict[str, Any]]) -> None:
|
||||||
|
async with world.db.session() as session:
|
||||||
|
for row in rows:
|
||||||
|
session.add(Usage(**row))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_usage_groups_by_agent_day_and_model(world: World) -> None:
|
||||||
|
api = Api(world)
|
||||||
|
now = datetime.now(UTC).replace(tzinfo=None)
|
||||||
|
await seed_usage(
|
||||||
|
world,
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"ts": now - timedelta(hours=1),
|
||||||
|
"agent_name": "a",
|
||||||
|
"conversation_id": "c1",
|
||||||
|
"model": "claude-opus-5",
|
||||||
|
"input_tokens": 10,
|
||||||
|
"output_tokens": 20,
|
||||||
|
"cache_read_tokens": 100,
|
||||||
|
"cache_creation_tokens": 5,
|
||||||
|
"cost_usd": 0.5,
|
||||||
|
"model_usage": {
|
||||||
|
"claude-opus-5": {
|
||||||
|
"inputTokens": 10,
|
||||||
|
"outputTokens": 20,
|
||||||
|
"cacheReadInputTokens": 100,
|
||||||
|
"cacheCreationInputTokens": 5,
|
||||||
|
"webSearchRequests": 2,
|
||||||
|
"costUSD": 0.5,
|
||||||
|
}
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"ts": now - timedelta(days=2),
|
||||||
|
"agent_name": "d",
|
||||||
|
"conversation_id": "c2",
|
||||||
|
"model": "claude-sonnet-5",
|
||||||
|
"input_tokens": 1,
|
||||||
|
"output_tokens": 2,
|
||||||
|
"cost_usd": 0.1,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
)
|
||||||
|
by_agent = await api.get("/usage", {"group_by": "agent"})
|
||||||
|
assert [r["agent"] for r in by_agent["rows"]] == ["a"]
|
||||||
|
assert by_agent["total"] == {
|
||||||
|
"turns": 1,
|
||||||
|
"input": 10,
|
||||||
|
"output": 20,
|
||||||
|
"cache_read": 100,
|
||||||
|
"cache_creation": 5,
|
||||||
|
"cost_usd": 0.5,
|
||||||
|
"web_searches": 2,
|
||||||
|
}
|
||||||
|
since = (datetime.now(UTC) - timedelta(days=3)).isoformat()
|
||||||
|
by_day = await api.get("/usage", {"group_by": "day", "since": since})
|
||||||
|
assert len(by_day["rows"]) == 2
|
||||||
|
assert by_day["rows"][0]["day"] < by_day["rows"][1]["day"]
|
||||||
|
by_model = await api.get("/usage", {"group_by": "model", "since": since})
|
||||||
|
models = {r["model"]: r for r in by_model["rows"]}
|
||||||
|
assert models["claude-opus-5"]["web_searches"] == 2
|
||||||
|
assert models["claude-sonnet-5"]["output"] == 2
|
||||||
|
by_conv = await api.get("/usage", {"group_by": "conversation", "since": since})
|
||||||
|
assert {r["conversation"] for r in by_conv["rows"]} == {"c1", "c2"}
|
||||||
|
bad = await api.http.get("/usage", params={"group_by": "x"}, headers=HEADERS)
|
||||||
|
assert bad.status_code == 400
|
||||||
|
|
||||||
|
|
||||||
|
async def test_limits_report_latest_window_with_gateway_spend(world: World) -> None:
|
||||||
|
api = Api(world)
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
async with world.db.session() as session:
|
||||||
|
session.add(
|
||||||
|
RateLimit(
|
||||||
|
window="five_hour",
|
||||||
|
status="allowed",
|
||||||
|
utilization=0.2,
|
||||||
|
resets_at=now + timedelta(hours=4),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
session.add(
|
||||||
|
RateLimit(
|
||||||
|
window="five_hour",
|
||||||
|
status="allowed_warning",
|
||||||
|
utilization=0.9,
|
||||||
|
resets_at=now + timedelta(hours=4),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
session.add(RateLimit(window="seven_day", status="allowed", utilization=0.3))
|
||||||
|
await session.commit()
|
||||||
|
await seed_usage(
|
||||||
|
world,
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"ts": (now - timedelta(minutes=30)).replace(tzinfo=None),
|
||||||
|
"agent_name": "a",
|
||||||
|
"model": "m",
|
||||||
|
"output_tokens": 7,
|
||||||
|
"cost_usd": 0.25,
|
||||||
|
}
|
||||||
|
],
|
||||||
|
)
|
||||||
|
out = await api.get("/limits")
|
||||||
|
windows = {w["window"]: w for w in out["windows"]}
|
||||||
|
assert windows["five_hour"]["utilization"] == 0.9
|
||||||
|
assert windows["five_hour"]["status"] == "allowed_warning"
|
||||||
|
assert windows["five_hour"]["gateway"]["output"] == 7
|
||||||
|
assert windows["five_hour"]["gateway"]["cost_usd"] == 0.25
|
||||||
|
assert windows["seven_day"]["utilization"] == 0.3
|
||||||
|
assert [w["window"] for w in out["windows"]] == ["five_hour", "seven_day"]
|
||||||
|
assert len(out["history"]) == 3
|
||||||
|
|
||||||
|
|
||||||
|
async def test_memory_tree_and_file(world: World) -> None:
|
||||||
|
root = world.root / "zone"
|
||||||
|
(root / "дни").mkdir(parents=True)
|
||||||
|
(root / "дни" / "2026-08-27.md").write_text("# day", encoding="utf-8")
|
||||||
|
(root / "состояние.md").write_text("# state", encoding="utf-8")
|
||||||
|
(root / ".hidden").write_text("x", encoding="utf-8")
|
||||||
|
api = Api(world, memory_root=root)
|
||||||
|
tree = await api.get("/memory")
|
||||||
|
names = [n["name"] for n in tree["tree"]]
|
||||||
|
assert names == ["дни", "состояние.md"]
|
||||||
|
assert tree["tree"][0]["children"][0]["path"] == "дни/2026-08-27.md"
|
||||||
|
file = await api.get("/memory/file", {"path": "дни/2026-08-27.md"})
|
||||||
|
assert file["content"] == "# day"
|
||||||
|
escape = await api.http.get(
|
||||||
|
"/memory/file", params={"path": "../w.db"}, headers=HEADERS
|
||||||
|
)
|
||||||
|
assert escape.status_code == 404
|
||||||
|
unset = Api(world)
|
||||||
|
res = await unset.http.get("/memory", headers=HEADERS)
|
||||||
|
assert res.status_code == 404
|
||||||
|
|
||||||
|
|
||||||
|
async def test_describe_snapshots_open_tools_and_records_rate_limit(
|
||||||
|
world: World,
|
||||||
|
) -> None:
|
||||||
|
world.backend._factory = ToolClient # noqa: SLF001
|
||||||
|
api = Api(world)
|
||||||
|
|
||||||
|
async def record(event: Any) -> None:
|
||||||
|
async with world.db.session() as session:
|
||||||
|
session.add(
|
||||||
|
Usage(
|
||||||
|
agent_name=event.agent_name,
|
||||||
|
conversation_id=event.conversation_id,
|
||||||
|
model=event.model,
|
||||||
|
cost_usd=event.usage.cost_usd,
|
||||||
|
model_usage=event.usage.model_usage,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
world.backend._usage_sink = record # noqa: SLF001
|
||||||
|
conv = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
ScriptedClient.hold = asyncio.Event()
|
||||||
|
await world.conversations.post(conv, "go")
|
||||||
|
await asyncio.sleep(0.3)
|
||||||
|
described = await api.get(f"/conversations/{conv.external_id}")
|
||||||
|
turn = described["turn"]
|
||||||
|
assert turn is not None and turn["id"] == described["running_turn"]
|
||||||
|
tools = {t["tool_use_id"]: t for t in turn["tools"]}
|
||||||
|
assert tools["tu_1"]["name"] == "Bash" and tools["tu_1"]["ended_at"] is None
|
||||||
|
assert tools["tu_2"]["parent_tool_use_id"] == "tu_1"
|
||||||
|
ScriptedClient.hold.set()
|
||||||
|
await world.settle(conv, 1)
|
||||||
|
described = await api.get(f"/conversations/{conv.external_id}")
|
||||||
|
assert described["turn"] is None
|
||||||
|
limits = await api.get("/limits")
|
||||||
|
assert limits["windows"][0]["utilization"] == 0.8
|
||||||
|
assert limits["windows"][0]["gateway"]["cost_usd"] == 0.5
|
||||||
|
usage = await api.get("/usage", {"group_by": "model"})
|
||||||
|
assert {r["model"] for r in usage["rows"]} == {"claude-opus-5", "claude-haiku-4-5"}
|
||||||
|
assert usage["total"]["web_searches"] == 1
|
||||||
|
conv = await world.conversations.get(conv.external_id)
|
||||||
|
assert conv is not None and conv.session_id is not None
|
||||||
|
await world.store.append(
|
||||||
|
world.key(conv.session_id),
|
||||||
|
build_entries(
|
||||||
|
[
|
||||||
|
{"role": "user", "content": "go"},
|
||||||
|
{"role": "assistant", "content": [{"type": "text", "text": "ok"}]},
|
||||||
|
],
|
||||||
|
session_id=conv.session_id,
|
||||||
|
cwd=str(world.root),
|
||||||
|
model="m",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
history = await api.get(f"/conversations/{conv.external_id}/history")
|
||||||
|
assert [m["role"] for m in history["messages"]] == ["user", "assistant"]
|
||||||
|
entries = await api.get(f"/conversations/{conv.external_id}/entries")
|
||||||
|
assert entries["total"] == len(entries["entries"]) == 2
|
||||||
|
page = await api.get(
|
||||||
|
f"/conversations/{conv.external_id}/entries", {"limit": 1, "offset": 0}
|
||||||
|
)
|
||||||
|
assert page["offset"] == 0 and len(page["entries"]) == 1
|
||||||
|
|
||||||
|
|
||||||
|
async def test_tokens_and_audit_need_admin_scope(world: World) -> None:
|
||||||
|
api = Api(world)
|
||||||
|
api.store.grant("api-only", "weak", scope="api")
|
||||||
|
weak = {"Authorization": "Bearer weak"}
|
||||||
|
assert (await api.http.get("/tokens", headers=weak)).status_code == 403
|
||||||
|
created = await api.http.post(
|
||||||
|
"/tokens", json={"name": "cursor", "scope": "mcp"}, headers=HEADERS
|
||||||
|
)
|
||||||
|
assert created.status_code == 201
|
||||||
|
plaintext = created.json()["plaintext"]
|
||||||
|
await api.store.invalidate()
|
||||||
|
identity = await api.store.verify(plaintext)
|
||||||
|
assert identity is not None and identity.scope == "mcp"
|
||||||
|
listed = await api.get("/tokens")
|
||||||
|
assert [t["name"] for t in listed["tokens"]] == ["cursor"]
|
||||||
|
token_id = listed["tokens"][0]["id"]
|
||||||
|
revoked = await api.http.post(f"/tokens/{token_id}/revoke", headers=HEADERS)
|
||||||
|
assert revoked.status_code == 200
|
||||||
|
assert (await api.get("/tokens"))["tokens"] == []
|
||||||
|
audit = await api.get("/audit")
|
||||||
|
assert [r["kind"] for r in audit["records"]] == ["token_revoke", "token_create"]
|
||||||
|
|
||||||
|
|
||||||
|
async def test_admin_login_hands_out_the_ui_bearer(world: World) -> None:
|
||||||
|
api = Api(world)
|
||||||
|
ui_dir = world.root / "build"
|
||||||
|
ui_dir.mkdir()
|
||||||
|
(ui_dir / "index.html").write_text("<html>ui</html>", encoding="utf-8")
|
||||||
|
admin = AdminFrontend(ui_dir=ui_dir)
|
||||||
|
admin._app = build_admin(api.runtime, token="ui-bearer", ui_dir=ui_dir) # noqa: SLF001
|
||||||
|
api.store.grant("admin-ui", "ui-bearer")
|
||||||
|
api_fe = ApiFrontend()
|
||||||
|
api_fe._app = api.app # noqa: SLF001
|
||||||
|
http = api.root(admin, api_fe)
|
||||||
|
assert (await http.get("/admin/auth/session")).status_code == 401
|
||||||
|
bad = await http.post(
|
||||||
|
"/admin/auth/login", json={"username": "admin", "password": "nope"}
|
||||||
|
)
|
||||||
|
assert bad.status_code == 401
|
||||||
|
ok = await http.post(
|
||||||
|
"/admin/auth/login", json={"username": "admin", "password": "secret"}
|
||||||
|
)
|
||||||
|
assert ok.status_code == 200
|
||||||
|
session = (await http.get("/admin/auth/session")).json()
|
||||||
|
assert session == {"user": "admin", "token": "ui-bearer"}
|
||||||
|
spa = await http.get("/admin/conversations/abc")
|
||||||
|
assert spa.status_code == 200 and spa.text == "<html>ui</html>"
|
||||||
|
root = await http.get("/")
|
||||||
|
assert root.status_code == 307 and root.headers["location"] == "/admin/"
|
||||||
|
assert (await http.get("/admin")).status_code == 307
|
||||||
|
health = await http.get("/healthz")
|
||||||
|
assert health.json()["frontends"] == ["/admin", "/api"]
|
||||||
|
escape = await http.get("/admin/%2e%2e/pyproject.toml")
|
||||||
|
assert escape.status_code == 200 and escape.text == "<html>ui</html>"
|
||||||
|
me = await api.http.get("/agents", headers={"Authorization": "Bearer ui-bearer"})
|
||||||
|
assert me.status_code == 200
|
||||||
|
assert (await http.post("/admin/auth/logout")).status_code == 204
|
||||||
|
assert (await http.get("/admin/auth/session")).status_code == 401
|
||||||
|
|
||||||
|
|
||||||
|
async def test_pre_sdk_rows_show_canonical_history_and_a_title(world: World) -> None:
|
||||||
|
api = Api(world)
|
||||||
|
conv = await world.conversations.create(kind="deep", agent="d", origin="markdown")
|
||||||
|
async with world.db.session() as session:
|
||||||
|
await rewrite_messages(
|
||||||
|
session,
|
||||||
|
conversation_id=cast("int", conv.id),
|
||||||
|
messages=[
|
||||||
|
{"role": "user", "content": "так смотри, план на май\nвторая строка"},
|
||||||
|
{"role": "assistant", "content": [{"type": "text", "text": "ок"}]},
|
||||||
|
],
|
||||||
|
)
|
||||||
|
rows = (await api.get("/conversations"))["conversations"]
|
||||||
|
assert [r["title"] for r in rows if r["id"] == conv.external_id] == [
|
||||||
|
"так смотри, план на май"
|
||||||
|
]
|
||||||
|
shown = await api.get(f"/conversations/{conv.external_id}")
|
||||||
|
assert shown["title"] == "так смотри, план на май" and shown["session_id"] is None
|
||||||
|
history = await api.get(f"/conversations/{conv.external_id}/history")
|
||||||
|
assert [m["role"] for m in history["messages"]] == ["user", "assistant"]
|
||||||
|
|
||||||
|
|
||||||
|
async def test_close_distills_a_deep_chat(world: World) -> None:
|
||||||
|
from test_distill import DistillerClient, deep_chat, distiller
|
||||||
|
|
||||||
|
config = distiller(world, DistillerClient)
|
||||||
|
api = Api(world)
|
||||||
|
master = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
chat = await deep_chat(world)
|
||||||
|
res = await api.http.post(
|
||||||
|
f"/conversations/{chat.external_id}/close", headers=HEADERS
|
||||||
|
)
|
||||||
|
assert res.status_code == 200, res.text
|
||||||
|
body = res.json()
|
||||||
|
assert body["status"] == "closed" and body["error"] is None
|
||||||
|
assert body["digest"] == str(config.dir / "2026-08-29 - тема.md")
|
||||||
|
assert body["text"].count("\n") == 2
|
||||||
|
assert (await world.conversations.get(chat.external_id)).status == "closed"
|
||||||
|
assert (await world.conversations.queue.recent(master.id))[0].origin == "выжимка"
|
||||||
|
again = await api.http.post(
|
||||||
|
f"/conversations/{chat.external_id}/close", headers=HEADERS
|
||||||
|
)
|
||||||
|
assert again.status_code == 400
|
||||||
|
branch = await world.conversations.create(kind="branch", agent="a", origin="test")
|
||||||
|
plain = await api.http.post(
|
||||||
|
f"/conversations/{branch.external_id}/close", headers=HEADERS
|
||||||
|
)
|
||||||
|
assert plain.status_code == 200 and plain.json()["status"] == "closed"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_bind_without_a_window_materializes_one(world: World) -> None:
|
||||||
|
api = Api(world)
|
||||||
|
chat = await world.conversations.create(kind="deep", agent="d", origin="test")
|
||||||
|
world.markdown.materialized.clear()
|
||||||
|
res = await api.http.post(
|
||||||
|
f"/conversations/{chat.external_id}/bind",
|
||||||
|
json={"frontend": "markdown"},
|
||||||
|
headers=HEADERS,
|
||||||
|
)
|
||||||
|
assert res.status_code == 200, res.text
|
||||||
|
assert world.markdown.materialized == [chat.external_id]
|
||||||
|
assert res.json()["bindings"] == [
|
||||||
|
{
|
||||||
|
"frontend": "markdown",
|
||||||
|
"external_id": f"markdown:{chat.external_id}",
|
||||||
|
"visible": True,
|
||||||
|
}
|
||||||
|
]
|
||||||
|
hidden = await api.http.post(
|
||||||
|
f"/conversations/{chat.external_id}/bind",
|
||||||
|
json={
|
||||||
|
"frontend": "markdown",
|
||||||
|
"external_id": f"markdown:{chat.external_id}",
|
||||||
|
"visible": False,
|
||||||
|
},
|
||||||
|
headers=HEADERS,
|
||||||
|
)
|
||||||
|
assert hidden.json()["bindings"][0]["visible"] is False
|
||||||
|
no_window = await api.http.post(
|
||||||
|
f"/conversations/{chat.external_id}/bind",
|
||||||
|
json={"frontend": "api"},
|
||||||
|
headers=HEADERS,
|
||||||
|
)
|
||||||
|
assert no_window.status_code == 400
|
||||||
|
unknown = await api.http.post(
|
||||||
|
f"/conversations/{chat.external_id}/bind",
|
||||||
|
json={"frontend": "nope"},
|
||||||
|
headers=HEADERS,
|
||||||
|
)
|
||||||
|
assert unknown.status_code == 400
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
"""``require_token`` accepts ``?token=`` only when no auth header is present."""
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from types import SimpleNamespace
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi import HTTPException
|
||||||
|
from starlette.requests import Request
|
||||||
|
|
||||||
|
from beaver_gateway.core.auth import TokenStore
|
||||||
|
from beaver_gateway.frontends._auth import require_token
|
||||||
|
|
||||||
|
|
||||||
|
def _request(query: str = "", headers: dict[str, str] | None = None) -> Request:
|
||||||
|
scope = {
|
||||||
|
"type": "http",
|
||||||
|
"method": "POST",
|
||||||
|
"path": "/hooks/komodo",
|
||||||
|
"query_string": query.encode(),
|
||||||
|
"headers": [
|
||||||
|
(k.lower().encode(), v.encode()) for k, v in (headers or {}).items()
|
||||||
|
],
|
||||||
|
}
|
||||||
|
return Request(scope)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def runtime() -> SimpleNamespace:
|
||||||
|
return SimpleNamespace(token_store=TokenStore(bootstrap={"komodo": "qs-secret"}))
|
||||||
|
|
||||||
|
|
||||||
|
async def test_query_token_authorizes_a_headerless_webhook(runtime) -> None:
|
||||||
|
name = await require_token(_request("token=qs-secret"), runtime, scope="api")
|
||||||
|
assert name == "komodo"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_header_wins_over_query(runtime) -> None:
|
||||||
|
bad_header = _request("token=qs-secret", {"Authorization": "Bearer nope"})
|
||||||
|
with pytest.raises(HTTPException) as exc:
|
||||||
|
await require_token(bad_header, runtime, scope="api")
|
||||||
|
assert exc.value.status_code == 401
|
||||||
|
|
||||||
|
|
||||||
|
async def test_missing_everything_is_401(runtime) -> None:
|
||||||
|
with pytest.raises(HTTPException) as exc:
|
||||||
|
await require_token(_request(), runtime, scope="api")
|
||||||
|
assert exc.value.status_code == 401
|
||||||
|
|
||||||
|
|
||||||
|
async def test_bootstrap_entry_can_carry_a_scope() -> None:
|
||||||
|
raw = "admin:secret-a,komodo:hook-value:api"
|
||||||
|
store = TokenStore(
|
||||||
|
bootstrap=TokenStore.parse_bootstrap(raw),
|
||||||
|
bootstrap_scopes=TokenStore.parse_bootstrap_scopes(raw),
|
||||||
|
)
|
||||||
|
admin = await store.verify("secret-a")
|
||||||
|
hook = await store.verify("hook-value")
|
||||||
|
assert admin is not None and admin.scope == "*"
|
||||||
|
assert hook is not None and hook.scope == "api" and hook.name == "komodo"
|
||||||
|
assert not hook.allows("admin") and hook.allows("api")
|
||||||
|
|
||||||
|
|
||||||
|
def test_access_log_filter_masks_query_tokens() -> None:
|
||||||
|
from beaver_gateway.core.redact import RedactFilter
|
||||||
|
|
||||||
|
record = logging.LogRecord(
|
||||||
|
"uvicorn.access",
|
||||||
|
logging.INFO,
|
||||||
|
__file__,
|
||||||
|
1,
|
||||||
|
'%s - "%s %s HTTP/%s" %d',
|
||||||
|
("1.2.3.4:1", "POST", "/hooks/komodo?token=s3cret&x=1", "1.1", 202),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
assert RedactFilter().filter(record)
|
||||||
|
assert "s3cret" not in record.getMessage()
|
||||||
|
assert "/hooks/komodo?token=<…>&x=1" in record.getMessage()
|
||||||
@@ -0,0 +1,540 @@
|
|||||||
|
import asyncio
|
||||||
|
import tempfile
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from anthropic.types import (
|
||||||
|
RawContentBlockDeltaEvent,
|
||||||
|
RawContentBlockStartEvent,
|
||||||
|
RawContentBlockStopEvent,
|
||||||
|
RawMessageDeltaEvent,
|
||||||
|
RawMessageStartEvent,
|
||||||
|
RawMessageStopEvent,
|
||||||
|
)
|
||||||
|
from claude_agent_sdk import (
|
||||||
|
AssistantMessage,
|
||||||
|
InMemorySessionStore,
|
||||||
|
ResultMessage,
|
||||||
|
StreamEvent,
|
||||||
|
TextBlock,
|
||||||
|
ToolResultBlock,
|
||||||
|
ToolUseBlock,
|
||||||
|
UserMessage,
|
||||||
|
project_key_for_directory,
|
||||||
|
)
|
||||||
|
|
||||||
|
from beaver_gateway.agents.base import ExposedMcp
|
||||||
|
from beaver_gateway.agents.claude import ClaudeAgent, ClaudeOptions, Prompts
|
||||||
|
from beaver_gateway.backends.claude_sdk import (
|
||||||
|
ClaudeSdkBackend,
|
||||||
|
RunnerConfig,
|
||||||
|
UsageEvent,
|
||||||
|
fingerprint,
|
||||||
|
)
|
||||||
|
from beaver_gateway.core.transcript import messages_from_entries
|
||||||
|
from beaver_gateway.core.turn_capture import TurnCapture
|
||||||
|
|
||||||
|
|
||||||
|
def _stream(index: int, text: str) -> list[StreamEvent]:
|
||||||
|
def ev(event: dict[str, Any]) -> StreamEvent:
|
||||||
|
return StreamEvent(uuid="u", session_id="s", event=event)
|
||||||
|
|
||||||
|
return [
|
||||||
|
ev(
|
||||||
|
{
|
||||||
|
"type": "content_block_start",
|
||||||
|
"index": index,
|
||||||
|
"content_block": {"type": "text", "text": ""},
|
||||||
|
}
|
||||||
|
),
|
||||||
|
ev(
|
||||||
|
{
|
||||||
|
"type": "content_block_delta",
|
||||||
|
"index": index,
|
||||||
|
"delta": {"type": "text_delta", "text": text},
|
||||||
|
}
|
||||||
|
),
|
||||||
|
ev({"type": "content_block_stop", "index": index}),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _result(session_id: str) -> ResultMessage:
|
||||||
|
return ResultMessage(
|
||||||
|
subtype="success",
|
||||||
|
duration_ms=10,
|
||||||
|
duration_api_ms=5,
|
||||||
|
is_error=False,
|
||||||
|
num_turns=1,
|
||||||
|
session_id=session_id,
|
||||||
|
stop_reason="end_turn",
|
||||||
|
total_cost_usd=0.01,
|
||||||
|
usage={
|
||||||
|
"input_tokens": 3,
|
||||||
|
"output_tokens": 7,
|
||||||
|
"cache_read_input_tokens": 100,
|
||||||
|
"cache_creation_input_tokens": 20,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class FakeClient:
|
||||||
|
instances: list["FakeClient"] = []
|
||||||
|
|
||||||
|
def __init__(self, options: Any) -> None:
|
||||||
|
self.options = options
|
||||||
|
self.prompts: list[str] = []
|
||||||
|
self.connected = False
|
||||||
|
self.session_id = options.resume or "fresh-session"
|
||||||
|
FakeClient.instances.append(self)
|
||||||
|
|
||||||
|
async def connect(self) -> None:
|
||||||
|
self.connected = True
|
||||||
|
|
||||||
|
async def query(self, prompt: str) -> None:
|
||||||
|
self.prompts.append(prompt)
|
||||||
|
|
||||||
|
async def interrupt(self) -> None:
|
||||||
|
self.interrupted = True
|
||||||
|
|
||||||
|
async def receive_response(self):
|
||||||
|
start = {
|
||||||
|
"type": "message_start",
|
||||||
|
"message": {
|
||||||
|
"usage": {"input_tokens": 5, "cache_read_input_tokens": 20_000}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
yield StreamEvent(uuid="u", session_id="s", event=start)
|
||||||
|
for e in _stream(0, "calling "):
|
||||||
|
yield e
|
||||||
|
tool = ToolUseBlock(id="toolu_1", name="Read", input={"path": "x"})
|
||||||
|
yield StreamEvent(
|
||||||
|
uuid="u",
|
||||||
|
session_id="s",
|
||||||
|
event={
|
||||||
|
"type": "content_block_start",
|
||||||
|
"index": 1,
|
||||||
|
"content_block": {"type": "tool_use", "id": "toolu_1", "name": "Read"},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
yield StreamEvent(
|
||||||
|
uuid="u", session_id="s", event={"type": "content_block_stop", "index": 1}
|
||||||
|
)
|
||||||
|
yield AssistantMessage(content=[TextBlock(text="calling "), tool], model="m")
|
||||||
|
yield UserMessage(
|
||||||
|
content=[ToolResultBlock(tool_use_id="toolu_1", content="42")]
|
||||||
|
)
|
||||||
|
second = {
|
||||||
|
"type": "message_start",
|
||||||
|
"message": {
|
||||||
|
"usage": {
|
||||||
|
"input_tokens": 7,
|
||||||
|
"cache_read_input_tokens": 20_000,
|
||||||
|
"cache_creation_input_tokens": 3_000,
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
yield StreamEvent(uuid="u", session_id="s", event=second)
|
||||||
|
for e in _stream(0, "done"):
|
||||||
|
yield e
|
||||||
|
yield AssistantMessage(content=[TextBlock(text="done")], model="m")
|
||||||
|
yield StreamEvent(
|
||||||
|
uuid="sub",
|
||||||
|
session_id="s",
|
||||||
|
event={"type": "content_block_stop", "index": 5},
|
||||||
|
parent_tool_use_id="toolu_9",
|
||||||
|
)
|
||||||
|
yield _result(self.session_id)
|
||||||
|
|
||||||
|
async def disconnect(self) -> None:
|
||||||
|
self.connected = False
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def cwd() -> Path:
|
||||||
|
return Path(tempfile.mkdtemp(prefix="beaver-sdk-"))
|
||||||
|
|
||||||
|
|
||||||
|
def _backend(
|
||||||
|
cwd: Path, store: InMemorySessionStore, sink=None, **agent_kwargs
|
||||||
|
) -> ClaudeSdkBackend:
|
||||||
|
FakeClient.instances.clear()
|
||||||
|
agent = ClaudeAgent(
|
||||||
|
name="a",
|
||||||
|
model="claude-x",
|
||||||
|
system_prompt="hi",
|
||||||
|
cwd=cwd,
|
||||||
|
options=ClaudeOptions(effort="low"),
|
||||||
|
**agent_kwargs,
|
||||||
|
)
|
||||||
|
return ClaudeSdkBackend(
|
||||||
|
agent=agent,
|
||||||
|
mcp_internal_urls={"firefly": "http://127.0.0.1:1/mcp/firefly/"},
|
||||||
|
session_store=store,
|
||||||
|
mcp_tool_names={"firefly": ["list_account", "delete_account", "store_account"]},
|
||||||
|
usage_sink=sink,
|
||||||
|
client_factory=FakeClient,
|
||||||
|
work_dir=cwd / "work",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def _drain(events) -> list[Any]:
|
||||||
|
return [e async for e in events]
|
||||||
|
|
||||||
|
|
||||||
|
async def test_stream_envelope_and_index_rebase(cwd: Path) -> None:
|
||||||
|
backend = _backend(
|
||||||
|
cwd, InMemorySessionStore(), expose_mcps=(ExposedMcp(name="firefly"),)
|
||||||
|
)
|
||||||
|
capture = TurnCapture()
|
||||||
|
events = await _drain(
|
||||||
|
backend.complete(
|
||||||
|
agent=backend.agent,
|
||||||
|
messages=[{"role": "user", "content": "hi"}],
|
||||||
|
conversation_id="conv-1",
|
||||||
|
capture=capture,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
assert isinstance(events[0], RawMessageStartEvent)
|
||||||
|
assert isinstance(events[-1], RawMessageStopEvent)
|
||||||
|
assert isinstance(events[-2], RawMessageDeltaEvent)
|
||||||
|
assert events[-2].usage.output_tokens == 7
|
||||||
|
assert events[-2].usage.cache_read_input_tokens == 100
|
||||||
|
starts = [e.index for e in events if isinstance(e, RawContentBlockStartEvent)]
|
||||||
|
assert starts == [0, 1, 2]
|
||||||
|
stops = [e.index for e in events if isinstance(e, RawContentBlockStopEvent)]
|
||||||
|
assert stops == [0, 1, 2]
|
||||||
|
deltas = [e.delta.text for e in events if isinstance(e, RawContentBlockDeltaEvent)]
|
||||||
|
assert deltas == ["calling ", "done"]
|
||||||
|
|
||||||
|
assert capture.session_id == "fresh-session"
|
||||||
|
assert capture.usage is not None and capture.usage.cost_usd == 0.01
|
||||||
|
assert capture.usage.context_tokens == 23_007
|
||||||
|
assert capture.synthesized_messages == [
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": [
|
||||||
|
{"type": "text", "text": "calling "},
|
||||||
|
{
|
||||||
|
"type": "tool_use",
|
||||||
|
"id": "toolu_1",
|
||||||
|
"name": "Read",
|
||||||
|
"input": {"path": "x"},
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": [
|
||||||
|
{"type": "tool_result", "tool_use_id": "toolu_1", "content": "42"}
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{"role": "assistant", "content": [{"type": "text", "text": "done"}]},
|
||||||
|
]
|
||||||
|
client = FakeClient.instances[0]
|
||||||
|
assert client.prompts == ["hi"]
|
||||||
|
opts = client.options
|
||||||
|
assert opts.setting_sources == []
|
||||||
|
assert opts.strict_mcp_config is True
|
||||||
|
assert opts.permission_mode == "bypassPermissions"
|
||||||
|
assert opts.mcp_servers == {
|
||||||
|
"firefly": {"type": "http", "url": "http://127.0.0.1:1/mcp/firefly/"}
|
||||||
|
}
|
||||||
|
assert opts.session_store is not None
|
||||||
|
assert opts.resume is None
|
||||||
|
assert Path(opts.cli_path).exists()
|
||||||
|
wrapper = Path(opts.cli_path).read_text()
|
||||||
|
assert "CLAUDE_" in wrapper and "DATABASE_URL" not in wrapper
|
||||||
|
|
||||||
|
|
||||||
|
async def test_conversation_reuses_live_session(cwd: Path) -> None:
|
||||||
|
backend = _backend(cwd, InMemorySessionStore())
|
||||||
|
for i in range(2):
|
||||||
|
await _drain(
|
||||||
|
backend.complete(
|
||||||
|
agent=backend.agent,
|
||||||
|
messages=[{"role": "user", "content": f"turn {i}"}],
|
||||||
|
conversation_id="conv-1",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
assert len(FakeClient.instances) == 1
|
||||||
|
assert FakeClient.instances[0].prompts == ["turn 0", "turn 1"]
|
||||||
|
assert backend.sessions["conv-1"]["turns"] == 2
|
||||||
|
|
||||||
|
|
||||||
|
async def test_history_is_seeded_into_store_and_resumed(cwd: Path) -> None:
|
||||||
|
store = InMemorySessionStore()
|
||||||
|
backend = _backend(cwd, store)
|
||||||
|
history = [
|
||||||
|
{"role": "user", "content": "first"},
|
||||||
|
{"role": "assistant", "content": [{"type": "text", "text": "reply"}]},
|
||||||
|
{"role": "user", "content": "second"},
|
||||||
|
]
|
||||||
|
await _drain(
|
||||||
|
backend.complete(
|
||||||
|
agent=backend.agent, messages=history, conversation_id="conv-2"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
client = FakeClient.instances[0]
|
||||||
|
assert client.options.resume is not None
|
||||||
|
key = {
|
||||||
|
"project_key": project_key_for_directory(str(cwd)),
|
||||||
|
"session_id": client.options.resume,
|
||||||
|
}
|
||||||
|
entries = store.get_entries(key)
|
||||||
|
assert messages_from_entries(entries) == history[:2]
|
||||||
|
assert client.prompts == ["second"]
|
||||||
|
|
||||||
|
|
||||||
|
async def test_known_session_id_is_resumed_without_seeding(cwd: Path) -> None:
|
||||||
|
store = InMemorySessionStore()
|
||||||
|
backend = _backend(cwd, store)
|
||||||
|
history = [
|
||||||
|
{"role": "user", "content": "first"},
|
||||||
|
{"role": "assistant", "content": "reply"},
|
||||||
|
{"role": "user", "content": "second"},
|
||||||
|
]
|
||||||
|
await _drain(
|
||||||
|
backend.complete(
|
||||||
|
agent=backend.agent,
|
||||||
|
messages=history,
|
||||||
|
conversation_id="conv-3",
|
||||||
|
session_id="known-sid",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
assert FakeClient.instances[0].options.resume == "known-sid"
|
||||||
|
assert store.size == 0
|
||||||
|
|
||||||
|
|
||||||
|
async def test_stateless_caller_hits_same_session_next_turn(cwd: Path) -> None:
|
||||||
|
backend = _backend(cwd, InMemorySessionStore())
|
||||||
|
first = [{"role": "user", "content": "hi"}]
|
||||||
|
await _drain(backend.complete(agent=backend.agent, messages=first))
|
||||||
|
follow_up = [
|
||||||
|
*first,
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": [
|
||||||
|
{"type": "text", "text": "calling "},
|
||||||
|
{"type": "text", "text": "done"},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{"role": "user", "content": "more"},
|
||||||
|
]
|
||||||
|
assert list(backend.sessions) == [fingerprint(follow_up[:-1])]
|
||||||
|
await _drain(backend.complete(agent=backend.agent, messages=follow_up))
|
||||||
|
assert len(FakeClient.instances) == 1
|
||||||
|
assert FakeClient.instances[0].prompts == ["hi", "more"]
|
||||||
|
|
||||||
|
|
||||||
|
async def test_mcp_deny_and_usage_sink(cwd: Path) -> None:
|
||||||
|
seen: list[UsageEvent] = []
|
||||||
|
|
||||||
|
async def sink(event: UsageEvent) -> None:
|
||||||
|
seen.append(event)
|
||||||
|
|
||||||
|
backend = _backend(
|
||||||
|
cwd,
|
||||||
|
InMemorySessionStore(),
|
||||||
|
sink,
|
||||||
|
expose_mcps=(ExposedMcp(name="firefly", deny=("delete_*",)),),
|
||||||
|
)
|
||||||
|
await _drain(
|
||||||
|
backend.complete(
|
||||||
|
agent=backend.agent,
|
||||||
|
messages=[{"role": "user", "content": "x"}],
|
||||||
|
conversation_id="c",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
assert FakeClient.instances[0].options.disallowed_tools == [
|
||||||
|
"mcp__firefly__delete_account"
|
||||||
|
]
|
||||||
|
assert len(seen) == 1
|
||||||
|
assert seen[0].usage.input_tokens == 3
|
||||||
|
assert seen[0].conversation_id == "c"
|
||||||
|
assert seen[0].session_id == "fresh-session"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_skill_sets_become_sorted_plugins(cwd: Path) -> None:
|
||||||
|
sets = cwd / "skills"
|
||||||
|
for name in ("zeta", "общие"):
|
||||||
|
(sets / name / "demo").mkdir(parents=True)
|
||||||
|
(sets / name / "demo" / "SKILL.md").write_text("---\nname: demo\n---\n")
|
||||||
|
backend = _backend(
|
||||||
|
cwd, InMemorySessionStore(), skill_sets=(sets / "zeta", sets / "общие")
|
||||||
|
)
|
||||||
|
await _drain(
|
||||||
|
backend.complete(
|
||||||
|
agent=backend.agent,
|
||||||
|
messages=[{"role": "user", "content": "x"}],
|
||||||
|
conversation_id="c",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
plugins = FakeClient.instances[0].options.plugins
|
||||||
|
assert [p["type"] for p in plugins] == ["local", "local"]
|
||||||
|
paths = [Path(p["path"]) for p in plugins]
|
||||||
|
assert [p.name for p in paths] == ["zeta", "общие"]
|
||||||
|
for path in paths:
|
||||||
|
assert (path / ".claude-plugin" / "plugin.json").exists()
|
||||||
|
assert (path / "skills" / "demo" / "SKILL.md").exists()
|
||||||
|
assert FakeClient.instances[0].options.skills == "all"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_prompt_sources_are_assembled(cwd: Path) -> None:
|
||||||
|
(cwd / "a.md").write_text("alpha\n")
|
||||||
|
(cwd / "b.md").write_text("\nbeta\n\n")
|
||||||
|
backend = _backend(
|
||||||
|
cwd,
|
||||||
|
InMemorySessionStore(),
|
||||||
|
prompts=Prompts(deep=(("role", cwd / "a.md"), cwd / "b.md")),
|
||||||
|
)
|
||||||
|
await _drain(
|
||||||
|
backend.complete(
|
||||||
|
agent=backend.agent,
|
||||||
|
messages=[{"role": "user", "content": "x"}],
|
||||||
|
conversation_id="c",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
assert (
|
||||||
|
FakeClient.instances[0].options.system_prompt
|
||||||
|
== "<role>\nalpha\n</role>\n\nbeta\n"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def test_runner_user_lands_in_wrapper(cwd: Path) -> None:
|
||||||
|
import os
|
||||||
|
import pwd
|
||||||
|
|
||||||
|
me = pwd.getpwuid(os.getuid())
|
||||||
|
backend = _backend(cwd, InMemorySessionStore())
|
||||||
|
backend._runner = RunnerConfig(user=me.pw_name, home=cwd)
|
||||||
|
backend._uid, backend._gid = me.pw_uid, me.pw_gid
|
||||||
|
await _drain(
|
||||||
|
backend.complete(
|
||||||
|
agent=backend.agent,
|
||||||
|
messages=[{"role": "user", "content": "x"}],
|
||||||
|
conversation_id="c",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
opts = FakeClient.instances[0].options
|
||||||
|
assert opts.env["HOME"] == str(cwd)
|
||||||
|
assert opts.env["CLAUDE_CONFIG_DIR"] == str(cwd / ".claude")
|
||||||
|
wrapper = Path(opts.cli_path).read_text()
|
||||||
|
assert f"UID = {me.pw_uid}" in wrapper
|
||||||
|
assert "os.setuid(UID)" in wrapper
|
||||||
|
|
||||||
|
|
||||||
|
async def test_close_disconnects(cwd: Path) -> None:
|
||||||
|
backend = _backend(cwd, InMemorySessionStore())
|
||||||
|
async with backend:
|
||||||
|
await _drain(
|
||||||
|
backend.complete(
|
||||||
|
agent=backend.agent,
|
||||||
|
messages=[{"role": "user", "content": "x"}],
|
||||||
|
conversation_id="c",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
assert FakeClient.instances[0].connected is False
|
||||||
|
assert backend.sessions == {}
|
||||||
|
|
||||||
|
|
||||||
|
class GatedClient(FakeClient):
|
||||||
|
"""Streams one delta, then waits for ``gate`` before finishing the turn."""
|
||||||
|
|
||||||
|
gate: asyncio.Event
|
||||||
|
|
||||||
|
async def receive_response(self):
|
||||||
|
yield StreamEvent(
|
||||||
|
uuid="u", session_id="s", event={"type": "message_start", "message": {}}
|
||||||
|
)
|
||||||
|
for e in _stream(0, "first"):
|
||||||
|
yield e
|
||||||
|
await GatedClient.gate.wait()
|
||||||
|
yield AssistantMessage(content=[TextBlock(text="first")], model="m")
|
||||||
|
yield _result(self.session_id)
|
||||||
|
|
||||||
|
|
||||||
|
async def test_deltas_reach_the_caller_before_the_turn_ends(cwd: Path) -> None:
|
||||||
|
"""The turn is not buffered: a delta is observable while the CLI still runs."""
|
||||||
|
GatedClient.gate = asyncio.Event()
|
||||||
|
backend = _backend(cwd, InMemorySessionStore())
|
||||||
|
backend._factory = GatedClient
|
||||||
|
events = backend.complete(
|
||||||
|
agent=backend.agent,
|
||||||
|
messages=[{"role": "user", "content": "hi"}],
|
||||||
|
conversation_id="conv-live",
|
||||||
|
)
|
||||||
|
seen: list[Any] = []
|
||||||
|
async for ev in events:
|
||||||
|
seen.append(ev)
|
||||||
|
if isinstance(ev, RawContentBlockDeltaEvent):
|
||||||
|
break
|
||||||
|
assert seen[-1].delta.text == "first"
|
||||||
|
assert not GatedClient.gate.is_set()
|
||||||
|
GatedClient.gate.set()
|
||||||
|
rest = [e async for e in events]
|
||||||
|
assert isinstance(rest[-1], RawMessageStopEvent)
|
||||||
|
|
||||||
|
|
||||||
|
async def test_policy_hook_denies_and_audits(cwd: Path) -> None:
|
||||||
|
from beaver_gateway.core.policy import Deny, ToolCall
|
||||||
|
|
||||||
|
def no_days(c: ToolCall):
|
||||||
|
p = c.path()
|
||||||
|
if c.tool == "Write" and p is not None and "📅 дни" in p.parts:
|
||||||
|
return Deny(reason="дневник только читать")
|
||||||
|
c.state["seen"] = c.state.get("seen", 0) + 1
|
||||||
|
return None
|
||||||
|
|
||||||
|
audits = []
|
||||||
|
|
||||||
|
async def sink(a):
|
||||||
|
audits.append(a)
|
||||||
|
|
||||||
|
backend = _backend(cwd, InMemorySessionStore(), policy=(no_days,))
|
||||||
|
backend._audit_sink = sink
|
||||||
|
await _drain(
|
||||||
|
backend.complete(
|
||||||
|
agent=backend.agent,
|
||||||
|
messages=[{"role": "user", "content": "hi"}],
|
||||||
|
conversation_id="conv-1",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
hook = FakeClient.instances[0].options.hooks["PreToolUse"][0].hooks[0]
|
||||||
|
denied = await hook(
|
||||||
|
{"tool_name": "Write", "tool_input": {"file_path": "/vault/📅 дни/x.md"}},
|
||||||
|
"t1",
|
||||||
|
{},
|
||||||
|
)
|
||||||
|
assert denied["hookSpecificOutput"]["permissionDecision"] == "deny"
|
||||||
|
assert (
|
||||||
|
denied["hookSpecificOutput"]["permissionDecisionReason"]
|
||||||
|
== "дневник только читать"
|
||||||
|
)
|
||||||
|
allowed = await hook(
|
||||||
|
{"tool_name": "Write", "tool_input": {"file_path": "/vault/мета/бобер/x.md"}},
|
||||||
|
"t2",
|
||||||
|
{},
|
||||||
|
)
|
||||||
|
assert allowed == {}
|
||||||
|
await hook(
|
||||||
|
{"tool_name": "Read", "tool_input": {"file_path": "/vault/a.md"}}, "t3", {}
|
||||||
|
)
|
||||||
|
assert backend.live("conv-1").state["seen"] == 2
|
||||||
|
assert [(a.tool, a.decision) for a in audits] == [
|
||||||
|
("Write", "deny"),
|
||||||
|
("Write", "allow"),
|
||||||
|
("Read", "allow"),
|
||||||
|
]
|
||||||
|
assert audits[0].brief == "/vault/📅 дни/x.md"
|
||||||
|
assert audits[0].kind == "deep" and audits[0].conversation == "conv-1"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_no_policy_no_hooks(cwd: Path) -> None:
|
||||||
|
backend = _backend(cwd, InMemorySessionStore())
|
||||||
|
await _drain(
|
||||||
|
backend.complete(
|
||||||
|
agent=backend.agent, messages=[{"role": "user", "content": "hi"}]
|
||||||
|
)
|
||||||
|
)
|
||||||
|
assert FakeClient.instances[0].options.hooks is None
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
import tempfile
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from beaver_gateway import config_loader
|
||||||
|
|
||||||
|
|
||||||
|
def test_config_imports_sibling_modules():
|
||||||
|
root = Path(tempfile.mkdtemp(prefix="beaver-cfg-"))
|
||||||
|
(root / "policy_sibling.py").write_text("RULES = ('x',)\n")
|
||||||
|
(root / "config.py").write_text(
|
||||||
|
"from policy_sibling import RULES\n"
|
||||||
|
"assert RULES == ('x',)\n"
|
||||||
|
"gateway = Gateway()\n"
|
||||||
|
)
|
||||||
|
gw = config_loader.load(root / "config.py")
|
||||||
|
assert gw.agents == []
|
||||||
@@ -0,0 +1,844 @@
|
|||||||
|
import asyncio
|
||||||
|
import tempfile
|
||||||
|
import uuid
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from claude_agent_sdk import (
|
||||||
|
AssistantMessage,
|
||||||
|
InMemorySessionStore,
|
||||||
|
ResultMessage,
|
||||||
|
StreamEvent,
|
||||||
|
TextBlock,
|
||||||
|
ToolResultBlock,
|
||||||
|
ToolUseBlock,
|
||||||
|
UserMessage,
|
||||||
|
project_key_for_directory,
|
||||||
|
)
|
||||||
|
|
||||||
|
from beaver_gateway.agents.claude import ClaudeAgent, ClaudeOptions
|
||||||
|
from beaver_gateway.backends.claude_sdk import ClaudeSdkBackend
|
||||||
|
from beaver_gateway.core.bus import EventBus
|
||||||
|
from beaver_gateway.core.conversations import Conversations, ConversationTexts, parse_at
|
||||||
|
from beaver_gateway.core.gateway_tools import SAY_IN_USER_TURN, _tools
|
||||||
|
from beaver_gateway.core.registry import AgentRegistry
|
||||||
|
from beaver_gateway.core.sessions import SessionPool
|
||||||
|
from beaver_gateway.core.transcript import (
|
||||||
|
build_entries,
|
||||||
|
close_open_tool_uses,
|
||||||
|
open_tool_uses,
|
||||||
|
render_messages,
|
||||||
|
strip_tool_entries,
|
||||||
|
window_entries,
|
||||||
|
)
|
||||||
|
from beaver_gateway.frontends.base import Frontend
|
||||||
|
from beaver_gateway.storage import Database
|
||||||
|
from beaver_gateway.storage.models import (
|
||||||
|
Conversation,
|
||||||
|
ConversationBinding,
|
||||||
|
InjectQueueItem,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class StubFrontend(Frontend):
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
name: str,
|
||||||
|
kinds: tuple[str, ...],
|
||||||
|
agents: dict[str, str] | None = None,
|
||||||
|
*,
|
||||||
|
home: bool = False,
|
||||||
|
) -> None:
|
||||||
|
self.name = name
|
||||||
|
self.kinds = kinds
|
||||||
|
self.agents = agents or {}
|
||||||
|
self.home = home
|
||||||
|
self.materialized: list[str] = []
|
||||||
|
self.conversations = None
|
||||||
|
|
||||||
|
def configure(self, runtime) -> None:
|
||||||
|
pass
|
||||||
|
|
||||||
|
async def serve(self) -> None:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def agent_for(self, kind: str) -> str | None:
|
||||||
|
return self.agents.get(kind)
|
||||||
|
|
||||||
|
async def materialize(self, conv: Conversation) -> ConversationBinding | None:
|
||||||
|
if not self.home:
|
||||||
|
return None
|
||||||
|
self.materialized.append(conv.external_id)
|
||||||
|
return await self.conversations.bind(
|
||||||
|
conv, frontend=self.name, external_id=f"{self.name}:{conv.external_id}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ScriptedClient:
|
||||||
|
instances: list["ScriptedClient"] = []
|
||||||
|
hold: asyncio.Event | None = None
|
||||||
|
|
||||||
|
def __init__(self, options: Any) -> None:
|
||||||
|
self.options = options
|
||||||
|
self.prompts: list[str] = []
|
||||||
|
self.session_id = options.resume or str(uuid.uuid4())
|
||||||
|
self.interrupted = False
|
||||||
|
self.connected = False
|
||||||
|
ScriptedClient.instances.append(self)
|
||||||
|
|
||||||
|
async def connect(self) -> None:
|
||||||
|
self.connected = True
|
||||||
|
|
||||||
|
async def query(self, prompt: str) -> None:
|
||||||
|
self.prompts.append(prompt)
|
||||||
|
|
||||||
|
async def receive_response(self):
|
||||||
|
prompt = self.prompts[-1]
|
||||||
|
yield StreamEvent(
|
||||||
|
uuid="u",
|
||||||
|
session_id=self.session_id,
|
||||||
|
event={"type": "message_start", "message": {}},
|
||||||
|
)
|
||||||
|
yield StreamEvent(
|
||||||
|
uuid="u",
|
||||||
|
session_id=self.session_id,
|
||||||
|
event={
|
||||||
|
"type": "content_block_start",
|
||||||
|
"index": 0,
|
||||||
|
"content_block": {"type": "text", "text": ""},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
yield StreamEvent(
|
||||||
|
uuid="u",
|
||||||
|
session_id=self.session_id,
|
||||||
|
event={
|
||||||
|
"type": "content_block_delta",
|
||||||
|
"index": 0,
|
||||||
|
"delta": {"type": "text_delta", "text": f"ok:{prompt}"},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
yield StreamEvent(
|
||||||
|
uuid="u",
|
||||||
|
session_id=self.session_id,
|
||||||
|
event={"type": "content_block_stop", "index": 0},
|
||||||
|
)
|
||||||
|
yield AssistantMessage(content=[TextBlock(text=f"ok:{prompt}")], model="m")
|
||||||
|
hold = ScriptedClient.hold
|
||||||
|
if hold is not None and not self.interrupted:
|
||||||
|
await hold.wait()
|
||||||
|
cut = self.interrupted
|
||||||
|
self.interrupted = False
|
||||||
|
yield ResultMessage(
|
||||||
|
subtype="error_during_execution" if cut else "success",
|
||||||
|
duration_ms=1,
|
||||||
|
duration_api_ms=1,
|
||||||
|
is_error=cut,
|
||||||
|
num_turns=1,
|
||||||
|
session_id=self.session_id,
|
||||||
|
stop_reason="end_turn",
|
||||||
|
total_cost_usd=0.0,
|
||||||
|
usage={"input_tokens": 1, "output_tokens": 1},
|
||||||
|
)
|
||||||
|
|
||||||
|
async def interrupt(self) -> None:
|
||||||
|
self.interrupted = True
|
||||||
|
if ScriptedClient.hold is not None:
|
||||||
|
ScriptedClient.hold.set()
|
||||||
|
|
||||||
|
async def disconnect(self) -> None:
|
||||||
|
self.connected = False
|
||||||
|
|
||||||
|
|
||||||
|
class World:
|
||||||
|
def __init__(self, root: Path) -> None:
|
||||||
|
ScriptedClient.instances.clear()
|
||||||
|
ScriptedClient.hold = None
|
||||||
|
self.root = root
|
||||||
|
self.db = Database(f"sqlite:///{root / 'w.db'}")
|
||||||
|
self.store = InMemorySessionStore()
|
||||||
|
self.agent = ClaudeAgent(
|
||||||
|
name="a",
|
||||||
|
model="m",
|
||||||
|
system_prompt="hi",
|
||||||
|
cwd=root,
|
||||||
|
kinds=("master", "branch", "job", "fork"),
|
||||||
|
gateway_tools=("say",),
|
||||||
|
options=ClaudeOptions(effort="low"),
|
||||||
|
)
|
||||||
|
self.deep_agent = ClaudeAgent(
|
||||||
|
name="d", model="m", system_prompt="deep", cwd=root
|
||||||
|
)
|
||||||
|
self.markdown = StubFrontend("markdown", ("deep",), {"deep": "d"}, home=True)
|
||||||
|
self.api = StubFrontend("api", ("master", "branch", "deep", "job"))
|
||||||
|
self.pool = SessionPool(rss_limit=1 << 40, max_live=100)
|
||||||
|
self.backend, self.deep_backend = (
|
||||||
|
ClaudeSdkBackend(
|
||||||
|
agent=agent,
|
||||||
|
mcp_internal_urls={},
|
||||||
|
session_store=self.store,
|
||||||
|
client_factory=ScriptedClient,
|
||||||
|
work_dir=root / "work",
|
||||||
|
pool=self.pool,
|
||||||
|
)
|
||||||
|
for agent in (self.agent, self.deep_agent)
|
||||||
|
)
|
||||||
|
self.bus = EventBus()
|
||||||
|
self.conversations = Conversations(
|
||||||
|
db=self.db,
|
||||||
|
agents=AgentRegistry([self.agent, self.deep_agent]),
|
||||||
|
backends={"a": self.backend, "d": self.deep_backend},
|
||||||
|
bus=self.bus,
|
||||||
|
pool=self.pool,
|
||||||
|
store=self.store,
|
||||||
|
texts=ConversationTexts(),
|
||||||
|
frontends=[self.api, self.markdown],
|
||||||
|
idle_interval=3600,
|
||||||
|
)
|
||||||
|
self.markdown.conversations = self.conversations
|
||||||
|
self.api.conversations = self.conversations
|
||||||
|
|
||||||
|
async def setup(self) -> "World":
|
||||||
|
await self.db.create_all()
|
||||||
|
return self
|
||||||
|
|
||||||
|
def key(self, session_id: str) -> dict[str, str]:
|
||||||
|
return {
|
||||||
|
"project_key": project_key_for_directory(str(self.root)),
|
||||||
|
"session_id": session_id,
|
||||||
|
}
|
||||||
|
|
||||||
|
async def statuses(self, conv) -> list[tuple[str, str]]:
|
||||||
|
rows = await self.conversations.queue.recent(conv.id)
|
||||||
|
return [(r.priority, r.status) for r in reversed(rows)]
|
||||||
|
|
||||||
|
async def settle(self, conv, expected: int, timeout: float = 5.0) -> None:
|
||||||
|
deadline = asyncio.get_running_loop().time() + timeout
|
||||||
|
while asyncio.get_running_loop().time() < deadline:
|
||||||
|
rows = await self.conversations.queue.recent(conv.id)
|
||||||
|
if sum(1 for r in rows if r.status == "done") >= expected:
|
||||||
|
return
|
||||||
|
await asyncio.sleep(0.02)
|
||||||
|
msg = f"queue did not settle: {await self.statuses(conv)}"
|
||||||
|
raise AssertionError(msg)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
async def world() -> World:
|
||||||
|
root = Path(tempfile.mkdtemp(prefix="beaver-conv-"))
|
||||||
|
w = await World(root).setup()
|
||||||
|
yield w
|
||||||
|
await w.conversations.stop()
|
||||||
|
await w.pool.close_all()
|
||||||
|
await w.db.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_two_messages_run_one_at_a_time_in_order(world: World) -> None:
|
||||||
|
conv = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
ScriptedClient.hold = asyncio.Event()
|
||||||
|
await world.conversations.post(conv, "first")
|
||||||
|
await asyncio.sleep(0.2)
|
||||||
|
await world.conversations.post(conv, "second")
|
||||||
|
await asyncio.sleep(0.2)
|
||||||
|
assert await world.statuses(conv) == [("user", "running"), ("user", "queued")]
|
||||||
|
live = world.pool.get(conv.external_id)
|
||||||
|
assert live is not None and live.busy
|
||||||
|
assert (await world.conversations.get(conv.external_id)).running_turn is not None
|
||||||
|
ScriptedClient.hold.set()
|
||||||
|
await world.settle(conv, 2)
|
||||||
|
assert len(ScriptedClient.instances) == 1
|
||||||
|
assert ScriptedClient.instances[0].prompts == ["first", "second"]
|
||||||
|
row = await world.conversations.get(conv.external_id)
|
||||||
|
assert row.running_turn is None
|
||||||
|
assert row.session_id == ScriptedClient.instances[0].session_id
|
||||||
|
|
||||||
|
|
||||||
|
async def test_say_is_refused_in_user_turns_and_works_in_inject_turns(
|
||||||
|
world: World,
|
||||||
|
) -> None:
|
||||||
|
conv = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
say = next(
|
||||||
|
t for t in _tools(world.conversations, conv.external_id) if t.name == "say"
|
||||||
|
)
|
||||||
|
said: list[str] = []
|
||||||
|
|
||||||
|
async def watch() -> None:
|
||||||
|
async for event in world.bus.stream(conversation_id=conv.external_id):
|
||||||
|
if event["type"] == "say":
|
||||||
|
said.append(str(event["text"]))
|
||||||
|
|
||||||
|
watcher = asyncio.create_task(watch())
|
||||||
|
ScriptedClient.hold = asyncio.Event()
|
||||||
|
await world.conversations.post(conv, "hi")
|
||||||
|
await asyncio.sleep(0.2)
|
||||||
|
assert world.conversations.turn_origin(conv) == "user"
|
||||||
|
out = await say.handler({"text": "dup"})
|
||||||
|
assert out.get("is_error") and out["content"][0]["text"] == SAY_IN_USER_TURN
|
||||||
|
ScriptedClient.hold.set()
|
||||||
|
await world.settle(conv, 1)
|
||||||
|
assert world.conversations.turn_origin(conv) is None
|
||||||
|
ScriptedClient.hold = asyncio.Event()
|
||||||
|
await world.conversations.inject(conv, "tick", urgency="urgent", origin="крон")
|
||||||
|
await asyncio.sleep(0.2)
|
||||||
|
assert world.conversations.turn_origin(conv) == "inject"
|
||||||
|
out = await say.handler({"text": "psst"})
|
||||||
|
assert not out.get("is_error")
|
||||||
|
ScriptedClient.hold.set()
|
||||||
|
await world.settle(conv, 2)
|
||||||
|
await asyncio.sleep(0.1)
|
||||||
|
watcher.cancel()
|
||||||
|
assert said == ["psst"]
|
||||||
|
|
||||||
|
|
||||||
|
async def test_urgent_interrupts_and_goes_first(world: World) -> None:
|
||||||
|
conv = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
ScriptedClient.hold = asyncio.Event()
|
||||||
|
await world.conversations.post(conv, "first")
|
||||||
|
await asyncio.sleep(0.2)
|
||||||
|
await world.conversations.post(conv, "second")
|
||||||
|
await world.conversations.inject(conv, "ALERT", urgency="urgent", origin="крон")
|
||||||
|
await world.settle(conv, 2)
|
||||||
|
client = ScriptedClient.instances[0]
|
||||||
|
assert await world.statuses(conv) == [
|
||||||
|
("user", "interrupted"),
|
||||||
|
("user", "done"),
|
||||||
|
("urgent", "done"),
|
||||||
|
]
|
||||||
|
assert await world.statuses(conv) == [
|
||||||
|
("user", "interrupted"),
|
||||||
|
("user", "done"),
|
||||||
|
("urgent", "done"),
|
||||||
|
]
|
||||||
|
assert client.prompts[0] == "first"
|
||||||
|
assert client.prompts[1].startswith("[инжект: крон")
|
||||||
|
assert "прервал предыдущий тёрн" in client.prompts[1]
|
||||||
|
assert client.prompts[1].endswith("ALERT")
|
||||||
|
assert client.prompts[2] == "second"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_inject_header_is_configurable(world: World) -> None:
|
||||||
|
from beaver_gateway.core.conversations import ConversationTexts
|
||||||
|
|
||||||
|
world.conversations._texts = ConversationTexts( # noqa: SLF001
|
||||||
|
inject_header=lambda ctx: (
|
||||||
|
f"[от {ctx.origin}, {ctx.priority}, cut={ctx.interrupted_turn}]"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
conv = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
await world.conversations.inject(conv, "hi", urgency="urgent", origin="panel")
|
||||||
|
await world.settle(conv, 1)
|
||||||
|
assert ScriptedClient.instances[0].prompts[0] == "[от panel, urgent, cut=False]\nhi"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_normal_injects_ride_with_the_next_user_message(world: World) -> None:
|
||||||
|
conv = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
await world.conversations.inject(
|
||||||
|
conv, "vault changed", urgency="normal", origin="watch"
|
||||||
|
)
|
||||||
|
await asyncio.sleep(0.2)
|
||||||
|
assert await world.statuses(conv) == [("normal", "queued")]
|
||||||
|
await world.conversations.post(conv, "hello")
|
||||||
|
await world.settle(conv, 2)
|
||||||
|
prompts = ScriptedClient.instances[0].prompts
|
||||||
|
assert len(prompts) == 1
|
||||||
|
assert prompts[0].startswith("hello\n\n[инжекты")
|
||||||
|
assert "- [watch] vault changed" in prompts[0]
|
||||||
|
|
||||||
|
|
||||||
|
async def test_inject_turn_reply_is_not_routed(world: World) -> None:
|
||||||
|
conv = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
seen: list[dict[str, Any]] = []
|
||||||
|
|
||||||
|
async def collect() -> None:
|
||||||
|
async for event in world.bus.stream(conversation_id=conv.external_id):
|
||||||
|
seen.append(event)
|
||||||
|
|
||||||
|
task = asyncio.create_task(collect())
|
||||||
|
world.conversations._normal_window = 0.05
|
||||||
|
await world.conversations.inject(conv, "tick", urgency="normal", origin="крон")
|
||||||
|
await world.settle(conv, 1)
|
||||||
|
await asyncio.sleep(0.05)
|
||||||
|
task.cancel()
|
||||||
|
types = [e["type"] for e in seen]
|
||||||
|
assert "turn.start" in types and "turn.end" in types
|
||||||
|
assert "reply" not in types
|
||||||
|
assert all(e.get("origin") == "inject" for e in seen if e["type"] == "turn.start")
|
||||||
|
|
||||||
|
|
||||||
|
async def test_spawn_seeds_first_user_message(world: World) -> None:
|
||||||
|
conv = await world.conversations.spawn(
|
||||||
|
kind="branch", agent="a", seed="brief", text="do X", title="t"
|
||||||
|
)
|
||||||
|
await world.settle(conv, 1)
|
||||||
|
prompt = ScriptedClient.instances[0].prompts[0]
|
||||||
|
assert prompt.startswith("[сид: brief] branch «t», ")
|
||||||
|
assert prompt.endswith("\n\ndo X")
|
||||||
|
assert ScriptedClient.instances[0].options.system_prompt == "hi"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_fork_leaves_original_untouched(world: World) -> None:
|
||||||
|
sid = str(uuid.uuid4())
|
||||||
|
history = [
|
||||||
|
{"role": "user", "content": "one"},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": [{"type": "tool_use", "id": "t1", "name": "Read", "input": {}}],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": [{"type": "tool_result", "tool_use_id": "t1", "content": "x"}],
|
||||||
|
},
|
||||||
|
{"role": "assistant", "content": "done one"},
|
||||||
|
{"role": "user", "content": "two"},
|
||||||
|
{"role": "assistant", "content": "done two"},
|
||||||
|
{"role": "user", "content": "three"},
|
||||||
|
{"role": "assistant", "content": "done three"},
|
||||||
|
]
|
||||||
|
entries = build_entries(history, session_id=sid, cwd=str(world.root), model="m")
|
||||||
|
await world.store.append(world.key(sid), entries)
|
||||||
|
conv = await world.conversations.create(
|
||||||
|
kind="branch", agent="a", origin="test", session_id=sid
|
||||||
|
)
|
||||||
|
before = [dict(e) for e in await world.store.load(world.key(sid))]
|
||||||
|
|
||||||
|
result = await world.conversations.fork(
|
||||||
|
conv, "summarize", window=2, strip_tools=True
|
||||||
|
)
|
||||||
|
|
||||||
|
after = await world.store.load(world.key(sid))
|
||||||
|
assert after == before
|
||||||
|
assert result.text == "ok:summarize"
|
||||||
|
assert result.conversation.status == "closed"
|
||||||
|
assert result.conversation.parent_id == conv.id
|
||||||
|
fork_entries = await world.store.load(world.key(result.conversation.session_id))
|
||||||
|
assert not {e["uuid"] for e in fork_entries} & {e["uuid"] for e in before}
|
||||||
|
texts = [e["message"]["content"] for e in fork_entries]
|
||||||
|
assert texts[0] == "two"
|
||||||
|
assert len(fork_entries) == 4
|
||||||
|
assert fork_entries[0]["parentUuid"] is None
|
||||||
|
assert all(
|
||||||
|
b["type"] == "text"
|
||||||
|
for e in fork_entries[1:]
|
||||||
|
for b in e["message"]["content"]
|
||||||
|
if isinstance(e["message"]["content"], list)
|
||||||
|
)
|
||||||
|
client = ScriptedClient.instances[0]
|
||||||
|
assert client.options.resume == result.conversation.session_id
|
||||||
|
assert client.options.mcp_servers == {}
|
||||||
|
assert world.pool.get(result.conversation.external_id) is None
|
||||||
|
|
||||||
|
|
||||||
|
async def test_copy_seed_forks_parent_with_window(world: World) -> None:
|
||||||
|
sid = str(uuid.uuid4())
|
||||||
|
history = [
|
||||||
|
{"role": "user", "content": f"q{i}"}
|
||||||
|
if i % 2 == 0
|
||||||
|
else {"role": "assistant", "content": f"a{i}"}
|
||||||
|
for i in range(8)
|
||||||
|
]
|
||||||
|
await world.store.append(
|
||||||
|
world.key(sid),
|
||||||
|
build_entries(history, session_id=sid, cwd=str(world.root), model="m"),
|
||||||
|
)
|
||||||
|
parent = await world.conversations.create(
|
||||||
|
kind="master", agent="a", origin="test", session_id=sid
|
||||||
|
)
|
||||||
|
child = await world.conversations.spawn(
|
||||||
|
kind="branch", agent="a", seed="copy", parent=parent, window=1
|
||||||
|
)
|
||||||
|
assert child.session_id and child.session_id != sid
|
||||||
|
copied = await world.store.load(world.key(child.session_id))
|
||||||
|
assert [e["message"]["content"] for e in copied] == [
|
||||||
|
"q6",
|
||||||
|
[{"type": "text", "text": "a7"}],
|
||||||
|
]
|
||||||
|
assert (await world.conversations.get(child.external_id)).flags["seed"] == "copy"
|
||||||
|
await world.conversations.post(child, "go")
|
||||||
|
await world.settle(child, 1)
|
||||||
|
assert ScriptedClient.instances[0].options.resume == child.session_id
|
||||||
|
prompt = ScriptedClient.instances[0].prompts[0]
|
||||||
|
assert prompt.startswith("[сид: copy] branch, ")
|
||||||
|
assert "последние 1 тёрнов" in prompt and prompt.endswith("\n\ngo")
|
||||||
|
assert (await world.conversations.get(child.external_id)).flags["seed"] is None
|
||||||
|
|
||||||
|
|
||||||
|
async def test_merge_injects_summary_into_parent(world: World) -> None:
|
||||||
|
sid = str(uuid.uuid4())
|
||||||
|
await world.store.append(
|
||||||
|
world.key(sid),
|
||||||
|
build_entries(
|
||||||
|
[{"role": "user", "content": "q"}, {"role": "assistant", "content": "a"}],
|
||||||
|
session_id=sid,
|
||||||
|
cwd=str(world.root),
|
||||||
|
model="m",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
master = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
branch = await world.conversations.create(
|
||||||
|
kind="branch", agent="a", origin="test", parent=master, session_id=sid
|
||||||
|
)
|
||||||
|
result = await world.conversations.merge(branch)
|
||||||
|
assert result.text.startswith("ok:")
|
||||||
|
assert (await world.conversations.get(branch.external_id)).status == "merged"
|
||||||
|
assert await world.statuses(master) == [("normal", "queued")]
|
||||||
|
item = (await world.conversations.queue.recent(master.id))[0]
|
||||||
|
assert item.origin == "слив" and item.text == result.text
|
||||||
|
|
||||||
|
|
||||||
|
async def test_recover_closes_open_tool_use_and_injects_interrupted(
|
||||||
|
world: World,
|
||||||
|
) -> None:
|
||||||
|
sid = str(uuid.uuid4())
|
||||||
|
history = [
|
||||||
|
{"role": "user", "content": "go"},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": [
|
||||||
|
{
|
||||||
|
"type": "tool_use",
|
||||||
|
"id": "t9",
|
||||||
|
"name": "Bash",
|
||||||
|
"input": {"command": "sleep"},
|
||||||
|
}
|
||||||
|
],
|
||||||
|
},
|
||||||
|
]
|
||||||
|
await world.store.append(
|
||||||
|
world.key(sid),
|
||||||
|
build_entries(history, session_id=sid, cwd=str(world.root), model="m"),
|
||||||
|
)
|
||||||
|
conv = await world.conversations.create(
|
||||||
|
kind="master", agent="a", origin="test", session_id=sid
|
||||||
|
)
|
||||||
|
async with world.db.session() as session:
|
||||||
|
row = await session.get(type(conv), conv.id)
|
||||||
|
row.running_turn = "turn_dead"
|
||||||
|
session.add(row)
|
||||||
|
session.add(
|
||||||
|
InjectQueueItem(
|
||||||
|
conversation_id=conv.id,
|
||||||
|
priority="user",
|
||||||
|
origin="user",
|
||||||
|
text="go",
|
||||||
|
status="running",
|
||||||
|
turn_id="turn_dead",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
cut = await world.conversations.recover()
|
||||||
|
|
||||||
|
assert [c.external_id for c in cut] == [conv.external_id]
|
||||||
|
entries = await world.store.load(world.key(sid))
|
||||||
|
assert not open_tool_uses(entries)
|
||||||
|
tail = entries[-1]
|
||||||
|
assert tail["type"] == "user"
|
||||||
|
assert tail["message"]["content"][0] == {
|
||||||
|
"type": "tool_result",
|
||||||
|
"tool_use_id": "t9",
|
||||||
|
"content": "прервано",
|
||||||
|
"is_error": True,
|
||||||
|
}
|
||||||
|
assert tail["parentUuid"] == entries[-2]["uuid"]
|
||||||
|
assert (await world.conversations.get(conv.external_id)).running_turn is None
|
||||||
|
assert await world.statuses(conv) == [("user", "interrupted"), ("normal", "queued")]
|
||||||
|
note = (await world.conversations.queue.recent(conv.id))[0]
|
||||||
|
assert (
|
||||||
|
"turn_dead" in note.text
|
||||||
|
and "оборван" in note.text
|
||||||
|
and "1 незакрытых" in note.text
|
||||||
|
)
|
||||||
|
await asyncio.sleep(0.2)
|
||||||
|
assert ScriptedClient.instances == []
|
||||||
|
|
||||||
|
|
||||||
|
async def test_graceful_stop_keeps_running_turn_for_recover(world: World) -> None:
|
||||||
|
conv = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
ScriptedClient.hold = asyncio.Event()
|
||||||
|
await world.conversations.post(conv, "long")
|
||||||
|
await asyncio.sleep(0.2)
|
||||||
|
assert (await world.conversations.get(conv.external_id)).running_turn is not None
|
||||||
|
await world.conversations.stop()
|
||||||
|
row = await world.conversations.get(conv.external_id)
|
||||||
|
assert row.running_turn is not None
|
||||||
|
assert row.session_id == ScriptedClient.instances[0].session_id
|
||||||
|
cut = await world.conversations.recover()
|
||||||
|
assert [c.external_id for c in cut] == [conv.external_id]
|
||||||
|
assert (await world.conversations.get(conv.external_id)).running_turn is None
|
||||||
|
assert await world.statuses(conv) == [("user", "interrupted"), ("normal", "queued")]
|
||||||
|
ScriptedClient.hold = None
|
||||||
|
|
||||||
|
|
||||||
|
async def test_read_and_bindings(world: World) -> None:
|
||||||
|
sid = str(uuid.uuid4())
|
||||||
|
history = [
|
||||||
|
{"role": "user", "content": "q1"},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": [
|
||||||
|
{"type": "text", "text": "a1"},
|
||||||
|
{"type": "tool_use", "id": "t", "name": "Read", "input": {}},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": [{"type": "tool_result", "tool_use_id": "t", "content": "x"}],
|
||||||
|
},
|
||||||
|
{"role": "assistant", "content": "a1b"},
|
||||||
|
{"role": "user", "content": "q2"},
|
||||||
|
{"role": "assistant", "content": "a2"},
|
||||||
|
]
|
||||||
|
await world.store.append(
|
||||||
|
world.key(sid),
|
||||||
|
build_entries(history, session_id=sid, cwd=str(world.root), model="m"),
|
||||||
|
)
|
||||||
|
conv = await world.conversations.create(
|
||||||
|
kind="deep", agent="d", origin="test", session_id=sid
|
||||||
|
)
|
||||||
|
assert (
|
||||||
|
await world.conversations.read(conv)
|
||||||
|
== "user:\nq1\n\nassistant:\na1\n\na1b\n(tools: Read)\n\nuser:\nq2\n\nassistant:\na2"
|
||||||
|
)
|
||||||
|
assert (
|
||||||
|
await world.conversations.read(conv, window=1) == "user:\nq2\n\nassistant:\na2"
|
||||||
|
)
|
||||||
|
|
||||||
|
await world.conversations.bind(conv, frontend="markdown", external_id="a.md")
|
||||||
|
await world.conversations.bind(conv, frontend="markdown", external_id="b.md")
|
||||||
|
bindings = await world.conversations.bindings(conv)
|
||||||
|
assert [(b.external_id, b.visible) for b in bindings] == [
|
||||||
|
("a.md", False),
|
||||||
|
("b.md", True),
|
||||||
|
]
|
||||||
|
found = await world.conversations.find_bound(
|
||||||
|
frontend="markdown", external_id="b.md"
|
||||||
|
)
|
||||||
|
assert found is not None and found.id == conv.id
|
||||||
|
assert (
|
||||||
|
await world.conversations.find_bound(frontend="markdown", external_id="a.md")
|
||||||
|
is None
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def test_schedule_without_scheduler_and_parse_at(world: World) -> None:
|
||||||
|
conv = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
with pytest.raises(RuntimeError, match="no scheduler"):
|
||||||
|
await world.conversations.schedule(conv, "+15m", "push X")
|
||||||
|
assert await world.conversations.schedules(conv) == []
|
||||||
|
assert parse_at("2026-09-01T10:00:00+02:00") == datetime(
|
||||||
|
2026, 9, 1, 8, 0, tzinfo=UTC
|
||||||
|
)
|
||||||
|
delta = (parse_at("+15m") - datetime.now(UTC)).total_seconds()
|
||||||
|
assert 14 * 60 < delta <= 15 * 60
|
||||||
|
with pytest.raises(ValueError, match="Invalid isoformat"):
|
||||||
|
parse_at("tomorrow")
|
||||||
|
|
||||||
|
|
||||||
|
def test_transcript_helpers() -> None:
|
||||||
|
history = [
|
||||||
|
{"role": "user", "content": "q1"},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": [
|
||||||
|
{"type": "thinking", "thinking": "hm", "signature": "s"},
|
||||||
|
{"type": "tool_use", "id": "t", "name": "Read", "input": {}},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": [{"type": "tool_result", "tool_use_id": "t", "content": "x"}],
|
||||||
|
},
|
||||||
|
{"role": "assistant", "content": "a1"},
|
||||||
|
{"role": "user", "content": "q2"},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": [{"type": "tool_use", "id": "t2", "name": "Bash", "input": {}}],
|
||||||
|
},
|
||||||
|
]
|
||||||
|
entries = build_entries(history, session_id="s", cwd="/v", model="m")
|
||||||
|
assert [b["id"] for _, b in open_tool_uses(entries)] == ["t2"]
|
||||||
|
fixes = close_open_tool_uses(entries)
|
||||||
|
assert len(fixes) == 1 and fixes[0]["sessionId"] == "s" and fixes[0]["cwd"] == "/v"
|
||||||
|
assert not open_tool_uses([*entries, *fixes])
|
||||||
|
stripped = strip_tool_entries(entries)
|
||||||
|
assert [e["message"]["content"] for e in stripped] == [
|
||||||
|
"q1",
|
||||||
|
[{"type": "text", "text": "a1"}],
|
||||||
|
"q2",
|
||||||
|
]
|
||||||
|
assert (
|
||||||
|
stripped[0]["parentUuid"] is None
|
||||||
|
and stripped[1]["parentUuid"] == stripped[0]["uuid"]
|
||||||
|
)
|
||||||
|
windowed = window_entries(entries, window=1)
|
||||||
|
assert windowed[0]["message"]["content"] == "q2" and len(windowed) == 2
|
||||||
|
assert (
|
||||||
|
render_messages(
|
||||||
|
[{"role": "user", "content": "q"}, {"role": "assistant", "content": "a"}]
|
||||||
|
)
|
||||||
|
== "user:\nq\n\nassistant:\na"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def test_agent_kinds_gate_create(world: World) -> None:
|
||||||
|
with pytest.raises(ValueError, match="does not serve kind 'deep'"):
|
||||||
|
await world.conversations.create(kind="deep", agent="a", origin="test")
|
||||||
|
with pytest.raises(ValueError, match="does not serve kind 'master'"):
|
||||||
|
await world.conversations.create(kind="master", agent="d", origin="test")
|
||||||
|
with pytest.raises(ValueError, match="'d' does not serve kind 'master'"):
|
||||||
|
await world.conversations.spawn(kind="master", seed="clean", agent="d")
|
||||||
|
conv = await world.conversations.create(kind="deep", agent="d", origin="test")
|
||||||
|
assert conv.agent_name == "d"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_frontend_kinds_gate_bind(world: World) -> None:
|
||||||
|
master = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
with pytest.raises(ValueError, match="'markdown' does not show kind 'master'"):
|
||||||
|
await world.conversations.bind(master, frontend="markdown", external_id="m.md")
|
||||||
|
with pytest.raises(ValueError, match="unknown frontend 'telegram'"):
|
||||||
|
await world.conversations.bind(master, frontend="telegram", external_id="1")
|
||||||
|
await world.conversations.bind(master, frontend="api", external_id="x")
|
||||||
|
|
||||||
|
|
||||||
|
async def test_spawn_defaults_agent_and_materializes(world: World) -> None:
|
||||||
|
master = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
deep = await world.conversations.spawn(
|
||||||
|
kind="deep", seed="brief", text="dig", title="t", parent=master, origin="mcp"
|
||||||
|
)
|
||||||
|
assert deep.agent_name == "d"
|
||||||
|
assert world.markdown.materialized == [deep.external_id]
|
||||||
|
bindings = await world.conversations.bindings(deep)
|
||||||
|
assert [(b.frontend, b.external_id) for b in bindings] == [
|
||||||
|
("markdown", f"markdown:{deep.external_id}")
|
||||||
|
]
|
||||||
|
branch = await world.conversations.spawn(kind="branch", seed="clean", parent=master)
|
||||||
|
assert branch.agent_name == "a"
|
||||||
|
assert world.markdown.materialized == [deep.external_id]
|
||||||
|
with pytest.raises(ValueError, match="no default agent for kind 'job'"):
|
||||||
|
await world.conversations.spawn(kind="job", seed="clean")
|
||||||
|
await world.settle(deep, 1)
|
||||||
|
|
||||||
|
|
||||||
|
def test_agent_kinds_follow_prompts(tmp_path: Path) -> None:
|
||||||
|
from beaver_gateway.agents.claude import Prompts
|
||||||
|
|
||||||
|
plain = ClaudeAgent(name="p", model="m", system_prompt="hi", cwd=tmp_path)
|
||||||
|
assert plain.kinds == ("deep",)
|
||||||
|
dispatcher = ClaudeAgent(
|
||||||
|
name="x", model="m", cwd=tmp_path, prompts=Prompts(master=("a.md",), fork=())
|
||||||
|
)
|
||||||
|
assert dispatcher.kinds == ("master", "fork")
|
||||||
|
assert dispatcher.prompt_for("master") == ("a.md",)
|
||||||
|
assert dispatcher.prompt_for("deep") is None
|
||||||
|
with pytest.raises(ValueError, match=r"serves \['deep'\] without a prompt"):
|
||||||
|
ClaudeAgent(
|
||||||
|
name="y",
|
||||||
|
model="m",
|
||||||
|
cwd=tmp_path,
|
||||||
|
kinds=("master", "deep"),
|
||||||
|
prompts=Prompts(master=()),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def test_observer_publishes_tool_results_for_the_panel(world: World) -> None:
|
||||||
|
seen: list[dict[str, Any]] = []
|
||||||
|
conv = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
|
||||||
|
async def collect() -> None:
|
||||||
|
async for event in world.bus.stream(conversation_id=conv.external_id):
|
||||||
|
seen.append(event)
|
||||||
|
|
||||||
|
task = asyncio.create_task(collect())
|
||||||
|
await asyncio.sleep(0)
|
||||||
|
runner = world.conversations._runner(conv.id)
|
||||||
|
runner.turn_id = "t1"
|
||||||
|
observe = world.conversations._observer(conv, runner, "t1", "user")
|
||||||
|
observe(
|
||||||
|
AssistantMessage(
|
||||||
|
content=[ToolUseBlock(id="toolu_1", name="Bash", input={"command": "ls"})],
|
||||||
|
model="m",
|
||||||
|
parent_tool_use_id="toolu_0",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
observe(
|
||||||
|
UserMessage(
|
||||||
|
content=[
|
||||||
|
ToolResultBlock(tool_use_id="toolu_1", content="a\nb", is_error=False)
|
||||||
|
],
|
||||||
|
parent_tool_use_id="toolu_0",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
observe(UserMessage(content="plain text, no tool result"))
|
||||||
|
await asyncio.sleep(0.05)
|
||||||
|
task.cancel()
|
||||||
|
assert [e["type"] for e in seen] == ["tool", "tool.result"]
|
||||||
|
tool, result = seen
|
||||||
|
assert tool["tool_use_id"] == "toolu_1" and tool["parent_tool_use_id"] == "toolu_0"
|
||||||
|
snapshot = runner.snapshot()
|
||||||
|
assert snapshot is not None and snapshot["tools"][0]["content"] == "a\nb"
|
||||||
|
assert result["tool_use_id"] == "toolu_1"
|
||||||
|
assert result["parent_tool_use_id"] == "toolu_0"
|
||||||
|
assert result["turn_id"] == "t1" and result["is_error"] is False
|
||||||
|
assert result["content"] == "a\nb"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_wake_injects_start_a_turn_and_take_normals_along(world: World) -> None:
|
||||||
|
conv = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
await world.conversations.inject(conv, "digest", urgency="normal", origin="watch")
|
||||||
|
await asyncio.sleep(0.2)
|
||||||
|
assert await world.statuses(conv) == [("normal", "queued")]
|
||||||
|
await world.conversations.inject(
|
||||||
|
conv, "reminder", urgency="wake", origin="schedule"
|
||||||
|
)
|
||||||
|
await world.settle(conv, 2)
|
||||||
|
prompts = ScriptedClient.instances[0].prompts
|
||||||
|
assert len(prompts) == 1
|
||||||
|
assert prompts[0].startswith("[инжект: schedule")
|
||||||
|
assert "reminder" in prompts[0]
|
||||||
|
assert "[инжект: watch" in prompts[0]
|
||||||
|
assert "digest" in prompts[0]
|
||||||
|
assert await world.statuses(conv) == [("normal", "done"), ("wake", "done")]
|
||||||
|
|
||||||
|
|
||||||
|
async def test_wake_does_not_cut_a_running_turn(world: World) -> None:
|
||||||
|
conv = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
ScriptedClient.hold = asyncio.Event()
|
||||||
|
await world.conversations.post(conv, "first")
|
||||||
|
await asyncio.sleep(0.2)
|
||||||
|
await world.conversations.inject(conv, "later", urgency="wake", origin="schedule")
|
||||||
|
await asyncio.sleep(0.2)
|
||||||
|
assert await world.statuses(conv) == [("user", "running"), ("wake", "queued")]
|
||||||
|
assert not ScriptedClient.instances[0].interrupted
|
||||||
|
assert await world.conversations.busy(conv)
|
||||||
|
ScriptedClient.hold.set()
|
||||||
|
await world.settle(conv, 2)
|
||||||
|
assert await world.statuses(conv) == [("user", "done"), ("wake", "done")]
|
||||||
|
assert ScriptedClient.instances[0].prompts[1].endswith("later")
|
||||||
|
|
||||||
|
|
||||||
|
async def test_resolve_aliases_master_and_parent(world: World) -> None:
|
||||||
|
master = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
branch = await world.conversations.create(
|
||||||
|
kind="branch", agent="a", parent=master, origin="test"
|
||||||
|
)
|
||||||
|
resolve = world.conversations.resolve
|
||||||
|
assert (await resolve("master")).id == master.id
|
||||||
|
assert (await resolve(" master ")).id == master.id
|
||||||
|
assert (await resolve("parent", origin=branch)).id == master.id
|
||||||
|
assert await resolve("parent", origin=master) is None
|
||||||
|
assert await resolve("parent") is None
|
||||||
|
assert (await resolve(branch.external_id)).id == branch.id
|
||||||
|
assert await resolve("nope") is None
|
||||||
|
await world.conversations.set_status(master, "closed")
|
||||||
|
assert await resolve("master") is None
|
||||||
|
|
||||||
|
|
||||||
|
async def test_inject_into_a_closed_master_lands_in_the_open_one(world: World) -> None:
|
||||||
|
old = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
old = await world.conversations.set_status(old, "closed")
|
||||||
|
new = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
item = await world.conversations.inject(old, "late", urgency="normal", origin="x")
|
||||||
|
assert item.conversation_id == new.id
|
||||||
|
assert await world.statuses(old) == []
|
||||||
|
assert await world.statuses(new) == [("normal", "queued")]
|
||||||
@@ -0,0 +1,61 @@
|
|||||||
|
import sqlite3
|
||||||
|
import tempfile
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from sqlmodel import select
|
||||||
|
|
||||||
|
from beaver_gateway.storage import Database
|
||||||
|
from beaver_gateway.storage.models import Conversation
|
||||||
|
|
||||||
|
|
||||||
|
async def test_create_all_adds_missing_columns() -> None:
|
||||||
|
path = Path(tempfile.mkdtemp(prefix="beaver-migrate-")) / "old.db"
|
||||||
|
raw = sqlite3.connect(path)
|
||||||
|
raw.execute(
|
||||||
|
"CREATE TABLE conversations (id INTEGER PRIMARY KEY, frontend VARCHAR NOT NULL, "
|
||||||
|
"external_id VARCHAR NOT NULL, agent_name VARCHAR NOT NULL, "
|
||||||
|
"created_at DATETIME NOT NULL, updated_at DATETIME NOT NULL)"
|
||||||
|
)
|
||||||
|
raw.execute(
|
||||||
|
"INSERT INTO conversations VALUES (1, 'markdown', 'x', 'a', '2026-01-01', '2026-01-01')"
|
||||||
|
)
|
||||||
|
raw.commit()
|
||||||
|
raw.close()
|
||||||
|
|
||||||
|
db = Database(f"sqlite:///{path}")
|
||||||
|
await db.create_all()
|
||||||
|
async with db.session() as session:
|
||||||
|
conv = (await session.exec(select(Conversation))).one()
|
||||||
|
assert conv.session_id is None
|
||||||
|
conv.session_id = "sid"
|
||||||
|
session.add(conv)
|
||||||
|
await session.commit()
|
||||||
|
await db.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_create_all_backfills_defaults_for_old_rows() -> None:
|
||||||
|
path = Path(tempfile.mkdtemp(prefix="beaver-migrate-")) / "old.db"
|
||||||
|
raw = sqlite3.connect(path)
|
||||||
|
raw.execute(
|
||||||
|
"CREATE TABLE conversations (id INTEGER PRIMARY KEY, frontend VARCHAR NOT NULL, "
|
||||||
|
"external_id VARCHAR NOT NULL, agent_name VARCHAR NOT NULL, session_id VARCHAR, "
|
||||||
|
"created_at DATETIME NOT NULL, updated_at DATETIME NOT NULL)"
|
||||||
|
)
|
||||||
|
raw.execute(
|
||||||
|
"INSERT INTO conversations VALUES (1, 'markdown', 'x', 'a', NULL, '2026-01-01', '2026-01-01')"
|
||||||
|
)
|
||||||
|
raw.commit()
|
||||||
|
raw.close()
|
||||||
|
|
||||||
|
db = Database(f"sqlite:///{path}")
|
||||||
|
await db.create_all()
|
||||||
|
async with db.session() as session:
|
||||||
|
conv = (await session.exec(select(Conversation))).one()
|
||||||
|
assert (conv.kind, conv.status, conv.pending_question, conv.flags) == (
|
||||||
|
"deep",
|
||||||
|
"open",
|
||||||
|
False,
|
||||||
|
{},
|
||||||
|
)
|
||||||
|
assert conv.running_turn is None
|
||||||
|
await db.dispose()
|
||||||
@@ -0,0 +1,419 @@
|
|||||||
|
import asyncio
|
||||||
|
import uuid
|
||||||
|
from datetime import UTC, datetime, timedelta
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from claude_agent_sdk import (
|
||||||
|
AssistantMessage,
|
||||||
|
ResultMessage,
|
||||||
|
TextBlock,
|
||||||
|
ToolResultBlock,
|
||||||
|
ToolUseBlock,
|
||||||
|
UserMessage,
|
||||||
|
project_key_for_directory,
|
||||||
|
)
|
||||||
|
from test_conversations import ScriptedClient, World, world
|
||||||
|
|
||||||
|
from beaver_gateway.agents.claude import ClaudeAgent, ClaudeOptions
|
||||||
|
from beaver_gateway.backends.claude_sdk import ClaudeSdkBackend
|
||||||
|
from beaver_gateway.core.conversations import ConversationTexts
|
||||||
|
from beaver_gateway.core.distill import (
|
||||||
|
Distiller,
|
||||||
|
DistillContext,
|
||||||
|
LineCap,
|
||||||
|
check_digest,
|
||||||
|
trim_summary,
|
||||||
|
)
|
||||||
|
from beaver_gateway.core.gateway_tools import _tools, build_tool_server
|
||||||
|
from beaver_gateway.core.registry import AgentRegistry
|
||||||
|
from beaver_gateway.core.scheduler import Job, JobRun, Scheduler
|
||||||
|
from beaver_gateway.core.transcript import build_entries
|
||||||
|
from beaver_gateway.storage.models import Conversation
|
||||||
|
|
||||||
|
__all__ = ["world"]
|
||||||
|
|
||||||
|
DIGEST = """---
|
||||||
|
type: выжимка
|
||||||
|
source: "[[{chat}]]"
|
||||||
|
date: 2026-08-29
|
||||||
|
---
|
||||||
|
# тема
|
||||||
|
|
||||||
|
## контекст
|
||||||
|
зачем открывали
|
||||||
|
|
||||||
|
## решили
|
||||||
|
- одно
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
class DistillerClient(ScriptedClient):
|
||||||
|
"""The distiller: writes a digest file when told to, answers in N lines."""
|
||||||
|
|
||||||
|
digest_dir: Path | None = None
|
||||||
|
lines = 3
|
||||||
|
write = True
|
||||||
|
frontmatter = DIGEST
|
||||||
|
|
||||||
|
async def receive_response(self):
|
||||||
|
prompt = self.prompts[-1]
|
||||||
|
written: list[ToolUseBlock] = []
|
||||||
|
if self.write and self.digest_dir is not None and "файл не пиши" not in prompt:
|
||||||
|
chat = prompt.split("«", 1)[1].split("»", 1)[0] if "«" in prompt else "чат"
|
||||||
|
path = self.digest_dir / "2026-08-29 - тема.md"
|
||||||
|
path.write_text(self.frontmatter.format(chat=chat), encoding="utf-8")
|
||||||
|
written.append(
|
||||||
|
ToolUseBlock(id="w1", name="Write", input={"file_path": str(path)})
|
||||||
|
)
|
||||||
|
for block in written:
|
||||||
|
yield AssistantMessage(content=[block], model="m")
|
||||||
|
yield UserMessage(content=[ToolResultBlock(tool_use_id=block.id)])
|
||||||
|
text = "\n".join(f"строка {i} про {prompt[:20]!r}" for i in range(self.lines))
|
||||||
|
yield AssistantMessage(content=[TextBlock(text=text)], model="m")
|
||||||
|
yield ResultMessage(
|
||||||
|
subtype="success",
|
||||||
|
duration_ms=1,
|
||||||
|
duration_api_ms=1,
|
||||||
|
is_error=False,
|
||||||
|
num_turns=1,
|
||||||
|
session_id=self.session_id,
|
||||||
|
stop_reason="end_turn",
|
||||||
|
total_cost_usd=0.0,
|
||||||
|
usage={"input_tokens": 1, "output_tokens": 1},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ClosingClient(ScriptedClient):
|
||||||
|
"""A deep chat that calls ``close_chat`` inside its reply."""
|
||||||
|
|
||||||
|
conversations: Any = None
|
||||||
|
key = ""
|
||||||
|
|
||||||
|
async def receive_response(self):
|
||||||
|
prompt = self.prompts[-1]
|
||||||
|
if "обсудили" in prompt:
|
||||||
|
assert "gateway" in self.options.mcp_servers
|
||||||
|
tools = _tools(ClosingClient.conversations, ClosingClient.key)
|
||||||
|
close = next(t for t in tools if t.name == "close_chat")
|
||||||
|
result = await close.handler({})
|
||||||
|
assert "closes after this reply" in result["content"][0]["text"]
|
||||||
|
yield AssistantMessage(
|
||||||
|
content=[
|
||||||
|
ToolUseBlock(id="c1", name="mcp__gateway__close_chat", input={})
|
||||||
|
],
|
||||||
|
model="m",
|
||||||
|
)
|
||||||
|
yield AssistantMessage(content=[TextBlock(text=f"ok:{prompt}")], model="m")
|
||||||
|
yield ResultMessage(
|
||||||
|
subtype="success",
|
||||||
|
duration_ms=1,
|
||||||
|
duration_api_ms=1,
|
||||||
|
is_error=False,
|
||||||
|
num_turns=1,
|
||||||
|
session_id=self.session_id,
|
||||||
|
stop_reason="end_turn",
|
||||||
|
total_cost_usd=0.0,
|
||||||
|
usage={"input_tokens": 1, "output_tokens": 1},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class CapClient(ScriptedClient):
|
||||||
|
"""A job that rewrites the capped file with ``lines`` lines."""
|
||||||
|
|
||||||
|
target: Path | None = None
|
||||||
|
lines = 70
|
||||||
|
|
||||||
|
async def receive_response(self):
|
||||||
|
assert self.target is not None
|
||||||
|
self.target.write_text(
|
||||||
|
"\n".join(f"- строка {i}" for i in range(self.lines)), encoding="utf-8"
|
||||||
|
)
|
||||||
|
CapClient.lines = 10
|
||||||
|
async for event in super().receive_response():
|
||||||
|
yield event
|
||||||
|
|
||||||
|
|
||||||
|
def distiller(world: World, client: type[ScriptedClient]) -> Distiller:
|
||||||
|
vault = world.root / "vault"
|
||||||
|
digests = vault / "мета" / "бобер" / "выжимки"
|
||||||
|
digests.mkdir(parents=True)
|
||||||
|
agent = ClaudeAgent(
|
||||||
|
name="x",
|
||||||
|
model="m",
|
||||||
|
system_prompt="distill",
|
||||||
|
cwd=vault,
|
||||||
|
kinds=("fork", "job"),
|
||||||
|
options=ClaudeOptions(
|
||||||
|
effort="medium", tools=("Read", "Write"), include_partial_messages=False
|
||||||
|
),
|
||||||
|
)
|
||||||
|
backend = ClaudeSdkBackend(
|
||||||
|
agent=agent,
|
||||||
|
mcp_internal_urls={},
|
||||||
|
session_store=world.store,
|
||||||
|
client_factory=client,
|
||||||
|
work_dir=world.root / "work",
|
||||||
|
pool=world.pool,
|
||||||
|
)
|
||||||
|
world.conversations._agents = AgentRegistry( # noqa: SLF001
|
||||||
|
[world.agent, world.deep_agent, agent]
|
||||||
|
)
|
||||||
|
world.conversations._backends["x"] = backend # noqa: SLF001
|
||||||
|
config = Distiller(agent="x", dir=digests, index=digests.parent / "индекс.md")
|
||||||
|
world.conversations._distiller = config # noqa: SLF001
|
||||||
|
DistillerClient.digest_dir = digests
|
||||||
|
DistillerClient.lines = 3
|
||||||
|
DistillerClient.write = True
|
||||||
|
DistillerClient.frontmatter = DIGEST
|
||||||
|
return config
|
||||||
|
|
||||||
|
|
||||||
|
async def deep_chat(world: World, name: str = "2026-08-20 - тема чата") -> Conversation:
|
||||||
|
sid = str(uuid.uuid4())
|
||||||
|
await world.store.append(
|
||||||
|
world.key(sid),
|
||||||
|
build_entries(
|
||||||
|
[
|
||||||
|
{"role": "user", "content": "q"},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": [
|
||||||
|
{"type": "tool_use", "id": "t1", "name": "Read", "input": {}}
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": [
|
||||||
|
{"type": "tool_result", "tool_use_id": "t1", "content": "x"}
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{"role": "assistant", "content": "a"},
|
||||||
|
],
|
||||||
|
session_id=sid,
|
||||||
|
cwd=str(world.root),
|
||||||
|
model="m",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
conv = await world.conversations.create(
|
||||||
|
kind="deep", agent="d", origin="markdown", session_id=sid
|
||||||
|
)
|
||||||
|
await world.conversations.bind(
|
||||||
|
conv, frontend="markdown", external_id=f"2026-08/{name}.md"
|
||||||
|
)
|
||||||
|
return conv
|
||||||
|
|
||||||
|
|
||||||
|
async def test_distill_writes_the_digest_indexes_it_and_merges_short(
|
||||||
|
world: World,
|
||||||
|
) -> None:
|
||||||
|
config = distiller(world, DistillerClient)
|
||||||
|
prompts: list[DistillContext] = []
|
||||||
|
|
||||||
|
def distill_prompt(ctx: DistillContext) -> str:
|
||||||
|
prompts.append(ctx)
|
||||||
|
return f"Чат «{ctx.chat_name}» закрыт ({ctx.reason})."
|
||||||
|
|
||||||
|
world.conversations._texts = ConversationTexts(distill=distill_prompt) # noqa: SLF001
|
||||||
|
master = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
chat = await deep_chat(world)
|
||||||
|
|
||||||
|
result = await world.conversations.distill(chat, reason="api")
|
||||||
|
|
||||||
|
assert prompts[0].chat_name == "2026-08-20 - тема чата"
|
||||||
|
assert prompts[0].memory is True
|
||||||
|
assert result.digest is not None and result.error is None
|
||||||
|
assert result.digest.path == config.dir / "2026-08-29 - тема.md"
|
||||||
|
assert result.digest.source == "[[2026-08-20 - тема чата]]"
|
||||||
|
index = config.index.read_text(encoding="utf-8")
|
||||||
|
assert index.startswith("# индекс")
|
||||||
|
assert "- 2026-08-29 [[2026-08-20 - тема чата]] → [[2026-08-29 - тема]]" in index
|
||||||
|
assert result.text.count("\n") == 2 and not result.trimmed
|
||||||
|
closed = await world.conversations.get(chat.external_id)
|
||||||
|
assert closed.status == "closed"
|
||||||
|
assert closed.flags["digest"] == str(result.digest.path)
|
||||||
|
assert closed.flags["closed_reason"] == "api"
|
||||||
|
fork = await world.conversations.get(result.fork.external_id)
|
||||||
|
assert fork.kind == "fork" and fork.agent_name == "x" and fork.status == "closed"
|
||||||
|
items = await world.conversations.queue.recent(master.id)
|
||||||
|
assert items[0].origin == "выжимка"
|
||||||
|
assert items[0].text.startswith(
|
||||||
|
"Закрыт глубокий чат [[2026-08-20 - тема чата]], выжимка [[2026-08-29 - тема]]."
|
||||||
|
)
|
||||||
|
assert items[0].text.endswith(result.text)
|
||||||
|
forked = ScriptedClient.instances[-1]
|
||||||
|
assert forked.options.resume is not None
|
||||||
|
vault_key = {
|
||||||
|
"project_key": project_key_for_directory(str(config.dir.parents[2])),
|
||||||
|
"session_id": forked.options.resume,
|
||||||
|
}
|
||||||
|
entries = await world.store.load(vault_key)
|
||||||
|
assert entries
|
||||||
|
assert all(
|
||||||
|
not isinstance(e.get("message", {}).get("content"), list)
|
||||||
|
or all(b.get("type") == "text" for b in e["message"]["content"])
|
||||||
|
for e in entries
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def test_memory_off_merges_without_a_file(world: World) -> None:
|
||||||
|
config = distiller(world, DistillerClient)
|
||||||
|
master = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
chat = await deep_chat(world)
|
||||||
|
await world.conversations.set_flags(chat, {"memory": False})
|
||||||
|
|
||||||
|
result = await world.conversations.distill(chat, reason="api")
|
||||||
|
|
||||||
|
assert result.digest is None and result.error is None
|
||||||
|
assert list(config.dir.iterdir()) == []
|
||||||
|
assert not config.index.exists()
|
||||||
|
assert (await world.conversations.get(chat.external_id)).status == "closed"
|
||||||
|
items = await world.conversations.queue.recent(master.id)
|
||||||
|
assert len(items) == 1 and items[0].text.endswith(result.text)
|
||||||
|
assert ", выжимка" not in items[0].text
|
||||||
|
assert "файл не пиши" in ScriptedClient.instances[-1].prompts[0]
|
||||||
|
|
||||||
|
|
||||||
|
async def test_bad_frontmatter_and_long_merge_are_reported(world: World) -> None:
|
||||||
|
distiller(world, DistillerClient)
|
||||||
|
DistillerClient.frontmatter = "---\ntype: заметка\n---\n# тема\n"
|
||||||
|
DistillerClient.lines = 9
|
||||||
|
await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
chat = await deep_chat(world)
|
||||||
|
|
||||||
|
result = await world.conversations.distill(chat, reason="idle 2d")
|
||||||
|
|
||||||
|
assert result.digest is None
|
||||||
|
assert result.error is not None and "`type`" in result.error
|
||||||
|
assert result.trimmed and result.text.count("\n") == 4
|
||||||
|
closed = await world.conversations.get(chat.external_id)
|
||||||
|
assert closed.status == "closed"
|
||||||
|
assert closed.flags["digest_error"] == result.error
|
||||||
|
|
||||||
|
|
||||||
|
def test_check_digest_rejects_what_is_not_a_digest(tmp_path: Path) -> None:
|
||||||
|
config = Distiller(agent="x", dir=tmp_path, index=tmp_path / "i.md")
|
||||||
|
path = tmp_path / "d.md"
|
||||||
|
path.write_text("---\ntype: выжимка\nsource: ''\ndate: 2026-08-29\n---\nx\n")
|
||||||
|
assert check_digest(path, config) == "`source` пустой"
|
||||||
|
path.write_text("---\ntype: выжимка\nsource: '[[a]]'\ndate: вчера\n---\nx\n")
|
||||||
|
assert "`date`" in check_digest(path, config)
|
||||||
|
path.write_text("---\ntype: выжимка\nsource: '[[a]]'\ndate: 2026-08-29\n---\n\n")
|
||||||
|
assert check_digest(path, config) == "тело пустое"
|
||||||
|
path.write_text("---\ntype: выжимка\nsource: '[[a]]'\ndate: 2026-08-29\n---\nx\n")
|
||||||
|
digest = check_digest(path, config)
|
||||||
|
assert not isinstance(digest, str) and digest.date.isoformat() == "2026-08-29"
|
||||||
|
assert trim_summary("a\n\nb\nc\nd\ne\nf") == ("a\nb\nc\nd\ne", True)
|
||||||
|
|
||||||
|
|
||||||
|
async def test_close_chat_tool_closes_after_the_reply(world: World) -> None:
|
||||||
|
distiller(world, DistillerClient)
|
||||||
|
world.deep_agent = ClaudeAgent(
|
||||||
|
name="d",
|
||||||
|
model="m",
|
||||||
|
system_prompt="deep",
|
||||||
|
cwd=world.root,
|
||||||
|
gateway_tools=("close_chat",),
|
||||||
|
)
|
||||||
|
world.conversations._agents = AgentRegistry( # noqa: SLF001
|
||||||
|
[world.agent, world.deep_agent, world.conversations._agents.get("x")] # noqa: SLF001
|
||||||
|
)
|
||||||
|
world.conversations._backends["d"] = ClaudeSdkBackend( # noqa: SLF001
|
||||||
|
agent=world.deep_agent,
|
||||||
|
mcp_internal_urls={},
|
||||||
|
session_store=world.store,
|
||||||
|
client_factory=ClosingClient,
|
||||||
|
work_dir=world.root / "work",
|
||||||
|
pool=world.pool,
|
||||||
|
tool_server=lambda key, _kind: build_tool_server(
|
||||||
|
world.conversations, conversation_key=key, names=("close_chat",)
|
||||||
|
),
|
||||||
|
)
|
||||||
|
master = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
chat = await deep_chat(world)
|
||||||
|
ClosingClient.conversations = world.conversations
|
||||||
|
ClosingClient.key = chat.external_id
|
||||||
|
|
||||||
|
await world.conversations.post(chat, "ок, обсудили")
|
||||||
|
await world.settle(chat, 1)
|
||||||
|
for _ in range(100):
|
||||||
|
row = await world.conversations.get(chat.external_id)
|
||||||
|
if row.status == "closed":
|
||||||
|
break
|
||||||
|
await asyncio.sleep(0.05)
|
||||||
|
else:
|
||||||
|
raise AssertionError("chat did not close")
|
||||||
|
|
||||||
|
assert row.flags["closed_reason"] == "close_chat"
|
||||||
|
assert row.flags["digest"] is not None
|
||||||
|
items = await world.conversations.queue.recent(master.id)
|
||||||
|
assert items[0].origin == "выжимка"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_idle_picks_quiet_chats_after_launch_at_most_limit(world: World) -> None:
|
||||||
|
distiller(world, DistillerClient)
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
launch = now - timedelta(days=10)
|
||||||
|
|
||||||
|
async def aged(name: str, days: float) -> Conversation:
|
||||||
|
conv = await deep_chat(world, name)
|
||||||
|
|
||||||
|
async def apply(row: Conversation) -> None:
|
||||||
|
row.last_activity_at = now - timedelta(days=days)
|
||||||
|
|
||||||
|
return await world.conversations._update(conv, apply) # noqa: SLF001
|
||||||
|
|
||||||
|
old = await aged("old", 30)
|
||||||
|
a = await aged("a", 5)
|
||||||
|
b = await aged("b", 4)
|
||||||
|
c = await aged("c", 3)
|
||||||
|
d = await aged("d", 2.5)
|
||||||
|
fresh = await aged("fresh", 1)
|
||||||
|
no_session = await world.conversations.create(kind="deep", agent="d", origin="t")
|
||||||
|
|
||||||
|
idle = await world.conversations.idle(kind="deep", days=2, since=launch)
|
||||||
|
assert [x.external_id for x in idle] == [
|
||||||
|
a.external_id,
|
||||||
|
b.external_id,
|
||||||
|
c.external_id,
|
||||||
|
d.external_id,
|
||||||
|
]
|
||||||
|
assert old.external_id not in {x.external_id for x in idle}
|
||||||
|
assert fresh.external_id not in {x.external_id for x in idle}
|
||||||
|
assert no_session.external_id not in {x.external_id for x in idle}
|
||||||
|
|
||||||
|
scheduler = Scheduler(conversations=world.conversations)
|
||||||
|
run = JobRun(Job("закрытие", lambda _run: asyncio.sleep(0)), "cron", {}, scheduler)
|
||||||
|
closed = await run.close_idle(kind="deep", days=2, limit=3, since=launch)
|
||||||
|
assert [r.conversation.external_id for r in closed] == [
|
||||||
|
a.external_id,
|
||||||
|
b.external_id,
|
||||||
|
c.external_id,
|
||||||
|
]
|
||||||
|
assert (await world.conversations.get(d.external_id)).status == "open"
|
||||||
|
assert len(await world.conversations.idle(kind="deep", days=2, since=launch)) == 1
|
||||||
|
|
||||||
|
|
||||||
|
async def test_line_cap_bounces_a_long_rewrite_and_asks_to_shorten(
|
||||||
|
world: World,
|
||||||
|
) -> None:
|
||||||
|
distiller(world, CapClient)
|
||||||
|
state = world.root / "vault" / "состояние.md"
|
||||||
|
state.write_text("# состояние\n- было так\n", encoding="utf-8")
|
||||||
|
CapClient.target = state
|
||||||
|
CapClient.lines = 70
|
||||||
|
scheduler = Scheduler(conversations=world.conversations)
|
||||||
|
run = JobRun(Job("память", lambda _run: asyncio.sleep(0)), "cron", {}, scheduler)
|
||||||
|
|
||||||
|
job = await run.spawn_job(
|
||||||
|
agent="x", text="перепиши", line_cap=LineCap(state, max_lines=60)
|
||||||
|
)
|
||||||
|
await world.settle(job, 2)
|
||||||
|
|
||||||
|
client = ScriptedClient.instances[-1]
|
||||||
|
assert len(client.prompts) == 2
|
||||||
|
assert "70 строк при потолке 60" in client.prompts[1]
|
||||||
|
assert "[инжект: потолок" in client.prompts[1]
|
||||||
|
assert state.read_text(encoding="utf-8").count("\n") == 10 - 1
|
||||||
|
row = await world.conversations.get(job.external_id)
|
||||||
|
assert row.flags["line_cap_attempts"] == 1
|
||||||
@@ -0,0 +1,218 @@
|
|||||||
|
import asyncio
|
||||||
|
import tempfile
|
||||||
|
from datetime import UTC, datetime, timedelta
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from test_conversations import ScriptedClient, World, world
|
||||||
|
|
||||||
|
from beaver_gateway.core.conversations import UserSaid
|
||||||
|
from beaver_gateway.core.envelope import HEADER, Envelope, RecallContext, render
|
||||||
|
from beaver_gateway.core.watch import Change, VaultWatch, WatchRules
|
||||||
|
|
||||||
|
__all__ = ["world"]
|
||||||
|
|
||||||
|
RULES = WatchRules(
|
||||||
|
full=("дни/{today}.md",),
|
||||||
|
names=("дни/*", "люди/*", "мета/бобер/*"),
|
||||||
|
ignore=("чаты/*", "мета/*"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def vault() -> tuple[Path, VaultWatch]:
|
||||||
|
root = Path(tempfile.mkdtemp(prefix="beaver-vault-"))
|
||||||
|
for sub in ("дни", "люди", "чаты", "мета/бобер", "мета/чужое"):
|
||||||
|
(root / sub).mkdir(parents=True)
|
||||||
|
today = datetime.now(UTC).date().isoformat()
|
||||||
|
(root / "дни" / f"{today}.md").write_text("# день\n- 09:00 проснулся\n")
|
||||||
|
(root / "люди" / "Прохор.md").write_text("# Прохор\nстрока\n")
|
||||||
|
(root / "чаты" / "чат.md").write_text("чат\n")
|
||||||
|
(root / "мета" / "чужое" / "x.md").write_text("x\n")
|
||||||
|
watch = VaultWatch(root, RULES, tz="UTC")
|
||||||
|
watch.snapshot()
|
||||||
|
return root, watch
|
||||||
|
|
||||||
|
|
||||||
|
def append(path: Path, text: str) -> None:
|
||||||
|
with path.open("a", encoding="utf-8") as fh:
|
||||||
|
fh.write(text)
|
||||||
|
|
||||||
|
|
||||||
|
def test_rules_classify_paths() -> None:
|
||||||
|
today = "2026-08-29"
|
||||||
|
assert RULES.kind("дни/2026-08-29.md", today=today) == "full"
|
||||||
|
assert RULES.kind("дни/2026-08-28.md", today=today) == "names"
|
||||||
|
assert RULES.kind("люди/Прохор.md", today=today) == "names"
|
||||||
|
assert RULES.kind("мета/бобер/состояние.md", today=today) == "names"
|
||||||
|
assert RULES.kind("мета/чужое/x.md", today=today) is None
|
||||||
|
assert RULES.kind("чаты/чат.md", today=today) is None
|
||||||
|
assert RULES.kind("люди/фото.png", today=today) is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_watch_reports_only_added_lines_with_mtime() -> None:
|
||||||
|
root, watch = vault()
|
||||||
|
today = watch.today()
|
||||||
|
diary = root / "дни" / f"{today}.md"
|
||||||
|
append(diary, "- 12:40 вышел\n- 13:00 кофе\n")
|
||||||
|
append(root / "люди" / "Прохор.md", "ещё\n")
|
||||||
|
append(root / "чаты" / "чат.md", "ignored\n")
|
||||||
|
for path in (diary, root / "люди" / "Прохор.md", root / "чаты" / "чат.md"):
|
||||||
|
watch.note(path)
|
||||||
|
changes = {c.path: c for c in watch.take()}
|
||||||
|
assert set(changes) == {f"дни/{today}.md", "люди/Прохор.md"}
|
||||||
|
assert changes[f"дни/{today}.md"].added == ["- 12:40 вышел", "- 13:00 кофе"]
|
||||||
|
assert changes[f"дни/{today}.md"].full
|
||||||
|
assert changes["люди/Прохор.md"].added_count == 1
|
||||||
|
assert changes["люди/Прохор.md"].added == []
|
||||||
|
assert abs(
|
||||||
|
changes[f"дни/{today}.md"].mtime
|
||||||
|
- datetime.fromtimestamp(diary.stat().st_mtime, tz=UTC)
|
||||||
|
) < timedelta(seconds=1)
|
||||||
|
assert watch.take() == []
|
||||||
|
append(diary, "- 14:00 снова\n")
|
||||||
|
watch.note(diary)
|
||||||
|
(only,) = watch.take()
|
||||||
|
assert only.added == ["- 14:00 снова"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_envelope_respects_ceilings_and_names_only_window() -> None:
|
||||||
|
root, watch = vault()
|
||||||
|
today = watch.today()
|
||||||
|
diary = root / "дни" / f"{today}.md"
|
||||||
|
append(diary, "".join(f"- строка {i}\n" for i in range(200)))
|
||||||
|
for name in ("Петя", "Маша"):
|
||||||
|
(root / "люди" / f"{name}.md").write_text("новый\n" * 50)
|
||||||
|
watch.note(root / "люди" / f"{name}.md")
|
||||||
|
watch.note(diary)
|
||||||
|
envelope = Envelope(watch=watch, tz="Europe/Warsaw")
|
||||||
|
text = envelope.build()
|
||||||
|
lines = text.splitlines()
|
||||||
|
assert lines[0] == HEADER
|
||||||
|
assert "(Warsaw)" in lines[1]
|
||||||
|
assert lines[2].startswith("vault, изменено со старта: ")
|
||||||
|
assert f"дни/{today}.md (+200)" in lines[2]
|
||||||
|
assert "люди/Петя.md (+50)" in lines[2]
|
||||||
|
assert sum(1 for line in lines if line.startswith("+ ")) == 31
|
||||||
|
assert "+ … ещё 170" in lines
|
||||||
|
assert len(lines) <= 120
|
||||||
|
append(diary, "- ещё одна\n")
|
||||||
|
watch.note(diary)
|
||||||
|
second = envelope.build()
|
||||||
|
assert f"дни/{today}.md (+1)" in second
|
||||||
|
assert "+ - ещё одна" not in second
|
||||||
|
|
||||||
|
|
||||||
|
def test_render_hits_total_ceiling() -> None:
|
||||||
|
now = datetime(2026, 8, 26, 11, 4, tzinfo=UTC)
|
||||||
|
changes = [
|
||||||
|
Change(
|
||||||
|
path=f"дни/{i}.md",
|
||||||
|
mtime=now,
|
||||||
|
added=[f"l{j}" for j in range(30)],
|
||||||
|
added_count=30,
|
||||||
|
full=True,
|
||||||
|
)
|
||||||
|
for i in range(10)
|
||||||
|
]
|
||||||
|
text = render(
|
||||||
|
now=now,
|
||||||
|
tz="Europe/Warsaw",
|
||||||
|
changes=changes,
|
||||||
|
since=now - timedelta(hours=1),
|
||||||
|
names_only=False,
|
||||||
|
)
|
||||||
|
lines = text.splitlines()
|
||||||
|
assert len(lines) <= 120
|
||||||
|
assert "… (потолок конверта)" in lines
|
||||||
|
assert lines[1] == "время: 2026-08-26 13:04 (Warsaw)"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_master_turn_gets_envelope_after_text_and_before_injects(
|
||||||
|
world: World,
|
||||||
|
) -> None:
|
||||||
|
root, watch = vault()
|
||||||
|
world.conversations._envelope = Envelope(watch=watch, tz="UTC") # noqa: SLF001
|
||||||
|
master = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
append(root / "люди" / "Прохор.md", "новое\n")
|
||||||
|
watch.note(root / "люди" / "Прохор.md")
|
||||||
|
await world.conversations.inject(master, "tick", urgency="normal", origin="крон")
|
||||||
|
await world.conversations.post(master, "hello")
|
||||||
|
await world.settle(master, 2)
|
||||||
|
prompt = ScriptedClient.instances[0].prompts[0]
|
||||||
|
head, _, rest = prompt.partition("\n\n")
|
||||||
|
assert head == "hello"
|
||||||
|
assert rest.startswith(HEADER)
|
||||||
|
assert "люди/Прохор.md (+1)" in rest
|
||||||
|
assert rest.index("[инжекты") > rest.index(HEADER)
|
||||||
|
branch = await world.conversations.spawn(
|
||||||
|
kind="branch", parent=master, seed="brief", text="do X"
|
||||||
|
)
|
||||||
|
await world.settle(branch, 1)
|
||||||
|
assert "[конверт" not in ScriptedClient.instances[-1].prompts[0]
|
||||||
|
await asyncio.sleep(0)
|
||||||
|
|
||||||
|
|
||||||
|
async def test_recall_lines_follow_the_vault_block_and_reach_branches(
|
||||||
|
world: World,
|
||||||
|
) -> None:
|
||||||
|
root, watch = vault()
|
||||||
|
seen: list[tuple[str, str]] = []
|
||||||
|
|
||||||
|
def recall(ctx: RecallContext) -> str | None:
|
||||||
|
seen.append((ctx.kind, ctx.text))
|
||||||
|
return "👤 Прохор - карточка `люди/Прохор.md`" if "Прохор" in ctx.text else None
|
||||||
|
|
||||||
|
noted: list[UserSaid] = []
|
||||||
|
world.conversations._envelope = Envelope(watch=watch, tz="UTC", recall=recall) # noqa: SLF001
|
||||||
|
world.conversations._user_sink = noted.append # noqa: SLF001
|
||||||
|
master = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
append(root / "люди" / "Прохор.md", "новое\n")
|
||||||
|
watch.note(root / "люди" / "Прохор.md")
|
||||||
|
await world.conversations.post(master, "что там у Прохор")
|
||||||
|
await world.settle(master, 1)
|
||||||
|
prompt = ScriptedClient.instances[0].prompts[0]
|
||||||
|
head, _, rest = prompt.partition("\n\n")
|
||||||
|
assert head == "что там у Прохор"
|
||||||
|
lines = rest.splitlines()
|
||||||
|
assert lines[0] == HEADER
|
||||||
|
assert "люди/Прохор.md (+1)" in rest
|
||||||
|
assert lines[-1] == "👤 Прохор - карточка `люди/Прохор.md`"
|
||||||
|
assert rest.index("люди/Прохор.md (+1)") < rest.index("👤 Прохор")
|
||||||
|
assert seen == [("master", "что там у Прохор")]
|
||||||
|
assert [(m.kind, m.text) for m in noted] == [("master", "что там у Прохор")]
|
||||||
|
|
||||||
|
branch = await world.conversations.spawn(
|
||||||
|
kind="branch", parent=master, seed="clean", text="про Прохор подробнее"
|
||||||
|
)
|
||||||
|
await world.settle(branch, 1)
|
||||||
|
branch_prompt = ScriptedClient.instances[-1].prompts[0]
|
||||||
|
assert "про Прохор подробнее\n\n" + HEADER in branch_prompt
|
||||||
|
assert branch_prompt.endswith(f"{HEADER}\n👤 Прохор - карточка `люди/Прохор.md`")
|
||||||
|
assert "vault, изменено" not in branch_prompt
|
||||||
|
# the first branch turn carries the seed line above the text
|
||||||
|
assert seen[-1][0] == "branch" and seen[-1][1].endswith("про Прохор подробнее")
|
||||||
|
assert noted[-1].kind == "branch"
|
||||||
|
|
||||||
|
await world.conversations.post(master, "ничего про людей")
|
||||||
|
await world.settle(master, 2)
|
||||||
|
second = ScriptedClient.instances[0].prompts[1]
|
||||||
|
assert "👤" not in second
|
||||||
|
assert second.partition("\n\n")[2].startswith(HEADER)
|
||||||
|
await asyncio.sleep(0)
|
||||||
|
|
||||||
|
|
||||||
|
async def test_failing_recall_or_sink_never_blocks_the_turn(world: World) -> None:
|
||||||
|
root, watch = vault()
|
||||||
|
|
||||||
|
def boom(_: object) -> str:
|
||||||
|
msg = "nope"
|
||||||
|
raise RuntimeError(msg)
|
||||||
|
|
||||||
|
world.conversations._envelope = Envelope(watch=watch, tz="UTC", recall=boom) # noqa: SLF001
|
||||||
|
world.conversations._user_sink = boom # noqa: SLF001
|
||||||
|
master = await world.conversations.create(kind="master", agent="a", origin="test")
|
||||||
|
await world.conversations.post(master, "hello")
|
||||||
|
await world.settle(master, 1)
|
||||||
|
prompt = ScriptedClient.instances[0].prompts[0]
|
||||||
|
assert prompt.startswith("hello\n\n" + HEADER)
|
||||||
|
assert root.exists()
|
||||||
|
await asyncio.sleep(0)
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
from anthropic.types import Message, TextBlock, ThinkingBlock, ToolUseBlock, Usage
|
||||||
|
|
||||||
|
from beaver_gateway.frontends.markdown import renderer
|
||||||
|
|
||||||
|
|
||||||
|
def _message(*blocks: object) -> Message:
|
||||||
|
return Message(
|
||||||
|
id="m",
|
||||||
|
type="message",
|
||||||
|
role="assistant",
|
||||||
|
model="x",
|
||||||
|
content=list(blocks), # type: ignore[arg-type]
|
||||||
|
stop_reason="end_turn",
|
||||||
|
stop_sequence=None,
|
||||||
|
usage=Usage(input_tokens=1, output_tokens=1),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_tool_calls_leave_no_blank_lines() -> None:
|
||||||
|
message = _message(
|
||||||
|
TextBlock(type="text", text="first"),
|
||||||
|
ToolUseBlock(type="tool_use", id="t1", name="Bash", input={"command": "ls"}),
|
||||||
|
ToolUseBlock(type="tool_use", id="t2", name="Read", input={"file_path": "x"}),
|
||||||
|
ToolUseBlock(type="tool_use", id="t3", name="Grep", input={"pattern": "y"}),
|
||||||
|
TextBlock(type="text", text="second"),
|
||||||
|
)
|
||||||
|
assert renderer.render_assistant_message(message) == (
|
||||||
|
"### Assistant:\n\nfirst\n\nsecond\n"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_thinking_renders_as_collapsed_callout() -> None:
|
||||||
|
message = _message(
|
||||||
|
ThinkingBlock(type="thinking", thinking="a\nb", signature="s"),
|
||||||
|
TextBlock(type="text", text="answer"),
|
||||||
|
)
|
||||||
|
assert renderer.render_assistant_message(message) == (
|
||||||
|
"### Assistant:\n\n> [!thinking]-\n> a\n> b\n\nanswer\n"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_user_scaffold_matches_rendered_user_turn_shape() -> None:
|
||||||
|
body = renderer.append_to_body("### Assistant:\n\nhi\n", renderer.USER_SCAFFOLD)
|
||||||
|
assert body.endswith("\n\n---\n\n### User:\n\n")
|
||||||
|
assert renderer.render_user_text("q").startswith("### User:\n\n")
|
||||||
@@ -0,0 +1,208 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import mcp.types as mt
|
||||||
|
import pytest
|
||||||
|
from fastmcp import FastMCP
|
||||||
|
from fastmcp.tools.base import ToolResult
|
||||||
|
|
||||||
|
from beaver_gateway.core import redact as redact_mod
|
||||||
|
from beaver_gateway.core.redact import redact
|
||||||
|
from beaver_gateway.mcp.internal_app import build_internal_app
|
||||||
|
from beaver_gateway.mcp.redacting import RedactingMiddleware
|
||||||
|
from beaver_gateway.mcp.types import McpServer
|
||||||
|
|
||||||
|
# A trimmed but otherwise verbatim ``komodo deploy`` result: this is the
|
||||||
|
# shape Komodo returns for the ``Compose Config`` stage of a
|
||||||
|
# DeployStackService, which the komodo tool passes on to the model as
|
||||||
|
# ``-- <stage> ok`` plus the stage's stdout. Values are substituted.
|
||||||
|
KOMODO_DEPLOY = """DeployStackService Complete ok
|
||||||
|
-- Service/s ok
|
||||||
|
gateway
|
||||||
|
-- Compose Config ok
|
||||||
|
name: beaver-agent
|
||||||
|
services:
|
||||||
|
gateway:
|
||||||
|
container_name: beaver-gateway
|
||||||
|
environment:
|
||||||
|
ADMIN_PASS: 0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f50
|
||||||
|
ADMIN_USER: admin
|
||||||
|
BEAVERGRAM_MCP: https://tg.example.dev/mcp?token=bg-0123456789abcdef
|
||||||
|
BOOTSTRAP_TOKENS: admin:bs-0123456789abcdef,komodo:hook-0123456789
|
||||||
|
CALENDAR_MCPS: personal=https://cal.example.com/api/mcp?icsUrl=https%3A%2F%2Fcalendar.google.com%2Fcalendar%2Fical%2Fx%2Fprivate-0123456789abcdef%2Fbasic.ics
|
||||||
|
CLAUDE_CODE_OAUTH_TOKEN: sk-ant-oat01-0123456789abcdef
|
||||||
|
CLAUDE_HOME: /home/beaver-runner
|
||||||
|
COMPOSE_PROFILES: db,gateway,obsidian,t3
|
||||||
|
DATABASE_URL: postgresql+psycopg://beaver:pg-0123456789abcdef@postgres:5432/beaver
|
||||||
|
KOMODO_URL: https://komod.example.dev
|
||||||
|
POSTGRES_DB: beaver
|
||||||
|
POSTGRES_PASSWORD: pg-0123456789abcdef
|
||||||
|
POSTGRES_USER: beaver
|
||||||
|
ports:
|
||||||
|
- 62990:62990
|
||||||
|
-- Compose Up ok
|
||||||
|
Container beaver-gateway Started
|
||||||
|
"""
|
||||||
|
|
||||||
|
SECRETS = (
|
||||||
|
"0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f50",
|
||||||
|
"bg-0123456789abcdef",
|
||||||
|
"bs-0123456789abcdef",
|
||||||
|
"hook-0123456789",
|
||||||
|
"private-0123456789abcdef",
|
||||||
|
"sk-ant-oat01-0123456789abcdef",
|
||||||
|
"pg-0123456789abcdef",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _no_env_secrets() -> None:
|
||||||
|
"""Prove the patterns alone carry it — no literal env values loaded."""
|
||||||
|
redact_mod._ENV_SECRETS[:] = [] # noqa: SLF001
|
||||||
|
|
||||||
|
|
||||||
|
def test_compose_config_dump_loses_every_value_but_keeps_every_name() -> None:
|
||||||
|
out = redact(KOMODO_DEPLOY)
|
||||||
|
for secret in SECRETS:
|
||||||
|
assert secret not in out, secret
|
||||||
|
# Names stay: the dump still says what is configured.
|
||||||
|
for name in (
|
||||||
|
"ADMIN_PASS",
|
||||||
|
"BOOTSTRAP_TOKENS",
|
||||||
|
"CALENDAR_MCPS",
|
||||||
|
"POSTGRES_PASSWORD",
|
||||||
|
# a username is half of a login pair — in prod ADMIN_USER is a
|
||||||
|
# 64-hex string, not "admin"
|
||||||
|
"ADMIN_USER",
|
||||||
|
"POSTGRES_USER",
|
||||||
|
):
|
||||||
|
assert f"{name}: <…>" in out
|
||||||
|
# Non-secret values are untouched, so the result stays worth reading.
|
||||||
|
assert "POSTGRES_DB: beaver" in out
|
||||||
|
assert "CLAUDE_HOME: /home/beaver-runner" in out
|
||||||
|
assert "COMPOSE_PROFILES: db,gateway,obsidian,t3" in out
|
||||||
|
assert "KOMODO_URL: https://komod.example.dev" in out
|
||||||
|
assert "container_name: beaver-gateway" in out
|
||||||
|
assert "-- Compose Up ok" in out
|
||||||
|
assert " Container beaver-gateway Started" in out
|
||||||
|
|
||||||
|
|
||||||
|
def test_database_url_keeps_its_shape_and_loses_its_password() -> None:
|
||||||
|
# DATABASE_URL is not a secret-looking name, so the whole line is not
|
||||||
|
# masked; the password is caught as the userinfo of the connection
|
||||||
|
# string, which is what keeps the host and database readable.
|
||||||
|
out = redact(KOMODO_DEPLOY)
|
||||||
|
line = next(ln for ln in out.splitlines() if "DATABASE_URL" in ln)
|
||||||
|
assert "pg-0123456789abcdef" not in line
|
||||||
|
assert "postgres:5432/beaver" in line
|
||||||
|
|
||||||
|
|
||||||
|
def test_redaction_is_idempotent() -> None:
|
||||||
|
once = redact(KOMODO_DEPLOY)
|
||||||
|
assert redact(once) == once
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_log_line_is_not_swallowed_by_the_assignment_rule() -> None:
|
||||||
|
# ``token:`` in mid-line must not eat the rest of an audit line.
|
||||||
|
line = "mcp: actor=token:komodo namespace=calendar POST /mcp/ -> 200"
|
||||||
|
assert redact(line) == line
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_middleware_masks_a_tool_result() -> None:
|
||||||
|
def deploy(stack: str) -> str:
|
||||||
|
"""Deploy a stack."""
|
||||||
|
return f"{stack}\n{KOMODO_DEPLOY}"
|
||||||
|
|
||||||
|
server = FastMCP(name="komodo", tools=[deploy])
|
||||||
|
server.add_middleware(RedactingMiddleware())
|
||||||
|
result = await server.call_tool("deploy", {"stack": "beaver-agent"})
|
||||||
|
text = "\n".join(b.text for b in result.content if isinstance(b, mt.TextContent))
|
||||||
|
for secret in SECRETS:
|
||||||
|
assert secret not in text, secret
|
||||||
|
assert "ADMIN_PASS: <…>" in text
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_middleware_masks_structured_content() -> None:
|
||||||
|
def config() -> dict[str, object]:
|
||||||
|
"""Return a config blob."""
|
||||||
|
return {
|
||||||
|
"env": {"ADMIN_PASS": "0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f50"},
|
||||||
|
"port": 62990,
|
||||||
|
}
|
||||||
|
|
||||||
|
server = FastMCP(name="cfg", tools=[config])
|
||||||
|
server.add_middleware(RedactingMiddleware())
|
||||||
|
result = await server.call_tool("config", {})
|
||||||
|
dumped = str(result.structured_content) + str(result.content)
|
||||||
|
assert "0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f50" not in dumped
|
||||||
|
assert "62990" in dumped
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_clean_result_passes_through_untouched() -> None:
|
||||||
|
def ping() -> str:
|
||||||
|
"""Say hello."""
|
||||||
|
return "pong"
|
||||||
|
|
||||||
|
server = FastMCP(name="p", tools=[ping])
|
||||||
|
server.add_middleware(RedactingMiddleware())
|
||||||
|
result = await server.call_tool("ping", {})
|
||||||
|
assert isinstance(result, ToolResult)
|
||||||
|
assert result.content[0].text == "pong" # ty: ignore[unresolved-attribute]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_every_namespace_built_by_the_aggregator_is_filtered() -> None:
|
||||||
|
def deploy() -> str:
|
||||||
|
"""Deploy."""
|
||||||
|
return KOMODO_DEPLOY
|
||||||
|
|
||||||
|
_app, _urls, servers = build_internal_app(
|
||||||
|
[McpServer.python_tool(name="komodo", tools=[deploy])],
|
||||||
|
host="127.0.0.1",
|
||||||
|
port=8765,
|
||||||
|
)
|
||||||
|
for name, server in servers.items():
|
||||||
|
result = await server.call_tool(
|
||||||
|
"deploy" if name == "komodo" else "komodo_deploy", {}
|
||||||
|
)
|
||||||
|
text = "\n".join(
|
||||||
|
b.text for b in result.content if isinstance(b, mt.TextContent)
|
||||||
|
)
|
||||||
|
assert "0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f50" not in text, name
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_gateway_own_tools_are_filtered_too() -> None:
|
||||||
|
# These bypass the FastMCP middleware — the SDK mounts them
|
||||||
|
# in-process — so they carry their own wrapper.
|
||||||
|
from claude_agent_sdk import SdkMcpTool
|
||||||
|
|
||||||
|
from beaver_gateway.core.gateway_tools import _redacting
|
||||||
|
|
||||||
|
async def handler(_args: dict[str, object]) -> dict[str, object]:
|
||||||
|
return {"content": [{"type": "text", "text": KOMODO_DEPLOY}]}
|
||||||
|
|
||||||
|
spec = _redacting(
|
||||||
|
SdkMcpTool(
|
||||||
|
name="read_conversation", description="", input_schema={}, handler=handler
|
||||||
|
)
|
||||||
|
)
|
||||||
|
out = str(await spec.handler({}))
|
||||||
|
for secret in SECRETS:
|
||||||
|
assert secret not in out, secret
|
||||||
|
|
||||||
|
|
||||||
|
def test_source_code_the_dispatcher_reads_is_not_mangled() -> None:
|
||||||
|
# t3code hands back files and diffs. A SCREAMING_SNAKE constant is
|
||||||
|
# an assignment too, but ``NAME = expr`` is code, not a config dump.
|
||||||
|
source = (
|
||||||
|
'SESSION_SECRET = os.environ["SESSION_SECRET"]\nAPI_KEY_HEADER = "X-Api-Key"'
|
||||||
|
)
|
||||||
|
assert redact(source) == source
|
||||||
|
|
||||||
|
|
||||||
|
def test_env_file_lines_are_still_masked() -> None:
|
||||||
|
out = redact("ADMIN_PASS=0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f50\nPOSTGRES_DB=beaver")
|
||||||
|
assert out == "ADMIN_PASS=<…>\nPOSTGRES_DB=beaver"
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from beaver_gateway.core.policy import Deny, ToolCall, brief, evaluate, hook_output
|
||||||
|
|
||||||
|
|
||||||
|
def call(tool: str, **tool_input) -> ToolCall:
|
||||||
|
return ToolCall(
|
||||||
|
tool=tool,
|
||||||
|
input=tool_input,
|
||||||
|
agent="a",
|
||||||
|
kind="master",
|
||||||
|
conversation="c1",
|
||||||
|
cwd=Path("/vault"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_path_resolves_relative_against_cwd():
|
||||||
|
assert call("Write", file_path="мета/бобер/x.md").path() == Path(
|
||||||
|
"/vault/мета/бобер/x.md"
|
||||||
|
)
|
||||||
|
assert call("Write", file_path="/vault/a/../b.md").path() == Path("/vault/b.md")
|
||||||
|
assert call("Write", file_path="../../etc/passwd").path() == Path("/etc/passwd")
|
||||||
|
assert call("Bash", command="ls").path() is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_mcp_split():
|
||||||
|
assert call("mcp__firefly__store_transaction").mcp == (
|
||||||
|
"firefly",
|
||||||
|
"store_transaction",
|
||||||
|
)
|
||||||
|
assert call("Write").mcp is None
|
||||||
|
|
||||||
|
|
||||||
|
async def test_first_deny_wins_and_async_rules_work():
|
||||||
|
seen = []
|
||||||
|
|
||||||
|
def allow(c):
|
||||||
|
seen.append("allow")
|
||||||
|
|
||||||
|
async def deny(c):
|
||||||
|
return Deny(reason="no")
|
||||||
|
|
||||||
|
def never(c):
|
||||||
|
raise AssertionError
|
||||||
|
|
||||||
|
verdict = await evaluate((allow, deny, never), call("Write"))
|
||||||
|
assert verdict == Deny(reason="no")
|
||||||
|
assert seen == ["allow"]
|
||||||
|
|
||||||
|
|
||||||
|
async def test_raising_rule_fails_closed():
|
||||||
|
def boom(c):
|
||||||
|
raise RuntimeError("x")
|
||||||
|
|
||||||
|
verdict = await evaluate((boom,), call("Write"))
|
||||||
|
assert verdict is not None and "boom" in verdict.reason
|
||||||
|
|
||||||
|
|
||||||
|
def test_hook_output_shapes():
|
||||||
|
assert hook_output(None) == {}
|
||||||
|
out = hook_output(Deny(reason="r"))
|
||||||
|
assert out["hookSpecificOutput"]["permissionDecision"] == "deny"
|
||||||
|
assert out["hookSpecificOutput"]["permissionDecisionReason"] == "r"
|
||||||
|
|
||||||
|
|
||||||
|
def test_brief_prefers_path_then_command_and_truncates():
|
||||||
|
assert brief({"file_path": "/v/x.md", "content": "a" * 999}) == "/v/x.md"
|
||||||
|
assert brief({"command": "ls -la"}) == "ls -la"
|
||||||
|
assert len(brief({"q": "a" * 999})) == 240
|
||||||
Loaded 100 of 301 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user