26 Commits
Author SHA1 Message Date
hh ba348a35b7 fix(api): lifespan param named _app without dishka 2026-10-15 14:06:17 +02:00
hh 7874399cd7 fix: userbot base moves to trixie-slim, the bookworm tag died at uv 0.9.30; uv_build follows uv 0.13 2026-10-10 22:42:52 +00:00
hh ee0f00ece8 refactor: drop make deploy and its refs/deploy stamps, the fleet deploys from infra/komodo; short readme and claude.md 2026-10-08 21:34:39 +02:00
hh a5345a9e4c docs(readme): production in the fleet deploys from infra/komodo, make deploy stays for a checkout deployed by hand 2026-10-08 04:40:25 +00:00
hh 2be4367f9f fix(deploy): postgres comes up in the project, not one directory above, and an empty service list restarts nothing; image named with dashes 2026-10-08 04:37:26 +00:00
hh eed732fbcf feat(config): served from an in-process cache with a 5 s version check, writes bump the version atomically, mongo and postgres alike, redis keyed by version on top 2026-10-08 05:38:09 +02:00
hh 4cb3b85321 fix(logging): tracebacks show locals only with LOG__SHOW_LOCALS, prod logs stop dumping tokens and request bodies 2026-10-08 05:13:17 +02:00
hh 92716b79b6 fix(migrations): alembic autogenerate leaves the taskiq broker tables alone 2026-10-08 05:08:41 +02:00
hh 964b9edf67 fix(worker): postgres broker on taskiq-postgres, taskiq-pg is unmaintained, runs each task in every worker process and breaks on taskiq 0.13 2026-10-08 05:08:41 +02:00
hh c19cdc4e11 feat(check): make check fails when a service imports another service's sdk at startup 2026-10-08 04:53:54 +02:00
hh 97fdafa092 refactor(userbot): package init stops re-exporting the pyrogram client 2026-10-08 04:53:54 +02:00
hh 4d2ed061ea perf(llm): google sdk imported when the agent is built, in a worker thread through utils.imports.preload 2026-10-08 04:53:54 +02:00
hh 85ce05a55e perf(dishka): aiogram and fastapi integration providers left out of the shared container, services stop importing each other's framework 2026-10-08 04:53:54 +02:00
hh 47debebc0c perf(logging): rich finds the aiogram router path with find_spec, setup_logging no longer imports aiogram in every service 2026-10-08 04:53:54 +02:00
hh eadfa3d8ec chore(deps): raise lower bounds and uv_build, ty skips docs 2026-10-01 02:01:58 +02:00
hh 6028507874 fix(api,userbot): clean app.py ending for every option set, handlers typed for current ty 2026-10-01 02:01:58 +02:00
hh db8184cb37 feat(env): scripts connection urls for runs outside docker, sections ignore unknown keys 2026-10-01 02:01:58 +02:00
hh 2c1dfefe34 feat(compose): services listen on their configured port, publishing is commented out with a random host port 2026-10-01 02:01:58 +02:00
hh b4125407bf TZ for the migrator 2026-09-09 00:54:30 +02:00
hh e0d5defa97 Migrate arguments, restart api after SPA build 2026-09-09 00:51:34 +02:00
hh dae9532ef6 Run compose from the root in part layout 2026-09-09 00:39:12 +02:00
hh e316bc92a5 Deploy hooks 2026-09-09 00:37:04 +02:00
hh 3878837d25 README wording 2026-09-09 00:21:25 +02:00
hh b1db7c274f Single trailing newline 2026-09-09 00:19:34 +02:00
hh ac0ccf47e5 Override-based compose, profiles in .env, README 2026-09-09 00:18:15 +02:00
hh 1f416dbe60 PyroClient: device model, locale, transmissions 2026-09-09 00:05:28 +02:00
34 changed files with 534 additions and 251 deletions

No files matched your search

+1 -3
View File
@@ -1,8 +1,6 @@
# backend-python # backend-python
Copier template for a Python backend, either as a `backend/` part next to other parts or alone at the root of the project (`layout`): a uv project with one Docker image and one `python -m <module>` entrypoint per service (`api`, `bot`, `userbot`, `worker`, `scheduler`), pydantic-settings, rich logging, optional Dishka, MongoDB or PostgreSQL, Redis, taskiq (in-memory, Redis or PostgreSQL broker), CryptoPay, pydantic-ai and a DynamicConfig stored in the database. Copier template for a Python backend: a uv project, one Docker image, a `python -m <module>` entrypoint per service (`api`, `bot`, `userbot`, `worker`, `scheduler`), optional Dishka, MongoDB or PostgreSQL, Redis, taskiq, CryptoPay, pydantic-ai and a dynamic config in the database. A `backend/` part or the whole project (`layout`), on top of `infra`.
Ships `backend/docker-compose.yml` with root-relative paths, `docker-compose.local.yml` (published ports) and `docker-compose.prod.yml` (the `caddy` network), a `Makefile` with `fmt`, `check`, `build` and service-specific targets, `.env.example` and `CLAUDE.md`. Applied on top of the `infra` template, `layout: parts` or `layout: single` respectively.
```sh ```sh
copier copy --trust -d project_name="My Thing" <this-repo> <dest> copier copy --trust -d project_name="My Thing" <this-repo> <dest>
+1 -1
View File
@@ -71,7 +71,7 @@ taskiq_broker:
choices: choices:
in-memory: memory in-memory: memory
Redis: redis Redis: redis
PostgreSQL (taskiq-pg): PostgreSQL (taskiq-postgres):
value: postgres value: postgres
validator: "{% if database != 'postgres' %}Requires the PostgreSQL database{% endif %}" validator: "{% if database != 'postgres' %}Requires the PostgreSQL database{% endif %}"
default: memory default: memory
+1 -1
View File
@@ -2,4 +2,4 @@
python = "{{ prefix }}.venv" python = "{{ prefix }}.venv"
[src] [src]
exclude = ["{{ prefix }}migrations"] exclude = ["docs", "{{ prefix }}migrations"]
+14 -27
View File
@@ -1,27 +1,14 @@
{% if layout == 'root' %}COMPOSE_FILE=docker-compose.yml:docker-compose.local.yml {%- set groups = [] -%}
{% endif %}RUN_ENVIRONMENT=dev {%- if layout == 'root' %}{% set _ = groups.append(["COMPOSE_PROFILES=" ~ ("external," if database != 'none' or use_redis else "") ~ "services"]) %}{% endif -%}
LOG__LEVEL=INFO {%- set _ = groups.append(["RUN_ENVIRONMENT=prod", "TZ=UTC"]) -%}
LOG__LEVEL_EXTERNAL=WARNING {%- set _ = groups.append(["LOG__LEVEL=INFO", "LOG__LEVEL_EXTERNAL=WARNING", "LOG__SHOW_TIME=false", "LOG__SHOW_LOCALS=false", "LOG__CONSOLE_WIDTH=150"]) -%}
LOG__SHOW_TIME=false {%- if database == 'mongo' %}{% set _ = groups.append(["DB__HOST=mongodb", "DB__PORT=27017", "DB__USER=" ~ project_slug, "DB__PASSWORD=" ~ project_slug, "DB__DB_NAME=" ~ project_slug, "DB__SCRIPTS_CONNECTION_URL=mongodb://" ~ project_slug ~ ":" ~ project_slug ~ "@localhost:27017/" ~ project_slug ~ "?authSource=admin"]) %}{% endif -%}
LOG__CONSOLE_WIDTH=150 {%- if database == 'postgres' %}{% set _ = groups.append(["DB__HOST=postgres", "DB__PORT=5432", "DB__USER=" ~ project_slug, "DB__PASSWORD=" ~ project_slug, "DB__DB_NAME=" ~ project_slug, "DB__SCRIPTS_CONNECTION_URL=postgresql://" ~ project_slug ~ ":" ~ project_slug ~ "@localhost:5432/" ~ project_slug]) %}{% endif -%}
{% if database == 'mongo' %}DB__HOST=mongodb {%- if use_redis %}{% set _ = groups.append(["REDIS__HOST=redis", "REDIS__PORT=6379", "REDIS__CACHE_TTL=300", "REDIS__SCRIPTS_CONNECTION_URL=redis://localhost:6379/0"]) %}{% endif -%}
DB__PORT=27017 {%- if 'api' in backend_services %}{% set _ = groups.append(["API__HOST=0.0.0.0", "API__PORT=8080", "API__WORKERS=1", "API__DOCS=false"]) %}{% endif -%}
DB__USER={{ project_slug }} {%- if 'aiogram_bot' in backend_services %}{% set _ = groups.append(["BOT__TOKEN="]) %}{% endif -%}
DB__PASSWORD={{ project_slug }} {%- if include_payments %}{% set _ = groups.append(["CRYPTO_PAY__TOKEN="]) %}{% endif -%}
DB__DB_NAME={{ project_slug }} {%- if use_llm %}{% set _ = groups.append(["LLM__MODEL=google-gla:gemini-2.5-flash", "LLM__GEMINI_API_KEY="]) %}{% endif -%}
{% endif %}{% if database == 'postgres' %}DB__HOST=postgres {% for group in groups %}{{ group | join("\n") }}
DB__PORT=5432 {% if not loop.last %}
DB__USER={{ project_slug }} {% endif %}{% endfor %}
DB__PASSWORD={{ project_slug }}
DB__DB_NAME={{ project_slug }}
{% endif %}{% if use_redis %}REDIS__HOST=redis
REDIS__PORT=6379
REDIS__CACHE_TTL=300
{% endif %}{% if 'api' in backend_services %}API__HOST=0.0.0.0
API__PORT=8080
API__WORKERS=1
{% endif %}{% if 'aiogram_bot' in backend_services %}BOT__TOKEN=
{% endif %}{% if include_payments %}CRYPTO_PAY__TOKEN=
{% endif %}{% if use_llm %}LLM__MODEL=google-gla:gemini-2.5-flash
LLM__GEMINI_API_KEY=
{% endif %}
+3 -1
View File
@@ -5,11 +5,13 @@
{%- if 'kurigram_userbot' in backend_services %}{% set _ = svc_modules.append("userbot") %}{% endif -%} {%- if 'kurigram_userbot' in backend_services %}{% set _ = svc_modules.append("userbot") %}{% endif -%}
{%- if 'scheduler' in backend_services and 'taskiq_worker' not in backend_services %}{% set _ = svc_modules.append("scheduler") %}{% endif -%} {%- if 'scheduler' in backend_services and 'taskiq_worker' not in backend_services %}{% set _ = svc_modules.append("scheduler") %}{% endif -%}
{%- if 'kurigram_userbot' in backend_services -%} {%- if 'kurigram_userbot' in backend_services -%}
FROM ghcr.io/astral-sh/uv:python3.13-bookworm-slim FROM ghcr.io/astral-sh/uv:python3.13-trixie-slim
RUN apt-get update && apt-get install -y build-essential && rm -rf /var/lib/apt/lists/* RUN apt-get update && apt-get install -y build-essential && rm -rf /var/lib/apt/lists/*
{%- else -%} {%- else -%}
FROM ghcr.io/astral-sh/uv:python3.13-alpine FROM ghcr.io/astral-sh/uv:python3.13-alpine
RUN apk add --no-cache tzdata
{%- endif %} {%- endif %}
WORKDIR /app WORKDIR /app
+25 -20
View File
@@ -5,11 +5,9 @@
{%- if 'kurigram_userbot' in backend_services %}{% set _ = svc_modules.append("userbot") %}{% endif -%} {%- if 'kurigram_userbot' in backend_services %}{% set _ = svc_modules.append("userbot") %}{% endif -%}
{%- if 'scheduler' in backend_services and 'taskiq_worker' not in backend_services %}{% set _ = svc_modules.append("scheduler") %}{% endif -%} {%- if 'scheduler' in backend_services and 'taskiq_worker' not in backend_services %}{% set _ = svc_modules.append("scheduler") %}{% endif -%}
{%- set has_script = dynamic_config and svc_modules -%} {%- set has_script = dynamic_config and svc_modules -%}
COMPOSE := docker compose{% if layout == 'part' %} --project-directory ..{% endif %} COMPOSE := {% if layout == 'part' %}cd .. && {% endif %}docker compose
{% if layout == 'root' %}SERVICES := $(COMPOSE) --profile services
EXTERNAL := $(COMPOSE) --profile external .PHONY: {% if layout == 'root' %}env recreate rebuild restart logs down {% endif %}fmt check{% if svc_modules %} build{% endif %}{% if database == 'postgres' %} migrate revision{% endif %}{% if has_script %} script{% endif %}{% if 'kurigram_userbot' in backend_services %} session{% endif %}
{% endif %}
.PHONY: {% if layout == 'root' %}env external recreate rebuild restart logs down {% endif %}fmt check build{% if database == 'postgres' %} migrate revision{% endif %}{% if has_script %} script{% endif %}{% if 'kurigram_userbot' in backend_services %} session{% endif %}
fmt: fmt:
uv run ruff format uv run ruff format
@@ -19,39 +17,46 @@ check:
uv run ruff format --check uv run ruff format --check
uv run ruff check uv run ruff check
uv run ty check uv run ty check
{% if svc_modules %} uv run python scripts/check_imports.py
{% endif %}{% if svc_modules %}
build: build:
$(COMPOSE) build {{ svc_modules[0] }} $(COMPOSE) build {{ svc_modules[0] }}
{% if layout == 'root' %} {% endif %}{% if layout == 'root' %}
env: env:
@test -f .env || cp .env.example .env @fresh=; test -f .env || { cp .env.example .env; fresh=1; }; \
if ! test -f docker-compose.override.yml; then \
cp docker-compose.override.yml.example docker-compose.override.yml; \
for port in $$(sed -n 's/.*127\.0\.0\.1:\([0-9]*\):.*/\1/p' docker-compose.override.yml | sort -u); do \
random=$$((20000 + $$(od -An -N2 -tu2 /dev/urandom) % 40000)); \
sed "s/127\.0\.0\.1:$$port:/127.0.0.1:$$random:/" docker-compose.override.yml > docker-compose.override.yml.tmp && mv docker-compose.override.yml.tmp docker-compose.override.yml; \
test -z "$$fresh" || { sed "s|localhost:$$port/|localhost:$$random/|" .env > .env.tmp && mv .env.tmp .env; }; \
done; \
fi
external: recreate:
$(EXTERNAL) up -d $(COMPOSE) up -d --force-recreate
recreate: external rebuild: build
$(SERVICES) up -d --force-recreate $(COMPOSE) up -d
rebuild: build recreate
restart: restart:
$(SERVICES) restart $(COMPOSE) restart
logs: logs:
$(SERVICES) logs -f --tail=100 $(COMPOSE) logs -f --tail=100
down: down:
$(EXTERNAL) --profile services down $(COMPOSE) down
{% endif %}{% if database == 'postgres' %} {% endif %}{% if database == 'postgres' %}
migrate: migrate:
$(COMPOSE) --profile external run --rm migrator $(COMPOSE) run --rm migrator $(or $(filter-out $@,$(MAKECMDGOALS)),upgrade head)
revision: revision:
$(COMPOSE) --profile external run --rm migrator revision --autogenerate -m "$(m)" $(COMPOSE) run --rm migrator revision --autogenerate -m "$(m)"
{% endif %}{% if has_script %} {% endif %}{% if has_script %}
script: script:
@$(COMPOSE) run --rm {{ svc_modules[0] }} scripts.$(word 2,$(MAKECMDGOALS)) $(wordlist 3,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS)) @$(COMPOSE) run --rm {{ svc_modules[0] }} scripts.$(word 2,$(MAKECMDGOALS)) $(wordlist 3,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS))
{% endif %}{% if database == 'postgres' or has_script %}
%: %:
@: @:
{% endif %}{% if 'kurigram_userbot' in backend_services %} {% endif %}{% if 'kurigram_userbot' in backend_services %}
@@ -1,15 +0,0 @@
{%- set has_ports = 'api' in backend_services or database != 'none' or use_redis -%}
services:{% if not has_ports %} {}{% endif %}
{% if 'api' in backend_services %} api:
ports:
- "127.0.0.1:${API__PORT:-8080}:8080"
{% endif %}{% if database == 'mongo' %} mongodb:
ports:
- "127.0.0.1:${DB__PORT:-27017}:${DB__PORT:-27017}"
{% endif %}{% if database == 'postgres' %} postgres:
ports:
- "127.0.0.1:${DB__PORT:-5432}:5432"
{% endif %}{% if use_redis %} redis:
ports:
- "127.0.0.1:${REDIS__PORT:-6379}:6379"
{% endif %}
@@ -0,0 +1,30 @@
{%- set dash = project_slug | replace('_', '-') -%}
{%- set entries = [] -%}
{%- if 'api' in backend_services %}{% set _ = entries.append('api') %}{% endif -%}
{%- if database == 'mongo' %}{% set _ = entries.append('mongodb') %}{% endif -%}
{%- if database == 'postgres' %}{% set _ = entries.append('postgres') %}{% endif -%}
{%- if use_redis %}{% set _ = entries.append('redis') %}{% endif -%}
services:{% if not entries %} {}{% endif %}
{% for entry in entries %}{% if entry == 'api' %} api:
networks:
default: {}
caddy:
aliases:
- {{ dash }}-api
# ports:
# - "127.0.0.1:8080:${API__PORT}"
{% elif entry == 'mongodb' %} mongodb:
# ports:
# - "127.0.0.1:27017:${DB__PORT}"
{% elif entry == 'postgres' %} postgres:
# ports:
# - "127.0.0.1:5432:${DB__PORT}"
{% elif entry == 'redis' %} redis:
# ports:
# - "127.0.0.1:6379:${REDIS__PORT}"
{% endif %}{% if not loop.last %}
{% endif %}{% endfor %}{% if 'api' in backend_services %}
networks:
caddy:
external: true
{% endif %}
@@ -1,15 +0,0 @@
{%- set dash = project_slug | replace('_', '-') -%}
{% if 'api' in backend_services -%}
services:
api:
networks:
caddy:
aliases:
- {{ dash }}-api
networks:
caddy:
external: true
{%- else -%}
services: {}
{%- endif %}
@@ -2,6 +2,10 @@
{%- set svc_networks = [] -%} {%- set svc_networks = [] -%}
{%- if database != 'none' %}{% set _ = svc_networks.append('database') %}{% endif -%} {%- if database != 'none' %}{% set _ = svc_networks.append('database') %}{% endif -%}
{%- if use_redis %}{% set _ = svc_networks.append('redis') %}{% endif -%} {%- if use_redis %}{% set _ = svc_networks.append('redis') %}{% endif -%}
{%- set external_services = [] -%}
{%- if database == 'mongo' %}{% set _ = external_services.append('mongodb') %}{% endif -%}
{%- if database == 'postgres' %}{% set _ = external_services.append('postgres') %}{% endif -%}
{%- if use_redis %}{% set _ = external_services.append('redis') %}{% endif -%}
{%- set app_services = [] -%} {%- set app_services = [] -%}
{%- if 'api' in backend_services %}{% set _ = app_services.append({'name': 'api', 'command': '[api]'}) %}{% endif -%} {%- if 'api' in backend_services %}{% set _ = app_services.append({'name': 'api', 'command': '[api]'}) %}{% endif -%}
{%- if 'aiogram_bot' in backend_services %}{% set _ = app_services.append({'name': 'bot', 'command': '[bot]'}) %}{% endif -%} {%- if 'aiogram_bot' in backend_services %}{% set _ = app_services.append({'name': 'bot', 'command': '[bot]'}) %}{% endif -%}
@@ -10,7 +14,7 @@
{%- if 'scheduler' in backend_services %}{% set _ = app_services.append({'name': 'scheduler', 'command': ('[worker, scheduler]' if 'taskiq_worker' in backend_services else '[scheduler]')}) %}{% endif -%} {%- if 'scheduler' in backend_services %}{% set _ = app_services.append({'name': 'scheduler', 'command': ('[worker, scheduler]' if 'taskiq_worker' in backend_services else '[scheduler]')}) %}{% endif -%}
{%- macro backend_service(name, command) %} {{ name }}: {%- macro backend_service(name, command) %} {{ name }}:
build: {{ part_dir }} build: {{ part_dir }}
image: {{ project_slug }}/backend image: {{ project_slug | replace('_', '-') }}/backend
profiles: [{{ name }}, services] profiles: [{{ name }}, services]
restart: unless-stopped restart: unless-stopped
env_file: env_file:
@@ -20,14 +24,20 @@
required: false required: false
{% endif %} environment: {% endif %} environment:
RUN_ENVIRONMENT: prod RUN_ENVIRONMENT: prod
TZ: ${TZ:-UTC}
volumes: volumes:
- ./{{ prefix }}src:/app/src - ./{{ prefix }}src:/app/src
{% if has_scripts %} - ./{{ prefix }}scripts:/app/scripts {% if has_scripts %} - ./{{ prefix }}scripts:/app/scripts
{% endif %}{% if name == 'userbot' %} - ./{{ prefix }}sessions:/app/sessions {% endif %}{% if name == 'userbot' %} - ./{{ prefix }}sessions:/app/sessions
{% endif %}{% if name == 'api' and serve_spa %} - ./frontend/build:/app/static:ro {% endif %}{% if name == 'api' and serve_spa %} - ./frontend/build:/app/static:ro
{% endif %} command: {{ command }} {% endif %} command: {{ command }}
{% if svc_networks %} networks: {% if external_services %} depends_on:
{% for net in svc_networks %} {{ net }}: {% for ext in external_services %} {{ ext }}:
condition: service_healthy
required: false
{% endfor %}{% endif %}{% if svc_networks %} networks:
{% if name == 'api' %} default:
{% endif %}{% for net in svc_networks %} {{ net }}:
{% endfor %}{% endif %}{% endmacro -%} {% endfor %}{% endif %}{% endmacro -%}
services: services:
{% for svc in app_services %}{{ backend_service(svc.name, svc.command) }}{% if not loop.last %} {% for svc in app_services %}{{ backend_service(svc.name, svc.command) }}{% if not loop.last %}
@@ -40,13 +50,18 @@ services:
environment: environment:
MONGO_INITDB_ROOT_USERNAME: ${DB__USER} MONGO_INITDB_ROOT_USERNAME: ${DB__USER}
MONGO_INITDB_ROOT_PASSWORD: ${DB__PASSWORD} MONGO_INITDB_ROOT_PASSWORD: ${DB__PASSWORD}
command: mongod --port ${DB__PORT:-27017} command: mongod --port ${DB__PORT}
volumes: volumes:
- database:/data/db - database:/data/db
healthcheck:
test: ["CMD", "mongosh", "--quiet", "--port", "${DB__PORT}", "--eval", "db.adminCommand('ping')"]
interval: 5s
timeout: 5s
retries: 5
networks: networks:
database: database:
aliases: aliases:
- ${DB__HOST:-mongodb} - ${DB__HOST}
{% endif %} {% endif %}
{%- if database == 'postgres' %} {%- if database == 'postgres' %}
postgres: postgres:
@@ -57,6 +72,7 @@ services:
POSTGRES_USER: ${DB__USER} POSTGRES_USER: ${DB__USER}
POSTGRES_PASSWORD: ${DB__PASSWORD} POSTGRES_PASSWORD: ${DB__PASSWORD}
POSTGRES_DB: ${DB__DB_NAME} POSTGRES_DB: ${DB__DB_NAME}
PGPORT: ${DB__PORT}
volumes: volumes:
- database:/var/lib/postgresql/data - database:/var/lib/postgresql/data
healthcheck: healthcheck:
@@ -67,11 +83,11 @@ services:
networks: networks:
database: database:
aliases: aliases:
- ${DB__HOST:-postgres} - ${DB__HOST}
migrator: migrator:
build: {{ part_dir }} build: {{ part_dir }}
image: {{ project_slug }}/backend image: {{ project_slug | replace('_', '-') }}/backend
profiles: [migrate] profiles: [migrate]
env_file: env_file:
- path: .env - path: .env
@@ -80,6 +96,7 @@ services:
required: false required: false
{% endif %} environment: {% endif %} environment:
RUN_ENVIRONMENT: prod RUN_ENVIRONMENT: prod
TZ: ${TZ:-UTC}
volumes: volumes:
- ./{{ prefix }}src:/app/src - ./{{ prefix }}src:/app/src
- ./{{ prefix }}migrations:/app/migrations - ./{{ prefix }}migrations:/app/migrations
@@ -89,6 +106,7 @@ services:
depends_on: depends_on:
postgres: postgres:
condition: service_healthy condition: service_healthy
required: false
networks: networks:
database: database:
{% endif %} {% endif %}
@@ -97,13 +115,18 @@ services:
image: redis:7-alpine image: redis:7-alpine
profiles: [redis, external] profiles: [redis, external]
restart: unless-stopped restart: unless-stopped
command: ["redis-server", "--appendonly", "yes"] command: ["redis-server", "--port", "${REDIS__PORT}", "--appendonly", "yes"]
healthcheck:
test: ["CMD", "redis-cli", "-p", "${REDIS__PORT}", "ping"]
interval: 5s
timeout: 5s
retries: 5
volumes: volumes:
- redis_data:/data - redis_data:/data
networks: networks:
redis: redis:
aliases: aliases:
- ${REDIS__HOST:-redis} - ${REDIS__HOST}
{% endif %} {% endif %}
{%- if database != 'none' or use_redis %} {%- if database != 'none' or use_redis %}
volumes: volumes:
+6 -6
View File
@@ -1,15 +1,15 @@
{%- set deps = ["pydantic-settings>=2.14.2", "rich>=15.0.0"] -%} {%- set deps = ["pydantic-settings>=2.14.2", "rich>=15.0.0"] -%}
{%- if use_dishka %}{% set _ = deps.append("dishka>=1.10.1") %}{% endif -%} {%- if use_dishka %}{% set _ = deps.append("dishka>=1.10.1") %}{% endif -%}
{%- if 'api' in backend_services %}{% set _ = deps.append("fastapi>=0.138.0") %}{% set _ = deps.append("uvicorn[standard]>=0.49.0") %}{% endif -%} {%- if 'api' in backend_services %}{% set _ = deps.append("fastapi>=0.139.0") %}{% set _ = deps.append("uvicorn[standard]>=0.50.0") %}{% endif -%}
{%- if 'aiogram_bot' in backend_services %}{% set _ = deps.append("aiogram>=3.29.0") %}{% endif -%} {%- if 'aiogram_bot' in backend_services %}{% set _ = deps.append("aiogram>=3.29.0") %}{% endif -%}
{%- if include_payments %}{% set _ = deps.append("aiosend>=3.0.7") %}{% endif -%} {%- if include_payments %}{% set _ = deps.append("aiosend>=3.0.7") %}{% endif -%}
{%- if 'kurigram_userbot' in backend_services %}{% set _ = deps.append("anyio>=4.0.0") %}{% set _ = deps.append("kurigram>=2.2.7") %}{% set _ = deps.append("tgcrypto>=1.2.5") %}{% set _ = deps.append("uvloop>=0.21.0") %}{% endif -%} {%- if 'kurigram_userbot' in backend_services %}{% set _ = deps.append("anyio>=4.0.0") %}{% set _ = deps.append("kurigram>=2.2.15") %}{% set _ = deps.append("tgcrypto>=1.2.5") %}{% set _ = deps.append("uvloop>=0.21.0") %}{% endif -%}
{%- if 'taskiq_worker' in backend_services %}{% set _ = deps.append("taskiq>=0.12.4") %}{% if taskiq_broker == 'redis' %}{% set _ = deps.append("taskiq-redis>=1.2.3") %}{% endif %}{% if taskiq_broker == 'postgres' %}{% set _ = deps.append("taskiq-pg>=0.2.0") %}{% endif %}{% endif -%} {%- if 'taskiq_worker' in backend_services %}{% set _ = deps.append("taskiq>=0.13.0") %}{% if taskiq_broker == 'redis' %}{% set _ = deps.append("taskiq-redis>=1.2.3") %}{% endif %}{% if taskiq_broker == 'postgres' %}{% set _ = deps.append("taskiq-postgres[asyncpg]>=0.9.1") %}{% endif %}{% endif -%}
{%- if 'scheduler' in backend_services %}{% set _ = deps.append("apscheduler>=3.11.0") %}{% endif -%} {%- if 'scheduler' in backend_services %}{% set _ = deps.append("apscheduler>=3.11.0") %}{% endif -%}
{%- if database == 'mongo' %}{% set _ = deps.append("beanie>=2.0.1") %}{% endif -%} {%- if database == 'mongo' %}{% set _ = deps.append("beanie>=2.0.1") %}{% endif -%}
{%- if database == 'postgres' %}{% set _ = deps.append("sqlmodel>=0.0.38") %}{% set _ = deps.append("asyncpg>=0.31.0") %}{% set _ = deps.append("alembic>=1.18.4") %}{% set _ = deps.append("greenlet>=3.1.0") %}{% endif -%} {%- if database == 'postgres' %}{% set _ = deps.append("sqlmodel>=0.0.39") %}{% set _ = deps.append("asyncpg>=0.31.0") %}{% set _ = deps.append("alembic>=1.18.5") %}{% set _ = deps.append("greenlet>=3.5.3") %}{% endif -%}
{%- if use_redis %}{% set _ = deps.append("redis>=5.2.0") %}{% endif -%} {%- if use_redis %}{% set _ = deps.append("redis>=5.2.0") %}{% endif -%}
{%- if use_llm %}{% set _ = deps.append("pydantic-ai-slim[google]>=2.1.0") %}{% endif -%} {%- if use_llm %}{% set _ = deps.append("pydantic-ai-slim[google]>=2.15.0") %}{% endif -%}
{%- set modules = [] -%} {%- set modules = [] -%}
{%- if 'api' in backend_services %}{% set _ = modules.append("api") %}{% endif -%} {%- if 'api' in backend_services %}{% set _ = modules.append("api") %}{% endif -%}
{%- if 'aiogram_bot' in backend_services %}{% set _ = modules.append("bot") %}{% endif -%} {%- if 'aiogram_bot' in backend_services %}{% set _ = modules.append("bot") %}{% endif -%}
@@ -34,7 +34,7 @@ dependencies = [
] ]
[build-system] [build-system]
requires = ["uv_build>=0.11.14,<0.12.0"] requires = ["uv_build>=0.13.0,<0.14.0"]
build-backend = "uv_build" build-backend = "uv_build"
[tool.uv.build-backend] [tool.uv.build-backend]
@@ -2,7 +2,7 @@ import asyncio
import anyio import anyio
from userbot import PyroClient from userbot.modules.client import PyroClient
async def main() -> None: async def main() -> None:
@@ -0,0 +1,60 @@
import os
import subprocess
import sys
HEAVY = ["aiogram", "pyrogram", "fastapi", "google.genai"]
SERVICES = {
{%- if 'api' in backend_services %}
"api": (["api.app"], {"fastapi"}),
{%- endif %}
{%- if 'aiogram_bot' in backend_services %}
"bot": ({% if include_payments %}["bot"]{% else %}["bot.common", "bot.handlers"]{% endif %}, {"aiogram"}),
{%- endif %}
{%- if 'kurigram_userbot' in backend_services %}
"userbot": (["userbot.runner", "userbot.handlers"], {"pyrogram"}),
{%- endif %}
{%- if 'taskiq_worker' in backend_services %}
"worker": (["worker.broker", "worker.tasks"], {% if 'api' in backend_services %}{"fastapi"}{% else %}set(){% endif %}),
{%- elif 'scheduler' in backend_services %}
"scheduler": (["scheduler"], set()),
{%- endif %}
}
PROBE = """
import importlib, sys
for module in sys.argv[1].split(","):
importlib.import_module(module)
print(" ".join(name for name in sys.argv[2:] if name in sys.modules))
"""
{%- if 'aiogram_bot' in backend_services %}
ENV = {**os.environ, "BOT__TOKEN": "1:check"}
{%- else %}
ENV = dict(os.environ)
{%- endif %}
def loaded(modules: list[str]) -> tuple[set[str], str]:
result = subprocess.run( # noqa: S603
[sys.executable, "-c", PROBE, ",".join(modules), *HEAVY],
capture_output=True,
text=True,
cwd="src",
env=ENV,
check=False,
)
error = (result.stderr.strip().splitlines() or ["exit code"])[-1]
return set(result.stdout.split()) & set(HEAVY), error if result.returncode else ""
def main() -> int:
failed = False
for service, (modules, allowed) in SERVICES.items():
found, error = loaded(modules)
extra = sorted(found - allowed)
status = error or ("imports " + ", ".join(extra) if extra else "")
print(f"{service:10s} {'FAIL: ' + status if status else 'ok'}")
failed |= bool(status)
return int(failed)
if __name__ == "__main__":
sys.exit(main())
@@ -261,10 +261,9 @@ async def read_config() -> DynamicConfigBase:
async def write_config(config: DynamicConfigBase) -> DynamicConfigBase: async def write_config(config: DynamicConfigBase) -> DynamicConfigBase:
await init_db() await init_db()
doc = await DynamicConfig.get_or_create() await DynamicConfig.update_config(
for name in DynamicConfigBase.model_fields: **{name: getattr(config, name) for name in DynamicConfigBase.model_fields}
setattr(doc, name, getattr(config, name)) )
await doc.save()
return config return config
+38 -14
View File
@@ -1,3 +1,9 @@
{%- macro str_field(name, value) -%}
{%- set line = " " ~ name ~ ': str = "' ~ value ~ '"' -%}
{%- if line | length <= 88 or value | length + 10 > 88 %}{{ line }}{% else %} {{ name }}: str = (
"{{ value }}"
){% endif -%}
{%- endmacro -%}
{% if database != 'none' or use_redis %}import os {% if database != 'none' or use_redis %}import os
{% endif %}from pydantic import Field{% if database != 'none' or 'aiogram_bot' in backend_services or include_payments or use_llm or use_redis %}, SecretStr{% endif %} {% endif %}from pydantic import Field{% if database != 'none' or 'aiogram_bot' in backend_services or include_payments or use_llm or use_redis %}, SecretStr{% endif %}
@@ -8,30 +14,36 @@ def is_prod() -> bool:
return os.getenv("RUN_ENVIRONMENT") == "prod" return os.getenv("RUN_ENVIRONMENT") == "prod"
{% endif %} {% endif %}
class Section(BaseSettings):
model_config = SettingsConfigDict(extra="ignore")
class LogSettings(BaseSettings):
class LogSettings(Section):
level: str = "INFO" level: str = "INFO"
level_external: str = "WARNING" level_external: str = "WARNING"
show_time: bool = False show_time: bool = False
show_locals: bool = False
console_width: int = 150 console_width: int = 150
{% if database == 'mongo' %} {% if database == 'mongo' %}
class DatabaseSettings(BaseSettings): class DatabaseSettings(Section):
host: str = "mongodb" host: str = "mongodb"
port: int = 27017 port: int = 27017
user: str = "{{ project_slug }}" user: str = "{{ project_slug }}"
password: SecretStr = SecretStr("{{ project_slug }}") password: SecretStr = SecretStr("{{ project_slug }}")
db_name: str = "{{ project_slug }}" db_name: str = "{{ project_slug }}"
connection_params: str = "?authSource=admin" connection_params: str = "?authSource=admin"
{{ str_field("scripts_connection_url", "mongodb://" ~ project_slug ~ ":" ~ project_slug ~ "@localhost:27017/" ~ project_slug ~ "?authSource=admin") }}
@property @property
def connection_url(self) -> str: def connection_url(self) -> str:
host = self.host if is_prod() else "localhost" if not is_prod():
return self.scripts_connection_url
password = self.password.get_secret_value() password = self.password.get_secret_value()
return f"mongodb://{self.user}:{password}@{host}:{self.port}/{self.db_name}{self.connection_params}" return f"mongodb://{self.user}:{password}@{self.host}:{self.port}/{self.db_name}{self.connection_params}"
{% elif database == 'postgres' %} {% elif database == 'postgres' %}
class DatabaseSettings(BaseSettings): class DatabaseSettings(Section):
host: str = "postgres" host: str = "postgres"
port: int = 5432 port: int = 5432
user: str = "{{ project_slug }}" user: str = "{{ project_slug }}"
@@ -39,51 +51,56 @@ class DatabaseSettings(BaseSettings):
db_name: str = "{{ project_slug }}" db_name: str = "{{ project_slug }}"
min_pool_size: int = 5 min_pool_size: int = 5
max_pool_size: int = 20 max_pool_size: int = 20
{{ str_field("scripts_connection_url", "postgresql://" ~ project_slug ~ ":" ~ project_slug ~ "@localhost:5432/" ~ project_slug) }}
@property @property
def connection_url(self) -> str: def connection_url(self) -> str:
host = self.host if is_prod() else "localhost" if not is_prod():
return self.scripts_connection_url
password = self.password.get_secret_value() password = self.password.get_secret_value()
return f"postgresql://{self.user}:{password}@{host}:{self.port}/{self.db_name}" return f"postgresql://{self.user}:{password}@{self.host}:{self.port}/{self.db_name}"
@property @property
def async_connection_url(self) -> str: def async_connection_url(self) -> str:
return self.connection_url.replace("postgresql://", "postgresql+asyncpg://", 1) return self.connection_url.replace("postgresql://", "postgresql+asyncpg://", 1)
{% endif %}{% if use_redis %} {% endif %}{% if use_redis %}
class RedisSettings(BaseSettings): class RedisSettings(Section):
host: str = "redis" host: str = "redis"
port: int = 6379 port: int = 6379
db: int = 0 db: int = 0
password: SecretStr | None = None password: SecretStr | None = None
cache_ttl: int = 300 cache_ttl: int = 300
scripts_connection_url: str = "redis://localhost:6379/0"
@property @property
def url(self) -> str: def url(self) -> str:
host = self.host if is_prod() else "localhost" if not is_prod():
return self.scripts_connection_url
auth = f":{self.password.get_secret_value()}@" if self.password else "" auth = f":{self.password.get_secret_value()}@" if self.password else ""
return f"redis://{auth}{host}:{self.port}/{self.db}" return f"redis://{auth}{self.host}:{self.port}/{self.db}"
{% endif %}{% if 'api' in backend_services %} {% endif %}{% if 'api' in backend_services %}
class ApiSettings(BaseSettings): class ApiSettings(Section):
host: str = "0.0.0.0" # noqa: S104 host: str = "0.0.0.0" # noqa: S104
port: int = 8080 port: int = 8080
workers: int = 1 workers: int = 1
docs: bool = False
{%- if serve_spa %} {%- if serve_spa %}
static_dir: str = "static" static_dir: str = "static"
{%- endif %} {%- endif %}
{% endif %}{% if 'aiogram_bot' in backend_services %} {% endif %}{% if 'aiogram_bot' in backend_services %}
class BotSettings(BaseSettings): class BotSettings(Section):
token: SecretStr = SecretStr("") token: SecretStr = SecretStr("")
admins: list[int] = Field(default_factory=list) admins: list[int] = Field(default_factory=list)
{% endif %}{% if include_payments %} {% endif %}{% if include_payments %}
class CryptoPaySettings(BaseSettings): class CryptoPaySettings(Section):
token: SecretStr = SecretStr("") token: SecretStr = SecretStr("")
{% endif %}{% if use_llm %} {% endif %}{% if use_llm %}
class LlmSettings(BaseSettings): class LlmSettings(Section):
model: str = "google-gla:gemini-2.5-flash" model: str = "google-gla:gemini-2.5-flash"
gemini_api_key: SecretStr = SecretStr("") gemini_api_key: SecretStr = SecretStr("")
{% endif %} {% endif %}
@@ -110,7 +127,14 @@ class Settings(BaseSettings):
{%- endif %} {%- endif %}
model_config = SettingsConfigDict( model_config = SettingsConfigDict(
{%- if layout == 'part' %}
case_sensitive=False,
env_file=("../.env", ".env"),
env_nested_delimiter="__",
extra="ignore",
{%- else %}
case_sensitive=False, env_file=".env", env_nested_delimiter="__", extra="ignore" case_sensitive=False, env_file=".env", env_nested_delimiter="__", extra="ignore"
{%- endif %}
) )
@@ -0,0 +1,6 @@
import asyncio
from importlib import import_module
async def preload(module: str) -> None:
await asyncio.to_thread(import_module, module)
@@ -1,5 +1,7 @@
import logging import logging
{% if 'aiogram_bot' in backend_services %}from importlib.util import find_spec
from pathlib import Path
{% endif %}
from rich.console import Console from rich.console import Console
from rich.logging import RichHandler from rich.logging import RichHandler
from rich.traceback import install from rich.traceback import install
@@ -10,8 +12,12 @@ console = Console(width=env.log.console_width, color_system="auto", force_termin
def setup_logging() -> None: def setup_logging() -> None:
{% if 'aiogram_bot' in backend_services %} from aiogram.dispatcher import router # noqa: PLC0415 {% if 'aiogram_bot' in backend_services %} aiogram = find_spec("aiogram")
suppress = (
[str(Path(aiogram.origin).parent / "dispatcher")]
if aiogram and aiogram.origin
else []
)
{% endif %} logging.basicConfig( {% endif %} logging.basicConfig(
level=env.log.level_external, level=env.log.level_external,
format="", format="",
@@ -22,16 +28,16 @@ def setup_logging() -> None:
markup=True, markup=True,
rich_tracebacks=True, rich_tracebacks=True,
enable_link_path=False, enable_link_path=False,
tracebacks_show_locals=True, tracebacks_show_locals=env.log.show_locals,
omit_repeated_times=False, omit_repeated_times=False,
show_time=env.log.show_time, show_time=env.log.show_time,
{%- if 'aiogram_bot' in backend_services %} {%- if 'aiogram_bot' in backend_services %}
tracebacks_suppress=[router], tracebacks_suppress=suppress,
{%- endif %} {%- endif %}
) )
], ],
) )
install(console=console, show_locals=True) install(console=console, show_locals=env.log.show_locals)
logger = logging.getLogger("{{ project_slug }}") logger = logging.getLogger("{{ project_slug }}")
@@ -22,6 +22,7 @@ class DynamicConfig(SQLModel, table=True):
id: int = SQLField(default=1, primary_key=True) id: int = SQLField(default=1, primary_key=True)
data: dict[str, Any] = SQLField(sa_column=Column(postgresql.JSONB, nullable=False)) data: dict[str, Any] = SQLField(sa_column=Column(postgresql.JSONB, nullable=False))
version: int = SQLField(default=0)
updated_at: datetime | None = SQLField( updated_at: datetime | None = SQLField(
default=None, default=None,
sa_column=Column( sa_column=Column(
@@ -32,8 +33,19 @@ class DynamicConfig(SQLModel, table=True):
), ),
) )
{%- else -%} {%- else -%}
import time
from dataclasses import dataclass
from beanie import Document from beanie import Document
from beanie.odm.operators.update import BaseUpdateOperator
from beanie.operators import Inc, Set
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
from pymongo.errors import PyMongoError
from utils.logging import logger
CONFIG_CHECK_SECONDS = 5.0
CONFIG_TTL_SECONDS = 900.0
class BotConfig(BaseModel): class BotConfig(BaseModel):
@@ -44,15 +56,69 @@ class DynamicConfigBase(BaseModel):
bot: BotConfig = Field(default_factory=BotConfig) bot: BotConfig = Field(default_factory=BotConfig)
@dataclass
class _ConfigCache:
value: "DynamicConfig | None" = None
loaded_at: float = 0.0
checked_at: float = 0.0
_cache = _ConfigCache()
class DynamicConfig(DynamicConfigBase, Document): class DynamicConfig(DynamicConfigBase, Document):
version: int = 0
class Settings: class Settings:
name = "config" name = "config"
@classmethod @classmethod
async def get_or_create(cls) -> "DynamicConfig": async def get_or_create(cls) -> "DynamicConfig":
config = await cls.find_one() now = time.monotonic()
if config is None: current = _cache.value
config = cls() if current is None:
await config.save() current = await cls._reload()
return config elif now - _cache.loaded_at >= CONFIG_TTL_SECONDS:
current = await cls._reload_or_keep(current)
elif now - _cache.checked_at >= CONFIG_CHECK_SECONDS:
_cache.checked_at = now
current = await cls._refresh_if_changed(current)
return current.model_copy(deep=True)
@classmethod
async def update_config(
cls, *operations: BaseUpdateOperator, **sections: BaseModel
) -> "DynamicConfig":
current = await cls.get_or_create()
changes = [*operations, Set(sections)] if sections else [*operations]
await cls.find_one(cls.id == current.id).update(*changes, Inc({cls.version: 1}))
return (await cls._reload()).model_copy(deep=True)
@classmethod
async def _reload(cls) -> "DynamicConfig":
loaded = await cls.find_one() or await cls().insert()
_cache.value = loaded
_cache.loaded_at = _cache.checked_at = time.monotonic()
return loaded
@classmethod
async def _reload_or_keep(cls, current: "DynamicConfig") -> "DynamicConfig":
try:
return await cls._reload()
except PyMongoError as e:
logger.warning(f"Config reload failed, serving the cached copy: {e}")
return current
@classmethod
async def _refresh_if_changed(cls, current: "DynamicConfig") -> "DynamicConfig":
try:
stored = await cls.get_pymongo_collection().find_one(
{"_id": current.id}, {"version": 1}
)
except PyMongoError as e:
logger.warning(f"Config version check failed, serving the cached copy: {e}")
return current
if stored and stored.get("version", 0) == current.version:
return current
return await cls._reload_or_keep(current)
{%- endif %} {%- endif %}
@@ -1,88 +1,139 @@
{% if use_redis -%} import time
{%- if use_redis %}
from contextlib import suppress
{%- endif %}
from typing import Any
from pydantic import BaseModel
{%- if use_redis %}
from redis.exceptions import RedisError
{%- endif %}
from sqlalchemy import literal, update
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.exc import SQLAlchemyError
from sqlalchemy.ext.asyncio import async_sessionmaker from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlmodel import col, select
from sqlmodel.ext.asyncio.session import AsyncSession from sqlmodel.ext.asyncio.session import AsyncSession
from utils.db.models.config import DynamicConfig, DynamicConfigBase from utils.db.models.config import DynamicConfig, DynamicConfigBase
from utils.logging import logger
{%- if use_redis %}
from .cache import RedisCache from .cache import RedisCache
{%- endif %}
CONFIG_CACHE_KEY = "config:dynamic"
CONFIG_ROW_ID = 1 CONFIG_ROW_ID = 1
CONFIG_CHECK_SECONDS = 5.0
CONFIG_TTL_SECONDS = 900.0
{%- if use_redis %}
CONFIG_CACHE_KEY = "config:dynamic"
{%- endif %}
class ConfigRepository: class ConfigRepository:
def __init__( def __init__(
self, sessionmaker: async_sessionmaker[AsyncSession], cache: RedisCache self, sessionmaker: async_sessionmaker[AsyncSession]{% if use_redis %}, cache: RedisCache{% endif %}
) -> None: ) -> None:
self._sessionmaker = sessionmaker self._sessionmaker = sessionmaker
{%- if use_redis %}
self._cache = cache self._cache = cache
async def get(self) -> DynamicConfigBase:
return await self._cache.get_or_set_model(
CONFIG_CACHE_KEY, DynamicConfigBase, self._load
)
async def _load(self) -> DynamicConfigBase:
async with self._sessionmaker() as session:
row = await session.get(DynamicConfig, CONFIG_ROW_ID)
if row is None:
config = DynamicConfigBase()
session.add(DynamicConfig(id=CONFIG_ROW_ID, data=config.model_dump()))
await session.commit()
return config
return DynamicConfigBase.model_validate(row.data)
async def save(self, config: DynamicConfigBase) -> DynamicConfigBase:
async with self._sessionmaker() as session:
row = await session.get(DynamicConfig, CONFIG_ROW_ID)
if row is None:
session.add(DynamicConfig(id=CONFIG_ROW_ID, data=config.model_dump()))
else:
row.data = config.model_dump()
session.add(row)
await session.commit()
await self._cache.set_model(CONFIG_CACHE_KEY, config)
return config
async def reset(self) -> DynamicConfigBase:
return await self.save(DynamicConfigBase())
async def invalidate(self) -> None:
await self._cache.invalidate(CONFIG_CACHE_KEY)
{%- else -%}
from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlmodel.ext.asyncio.session import AsyncSession
from utils.db.models.config import DynamicConfig, DynamicConfigBase
CONFIG_ROW_ID = 1
class ConfigRepository:
def __init__(self, sessionmaker: async_sessionmaker[AsyncSession]) -> None:
self._sessionmaker = sessionmaker
async def get(self) -> DynamicConfigBase:
async with self._sessionmaker() as session:
row = await session.get(DynamicConfig, CONFIG_ROW_ID)
if row is None:
config = DynamicConfigBase()
session.add(DynamicConfig(id=CONFIG_ROW_ID, data=config.model_dump()))
await session.commit()
return config
return DynamicConfigBase.model_validate(row.data)
async def save(self, config: DynamicConfigBase) -> DynamicConfigBase:
async with self._sessionmaker() as session:
row = await session.get(DynamicConfig, CONFIG_ROW_ID)
if row is None:
session.add(DynamicConfig(id=CONFIG_ROW_ID, data=config.model_dump()))
else:
row.data = config.model_dump()
session.add(row)
await session.commit()
return config
async def reset(self) -> DynamicConfigBase:
return await self.save(DynamicConfigBase())
{%- endif %} {%- endif %}
self._value: DynamicConfigBase | None = None
self._version = -1
self._loaded_at = 0.0
self._checked_at = 0.0
async def get(self) -> DynamicConfigBase:
now = time.monotonic()
current = self._value
if current is None:
current = await self._reload()
elif now - self._loaded_at >= CONFIG_TTL_SECONDS:
current = await self._reload_or_keep(current)
elif now - self._checked_at >= CONFIG_CHECK_SECONDS:
self._checked_at = now
current = await self._refresh_if_changed(current)
return current.model_copy(deep=True)
async def save(self, config: DynamicConfigBase) -> DynamicConfigBase:
return await self._write(config.model_dump(mode="json"), merge=False)
async def update(self, **sections: BaseModel) -> DynamicConfigBase:
patch = {name: value.model_dump(mode="json") for name, value in sections.items()}
return await self._write(patch, merge=True)
async def reset(self) -> DynamicConfigBase:
return await self.save(DynamicConfigBase())
async def _write(self, data: dict[str, Any], *, merge: bool) -> DynamicConfigBase:
await self.get()
payload = literal(data, JSONB)
async with self._sessionmaker() as session:
await session.exec(
update(DynamicConfig)
.where(col(DynamicConfig.id) == CONFIG_ROW_ID)
.values(
data=col(DynamicConfig.data).op("||")(payload) if merge else payload,
version=col(DynamicConfig.version) + 1,
)
)
await session.commit()
return (await self._reload()).model_copy(deep=True)
async def _stored_version(self) -> int | None:
async with self._sessionmaker() as session:
query = select(DynamicConfig.version).where(
col(DynamicConfig.id) == CONFIG_ROW_ID
)
return (await session.exec(query)).first()
async def _reload(self) -> DynamicConfigBase:
{%- if use_redis %}
version = await self._stored_version()
if version is not None:
with suppress(RedisError):
key = f"{CONFIG_CACHE_KEY}:{version}"
cached = await self._cache.get_model(key, DynamicConfigBase)
if cached is not None:
return self._remember(cached, version)
{%- endif %}
async with self._sessionmaker() as session:
row = await session.get(DynamicConfig, CONFIG_ROW_ID)
if row is None:
row = DynamicConfig(
id=CONFIG_ROW_ID, data=DynamicConfigBase().model_dump(mode="json")
)
session.add(row)
await session.commit()
await session.refresh(row)
value = DynamicConfigBase.model_validate(row.data)
version = row.version
{%- if use_redis %}
with suppress(RedisError):
await self._cache.set_model(f"{CONFIG_CACHE_KEY}:{version}", value)
{%- endif %}
return self._remember(value, version)
def _remember(self, value: DynamicConfigBase, version: int) -> DynamicConfigBase:
self._value, self._version = value, version
self._loaded_at = self._checked_at = time.monotonic()
return value
async def _reload_or_keep(self, current: DynamicConfigBase) -> DynamicConfigBase:
try:
return await self._reload()
except SQLAlchemyError as e:
logger.warning(f"Config reload failed, serving the cached copy: {e}")
return current
async def _refresh_if_changed(
self, current: DynamicConfigBase
) -> DynamicConfigBase:
try:
version = await self._stored_version()
except SQLAlchemyError as e:
logger.warning(f"Config version check failed, serving the cached copy: {e}")
return current
if version == self._version:
return current
return await self._reload_or_keep(current)
@@ -5,7 +5,11 @@ from utils.env import env
def main() -> None: def main() -> None:
uvicorn.run( uvicorn.run(
"api.app:app", host=env.api.host, port=env.api.port, workers=env.api.workers "api.app:app",
host=env.api.host,
port=env.api.port,
workers=env.api.workers,
forwarded_allow_ips="*",
) )
@@ -10,15 +10,15 @@ from fastapi.middleware.cors import CORSMiddleware
from api import routers from api import routers
{% if use_dishka %}from dependencies.container import container {% if use_dishka %}from dependencies.container import container
{% endif %}{% if database != 'none' %}from utils.db import init_db {% endif %}{% if database != 'none' %}from utils.db import init_db
{% endif %}{% if serve_spa %}from utils.env import env {% endif %}from utils.env import env
{% endif %}from utils.logging import setup_logging from utils.logging import setup_logging
{% if serve_spa %} {% if serve_spa %}
IMMUTABLE_HEADERS = {"Cache-Control": "public, max-age=31536000, immutable"} IMMUTABLE_HEADERS = {"Cache-Control": "public, max-age=31536000, immutable"}
NO_CACHE_HEADERS = {"Cache-Control": "no-cache"} NO_CACHE_HEADERS = {"Cache-Control": "no-cache"}
{% endif %} {% endif %}
@asynccontextmanager @asynccontextmanager
async def lifespan(app_: FastAPI) -> AsyncGenerator[None]: async def lifespan({% if use_dishka %}app_{% else %}_app{% endif %}: FastAPI) -> AsyncGenerator[None]:
setup_logging() setup_logging()
{%- if database != 'none' %} {%- if database != 'none' %}
await init_db() await init_db()
@@ -29,14 +29,19 @@ async def lifespan(app_: FastAPI) -> AsyncGenerator[None]:
{%- endif %} {%- endif %}
app = FastAPI(title="{{ project_name }} API", lifespan=lifespan) app = FastAPI(
title="{{ project_name }} API",
lifespan=lifespan,
openapi_url="/openapi.json" if env.api.docs else None,
)
app.add_middleware( app.add_middleware(
CORSMiddleware, allow_origins=["*"], allow_methods=["*"], allow_headers=["*"] CORSMiddleware, allow_origins=["*"], allow_methods=["*"], allow_headers=["*"]
) )
app.include_router(routers.router) app.include_router(routers.router)
{% if serve_spa %} {%- if serve_spa %}
spa_dir = Path(env.api.static_dir).resolve() spa_dir = Path(env.api.static_dir).resolve()
if spa_dir.is_dir(): if spa_dir.is_dir():
@@ -48,6 +53,9 @@ if spa_dir.is_dir():
headers = IMMUTABLE_HEADERS if immutable else NO_CACHE_HEADERS headers = IMMUTABLE_HEADERS if immutable else NO_CACHE_HEADERS
return FileResponse(candidate, headers=headers) return FileResponse(candidate, headers=headers)
return FileResponse(spa_dir / "index.html", headers=NO_CACHE_HEADERS) return FileResponse(spa_dir / "index.html", headers=NO_CACHE_HEADERS)
{% endif %} {%- endif %}
{% if use_dishka %}setup_dishka(container, app) {%- if use_dishka %}
{% endif %}
{% if serve_spa %}
{% endif %}setup_dishka(container, app)
{%- endif %}
@@ -1,3 +0,0 @@
from .modules.client import PyroClient
__all__ = ["PyroClient"]
@@ -1,12 +1,11 @@
from pyrogram import filters, types from pyrogram import Client, filters, types
from pyrogram.handlers import MessageHandler
from userbot import PyroClient
from utils.logging import logger from utils.logging import logger
@PyroClient.on_message(filters.me) async def echo(_: Client, message: types.Message) -> None:
async def echo(_: PyroClient, message: types.Message) -> None:
logger.info(f"[cyan]me:[/] {message.text}") logger.info(f"[cyan]me:[/] {message.text}")
handlers = echo.handlers handlers = [(MessageHandler(echo, filters.me), 0)]
@@ -1,13 +1,12 @@
from pyrogram import filters, types from pyrogram import Client, filters, types
from pyrogram.handlers import MessageHandler
from userbot import PyroClient
from utils.logging import logger from utils.logging import logger
@PyroClient.on_message(filters.private & ~filters.me) async def pm_message(_: Client, message: types.Message) -> None:
async def pm_message(_: PyroClient, message: types.Message) -> None:
sender = message.from_user.id if message.from_user else "unknown" sender = message.from_user.id if message.from_user else "unknown"
logger.info(f"[green]pm from {sender}:[/] {message.text}") logger.info(f"[green]pm from {sender}:[/] {message.text}")
handlers = pm_message.handlers handlers = [(MessageHandler(pm_message, filters.private & ~filters.me), 0)]
@@ -1,20 +1,30 @@
from pyrogram import Client, enums from pyrogram import Client, enums
DEVICE_MODEL = "{{ project_name[:32] }}"
class PyroClient(Client): class PyroClient(Client):
def __init__( def __init__(
self, name: str, *, workdir: str = "sessions", load_handlers: bool = True self,
name: str,
*,
workdir: str = "sessions",
device_model: str | None = None,
load_handlers: bool = True,
) -> None: ) -> None:
super().__init__( super().__init__(
name, name,
workdir=workdir, workdir=workdir,
api_id=2040, api_id=2040,
api_hash="b18441a1ff607e10a989891a5462e627", api_hash="b18441a1ff607e10a989891a5462e627",
device_model="Desktop", device_model=device_model or DEVICE_MODEL,
system_version="Windows 11 x64", system_version="Windows 11 x64",
app_version="6.2.4 x64", app_version="7.0.8 x64",
lang_pack="tdesktop", lang_pack="tdesktop",
lang_code="en",
system_lang_code="en-US",
client_platform=enums.ClientPlatform.DESKTOP, client_platform=enums.ClientPlatform.DESKTOP,
max_concurrent_transmissions=4,
) )
if load_handlers: if load_handlers:
@@ -24,4 +34,4 @@ class PyroClient(Client):
self.add_handler(*handler) self.add_handler(*handler)
__all__ = ["PyroClient"] __all__ = ["DEVICE_MODEL", "PyroClient"]
@@ -1,5 +1,6 @@
import asyncio import asyncio
import contextlib import contextlib
import signal
import anyio import anyio
import uvloop import uvloop
@@ -11,8 +12,15 @@ import uvloop
setup_logging() setup_logging()
SCAN_INTERVAL = 10
async def runner() -> None: async def runner() -> None:
stop = asyncio.Event()
loop = asyncio.get_running_loop()
for sig in (signal.SIGINT, signal.SIGTERM):
loop.add_signal_handler(sig, stop.set)
{% if database != 'none' %} await init_db() {% if database != 'none' %} await init_db()
{% endif %} sessions_dir = anyio.Path("sessions") {% endif %} sessions_dir = anyio.Path("sessions")
@@ -22,7 +30,7 @@ async def runner() -> None:
clients: dict[int, PyroClient] = {} clients: dict[int, PyroClient] = {}
try: try:
while True: while not stop.is_set():
current = {path.name async for path in sessions_dir.glob("*.session")} current = {path.name async for path in sessions_dir.glob("*.session")}
for session_file in current - started: for session_file in current - started:
@@ -43,7 +51,8 @@ async def runner() -> None:
if me: if me:
clients[me.id] = client clients[me.id] = client
await asyncio.sleep(10) with contextlib.suppress(TimeoutError):
await asyncio.wait_for(stop.wait(), SCAN_INTERVAL)
finally: finally:
for client in clients.values(): for client in clients.values():
with contextlib.suppress(Exception): with contextlib.suppress(Exception):
@@ -55,8 +64,5 @@ def main() -> None:
uvloop.install() uvloop.install()
logger.info("Starting...") logger.info("Starting...")
with contextlib.suppress(KeyboardInterrupt):
asyncio.run(runner()) asyncio.run(runner())
logger.info("[red]Stopped.[/]") logger.info("[red]Stopped.[/]")
@@ -1,5 +1,5 @@
import asyncio import asyncio
import contextlib import signal
from apscheduler.schedulers.asyncio import AsyncIOScheduler from apscheduler.schedulers.asyncio import AsyncIOScheduler
from apscheduler.triggers.interval import IntervalTrigger from apscheduler.triggers.interval import IntervalTrigger
@@ -15,6 +15,10 @@ async def example_job() -> None:
async def runner() -> None: async def runner() -> None:
stop = asyncio.Event()
loop = asyncio.get_running_loop()
for sig in (signal.SIGINT, signal.SIGTERM):
loop.add_signal_handler(sig, stop.set)
{%- if database != 'none' %} {%- if database != 'none' %}
await init_db() await init_db()
{%- endif %} {%- endif %}
@@ -27,14 +31,13 @@ async def runner() -> None:
scheduler.start() scheduler.start()
logger.info("Scheduler started") logger.info("Scheduler started")
with contextlib.suppress(asyncio.CancelledError): await stop.wait()
await asyncio.Event().wait() scheduler.shutdown()
def main() -> None: def main() -> None:
setup_logging() setup_logging()
logger.info("Starting...") logger.info("Starting...")
with contextlib.suppress(KeyboardInterrupt):
asyncio.run(runner()) asyncio.run(runner())
logger.info("[red]Stopped.[/]") logger.info("[red]Stopped.[/]")
@@ -43,7 +43,7 @@ setup_dishka(container, broker)
{% if use_dishka %}from dishka.integrations.taskiq import setup_dishka {% if use_dishka %}from dishka.integrations.taskiq import setup_dishka
{% endif %}{% if 'scheduler' in backend_services %}from taskiq import TaskiqScheduler {% endif %}{% if 'scheduler' in backend_services %}from taskiq import TaskiqScheduler
from taskiq.schedule_sources import LabelScheduleSource from taskiq.schedule_sources import LabelScheduleSource
{% endif %}from taskiq_pg import AsyncpgBroker, AsyncpgResultBackend {% endif %}from taskiq_pg.asyncpg import AsyncpgBroker, AsyncpgResultBackend
{% if use_dishka %}from dependencies.container import container {% if use_dishka %}from dependencies.container import container
{% endif %}from utils.env import env {% endif %}from utils.env import env
@@ -4,11 +4,7 @@
{%- if database == 'postgres' %}{% set _ = providers.append('RepositoryProvider()') %}{% endif -%} {%- if database == 'postgres' %}{% set _ = providers.append('RepositoryProvider()') %}{% endif -%}
{%- if use_redis %}{% set _ = providers.append('RedisProvider()') %}{% endif -%} {%- if use_redis %}{% set _ = providers.append('RedisProvider()') %}{% endif -%}
{%- if use_llm %}{% set _ = providers.append('LlmProvider()') %}{% endif -%} {%- if use_llm %}{% set _ = providers.append('LlmProvider()') %}{% endif -%}
{%- if 'aiogram_bot' in backend_services %}{% set _ = providers.append('AiogramProvider()') %}{% endif -%}
{%- if 'api' in backend_services %}{% set _ = providers.append('FastapiProvider()') %}{% endif -%}
{%- set third = ['from dishka import make_async_container'] -%} {%- set third = ['from dishka import make_async_container'] -%}
{%- if 'aiogram_bot' in backend_services %}{% set _ = third.append('from dishka.integrations.aiogram import AiogramProvider') %}{% endif -%}
{%- if 'api' in backend_services %}{% set _ = third.append('from dishka.integrations.fastapi import FastapiProvider') %}{% endif -%}
{%- set first = [] -%} {%- set first = [] -%}
{%- if dynamic_config %}{% set _ = first.append('from dependencies.providers.config import ConfigProvider') %}{% endif -%} {%- if dynamic_config %}{% set _ = first.append('from dependencies.providers.config import ConfigProvider') %}{% endif -%}
{%- if database != 'none' %}{% set _ = first.append('from dependencies.providers.db import DbProvider') %}{% endif -%} {%- if database != 'none' %}{% set _ = first.append('from dependencies.providers.db import DbProvider') %}{% endif -%}
@@ -1,14 +1,20 @@
from dishka import Provider, Scope, provide from dishka import Provider, Scope, provide
from pydantic_ai import Agent from pydantic_ai import Agent
from pydantic_ai.models.google import GoogleModel, GoogleModelSettings
from pydantic_ai.providers.google import GoogleProvider
from utils.env import env from utils.env import env
from utils.imports import preload
class LlmProvider(Provider): class LlmProvider(Provider):
@provide(scope=Scope.APP) @provide(scope=Scope.APP)
def agent(self) -> Agent: async def agent(self) -> Agent:
await preload("pydantic_ai.models.google")
from pydantic_ai.models.google import ( # noqa: PLC0415
GoogleModel,
GoogleModelSettings,
)
from pydantic_ai.providers.google import GoogleProvider # noqa: PLC0415
model = GoogleModel( model = GoogleModel(
env.llm.model.split(":", 1)[-1], env.llm.model.split(":", 1)[-1],
provider=GoogleProvider(api_key=env.llm.gemini_api_key.get_secret_value()), provider=GoogleProvider(api_key=env.llm.gemini_api_key.get_secret_value()),
@@ -16,10 +16,15 @@ if config.config_file_name is not None:
target_metadata = SQLModel.metadata target_metadata = SQLModel.metadata
def include_name(name: str | None, type_: str, _parents: object) -> bool:
return not (type_ == "table" and name is not None and name.startswith("taskiq_"))
def run_migrations_offline() -> None: def run_migrations_offline() -> None:
context.configure( context.configure(
url=env.db.connection_url, url=env.db.connection_url,
target_metadata=target_metadata, target_metadata=target_metadata,
include_name=include_name,
literal_binds=True, literal_binds=True,
dialect_opts={"paramstyle": "named"}, dialect_opts={"paramstyle": "named"},
) )
@@ -28,7 +33,11 @@ def run_migrations_offline() -> None:
def do_run_migrations(connection: Connection) -> None: def do_run_migrations(connection: Connection) -> None:
context.configure(connection=connection, target_metadata=target_metadata) context.configure(
connection=connection,
target_metadata=target_metadata,
include_name=include_name,
)
with context.begin_transaction(): with context.begin_transaction():
context.run_migrations() context.run_migrations()
@@ -1,7 +0,0 @@
## Checking commands
After writing code, always run from `backend/`:
```shell
ruff format
ruff check --fix
ty check
```
@@ -0,0 +1,10 @@
## Checking commands
After writing code, always run from `backend/`:
```shell
make fmt
make check
```
{% if backend_services or dynamic_config %}
{% endif %}{% if backend_services %}Mind startup imports: `make check` runs `scripts/check_imports.py`.
{% endif %}{% if dynamic_config %}Change dynamic config only through its update methods.
{% endif %}
@@ -0,0 +1,16 @@
{%- set targets = [] -%}
{%- if database == 'postgres' %}{% set _ = targets.append("`make migrate`, `make revision m=\"message\"`") %}{% endif -%}
{%- if dynamic_config and backend_services %}{% set _ = targets.append("`make script name`") %}{% endif -%}
{%- if 'kurigram_userbot' in backend_services %}{% set _ = targets.append("`make session`") %}{% endif -%}
# {{ project_name }}
{% if project_description and project_description != project_name %}
{{ project_description }}
{% endif %}
```sh
make env # .env and docker-compose.override.yml from the examples
make recreate # the services in COMPOSE_PROFILES
make check
```
{% if targets %}
Also {{ targets | join(", ") }}.
{% endif %}