Compare commits
24
Commits
v0.2.0
..
a5345a9e4c
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a5345a9e4c | ||
|
|
2be4367f9f | ||
|
|
eed732fbcf | ||
|
|
4cb3b85321 | ||
|
|
92716b79b6 | ||
|
|
964b9edf67 | ||
|
|
c19cdc4e11 | ||
|
|
97fdafa092 | ||
|
|
4d2ed061ea | ||
|
|
85ce05a55e | ||
|
|
47debebc0c | ||
|
|
eadfa3d8ec | ||
|
|
6028507874 | ||
|
|
db8184cb37 | ||
|
|
2c1dfefe34 | ||
|
|
b4125407bf | ||
|
|
e0d5defa97 | ||
|
|
dae9532ef6 | ||
|
|
e316bc92a5 | ||
|
|
3878837d25 | ||
|
|
b1db7c274f | ||
|
|
ac0ccf47e5 | ||
|
|
1f416dbe60 | ||
|
|
9afa7da9c4 |
No files matched your search
@@ -1,8 +1,10 @@
|
||||
# backend-python
|
||||
|
||||
Copier template for a Python backend, either as a `backend/` part next to other parts or alone at the root of the project (`layout`): a uv project with one Docker image and one `python -m <module>` entrypoint per service (`api`, `bot`, `userbot`, `worker`, `scheduler`), pydantic-settings, rich logging, optional Dishka, MongoDB or PostgreSQL, Redis, taskiq, CryptoPay, pydantic-ai and a DynamicConfig stored in the database.
|
||||
Copier template for a Python backend, either as a `backend/` part next to other parts or alone at the root of the project (`layout`): a uv project with one Docker image and one `python -m <module>` entrypoint per service (`api`, `bot`, `userbot`, `worker`, `scheduler`), pydantic-settings, rich logging, optional Dishka, MongoDB or PostgreSQL, Redis, taskiq (in-memory, Redis or PostgreSQL broker), CryptoPay, pydantic-ai and a DynamicConfig stored in the database.
|
||||
|
||||
Ships `backend/docker-compose.yml` with root-relative paths, `docker-compose.local.yml` (published ports) and `docker-compose.prod.yml` (the `caddy` network), a `Makefile` with `fmt`, `check`, `build` and service-specific targets, `.env.example` and `CLAUDE.md`. Applied on top of the `infra` template, `layout: parts` or `layout: single` respectively.
|
||||
Ships `backend/docker-compose.yml` with root-relative paths and profiles per service, `docker-compose.override.yml.example` (published ports and the `caddy` network aliases, copied by `make env`), a `Makefile` with `fmt`, `check` (ruff, ty and `scripts/check_imports.py`, which keeps each service from importing the others' SDKs at startup), `build` and service-specific targets, `.env.example` and `CLAUDE.md`. Applied on top of the `infra` template, `layout: parts` or `layout: single` respectively.
|
||||
|
||||
`DB__HOST`/`DB__PORT` and `REDIS__HOST`/`REDIS__PORT` say where the service really is. The bundled container takes the host as its network alias and listens on the port, the override publishes it on the machine. For an external database, point them at it and leave its profile out of `COMPOSE_PROFILES`.
|
||||
|
||||
```sh
|
||||
copier copy --trust -d project_name="My Thing" <this-repo> <dest>
|
||||
|
||||
@@ -71,6 +71,9 @@ taskiq_broker:
|
||||
choices:
|
||||
in-memory: memory
|
||||
Redis: redis
|
||||
PostgreSQL (taskiq-postgres):
|
||||
value: postgres
|
||||
validator: "{% if database != 'postgres' %}Requires the PostgreSQL database{% endif %}"
|
||||
default: memory
|
||||
|
||||
include_payments:
|
||||
|
||||
@@ -2,4 +2,4 @@
|
||||
python = "{{ prefix }}.venv"
|
||||
|
||||
[src]
|
||||
exclude = ["{{ prefix }}migrations"]
|
||||
exclude = ["docs", "{{ prefix }}migrations"]
|
||||
@@ -1,27 +1,14 @@
|
||||
{% if layout == 'root' %}COMPOSE_FILE=docker-compose.yml:docker-compose.local.yml
|
||||
{% endif %}RUN_ENVIRONMENT=dev
|
||||
LOG__LEVEL=INFO
|
||||
LOG__LEVEL_EXTERNAL=WARNING
|
||||
LOG__SHOW_TIME=false
|
||||
LOG__CONSOLE_WIDTH=150
|
||||
{% if database == 'mongo' %}DB__HOST=mongodb
|
||||
DB__PORT=27017
|
||||
DB__USER={{ project_slug }}
|
||||
DB__PASSWORD={{ project_slug }}
|
||||
DB__DB_NAME={{ project_slug }}
|
||||
{% endif %}{% if database == 'postgres' %}DB__HOST=postgres
|
||||
DB__PORT=5432
|
||||
DB__USER={{ project_slug }}
|
||||
DB__PASSWORD={{ project_slug }}
|
||||
DB__DB_NAME={{ project_slug }}
|
||||
{% endif %}{% if use_redis %}REDIS__HOST=redis
|
||||
REDIS__PORT=6379
|
||||
REDIS__CACHE_TTL=300
|
||||
{% endif %}{% if 'api' in backend_services %}API__HOST=0.0.0.0
|
||||
API__PORT=8080
|
||||
API__WORKERS=1
|
||||
{% endif %}{% if 'aiogram_bot' in backend_services %}BOT__TOKEN=
|
||||
{% endif %}{% if include_payments %}CRYPTO_PAY__TOKEN=
|
||||
{% endif %}{% if use_llm %}LLM__MODEL=google-gla:gemini-2.5-flash
|
||||
LLM__GEMINI_API_KEY=
|
||||
{% endif %}
|
||||
{%- set groups = [] -%}
|
||||
{%- if layout == 'root' %}{% set _ = groups.append(["COMPOSE_PROFILES=" ~ ("external," if database != 'none' or use_redis else "") ~ "services"]) %}{% endif -%}
|
||||
{%- set _ = groups.append(["RUN_ENVIRONMENT=prod", "TZ=UTC"]) -%}
|
||||
{%- set _ = groups.append(["LOG__LEVEL=INFO", "LOG__LEVEL_EXTERNAL=WARNING", "LOG__SHOW_TIME=false", "LOG__SHOW_LOCALS=false", "LOG__CONSOLE_WIDTH=150"]) -%}
|
||||
{%- if database == 'mongo' %}{% set _ = groups.append(["DB__HOST=mongodb", "DB__PORT=27017", "DB__USER=" ~ project_slug, "DB__PASSWORD=" ~ project_slug, "DB__DB_NAME=" ~ project_slug, "DB__SCRIPTS_CONNECTION_URL=mongodb://" ~ project_slug ~ ":" ~ project_slug ~ "@localhost:27017/" ~ project_slug ~ "?authSource=admin"]) %}{% endif -%}
|
||||
{%- if database == 'postgres' %}{% set _ = groups.append(["DB__HOST=postgres", "DB__PORT=5432", "DB__USER=" ~ project_slug, "DB__PASSWORD=" ~ project_slug, "DB__DB_NAME=" ~ project_slug, "DB__SCRIPTS_CONNECTION_URL=postgresql://" ~ project_slug ~ ":" ~ project_slug ~ "@localhost:5432/" ~ project_slug]) %}{% endif -%}
|
||||
{%- if use_redis %}{% set _ = groups.append(["REDIS__HOST=redis", "REDIS__PORT=6379", "REDIS__CACHE_TTL=300", "REDIS__SCRIPTS_CONNECTION_URL=redis://localhost:6379/0"]) %}{% endif -%}
|
||||
{%- if 'api' in backend_services %}{% set _ = groups.append(["API__HOST=0.0.0.0", "API__PORT=8080", "API__WORKERS=1", "API__DOCS=false"]) %}{% endif -%}
|
||||
{%- if 'aiogram_bot' in backend_services %}{% set _ = groups.append(["BOT__TOKEN="]) %}{% endif -%}
|
||||
{%- if include_payments %}{% set _ = groups.append(["CRYPTO_PAY__TOKEN="]) %}{% endif -%}
|
||||
{%- if use_llm %}{% set _ = groups.append(["LLM__MODEL=google-gla:gemini-2.5-flash", "LLM__GEMINI_API_KEY="]) %}{% endif -%}
|
||||
{% for group in groups %}{{ group | join("\n") }}
|
||||
{% if not loop.last %}
|
||||
{% endif %}{% endfor %}
|
||||
@@ -10,6 +10,8 @@ FROM ghcr.io/astral-sh/uv:python3.13-bookworm-slim
|
||||
RUN apt-get update && apt-get install -y build-essential && rm -rf /var/lib/apt/lists/*
|
||||
{%- else -%}
|
||||
FROM ghcr.io/astral-sh/uv:python3.13-alpine
|
||||
|
||||
RUN apk add --no-cache tzdata
|
||||
{%- endif %}
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
@@ -4,12 +4,23 @@
|
||||
{%- if 'taskiq_worker' in backend_services %}{% set _ = svc_modules.append("worker") %}{% endif -%}
|
||||
{%- if 'kurigram_userbot' in backend_services %}{% set _ = svc_modules.append("userbot") %}{% endif -%}
|
||||
{%- if 'scheduler' in backend_services and 'taskiq_worker' not in backend_services %}{% set _ = svc_modules.append("scheduler") %}{% endif -%}
|
||||
{%- set app_services = [] -%}
|
||||
{%- if 'api' in backend_services %}{% set _ = app_services.append("api") %}{% endif -%}
|
||||
{%- if 'aiogram_bot' in backend_services %}{% set _ = app_services.append("bot") %}{% endif -%}
|
||||
{%- if 'taskiq_worker' in backend_services %}{% set _ = app_services.append("worker") %}{% endif -%}
|
||||
{%- if 'kurigram_userbot' in backend_services %}{% set _ = app_services.append("userbot") %}{% endif -%}
|
||||
{%- if 'scheduler' in backend_services %}{% set _ = app_services.append("scheduler") %}{% endif -%}
|
||||
{%- set has_script = dynamic_config and svc_modules -%}
|
||||
COMPOSE := docker compose{% if layout == 'part' %} --project-directory ..{% endif %}
|
||||
{% if layout == 'root' %}SERVICES := $(COMPOSE) --profile services
|
||||
EXTERNAL := $(COMPOSE) --profile external
|
||||
{%- set has_scripts = dynamic_config or 'kurigram_userbot' in backend_services -%}
|
||||
COMPOSE := {% if layout == 'part' %}cd .. && {% endif %}docker compose
|
||||
{% if svc_modules %}IMAGE := {{ project_slug | replace('_', '-') }}/backend
|
||||
IMAGE_INPUTS := pyproject.toml uv.lock Dockerfile
|
||||
CODE := src{% if has_scripts %} scripts{% endif %}{% if serve_spa %} ../frontend{% endif %}
|
||||
SERVICES := {{ app_services | join(' ') }}
|
||||
changed = ! git diff --quiet $(1) -- $(2) 2>/dev/null
|
||||
stamp = { git update-ref $(1) HEAD 2>/dev/null || true; }
|
||||
{% endif %}
|
||||
.PHONY: {% if layout == 'root' %}env external recreate rebuild restart logs down {% endif %}fmt check build{% if database == 'postgres' %} migrate revision{% endif %}{% if has_script %} script{% endif %}{% if 'kurigram_userbot' in backend_services %} session{% endif %}
|
||||
.PHONY: {% if layout == 'root' %}env recreate rebuild restart logs down deploy {% endif %}fmt check{% if svc_modules %} build pre-deploy post-deploy{% endif %}{% if database == 'postgres' %} migrate revision{% endif %}{% if has_script %} script{% endif %}{% if 'kurigram_userbot' in backend_services %} session{% endif %}
|
||||
|
||||
fmt:
|
||||
uv run ruff format
|
||||
@@ -19,39 +30,64 @@ check:
|
||||
uv run ruff format --check
|
||||
uv run ruff check
|
||||
uv run ty check
|
||||
|
||||
{% if svc_modules %} uv run python scripts/check_imports.py
|
||||
{% endif %}{% if svc_modules %}
|
||||
build:
|
||||
$(COMPOSE) build {{ svc_modules[0] }}
|
||||
{% if layout == 'root' %}
|
||||
|
||||
pre-deploy:
|
||||
@if $(call changed,refs/deploy/backend,$(IMAGE_INPUTS)) || ! docker image inspect $(IMAGE) >/dev/null 2>&1; then \
|
||||
$(MAKE) build && $(call stamp,refs/deploy/backend) && $(call stamp,refs/deploy/backend-code); \
|
||||
fi
|
||||
{% if database == 'postgres' %} @if ($(COMPOSE) config --services) | grep -qx postgres; then ($(COMPOSE) up -d --wait postgres); fi
|
||||
$(MAKE) migrate
|
||||
{% endif %}
|
||||
post-deploy:
|
||||
@if $(call changed,refs/deploy/backend-code,$(CODE)); then \
|
||||
running=$$($(COMPOSE) ps --services --status running | grep -x $(addprefix -e ,$(SERVICES))); \
|
||||
if [ -n "$$running" ]; then ($(COMPOSE) restart $$running); fi; \
|
||||
$(call stamp,refs/deploy/backend-code); \
|
||||
fi
|
||||
{% endif %}{% if layout == 'root' %}
|
||||
env:
|
||||
@test -f .env || cp .env.example .env
|
||||
@fresh=; test -f .env || { cp .env.example .env; fresh=1; }; \
|
||||
if ! test -f docker-compose.override.yml; then \
|
||||
cp docker-compose.override.yml.example docker-compose.override.yml; \
|
||||
for port in $$(sed -n 's/.*127\.0\.0\.1:\([0-9]*\):.*/\1/p' docker-compose.override.yml | sort -u); do \
|
||||
random=$$((20000 + $$(od -An -N2 -tu2 /dev/urandom) % 40000)); \
|
||||
sed "s/127\.0\.0\.1:$$port:/127.0.0.1:$$random:/" docker-compose.override.yml > docker-compose.override.yml.tmp && mv docker-compose.override.yml.tmp docker-compose.override.yml; \
|
||||
test -z "$$fresh" || { sed "s|localhost:$$port/|localhost:$$random/|" .env > .env.tmp && mv .env.tmp .env; }; \
|
||||
done; \
|
||||
fi
|
||||
|
||||
external:
|
||||
$(EXTERNAL) up -d
|
||||
recreate:
|
||||
$(COMPOSE) up -d --force-recreate
|
||||
|
||||
recreate: external
|
||||
$(SERVICES) up -d --force-recreate
|
||||
|
||||
rebuild: build recreate
|
||||
rebuild: build
|
||||
$(COMPOSE) up -d
|
||||
|
||||
restart:
|
||||
$(SERVICES) restart
|
||||
$(COMPOSE) restart
|
||||
|
||||
logs:
|
||||
$(SERVICES) logs -f --tail=100
|
||||
$(COMPOSE) logs -f --tail=100
|
||||
|
||||
down:
|
||||
$(EXTERNAL) --profile services down
|
||||
{% endif %}{% if database == 'postgres' %}
|
||||
$(COMPOSE) down
|
||||
{% if svc_modules %}
|
||||
deploy: pre-deploy
|
||||
$(COMPOSE) up -d
|
||||
$(MAKE) post-deploy
|
||||
{% endif %}{% endif %}{% if database == 'postgres' %}
|
||||
migrate:
|
||||
$(COMPOSE) --profile external run --rm migrator
|
||||
$(COMPOSE) run --rm migrator $(or $(filter-out $@,$(MAKECMDGOALS)),upgrade head)
|
||||
|
||||
revision:
|
||||
$(COMPOSE) --profile external run --rm migrator revision --autogenerate -m "$(m)"
|
||||
$(COMPOSE) run --rm migrator revision --autogenerate -m "$(m)"
|
||||
{% endif %}{% if has_script %}
|
||||
script:
|
||||
@$(COMPOSE) run --rm {{ svc_modules[0] }} scripts.$(word 2,$(MAKECMDGOALS)) $(wordlist 3,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS))
|
||||
|
||||
{% endif %}{% if database == 'postgres' or has_script %}
|
||||
%:
|
||||
@:
|
||||
{% endif %}{% if 'kurigram_userbot' in backend_services %}
|
||||
|
||||
@@ -1,15 +0,0 @@
|
||||
{%- set has_ports = 'api' in backend_services or database != 'none' or use_redis -%}
|
||||
services:{% if not has_ports %} {}{% endif %}
|
||||
{% if 'api' in backend_services %} api:
|
||||
ports:
|
||||
- "127.0.0.1:${API__PORT:-8080}:8080"
|
||||
{% endif %}{% if database == 'mongo' %} mongodb:
|
||||
ports:
|
||||
- "127.0.0.1:${DB__PORT:-27017}:${DB__PORT:-27017}"
|
||||
{% endif %}{% if database == 'postgres' %} postgres:
|
||||
ports:
|
||||
- "127.0.0.1:${DB__PORT:-5432}:5432"
|
||||
{% endif %}{% if use_redis %} redis:
|
||||
ports:
|
||||
- "127.0.0.1:${REDIS__PORT:-6379}:6379"
|
||||
{% endif %}
|
||||
@@ -0,0 +1,30 @@
|
||||
{%- set dash = project_slug | replace('_', '-') -%}
|
||||
{%- set entries = [] -%}
|
||||
{%- if 'api' in backend_services %}{% set _ = entries.append('api') %}{% endif -%}
|
||||
{%- if database == 'mongo' %}{% set _ = entries.append('mongodb') %}{% endif -%}
|
||||
{%- if database == 'postgres' %}{% set _ = entries.append('postgres') %}{% endif -%}
|
||||
{%- if use_redis %}{% set _ = entries.append('redis') %}{% endif -%}
|
||||
services:{% if not entries %} {}{% endif %}
|
||||
{% for entry in entries %}{% if entry == 'api' %} api:
|
||||
networks:
|
||||
default: {}
|
||||
caddy:
|
||||
aliases:
|
||||
- {{ dash }}-api
|
||||
# ports:
|
||||
# - "127.0.0.1:8080:${API__PORT}"
|
||||
{% elif entry == 'mongodb' %} mongodb:
|
||||
# ports:
|
||||
# - "127.0.0.1:27017:${DB__PORT}"
|
||||
{% elif entry == 'postgres' %} postgres:
|
||||
# ports:
|
||||
# - "127.0.0.1:5432:${DB__PORT}"
|
||||
{% elif entry == 'redis' %} redis:
|
||||
# ports:
|
||||
# - "127.0.0.1:6379:${REDIS__PORT}"
|
||||
{% endif %}{% if not loop.last %}
|
||||
{% endif %}{% endfor %}{% if 'api' in backend_services %}
|
||||
networks:
|
||||
caddy:
|
||||
external: true
|
||||
{% endif %}
|
||||
@@ -1,15 +0,0 @@
|
||||
{%- set dash = project_slug | replace('_', '-') -%}
|
||||
{% if 'api' in backend_services -%}
|
||||
services:
|
||||
api:
|
||||
networks:
|
||||
caddy:
|
||||
aliases:
|
||||
- {{ dash }}-api
|
||||
|
||||
networks:
|
||||
caddy:
|
||||
external: true
|
||||
{%- else -%}
|
||||
services: {}
|
||||
{%- endif %}
|
||||
@@ -2,6 +2,10 @@
|
||||
{%- set svc_networks = [] -%}
|
||||
{%- if database != 'none' %}{% set _ = svc_networks.append('database') %}{% endif -%}
|
||||
{%- if use_redis %}{% set _ = svc_networks.append('redis') %}{% endif -%}
|
||||
{%- set external_services = [] -%}
|
||||
{%- if database == 'mongo' %}{% set _ = external_services.append('mongodb') %}{% endif -%}
|
||||
{%- if database == 'postgres' %}{% set _ = external_services.append('postgres') %}{% endif -%}
|
||||
{%- if use_redis %}{% set _ = external_services.append('redis') %}{% endif -%}
|
||||
{%- set app_services = [] -%}
|
||||
{%- if 'api' in backend_services %}{% set _ = app_services.append({'name': 'api', 'command': '[api]'}) %}{% endif -%}
|
||||
{%- if 'aiogram_bot' in backend_services %}{% set _ = app_services.append({'name': 'bot', 'command': '[bot]'}) %}{% endif -%}
|
||||
@@ -10,7 +14,7 @@
|
||||
{%- if 'scheduler' in backend_services %}{% set _ = app_services.append({'name': 'scheduler', 'command': ('[worker, scheduler]' if 'taskiq_worker' in backend_services else '[scheduler]')}) %}{% endif -%}
|
||||
{%- macro backend_service(name, command) %} {{ name }}:
|
||||
build: {{ part_dir }}
|
||||
image: {{ project_slug }}/backend
|
||||
image: {{ project_slug | replace('_', '-') }}/backend
|
||||
profiles: [{{ name }}, services]
|
||||
restart: unless-stopped
|
||||
env_file:
|
||||
@@ -20,14 +24,20 @@
|
||||
required: false
|
||||
{% endif %} environment:
|
||||
RUN_ENVIRONMENT: prod
|
||||
TZ: ${TZ:-UTC}
|
||||
volumes:
|
||||
- ./{{ prefix }}src:/app/src
|
||||
{% if has_scripts %} - ./{{ prefix }}scripts:/app/scripts
|
||||
{% endif %}{% if name == 'userbot' %} - ./{{ prefix }}sessions:/app/sessions
|
||||
{% endif %}{% if name == 'api' and serve_spa %} - ./frontend/build:/app/static:ro
|
||||
{% endif %} command: {{ command }}
|
||||
{% if svc_networks %} networks:
|
||||
{% for net in svc_networks %} {{ net }}:
|
||||
{% if external_services %} depends_on:
|
||||
{% for ext in external_services %} {{ ext }}:
|
||||
condition: service_healthy
|
||||
required: false
|
||||
{% endfor %}{% endif %}{% if svc_networks %} networks:
|
||||
{% if name == 'api' %} default:
|
||||
{% endif %}{% for net in svc_networks %} {{ net }}:
|
||||
{% endfor %}{% endif %}{% endmacro -%}
|
||||
services:
|
||||
{% for svc in app_services %}{{ backend_service(svc.name, svc.command) }}{% if not loop.last %}
|
||||
@@ -40,13 +50,18 @@ services:
|
||||
environment:
|
||||
MONGO_INITDB_ROOT_USERNAME: ${DB__USER}
|
||||
MONGO_INITDB_ROOT_PASSWORD: ${DB__PASSWORD}
|
||||
command: mongod --port ${DB__PORT:-27017}
|
||||
command: mongod --port ${DB__PORT}
|
||||
volumes:
|
||||
- database:/data/db
|
||||
healthcheck:
|
||||
test: ["CMD", "mongosh", "--quiet", "--port", "${DB__PORT}", "--eval", "db.adminCommand('ping')"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
networks:
|
||||
database:
|
||||
aliases:
|
||||
- ${DB__HOST:-mongodb}
|
||||
- ${DB__HOST}
|
||||
{% endif %}
|
||||
{%- if database == 'postgres' %}
|
||||
postgres:
|
||||
@@ -57,6 +72,7 @@ services:
|
||||
POSTGRES_USER: ${DB__USER}
|
||||
POSTGRES_PASSWORD: ${DB__PASSWORD}
|
||||
POSTGRES_DB: ${DB__DB_NAME}
|
||||
PGPORT: ${DB__PORT}
|
||||
volumes:
|
||||
- database:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
@@ -67,11 +83,11 @@ services:
|
||||
networks:
|
||||
database:
|
||||
aliases:
|
||||
- ${DB__HOST:-postgres}
|
||||
- ${DB__HOST}
|
||||
|
||||
migrator:
|
||||
build: {{ part_dir }}
|
||||
image: {{ project_slug }}/backend
|
||||
image: {{ project_slug | replace('_', '-') }}/backend
|
||||
profiles: [migrate]
|
||||
env_file:
|
||||
- path: .env
|
||||
@@ -80,6 +96,7 @@ services:
|
||||
required: false
|
||||
{% endif %} environment:
|
||||
RUN_ENVIRONMENT: prod
|
||||
TZ: ${TZ:-UTC}
|
||||
volumes:
|
||||
- ./{{ prefix }}src:/app/src
|
||||
- ./{{ prefix }}migrations:/app/migrations
|
||||
@@ -89,6 +106,7 @@ services:
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
required: false
|
||||
networks:
|
||||
database:
|
||||
{% endif %}
|
||||
@@ -97,13 +115,18 @@ services:
|
||||
image: redis:7-alpine
|
||||
profiles: [redis, external]
|
||||
restart: unless-stopped
|
||||
command: ["redis-server", "--appendonly", "yes"]
|
||||
command: ["redis-server", "--port", "${REDIS__PORT}", "--appendonly", "yes"]
|
||||
healthcheck:
|
||||
test: ["CMD", "redis-cli", "-p", "${REDIS__PORT}", "ping"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
volumes:
|
||||
- redis_data:/data
|
||||
networks:
|
||||
redis:
|
||||
aliases:
|
||||
- ${REDIS__HOST:-redis}
|
||||
- ${REDIS__HOST}
|
||||
{% endif %}
|
||||
{%- if database != 'none' or use_redis %}
|
||||
volumes:
|
||||
|
||||
@@ -1,15 +1,15 @@
|
||||
{%- set deps = ["pydantic-settings>=2.14.2", "rich>=15.0.0"] -%}
|
||||
{%- if use_dishka %}{% set _ = deps.append("dishka>=1.10.1") %}{% endif -%}
|
||||
{%- if 'api' in backend_services %}{% set _ = deps.append("fastapi>=0.138.0") %}{% set _ = deps.append("uvicorn[standard]>=0.49.0") %}{% endif -%}
|
||||
{%- if 'api' in backend_services %}{% set _ = deps.append("fastapi>=0.139.0") %}{% set _ = deps.append("uvicorn[standard]>=0.50.0") %}{% endif -%}
|
||||
{%- if 'aiogram_bot' in backend_services %}{% set _ = deps.append("aiogram>=3.29.0") %}{% endif -%}
|
||||
{%- if include_payments %}{% set _ = deps.append("aiosend>=3.0.7") %}{% endif -%}
|
||||
{%- if 'kurigram_userbot' in backend_services %}{% set _ = deps.append("anyio>=4.0.0") %}{% set _ = deps.append("kurigram>=2.2.7") %}{% set _ = deps.append("tgcrypto>=1.2.5") %}{% set _ = deps.append("uvloop>=0.21.0") %}{% endif -%}
|
||||
{%- if 'taskiq_worker' in backend_services %}{% set _ = deps.append("taskiq>=0.12.4") %}{% if taskiq_broker == 'redis' %}{% set _ = deps.append("taskiq-redis>=1.2.3") %}{% endif %}{% endif -%}
|
||||
{%- if 'kurigram_userbot' in backend_services %}{% set _ = deps.append("anyio>=4.0.0") %}{% set _ = deps.append("kurigram>=2.2.15") %}{% set _ = deps.append("tgcrypto>=1.2.5") %}{% set _ = deps.append("uvloop>=0.21.0") %}{% endif -%}
|
||||
{%- if 'taskiq_worker' in backend_services %}{% set _ = deps.append("taskiq>=0.13.0") %}{% if taskiq_broker == 'redis' %}{% set _ = deps.append("taskiq-redis>=1.2.3") %}{% endif %}{% if taskiq_broker == 'postgres' %}{% set _ = deps.append("taskiq-postgres[asyncpg]>=0.9.1") %}{% endif %}{% endif -%}
|
||||
{%- if 'scheduler' in backend_services %}{% set _ = deps.append("apscheduler>=3.11.0") %}{% endif -%}
|
||||
{%- if database == 'mongo' %}{% set _ = deps.append("beanie>=2.0.1") %}{% endif -%}
|
||||
{%- if database == 'postgres' %}{% set _ = deps.append("sqlmodel>=0.0.38") %}{% set _ = deps.append("asyncpg>=0.31.0") %}{% set _ = deps.append("alembic>=1.18.4") %}{% set _ = deps.append("greenlet>=3.1.0") %}{% endif -%}
|
||||
{%- if database == 'postgres' %}{% set _ = deps.append("sqlmodel>=0.0.39") %}{% set _ = deps.append("asyncpg>=0.31.0") %}{% set _ = deps.append("alembic>=1.18.5") %}{% set _ = deps.append("greenlet>=3.5.3") %}{% endif -%}
|
||||
{%- if use_redis %}{% set _ = deps.append("redis>=5.2.0") %}{% endif -%}
|
||||
{%- if use_llm %}{% set _ = deps.append("pydantic-ai-slim[google]>=2.1.0") %}{% endif -%}
|
||||
{%- if use_llm %}{% set _ = deps.append("pydantic-ai-slim[google]>=2.15.0") %}{% endif -%}
|
||||
{%- set modules = [] -%}
|
||||
{%- if 'api' in backend_services %}{% set _ = modules.append("api") %}{% endif -%}
|
||||
{%- if 'aiogram_bot' in backend_services %}{% set _ = modules.append("bot") %}{% endif -%}
|
||||
@@ -34,7 +34,7 @@ dependencies = [
|
||||
]
|
||||
|
||||
[build-system]
|
||||
requires = ["uv_build>=0.11.14,<0.12.0"]
|
||||
requires = ["uv_build>=0.12.10,<0.13.0"]
|
||||
build-backend = "uv_build"
|
||||
|
||||
[tool.uv.build-backend]
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@ import asyncio
|
||||
|
||||
import anyio
|
||||
|
||||
from userbot import PyroClient
|
||||
from userbot.modules.client import PyroClient
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
|
||||
+60
@@ -0,0 +1,60 @@
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
HEAVY = ["aiogram", "pyrogram", "fastapi", "google.genai"]
|
||||
SERVICES = {
|
||||
{%- if 'api' in backend_services %}
|
||||
"api": (["api.app"], {"fastapi"}),
|
||||
{%- endif %}
|
||||
{%- if 'aiogram_bot' in backend_services %}
|
||||
"bot": ({% if include_payments %}["bot"]{% else %}["bot.common", "bot.handlers"]{% endif %}, {"aiogram"}),
|
||||
{%- endif %}
|
||||
{%- if 'kurigram_userbot' in backend_services %}
|
||||
"userbot": (["userbot.runner", "userbot.handlers"], {"pyrogram"}),
|
||||
{%- endif %}
|
||||
{%- if 'taskiq_worker' in backend_services %}
|
||||
"worker": (["worker.broker", "worker.tasks"], {% if 'api' in backend_services %}{"fastapi"}{% else %}set(){% endif %}),
|
||||
{%- elif 'scheduler' in backend_services %}
|
||||
"scheduler": (["scheduler"], set()),
|
||||
{%- endif %}
|
||||
}
|
||||
PROBE = """
|
||||
import importlib, sys
|
||||
for module in sys.argv[1].split(","):
|
||||
importlib.import_module(module)
|
||||
print(" ".join(name for name in sys.argv[2:] if name in sys.modules))
|
||||
"""
|
||||
{%- if 'aiogram_bot' in backend_services %}
|
||||
ENV = {**os.environ, "BOT__TOKEN": "1:check"}
|
||||
{%- else %}
|
||||
ENV = dict(os.environ)
|
||||
{%- endif %}
|
||||
|
||||
|
||||
def loaded(modules: list[str]) -> tuple[set[str], str]:
|
||||
result = subprocess.run( # noqa: S603
|
||||
[sys.executable, "-c", PROBE, ",".join(modules), *HEAVY],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
cwd="src",
|
||||
env=ENV,
|
||||
check=False,
|
||||
)
|
||||
error = (result.stderr.strip().splitlines() or ["exit code"])[-1]
|
||||
return set(result.stdout.split()) & set(HEAVY), error if result.returncode else ""
|
||||
|
||||
|
||||
def main() -> int:
|
||||
failed = False
|
||||
for service, (modules, allowed) in SERVICES.items():
|
||||
found, error = loaded(modules)
|
||||
extra = sorted(found - allowed)
|
||||
status = error or ("imports " + ", ".join(extra) if extra else "")
|
||||
print(f"{service:10s} {'FAIL: ' + status if status else 'ok'}")
|
||||
failed |= bool(status)
|
||||
return int(failed)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
+3
-4
@@ -261,10 +261,9 @@ async def read_config() -> DynamicConfigBase:
|
||||
|
||||
async def write_config(config: DynamicConfigBase) -> DynamicConfigBase:
|
||||
await init_db()
|
||||
doc = await DynamicConfig.get_or_create()
|
||||
for name in DynamicConfigBase.model_fields:
|
||||
setattr(doc, name, getattr(config, name))
|
||||
await doc.save()
|
||||
await DynamicConfig.update_config(
|
||||
**{name: getattr(config, name) for name in DynamicConfigBase.model_fields}
|
||||
)
|
||||
return config
|
||||
|
||||
|
||||
|
||||
@@ -1,3 +1,9 @@
|
||||
{%- macro str_field(name, value) -%}
|
||||
{%- set line = " " ~ name ~ ': str = "' ~ value ~ '"' -%}
|
||||
{%- if line | length <= 88 or value | length + 10 > 88 %}{{ line }}{% else %} {{ name }}: str = (
|
||||
"{{ value }}"
|
||||
){% endif -%}
|
||||
{%- endmacro -%}
|
||||
{% if database != 'none' or use_redis %}import os
|
||||
|
||||
{% endif %}from pydantic import Field{% if database != 'none' or 'aiogram_bot' in backend_services or include_payments or use_llm or use_redis %}, SecretStr{% endif %}
|
||||
@@ -8,30 +14,36 @@ def is_prod() -> bool:
|
||||
return os.getenv("RUN_ENVIRONMENT") == "prod"
|
||||
{% endif %}
|
||||
|
||||
class Section(BaseSettings):
|
||||
model_config = SettingsConfigDict(extra="ignore")
|
||||
|
||||
class LogSettings(BaseSettings):
|
||||
|
||||
class LogSettings(Section):
|
||||
level: str = "INFO"
|
||||
level_external: str = "WARNING"
|
||||
show_time: bool = False
|
||||
show_locals: bool = False
|
||||
console_width: int = 150
|
||||
{% if database == 'mongo' %}
|
||||
|
||||
class DatabaseSettings(BaseSettings):
|
||||
class DatabaseSettings(Section):
|
||||
host: str = "mongodb"
|
||||
port: int = 27017
|
||||
user: str = "{{ project_slug }}"
|
||||
password: SecretStr = SecretStr("{{ project_slug }}")
|
||||
db_name: str = "{{ project_slug }}"
|
||||
connection_params: str = "?authSource=admin"
|
||||
{{ str_field("scripts_connection_url", "mongodb://" ~ project_slug ~ ":" ~ project_slug ~ "@localhost:27017/" ~ project_slug ~ "?authSource=admin") }}
|
||||
|
||||
@property
|
||||
def connection_url(self) -> str:
|
||||
host = self.host if is_prod() else "localhost"
|
||||
if not is_prod():
|
||||
return self.scripts_connection_url
|
||||
password = self.password.get_secret_value()
|
||||
return f"mongodb://{self.user}:{password}@{host}:{self.port}/{self.db_name}{self.connection_params}"
|
||||
return f"mongodb://{self.user}:{password}@{self.host}:{self.port}/{self.db_name}{self.connection_params}"
|
||||
{% elif database == 'postgres' %}
|
||||
|
||||
class DatabaseSettings(BaseSettings):
|
||||
class DatabaseSettings(Section):
|
||||
host: str = "postgres"
|
||||
port: int = 5432
|
||||
user: str = "{{ project_slug }}"
|
||||
@@ -39,51 +51,56 @@ class DatabaseSettings(BaseSettings):
|
||||
db_name: str = "{{ project_slug }}"
|
||||
min_pool_size: int = 5
|
||||
max_pool_size: int = 20
|
||||
{{ str_field("scripts_connection_url", "postgresql://" ~ project_slug ~ ":" ~ project_slug ~ "@localhost:5432/" ~ project_slug) }}
|
||||
|
||||
@property
|
||||
def connection_url(self) -> str:
|
||||
host = self.host if is_prod() else "localhost"
|
||||
if not is_prod():
|
||||
return self.scripts_connection_url
|
||||
password = self.password.get_secret_value()
|
||||
return f"postgresql://{self.user}:{password}@{host}:{self.port}/{self.db_name}"
|
||||
return f"postgresql://{self.user}:{password}@{self.host}:{self.port}/{self.db_name}"
|
||||
|
||||
@property
|
||||
def async_connection_url(self) -> str:
|
||||
return self.connection_url.replace("postgresql://", "postgresql+asyncpg://", 1)
|
||||
{% endif %}{% if use_redis %}
|
||||
|
||||
class RedisSettings(BaseSettings):
|
||||
class RedisSettings(Section):
|
||||
host: str = "redis"
|
||||
port: int = 6379
|
||||
db: int = 0
|
||||
password: SecretStr | None = None
|
||||
cache_ttl: int = 300
|
||||
scripts_connection_url: str = "redis://localhost:6379/0"
|
||||
|
||||
@property
|
||||
def url(self) -> str:
|
||||
host = self.host if is_prod() else "localhost"
|
||||
if not is_prod():
|
||||
return self.scripts_connection_url
|
||||
auth = f":{self.password.get_secret_value()}@" if self.password else ""
|
||||
return f"redis://{auth}{host}:{self.port}/{self.db}"
|
||||
return f"redis://{auth}{self.host}:{self.port}/{self.db}"
|
||||
{% endif %}{% if 'api' in backend_services %}
|
||||
|
||||
class ApiSettings(BaseSettings):
|
||||
class ApiSettings(Section):
|
||||
host: str = "0.0.0.0" # noqa: S104
|
||||
port: int = 8080
|
||||
workers: int = 1
|
||||
docs: bool = False
|
||||
{%- if serve_spa %}
|
||||
static_dir: str = "static"
|
||||
{%- endif %}
|
||||
{% endif %}{% if 'aiogram_bot' in backend_services %}
|
||||
|
||||
class BotSettings(BaseSettings):
|
||||
class BotSettings(Section):
|
||||
token: SecretStr = SecretStr("")
|
||||
admins: list[int] = Field(default_factory=list)
|
||||
{% endif %}{% if include_payments %}
|
||||
|
||||
class CryptoPaySettings(BaseSettings):
|
||||
class CryptoPaySettings(Section):
|
||||
token: SecretStr = SecretStr("")
|
||||
{% endif %}{% if use_llm %}
|
||||
|
||||
class LlmSettings(BaseSettings):
|
||||
class LlmSettings(Section):
|
||||
model: str = "google-gla:gemini-2.5-flash"
|
||||
gemini_api_key: SecretStr = SecretStr("")
|
||||
{% endif %}
|
||||
@@ -110,7 +127,14 @@ class Settings(BaseSettings):
|
||||
{%- endif %}
|
||||
|
||||
model_config = SettingsConfigDict(
|
||||
{%- if layout == 'part' %}
|
||||
case_sensitive=False,
|
||||
env_file=("../.env", ".env"),
|
||||
env_nested_delimiter="__",
|
||||
extra="ignore",
|
||||
{%- else %}
|
||||
case_sensitive=False, env_file=".env", env_nested_delimiter="__", extra="ignore"
|
||||
{%- endif %}
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
import asyncio
|
||||
from importlib import import_module
|
||||
|
||||
|
||||
async def preload(module: str) -> None:
|
||||
await asyncio.to_thread(import_module, module)
|
||||
@@ -1,5 +1,7 @@
|
||||
import logging
|
||||
|
||||
{% if 'aiogram_bot' in backend_services %}from importlib.util import find_spec
|
||||
from pathlib import Path
|
||||
{% endif %}
|
||||
from rich.console import Console
|
||||
from rich.logging import RichHandler
|
||||
from rich.traceback import install
|
||||
@@ -10,8 +12,12 @@ console = Console(width=env.log.console_width, color_system="auto", force_termin
|
||||
|
||||
|
||||
def setup_logging() -> None:
|
||||
{% if 'aiogram_bot' in backend_services %} from aiogram.dispatcher import router # noqa: PLC0415
|
||||
|
||||
{% if 'aiogram_bot' in backend_services %} aiogram = find_spec("aiogram")
|
||||
suppress = (
|
||||
[str(Path(aiogram.origin).parent / "dispatcher")]
|
||||
if aiogram and aiogram.origin
|
||||
else []
|
||||
)
|
||||
{% endif %} logging.basicConfig(
|
||||
level=env.log.level_external,
|
||||
format="",
|
||||
@@ -22,16 +28,16 @@ def setup_logging() -> None:
|
||||
markup=True,
|
||||
rich_tracebacks=True,
|
||||
enable_link_path=False,
|
||||
tracebacks_show_locals=True,
|
||||
tracebacks_show_locals=env.log.show_locals,
|
||||
omit_repeated_times=False,
|
||||
show_time=env.log.show_time,
|
||||
{%- if 'aiogram_bot' in backend_services %}
|
||||
tracebacks_suppress=[router],
|
||||
tracebacks_suppress=suppress,
|
||||
{%- endif %}
|
||||
)
|
||||
],
|
||||
)
|
||||
install(console=console, show_locals=True)
|
||||
install(console=console, show_locals=env.log.show_locals)
|
||||
|
||||
|
||||
logger = logging.getLogger("{{ project_slug }}")
|
||||
|
||||
+71
-5
@@ -22,6 +22,7 @@ class DynamicConfig(SQLModel, table=True):
|
||||
|
||||
id: int = SQLField(default=1, primary_key=True)
|
||||
data: dict[str, Any] = SQLField(sa_column=Column(postgresql.JSONB, nullable=False))
|
||||
version: int = SQLField(default=0)
|
||||
updated_at: datetime | None = SQLField(
|
||||
default=None,
|
||||
sa_column=Column(
|
||||
@@ -32,8 +33,19 @@ class DynamicConfig(SQLModel, table=True):
|
||||
),
|
||||
)
|
||||
{%- else -%}
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
|
||||
from beanie import Document
|
||||
from beanie.odm.operators.update import BaseUpdateOperator
|
||||
from beanie.operators import Inc, Set
|
||||
from pydantic import BaseModel, Field
|
||||
from pymongo.errors import PyMongoError
|
||||
|
||||
from utils.logging import logger
|
||||
|
||||
CONFIG_CHECK_SECONDS = 5.0
|
||||
CONFIG_TTL_SECONDS = 900.0
|
||||
|
||||
|
||||
class BotConfig(BaseModel):
|
||||
@@ -44,15 +56,69 @@ class DynamicConfigBase(BaseModel):
|
||||
bot: BotConfig = Field(default_factory=BotConfig)
|
||||
|
||||
|
||||
@dataclass
|
||||
class _ConfigCache:
|
||||
value: "DynamicConfig | None" = None
|
||||
loaded_at: float = 0.0
|
||||
checked_at: float = 0.0
|
||||
|
||||
|
||||
_cache = _ConfigCache()
|
||||
|
||||
|
||||
class DynamicConfig(DynamicConfigBase, Document):
|
||||
version: int = 0
|
||||
|
||||
class Settings:
|
||||
name = "config"
|
||||
|
||||
@classmethod
|
||||
async def get_or_create(cls) -> "DynamicConfig":
|
||||
config = await cls.find_one()
|
||||
if config is None:
|
||||
config = cls()
|
||||
await config.save()
|
||||
return config
|
||||
now = time.monotonic()
|
||||
current = _cache.value
|
||||
if current is None:
|
||||
current = await cls._reload()
|
||||
elif now - _cache.loaded_at >= CONFIG_TTL_SECONDS:
|
||||
current = await cls._reload_or_keep(current)
|
||||
elif now - _cache.checked_at >= CONFIG_CHECK_SECONDS:
|
||||
_cache.checked_at = now
|
||||
current = await cls._refresh_if_changed(current)
|
||||
return current.model_copy(deep=True)
|
||||
|
||||
@classmethod
|
||||
async def update_config(
|
||||
cls, *operations: BaseUpdateOperator, **sections: BaseModel
|
||||
) -> "DynamicConfig":
|
||||
current = await cls.get_or_create()
|
||||
changes = [*operations, Set(sections)] if sections else [*operations]
|
||||
await cls.find_one(cls.id == current.id).update(*changes, Inc({cls.version: 1}))
|
||||
return (await cls._reload()).model_copy(deep=True)
|
||||
|
||||
@classmethod
|
||||
async def _reload(cls) -> "DynamicConfig":
|
||||
loaded = await cls.find_one() or await cls().insert()
|
||||
_cache.value = loaded
|
||||
_cache.loaded_at = _cache.checked_at = time.monotonic()
|
||||
return loaded
|
||||
|
||||
@classmethod
|
||||
async def _reload_or_keep(cls, current: "DynamicConfig") -> "DynamicConfig":
|
||||
try:
|
||||
return await cls._reload()
|
||||
except PyMongoError as e:
|
||||
logger.warning(f"Config reload failed, serving the cached copy: {e}")
|
||||
return current
|
||||
|
||||
@classmethod
|
||||
async def _refresh_if_changed(cls, current: "DynamicConfig") -> "DynamicConfig":
|
||||
try:
|
||||
stored = await cls.get_pymongo_collection().find_one(
|
||||
{"_id": current.id}, {"version": 1}
|
||||
)
|
||||
except PyMongoError as e:
|
||||
logger.warning(f"Config version check failed, serving the cached copy: {e}")
|
||||
return current
|
||||
if stored and stored.get("version", 0) == current.version:
|
||||
return current
|
||||
return await cls._reload_or_keep(current)
|
||||
{%- endif %}
|
||||
+123
-72
@@ -1,88 +1,139 @@
|
||||
{% if use_redis -%}
|
||||
import time
|
||||
{%- if use_redis %}
|
||||
from contextlib import suppress
|
||||
{%- endif %}
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel
|
||||
{%- if use_redis %}
|
||||
from redis.exceptions import RedisError
|
||||
{%- endif %}
|
||||
from sqlalchemy import literal, update
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.exc import SQLAlchemyError
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||
from sqlmodel import col, select
|
||||
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||
|
||||
from utils.db.models.config import DynamicConfig, DynamicConfigBase
|
||||
from utils.logging import logger
|
||||
{%- if use_redis %}
|
||||
|
||||
from .cache import RedisCache
|
||||
{%- endif %}
|
||||
|
||||
CONFIG_CACHE_KEY = "config:dynamic"
|
||||
CONFIG_ROW_ID = 1
|
||||
CONFIG_CHECK_SECONDS = 5.0
|
||||
CONFIG_TTL_SECONDS = 900.0
|
||||
{%- if use_redis %}
|
||||
CONFIG_CACHE_KEY = "config:dynamic"
|
||||
{%- endif %}
|
||||
|
||||
|
||||
class ConfigRepository:
|
||||
def __init__(
|
||||
self, sessionmaker: async_sessionmaker[AsyncSession], cache: RedisCache
|
||||
self, sessionmaker: async_sessionmaker[AsyncSession]{% if use_redis %}, cache: RedisCache{% endif %}
|
||||
) -> None:
|
||||
self._sessionmaker = sessionmaker
|
||||
{%- if use_redis %}
|
||||
self._cache = cache
|
||||
|
||||
async def get(self) -> DynamicConfigBase:
|
||||
return await self._cache.get_or_set_model(
|
||||
CONFIG_CACHE_KEY, DynamicConfigBase, self._load
|
||||
)
|
||||
|
||||
async def _load(self) -> DynamicConfigBase:
|
||||
async with self._sessionmaker() as session:
|
||||
row = await session.get(DynamicConfig, CONFIG_ROW_ID)
|
||||
if row is None:
|
||||
config = DynamicConfigBase()
|
||||
session.add(DynamicConfig(id=CONFIG_ROW_ID, data=config.model_dump()))
|
||||
await session.commit()
|
||||
return config
|
||||
return DynamicConfigBase.model_validate(row.data)
|
||||
|
||||
async def save(self, config: DynamicConfigBase) -> DynamicConfigBase:
|
||||
async with self._sessionmaker() as session:
|
||||
row = await session.get(DynamicConfig, CONFIG_ROW_ID)
|
||||
if row is None:
|
||||
session.add(DynamicConfig(id=CONFIG_ROW_ID, data=config.model_dump()))
|
||||
else:
|
||||
row.data = config.model_dump()
|
||||
session.add(row)
|
||||
await session.commit()
|
||||
await self._cache.set_model(CONFIG_CACHE_KEY, config)
|
||||
return config
|
||||
|
||||
async def reset(self) -> DynamicConfigBase:
|
||||
return await self.save(DynamicConfigBase())
|
||||
|
||||
async def invalidate(self) -> None:
|
||||
await self._cache.invalidate(CONFIG_CACHE_KEY)
|
||||
{%- else -%}
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||
|
||||
from utils.db.models.config import DynamicConfig, DynamicConfigBase
|
||||
|
||||
CONFIG_ROW_ID = 1
|
||||
|
||||
|
||||
class ConfigRepository:
|
||||
def __init__(self, sessionmaker: async_sessionmaker[AsyncSession]) -> None:
|
||||
self._sessionmaker = sessionmaker
|
||||
|
||||
async def get(self) -> DynamicConfigBase:
|
||||
async with self._sessionmaker() as session:
|
||||
row = await session.get(DynamicConfig, CONFIG_ROW_ID)
|
||||
if row is None:
|
||||
config = DynamicConfigBase()
|
||||
session.add(DynamicConfig(id=CONFIG_ROW_ID, data=config.model_dump()))
|
||||
await session.commit()
|
||||
return config
|
||||
return DynamicConfigBase.model_validate(row.data)
|
||||
|
||||
async def save(self, config: DynamicConfigBase) -> DynamicConfigBase:
|
||||
async with self._sessionmaker() as session:
|
||||
row = await session.get(DynamicConfig, CONFIG_ROW_ID)
|
||||
if row is None:
|
||||
session.add(DynamicConfig(id=CONFIG_ROW_ID, data=config.model_dump()))
|
||||
else:
|
||||
row.data = config.model_dump()
|
||||
session.add(row)
|
||||
await session.commit()
|
||||
return config
|
||||
|
||||
async def reset(self) -> DynamicConfigBase:
|
||||
return await self.save(DynamicConfigBase())
|
||||
{%- endif %}
|
||||
self._value: DynamicConfigBase | None = None
|
||||
self._version = -1
|
||||
self._loaded_at = 0.0
|
||||
self._checked_at = 0.0
|
||||
|
||||
async def get(self) -> DynamicConfigBase:
|
||||
now = time.monotonic()
|
||||
current = self._value
|
||||
if current is None:
|
||||
current = await self._reload()
|
||||
elif now - self._loaded_at >= CONFIG_TTL_SECONDS:
|
||||
current = await self._reload_or_keep(current)
|
||||
elif now - self._checked_at >= CONFIG_CHECK_SECONDS:
|
||||
self._checked_at = now
|
||||
current = await self._refresh_if_changed(current)
|
||||
return current.model_copy(deep=True)
|
||||
|
||||
async def save(self, config: DynamicConfigBase) -> DynamicConfigBase:
|
||||
return await self._write(config.model_dump(mode="json"), merge=False)
|
||||
|
||||
async def update(self, **sections: BaseModel) -> DynamicConfigBase:
|
||||
patch = {name: value.model_dump(mode="json") for name, value in sections.items()}
|
||||
return await self._write(patch, merge=True)
|
||||
|
||||
async def reset(self) -> DynamicConfigBase:
|
||||
return await self.save(DynamicConfigBase())
|
||||
|
||||
async def _write(self, data: dict[str, Any], *, merge: bool) -> DynamicConfigBase:
|
||||
await self.get()
|
||||
payload = literal(data, JSONB)
|
||||
async with self._sessionmaker() as session:
|
||||
await session.exec(
|
||||
update(DynamicConfig)
|
||||
.where(col(DynamicConfig.id) == CONFIG_ROW_ID)
|
||||
.values(
|
||||
data=col(DynamicConfig.data).op("||")(payload) if merge else payload,
|
||||
version=col(DynamicConfig.version) + 1,
|
||||
)
|
||||
)
|
||||
await session.commit()
|
||||
return (await self._reload()).model_copy(deep=True)
|
||||
|
||||
async def _stored_version(self) -> int | None:
|
||||
async with self._sessionmaker() as session:
|
||||
query = select(DynamicConfig.version).where(
|
||||
col(DynamicConfig.id) == CONFIG_ROW_ID
|
||||
)
|
||||
return (await session.exec(query)).first()
|
||||
|
||||
async def _reload(self) -> DynamicConfigBase:
|
||||
{%- if use_redis %}
|
||||
version = await self._stored_version()
|
||||
if version is not None:
|
||||
with suppress(RedisError):
|
||||
key = f"{CONFIG_CACHE_KEY}:{version}"
|
||||
cached = await self._cache.get_model(key, DynamicConfigBase)
|
||||
if cached is not None:
|
||||
return self._remember(cached, version)
|
||||
{%- endif %}
|
||||
async with self._sessionmaker() as session:
|
||||
row = await session.get(DynamicConfig, CONFIG_ROW_ID)
|
||||
if row is None:
|
||||
row = DynamicConfig(
|
||||
id=CONFIG_ROW_ID, data=DynamicConfigBase().model_dump(mode="json")
|
||||
)
|
||||
session.add(row)
|
||||
await session.commit()
|
||||
await session.refresh(row)
|
||||
value = DynamicConfigBase.model_validate(row.data)
|
||||
version = row.version
|
||||
{%- if use_redis %}
|
||||
with suppress(RedisError):
|
||||
await self._cache.set_model(f"{CONFIG_CACHE_KEY}:{version}", value)
|
||||
{%- endif %}
|
||||
return self._remember(value, version)
|
||||
|
||||
def _remember(self, value: DynamicConfigBase, version: int) -> DynamicConfigBase:
|
||||
self._value, self._version = value, version
|
||||
self._loaded_at = self._checked_at = time.monotonic()
|
||||
return value
|
||||
|
||||
async def _reload_or_keep(self, current: DynamicConfigBase) -> DynamicConfigBase:
|
||||
try:
|
||||
return await self._reload()
|
||||
except SQLAlchemyError as e:
|
||||
logger.warning(f"Config reload failed, serving the cached copy: {e}")
|
||||
return current
|
||||
|
||||
async def _refresh_if_changed(
|
||||
self, current: DynamicConfigBase
|
||||
) -> DynamicConfigBase:
|
||||
try:
|
||||
version = await self._stored_version()
|
||||
except SQLAlchemyError as e:
|
||||
logger.warning(f"Config version check failed, serving the cached copy: {e}")
|
||||
return current
|
||||
if version == self._version:
|
||||
return current
|
||||
return await self._reload_or_keep(current)
|
||||
+5
-1
@@ -5,7 +5,11 @@ from utils.env import env
|
||||
|
||||
def main() -> None:
|
||||
uvicorn.run(
|
||||
"api.app:app", host=env.api.host, port=env.api.port, workers=env.api.workers
|
||||
"api.app:app",
|
||||
host=env.api.host,
|
||||
port=env.api.port,
|
||||
workers=env.api.workers,
|
||||
forwarded_allow_ips="*",
|
||||
)
|
||||
|
||||
|
||||
|
||||
+15
-7
@@ -10,8 +10,8 @@ from fastapi.middleware.cors import CORSMiddleware
|
||||
from api import routers
|
||||
{% if use_dishka %}from dependencies.container import container
|
||||
{% endif %}{% if database != 'none' %}from utils.db import init_db
|
||||
{% endif %}{% if serve_spa %}from utils.env import env
|
||||
{% endif %}from utils.logging import setup_logging
|
||||
{% endif %}from utils.env import env
|
||||
from utils.logging import setup_logging
|
||||
{% if serve_spa %}
|
||||
IMMUTABLE_HEADERS = {"Cache-Control": "public, max-age=31536000, immutable"}
|
||||
NO_CACHE_HEADERS = {"Cache-Control": "no-cache"}
|
||||
@@ -29,14 +29,19 @@ async def lifespan(app_: FastAPI) -> AsyncGenerator[None]:
|
||||
{%- endif %}
|
||||
|
||||
|
||||
app = FastAPI(title="{{ project_name }} API", lifespan=lifespan)
|
||||
app = FastAPI(
|
||||
title="{{ project_name }} API",
|
||||
lifespan=lifespan,
|
||||
openapi_url="/openapi.json" if env.api.docs else None,
|
||||
)
|
||||
|
||||
app.add_middleware(
|
||||
CORSMiddleware, allow_origins=["*"], allow_methods=["*"], allow_headers=["*"]
|
||||
)
|
||||
|
||||
app.include_router(routers.router)
|
||||
{% if serve_spa %}
|
||||
{%- if serve_spa %}
|
||||
|
||||
spa_dir = Path(env.api.static_dir).resolve()
|
||||
if spa_dir.is_dir():
|
||||
|
||||
@@ -48,6 +53,9 @@ if spa_dir.is_dir():
|
||||
headers = IMMUTABLE_HEADERS if immutable else NO_CACHE_HEADERS
|
||||
return FileResponse(candidate, headers=headers)
|
||||
return FileResponse(spa_dir / "index.html", headers=NO_CACHE_HEADERS)
|
||||
{% endif %}
|
||||
{% if use_dishka %}setup_dishka(container, app)
|
||||
{% endif %}
|
||||
{%- endif %}
|
||||
{%- if use_dishka %}
|
||||
|
||||
{% if serve_spa %}
|
||||
{% endif %}setup_dishka(container, app)
|
||||
{%- endif %}
|
||||
-3
@@ -1,3 +0,0 @@
|
||||
from .modules.client import PyroClient
|
||||
|
||||
__all__ = ["PyroClient"]
|
||||
+4
-5
@@ -1,12 +1,11 @@
|
||||
from pyrogram import filters, types
|
||||
from pyrogram import Client, filters, types
|
||||
from pyrogram.handlers import MessageHandler
|
||||
|
||||
from userbot import PyroClient
|
||||
from utils.logging import logger
|
||||
|
||||
|
||||
@PyroClient.on_message(filters.me)
|
||||
async def echo(_: PyroClient, message: types.Message) -> None:
|
||||
async def echo(_: Client, message: types.Message) -> None:
|
||||
logger.info(f"[cyan]me:[/] {message.text}")
|
||||
|
||||
|
||||
handlers = echo.handlers
|
||||
handlers = [(MessageHandler(echo, filters.me), 0)]
|
||||
+4
-5
@@ -1,13 +1,12 @@
|
||||
from pyrogram import filters, types
|
||||
from pyrogram import Client, filters, types
|
||||
from pyrogram.handlers import MessageHandler
|
||||
|
||||
from userbot import PyroClient
|
||||
from utils.logging import logger
|
||||
|
||||
|
||||
@PyroClient.on_message(filters.private & ~filters.me)
|
||||
async def pm_message(_: PyroClient, message: types.Message) -> None:
|
||||
async def pm_message(_: Client, message: types.Message) -> None:
|
||||
sender = message.from_user.id if message.from_user else "unknown"
|
||||
logger.info(f"[green]pm from {sender}:[/] {message.text}")
|
||||
|
||||
|
||||
handlers = pm_message.handlers
|
||||
handlers = [(MessageHandler(pm_message, filters.private & ~filters.me), 0)]
|
||||
+14
-4
@@ -1,20 +1,30 @@
|
||||
from pyrogram import Client, enums
|
||||
|
||||
DEVICE_MODEL = "{{ project_name[:32] }}"
|
||||
|
||||
|
||||
class PyroClient(Client):
|
||||
def __init__(
|
||||
self, name: str, *, workdir: str = "sessions", load_handlers: bool = True
|
||||
self,
|
||||
name: str,
|
||||
*,
|
||||
workdir: str = "sessions",
|
||||
device_model: str | None = None,
|
||||
load_handlers: bool = True,
|
||||
) -> None:
|
||||
super().__init__(
|
||||
name,
|
||||
workdir=workdir,
|
||||
api_id=2040,
|
||||
api_hash="b18441a1ff607e10a989891a5462e627",
|
||||
device_model="Desktop",
|
||||
device_model=device_model or DEVICE_MODEL,
|
||||
system_version="Windows 11 x64",
|
||||
app_version="6.2.4 x64",
|
||||
app_version="7.0.8 x64",
|
||||
lang_pack="tdesktop",
|
||||
lang_code="en",
|
||||
system_lang_code="en-US",
|
||||
client_platform=enums.ClientPlatform.DESKTOP,
|
||||
max_concurrent_transmissions=4,
|
||||
)
|
||||
|
||||
if load_handlers:
|
||||
@@ -24,4 +34,4 @@ class PyroClient(Client):
|
||||
self.add_handler(*handler)
|
||||
|
||||
|
||||
__all__ = ["PyroClient"]
|
||||
__all__ = ["DEVICE_MODEL", "PyroClient"]
|
||||
+12
-6
@@ -1,5 +1,6 @@
|
||||
import asyncio
|
||||
import contextlib
|
||||
import signal
|
||||
|
||||
import anyio
|
||||
import uvloop
|
||||
@@ -11,8 +12,15 @@ import uvloop
|
||||
|
||||
setup_logging()
|
||||
|
||||
SCAN_INTERVAL = 10
|
||||
|
||||
|
||||
async def runner() -> None:
|
||||
stop = asyncio.Event()
|
||||
loop = asyncio.get_running_loop()
|
||||
for sig in (signal.SIGINT, signal.SIGTERM):
|
||||
loop.add_signal_handler(sig, stop.set)
|
||||
|
||||
{% if database != 'none' %} await init_db()
|
||||
|
||||
{% endif %} sessions_dir = anyio.Path("sessions")
|
||||
@@ -22,7 +30,7 @@ async def runner() -> None:
|
||||
clients: dict[int, PyroClient] = {}
|
||||
|
||||
try:
|
||||
while True:
|
||||
while not stop.is_set():
|
||||
current = {path.name async for path in sessions_dir.glob("*.session")}
|
||||
|
||||
for session_file in current - started:
|
||||
@@ -43,7 +51,8 @@ async def runner() -> None:
|
||||
if me:
|
||||
clients[me.id] = client
|
||||
|
||||
await asyncio.sleep(10)
|
||||
with contextlib.suppress(TimeoutError):
|
||||
await asyncio.wait_for(stop.wait(), SCAN_INTERVAL)
|
||||
finally:
|
||||
for client in clients.values():
|
||||
with contextlib.suppress(Exception):
|
||||
@@ -55,8 +64,5 @@ def main() -> None:
|
||||
uvloop.install()
|
||||
|
||||
logger.info("Starting...")
|
||||
|
||||
with contextlib.suppress(KeyboardInterrupt):
|
||||
asyncio.run(runner())
|
||||
|
||||
asyncio.run(runner())
|
||||
logger.info("[red]Stopped.[/]")
|
||||
+8
-5
@@ -1,5 +1,5 @@
|
||||
import asyncio
|
||||
import contextlib
|
||||
import signal
|
||||
|
||||
from apscheduler.schedulers.asyncio import AsyncIOScheduler
|
||||
from apscheduler.triggers.interval import IntervalTrigger
|
||||
@@ -15,6 +15,10 @@ async def example_job() -> None:
|
||||
|
||||
|
||||
async def runner() -> None:
|
||||
stop = asyncio.Event()
|
||||
loop = asyncio.get_running_loop()
|
||||
for sig in (signal.SIGINT, signal.SIGTERM):
|
||||
loop.add_signal_handler(sig, stop.set)
|
||||
{%- if database != 'none' %}
|
||||
await init_db()
|
||||
{%- endif %}
|
||||
@@ -27,14 +31,13 @@ async def runner() -> None:
|
||||
scheduler.start()
|
||||
logger.info("Scheduler started")
|
||||
|
||||
with contextlib.suppress(asyncio.CancelledError):
|
||||
await asyncio.Event().wait()
|
||||
await stop.wait()
|
||||
scheduler.shutdown()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
setup_logging()
|
||||
|
||||
logger.info("Starting...")
|
||||
with contextlib.suppress(KeyboardInterrupt):
|
||||
asyncio.run(runner())
|
||||
asyncio.run(runner())
|
||||
logger.info("[red]Stopped.[/]")
|
||||
+16
@@ -39,6 +39,22 @@ broker = ResilientListQueueBroker(env.redis.url).with_result_backend(
|
||||
{% endif %}{% if use_dishka %}
|
||||
setup_dishka(container, broker)
|
||||
{% endif %}
|
||||
{%- elif taskiq_broker == 'postgres' -%}
|
||||
{% if use_dishka %}from dishka.integrations.taskiq import setup_dishka
|
||||
{% endif %}{% if 'scheduler' in backend_services %}from taskiq import TaskiqScheduler
|
||||
from taskiq.schedule_sources import LabelScheduleSource
|
||||
{% endif %}from taskiq_pg.asyncpg import AsyncpgBroker, AsyncpgResultBackend
|
||||
|
||||
{% if use_dishka %}from dependencies.container import container
|
||||
{% endif %}from utils.env import env
|
||||
|
||||
broker = AsyncpgBroker(env.db.connection_url).with_result_backend(
|
||||
AsyncpgResultBackend(env.db.connection_url)
|
||||
)
|
||||
{% if 'scheduler' in backend_services %}scheduler = TaskiqScheduler(broker, sources=[LabelScheduleSource(broker)])
|
||||
{% endif %}{% if use_dishka %}
|
||||
setup_dishka(container, broker)
|
||||
{% endif %}
|
||||
{%- else -%}
|
||||
{% if use_dishka %}from dishka.integrations.taskiq import setup_dishka
|
||||
{% endif %}from taskiq import InMemoryBroker{% if 'scheduler' in backend_services %}, TaskiqScheduler
|
||||
|
||||
-4
@@ -4,11 +4,7 @@
|
||||
{%- if database == 'postgres' %}{% set _ = providers.append('RepositoryProvider()') %}{% endif -%}
|
||||
{%- if use_redis %}{% set _ = providers.append('RedisProvider()') %}{% endif -%}
|
||||
{%- if use_llm %}{% set _ = providers.append('LlmProvider()') %}{% endif -%}
|
||||
{%- if 'aiogram_bot' in backend_services %}{% set _ = providers.append('AiogramProvider()') %}{% endif -%}
|
||||
{%- if 'api' in backend_services %}{% set _ = providers.append('FastapiProvider()') %}{% endif -%}
|
||||
{%- set third = ['from dishka import make_async_container'] -%}
|
||||
{%- if 'aiogram_bot' in backend_services %}{% set _ = third.append('from dishka.integrations.aiogram import AiogramProvider') %}{% endif -%}
|
||||
{%- if 'api' in backend_services %}{% set _ = third.append('from dishka.integrations.fastapi import FastapiProvider') %}{% endif -%}
|
||||
{%- set first = [] -%}
|
||||
{%- if dynamic_config %}{% set _ = first.append('from dependencies.providers.config import ConfigProvider') %}{% endif -%}
|
||||
{%- if database != 'none' %}{% set _ = first.append('from dependencies.providers.db import DbProvider') %}{% endif -%}
|
||||
|
||||
+9
-3
@@ -1,14 +1,20 @@
|
||||
from dishka import Provider, Scope, provide
|
||||
from pydantic_ai import Agent
|
||||
from pydantic_ai.models.google import GoogleModel, GoogleModelSettings
|
||||
from pydantic_ai.providers.google import GoogleProvider
|
||||
|
||||
from utils.env import env
|
||||
from utils.imports import preload
|
||||
|
||||
|
||||
class LlmProvider(Provider):
|
||||
@provide(scope=Scope.APP)
|
||||
def agent(self) -> Agent:
|
||||
async def agent(self) -> Agent:
|
||||
await preload("pydantic_ai.models.google")
|
||||
from pydantic_ai.models.google import ( # noqa: PLC0415
|
||||
GoogleModel,
|
||||
GoogleModelSettings,
|
||||
)
|
||||
from pydantic_ai.providers.google import GoogleProvider # noqa: PLC0415
|
||||
|
||||
model = GoogleModel(
|
||||
env.llm.model.split(":", 1)[-1],
|
||||
provider=GoogleProvider(api_key=env.llm.gemini_api_key.get_secret_value()),
|
||||
|
||||
+10
-1
@@ -16,10 +16,15 @@ if config.config_file_name is not None:
|
||||
target_metadata = SQLModel.metadata
|
||||
|
||||
|
||||
def include_name(name: str | None, type_: str, _parents: object) -> bool:
|
||||
return not (type_ == "table" and name is not None and name.startswith("taskiq_"))
|
||||
|
||||
|
||||
def run_migrations_offline() -> None:
|
||||
context.configure(
|
||||
url=env.db.connection_url,
|
||||
target_metadata=target_metadata,
|
||||
include_name=include_name,
|
||||
literal_binds=True,
|
||||
dialect_opts={"paramstyle": "named"},
|
||||
)
|
||||
@@ -28,7 +33,11 @@ def run_migrations_offline() -> None:
|
||||
|
||||
|
||||
def do_run_migrations(connection: Connection) -> None:
|
||||
context.configure(connection=connection, target_metadata=target_metadata)
|
||||
context.configure(
|
||||
connection=connection,
|
||||
target_metadata=target_metadata,
|
||||
include_name=include_name,
|
||||
)
|
||||
with context.begin_transaction():
|
||||
context.run_migrations()
|
||||
|
||||
|
||||
@@ -5,3 +5,12 @@ ruff format
|
||||
ruff check --fix
|
||||
ty check
|
||||
```
|
||||
|
||||
## Imports
|
||||
Every service imports only what it needs to start. `make check` runs `scripts/check_imports.py`, which fails when a service loads a heavy package it is not allowed; `SERVICES` there is the contract, extend it on purpose.
|
||||
- Package `__init__` files don't re-export clients or SDKs.
|
||||
- An SDK used by one function is imported inside it. On an async path load it first with `await preload("<module>")` from `utils.imports`, so the event loop never waits on the import.
|
||||
- Types used in Dishka provider signatures and `FromDishka[...]` stay real top-level imports.
|
||||
|
||||
## Dynamic config
|
||||
`config()` returns a copy of an in-process snapshot: no database round-trip per call, changes from any process show up within 5 s through a version number on the config record. Never `save()` a config you read; write through `DynamicConfig.update_config(section=...)` (Mongo) or `ConfigRepository.update(section=...)`/`save(...)` (Postgres), which bump the version atomically. Counters and other hot data don't belong in the config, give them their own collection or table with atomic increments.
|
||||
@@ -0,0 +1,22 @@
|
||||
{%- set targets = [] -%}
|
||||
{%- if database == 'postgres' %}{% set _ = targets.append("`make migrate` (`make migrate downgrade -1` to roll back), `make revision m=\"message\"`") %}{% endif -%}
|
||||
{%- if dynamic_config and backend_services %}{% set _ = targets.append("`make script name`") %}{% endif -%}
|
||||
{%- if 'kurigram_userbot' in backend_services %}{% set _ = targets.append("`make session`") %}{% endif -%}
|
||||
# {{ project_name }}
|
||||
{% if project_description and project_description != project_name %}
|
||||
{{ project_description }}
|
||||
{% endif %}
|
||||
Runs in Docker, locally the same way as in production. `COMPOSE_PROFILES` in `.env` selects the services that run on this machine. `docker-compose.override.yml` holds what differs between machines: published ports and the aliases on the external `caddy` network.
|
||||
|
||||
```sh
|
||||
make env
|
||||
make recreate
|
||||
```
|
||||
|
||||
`make env` creates `.env` and `docker-compose.override.yml` from their examples. `make recreate` starts the selected profiles, `make rebuild` builds the image first, `make logs`, `make restart` and `make down` do what they say.
|
||||
|
||||
{% if backend_services %}To deploy, `git pull && make deploy`. It runs `make pre-deploy`, which builds the image only when `pyproject.toml`, `uv.lock` or `Dockerfile` changed{% if database == 'postgres' %} and applies migrations{% endif %}, then `docker compose up -d`, then `make post-deploy`, which restarts the services whose mounted code changed. A deploy system with its own `up` step calls the two hooks around it. What was built and started is remembered as `refs/deploy/*` in the clone.
|
||||
|
||||
In the fleet, production is deployed from `infra/komodo` instead (git mode of Komodo, see `docs/architecture.md` there): Komodo clones this repo, and a layer plus `deploy.sh` in the stack's directory tag images by the hash of what they bake in, write `.env` from sops and run migrations before the code. The repo only has to stay a plain recipe: images built from `build:` in compose, migrations as the `migrator` service when there is a database, `caddy/site.caddy`. `make deploy` and `refs/deploy/*` are for a checkout deployed by hand.
|
||||
|
||||
{% endif %}`make fmt` formats, `make check` runs ruff and ty.{% if targets %} Also {{ targets | join(", ") }}.{% endif %}
|
||||
Reference in new issue
Block a user