feat(transcribe,telegram,config): voice notes, round videos and audio get a transcript from pluggable providers, an openai-compatible one built in
This commit is contained in:
1 parent
944b68e275
commit
42a7058f75
12 files changed
+454
-28
No files matched your search
@@ -38,6 +38,9 @@ is a full setup built on them.
|
||||
write-only from the admin, handed out only through `secret run NAME --
|
||||
cmd` (one `sudo` rule for `beaver-runner`, a root-only socket, files in
|
||||
`/dev/shm`, values masked in the command's output).
|
||||
- **transcribe/** - speech to text for the voice notes, round videos and
|
||||
audio a window receives: providers tried in order, `OpenAITranscriber`
|
||||
for any OpenAI-compatible endpoint built in.
|
||||
- **events/** - the in-process bus every frontend taps, and the stream
|
||||
event protocol backends emit.
|
||||
- **storage/** - SQLModel tables and the Postgres session store.
|
||||
@@ -67,6 +70,46 @@ file next to it adds Postgres. Without Docker: `uv sync`, build the SPA with
|
||||
`DATABASE_URL` and `CONFIG_PATH` set. A setup's own secrets come from `.env`
|
||||
next to its config (`load_dotenv` runs before the config is executed).
|
||||
|
||||
## Transcription
|
||||
|
||||
A voice note, a round video or an audio file sent to the Telegram window is
|
||||
downloaded as any attachment; with `Gateway.transcribers` set, the frontend
|
||||
runs them in order and the first non-empty text rides under the attachment
|
||||
note (`TelegramTexts.attachment_transcript`). A provider that raises is
|
||||
logged and the next one tried; the whole chain gives up after the
|
||||
frontend's `transcribe_timeout` (90 s) and the note goes without the text.
|
||||
No transcribers, nothing changes.
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
from beaver_gateway.transcribe import Audio, OpenAITranscriber
|
||||
|
||||
|
||||
async def voice_only(audio: Audio) -> str | None:
|
||||
if audio.media_type != "audio/ogg":
|
||||
return None
|
||||
return await my_service.transcribe(audio.path, seconds=audio.duration)
|
||||
|
||||
|
||||
gateway = Gateway(
|
||||
transcribers=[
|
||||
voice_only,
|
||||
OpenAITranscriber(
|
||||
base_url="https://api.groq.com/openai/v1",
|
||||
api_key=os.environ["GROQ_API_KEY"],
|
||||
model="whisper-large-v3",
|
||||
language="ru",
|
||||
),
|
||||
],
|
||||
)
|
||||
```
|
||||
|
||||
A provider is any `async (Audio) -> str | None`; `None` passes the audio
|
||||
on. `OpenAITranscriber` posts the file to `{base_url}/audio/transcriptions`,
|
||||
the wire of OpenAI, Groq, faster-whisper-server and a whisper.cpp server;
|
||||
`api_key=None` for a local one without auth.
|
||||
|
||||
## Develop
|
||||
|
||||
```sh
|
||||
|
||||
Reference in new issue
Block a user