Content
# telegram-rest-gateway
Self-hosted REST/WebSocket wrapper over a user Telegram account based on
**TDLib** (native C++ interface, `td::ClientManager`) and **Drogon**. Provides language-independent
HTTP access to the account, hiding the asynchronous nature of TDLib behind a "request → response" model
and delivering updates via WebSocket.
> Status: **Stage 0 (framework)**. Bridge/authorization/messages functionality is under development
> in stages (see `docs/SPEC_ELABORATION.md` §12). Build only in Docker/CI.
## Documentation
| File | Purpose |
|---|---|
| [`TZ_TDLib_Drogon_REST.md`](TZ_TDLib_Drogon_REST.md) | Main requirements (v2.0) |
| [`docs/SPEC_ELABORATION.md`](docs/SPEC_ELABORATION.md) | Detailed implementation level (v3.0) |
| [`docs/DECISIONS.md`](docs/DECISIONS.md) | Accepted decisions log |
| [`docs/GITFLOW.md`](docs/GITFLOW.md) | Branching model |
| [`openapi/openapi.yaml`](openapi/openapi.yaml) | Single source of truth for HTTP contract |
## Stack and key decisions
- **C++20** (Drogon coroutines), TDLib built with C++17.
- **TDLib linked statically** via `Td::TdStatic` — native C++ API not exported as `libtdjson.so`.
- Unified **OpenSSL 3.0** for TDLib, Drogon, and our code.
- Single-account, WebSocket only for updates, TDLib DB encryption, JSON IDs as strings.
- Deployment: multi-arch (amd64 + arm64) Docker, final — distroless nonroot.
## Structure
```
src/
main.cpp — entry point, /v1/health, --healthcheck
bridge/
transport.hpp — ITdTransport (seam over td::ClientManager)
real_transport.* — RealTdTransport (single point of real TDLib)
clock.hpp — time injection (SteadyClock / FakeClock in tests)
tests/
fake_transport.hpp — FakeTdTransport (scripted, thread-safe)
transport_smoke_test.cpp
openapi/openapi.yaml
Dockerfile.builder — toolchain + TDLib + Drogon image (pinned TDLIB_REF/DROGON_REF)
Dockerfile — service image (FROM builder) → distroless
```
CI/CD — GitHub Actions ([`.github/workflows/ci.yml`](.github/workflows/ci.yml)): lint (clang-format), suppressions-guard, cached GHCR builder image (TDLib+Drogon), tests under ASan/TSan, clang-tidy, and Release service build. Historical note on old pipeline variables — [`docs/CICD.md`](docs/CICD.md).
## Build (in container)
```bash
# 1. Fix TDLIB_REF (full git SHA!) and DROGON_REF.
# 2. Build builder image (long; only when changing pins):
docker build -f Dockerfile.builder \
--build-arg TDLIB_REF="$(grep -oP '(?<=^TDLIB_REF=).*' TDLIB_REF)" \
--build-arg DROGON_REF="$(grep -oP '(?<=^DROGON_REF=).*' DROGON_REF)" \
-t tgw-builder:local .
# 3. Build service:
docker build --build-arg BUILDER_IMAGE=tgw-builder:local -t telegram-rest-gateway .
```
Local build outside Docker requires installed TDLib (`Td::TdStatic`), Drogon, and
OpenSSL 3.0; configuration via `CMakePresets.json` (`cmake --preset dev-debug`).
## Configuration (env)
| Variable | Default | Purpose |
|---|---|---|
| `TGW_LISTEN_ADDRESS` | `127.0.0.1` | Listen address (in Docker — `0.0.0.0`) |
| `TGW_LISTEN_PORT` | `8080` | HTTP port |
| `TGW_SESSION` / `TGW_SESSION_FILE` | — | Session string (base64 from `td.binlog`) for stateless start |
| `BEARER_TOKENS` | — | One string per token: `<token>[ <scopes>]`; scopes: `read`,`write`,`admin` (no scopes = all) |
| `TGW_WS_MAX_PENDING_BYTES` | `8388608` | WS back-pressure: byte limit from last pong; 0 — off |
| `TGW_MAX_MEMORY_BODY_BYTES` | `1048576` | Threshold for spooling to disk for request bodies (RSS during uploads) |
| `TGW_KAFKA_BROKERS` | — | Kafka/Redpanda bootstrap; empty — events in Kafka are disabled |
| `TGW_KAFKA_TOPIC` | `tgw.updates` | Events topic |
| `TGW_KAFKA_CLIENT_ID` | `tgw-<session_id>` | Producer client.id |
### Storage session in S3/MinIO (optional)
If set, `td.binlog` is pulled from S3 at startup (if local one is absent), periodically and on
graceful shutdown pushed back. Allows running service fully stateless without mounting
a volume. Enabled only when `bucket` + credentials + `endpoint` are filled.
| Variable | Default | Purpose |
|---|---|---|
| `TGW_S3_ENDPOINT` | — | `http(s)://host[:port]` (AWS: `https://s3.<region>.amazonaws.com`) |
| `TGW_S3_REGION` | `us-east-1` | Region for SigV4 signature |
| `TGW_S3_BUCKET` | — | Bucket name |
| `TGW_SESSION_ID` | `default` | Instance label → S3 path (segment `[A-Za-z0-9._-]`) |
| `TGW_S3_PREFIX` | `telegram-sessions` | Key prefix |
| `TGW_S3_KEY` | *(derived)* | Explicit key override; otherwise `<prefix>/<session_id>/td.binlog` |
| `TGW_S3_ACCESS_KEY_ID` / `_FILE` | — | Access key |
| `TGW_S3_SECRET_ACCESS_KEY` / `_FILE` | — | Secret key |
| `TGW_S3_PATH_STYLE` | `true` | `true` — path-style (MinIO); `false` — virtual-host (AWS) |
| `TGW_S3_SYNC_INTERVAL_SECONDS` | `300` | Background session backup to S3 interval |
**Multiple accounts on one bucket.** Each instance gets its own path by `TGW_SESSION_ID`:
`telegram-sessions/<session_id>/td.binlog`. Set different `TGW_SESSION_ID` for each account
(Telegram account_id is not available before login, so the label is assigned by the operator). The same
`session_id` cannot be used in two instances simultaneously — Telegram will kill the session (`AUTH_KEY_DUPLICATED`).
**Scoped tokens.** String `BEARER_TOKENS` like `tgw_xxx read` gives a token only for reading
(GET + WS), `read,write` — plus mutations, but without `/v1/auth/*` (login and session export — only
`admin`/full token). Insufficient scope → `403 INSUFFICIENT_SCOPE`. Issue minimal necessary scope to agents (MCP).
Secrets (`api_id`, `api_hash`, `database_encryption_key`, Bearer tokens, S3 credentials) — only
via `*_FILE` / secret manager, never directly in image/env.
## Events in Kafka
If `TGW_KAFKA_BROKERS` is set, each update from the allowlist (the same set as in WS) is published
to the topic `TGW_KAFKA_TOPIC`. Message body format is like a WS frame (`type/update_type/seq/session_id/data`);
**message key is `<session_id>:<chat_id>`** (account ID prefix; order within a chat is guaranteed by partitioning). Delivery is at-least-once: deduplication by consumer by
`(session_id, seq)`; gap in `seq` = loss (see `tgw_kafka_dropped_total`). Producer never blocks
Telegram updates: on queue overflow, events are dropped with a metric.
## Archiver: conversation search
[`archiver/`](archiver/README.md) — Kafka events consumer → SQLite+FTS5: full-text
search over entire saved history (`GET :8090/search?q=`), including edits and deleted.
Available to the agent as MCP-tool `telegram_search_history`.
## MCP: agent connector
[`mcp/`](mcp/README.md) — MCP server (TypeScript, stdio): Claude Code/Desktop and any
MCP client get 14 account tools (messages, reactions, resolve, media).
One server = one account; token = full access, store as a secret.
## Deployment to Kubernetes (Helm)
Chart: [`deploy/helm/telegram-rest-gateway`](deploy/helm/telegram-rest-gateway). Features:
one session = one pod (**strategy Recreate, no replicas scaling** — otherwise
`AUTH_KEY_DUPLICATED`), no PVC (sessions in S3), readiness on `/v1/health` (unauthorized pod
should accept traffic for login via `/ui`). Secrets — via `existingSecret`.
```bash
kubectl create secret generic telegram-rest-gateway \
--from-literal=API_ID=... --from-literal=API_HASH=... \
--from-literal=DATABASE_ENCRYPTION_KEY=... --from-literal=BEARER_TOKENS=... \
--from-literal=TGW_S3_ACCESS_KEY_ID=... --from-literal=TGW_S3_SECRET_ACCESS_KEY=...
helm install tgw deploy/helm/telegram-rest-gateway \
--set 'accounts[0].sessionId=<account_id>'
```
## Metrics
`GET /metrics` — Prometheus text format (no Bearer): auth state, WS subscribers,
bridge inflight, HTTP/update counters.
## Web interface for login
`GET /ui` — self-contained login page (no Bearer filter; token is entered in a form):
choose **QR code** method (auto-renewal during TDLib link rotation) or **phone**
(number → code → 2FA). After authorization, shows the account.
## OpenAPI
Specification of all endpoints: [`docs/openapi.yaml`](docs/openapi.yaml).
## License
[MIT](LICENSE).
