FinEventa

# FinEventa Self-hosted, bring-your-own-keys financial event API for AI agents and quant traders. Multi-source ingestion (KAP, SEC EDGAR, TCMB EVDS), bilingual AI enrichment, and first-class MCP + Python SDK access. [](https://github.com/yigitbuyuktunaa/FinEventa/actions/workflows/ci.yml) [](LICENSE) [](https://www.python.org) --- ## What to Try First - Run the local API and inspect `GET /v1/filings/latest`. - Connect Claude Desktop or Cursor through the MCP server in `mcp-server/`. - Stream new filings with the Python SDK or try the Telegram/RAG recipes. --- ## What It Does FinEventa ingests financial disclosures and macro data from multiple sources and exposes them through a single, consistent REST API. Sources today: SEC EDGAR for US 8-K filings; TCMB EVDS for Turkish macro indicators (CPI, policy rate, USD/TRY, FX reserves); KAP for BIST corporate disclosures (self-host only, disabled by default). Every filing is AI-enriched with a bilingual (TR/EN) summary, sentiment score, and category using source-aware prompts. Built to be self-hosted and plugged directly into trading systems, backtests, and AI agents. --- ## Why Self-Host - **Bring your own keys.** You supply an Anthropic API key; all AI enrichment runs against your account. Optionally add TCMB EVDS and SEC User-Agent credentials. - **Your data stays yours.** Nothing leaves your machine unless you point it at an external database or monitoring service. - **No vendor lock-in.** No SaaS account, no billing, no usage limits except those you configure in your own rate-limit settings. --- ## Architecture at a Glance ```mermaid flowchart TB sec["SEC EDGAR<br/>8-K atom feed"] tcmb["TCMB EVDS<br/>macro indicators"] kap["KAP opt-in<br/>BIST disclosures"] sec --> adapters["Source Adapters<br/>env gated credentials"] tcmb --> adapters kap --> adapters adapters --> registry["Source Registry"] registry --> worker["Celery Worker + Beat<br/>at-least-once ingest"] worker --> db[("PostgreSQL 16<br/>filings, enrichments, companies")] worker --> ai["AI Enricher<br/>Claude Haiku primary<br/>Claude Sonnet failover"] ai --> db db --> api["FastAPI REST API v0.5.0<br/>/v1/filings + legacy KAP routes"] api --> sdk["Python SDK"] api --> mcp["MCP Server"] api --> ws["WebSocket Stream"] sdk --> users["AI agents, quant tools, recipes"] mcp --> users ws --> users ``` Worker and API share the same `app/` package (`infra/Dockerfile.worker` vs `infra/Dockerfile.api`). --- ## Supported Sources | Source | Status | Cadence | Notes | |---|---|---|---| | SEC EDGAR | Live | 8-K atom poll every minute; weekly company refresh (Sunday 04:00 UTC) | Set `SEC_EDGAR_ENABLED=true` and `SEC_USER_AGENT` | | TCMB EVDS | Live | Hourly macro poll | Set `TCMB_EVDS_ENABLED=true` and `TCMB_EVDS_API_KEY` | | KAP | Live (disabled by default) | 30-second poll | Optional self-hosted connector. Users are responsible for KAP, MKK, and Borsa Istanbul data usage, redistribution, and licensing requirements. Enable only when your usage is covered. | --- ## Quickstart Prerequisites: Docker and Docker Compose. ```bash git clone https://github.com/yigitbuyuktunaa/FinEventa.git cd FinEventa cp .env.example .env # Edit .env: set ANTHROPIC_API_KEY (required for AI enrichment). # Optionally set SEC_USER_AGENT and TCMB_EVDS_API_KEY if enabling those sources. docker compose up -d # Run database migrations docker compose exec api alembic upgrade head # Create your first API key (printed once; copy it now) docker compose exec api python -m app.cli create-key --tier pro curl http://localhost:8000/v1/health curl -H "Authorization: Bearer <KEY>" http://localhost:8000/v1/filings/latest ``` Sources are off by default. To start ingesting, enable one in `.env`: ```bash # .env -- SEC EDGAR (US 8-K filings); needs a User-Agent per SEC policy SEC_EDGAR_ENABLED=true SEC_USER_AGENT="YourApp you@example.com" ``` Recreate the containers so the new `.env` is loaded (a plain `restart` does not reload env vars), then seed the company list: ```bash docker compose up -d --force-recreate api worker # Seed the full company list so filings resolve to tickers # (required for ticker filtering and historical-context): docker compose exec api python -m scripts.seed_sec_companies ``` Interactive API docs are available at `http://localhost:8000/docs` when `APP_ENV` is not `production`. --- ## API Surface All authenticated endpoints require `Authorization: Bearer <KEY>`. | Endpoint | Description | |---|---| | `GET /v1/filings` | List filings across all sources with filters (source, ticker, filing_type, category, date range, enrichment_status). Offset-paginated, max 200 per page. | | `GET /v1/filings/latest` | Most recent filings, non-paginated, capped at 50. Returns only AI-enriched rows by default. | | `GET /v1/filings/{id}` | Single filing with its enrichment record. Returns 404 if not found. | | `GET /v1/filings/{id}/summary` | Stored AI summary for a processed filing. Returns 202 + Retry-After while enrichment is in progress. | | `GET /v1/filings/{id}/sentiment` | Stored AI sentiment score. Same 4-state response pattern as `/summary`. | | `GET /v1/filings/{id}/classify` | Stored AI category classification. Same 4-state response pattern as `/summary`. | | `GET /v1/filings/{id}/historical-context` | Historical reaction distribution for similar past filings (base rate, not a forecast). Returns sample size, percentile stats, and a fixed disclaimer. Reaction data available for SEC EDGAR filings only. | | `GET /v1/filings/{id}/extraction` | AI-extracted typed facts with provenance (evidence snippets per field). Verify against the source; not a forecast. Non-extractable and pending filings return an explicit status rather than an error. | | `GET /v1/kap/notifications` | KAP-only filings (backward-compatible namespace; returns `source="kap"` rows only). | | `GET /v1/kap/notifications/latest` | Most recent KAP notifications, non-paginated. | | `GET /v1/kap/notifications/{id}` | Single KAP notification with enrichment. Returns 404 for non-KAP IDs. | | `WS /v1/stream/filings` | Real-time stream of newly ingested filings. Optional `source` and `ticker` filters. Auth via `Authorization: Bearer` header or `?api_key=`. | | `GET /v1/health` | Postgres + Redis health probe. Public, no API key required. | The `/v1/kap/notifications` namespace is preserved for backward compatibility. It will not return SEC EDGAR or TCMB data. Prefer `/v1/filings` for new integrations. Full endpoint contracts, query parameters, error codes, and example payloads: [`docs/api-reference.md`](docs/api-reference.md) --- ## Authentication All endpoints except `/v1/health` require an API key in the `Authorization` header: ``` Authorization: Bearer fev_live_<key> ``` Keys are created with the CLI (see Quickstart). The plaintext key is printed once and never stored -- only a bcrypt hash is persisted. ```bash # Create a key docker compose exec api python -m app.cli create-key --tier pro # Optional flags: # --email admin@example.com (default: admin@localhost) # --name my-agent (human-readable label) # --env live|test (key prefix; default: live) # --expires-days 90 (omit for a non-expiring key) # List all keys (metadata only, no hashes) docker compose exec api python -m app.cli list-keys ``` The `--tier` flag selects a local rate-limit profile. In a self-hosted single-user setup you can choose `pro` or `enterprise`; the smaller profiles exist for multi-tenant deployments, demos, and tests. Valid profiles are `free`, `hobby`, `startup`, `pro`, and `enterprise`. Limits are configured in `app/middleware/rate_limit.py`; see [`docs/configuration.md`](docs/configuration.md) for the table. --- ## Connect an AI Agent (MCP) The [`mcp-server/`](mcp-server/) package ships a ready-to-use MCP server that exposes FinEventa as 6 tools and a taxonomy resource. Wire it into Claude Desktop or Cursor in under five minutes: ```bash pip install -e ./mcp-server # Then add "fineventa-mcp" to your claude_desktop_config.json or ~/.cursor/mcp.json. ``` Full setup guide, config JSON blocks, and tool reference: [`mcp-server/README.md`](mcp-server/README.md). --- ## Python SDK The [`sdk/`](sdk/) package ships `fineventa-sdk`, a typed client wrapping the REST API with both sync and async surfaces. ```python from fineventa import FineventaClient with FineventaClient(api_url="http://localhost:8000", api_key="fev_live_<key>") as client: page = client.filings.list(ticker="AKBNK", limit=20) detail = client.filings.get(page.items[0].id) ``` An `AsyncFineventaClient` mirrors the same surface for asyncio code. Install, async usage, and the full API reference: [`sdk/README.md`](sdk/README.md). --- ## Real-time Streaming (WebSocket) Connect once and receive new filings the moment they are ingested, instead of polling. A trading bot or AI agent subscribes to `WS /v1/stream/filings` with optional `source` and `ticker` filters and gets a JSON event per matching filing. ```python import json from websockets.sync.client import connect url = "ws://localhost:8000/v1/stream/filings?source=kap&ticker=AKBNK" with connect(url, additional_headers={"Authorization": "Bearer fev_live_<key>"}) as ws: for raw in ws: event = json.loads(raw) if event["type"] == "filing": print(event["data"]["subject"]) ``` Event types: `connected` (initial ack with active filters), `filing` (a new filing, payload identical to `GET /v1/filings` rows), and `heartbeat` (periodic keepalive). Delivery is at-least-once on `external_id`; dedup client-side if you need exact-once. Browser clients that cannot set headers may pass the key as `?api_key=` (be aware it can appear in reverse-proxy logs; prefer the header). Under the hood, ingestion workers publish each new filing to Redis pub/sub and the API process fans out to all subscribed connections. --- ## How We Evaluate Our AI Most projects ship an "AI sentiment" field and never say whether it means anything. We measured ours. [`fineventa-eval/`](fineventa-eval/) is a self-contained event-study pipeline that asks one honest question: > Does the LLM's sentiment, formed from the **filing text alone**, predict the > stock's short-term market reaction? It scores ~500 S&P 100 SEC 8-K filings (text only, strict no-leakage), computes the realized abnormal return around each filing independently, and compares the two. The headline finding, after correcting an event-window bug that first produced a spurious negative result: **no reliable short-term predictive relationship** (Spearman rho = +0.03, p = 0.53). In other words, the model produces a textual tone score, but that score does not reliably forecast next-day price moves. This is exactly why the API documents `sentiment_score` as "textual tone, not a price forecast or investment advice", and the finding now backs that wording with data. The pipeline is reproducible and runs as an offline CI gate: it recomputes the published metrics from committed recorded outputs with no API call and no network, so a future prompt or model change cannot silently break the result. Full methodology and limitations: [`docs/eval-methodology.md`](docs/eval-methodology.md) and [`fineventa-eval/reports/SONUC_RAPORU.md`](fineventa-eval/reports/SONUC_RAPORU.md). That same base-rate idea is now live in the API: `GET /v1/filings/{id}/historical-context` returns the historical reaction distribution for filings of the same event type, so callers can see what typically happened in the past -- without any claim about what will happen next. The response always includes a disclaimer to that effect. --- ## Coming Next - Historical backfill scripts -- seed months of SEC EDGAR and TCMB data on first run. --- ## Configuration All settings are environment variables. Copy `.env.example` to `.env` and edit. Full reference: [`docs/configuration.md`](docs/configuration.md). Essential variables: ```bash # Required -- instance will not start without these DATABASE_URL=postgresql+asyncpg://fineventa:fineventa@localhost:5432/fineventa DATABASE_URL_SYNC=postgresql+psycopg2://fineventa:fineventa@localhost:5432/fineventa REDIS_URL=redis://localhost:6379/0 APP_SECRET_KEY=<output of: openssl rand -hex 32> # Required for AI enrichment ANTHROPIC_API_KEY=sk-ant-... # Source adapters (all off by default) SEC_EDGAR_ENABLED=false SEC_USER_AGENT= # "AppName contact@email.com" TCMB_EVDS_ENABLED=false TCMB_EVDS_API_KEY= KAP_SCRAPING_ENABLED=false ``` The local `docker-compose.yml` wires `DATABASE_URL` and `REDIS_URL` automatically to the bundled Postgres and Redis containers, so you only need to set `APP_SECRET_KEY` and `ANTHROPIC_API_KEY` to get a working local instance. --- ## Development ```bash # Run the full stack docker compose up -d # Tests cd api export DATABASE_URL=postgresql+asyncpg://finevent:finevent@localhost:5432/finevent export REDIS_URL=redis://localhost:6379/0 export APP_SECRET_KEY=test-secret-key-with-at-least-32-bytes pytest tests/unit/ # fast, external services mocked pytest tests/integration/ # requires Docker stack # Lint and type-check ruff check . mypy app/ # Smoke test (verifies health, auth, rate-limit headers, multi-source query) bash api/scripts/smoke.sh ``` This is a monorepo of independent Python packages plus runnable examples. The main installable packages are `api/`, `mcp-server/`, and `sdk/`; `recipes/` and `fineventa-eval/` have their own dependency files. CI tests each surface with the matching environment. Run a package's tooling from its own directory after an editable install, for example: ```bash cd mcp-server && pip install -e ".[dev]" && mypy fineventa_mcp && pytest cd sdk && pip install -e ".[dev]" && mypy fineventa && pytest ``` Running `mypy`/`pytest` for `mcp-server` or `sdk` from a venv that does not have that package installed will report spurious import or untyped-decorator errors; install the package first. See [`docs/local-development.md`](docs/local-development.md) for the full local setup guide including migrations, seed data, worker behaviour, and troubleshooting. --- ## Documentation Index | File | Contents | |---|---| | [`docs/architecture.md`](docs/architecture.md) | System architecture, multi-source ingest pattern, Celery beat schedule, AI flow, DB schema notes | | [`docs/api-reference.md`](docs/api-reference.md) | Endpoint contracts, query parameters, error codes, pagination, example payloads | | [`docs/configuration.md`](docs/configuration.md) | All environment variables, per-source enable flags, production boot validators | | [`docs/local-development.md`](docs/local-development.md) | Local setup, migrations, seed data, worker, troubleshooting | | [`docs/operations-runbook.md`](docs/operations-runbook.md) | Deploy, Celery beat tasks, source enable/disable, AI model swap, incident response | | [`docs/eval-methodology.md`](docs/eval-methodology.md) | How we evaluate the AI enrichment (event-study methodology) | For roadmap and project direction, see [`ROADMAP.md`](ROADMAP.md). Project history lives in the commit and pull-request log. --- ## Contributing Contributions are welcome. See [`CONTRIBUTING.md`](CONTRIBUTING.md) for the dev setup, test/lint commands, and PR conventions. Security reports: see [`SECURITY.md`](SECURITY.md). --- ## License MIT. See [LICENSE](LICENSE). **Data disclaimer:** Data and AI-generated content produced by this software are for informational purposes only. Nothing produced by this API constitutes investment advice. KAP support is provided as an optional, self-hosted connector and is disabled by default. Users are responsible for complying with KAP, MKK, and Borsa Istanbul data usage, redistribution, and licensing requirements. For production or commercial redistribution, use the official KAP Data Dissemination Service and obtain the required agreements.