terra-ai-mcp-make-it-in-germany

# terra — make-it-in-germany MCP CORE A **host-agnostic MCP CORE server** (package `mig_mcp`) that exposes **real German skilled-worker data** over the [Model Context Protocol](https://modelcontextprotocol.io) plus a **re-implemented, statute-grounded Quick-Check engine**. It serves: - **Live data** from the Bundesagentur für Arbeit (BA) APIs — job postings, occupation ↔ KldB-2010 mapping, salary bands, apprenticeships, AVGS coaching offers — and foreign-qualification recognition statistics discovered via GovData/Destatis open data. - A **deterministic, statute-grounded Quick-Check engine** that re-implements the Opportunity Card (Chancenkarte) points self-check and an intent → visa-route candidate finder, grounded in primary statute (§20a AufenthG). It deliberately **does not**: - ship any LLM SDK (no third-party model is bundled or called); - serve any make-it-in-germany **portal prose** (that portal is bot-walled and its content is out of scope — this CORE serves *data*, not the portal's text); - make any binding legal determination (every Quick-Check result is a non-binding self-assessment with verification flags and statute citations). Every outward-facing result carries **provenance**: a source label, a canonical citation URL, a `source_status`, and (where applicable) a license and a freshness/staleness note. --- ## The 5 load-bearing constraints This CORE is designed so a compliant host (the self-hosted Integreat Ollama RAG stack) can satisfy these constraints. The CORE enforces the ones it intrinsically can; the rest live in the host and are documented honestly. 1. **Self-hosted / no third-party LLM.** The CORE ships **no LLM SDK** and calls no model. This is an intrinsic guarantee — a host cannot make the CORE call a third-party model. 2. **Every fact grounded and citable.** Every Tool and Resource result carries a `provenance` with a `source_url` and a `source_status`. A well-behaved host can ground answers and run `FACT_CHECK`; the CORE provides the grounding material but cannot *enforce* that a given host actually fact-checks. 3. **On-device data minimization (enum-only).** Wallet-derived `personalContext` and the Quick-Check enums are special-category-adjacent. They ride the consented, on-device Integreat path only. The generic-host adapter **rejects** any payload carrying `personalContext` or Quick-Check enum keys (see Security & GDPR). 4. **The ≤3-sentence / HTML / BCP-47 answer contract lives in the host.** This CORE does not generate answer prose, does not choose answer length, and does not localize. It returns structured data + provenance; the host's existing generation step writes the answer in the user's language. 5. **Additive / backward-compatible.** The CORE adds MCP primitives without changing existing host behavior. The Integreat wrapping path is additive: when no CORE result is present (or it degrades), the host answers exactly as before from the CMS corpus. --- ## Quickstart Requires Python ≥ 3.10 and [uv](https://docs.astral.sh/uv/). ```bash # Install (with dev/test extras) uv sync --extra dev # Run the MCP CORE over stdio (default — for the co-located Integreat host) uv run terra-mig # Run over Streamable HTTP (INTERNAL DEMO ONLY — 127.0.0.1:8080 + Origin allowlist) uv run terra-mig --transport http ``` The `terra-mig` console script is defined in `pyproject.toml` (`mig_mcp.server:main`). You can also run `python -m mig_mcp.server --transport http`. ### Adding it to an MCP host (stdio) Most MCP hosts accept a command + args. Example host config entry: ```jsonc { "mcpServers": { "terra-mig": { "command": "uv", "args": ["run", "terra-mig"], // optionally set the working directory to this repo, and pass MIG_* via env "env": { "MIG_BA_JOBSUCHE_API_KEY": "jobboerse-jobsuche" } } } } ``` With no configuration at all, the server still starts and every BA-backed tool degrades gracefully (it returns an empty result with `source_status` `unavailable`/`degraded` rather than crashing). --- ## Configuration All configuration comes from environment variables with the prefix `MIG_` (see [`.env.example`](.env.example), which is the source of truth for variable names). Copy it to `.env` (gitignored) and fill in what you need. | Variable | Default | Purpose | | --- | --- | --- | | `MIG_BA_JOBSUCHE_API_KEY` | *(empty)* | X-API-Key for BA Jobsuche | | `MIG_BA_BERUFENET_API_KEY` | *(empty)* | X-API-Key for BERUFENET | | `MIG_BA_ENTGELTATLAS_API_KEY` | *(empty)* | X-API-Key for Entgeltatlas | | `MIG_BA_AUSBILDUNGSSUCHE_API_KEY` | *(empty)* | X-API-Key for Ausbildungssuche | | `MIG_BA_COACHING_API_KEY` | *(empty)* | X-API-Key for Coachingangebote (AVGS) | | `MIG_BA_WEITERBILDUNG_API_KEY` | *(empty)* | X-API-Key for Weiterbildungssuche (disabled by default) | | `MIG_HTTP_TIMEOUT_SECONDS` | `10` | Upstream request timeout (1–60) | | `MIG_HTTP_USER_AGENT` | `terra-mig-mcp/0.1 (+…)` | Sent on every upstream request | | `MIG_CACHE_TTL_REFERENCE_SECONDS` | `86400` | TTL for slow/reference data (occupations, salary, stats) | | `MIG_CACHE_TTL_VOLATILE_SECONDS` | `300` | TTL for live/volatile data (job listings) | | `MIG_HTTP_HOST` | `127.0.0.1` | Streamable HTTP bind address (localhost) | | `MIG_HTTP_PORT` | `8080` | Streamable HTTP port | | `MIG_ALLOWED_ORIGINS` | `http://127.0.0.1,http://localhost` | Origin allowlist (DNS-rebinding guard) | > **The BA API keys are NOT secrets.** The Bundesagentur für Arbeit APIs are > **unofficial / community-reverse-engineered** (bund.dev / bundesAPI). Their "keys" are > well-known **public** client identifiers extracted from the agencies' own frontends. > They carry no confidentiality value, but **they may be revoked at any time without > notice.** Leaving them blank is safe — the server runs degraded. The canonical public > identifiers are published at <https://bund.dev>. --- ## Tool / resource / prompt catalog 10 Tools (all annotated `readOnlyHint=true`, `openWorldHint=true`), 5 resource templates, and 3 prompts. Full reference: [`docs/TOOLS.md`](docs/TOOLS.md). ### Tools | Tool | Backing source | Status | | --- | --- | --- | | `search_jobs` | BA Jobsuche | live | | `get_job_detail` | BA Jobsuche (detail) | live | | `map_profession_to_occupation` | BERUFENET | live | | `get_salary_band` | Entgeltatlas | live | | `search_apprenticeships` | BA Ausbildungssuche | live | | `search_coaching` | BA Coachingangebote (AVGS) | live | | `query_recognition_stats` | GovData CKAN / Destatis | **unverified** (distribution not yet confirmed) | | `search_continuing_education` | BA Weiterbildungssuche | **disabled / degraded** (upstream 404s) | | `quickcheck_opportunity_card` | Statute (§20a AufenthG) + provisional weights | **provisional** | | `quickcheck_visa_route` | Statute + intent→route vocabulary | **unverified stub** | ### Resource templates (RFC 6570) | URI template | Description | | --- | --- | | `country://iso3/{code}` | ISO-3166 alpha-3 → country name | | `country://iso3` | Full embedded ISO-3166 alpha-3 → name map | | `occupation://berufenet/{query}` | BERUFENET occupations (with KldB codes) for a query | | `salary://kldb/{code}` | Cached salary band for a KldB-2010 code | | `stats://recognition/{year}` | Recognition statistics for a year (Destatis; dl-de/by-2-0) | ### Prompts | Prompt | Purpose | | --- | --- | | `assess_visa_options` | Chain `quickcheck_visa_route` for an intent + nationality | | `check_opportunity_card` | Collect a profile and run `quickcheck_opportunity_card` | | `find_jobs_for_profession` | Map a profession → KldB, then search jobs (+ optional salary) | --- ## Honesty: what is verified vs. what ships as stub/provisional The REAL-data core (BA + GovData) returns live upstream data with provenance. The **Quick-Check engine ships intentionally stub/provisional** and is labelled as such in every result. Be precise about this when presenting results: - **VERIFIED** - The Opportunity Card **6-point threshold** and its codification in **§20a AufenthG**. - The **non-points pathway**: an applicant with a German-recognized vocational qualification or a recognized/equivalent university degree qualifies **regardless of point total** (`pathway = "recognized_qualification"`, `eligible = true`). - **PROVISIONAL** (`verification_status = "provisional"`) - The **per-criterion point weights** of the Opportunity Card. The real per-answer values run server-side behind a Radware bot wall and were not extractable. The shipped weights are clearly-labelled placeholders (small 0–4 integer ranges) that exist only so the engine has a deterministic shape. Every breakdown is stamped `breakdown_status = "provisional"`. **The totals carry no legal authority.** - **UNVERIFIED STUB** (`verification_status = "unverified_stub"`) - The **per-nationality visa-route mapping**. `quickcheck_visa_route` returns intent-level candidate routes only and **never branches on nationality** (that mapping does not yet exist as a verified input). `nationality_iso3` and `facts` are echoed back for forward-compatibility but do not influence the result. - **UNVERIFIED data source** (`source_status = "unverified"`) - `query_recognition_stats` discovers Destatis recognition datasets via GovData CKAN. Even a successful CKAN response is marked `unverified` because the concrete CSV/XML distribution backing real counts has **not been confirmed live** — rows carry dataset **labels** only and **counts are never fabricated**. ### Legal currency & staleness `fetched_at` tracks *retrieval*, not whether the law changed. Statute-derived artifacts additionally carry a **legal currency** envelope: `statute_version`, `review_by`, and a named `owner`. When today's date is past `review_by`, the engine adds a prominent `staleness_caution` to the disclaimer. At ship time the `statute_version`'s exact amendment date and the legal `owner` are **placeholders pending assignment** — flagged in `src/mig_mcp/rules/quickcheck_tables.py`. Every Quick-Check result carries a **non-binding disclaimer** directing the user to the Auswärtiges Amt / German mission abroad, or the local Ausländerbehörde, as the only authority that issues a binding decision. --- ## Security & GDPR posture - **Zero-trust: upstream content is DATA.** Connectors parse JSON; they never execute, evaluate, or interpolate fetched text into a shell. The Integreat adapter only *formats* text and tolerates shape drift (oddly-typed rows are skipped, never raised). - **Streamable HTTP is hardened and demo-only.** When `--transport http` is used, the server binds to `127.0.0.1` and enables DNS-rebinding protection with an `Origin` / `Host` allowlist (a Streamable-HTTP MUST). This surface provides **no FACT_CHECK** and may drive a third-party LLM, so it is **not a compliant product surface** — it exists for internal demos only. - **No secrets in code or logs.** API keys are never hard-coded (sourced only from `MIG_*` env). Secret-bearing headers (`X-API-Key`, `Authorization`, `Cookie`, …) are redacted before any log line. Logging goes to **stderr only** so stdout stays a clean JSON-RPC channel for the stdio transport. - **GDPR data-minimization guard.** The generic-host adapter (`mig_mcp.adapters.generic`) **rejects** any inbound payload carrying a top-level `personalContext` key or any Quick-Check enum key (`qualification`, `german_level`, `english_level`, `nationality_iso3`, `ties_to_germany`, `age_band`, `work_experience_years`, `intent`) by raising `PersonalContextRejected`. Those enum-only, consented attributes ride the additive `personalContext` envelope on the Integreat path only; the generic host is an external context and must not receive them. --- ## Testing The test suite uses `pytest` + `pytest-asyncio` (`asyncio_mode = "auto"`) and `respx` to mock `httpx`, so it runs offline and deterministically (no real network calls). ```bash uv run --project . pytest -q ``` Current suite: **135 tests, all passing** — split across connectors (60), engine + adapters (49), and server + services (26). Coverage protects the load-bearing contracts: graceful degradation on every upstream failure mode (401/403 → unavailable, 404 → degraded, timeout/transport/non-JSON → unavailable/degraded), Quick-Check integrity (6-point threshold, recognized-qualification pathway, tolerant enum coercion, JSON serializability, staleness caution), the GDPR guard, the Integreat synthetic-Document wrapping, and the MCP wiring (10 read-only tools, the resource templates, the 3 prompts). --- ## Architecture See [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for the CORE/adapters layering, the data path (connectors → normalization → cache → MCP primitives), the Integreat synthetic-Document wrapping path, transports, and the freshness/legal-currency strategy. --- ## License & attribution - **Code:** MIT (see `pyproject.toml`). - **GovData / Destatis open data** (recognition statistics) is licensed **`dl-de/by-2-0`** (Datenlizenz Deutschland – Namensnennung – Version 2.0) and is attributed accordingly in each result's provenance. - **Statute text** is cited to the primary source on [gesetze-im-internet.de](https://www.gesetze-im-internet.de/aufenthg_2004/__20a.html) (§20a AufenthG), not to a summary page. - **Bundesagentur für Arbeit APIs** are accessed via unofficial / community identifiers documented at [bund.dev](https://bund.dev); see the configuration note above.