Content
# vault-mcp
`vault-mcp` is not just a simple HashiCorp Vault management tool, but an Ops Broker that centrally authenticates, authorizes, audits, and manages credentials for server, website, and API access.
LLM (Claude) or CLI sends user requests to the central API, which controls user authentication, authorization, resource interpretation, Vault credential lookup, result masking, and audit logging. Actual secrets are stored only in Vault, and not leaked to LLM, CLI, DB, or API responses.
> **Current stage: STEP 8.7 — Resource type expansion (SERVER/WEBSITE/API) completed.**
> Next is STEP 9 (authentication improvement). SSH execution (Runner) is STEP 10 and not yet implemented.
> Detailed stages are in [docs/development-roadmap.md](docs/development-roadmap.md).
## Table of Contents
- [Core Principles](#core-principles)
- [Architecture](#architecture)
- [Resource Types and Storage Responsibilities](#resource-types-and-storage-responsibilities)
- [API Reference](#api-reference)
- [Authentication Methods](#authentication-methods)
- [Vault Paths and Deletion Policies](#vault-paths-and-deletion-policies)
- [Data Model](#data-model)
- [Quick Start](#quick-start)
- [Environment Variables](#environment-variables)
- [Test and Quality Gates](#test-and-quality-gates)
- [Directory Structure](#directory-structure)
- [Development Stages](#development-stages)
- [Deployment](#deployment)
- [Technology Stack](#technology-stack)
- [Document Index](#document-index)
## Core Principles
| Principle | Description |
|---|---|
| **Secret Isolation** | Actual secrets are stored only in Vault. DB, API responses, logs, and audits contain only `credential_ref` (logical paths). |
| **1 Resource = 1 Credential** | One registration item = one access target = one credential. No `credential_id` concept (not 1:N). |
| **Owner-Exclusive** | Resources are managed only by their owners. Non-owner resources are not visible (404, not 403). |
| **Client Input Distrust** | Server determines `user_id`, `credential_ref`, `vault_path`, and `host` from request bodies. |
| **LLM is an Alias** | Claude extracts only `alias` and `operation`. Server interprets and retrieves credentials within user permissions. |
| **Mandatory Auditing** | All credential accesses (registration, replacement, deletion, reveal) are logged in audit logs (excluding secret contents). |
## Architecture
Current implementation of data flow (registration, lookup, deletion). SSH execution (Runner) is not yet implemented (STEP 10).
```mermaid
flowchart TD
subgraph client [Client]
W[Web UI · Jinja2]
S[Claude Skill · REST]
end
W -->|Cookie session + CSRF| API
S -->|API Key| API
API[vault-mcp FastAPI]
API -->|Authentication 4 types → users.id| AUTH{Authentication · Authorization}
AUTH -->|Non-sensitive metadata| PG[(PostgreSQL<br/>vault_mcp schema)]
AUTH -->|credential_ref for Secret| VAULT[(HashiCorp Vault<br/>KV v2)]
API --> AUDIT[(audit_logs)]
PG -.->|credential_ref = logical path| VAULT
```
| Flow | Path |
|---|---|
| **Registration** | Resource registration (PG) → Credential registration → Vault write → `credential_ref` stored in PG |
| **Lookup (reveal)** | Ownership verification (PG) → `credential_ref` for Vault read → plain text secret return (no-store) |
| **Deletion** | Vault soft delete → PG status/`credential_ref` update → commit. **Compensate with Vault undelete if commit fails** |
Component responsibilities:
| Component | Responsibilities | Does not |
|---|---|---|
| **Claude Skill** | Extract `alias`/`operation`, analyze results, output only non-sensitive summaries | Output secret contents (prohibited even if user requests), access Vault/paths |
| **Web UI** | Login, resource CRUD, credential registration/replacement/deletion, API Key issuance | Re-display secrets (no reveal in Web) |
| **FastAPI** | Authentication, authorization, resource interpretation, Vault integration, masking, auditing | Expose secrets in responses/logs |
| **Vault** | Store actual secrets | — |
| **PostgreSQL** | Store non-sensitive metadata + `credential_ref` | Store actual secrets |
## Resource Types and Storage Responsibilities
As of STEP 8.7, registration targets are **SERVER**, **WEBSITE**, and **API** types (`OTHER` is out of scope). Types are immutable after registration (409). Reference: [docs/resource-types.md](docs/resource-types.md).
`✓` used · `—` always NULL (returns 400 if sent) · **bold** = required
| Field | Storage | SERVER | WEBSITE | API |
|---|---|:--:|:--:|:--:|
| `alias` / `description` / `service_name` | PG | **✓** / ✓ / **✓** | **✓** / ✓ / ✓ | **✓** / ✓ / ✓ |
| `status` / `environment` | PG | ✓ / ✓ | ✓ / — | ✓ / — |
| `host` / `ssh_port` | PG | **✓** / **✓** | — | — |
| `site_url` | PG | — | **✓** | — |
| `base_url` / `auth_type`(ApiAuthType) | PG | — | — | **✓** / **✓** |
| `header_name` / `token_prefix` / `scopes` / `expires_at` | PG | — | — | ✓ |
| `login_id` | PG | **—**(¹) | **✓** | ✓ (= `client_id`) |
| `memo` | PG | — | ✓ | ✓ |
| `username` / `port` | **Vault** | **✓** / ✓ | — | — |
| `password` | **Vault** | ✓(²) | **✓** | — |
| `private_key` / `passphrase` | **Vault** | ✓(²) | — | — |
| `api_key` / `token` / `refresh_token` / `client_secret` / `webhook_secret` | **Vault** | — | — | ✓(³) |
- (¹) **SERVER's SSH `username` is only in Vault** (backward compatibility). So, `login_id` in list responses is always `null` for SERVER.
- (²) SERVER uses `auth_type`(SshAuthType) to determine `password` **or** `private_key` (+ optional `passphrase`).
- (³) API's `auth_type`(ApiAuthType) **determines which secret is required**. `client_id` is not a secret (in PG), and **`client_secret` is in Vault** — returned as a pair with reveal.
**Authentication type enums are maintained separately**: SERVER = `SshAuthType`(PASSWORD/PRIVATE_KEY, Vault), API = `ApiAuthType`(API_KEY/ACCESS_TOKEN/BEARER_TOKEN/PERSONAL_ACCESS_TOKEN/CLIENT_CREDENTIALS/OAUTH_TOKEN/WEBHOOK_SECRET/CUSTOM, PostgreSQL). WEBSITE has no authentication type concept.
## API Reference
Base prefix `/api/v1`. Schema, Authorize, and curl examples: [docs/swagger-guide.md](docs/swagger-guide.md) (`/docs`, `/redoc`, `/openapi.json` are only exposed in local/dev).
**Secret return** is only through reveal. Other credential APIs return only metadata.
| Method | Path | Purpose | Authentication / scope | Secret |
|---|---|---|---|:--:|
| GET | `/health` · `/ready` | Liveness / Readiness (DB+Vault) | None | — |
| POST | `/auth/register` · `/auth/login` | LOCAL registration / login (JWT issuance) | None | — |
| GET | `/me` | Current user profile | Authentication | — |
| POST/GET/DELETE | `/api-keys` · `/api-keys/{id}` | API Key issuance (exposed once)/list/disposal | Only Web users (not API Key) | Issued once |
| GET | `/servers` · `/servers/{alias}` | Operation-authorized server list/details | `servers.read` | — |
| POST/PUT/GET/DELETE | `/resources/{id}/credentials` | **All types** credential registration/replacement/status/deletion | Authentication | — |
| POST/PUT/GET/DELETE | `/servers/{id}/credentials/ssh` | SSH Credential (**SERVER-only compatibility**) | Authentication | — |
| GET | `/my/resources` · `/my/resources/search` | My resource list/search (all types) | `resources:read` | — |
| POST | `/my/resources/{id}/credentials/reveal` | ⚠️ Credential original query | `credentials:reveal` | **Plain text** |
- `/resources/{id}/credentials` (all types, STEP 8.7 new) and `/servers/{id}/credentials/ssh` (SERVER-only compatibility) have **same behavior and storage results**. The latter returns 409 for WEBSITE/API.
- Reveal responses are oneOf(SERVER/WEBSITE/API) with a discriminator. SERVER responses add only the `resource_type` key.
- Scope notation inconsistencies are as in the code: `servers.read` (dot) vs `resources:read`/`credentials:reveal` (colon).
**Web UI** (`/web/*`, HTML+Cookie, OpenAPI not exposed): login, dashboard, resource CRUD (type selection → type-specific form), credential registration/deletion (no reveal), API Key issuance/disposal. Details: [docs/web-ui.md](docs/web-ui.md).
## Authentication Methods
One `Authorization` header routes four types, all resolving to internal `users.id`. Client-sent `user_id`/`tenant_id`/`oid` is not trusted.
| Method | Header | Algorithm | User identification | Purpose · Active conditions |
|---|---|---|---|---|
| **LOCAL JWT** | `Bearer <JWT>` | HS256 | `sub` = internal user_id | Development login / Web UI. `local_auth_enabled` + issuer match |
| **Microsoft Entra** | `Bearer <JWT>` | RS256 (only, JWKS) | `(tid, oid)` mapping | In-house SSO. Bearer JWT default path |
| **API Key** | `ApiKey <dv_...>` (or `Bearer dv_...`) | HMAC-SHA256 hash | `key_hash` → user_id | Claude/CLI. Prefix `dv_dev_`/`dv_live_` |
| **DEV** | No header | — | `dev_auth_user_id` | Local convenience. `dev_auth_enabled` + `is_local` (prod fail-fast) |
- **Not stored**: plain text password (Argon2id hash only), Access/Refresh Token, **API Key original text** (exposed only once during issuance). Details: [docs/token-storage-policy.md](docs/token-storage-policy.md).
- LOCAL (HS256) and Entra (RS256) are separated by algorithm, issuer, and audience, with cross-verification prohibited.
- API Key scope: `resources:read`, `credentials:reveal`, `servers.read`, etc. Web users pass scope checks (scope concept is API Key-specific).
- Details: [docs/local-authentication.md](docs/local-authentication.md), [docs/entra-authentication.md](docs/entra-authentication.md), [docs/api-key-authentication.md](docs/api-key-authentication.md).
## Vault Paths and Deletion Policies
Logical paths (stored in DB `credential_ref`). **DB values are authoritative** — no regeneration during read/replacement/deletion.
| Type | Format |
|---|---|
| **New** (all types) | `users/{user_id}/resources/{resource_id}/credential` |
| **Legacy** (STEP 6~8 SERVER) | `users/{user_id}/servers/{server_resource_id}/ssh` |
KV v2 physical path = `{mount}/data/{credential_ref}`. VaultClient uses only endpoints allowed by App Token Policy:
| Operation | HTTP endpoint | Required capability |
|---|---|---|
| write / read | `POST/GET /v1/{mount}/data/{path}` | `data/*` create·read·update |
| metadata read | `GET /v1/{mount}/metadata/{path}` | `metadata/*` read |
| **Soft delete** | `POST /v1/{mount}/delete/{path}` | `delete/*` update |
| Undelete (compensation) | `POST /v1/{mount}/undelete/{path}` | `undelete/*` update |
> **Deletion is soft delete** (not destroy, recoverable). Policy does not grant `data/*` delete, so **`POST /delete/`** (version-specific, metadata current_version pre-query) is used — using former would result in 403 failure for all deletions. **Resource deletion also soft deletes Vault Secret**, and compensates with undelete if DB commit fails.
> Reference: [docs/vault-client.md](docs/vault-client.md), [docs/credential-storage-flow.md](docs/credential-storage-flow.md).
## Data Model
Dedicated PostgreSQL schema `vault_mcp` for isolation (shared DB `vault_mcp` public and other schemas are not used).
enum is non-native VARCHAR(32). Reference: [docs/data-model.md](docs/data-model.md).
| Table | Role |
|---|---|
| `users` | Identity. LOCAL(login_id+password_hash) / MICROSOFT(tenant_id+entra_oid) from two sources |
| `api_keys` | API Key metadata. Original text not stored (only key_hash/key_prefix) |
| `server_resources` | Registered Resource (SERVER/WEBSITE/API all). Name maintained for backward compatibility |
| `server_permissions` | Authorization unit: user × resource × operation. Default Deny, DENY priority |
| `vault_connections` | Vault connection metadata. Central Vault (system-owned) has owner NULL |
| `audit_logs` | Audit log. No Secret column |
Key constraints (`server_resources`):
- `uq_server_resources_owner_user_id_alias` — UNIQUE(owner_user_id, alias) **WHERE status ≠ 'DELETED'**(partial). Soft delete leaves the row, so DELETED is excluded to allow reusing deleted aliases (0006). → alias query filters out DELETED.
- `resource_type_allowed` — `resource_type IN ('SERVER','WEBSITE','API')`
- `auth_type_api_only` — `auth_type IS NULL OR resource_type = 'API'`(SshAuthType/ApiAuthType mixed use blocked)
**Migration history**(revision id is `alembic_version.version_num` VARCHAR(32) constraint):
| revision | content |
|---|---|
| `0001_initial` | Initial schema with 5 tables (vault_mcp schema, enum=VARCHAR(32)) |
| `0002_local_auth_and_api_keys` | LOCAL login fields + api_keys table, Entra fields made nullable |
| `0003_credential_ref_nullable` | `server_resources.credential_ref` made nullable (unregistered=NULL) |
| `0004_server_environment` | `environment` column added (DEV/STAGING/PROD indication) |
| `0005_resource_types` | `resource_type` + type-specific fields, host/ssh_port/service_name NOT NULL relaxed, existing rows backfilled as SERVER |
| `0006_alias_unique_active_only` | alias unique constraint replaced with partial index excluding DELETED |
> SQLite (for testing) does not support ALTER COLUMN / ADD CONSTRAINT → NOT NULL relaxation and CHECK addition are **only performed in PostgreSQL**, and SQLite creates the same final schema using model `create_all`.
---
## Quick Start
**Requirements:** Python 3.12+, [uv](https://docs.astral.sh/uv/). `/api/v1/health` works without DB/Vault.
```bash
# [Local PC / WSL·zsh]
uv sync # virtual environment + dependencies
cp .env.example .env # fill in values (see environment variables below)
uv run uvicorn app.main:app --reload # or: bash scripts/run-local.sh
```
- API documentation: `http://127.0.0.1:8000/docs` (only on local/dev)
- Web UI: `http://127.0.0.1:8000/web/login`
- PoC login account creation: seed script in `docs/poc-user-seed.md`
> LOCAL login is enabled only when `.env` has `LOCAL_AUTH_ENABLED=true` (default false).
> Credential registration/query requires Vault connection — prepare development Vault tunnel and App Token as described in [docs/vault-local-development.md](docs/vault-local-development.md).
---
## Environment Variables
Managed by `.env` (do not commit `.env`, only `.env.example`). Secret values are stored in **Vault or K8S Secret**, not in `.env`.
| prefix | example key | notes |
|---|---|---|
| `APP_*` | APP_ENV, APP_HOST, APP_PORT | `APP_ENV` = local/dev/test/production |
| `DB_*` | DB_HOST/PORT/NAME/USERNAME/**PASSWORD**/SCHEMA | shared DB `vault_mcp`, schema `vault_mcp` |
| `LOCAL_*` / `PASSWORD_*` | LOCAL_AUTH_ENABLED, **LOCAL_JWT_SECRET**, LOCAL_JWT_ISSUER | HS256 self-signed JWT |
| `API_KEY_*` | **API_KEY_HASH_SECRET**, API_KEY_MAX_ACTIVE_PER_USER | defaults to SHA-256 if not set |
| `ENTRA_*` | ENTRA_TENANT_ID, **ENTRA_CLIENT_SECRET**, ENTRA_AUDIENCE, ENTRA_ISSUER | Resource Server validation |
| `VAULT_*` | VAULT_ADDR, **VAULT_TOKEN** / VAULT_TOKEN_FILE, VAULT_KV_MOUNT | token or 0600 file |
| `DEV_AUTH_*` | DEV_AUTH_ENABLED, DEV_AUTH_USER_ID | local/dev only (prod fail-fast) |
| others | LOG_LEVEL, DOCS_ENABLED, CORS_ALLOWED_ORIGINS | |
**Bold** = `SecretStr`(not logged or exposed in responses): `DB_PASSWORD`, `LOCAL_JWT_SECRET`, `API_KEY_HASH_SECRET`, `ENTRA_CLIENT_SECRET`, `VAULT_TOKEN`.
---
## Testing and Quality Gates
```bash
uv run pytest # 493 passed, 7 skipped (skipped = real Vault integration tests)
uv run ruff check . # All checks passed
uv run ruff format --check .
uv run mypy app # strict, no issues (87 files)
```
- 32 test files. Security boundaries (access by other users, unauthorized operations, Secret exposure, alias forgery, Vault failure consistency) are tested.
- Real Vault integration tests are skipped by default. Run: `VAULT_MCP_INTEGRATION=1 VAULT_ADDR=... VAULT_TOKEN_FILE=... uv run pytest tests/vault/test_real_vault_integration.py`.
---
## Directory Structure
```
app/
main.py FastAPI app assembly + OpenAPI configuration
core/ config, exceptions, openapi, api_docs, logging, middleware, security
api/
dependencies.py auth/service dependency injection
v1/router.py, v1/endpoints/ health, auth, me, api_keys, servers,
credentials(SSH backward compatible), resource_credentials(all types),
my_resources(list/search/reveal)
internal/ dev-only router (only registered on local/dev)
auth/ verifier(Entra), local_token/local_service, dependencies, context, jwks, password
api_keys/ generator, hasher, service, repository, schemas
credentials/ service, compensation, validation, schemas, errors
server_resources/ crud, service(alias resolution/authorization), repository, validation, schemas
vault/ client(KV v2), factory, token_file, credential_ref, errors
db/ base(schema isolation), session, models/{user,api_key,server_resource,...}
domain/ enums, errors, resolved
audit/ service(audit log)
web/ router(Jinja2), csrf, dependencies
templates/ static/{css,js} Web UI
scripts/seed_poc_user.py PoC user seed
ssh/ operations/ SSH Runner·operational API placeholder (STEP 10~11, not implemented)
migrations/versions/ Alembic 0001~0006
tests/ api, api_keys, auth, services, vault, web, skills, db, models
skills/vault-mcp/ company-wide common Claude Skill(REST call, Secret not output)
deploy/ gitops-repo(Kustomize+ArgoCD), k8s/dev(manual kubectl alternative)
docs/ design/operational documentation (below index)
Dockerfile Jenkinsfile multi-stage image / CI pipeline
```
---
## Development Stages
| STEP | content | status |
|---|---|:--:|
| 1 | FastAPI skeleton | ✅ |
| 2 | DB model/authorization/server resolution (Alembic, schema isolation) | ✅ |
| 3 | Microsoft Entra authentication | ✅ |
| 4 | LOCAL login + API Key + user-specific Vault path model | ✅ |
| 5 | development Vault installation/standalone validation (NCP) | ✅ |
| 6 | FastAPI ↔ Vault integration + Credential registration | ✅ |
| 7 | Skill-based REST API PoC | ✅ |
| 8 | user Web UI(Jinja2) | ✅ |
| 8.5 / 8.6 | development K8S deployment / Jenkins+Harbor+ArgoCD GitOps | ✅ |
| **8.7** | **Resource type extension(SERVER/WEBSITE/API)** | ✅ |
| 9 | user authentication improvement(Device Flow/short Access Token) | planned |
| 10 | SSH Runner etc. Credential utilization | not implemented |
| 11 / 12 / 13 | read-only operational API / CLI+Keyring / K8S deployment refinement | not implemented |
**Not implemented yet:** SSH execution(Runner), operational API(logs/status/disk), CLI/OS Keyring. Entra actual App Registration integration verification required in organization environment.
---
## Deployment
Jenkins(CI) → Harbor(registry) → ArgoCD(GitOps). Reference: [docs/ci-gitops.md](docs/ci-gitops.md), [deploy/gitops-repo/README.md](deploy/gitops-repo/README.md).
```mermaid
flowchart LR
GIT[code push] -->|pollSCM| J[Jenkins]
J -->|build| IMG[image sha-tag]
IMG -->|push| H[Harbor]
J -->|newTag update commit| D[vault-mcp-deploy repo]
D -->|watch| A[ArgoCD]
A -->|PreSync: migration Job| K[K8S vault-mcp ns]
A -->|sync: Deployment| K
```
- **Image:** `Dockerfile` multi-stage, non-root(uid 10001), uvicorn :8000. No Secret included (K8S Secret injection).
- **Jenkinsfile:** Init(short SHA) → Build → Push Harbor → deploy repo's `newTag` updated to `sha-<SHA>`. Deployment and migration Job reflect **same SHA** (mismatch prevention).
- **GitOps:** `deploy/gitops-repo/`(Kustomize + ArgoCD Application). migration is PreSync Hook, seed is one-time manual.
- `deploy/k8s/dev/` is an alternative for **manual kubectl apply verification** without GitOps.
---
## Technology Stack
| area | stack |
|---|---|
| language | Python 3.12+ |
| web | FastAPI ≥0.115, Uvicorn ≥0.34 |
| validation | Pydantic v2 ≥2.9, pydantic-settings ≥2.6 |
| DB | SQLAlchemy 2.x async, asyncpg ≥0.30, Alembic ≥1.14, PostgreSQL |
| authentication | PyJWT[crypto] ≥2.9, pwdlib[argon2] ≥0.2 |
| Vault/HTTP | httpx ≥0.28 (KV v2 direct call) |
| web | Jinja2 ≥3.1, python-multipart |
| development | pytest + pytest-asyncio, aiosqlite, ruff(line 100, py312), mypy(strict) |
| package | uv(non-package application mode) |
## Document Index Topic | Document |
|---|---|
| **Overview** | [architecture.md](docs/architecture.md) · [security-principles.md](docs/security-principles.md) · [development-roadmap.md](docs/development-roadmap.md) |
| **Resource / Data** | [resource-types.md](docs/resource-types.md) · [data-model.md](docs/data-model.md) · [authorization-model.md](docs/authorization-model.md) · [server-resolution-flow.md](docs/server-resolution-flow.md) |
| **Authentication** | [local-authentication.md](docs/local-authentication.md) · [entra-authentication.md](docs/entra-authentication.md) · [entra-setup-guide.md](docs/entra-setup-guide.md) · [api-key-authentication.md](docs/api-key-authentication.md) · [token-storage-policy.md](docs/token-storage-policy.md) |
| **Vault / Credential** | [vault-client.md](docs/vault-client.md) · [vault-architecture-and-key-management.md](docs/vault-architecture-and-key-management.md) · [user-owned-vault-layout.md](docs/user-owned-vault-layout.md) · [credential-storage-flow.md](docs/credential-storage-flow.md) · [credential-reveal-api.md](docs/credential-reveal-api.md) · [vault-error-handling.md](docs/vault-error-handling.md) · [vault-local-development.md](docs/vault-local-development.md) · [vault-production-readiness.md](docs/vault-production-readiness.md) |
| **API / Swagger** | [swagger-guide.md](docs/swagger-guide.md) · [skill-rest-api-poc.md](docs/skill-rest-api-poc.md) · [api-key-auth-flow.md](docs/api-key-auth-flow.md) |
| **Web UI** | [web-ui.md](docs/web-ui.md) · [web-authentication.md](docs/web-authentication.md) · [api-key-web-flow.md](docs/api-key-web-flow.md) · [poc-user-seed.md](docs/poc-user-seed.md) |
| **Skill Deployment** | [cowork-skill-deployment.md](docs/cowork-skill-deployment.md) · [security-model.md](docs/security-model.md) |
| **Deployment** | [ci-gitops.md](docs/ci-gitops.md) · [deploy/gitops-repo/README.md](deploy/gitops-repo/README.md) · [deploy/k8s/dev/README.md](deploy/k8s/dev/README.md) |
| **Work Log** | [docs/work-log/](docs/work-log/) |
---
Git Policy: Default branch`, no commit/push without explicit request. Do not commit `.env`, Token, certificates, or Private Keys. Project principles and work harness are in [CLAUDE.md](CLAUDE.md).
