Content
# ☁️ OpsYield — Multi-Cloud FinOps MCP Server
<!-- GitHub Topics: finops, mcp, multi-cloud, cloud-cost-optimization, devops, ai-agent, kubernetes, opencost, redis, postgresql, rag -->
<p align="center">
<img src="https://img.shields.io/badge/Python-3.11%2B-blue?logo=python&logoColor=white" alt="Python"/>
<img src="https://img.shields.io/badge/MCP-Model%20Context%20Protocol-purple" alt="MCP"/>
<img src="https://img.shields.io/badge/Clouds-AWS%20%7C%20GCP%20%7C%20Azure%20%7C%20K8s-orange" alt="Clouds"/>
<img src="https://img.shields.io/badge/Cache-Redis-red?logo=redis&logoColor=white" alt="Redis"/>
<img src="https://img.shields.io/badge/DB-PostgreSQL-336791?logo=postgresql&logoColor=white" alt="PostgreSQL"/>
<img src="https://img.shields.io/badge/RAG-ChromaDB-green" alt="RAG"/>
<img src="https://img.shields.io/badge/License-MIT-lightgrey" alt="MIT License"/>
</p>
<p align="center">
<em>Ask Claude "What is my most expensive AWS service?" and get a real answer — backed by live cloud APIs, Redis caching, PostgreSQL history, and semantic RAG search.</em>
</p>
---
## Table of Contents
- [What It Does](#-what-it-does)
- [Key Features](#-key-features)
- [Architecture](#-architecture)
- [Quick Start](#-quick-start)
- [Environment Configuration](#-environment-configuration)
- [Claude Desktop Setup](#-claude-desktop-setup)
- [MCP Tool Reference](#-mcp-tool-reference)
- [Example Queries](#-example-queries)
- [Project Structure](#-project-structure)
- [Kubernetes via OpenCost](#-kubernetes-via-opencost)
- [Running the REST API](#-running-the-rest-api)
- [Development](#-development)
- [Contributing](#-contributing)
- [License](#-license)
---
## 🔍 What It Does
**OpsYield** is a production-grade MCP server that gives AI agents (Claude Desktop, Claude CLI) direct access to your cloud billing and infrastructure data.
Ask Claude natural language questions. OpsYield handles everything underneath:
- Fetches live data from **AWS Cost Explorer**, **GCP BigQuery Billing Export**, **Azure Cost Management**, and **OpenCost**
- Caches responses in **Redis** (sub-millisecond hits, 7000× faster than live API calls)
- Persists every snapshot to **PostgreSQL** for historical trend queries
- Indexes all data in a **ChromaDB vector store** for semantic RAG search — ask "What spiked my costs in January?" without touching the cloud API
> [!WARNING]
> This tool queries live billing APIs. High-frequency polling may incur API costs depending on your provider's pricing tier (GCP BigQuery, AWS Cost Explorer).
---
## ✨ Key Features
| Feature | Detail |
|---|---|
| **Multi-Cloud** | AWS, GCP, Azure, Kubernetes (OpenCost) from a single interface |
| **Redis Cache** | 6-hour TTL for billing, 30-min for infra — automatic file fallback if Redis is down |
| **PostgreSQL Persistence** | Every billing snapshot + per-service cost records stored, queryable without cloud API |
| **RAG / Semantic Search** | ChromaDB + `all-MiniLM-L6-v2` embeddings — ask historical questions in plain English |
| **Zero-Config Fallbacks** | Redis down → file cache. PostgreSQL down → skip silently. Nothing breaks. |
| **MCP Native** | `stdio` and `SSE` transport — works with Claude Desktop, Claude CLI, any MCP client |
| **AI-Queryable History** | `ask_finops("When did Redshift costs spike?")` — answered from local vector store, no API call |
| **Idle Resource Detection** | Finds unattached disks, idle load balancers, orphaned IPs |
| **Rightsizing** | Down-scaling suggestions from trailing utilization metrics |
---
## 🏗️ Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ Claude Desktop / CLI │
│ (MCP Client — natural language) │
└────────────────────────┬────────────────────────────────────┘
│ MCP Protocol (stdio / SSE)
┌────────────────────────▼────────────────────────────────────┐
│ OpsYield MCP Server │
│ mcp_stdio.py │
│ │
│ ┌──────────┐ ┌────────────┐ ┌────────────────────────┐ │
│ │ Redis │ │ PostgreSQL │ │ ChromaDB (RAG) │ │
│ │ Cache │ │ History │ │ Vector Store │ │
│ │ 6h / 30m │ │ 3 tables │ │ all-MiniLM-L6-v2 │ │
│ └──────────┘ └────────────┘ └────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Cloud Providers │ │
│ │ AWS Cost Explorer │ GCP BigQuery │ Azure Cost Mgmt │ │
│ │ Kubernetes / OpenCost │ │
│ └─────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
```
**Data flow on every tool call:**
```
Request → Redis HIT? ──yes──→ return in <1ms
│
no
↓
Cloud API call (~5-10s)
↓
┌────┴─────────────┐
│ Redis SET (TTL) │
│ PostgreSQL INSERT│
│ ChromaDB UPSERT │
└──────────────────┘
↓
Return to Claude
```
---
## 🚀 Quick Start
```bash
# 1. Clone and install
git clone https://github.com/Moiz-Ali-Moomin/mcp-cloud-finops-ai-agent.git
cd mcp-cloud-finops-ai-agent
pip install -e .
# 2. (Optional but recommended) Start Redis and PostgreSQL
# Redis: redis-server
# PostgreSQL: createdb opsyield
# Both are optional — the server falls back gracefully without them.
# 3. Set your cloud credentials
export GOOGLE_CLOUD_PROJECT=my-gcp-project # GCP
export AWS_PROFILE=default # AWS
export REDIS_URL=redis://localhost:6379 # Cache
export DATABASE_URL=postgresql://localhost/opsyield # History
# 4. Run the MCP server
opsyield-mcp
```
The server starts silently and waits for MCP client connections. Add it to Claude Desktop (see below) and start asking questions.
---
## ⚙️ Environment Configuration
### 🔵 GCP
| Variable | Required | Description |
|---|---|---|
| `GOOGLE_CLOUD_PROJECT` | Yes | GCP project ID |
| `GOOGLE_APPLICATION_CREDENTIALS` | No | Path to service account JSON. Omit to use `gcloud` ADC |
| `CLOUDSDK_CONFIG` | No | Path to gcloud config dir (default: `~/.config/gcloud`) |
| `NO_GCE_CHECK` | Recommended | Set `true` to skip GCE metadata probe (prevents 30s hang outside GCE) |
Required IAM roles: `BigQuery Data Viewer`, `BigQuery Job User`, `Compute Viewer`. Billing export to BigQuery must be enabled.
### 🟠 AWS
| Variable | Required | Description |
|---|---|---|
| `AWS_PROFILE` | One of these | Named profile from `~/.aws/credentials` |
| `AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY` | One of these | Direct key auth |
| `AWS_DEFAULT_REGION` | Yes | e.g. `us-east-1` |
Required IAM permissions: `ce:GetCostAndUsage`, `ec2:Describe*`, `s3:ListAllMyBuckets`, `rds:Describe*`.
### ⚪ Azure
| Variable | Required | Description |
|---|---|---|
| `AZURE_CLIENT_ID` | Yes | Service principal app ID |
| `AZURE_CLIENT_SECRET` | Yes | Service principal secret |
| `AZURE_TENANT_ID` | Yes | Azure AD tenant ID |
| `AZURE_SUBSCRIPTION_ID` | Yes | Target subscription |
Required role: `Cost Management Reader`.
### 🔴 Redis (Cache)
| Variable | Default | Description |
|---|---|---|
| `REDIS_URL` | `redis://localhost:6379` | Redis connection URL. Supports `redis://:password@host:port/db` |
Falls back to file cache at `~/.opsyield/cache/` automatically if Redis is unreachable.
### 🐘 PostgreSQL (History)
| Variable | Default | Description |
|---|---|---|
| `DATABASE_URL` | `postgresql://postgres:@localhost:5432/opsyield` | Full DSN |
| `PGHOST` / `PGPORT` / `PGDATABASE` / `PGUSER` / `PGPASSWORD` | — | Alternative to `DATABASE_URL` |
The `opsyield` database must exist (`CREATE DATABASE opsyield`). All tables are created automatically on first run.
### ☸️ Kubernetes
| Variable | Default | Description |
|---|---|---|
| `OPENCOST_URL` | `http://localhost:9003` | OpenCost REST API endpoint |
---
## 🖥️ Claude Desktop Setup
Add to your Claude Desktop config file:
**Windows:** `%APPDATA%\Claude\claude_desktop_config.json`
**Mac:** `~/Library/Application Support/Claude/claude_desktop_config.json`
```json
{
"mcpServers": {
"opsyield-finops": {
"command": "/path/to/python",
"args": ["/path/to/mcp-cloud-finops-ai-agents/opsyield/mcp_stdio.py"],
"env": {
"PYTHONPATH": "/path/to/mcp-cloud-finops-ai-agents",
"PYTHONDONTWRITEBYTECODE": "1",
"NO_GCE_CHECK": "true",
"GOOGLE_CLOUD_PROJECT": "your-gcp-project-id",
"AWS_PROFILE": "default",
"AWS_DEFAULT_REGION": "us-east-1",
"REDIS_URL": "redis://localhost:6379",
"DATABASE_URL": "postgresql://postgres:password@localhost:5432/opsyield"
}
}
}
}
```
> Ready-to-use templates (Windows, Mac, annotated reference) are in [`examples/`](examples/).
Restart Claude Desktop after saving. You should see the OpsYield tools listed in the MCP tools panel.
---
## 🛠️ MCP Tool Reference
| Tool | Description |
|---|---|
| `configure_project(project_id)` | Set active GCP project for all subsequent calls |
| `get_billing_costs(provider, days, project_id, force_refresh)` | Fast billing costs — Redis cached, PostgreSQL persisted, RAG indexed |
| `get_infrastructure(provider, project_id, force_refresh)` | Cloud resources — VMs, disks, DBs, buckets |
| `run_finops_intelligence(provider, days, project_id, force_refresh)` | Full analysis — billing + infra combined |
| `aggregate_finops(providers, days, project_id)` | Combined view across multiple providers (e.g. `"gcp,aws"`) |
| `ask_finops(question, provider, project_id)` | **RAG semantic search** — natural language Q&A over historical data |
| `get_cost_history(provider, project_id, limit)` | Query past snapshots from PostgreSQL — no cloud API call |
| `get_top_services_db(provider, project_id, limit)` | Accumulated top services from PostgreSQL across all snapshots |
| `db_stats()` | PostgreSQL row counts for all tables |
| `rag_store_info()` | ChromaDB vector store stats — chunks stored, persist dir |
| `cache_info()` | Redis/file cache stats — backend, entries, memory |
| `cache_clear(confirm)` | Flush all cached responses |
All data tools support `force_refresh=true` to bypass cache and re-fetch live data.
---
## 💬 Example Queries
Once connected to Claude Desktop, try these:
**Cost analysis:**
- *"Show me AWS costs for the last 30 days grouped by service"*
- *"What is my GCP spend for project ecommerce-microservice-53 over the last 90 days?"*
- *"Which cloud is more expensive — AWS or GCP?"*
**RAG historical queries (no API call — instant):**
- *"What was my most expensive AWS service last month?"*
- *"When did GCP Compute Engine costs spike?"*
- *"Compare Kubernetes Engine costs between 30-day and 90-day windows"*
**Infrastructure:**
- *"List all running EC2 instances and their estimated monthly cost"*
- *"Find idle GCP resources — unattached disks, stopped VMs"*
**Database queries (no API call):**
- *"Show me the last 10 billing snapshots for AWS from the database"*
- *"What are the top 5 accumulated AWS services across all my historical data?"*
**Optimization:**
- *"Are there rightsizing recommendations for my EC2 environment?"*
- *"What Kubernetes namespaces are consuming the most cost via OpenCost?"*
---
## 📂 Project Structure
```
mcp-cloud-finops-ai-agents/
│
├── opsyield/
│ ├── mcp_stdio.py # MCP server entry point — all tools defined here
│ ├── mcp_sse.py # SSE transport variant
│ ├── mcp_entry.py # CLI entry point (opsyield-mcp command)
│ │
│ ├── core/
│ │ ├── cache.py # Redis cache (+ file fallback)
│ │ ├── models.py # NormalizedCost, Resource, AnalysisResult
│ │ ├── orchestrator.py # Dispatches provider calls, aggregates results
│ │ ├── aggregation.py # Multi-provider merge logic
│ │ ├── snapshot.py # CI/CD snapshot diff (cost regression guard)
│ │ ├── config.py # Environment variable loading
│ │ ├── context.py # Per-session GCP project context
│ │ └── logging.py # Structured JSON logging
│ │
│ ├── db/
│ │ ├── connection.py # asyncpg pool + auto schema migration
│ │ └── repository.py # save/query billing snapshots, cost records, infra
│ │
│ ├── rag/
│ │ ├── store.py # ChromaDB persistent vector store
│ │ ├── ingester.py # Billing data → text chunks → embeddings
│ │ └── query_engine.py # Semantic search + metadata filtering
│ │
│ ├── providers/ # Provider factory + per-cloud implementations
│ │ ├── factory.py
│ │ ├── base.py
│ │ ├── gcp.py
│ │ ├── aws.py
│ │ ├── azure.py
│ │ ├── kubernetes.py
│ │ └── cli_utils.py
│ │
│ ├── collectors/ # Low-level API collectors (compute, storage, SQL, metrics)
│ │ ├── base.py
│ │ ├── gcp/
│ │ ├── aws/
│ │ ├── azure/
│ │ └── kubernetes/
│ │
│ ├── billing/ # Billing export clients (BigQuery, Cost Explorer, etc.)
│ │ ├── base.py
│ │ ├── gcp.py
│ │ ├── aws.py
│ │ └── azure.py
│ │
│ ├── analysis/ # Heuristic engines
│ │ ├── cost_analyzer.py
│ │ ├── waste_detector.py
│ │ ├── rightsizer.py
│ │ ├── idle_scoring.py
│ │ ├── savings.py
│ │ └── recommendations.py
│ │
│ ├── intelligence/ # Cross-provider analytics
│ │ └── analytics.py
│ │
│ ├── optimization/ # Cost optimization strategies
│ │ └── strategies.py
│ │
│ ├── utils/ # Shared helpers
│ │ └── helpers.py
│ │
│ └── api/ # FastAPI REST server (optional)
│ ├── main.py
│ ├── server.py
│ └── adapters/
│ └── analysis_adapter.py
│
├── examples/
│ ├── claude_desktop_config.json # Fully annotated reference config
│ ├── claude_desktop_config.windows.json # Windows paste-ready template
│ ├── claude_desktop_config.mac.json # Mac/Linux paste-ready template
│ ├── claude_desktop_config.local.json # Your machine (pre-filled paths)
│ ├── docker-compose.yml
│ └── sample_queries.md
│
├── docs/
│ ├── architecture.md
│ ├── providers.md
│ ├── collectors.md
│ ├── development.md
│ └── kubernetes-costs.md
│
├── .github/workflows/python-ci.yml
├── Dockerfile
├── Makefile
├── requirements.txt
├── pyproject.toml
└── README.md
```
---
## ☸️ Kubernetes via OpenCost
OpsYield treats Kubernetes as a first-class cloud provider. It integrates with the [OpenCost](https://www.opencost.io/) REST API to normalize container spend (cost-per-namespace, cost-per-workload) into the same FinOps engine used for AWS/GCP/Azure.
```bash
# Install OpenCost in your cluster
kubectl apply --server-side -f https://raw.githubusercontent.com/opencost/opencost/develop/kubernetes/opencost.yaml
# Forward the port
kubectl port-forward -n opencost service/opencost 9003:9003
# Set the env var
export OPENCOST_URL=http://localhost:9003
```
Ask Claude: *"What are my Kubernetes costs by namespace this week?"*
---
## 🌐 Running the REST API
OpsYield also runs as a standalone FastAPI server for non-MCP integrations:
```bash
uvicorn opsyield.api.main:app --host 0.0.0.0 --port 8000
```
- Swagger UI: `http://localhost:8000/docs`
- SSE MCP endpoint: `http://localhost:8000/mcp`
---
## 💻 Development
```bash
# Clone and set up dev environment
git clone https://github.com/Moiz-Ali-Moomin/mcp-cloud-finops-ai-agent.git
cd mcp-cloud-finops-ai-agent
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install -e ".[dev]"
# Run tests
pytest opsyield/tests/
# Lint and format
ruff check .
black .
```
### Adding a new cloud provider
1. Create `opsyield/providers/myprovider.py` extending `opsyield.providers.base.BaseProvider`
2. Add a collector in `opsyield/collectors/myprovider/`
3. Add a billing client in `opsyield/billing/myprovider.py`
4. Register in `opsyield/providers/factory.py`
---
## 🤝 Contributing
Contributions are welcome. Please:
1. Fork the repo and create a feature branch
2. Run `ruff check .` and `black .` before submitting
3. Add or update tests in `opsyield/tests/`
4. Open a PR with a clear description of what changed and why
---
## 📜 License
MIT License — see [`LICENSE`](LICENSE) for details.
---
<p align="center">
Built by <a href="https://github.com/Moiz-Ali-Moomin">Moiz Ali Moomin</a> · Star ⭐ if this saved you money
</p>
