argus

<p align="center"> <img src="https://em-content.zobj.net/source/apple/391/eye_1f441-fe0f.png" width="110" alt="Argus" /> </p> <h1 align="center">Argus</h1> <p align="center"> <strong>Own the agent loop, not the framework.</strong> </p> <p align="center"> A framework-free, multi-agent deep-research engine — a planner fans out parallel<br/> agents over the web and your documents, and a synthesizer writes a cited answer. </p> <p align="center"> <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue?style=flat" alt="License: MIT"></a> <img src="https://img.shields.io/badge/python-3.12%2B-blue?style=flat" alt="Python 3.12+"> <img src="https://img.shields.io/badge/tests-150%20passing-brightgreen?style=flat" alt="Tests: 150 passing"> <img src="https://img.shields.io/badge/local--first-Ollama-orange?style=flat" alt="Local-first"> </p> <!-- Once the repo is PUBLIC, add the live GitHub badges below (they show "repo not found" while private): <p align="center"> <a href="https://github.com/subratamondal1/argus/actions/workflows/ci.yml"><img src="https://github.com/subratamondal1/argus/actions/workflows/ci.yml/badge.svg" alt="CI"></a> <a href="https://github.com/subratamondal1/argus/stargazers"><img src="https://img.shields.io/github/stars/subratamondal1/argus?style=flat&color=yellow" alt="Stars"></a> <a href="https://github.com/subratamondal1/argus/commits/main"><img src="https://img.shields.io/github/last-commit/subratamondal1/argus?style=flat" alt="Last commit"></a> </p> --> <p align="center"> <a href="#features">Features</a> • <a href="#quickstart">Quickstart</a> • <a href="#how-it-works">How it works</a> • <a href="#eval-gate">Eval</a> • <a href="#configuration">Configuration</a> • <a href="#development">Development</a> </p> --- Argus answers a hard question the way a research team would: a **planner** breaks it into sub-questions, a fan-out of hand-written **searcher agents** researches them in parallel over the live web and a local document corpus, and a **synthesizer** writes a cited answer. Retrieval quality is enforced by an **eval gate** that blocks regressions, the searcher fan-out **autoscales from zero** on Kubernetes, and a run can be made **crash-resumable**. Built on Python 3.12 · LiteLLM · PostgreSQL + pgvector · FastAPI · Next.js — with **no agent framework** (no LangChain/LangGraph): the loop, the budget, and the failure handling are owned directly. **Local-first** — it runs at zero cost on Ollama; OpenAI and Anthropic are optional drop-ins. <p align="center"> <img src="docs/assets/argus-demo.gif" width="100%" alt="Argus deep-research run: a planner decomposes the question into four sub-questions, four searcher agents run in parallel, and a synthesizer writes a cited answer — streamed live." /> </p> <p align="center"> <em>One <strong>Deep research</strong> run, end to end: the planner decomposes the question, four searcher agents fan out in parallel, and the synthesizer streams a cited answer — running locally on Ollama at zero cost.</em> </p> ## Features | Capability | Detail | |---|---| | **Framework-free agent loop** | Hand-written tool-use loop over LiteLLM with a 3-axis budget (turns / tokens / wall-clock) + hard cost cap, a retry/fallback ladder, and a self-registering, permission-gated tool registry. | | **Multi-agent orchestration** | Planner → parallel searcher agents (isolated context each) → synthesizer → reflect/replan. | | **Contextual RAG** | Anthropic-style contextual chunking; hybrid dense-HNSW + lexical-FTS retrieval fused with Reciprocal Rank Fusion; optional `bge-reranker-v2-m3` cross-encoder — all on a single pgvector store. | | **Eval gate** | A curated, source-grounded benchmark with RAGAS-style metrics (context precision/recall, faithfulness, answer relevancy) + a Cohen's-κ-calibrated LLM judge; fails the build below committed thresholds. | | **Horizontal scale** | Searcher fan-out on an ARQ-on-Redis queue; Kubernetes + KEDA scale searcher pods from zero on queue depth. | | **Durable execution** | Opt-in DBOS workflows — a crashed research run resumes from its last checkpointed step (Postgres-backed). | | **MCP server** | The tool registry exposed over the Model Context Protocol (`argus mcp`) for any MCP host. | | **Multi-tenant + auth** | Email/password → argon2id + HS256 JWT in an httpOnly cookie with signed double-submit CSRF; per-tenant data isolation. | | **Streaming UI** | FastAPI Server-Sent Events streaming live multi-agent progress to a Next.js 16 / React 19 client. | | **Sandboxed code execution** | `execute_python` runs model-generated code in a subprocess sandbox (rlimits, timeout, no network) behind a permission gate. | ## Quickstart ```bash # 1. Install (uv manages the Python 3.12 toolchain and the venv). uv sync # 2. Start the local backing stack (Postgres + pgvector, SearXNG). make up # 3. Run the LLM and embeddings locally on Ollama — zero cost, no API key. ollama pull qwen2.5:3b && ollama pull nomic-embed-text # 4. Ask. uv run argus "What changed in the EU AI Act timeline in 2026?" # or: make ask Q="..." ``` `cp .env.example .env` first if you want to override defaults. To use a hosted model, set `OPENAI_API_KEY` and `ARGUS_MODEL=openai/...` in `.env` (see [Configuration](#configuration)). Stack controls: `make status` / `make down`. ## How it works ``` question │ planner (LLM) ▼ sub-questions ──► searcher agent ─┐ each: own tool-use loop + budget, ──► searcher agent ─┤ rag_search over the corpus + web_search, ──► searcher agent ─┘ run in parallel (asyncio / ARQ+KEDA) │ findings ▼ synthesizer (LLM) ──► reflect/replan ──► cited answer ``` Every LLM call is structured-logged and cost-attributed; the agent loop stops on the first of its turn/token/wall-clock/cost limits. The RAG path ingests documents with LLM-written contextual prefixes, embeds them (locally on Ollama), and indexes for both dense (HNSW) and lexical (full-text) search; a query fuses the two with Reciprocal Rank Fusion. Design decisions are recorded as ADRs in [`docs/adr/`](docs/adr/) (e.g. [why a single pgvector store](docs/adr/0001-datastore-postgres-pgvector.md)). ## Eval gate Retrieval quality is a number, not a vibe. `argus eval` ingests a curated RAG/vector-search corpus ([`eval/corpus/`](eval/corpus/)), runs a committed golden set ([`eval/golden.jsonl`](eval/golden.jsonl)) — 48 questions including negative/unanswerable cases — through real retrieval and a judged agent answer, and exits non-zero when any metric falls below [`eval/thresholds.json`](eval/thresholds.json). ```bash make eval # the gate (corpus-only; reports the metrics below) make eval-calibrate # prove the judge agrees with humans (Cohen's κ ≥ floor) ``` Latest run (`make eval`, 48-item benchmark, corpus-only): | Metric (RAGAS vocabulary) | gpt-5.4-mini | qwen3.5:4b (local) | qwen2.5:3b (local) | threshold | |---|---|---|---|---| | context_recall (hit@k) | 1.000 | 0.976 | 0.976 | ≥ 0.80 | | context_precision | 0.662 | 0.676 | 0.624 | ≥ 0.20 | | mrr | 0.948 | 0.952 | 0.952 | ≥ 0.60 | | faithfulness | 0.976 | 0.976 | 1.000 | ≥ 0.70 | | answer_relevancy | 1.000 | 0.976 | 0.690 | — | | judge_pass_rate | 0.976 | 0.643 | 0.000 | ≥ 0.70 | | keyword_pass_rate | 0.881 | 0.833 | 0.190 | ≥ 0.60 | | abstention_rate (negatives declined) | 1.000 | 0.833 | 0.000 | ≥ 0.70 | | **Gate** | **PASS** | FAIL (1 metric, by 0.057) | FAIL | | Metrics are implemented in-repo ([`eval/`](src/argus/eval/)), not via RAGAS, to stay dependency-light and offline. Retrieval and generation signals are gated independently, and negatives are scored by abstention (a faithful system must decline what the corpus can't answer). The thresholds are tuned for a frontier model: the hosted gpt-5.4-mini clears the gate, while a **fully-local $0 stack** (qwen3.5:4b answer with thinking disabled, nomic embeddings, a non-reasoning qwen2.5:7b judge at κ=1.0) passes seven of eight metrics and misses only `judge_pass_rate` by 0.057 — one notch of reasoning short, with retrieval/faithfulness/abstention all green. The qwen2.5:3b column shows where a smaller model collapses (it fabricates on every negative: abstention 0.000). Getting a local reasoning model through the gate took disabling Qwen3's default thinking (`reasoning_effort="disable"` → Ollama `think:false`, ~84s→~2s/call) while keeping it on for structured judge output, which Ollama drops when thinking is off. ## Document ingestion (RAG) ```bash ollama pull nomic-embed-text # one-time, local embeddings uv run argus ingest ./notes/architecture.md # a file uv run argus ingest https://example.com/post # or a URL uv run argus --deep "How does our system handle retries?" ``` Embeddings and rerank run locally, so document text never leaves the machine. PDF/DOCX/PPTX ingest needs `uv sync --extra parse`; the cross-encoder rerank stage needs `uv sync --extra rerank` and `ARGUS_RERANK_ENABLED=true`. ## Web UI ```bash make web-install # first time only: install frontend deps (bun) make web # FastAPI on :8000 + Next.js on :3000 → http://localhost:3000 ``` The UI streams the multi-agent flow live (plan → parallel search → tool calls → synthesize → reflect) and renders a cited Markdown answer, with document upload and a deep-research toggle. The backend is standalone — `make serve` runs the API alone, and the CLI works without the UI. ## Configuration Settings are read by [Pydantic Settings](src/argus/config.py) from the environment (`ARGUS_` prefix) or `.env`. The defaults are fully local and free. Common overrides: | Variable | Default | Purpose | |---|---|---| | `ARGUS_MODEL` | `ollama_chat/qwen2.5:3b` | Agent / contextualization / judge LLM (`openai/...` for hosted). | | `ARGUS_EMBEDDING_MODEL` | `ollama/nomic-embed-text` | Embedding model (768-d; must match the column). | | `ARGUS_USE_QUEUE` | `false` | Fan searchers onto the ARQ-on-Redis queue (KEDA-autoscalable). | | `ARGUS_USE_DURABLE` | `false` | Run deep research as a crash-resumable DBOS workflow (`--extra durable`). | | `ARGUS_RERANK_ENABLED` | `false` | Enable the cross-encoder rerank stage (`--extra rerank`). | Optional extras: `parse` (document parsing), `rerank` (cross-encoder), `otel` (OpenTelemetry), `durable` (DBOS), `mcp` (MCP server). ## Development ```bash make ci # format-check + lint (ruff) + typecheck (ty) + tests (pytest) make test # tests only — hermetic (LLM, DB, and search are faked/marker-gated) make mcp # run the tool registry as an MCP server over stdio (or: argus mcp) ``` CI runs the hermetic suite plus a Postgres + Redis integration job and a kind + KEDA autoscaling smoke on every push. Integration tests are behind a `pytest -m integration` marker so the default suite needs no services. <!-- Uncomment once the repo has stars (renders an empty chart at 0): ## Star history <p align="center"> <a href="https://star-history.com/#subratamondal1/argus&Date"> <img src="https://api.star-history.com/svg?repos=subratamondal1/argus&type=Date" width="600" alt="Star history"> </a> </p> --> ## License MIT — see [LICENSE](LICENSE).