ai-counsel

<p align="center"> <img src="assets/ai-counsel.png" alt="AI Counsel Logo" width="400"> </p> # AI Counsel [](https://smithery.ai/skills?ns=blueman82&utm_source=github&utm_medium=badge) True deliberative consensus MCP server where AI models debate and refine positions across multiple rounds.      ## 🎬 See It In Action **Cloud Models Debate** (Claude Sonnet, GPT-5.1 Codex, Gemini): ```javascript mcp__ai-counsel__deliberate({ question: "Should we use REST or GraphQL for our new API?", participants: [ {cli: "claude", model: "claude-sonnet-4-5-20250929"}, {cli: "codex", model: "gpt-5.2-codex"}, {cli: "gemini", model: "gemini-2.5-pro"} ], mode: "conference", rounds: 3 }) ``` **Result**: Converged on hybrid architecture (0.82-0.95 confidence) • [View full transcript](transcripts/20251030_153509_Should_we_use_REST_or_GraphQL_for_our_new_API_Con.md) **Local Models Debate** (100% private, zero API costs): ```javascript mcp__ai-counsel__deliberate({ question: "Should we prioritize code quality or delivery speed?", participants: [ {cli: "ollama", model: "llama3.1:8b"}, {cli: "ollama", model: "mistral:7b"}, {cli: "ollama", model: "deepseek-r1:8b"} ], mode: "conference", rounds: 2 }) ``` **Result**: 2 models switched positions after Round 1 debate • [View full transcript](transcripts/20251030_153834_Should_we_prioritize_code_quality_or_delivery_spee.md) --- ## What Makes This Different **AI Counsel enables TRUE deliberative consensus** where models see each other's responses and refine positions across multiple rounds: - Models engage in actual debate (see and respond to each other) - Multi-round convergence with voting and confidence levels - Full audit trail with AI-generated summaries - Automatic early stopping when consensus reaches (saves API costs) ## Features - 🎯 **Two Modes**: `quick` (single-round) or `conference` (multi-round debate) - 🤖 **Mixed Adapters**: CLI tools (claude, codex, droid, gemini) + HTTP services (ollama, lmstudio, openrouter, nebius) - ⚡ **Auto-Convergence**: Stops when opinions stabilize (saves API costs) - 🗳️ **Structured Voting**: Models cast votes with confidence levels and rationale - 🧮 **Semantic Grouping**: Similar vote options automatically merged (0.70+ similarity) - 🎛️ **Model-Controlled Stopping**: Models decide when to stop deliberating - 🔬 **Evidence-Based Deliberation**: Models can read files, search code, list files, and run commands to ground decisions in reality - 💰 **Local Model Support**: Zero API costs with Ollama, LM Studio, llamacpp - 🔐 **Data Privacy**: Keep all data on-premises with self-hosted models - 🧠 **Context Injection**: Automatically finds similar past debates and injects context for faster convergence - 🔍 **Semantic Search**: Query past decisions with `query_decisions` tool (finds contradictions, traces evolution, analyzes patterns) - 🛡️ **Fault Tolerant**: Individual adapter failures don't halt deliberation - 📝 **Full Transcripts**: Markdown exports with AI-generated summaries ## Quick Start Get up and running in minutes: 1. **Install** – follow the commands in [Installation](#installation) to clone the repo, create a virtualenv, and install requirements. 2. **Configure** – set up your MCP client using the `.mcp.json` example in [Configure in Claude Code](#configure-in-claude-code). 3. **Run** – start the server with `python server.py` and trigger the `deliberate` tool using the examples in [Usage](#usage). **Try a Deliberation:** ```javascript // Mix local + cloud models, zero API costs for local models mcp__ai-counsel__deliberate({ question: "Should we add unit tests to new features?", participants: [ {cli: "ollama", model: "llama2"}, // Local {cli: "lmstudio", model: "mistral"}, // Local {cli: "claude", model: "sonnet"} // Cloud ], mode: "quick" }) ``` > **⚠️ Model Size Matters for Deliberations** > > **Recommended**: Use 7B-8B+ parameter models (Llama-3-8B, Mistral-7B, Qwen-2.5-7B) for reliable structured output and vote formatting. > > **Not Recommended**: Models under 3B parameters (e.g., Llama-3.2-1B) may struggle with complex instructions and produce invalid votes. **Available Models**: `claude` (opus 4.5, sonnet, haiku), `codex` (gpt-5.2-codex, gpt-5.1-codex-max, gpt-5.1-codex-mini, gpt-5.2), `droid`, `gemini`, HTTP adapters (ollama, lmstudio, openrouter). See [CLI Model Reference](docs/CLI_MODEL_REFERENCE.md) for complete details. > **🧠 Reasoning Effort Control** > > Control reasoning depth per-participant for codex and droid adapters: > ```javascript > participants: [ > {cli: "codex", model: "gpt-5.2-codex", reasoning_effort: "high"}, // Deep reasoning > {cli: "droid", model: "gpt-5.1-codex-max", reasoning_effort: "low"} // Fast response > ] > ``` > - **Codex**: `none`, `minimal`, `low`, `medium`, `high`, `xhigh` > - **Droid**: `off`, `low`, `medium`, `high` > - Config defaults set in `config.yaml`, per-participant overrides at runtime For model choices and picker workflow, see [Model Registry & Picker](docs/model-registry-and-picker.md). ## Installation ### Prerequisites 1. **Python 3.11+**: `python3 --version` 2. **At least one AI tool** (optional - HTTP adapters work without CLI): - **Claude CLI**: https://docs.claude.com/en/docs/claude-code/setup - **Codex CLI**: https://github.com/openai/codex - **Droid CLI**: https://github.com/Factory-AI/factory - **Gemini CLI**: https://github.com/google-gemini/gemini-cli ### Setup ```bash git clone https://github.com/blueman82/ai-counsel.git cd ai-counsel python3 -m venv .venv source .venv/bin/activate # macOS/Linux; Windows: .venv\Scripts\activate pip install -r requirements.txt python3 -m pytest tests/unit -v # Verify installation ``` ✅ Ready to use! Server includes core dependencies plus optional convergence backends (scikit-learn, sentence-transformers) for best accuracy. ## Configuration Edit `config.yaml` to configure adapters and settings: ```yaml adapters: claude: type: cli command: "claude" args: ["-p", "--model", "{model}", "--settings", "{\"disableAllHooks\": true}", "{prompt}"] timeout: 300 ollama: type: http base_url: "http://localhost:11434" timeout: 120 max_retries: 3 defaults: mode: "quick" rounds: 2 max_rounds: 5 ``` **Note:** Use `type: cli` for CLI tools and `type: http` for HTTP adapters (Ollama, LM Studio, OpenRouter). ### Model Registry Configuration Control which models are available for selection in the model registry. Each model can be enabled or disabled without removing its definition: ```yaml model_registry: claude: - id: "claude-sonnet-4-5-20250929" label: "Claude Sonnet 4.5" tier: "balanced" default: true enabled: true # Model is active and available - id: "claude-opus-4-20250514" label: "Claude Opus 4" tier: "premium" enabled: false # Temporarily disabled (cost control, testing, etc.) ``` **Enabled Field Behavior:** - `enabled: true` (default) - Model appears in `list_models` and can be selected for deliberations - `enabled: false` - Model is hidden from selection but definition retained for easy re-enabling - Disabled models cannot be used even if explicitly specified in `deliberate` calls - Default model selection skips disabled models automatically **Use Cases:** - **Cost Control**: Disable expensive models temporarily without losing configuration - **Testing**: Enable/disable specific models during integration tests - **Staged Rollout**: Configure new models as disabled, enable when ready - **Performance Tuning**: Disable slow models during rapid iteration - **Compliance**: Temporarily restrict models pending approval ## Core Features Deep Dive ### Convergence Detection & Auto-Stop Models automatically converge and stop deliberating when opinions stabilize, saving time and API costs. Status: Converged (≥85% similarity), Refining (40-85%), Diverging (<40%), or Impasse (stable disagreement). Voting takes precedence: when models cast votes, convergence reflects voting outcome. → **[Complete Guide](docs/convergence-detection.md)** - Thresholds, backends, configuration ### Structured Voting Models cast votes with confidence levels (0.0-1.0), rationale, and continue_debate signals. Votes determine consensus: Unanimous (3-0), Majority (2-1), or Tie. Similar options automatically merged at 0.70+ similarity threshold. → **[Complete Guide](docs/structured-voting.md)** - Vote structure, examples, integration ### HTTP Adapters & Local Models Run Ollama, LM Studio, OpenRouter, or Nebius for flexible API costs and privacy options. Mix with cloud models (Claude, GPT-4) in single deliberation. → **[Setup Guides](docs/http-adapters/intro.md)** - Ollama, LM Studio, OpenRouter, cost analysis ### Extending AI Counsel Add new CLI tools or HTTP adapters to fit your infrastructure. Simple 3-5 step process with examples and testing patterns. → **[Developer Guide](docs/adding-adapters.md)** - Step-by-step tutorials, real-world examples ## Evidence-Based Deliberation Ground design decisions in reality by querying actual code, files, and data: ```javascript // MCP client example (e.g., Claude Code) mcp__ai_counsel__deliberate({ question: "Should we migrate from SQLite to PostgreSQL?", participants: [ {cli: "claude", model: "sonnet"}, {cli: "codex", model: "gpt-4"} ], rounds: 3, working_directory: process.cwd() // Required - enables tools to access your files }) ``` **During deliberation, models can:** - 📄 Read files: `TOOL_REQUEST: {"name": "read_file", "arguments": {"path": "config.yaml"}}` - 🔍 Search code: `TOOL_REQUEST: {"name": "search_code", "arguments": {"pattern": "database.*connect"}}` - 📋 List files: `TOOL_REQUEST: {"name": "list_files", "arguments": {"pattern": "*.sql"}}` - ⚙️ Run commands: `TOOL_REQUEST: {"name": "run_command", "arguments": {"command": "git", "args": ["log", "--oneline"]}}` **Example workflow:** 1. Model A proposes PostgreSQL based on assumptions 2. Model B requests: `read_file` to check current config 3. Tool returns: `database: sqlite, max_connections: 10` 4. Model B searches: `search_code` for database queries 5. Tool returns: 50+ queries with complex JOINs 6. Models converge: "PostgreSQL needed for query complexity and scale" 7. Decision backed by evidence, not opinion **Benefits:** - Decisions rooted in current state, not assumptions - Applies to code reviews, architecture choices, testing strategy - Full audit trail of evidence in transcripts **Supported Tools:** - `read_file` - Read file contents (max 1MB) - `search_code` - Search regex patterns (ripgrep or Python fallback) - `list_files` - List files matching glob patterns - `run_command` - Execute safe read-only commands (ls, git, grep, etc.) ### Configuration Control tool behavior in `config.yaml`: **Working Directory** (Required): - Set `working_directory` parameter when calling `deliberate` tool - Tools resolve relative paths from this directory - Example: `working_directory: process.cwd()` in JavaScript MCP clients **Tool Security** (`deliberation.tool_security`): - `exclude_patterns`: Block access to sensitive directories (default: `transcripts/`, `.git/`, `node_modules/`) - `max_file_size_bytes`: File size limit for `read_file` (default: 1MB) - `command_whitelist`: Safe commands for `run_command` (ls, grep, find, cat, head, tail) **File Tree** (`deliberation.file_tree`): - `enabled`: Inject repository structure into Round 1 prompts (default: true) - `max_depth`: Directory depth limit (default: 3) - `max_files`: Maximum files to include (default: 100) **Adapter-Specific Requirements:** | Adapter | Working Directory Behavior | Configuration | |---------|---------------------------|---------------| | **Claude** | Automatic isolation via subprocess `{working_directory}` | No special config needed | | **Codex** | No true isolation - can access any file | Security consideration: models can read outside `{working_directory}` | | **Droid** | Automatic isolation via subprocess `{working_directory}` | No special config needed | | **Gemini** | Enforces workspace boundaries | **Required**: `--include-directories {working_directory}` flag | | **Ollama/LMStudio** | N/A - HTTP adapters | No file system access restrictions | **Learn More:** - [Complete Configuration Reference](CLAUDE.md#configuration-notes) - All config.yaml settings explained - [Working Directory Isolation](CLAUDE.md#core-components) - How adapters handle file paths - [Tool Security Model](CLAUDE.md#evidence-based-deliberation) - Whitelists, limits, and exclusions - [Adding Custom Tools](docs/adding-tool.md) - Developer guide for extending the tool system ### Troubleshooting **"File not found" errors:** - Ensure `working_directory` is set correctly in your MCP client call - Use discovery pattern: `list_files` → `read_file` - Check file paths are relative to working directory **"Access denied: Path matches exclusion pattern":** - Tools block `transcripts/`, `.git/`, `node_modules/` by default - Customize via `deliberation.tool_security.exclude_patterns` in config.yaml **Gemini "File path must be within workspace" errors:** - Verify Gemini's `--include-directories` flag uses `{working_directory}` placeholder - See adapter-specific setup above **Tool timeout errors:** - Increase `deliberation.tool_security.tool_timeout` for slow operations - Default: 10 seconds for file operations, 30 seconds for commands **Learn More:** - [Adding Custom Tools](docs/adding-tool.md) - Developer guide for extending tool system - [Architecture & Security](CLAUDE.md#evidence-based-deliberation) - How tools work under the hood - [Common Gotchas](CLAUDE.md#common-gotchas) - Advanced settings and known issues ## Decision Graph Memory AI Counsel learns from past deliberations to accelerate future decisions. Two core capabilities: ### 1. Automatic Context Injection When starting a new deliberation, the system: - Searches past debates for similar questions (semantic similarity) - Finds the top-k most relevant decisions (configurable, default: 3) - Injects context into Round 1 prompts automatically - Result: Models start with institutional knowledge, converge faster ### 2. Semantic Search with `query_decisions` Query past deliberations programmatically: - **Search similar**: Find decisions related to a question - **Find contradictions**: Detect conflicting past decisions - **Trace evolution**: See how opinions changed over time - **Analyze patterns**: Identify recurring themes **Configuration** (optional - defaults work out-of-box): ```yaml decision_graph: enabled: true # Auto-injection on by default db_path: "decision_graph.db" # Resolves to project root (works for any user/folder) similarity_threshold: 0.6 # Adjust to control context relevance max_context_decisions: 3 # How many past decisions to inject ``` **Works for any user from any directory** - database path is resolved relative to project root. → **[Quickstart](docs/decision-graph/quickstart.md)** | **[Configuration](docs/decision-graph/configuration.md)** | **[Context Injection](docs/decision-graph/using-context-injection.md)** ## Usage ### Start the Server ```bash python server.py ``` ### Configure in Claude Code **Option A: Project Config (Recommended)** - Create `.mcp.json`: ```json { "mcpServers": { "ai-counsel": { "type": "stdio", "command": ".venv/bin/python", "args": ["server.py"], "env": {} } } } ``` **Option B: User Config** - Add to `~/.claude.json` with absolute paths. After configuration, restart Claude Code. ### Model Selection & Session Defaults - Discover the allowlisted models for each adapter by running the MCP tool `list_models`. - Set per-session defaults with `set_session_models`; leave `model` blank in `deliberate` to use those defaults. - Full instructions and request examples live in [Model Registry & Picker](docs/model-registry-and-picker.md). ### Examples **Quick Mode:** ```javascript mcp__ai-counsel__deliberate({ question: "Should we migrate to TypeScript?", participants: [{cli: "claude", model: "sonnet"}, {cli: "codex", model: "gpt-5.2-codex"}], mode: "quick" }) ``` **Conference Mode (multi-round):** ```javascript mcp__ai-counsel__deliberate({ question: "JWT vs session-based auth?", participants: [ {cli: "claude", model: "sonnet"}, {cli: "codex", model: "gpt-5.2-codex"} ], rounds: 3, mode: "conference" }) ``` **Search Past Decisions:** ```javascript mcp__ai-counsel__query_decisions({ query_text: "database choice", threshold: 0.5, // NEW! Adjust sensitivity (0.0-1.0, default 0.6) limit: 5 }) // Returns: Similar past deliberations with consensus and similarity scores // NEW! Empty results include helpful diagnostics: { "type": "similar_decisions", "count": 0, "results": [], "diagnostics": { "total_decisions": 125, "best_match_score": 0.45, "near_misses": [{"question": "Database indexing...", "score": 0.45}], "suggested_threshold": 0.45, "message": "No results found above threshold 0.6. Best match scored 0.450. Try threshold=0.45..." } } // Find contradictions mcp__ai-counsel__query_decisions({ operation: "find_contradictions" }) // Returns: Decisions where consensus conflicts // Trace evolution mcp__ai-counsel__query_decisions({ query: "microservices architecture", operation: "trace_evolution" }) // Returns: How opinions evolved over time on this topic ``` ### Transcripts All deliberations saved to `transcripts/` with AI-generated summaries and full debate history. ## Architecture ``` ai-counsel/ ├── server.py # MCP server entry point ├── config.yaml # Configuration ├── adapters/ # CLI/HTTP adapters │ ├── base.py # Abstract base │ ├── base_http.py # HTTP base │ └── [adapter implementations] ├── deliberation/ # Core engine │ ├── engine.py # Orchestration │ ├── convergence.py # Similarity detection │ └── transcript.py # Markdown generation ├── models/ # Data models (Pydantic) ├── tests/ # Unit/integration/e2e tests └── decision_graph/ # Optional memory system ``` ## Documentation Hub ### Getting Started - **[Quick Start](README.md#quick-start)** - 5-minute setup - **[Installation](README.md#installation)** - Detailed prerequisites and setup - **[Usage Examples](README.md#usage)** - Quick and conference modes ### Core Concepts - **[Convergence Detection](docs/convergence-detection.md)** - Auto-stop, thresholds, backends - **[Structured Voting](docs/structured-voting.md)** - Vote structure, consensus types, vote grouping - **[Evidence-Based Deliberation](README.md#evidence-based-deliberation)** - Ground decisions in reality with read_file, search_code, list_files, run_command - **[Decision Graph Memory](docs/decision-graph/quickstart.md)** - Learning from past decisions ### Setup & Configuration - **[HTTP Adapters](docs/http-adapters/intro.md)** - Ollama, LM Studio, OpenRouter setup - **[Configuration Reference](docs/convergence-detection.md#configuration)** - All YAML options - **[Migration Guide](docs/migration/cli_tools_to_adapters.md)** - From cli_tools to adapters ### Development - **[Adding Adapters](docs/adding-adapters.md)** - CLI and HTTP adapter development - **[CLAUDE.md](CLAUDE.md)** - Architecture, development workflow, gotchas - **[Model Registry & Picker](docs/model-registry-and-picker.md)** - Managing allowlisted models and MCP picker tools ### Reference - **[Troubleshooting](docs/troubleshooting/http-adapters.md)** - HTTP adapter issues - **[Decision Graph Docs](docs/decision-graph/)** - Advanced memory features ## Development ### Running Tests ```bash pytest tests/unit -v # Unit tests (fast) pytest tests/integration -v -m integration # Integration tests pytest --cov=. --cov-report=html # Coverage report ``` See [CLAUDE.md](CLAUDE.md) for development workflow and architecture notes. ### Contributing 1. Fork the repository 2. Create a feature branch (`git checkout -b feature/your-feature`) 3. Write tests first (TDD workflow) 4. Implement feature 5. Ensure all tests pass 6. Submit PR with clear description ## License MIT License - see LICENSE file ## Credits Built with: - [MCP SDK](https://modelcontextprotocol.io/) - Model Context Protocol - [Pydantic](https://docs.pydantic.dev/) - Data validation - [pytest](https://pytest.org/) - Testing framework Inspired by the need for true deliberative AI consensus beyond parallel opinion gathering. --- ## Status       **Production Ready** - Multi-model deliberative consensus with cross-user decision graph memory, structured voting, and adaptive early stopping for critical technical decisions!