OmniHawk-AI

align="center"> # OpenHawk ### Global AI Intelligence OS for the Age of Intelligent Agents From paper and model releases to capital markets, policy regulation, and open-source ecosystems, Unified crawling, deduplication, analysis, subscription, and push, and provide programmable interfaces through MCP + CLI. [](#) [](#docker-启动) [](#mcp-服务) [](#agent-cli-接口新增) [](LICENSE) **Multi-Channel Subscription and Push**          **Important Tags**        <p align="center"> <img src.png" altHawk project banner" width="920" /> </p> **English** | [English](README-EN.md) | [हिन्दी](README-HI.md) | [Español](README-ES.md) | [العربية](README-AR.md) | [Français](README-FR | [Português](README-PT.md) | [বাংলা](README-BN.md) | [日本語](README-JA.md) | [한국어](README-KO.md) </div> ## 🚀 Why We Built This Project AI information is scattered across different sources and updates rapidly. Manual tracking often leads to: - Fragmented signals: papers, vendor announcements, financial reports, policies, and open-source trends are disjointed. - Timeliness and noise: old news reappears, making it difficult to filter out actual incremental information. - Deduplication challenges: cross-source reposts lead to duplicate crawling and pushing. - Insufficient automation: it's hard to directly integrate "information acquisition" into agent workflows. `OpenHawk` aims to create a sustainable, scalable, and intelligent infrastructure that can be directly called by agents. ## 👥 Who Is This For? - AI researchers: continuously track papers and method evolution. - Product and engineering teams: track model releases, open-source toolchains, and developer signals. - Investment and business analysis teams: track AI-related financial reports, capital markets, and industry reports. - Policy and compliance teams: track regulatory and security events across regions. - Agent builders: need programmableCLI tool interfaces. ## 🧭 Six Independent Pages (Parallel Relationship) | Page | Primary Purpose | Typical Sources | | --- | --- | | AI Paper Radar | Academic tracking and in-depth analysis | arXiv and other academic RSS feeds | | AI Frontier Radar | Model/product/technology advancements | Vendor websites, tech blogs, official news | | AI Financial Information | Capital markets and company dynamics | Financial reports, earnings calls, market news, and investment and financing information | | AI Industry Report | Industry research insights | Global institutional reports and whitepapers | | AI Policy and Security | Regulatory and risk events | Policy institutions, regulatory announcements, and security event sources | | AI Open-Source Ecosystem and Developer Signals | Open-source tools and community signals | GitHub Trending and open-source project dynamics | > These six pages are independent. Each page has its own parameters, subscriptions, and push configurations. ## ⚙️ Core Capabilities - Multi-source crawling and regional organization (organizing channels by country/region and source type). - Historical persistence and deduplication (avoiding duplicate crawling and pushing). - Default freshness control (e.g., 90-day to reduce stale information noise. - Unified translation pipeline (arbitrary target language) covering titles, LLM analysis, and push content-channel notifications: `feishu`, `wework`, `wechat`, `telegram`, `dingtalk`, `ntfy`, `bark`, `slack`,email`. - Intelligent push strategies: `daily`, `incremental`, and `realtime`. - Scheduled crawling and automated subscription- MCP tool interfaces. - Agent CLI (newly added) supports scripted direct tool invocation. ## 🌐 Unified Translation Pipeline for Arbitrary Target Languages - Unified scope: same pipeline covers `titles`, `LLM summaries/analysis`, and `push to avoid fragmented translation logic. - Target languages: supports `English`, `Korean`, `French`, `Chinese`, `Traditional Chinese custom target languages. - Cost control: batch translation, incremental completion, historical persistence reuse, and only translating missing fields to reduce repetitive API calls. - Consistency: Web, MCP, and CLI use theoutput_language` configuration semantics, ensuring consistent language output across page displays and notification content. Example (CLI): ```# Set the output language for the AI finance page to Japanese openhawk-ai-cli call save_scope_settings --args '{"scope":"market_finance","output_language":"Japanese"}' # Trigger crawling and subsequent content presentation/pushing in the specified language openhawk-ai-cli call fetch_scope_items --args '{"scope":"market_finance","max_per_source":20}' ``` ## 🧠 Intelligent Push Strategies | Strategy | Trigger Method | Applicable Scenarios | Characteristics | | --- | --- | --- | --- | | `daily` | Daily scheduled aggregation push | Management daily reports, team morning reports | Comprehensive topic aggregation, suitable for stable rhythms | | `incremental` | Push new items within a timed window | Daily monitoring | Deduplicates, lower noise `realtime` | Event-triggered immediate push | Major model releases, policy breaks, financing news | No waiting, high priority, and strong timeliness | - Strategies take effect independently based on "subscription rules" and can separately on each of the six pages. - Can be combined conditions (source, region, keywords) to achieve refined alerts. ## 🧱 System Architecture +---------------------------+ | Data Sources | | papers / frontier / ... | +-------------+-------------+ | v +----------------+ +------------+ +-------------------+ | Fetch & Dedupe |-->| Persistence|-->| Analysis & Routing| +----------------+ +------------+ +-------------------+ | v v +-------------+ +------------------+ Web Console | Push | +------+------+ +------------------+ +----------+----------+ MCP Server / CLI | | (agent automation +---------------------## 🖼 Screenshots ### Web Frontend Pages (img1 -| Page 1 | Page 2 | | --- | --- | |  |  | |  |  | ### Channel Subscriptionimg5)  --- ## ⚡### 1🧩 Environment Requirements `>= 12` - Recommended touv` - Docker to install `Docker `Docker Compose` 2) 🖥 Launch (Development Modebash uv sync``` 1. crawling main program (): ```bash openh``` 2. interactive Web console (6): ```bash openhawk_ai.web.panel_server --port 8080 --output-dir output ``` 3. Start the (HTTP): ```bashawk-ai-mcp -- --host 0.0.0. 3333### 3) Launch ```docker compose -f-compose.yml up -build ``` Default- Main service (container-internal Web port mapping): `WEBSERVER_PORT` (default 8080) - `http://127.0.1:cp` Stop: ```bash docker compose docker/docker-compose.yml``` View logs: compose --compose.yml logs -``` --- ## CLI Interface (Newly Added) To allow intelligent agents or scripts to directly invoke tools without using MCP transport, the `openhawk-ai-cli` added. ### Goals - Consistent tools ( names, same parameter semantics). - JSON input / JSON output for easy automation orchestration. - Suitable for Shell, CI, Python subprocess, and agent executors. ### 🧪 Commands After Installation ```bash openhawk-ai-cli tools ``` ### 📌 Common Examples 1. List all callable tools and parameters: ```bash -ai-cli tools ``` 2. Invoke a JSON parameters): openhawk-ai list_scope_items --scope":"market_finance20}' ``` Invoke a tool (): ```bash -ai-cli call up_subscription --args-file.json ``` 4 the project root directory and output directory: ```bashawk-ai-cli --project --output-dir ./_project_overview5. Compact JSONuitable for pipeline processing): ```bash openh call list_sccompact ``` ### Windows PowerShell Example (Recommended. Use `Convert` + `--` (most reliablepowershell $payload = = "market_finance = 20 ConvertTo-Json$payload | -Encoding utfpayload.json openhawk-ai-cli call list_scopeargs-file .\payloadcompact ``` 2 a parameter file usingString: ```powershell' { " "frontier", _per_source":, "source_ids": ["openai-news", "anthropic-news"] } '@-Content -Encoding utf8 .\payloadopenhawk-ai-cli call fetch_scope_items --args-file .\payload.json --compact ``` 3. parameters can be called```powershell openh-cli call get_project_overcompact ``` ### 🧾 Exit Code - `0`: successful invocation. - ` execution exception. -`: parameter error / not exist / JSON format error. ### Parameters and Coverage has fixed parameters as follows Level | Parameter || --- | --- | --- | | Global | `--project-root` | overrides project root directory | | Global | `--output-dir` runtime output directory | `--compact` | compact JSON output | | | `tools` | list callable tools | | | `call <tool>` | invoke a specified| `call` `--args` | inline JSON parameters | | `call` options | `--args-file` | read parameters from a JSON file | CLI business parameters are determined by specific tools. Execute the following command to view all tool parameter definitions: ```bash openhawk-ai-cli tools --compact ``` Coverage: - Covers all 22 tool capabilities currently exposed by MCP (project overview, page crawling, page settings, page subscriptions, paper analysis, etc Does not directly cover lifecycle management (e or stopping Docker Web service processes). directly cover behavior itself (e browser clicks), but covers the corresponding data layer operations (settings, crawling, subscriptions, pushing## 🔌 ### Start ```bash # stdio -m m.server --transport std python mcp_server.server --transport http --host 0.0.0.0 3333 Endpoint: `http.0.1:3333### MCP Tool Group1. Project- `get_project` - `list_pages` - `list` 2. Global- `get_global_settings` - `save_global_settings` 3. Page Data and Crawling - `list_scope_sources` - `list_scope_items` - `_items` - `_settings` - `` 4. Page Subscriptions -_scope_subscriptions` upsert_scope_subscriptiondelete_scope_subscription` - `run_scope_sub5. Paper Capabilities `list_papers `get_p` - `deep_paper` -_paper_action` 6. Paper Subscriptions - `list_paper_subscriptions` - `upsert_paper_subscription` - `delete_paper_subscription` - `run_paper` --- 🧠 Configuration Description Main configuration directory: `config/` Key files: - `config/config.yaml`: Main configuration (ling, pushing, AI, storage, etc.) - `config/timeline.yaml`: Timing strategy (preset and custom) - `config/frequency_words.txt`: Keyword rules - `config/ai_interests.txt`: Interest direction definition - `config/ai_analysis_prompt.txt`: Analysis prompt words - `config/ai_translation_prompt.txt`: Translation prompt words directory: `output/` Common persistent data: - `output/ai_progress_items.json` - `output/ai_progress_seen.json` - `output/panel_settings.json` - `output/progress_page_settings.json` - `output/panel_subscriptions.json` - `output/progress_subscriptions.json` - `output/news/*.db` - `output/rss/*.db` --- ## 📣 Push Channels Already supported and unified in backend + frontend + MCP + CLI: - `feishu` - `wework` - `wechat` (personal WeChat, based on WeCom text) - `telegram` - `dingtalk` - `ntfy` - `bark` - `slack` - `email` --- ## 🗂️ Project Structure ```text . ├─ open/ # Core capabilities (crawling/analysis/notification/Web) │ ├─ __main__.py # Main program entry │ ├─ agent_cli.py # Agent CLI entry (new) │ └─ web/panel_server.py console service ├─ mcp_server/ # MCP server ├─ config/ # Configuration and prompt words ├─ docker/ # Dockerfile / compose / entry script ├─ docs/assets/ # README visual resources (including OpenHawk SVG) ├─ output/ # Runtime persistent data ├─ README.md └─ README-EN.md ``` --- ## ❓Frequently Asked Questions ### Q1: What is the relationship between CLI and MCP? CLI directly calls MCP isomorphic tool functions; MCP is suitable for protocol access, and CLI is suitable for scripting and local automation. ### Q2: Can CLI cover all functions? CLI can cover all the capabilities currently exposed by MCP (22 tools). If you can do data layer operations on the UI (crawling, checking, setting, subscribing, pushing), you can usually implement them through CLI. ### Q3: Why do I see duplicate messages? Please check: - Whether `output/ai_progress_seen.json` is mounted and persisted. - Whether multiple duplicate subscription rules are enabled. - Whether `timeline` and page-level subscriptions trigger repeatedly in the same time window. ### Q4: How to minimize access to Agent? The simplest Let Agent call first ```bash openhawk-ai-cli call get_project_overview ``` Then call `list_scope_items` / `list_papers` / `run_*_subscriptions` as needed. --- ## 🙏 Acknowledgements and References - This project references and acknowledges [TrendRadar](https://github.com/sansan0/TrendRadar) in product ideas and engineering practices. - OpenHawk has independently evolved on this basis, including 6-page parallel system, regional data sources, multi-channel subscription push, MCP + Agent CLI integration, etc. --- ## 📄 License This project uses the [MIT License).