Content
# DualGraph-Agent-for-OpenHarmony
[](LICENSE)
A Graph-Augmented Agent for OpenHarmony. The system consists of two layers:
- **Agent Layer (`OpenHarmonyAssistant/`)**: Responsible for dialogue, planning, and Generative UI output, routing user queries to the lower-level tools.
- **DualGraphRAG Tool Layer (`src/hipporag/` + `OpenHarmonyAssistant/chatbox/`)**: Built on top of [HippoRAG](https://github.com/OSU-NLP-Group/HippoRAG), deeply transformed to parse Markdown documents into **Dual Graph** and use **Cost-Aware Best-First Graph Diffusion** to complete 4-stage multimodal retrieval as the core retrieval tool for the Agent.
The overall system is oriented towards three types of scenarios: "document-level question answering + cross-file API reasoning + rich interactive UI response".
> The above figure summarizes the online retrieval process:
> ① Multi-route vector recall and sorting (paragraphs/code/images/tables + triples)
> → ② Triplet filtering
> → ③ Assigning seed node weights
> → ④ Cost-aware graph diffusion
> → ⑤ Using selected multimodal nodes for question answering reading.
> The following [retrieval process (4 stages)](#retrieval-process-4-stages) will be expanded stage by stage.
> Naming instructions: The root package of the warehouse is still called `hipporag` (code import path), but this project has made significant modifications to the node structure, retrieval process, and service method. It is recommended to follow this README instead of the upstream documentation.
---
## Table of Contents
- [Project Highlights](#project-highlights)
- [System Architecture](#system-architecture)
- [Knowledge Graph: Nodes and Edges](#knowledge-graph-nodes-and-edges)
- [Construction Process (5 Steps)](#construction-process-5-steps)
- [Retrieval Process (4 Stages)](#retrieval-process-4-stages)
- [Quick Start](#quick-start)
- [Service and API](#service-and-api)
- [Configuration Parameters](#configuration-parameters)
- [Project Structure](#project-structure)
- [Frequently Asked Questions](#frequently-asked-questions)
---
## Project Highlights
1. **Fine-grained multimodal domain embedding**:
Document paragraphs are parsed into 6 types of graph nodes + 1 type of pure vector node, **each type has an independent vector library**, avoiding long text overwhelming code/table/image/entity signals:
- Graph nodes: `file` / `chunk` / `code` / `table` / `image` / `entity`
- Pure vector nodes (not in graph, only as seed source): `fact` (OpenIE triplets)
2. **Structural + Semantic Two-layer Heterogeneous Graph**:
- **Structural layer**: From Markdown title level and hyperlink — `File → Chunk → SubChunk`, `Chunk → Code/Table/Image`, `Chunk → File` (jump).
- **Semantic layer**: From OpenIE extracted triplets + entity KNN synonyms edge + paragraph↔entity association edge.
- The two layers are bridged by `Chunk ↔ Entity` to form a "outer circle structure + inner circle semantics" double-layer network.
3. **Cost-aware Best-First Graph Diffusion (non-PPR)**:
Online retrieval (`retrieve_v2`) uses heap-driven Best-First diffusion, starting from Fact hitting entity + hitting file, and spreading along "low-cost edge" multi-hop:
`score = sim(node, query) · init_weight / (1 + cumulative_cost)`
Different types of edges have different costs (`synonymy / semantic / passage / structural`), and are equipped with time budget, expansion number, frontier size and other multi-stop, **than PPR full graph iteration delay is more controllable**, more suitable for online API reasoning.
4. **Image perceptual hash deduplication**:
Build a graph using dHash/pHash + LSH bucket + optional SSIM for near-duplicate merging, and retain the highest resolution representative. The merged image edges will be automatically redirected.
5. **Complete service capability**:
Dual-process architecture: `hipporag_service` (preloaded graph, focused retrieval) + `server_text` (API gateway, calling LLM to generate final answer), with a built-in front-end page.
---
## System Architecture
```
┌────────────────────────────────────────────────────────────────────┐
│ User · Browser (frontend.html) │
└─────────────────────────────┬──────────────────────────────────────┘
│ HTTP
▼
┌────────────────────────────────────────────────────────────────────┐
│ Agent Layer · OpenHarmonyAssistant (server_text.py · port 8000) │
│ • Dialogue planning / tool routing / Generative UI output │
│ • Feed user queries to lower-level DualGraphRAG, and feed retrieval results to Chat LLM │
└─────────────────────────────┬──────────────────────────────────────┘
│ HTTP (as Agent's retrieval tool call)
▼
┌────────────────────────────────────────────────────────────────────┐
│ DualGraphRAG Tool Layer · hipporag_service.py (port 8001) │
│ • Preload Embedding / Reranker / dual-layer heterogeneous knowledge graph │
│ • Execute 4-stage retrieval: recall → Rerank → cost-aware graph diffusion → final ranking │
│ • Return multimodal retrieval results (chunks / codes / tables / images) │
└────────────────────────────────────────────────────────────────────┘
```
### Core Components
| Component | Location | Description |
| --- | --- | --- |
| `HippoRAG` | `src/hipporag/HippoRAG.py` | Core engine: graph construction + retrieval |
| `DocumentProcessor` | `src/hipporag/document_processor.py` | Markdown → hierarchical JSON parsing |
| `OpenIE` | `src/hipporag/information_extraction/` | Entity and triplet extraction (online / offline) |
| `EmbeddingStoreV2` | `src/hipporag/embedding_store_v2.py` | Multi-type vector storage |
| `TransformersCrossEncoderReranker` | `src/hipporag/rerankers/` | Local bge-reranker fine-tuning |
| `hipporag_service.py` | `OpenHarmonyAssistant/chatbox/` | FastAPI retrieval service |
| `server_text.py` | `OpenHarmonyAssistant/chatbox/` | FastAPI gateway + command line |
| `frontend.html` | `OpenHarmonyAssistant/chatbox/` | Web front-end |
---
## Knowledge Graph: Nodes and Edges
### Nodes (6 types in graph + 1 type pure vector)
| Node | ID Prefix | Embedded Text | In Graph |
| --- | --- | --- | --- |
| File | `file-` | File summary | Yes |
| Chunk | `chunk-` | Chunk summary (embedding content uses `filter_chunk.content`: pure text after removing code/table/image references) | Yes |
| Code Block | `code-` | Code block summary (LLM generated) | Yes |
| Table | `table-` | Table summary (LLM generated) | Yes |
| Image | `image-` | Image caption (MLLM generated, with perceptual hash deduplication) | Yes |
| Entity | `entity-` | `"name: description"` | Yes |
| Fact | `fact-` | Triplet string `(h, r, t)` | **No (only vector library, as seed generator)** |
> Note: The original HippoRAG's "Fact node in graph + PPR" chain has been **discontinued for online retrieval**. Online `retrieve_v2` uses cost-based Best-First diffusion, and Fact only serves as a seed generator in the vector layer.
### Edges (5 types, unified classification by `_classify_edge_type`)
```
① structural Containment relationship (contains edge automatically adds reverse)
File ──contains──▶ Chunk ──contains──▶ SubChunk / Code / Table / Image
② jump Paragraph-level hyperlink (Markdown [text](xxx.md))
Chunk ──jump──▶ File
③ passage Paragraph↔entity (entities appearing in chunk)
Chunk ──contains──▶ Entity
④ semantic Entity triplet (bidirectional, weight = co-occurrence frequency)
Entity ◀──(h, r, t)──▶ Entity
⑤ synonymy Synonym extension (entity vector KNN + threshold ≥ synonymy_edge_sim_threshold)
Entity ◀──sim──▶ Entity
```
### Edge Cost (for diffusion scoring)
| Edge Type | Cost Formula |
| --- | --- |
| `synonymy` | `max(0.01, 1 - weight)` |
| `semantic` | `0.5 / max(weight, 0.1)` |
| `passage` | `0.2` (fixed) |
| `structural` | `0.3` (fixed) |
| Others/Unknown | `0.5` |
Intuitive meaning: **Structural and paragraph-entity edges are cheap, weak semantic/weak synonym edges are expensive**, thus controlling noise and controlling expansion scale.
---
## Construction Process (5 Steps)
> All commands default to the warehouse root directory, output root directory is `outputs/Harmony_docs_zh_cn/`.
```bash
conda activate hipporag
```
### Step 1 · Document Structure Parsing
`DocumentProcessor` parses Markdown into hierarchical JSON:
- Recursively block by `#` ~ `######`
- Extract code blocks / Markdown / HTML tables (with front and back `context_lines` context)
- Extract image references and parse relative paths
- Extract paragraph internal `[text](*.md)` hyperlinks to generate `jump`
- Output `filter_chunk.content` to remove code/table/image references for subsequent OpenIE use
```bash
python src/hipporag/document_processor.py >> index.log 2>&1
# Default correspondence: /root/code/docs/zh-cn → outputs/Harmony_docs_zh_cn/markdown_parse/structure.json
```
Also can be called by code:
```python
from src.hipporag.document_processor import DocumentProcessor
proc = DocumentProcessor(context_lines=15)
proc.process_directory(
"/path/to/docs/zh-cn",
"outputs/Harmony_docs_zh_cn/markdown_parse/structure.json",
)
```
### Step 2 · Summary Generation (File / Chunk / Code / Table)
```bash
python generate_abstracts.py \
outputs/Harmony_docs_zh_cn/markdown_parse/structure.json \
outputs/Harmony_docs_zh_cn/markdown_parse/abstract.json \
--backup outputs/Harmony_docs_zh_cn/markdown_parse_bak/abstract.json \
--max-workers 20 \
>> index.log 2>&1
```
| Parameter | Default | Description |
| --- | --- | --- |
| `--max-workers` | 20 | Concurrent thread count |
| `--backup` | - | Reuse existing summary on failure |
| `--disable-fallback` | False | Disable backup LLM configuration |
| `--dry-run` | False | Only statistics |
### Step 3 · Image Description (MLLM Caption)
```bash
# Round 1
python generate_image_captions.py \
outputs/Harmony_docs_zh_cn/markdown_parse/abstract.json \
outputs/Harmony_docs_zh_cn/markdown_parse/with_captions.json \
--backup outputs/Harmony_docs_zh_cn/markdown_parse_bak/abstract.json \
--max-workers 10
# Retry failed items
python retry_failed_captions.py \
outputs/Harmony_docs_zh_cn/markdown_parse/with_captions.json \
outputs/Harmony_docs_zh_cn/markdown_parse/with_captions_final.json
```
### Step 4 · Entity and Triplet Extraction (OpenIE)
`extract_entities_triples.py` writes OpenIE results back to each `chunk.filter_chunk.extracted_entities / extracted_triples`:
```bash
python extract_entities_triples.py \
outputs/Harmony_docs_zh_cn/markdown_parse/with_captions_final.json \
outputs/Harmony_docs_zh_cn/markdown_parse/triples.json \
--batch-size 200 \
--openie-mode online \
>> index.log 2>&1
```
| Parameter | Default | Description |
| --- | --- | --- |
| `--batch-size` | 200 | Batch size |
| `--openie-mode` | online | `online` uses OpenAI protocol; `offline` uses vLLM |
| `--llm-name` | See script | LLM name |
| `--llm-base-url` | - | Custom endpoint |
| `--disable-fallback` | False | Disable backup configuration |
Each chunk outputs as:
```json
"filter_chunk": {
"content": "ArkTS is the main development language of OpenHarmony…",
"extracted_entities": [
["ArkTS", "OpenHarmony main push application development language"],
["OpenHarmony", "operating system"]
],
"extracted_triples": [
["ArkTS", "is…the main development language of", "OpenHarmony"]
]
}
```
### Step 5 · Knowledge Graph Construction and Indexing
Launching `hipporag_service.py` will automatically call `HippoRAG.index_from_json` to complete:
1. Various node vectorization and writing to corresponding embedding storage (`file/chunk/code/table/image/entity/fact`)
2. Image perceptual hash deduplication (dHash + optional SSIM + LSH bucket)
3. Add 5 types of edges (structural / jump / passage / semantic / synonymy)
4. Synonym edge completion → `augment_graph` → write to disk
5. Save node metadata (breadcrumb navigation `breadcrumb` + Gitee URL)
Also can be triggered manually:
```python
import json
from src.hipporag import HippoRAG
from src.hipporag.utils.config_utils import BaseConfig
cfg = BaseConfig()
cfg.save_dir = "outputs/Harmony_docs_zh_cn"
cfg.llm_name = "deepseek-v3.2-exp"
cfg.llm_base_url = "https://api.modelarts-maas.com/openai/v1"
cfg.embedding_model_name = "Qwen3-Embedding-4B"
hipporag = HippoRAG(global_config=cfg)
with open("outputs/Harmony_docs_zh_cn/markdown_parse/triples.json") as f:
json_structure = json.load(f)
hipporag.index_from_json(json_structure)
```
### Data Flow
```
structure.json → abstract.json → with_captions.json
→ with_captions_final.json → triples.json → graph (igraph)
```
## Retrieval Process (4 Stages)
The overall process can be visualized in [`figures/retrieval_overview.png`](figures/retrieval_overview.png). `retrieve_v2` is the actual online method, divided into 4 stages:
```
┌──────────────────────────────────────────────────────────────────────┐
│ Stage 1 · Multi-Vector Recall │
│ Fact-top100 / File-top100 / Chunk-top100 │
└──────────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────┐
│ Stage 2 · Rerank (Cross-encoder bge-reranker-v2-m3, local) │
│ Fact → top50 / File → top50 / Chunk → top50 │
└──────────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────┐
│ Stage 3 · Cost-Aware Best-First Graph Diffusion │
│ • Seeds: top-Fact resolved Entity (init_weight=0.8) │
│ + top-File (init_weight=0.9) │
│ • Mandatory Chunk: top-Fact source chunk + Stage 2 top-Chunk │
│ • Scoring: score = sim(node, q) · init_weight / (1 + cumulative_cost)│
│ • Collection: Chunk / Code / Table / Image │
│ • Early Stopping: time_budget_s / max_expansions / │
│ per_node_neighbor_limit / max_frontier_size / │
│ min_enqueue_score / candidate already full │
└──────────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────┐
│ Stage 4 · Final Rerank (default LLM backend) + Merging Output │
│ Chunk → 10 / Code → 2 / Table → 2 / Image → 2 │
│ Merged into directly displayable Markdown (with breadcrumbs, Gitee URL, ID) │
└──────────────────────────────────────────────────────────────────────┘
```
### Key Parameters
| Parameter | Default Value | Description |
| --- | --- | --- |
| `fact_candidate_k` / `file_candidate_k` / `chunk_candidate_k` | 100 | Stage 1 three-way recall number |
| `fact_top_k` / `file_top_k` / `chunk_top_k` | 50 | Stage 2 precision ranking retention number |
| `spread_chunk_k` | 100 | Stage 3 chunk candidate upper limit |
| `spread_code_k` / `spread_table_k` / `spread_image_k` | 5 | Stage 3 modal candidate upper limit |
| `spread_time_budget_s` | 2.0 | Diffusion time budget (seconds) |
| `spread_max_expansions` | 12000 | Maximum expansion node number |
| `spread_per_node_neighbor_limit` | 96 | Maximum number of neighbors expanded per node (prioritized by low cost) |
| `spread_max_frontier_size` | 20000 | Maximum priority queue length |
| `spread_min_enqueue_score` | 0.01 | Minimum score threshold for enqueuing |
| `final_chunk_k` | 10 | Final returned chunk number |
| `final_code_k` / `final_table_k` / `final_image_k` | 2 | Final returned modal number |
| `generate_report` | false | Whether to generate LLM integrated report |
| `verbose` | true | Whether to print retrieval process |
> The default gateway configuration of `server_text.py` is slightly different ( `fact_candidate_k=300`, `chunk_candidate_k=300`, `fact_top_k=10`, etc.), see `DEFAULT_RETRIEVAL_CONFIG`.
---
## Quick Start
### Environment Requirements
- Python 3.10+
- CUDA 11.8+ (GPU recommended; ≥ 24GB video memory suggested)
- Network accessible LLM/Embedding API (or local vLLM)
### Installation
```bash
pip install -r requirements.txt
pip install -e .
```
### Start Service
**1. HippoRAG Service (port 8001, pre-loading slower)**
```bash
cd OpenHarmonyAssistant/chatbox
python hipporag_service.py --port 8001
```
Once you see `✅ HippoRAG initialization complete`, it's ready.
**2. API Gateway (port 8000, second-level startup)**
```bash
# Open a new terminal
cd OpenHarmonyAssistant/chatbox
python server_text.py --server --port 8000
```
**3. Open browser and navigate to** `http://localhost:8000/`
### Command Line Interaction
```bash
cd OpenHarmonyAssistant/chatbox
python server_text.py -i # Interactive mode (default RAG + LLM)
python server_text.py "How to create an ArkTS page?" # Single query
python server_text.py "How to use @State?" --no-llm # Only retrieval
```
---
## Service and API
### Endpoint Quick Check
| Endpoint | Method | Description |
| --- | --- | --- |
| `/` (8000) | GET | Return frontend page |
| `/chat` (8000) | POST | RAG + LLM complete Q&A |
| `/retrieve` (8000) | POST | Only retrieval (no LLM call) |
| `/health` (8000) | GET | Health check (including downstream 8001 status) |
| `/retrieve` (8001) | POST | Directly call underlying HippoRAG retrieval |
| `/health` (8001) | GET | HippoRAG service health check |
### `/chat` Example
```bash
curl -X POST http://localhost:8000/chat \
-H "Content-Type: application/json" \
-d '{
"query": "How to create an ArkTS page?",
"use_rag": true,
"use_llm": true
}'
```
Only retrieval:
```bash
curl -X POST http://localhost:8000/chat \
-H "Content-Type: application/json" \
-d '{"query": "How to use @State decorator?", "use_rag": true, "use_llm": false}'
```
Custom retrieval parameters:
```bash
curl -X POST http://localhost:8000/chat \
-H "Content-Type: application/json" \
-d '{
"query": "How to create an ArkTS page?",
"retrieval_config": {
"final_chunk_k": 5,
"final_code_k": 3,
"spread_chunk_k": 80,
"verbose": true
}
}'
```
### Response (selected)
```json
{
"query": "How to create an ArkTS page?",
"rag_context": "# Relevant document paragraphs 1\n- ID: chunk-...\n- File: https://gitee.com/...\n- Location: H1 / H2 / H3\n\n...",
"llm_response": "To create an ArkTS page, you need to…",
"rag_chunks_count": 10,
"rag_codes_count": 2,
"rag_tables_count": 2,
"rag_images_count": 2,
"timing": {
"stage1_embedding": 1.23,
"stage2_rerank": 2.34,
"stage3_spread": 0.56,
"stage4_final_rerank": 3.45,
"stage4_merge": 0.01,
"rag_total": 7.59,
"llm": 5.67,
"total": 13.26
}
}
```
For more curl / jq usage, see [`OpenHarmonyAssistant/chatbox/API_README.md`](OpenHarmonyAssistant/chatbox/API_README.md).
---
## Configuration Parameters
### `BaseConfig` (`src/hipporag/utils/config_utils.py`)
**LLM**
| Parameter | Default Value | Description |
| --- | --- | --- |
| `llm_name` | gpt-4o-mini | OpenIE / final precision ranking / QA used LLM |
| `llm_base_url` | None | Custom OpenAI-compatible endpoint |
| `max_new_tokens` | 2048 | Single inference maximum token |
| `temperature` | 0 | Sampling temperature |
**Embedding**
| Parameter | Default Value | Description |
| --- | --- | --- |
| `embedding_model_name` | nvidia/NV-Embed-v2 | Embedding model |
| `embedding_batch_size` | 10000 | Embedding batch size |
| `embedding_max_seq_len` | 2048 | Maximum sequence length |
| `embedding_return_as_normalized` | True | Whether to normalize |
**Graph Construction**
| Parameter | Default Value | Description |
| --- | --- | --- |
| `synonymy_edge_topk` | 2047 | Entity KNN candidate number |
| `synonymy_edge_sim_threshold` | 0.95 | Synonym edge similarity threshold |
| `is_directed_graph` | False | Whether directed |
| `enable_image_content_dedup` | True | Whether to enable image perceptual hash deduplication |
| `image_dedup_hash_method` | dhash | `dhash` / `phash` |
| `image_dedup_hamming_threshold` | 6 | Hamming distance threshold |
| `enable_image_dedup_ssim` | False | Whether to additionally review SSIM |
### Service-side Default Configuration (`hipporag_service.py`)
| Item | Default |
| --- | --- |
| `save_dir` | `outputs/Harmony_docs_zh_cn` |
| `llm_name` | `deepseek-v3.2-exp` |
| `llm_base_url` | `https://api.modelarts-maas.com/openai/v1` |
| `embedding_model_name` | `Qwen3-Embedding-4B` |
| `rerank_backend` | `transformers` |
| `rerank_model_name` | Local `bge-reranker-v2-m3` |
| `final_rerank_backend` | `llm` |
### Environment Variables
| Variable | Default | Description |
| --- | --- | --- |
| `HIPPORAG_SERVICE_URL` | `http://localhost:8001` | Gateway access downstream HippoRAG address |
| `CHAT_MODEL_NAME` | `qwen3-coder-480b-a35b-instruct` | Gateway side Chat LLM |
| `CHAT_BASE_URL` | `https://api.modelarts-maas.com/v2` | Gateway side Chat LLM endpoint |
| `OPENAI_API_KEY` / `MAAS_API_KEY` | - | LLM authentication |
| `RERANK_BACKEND` | `transformers` | Stage 2 rerank backend |
| `RERANK_MODEL_NAME` | `bge-reranker-v2-m3` path | Rerank model |
| `RERANK_DEVICE` / `RERANK_BATCH_SIZE` / `RERANK_MAX_LENGTH` | See code | Rerank performance tuning |
| `FINAL_RERANK_BACKEND` | `llm` | Stage 4 final rerank backend |
---
## Project Structure
```
Graph-Agent-for-OpenHarmony/
├── src/hipporag/ # Core code
│ ├── HippoRAG.py # Main class: graph construction + 4-stage retrieval
│ ├── document_processor.py # Markdown → hierarchical JSON
│ ├── abstract_generator.py # Abstract generator
│ ├── embedding_store_v2.py # Multi-type vector storage
│ ├── rerank.py # DSPy-style fact rerank
│ ├── information_extraction/ # OpenIE (online / offline)
│ ├── embedding_model/ # Embedding adaptation
│ ├── llm/ # LLM client
│ ├── rerankers/ # Cross-encoder rerank
│ ├── prompts/ # Prompt templates
│ └── utils/ # Configuration / tools
│
├── OpenHarmonyAssistant/chatbox/ # Web service + frontend
│ ├── hipporag_service.py # FastAPI retrieval service (port 8001)
│ ├── server_text.py # FastAPI gateway + CLI (port 8000)
│ ├── frontend.html # Web frontend
│ └── API_README.md # API detailed documentation
│
├── extract_entities_triples.py # Step 4: entity triple extraction
├── generate_abstracts.py # Step 2: abstract generation
├── generate_image_captions.py # Step 3: image captions
├── retry_failed_captions.py # Step 3 retry
├── markdown_parser.py # Markdown parsing assistance
├── demo_retrieval.py / demo_*.py # Various examples
│
├── outputs/Harmony_docs_zh_cn/ # Default output directory (index + embedding)
├── bge-reranker-v2-m3/ # Local rerank model (self-provided)
│
├── figures/ # README diagrams (e.g., retrieval_overview.png)
├── requirements.txt
├── setup.py
└── README.md
```
## Frequently Asked Questions
**Q1. `hipporag_service.py` startup gets stuck or OOM?**
- Check `nvidia-smi` to confirm CUDA availability; reduce `embedding_batch_size` if memory is insufficient.
- The initial startup reads all embeddings under `outputs/Harmony_docs_zh_cn/` and constructs the igraph, **warm-up time is normal in minutes**.
**Q2. Retrieval results are empty?**
- Confirm the existence of embedding files under `outputs/Harmony_docs_zh_cn/`;
- Use `curl http://localhost:8001/health` to check if `hipporag_initialized` is `true`;
- Call `/retrieve` with `verbose: true` to view logs at each stage.
**Q3. `/chat` timeout?**
- Reduce `final_chunk_k` / `spread_chunk_k`;
- Shorten `spread_time_budget_s`;
- Confirm that the downstream LLM API is reachable.
**Q4. How to incrementally add documents?**
1. Rerun `DocumentProcessor.process_directory` to generate a new `structure.json`;
2. Perform steps 2-4 sequentially;
3. Rerun `index_from_json` (or restart `hipporag_service.py`).
The code supports incremental merging with `add_synonymy_edges` / `add_new_nodes` / `add_new_edges`.
**Q5. Want to change LLM / Embedding?**
- Change LLM: modify `llm_model_name` / `llm_base_url` in `hipporag_service.py`, or use `OPENAI_API_KEY` + custom `BaseConfig`;
- Change Embedding: modify `embedding_model_name`; if the vector dimension changes, **the index must be rebuilt**.
---
## Maintainer's Notes
- **`HippoRAG.py` has nearly 8000 lines**, before modifying `index_from_json` / `retrieve_v2` / `graph_spread_with_similarity`, please understand the three core dictionaries: `node_to_node_stats / fact_to_chunk_id / fact_to_entities`.
- Adding a new node type: add `*_embedding_store` in `index_from_json`, add parsing in `_extract_nodes_from_json`, and add classification in `_classify_edge_type`; also, add candidate collection and final ranking logic in `retrieve_v2` stages 3 and 4.
- Adding a new edge type: classify in `_classify_edge_type` and provide cost in `graph_spread_with_similarity::get_edge_cost`.
- Debugging a single query:
```python
from src.hipporag import HippoRAG
from src.hipporag.utils.config_utils import BaseConfig
hipporag = HippoRAG(global_config=BaseConfig(save_dir="outputs/Harmony_docs_zh_cn"))
hipporag.prepare_retrieval_objects()
result = hipporag.retrieve_v2(["How to use @State?"], verbose=True)
```
---
## License
MIT. See [LICENSE](LICENSE) for details.
## Update Log
- **v2.3** README adds an online retrieval and Q&A overview diagram (`figures/retrieval_overview.png`), visually presenting the 5-step online retrieval process.
- **v2.2** README rewritten to align with the actual behavior of `retrieve_v2` (cost-aware Best-First diffusion, not PPR); added image perceptual hash deduplication instructions.
- **v2.1** Introduced hierarchical JSON indexing (`index_from_json`).
- **v2.0** 4-stage retrieval process launched.
- **v1.0** Basic HippoRAG implementation.
