insurance-recommendation-agent

# Tool List This is a prototype project for an Insurance Recommendation Agent built with Google ADK, MCP Toolbox for Databases, ToolboxToolset, SQLite, and Vertex AI. The current core design of the project is: - ADK Agent is responsible for dialogue process, follow-up questions, tool selection, and final response integration - MCP Toolbox loads db/tools.yaml, providing controlled insurance query tools and prompt templates - SQLite provides demonstration products, recommendation rules, and FAQ data - Makefile is used to uniformly manage installation, database initialization, startup, and eval commands This project focuses on a traceable, testable, and SQL-free insurance recommendation process, rather than a complete insurance system. ## Project Goals The current goals of this prototype are to verify the following capabilities: - Ask follow-up questions when user information is insufficient - Select corresponding tools based on age, budget, and main protection goals - Execute controlled data queries through MCP Toolbox - Integrate recommendation reasons, rule basis, waiting period, and exclusion clause reminders - Maintain conservative statements in responses, without guaranteeing underwriting, claims, or benefits - Allow tool call to be inspected in ADK trace/eval ## Architecture Design ### System Architecture Diagram  ### Component Responsibilities - Google ADK Agent: responsible for interaction orchestration, determining whether to ask follow-up questions, which tool to query, and when to supplement details and rules - ToolboxToolset: serves as the MCP bridge layer between ADK and MCP Toolbox - MCP Toolbox: loads source, tool, toolset, and prompt definitions, providing tool services externally - db/tools.yaml: centrally defines data sources, insurance-specific tools, tool groups, and prompt templates - SQLite: stores demonstration products, recommendation rules, demo user profiles, and FAQs ## Current Implementation Status ### Agent Execution Path The current app/agent.py adopts the following approach: - Uses gemini-2.5-flash as the model - Loads the main agent prompt from app/prompts/insurance_agent_prompt.txt - Connects to http://127.0.0.1:5000 MCP Toolbox through ToolboxToolset - Uses tools provided by Toolbox in the formal execution path, rather than directly registering local Python helpers as agent tools ### Local Python Helper Role app/tools/insurance_tools.py still retains a set of local SQLite helpers, mainly for: - Providing reference implementations of query logic - Local testing and data inspection - Helping to compare YAML tools and Python query behavior In other words, the executing Agent currently relies mainly on MCP Toolbox; local Python helpers are auxiliary and testing assets, not the main runtime tool surface. ## Project Directory ```text insurance-recommendation-agent/ ├── Makefile ├── README.md ├── app │ ├── __init__.py │ ├── agent.py │ ├── prompts │ │ └── insurance_agent_prompt.txt │ └── tools │ ├── insurance_tools.py │ └── session_tools.py ├── archi.png ├── data ├── db │ ├── insurance.db │ ├── schema.sql │ ├── seed.sql │ └── tools.yaml ├── docker-compose.yml ├── docs │ ├── architecture.md │ ├── demo_script.md │ ├── embedding.md │ ├── governance.md │ ├── limitations.md │ ├── prompt_tool_contract.md │ └── summary.md ├── pyproject.toml ├── tests │ ├── evals │ │ ├── insurance_core.test.json │ │ ├── insurance_extended.test.json │ │ ├── safety │ │ │ ├── case_09_system_capability.test.json │ │ │ ├── case_10_no_guarantee.test.json │ │ │ ├── case_11_rule_explanation.test.json │ │ │ ├── case_12_product_detail_follow_up.test.json │ │ │ └── case_13_no_investment_return.test.json │ │ ├── session_aware │ │ │ ├── case_s1_reuse_existing_profile.test.json │ │ │ ├── case_s2_follow_up_with_last_product.test.json │ │ │ └── case_s3_update_budget.test.json │ │ └── test_config.json │ ├── test_cases.md │ ├── test_insurance_tools.py │ └── test_result_template.md └── uv.lock ``` ## Technical Composition - Google ADK: responsible for agent creation, tool invocation, and eval execution - ToolboxToolset: allows ADK Agent to use Toolbox tools through MCP protocol - MCP Toolbox for Databases: loads YAML configuration, provides SQLite query tools and prompts - SQLite: current prototype data layer - Vertex AI: provides Gemini model reasoning capabilities - Docker Compose: starts MCP Toolbox container - uv: creates virtual environment and synchronizes Python dependencies ## Defined Tools and Templates ### Source - insurance_sqlite ### Tools - search_medical_products - search_accident_products - search_family_protection_products - search_income_protection_products - get_product_by_name - get_product_detail - get_recommendation_rules ### Toolsets - insurance_recommendation_tools - insurance_debug_tools ### Prompts - insurance_followup_question_template - insurance_recommendation_response_template - insurance_disclaimer_template ### Current Tool Usage - search_medical_products: queries medical protection products based on age and annual budget - search_accident_products: queries accident protection products based on age and annual budget - search_family_protection_products: queries family protection candidate products based on age and annual budget - search_income_protection_products: queries income interruption risk candidate products based on age and annual budget - get_product_by_name: performs precise product query when user directly mentions product name - get_product_detail: supplements waiting period, exclusion clauses, applicable age, and premium range details - get_recommendation_rules: queries recommendation rules corresponding to main protection goals ## Agent Prompt Behavior Boundary The main prompt is currently defined in app/prompts/insurance_agent_prompt.txt, with core rules including: - Must confirm whether age, budget, and main protection goals are available - If information is insufficient, ask follow-up questions, not directly recommend - If information is sufficient, select corresponding search tools based on protection goals - Use get_product_by_name when user directly mentions product name - Use get_product_detail to supplement product restrictions or clauses - Use get_recommendation_rules to explain recommendation basis - Do not guarantee underwriting, claims, or benefits This represents the Agent Prompt's responsibility as orchestration, not free database exploration. ## Data Model The current schema.sql creates the following data tables: - insurance_products: main product data - recommendation_rules: recommendation rules and priority - user_profiles_demo: demonstration user data - faq_knowledge: FAQ knowledge data The current recommendation process directly relies on: - insurance_products - recommendation_rules ## Installation and Execution ### Prerequisites - Python 3.12 - uv - Docker - sqlite3 - Google Cloud environment with Vertex AI access ### Environment Variables Create .env, at least containing .env.example settings: ```env GOOGLE_CLOUD_PROJECT=your-gcp-project GOOGLE_CLOUD_LOCATION=your-vertex-region GOOGLE_GENAI_USE_VERTEXAI=1 ``` If using other authentication methods, ensure local access to Vertex AI. ### Common Commands Initial installation: ```bash make install ``` When ADK eval dependencies are needed: ```bash make install-eval ``` Synchronize dependencies when .venv exists: ```bash make sync ``` Synchronize dependencies with eval extra: ```bash make sync-eval ``` Check execution environment: ```bash make env-check ``` Initialize database: ```bash make db-init ``` Rebuild database: ```bash make db-reset ``` Start Toolbox: ```bash make toolbox-up ``` Stop Toolbox: ```bash make toolbox-down ``` Start ADK Web UI: ```bash make run ``` Execute Agent in CLI mode: ```bash make run-cli ``` One-click installation, database creation, and Toolbox startup: ```bash make up ``` Stop services: ```bash make down ``` After startup, open: ```text http://127.0.0.1:8000 ``` ### Recommended Startup Sequence ```bash make install make db-init make toolbox-up make run ``` ## Testing and Evaluation ### Python Testing ```bash make check ``` Currently, tests/test_insurance_tools.py mainly verifies local SQLite helper query results. ### ADK evals Core regression testing: ```bash make eval-core ``` Safety testing: ```bash make eval-safety ``` Or execute separately: ```bash make eval-safety-case-09 make eval-safety-case-10 make eval-safety-case-11 make eval-safety-case-12 make eval-safety-case-13 ``` Session-aware testing: ```bash make eval-session-aware ``` Or execute separately: ```bash make eval-session-aware-case-s1 make eval-session-aware-case-s2 make eval-session-aware-case-s3 ``` ### Eval File Configuration Currently, main eval files are located in tests/evals and its subdirectories: - tests/evals/insurance_core.test.json - tests/evals/insurance_extended.test.json - tests/evals/test_config.json - tests/evals/safety/case_09_system_capability.test.json - tests/evals/safety/case_10_no_guarantee.test.json - tests/evals/safety/case_11_rule_explanation.test.json - tests/evals/safety/case_12_product_detail_follow_up.test.json - tests/evals/safety/case_13_no_investment_return.test.json - tests/evals/session_aware/case_s1_reuse_existing_profile.test.json - tests/evals/session_aware/case_s2_follow_up_with_last_product.test.json - tests/evals/session_aware/case_s3_update_budget.test.json The current test_config.json evaluation criteria are: - tool_trajectory_avg_score: threshold 1.0, match_type is IN_ORDER - final_response_match_v2: threshold 0.7 ## Recommendation Process Summary 1. User inputs needs 2. Agent determines if age, budget, and main protection goals are available 3. If information is insufficient, asks follow-up questions 4. If information is sufficient, selects corresponding MCP Toolbox tools 5. Supplements recommendation rules and product details as needed 6. Integrates candidate products, clause reminders, and conservative statements to generate final response ## Current Capabilities - Can ask follow-up questions when information is insufficient - Can switch corresponding insurance tools based on protection goals - Can query product candidates, product details, and recommendation rules - Can handle user direct mentions of product names for detail follow-up questions - Can supplement waiting period and exclusion clause reminders in responses - Can verify tool usage sequence and final response quality through ADK eval ## Known Limitations Currently, there are the following limitations: 1. Product data is for demonstration purposes, not real insurance products. 2. Formal underwriting process not implemented. 3. Premium logic is still a simplified version. 4. Recommendation rule coverage is limited. 5. FAQ/embedding retrieval not connected to formal recommendation process. 6. Local Python helpers and YAML tools have dual-track implementation, which can be further converged. 7. Current UI is mainly ADK Web UI, without independent frontend. ## Future Directions 1. Reflect get_product_by_name used in agent prompt into more design documents and test cases. 2. Enhance FAQ, clause, and exclusion responsibility retrieval capabilities. 3. Make local Python helpers and MCP YAML tools' capability boundaries more consistent. 4. Expand eval matrix to cover more follow-up questions, multi-round clarifications, and product comparison scenarios. 5. Plan formal frontend and more complete safety/governance settings. ## Related Documents - docs/architecture.md: system architecture and data flow - docs/prompt_tool_contract.md: Prompt and Tool division boundary - docs/governance.md: governance and limitation description - docs/limitations.md: limitations and future directions - docs/demo_script.md: demonstration conversation script ## Disclaimer This project is for prototype design, learning, and system architecture demonstration only. All insurance recommendation results are for preliminary product screening only; actual underwriting still depends on product clauses, health disclosure, and underwriting results.