bedrock-search

# bedrock-search A remote **MCP (Model Context Protocol) server** that turns **AWS Bedrock AgentCore** managed tools into a clean, token-authenticated HTTP endpoint any MCP client (Claude Code, Cline, Cursor, your own agent…) can use: - **`search`** → forwards the query, as-is, to the AgentCore **Web Search Tool** (Amazon's own web index + knowledge graph), reached through an AgentCore **Gateway** MCP endpoint (SigV4 / `AWS_IAM`). - **`fetch`** → retrieves a URL's readable content via the AgentCore **Browser Tool**: a remote headless Chrome driven over **raw Chrome DevTools Protocol (CDP)** on a SigV4-signed WebSocket. - **`browser`** *(optional, off by default)* → full stateful browser automation (navigate / click / type / screenshot / multi-tab / network intercept / `execute_cdp`). > **No LLM is in the request path.** `search` and `fetch` are pure API/CDP calls, > so they don't consume any model tokens — you pay only for the underlying > AgentCore usage. The server is a thin, deterministic adapter. The same Go code runs two ways: as a long-running service on **Kubernetes/EKS**, or as an **AWS Lambda** function — see [Deployment](#deployment). --- ## Architecture ```mermaid flowchart LR subgraph client["MCP client (agent)"] A["Claude Code / Cline / Cursor / custom agent"] end subgraph svc["bedrock-search (this repo, Go)"] B["MCP server<br/>Streamable HTTP /mcp"] BA["Bearer-token auth"] S["search engine"] F["fetch engine<br/>(internal/cdp + sigv4ws)"] BR["browser engine<br/>(optional, stateful)"] end subgraph aws["AWS Bedrock AgentCore"] GW["AgentCore Gateway<br/>(web-search connector)"] WS["Web Search Tool<br/>(Amazon web index + KG)"] BT["Browser Tool<br/>(remote headless Chrome)"] end A -- "MCP + Authorization: Bearer" --> BA --> B B --> S -- "SigV4 MCP tools/call" --> GW --> WS B --> F -- "SigV4 WS / raw CDP" --> BT B -.-> BR -. "SigV4 WS / raw CDP" .-> BT ``` **Request flow for `search`:** ```mermaid sequenceDiagram participant C as MCP client participant S as bedrock-search participant G as AgentCore Gateway participant W as Web Search Tool C->>S: tools/call search {query} + Bearer token S->>S: validate token S->>G: SigV4-signed MCP tools/call (web-search___WebSearch) G->>W: invoke managed connector W-->>G: results (title, url, snippet, date) G-->>S: MCP result (content[].text JSON) S-->>C: raw results, verbatim ``` **Request flow for `fetch`:** ```mermaid sequenceDiagram participant C as MCP client participant S as bedrock-search participant B as AgentCore Browser Tool C->>S: tools/call fetch {url} + Bearer token S->>B: StartBrowserSession B-->>S: automation stream (wss://…) S->>B: SigV4-signed WS upgrade → raw CDP S->>B: Target.createTarget / Page.navigate / Runtime.evaluate B-->>S: page title + readable text S->>B: StopBrowserSession S-->>C: {url, title, text} ``` --- ## Why a custom backend (vs. calling AgentCore directly) - The **Web Search Tool** is only reachable as an MCP tool **behind an AgentCore Gateway** — there is no public `InvokeWebSearch` API. This server fronts it so any MCP client can use it with a simple Bearer token. - The **Browser Tool**'s automation stream is **CDP over a SigV4-signed WebSocket**. Standard Go CDP libraries can't inject SigV4 headers on the WS upgrade, so this repo owns the dial (`internal/sigv4ws`) and speaks CDP itself (`internal/cdp`). The browser runs **remotely in AgentCore** — there is no Chromium or Node in the container image. --- ## Repository layout ``` backend/ Go service (the MCP server, engines, CDP client). See backend/README.md. pkg/app/ shared handler builder used by BOTH the server and the Lambda entry. aws/lambda/ AWS Lambda entrypoint — reuses backend/pkg/app verbatim. charts/ Helm chart: Deployment, Service, token Secret, HTTPRoute, ServiceAccount (IRSA). scripts/ setup_aws.sh (Gateway), setup_irsa.sh (IAM role), deploy.sh, mcp.json. .github/ CI: lint, build/publish image + chart, deploy to EKS. ``` --- ## Prerequisites - An **AWS account** with **Bedrock AgentCore** available. The Web Search Tool and Browser Tool are currently **`us-east-1`** only — set the `*.region` values to `us-east-1` even if you deploy the service elsewhere. - AWS CLI v2 + `boto3`/`botocore` (a recent version that knows the AgentCore `connector` target type), `jq`, and credentials with permission to create AgentCore gateways + IAM roles for the one-time setup. - For the Kubernetes path: an EKS cluster with an OIDC provider, an Envoy Gateway (or adapt the `HTTPRoute`), `kubectl`, `helm`, `docker`. - For the Lambda path: just the AWS CLI + Go (`go build`) + `zip`. --- ## Enable on your own AWS account ### Step 1 — Create the AgentCore Gateway (for `search`) The Web Search Tool is exposed through an AgentCore Gateway with the managed `web-search` connector. `scripts/setup_aws.sh` creates the gateway, its execution role, and the connector target, then prints the gateway's MCP URL: ```bash AWS_REGION=us-east-1 ./scripts/setup_aws.sh # ... prints: # Gateway MCP endpoint (set as search.gatewayUrl): # https://<your-gateway-id>.gateway.bedrock-agentcore.us-east-1.amazonaws.com/mcp ``` Copy that URL into `charts/values.yaml` (`bedrockSearch.search.gatewayUrl`) or pass it as the `SEARCH_GATEWAY_URL` env var for Lambda. > The Browser Tool (for `fetch` / `browser`) needs **no setup** — it uses the > built-in `aws.browser.v1` browser. ### Step 2 — Generate API tokens Clients authenticate with `Authorization: Bearer <token>`. Generate strong tokens and keep them out of source control: ```bash openssl rand -hex 32 ``` ### Step 3 — Deploy Pick one of the two paths below. --- ## Deployment ### Option A — Kubernetes / EKS (long-running, supports the stateful `browser` tool) 1. **Create the IRSA role** (gives the pod least-privilege AWS credentials — AgentCore browser sessions + `InvokeGateway`, nothing else): ```bash AWS_REGION=<cluster-region> EKS_CLUSTER=<cluster-name> ./scripts/setup_irsa.sh # -> prints the role ARN; set it as bedrockSearch.irsaRoleArn in charts/values.yaml ``` 2. **Edit `charts/values.yaml`**: set `image.repository`, `host`, `search.gatewayUrl`, `irsaRoleArn`, and supply tokens at deploy time. 3. **Build, push, and deploy**: ```bash GHCR_OWNER=<your-org> ./scripts/deploy.sh all \ --set 'bedrockSearch.auth.tokens={REPLACE_TOKEN_1,REPLACE_TOKEN_2}' ``` `deploy.sh all` builds the arm64 image, pushes the Helm chart (OCI), and runs `helm upgrade`. The chart renders the tokens into a Kubernetes `Secret` and binds the IRSA ServiceAccount. Your MCP endpoint is then `https://<host>/mcp`. ### Option B — AWS Lambda (serverless; ideal for `search` + `fetch`) The same Go code runs on Lambda behind a **Function URL**: ```bash SEARCH_GATEWAY_URL="https://<your-gateway-id>.gateway.bedrock-agentcore.us-east-1.amazonaws.com/mcp" \ AUTH_TOKENS="REPLACE_TOKEN_1,REPLACE_TOKEN_2" \ AWS_REGION=us-east-1 \ ./scripts/deploy.sh lambda # -> prints the Function URL + the MCP endpoint (<url>/mcp) ``` `deploy.sh lambda` cross-compiles `provided.al2023`/arm64, zips a `bootstrap`, creates a least-privilege execution role, and configures a Function URL. > **Note:** the MCP transport runs in *stateless* mode on Lambda (each > invocation is independent), so the multi-call `browser` tool isn't suitable > there — keep `BROWSER_ENABLED=false`. `search` and `fetch` are stateless and > ideal. Some AWS Organizations block anonymous Function URLs via an SCP; if so, > front the function with an API Gateway HTTP API (payload v2.0, no code change). --- ## Using the server from an MCP client The server exposes bare tools `search` / `fetch` (and `browser` when enabled). MCP clients **prefix the configured server key** to the tool name, so with key `web` they surface as `web_search` / `web_fetch` / `web_browser`. `scripts/mcp.json`: ```json { "mcpServers": { "web": { "type": "http", "url": "https://<your-host>/mcp", "headers": { "Authorization": "Bearer <your-token>" } } } } ``` Or via the Claude Code CLI: ```bash claude mcp add --transport http web https://<your-host>/mcp \ --header "Authorization: Bearer <your-token>" ``` ### Per-client `browser` toggle (`?browser=off`) The `browser` tool is enabled by default (server-wide). A specific client can **hide it for its own connection** by appending `?browser=off` to the MCP URL — no server change needed: ```json { "mcpServers": { "web": { "type": "http", "url": "https://<your-host>/mcp?browser=off", "headers": { "Authorization": "Bearer <your-token>" } } } } ``` That client then sees only `search` / `fetch`. Accepted "off" values: `off`, `false`, `0`, `no`, `disable`. (Any other value, or omitting the parameter, keeps the `browser` tool as configured server-side.) ### Quick curl check ```bash curl -sS -X POST "https://<your-host>/mcp" \ -H "Authorization: Bearer <your-token>" \ -H "Content-Type: application/json" \ -H "Accept: application/json, text/event-stream" \ -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' ``` --- ## The `browser` tool Enabled by default. Disable it server-wide via `bedrockSearch.browser.enabled: false` (chart) / `browser.enabled: false` (config), or hide it per-connection with `?browser=off` on the MCP URL (see above). It is a single, action-switched tool over the AgentCore Browser Tool: | Category | Actions | |----------|---------| | Session | `init_session` (returns a `sessionId`), `close` | | Navigation | `navigate`, `refresh`, `back`, `forward` | | Interaction | `click`, `type`, `press_key`, `evaluate` | | Extraction | `get_text`, `get_html`, `screenshot`, `get_cookies` | | Advanced | `set_cookies`, `new_tab`, `switch_tab`, `close_tab`, `list_tabs`, `network_intercept`, `execute_cdp` | Sessions are server-managed (pass the `sessionId` to every call after `init_session`), reaped on idle (`idleTimeoutSec`), and capped (`maxSessions`). Every operation logs with the session id, and CDP calls retry once on transient failure (CDP can be flaky by nature). Because sessions are stateful, this tool is for the **Kubernetes** path, not Lambda. --- ## Security notes - **Never commit real Bearer tokens.** `charts/values.yaml` ships with an empty token list — supply tokens at deploy time (`--set`) or via a sealed/external secret. `/healthz` is intentionally open; `/mcp` always requires a token when any token is configured. - The pod/function gets AWS access via **IRSA / a least-privilege execution role** — no static AWS keys in the image. - Treat the MCP endpoint as a credentialed API: put it behind TLS (the chart's `HTTPRoute` assumes TLS terminates at your gateway) and rotate tokens. --- ## License See [LICENSE](LICENSE).