HostShift

 # HostShift Deterministic Ubuntu and Debian server migrations with auditable planning and verification. HostShift discovers a live server over SSH, builds a reviewable migration profile, prepares a clean target, streams data across, and verifies the result. It enforces a strictly read-only source policy: no source-side writes, no `sudo`, no service restarts, no package installs, no firewall changes, no keys added, no snapshots, no maintenance mode. [Documentation](https://hostshift.karacabay.com) | [Install](#install) | [AI Integrations](#ai-integrations) | [What You Get](#what-you-get) | [How It Works](#how-it-works) | [Validation](#validation) | [Safety Model](#safety-model) --- HostShift is a CLI-first migration tool and Codex plugin for practical Linux web-stack moves. It is built for the annoying real case: you have working production services on one Ubuntu or Debian host, you need to recreate them somewhere else, and stopping or modifying the source would be unacceptable. The migration engine is deterministic Go code. The Codex plugin is an operator layer that helps Codex follow the workflow, keep the source read-only contract, and run the right `hostshift` commands in the right order. ## Before / After Manual server move: > Inspect services by hand, copy configs, guess package names, dump databases, chase missing firewall rules, forget an SSH setting, retry under pressure, and hope the old server still behaves. HostShift move: ```bash hostshift discover --source old-server --name rehearsal --profile rehearsal.profile.yaml --json hostshift plan --profile rehearsal.profile.yaml --target new-server --json hostshift explain --profile rehearsal.profile.yaml --target new-server --json hostshift prepare --profile rehearsal.profile.yaml --target new-server --json hostshift sync --profile rehearsal.profile.yaml --target new-server --json hostshift verify --profile rehearsal.profile.yaml --target new-server --json ``` Same goal. Less guessing. Source stays read-only. ```text source mutation policy blocked target mutations planned and reviewable transfer path streamed over SSH profile format YAML + JSON Schema verification workload checks, not vibes resume/audit state + JSONL journal ``` ## Install Use release binaries for migration rehearsals and production work. Full release verification lives in [docs/install.md](./docs/install.md). ```bash tar xzf hostshift_<version>_<os>_<arch>.tar.gz install -m 0755 hostshift /usr/local/bin/hostshift hostshift version ``` Build from source during development: ```bash git clone https://github.com/oguzhankrcb/HostShift.git cd HostShift make build ./dist/hostshift version ``` Run directly from source: ```bash go run ./cmd/hostshift version go run ./cmd/hostshift doctor --source old-server --target new-server --json go run ./cmd/hostshift discover --source old-server --name example --profile example.profile.yaml --json go run ./cmd/hostshift plan --profile examples/profile.yaml --target new-server --json go run ./cmd/hostshift explain --profile examples/profile.yaml --target new-server --json go run ./cmd/hostshift prepare --profile examples/profile.yaml --target new-server --json go run ./cmd/hostshift sync --profile examples/profile.yaml --target new-server --json go run ./cmd/hostshift verify --profile examples/profile.yaml --target new-server --json go run ./cmd/hostshift profile migrate --input examples/profile.yaml --output /tmp/profile.v2.json ``` With an installed binary, replace `go run ./cmd/hostshift` with `hostshift`. > [!IMPORTANT] > `prepare`, `sync`, and `verify` default to dry-run mode and write resumable state. Add `--apply` only after reviewing blockers, actions, streams, and rollback metadata. ## Codex Plugin HostShift ships as a Codex plugin under [plugins/hostshift](./plugins/hostshift). The plugin does not replace the CLI. It gives Codex the migration workflow, safety rules, and operator guidance while the deterministic `hostshift` binary performs discovery, planning, transfer, and verification. Install from this repo marketplace: ```bash codex plugin marketplace add https://github.com/oguzhankrcb/HostShift.git codex plugin add hostshift@hostshift ``` For local development from a checkout: ```bash codex plugin marketplace add . codex plugin add hostshift@hostshift ``` Start a new Codex thread after installing or updating the plugin so the bundled `migrate-server` skill is loaded. ## AI Integrations HostShift exposes AI integrations as operator layers around the Go CLI: - Codex plugin: `plugins/hostshift` - MCP stdio server: `hostshift mcp stdio` - Claude Desktop example: [integrations/claude/claude_desktop_config.example.json](./integrations/claude/claude_desktop_config.example.json) Validate the MCP and Claude integration surface: ```bash hostshift mcp doctor --json hostshift capabilities --json ``` MCP tools are intentionally non-apply: they can run discovery, planning, AI-friendly explanation, structured review, dry-runs, cutover dry-runs, profile migration, source policy introspection, capability catalog inspection, and rollback metadata. The MCP server also exposes the `hostshift_migration_operator` prompt and `hostshift://source-safety`, `hostshift://migration-workflow`, and `hostshift://capabilities` resources so Claude or other clients can load the safety workflow directly. Target mutation still requires a reviewed human CLI command. ## Supported Platforms The first support matrix focuses on current Ubuntu and Debian server targets. | Family | Supported releases | | --- | --- | | Ubuntu | 22.04 LTS, 24.04 LTS, 25.10 interim, 26.04 LTS | | Debian | 12, 13 | EOL targets are blocked by default. EOL sources can still be read when the migration remains source-safe. Source and target versions do not need to match; HostShift checks platform capabilities instead of assuming identical images. ## What You Get | Area | What HostShift does | | --- | --- | | Source discovery | Reads OS, packages, services, Docker, web server, SSH, firewall, and database facts through allowlisted commands. | | Profiles | Writes v2 YAML profiles with JSON Schema validation, workload metadata, checks, target settings, and secret env references. | | Planning | Emits reviewable `Action{id, phase, hostRole, impact, command, preconditions, rollback}` records before apply. | | Target prepare | Plans target-only package installs and config writes for Docker, databases, Nginx/Apache, SSH, UFW, nftables, and app dependencies. | | Transfers | Streams files, container images, MySQL/MariaDB, and PostgreSQL from source stdout to target stdin. | | Verification | Checks HTTP, Laravel database connectivity, files, DB scalar queries, systemd services, firewall rules, nftables, and Nginx config. | | State | Keeps resumable run state and JSONL audit logs. | | Safety | Blocks source-side mutation attempts in code and tests. | See [examples/profile.v2.yaml](./examples/profile.v2.yaml) and [examples/web-stack-v2.yaml](./examples/web-stack-v2.yaml) for real profile shapes. ## Workloads Current workload coverage includes: - Docker Compose projects - standalone Docker containers - Docker named volumes through explicit snapshot, disposable, database-backed, or external strategies - bind-mounted file sets - streamed Docker images - MySQL and MariaDB - PostgreSQL - Redis through existing RDB snapshots or read-only replica streams - Nginx configuration and reload validation - Apache vhost activation and reload validation - Caddy configuration and reload validation - systemd application service cutover and service checks - cron file transfer, target cron package planning, and target cron reload - PHP-FPM config transfer, target package planning, and target service reload - Supervisor config transfer, target package planning, and target reread/update - Fail2ban config transfer, target package planning, and target service reload - Memcached config transfer, target package planning, and target service restart - RabbitMQ config transfer, target package planning, and target diagnostics; live queues/messages are not migrated - Certbot/Let's Encrypt state transfer, target package planning, certificate listing, and renewal timer activation when available - Logrotate config transfer, target package planning, and target config validation - SSH server settings - UFW and nftables firewall rules - Laravel-style database connectivity checks Redis workloads are blocked unless the profile names an existing RDB snapshot or a read-only replica endpoint. HostShift never runs source-side `SAVE`, `BGSAVE`, or Redis config changes. Discovered Docker named volumes are blockers until the profile explicitly selects `snapshot`, `disposable`, `database-backed`, or `external`. The `snapshot` strategy reads a tar file that already exists on the source and extracts it into a reviewed target path; HostShift never creates a source-side volume snapshot. ## How It Works 1. **Doctor** checks connectivity, platform support, and obvious blockers. 2. **Discover** reads source facts through allowlisted, read-only operations. 3. **Profile** records workloads, target config, checks, and env-var secret references. 4. **Plan** turns the profile into reviewable actions and streams. 5. **Explain** summarizes blockers, warnings, streams, target impacts, and safe next actions for AI-assisted review. 6. **Review** emits structured findings, workload-aware recommendations, suggested YAML snippets, an operator checklist, and an AI safety brief. 7. **Prepare** applies target-only package and config changes when approved. 8. **Sync** streams data from source stdout into target-side writes. 9. **Verify** runs target-side checks and records audit output. 10. **Cutover** runs reviewed target-only start actions such as Compose `up` or standalone container `run`. 11. **Rollback/status/resume** report target rollback metadata and continue interrupted runs from state. Sync plans may include streams such as `tar --create -> tar --extract`, existing Docker volume snapshot tar `cat -> tar --extract`, `docker image save -> docker image load`, `mysqldump -> mysql`, and `pg_dump -> pg_restore`. The source side produces stdout; the target side mutates only the target. Interrupted apply runs can be inspected and resumed without replaying completed steps: ```bash hostshift status --state-dir .hostshift --run-id sync-001 --json hostshift resume --profile migration.profile.yaml --state-dir .hostshift --run-id sync-001 --json hostshift resume --profile migration.profile.yaml --state-dir .hostshift --run-id sync-001 --apply --json ``` State is written atomically after every completed action or stream, and a per-run OS lock prevents concurrent apply processes from replaying the same work. Resume refuses to run when the profile, target, blockers, or generated commands no longer match the saved plan fingerprint. If an interrupted command may have partially changed the target, apply remains blocked until the operator explicitly names it with `--retry-failed <action-id>`. ## Validation Quick checks: ```bash make test make test-go make build npm run docs:build npm run docs:compose:config ``` Docker migration matrix: ```bash make test-integration-docker HOSTSHIFT_RUN_DOCKER_MATRIX=1 make test-integration-docker ``` Real VM e2e: ```bash make test-e2e-vm HOSTSHIFT_RUN_VM_E2E=1 make test-e2e-vm make build HOSTSHIFT_RUN_VM_E2E=1 bash tests/e2e/vm/run-vm-e2e.sh --pair 'ubuntu22->debian12' --apply ``` The Docker matrix runs SSH-driven source/target containers. The VM e2e layer boots real Lima instances, captures source snapshots, runs `discover`, `plan`, `prepare`, `sync`, and `verify` over SSH, validates workload parity, reboots the target, verifies persistence, and checks that the source snapshot did not change. Release candidates must satisfy the gates in [docs/validation.md](./docs/validation.md), including source immutability, cross-distro Docker coverage, real VM apply coverage, checksums, and SBOM output. ## Safety Model HostShift's core invariant is strict: > The source server is an immutable observation endpoint. Allowed on source: - read inventory - inspect configuration - stream typed exports to stdout - run allowlisted fact commands Forbidden on source: - `sudo` - writing files - installing packages - restarting or stopping services - changing firewall rules - adding SSH keys - creating snapshots - placing apps into maintenance mode - creating database dump files on disk Known limits are reported instead of hidden. Live filesystem streaming is not a point-in-time snapshot. MySQL dumps can acquire metadata locks. PostgreSQL dumps are consistent per selected database, not across unrelated databases. Workloads that cannot be read safely are blockers, not silent skips. ## Documentation The documentation website is published at [hostshift.karacabay.com](https://hostshift.karacabay.com). If you prefer the full guide format, click there for installation, profiles, workloads, validation, and reference pages. The site source is built with Astro Starlight under [docs-site](./docs-site). ```bash cd docs-site npm install npm run dev ``` Or run it with Docker Compose: ```bash docker compose -f docs-site/compose.yml up --build ``` Open `http://localhost:4321`. Useful docs: - [Install](./docs/install.md) - [Architecture](./docs/architecture.md) - [Threat model](./docs/threat-model.md) - [Validation](./docs/validation.md) - [Release checklist](./docs/release.md) - [Security policy](./SECURITY.md) - [Contributing](./CONTRIBUTING.md) ## Release ```bash make release-snapshot make checksum make sbom ``` `make release-snapshot` uses GoReleaser when it is installed. Without GoReleaser it still builds `dist/hostshift`, writes `dist/checksums.txt`, and generates `dist/hostshift.sbom.spdx.json` from the Go module graph. Tagged GitHub releases use [.github/workflows/release.yml](./.github/workflows/release.yml). SBOM generation is implemented in the Go CLI: ```bash hostshift sbom --output dist/hostshift.sbom.spdx.json --json ``` Before publishing a public tag, complete [docs/release.md](./docs/release.md): hosted CI candidate, self-hosted or local VM apply, clean working tree, and signed checksum verification. ## Status HostShift is in the validation milestone. The Go `hostshift` CLI is the implementation used for discovery, planning, transfer, verification, cutover, and rollback. Cloud and DNS automation are intentionally outside the core. ## License Apache-2.0. See [LICENSE](./LICENSE).