Content
<div align="center">
# 📖 howto
### An open library of everyday procedural knowledge
**Structured how-to recipes for training agents 🤖 and robots 🦾, readable by humans 🧑**
[](LICENSE)
[](https://creativecommons.org/licenses/by/4.0/)
[](INDEX.md)
[](journeys/)
[](packs/)
[](domains.json)
[](CONTRIBUTING.md)
[**Use it**](#use) · [**Why**](#why) · [**What's inside**](#inside) · [**Packs**](#packs) · [**Journeys**](#journeys) · [**Quick start**](#quickstart) · [**Roadmap**](#roadmap) · [**Contributing**](#contributing) · [**Citing**](#citing)
</div>
---
## 🐘 How do you put an elephant into a fridge?
<div align="center">
<img src="assets/elephant-in-fridge.png" alt="An elephant crammed into a fridge with the door held open" width="420">
</div>
Easy:
> 1. Open the fridge. 2. Put the elephant in. 3. Close the fridge.
That is how most datasets think tasks work. Here is how `howto` writes it:
```markdown
## Steps
1. **Open the fridge.** → *Expect:* door open, shelves visible, light on.
2. **Insert the elephant.** [BRANCH: elephant does not fit → F1]
3. **Close the fridge.** → *Expect:* door latched; no trunk protruding.
## Failure modes & recovery
- **F1 Elephant exceeds fridge capacity:** acquire a larger fridge, or a smaller elephant.
## Verification
Opening the fridge reveals one chilled elephant.
```
**Things are always more complicated than "open, put, close", and the complications are exactly the knowledge agents are missing.** Every human knows the real steps, the expected observations, the branches, the failure modes. No corpus writes them down. This one does.
And here is a real one, from [`transit/ride-a-subway.md`](transit/ride-a-subway.md):
```markdown
3. **Obtain or validate fare.** [BRANCH: tap-to-pay bank card | transit card with
balance | ticket machine | transit app QR] At a machine: select destination or fare
type, pay, take the ticket. ⚠️ *Irreversible:* multi-day passes and non-refundable
tickets — confirm fare type before paying. → *Expect:* you hold a valid fare medium.
4. **Pass the fare gate.** Tap or insert; walk through on green/open. → *Expect:* gate
opens and (on card systems) displays balance or fare; if rejected, F1.
...
- **F2 Boarded the wrong direction:** exit at the next station, cross to the opposite
platform (may require staying inside the paid area), reverse.
```
---
<a id="use"></a>
## 🚀 Use it, don't just read it
Two ways to get value in under a minute:
- **🔎 Browse** — a live, searchable viewer over the whole corpus, filter by domain, and **copy any recipe as agent context** with one click: **[shirleyhuang11.github.io/howto](https://shirleyhuang11.github.io/howto/)**
- **🤖 Give it to your agent** — the **[MCP server](mcp/)** exposes `search_howto` / `get_howto` so Claude Code, Claude Desktop, or Cursor can pull the verified steps at inference time — warnings and all — instead of guessing:
```bash
pip install -r mcp/requirements.txt
claude mcp add howto -- python3 "$(pwd)/mcp/server.py"
```
---
<a id="why"></a>
## 🧠 Why this exists
Web agents, computer-use agents, and household robots fail at everyday tasks less from weak reasoning than from **missing grounded procedural knowledge**: canonical step order, the observation that confirms each step worked, the decision branches, the recovery paths, and which steps are irreversible.
[`agency-agents`](https://github.com/msitarzewski/agency-agents) (134k stars) covers **who an agent is**. `howto` is the complement: **what an agent knows how to do**. Ride a subway, buy and return a product, create an account, make coffee, load a dishwasher, wait in line, pay at a cashier.
The schema turns prose into training substrate. One corpus yields, simultaneously:
| Training product | Derived from |
|---|---|
| SFT / midtraining planning traces | Steps + expected observations |
| Agentic RL tasks with **verifiable rewards** | Goal + Preconditions + Verification |
| Curriculum / skill DAG | `prerequisites` links between recipes |
| Inference-time skill library (RAG) | Recipe chunks |
| Contamination-controlled evals | Held-out recipes and locale variants |
| Robot task plans & sim task generation | Embodied `objects` / `affordances` / `workspace` / `safety` |
| **Long-horizon planning traces** | **[Journeys](journeys/): milestone DAGs over recipes** (gates, waits, re-plan triggers) |
---
<a id="quickstart"></a>
## ⚡ Quick start
```bash
git clone https://github.com/ShirleyHuang11/howto.git && cd howto
pip install pyyaml # the only dependency
./scripts/validate.sh # validate every recipe against the schema
python3 scripts/stats.py # per-domain coverage dashboard
python3 scripts/export.py --format sft # instruction-tuning pairs (JSONL)
python3 scripts/export.py --format eval # task specs with success criteria (JSONL)
python3 scripts/export.py --format json # full structured recipes
```
Or just read one: [`transit/ride-a-subway.md`](transit/ride-a-subway.md) · [`embodied/kitchen/load-a-dishwasher.md`](embodied/kitchen/load-a-dishwasher.md) · [`daily/social/pay-at-a-cashier.md`](daily/social/pay-at-a-cashier.md)
---
<a id="inside"></a>
## 🗂️ What's inside
One task = one markdown file, separated by domain ([`domains.json`](domains.json) is the registry, [`INDEX.md`](INDEX.md) is the full linked index, and the frontmatter is formally specified in [`schema/recipe.schema.json`](schema/recipe.schema.json)). Every recipe shares one skeleton:
```
Goal → Preconditions → Steps (every step ends with → *Expect:* …)
→ Decision points → Failure modes & recovery → Verification
→ Variations (locale/platform) → Safety & privacy (⚠️ irreversible steps inline)
```
<details open>
<summary><b>🧠 Common sense — <code>daily/</code></b> (the largest track: what everyone knows and no one wrote down)</summary>
<br>
| Subdomain | Recipes |
|---|---|
| `self-care/` | brush your teeth · floss · shower · tie shoelaces · bedtime · trim nails · dress for the weather |
| `food/` | boil water · make tea · cook rice · fry an egg · sandwich · cut an onion · microwave · instant noodles · pack a lunch · store leftovers · read food labels |
| `home/` | do the laundry · make a bed · sweep & mop · vacuum · clean a bathroom · sort recycling · unclog a drain · change a lightbulb · change batteries |
| `social/` | pay at a cashier · wait in line · order at a restaurant · split a bill · answer a phone call · greet a neighbor · give directions · ask for help in a store · small talk · borrow & return |
| `errands/` | use an ATM · mail a letter · refuel a car · vending machine · self-checkout · buy groceries · get a haircut · public library |
| `parenting/` | feed & burp a newborn · swaddle · safe infant sleep · introduce solids · potty-train · handle a tantrum · teach a kid to tie shoes / ride a bike · pack a school lunch · hire a babysitter |
</details>
<details>
<summary><b>🦾 Robot track — <code>embodied/</code></b> (recipes that compile into simulator tasks)</summary>
<br>
| Subdomain | Recipes |
|---|---|
| `kitchen/` | make drip coffee · load a dishwasher · set a table · store groceries · make toast |
| `household/` | fold a t-shirt · water houseplants · take out the trash · tidy a room |
| `mobility/` | open a door · use an elevator · carry & deliver · cross a street |
| `care/` | fetch an item for a person |
Embodied frontmatter specifies the sim scene:
```yaml
objects: [dishwasher, plates, cutlery, detergent-pod]
affordances: [door-open, rack-slide, grasp, place]
workspace: kitchen
safety: {sharp_objects: true, fragile: [glasses], human_proximity: pause}
```
</details>
<details>
<summary><b>💻 Digital — <code>accounts/</code> <code>shopping/</code> <code>digital/</code> <code>finance/</code></b></summary>
<br>
create an account · log in · enable 2FA · recover a password · delete an account · security review · buy & return a product · track a delivery · compare before buying · install an app · unsubscribe · back up files · connect to wifi · update safely · pay a bill · dispute a charge · send money to a friend
**Account security & privacy** (`accounts/security/`): migrate 2FA to a new phone · recover a hacked email · secure an account after a breach · find & remotely wipe a lost device · set up a legacy contact · revoke third-party app access · opt out of data-broker sites — the identity/access flows an agent has to get exactly right.
**Computer-use atomics** (the leaf layer agents can perform directly): rename a file · find text on a page · copy-paste · reopen a closed tab · a spreadsheet formula · crop an image · cc/bcc an email · take a region screenshot — 100 single-action primitives under `digital/`.
**Agentic commerce** (buy/sell/transact flows written as verifiable-reward tasks): sell on a marketplace · price an item to sell fast · stack coupons & cashback · bid on an auction · checkout under a budget · audit subscriptions · negotiate an APR or a bill · maximize card rewards · book the cheapest fare · use a virtual card — 200 transactional recipes across `shopping/`, `finance/optimize/`, `travel/booking/`, `digital/transactions/`, each with an explicit target and a checkable end-state.
</details>
<details>
<summary><b>💼 Business ops — <code>business/</code></b> (atomic workflows an agent can run end-to-end)</summary>
<br>
**Invoicing & books:** create an invoice · send a payment reminder · reconcile a statement · record an expense · issue a refund · P&L report · **CRM & sales:** add a lead · move a deal stage · log a call · forecast sales · **HR & hiring:** post a job · screen a resume · send an offer · run payroll · onboard a hire · **Project & support:** create a project · triage a ticket · run a standup · close a sprint · **Meetings, marketing & e-commerce:** schedule across time zones · set up an email campaign · process an order · handle a chargeback · update inventory · reprice inventory · run a shopping ad · fulfill a dropship order · calculate a product margin · flag a fraudulent order
</details>
<details>
<summary><b>🎓 Learning & academics — <code>education/</code></b> (the 14th domain)</summary>
<br>
**Study skills:** active recall · spaced repetition · Cornell notes · the Pomodoro technique · make a one-page study sheet · **Test prep:** SAT · ACT · GRE · GMAT · TOEFL/IELTS · manage test anxiety · **Language learning:** set a goal · daily habit · language exchange · comprehensible input · **Academic writing:** thesis statement · literature review · APA/MLA/Chicago citations (with copyable examples) · avoid plagiarism · **Admissions & aid:** build a college list · Common App essay · FAFSA · scholarships · compare aid offers · grad-school SOP · **Online & K-12:** finish a MOOC · earn a certificate · help with homework · IEP/504 meetings
</details>
<details>
<summary><b>🏋️ Fitness & exercise — <code>fitness/</code></b> (the 15th domain; coach-accurate form, measurable "done when")</summary>
<br>
**Strength:** squat · deadlift · bench · overhead press · row · pull-up & push-up progressions · progressive overload · **Cardio:** run a 5k → marathon · zone-2 · HIIT · rowing · pacing · **Mobility:** full-body & targeted stretches · dynamic warmups · hip/ankle/thoracic mobility · **Home & bodyweight:** no-equipment circuits · calisthenics · band & kettlebell routines · hotel-room workouts · **Sports skills:** basketball · tennis · golf · swimming · cycling · skating · **Recovery:** soreness · injury prevention · protein/sleep/hydration · rest days · **Planning:** goal-setting · splits · tracking · fat-loss/muscle-gain plans · building the habit
</details>
<details>
<summary><b>💻 Software engineering — <code>engineering/</code></b> (the 16th domain — the coding-agent workflows, agent-executable with a programmatic "done when")</summary>
<br>
The tasks a Claude Code / Cursor agent runs all day, each with a *checkable* Verification (tests exit 0, CI green, `curl` 200, migration applied):
**Version control:** open & review a PR · resolve a merge conflict · rebase · bisect a regression · recover a lost commit · **Testing:** write a failing test first · mocks & fixtures · coverage · fix a flaky test · **Debugging:** read a stack trace · reproduce a bug · memory leaks · race conditions · production-only bugs · **CI/CD:** pipelines · caching · matrix builds · required checks · deploy-on-merge · **Deploy:** zero-downtime · blue-green · canary · rollback · feature flags · safe DB migrations · **Database:** SQL & joins · schema design · indexes · backup/restore · prevent SQL injection · **API:** REST/GraphQL · OAuth2 · rate limits · retries with backoff · webhook signature verification · **Containers:** Dockerfiles · compose · Kubernetes · healthchecks · debug a crashloop · **Observability:** structured logs · tracing · SLOs · alerting · postmortems · **Security:** secret handling · key rotation · dependency scanning · XSS/CORS · JWTs. Destructive ops (prod migrations, key rotation, history rewrite) are marked ⚠️ and `risk: high`.
</details>
<details>
<summary><b>🤖 AI & LLM engineering — <code>ai/</code></b> (the 17th domain: how to build LLM apps, RAG systems, and agents)</summary>
<br>
Each of the 200 recipes carries a checkable Verification: an eval clears its threshold, structured output validates against a schema, or retrieval returns the gold document in the top k.
**Prompting:** system prompts · few-shot · chain-of-thought · structured JSON output · prompt-injection defense · **RAG:** chunking · embeddings · vector databases · hybrid search · reranking · answer faithfulness · **Agents:** tool-calling · ReAct loops · memory · planning · MCP servers · guardrails · human-in-the-loop · **Evals:** eval sets · LLM-as-judge · regression evals · contamination checks · significance tests · **Fine-tuning:** dataset prep · LoRA · DPO · quantization · the fine-tune vs RAG vs prompt decision · **LLMOps:** API calls · streaming · caching · model routing · cost caps · PII redaction · **Data:** cleaning · labeling · drift detection · pipelines · anonymization · **Multimodal:** captioning · OCR · transcription · image generation and QA · **Safety:** red-teaming · jailbreak filters · runtime injection detection · output moderation. Steps that send data to a third-party API or deploy to production are marked ⚠️ and `risk: high`.
</details>
<details>
<summary><b>🚇 Physical & services — <code>transit/</code> <code>travel/</code> <code>communication/</code> <code>government/</code> <code>healthcare/</code> <code>housing/</code></b></summary>
<br>
ride a subway · take a bus · hail a rideshare · navigate with maps · book a flight · book the cheapest fare for your dates · rebook after a price drop · claim delayed-flight compensation · check in · book a hotel · send a package · schedule a meeting · renew a passport · book a doctor's appointment · refill a prescription · set up utilities
**Car care** (`transit/vehicle/`): change a flat · jump-start a dead battery · check tire pressure & oil · replace wipers · handle a blowout or an overheating engine · what to do after a fender bender — hands-on, safety-first (hazard steps marked). **Civic & legal** (`government/civic/`): REAL ID · passports · small-claims filing · car titles · vital records · FOIA. **Health navigation** (`healthcare/navigation/`): patient portals · specialist referrals · advance directives · pre-op & discharge · financial assistance — the paperwork and coordination around care, not just first aid. **Home owning** (`housing/owning/`): mortgage preapproval to closing · sell & stage a home · HVAC/water-heater/roof upkeep · HOA · refinancing · roommate agreements.
</details>
📊 Live coverage anytime: `python3 scripts/stats.py`
---
<a id="packs"></a>
## 🎒 Packs — grab a whole kit, install it as a skill
Nobody assembles a scam-defense toolkit from a 2,900-recipe index. A [**pack**](packs/) is a curated, cross-domain kit for one real situation — and it installs into your agent as a self-contained **skill** in one command:
```bash
python3 scripts/build_skills.py scam-and-fraud-defense # → packs/skills/.../SKILL.md
```
Drop it into Claude Code (`.claude/skills/`) or point your agent at the [MCP server](mcp/) and say *"use the scam-defense pack."* Your agent now follows verified steps — *Expect* observations and ⚠ warnings included — instead of guessing. Every pack's recipes are [machine-checked to resolve](scripts/validate_packs.py).
| 🛡️ [Scam & Fraud Defense](packs/scam-and-fraud-defense.md) | 🚑 [First-Aid & Emergencies](packs/first-aid-emergencies.md) | 📦 [Move Into a New Place](packs/move-into-a-new-place.md) | 🍳 [Kitchen Starter](packs/kitchen-starter.md) |
|---|---|---|---|
| 🦾 **[Robot Manipulation Primitives](packs/robot-manipulation-primitives.md)** | 🔐 **[Digital Security Basics](packs/digital-security-basics.md)** | 🚗 **[New Car Owner](packs/new-car-owner.md)** | 💰 **[Money Starter](packs/money-starter.md)** |
---
<a id="journeys"></a>
## 🗺️ Journeys — long-horizon tasks
Some tasks a single recipe can't hold: they span months, wait on third parties, accumulate state, and force **re-planning** when new information arrives. A [**journey**](journeys/) is a temporal DAG whose leaf nodes are the recipes above — each milestone carries a **gate** (what unblocks it), a **wait** (the third-party clock), a checkable **verify**, and a **re-plan trigger**. And **every leaf is a real recipe id, machine-checked** by [`scripts/validate_journeys.py`](scripts/validate_journeys.py) — 66 references across the four below, all resolving — so a journey can't decay into a listicle. Counted and validated separately from recipes.
| Journey | Horizon | Milestones | What's hard about it |
|---|---|---|---|
| [`recover-from-identity-theft`](journeys/recover-from-identity-theft.md) | 1mo–12mo | 8 | parallel bureau tracks, 30-day dispute clocks, a monitoring loop that can restart everything |
| [`buy-a-home`](journeys/buy-a-home.md) | 2mo–12mo | 7 | irreversible offer and closing; financing gated on appraisal + underwriting |
| [`get-out-of-debt`](journeys/get-out-of-debt.md) | 3mo–36mo | 6 | a control loop, not a one-shot; reward delayed over years |
| [`job-search-and-switch`](journeys/job-search-and-switch.md) | 4wk–4mo | 6 | never resign before the offer is signed; constant re-planning |
Export the planning traces with `python3 scripts/export.py --format journeys`. New journeys welcome — see [`journeys/README.md`](journeys/README.md).
---
## 📐 Design principles
1. **Every step states its expected observation.** A step list without expectations is prose; the *Expect:* clause is what makes a recipe executable and trainable.
2. **Verification is a predicate, not a vibe.** "You are done when…" must be checkable by a program or a stranger — it compiles into a reward function.
3. **Irreversible steps are marked ⚠️ inline** — payments, deletions, hot liquids, handed-over packages. Agents must learn *where to stop and confirm*, and that signal lives in the data.
4. **Failure modes are first-class.** Real procedures go wrong in known ways; the recovery paths are half the knowledge.
5. **Generic first, locale variants second.** The `generic` recipe stands alone; `us-nyc` / `jp-tokyo` / `cn-beijing` variants layer on top.
6. **Multimodal, with text as the floor.** Recipes can attach diagrams, photos, and clips per step via the `media` frontmatter (roles: action-demo, expected-observation, diagram, warning, overview), each with a full alt description so every recipe still trains text-only models. See the fare-gate diagram in [`transit/ride-a-subway.md`](transit/ride-a-subway.md) and the rack layout in [`embodied/kitchen/load-a-dishwasher.md`](embodied/kitchen/load-a-dishwasher.md).
---
<a id="roadmap"></a>
## 🗺️ Roadmap
The corpus spans **2937 recipes across all 17 domains** and keeps growing. Where help lands hardest:
- [ ] Locale variant packs: `jp-tokyo`, `cn-beijing`, `de-berlin`, and yours
- [ ] `reviewed` and `verified` promotions (agent-executability evidence; see CONTRIBUTING)
- [ ] Embodied recipes → simulator task compiler
- [ ] Multimodal coverage: a diagram for every embodied recipe
- [ ] Pre-built SFT/eval exports as tagged releases
---
<a id="contributing"></a>
## 🤝 Contributing
One recipe = one markdown file = one PR.
1. Copy [`TEMPLATE.md`](TEMPLATE.md) into the right domain directory
2. Write the procedure you actually know (original content only)
3. `./scripts/validate.sh` until green ✅
4. Open the PR
The short rules: every step needs an *Expect:* · verification must be checkable · irreversible steps get ⚠️ · embodied recipes need the robot frontmatter. Full rules in [CONTRIBUTING.md](CONTRIBUTING.md).
**🌍 Locale expertise is especially welcome.** Recipes for how things work in *your* city make agents useful beyond the defaults.
---
<a id="citing"></a>
## 📎 Citing and using the data
Recipe content is CC-BY-4.0: training (including commercial) is fine with attribution; see [LICENSE](LICENSE). Export formats are documented in [`scripts/export.py`](scripts/export.py)'s header, and each JSONL record carries the recipe `id` for attribution. If you use the corpus in research:
```bibtex
@misc{howto2026,
title = {howto: an open library of everyday procedural knowledge for training agents and robots},
author = {{howto contributors}},
year = {2026},
url = {https://github.com/ShirleyHuang11/howto}
}
```
---
<div align="center">
Code: [MIT](LICENSE) · Content: [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)
**⭐ Star this repo if your agent ever confidently deleted the wrong file, skipped the step that actually mattered, and reported success.**
</div>
Connection Info
You Might Also Like
No recommendations available