commercial-creator

# Commercial Creator An agentic pipeline that turns a product description into a finished video commercial: creative brief → concepts → script → adversarially-reviewed shot list → styled storyboard keyframes → VO-timed animatic → Seedance 2.0 video generation (with storyboard-composed multi-shot windows) → QC judging with corrective regeneration → assembly → music + expressive VO mix → final cut with themed overlays. A FastAPI server drives it; a single-file web UI and an MCP connector (for Claude Code and other agents) sit on top. Built with a **human-in-the-loop by design**: the brief, the shot list and the animatic are approval gates, and **video generation never spends money without an explicit approval step** — in the UI and in the MCP tools alike. ## What it produces 30–60s commercials/trailers at draft (480–720p) or final (1080p) tier, with: - multi-agent planning: intake strategist, creative director (3 concepts, beat-scripted VO at ~2.3 words/s), a **shot room** where a cinematographer critic and an editor critic attack the draft shot list in parallel before the director revises it - storyboard keyframes (Gemini nano-banana via fal.ai) locked to a style recipe, with product reference images for identity - an **audio-first animatic**: VO takes are generated and measured first, the timeline stretches to fit, and the shot plan reconciles the locked timeline onto Seedance's 4–15s generation windows (scale / trim / split / multi-shot collapse / storyboard-composed windows) - Seedance 2.0 generation via fal.ai with per-route handling (t2v / i2v / reference-composed r2v / last-frame chaining), transient retries, a billing fail-fast, and cancellable queue polling - a 6-dimension vision QC judge that stages corrective regens (never auto-spends) and a **self-improvement loop** that distills recurring failures into durable prompt rules (per-project lessons + global house rules) - ElevenLabs v3 expressive VO (audio tags per beat kind, per-brand voice casting by an agent) + generated music bed, sidechain-ducked under the VO - Remotion-rendered overlays: theme-palette lower thirds and end cards, with a presentation setting (cinematic / balanced / informational) chosen at intake or recommended by the brief agent Typical draft-tier cost: **$10–20 per 30s spot** (Seedance dominates; LLM/images/VO are cents). See `docs/` for the research that shaped the pipeline. ## See it in action Five finished spots produced end-to-end by this pipeline — each was one product description in, one video out (every brand fictional). Uploaded as demos on [The Bearded AI Guy](https://www.youtube.com/@bearded-ai-guy): ### 🎬 [PocketBrew — coffee commercial](https://youtu.be/zSaz230ygl4) (45s, ~$17) The pipeline's first complete spot: theme-styled lower thirds and end card, ElevenLabs music bed ducked under the VO. Every shot generated per-keyframe (image-to-video) from the styled storyboard. <p> <a href="https://youtu.be/zSaz230ygl4"><img src="assets/pocketbrew_a.jpg" width="49%" alt="PocketBrew — cabin scene with themed lower third"></a> <a href="https://youtu.be/zSaz230ygl4"><img src="assets/pocketbrew_b.jpg" width="49%" alt="PocketBrew — product beat with 3-minute timer"></a> </p> ### 🛸 [ScoutFly — FOMO drone commercial](https://youtu.be/ddMKuNCeo-Y) (31s, ~$15) The QC loop on camera: the vision judge failed 4 of 6 draft clips (garbled text in the blueprint theme), staged corrective regenerations with its feedback spliced into the prompts, and every fix passed. That failure pattern then became a permanent theme rule — the self-improvement loop's origin story. <p> <a href="https://youtu.be/ddMKuNCeo-Y"><img src="assets/scoutfly_a.jpg" width="49%" alt="ScoutFly — blueprint-theme flight path"></a> <a href="https://youtu.be/ddMKuNCeo-Y"><img src="assets/scoutfly_b.jpg" width="49%" alt="ScoutFly — the drone with a caller's face on its screen"></a> </p> ### ⌚ [TIMEPIECE — 60s streaming-series trailer](https://youtu.be/irapcuaeudw) (~$27) A prestige drama trailer for a show that doesn't exist: custom cinematic theme, tick-synced cuts, a custom-prompted orchestral score, and a measured animatic that landed the 60s target exactly. <p> <a href="https://youtu.be/irapcuaeudw"><img src="assets/timepiece_a.jpg" width="49%" alt="TIMEPIECE — cliff beneath three moons"></a> <a href="https://youtu.be/irapcuaeudw"><img src="assets/timepiece_b.jpg" width="49%" alt="TIMEPIECE — the proof beat close-up"></a> </p> ### ☁️ [SKYSILVER — action-comedy trailer](https://youtu.be/neDwwKh56Pw) (31s, ~$14) The storyboard-composed route at work: two of its montages are SINGLE Seedance generations following a numbered storyboard grid (multiple hard cuts per call, scene-detection validated), roughly halving the cost of per-shot generation. The adversarial shot room made 14 revisions before a dollar was spent — including a new VO-synced cutaway. <p> <a href="https://youtu.be/neDwwKh56Pw"><img src="assets/skysilver_a.jpg" width="49%" alt="SKYSILVER — brass airship in the cloud sea"></a> <a href="https://youtu.be/neDwwKh56Pw"><img src="assets/skysilver_b.jpg" width="49%" alt="SKYSILVER — Captain Silver's wink"></a> </p> ### ✨ [Wear Your Sparkles — beauty commercial](https://youtu.be/s1I4wbIcFQk) (33s, ~$17) The hardest photoreal test (macro glitter, skin, eyes): the judge ran 7 corrective regenerations against glitter-physics hallucinations and distilled two durable lessons about macro realism that now ride every prompt in the project. <p> <a href="https://youtu.be/s1I4wbIcFQk"><img src="assets/wys_a.jpg" width="49%" alt="Wear Your Sparkles — applicator catching light on the lid"></a> <a href="https://youtu.be/s1I4wbIcFQk"><img src="assets/wys_b.jpg" width="49%" alt="Wear Your Sparkles — glitter eye macro with product"></a> </p> ## Requirements - **Python 3.12** (pydub/audioop compatibility) - **ffmpeg + ffprobe** on PATH - **Node.js 18+** (only for the final overlay render via Remotion; every other stage works without it) - API keys: - `FAL_KEY` — [fal.ai](https://fal.ai) (Seedance video + nano-banana images) - `ANTHROPIC_API_KEY` — planning/judging agents - `ELEVENLABS_API_KEY` — VO + music (optional; the pipeline degrades to silent scratch timing without it) ## Setup ```bash python -m venv .venv .venv/Scripts/activate # Windows; source .venv/bin/activate elsewhere pip install -r requirements.txt cp .env.example .env # fill in your keys uvicorn commercial_server:app --port 8700 # open http://127.0.0.1:8700 (the web UI) ``` Health check: `GET /api/v1/health` reports missing binaries/keys. > **Google Drive / synced-folder warning:** if this folder lives on a synced > drive, set `REMOTION_BUILD_DIR` to a local path (e.g. `C:/remotion-build/projects`) > — `npm install` fails on synced filesystems, and renders stage off-drive > then copy back. Non-synced installs can set `REMOTION_WORK_MODE=project`. ## The pipeline ``` intake → brief* → script → shots* → refs → storyboard → animatic* → generate† → assemble → audio → variants (*human approval gates) (†explicit spend approval) ``` Every stage is a job with live logs (SSE to the UI), and artifacts are versioned append-only — editing an upstream artifact marks everything downstream stale rather than deleting it. ## Claude Code / MCP connector `mcp_server.py` exposes the pipeline as typed MCP tools so an agent can drive it end-to-end: ```bash pip install "mcp[cli]" claude mcp add commercial-creator -- python /absolute/path/to/mcp_server.py ``` Then, in Claude Code: *"Create a 30-second commercial for <product>"* — the agent walks the stages with `run_step`, honors the approval gates via `approve_artifact`, and **cannot generate video in one step**: `stage_video_batch` returns the dollar estimate, and only the separate `approve_video_jobs` call spends. Point the connector at a non-default server with `COMMERCIAL_CREATOR_URL`. ## Field notes baked into the code (the expensive lessons) - **Seedance 2.0 has no seed input** — reproducibility lives in reference images + verbatim prompts. Consistency comes from style-recipe keyframes, reference binding (`@Image1…`) and single-generation multi-shot windows. - **The likeness gate**: the provider rejects *detectable photoreal human faces* in reference-to-video **reference images and video refs** (facial fragments — a macro eye — count). The same images pass as image-to-video start frames. Stylized/animated character sheets pass, and carry identity across composed multi-shot generations remarkably well. The planner routes around all of this automatically via a per-shot `face_prominent` flag and a per-project `visual_mode` (photoreal | stylized). - **Storyboard-composed windows**: a numbered grid of keyframes (≤3×3, black borders, number badges) as `@Image1` + timed `Shot N` blocks reliably yields one generation with native hard cuts — ~1.2s/shot floor, 6–7 shots per 10s is the readable ceiling. Cut counts are validated by scene detection after download. - **Output-side "sensitive content" flags are noisy** — one identical retry is automatic and usually passes. Input-side likeness rejections are deterministic — never retried, rerouted instead. - **ElevenLabs v3** does not support `previous_text`/`next_text`, and its stability tiers map to floats (0.5 = natural). The *voice* is the quality lever: conversational-tagged voices respond to `[excited]`-style audio tags; narration voices ignore them. - **Assembly policy**: never concat stream-copied and re-encoded segments; uniform per-segment encode trimmed to edit duration, 4-frame crossfades on take-chained joins. **📖 [Full illustrated walkthrough](docs/walkthrough.md)** — installation to finished commercial, with screenshots of every panel (also published as a Medium article). `docs/seedance-commercial-workflow-best-practices.md` is the full research playbook; `docs/seedance-commercial-app-implementation-plan.md` is the original architecture spec. ## Costs & safety rails - Per-project **budget cap**: jobs estimate before running; the queue refuses to start work that would blow the cap (402) counting in-flight estimates. - All video generation jobs are created `needs_approval` and survive server restarts without auto-running. - The QC judge stages corrective regens with feedback spliced into the prompt — approval is still explicit, and after two failed fixes it escalates to a human instead of burning more money. ## Repository layout ``` commercial_server.py FastAPI app (port 8700): REST + SSE + static UI commercial/ the pipeline package agents/ LLM agents (intake, director, breakdown+shot room, reference, judge, continuation) — Claude via tool-forced structured output adapters.py Seedance capability descriptors + fal queue client shot_plan.py timing contract: animatic timeline -> generation rows compiler.py deterministic prompt compiler (inspectable pre-spend) storyboard_sheet.py numbered grid compositor for composed windows keyframes.py/animatic.py/assembly.py/audio_vo.py/audio_final.py/music.py overlay.py/themes.py Remotion overlay render + theme palettes jobs.py/store.py approval-gated job queue + single-writer JSON store lessons.py self-improvement: QC failures -> durable prompt rules static/commercial.html the whole UI (no build step) mcp_server.py MCP connector for Claude Code remotion_template/ Remotion project (CommercialFinal composition) ``` ## License MIT — see `LICENSE`.