7517
Cd
clanker-and-the-dev / files
clanker-and-the-dev / .agents / skills / add-new-comic / SKILL.md
7.5 KB
1--- 2name: add-new-comic 3description: Use when the user wants to add, generate, render, regenerate, or submit a new Clanker and the Dev comic/joke. This skill is the formal entrypoint for comic generation and uses the SDXL image pipeline in this skill folder. 4allowed-tools: Read, Write, Edit, Bash 5--- 6 7# Add a New Clanker and the Dev Comic 8 9This repo is a Jekyll/GitHub Pages comic-strip blog. Comic generation is driven by this agent skill and the Python SDXL pipeline in `scripts/`. 10 11The skill is the only supported generation path. Do not use the old deterministic SVG renderer or `job.sh` workflow. 12 13## Important project rules 14 15- Do not install Ruby gems locally. 16- Local Jekyll preview runs through Docker with `./dev.sh`. 17- Use `uv` for Python environment management and Python package installation. 18- Python generation code lives inside this skill folder. 19- Generated image assets live under `assets/comics/<slug>/`. 20- Generated Jekyll posts live in `_posts/`. 21- Comic specs live in `comic/specs/` and use the current image-generation schema in `comic/schema.md`. 22- Do not ask Stable Diffusion to render dialogue, captions, terminal text, lettering, title cards, headers, footers, watermarks, or UI chrome. The Python compositor adds readable comic text with Pillow, and the site adds page chrome with HTML/CSS. 23- Default SDXL model: `stabilityai/stable-diffusion-xl-base-1.0`. 24- Default IP-Adapter: `h94/IP-Adapter`, subfolder `sdxl_models`, weight `ip-adapter_sdxl.bin`. 25- Default IP-Adapter reference images: `art-reference/training-reference.png` plus `art-reference/ip-character-reference.png`. 26- The generator extracts per-panel crops from the training sheet plus character-only crops from `ip-character-reference.png`, then averages IP-Adapter image embeddings so the model sees the characters without copying sheet grids/cards. 27- ControlNet is enabled by default with reference-derived layout guides (`control-panel-XX.png`) to stabilize composition and character proportions. 28- The compositor converts soft SDXL output back into crisp terminal line art, clips generated strokes to the reference-derived guide to remove scanlines/blobs/header-footer bars, and merges that guide to repair character drift. 29- Model CPU offload is enabled by default so SDXL/IP-Adapter/ControlNet can run on a 12 GB GPU. 30 31## Standard workflow 32 33### 1. Turn the user's joke into an image-generation spec 34 35Create or update: 36 37```txt 38comic/specs/<slug>.json 39``` 40 41The spec must be valid JSON only. It should follow `comic/schema.md`. 42 43Use this shape: 44 45```json 46{ 47 "title": "My Great Joke", 48 "slug": "my-great-joke", 49 "timestamp": "YYYY-MM-DD", 50 "credit": "User / handle / inspiration", 51 "summary": "One sentence premise for the whole strip.", 52 "generation": { 53 "modelId": "stabilityai/stable-diffusion-xl-base-1.0", 54 "ipAdapterRepo": "h94/IP-Adapter", 55 "ipAdapterSubfolder": "sdxl_models", 56 "ipAdapterWeight": "ip-adapter_sdxl.bin", 57 "ipAdapterScale": 0.82, 58 "referenceImages": [ 59 "art-reference/training-reference.png", 60 "art-reference/ip-character-reference.png" 61 ], 62 "referenceCropMode": "auto", 63 "width": 1024, 64 "height": 768, 65 "steps": 34, 66 "guidanceScale": 5.0, 67 "enableControlNet": true, 68 "controlNetModelId": "diffusers/controlnet-canny-sdxl-1.0", 69 "controlGuideMode": "layout", 70 "controlNetScale": 0.88, 71 "controlNetStart": 0.0, 72 "controlNetEnd": 0.90, 73 "artPostprocessMode": "terminal-lineart", 74 "lineArtThreshold": 112, 75 "lineArtStroke": 3 76 }, 77 "panels": [ 78 { 79 "artPrompt": "Dev leans toward a terminal laptop while Clanker, a boxy robot assistant, offers help in a pure black terminal room.", 80 "composition": "Dev on the left, Clanker on the right, laptop in the foreground, empty dark upper area for speech bubbles.", 81 "shot": "medium", 82 "dialogue": [ 83 { "speaker": "dev", "text": "Can you make this simpler?" }, 84 { "speaker": "clanker", "text": "Absolutely." } 85 ], 86 "terminalLines": ["$ git status", "legacy code: haunted"] 87 } 88 ] 89} 90``` 91 92Guidelines: 93 94- Usually make 3–5 panels. 95- Preserve the user's core joke and attribution. 96- Keep dialogue short; the compositor has limited bubble space. 97- `artPrompt` describes only visuals, acting, mood, props, and setting. 98- Never put dialogue text, title/header/footer copy, watermarks, or UI chrome inside `artPrompt`; comic text is overlaid later and page chrome is HTML/CSS. 99- Prompt for the recurring characters explicitly: Dev is a human developer with simple glasses; Clanker is a boxy robot assistant with screen eyes and antenna. 100- Include composition directions such as character positions and reserved empty space for bubbles. 101- Use optional explicit `seed` values only for debugging/reruns. Default generation is non-deterministic. 102 103### 2. Generate the comic with uv 104 105From the repo root, run: 106 107```bash 108HF_HOME="$PWD/.cache/huggingface" PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True uv run --project .agents/skills/add-new-comic python .agents/skills/add-new-comic/scripts/generate_comic.py comic/specs/<slug>.json 109``` 110 111This creates or updates: 112 113```txt 114assets/comics/<slug>/control-panel-01.png 115assets/comics/<slug>/raw-panel-01.png 116assets/comics/<slug>/panel-01.png 117assets/comics/<slug>/panel-02.png 118assets/comics/<slug>/strip.png 119assets/comics/<slug>/generation.json 120_posts/YYYY-MM-DD-<slug>.md 121``` 122 123Notes: 124 125- `control-panel-XX.png` files are generated ControlNet layout guides for review/debugging. 126- Raw panels are SDXL output before text/borders. 127- Final `panel-XX.png` files include cleaned line art, the merged reference guide, composited comic lettering/code snippets, and a simple panel frame. 128- `strip.png` is a panels-only comic image with no baked-in title/header/footer/metadata chrome; the Jekyll page adds those with HTML/CSS. 129- Strip composition uses Pillow; no ImageMagick dependency is required for this step. 130- The first run downloads Hugging Face model weights and may require accepted model terms / Hugging Face credentials in the environment. 131- SDXL + IP-Adapter + ControlNet is VRAM-heavy; generation uses model CPU offload and one isolated worker process per panel to avoid VRAM accumulation. 132 133### 3. Review generated files 134 135Check status and diff: 136 137```bash 138git status 139git diff 140``` 141 142Review: 143 144```txt 145comic/specs/<slug>.json 146assets/comics/<slug>/strip.png 147_posts/YYYY-MM-DD-<slug>.md 148``` 149 150If the result is not good enough: 151 1521. Edit visual prompts, composition, dialogue length, reference strength, or generation settings in the spec. 1532. Re-run the `uv run --project ... generate_comic.py ...` command. 154 155### 4. Preview locally when useful 156 157Use Docker only: 158 159```bash 160./dev.sh 161``` 162 163Then open: 164 165```txt 166http://localhost:4000 167``` 168 169Do not install or run Bundler/Ruby gems locally outside Docker. 170 171### 5. Commit or prepare for PR 172 173If the user asks to commit, include the generated spec, assets, and post: 174 175```bash 176git add comic/specs assets/comics _posts 177git commit -m "Add <comic title> comic" 178``` 179 180If the user is contributing from a fork, remind them to open a pull request with their generated spec, image assets, and post. 181 182## Regenerating existing comics 183 184To regenerate an existing comic, update its spec in `comic/specs/<slug>.json`, then run: 185 186```bash 187HF_HOME="$PWD/.cache/huggingface" PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True uv run --project .agents/skills/add-new-comic python .agents/skills/add-new-comic/scripts/generate_comic.py comic/specs/<slug>.json 188``` 189 190Existing generated SVG posts should be replaced by image posts that embed the panels-only strip through `relative_url`: 191 192```liquid 193{{ '/assets/comics/<slug>/strip.png' | relative_url }} 194```