xonecas.io
7517
Cd

clanker-and-the-dev / files

clanker-and-the-dev / .agents / skills / add-new-comic / SKILL.md

7.5 KB
1---
2name: add-new-comic
3description: Use when the user wants to add, generate, render, regenerate, or submit a new Clanker and the Dev comic/joke. This skill is the formal entrypoint for comic generation and uses the SDXL image pipeline in this skill folder.
4allowed-tools: Read, Write, Edit, Bash
5---
6
7# Add a New Clanker and the Dev Comic
8
9This repo is a Jekyll/GitHub Pages comic-strip blog. Comic generation is driven by this agent skill and the Python SDXL pipeline in `scripts/`.
10
11The skill is the only supported generation path. Do not use the old deterministic SVG renderer or `job.sh` workflow.
12
13## Important project rules
14
15- Do not install Ruby gems locally.
16- Local Jekyll preview runs through Docker with `./dev.sh`.
17- Use `uv` for Python environment management and Python package installation.
18- Python generation code lives inside this skill folder.
19- Generated image assets live under `assets/comics/<slug>/`.
20- Generated Jekyll posts live in `_posts/`.
21- Comic specs live in `comic/specs/` and use the current image-generation schema in `comic/schema.md`.
22- Do not ask Stable Diffusion to render dialogue, captions, terminal text, lettering, title cards, headers, footers, watermarks, or UI chrome. The Python compositor adds readable comic text with Pillow, and the site adds page chrome with HTML/CSS.
23- Default SDXL model: `stabilityai/stable-diffusion-xl-base-1.0`.
24- Default IP-Adapter: `h94/IP-Adapter`, subfolder `sdxl_models`, weight `ip-adapter_sdxl.bin`.
25- Default IP-Adapter reference images: `art-reference/training-reference.png` plus `art-reference/ip-character-reference.png`.
26- The generator extracts per-panel crops from the training sheet plus character-only crops from `ip-character-reference.png`, then averages IP-Adapter image embeddings so the model sees the characters without copying sheet grids/cards.
27- ControlNet is enabled by default with reference-derived layout guides (`control-panel-XX.png`) to stabilize composition and character proportions.
28- The compositor converts soft SDXL output back into crisp terminal line art, clips generated strokes to the reference-derived guide to remove scanlines/blobs/header-footer bars, and merges that guide to repair character drift.
29- Model CPU offload is enabled by default so SDXL/IP-Adapter/ControlNet can run on a 12 GB GPU.
30
31## Standard workflow
32
33### 1. Turn the user's joke into an image-generation spec
34
35Create or update:
36
37```txt
38comic/specs/<slug>.json
39```
40
41The spec must be valid JSON only. It should follow `comic/schema.md`.
42
43Use this shape:
44
45```json
46{
47  "title": "My Great Joke",
48  "slug": "my-great-joke",
49  "timestamp": "YYYY-MM-DD",
50  "credit": "User / handle / inspiration",
51  "summary": "One sentence premise for the whole strip.",
52  "generation": {
53    "modelId": "stabilityai/stable-diffusion-xl-base-1.0",
54    "ipAdapterRepo": "h94/IP-Adapter",
55    "ipAdapterSubfolder": "sdxl_models",
56    "ipAdapterWeight": "ip-adapter_sdxl.bin",
57    "ipAdapterScale": 0.82,
58    "referenceImages": [
59      "art-reference/training-reference.png",
60      "art-reference/ip-character-reference.png"
61    ],
62    "referenceCropMode": "auto",
63    "width": 1024,
64    "height": 768,
65    "steps": 34,
66    "guidanceScale": 5.0,
67    "enableControlNet": true,
68    "controlNetModelId": "diffusers/controlnet-canny-sdxl-1.0",
69    "controlGuideMode": "layout",
70    "controlNetScale": 0.88,
71    "controlNetStart": 0.0,
72    "controlNetEnd": 0.90,
73    "artPostprocessMode": "terminal-lineart",
74    "lineArtThreshold": 112,
75    "lineArtStroke": 3
76  },
77  "panels": [
78    {
79      "artPrompt": "Dev leans toward a terminal laptop while Clanker, a boxy robot assistant, offers help in a pure black terminal room.",
80      "composition": "Dev on the left, Clanker on the right, laptop in the foreground, empty dark upper area for speech bubbles.",
81      "shot": "medium",
82      "dialogue": [
83        { "speaker": "dev", "text": "Can you make this simpler?" },
84        { "speaker": "clanker", "text": "Absolutely." }
85      ],
86      "terminalLines": ["$ git status", "legacy code: haunted"]
87    }
88  ]
89}
90```
91
92Guidelines:
93
94- Usually make 3–5 panels.
95- Preserve the user's core joke and attribution.
96- Keep dialogue short; the compositor has limited bubble space.
97- `artPrompt` describes only visuals, acting, mood, props, and setting.
98- Never put dialogue text, title/header/footer copy, watermarks, or UI chrome inside `artPrompt`; comic text is overlaid later and page chrome is HTML/CSS.
99- Prompt for the recurring characters explicitly: Dev is a human developer with simple glasses; Clanker is a boxy robot assistant with screen eyes and antenna.
100- Include composition directions such as character positions and reserved empty space for bubbles.
101- Use optional explicit `seed` values only for debugging/reruns. Default generation is non-deterministic.
102
103### 2. Generate the comic with uv
104
105From the repo root, run:
106
107```bash
108HF_HOME="$PWD/.cache/huggingface" PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True uv run --project .agents/skills/add-new-comic python .agents/skills/add-new-comic/scripts/generate_comic.py comic/specs/<slug>.json
109```
110
111This creates or updates:
112
113```txt
114assets/comics/<slug>/control-panel-01.png
115assets/comics/<slug>/raw-panel-01.png
116assets/comics/<slug>/panel-01.png
117assets/comics/<slug>/panel-02.png
118assets/comics/<slug>/strip.png
119assets/comics/<slug>/generation.json
120_posts/YYYY-MM-DD-<slug>.md
121```
122
123Notes:
124
125- `control-panel-XX.png` files are generated ControlNet layout guides for review/debugging.
126- Raw panels are SDXL output before text/borders.
127- Final `panel-XX.png` files include cleaned line art, the merged reference guide, composited comic lettering/code snippets, and a simple panel frame.
128- `strip.png` is a panels-only comic image with no baked-in title/header/footer/metadata chrome; the Jekyll page adds those with HTML/CSS.
129- Strip composition uses Pillow; no ImageMagick dependency is required for this step.
130- The first run downloads Hugging Face model weights and may require accepted model terms / Hugging Face credentials in the environment.
131- SDXL + IP-Adapter + ControlNet is VRAM-heavy; generation uses model CPU offload and one isolated worker process per panel to avoid VRAM accumulation.
132
133### 3. Review generated files
134
135Check status and diff:
136
137```bash
138git status
139git diff
140```
141
142Review:
143
144```txt
145comic/specs/<slug>.json
146assets/comics/<slug>/strip.png
147_posts/YYYY-MM-DD-<slug>.md
148```
149
150If the result is not good enough:
151
1521. Edit visual prompts, composition, dialogue length, reference strength, or generation settings in the spec.
1532. Re-run the `uv run --project ... generate_comic.py ...` command.
154
155### 4. Preview locally when useful
156
157Use Docker only:
158
159```bash
160./dev.sh
161```
162
163Then open:
164
165```txt
166http://localhost:4000
167```
168
169Do not install or run Bundler/Ruby gems locally outside Docker.
170
171### 5. Commit or prepare for PR
172
173If the user asks to commit, include the generated spec, assets, and post:
174
175```bash
176git add comic/specs assets/comics _posts
177git commit -m "Add <comic title> comic"
178```
179
180If the user is contributing from a fork, remind them to open a pull request with their generated spec, image assets, and post.
181
182## Regenerating existing comics
183
184To regenerate an existing comic, update its spec in `comic/specs/<slug>.json`, then run:
185
186```bash
187HF_HOME="$PWD/.cache/huggingface" PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True uv run --project .agents/skills/add-new-comic python .agents/skills/add-new-comic/scripts/generate_comic.py comic/specs/<slug>.json
188```
189
190Existing generated SVG posts should be replaced by image posts that embed the panels-only strip through `relative_url`:
191
192```liquid
193{{ '/assets/comics/<slug>/strip.png' | relative_url }}
194```