7517
Cd
clanker-and-the-dev / files
clanker-and-the-dev / comic / schema.md
7.8 KB
1# Comic Image-Generation Spec Schema 2 3The `add-new-comic` agent skill writes one JSON file matching this schema. The SDXL pipeline in `.agents/skills/add-new-comic/scripts/` reads the spec, generates individual PNG panels, composites readable comic lettering, glues panels into a final panels-only strip image, and writes the Jekyll post. Titles, credits, headers, footers, metadata, and other page chrome belong in HTML/CSS, not inside the PNG. 4 5Specs live in: 6 7```txt 8comic/specs/<slug>.json 9``` 10 11Generated assets live in: 12 13```txt 14assets/comics/<slug>/ 15``` 16 17ControlNet guide images are also written for review: 18 19```txt 20assets/comics/<slug>/control-panel-XX.png 21``` 22 23## Required shape 24 25```json 26{ 27 "title": "The Refactor", 28 "slug": "the-refactor", 29 "timestamp": "2026-06-08", 30 "credit": "dev and clanker", 31 "summary": "The dev asks for a refactor and Clanker optimizes by deleting the code.", 32 "generation": { 33 "modelId": "stabilityai/stable-diffusion-xl-base-1.0", 34 "ipAdapterRepo": "h94/IP-Adapter", 35 "ipAdapterSubfolder": "sdxl_models", 36 "ipAdapterWeight": "ip-adapter_sdxl.bin", 37 "ipAdapterScale": 0.82, 38 "referenceImages": [ 39 "art-reference/training-reference.png", 40 "art-reference/ip-character-reference.png" 41 ], 42 "referenceCropMode": "auto", 43 "width": 1024, 44 "height": 768, 45 "steps": 34, 46 "guidanceScale": 5.0, 47 "enableControlNet": true, 48 "controlNetModelId": "diffusers/controlnet-canny-sdxl-1.0", 49 "controlGuideMode": "layout", 50 "controlNetScale": 0.88, 51 "controlNetStart": 0.0, 52 "controlNetEnd": 0.90, 53 "artPostprocessMode": "terminal-lineart", 54 "lineArtThreshold": 112, 55 "lineArtStroke": 3 56 }, 57 "panels": [ 58 { 59 "artPrompt": "Dev leans over a terminal laptop while Clanker, a boxy robot assistant, confidently offers help in a pure black terminal room.", 60 "composition": "Dev on the left, Clanker on the right, laptop foreground, empty dark upper area for speech bubbles.", 61 "shot": "medium", 62 "dialogue": [ 63 { "speaker": "dev", "text": "Can you refactor this?" }, 64 { "speaker": "clanker", "text": "Absolutely." } 65 ], 66 "terminalLines": ["$ git status", "legacy code: haunted"] 67 } 68 ] 69} 70``` 71 72## Top-level fields 73 74- `title` string, required. Human-readable post title. 75- `slug` string, required. Lowercase URL slug, `a-z`, `0-9`, and hyphens only. 76- `timestamp` string, required. Use `YYYY-MM-DD`; becomes the Jekyll post date. 77- `credit` string, optional. Attribution line. 78- `summary` string, optional but recommended. One sentence premise for prompt context. 79- `generation` object, optional. Defaults are provided by the Python model if omitted. 80- `panels` array, required. 1 to 6 panel objects; usually 3 to 5. 81 82## Generation fields 83 84All generation fields are optional; defaults are documented here. 85 86- `modelId`: default `stabilityai/stable-diffusion-xl-base-1.0`. 87- `ipAdapterRepo`: default `h94/IP-Adapter`. 88- `ipAdapterSubfolder`: default `sdxl_models`. 89- `ipAdapterWeight`: default `ip-adapter_sdxl.bin`. 90- `ipAdapterScale`: default `0.82`. Increase for stronger reference adherence; decrease if output is too constrained. 91- `referenceImages`: array of repo-relative image paths. Defaults to `art-reference/training-reference.png` plus `art-reference/ip-character-reference.png`. 92- `referenceCropMode`: default `auto`. When the default reference files are used, the generator extracts large character/pose crops from `training-reference.png` and character-only crops from `ip-character-reference.png`, then averages their IP-Adapter image embeddings so SDXL sees the characters without copying sheet grids/cards. Use `none` to feed each listed image as-is. 93- `width`: default `1024`. 94- `height`: default `768`. 95- `steps`: default `34`. 96- `guidanceScale`: default `5.0`. 97- `enableControlNet`: default `true`. Enables SDXL ControlNet conditioning for panel structure. 98- `controlNetModelId`: default `diffusers/controlnet-canny-sdxl-1.0`. 99- `controlGuideMode`: default `layout`. With no per-panel `controlImage`, the generator builds a black/white guide from art-reference character sprites plus generated props/effects, then saves it as `control-panel-XX.png`. Use `canny` to Canny-preprocess a supplied `controlImage`, or `raw` to pass the supplied guide as-is. 100- `controlNetScale`: default `0.88`. Lower values give IP-Adapter/style more freedom; higher values follow the guide more strongly. 101- `controlNetStart`: default `0.0`. 102- `controlNetEnd`: default `0.90`. 103- `artPostprocessMode`: default `terminal-lineart`. Converts soft SDXL output back into crisp green-on-black line art, clips generated strokes to the reference-derived control guide to remove scanlines/blobs/header-footer bars, and merges the guide to repair character drift. Use `monochrome` for simple grayscale colorizing or `none` to keep raw art. 104- `lineArtThreshold`: default `112`. Brightness cutoff used by `terminal-lineart` cleanup. 105- `lineArtStroke`: default `3`. Stroke restoration width used by `terminal-lineart` cleanup. 106- `negativePrompt`: optional global negative prompt override. 107- `seed`: optional global seed. Omit for non-deterministic generation. 108- `device`: default `cuda`. 109- `torchDtype`: default `float16`. 110- `enableVaeSlicing`: default `true`. 111- `enableVaeTiling`: default `true`. 112- `enableAttentionSlicing`: default `false`. Attention slicing is disabled because it can conflict with IP-Adapter attention processors. 113- `enableSequentialCpuOffload`: default `false`. More aggressive and slower offload mode; enable only if model CPU offload still exceeds VRAM. 114- `enableModelCpuOffload`: default `true`. Keeps SDXL/IP-Adapter/ControlNet usable on a 12 GB GPU by offloading inactive models through Accelerate. 115 116## Panel fields 117 118- `artPrompt` string, required. Visual description for SDXL. Do not include dialogue or text that should appear in the final image. 119- `composition` string, optional but recommended. Character positions, framing, key props, and reserved bubble space. 120- `shot` string, optional. One of `wide`, `medium`, `close`, or `over-the-shoulder`; defaults to `medium`. 121- `dialogue` array, optional. Up to 3 objects like `{ "speaker": "dev", "text": "ship it?" }`; `speaker` must be `narrator`, `dev`, or `clanker`. 122- `caption` string, optional. Narration/caption text. Use instead of or before dialogue. 123- `terminalLines` array, optional. Up to 4 short terminal commands/output lines. 124- `controlImage` string, optional. Repo-relative sketch/edge image for this panel's ControlNet guide. Omit to let the generator build a reference-derived layout guide from `composition`. 125- `controlNetScale` number, optional. Panel-specific override for global `controlNetScale`. 126- `negativePrompt` string, optional panel-specific additions to the global negative prompt. 127- `seed` integer, optional panel seed for debugging/reruns. Omit for non-deterministic generation. 128 129## Text and chrome rules 130 131Stable Diffusion should not render readable lettering or page/interface chrome. 132 133Do not put final dialogue, captions, terminal output, signs, labels, title cards, headers, footers, credits, watermarks, or UI copy in `artPrompt`. Put comic text fields in: 134 135- `dialogue` 136- `caption` 137- `terminalLines` 138 139The compositor renders comic text with the local VT323 font. The generated `strip.png` must contain only the comic panels; the Jekyll layout/post HTML supplies the page title, credit, borders, and any surrounding UI. 140 141## Prompting rules 142 143- Describe Clanker and Dev explicitly in each panel. 144- Keep prompts visual: acting, pose, mood, props, environment, lighting, composition. 145- Ask for black/neon terminal comic style, but avoid browser/app/window chrome, title bars, headers, footers, dashboards, logos, and watermarks. 146- Reserve empty dark space near the top for speech bubbles. 147- Keep dialogue short so it fits in bubbles. 148- Usually map one joke beat to one panel. 149- Output JSON only. No markdown fences, comments, trailing commas, SVG, or HTML.