xonecas.io
7517
Cd

clanker-and-the-dev / files

clanker-and-the-dev / comic / schema.md

7.8 KB
1# Comic Image-Generation Spec Schema
2
3The `add-new-comic` agent skill writes one JSON file matching this schema. The SDXL pipeline in `.agents/skills/add-new-comic/scripts/` reads the spec, generates individual PNG panels, composites readable comic lettering, glues panels into a final panels-only strip image, and writes the Jekyll post. Titles, credits, headers, footers, metadata, and other page chrome belong in HTML/CSS, not inside the PNG.
4
5Specs live in:
6
7```txt
8comic/specs/<slug>.json
9```
10
11Generated assets live in:
12
13```txt
14assets/comics/<slug>/
15```
16
17ControlNet guide images are also written for review:
18
19```txt
20assets/comics/<slug>/control-panel-XX.png
21```
22
23## Required shape
24
25```json
26{
27  "title": "The Refactor",
28  "slug": "the-refactor",
29  "timestamp": "2026-06-08",
30  "credit": "dev and clanker",
31  "summary": "The dev asks for a refactor and Clanker optimizes by deleting the code.",
32  "generation": {
33    "modelId": "stabilityai/stable-diffusion-xl-base-1.0",
34    "ipAdapterRepo": "h94/IP-Adapter",
35    "ipAdapterSubfolder": "sdxl_models",
36    "ipAdapterWeight": "ip-adapter_sdxl.bin",
37    "ipAdapterScale": 0.82,
38    "referenceImages": [
39      "art-reference/training-reference.png",
40      "art-reference/ip-character-reference.png"
41    ],
42    "referenceCropMode": "auto",
43    "width": 1024,
44    "height": 768,
45    "steps": 34,
46    "guidanceScale": 5.0,
47    "enableControlNet": true,
48    "controlNetModelId": "diffusers/controlnet-canny-sdxl-1.0",
49    "controlGuideMode": "layout",
50    "controlNetScale": 0.88,
51    "controlNetStart": 0.0,
52    "controlNetEnd": 0.90,
53    "artPostprocessMode": "terminal-lineart",
54    "lineArtThreshold": 112,
55    "lineArtStroke": 3
56  },
57  "panels": [
58    {
59      "artPrompt": "Dev leans over a terminal laptop while Clanker, a boxy robot assistant, confidently offers help in a pure black terminal room.",
60      "composition": "Dev on the left, Clanker on the right, laptop foreground, empty dark upper area for speech bubbles.",
61      "shot": "medium",
62      "dialogue": [
63        { "speaker": "dev", "text": "Can you refactor this?" },
64        { "speaker": "clanker", "text": "Absolutely." }
65      ],
66      "terminalLines": ["$ git status", "legacy code: haunted"]
67    }
68  ]
69}
70```
71
72## Top-level fields
73
74- `title` string, required. Human-readable post title.
75- `slug` string, required. Lowercase URL slug, `a-z`, `0-9`, and hyphens only.
76- `timestamp` string, required. Use `YYYY-MM-DD`; becomes the Jekyll post date.
77- `credit` string, optional. Attribution line.
78- `summary` string, optional but recommended. One sentence premise for prompt context.
79- `generation` object, optional. Defaults are provided by the Python model if omitted.
80- `panels` array, required. 1 to 6 panel objects; usually 3 to 5.
81
82## Generation fields
83
84All generation fields are optional; defaults are documented here.
85
86- `modelId`: default `stabilityai/stable-diffusion-xl-base-1.0`.
87- `ipAdapterRepo`: default `h94/IP-Adapter`.
88- `ipAdapterSubfolder`: default `sdxl_models`.
89- `ipAdapterWeight`: default `ip-adapter_sdxl.bin`.
90- `ipAdapterScale`: default `0.82`. Increase for stronger reference adherence; decrease if output is too constrained.
91- `referenceImages`: array of repo-relative image paths. Defaults to `art-reference/training-reference.png` plus `art-reference/ip-character-reference.png`.
92- `referenceCropMode`: default `auto`. When the default reference files are used, the generator extracts large character/pose crops from `training-reference.png` and character-only crops from `ip-character-reference.png`, then averages their IP-Adapter image embeddings so SDXL sees the characters without copying sheet grids/cards. Use `none` to feed each listed image as-is.
93- `width`: default `1024`.
94- `height`: default `768`.
95- `steps`: default `34`.
96- `guidanceScale`: default `5.0`.
97- `enableControlNet`: default `true`. Enables SDXL ControlNet conditioning for panel structure.
98- `controlNetModelId`: default `diffusers/controlnet-canny-sdxl-1.0`.
99- `controlGuideMode`: default `layout`. With no per-panel `controlImage`, the generator builds a black/white guide from art-reference character sprites plus generated props/effects, then saves it as `control-panel-XX.png`. Use `canny` to Canny-preprocess a supplied `controlImage`, or `raw` to pass the supplied guide as-is.
100- `controlNetScale`: default `0.88`. Lower values give IP-Adapter/style more freedom; higher values follow the guide more strongly.
101- `controlNetStart`: default `0.0`.
102- `controlNetEnd`: default `0.90`.
103- `artPostprocessMode`: default `terminal-lineart`. Converts soft SDXL output back into crisp green-on-black line art, clips generated strokes to the reference-derived control guide to remove scanlines/blobs/header-footer bars, and merges the guide to repair character drift. Use `monochrome` for simple grayscale colorizing or `none` to keep raw art.
104- `lineArtThreshold`: default `112`. Brightness cutoff used by `terminal-lineart` cleanup.
105- `lineArtStroke`: default `3`. Stroke restoration width used by `terminal-lineart` cleanup.
106- `negativePrompt`: optional global negative prompt override.
107- `seed`: optional global seed. Omit for non-deterministic generation.
108- `device`: default `cuda`.
109- `torchDtype`: default `float16`.
110- `enableVaeSlicing`: default `true`.
111- `enableVaeTiling`: default `true`.
112- `enableAttentionSlicing`: default `false`. Attention slicing is disabled because it can conflict with IP-Adapter attention processors.
113- `enableSequentialCpuOffload`: default `false`. More aggressive and slower offload mode; enable only if model CPU offload still exceeds VRAM.
114- `enableModelCpuOffload`: default `true`. Keeps SDXL/IP-Adapter/ControlNet usable on a 12 GB GPU by offloading inactive models through Accelerate.
115
116## Panel fields
117
118- `artPrompt` string, required. Visual description for SDXL. Do not include dialogue or text that should appear in the final image.
119- `composition` string, optional but recommended. Character positions, framing, key props, and reserved bubble space.
120- `shot` string, optional. One of `wide`, `medium`, `close`, or `over-the-shoulder`; defaults to `medium`.
121- `dialogue` array, optional. Up to 3 objects like `{ "speaker": "dev", "text": "ship it?" }`; `speaker` must be `narrator`, `dev`, or `clanker`.
122- `caption` string, optional. Narration/caption text. Use instead of or before dialogue.
123- `terminalLines` array, optional. Up to 4 short terminal commands/output lines.
124- `controlImage` string, optional. Repo-relative sketch/edge image for this panel's ControlNet guide. Omit to let the generator build a reference-derived layout guide from `composition`.
125- `controlNetScale` number, optional. Panel-specific override for global `controlNetScale`.
126- `negativePrompt` string, optional panel-specific additions to the global negative prompt.
127- `seed` integer, optional panel seed for debugging/reruns. Omit for non-deterministic generation.
128
129## Text and chrome rules
130
131Stable Diffusion should not render readable lettering or page/interface chrome.
132
133Do not put final dialogue, captions, terminal output, signs, labels, title cards, headers, footers, credits, watermarks, or UI copy in `artPrompt`. Put comic text fields in:
134
135- `dialogue`
136- `caption`
137- `terminalLines`
138
139The compositor renders comic text with the local VT323 font. The generated `strip.png` must contain only the comic panels; the Jekyll layout/post HTML supplies the page title, credit, borders, and any surrounding UI.
140
141## Prompting rules
142
143- Describe Clanker and Dev explicitly in each panel.
144- Keep prompts visual: acting, pose, mood, props, environment, lighting, composition.
145- Ask for black/neon terminal comic style, but avoid browser/app/window chrome, title bars, headers, footers, dashboards, logos, and watermarks.
146- Reserve empty dark space near the top for speech bubbles.
147- Keep dialogue short so it fits in bubbles.
148- Usually map one joke beat to one panel.
149- Output JSON only. No markdown fences, comments, trailing commas, SVG, or HTML.