xonecas.io
7517
Cd

clanker-and-the-dev / commit db01b14

Add SDXL LoRA experiment checkpoint

xonecas2026-06-13 db01b14c19defdcd356080169605307d4d2ea065 parent 893385e

157 files changed

This diff is too big to show in full. The first 1000 lines are below, and every file is listed with its counts.

.gitignore+3−0
@@ -9,3 +9,6 @@ __pycache__/
99 *.py[cod]
1010 .venv/
1111 .cache/
12+comic/models/
13+comic/runs/
14+comic/synthetic-training-candidates/
AGENTS.md+171−11
@@ -1,6 +1,6 @@
11 # Clanker and the Dev
22
3-Minimal project notes for agents.
3+Project notes for coding agents.
44
55 ## Site
66
@@ -9,22 +9,182 @@ Minimal project notes for agents.
99 - Do not install Ruby gems locally.
1010 - Existing posts live in `_posts/` and may embed images from `assets/comics/<slug>/`.
1111
12-## New generation direction
12+## Current generation direction
1313
14-- The old generation scripts have been removed.
15-- We are experimenting with a new Python generation workflow based on training a LoRA over SDXL.
16-- No compositor yet: generate raw image panels only, with no dialogue, captions, terminal text, or baked-in page chrome.
17-- Initial goal: iterate on LoRA training, schema, prompts, and instructions until raw panel quality is good.
14+- Python-based experimental SDXL + LoRA workflow.
15+- Generate raw image panels only for now.
16+- No compositor yet: do not bake in dialogue, captions, speech bubbles, terminal text, panel borders, strip layout, or page chrome.
17+- Goal is to iterate on training data, conditioning, prompts, and generation settings until raw panel quality is close to the simple neon stick-figure reference style.
18+- Status at pause on 2026-06-11: v3 LoRA was trained from 64 clean image/caption pairs, including a strong Gemini batch based on the original `art-reference/` sheets, but v3 generation still did not reach target quality. Avoid more v2/v3 prompt sweeps unless explicitly requested; the next pass should consider a different conditioning approach such as ControlNet/IP-Adapter, or a more substantial v4 training change rather than small prompt tweaks.
1819
1920 ## Python project
2021
2122 - Treat `comic/` as the Python project root, separate from the Jekyll website.
22-- New generation/training code should live under `comic/`.
23-- Main generation entrypoint should be `comic/generate.py`.
24-- Use Python for the new workflow.
23+- Use `uv` for Python package management. Do not use global/local ad-hoc pip installs.
24+- Python project files live in `comic/`:
25+ - `comic/pyproject.toml`
26+ - `comic/uv.lock`
27+ - `comic/train_lora.py` — SDXL LoRA training
28+ - `comic/generate.py` — raw panel generation from specs
29+ - `comic/augment_training_data.py` — Gemini image-generation helper for synthetic training candidates
30+- Typical command prefix:
2531
26-## Content conventions
32+```bash
33+cd comic
34+uv run python <script>.py ...
35+```
36+
37+## Specs and assets
2738
2839 - Comic specs live in `comic/specs/`.
40+- Current specs are intentionally lightweight: mostly `title`, `slug`, metadata, and each panel's `artPrompt` plus `dialogue`/`caption` for future composition.
41+- `generate.py` should use only `artPrompt` for image generation; dialogue/caption text is not rendered yet.
2942 - Generated comic image assets live in `assets/comics/<slug>/`.
30-- For now, generated panels should be raw art only: no rendered text, speech bubbles, captions, or UI chrome.
43+- Raw panel outputs are named `raw-panel-XX.png`.
44+- `generation.json` records generation metadata.
45+
46+## Training data
47+
48+- LoRA training data lives in `comic/training-data/` as paired files:
49+ - `name.png`
50+ - `name.txt`
51+- Captions should be concise positive LoRA captions. Avoid putting long negative instructions in training captions.
52+- Use trigger/style token `ctdstyle` in captions.
53+- Desired style: pure black background, neon green simple line art, Dev as a stick-figure developer with rectangular glasses, Clanker as a small boxy robot with screen head/antenna/claw hands.
54+- Original user drawings/reference sheets live in repo-root `art-reference/` (singular path; sometimes referred to conversationally as art-references). Use these as Gemini reference images for the next synthetic data round.
55+- Synthetic candidates may be generated into `comic/synthetic-training-candidates/`, manually reviewed, then promoted into `comic/training-data/`.
56+- `comic/training-data/synthetic-promoted-manifest.json` may record promoted synthetic sources.
57+
58+## LoRA training
59+
60+- Main script: `comic/train_lora.py`.
61+- Validate pairs without importing heavy training deps:
62+
63+```bash
64+cd comic
65+uv run --no-sync python train_lora.py --validate-only
66+```
67+
68+- Trained LoRA directories live under `comic/models/` and are gitignored.
69+- Training logs/PIDs live under `comic/runs/` and are gitignored.
70+- Use project-local Hugging Face cache if needed: `HF_HOME=$PWD/.cache/huggingface` from inside `comic/`.
71+- Previous/current model names used in this experiment:
72+ - `models/ctdstyle-sdxl-lora` — first LoRA, trained from the tiny original set, incomplete/rough.
73+ - `models/ctdstyle-sdxl-lora-v2` — expanded training set run using original + curated synthetic data.
74+ - `models/ctdstyle-sdxl-lora-v3` — trained 2026-06-11 from 64 validated pairs after adding the strong `20260610-233051` Gemini/art-reference batch; final plus checkpoints at 200/400/600/800/1000 exist locally under `comic/models/`.
75+
76+Example v2-style training command:
77+
78+```bash
79+cd comic
80+export HF_HOME="$PWD/.cache/huggingface"
81+uv run python train_lora.py \
82+ --data-dir training-data \
83+ --output-dir models/ctdstyle-sdxl-lora-v2 \
84+ --pretrained-model-name-or-path stabilityai/stable-diffusion-xl-base-1.0 \
85+ --vae-model-name-or-path madebyollin/sdxl-vae-fp16-fix \
86+ --resolution 1024 \
87+ --train-batch-size 1 \
88+ --gradient-accumulation-steps 4 \
89+ --max-train-steps 1000 \
90+ --learning-rate 5e-5 \
91+ --rank 16 \
92+ --mixed-precision fp16
93+```
94+
95+## Recent v2 generation findings
96+
97+- Review contact sheets from the final v2 sweeps are in `comic/runs/review-contact-sheets/`, especially:
98+ - `next-sweep-v2-final-all.png`
99+ - `next-sweep-v2-ckpt200.png`
100+ - `next-sweep-prompt-black-all.png`
101+ - `next-sweep-tiny-centered.png`
102+- Avoid the old default-ish high-strength settings for v2: `--lora-scale 0.85 --guidance-scale 7`. They produce dense cyberpunk/texture soup, panel grids, or overfit repeated robots.
103+- Best v2 lane found was roughly: 1024px, 30 steps, `--prompt-2-mode same`, short direct prompts, `--lora-scale 0.10` to `0.20`, `--guidance-scale 4` to `6`.
104+- Even the best v2 lane is not good enough: backgrounds drift gray/beige instead of pure black, Dev often becomes a robot or realistic/cartoon human, and two-character Dev/Clanker identity is not reliable.
105+- Conclusion: stop spending time on v2 prompt/settings sweeps unless explicitly requested. v3 has since been trained/tested; see the v3 findings below.
106+
107+## Recent v3 generation findings
108+
109+- New art-reference Gemini batch: `comic/synthetic-training-candidates/20260610-233051/contact-sheet.png`. This source batch looked much closer to target: pure black background, thin neon-green line art, sparse Dev/Clanker compositions.
110+- Promoted v3 candidates into `comic/training-data/`; validation currently finds 64 image/caption pairs. Review sheet: `comic/runs/review-contact-sheets/training-data-64-contact.png`.
111+- v3 training completed with `--max-train-steps 1000`, rank 16, learning rate `5e-5`; outputs in `comic/models/ctdstyle-sdxl-lora-v3/`. Current `train_lora.py` trains a UNet-only LoRA: both SDXL text encoders are frozen and captions are pre-embedded before training, so trigger/identity token learning may be limited.
112+- Key v3 smoke/review sheets:
113+ - `comic/runs/v3-smoke/final-scale-020/contact-sheet.png` — final v3 at low scale remained base-model dominated, realistic/dense.
114+ - `comic/runs/v3-smoke/fullprompt-scale-sweep-contact.png` — high scales overpowered/collapsed into robot-only or texture soup.
115+ - `comic/runs/v3-smoke/shortprompt-final-scale-sweep-contact.png` — still dense, gray/green backgrounds, weak identity.
116+ - `comic/runs/v3-smoke/checkpoint-sweep-low-scales-contact.png` — checkpoints 200/400/600/800 at low scales are not good enough; still robot-heavy, gray backgrounds, scene clutter.
117+ - `comic/runs/v3-smoke/simplify-hard-sweep-contact.png` — aggressive “icon/isolated/no scene” prompts made grid/diagram artifacts worse.
118+ - `comic/runs/v3-smoke/captionlike-sweep-contact.png` — prompts closely matching training captions still produced gray backgrounds, non-stick robot people, and too much scene detail.
119+- Conclusion: v3 data quality is much better than v2, but SDXL LoRA generation is still not enforcing the desired raw panel style or Dev/Clanker identity. Do not spend more time on minor v3 prompt/scale/checkpoint sweeps unless the user asks.
120+
121+## Raw panel generation
122+
123+- Main script: `comic/generate.py`.
124+- It injects shared style and Dev/Clanker character descriptions, so specs can keep short panel-specific art prompts.
125+- It uses SDXL plus a LoRA by default. Pass `--lora-path` to choose a model directory.
126+- `generate.py` now handles local LoRA checkpoint directories/files in offline-cache mode by passing `pytorch_lora_weights.safetensors` as `weight_name` when present. Use `HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1` only after the base model/VAE are cached.
127+- Useful commands:
128+
129+```bash
130+cd comic
131+uv run python generate.py --slug limb-budget --lora-path models/ctdstyle-sdxl-lora-v2 --overwrite
132+uv run python generate.py --all --lora-path models/ctdstyle-sdxl-lora-v2 --overwrite
133+uv run python generate.py --slug limb-budget --panel 1 --num-images-per-panel 4 --seed 2000
134+uv run python generate.py --slug limb-budget --dry-run --print-prompts
135+```
136+
137+- If outputs drift anime/cyberpunk/detailed, first try prompt/negative prompt/LoRA scale changes, then improve training data.
138+
139+## Synthetic training candidate generation
140+
141+- Main script: `comic/augment_training_data.py`.
142+- Uses Gemini image generation through the REST API.
143+- Read API keys from environment or `.env`; do not commit keys. `.env` is gitignored.
144+- Candidate outputs are gitignored under `comic/synthetic-training-candidates/`.
145+- Review candidates manually before promoting to `comic/training-data/`.
146+
147+Example:
148+
149+```bash
150+cd comic
151+set -a; . ../.env; set +a
152+uv run python augment_training_data.py \
153+ --reference-image training-data/dev-neutral.png \
154+ --reference-image training-data/dev-laptop-sitting.png \
155+ --reference-image training-data/clanker-neutral.png \
156+ --reference-image training-data/clanker-happy.png \
157+ --count-per-prompt 2
158+```
159+
160+Previously used v3 data command:
161+
162+```bash
163+cd comic
164+set -a; . ../.env; set +a
165+uv run python augment_training_data.py \
166+ --reference-image ../art-reference/dev-expressions-sheet.png \
167+ --reference-image ../art-reference/dev-poses-sheet.png \
168+ --reference-image ../art-reference/clanker-expressions-sheet.png \
169+ --reference-image ../art-reference/clanker-poses-actions-sheet.png \
170+ --reference-image ../art-reference/fx-props-ui-asset-sheet.png \
171+ --count-per-prompt 2
172+```
173+
174+- The 2026-06-11 v3 pass already generated and promoted a good art-reference batch. More Gemini data may still help, but do not assume more of the same will fix the SDXL LoRA behavior; first decide whether to pivot conditioning/training strategy.
175+- If generating more candidates, cull aggressively. Prefer candidates with pure black backgrounds, thin neon-green line art, sparse compositions, clear Dev-as-stick-figure-with-rectangular-glasses, clear Clanker-as-small-boxy-robot, and paired Dev+Clanker scenes.
176+- Captions should stay concise/positive and explicitly distinguish Dev from Clanker in paired scenes.
177+
178+## Future generation controls
179+
180+- The improved v3 data/training pass has now been tried and did not give significant improvements. ControlNet/ControlNET, IP-Adapter/reference conditioning, img2img from simple sketches, or an alternate base/model family are now reasonable next investigations if the user wants to keep pursuing generated raw panels.
181+
182+## Git hygiene
183+
184+- Gitignored generated/heavy/local paths include:
185+ - `comic/.venv/`
186+ - `comic/.cache/`
187+ - `comic/models/`
188+ - `comic/runs/`
189+ - `comic/synthetic-training-candidates/`
190+- Be careful not to delete or overwrite user-created art references unless explicitly asked.
art-reference/ip-character-reference.png+0−0
Binary file, not shown.
art-reference/sample-comic-strip.png+0−0
Binary file, not shown.
art-reference/style-character-reference.png+0−0
Binary file, not shown.
art-reference/training-reference.png+0−0
Binary file, not shown.
assets/comics/limb-budget/generation.json+55−24
@@ -1,29 +1,60 @@
11 {
2+ "generatedAt": "2026-06-10T22:02:00.793273+00:00",
3+ "spec": "comic/specs/limb-budget.json",
24 "title": "Limb Budget",
35 "slug": "limb-budget",
4- "timestamp": "2026-06-09",
5- "modelId": "stabilityai/stable-diffusion-xl-base-1.0",
6- "ipAdapterRepo": "h94/IP-Adapter",
7- "ipAdapterWeight": "ip-adapter_sdxl.bin",
8- "ipAdapterScale": 0.82,
9- "referenceImages": [
10- "art-reference/training-reference.png",
11- "art-reference/ip-character-reference.png"
12- ],
13- "referenceCropMode": "auto",
14- "enableControlNet": true,
15- "controlNetModelId": "diffusers/controlnet-canny-sdxl-1.0",
16- "controlGuideMode": "layout",
17- "controlNetScale": 0.88,
18- "controlNetStart": 0.0,
19- "controlNetEnd": 0.9,
20- "artPostprocessMode": "terminal-lineart",
21- "lineArtThreshold": 112,
22- "lineArtStroke": 3,
23- "panelSeeds": [
24- 1633277776,
25- 2817340413,
26- 2583630545,
27- 1808476976
6+ "rawPanelsOnly": true,
7+ "records": [
8+ {
9+ "panel": 1,
10+ "seed": 1234,
11+ "output": "assets/comics/limb-budget/raw-panel-01.png",
12+ "artPrompt": "The Dev is sitting behind the laptop, with an inquisitive look, while Clanker is busy to the side with his own laptop.",
13+ "prompt": "ctdstyle, neon green outline doodle, pure black background, simple sparse comic, The Dev is sitting behind the laptop, with an inquisitive look, while Clanker is busy to the side with his own laptop., raw art only, no text, no speech bubbles",
14+ "prompt2": "ctdstyle, neon green outline doodle, pure black background, simple sparse comic, Dev: stick figure developer, round head, rectangular glasses, thin limbs. Clanker: small boxy robot, screen head, antenna, claw hands, raw art only, no text, no speech bubbles",
15+ "stylePrompt": "ctdstyle, neon green outline doodle, pure black background, simple sparse comic",
16+ "characterPrompt": "Dev: stick figure developer, round head, rectangular glasses, thin limbs. Clanker: small boxy robot, screen head, antenna, claw hands",
17+ "negativePrompt": "anime, manga, cyberpunk, armored suit, helmet, realistic human, hoodie, detailed clothing, text, words, letters, speech bubble, caption, watermark, logo, panel border, white background, gray background, complex background, photorealistic, 3d render, clutter, extra limbs, malformed limbs, blurry, low quality",
18+ "width": 1024,
19+ "height": 1024,
20+ "numInferenceSteps": 35,
21+ "guidanceScale": 7.0,
22+ "loraPath": "models/ctdstyle-sdxl-lora-v2",
23+ "loraScale": 0.85
24+ },
25+ {
26+ "panel": 2,
27+ "seed": 1335,
28+ "output": "assets/comics/limb-budget/raw-panel-02.png",
29+ "artPrompt": "The Dev is sitting behind the laptop, with an inquisitive look, while Clanker is busy to the side with his own laptop.",
30+ "prompt": "ctdstyle, neon green outline doodle, pure black background, simple sparse comic, The Dev is sitting behind the laptop, with an inquisitive look, while Clanker is busy to the side with his own laptop., raw art only, no text, no speech bubbles",
31+ "prompt2": "ctdstyle, neon green outline doodle, pure black background, simple sparse comic, Dev: stick figure developer, round head, rectangular glasses, thin limbs. Clanker: small boxy robot, screen head, antenna, claw hands, raw art only, no text, no speech bubbles",
32+ "stylePrompt": "ctdstyle, neon green outline doodle, pure black background, simple sparse comic",
33+ "characterPrompt": "Dev: stick figure developer, round head, rectangular glasses, thin limbs. Clanker: small boxy robot, screen head, antenna, claw hands",
34+ "negativePrompt": "anime, manga, cyberpunk, armored suit, helmet, realistic human, hoodie, detailed clothing, text, words, letters, speech bubble, caption, watermark, logo, panel border, white background, gray background, complex background, photorealistic, 3d render, clutter, extra limbs, malformed limbs, blurry, low quality",
35+ "width": 1024,
36+ "height": 1024,
37+ "numInferenceSteps": 35,
38+ "guidanceScale": 7.0,
39+ "loraPath": "models/ctdstyle-sdxl-lora-v2",
40+ "loraScale": 0.85
41+ },
42+ {
43+ "panel": 3,
44+ "seed": 1436,
45+ "output": "assets/comics/limb-budget/raw-panel-03.png",
46+ "artPrompt": "The Dev sits defeated with his head on the keyboard, while Clanker excitedly works to his side with his own laptop.",
47+ "prompt": "ctdstyle, neon green outline doodle, pure black background, simple sparse comic, The Dev sits defeated with his head on the keyboard, while Clanker excitedly works to his side with his own laptop., raw art only, no text, no speech bubbles",
48+ "prompt2": "ctdstyle, neon green outline doodle, pure black background, simple sparse comic, Dev: stick figure developer, round head, rectangular glasses, thin limbs. Clanker: small boxy robot, screen head, antenna, claw hands, raw art only, no text, no speech bubbles",
49+ "stylePrompt": "ctdstyle, neon green outline doodle, pure black background, simple sparse comic",
50+ "characterPrompt": "Dev: stick figure developer, round head, rectangular glasses, thin limbs. Clanker: small boxy robot, screen head, antenna, claw hands",
51+ "negativePrompt": "anime, manga, cyberpunk, armored suit, helmet, realistic human, hoodie, detailed clothing, text, words, letters, speech bubble, caption, watermark, logo, panel border, white background, gray background, complex background, photorealistic, 3d render, clutter, extra limbs, malformed limbs, blurry, low quality",
52+ "width": 1024,
53+ "height": 1024,
54+ "numInferenceSteps": 35,
55+ "guidanceScale": 7.0,
56+ "loraPath": "models/ctdstyle-sdxl-lora-v2",
57+ "loraScale": 0.85
58+ }
2859 ]
2960 }
assets/comics/limb-budget/raw-panel-01.png+0−0
Binary file, not shown.
assets/comics/limb-budget/raw-panel-02.png+0−0
Binary file, not shown.
assets/comics/limb-budget/raw-panel-03.png+0−0
Binary file, not shown.
assets/comics/limb-budget/raw-panel-04.png+0−0
Binary file, not shown.
assets/comics/the-cycle-of-ai-coding-burnout/generation.json+74−25
@@ -1,30 +1,79 @@
11 {
2+ "generatedAt": "2026-06-10T19:51:26.555687+00:00",
3+ "spec": "comic/specs/the-cycle-of-ai-coding-burnout.json",
24 "title": "The Cycle of AI Coding Burnout",
35 "slug": "the-cycle-of-ai-coding-burnout",
4- "timestamp": "2026-06-08",
5- "modelId": "stabilityai/stable-diffusion-xl-base-1.0",
6- "ipAdapterRepo": "h94/IP-Adapter",
7- "ipAdapterWeight": "ip-adapter_sdxl.bin",
8- "ipAdapterScale": 0.82,
9- "referenceImages": [
10- "art-reference/training-reference.png",
11- "art-reference/ip-character-reference.png"
12- ],
13- "referenceCropMode": "auto",
14- "enableControlNet": true,
15- "controlNetModelId": "diffusers/controlnet-canny-sdxl-1.0",
16- "controlGuideMode": "layout",
17- "controlNetScale": 0.88,
18- "controlNetStart": 0.0,
19- "controlNetEnd": 0.9,
20- "artPostprocessMode": "terminal-lineart",
21- "lineArtThreshold": 112,
22- "lineArtStroke": 3,
23- "panelSeeds": [
24- 3302277821,
25- 2092313095,
26- 411100092,
27- 3678133775,
28- 1594613776
6+ "rawPanelsOnly": true,
7+ "records": [
8+ {
9+ "panel": 1,
10+ "seed": 1234,
11+ "output": "assets/comics/the-cycle-of-ai-coding-burnout/raw-panel-01.png",
12+ "artPrompt": "The Dev is standing next to Clanker who is focused on working on his laptop.",
13+ "prompt": "ctdstyle, neon green line art, pure black background, minimalist comic, The Dev is standing next to Clanker who is focused on working on his laptop., raw art only, no text, no speech bubbles, no captions",
14+ "negativePrompt": "text, words, letters, speech bubble, caption, watermark, logo, panel border, photorealistic, 3d render, clutter, extra limbs, malformed limbs, blurry, low quality",
15+ "width": 1024,
16+ "height": 1024,
17+ "numInferenceSteps": 35,
18+ "guidanceScale": 7.0,
19+ "loraPath": "comic/models/ctdstyle-sdxl-lora",
20+ "loraScale": 0.85
21+ },
22+ {
23+ "panel": 2,
24+ "seed": 1335,
25+ "output": "assets/comics/the-cycle-of-ai-coding-burnout/raw-panel-02.png",
26+ "artPrompt": "Dev is standing next to Clanker, amazed and excited with the work Clanker is doing on his laptop. Clanker is sitting in front of his laptop",
27+ "prompt": "ctdstyle, neon green line art, pure black background, minimalist comic, Dev is standing next to Clanker, amazed and excited with the work Clanker is doing on his laptop. Clanker is sitting in front of his laptop, raw art only, no text, no speech bubbles, no captions",
28+ "negativePrompt": "text, words, letters, speech bubble, caption, watermark, logo, panel border, photorealistic, 3d render, clutter, extra limbs, malformed limbs, blurry, low quality",
29+ "width": 1024,
30+ "height": 1024,
31+ "numInferenceSteps": 35,
32+ "guidanceScale": 7.0,
33+ "loraPath": "comic/models/ctdstyle-sdxl-lora",
34+ "loraScale": 0.85
35+ },
36+ {
37+ "panel": 3,
38+ "seed": 1436,
39+ "output": "assets/comics/the-cycle-of-ai-coding-burnout/raw-panel-03.png",
40+ "artPrompt": "The Dev is standing next to clanker, now mind-blown by the work Clanker is doing in his laptop. Clanker is sitting with his laptop, focused on his work.",
41+ "prompt": "ctdstyle, neon green line art, pure black background, minimalist comic, The Dev is standing next to clanker, now mind-blown by the work Clanker is doing in his laptop. Clanker is sitting with his laptop, focused on his work., raw art only, no text, no speech bubbles, no captions",
42+ "negativePrompt": "text, words, letters, speech bubble, caption, watermark, logo, panel border, photorealistic, 3d render, clutter, extra limbs, malformed limbs, blurry, low quality",
43+ "width": 1024,
44+ "height": 1024,
45+ "numInferenceSteps": 35,
46+ "guidanceScale": 7.0,
47+ "loraPath": "comic/models/ctdstyle-sdxl-lora",
48+ "loraScale": 0.85
49+ },
50+ {
51+ "panel": 4,
52+ "seed": 1537,
53+ "output": "assets/comics/the-cycle-of-ai-coding-burnout/raw-panel-04.png",
54+ "artPrompt": "The Dev is laying on the ground seemingly dead, while clanker is now standing next to the looking confused.",
55+ "prompt": "ctdstyle, neon green line art, pure black background, minimalist comic, The Dev is laying on the ground seemingly dead, while clanker is now standing next to the looking confused., raw art only, no text, no speech bubbles, no captions",
56+ "negativePrompt": "text, words, letters, speech bubble, caption, watermark, logo, panel border, photorealistic, 3d render, clutter, extra limbs, malformed limbs, blurry, low quality",
57+ "width": 1024,
58+ "height": 1024,
59+ "numInferenceSteps": 35,
60+ "guidanceScale": 7.0,
61+ "loraPath": "comic/models/ctdstyle-sdxl-lora",
62+ "loraScale": 0.85
63+ },
64+ {
65+ "panel": 5,
66+ "seed": 1638,
67+ "output": "assets/comics/the-cycle-of-ai-coding-burnout/raw-panel-05.png",
68+ "artPrompt": "The dev is sitting with his laptop, looking tired and defeated.",
69+ "prompt": "ctdstyle, neon green line art, pure black background, minimalist comic, The dev is sitting with his laptop, looking tired and defeated., raw art only, no text, no speech bubbles, no captions",
70+ "negativePrompt": "text, words, letters, speech bubble, caption, watermark, logo, panel border, photorealistic, 3d render, clutter, extra limbs, malformed limbs, blurry, low quality",
71+ "width": 1024,
72+ "height": 1024,
73+ "numInferenceSteps": 35,
74+ "guidanceScale": 7.0,
75+ "loraPath": "comic/models/ctdstyle-sdxl-lora",
76+ "loraScale": 0.85
77+ }
2978 ]
3079 }
assets/comics/the-cycle-of-ai-coding-burnout/raw-panel-01.png+0−0
Binary file, not shown.
assets/comics/the-cycle-of-ai-coding-burnout/raw-panel-02.png+0−0
Binary file, not shown.
assets/comics/the-cycle-of-ai-coding-burnout/raw-panel-03.png+0−0
Binary file, not shown.
assets/comics/the-cycle-of-ai-coding-burnout/raw-panel-04.png+0−0
Binary file, not shown.
assets/comics/the-cycle-of-ai-coding-burnout/raw-panel-05.png+0−0
Binary file, not shown.
comic/.python-version+1−0
@@ -0,0 +1 @@
1+3.12
comic/README.md+110−0
@@ -0,0 +1,110 @@
1+# Comic generation experiments
2+
3+This is the Python project root for the experimental SDXL + LoRA raw-panel workflow for Clanker and the Dev.
4+
5+## Current status
6+
7+- Training data: `training-data/` contains 64 validated image/caption pairs using the `ctdstyle` token.
8+- Models: trained LoRA outputs live locally under `models/` and are intentionally gitignored.
9+- Latest run: `models/ctdstyle-sdxl-lora-v3` trained successfully with checkpoints at 200/400/600/800/1000 steps.
10+- Result: v3 data is much better than the original set, but SDXL LoRA generation still misses the target style and character identity often enough that more small prompt/scale sweeps are not the recommended next step.
11+- Next likely direction: ControlNet, IP-Adapter/reference conditioning, img2img from simple sketches, an alternate model family, or a more substantial v4 training change.
12+
13+This workflow generates raw image panels only. It does not compose dialogue, captions, speech bubbles, panel borders, strips, or page chrome.
14+
15+## Setup
16+
17+Use `uv` from this directory:
18+
19+```bash
20+cd comic
21+uv sync
22+```
23+
24+If the SDXL base model is not already available locally, log in to Hugging Face and accept the model license:
25+
26+```bash
27+uv run huggingface-cli login
28+```
29+
30+## Validate training data
31+
32+The training data is paired as `training-data/name.png` + `training-data/name.txt`.
33+
34+```bash
35+uv run --no-sync python train_lora.py --validate-only
36+```
37+
38+## Train a LoRA
39+
40+Example v3-style command:
41+
42+```bash
43+export HF_HOME="$PWD/.cache/huggingface"
44+uv run python train_lora.py \
45+ --data-dir training-data \
46+ --output-dir models/ctdstyle-sdxl-lora-v3 \
47+ --pretrained-model-name-or-path stabilityai/stable-diffusion-xl-base-1.0 \
48+ --vae-model-name-or-path madebyollin/sdxl-vae-fp16-fix \
49+ --resolution 1024 \
50+ --train-batch-size 1 \
51+ --gradient-accumulation-steps 4 \
52+ --max-train-steps 1000 \
53+ --learning-rate 5e-5 \
54+ --rank 16 \
55+ --mixed-precision fp16
56+```
57+
58+Final LoRA weights are written to:
59+
60+```txt
61+models/<run-name>/pytorch_lora_weights.safetensors
62+```
63+
64+Intermediate checkpoints are written below the same output directory as `checkpoint-<step>/`.
65+
66+## Generate raw panels
67+
68+Generate one spec panel-by-panel into `../assets/comics/<slug>/raw-panel-XX.png`:
69+
70+```bash
71+uv run python generate.py --slug limb-budget --lora-path models/ctdstyle-sdxl-lora-v3 --overwrite
72+```
73+
74+Generate every spec:
75+
76+```bash
77+uv run python generate.py --all --lora-path models/ctdstyle-sdxl-lora-v3 --overwrite
78+```
79+
80+Useful iteration flags:
81+
82+```bash
83+uv run python generate.py --slug limb-budget --panel 1 --num-images-per-panel 4 --seed 2000
84+uv run python generate.py --slug limb-budget --dry-run --print-prompts
85+uv run python generate.py --slug limb-budget --panel 1 --lora-scale 0.2 --guidance-scale 4 --prompt-2-mode same
86+```
87+
88+`generate.py` injects shared style and main-character descriptions automatically, so spec `artPrompt` fields can stay focused on panel-specific action/composition. Override that text with:
89+
90+```bash
91+uv run python generate.py --slug limb-budget --character-prompt-file prompts/characters.txt
92+uv run python generate.py --slug limb-budget --no-character-prompt
93+```
94+
95+## Generate synthetic training candidates
96+
97+Use Gemini image generation to create candidate image/caption pairs for manual review:
98+
99+```bash
100+set -a; . ../.env; set +a
101+uv run python augment_training_data.py \
102+ --reference-image ../art-reference/dev-expressions-sheet.png \
103+ --reference-image ../art-reference/dev-poses-sheet.png \
104+ --reference-image ../art-reference/clanker-expressions-sheet.png \
105+ --reference-image ../art-reference/clanker-poses-actions-sheet.png \
106+ --reference-image ../art-reference/fx-props-ui-asset-sheet.png \
107+ --count-per-prompt 2
108+```
109+
110+Candidates are written to ignored directories under `synthetic-training-candidates/`. Review them manually and copy only good `.png`/`.txt` pairs into `training-data/` before retraining.
comic/augment_training_data.py+224−0
@@ -0,0 +1,224 @@
1+#!/usr/bin/env python3
2+"""Generate candidate synthetic LoRA training images with Gemini image generation.
3+
4+Outputs are candidates only. Review them manually, then copy good image/text pairs into
5+`training-data/` before retraining.
6+"""
7+
8+from __future__ import annotations
9+
10+import argparse
11+import base64
12+import json
13+import mimetypes
14+import os
15+import re
16+import time
17+import urllib.error
18+import urllib.request
19+from dataclasses import dataclass
20+from datetime import datetime, timezone
21+from io import BytesIO
22+from pathlib import Path
23+from typing import Any
24+
25+from PIL import Image
26+
27+DEFAULT_MODEL = "gemini-3.1-flash-image"
28+DEFAULT_API_VERSION = "v1beta"
29+DEFAULT_STYLE_INSTRUCTION = """
30+Create one clean training image for a LoRA dataset in the exact Clanker and the Dev style.
31+Style requirements: simple neon green monoline doodle on a pure black background, thin glowing outlines, sparse centered composition, no shading, no fill, no white or gray background.
32+Character requirements: Dev is a minimalist stick figure software developer with rectangular glasses, round head, and thin limbs. Clanker is a small cute boxy robot assistant with a rectangular screen head, antenna, segmented arms, claw hands, and simple screen eyes.
33+Image requirements: square 1024x1024, raw art only, no speech bubbles, no captions, no readable text, no watermark, no border.
34+Avoid: anime, manga, cyberpunk armor, realistic humans, detailed clothing, complex backgrounds, filled green bodies.
35+""".strip()
36+
37+DEFAULT_PROMPTS = [
38+ "Dev standing next to Clanker, both small and centered, neutral pose.",
39+ "Dev sitting at a simple laptop while Clanker stands beside the desk looking helpful.",
40+ "Clanker sitting at a simple laptop while Dev stands beside him looking amazed.",
41+ "Dev and Clanker both looking at one laptop on a simple desk, sparse composition.",
42+ "Dev exhausted with head on keyboard while Clanker stands nearby looking confused.",
43+ "Dev thinking with one hand on chin while Clanker waits beside him.",
44+ "Dev celebrating with arms raised while Clanker raises both claw hands happily.",
45+ "Clanker confused error pose with Dev standing nearby pointing at a laptop.",
46+ "Dev walking while carrying an open laptop, Clanker walking beside him.",
47+ "Simple empty desk with laptop, Dev seated on one side, Clanker seated on the other side.",
48+]
49+
50+
51+@dataclass(frozen=True)
52+class Candidate:
53+ prompt_index: int
54+ variant_index: int
55+ prompt: str
56+ full_prompt: str
57+ image_path: Path
58+ caption_path: Path
59+ metadata_path: Path
60+
61+
62+def slugify(value: str, max_length: int = 48) -> str:
63+ value = value.lower()
64+ value = re.sub(r"[^a-z0-9]+", "-", value).strip("-")
65+ return (value[:max_length].strip("-") or "candidate")
66+
67+
68+def parse_args() -> argparse.Namespace:
69+ comic_root = Path(__file__).resolve().parent
70+ default_output = comic_root / "synthetic-training-candidates" / datetime.now(timezone.utc).strftime("%Y%m%d-%H%M%S")
71+
72+ parser = argparse.ArgumentParser(description="Generate synthetic candidate training images with Gemini.")
73+ parser.add_argument("--output-dir", type=Path, default=default_output)
74+ parser.add_argument("--model", default=DEFAULT_MODEL)
75+ parser.add_argument("--api-version", default=DEFAULT_API_VERSION)
76+ parser.add_argument("--api-key-env", default=None, help="Env var containing the API key. Defaults to GEMINI_API_KEY then GOOGLE_API_KEY.")
77+ parser.add_argument("--api-key", default=None, help="API key. Prefer env vars; this is only for local one-off use.")
78+
79+ parser.add_argument("--style-instruction", default=DEFAULT_STYLE_INSTRUCTION)
80+ parser.add_argument("--prompt", action="append", help="Panel/candidate prompt. Repeat for multiple prompts.")
81+ parser.add_argument("--prompts-file", type=Path, help="One prompt per line, blank lines and # comments ignored.")
82+ parser.add_argument("--count-per-prompt", type=int, default=1)
83+ parser.add_argument("--start-index", type=int, default=1)
84+ parser.add_argument("--sleep", type=float, default=1.0, help="Seconds to sleep between API calls.")
85+
86+ parser.add_argument("--reference-image", type=Path, action="append", default=[], help="Optional style reference image to send with every request. Repeatable.")
87+ parser.add_argument("--use-training-references", action="store_true", help="Use current comic/training-data/*.png as reference images.")
88+ parser.add_argument("--max-training-references", type=int, default=6)
89+
90+ parser.add_argument("--dry-run", action="store_true")
91+ return parser.parse_args()
92+
93+
94+def resolve_api_key(args: argparse.Namespace) -> str:
95+ if args.api_key:
96+ return args.api_key
97+ env_names = [args.api_key_env] if args.api_key_env else ["GEMINI_API_KEY", "GOOGLE_API_KEY"]
98+ for env_name in env_names:
99+ if env_name and os.environ.get(env_name):
100+ return os.environ[env_name]
101+ raise SystemExit("Missing API key. Set GEMINI_API_KEY or GOOGLE_API_KEY, or pass --api-key.")
102+
103+
104+def load_prompts(args: argparse.Namespace) -> list[str]:
105+ prompts: list[str] = []
106+ if args.prompts_file:
107+ for line in args.prompts_file.read_text(encoding="utf-8").splitlines():
108+ line = line.strip()
109+ if line and not line.startswith("#"):
110+ prompts.append(line)
111+ if args.prompt:
112+ prompts.extend(args.prompt)
113+ if not prompts:
114+ prompts = DEFAULT_PROMPTS
115+ return prompts
116+
117+
118+def reference_images(args: argparse.Namespace) -> list[Path]:
119+ images = list(args.reference_image)
120+ if args.use_training_references:
121+ training_dir = Path(__file__).resolve().parent / "training-data"
122+ images.extend(sorted(training_dir.glob("*.png"))[: args.max_training_references])
123+ return images
124+
125+
126+def image_part(path: Path) -> dict[str, Any]:
127+ mime_type = mimetypes.guess_type(path.name)[0] or "image/png"
128+ data = base64.b64encode(path.read_bytes()).decode("ascii")
129+ return {"inlineData": {"mimeType": mime_type, "data": data}}
130+
131+
132+def request_image(api_key: str, model: str, api_version: str, prompt: str, references: list[Path]) -> tuple[bytes, str | None, dict[str, Any]]:
133+ endpoint = f"https://generativelanguage.googleapis.com/{api_version}/models/{model}:generateContent"
134+ parts: list[dict[str, Any]] = [{"text": prompt}]
135+ parts.extend(image_part(path) for path in references)
136+ payload = {"contents": [{"parts": parts}]}
137+
138+ request = urllib.request.Request(
139+ endpoint,
140+ data=json.dumps(payload).encode("utf-8"),
141+ headers={"Content-Type": "application/json", "x-goog-api-key": api_key},
142+ method="POST",
143+ )
144+
145+ try:
146+ with urllib.request.urlopen(request, timeout=180) as response:
147+ body = json.loads(response.read().decode("utf-8"))
148+ except urllib.error.HTTPError as exc:
149+ detail = exc.read().decode("utf-8", "replace")
150+ raise RuntimeError(f"Gemini API error {exc.code}: {detail}") from exc
151+
152+ text_parts: list[str] = []
153+ for candidate in body.get("candidates", []):
154+ for part in candidate.get("content", {}).get("parts", []):
155+ if "text" in part:
156+ text_parts.append(part["text"])
157+ inline = part.get("inlineData") or part.get("inline_data")
158+ if inline and inline.get("data"):
159+ return base64.b64decode(inline["data"]), "\n".join(text_parts).strip() or None, body
160+
161+ raise RuntimeError(f"No image returned by Gemini API. Response: {json.dumps(body)[:2000]}")
162+
163+
164+def save_candidate(candidate: Candidate, image_bytes: bytes, response_text: str | None, raw_response: dict[str, Any], model: str, references: list[Path]) -> None:
165+ candidate.image_path.parent.mkdir(parents=True, exist_ok=True)
166+
167+ # Normalize whatever image type the API returns into PNG.
168+ image = Image.open(BytesIO(image_bytes)).convert("RGBA")
169+ image.save(candidate.image_path)
170+ candidate.caption_path.write_text(candidate.full_prompt + "\n", encoding="utf-8")
171+
172+ metadata = {
173+ "generatedAt": datetime.now(timezone.utc).isoformat(),
174+ "model": model,
175+ "prompt": candidate.prompt,
176+ "fullPrompt": candidate.full_prompt,
177+ "responseText": response_text,
178+ "references": [str(path) for path in references],
179+ "rawResponseKeys": sorted(raw_response.keys()),
180+ }
181+ candidate.metadata_path.write_text(json.dumps(metadata, indent=2) + "\n", encoding="utf-8")
182+
183+
184+def main() -> None:
185+ args = parse_args()
186+ prompts = load_prompts(args)
187+ refs = reference_images(args)
188+
189+ print(f"Output: {args.output_dir}")
190+ print(f"Model: {args.model}")
191+ print(f"Prompts: {len(prompts)} x {args.count_per_prompt}")
192+ if refs:
193+ print("References:")
194+ for ref in refs:
195+ print(f"- {ref}")
196+
197+ api_key = "" if args.dry_run else resolve_api_key(args)
198+
199+ for prompt_offset, prompt in enumerate(prompts, start=args.start_index):
200+ full_prompt = f"{args.style_instruction}\n\nCandidate subject: {prompt}"
201+ name = f"synthetic-{prompt_offset:03d}-{slugify(prompt)}"
202+ for variant in range(1, args.count_per_prompt + 1):
203+ suffix = f"-{variant:02d}" if args.count_per_prompt > 1 else ""
204+ candidate = Candidate(
205+ prompt_index=prompt_offset,
206+ variant_index=variant,
207+ prompt=prompt,
208+ full_prompt=full_prompt,
209+ image_path=args.output_dir / f"{name}{suffix}.png",
210+ caption_path=args.output_dir / f"{name}{suffix}.txt",
211+ metadata_path=args.output_dir / f"{name}{suffix}.json",
212+ )
213+ print(f"generate {candidate.image_path.name}")
214+ if args.dry_run:
215+ print(full_prompt)
216+ continue
217+ image_bytes, response_text, raw_response = request_image(api_key, args.model, args.api_version, full_prompt, refs)
218+ save_candidate(candidate, image_bytes, response_text, raw_response, args.model, refs)
219+ if args.sleep:
220+ time.sleep(args.sleep)
221+
222+
223+if __name__ == "__main__":
224+ main()
comic/generate.py+351−0
@@ -0,0 +1,351 @@
1+#!/usr/bin/env python3
2+"""Generate raw SDXL LoRA panels from comic specs.
3+
4+This script intentionally generates only raw image panels. It does not compose speech
5+bubbles, captions, terminal text, borders, or final strips.
6+"""
7+
8+from __future__ import annotations
9+
10+import argparse
11+import json
12+import os
13+import random
14+from dataclasses import dataclass
15+from datetime import datetime, timezone
16+from pathlib import Path
17+from typing import Any
18+
19+DEFAULT_MODEL = "stabilityai/stable-diffusion-xl-base-1.0"
20+DEFAULT_VAE = "madebyollin/sdxl-vae-fp16-fix"
21+DEFAULT_STYLE_PROMPT = "ctdstyle, neon green outline doodle, pure black background, simple sparse comic"
22+DEFAULT_CHARACTER_PROMPT = (
23+ "Dev: stick figure developer, round head, rectangular glasses, thin limbs. "
24+ "Clanker: small boxy robot, screen head, antenna, claw hands"
25+)
26+DEFAULT_PANEL_SUFFIX = "raw art only, no text, no speech bubbles"
27+DEFAULT_NEGATIVE_PROMPT = (
28+ "anime, manga, cyberpunk, armored suit, helmet, realistic human, hoodie, detailed clothing, "
29+ "text, words, letters, speech bubble, caption, watermark, logo, panel border, white background, "
30+ "gray background, complex background, photorealistic, 3d render, clutter, extra limbs, "
31+ "malformed limbs, blurry, low quality"
32+)
33+
34+
35+@dataclass(frozen=True)
36+class PanelJob:
37+ index: int
38+ art_prompt: str
39+ output_path: Path
40+ seed: int
41+
42+
43+def project_paths() -> tuple[Path, Path]:
44+ comic_root = Path(__file__).resolve().parent
45+ site_root = comic_root.parent
46+ return comic_root, site_root
47+
48+
49+def display_path(path: Path) -> str:
50+ """Return repo-relative metadata paths when possible."""
51+ _, site_root = project_paths()
52+ try:
53+ return path.resolve().relative_to(site_root).as_posix()
54+ except ValueError:
55+ return str(path)
56+
57+
58+def parse_args() -> argparse.Namespace:
59+ comic_root, site_root = project_paths()
60+
61+ parser = argparse.ArgumentParser(description="Generate raw image panels from comic specs.")
62+ parser.add_argument("spec", nargs="?", type=Path, help="Spec JSON path. If omitted, use --slug or generate all specs.")
63+ parser.add_argument("--slug", help="Generate comic/specs/<slug>.json")
64+ parser.add_argument("--all", action="store_true", help="Generate every JSON spec in comic/specs")
65+
66+ parser.add_argument("--specs-dir", type=Path, default=comic_root / "specs")
67+ parser.add_argument("--assets-dir", type=Path, default=site_root / "assets" / "comics")
68+ parser.add_argument("--output-dir", type=Path, default=None, help="Override output directory for a single spec")
69+
70+ parser.add_argument("--pretrained-model-name-or-path", default=DEFAULT_MODEL)
71+ parser.add_argument("--vae-model-name-or-path", default=DEFAULT_VAE)
72+ parser.add_argument("--lora-path", type=Path, default=comic_root / "models" / "ctdstyle-sdxl-lora")
73+ parser.add_argument("--no-lora", action="store_true")
74+ parser.add_argument("--lora-scale", type=float, default=0.85)
75+ parser.add_argument("--hf-home", type=Path, default=comic_root / ".cache" / "huggingface")
76+
77+ parser.add_argument("--style-prompt", default=DEFAULT_STYLE_PROMPT)
78+ parser.add_argument("--character-prompt", default=DEFAULT_CHARACTER_PROMPT)
79+ parser.add_argument("--character-prompt-file", type=Path, help="Optional text file overriding --character-prompt")
80+ parser.add_argument("--no-character-prompt", action="store_true", help="Do not inject shared Dev/Clanker descriptions")
81+ parser.add_argument("--panel-suffix", default=DEFAULT_PANEL_SUFFIX)
82+ parser.add_argument(
83+ "--prompt-2-mode",
84+ choices=["split", "same", "none"],
85+ default="split",
86+ help="How to populate SDXL prompt_2: split keeps reusable character context there, same mirrors prompt, none leaves it empty.",
87+ )
88+ parser.add_argument("--negative-prompt", default=DEFAULT_NEGATIVE_PROMPT)
89+ parser.add_argument("--guidance-scale", type=float, default=7.0)
90+ parser.add_argument("--num-inference-steps", type=int, default=35)
91+ parser.add_argument("--width", type=int, default=1024)
92+ parser.add_argument("--height", type=int, default=1024)
93+ parser.add_argument("--seed", type=int, default=1234)
94+ parser.add_argument("--seed-stride", type=int, default=101)
95+
96+ parser.add_argument("--panel", type=int, action="append", help="1-based panel number to generate. Repeat for multiple panels.")
97+ parser.add_argument("--num-images-per-panel", type=int, default=1)
98+ parser.add_argument("--overwrite", action="store_true")
99+ parser.add_argument(
100+ "--clean-stale",
101+ action=argparse.BooleanOptionalAction,
102+ default=True,
103+ help="When generating a full spec with --overwrite, remove stale raw-panel-*.png files for panels no longer in the spec.",
104+ )
105+ parser.add_argument("--dry-run", action="store_true")
106+ parser.add_argument("--print-prompts", action="store_true", help="Print full prompts as panels are planned")
107+ parser.add_argument("--cpu-offload", action=argparse.BooleanOptionalAction, default=True)
108+ parser.add_argument("--mixed-precision", choices=["fp16", "bf16", "no"], default="fp16")
109+ return parser.parse_args()
110+
111+
112+def resolve_specs(args: argparse.Namespace) -> list[Path]:
113+ if args.all:
114+ return sorted(args.specs_dir.glob("*.json"))
115+ if args.spec is not None:
116+ return [args.spec]
117+ if args.slug:
118+ return [args.specs_dir / f"{args.slug}.json"]
119+ raise SystemExit("Provide a spec path, --slug <slug>, or --all")
120+
121+
122+def load_spec(spec_path: Path) -> dict[str, Any]:
123+ if not spec_path.exists():
124+ raise FileNotFoundError(f"Spec not found: {spec_path}")
125+ spec = json.loads(spec_path.read_text(encoding="utf-8"))
126+ if not isinstance(spec.get("panels"), list) or not spec["panels"]:
127+ raise ValueError(f"Spec has no panels: {spec_path}")
128+ for index, panel in enumerate(spec["panels"], start=1):
129+ if not str(panel.get("artPrompt", "")).strip():
130+ raise ValueError(f"Panel {index} in {spec_path} is missing artPrompt")
131+ if not str(spec.get("slug", "")).strip():
132+ spec["slug"] = spec_path.stem
133+ return spec
134+
135+
136+def resolve_character_prompt(args: argparse.Namespace) -> str:
137+ if args.no_character_prompt:
138+ return ""
139+ if args.character_prompt_file:
140+ return args.character_prompt_file.read_text(encoding="utf-8").strip()
141+ return args.character_prompt.strip()
142+
143+
144+def join_prompt_parts(*parts: str) -> str:
145+ return ", ".join(part.strip() for part in parts if part and part.strip())
146+
147+
148+def build_prompts(
149+ art_prompt: str,
150+ style_prompt: str,
151+ character_prompt: str,
152+ panel_suffix: str,
153+ prompt_2_mode: str,
154+) -> tuple[str, str]:
155+ # SDXL has two text encoders, each with a short CLIP token limit. The default
156+ # split mode keeps panel/action direction in prompt and reusable character
157+ # definitions in prompt_2. For small LoRAs, mirroring prompt into prompt_2 is
158+ # useful because training captions conditioned both text encoders on the same
159+ # caption text.
160+ prompt = join_prompt_parts(style_prompt, art_prompt, panel_suffix)
161+ if prompt_2_mode == "same":
162+ prompt_2 = prompt
163+ elif prompt_2_mode == "none":
164+ prompt_2 = ""
165+ else:
166+ prompt_2 = join_prompt_parts(style_prompt, character_prompt, panel_suffix)
167+ return prompt, prompt_2
168+
169+
170+def output_name(panel_index: int, variant_index: int, num_variants: int) -> str:
171+ if num_variants == 1:
172+ return f"raw-panel-{panel_index:02d}.png"
173+ return f"raw-panel-{panel_index:02d}-{variant_index:02d}.png"
174+
175+
176+def plan_jobs(spec: dict[str, Any], spec_output_dir: Path, args: argparse.Namespace) -> list[PanelJob]:
177+ requested_panels = set(args.panel or [])
178+ jobs: list[PanelJob] = []
179+
180+ for panel_index, panel in enumerate(spec["panels"], start=1):
181+ if requested_panels and panel_index not in requested_panels:
182+ continue
183+ for variant_index in range(1, args.num_images_per_panel + 1):
184+ seed = args.seed + (panel_index - 1) * args.seed_stride + (variant_index - 1)
185+ jobs.append(
186+ PanelJob(
187+ index=panel_index,
188+ art_prompt=panel["artPrompt"],
189+ output_path=spec_output_dir / output_name(panel_index, variant_index, args.num_images_per_panel),
190+ seed=seed,
191+ )
192+ )
193+
194+ if not jobs:
195+ raise ValueError("No panels selected")
196+ return jobs
197+
198+
199+def load_pipeline(args: argparse.Namespace) -> Any:
200+ import torch
201+ from diffusers import AutoencoderKL, DPMSolverMultistepScheduler, StableDiffusionXLPipeline
202+
203+ dtype = {"fp16": torch.float16, "bf16": torch.bfloat16, "no": torch.float32}[args.mixed_precision]
204+
205+ vae = None
206+ if args.vae_model_name_or_path:
207+ vae = AutoencoderKL.from_pretrained(args.vae_model_name_or_path, torch_dtype=dtype)
208+
209+ pipe = StableDiffusionXLPipeline.from_pretrained(
210+ args.pretrained_model_name_or_path,
211+ vae=vae,
212+ torch_dtype=dtype,
213+ variant="fp16" if args.mixed_precision == "fp16" else None,
214+ use_safetensors=True,
215+ )
216+ pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)
217+
218+ if not args.no_lora:
219+ if not args.lora_path.exists():
220+ raise FileNotFoundError(f"LoRA path not found: {args.lora_path}")
221+ if args.lora_path.is_file():
222+ pipe.load_lora_weights(str(args.lora_path.parent), weight_name=args.lora_path.name, adapter_name="ctdstyle")
223+ else:
224+ lora_kwargs: dict[str, Any] = {}
225+ default_weight = args.lora_path / "pytorch_lora_weights.safetensors"
226+ if default_weight.exists():
227+ lora_kwargs["weight_name"] = default_weight.name
228+ pipe.load_lora_weights(str(args.lora_path), adapter_name="ctdstyle", **lora_kwargs)
229+ if hasattr(pipe, "set_adapters"):
230+ pipe.set_adapters(["ctdstyle"], adapter_weights=[args.lora_scale])
231+
232+ if args.cpu_offload:
233+ pipe.enable_model_cpu_offload()
234+ else:
235+ pipe.to("cuda" if torch.cuda.is_available() else "cpu")
236+
237+ return pipe
238+
239+
240+def remove_stale_raw_panels(spec_output_dir: Path, jobs: list[PanelJob], args: argparse.Namespace) -> None:
241+ if args.dry_run or not args.overwrite or not args.clean_stale or args.panel:
242+ return
243+
244+ expected = {job.output_path.resolve() for job in jobs}
245+ for path in sorted(spec_output_dir.glob("raw-panel-*.png")):
246+ if path.resolve() not in expected:
247+ print(f"remove stale {path}")
248+ path.unlink()
249+
250+
251+def generate_spec(spec_path: Path, pipe: Any | None, args: argparse.Namespace) -> None:
252+ import torch
253+
254+ spec = load_spec(spec_path)
255+ slug = spec["slug"]
256+ spec_output_dir = args.output_dir if args.output_dir else args.assets_dir / slug
257+ spec_output_dir.mkdir(parents=True, exist_ok=True)
258+
259+ jobs = plan_jobs(spec, spec_output_dir, args)
260+ remove_stale_raw_panels(spec_output_dir, jobs, args)
261+ character_prompt = resolve_character_prompt(args)
262+ records: list[dict[str, Any]] = []
263+
Cut short: this commit's diff is over 1000 lines.
comic/pyproject.toml+30−0
Cut short: this commit's diff is over 1000 lines.
comic/specs/limb-budget.json+6−63
Cut short: this commit's diff is over 1000 lines.
comic/specs/the-cycle-of-ai-coding-burnout.json+5−18
Cut short: this commit's diff is over 1000 lines.
comic/specs/the-dumb-zone.json+0−33
Cut short: this commit's diff is over 1000 lines.
comic/specs/the-refactor.json+0−44
Cut short: this commit's diff is over 1000 lines.
comic/train_lora.py+442−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/clanker-error.png+0−0
Binary file, not shown.
comic/training-data/clanker-error.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/clanker-happy.png+0−0
Binary file, not shown.
comic/training-data/clanker-happy.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/clanker-neutral.png+0−0
Binary file, not shown.
comic/training-data/clanker-neutral.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/dev-dead.png+0−0
Binary file, not shown.
comic/training-data/dev-dead.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/dev-happy.png+0−0
Binary file, not shown.
comic/training-data/dev-happy.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/dev-laptop-sitting.png+0−0
Binary file, not shown.
comic/training-data/dev-laptop-sitting.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/dev-laptop-walking.png+0−0
Binary file, not shown.
comic/training-data/dev-laptop-walking.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/dev-neutral.png+0−0
Binary file, not shown.
comic/training-data/dev-neutral.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/dev-thinking.png+0−0
Binary file, not shown.
comic/training-data/dev-thinking.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-204225-001-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-204225-001-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-204225-001-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-204225-001-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-204225-002-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-204225-002-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-204225-002-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-204225-002-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-204225-003-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-204225-003-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-204225-003-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-204225-003-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-204225-004-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-204225-004-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-204225-004-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-204225-004-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-204225-006-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-204225-006-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-204225-006-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-204225-006-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-204225-007-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-204225-007-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-204225-007-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-204225-007-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-204225-008-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-204225-008-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-204225-009-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-204225-009-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-204225-009-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-204225-009-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-204225-010-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-204225-010-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-204225-010-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-204225-010-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-001-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-001-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-001-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-001-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-002-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-002-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-002-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-002-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-003-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-003-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-003-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-003-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-004-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-004-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-004-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-004-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-005-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-005-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-005-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-005-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-006-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-006-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-006-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-006-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-007-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-007-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-007-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-007-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-008-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-008-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-008-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-008-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-009-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-009-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-009-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-009-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-010-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-010-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-20260610-205131-010-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-20260610-205131-010-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-promoted-manifest.json+476−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-v3-20260610-233051-001-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-v3-20260610-233051-001-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-v3-20260610-233051-001-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-v3-20260610-233051-001-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-v3-20260610-233051-002-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-v3-20260610-233051-002-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-v3-20260610-233051-002-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-v3-20260610-233051-002-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-v3-20260610-233051-003-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-v3-20260610-233051-003-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-v3-20260610-233051-003-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-v3-20260610-233051-003-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-v3-20260610-233051-004-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-v3-20260610-233051-004-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-v3-20260610-233051-004-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-v3-20260610-233051-004-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-v3-20260610-233051-006-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-v3-20260610-233051-006-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-v3-20260610-233051-006-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-v3-20260610-233051-006-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-v3-20260610-233051-007-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-v3-20260610-233051-007-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-v3-20260610-233051-007-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-v3-20260610-233051-007-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-v3-20260610-233051-008-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-v3-20260610-233051-008-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-v3-20260610-233051-008-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-v3-20260610-233051-008-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-v3-20260610-233051-009-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-v3-20260610-233051-009-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-v3-20260610-233051-009-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-v3-20260610-233051-009-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-v3-20260610-233051-010-01.png+0−0
Binary file, not shown.
comic/training-data/synthetic-v3-20260610-233051-010-01.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/training-data/synthetic-v3-20260610-233051-010-02.png+0−0
Binary file, not shown.
comic/training-data/synthetic-v3-20260610-233051-010-02.txt+1−0
Cut short: this commit's diff is over 1000 lines.
comic/uv.lock+1247−0
Cut short: this commit's diff is over 1000 lines.