@@ -0,0 +1,82 @@
1+# Spike 7: audio, video, text editing, a layout per project
2+
3+Goal: probe the three media the ceiling table left open, with each project laid out its own
4+way. Three test projects in `/tmp`, made with ffmpeg and 0 slots: `music` (ogg, flac, mp3),
5+`film` (two mp4 with sound, one silent webm), `notes` (Markdown and Rust).
6+
7+## Builds
8+
9+Each was one Ctrl+click on the board, with the project's tab selected, and one prompt. All
10+three were done first try.
11+
12+| # | Project | Time | Lines | What the agent built |
13+|---|---|---|---|---|
14+| 5 | music | 7.3 min | 432 | `layout_<slug>` per project in `canvas`. `music_player`: playlist, waveform from ffmpeg (2 kHz mono, cached), seek by click or drag, prev/next, volume, auto-advance. Playback is a detached `ffplay` with a heartbeat watchdog |
15+| 6 | film | 8.7 min | 348 | `cinema`: ffmpeg decodes to raw RGBA on a pipe, into `rune::Pixels` at 30 fps (`-re` paced). The same ffmpeg sends the sound to PulseAudio. Live 12 fps thumbnails, controls that fade, status pane over the video |
16+| 7 | notes | 11.3 min | 644 | `file_tree`, `note_editor` (line numbers, Markdown and Rust colours in the field itself, Tab, Ctrl+S, unsaved dot, buffers kept across restarts), `note_preview` (live Markdown), board in a drawer |
17+
18+The library is now 2097 lines over 17 widgets.
19+
20+```
21+ canvas ── selected project? ── layout_<slug> exists? ──yes──▶ that layout
22+ └──no───▶ board + status pane
23+ music : music_player over board strip
24+ film : cinema under status pane
25+ notes : file_tree | note_editor | note_preview, board drawer
26+ others : board under status pane
27+```
28+
29+## Checked on the desktop
30+
31+| Check | Result |
32+|---|---|
33+| Music reaches the speakers | sink monitor RMS 5304 on drift. Seek to 75% ran `ffplay -ss 29.81` |
34+| Auto-advance, music on other tabs | drift → pulse while the film tab was up |
35+| Video with sound | 30.0 fps (frame counter 566 at 18.867 s). Sink RMS 2886 from ffmpeg's own Pulse stream |
36+| Cost | the app at 3.5% CPU while playing. ffmpeg at 189% decoding and scaling 1080p on the CPU |
37+| Controls fade, status pane blurs the video | yes |
38+| Off-screen thumbnails | their decoders stop when another tab is up |
39+| Editor | typing, live preview, unsaved dot, Ctrl+S wrote the file. Ctrl+S didn't open the inspector |
40+| Media across an app restart | music kept its ffplay (re-adopted by pid), video restarted at its spot after ~2 s. See the restart limit below |
41+
42+## What the agent couldn't do
43+
44+- **Hear.** The preview has no sound, so it reported "I couldn't hear the sound or check that it
45+ stays in sync with the picture."
46+- **See the project's files.** `/tmp/notes` and the media folders don't exist in the VM. It
47+ made its own sample files with ffmpeg in the guest.
48+- **Know the host's programs.** It assumed `ffplay` and `ffmpeg` are on the host. They are,
49+ but nothing told it so.
50+- **Publish from `update`.** The editor shares its buffer and unsaved set through the
51+ `on_layout` marker hack again, as in spike 6.
52+
53+## Core fixes in this spike
54+
55+- **Placement**, which misplaced widgets twice more. xonecas's `ticket_editor` and this
56+ spike's `layout_film` each landed as a stray card on the canvas, the second one inside the
57+ *music* project, because their parents only draw them sometimes. Two rules now:
58+ - conveyord leaves a kind out of `created` when a widget names it in `child("kind"`;
59+ - `Live::mount` never places a kind with a zero size (a layout).
60+
61+ This is still guessing. Explicit placement stays on the list.
62+- Toast titles truncate instead of running off the toast.
63+
64+## Where the ceiling is now
65+
66+The media ceiling is the host's programs, not Rune or the API: anything ffmpeg can decode, a
67+widget can show and play. The limits:
68+
69+| Limit | Seen as | API to aim for |
70+|---|---|---|
71+| Instances exist only once rendered | after a restart, music playing on a hidden project dies within 10 s: nothing runs its `update` | restore every instance at start, or a `background` flag on `Widget` |
72+| Every library change restarts the app | video stops for ~2 s; players re-adopt detached processes by pid; each media widget reinvents a watchdog | a host-owned media service that outlives restarts (`cx.play(path)` returns a handle), or hot reload |
73+| Rune's `Player` is macOS-only | widgets shell out to ffplay/ffmpeg; seek and volume restart the process | a Linux `Player` backend upstream (GStreamer or ffmpeg), and the API above built on it |
74+| The preview is silent | the agent can't check sound or sync | `preview --audio out.wav` and `--frames DIR`, so the agent can measure what it can't hear |
75+| The agent can't see the project's files | it tested on samples it made itself | a build ticket carries a listing (and small samples) of the projects the widget reads |
76+| The host's programs are unknown | the agent assumed ffplay | the ticket lists the host's relevant programs, or the media service above makes it moot |
77+| Actions only from handlers | the marker hack, a second time | `Cx` carries actions (`cx.share`, `cx.new_ticket`), as spike 6 found |
78+| Ctrl is the builder's key | Ctrl+S worked, but by design holding Ctrl (word moves) shows the inspector, and it takes a Ctrl+click in an editor | a chord or a long-hold to inspect, so editors keep Ctrl |
79+| No global now-playing | music keeps playing on other tabs, and only the music tab can stop it | product question: a media slot in the bottom bar |
80+
81+Still open as probes: a VM's live desktop in a widget, large files in the editor, and widgets
82+across several windows.