xonecas.io
8447
Co

conveyor / commit e47a7f7

app: load widget builds in place instead of restarting

The app is a thin bin over the conveyor_app dylib; the widgets build into
conveyor_widgets, which the running app dlopens and swaps in with
Live::reload. A build against another app build waits for the exec restart.
cargo app (and conveyord) build both with prefer-dynamic into target/app;
conveyord stays static. Spike 8 notes, DESIGN and the library's CLAUDE.md.

Co-Authored-By: Claude Opus 5.5

sacenox2026-11-09 e47a7f7bc5d07f0096593f38f1ef78f42374481a parent 7aeeb8c

23 files changed

.cargo/config.toml+4−0
@@ -0,0 +1,4 @@
1+[alias]
2+# The app and the widgets it loads share one std and one Rune, so both link them dynamically.
3+# Their own target dir, since the flag rebuilds everything. Run it from the repo's root.
4+app = ["build", "--release", "-p", "conveyor-app", "-p", "conveyor-widgets", "--target-dir", "target/app", "--config", "build.rustflags=['-Cprefer-dynamic']"]
Cargo.lock+8−0
@@ -605,6 +605,14 @@ dependencies = [
605605 "serde_json",
606606 ]
607607
608+[[package]]
609+name = "conveyor-widgets"
610+version = "0.0.1"
611+dependencies = [
612+ "conveyor-app",
613+ "rune",
614+]
615+
608616 [[package]]
609617 name = "core-foundation"
610618 version = "0.9.4"
Cargo.toml+3−1
@@ -1,5 +1,7 @@
11 [workspace]
2-members = ["crates/conveyor", "crates/conveyor-app"]
2+members = ["crates/conveyor", "crates/conveyor-app", "crates/conveyor-widgets"]
3+# The app and widgets link std dynamically: build them with `cargo app`.
4+default-members = ["crates/conveyor"]
35 resolver = "3"
46
57 [workspace.dependencies]
DESIGN.md+7−5
@@ -174,7 +174,7 @@ Measured in spike 1, on the host:
174174 - Verifiable by agents: headless PNG, element tree and scripted input (Rune), plus AT-SPI through AccessKit on the real desktop.
175175 - **Inbox first**: every CTA across all projects, one line each. This is the main view.
176176 - Project → tickets → ticket detail: the contract, the timeline from `events.jsonl`, the branch diff, and VNC.
177-- **Widgets are single files in the library**, `widgets/<name>.rs`, compiled into the app. The app is a view and restarts with no state to lose, so a new widget costs one incremental build. The core views are built the same way, so conveyor's own agents can change conveyor.
177+- **Widgets are single files in the library**, `widgets/<name>.rs`, compiled into a library the running app loads in place. A new widget costs one incremental build, and the window stays. The core views are built the same way, so conveyor's own agents can change conveyor.
178178
179179 ### The widget builder
180180
@@ -191,7 +191,7 @@ window ─ core draws ─► root (library) core, drawn above t
191191 Ctrl+click ─► conveyord: widgets ticket (kind, instance chain, tree, shared, thread)
192192 ─► VM: claude starts from the trial, edits widgets/, checks with the preview,
193193 `conveyor place <kind> --in <instance>` for free children, `conveyor done`
194- ─► review: trial = main + every build in review ─► cargo build ─► app exec()s itself
194+ ─► review: trial = main + every build in review ─► cargo app ─► the app loads it in place
195195 ─► the user tries it: Keep ─► merged into main, done
196196 Discard ─► back to draft, trial rebuilt without it
197197 ```
@@ -205,10 +205,11 @@ Ctrl+click ─► conveyord: widgets ticket (kind, instance chain, tree, shared,
205205 - **The trial's edges are tested**: a branch that doesn't build, two that conflict, a follow-up built on the trial, a discard (`cargo test --release -p conveyor trial`, on a throwaway library and target dir).
206206 - **Placement is explicit.** The app places nothing unless the agent ran `conveyor place`. Guessing placed widgets twice, four times over.
207207 - **Widgets act from anywhere.** `cx.act(|live| ..)` does from `view`, `update` and `event` what the free functions do from handlers.
208-- **Hidden work survives restarts.** Instances exist once rendered; `cx.keep_running()` from `update` makes the app bring one back after a restart even when nothing shows it.
208+- **Library changes load in place** (spike 8). The app is a 7 KB binary over `libconveyor_app.so` (app, `api.rs`, Rune, std); the widgets are `libconveyor_widgets.so`, rebuilt in 0.45 s and `dlopen`ed by the running app. Every instance is dropped and made again from its folder, so widgets save as they go and stop their threads and processes in `Drop`. Widgets built against another build of the app wait for its `exec` restart, which now happens only when conveyor itself changes. Both are built wi …
209+- **Hidden work survives reloads.** Instances exist once rendered; `cx.keep_running()` from `update` makes the app bring one back after a reload or restart even when nothing shows it.
209210 - **Widgets run with every permission the user has.** They are native code in the app, on the host: any file, any program, the network. The agent that writes them is sandboxed in a VM, the code it writes is not. That's deliberate for now, and may be a feature (a widget can drive the whole machine). Revisit before anyone runs someone else's library.
210211 - **`api.rs` is conveyor's contract**, copied into the library on every start along with `gen.rs`, `CLAUDE.md` and the preview. The preview is the app's own host (`api::Live`) with sample data, so the agent checks exactly what the app runs.
211-- The app restarts with `exec` on the path it read at launch, and conveyord copies itself into VMs from `/proc/self/exe`: after cargo replaces a binary, `current_exe()` names the deleted file.
212+- The app restarts (on a conveyor change) with `exec` on the path it read at launch, and conveyord copies itself into VMs from `/proc/self/exe`: after cargo replaces a binary, `current_exe()` names the deleted file.
212213 - The base image carries the pinned toolchain and a warm `~/.cache/conveyor-target`, so the agent's first preview build is incremental.
213214
214215 #### What spike 5 showed (`spikes/5-builder/NOTES.md`)
@@ -244,7 +245,7 @@ Why Rune:
244245 2. **The fastest edit loop.** A mod rebuilds in 0.55 s, and edit-to-visible is about 0.9 s, restart included.
245246 3. **The least code (208 lines) and the fastest first frame (284 ms).** Its typography was also the best of the four, and the OpenCode app proves it holds up in a real product.
246247
247-Mods are a **rebuild**, not a runtime-loaded file. That changes criterion 2 below. It works because the UI is only a view: conveyord holds all state, so restarting the app costs a frame. Mods get the full language rather than a DSL, at the cost of a Rust toolchain on the host.
248+Mods are a **rebuild**, not a runtime-loaded file (since spike 8, the rebuilt widgets load into the running app, but they are still compiled Rust). That changes criterion 2 below. It works because the UI is only a view: conveyord holds all state, so restarting the app costs a frame. Mods get the full language rather than a DSL, at the cost of a Rust toolchain on the host.
248249
249250 Risks we accept for a personal tool:
250251
@@ -338,6 +339,7 @@ Each spike is a throwaway script that answers one question with numbers. They ra
338339 | 5 | What API does a self-modifying client need? | done | 4/4 builds first try, 3 to 4 min each. Gaps ranked in [The widget builder](#the-widget-builder) |
339340 | 6 | Where is the builder's ceiling? | done | 3/3 hard builds first try (blur, vector diagram, paste → ticket). Limits are in the harness. See `spikes/6-ceiling/NOTES.md` |
340341 | 7 | Audio, video, text editing, a layout per project | done | 3/3 first try, 7 to 11 min each. ffmpeg into `rune::Pixels` plays 30 fps video. The limits are app restarts and instances that only exist once rendered. See `spikes/7-media/NOTES.md` |
342+| 8 | Library changes without reopening the app | done | Widgets are a dylib the app `dlopen`s: a build or discard shows up in the same window, 0.45 s build, 0.5 s after Discard. See `spikes/8-reload/NOTES.md` |
341343
342344 ### What the spikes taught us
343345
crates/conveyor-app/Cargo.toml+5−0
@@ -3,6 +3,11 @@ name = "conveyor-app"
33 version = "0.0.1"
44 edition = "2024"
55
6+# A dylib, with std and Rune linked dynamically (cargo app), so the widgets load against it.
7+[lib]
8+name = "conveyor_app"
9+crate-type = ["dylib"]
10+
611 [dependencies]
712 rune.workspace = true
813 serde.workspace = true
crates/conveyor-app/build.rs+10−10
@@ -1,12 +1,12 @@
1-//! Compiles the user's widget library into the app: every widgets/<name>.rs becomes a module.
2-include!("../../templates/library/gen.rs");
3-
1+//! Stamps the build, and lets the binary find this library and Rust's std next to it.
42 fn main() {
5- println!("cargo:rerun-if-env-changed=CONVEYOR_LIBRARY");
6- let home = std::path::PathBuf::from(std::env::var("HOME").unwrap());
7- let lib = std::env::var("CONVEYOR_LIBRARY").map(std::path::PathBuf::from)
8- .unwrap_or_else(|_| home.join(".local/share/conveyor/library"));
9- // Before conveyord has created the library, build with the empty template.
10- let lib = if lib.join("api.rs").exists() { lib } else { std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../templates/library") };
11- generate_widgets(&lib);
3+ println!("cargo:rerun-if-changed=src");
4+ println!("cargo:rerun-if-changed=../../templates/library/api.rs");
5+ let now = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos();
6+ println!("cargo:rustc-env=CONVEYOR_BUILD={now}");
7+ let rustc = std::env::var("RUSTC").unwrap_or("rustc".into());
8+ let out = std::process::Command::new(rustc).args(["--print", "sysroot"]).output().unwrap();
9+ let sysroot = String::from_utf8(out.stdout).unwrap();
10+ let target = std::env::var("TARGET").unwrap();
11+ println!("cargo:rustc-link-arg-bins=-Wl,-rpath,$ORIGIN:{}/lib/rustlib/{target}/lib", sysroot.trim());
1212 }
crates/conveyor-app/src/app.rs+11−6
@@ -1,7 +1,7 @@
11 //! The app's state and loop. It draws the library's `root` widget and three things of its own
22 //! above it: the Ctrl inspector, the prompt form, and build status. Everything else is widgets.
3-use crate::{client, form::{self, Form}, inspect, status};
4-use crate::widgets::api::{Event, Host, Live};
3+use crate::{client, form::{self, Form}, inspect, reload::Loader, status};
4+use crate::api::{Event, Host, Live};
55 use rune::{El, Tx, Waker, el};
66 use serde_json::Value;
77 use std::{cell::RefCell, collections::BTreeSet, path::{Path, PathBuf}, sync::{Arc, Mutex}, time::{Duration, Instant, SystemTime}};
@@ -39,6 +39,7 @@ pub struct State {
3939 /// Builds whose reply was dismissed or decided, and builds whose new widgets were placed.
4040 pub seen: BTreeSet<u64>,
4141 placed: BTreeSet<u64>,
42+ widgets: Loader,
4243 }
4344
4445 impl Host for State {
@@ -62,7 +63,9 @@ pub fn save_ids(base: &Path, name: &str, set: &BTreeSet<u64>) {
6263 pub fn init() -> State {
6364 let base = PathBuf::from(std::env::var("HOME").unwrap_or_default()).join(".local/share/conveyor/app");
6465 let _ = std::fs::create_dir_all(&base);
65- State { seen: ids(&base, "seen").unwrap_or_default(), placed: ids(&base, "placed").unwrap_or_default(), live: Live::new(base),
66+ let mut widgets = Loader::new();
67+ let live = Live::new(base.clone(), widgets.poll().unwrap_or(&[]));
68+ State { seen: ids(&base, "seen").unwrap_or_default(), placed: ids(&base, "placed").unwrap_or_default(), live, widgets,
6669 builds: vec![], connected: false, sent: Instant::now(), started: Instant::now(), inbox: Default::default(), exe: None, built: None, ctrl: false, pointer: (0.0, 0.0),
6770 picked: vec![], element: String::new(), rects: RefCell::new(vec![]), form: None, window: Default::default(), spare: rune::TextEdit::new("") }
6871 }
@@ -81,8 +84,9 @@ pub fn setup(s: &mut State, waker: Waker) {
8184 }
8285
8386 pub fn poll(s: &mut State, _t: f64, tx: &mut Tx) -> bool {
87+ let reloaded = s.widgets.poll().map(|kinds| s.live.reload(kinds)).is_some();
8488 let made = s.live.make_wanted();
85- let mut again = made;
89+ let mut again = made || reloaded;
8690 if let Some((asked, snap)) = s.inbox.lock().unwrap().take() {
8791 s.connected = snap.is_some();
8892 if let Some(snap) = snap.filter(|_| asked > s.sent) {
@@ -122,7 +126,8 @@ fn place(s: &mut State) {
122126 save_ids(s.live.base(), "placed", &s.placed);
123127 }
124128
125-/// conveyord rebuilt the app with a changed library: become the new binary, in place.
129+/// conveyord rebuilt the app itself (a change to conveyor, not to the widgets): become the
130+/// new binary, in place.
126131 fn restart(s: &mut State) {
127132 let Some(exe) = &s.exe else { return };
128133 let Some(t) = mtime(exe) else { return };
@@ -145,7 +150,7 @@ pub fn on_modifiers(s: &mut State, mods: rune::Mods, tx: &mut Tx) {
145150 fn paste(s: &mut State, c: &rune::ClipboardContent, _: &mut Tx) {
146151 let image = c.image().and_then(|img| {
147152 let dir = s.live.base().join("clips");
148- let path = dir.join(format!("{}.png", (crate::widgets::api::now() * 1000.0) as u64));
153+ let path = dir.join(format!("{}.png", (crate::api::now() * 1000.0) as u64));
149154 std::fs::create_dir_all(&dir).ok()?;
150155 std::fs::write(&path, img.png()).ok()?;
151156 Some(path)
crates/conveyor-app/src/client.rs+1−1
@@ -1,5 +1,5 @@
11 //! Talks to conveyord over its socket, and turns its snapshot into what widgets read.
2-use crate::widgets::api::{Activity, Data, Project, Ticket};
2+use crate::api::{Activity, Data, Project, Ticket};
33 use serde_json::{Value, json};
44 use std::{io::{BufRead, BufReader, Write}, os::unix::net::UnixStream, path::PathBuf};
55
crates/conveyor-app/src/lib.rs+23−0
@@ -0,0 +1,23 @@
1+//! The conveyor app: it draws the user's widget library, starting from its `root` widget, and
2+//! the building tool over it: hold Ctrl, click any widget, and say what it should become.
3+mod app;
4+mod client;
5+mod form;
6+mod inspect;
7+mod reload;
8+mod status;
9+#[path = "../../../templates/library/api.rs"]
10+pub mod api;
11+
12+use rune::{App, AppSpec, WindowOptions};
13+
14+/// The state every widget handler gets (see the library's api.rs).
15+pub type HostState = app::State;
16+
17+/// Changes with every build of this library: widgets built against another one can't load.
18+pub const BUILD: &str = env!("CONVEYOR_BUILD");
19+
20+pub fn run() {
21+ let spec = AppSpec { setup: Some(app::setup), poll: Some(app::poll), ..AppSpec::new("Conveyor", (1400.0, 900.0), app::init, app::view) };
22+ App::new(spec).on_modifiers(app::on_modifiers).window_options(WindowOptions { borderless: true, ..Default::default() }).run();
23+}
crates/conveyor-app/src/main.rs+3−18
@@ -1,20 +1,5 @@
1-//! The conveyor app: it draws the user's widget library, starting from its `root` widget, and
2-//! the building tool over it: hold Ctrl, click any widget, and say what it should become.
3-mod app;
4-mod client;
5-mod form;
6-mod inspect;
7-mod status;
8-mod widgets {
9- include!(concat!(env!("OUT_DIR"), "/widgets.rs"));
10-}
11-
12-use rune::{App, AppSpec, WindowOptions};
13-
14-/// The state every widget handler gets (see the library's api.rs).
15-pub type HostState = app::State;
16-
1+//! The conveyor app's binary. Everything is in the conveyor_app library, which the widget
2+//! library's builds link against, so they load into this process without a restart.
173 fn main() {
18- let spec = AppSpec { setup: Some(app::setup), poll: Some(app::poll), ..AppSpec::new("Conveyor", (1400.0, 900.0), app::init, app::view) };
19- App::new(spec).on_modifiers(app::on_modifiers).window_options(WindowOptions { borderless: true, ..Default::default() }).run();
4+ conveyor_app::run();
205 }
crates/conveyor-app/src/reload.rs+55−0
@@ -0,0 +1,55 @@
1+//! Loads the widget library's latest build into this process: libconveyor_widgets.so, next to
2+//! the binary. Old builds stay mapped, since their threads may still be winding down. A build
3+//! made against another build of this app waits for the app's own restart.
4+use crate::api::Kinds;
5+use std::{ffi::{CStr, CString, c_char, c_void}, path::PathBuf, time::SystemTime};
6+
7+unsafe extern "C" {
8+ fn dlopen(file: *const c_char, mode: i32) -> *mut c_void;
9+ fn dlsym(handle: *mut c_void, name: *const c_char) -> *mut c_void;
10+ fn dlerror() -> *const c_char;
11+}
12+
13+pub struct Loader {
14+ file: PathBuf,
15+ loaded: Option<SystemTime>,
16+ count: u32,
17+}
18+
19+impl Loader {
20+ pub fn new() -> Self {
21+ let dir = std::env::current_exe().ok().and_then(|e| Some(e.parent()?.to_owned())).unwrap_or_default();
22+ Loader { file: dir.join("libconveyor_widgets.so"), loaded: None, count: 0 }
23+ }
24+
25+ /// The widgets of a build not loaded yet, when there is one this app can run.
26+ pub fn poll(&mut self) -> Option<Kinds> {
27+ let t = std::fs::metadata(&self.file).and_then(|m| m.modified()).ok()?;
28+ if Some(t) == self.loaded { return None; }
29+ self.count += 1;
30+ // dlopen keys on the path: each build gets its own.
31+ let dir = std::env::var("XDG_RUNTIME_DIR").map_or(std::env::temp_dir(), PathBuf::from);
32+ let copy = dir.join(format!("conveyor-widgets-{}-{}.so", std::process::id(), self.count));
33+ std::fs::copy(&self.file, &copy).ok()?;
34+ let loaded = unsafe { open(&copy) };
35+ let _ = std::fs::remove_file(&copy);
36+ self.loaded = Some(t);
37+ match loaded {
38+ Ok((build, kinds)) if build == crate::BUILD => Some(kinds),
39+ Ok(_) => { eprintln!("widgets built against a newer app: waiting for its restart"); None }
40+ Err(e) => { eprintln!("widgets didn't load: {e}"); None }
41+ }
42+ }
43+}
44+
45+/// Safe while the library was built by the same compiler against this app: the build id says so.
46+unsafe fn open(path: &std::path::Path) -> Result<(&'static str, Kinds), String> {
47+ let err = || unsafe { CStr::from_ptr(dlerror()).to_string_lossy().into_owned() };
48+ let path = CString::new(path.as_os_str().as_encoded_bytes()).map_err(|e| e.to_string())?;
49+ let handle = unsafe { dlopen(path.as_ptr(), 2) }; // RTLD_NOW
50+ if handle.is_null() { return Err(err()); }
51+ let entry = unsafe { dlsym(handle, c"conveyor_widgets".as_ptr()) };
52+ if entry.is_null() { return Err(err()); }
53+ let entry: fn() -> (&'static str, Kinds) = unsafe { std::mem::transmute(entry) };
54+ Ok(entry())
55+}
crates/conveyor-app/src/status.rs+1−1
@@ -54,7 +54,7 @@ fn decide(id: u64, op: &'static str, label: &'static str, fill: rune::Color) ->
5454
5555 pub fn view(s: &State) -> El {
5656 let t = tok::theme(true);
57- let now = crate::widgets::api::now() as u64;
57+ let now = crate::api::now() as u64;
5858 let mut out = col().key("status").absolute(0.0, 0.0).w_full().h_full().padding(12.0).gap(8.0).items_end().justify_end();
5959 if !s.connected { out = out.child(text("conveyord isn't running").color(t.text_secondary)); }
6060 for b in &s.builds {
crates/conveyor-widgets/Cargo.toml+13−0
@@ -0,0 +1,13 @@
1+[package]
2+name = "conveyor-widgets"
3+version = "0.0.1"
4+edition = "2024"
5+
6+# The user's widget library, loaded into the running app (see conveyor-app's reload.rs).
7+[lib]
8+name = "conveyor_widgets"
9+crate-type = ["dylib"]
10+
11+[dependencies]
12+conveyor-app = { path = "../conveyor-app" }
13+rune.workspace = true
crates/conveyor-widgets/build.rs+12−0
@@ -0,0 +1,12 @@
1+//! Compiles the user's widget library: every widgets/<name>.rs becomes a module.
2+include!("../../templates/library/gen.rs");
3+
4+fn main() {
5+ println!("cargo:rerun-if-env-changed=CONVEYOR_LIBRARY");
6+ let home = std::path::PathBuf::from(std::env::var("HOME").unwrap());
7+ let lib = std::env::var("CONVEYOR_LIBRARY").map(std::path::PathBuf::from)
8+ .unwrap_or_else(|_| home.join(".local/share/conveyor/library"));
9+ // Before conveyord has created the library, build with the empty template.
10+ let lib = if lib.join("api.rs").exists() { lib } else { std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../templates/library") };
11+ generate_widgets(&lib, Some("pub use conveyor_app::api;"));
12+}
crates/conveyor-widgets/src/lib.rs+8−0
@@ -0,0 +1,8 @@
1+//! The user's widget library, as the app loads it.
2+include!(concat!(env!("OUT_DIR"), "/widgets.rs"));
3+
4+/// The app build these widgets were made against, and the widgets.
5+#[unsafe(no_mangle)]
6+pub fn conveyor_widgets() -> (&'static str, api::Kinds) {
7+ (conveyor_app::BUILD, ALL)
8+}
crates/conveyor/src/library.rs+4−1
@@ -168,9 +168,12 @@ fn trial(fresh: Option<u32>) -> Result<(), String> {
168168 Ok(())
169169 }
170170
171+/// Builds the app and the widgets from `library`, as `cargo app` does. The running app loads
172+/// the new widgets in place.
171173 fn build_app(library: &Path) -> Result<(), String> {
172174 vm::run(Command::new("cargo").current_dir(paths::source()).env("CONVEYOR_LIBRARY", library)
173- .args(["build", "--release", "-q", "-p", "conveyor-app"])).map(|_| ())
175+ .args(["build", "--release", "-q", "-p", "conveyor-app", "-p", "conveyor-widgets", "--target-dir"]).arg(paths::app_target())
176+ .args(["--config", "build.rustflags=['-Cprefer-dynamic']"])).map(|_| ())
174177 }
175178
176179 /// Each build ticket with what it was asked about and changed, for the app's status.
crates/conveyor/src/paths.rs+5−0
@@ -44,6 +44,11 @@ pub fn vm_sock(name: &str, kind: &str) -> PathBuf {
4444 PathBuf::from(format!("/tmp/cv-{name}.{kind}"))
4545 }
4646
47+/// Where the app and its widgets are built (`cargo app`): CARGO_TARGET_DIR's `app`.
48+pub fn app_target() -> PathBuf {
49+ std::env::var("CARGO_TARGET_DIR").map_or(source().join("target"), PathBuf::from).join("app")
50+}
51+
4752 /// Conveyor's own source, to rebuild the app when the widget library changes.
4853 pub fn source() -> PathBuf {
4954 PathBuf::from(concat!(env!("CARGO_MANIFEST_DIR"), "/../.."))
spikes/8-reload/NOTES.md+68−0
@@ -0,0 +1,68 @@
1+# Spike 8: library changes without reopening the app
2+
3+Goal: a widget build shows up in the window the user is looking at. Until now, every library
4+change rebuilt the app and `exec`ed it: the window closed and reopened, video stopped for ~2 s,
5+and every media widget reinvented a watchdog to survive it.
6+
7+## Phase 1: does Rust hot reload work with Rune? (this folder)
8+
9+`shared` (the state type and a `View` trait), `widgets` (a card with a 30 fps `Pixels` thread
10+and a button) and `host`, which polls `libwidgets.so` and loads each new build with `dlopen`.
11+
12+| Check | Result |
13+|---|---|
14+| Same window and pid through 7 reloads | yes, no flash |
15+| Handlers from the new build | work. Host state survived: presses 3 → 4 |
16+| A struct change in the widget | works: instances are made again from the new code |
17+| The old build's thread | stops in `Drop`; thread count stayed at 12 |
18+| Rebuild, load | 0.29 to 0.30 s, then 0.2 to 0.6 ms to load |
19+| Memory | about +75 KB per reload: old builds stay mapped |
20+
21+What it takes:
22+
23+- std and Rune must exist once in the process. `-C prefer-dynamic` links them dynamically,
24+ into the host and every widget build.
25+- `dlopen` keys on the path, so each build is copied to its own path first.
26+- Old builds are never unloaded: their threads and queued handlers may still run.
27+- Rune's handlers are typed on the runtime's state, so the state type lives in the shared dylib.
28+- Rejected: `rune`'s `hot.rs` (subsecond) patches function bodies only; no struct changes.
29+
30+## Phase 2: in conveyor
31+
32+```
33+conveyor-app (7 KB bin) ─links─▶ libconveyor_app.so app, api.rs (Live), Rune, std
34+ ▲ links changes only when conveyor changes
35+libconveyor_widgets.so every widgets/*.rs, and conveyor_widgets() → (build id, kinds)
36+ rebuilt per build: 0.45 s
37+ app polls its mtime ─▶ copy to $XDG_RUNTIME_DIR, dlopen, unlink
38+ ─▶ same build id? ─yes─▶ live.reload(kinds): every instance dropped, made again
39+ └─no──▶ wait: the app binary changes too, and exec()s as before
40+```
41+
42+- `cargo app` builds both into `target/app`, with `-C prefer-dynamic`. conveyord does the same.
43+ conveyord itself stays static: it copies itself into VMs.
44+- The bin's rpath is `$ORIGIN` and the toolchain's lib dir, so it finds both .so files.
45+- `Live` holds its kinds instead of a compiled-in `ALL`. `Live::reload` runs pending `cx.act`s,
46+ drops every instance, then makes them again from their folders before the next render.
47+
48+| Check | Result |
49+|---|---|
50+| A title change from a scratch library | same pid, in place, 0.45 s build |
51+| A change to the app itself | "widgets built against a newer app: waiting for its restart", then one `exec` |
52+| A real build (#6, a clock in the title bar) | 2.5 min in its VM. On trial in the same window, pid unchanged |
53+| Discard #6 | gone 0.5 s after the click, same pid |
54+| After 4 reloads | 148 MB RSS, 12 threads |
55+| `cargo test --release -p conveyor trial` | passes with the new build (97 s) |
56+
57+## Left open
58+
59+- **Widgets must clean up in `Drop`.** A thread or child process the old instance doesn't stop
60+ keeps running beside the new one. The library's CLAUDE.md now says so. spike 7's `cinema`
61+ may not stop its ffmpeg in Drop.
62+- **Every reload remakes every instance**, which costs a widget its in-memory state (scroll,
63+ an unsaved buffer kept in memory only). Saving as it changes was already the rule.
64+- **TypeIds don't change between builds**, so a handler queued by an old instance could reach
65+ the new one of the same kind. The app renders again right after a reload, which keeps that
66+ window to one frame.
67+- **A change to `api.rs`** (even a comment) changes the app's build id, so it still restarts
68+ the app. Fine: that's a conveyor change, not a widget build.
templates/library/CLAUDE.md+3−2
@@ -33,10 +33,11 @@ A widget is a struct with its own state that renders itself (`api::View`). Read
3333 - **Input.** Buttons, fields, keys, drags, file drops on an element (`on_drop_files`). Reach the widget's state from a handler with `on`, `edit` or `get`.
3434 - **Window-wide events.** A paste with no text field focused, or files dropped on the window, arrive at `View::event` (pasted images are saved as PNGs first). Every instance is offered it, innermost first.
3535 - **Writable state each frame.** `View::update(&mut self, cx)` runs before every render, for every instance, shown or not: note what changed in `cx.data`, step an animation's state. Keep `view` a pure drawing of state.
36-- **Work that goes on off screen.** An instance exists once something renders it, and the app restarts on every library change. If it must keep going while hidden, such as music playing, call `cx.keep_running()` in `update` while it does: the app then brings it back after a restart even when no parent shows it.
36+- **New code, same window.** When the library changes, the app keeps running and swaps in the new build: every instance is dropped and made again with `new`, from its folder. So save what the user typed or arranged as it changes, and stop your threads and child processes in `Drop` (a flag the thread checks, a `kill`), or the old build's keep running beside the new one's. A widget that holds a process the user would notice stopping, such as music, can leave it running detached and adopt it again …
37+- **Work that goes on off screen.** An instance exists once something renders it. If it must keep going while hidden, such as music playing, call `cx.keep_running()` in `update` while it does: the app then brings it back after a reload or a restart even when no parent shows it.
3738 - **Key what a user can point at.** Give cards, buttons, columns and panes stable, meaningful keys (`card-{id}`, `col-done`, `add-project`). The Ctrl+click outline and the ticket's element path come from keys; unkeyed parts are invisible to the builder.
3839 - **Animate.** Call `cx.animate()` in `view` while something moves, and draw from `cx.now`. Without it the app renders once a second.
39-- **Files.** `cx.dir` is the instance's own folder and survives restarts. The app restarts whenever the library changes, so save what the user typed or arranged, as it changes.
40+- **Files.** `cx.dir` is the instance's own folder and survives reloads and restarts.
4041 - **Background work.** `std::thread`, and the system's programs with `std::process::Command` (`curl` for the network: std has no HTTPS). `rune::ImageCache` and `rune::Pixels` decode PNG and JPEG.
4142
4243 ## When to block instead
templates/library/api.rs+26−11
@@ -56,6 +56,9 @@ pub struct Data {
5656
5757 // ── Widgets ─────────────────────────────────────────────────────────────────────────────────
5858
59+/// Every widget kind in the library, as (file name, widget): what a host loads.
60+pub type Kinds = &'static [(&'static str, &'static Widget)];
61+
5962 /// Each widget file declares `pub const WIDGET: Widget`.
6063 pub struct Widget {
6164 pub title: &'static str,
@@ -101,8 +104,8 @@ pub struct Cx<'a> {
101104 /// Unix seconds. The app renders at least once a second; `animate` asks for more.
102105 pub now: f64,
103106 pub data: &'a Data,
104- /// This instance's own folder. What it keeps there survives restarts, and the app
105- /// restarts every time the library changes.
107+ /// This instance's own folder. What it keeps there survives restarts, and every library
108+ /// change, which drops the instance and makes it again from the new code.
106109 pub dir: PathBuf,
107110 live: &'a Live,
108111 }
@@ -125,8 +128,8 @@ impl Cx<'_> {
125128 }
126129
127130 /// Every widget kind in the library, as (name, widget).
128- pub fn kinds(&self) -> &'static [(&'static str, &'static Widget)] {
129- super::ALL
131+ pub fn kinds(&self) -> Kinds {
132+ self.live.kinds
130133 }
131134
132135 /// Acts on the app from anywhere a widget runs (view, update, event), not only handlers:
@@ -137,8 +140,8 @@ impl Cx<'_> {
137140 }
138141
139142 /// Call it from `update` while this instance has work that must go on when nothing shows
140- /// it, such as music playing: the app brings it back after a restart (every library change
141- /// restarts the app), even when no parent renders it. Stop calling it when the work stops.
143+ /// it, such as music playing: the app brings it back after a library change or a restart,
144+ /// even when no parent renders it. Stop calling it when the work stops.
142145 pub fn keep_running(&self) {
143146 self.live.running.borrow_mut().insert(self.id.to_owned());
144147 }
@@ -232,6 +235,7 @@ pub fn quote(s: &str) -> String {
232235 /// Every widget instance, what they share, and what they asked of the host.
233236 pub struct Live {
234237 pub data: Data,
238+ kinds: Kinds,
235239 base: PathBuf,
236240 instances: HashMap<String, (String, Box<dyn View>)>,
237241 shared: BTreeMap<String, String>,
@@ -302,15 +306,15 @@ impl Live {
302306 }
303307
304308 impl Live {
305- /// Instances keep their folders under `base`, and shared values in `base/shared.tsv`.
306- pub fn new(base: PathBuf) -> Self {
309+ /// Instances of `kinds` keep their folders under `base`, and shared values in `base/shared.tsv`.
310+ pub fn new(base: PathBuf, kinds: Kinds) -> Self {
307311 let shared = std::fs::read_to_string(base.join("shared.tsv")).unwrap_or_default().lines()
308312 .filter_map(|l| l.split_once('\t').map(|(k, v)| (k.to_owned(), v.replace("\\n", "\n"))))
309313 .collect();
310314 // Instances that were keeping work going come back now, shown or not.
311315 let wanted: Vec<(String, String)> = std::fs::read_to_string(base.join("running.tsv")).unwrap_or_default().lines()
312316 .filter_map(|l| l.split_once('\t').map(|(id, kind)| (id.to_owned(), kind.to_owned()))).collect();
313- Live { data: Data::default(), base, instances: HashMap::new(), shared, wanted: RefCell::new(wanted),
317+ Live { data: Data::default(), kinds, base, instances: HashMap::new(), shared, wanted: RefCell::new(wanted),
314318 animate: Cell::new(false), outbox: vec![], spare: TextEdit::new(""), dirty: false,
315319 later: RefCell::new(vec![]), running: RefCell::new(BTreeSet::new()), kept: BTreeSet::new() }
316320 }
@@ -337,7 +341,7 @@ impl Live {
337341 let wanted = std::mem::take(self.wanted.get_mut());
338342 let mut made = false;
339343 for (id, kind) in wanted {
340- let Some((_, w)) = super::ALL.iter().find(|(n, _)| *n == kind) else { continue };
344+ let Some((_, w)) = self.kinds.iter().find(|(n, _)| *n == kind) else { continue };
341345 if self.instances.get(&id).is_some_and(|(k, _)| *k == kind) { continue; }
342346 let cx = self.cx(&id, now());
343347 let _ = std::fs::create_dir_all(&cx.dir);
@@ -350,7 +354,18 @@ impl Live {
350354
351355 /// Offers new widget `kind` to instance `id`, at `at` in its points. Whether it kept it.
352356 pub fn mount(&mut self, id: &str, at: (f32, f32), kind: &str) -> bool {
353- super::ALL.iter().any(|(n, _)| *n == kind) && self.with(id, |w, cx| w.mount(kind, at, cx)) == Some(true)
357+ self.kinds.iter().any(|(n, _)| *n == kind) && self.with(id, |w, cx| w.mount(kind, at, cx)) == Some(true)
358+ }
359+
360+ /// A new build of the library: every instance is dropped, then made again from its kind's
361+ /// new code, before the next render. What they asked with `cx.act` runs first, against
362+ /// the old instances. Kinds the new build doesn't have are gone.
363+ pub fn reload(&mut self, kinds: Kinds) {
364+ self.apply();
365+ let all: Vec<(String, String)> = self.instances.drain().map(|(id, (kind, _))| (id, kind)).collect();
366+ self.kinds = kinds;
367+ self.wanted.get_mut().extend(all);
368+ self.make_wanted();
354369 }
355370
356371 /// Instances of `kind`, or any instance if `kind` is empty.
templates/library/gen.rs+8−4
@@ -1,7 +1,8 @@
11 // Writes $OUT_DIR/widgets.rs: the API, one module per widgets/<name>.rs, and ALL.
2-// The including crate defines `HostState: api::Host`, the state every handler gets.
3-// Shared by the app's and the preview's build.rs with include!, so keep it dependency free.
4-fn generate_widgets(lib: &std::path::Path) {
2+// `api` is how the API comes in: the preview compiles api.rs itself (and defines `HostState:
3+// api::Host`, the state every handler gets); the app's widgets build uses the app's.
4+// Shared by both build.rs files with include!, so keep it dependency free.
5+fn generate_widgets(lib: &std::path::Path, api: Option<&str>) {
56 let dir = lib.join("widgets");
67 println!("cargo:rerun-if-changed={}", dir.display());
78 println!("cargo:rerun-if-changed={}", lib.join("api.rs").display());
@@ -9,7 +10,10 @@ fn generate_widgets(lib: &std::path::Path) {
910 .filter_map(|e| { let p = e.path(); (p.extension()? == "rs").then_some(p.file_stem()?.to_str()?.to_owned()) })
1011 .collect();
1112 names.sort();
12- let mut out = format!("#[allow(dead_code)]\n#[path = {:?}] pub mod api;\n", lib.join("api.rs"));
13+ let mut out = match api {
14+ Some(line) => format!("{line}\n"),
15+ None => format!("#[allow(dead_code)]\n#[path = {:?}] pub mod api;\n", lib.join("api.rs")),
16+ };
1317 for n in &names { out += &format!("#[path = {:?}] pub mod {n};\n", dir.join(format!("{n}.rs"))); }
1418 out += "pub const ALL: &[(&str, &api::Widget)] = &[";
1519 for n in &names { out += &format!("({n:?}, &{n}::WIDGET), "); }
templates/library/preview/build.rs+1−1
@@ -1,5 +1,5 @@
11 include!("../gen.rs");
22
33 fn main() {
4- generate_widgets(&std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join(".."));
4+ generate_widgets(&std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join(".."), None);
55 }
templates/library/preview/src/main.rs+1−1
@@ -89,7 +89,7 @@ fn init() -> State {
8989 Some(dir) => replay(&dir, &base),
9090 None => (sample(), None),
9191 };
92- let mut live = Live::new(base);
92+ let mut live = Live::new(base, widgets::ALL);
9393 live.data = data;
9494 let mut args = args.into_iter();
9595 State { live, kind: args.next().unwrap_or_default(), mounts: args.collect(), click }