70160
Ce
cel-tui / files
cel-tui / benchmarks / RESULTS.md
15.6 KB
1# cel-tui Benchmark Results 2 3> These are historical snapshots, not universal performance claims. For 4> before/after work, run `bun run bench:regression` on the same machine, Bun 5> version, and power profile; it emits environment metadata and raw fixed-work 6> samples as JSON. 7 8## 2026-07-09 quality-pass comparison 9 10Same-machine, same-runtime (`Bun 1.3.13`) fixed-work medians recorded before 11and after the rendering quality pass: 12 13| Case | Before | After | Change | 14| ----------------------------------------- | ------- | ------- | ------ | 15| Allocate `CellBuffer(120, 40)` | 92.8 µs | 3.4 µs | -96.3% | 16| Allocate + paint 1,000 messages at top | 1.33 ms | 0.44 ms | -67.1% | 17| Allocate + paint 1,000 messages at bottom | 8.61 ms | 0.45 ms | -94.8% | 18 19The bottom-scroll gain comes from translating and clipping during traversal 20instead of cloning every layout node in a scrolled subtree. The allocation 21gain comes from sharing an immutable empty-cell sentinel. The checked-in 22`bench:regression` harness now makes equivalent future comparisons 23repeatable and records its raw samples and environment as JSON. 24 25Two mid-pass checkpoints isolate the later ASCII painting fast path: the 26Ink-shaped end-to-end case moved from 0.663 ms to 0.198 ms (-70.1%), and the 2720-message app case from 4.29 ms to 0.466 ms (-89.1%). These are not the 28original repository baseline, so they are intentionally separate from the 29table above. 30 31The full test suite also moved from 561 tests in 4.77 s to 608 tests in 320.40 s after replacing fixed timer sleeps with a deterministic render-flush 33seam. Line coverage increased from 93.20% to 93.52%. 34 35### SyntaxHighlight cache scaling 36 37The same quality pass replaced the global four-state content-prefix heuristic 38with a bounded exact-render cache plus explicit `createSyntaxHighlight()` 39instances. The fixed workload below renders 24-line TypeScript snippets for 30 40frames after three warmup frames on the same Bun 1.3.13 process shape: 41 42| Static snippets per frame | Before | After | 43| ------------------------- | --------- | --------- | 44| 4 | 0.0058 ms | 0.0067 ms | 45| 5 | 1.7105 ms | 0.0082 ms | 46| 5 / 4 scaling ratio | 297.3x | 1.22x | 47 48Crossing the old four-entry boundary made every lookup miss. At five snippets, 49the new cache removes 99.5% of that frame cost while scaling close to the ideal 501.25x increase in work. `bench:regression` now includes fixed-work 4-snippet, 515-snippet, and 32-isolated-instance cases; one recorded run measured medians of 523.40 µs, 4.17 µs, and 0.93 µs per frame respectively. 53 54### Variable-height VirtualList 55 56The first-class `VirtualList` regression case uses 5,000 keyed, variable-height 57mini-coder message rows at 120×40. It includes callable component rendering, 58visible-row measurement, prefix/window calculation, spacer construction, and 59layout. A same-process steady-state run on Bun 1.3.13 measured: 60 61| Case | Median | 62| ----------------------------------- | ------------- | 63| Build + layout all 5,000 rows | 134.39 ms | 64| VirtualList + layout visible window | 2.170 ms | 65| Relative reduction / speedup | 98.4% / 61.9× | 66 67The virtual result retains cell-accurate measured heights for active rows and 68uses bounded estimates for cold rows; only the viewport plus configured 69overscan appears in the returned node tree. Both cases remain in 70`bench:regression` so future changes can be compared with the same fixed-work 71harness and raw samples. 72 73### Editing-key lookup 74 75The exploratory key benchmark exposed an immutable `Set` being rebuilt for 76every named key event. Hoisting that lookup removed the hot-path allocation on 77the same machine and similar measured clock (~1.7 GHz): 78 79| Case | Before | After | Change | 80| ------------------------ | ------- | ------- | ------------- | 81| `isEditingKey("enter")` | 1.39 µs | 3.94 ns | -99.7% / 353× | 82| `isEditingKey("ctrl+s")` | 1.26 µs | 6.03 ns | -99.5% / 209× | 83 84`bench:regression` now includes the combined named-key/shortcut lookup; one 85recorded fixed-work median was 11.09 ns for both calls together. 86 87**Machine:** 12th Gen Intel Core i9-12900, ~4.4 GHz 88**Runtime:** Bun 1.3.9 (x64-linux) 89**Date:** 2026-04-05 (updated from 2026-04-04 baseline) 90 91## Ink Comparison (caveat emptor) 92 93These numbers compare cel-tui's end-to-end pipeline against Ink using a 94visually similar tree (styled text, flexbox columns, word-wrap, nested 95containers with colors — the same shape as Ink's own `benchmark/simple`). 96 97**This is not an apples-to-apples comparison.** Ink is a React-based 98framework that does significantly more work per render: JSX → React 99reconciliation → Yoga layout (C++ via NAPI) → segment compositing → 100ANSI output. It supports React hooks, component lifecycle, concurrent 101mode, and a rich ecosystem of composable components. cel-tui is a 102minimal stateless framework that builds plain JS objects and writes 103cells into a flat buffer — it's a fundamentally different (and much 104simpler) architecture. 105 106The comparison is included as a rough reference point, not a claim of 107superiority. Ink solves a broader set of problems; cel-tui trades 108features for raw throughput. 109 110| What | cel-tui | Ink 6.8.0 | Ratio | 111| ------------------------------------------ | ---------- | ------------------------------------------------- | --------- | 112| **Single render (comparable tree, 80×24)** | **65 µs** | 632 µs (`renderToString`) | **~10×** | 113| **1,000 re-renders** | **67 ms** | 632 ms (`renderToString`) / 1,462 ms (`rerender`) | **9–22×** | 114| **Projected 100k re-renders** | **~6.7 s** | ~63 s (`renderToString`) / ~146 s (`rerender`) | **9–22×** | 115 116The Ink benchmark script we used is a custom `renderToString` loop (see 117[How to Reproduce](#how-to-reproduce)), not Ink's own shipped benchmark, 118which does 100k `rerender()` calls that also write to stdout. Both 119numbers are included above. 120 121### What each framework does per render 122 123| Step | cel-tui | Ink | 124| -------------------- | -------------------------------- | ------------------------------- | 125| Tree construction | Plain JS objects (~0 cost) | React.createElement + JSX | 126| Layout | Custom flexbox (TypeScript) | Yoga (C++ via NAPI) | 127| Painting | Cell buffer (typed array writes) | Segment compositor (string ops) | 128| Output | ANSI emitter (string concat) | ANSI string builder (chalk) | 129| State reconciliation | None (stateless) | React reconciler | 130 131## cel-tui Pipeline Breakdown 132 133### visibleWidth (character measurement) 134 135| Input | Time | 136| ---------------------------- | -------- | 137| ASCII short (5 chars) | 4.1 ns | 138| ASCII (64 chars) | 39.1 ns | 139| ASCII long (640 chars) | 386.6 ns | 140| CJK + emoji (26 graphemes) | 26.6 ns | 141| Mixed ASCII + CJK (60 chars) | 4.9 ns | 142| ANSI escape sequences | 3.2 ns | 143| Empty string | 0.1 ns | 144 145The fast ASCII path dominates — pure ASCII strings bypass grapheme segmentation entirely. 146 147### Layout Engine 148 149| Tree | Time | 150| ----------------------------- | -------- | 151| 10 children (flat) | 1.5 µs | 152| 100 children (flat) | 12.0 µs | 153| 1,000 children (flat) | 125.1 µs | 154| depth=3 breadth=3 (40 nodes) | 3.0 µs | 155| depth=4 breadth=3 (121 nodes) | 15.1 µs | 156| depth=3 breadth=5 (156 nodes) | 9.5 µs | 157| 50 styled groups (200 nodes) | 52.6 µs | 158| 10 word-wrap paragraphs | 2.6 µs | 159| 50 word-wrap paragraphs | 13.1 µs | 160| Ink-comparable tree | 1.7 µs | 161| App tree (20 messages) | 10.1 µs | 162| App tree (100 messages) | 39.6 µs | 163 164Layout scales linearly with node count (~120 ns/node for flat trees). 165 166### Paint (cell buffer writing) 167 168| Scenario | Time | 169| ----------------------------- | ------ | 170| 10 children (flat, 120×40) | 93 µs | 171| 100 children (flat, 120×40) | 193 µs | 172| 1,000 children (flat, 120×40) | 192 µs | 173| Nested 121 nodes (120×40) | 180 µs | 174| 50 styled groups (120×40) | 207 µs | 175| 200 lines scrollable (120×40) | 185 µs | 176| 50 word-wrapped paragraphs | 286 µs | 177| Ink-comparable (80×24) | 45 µs | 178| App tree 20 msgs (120×40) | 208 µs | 179| 80×24 (small terminal) | 109 µs | 180| 120×40 (medium terminal) | 196 µs | 181| 200×50 (large terminal) | 284 µs | 182 183Paint is O(cells), not O(nodes) — 100 vs 1,000 children produce similar times because only visible cells are painted. Cost scales with terminal size. 184 185### CellBuffer & ANSI Emission 186 187| Operation | Time | 188| ---------------------------- | -------------------------- | 189| Buffer creation 80×24 | 24.7 µs | 190| Buffer creation 120×40 | 58.2 µs | 191| Buffer creation 200×50 | 117.0 µs | 192| emitBuffer 80×24 (full) | 21.4 µs | 193| emitBuffer 120×40 (full) | 46.4 µs | 194| emitDiff 80×24 (no changes) | 11.8 µs | 195| emitDiff 80×24 (partial) | invalid historical fixture | 196| emitDiff 80×24 (full change) | invalid historical fixture | 197 198The original partial/full-change fixtures accidentally produced identical 199visible buffers, so their published timings did not measure the named cases. 200The fixtures now assert that exactly one or every cell changes, respectively; 201use `bun run benchmarks/cell-buffer.bench.ts` for corrected measurements. 202 203### Hit Test & Focus 204 205| Operation | Time | 206| ---------------------------- | ------- | 207| hitTest flat 100, center | 145 ns | 208| hitTest nested 121, center | 41 ns | 209| hitTest app 50 msgs, center | 108 ns | 210| hitTest miss (out of bounds) | 7.7 ns | 211| collectFocusable 100 buttons | 1.13 µs | 212| collectFocusable app tree | 0.96 µs | 213 214### Key Parsing 215 216| Input | Time | 217| --------------------- | ------ | 218| Printable char 'a' | 15 ns | 219| Ctrl+C (0x03) | 18 ns | 220| Escape | ~1 ns | 221| Arrow up (CSI) | 58 ns | 222| Shift+Tab (CSI Z) | <1 ns | 223| F5 (CSI 15~) | 111 ns | 224| Backspace (0x7F) | 7.1 ns | 225| normalizeKey "ctrl+s" | 84 ns | 226 227### End-to-End Pipeline 228 229| Scenario | Time | Renders/sec | 230| ------------------------- | ------ | ----------- | 231| Ink-comparable (80×24) | 67 µs | **14,900** | 232| Flat 100 lines (80×24) | 152 µs | 6,600 | 233| App tree 20 msgs (120×40) | 268 µs | 3,700 | 234| Re-render no-change diff | 61 µs | 16,300 | 235| Re-render +1 message diff | 248 µs | 4,030 | 236| 1,000 re-renders batch | 67 ms | 14,800 | 237 238## Real-World Synthetic App Shapes 239 240Focused run for the new `benchmarks/real-world.bench.ts` cases. 241These numbers use fixtures shaped after `mini-coder` and `mini-coder-next`: 242long unvirtualized conversations, syntax-highlight-like wrapping HStack token 243rows, and mini-coder-style virtualized scrollback slices. 244 245**Runtime:** Bun 1.3.13 (x64-linux) 246**Date:** 2026-04-25 247 248### mini-coder-next-style unvirtualized conversation 249 250| Scenario | Time | 251| -------------------------------------- | ------ | 252| Layout 100 messages (120×40) | 71 ms | 253| Layout 500 messages (120×40) | 348 ms | 254| Layout 1,000 messages (120×40) | 740 ms | 255| Paint pre-laid 500 messages (120×40) | 4.0 ms | 256| Paint pre-laid 1,000 messages (120×40) | 6.5 ms | 257| Full render 250 messages (120×40) | 172 ms | 258| Diff render 250 → 251 messages | 179 ms | 259 260The new benchmark exposes the real bottleneck that the older app-tree cases 261missed: long unvirtualized conversation layout dominates render time. Paint is 262much smaller but still grows for token-heavy visible content. 263 264### Long wrapping HStack token lists 265 266| Scenario | Time | 267| ---------------------------- | ------ | 268| Layout 100 tokens (120×40) | 314 µs | 269| Layout 500 tokens (120×40) | 1.6 ms | 270| Layout 1,000 tokens (120×40) | 3.2 ms | 271| Paint 100 tokens (120×40) | 771 µs | 272| Paint 500 tokens (120×40) | 3.5 ms | 273| Paint 1,000 tokens (120×40) | 6.0 ms | 274 275These rows mimic `SyntaxHighlight` output: each logical line becomes an 276`HStack({ flexWrap: "wrap" })` with many styled `Text` spans. 277 278### mini-coder-style virtualized conversation 279 280| Scenario | Time | 281| ----------------------------------------------- | ------ | 282| Layout total 1,000 / visible 40 (120×40) | 31 ms | 283| Layout total 5,000 / visible 40 (120×40) | 26 ms | 284| Paint total 1,000 / visible 40 (120×40) | 1.6 ms | 285| Paint total 5,000 / visible 40 (120×40) | 1.6 ms | 286| Measure content height, 250 messages, width 118 | 56 ms | 287 288The virtualized shape is mostly insensitive to total message count because the 289rendered node count is bounded by the visible slice plus spacers. The remaining 290cost is from the visible token-heavy message nodes and cold height measurement. 291 292## Ink Raw Numbers 293 294Ink v6.8.0, visually similar tree to cel-tui's "ink-comparable" 295scenario. Run with Bun 1.3.9 on the same machine. Note: the 296`renderToString` benchmark was written by us (not shipped by Ink) to 297isolate the render pipeline from terminal I/O. 298 299### `renderToString` (pure pipeline, no I/O) 300 301``` 302Run 1: 1,000 renders: 632.75 ms — 632 µs/render 303Run 2: 1,000 renders: 632.66 ms — 633 µs/render 304Run 3: 1,000 renders: 630.99 ms — 631 µs/render 305``` 306 307### `rerender` (full render with React reconciler + stdout) 308 309``` 310Run 1: 1,000 re-renders: 1473.63 ms — 1,474 µs/render 311Run 2: 1,000 re-renders: 1461.48 ms — 1,461 µs/render 312Run 3: 1,000 re-renders: 1450.79 ms — 1,451 µs/render 313``` 314 315## How to Reproduce 316 317```bash 318# cel-tui benchmarks 319cd cel-tui 320bun install 321bun run benchmarks/run-all.ts # full suite 322bun run benchmarks/e2e.bench.ts # just the headline numbers 323bun run benchmarks/real-world.bench.ts # real app-shaped stress cases 324 325# Ink benchmarks 326git clone https://github.com/vadimdemedes/ink.git /tmp/ink-bench 327cd /tmp/ink-bench 328npm install && npm run build 329 330# Create a renderToString benchmark (Ink ships rerender-to-stdout only) 331cat > benchmark/simple/bench-rts.tsx << 'BENCH' 332import React from "react"; 333import { renderToString, Box, Text } from "../../src/index.js"; 334 335function App() { 336 return ( 337 <Box flexDirection="column" padding={1}> 338 <Text underline bold color="red">{"Hello World"}</Text> 339 <Box marginTop={1} width={60}> 340 <Text> 341 Cupcake ipsum dolor sit amet candy candy. Sesame snaps cookie 342 I love tootsie roll apple pie bonbon wafer. Caramels sesame 343 snaps icing cotton candy I love cookie sweet roll. 344 </Text> 345 </Box> 346 <Box marginTop={1} flexDirection="column"> 347 <Text backgroundColor="white" color="black">Colors:</Text> 348 <Box flexDirection="column" paddingLeft={1}> 349 <Text>- <Text color="red">Red</Text></Text> 350 <Text>- <Text color="blue">Blue</Text></Text> 351 <Text>- <Text color="green">Green</Text></Text> 352 </Box> 353 </Box> 354 </Box> 355 ); 356} 357 358renderToString(<App />, { columns: 80 }); // warmup 359const start = performance.now(); 360for (let i = 0; i < 1_000; i++) renderToString(<App />, { columns: 80 }); 361const elapsed = performance.now() - start; 362console.log(`1,000 renders: ${elapsed.toFixed(2)} ms (${(elapsed / 1000).toFixed(3)} ms/render)`); 363BENCH 364 365bun run benchmark/simple/bench-rts.tsx 366```