xonecas.io
70160
Ce

cel-tui / files

cel-tui / benchmarks / RESULTS.md

15.6 KB
1# cel-tui Benchmark Results
2
3> These are historical snapshots, not universal performance claims. For
4> before/after work, run `bun run bench:regression` on the same machine, Bun
5> version, and power profile; it emits environment metadata and raw fixed-work
6> samples as JSON.
7
8## 2026-07-09 quality-pass comparison
9
10Same-machine, same-runtime (`Bun 1.3.13`) fixed-work medians recorded before
11and after the rendering quality pass:
12
13| Case                                      | Before  | After   | Change |
14| ----------------------------------------- | ------- | ------- | ------ |
15| Allocate `CellBuffer(120, 40)`            | 92.8 µs | 3.4 µs  | -96.3% |
16| Allocate + paint 1,000 messages at top    | 1.33 ms | 0.44 ms | -67.1% |
17| Allocate + paint 1,000 messages at bottom | 8.61 ms | 0.45 ms | -94.8% |
18
19The bottom-scroll gain comes from translating and clipping during traversal
20instead of cloning every layout node in a scrolled subtree. The allocation
21gain comes from sharing an immutable empty-cell sentinel. The checked-in
22`bench:regression` harness now makes equivalent future comparisons
23repeatable and records its raw samples and environment as JSON.
24
25Two mid-pass checkpoints isolate the later ASCII painting fast path: the
26Ink-shaped end-to-end case moved from 0.663 ms to 0.198 ms (-70.1%), and the
2720-message app case from 4.29 ms to 0.466 ms (-89.1%). These are not the
28original repository baseline, so they are intentionally separate from the
29table above.
30
31The full test suite also moved from 561 tests in 4.77 s to 608 tests in
320.40 s after replacing fixed timer sleeps with a deterministic render-flush
33seam. Line coverage increased from 93.20% to 93.52%.
34
35### SyntaxHighlight cache scaling
36
37The same quality pass replaced the global four-state content-prefix heuristic
38with a bounded exact-render cache plus explicit `createSyntaxHighlight()`
39instances. The fixed workload below renders 24-line TypeScript snippets for 30
40frames after three warmup frames on the same Bun 1.3.13 process shape:
41
42| Static snippets per frame | Before    | After     |
43| ------------------------- | --------- | --------- |
44| 4                         | 0.0058 ms | 0.0067 ms |
45| 5                         | 1.7105 ms | 0.0082 ms |
46| 5 / 4 scaling ratio       | 297.3x    | 1.22x     |
47
48Crossing the old four-entry boundary made every lookup miss. At five snippets,
49the new cache removes 99.5% of that frame cost while scaling close to the ideal
501.25x increase in work. `bench:regression` now includes fixed-work 4-snippet,
515-snippet, and 32-isolated-instance cases; one recorded run measured medians of
523.40 µs, 4.17 µs, and 0.93 µs per frame respectively.
53
54### Variable-height VirtualList
55
56The first-class `VirtualList` regression case uses 5,000 keyed, variable-height
57mini-coder message rows at 120×40. It includes callable component rendering,
58visible-row measurement, prefix/window calculation, spacer construction, and
59layout. A same-process steady-state run on Bun 1.3.13 measured:
60
61| Case                                | Median        |
62| ----------------------------------- | ------------- |
63| Build + layout all 5,000 rows       | 134.39 ms     |
64| VirtualList + layout visible window | 2.170 ms      |
65| Relative reduction / speedup        | 98.4% / 61.9× |
66
67The virtual result retains cell-accurate measured heights for active rows and
68uses bounded estimates for cold rows; only the viewport plus configured
69overscan appears in the returned node tree. Both cases remain in
70`bench:regression` so future changes can be compared with the same fixed-work
71harness and raw samples.
72
73### Editing-key lookup
74
75The exploratory key benchmark exposed an immutable `Set` being rebuilt for
76every named key event. Hoisting that lookup removed the hot-path allocation on
77the same machine and similar measured clock (~1.7 GHz):
78
79| Case                     | Before  | After   | Change        |
80| ------------------------ | ------- | ------- | ------------- |
81| `isEditingKey("enter")`  | 1.39 µs | 3.94 ns | -99.7% / 353× |
82| `isEditingKey("ctrl+s")` | 1.26 µs | 6.03 ns | -99.5% / 209× |
83
84`bench:regression` now includes the combined named-key/shortcut lookup; one
85recorded fixed-work median was 11.09 ns for both calls together.
86
87**Machine:** 12th Gen Intel Core i9-12900, ~4.4 GHz  
88**Runtime:** Bun 1.3.9 (x64-linux)  
89**Date:** 2026-04-05 (updated from 2026-04-04 baseline)
90
91## Ink Comparison (caveat emptor)
92
93These numbers compare cel-tui's end-to-end pipeline against Ink using a
94visually similar tree (styled text, flexbox columns, word-wrap, nested
95containers with colors — the same shape as Ink's own `benchmark/simple`).
96
97**This is not an apples-to-apples comparison.** Ink is a React-based
98framework that does significantly more work per render: JSX → React
99reconciliation → Yoga layout (C++ via NAPI) → segment compositing →
100ANSI output. It supports React hooks, component lifecycle, concurrent
101mode, and a rich ecosystem of composable components. cel-tui is a
102minimal stateless framework that builds plain JS objects and writes
103cells into a flat buffer — it's a fundamentally different (and much
104simpler) architecture.
105
106The comparison is included as a rough reference point, not a claim of
107superiority. Ink solves a broader set of problems; cel-tui trades
108features for raw throughput.
109
110| What                                       | cel-tui    | Ink 6.8.0                                         | Ratio     |
111| ------------------------------------------ | ---------- | ------------------------------------------------- | --------- |
112| **Single render (comparable tree, 80×24)** | **65 µs**  | 632 µs (`renderToString`)                         | **~10×**  |
113| **1,000 re-renders**                       | **67 ms**  | 632 ms (`renderToString`) / 1,462 ms (`rerender`) | **9–22×** |
114| **Projected 100k re-renders**              | **~6.7 s** | ~63 s (`renderToString`) / ~146 s (`rerender`)    | **9–22×** |
115
116The Ink benchmark script we used is a custom `renderToString` loop (see
117[How to Reproduce](#how-to-reproduce)), not Ink's own shipped benchmark,
118which does 100k `rerender()` calls that also write to stdout. Both
119numbers are included above.
120
121### What each framework does per render
122
123| Step                 | cel-tui                          | Ink                             |
124| -------------------- | -------------------------------- | ------------------------------- |
125| Tree construction    | Plain JS objects (~0 cost)       | React.createElement + JSX       |
126| Layout               | Custom flexbox (TypeScript)      | Yoga (C++ via NAPI)             |
127| Painting             | Cell buffer (typed array writes) | Segment compositor (string ops) |
128| Output               | ANSI emitter (string concat)     | ANSI string builder (chalk)     |
129| State reconciliation | None (stateless)                 | React reconciler                |
130
131## cel-tui Pipeline Breakdown
132
133### visibleWidth (character measurement)
134
135| Input                        | Time     |
136| ---------------------------- | -------- |
137| ASCII short (5 chars)        | 4.1 ns   |
138| ASCII (64 chars)             | 39.1 ns  |
139| ASCII long (640 chars)       | 386.6 ns |
140| CJK + emoji (26 graphemes)   | 26.6 ns  |
141| Mixed ASCII + CJK (60 chars) | 4.9 ns   |
142| ANSI escape sequences        | 3.2 ns   |
143| Empty string                 | 0.1 ns   |
144
145The fast ASCII path dominates — pure ASCII strings bypass grapheme segmentation entirely.
146
147### Layout Engine
148
149| Tree                          | Time     |
150| ----------------------------- | -------- |
151| 10 children (flat)            | 1.5 µs   |
152| 100 children (flat)           | 12.0 µs  |
153| 1,000 children (flat)         | 125.1 µs |
154| depth=3 breadth=3 (40 nodes)  | 3.0 µs   |
155| depth=4 breadth=3 (121 nodes) | 15.1 µs  |
156| depth=3 breadth=5 (156 nodes) | 9.5 µs   |
157| 50 styled groups (200 nodes)  | 52.6 µs  |
158| 10 word-wrap paragraphs       | 2.6 µs   |
159| 50 word-wrap paragraphs       | 13.1 µs  |
160| Ink-comparable tree           | 1.7 µs   |
161| App tree (20 messages)        | 10.1 µs  |
162| App tree (100 messages)       | 39.6 µs  |
163
164Layout scales linearly with node count (~120 ns/node for flat trees).
165
166### Paint (cell buffer writing)
167
168| Scenario                      | Time   |
169| ----------------------------- | ------ |
170| 10 children (flat, 120×40)    | 93 µs  |
171| 100 children (flat, 120×40)   | 193 µs |
172| 1,000 children (flat, 120×40) | 192 µs |
173| Nested 121 nodes (120×40)     | 180 µs |
174| 50 styled groups (120×40)     | 207 µs |
175| 200 lines scrollable (120×40) | 185 µs |
176| 50 word-wrapped paragraphs    | 286 µs |
177| Ink-comparable (80×24)        | 45 µs  |
178| App tree 20 msgs (120×40)     | 208 µs |
179| 80×24 (small terminal)        | 109 µs |
180| 120×40 (medium terminal)      | 196 µs |
181| 200×50 (large terminal)       | 284 µs |
182
183Paint is O(cells), not O(nodes) — 100 vs 1,000 children produce similar times because only visible cells are painted. Cost scales with terminal size.
184
185### CellBuffer & ANSI Emission
186
187| Operation                    | Time                       |
188| ---------------------------- | -------------------------- |
189| Buffer creation 80×24        | 24.7 µs                    |
190| Buffer creation 120×40       | 58.2 µs                    |
191| Buffer creation 200×50       | 117.0 µs                   |
192| emitBuffer 80×24 (full)      | 21.4 µs                    |
193| emitBuffer 120×40 (full)     | 46.4 µs                    |
194| emitDiff 80×24 (no changes)  | 11.8 µs                    |
195| emitDiff 80×24 (partial)     | invalid historical fixture |
196| emitDiff 80×24 (full change) | invalid historical fixture |
197
198The original partial/full-change fixtures accidentally produced identical
199visible buffers, so their published timings did not measure the named cases.
200The fixtures now assert that exactly one or every cell changes, respectively;
201use `bun run benchmarks/cell-buffer.bench.ts` for corrected measurements.
202
203### Hit Test & Focus
204
205| Operation                    | Time    |
206| ---------------------------- | ------- |
207| hitTest flat 100, center     | 145 ns  |
208| hitTest nested 121, center   | 41 ns   |
209| hitTest app 50 msgs, center  | 108 ns  |
210| hitTest miss (out of bounds) | 7.7 ns  |
211| collectFocusable 100 buttons | 1.13 µs |
212| collectFocusable app tree    | 0.96 µs |
213
214### Key Parsing
215
216| Input                 | Time   |
217| --------------------- | ------ |
218| Printable char 'a'    | 15 ns  |
219| Ctrl+C (0x03)         | 18 ns  |
220| Escape                | ~1 ns  |
221| Arrow up (CSI)        | 58 ns  |
222| Shift+Tab (CSI Z)     | <1 ns  |
223| F5 (CSI 15~)          | 111 ns |
224| Backspace (0x7F)      | 7.1 ns |
225| normalizeKey "ctrl+s" | 84 ns  |
226
227### End-to-End Pipeline
228
229| Scenario                  | Time   | Renders/sec |
230| ------------------------- | ------ | ----------- |
231| Ink-comparable (80×24)    | 67 µs  | **14,900**  |
232| Flat 100 lines (80×24)    | 152 µs | 6,600       |
233| App tree 20 msgs (120×40) | 268 µs | 3,700       |
234| Re-render no-change diff  | 61 µs  | 16,300      |
235| Re-render +1 message diff | 248 µs | 4,030       |
236| 1,000 re-renders batch    | 67 ms  | 14,800      |
237
238## Real-World Synthetic App Shapes
239
240Focused run for the new `benchmarks/real-world.bench.ts` cases.
241These numbers use fixtures shaped after `mini-coder` and `mini-coder-next`:
242long unvirtualized conversations, syntax-highlight-like wrapping HStack token
243rows, and mini-coder-style virtualized scrollback slices.
244
245**Runtime:** Bun 1.3.13 (x64-linux)  
246**Date:** 2026-04-25
247
248### mini-coder-next-style unvirtualized conversation
249
250| Scenario                               | Time   |
251| -------------------------------------- | ------ |
252| Layout 100 messages (120×40)           | 71 ms  |
253| Layout 500 messages (120×40)           | 348 ms |
254| Layout 1,000 messages (120×40)         | 740 ms |
255| Paint pre-laid 500 messages (120×40)   | 4.0 ms |
256| Paint pre-laid 1,000 messages (120×40) | 6.5 ms |
257| Full render 250 messages (120×40)      | 172 ms |
258| Diff render 250 → 251 messages         | 179 ms |
259
260The new benchmark exposes the real bottleneck that the older app-tree cases
261missed: long unvirtualized conversation layout dominates render time. Paint is
262much smaller but still grows for token-heavy visible content.
263
264### Long wrapping HStack token lists
265
266| Scenario                     | Time   |
267| ---------------------------- | ------ |
268| Layout 100 tokens (120×40)   | 314 µs |
269| Layout 500 tokens (120×40)   | 1.6 ms |
270| Layout 1,000 tokens (120×40) | 3.2 ms |
271| Paint 100 tokens (120×40)    | 771 µs |
272| Paint 500 tokens (120×40)    | 3.5 ms |
273| Paint 1,000 tokens (120×40)  | 6.0 ms |
274
275These rows mimic `SyntaxHighlight` output: each logical line becomes an
276`HStack({ flexWrap: "wrap" })` with many styled `Text` spans.
277
278### mini-coder-style virtualized conversation
279
280| Scenario                                        | Time   |
281| ----------------------------------------------- | ------ |
282| Layout total 1,000 / visible 40 (120×40)        | 31 ms  |
283| Layout total 5,000 / visible 40 (120×40)        | 26 ms  |
284| Paint total 1,000 / visible 40 (120×40)         | 1.6 ms |
285| Paint total 5,000 / visible 40 (120×40)         | 1.6 ms |
286| Measure content height, 250 messages, width 118 | 56 ms  |
287
288The virtualized shape is mostly insensitive to total message count because the
289rendered node count is bounded by the visible slice plus spacers. The remaining
290cost is from the visible token-heavy message nodes and cold height measurement.
291
292## Ink Raw Numbers
293
294Ink v6.8.0, visually similar tree to cel-tui's "ink-comparable"
295scenario. Run with Bun 1.3.9 on the same machine. Note: the
296`renderToString` benchmark was written by us (not shipped by Ink) to
297isolate the render pipeline from terminal I/O.
298
299### `renderToString` (pure pipeline, no I/O)
300
301```
302Run 1: 1,000 renders: 632.75 ms — 632 µs/render
303Run 2: 1,000 renders: 632.66 ms — 633 µs/render
304Run 3: 1,000 renders: 630.99 ms — 631 µs/render
305```
306
307### `rerender` (full render with React reconciler + stdout)
308
309```
310Run 1: 1,000 re-renders: 1473.63 ms — 1,474 µs/render
311Run 2: 1,000 re-renders: 1461.48 ms — 1,461 µs/render
312Run 3: 1,000 re-renders: 1450.79 ms — 1,451 µs/render
313```
314
315## How to Reproduce
316
317```bash
318# cel-tui benchmarks
319cd cel-tui
320bun install
321bun run benchmarks/run-all.ts          # full suite
322bun run benchmarks/e2e.bench.ts        # just the headline numbers
323bun run benchmarks/real-world.bench.ts # real app-shaped stress cases
324
325# Ink benchmarks
326git clone https://github.com/vadimdemedes/ink.git /tmp/ink-bench
327cd /tmp/ink-bench
328npm install && npm run build
329
330# Create a renderToString benchmark (Ink ships rerender-to-stdout only)
331cat > benchmark/simple/bench-rts.tsx << 'BENCH'
332import React from "react";
333import { renderToString, Box, Text } from "../../src/index.js";
334
335function App() {
336  return (
337    <Box flexDirection="column" padding={1}>
338      <Text underline bold color="red">{"Hello World"}</Text>
339      <Box marginTop={1} width={60}>
340        <Text>
341          Cupcake ipsum dolor sit amet candy candy. Sesame snaps cookie
342          I love tootsie roll apple pie bonbon wafer. Caramels sesame
343          snaps icing cotton candy I love cookie sweet roll.
344        </Text>
345      </Box>
346      <Box marginTop={1} flexDirection="column">
347        <Text backgroundColor="white" color="black">Colors:</Text>
348        <Box flexDirection="column" paddingLeft={1}>
349          <Text>- <Text color="red">Red</Text></Text>
350          <Text>- <Text color="blue">Blue</Text></Text>
351          <Text>- <Text color="green">Green</Text></Text>
352        </Box>
353      </Box>
354    </Box>
355  );
356}
357
358renderToString(<App />, { columns: 80 }); // warmup
359const start = performance.now();
360for (let i = 0; i < 1_000; i++) renderToString(<App />, { columns: 80 });
361const elapsed = performance.now() - start;
362console.log(`1,000 renders: ${elapsed.toFixed(2)} ms (${(elapsed / 1000).toFixed(3)} ms/render)`);
363BENCH
364
365bun run benchmark/simple/bench-rts.tsx
366```