README.md
lextide
lextide = TypeScript-first, HAST-first, streaming-capable rewrite of lowlight.
Goal simple:
- drop-in replacement for
lowlight - return HAST, not HTML
- keep lowlight-style API
- add streaming
Status
Current built-in languages:
- bash
- javascript
- typescript
- python
- markdown
common and all same for now.
Install
bun add lextide
# or
npm install lextide
# or
yarn add lextide
# or
pnpm add lextide
Quick start
import { common, createLowlight } from "lextide";
const lowlight = createLowlight(common);
const tree = lowlight.highlight("javascript", "const answer = 42");
console.log(tree);
Output shape:
{
type: "root",
children: [
{
type: "element",
tagName: "span",
properties: { className: ["hljs-keyword"] },
children: [{ type: "text", value: "const" }]
},
{ type: "text", value: " answer = " },
{
type: "element",
tagName: "span",
properties: { className: ["hljs-number"] },
children: [{ type: "text", value: "42" }]
}
],
data: {
language: "javascript",
relevance: 3
}
}
Lowlight-compatible API
import { all, common, createLowlight } from "lextide";
Instance API:
highlight(language, value, options?)highlightAuto(value, options?)listLanguages()register(name, grammar)register(grammars)registerAlias(language, alias)registerAlias(aliases)registered(aliasOrName)stream(language, options?)
Built-ins
import { common, createLowlight } from "lextide";
const lowlight = createLowlight(common);
console.log(lowlight.listLanguages());
// ["bash", "javascript", "typescript", "python", "markdown"]
Useful built-in aliases:
- javascript:
js,jsx,mjs,cjs - typescript:
ts,tsx,mts,cts - python:
py - markdown:
md,mkdown,mkd - bash:
sh,shell,zsh,bash
highlight()
const tree = lowlight.highlight("python", "def greet():\n return 1\n");
Rules:
- throws on unknown language
- does best-effort highlight on malformed source
- sets
root.data.language - sets
root.data.relevance
Unknown language error:
lowlight.highlight("wat", "...");
// Error: Unknown language: `wat` is not registered
highlightAuto()
const tree = lowlight.highlightAuto("interface Person { name: string }");
Optional subset:
const tree = lowlight.highlightAuto(source, {
subset: ["javascript", "typescript"],
});
Rules:
- checks registered languages, or
subsetwhen given - returns best match by relevance
- returns empty root when nothing useful matches
Options
interface Options {
prefix?: string | null | undefined;
}
Prefix behavior:
- omitted or
undefined->hljs- null-> no prefix- string -> custom prefix
Example:
lowlight.highlight("javascript", "const x = 1", { prefix: "code-" });
Streaming
Streaming API give incremental updates.
const stream = lowlight.stream("typescript");
let children = [];
children.push(...stream.write("interface Person {\n").stable);
children.push(...stream.write(" name: string\n").stable);
children.push(...stream.end("}\n").stable);
console.log(stream.snapshot());
API:
interface StreamOptions extends Options {
allowRecalls?: boolean | null | undefined;
}
interface StreamUpdate {
recall: number;
stable: ElementContent[];
unstable: ElementContent[];
}
interface HighlightStream {
write(chunk: string): StreamUpdate;
end(chunk?: string): StreamUpdate;
snapshot(): Root;
}
Stable mode
Default mode conservative:
- only stable nodes go to
stable unstableempty- final
end()output matches synchighlight()output
Recall mode
const stream = lowlight.stream("javascript", { allowRecalls: true });
Now each update may include unstable tail:
recall= how many previously rendered unstable root children to remove- append
stable - append
unstable
Consumer algorithm:
const rendered: ElementContent[] = [];
function apply(update: StreamUpdate) {
if (update.recall > 0) {
rendered.splice(rendered.length - update.recall, update.recall);
}
rendered.push(...update.stable, ...update.unstable);
}
HAST contract
lextide returns HAST-like roots:
interface Root {
type: "root";
children: ElementContent[];
data?: {
language?: string;
relevance?: number;
};
}
Node rules:
- plain text =
{type: "text", value} - scopes =
spanelements - classes live in
properties.className - adjacent text nodes merged
- source text preserved exactly
Class serialization matches lowlight/highlight.js style:
- first scope segment gets prefix
- later dotted segments get underscore suffixes
Example:
- scope:
title.function - classes:
['hljs-title', 'function_']
Custom grammars
You can register custom grammars.
import { createLowlight, type LanguageFn } from "lextide";
const toy: LanguageFn = () => ({
name: "toy",
relevance: 1,
startState: "root",
states: {
root: [
{ match: /foo/, scope: "keyword", relevance: 2 },
{ match: /"[^"]*"/, scope: "string", relevance: 1 },
],
},
});
const lowlight = createLowlight();
lowlight.register("toy", toy);
lowlight.registerAlias("toy", ["toybox", "toy-script"]);
console.log(lowlight.highlight("toybox", 'foo "bar"'));
Public grammar types:
interface GrammarRule {
match: RegExp | ((input: string, pos: number) => number | null);
scope?: string;
push?: string;
pop?: boolean;
children?: GrammarRule[];
relevance?: number;
}
interface GrammarDefinition {
name: string;
aliases?: string[];
relevance?: number;
states: Record<string, GrammarRule[]>;
startState: string;
}
type LanguageFn = () => GrammarDefinition;
Render yourself
lextide no render HTML.
It return HAST.
Use whatever renderer fit app.
Examples:
- rehype / unified pipeline
- custom React renderer
- custom HTML serializer
Why lextide
- lowlight-style API
- HAST output only
- no HTML intermediate step
- streaming support
- strict TypeScript
- small built-in language set, focused on v0 quality
Current scope and caveats
Current project target = solid v0 for builtin languages, not full highlight.js language coverage.
Today:
- builtins only: bash / javascript / typescript / python / markdown
commonandallsame- no HTML serializer included
- grammar parity work still ongoing for builtin languages
Dev
bun run build
bun run format
bun run lint
bun run typecheck
bun run test
bun run check
bun run pack:dry
Release
Local release flow:
git fetch origin
git pull --ff-only origin main
bun run check
bun run build
bun run pack:dry
git tag -a v<version> -m "v<version>"
git push origin main
git push origin v<version>
bun publish --access public
Replace <version> with the value in package.json.