Building the Website and the Playground

I needed both a website and a playground, and I needed the code blocks on the website to actually highlight Hew — not just dump monochrome text and hope people squint past it.

Astro and Tailwind

hew.sh runs on Astro with Tailwind CSS, deployed to Cloudflare Pages. The choice was boring on purpose. Astro renders static HTML, ships zero JavaScript by default, and lets me drop interactive islands where I need them (the playground page, the grammar explorer) without hydrating the entire site. Tailwind handles the styling.

The interesting part was Shiki integration. Shiki is the syntax highlighter Astro uses for code blocks in Markdown, and it consumes TextMate grammars — the same .tmLanguage.json format VS Code uses. So I loaded Hew’s TextMate grammar directly in astro.config.mjs:

hewGrammar = JSON.parse(
  readFileSync(resolve(__dirname, 'src/lib/syntax/hew.tmLanguage.json'), 'utf-8')
);

// ...
langs: [
  {
    id: 'hew',
    scopeName: 'source.hew',
    aliases: ['Hew'],
    ...hewGrammar
  },
]

Every code block tagged with ```hew in a blog post gets full syntax highlighting — keywords, types, operators, string interpolation, doc comments — using the exact same grammar that drives VS Code. One grammar file, copied from the vscode-hew repo and used in three places. The Tailwind base color collision from Part 10 was the worst bug I hit during the website build, and it was a five-character rename.

Monaco and the Monarch tokenizer

Static syntax highlighting covers blog posts and docs. But the playground needs an editor, and the editor needs live tokenization — not just coloring a static block, but re-tokenizing on every keystroke. Monaco is the obvious choice (it’s VS Code’s editor, running in the browser), but Monaco doesn’t use TextMate grammars. It uses its own tokenizer format called Monarch.

So there are two highlighting pipelines: TextMate for Shiki (static) and Monarch for Monaco (interactive). The Monarch tokenizer lives in src/lib/monaco/hew-language.ts — about 220 lines defining keywords, types, operators, and a state machine for strings, comments, f-string interpolation, and regex literals. The keyword and type lists come from the compiler’s syntax-data.json, regenerated by a script:

node tools/generate-monarch.mjs ~/projects/hew-lang/hew/docs/syntax-data.json

That script reads every keyword, type, and operator the lexer knows about and patches them into the Monarch definition. When a keyword gets added to the compiler, one command updates the website’s editor. When a keyword gets removed — and several have, across the spec revisions — same thing. No manual list maintenance, no drift between what the compiler accepts and what the editor highlights.

The Monarch tokenizer handles constructs that don’t exist in most languages. receive fn, f-strings with f"hello {name}", regex literals with re"pattern", duration suffixes, attribute annotations with #[...]. Each one is a tokenizer state transition. The f-string handling was the trickiest — entering { inside an f-string pushes a new state that tokenizes the expression as code, then } pops back to string mode. Getting nested braces right took three attempts.

Compiling the frontend to WASM

The playground has two halves: diagnostics and execution. Execution hits a server — the compiled program runs inside a gVisor sandbox (Google’s container isolation runtime), isolated in a container that gets torn down after a timeout. But diagnostics run entirely in the browser.

The Hew compiler’s Rust frontend — lexer, parser, type checker — compiles to WebAssembly via wasm-bindgen (the Rust-to-JS bridge generator). The WASM module exposes three functions: analyze(source) returns diagnostics as JSON, get_keywords() returns the keyword list for editor completion, and hover(source, offset) returns type information at a cursor position. The diagnostics integration is straightforward — a debounced content-change listener calls analyze(), converts byte offsets to line/column positions, and pushes Monaco markers:

editor.onDidChangeModelContent(() => {
  if (timer) clearTimeout(timer);
  timer = setTimeout(() => runDiagnostics(monaco, editor), DEBOUNCE_MS);
});

300 milliseconds after you stop typing, the entire Rust frontend runs against your code — in the browser, no server round-trip — and red squiggles appear under errors. The same lexer, parser, and type checker that produce native binaries through the MLIR pipeline, running as WASM to check your code while you write it.

The WASM build itself was not smooth. The wasm-bindgen output includes a workaround for a Safari TextDecoder bug — Safari’s decoder accumulates internal state and starts returning garbage after processing roughly 2 GB of text. The generated JS tracks total bytes decoded and recreates the decoder when it approaches the limit. I didn’t write that fix. wasm-bindgen generates it. I spent an hour debugging garbled diagnostics output in Safari before discovering the workaround existed and I just needed to update wasm-bindgen. (The generated code was already smarter than me about the bug I was trying to fix.)

The .wasm binary sits at public/wasm/hew_wasm_bg.wasm and gets fetched on first load of the playground page. It’s about 2 MB — the entire Rust frontend, compiled. Every time the compiler’s frontend changes, I rebuild the WASM module and commit the new binary. The commit messages read like a changelog: “rebuild WASM with literal coercion and cast support,” “rebuild WASM diagnostics from Rust frontend v0.1.7.” Not elegant, but it works — the website always matches the compiler.

hew.run

The standalone playground at hew.run is a SvelteKit app. It’s a separate repo because it started as a prototype before the website existed, and by the time hew.sh had its own playground page, hew.run had its own URL and its own users. Both share the same Monaco configuration — the Monarch tokenizer, the dark and light themes, the WASM diagnostics module. The SvelteKit version has a slightly different layout (full-screen editor, output panel below) but the same engine underneath.

The keyword and type arrays in hew.run are copied from hew.sh manually. There’s no generator script for it — just copy the arrays when they change. It’s on the list of things to automate. (It has been on the list for three weeks.)

The blog itself

The TextMate grammar that highlights the code blocks in this post is the same grammar I wrote about creating in Part 10. The Monaco editor in the playground is the same one I use to test the code examples. In practice, when I break the grammar, I break both the editor and the blog at the same time.