chore: Added syntax highlighting to docs site.

A can of worms has been opened.

ASSET_PIPELINE_PLAN.md

@@ -0,0 +1,135 @@
+# Asset Pipeline Plan
+
+Design for letting thor users bring their own CSS toolchain (Sass, PostCSS,
+Lightning CSS, Tailwind, …).
+
+## Thesis
+
+Let thor users bring their own CSS toolchain by **orchestrating the tool they
+already installed** — never bundling or reimplementing one. This is expressed
+through thor's existing mustache pipe grammar with two new ops: `tool` and
+`fingerprint`.
+
+The earlier research design (content-addressed registry + deferred token
+resolution + a separate post-render path for "whole-output scanners") is
+**discarded**. It was overbuilt. Every tool — Tailwind included — scans
+*source*, which exists at render time, so every transform runs **eagerly,
+synchronously, and memoized**. No placeholder tokens, no post-render
+string-replace pass, no second integration surface.
+
+## The two ops
+
+```html
+<link href="{{ "css/main.scss" | tool sass | fingerprint | rel_url }}">
+```
+
+```json5
+// thor.json
+{
+  "tools": {
+    "sass":         "sass $in $out",
+    "postcss":      "postcss $in -o $out",
+    "lightningcss": "lightningcss --minify $in -o $out",
+    "tailwind":     "tailwindcss -i $in -o $out"
+  }
+}
+```
+
+## Core model
+
+The value flowing through the pipe is **bytes held by thor**, with a logical
+name attached (`css/main.css`). Each stage transforms the bytes; the **final
+stage publishes** one file to the output dir and returns its URL.
+
+- A literal (`"css/main.scss"`) or context key names the **source** file on disk.
+- `| tool <name>` runs a subprocess, replacing the carried bytes with its output.
+- `| fingerprint` writes the content-addressed file and returns its hashed URL.
+- The chain composes with the existing `rel_url` for base_url subpaths.
+
+**Memoization:** results are cached per build, keyed by (op-chain, source). The
+first page to reference an asset runs the tool; every other page hits the cache.
+This is what stops N pages from shelling out N times.
+
+## `| tool <name>`
+
+Looks up `<name>` in the `thor.json` `tools` table and runs its command
+template. `$in` and `$out` are **optional per-tool I/O adapters**:
+
+| Template has | Behavior |
+|---|---|
+| `$in` | thor writes current bytes to a scratch path, substitutes it |
+| no `$in` | thor pipes current bytes to the tool's **stdin** |
+| `$out` | thor substitutes a scratch path, reads it back after |
+| no `$out` | thor captures the tool's **stdout** |
+
+Defaulting to stdin/stdout avoids scratch-file overhead when the tool supports
+it. Tools that can't stream (Lightning CSS wants a path; Tailwind v3 needs an
+input file) just get `$in`/`$out` in their template. Either way the bytes are
+thor's, so chaining works regardless of each tool's preference:
+`| tool sass | tool postcss` composes even if sass streams and postcss wants
+files.
+
+- **`$out` is always thor-controlled scratch** — the tool never picks the
+  published filename.
+- **Extension derivation:** thor owns the output name; `main.scss` through
+  `tool sass` becomes `main.css`.
+- **Errors:** thor gates on exit code and surfaces the tool's stderr through one
+  code path. If the tool is missing from PATH, fail with its stderr — never
+  bundle, never assume a runtime.
+- **A bare `tool` (no `fingerprint`) still publishes** — to the logical unhashed
+  path (`output/css/main.css`), returning `/css/main.css`.
+
+## `| fingerprint`
+
+Content-addresses whatever bytes it currently holds. It has **two outputs**:
+
+- **Side effect:** writes `css/main.<hash>.css` to the output dir.
+- **Return value:** the hashed URL string `/css/main.<hash>.css` (feeds
+  `rel_url`).
+
+Properties:
+
+- **Works standalone**, no `tool` needed: `{{ "css/main.css" | fingerprint }}`
+  is plain cache-busting of a hand-written file — probably the most common use.
+- **Position decides what's hashed** — it hashes current bytes, so it belongs
+  **last** in the chain. No terminal-only special-casing; the rule is just "put
+  it last."
+- **Hash format:** `name.<hash>.ext` (CDN-friendly), 8 hex chars of SHA-256.
+
+## Tailwind is not special
+
+It scans your **source** (`layouts/`, `content/`, partials) — which exists at
+render time — so it runs synchronously like sass. Its `thor.json` entry just
+needs content globs pointing at the source tree (or v4 auto-detection). The
+**one** asymmetry is watch invalidation: sass re-runs when `main.scss` changes;
+Tailwind must re-run when *any* scanned source changes. That's a cache-key
+breadth difference, not an architectural one. The cost of scanning source vs.
+output is losing dynamically-assembled class tokens (`col-{{n}}`) — a Tailwind
+anti-pattern whose sanctioned fix is a safelist, not warping the build.
+
+## Interaction: `copy_assets_dir`
+
+Today thor copies everything under `assets/` to the output dir. Any source
+consumed by a pipe must be **excluded from that plain copy** — otherwise you'd
+ship the raw `main.scss` (or a dead unhashed `main.css` alongside the hashed
+one). The pipe owns the write for assets it touches.
+
+## Implementation steps (separable)
+
+1. **`| fingerprint`** — no subprocess, no config. Hash + write + return URL,
+   plus the `copy_assets_dir` exclusion. Smallest blast radius, independently
+   useful, ships first.
+2. **`| tool <name>`** — `thor.json` tools table, the `$in`/`$out`/stdin/stdout
+   adapter, subprocess + exit-code/stderr handling, extension derivation,
+   memoization. Composes with step 1.
+
+## Open questions for implementation
+
+- **Watch invalidation:** simplest correct v1 recomputes the memo each build
+  (cheap for the common single-asset case). A cross-build skip keyed on source
+  content/mtime is a later optimization — and Tailwind's broad content-glob key
+  is where it matters most.
+- **Source-map sidecars:** sass/lightning emit `.map` files next to `$out`;
+  disable in the template or ignore the sidecar. Minor.
+- **Error policy default:** hard-fail the build vs. warn-and-skip on tool
+  failure/absence. Lean fail; possibly configurable later.

ASSET_PIPELINE_RESEARCH.md

@@ -1,298 +0,0 @@
-# Asset Pipeline Research: Bring-Your-Own CSS Preprocessor
-
-Research and design notes for letting thor users bring their own CSS toolchain
-(Sass, PostCSS/Tailwind, Less, Lightning CSS, …) instead of thor embedding or
-tracking any of them.
-
-## Problem statement
-
-The CSS pre-processor landscape is large and moving (Tailwind, PostCSS, Less,
-Sass, Lightning CSS, …). thor is a ~1.9 MB self-contained binary that tracks its
-own size (see `AUDITING_BINARY_SIZE.md`); bundling any one preprocessor — let
-alone keeping up with all of them — is a non-starter on both size and
-maintenance grounds. So "bring your own" has to mean **thor orchestrates the
-tool the user already installed**, never reimplements it.
-
-Today `copy_assets_dir` (`assets.odin`) copies everything under `assets/` to the
-output dir and optionally minifies `.css` via the tree-sitter pass in
-`minify.odin`. There is no preprocessing seam.
-
-## What we are NOT doing, and why
-
-- **Bundle a specific preprocessor** — rejected: binary-size-hostile, and picks
-  winners in a space we said we can't keep up with.
-- **Hugo Pipes-style template resource pipeline** (`resources.Get | css.Sass |
-  minify | fingerprint`) — powerful, but needs a resource abstraction and a
-  non-logic-less template language to host the chained syntax. thor's engine is
-  logic-less mustache; wrong fit.
-- **Curated shell-out for named tools only** (what Hugo actually ships — only
-  `css.Sass` and `css.PostCSS` shell out) — this just re-picks winners.
-
-The chosen direction reuses thor's **existing mustache pipe grammar** (the same
-mechanism as the `rel_url` pipe) to express asset transforms declaratively,
-where the asset is referenced.
-
-## Correcting the record on Hugo
-
-An earlier draft attributed a generic "pre/post-build shell hook" model to Hugo.
-That is wrong. Hugo has **no** generic build hooks. Its CSS handling is Hugo
-Pipes — a *template-invoked resource pipeline* of fixed, named functions
-(`css.Sass`, `css.PostCSS`, `minify`, `fingerprint`, `resources.Concat`, …).
-Only two of those shell out: `css.PostCSS` (to a local `postcss-cli` under Node)
-and `css.Sass` (embedded LibSass, or a `dart-sass` binary). Generic
-"run any command pre/post build" is closer to npm-scripts / Makefile glue, or
-Eleventy's `before`/`after` events (which run JavaScript, not shell commands).
-No mainstream SSG markets a generic command hook as its asset story.
-
----
-
-## Tool CLI research
-
-Five representative tools were researched for their exact shell-out contract.
-The questions that determine thor's interface are the same for each: stdin/stdout
-vs. file-path args, whether a config file is mandatory, whether it needs to scan
-rendered HTML, watch support, minify overlap, and runtime dependencies.
-
-### Mechanical matrix
-
-| Tool | stdin→stdout | Config file | Self-contained (no Node) | Self-minifies | Errors |
-|---|---|---|---|---|---|
-| **Dart Sass** | yes (`sass -`) | none | yes — standalone binary | `--style=compressed` | stderr, exit 65 |
-| **PostCSS** (`postcss-cli`) | yes (`< in > out`) | optional (`--use`) | **no** — needs Node + node_modules | via cssnano plugin | stderr, exit 1 |
-| **lessc** | yes (`-` / positional) | none | **no** — needs Node | weak/deprecated (`-x`) | stderr, nonzero |
-| **Lightning CSS** | file-in, stdout-out | none (flags) | yes — Rust binary (npm-distributed) | `--minify` (its core purpose) | stderr, nonzero |
-| **Tailwind v4** | yes (`-i - -o -`) | none (CSS-first) | yes — standalone binary (Bun) | `--minify` | stderr, nonzero |
-| **Tailwind v3** | no (needs `-i` file) | `tailwind.config.js` | yes — standalone (pkg/Node embed) | `--minify` | stderr, nonzero |
-
-Two properties are uniform and load-bearing:
-
-- **Every tool writes errors to stderr and returns nonzero on failure.** thor can
-  gate on exit code and surface stderr through a single code path for all tools.
-- **Every tool can stream to stdout.** Differences (Lightning CSS wants a path
-  not stdin; lessc's output is positional; Tailwind v3 needs an input file) are
-  all absorbable by a per-tool command template with placeholders.
-
-### Per-tool detail
-
-**Dart Sass** — `sass input.scss output.css`, or `sass -` for stdin, prints to
-stdout when no output arg. No config file exists or is supported; configured by
-CLI flags + `@use`/`@import` in source. Load paths via `--load-path`/`-I`.
-`--style=compressed` minifies (overlaps thor's minifier). Source maps on by
-default (`--no-source-map` to disable). Exit 64 = usage error, 65 = compile
-error. **Distribution:** standalone release bundles the Dart VM + snapshot, no
-Node; the npm `sass` package is dart2js and needs Node. Prefer the standalone.
-
-**PostCSS** (`postcss-cli`, binary `postcss`) — positional input files or stdin;
-stdout is the default when no `-o`/`--dir`/`--replace`. Config (`postcss.config.js`
-/ `.postcssrc`, cosmiconfig, walks up) is **not required** — plugins can be given
-with `-u/--use` (e.g. `postcss in.css -u autoprefixer -u cssnano -o out.css`).
-Zero plugins = near pass-through. **postcss is only a plugin host** — autoprefixer,
-tailwind, cssnano are separate npm packages resolved from `node_modules`.
-`-w/--watch` tracks the full `@import` dependency graph via PostCSS `dependency`
-messages. Exit 1 on error, errors to stderr, stdout stays clean for piping.
-**Hard Node + node_modules dependency.** Invoke via `npx postcss` to resolve
-local `.bin`.
-
-**lessc** — `lessc [opts] <source> [destination]`; `-` reads stdin; stdout when
-destination omitted (**no `-o` flag** — output is positional). No config file;
-`.less` source + flags. `--include-path` for import search. `-x`/`--compress` is
-deprecated/weak — let thor's minifier compress. **No CLI watch mode** (watch only
-exists in the browser runtime). Errors to stderr, nonzero on compile error.
-**Hard Node dependency** (`bin/lessc` is a Node script).
-
-**Lightning CSS** (`lightningcss`) — input file positional, `-o`/`--output` or
-stdout when omitted. No config, flag-driven. `--minify` is the tool's core
-purpose (**directly overlaps thor's minifier** — pick one). `--bundle` inlines
-`@import`s; `--targets "<browserslist>"` downlevels modern CSS. No built-in watch.
-Errors to stderr, nonzero. **Runtime: self-contained Rust binary, no Node to
-run** — but distribution is npm-only (`lightningcss-cli` pulls a platform package
-like `lightningcss-cli-linux-x64-gnu`); no official GitHub/Homebrew/Winget
-binaries. Can extract the binary from the npm registry tarball and run it
-standalone.
-
-**Tailwind CSS standalone CLI** (v3 and v4) — both ship a self-contained
-per-platform binary (`./tailwindcss`), no Node required (v3 via Vercel pkg, v4
-via Bun). `-i`/`--input`, `-o`/`--output` (defaults to stdout in v4). v4 reads
-stdin with `-i -`; v3 effectively needs an input file. Input is a CSS entry file:
-v3 uses `@tailwind base/components/utilities;`, v4 uses `@import "tailwindcss";`.
-**Config:** v3 uses `tailwind.config.js` (auto-discovered, `-c` to override) and
-in practice needs it (or `--content`) to know what to scan; v4 is CSS-first and
-genuinely zero-config. `--minify` in both. Errors to stderr, nonzero (codes not
-formally enumerated — check `exit != 0`).
-
-**Content scanning (the critical Tailwind property):** Tailwind scans
-source/template files **as plain text** for complete, static class-name tokens.
-It does not parse code and does not strictly require rendered HTML — it scans
-whatever files you point it at. But for an SSG the safe strategy is to scan the
-**generated output** after render, so classes that only appear post-render aren't
-missed. v3: `content: ["./public/**/*.html"]` or `--content`. v4: automatic
-detection, but it **ignores `.gitignore`d paths** — so a git-ignored `public/`
-must be added explicitly with `@source "../public"`.
-
----
-
-## Design conclusions
-
-### The one split that matters: two tool shapes
-
-The research draws a hard line between two categories that need **different
-integration surfaces**:
-
-1. **Per-asset transformers** — Sass, Less, Lightning CSS, PostCSS-as-compiler.
-   Input = one source file → output = one CSS file. These fit a mustache pipe:
-   `{{ "css/main.scss" | tool sass }}`. Each tool's CLI quirk (stdin vs. path,
-   positional output, etc.) is absorbed by a per-tool command template.
-
-2. **Whole-output scanners** — Tailwind, alone. Its "input" is not one asset; it
-   is an entry CSS file **plus every rendered HTML file it scans**. At the moment
-   a per-page pipe would fire, the full output does not exist yet. Tailwind must
-   run **once, after all HTML is written, pointed at `output_dir`**. That is a
-   post-render step, categorically not a pipe.
-
-### The chicken-and-egg, and the fix
-
-A `<link>` wants a **fingerprinted (cache-busted) URL**, but for Tailwind the CSS
-isn't produced until **after** render — so at render time the template cannot
-know the hash. Hugo dodges this with a lazy multi-pass resource graph; thor is
-single-pass.
-
-**Resolution — emit a placeholder at render, resolve at the end.** A pipe like
-`{{ "css/main.scss" | tool sass | fingerprint }}` writes a **stable token** (e.g.
-`⟦asset:css/main.scss⟧`) into the HTML. After render, thor runs the tools,
-populates a **content-addressed asset registry** (logical name → final hashed
-URL), then does one cheap string-replace pass over the output swapping tokens →
-real URLs.
-
-This single mechanism dissolves every constraint encountered:
-
-- **Cache-busting:** the hash is known at resolve time, so the token becomes
-  `/css/main.a1b2c3.css` uniformly, even for tools that rename output themselves.
-- **Hash-after-minify:** resolution is the last step, so the hash is always of the
-  exact final served bytes.
-- **Same asset → same hash across N pages:** the registry is computed once; every
-  token resolves identically. (This is also the memoization that keeps a
-  per-page pipe from shelling out N times for the same file.)
-- **Tailwind:** its post-render output lands in the registry like any other asset;
-  tokens that referenced it resolve correctly despite the CSS not existing at
-  render time.
-
-### Cache-busting: two regimes
-
-- **Regime A — thor owns the hash (recommended, and what every researched
-  compiler needs).** Tool → stdout → thor minifies → thor hashes the final bytes →
-  writes `main.<hash>.css` → registers the URL. None of Sass/PostCSS/Less/
-  Lightning CSS/Tailwind fingerprint on their own; fingerprinting is a
-  bundler/plugin behavior. thor stays in full control of the name because it
-  writes the file. Mirrors Hugo's `| fingerprint` → `.RelPermalink`.
-- **Regime B — the tool owns the hash** (bundlers: esbuild/vite/parcel, or
-  PostCSS + `postcss-hash`). The tool renames output to a name thor can't predict,
-  and stdout capture doesn't reveal it. The standard answer these tools already
-  provide is a **manifest** (`manifest.json` mapping `"main.css"` →
-  `"main.a1b2c3.css"`). Supporting this class means thor ingests that manifest
-  into the same registry rather than scanning for files.
-
-Both regimes converge on the same abstraction: **a build-scoped, content-addressed
-asset registry** (logical name → final emitted URL), populated by thor's own
-transform+fingerprint or by an external tool's manifest, and resolved through the
-placeholder pass.
-
-## Proposed interface shape
-
-- **`thor.json` `tools` table** — one command template per tool. Keeps shell
-  commands out of templates (a single trust surface), and absorbs each CLI's
-  quirks. `$in` = source path; stdout is captured:
-
-  ```json
-  "tools": {
-    "sass":         "sass --no-source-map $in",
-    "postcss":      "postcss -u autoprefixer $in",
-    "lightningcss": "lightningcss --minify $in",
-    "tailwind":     { "run": "tailwindcss -i $in -o -", "scan": "output" }
-  }
-  ```
-
-  The `scan: output` marker flips a tool from per-asset (pipe) to post-render
-  (whole-output) mode.
-
-- **Pipe ops** — `tool <name>` (transform → asset handle), `minify`,
-  `fingerprint` — all resolving through the registry, chainable with the existing
-  `rel_url`:
-
-  ```
-  <link href="{{ "css/main.scss" | tool sass | minify | fingerprint | rel_url }}">
-  ```
-
-  A lookup-direction op resolves a logical name to its final URL:
-  `{{ "css/main.css" | asset }}` → `/css/main.a1b2c3.css`.
-
-- **Final resolve pass** over `output_dir` replacing asset tokens with registry
-  URLs. This is also where post-render scanners (Tailwind) have already run.
-
-### Fit with the existing mustache pipe grammar
-
-`{{ "css/main.css" | tool postcss }}` already parses under `mustache/pipes.odin`:
-the key is a quoted literal (`tokenize_fields` keeps the quotes so a literal is
-distinguishable from a context key), `tool` is the op, `postcss` is one arg
-(under `MAX_PIPE_ARGS = 2`). Adding an op is the documented one-liner path — a
-variant on `Pipe_Op` plus a `case` in `apply_filter`. The only architectural
-departure from existing ops (all pure value→value string transforms) is that
-`tool`/`fingerprint` have side effects (subprocess, file write) and must be
-backed by the registry — which is why the placeholder + deferred-resolve model
-matters: it keeps render itself side-effect-light and pushes the real work to a
-defined post-render phase.
-
-## Cross-cutting decisions
-
-1. **Node dependency is not thor's problem.** PostCSS and lessc require Node +
-   node_modules; Sass/Lightning CSS/Tailwind ship self-contained binaries. thor's
-   job is to exec whatever is on PATH and **fail gracefully with the tool's
-   stderr** if it's absent — never bundle, never assume a runtime. The Nix flake
-   makes pinning the toolchain pleasant for users who want reproducibility.
-2. **Minification must be explicit, never doubled.** Every one of these tools can
-   self-minify, and thor has its own minifier. Make `minify` an explicit pipe
-   step and do **not** auto-apply thor's global minify to tool-produced assets —
-   otherwise `| tool lightningcss` (already minified) gets re-minified. The pipe
-   chain is the single source of truth for whether/when an asset is minified.
-3. **thor drives the watch; ignore the tools' own `--watch`.** thor already polls
-   (5 s). Invoke each tool once per build as a pipeline step. Sass/PostCSS/
-   Tailwind have watch modes, but running them would put a second watcher in
-   competition with thor's loop — avoid it.
-
-## Headline
-
-The feature is not "a `tool` pipe." It is a **content-addressed asset registry
-with deferred token resolution**, exposed through two pipe ops (`tool`,
-`fingerprint`/`asset`) plus one post-render hook (for whole-output scanners like
-Tailwind). That is the minimum that handles both tool shapes *and* cache-busting
-without introducing a multi-pass render engine.
-
-## Open questions for the design phase
-
-- Placeholder token format and collision-safety (must survive HTML minification
-  untouched — likely treat like the `pre`/`code` preserve ranges in `minify.odin`).
-- Registry lifetime and keying (content hash vs. path+mtime) and how it threads
-  through the watch loop for incremental rebuilds.
-- Whether `fingerprint` is terminal-only, or the resolve pass always hashes
-  post-minify regardless of chain position.
-- Manifest ingestion format for Regime B (adopt Vite/webpack `manifest.json`
-  shape?).
-- Error policy: hard-fail the build vs. warn-and-skip when a tool exits nonzero
-  or is missing from PATH (probably configurable, defaulting to fail).
-
-## Sources
-
-- Tailwind CLI (v4): https://tailwindcss.com/docs/installation/tailwind-cli ·
-  content detection: https://tailwindcss.com/docs/detecting-classes-in-source-files ·
-  standalone: https://tailwindcss.com/blog/standalone-cli ·
-  v3: https://v3.tailwindcss.com/docs/installation ·
-  https://v3.tailwindcss.com/docs/content-configuration
-- PostCSS CLI: https://github.com/postcss/postcss-cli (README + `index.js` +
-  `lib/args.js`)
-- Dart Sass CLI: https://sass-lang.com/documentation/cli/dart-sass/ ·
-  https://github.com/sass/dart-sass
-- Less: https://lesscss.org/usage/ ·
-  https://github.com/less/less-docs/blob/master/content/usage/command-line-usage.md
-- Lightning CSS: https://lightningcss.dev/docs.html ·
-  https://github.com/parcel-bundler/lightningcss ·
-  https://www.npmjs.com/package/lightningcss-cli ·
-  distribution discussion: https://github.com/parcel-bundler/lightningcss/discussions/732

GRAMMAR_CACHE_THREADING.md

@@ -1,9 +1,32 @@
 # Grammar cache thread-safety & allocator lifetime
 
-Status: **investigation complete, fix deferred.** This document records what we
+Status: **resolved.** All three issues below are fixed. The cache was redesigned
-found and the direction we chose, so the next person doesn't have to rediscover
+so the library caches only the immutable, shareable objects and callers own the
-it. No behavior has been changed yet — only documentation comments were added to
+per-thread ones. The rest of this document is kept as the rationale.
-`treesitter/treesitter.odin`.
+
+## Resolution (what shipped)
+
+The `Grammar_Cache` (which mixed shareable `language`/`query` with per-thread
+`parser`/`cursor`) was split:
+
+- **`Grammar`** holds only the immutable `language` + `query`. It lives in a
+  process-lifetime **`registry`** (`map[string]^Grammar`) guarded by one mutex.
+- **`grammar(lang)`** is the thread-safe, self-initializing, lazy loader — fixes
+  **#1a** (all map access under the lock) and lets tests run with no setup.
+- The registry allocator is pinned with `os.heap_allocator()` — fixes **#2**; the
+  cache no longer captures a transient `context.allocator`.
+- Parsers/cursors are **caller-owned**: `open_parser(g)` mints one, the caller
+  `defer`s `parser_delete` / `query_cursor_delete`. No parser is shared, so
+  **#1b** is gone. `open_parser` is the seam where a future pool / thread-local
+  strategy plugs in without touching call sites.
+- Deleted: `ensure_parser`, `load_grammar`, `preload_grammar`, `grammar_worker`,
+  `cache_mutex`; `preload_grammars` is now a serial registry warm-up. The minify
+  tests dropped their `parse_mu` serialization and preload scaffolding entirely.
+
+The **later** decision — pool vs per-thread parser reuse for genuine parallel
+parsing — is still open, but intentionally so: it only matters once we parse in
+parallel, and it now changes only `open_parser`, not the rest of the library.
+The original investigation follows.
 
 ## How we got here
 

TODOS.md

@@ -2,16 +2,32 @@
 
 ## Priority Unclear (Evaluate Later)
 
-- [ ] Decide whether the grammar cache should support multi-threaded parsing.
+- [ ] Decide the parallel-parsing strategy for the grammar cache (pool vs
-  - For now it stays a single global cache used single-threaded; the plan is to
+      thread-local parsers) — deferred until the parallel usage shape is clear.
-    guard the non-thread-safe parser/cursor use with a mutex when tests (or any
+  - Resolved already: the registry (`grammar()`) is thread-safe and self-
-    future parallel caller) need it. A `TSParser`/`TSQueryCursor` is one-per-thread.
+    initializing, its allocator is pinned to the heap, and parsers/cursors are now
-  - Alternative later: give each thread its own parser+cursor for real parallelism.
+    caller-owned (`open_parser`) instead of cached — so the library holds no
-  - Also pin `init_persistent` to the heap allocator (lifetime bug, independent).
+    single-thread-use state and the multi-threaded tests pass without locks.
+  - Remaining choice only matters once we actually parse in parallel: reuse
+    parsers via a pool or per-thread storage, plugged in behind `open_parser`
+    without touching call sites. A `TSParser`/`TSQueryCursor` is one-per-thread.
   - Full write-up: `GRAMMAR_CACHE_THREADING.md`.
+- [x] Restore parallel grammar preload (serialized in `edb2e64`); `preload_grammars`
+      now fans `build_grammar` across a bounded `core:thread.Pool`.
+  - [ ] Add tests: pool fan-out correctness + `-sanitize:thread` for race-freedom.
+  - [ ] Concurrent-read path: an `RW_Mutex` on `grammar()` only helps a future
+        multi-threaded render, not the load side. Revisit if render goes parallel.
 - [ ] instead of `warnings: [dynamic]Error` we should use `warnings: [dynamic; 8]Error`
   - when limit reached, the template fails and stops rendering.
 - [ ] Brainstorm plugin/tool/(postccs/less/tailwind/sass) support.
+- [ ] Reconsider `"menus": {}` opt-out: it currently discards all page frontmatter
+      menus (with only a warning). Decide whether empty config should instead just
+      suppress the auto-generated menu while still honoring frontmatter.
+- [ ] asset css files not  automatically  included in site.
+  - should we at least warn about them?
+- [ ] warn if stylesheet url starts with /assets  (and is therefore invalid).
+- [ ] warn when heading levels are skipped in a toc.
+- [ ] extract `[TOC]` from `.md` body (or at least from  the start.)
 
 ## General
 

assets.odin

@@ -5,7 +5,20 @@ import "core:log"
 import "core:os"
 import "core:strings"
 
+import ts "treesitter"
+
 copy_assets_dir :: proc(vfs: ^VFS, output_dir: string, features: bit_set[Feature]) {
+	// One CSS parser, reused for every .css asset (single-threaded). Owned here.
+	css_parser: ts.Parser
+	if .Minify in features {
+		if g, ok := ts.grammar("css"); ok {
+			css_parser = ts.open_parser(g)
+		}
+	}
+	defer if css_parser != nil {
+		ts.parser_delete(css_parser)
+	}
+
 	for virtual_path, entry in vfs.files {
 		if !strings.has_prefix(virtual_path, "assets/") {
 			continue
@@ -24,7 +37,11 @@ copy_assets_dir :: proc(vfs: ^VFS, output_dir: string, features: bit_set[Feature
 		if .Minify in features && strings.has_suffix(rel, ".css") {
 			data, ok := vfs_get(vfs, virtual_path)
 			if ok {
-				write_file(dest, minify_css(string(data)))
+				if m, mok := minify_css(css_parser, string(data)); mok {
+					write_file(dest, m)
+				} else {
+					write_file(dest, string(data))
+				}
 			}
 		} else if entry.data != nil {
 			if err := os.write_entire_file(dest, entry.data); err != nil {

defaults/layouts/partials/styles.html

@@ -1,3 +1,4 @@
+{{^params.stylesheets}}
 <style>
 /* Nav */
 header nav ul {
@@ -228,4 +229,66 @@ label.margin-toggle:not(.sidenote-number) {
   }
 }
 </style>
-{{#params.stylesheets}}<link rel="stylesheet" href="{{.|rel_url}}">{{/params.stylesheets}}
+{{/params.stylesheets}}
+<style>
+/* Syntax highlighting. Build-time <span class="hl-*"> wrappers only appear
+   inside fenced code blocks. The base class (e.g. hl-keyword) is always
+   emitted alongside any dotted variant (hl-keyword-control), so styling the
+   base names covers every capture. Re-theme by overriding the --hl-* vars. */
+:root {
+  --hl-bg:        #282c34;
+  --hl-fg:        #abb2bf;
+  --hl-comment:   #5c6370;
+  --hl-keyword:   #c678dd;
+  --hl-string:    #98c379;
+  --hl-escape:    #56b6c2;
+  --hl-number:    #d19a66;
+  --hl-function:  #61afef;
+  --hl-type:      #e5c07b;
+  --hl-variable:  #e06c75;
+  --hl-attribute: #d19a66;
+}
+
+pre:has(> code [class^="hl-"]) {
+  background: var(--hl-bg);
+  color: var(--hl-fg);
+  padding: 0.75rem 1rem;
+  border-radius: 4px;
+}
+
+.hl-comment { color: var(--hl-comment); font-style: italic; }
+.hl-keyword,
+.hl-conditional,
+.hl-repeat,
+.hl-exception,
+.hl-include,
+.hl-storageclass { color: var(--hl-keyword); }
+.hl-operator,
+.hl-punctuation { color: var(--hl-fg); }
+.hl-string,
+.hl-character { color: var(--hl-string); }
+.hl-escape,
+.hl-string-escape,
+.hl-string-special { color: var(--hl-escape); }
+.hl-number,
+.hl-float,
+.hl-boolean,
+.hl-constant { color: var(--hl-number); }
+.hl-function,
+.hl-method,
+.hl-constructor { color: var(--hl-function); }
+.hl-type,
+.hl-namespace,
+.hl-module { color: var(--hl-type); }
+.hl-variable,
+.hl-property,
+.hl-field,
+.hl-parameter,
+.hl-tag,
+.hl-symbol { color: var(--hl-variable); }
+.hl-attribute,
+.hl-label,
+.hl-preproc { color: var(--hl-attribute); }
+</style>
+{{#params.stylesheets}}<link rel="stylesheet" href="{{.|rel_url}}">
+{{/params.stylesheets}}

feed.odin

@@ -1,5 +1,6 @@
 package main
 
+import "base:runtime"
 import "core:fmt"
 import "core:strings"
 import "core:time"
@@ -110,8 +111,7 @@ generate_sitemap :: proc(site: ^Site) -> string {
 	return strings.to_string(sb)
 }
 
-// TODO: Leaks
+format_rfc822 :: proc(iso: string, allocator := context.temp_allocator) -> string {
-format_rfc822 :: proc(iso: string) -> string {
 	if len(iso) < 19 {
 		// TODO: should indicate error somehow
 		return iso
@@ -124,21 +124,66 @@ format_rfc822 :: proc(iso: string) -> string {
 	buf: [8]byte
 	t := time.to_string_hms(date, buf[:])
 
+	// RFC-822 zone: sign followed by two-digit hours and minutes (e.g. +0000,
+	// +0530, -0800). offset is in minutes.
+	sign := '+'
+	off := offset
+	if off < 0 {
+		sign = '-'
+		off = -off
+	}
+
 	return fmt.aprintf(
-		"%s, %02d %s %d %s %3d%2d",
+		"%s, %02d %s %d %s %c%02d%02d",
 		weekday[:3],
 		time.day(date),
 		month[:3],
 		time.year(date),
 		t,
-		offset / 60,
+		sign,
-		offset % 60,
+		off / 60,
+		off % 60,
+		allocator = allocator,
 	)
 }
 
-xml_escape :: proc(s: string) -> string {
+// xml_escape escapes the XML metacharacters '&', '<' and '>' in s. Only these
-	r, _ := strings.replace_all(s, "&", "&amp;")
+// ASCII characters are touched; all other bytes (including UTF-8 multibyte
-	r, _ = strings.replace_all(r, "<", "&lt;")
+// sequences) pass through verbatim, so s must already be valid UTF-8.
-	r, _ = strings.replace_all(r, ">", "&gt;")
+//
-	return r
+// The result is allocated in `allocator`, which defaults to
+// context.temp_allocator — the returned string is only valid until the next
+// temp allocator reset, so callers must not retain it across frames.
+xml_escape :: proc(
+	s: string,
+	allocator := context.temp_allocator,
+) -> (
+	escaped: string,
+	err: runtime.Allocator_Error,
+) #optional_allocator_error {
+	sb := strings.builder_make_len_cap(0, len(s), allocator) or_return
+
+	// Single pass over the bytes, copying runs of ordinary text in bulk and
+	// emitting an entity for each metacharacter. The switch is both the
+	// membership test and the replacement lookup, so no per-match scan.
+	start := 0
+	for i := 0; i < len(s); i += 1 {
+		esc: string
+		switch s[i] {
+		case '&':
+			esc = "&amp;"
+		case '<':
+			esc = "&lt;"
+		case '>':
+			esc = "&gt;"
+		case:
+			continue // ordinary byte: copied as part of the next run
+		}
+		strings.write_string(&sb, s[start:i]) // run before the metacharacter
+		strings.write_string(&sb, esc)
+		start = i + 1
+	}
+	strings.write_string(&sb, s[start:]) // tail
+	return strings.to_string(sb), nil
 }
+

feed_test.odin

@@ -0,0 +1,166 @@
+#+test
+package main
+
+import "core:strings"
+import "core:testing"
+
+// ---------------------------------------------------------------------------
+// xml_escape
+// ---------------------------------------------------------------------------
+
+@(test)
+test_xml_escape_passthrough :: proc(t: ^testing.T) {
+	testing.expect_value(t, xml_escape(""), "")
+	testing.expect_value(t, xml_escape("hello world"), "hello world")
+	// UTF-8 multibyte bytes are not metacharacters and pass through untouched.
+	testing.expect_value(t, xml_escape("héllo — 世界"), "héllo — 世界")
+}
+
+@(test)
+test_xml_escape_entities :: proc(t: ^testing.T) {
+	testing.expect_value(t, xml_escape("&"), "&")
+	testing.expect_value(t, xml_escape("<"), "&lt;")
+	testing.expect_value(t, xml_escape(">"), "&gt;")
+	// Adjacent specials, and specials at the very start/end of the string.
+	testing.expect_value(t, xml_escape("&<>"), "&amp;&lt;&gt;")
+	testing.expect_value(t, xml_escape("<a>"), "&lt;a&gt;")
+}
+
+@(test)
+test_xml_escape_mixed :: proc(t: ^testing.T) {
+	testing.expect_value(t, xml_escape("a<b>c&d"), "a&lt;b&gt;c&amp;d")
+	testing.expect_value(
+		t,
+		xml_escape("<p>Tom & Jerry</p>"),
+		"&lt;p&gt;Tom &amp; Jerry&lt;/p&gt;",
+	)
+}
+
+// ---------------------------------------------------------------------------
+// format_rfc822
+// ---------------------------------------------------------------------------
+
+@(test)
+test_format_rfc822_short_passthrough :: proc(t: ^testing.T) {
+	// Inputs shorter than a full ISO-8601 timestamp are returned unchanged
+	// (and are the input slice itself, so they must not be deleted).
+	testing.expect_value(t, format_rfc822("2024"), "2024")
+	testing.expect_value(t, format_rfc822(""), "")
+}
+
+@(test)
+test_format_rfc822_full_date :: proc(t: ^testing.T) {
+	// Results are temp-allocated (reset by the test runner), so no delete.
+	// 2024-01-15 is a Monday; wall-clock time is preserved and the zone is
+	// rendered as an RFC-822 ±HHMM offset.
+	testing.expect_value(
+		t,
+		format_rfc822("2024-01-15T10:30:00Z"),
+		"Mon, 15 Jan 2024 10:30:00 +0000",
+	)
+	testing.expect_value(
+		t,
+		format_rfc822("2024-01-15T10:30:00+05:30"),
+		"Mon, 15 Jan 2024 10:30:00 +0530",
+	)
+	testing.expect_value(
+		t,
+		format_rfc822("2024-01-15T10:30:00-08:00"),
+		"Mon, 15 Jan 2024 10:30:00 -0800",
+	)
+}
+
+// ---------------------------------------------------------------------------
+// generate_rss
+// ---------------------------------------------------------------------------
+
+@(test)
+test_generate_rss_structure :: proc(t: ^testing.T) {
+	site: Site
+	site.title = "My & Site"
+	site.description = "Desc <here>"
+	site.base_url = "https://example.com"
+	site.pages = make(#soa[dynamic]Page)
+	defer delete_soa(site.pages)
+
+	// A real content page.
+	append(
+		&site.pages,
+		Page {
+			section = "posts",
+			title = "First <Post>",
+			url = "https://example.com/posts/first/",
+			content = "<p>Hello & welcome</p>",
+		},
+	)
+	// The home index page (section == "" && _is_index) must be skipped.
+	append(
+		&site.pages,
+		Page {
+			section = "",
+			_is_index = true,
+			title = "HOME_SHOULD_BE_SKIPPED",
+			url = "https://example.com/",
+			content = "home",
+		},
+	)
+	// A second content page.
+	append(
+		&site.pages,
+		Page {
+			section = "posts",
+			title = "Second Post",
+			url = "https://example.com/posts/second/",
+			content = "plain body",
+		},
+	)
+
+	rss := generate_rss(&site)
+	defer delete(rss)
+
+	// Channel header: site title/description are XML-escaped, link gets a
+	// trailing slash, and the atom self-link points at index.xml.
+	testing.expect(t, strings.contains(rss, "<title>My &amp; Site</title>"), rss)
+	testing.expect(
+		t,
+		strings.contains(rss, "<description>Desc &lt;here&gt;</description>"),
+		rss,
+	)
+	testing.expect(t, strings.contains(rss, "<link>https://example.com/</link>"), rss)
+	testing.expect(
+		t,
+		strings.contains(rss, `<atom:link href="https://example.com/index.xml"`),
+		rss,
+	)
+
+	// Exactly two <item>s — the home index page is skipped.
+	testing.expect_value(t, strings.count(rss, "<item>"), 2)
+	testing.expect(t, !strings.contains(rss, "HOME_SHOULD_BE_SKIPPED"), rss)
+
+	// First item: title and body are XML-escaped, link is verbatim.
+	testing.expect(t, strings.contains(rss, "<title>First &lt;Post&gt;</title>"), rss)
+	testing.expect(
+		t,
+		strings.contains(
+			rss,
+			"<description>&lt;p&gt;Hello &amp; welcome&lt;/p&gt;</description>",
+		),
+		rss,
+	)
+	testing.expect(
+		t,
+		strings.contains(rss, "<link>https://example.com/posts/first/</link>"),
+		rss,
+	)
+
+	// Undated pages get the sentinel pubDate (no date formatting/allocation).
+	testing.expect(
+		t,
+		strings.contains(rss, "<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>"),
+		rss,
+	)
+
+	// Well-formed at the boundaries.
+	testing.expect(t, strings.has_prefix(rss, `<?xml version="1.0"`), rss)
+	testing.expect(t, strings.has_suffix(rss, "</channel>\n</rss>"), rss)
+}

flake.nix

@@ -169,6 +169,10 @@
 
               bloaty # du for binaries?
 
+              # temp
+              hugo
+              pandoc
+
               # IDE
               unstable.helix
               claude-code

markdown/deflists.odin

@@ -4,140 +4,288 @@ import cm "vendor:commonmark"
 
 import "core:strings"
 
-// DefList_Entry represents a single term-definition pair in a definition list.
-DefList_Entry :: struct {
-	term:       string,
-	definition: string,
-}
-
 // convert_deflists scans markdown text for definition list patterns and
 // converts them to <dl><dt><dd> HTML blocks before cmark processing.
 //
-// A definition line starts with optional whitespace followed by a colon and
+// A definition line starts with fewer than four columns of leading whitespace
-// a space. The term is the nearest preceding non-blank line (immediately or
+// followed by a colon and a space (or end-of-line). The term is the nearest
-// within one blank line). Consecutive term+definition pairs are grouped into
+// preceding non-blank line (immediately or within one blank line). Consecutive
-// a single <dl> block.
+// term+definition pairs are grouped into a single <dl> block, and consecutive
+// definition lines under one term become multiple <dd> elements.
 //
-// Terms and definitions are rendered through cmark individually so that
+// The scanner walks the body one line at a time using byte offsets: it never
-// inline markdown (code, links, emphasis) is processed.
+// splits the input, allocates nothing per line, and copies non-deflist text
+// through verbatim. Matching is disabled inside fenced code blocks. Terms and
+// definitions are rendered through cmark individually so that inline markdown
+// (code, links, emphasis) is processed.
+// Line_Kind classifies a line for convert_deflists's group-finding scan. Other
+// covers fenced-code delimiters and their contents, where definition syntax is
+// inert.
+Line_Kind :: enum {
+	None,
+	Blank,
+	Term,
+	Def,
+	Other,
+}
+
 convert_deflists :: proc(body: string, allocator := context.allocator) -> string {
-	lines := strings.split(body, "\n", allocator = context.temp_allocator)
+	// Fast path: a body with no definition line at all cannot contain a
+	// definition list, so return it untouched — skipping the builder allocation
+	// and the classifying scan below. This is the common case (most pages have
+	// no definition lists), and the guard scan is leaner than the full loop.
+	if !has_def_line(body) {
+		return body
+	}
 
 	sb := strings.builder_make(context.temp_allocator)
 
-	first := true
+	// emitted_up_to is the byte offset up to which `body` has been copied into
-	need_blank := false
+	// `sb`. It stays at 0 until the first deflist is emitted; if it is still 0
+	// at the end, nothing matched and we can return the body unchanged.
+	emitted_up_to := 0
 
-	i := 0
+	in_fence := false
-	for i < len(lines) {
+	fence_marker: u8 = 0
-		entries, matched, next := try_match_deflist(lines, i)
+
-		if matched {
+	// The scan is definition-driven: it walks line by line, remembering the two
-			html := render_deflist(entries)
+	// previous lines' kinds and offsets, and only does group-finding work when it
-			if !first {
+	// reaches a (rare) definition line — whose term is the immediately preceding
-				strings.write_string(&sb, "\n\n")
+	// line, or the one before a single blank line. This avoids a per-line forward
+	// lookahead: each line is classified once, with no rescanning.
+	prev1_kind, prev2_kind := Line_Kind.None, Line_Kind.None
+	prev1_pos, prev2_pos := 0, 0
+
+	pos := 0
+	for {
+		line, next, ok := line_at(body, pos)
+		if !ok {
+			break
+		}
+
+		// Classify the line. Definition-list syntax is inert inside fenced code
+		// blocks, so fence delimiters and their contents are classified Other.
+		kind: Line_Kind
+		if in_fence {
+			if is_fence_line(line, fence_marker) {
+				in_fence = false
 			}
-			strings.write_string(&sb, html)
+			kind = .Other
-			first = false
+		} else if marker, opened := fence_open(line); opened {
-			need_blank = true
+			in_fence = true
-			i = next
+			fence_marker = marker
-			continue
+			kind = .Other
+		} else if is_def_line(line) {
+			kind = .Def
+		} else if is_blank_line(line) {
+			kind = .Blank
+		} else {
+			kind = .Term
 		}
 
-		if need_blank {
+		// A definition-list group begins at the term preceding this def line
-			strings.write_string(&sb, "\n\n")
+		// (immediately, or across a single blank line). Flush pending passthrough
-			need_blank = false
+		// text verbatim, then render the whole group directly into the builder.
-		} else if !first {
+		if kind == .Def {
-			strings.write_string(&sb, "\n")
+			term_pos := -1
+			if prev1_kind == .Term {
+				term_pos = prev1_pos
+			} else if prev1_kind == .Blank && prev2_kind == .Term {
+				term_pos = prev2_pos
+			}
+			if term_pos >= 0 {
+				strings.write_string(&sb, body[emitted_up_to:term_pos])
+				group_end := write_deflist_group(&sb, body, term_pos)
+				// write_deflist_group emits a raw <dl> HTML block with no
+				// trailing newline, so the blank line the author left after the
+				// list is consumed as the </dl> line's terminator rather than a
+				// separating blank line. CommonMark (type-6 HTML block) then
+				// swallows the following block — e.g. a fenced code example —
+				// as raw, unescaped HTML, leaking its markup (a template's
+				// <h1>) into the document. Re-emit a newline so </dl> stands on
+				// its own line and the author's blank line still separates the
+				// block. Only when a blank line actually follows: fused
+				// non-blank fall-through content is left as-is.
+				if line, _, ok := line_at(body, group_end); ok && is_blank_line(line) {
+					strings.write_byte(&sb, '\n')
+				}
+				emitted_up_to = group_end
+				pos = group_end
+				prev1_kind, prev2_kind = .None, .None
+				in_fence = false
+				continue
+			}
 		}
-		strings.write_string(&sb, lines[i])
+
-		first = false
+		prev2_kind, prev2_pos = prev1_kind, prev1_pos
-		i += 1
+		prev1_kind, prev1_pos = kind, pos
+		pos = next
 	}
 
+	if emitted_up_to == 0 {
+		// No definition list anywhere; the body passes through unchanged.
+		return body
+	}
+	strings.write_string(&sb, body[emitted_up_to:])
 	return strings.clone(strings.to_string(sb), allocator)
 }
 
-// try_match_deflist attempts to match a definition list group starting at
+// line_at returns the line beginning at byte offset `pos` (excluding its
-// lines[start]. A group is one or more term+definition pairs. Returns the
+// trailing newline) along with the offset of the following line. `pos` must be
-// matched entries, whether a match was found, and the index past the group.
+// 0 or one past a '\n'. ok is false once `pos` reaches the end of `body`.
-try_match_deflist :: proc(
+line_at :: proc(body: string, pos: int) -> (line: string, next: int, ok: bool) {
-	lines: []string,
+	if pos >= len(body) {
-	start: int,
+		return "", pos, false
-) -> (
+	}
-	entries: [dynamic]DefList_Entry,
+	nl := strings.index_byte(body[pos:], '\n')
-	ok: bool,
+	if nl < 0 {
-	end: int,
+		return body[pos:], len(body), true
-) {
+	}
-	entries = make([dynamic]DefList_Entry, 0, allocator = context.temp_allocator)
+	return body[pos:pos + nl], pos + nl + 1, true
-	end = start
+}
 
-	i := start
+// has_def_line reports whether any line in `body` is a definition line (see
-	for i < len(lines) {
+// is_def_line). It is a cheap guard for convert_deflists: a body with no
-		// A term must be non-blank and not itself a def line
+// definition line cannot contain a definition list. There are no false
-		if is_blank_line(lines[i]) || is_def_line(lines[i]) {
+// negatives (every definition list requires a definition line); a false
+// positive — e.g. a colon line that only appears inside a code fence — simply
+// falls through to the full scan, which then finds no match and returns `body`.
+has_def_line :: proc(body: string) -> bool {
+	pos := 0
+	for {
+		line, next := line_at(body, pos) or_break
+		if is_def_line(line) {
+			return true
+		}
+		pos = next
+	}
+	return false
+}
+
+// write_deflist_group renders the definition-list group beginning at `start`
+// directly into `sb`, wrapped in <dl>...</dl>, and returns the byte offset one
+// past the group (the start of the first line not consumed). The caller must
+// have confirmed via term_has_def that a group starts at `start`, so the group
+// always contains at least one term+definition pair. A trailing blank line that
+// separates the group from following content is left unconsumed.
+write_deflist_group :: proc(sb: ^strings.Builder, body: string, start: int) -> (end: int) {
+	strings.write_string(sb, "<dl>")
+
+	pos := start
+	for {
+		term_line, term_next, term_ok := line_at(body, pos)
+		if !term_ok || is_blank_line(term_line) || is_def_line(term_line) {
 			break
 		}
 
-		// Look for a def line: immediately after or with one blank line
+		// The definition follows immediately or across a single blank line.
-		def_idx := i + 1
+		def_pos := term_next
-		if def_idx < len(lines) && is_blank_line(lines[def_idx]) {
+		def_line, def_next, def_ok := line_at(body, def_pos)
-			def_idx += 1
+		if def_ok && is_blank_line(def_line) {
+			def_pos = def_next
+			def_line, def_next, def_ok = line_at(body, def_pos)
 		}
-
+		if !def_ok || !is_def_line(def_line) {
-		if def_idx >= len(lines) || !is_def_line(lines[def_idx]) {
 			break
 		}
 
-		// Found a term + def pair
+		// Emit the term with its first definition, then any consecutive
-		append(
+		// definition lines as additional <dd>s under the same term.
-			&entries,
+		strings.write_string(sb, "<dt>")
-			DefList_Entry {
+		strings.write_string(sb, render_inline_md(strings.trim_space(term_line)))
-				term = strings.trim_space(lines[i]),
+		strings.write_string(sb, "</dt><dd>")
-				definition = def_content(lines[def_idx]),
+		strings.write_string(sb, render_inline_md(def_content(def_line)))
-			},
+		strings.write_string(sb, "</dd>")
-		)
+		pos = def_next
-		i = def_idx + 1
+		for {
+			cont, cont_next, cont_ok := line_at(body, pos)
+			if !cont_ok || !is_def_line(cont) {
+				break
+			}
+			strings.write_string(sb, "<dd>")
+			strings.write_string(sb, render_inline_md(def_content(cont)))
+			strings.write_string(sb, "</dd>")
+			pos = cont_next
+		}
 
-		// After a pair, check if another pair follows (optionally
+		// Continue the group if another term+definition pair follows,
-		// separated by one blank line). If so, continue the group.
+		// optionally separated by one blank line. Otherwise stop here without
-		// If not, break without consuming the blank line.
+		// consuming a trailing blank line.
-		if i < len(lines) && is_blank_line(lines[i]) {
+		blank_line, after_blank, blank_ok := line_at(body, pos)
-			after_blank := i + 1
+		if blank_ok && is_blank_line(blank_line) {
-			if after_blank < len(lines) &&
+			if term_has_def(body, after_blank) {
-			   !is_blank_line(lines[after_blank]) &&
+				pos = after_blank
-			   !is_def_line(lines[after_blank]) {
+				continue
-				// Check whether a def follows this potential term
-				check_def := after_blank + 1
-				if check_def < len(lines) && is_blank_line(lines[check_def]) {
-					check_def += 1
-				}
-				if check_def < len(lines) && is_def_line(lines[check_def]) {
-					i = after_blank
-					continue
-				}
 			}
 			break
 		}
+		// No blank line: loop and let the top re-test this line as a term.
 	}
 
-	if len(entries) > 0 {
+	strings.write_string(sb, "</dl>")
-		ok = true
+	return pos
-		end = i
-	}
-	return
 }
 
-// is_def_line returns true if the line is a definition line:
+// term_has_def reports whether the line at `pos` is a term that is immediately
-// optional leading whitespace, a colon, then whitespace or end-of-line.
+// (or across one blank line) followed by a definition line.
-is_def_line :: proc(line: string) -> bool {
+term_has_def :: proc(body: string, pos: int) -> bool {
-	trimmed := strings.trim_left(line, " \t")
+	line, next, ok := line_at(body, pos)
-	if len(trimmed) < 1 || trimmed[0] != ':' {
+	if !ok || is_blank_line(line) || is_def_line(line) {
 		return false
 	}
-	if len(trimmed) == 1 {
+	def_pos := next
+	def_line, def_next, def_ok := line_at(body, def_pos)
+	if def_ok && is_blank_line(def_line) {
+		def_pos = def_next
+		def_line, def_next, def_ok = line_at(body, def_pos)
+	}
+	return def_ok && is_def_line(def_line)
+}
+
+// fence_open reports whether `line` opens a fenced code block and, if so, which
+// marker character it uses. Leading spaces are ignored.
+fence_open :: proc(line: string) -> (marker: u8, ok: bool) {
+	trimmed := strings.trim_left(line, " ")
+	if strings.has_prefix(trimmed, "```") {
+		return '`', true
+	}
+	if strings.has_prefix(trimmed, "~~~") {
+		return '~', true
+	}
+	return 0, false
+}
+
+// is_fence_line reports whether `line` is a fence using the given marker
+// character (used to detect the closing fence of an open block).
+is_fence_line :: proc(line: string, marker: u8) -> bool {
+	trimmed := strings.trim_left(line, " ")
+	if marker == '~' {
+		return strings.has_prefix(trimmed, "~~~")
+	}
+	return strings.has_prefix(trimmed, "```")
+}
+
+// is_def_line returns true if the line is a definition line: fewer than four
+// columns of leading whitespace, a colon, then whitespace or end-of-line. Four
+// or more columns of indentation is a code block, not a definition.
+is_def_line :: proc(line: string) -> bool {
+	indent := 0
+	n := 0
+	for n < len(line) && (line[n] == ' ' || line[n] == '\t') {
+		indent += line[n] == '\t' ? 4 : 1
+		n += 1
+	}
+	if indent >= 4 {
+		return false
+	}
+	rest := line[n:]
+	if len(rest) < 1 || rest[0] != ':' {
+		return false
+	}
+	if len(rest) == 1 {
 		return true
 	}
-	return trimmed[1] == ' ' || trimmed[1] == '\t'
+	return rest[1] == ' ' || rest[1] == '\t'
 }
 
 // def_content extracts the definition text from a definition line,
@@ -154,27 +302,6 @@ is_blank_line :: proc(line: string) -> bool {
 	return strings.trim_space(line) == ""
 }
 
-// render_deflist builds the <dl> HTML block from a list of entries.
-// Each term and definition is rendered through cmark to process inline
-// markdown. Result lives in context.temp_allocator.
-render_deflist :: proc(entries: [dynamic]DefList_Entry) -> string {
-	sb := strings.builder_make(context.temp_allocator)
-
-	strings.write_string(&sb, "<dl>")
-	for entry in entries {
-		term_html := render_inline_md(entry.term)
-		def_html := render_inline_md(entry.definition)
-		strings.write_string(&sb, "<dt>")
-		strings.write_string(&sb, term_html)
-		strings.write_string(&sb, "</dt><dd>")
-		strings.write_string(&sb, def_html)
-		strings.write_string(&sb, "</dd>")
-	}
-	strings.write_string(&sb, "</dl>")
-
-	return strings.to_string(sb)
-}
-
 // render_inline_md renders a snippet of markdown through cmark and strips
 // the surrounding <p> tags. Result lives in context.temp_allocator.
 render_inline_md :: proc(text: string) -> string {

markdown/deflists_fixtures_test.odin

@@ -0,0 +1,92 @@
+#+test
+package markdown
+
+import "core:testing"
+
+// Fixture-driven tests for convert_deflists. Every fixture is exact-matched
+// against its oracle in testdata/deflists/expected/. See
+// testdata/deflists/reference/README.md for how each expected file was decided
+// (cross-checked against Pandoc and Hugo/Goldmark).
+//
+// The fixture inputs are the definition-list examples from
+// https://www.markdownlang.com/extended/definition-lists.html
+//
+// Provenance of each expected file:
+//   01,02,08,09  Pandoc and Hugo agree; oracle = that agreed HTML.
+//   06           Both agree it is NOT a deflist; convert_deflists passes the
+//                markdown through unchanged, so expected = the raw input.
+//   07           CommonMark's <4-space marker rule: we accept the 2-space colon
+//                (Pandoc does; Hugo does not).
+//   05           Block nesting is unsupported; our flatten happens to match
+//                Pandoc's output exactly.
+//   03,04        Block content in definitions is unsupported; expected captures
+//                our current degraded behavior (a <dl> for the term/first line,
+//                the rest falling through as markdown) as a regression guard.
+Deflist_Fixture :: struct {
+	name:     string,
+	input:    string,
+	expected: string,
+}
+
+@(test)
+test_deflist_fixtures :: proc(t: ^testing.T) {
+	cases := []Deflist_Fixture {
+		{
+			"01-basic",
+			#load(#directory + "testdata/deflists/01-basic.md", string),
+			#load(#directory + "testdata/deflists/expected/01-basic.html", string),
+		},
+		{
+			"02-multiple-defs",
+			#load(#directory + "testdata/deflists/02-multiple-defs.md", string),
+			#load(#directory + "testdata/deflists/expected/02-multiple-defs.html", string),
+		},
+		{
+			"03-multiline-def",
+			#load(#directory + "testdata/deflists/03-multiline-def.md", string),
+			#load(#directory + "testdata/deflists/expected/03-multiline-def.html", string),
+		},
+		{
+			"04-markdown-in-def",
+			#load(#directory + "testdata/deflists/04-markdown-in-def.md", string),
+			#load(#directory + "testdata/deflists/expected/04-markdown-in-def.html", string),
+		},
+		{
+			"05-nested",
+			#load(#directory + "testdata/deflists/05-nested.md", string),
+			#load(#directory + "testdata/deflists/expected/05-nested.html", string),
+		},
+		{
+			"06-compact",
+			#load(#directory + "testdata/deflists/06-compact.md", string),
+			#load(#directory + "testdata/deflists/expected/06-compact.html", string),
+		},
+		{
+			"07-extra-space",
+			#load(#directory + "testdata/deflists/07-extra-space.md", string),
+			#load(#directory + "testdata/deflists/expected/07-extra-space.html", string),
+		},
+		{
+			"08-glossary",
+			#load(#directory + "testdata/deflists/08-glossary.md", string),
+			#load(#directory + "testdata/deflists/expected/08-glossary.html", string),
+		},
+		{
+			"09-api-params",
+			#load(#directory + "testdata/deflists/09-api-params.md", string),
+			#load(#directory + "testdata/deflists/expected/09-api-params.html", string),
+		},
+	}
+
+	for c in cases {
+		result := convert_deflists(c.input, context.temp_allocator)
+		testing.expectf(
+			t,
+			result == c.expected,
+			"[%s]\n  expected: %q\n  got:      %q",
+			c.name,
+			c.expected,
+			result,
+		)
+	}
+}

markdown/deflists_test.odin

@@ -90,6 +90,17 @@ test_empty_body :: proc(t: ^testing.T) {
 	testing.expect_value(t, result, "")
 }
 
+// Regression: a fenced code block following a definition list must stay a
+// separate block. Without a blank line after the emitted </dl>, CommonMark
+// treats the fence as raw HTML continuation of the <dl> block and passes its
+// contents through unescaped (leaking e.g. an example <h1> into the document).
+@(test)
+test_deflist_before_code_fence :: proc(t: ^testing.T) {
+	input := "term\n\n: def\n\n```\n<h1>example</h1>\n```"
+	result := convert_deflists(input, context.temp_allocator)
+	testing.expect(t, strings.contains(result, "</dl>\n\n```"))
+}
+
 @(test)
 test_docs_md_pattern :: proc(t: ^testing.T) {
 	input := "content\n\n: `content` holds your pages.\n\n assets\n : `assets` contains files."

markdown/highlight.odin

@@ -90,9 +90,12 @@ write_escaped :: proc(b: ^strings.Builder, s: string) {
 	}
 }
 
+// unescape_html returns a caller-owned string (free it with delete). When the
+// input has no entities it returns a clone; otherwise it returns the builder's
+// buffer. It must NOT destroy the builder — to_string aliases that buffer, so the
+// returned string owns it.
 unescape_html :: proc(s: string) -> string {
 	sb := strings.builder_make()
-	defer strings.builder_destroy(&sb)
 
 	start := 0
 	for i in 0 ..< len(s) {
@@ -119,22 +122,28 @@ unescape_html :: proc(s: string) -> string {
 		strings.write_string(&sb, replacement)
 		start = i + semi + 1
 	}
-	if start == 0 do return s
+	if start == 0 do return strings.clone(s)
 	if start < len(s) do strings.write_string(&sb, s[start:])
 	return strings.to_string(sb)
 }
 
 highlight_block :: proc(code: string, lang: string, file_path: string) -> string {
-	gc := ts.load_grammar(lang)
+	g, ok := ts.grammar(lang)
-	if gc == nil {
+	if !ok {
 		return code
 	}
+	parser := ts.open_parser(g)
+	if parser == nil {
+		return code
+	}
+	defer ts.parser_delete(parser)
 
 	raw_code := unescape_html(code)
+	defer delete(raw_code)
 	raw_c := strings.clone_to_cstring(raw_code)
 	defer delete(raw_c)
 
-	tree := ts.parser_parse_string(gc.parser, nil, raw_c, u32(len(raw_code)))
+	tree := ts.parser_parse_string(parser, nil, raw_c, u32(len(raw_code)))
 	if tree == nil {
 		return code
 	}
@@ -156,12 +165,13 @@ highlight_block :: proc(code: string, lang: string, file_path: string) -> string
 		}
 	}
 
-	cursor := gc.cursor
+	if g.query == nil {
-	if cursor == nil {
 		return code
 	}
+	cursor := ts.query_cursor_new()
+	defer ts.query_cursor_delete(cursor)
 
-	ts.query_cursor_exec(cursor, gc.query, root)
+	ts.query_cursor_exec(cursor, g.query, root)
 
 	captures := make([dynamic]Capture, 0, 64, context.temp_allocator)
 	defer delete(captures)
@@ -174,7 +184,7 @@ highlight_block :: proc(code: string, lang: string, file_path: string) -> string
 		}
 		cap := match.captures[capture_idx]
 		name_len: u32
-		name_c := ts.query_capture_name_for_id(gc.query, cap.index, &name_len)
+		name_c := ts.query_capture_name_for_id(g.query, cap.index, &name_len)
 		if name_c == nil {
 			continue
 		}

markdown/highlight_test.odin

@@ -0,0 +1,103 @@
+#+test
+package markdown
+
+import ts "../treesitter"
+
+import "core:mem"
+import "core:strings"
+import "core:testing"
+
+// highlight_block returns the input string unchanged (borrowed) when it can't
+// highlight — no grammar, no query, or zero captures — and a freshly-allocated
+// string when it does. Free only the latter; identity (raw_data) tells them
+// apart regardless of content.
+@(private = "file")
+free_highlight :: proc(out, code: string) {
+	if raw_data(out) != raw_data(code) {
+		delete(out)
+	}
+}
+
+// A builtin grammar (css) highlights into hl- spans, preserving the source text.
+@(test)
+test_highlight_block_css :: proc(t: ^testing.T) {
+	code := "a{color:red}"
+	out := highlight_block(code, "css", "test")
+	defer free_highlight(out, code)
+
+	testing.expect(t, strings.contains(out, "<span class=\"hl-"), "css should be highlighted")
+	testing.expect(t, strings.contains(out, "</span>"))
+	testing.expect(t, strings.contains(out, "color"))
+	testing.expect(t, strings.contains(out, "red"))
+}
+
+// The other builtin grammar (html) highlights, and the source's own '<'/'>' are
+// HTML-escaped in the output (so they can't be confused with the wrapping spans).
+@(test)
+test_highlight_block_html_escapes_source :: proc(t: ^testing.T) {
+	code := "<p>hi</p>"
+	out := highlight_block(code, "html", "test")
+	defer free_highlight(out, code)
+
+	testing.expect(t, strings.contains(out, "<span class=\"hl-"), "html should be highlighted")
+	testing.expect(t, strings.contains(out, "hi"))
+	// The literal angle brackets of the source are escaped in the output.
+	testing.expect(t, strings.contains(out, "&lt;"))
+	testing.expect(t, strings.contains(out, "&gt;"))
+}
+
+// A language with no available grammar returns the input verbatim (best-effort:
+// highlighting is optional). The returned string must be the same buffer, so
+// free_highlight leaves it alone.
+@(test)
+test_highlight_block_unknown_lang_passthrough :: proc(t: ^testing.T) {
+	code := "fn main() {}"
+	out := highlight_block(code, "no-such-language", "test")
+	defer free_highlight(out, code)
+
+	testing.expect_value(t, out, code)
+	testing.expect(t, raw_data(out) == raw_data(code), "passthrough must return the input buffer")
+}
+
+// End-to-end: highlight_code finds a fenced code block in rendered HTML and
+// rewrites its body with highlight spans while preserving the <pre><code> shell.
+// Run under a scratch arena (as production does) so the intermediate allocations
+// highlight_code doesn't individually free are reclaimed in one shot.
+@(test)
+test_highlight_code_wraps_css_block :: proc(t: ^testing.T) {
+	arena: mem.Dynamic_Arena
+	mem.dynamic_arena_init(&arena)
+	defer mem.dynamic_arena_destroy(&arena)
+	context.allocator = mem.dynamic_arena_allocator(&arena)
+
+	html := `<pre><code class="language-css">a{color:red}</code></pre>`
+	out := highlight_code(html, "test")
+
+	testing.expect(t, strings.contains(out, `<pre><code class="language-css">`))
+	testing.expect(t, strings.contains(out, "</code></pre>"))
+	testing.expect(
+		t,
+		strings.contains(out, "<span class=\"hl-"),
+		"block body should be highlighted",
+	)
+}
+
+// unescape_html must return a caller-owned allocation the caller frees exactly
+// once. The buggy version frees its builder internally (defer builder_destroy)
+// and returns the freed alias, so the caller's delete becomes a double-free that
+// the test runner's tracking allocator flags.
+@(test)
+test_unescape_html_entities_owned :: proc(t: ^testing.T) {
+	got := unescape_html("&lt;a&gt;")
+	defer delete(got)
+	testing.expect_value(t, got, "<a>")
+}
+
+// Same contract on the no-entity path: the buggy version returns the borrowed
+// input, so freeing it is a bad free (pointer not from the allocator).
+@(test)
+test_unescape_html_passthrough_owned :: proc(t: ^testing.T) {
+	got := unescape_html("plain text")
+	defer delete(got)
+	testing.expect_value(t, got, "plain text")
+}

markdown/testdata/deflists/01-basic.md

@@ -0,0 +1,2 @@
+Term
+: Definition content

markdown/testdata/deflists/02-multiple-defs.md

@@ -0,0 +1,7 @@
+Markdown
+: A lightweight markup language
+: Created by John Gruber in 2004
+
+HTML
+: A standard markup language used to create web pages
+: Used to create web pages

markdown/testdata/deflists/03-multiline-def.md

@@ -0,0 +1,5 @@
+Term
+: This is the first line of definition content
+   This is the second line, needs at least one space indentation
+  
+   This is a new paragraph, needs at least one space indentation and a preceding blank line

markdown/testdata/deflists/04-markdown-in-def.md

@@ -0,0 +1,7 @@
+Markdown Syntax
+: **Markdown** supports multiple text formatting:
+  - *Italic* and **Bold**
+  - [Link](https://www.markdownlang.com)
+  - `Inline code`
+  - > Block quote
+  - Lists and other elements

markdown/testdata/deflists/05-nested.md

@@ -0,0 +1,6 @@
+Outer Term
+: Outer Definition
+
+   Inner Term
+  : Inner Definition
+  : Another Inner Definition

markdown/testdata/deflists/06-compact.md

@@ -0,0 +1 @@
+Term: Definition

markdown/testdata/deflists/07-extra-space.md

@@ -0,0 +1,2 @@
+Term
+  : Definition

markdown/testdata/deflists/08-glossary.md

@@ -0,0 +1,11 @@
+API
+: An application programming interface, a set of rules that allow different software applications to communicate with each other.
+
+Framework
+: A software library that provides a standardized structure for application development.
+
+Git
+: A distributed version control system used to track changes in a project development process.
+
+IDE
+: An integrated development environment, a software application that includes a code editor and various development tools.

markdown/testdata/deflists/09-api-params.md

@@ -0,0 +1,14 @@
+user_id
+: **Required** - The unique identifier for the user.
+: Type: `integer`
+
+name
+: **Optional** - The display name for the user.
+: Type: `string`
+: Default: `null`
+
+status
+: **Optional** - The user status.
+: Type: `string`
+: Allowed values: `active`, `inactive`, `suspended`
+: Default: `active`

markdown/testdata/deflists/expected/01-basic.html

@@ -0,0 +1 @@
+<dl><dt>Term</dt><dd>Definition content</dd></dl>

markdown/testdata/deflists/expected/02-multiple-defs.html

@@ -0,0 +1 @@
+<dl><dt>Markdown</dt><dd>A lightweight markup language</dd><dd>Created by John Gruber in 2004</dd><dt>HTML</dt><dd>A standard markup language used to create web pages</dd><dd>Used to create web pages</dd></dl>

markdown/testdata/deflists/expected/03-multiline-def.html

@@ -0,0 +1,3 @@
+<dl><dt>Term</dt><dd>This is the first line of definition content</dd></dl>   This is the second line, needs at least one space indentation
+  
+   This is a new paragraph, needs at least one space indentation and a preceding blank line

markdown/testdata/deflists/expected/04-markdown-in-def.html

@@ -0,0 +1,5 @@
+<dl><dt>Markdown Syntax</dt><dd><strong>Markdown</strong> supports multiple text formatting:</dd></dl>  - *Italic* and **Bold**
+  - [Link](https://www.markdownlang.com)
+  - `Inline code`
+  - > Block quote
+  - Lists and other elements

markdown/testdata/deflists/expected/05-nested.html

@@ -0,0 +1 @@
+<dl><dt>Outer Term</dt><dd>Outer Definition</dd><dt>Inner Term</dt><dd>Inner Definition</dd><dd>Another Inner Definition</dd></dl>

markdown/testdata/deflists/expected/06-compact.html

@@ -0,0 +1 @@
+Term: Definition

markdown/testdata/deflists/expected/07-extra-space.html

@@ -0,0 +1 @@
+<dl><dt>Term</dt><dd>Definition</dd></dl>

markdown/testdata/deflists/expected/08-glossary.html

@@ -0,0 +1 @@
+<dl><dt>API</dt><dd>An application programming interface, a set of rules that allow different software applications to communicate with each other.</dd><dt>Framework</dt><dd>A software library that provides a standardized structure for application development.</dd><dt>Git</dt><dd>A distributed version control system used to track changes in a project development process.</dd><dt>IDE</dt><dd>An integrated development environment, a software application that includes a code editor and various development tools.</dd></dl>

markdown/testdata/deflists/expected/09-api-params.html

@@ -0,0 +1 @@
+<dl><dt>user_id</dt><dd><strong>Required</strong> - The unique identifier for the user.</dd><dd>Type: <code>integer</code></dd><dt>name</dt><dd><strong>Optional</strong> - The display name for the user.</dd><dd>Type: <code>string</code></dd><dd>Default: <code>null</code></dd><dt>status</dt><dd><strong>Optional</strong> - The user status.</dd><dd>Type: <code>string</code></dd><dd>Allowed values: <code>active</code>, <code>inactive</code>, <code>suspended</code></dd><dd>Default: <code>active</code></dd></dl>

markdown/testdata/deflists/reference/COMPARISON.md

@@ -0,0 +1,60 @@
+# 3-way behavior comparison: ours vs Pandoc vs Hugo
+
+Captured for later review (e.g. if we ever pursue "Option B" — full CommonMark
+block semantics inside definitions). Compares `convert_deflists` output (Option A,
+inline/flat) against Pandoc (`-f markdown -t html5`) and Hugo/Goldmark for the
+four fixtures where behavior is interesting. Raw renderings live in `pandoc/` and
+`hugo/`; ours is in `../expected/`. Whitespace/formatting differences ignored
+below — only structural differences are described.
+
+## 03-multiline-def — definition with a continuation line + a second paragraph
+*How much multi-line content belongs to the `<dd>`?*
+- **Ours:** only the **first line** goes in the `<dd>`. The continuation line and
+  the second paragraph fall out of the `<dl>` as **raw, unrendered markdown**.
+- **Pandoc:** folds the continuation line **into** the `<dd>` (lazy continuation),
+  then **ejects** the second paragraph as a sibling `<p>` after `</dl>`.
+- **Hugo:** keeps **everything inside** the `<dd>` — continuation line as text,
+  second paragraph as a nested `<p>`.
+- All three differ. We capture the least, Pandoc the middle, Hugo the most.
+
+## 04-markdown-in-def — definition containing a bullet list (with a blockquote)
+*What happens to the nested list?*
+- **Ours:** intro line becomes the `<dd>`; the **entire bullet list falls out as
+  raw markdown** (unrendered `- …` lines after `</dl>`).
+- **Pandoc:** bullets **flattened into the `<dd>`** as inline text — no `<ul>`, the
+  `-` become literal dashes.
+- **Hugo:** bullets become a **real nested `<ul>`/`<li>`** (blockquote and all)
+  inside the `<dd>`.
+- All three differ; same shape as 03. We drop it, Pandoc absorbs-but-flattens,
+  Hugo builds true structure.
+
+## 05-nested — a definition list nested under another definition
+*Nest or flatten?*
+- **Ours:** **flatten** — one flat `<dl>`; "Inner Term" is just another `<dt>`.
+- **Pandoc:** **flatten** — structurally **identical to ours**.
+- **Hugo:** **nests** a real inner `<dl>` inside the outer `<dd>`.
+- Ours == Pandoc; Hugo is the odd one out.
+
+## 07-extra-space — term then a 2-space-indented `: Definition`
+*Is a 2-space colon a definition marker?*
+- **Ours:** **yes** → clean `<dl>`.
+- **Pandoc:** **yes** → structurally **identical to ours**.
+- **Hugo:** **no** → rejects it, emits a plain `<p>`.
+- Ours == Pandoc; Hugo is the odd one out.
+
+## Pattern
+
+Two clean groups:
+
+- **Marker / flattening decisions (05, 07):** we land **exactly on Pandoc**. These
+  are the calls our CommonMark-marker + flat-inline design makes cleanly; Pandoc
+  agrees, Hugo diverges (nests / stricter marker).
+- **Block-content-in-a-definition (03, 04):** we diverge from **both** tools, doing
+  **less** — we emit the `<dl>` for the term/first line and let the rest fall
+  through as raw markdown, rather than absorbing it into the `<dd>` (flattened like
+  Pandoc, or structured like Hugo).
+
+In short: our behavior is **"Pandoc, minus the ability to pull block content into
+a `<dd>`."** Single inline-line definitions (01, 02, 05, 07, 08, 09) match a real
+engine; genuinely multi-block definitions (03, 04) stop at the first line and pass
+the remainder through. Closing that gap is what Option B would entail.

markdown/testdata/deflists/reference/README.md

@@ -0,0 +1,44 @@
+# Definition-list test fixtures
+
+Inputs (`../NN-name.md`) are the definition-list examples from
+<https://www.markdownlang.com/extended/definition-lists.html>.
+
+`pandoc/` and `hugo/` hold each input rendered by Pandoc (`-f markdown -t html5`)
+and Hugo (Goldmark with `markup.goldmark.extensions.definitionList = true`).
+They are retained as reference for a possible future "Option B" (full CommonMark
+block semantics inside definitions). They are NOT the test oracle for the current
+implementation.
+
+`../expected/NN.html` is the oracle for `convert_deflists`: our inline fragment
+format (no inter-tag newlines, no trailing newline), derived from the trusted
+tool per the decisions below.
+
+## Decisions (Option A — simple / inline)
+
+Marker whitespace follows CommonMark: `:` with 0–3 spaces of indent is a
+definition; 4+ (tab = 4) is a code block. Definition bodies are inline/flat.
+
+Every fixture has a concrete exact-match oracle in `../expected/NN.html`.
+
+| Fixture | Pandoc | Hugo | Decision | expected/ source |
+|---------|--------|------|----------|------------------|
+| 01-basic | `<dl>` | `<dl>` | support | hugo (agreed) |
+| 02-multiple-defs | multi `<dd>` | multi `<dd>` | support | hugo (agreed) |
+| 06-compact (`Term: Definition`) | `<p>` | `<p>` | NOT a deflist → passthrough | raw input |
+| 07-extra-space (2-space `:`) | `<dl>` | `<p>` | support (accept, CommonMark ≤3-space rule) | hand-written |
+| 08-glossary | `<dl>` | `<dl>` | support | hugo (agreed) |
+| 09-api-params | multi `<dd>` | multi `<dd>` | support | hugo (agreed) |
+| 05-nested | flattens | nested `<dl>` | flatten (matches Pandoc) | snapshot (== pandoc) |
+| 03-multiline-def | ejects 2nd para | nests 2nd para in `<dd>` | degrade | snapshot (current output) |
+| 04-markdown-in-def | flattens list | nested `<ul>`/`<blockquote>` | degrade | snapshot (current output) |
+
+A detailed 3-way behavior comparison (ours vs Pandoc vs Hugo) for the diverging
+fixtures is in `COMPARISON.md`, kept for Option-B review.
+
+03/04/05 require block-level content inside `<dd>` (multi-paragraph, nested
+lists/blockquotes, nested `<dl>`), which the inline preprocessor does not
+produce. 05 flattens exactly as Pandoc does, so its snapshot equals Pandoc's
+output. 03/04 emit a `<dl>` for the term and first line, then let the remaining
+block content fall through as markdown; their `expected/` files snapshot that
+current degraded behavior as a regression guard, not an endorsed rendering.
+Revisit via Option B (full CommonMark block semantics) if real content needs it.

markdown/testdata/deflists/reference/hugo/01-basic.html

@@ -0,0 +1,4 @@
+<dl>
+<dt>Term</dt>
+<dd>Definition content</dd>
+</dl>

markdown/testdata/deflists/reference/hugo/02-multiple-defs.html

@@ -0,0 +1,8 @@
+<dl>
+<dt>Markdown</dt>
+<dd>A lightweight markup language</dd>
+<dd>Created by John Gruber in 2004</dd>
+<dt>HTML</dt>
+<dd>A standard markup language used to create web pages</dd>
+<dd>Used to create web pages</dd>
+</dl>

markdown/testdata/deflists/reference/hugo/03-multiline-def.html

@@ -0,0 +1,7 @@
+<dl>
+<dt>Term</dt>
+<dd>This is the first line of definition content
+This is the second line, needs at least one space indentation
+<p>This is a new paragraph, needs at least one space indentation and a preceding blank line</p>
+</dd>
+</dl>

markdown/testdata/deflists/reference/hugo/04-markdown-in-def.html

@@ -0,0 +1,16 @@
+<dl>
+<dt>Markdown Syntax</dt>
+<dd><strong>Markdown</strong> supports multiple text formatting:
+<ul>
+<li><em>Italic</em> and <strong>Bold</strong></li>
+<li><a href="https://www.markdownlang.com">Link</a></li>
+<li><code>Inline code</code></li>
+<li>
+<blockquote>
+<p>Block quote</p>
+</blockquote>
+</li>
+<li>Lists and other elements</li>
+</ul>
+</dd>
+</dl>

markdown/testdata/deflists/reference/hugo/05-nested.html

@@ -0,0 +1,10 @@
+<dl>
+<dt>Outer Term</dt>
+<dd>Outer Definition
+<dl>
+<dt>Inner Term</dt>
+<dd>Inner Definition</dd>
+<dd>Another Inner Definition</dd>
+</dl>
+</dd>
+</dl>

markdown/testdata/deflists/reference/hugo/06-compact.html

@@ -0,0 +1 @@
+<p>Term: Definition</p>

markdown/testdata/deflists/reference/hugo/07-extra-space.html

@@ -0,0 +1,2 @@
+<p>Term
+: Definition</p>

markdown/testdata/deflists/reference/hugo/08-glossary.html

@@ -0,0 +1,10 @@
+<dl>
+<dt>API</dt>
+<dd>An application programming interface, a set of rules that allow different software applications to communicate with each other.</dd>
+<dt>Framework</dt>
+<dd>A software library that provides a standardized structure for application development.</dd>
+<dt>Git</dt>
+<dd>A distributed version control system used to track changes in a project development process.</dd>
+<dt>IDE</dt>
+<dd>An integrated development environment, a software application that includes a code editor and various development tools.</dd>
+</dl>

markdown/testdata/deflists/reference/hugo/09-api-params.html

@@ -0,0 +1,14 @@
+<dl>
+<dt>user_id</dt>
+<dd><strong>Required</strong> - The unique identifier for the user.</dd>
+<dd>Type: <code>integer</code></dd>
+<dt>name</dt>
+<dd><strong>Optional</strong> - The display name for the user.</dd>
+<dd>Type: <code>string</code></dd>
+<dd>Default: <code>null</code></dd>
+<dt>status</dt>
+<dd><strong>Optional</strong> - The user status.</dd>
+<dd>Type: <code>string</code></dd>
+<dd>Allowed values: <code>active</code>, <code>inactive</code>, <code>suspended</code></dd>
+<dd>Default: <code>active</code></dd>
+</dl>

markdown/testdata/deflists/reference/pandoc/01-basic.html

@@ -0,0 +1,6 @@
+<dl>
+<dt>Term</dt>
+<dd>
+Definition content
+</dd>
+</dl>

markdown/testdata/deflists/reference/pandoc/02-multiple-defs.html

@@ -0,0 +1,16 @@
+<dl>
+<dt>Markdown</dt>
+<dd>
+A lightweight markup language
+</dd>
+<dd>
+Created by John Gruber in 2004
+</dd>
+<dt>HTML</dt>
+<dd>
+A standard markup language used to create web pages
+</dd>
+<dd>
+Used to create web pages
+</dd>
+</dl>

markdown/testdata/deflists/reference/pandoc/03-multiline-def.html

@@ -0,0 +1,9 @@
+<dl>
+<dt>Term</dt>
+<dd>
+This is the first line of definition content This is the second line,
+needs at least one space indentation
+</dd>
+</dl>
+<p>This is a new paragraph, needs at least one space indentation and a
+preceding blank line</p>

markdown/testdata/deflists/reference/pandoc/04-markdown-in-def.html

@@ -0,0 +1,9 @@
+<dl>
+<dt>Markdown Syntax</dt>
+<dd>
+<strong>Markdown</strong> supports multiple text formatting: -
+<em>Italic</em> and <strong>Bold</strong> - <a
+href="https://www.markdownlang.com">Link</a> - <code>Inline code</code>
+- &gt; Block quote - Lists and other elements
+</dd>
+</dl>

markdown/testdata/deflists/reference/pandoc/05-nested.html

@@ -0,0 +1,13 @@
+<dl>
+<dt>Outer Term</dt>
+<dd>
+Outer Definition
+</dd>
+<dt>Inner Term</dt>
+<dd>
+Inner Definition
+</dd>
+<dd>
+Another Inner Definition
+</dd>
+</dl>

markdown/testdata/deflists/reference/pandoc/06-compact.html

@@ -0,0 +1 @@
+<p>Term: Definition</p>

markdown/testdata/deflists/reference/pandoc/07-extra-space.html

@@ -0,0 +1,6 @@
+<dl>
+<dt>Term</dt>
+<dd>
+Definition
+</dd>
+</dl>

markdown/testdata/deflists/reference/pandoc/08-glossary.html

@@ -0,0 +1,22 @@
+<dl>
+<dt>API</dt>
+<dd>
+An application programming interface, a set of rules that allow
+different software applications to communicate with each other.
+</dd>
+<dt>Framework</dt>
+<dd>
+A software library that provides a standardized structure for
+application development.
+</dd>
+<dt>Git</dt>
+<dd>
+A distributed version control system used to track changes in a project
+development process.
+</dd>
+<dt>IDE</dt>
+<dd>
+An integrated development environment, a software application that
+includes a code editor and various development tools.
+</dd>
+</dl>

markdown/testdata/deflists/reference/pandoc/09-api-params.html

@@ -0,0 +1,33 @@
+<dl>
+<dt>user_id</dt>
+<dd>
+<strong>Required</strong> - The unique identifier for the user.
+</dd>
+<dd>
+Type: <code>integer</code>
+</dd>
+<dt>name</dt>
+<dd>
+<strong>Optional</strong> - The display name for the user.
+</dd>
+<dd>
+Type: <code>string</code>
+</dd>
+<dd>
+Default: <code>null</code>
+</dd>
+<dt>status</dt>
+<dd>
+<strong>Optional</strong> - The user status.
+</dd>
+<dd>
+Type: <code>string</code>
+</dd>
+<dd>
+Allowed values: <code>active</code>, <code>inactive</code>,
+<code>suspended</code>
+</dd>
+<dd>
+Default: <code>active</code>
+</dd>
+</dl>

markdown/toc.odin

@@ -1,5 +1,6 @@
 package markdown
 
+import "core:log"
 import "core:strings"
 
 // generate_toc scans rendered HTML for <h1>-<h6> tags with id attributes and
@@ -28,8 +29,23 @@ generate_toc :: proc(html: string, allocator := context.allocator) -> string {
 			current_level = level
 			strings.write_string(&b, "<ul>\n")
 		} else if level > current_level {
+			if level - current_level > 1 {
+				log.warnf(
+					"heading level skips from h%d to h%d (\"%s\"); " +
+					"table of contents inserts empty levels to keep valid nesting",
+					current_level,
+					level,
+					text,
+				)
+			}
+			// The first step down nests inside the current, still-open <li>.
+			// Each further skipped level needs its own <li> to host the deeper
+			// <ul>, since a <ul> may only contain <li> — otherwise a jump of
+			// more than one level emits stacked, parentless <ul> tags.
+			strings.write_string(&b, "<ul>\n")
+			current_level += 1
 			for current_level < level {
-				strings.write_string(&b, "<ul>\n")
+				strings.write_string(&b, "<li>\n<ul>\n")
 				current_level += 1
 			}
 		} else if level < current_level {

markdown/toc_test.odin

@@ -0,0 +1,35 @@
+#+test
+package markdown
+
+import "core:strings"
+import "core:testing"
+
+@(test)
+test_toc_sequential_nesting :: proc(t: ^testing.T) {
+	html := `<h1 id="a">A</h1><h2 id="b">B</h2><h1 id="c">C</h1>`
+	result := generate_toc(html, context.temp_allocator)
+	testing.expect(t, strings.contains(result, `<a href="#a">A</a>`))
+	testing.expect(t, strings.contains(result, `<a href="#b">B</a>`))
+	testing.expect(t, strings.contains(result, `<a href="#c">C</a>`))
+	// A well-formed list never places a <ul> directly inside a <ul>.
+	testing.expect(t, !strings.contains(result, "<ul>\n<ul>"))
+}
+
+// Regression: a heading level that jumps by more than one (h1 -> h3) must not
+// emit stacked, parentless <ul> tags. Each skipped level is wrapped in an <li>
+// so the list stays valid.
+@(test)
+test_toc_level_jump_stays_valid :: proc(t: ^testing.T) {
+	html := `<h1 id="a">A</h1><h3 id="b">B</h3>`
+	result := generate_toc(html, context.temp_allocator)
+	testing.expect(t, !strings.contains(result, "<ul>\n<ul>"))
+	// The intermediate level is hosted by an <li>.
+	testing.expect(t, strings.contains(result, "<li>\n<ul>"))
+	testing.expect(t, strings.contains(result, `<a href="#b">B</a>`))
+}
+
+@(test)
+test_toc_empty_when_no_headings :: proc(t: ^testing.T) {
+	result := generate_toc("<p>no headings here</p>", context.temp_allocator)
+	testing.expect_value(t, result, "")
+}

menus.odin

@@ -4,7 +4,6 @@ import "core:encoding/json"
 import "core:fmt"
 import "core:log"
 import "core:mem"
-import "core:os"
 import "core:strings"
 
 DEFAULT_WEIGHT :: 10
@@ -112,42 +111,32 @@ parse_page_menus :: proc(
 	return result
 }
 
-// build_menus populates site.menus:
+// build_menus populates site.menus by merging every source into one menu set:
-//   1. Config menus (thor.json "menus" key present) — exclusive, preserves array order
+//   1. Auto-menus (sections + root-level pages)
-//   2. Auto-menus (sections + root-level pages) + page frontmatter menus — merged, sorted
+//   2. Config menus (thor.json "menus" key), already parsed into site.menus
+//   3. Page frontmatter menus
+// All three are combined per menu name and sorted by weight, then name.
 //
-// If "menus" is present but empty ({}) it means explicit opt-out: no menus.
+// If "menus" is present but empty ({}) it means explicit opt-out: no menus at
-// Config menus cannot be mixed with page frontmatter menus (error).
+// all, and page frontmatter menus are ignored.
 build_menus :: proc(site: ^Site) {
-	if site.menus != nil {
+	// Explicit opt-out: "menus": {} present but empty → no menus; page
-		has_menus := false
+	// frontmatter menus are ignored (with a warning).
+	// TODO(TODOS.md): silently discarding frontmatter here is harsh; revisit.
+	if site.menus != nil && len(site.menus) == 0 {
 		for page in site.pages {
 			if len(page.menus) > 0 {
-				has_menus = true
-				break
-			}
-		}
-
-		// Already populated from config in site_apply_config
-		if len(site.menus) == 0 {
-			// Explicit opt-out ("menus": {})
-			if has_menus {
 				log.warnf(
 					"menus: config has empty menus but pages have frontmatter menu entries; ignoring page menus",
 				)
+				break
 			}
-			return
 		}
-		// Config menus active
-		if has_menus {
-			log.fatalf("menus: cannot mix config menus with frontmatter menus")
-			os.exit(1)
-		}
-		warn_all_duplicate_weights(site)
 		return
 	}
 
-	// No config menus — auto-generate, then merge page menus on top
+	// Merge all sources: auto section/root menus, config menus (already in
+	// site.menus if present), then page frontmatter menus.
 	collect_auto_menus(site)
 	merge_page_menus(site)
 	warn_all_duplicate_weights(site)
@@ -253,8 +242,17 @@ collect_auto_menus :: proc(site: ^Site) {
 		return
 	}
 
+	if site.menus == nil {
+		site.menus = make(map[string][]Menu_Entry, alloc)
+	}
+	// Merge into any existing "main" (e.g. config menus) rather than clobbering.
+	if existing, ok := site.menus["main"]; ok {
+		merged := make([dynamic]Menu_Entry, 0, len(existing) + len(entries), alloc)
+		append(&merged, ..existing)
+		append(&merged, ..entries[:])
+		entries = merged
+	}
 	sort_menu_entries(entries[:])
-	site.menus = make(map[string][]Menu_Entry, alloc)
 	site.menus["main"] = entries[:]
 }
 

menus_test.odin

@@ -421,6 +421,71 @@ test_auto_menus_no_duplicate_with_frontmatter :: proc(t: ^testing.T) {
 	testing.expect_value(t, main[0].name, "Ideas")
 }
 
+// --- config + frontmatter + auto merge tests ---
+
+@(test)
+test_build_menus_config_merges_frontmatter :: proc(t: ^testing.T) {
+	site: Site
+	mem.dynamic_arena_init(&site.arena)
+	defer mem.dynamic_arena_destroy(&site.arena)
+	context.allocator = site_allocator(&site)
+
+	// Simulate config menus as site_apply_config would populate them.
+	site.menus = make(map[string][]Menu_Entry, site_allocator(&site))
+	cfg := make([dynamic]Menu_Entry, 0, 1, site_allocator(&site))
+	append(&cfg, Menu_Entry{name = "GitHub", url = "https://example.com", weight = 20})
+	site.menus["main"] = cfg[:]
+
+	// A root-level page opting into "main" via frontmatter.
+	page := make_page("Ideas", "/ideas/")
+	page.menus = parse_page_menus(parse_raw(`"main"`), page, site_allocator(&site))
+
+	site.pages = make(#soa[dynamic]Page, 0, 1, site_allocator(&site))
+	append(&site.pages, page)
+
+	build_menus(&site)
+
+	main, ok := site.menus["main"]
+	testing.expect(t, ok)
+	// GitHub (config, weight 20) + Ideas (frontmatter, nil → DEFAULT_WEIGHT 10).
+	testing.expect(t, len(main) == 2, "expected config + frontmatter merged")
+	testing.expect_value(t, main[0].name, "Ideas")
+	testing.expect_value(t, main[1].name, "GitHub")
+}
+
+@(test)
+test_build_menus_config_merges_auto :: proc(t: ^testing.T) {
+	site: Site
+	mem.dynamic_arena_init(&site.arena)
+	defer mem.dynamic_arena_destroy(&site.arena)
+	context.allocator = site_allocator(&site)
+
+	// Config menu in a different bucket ("footer"); auto-gen owns "main".
+	site.menus = make(map[string][]Menu_Entry, site_allocator(&site))
+	cfg := make([dynamic]Menu_Entry, 0, 1, site_allocator(&site))
+	append(&cfg, Menu_Entry{name = "GitHub", url = "https://example.com", weight = 20})
+	site.menus["footer"] = cfg[:]
+
+	// A section (non-index) page triggers an auto "main" section entry.
+	page := make_page("Post", "/blog/post/")
+	page.section = "blog"
+
+	site.pages = make(#soa[dynamic]Page, 0, 1, site_allocator(&site))
+	append(&site.pages, page)
+
+	build_menus(&site)
+
+	footer, ok_f := site.menus["footer"]
+	testing.expect(t, ok_f, "config footer menu should be preserved")
+	testing.expect(t, len(footer) == 1)
+	testing.expect_value(t, footer[0].name, "GitHub")
+
+	main, ok_m := site.menus["main"]
+	testing.expect(t, ok_m, "auto-generated main menu should exist alongside config")
+	testing.expect(t, len(main) == 1)
+	testing.expect_value(t, main[0].name, "Blog")
+}
+
 // --- warn_duplicate_weights tests ---
 
 @(test)

minify.odin

@@ -12,18 +12,31 @@ Range :: struct {
 	end:   u32,
 }
 
-minify_html :: proc(source: string) -> string {
+// minify_html collapses insignificant whitespace, strips comments, and minifies
-	gc := ts.ensure_parser("html")
+// inline <style> bodies (via minify_css). The caller owns html_parser and
-	if gc == nil {
+// css_parser and passes them in; both may be reused across calls. On success it
-		return source
+// returns (owned string, true) and the caller must free the string. On failure
+// (nil html_parser, parse failure, or syntax errors) it returns ("", false) and
+// the caller should keep the original source unchanged.
+minify_html :: proc(
+	html_parser: ts.Parser,
+	css_parser: ts.Parser,
+	source: string,
+) -> (
+	minified: string,
+	ok: bool,
+) {
+	if html_parser == nil {
+		return minified, ok
 	}
 
 	source_c := strings.clone_to_cstring(source)
 	defer delete(source_c)
 
-	tree := ts.parser_parse_string(gc.parser, nil, source_c, u32(len(source)))
+	tree := ts.parser_parse_string(html_parser, nil, source_c, u32(len(source)))
 	if tree == nil {
-		return source
+		ok = false
+		return minified, ok
 	}
 	defer ts.tree_delete(tree)
 
@@ -31,7 +44,8 @@ minify_html :: proc(source: string) -> string {
 
 	if ts.node_has_error(root) {
 		log.warnf("minify: HTML parse errors, skipping minification")
-		return source
+		ok = false
+		return minified, ok
 	}
 
 	comments: [dynamic]Range
@@ -66,13 +80,19 @@ minify_html :: proc(source: string) -> string {
 
 		if si < len(styles) && u32(i) >= styles[si].start {
 			s := styles[si]
-			// NOTE: minify_css may return either a fresh allocation or the
+			seg := source[i:s.end]
-			// input slice unchanged (on parser failure), so the result is not
+			if minified, mok := minify_css(css_parser, seg); mok {
-			// freed here — matching this package's one-shot allocation style.
+				strings.write_string(&sb, minified)
-			minified := minify_css(source[i:s.end])
+				if len(minified) > 0 {
-			strings.write_string(&sb, minified)
+					last_written = minified[len(minified) - 1]
-			if len(minified) > 0 {
+				}
-				last_written = minified[len(minified) - 1]
+				delete(minified)
+			} else {
+				// CSS minify skipped/failed: keep the style body verbatim.
+				strings.write_string(&sb, seg)
+				if len(seg) > 0 {
+					last_written = seg[len(seg) - 1]
+				}
 			}
 			i = int(s.end)
 			si += 1
@@ -111,7 +131,10 @@ minify_html :: proc(source: string) -> string {
 		}
 	}
 
-	return strings.to_string(sb)
+
+	minified, ok = strings.to_string(sb), true
+
+	return minified, ok
 }
 
 collect_html_ranges :: proc(
@@ -136,10 +159,7 @@ collect_html_ranges :: proc(
 			// child) via the CSS-aware pass.
 			raw, ok := style_raw_text(child)
 			if ok {
-				append(
+				append(styles, Range{start = ts.node_start_byte(raw), end = ts.node_end_byte(raw)})
-					styles,
-					Range{start = ts.node_start_byte(raw), end = ts.node_end_byte(raw)},
-				)
 			}
 		} else if type_str == "script_element" {
 			append(
@@ -210,18 +230,23 @@ is_css_delim :: proc(c: u8) -> bool {
 	return false
 }
 
-minify_css :: proc(source: string) -> string {
+// minify_css collapses insignificant whitespace and strips comments. The caller
-	gc := ts.ensure_parser("css")
+// owns `parser` and passes it in; it may be reused across calls. On success it
-	if gc == nil {
+// returns (owned string, true) and the caller must free the string. On failure
-		return source
+// (nil parser, parse failure, or syntax errors) it returns ("", false) and the
+// caller should keep the original source unchanged.
+minify_css :: proc(parser: ts.Parser, source: string) -> (minified: string, ok: bool) {
+	if parser == nil {
+		return minified, ok
 	}
 
 	source_c := strings.clone_to_cstring(source)
 	defer delete(source_c)
 
-	tree := ts.parser_parse_string(gc.parser, nil, source_c, u32(len(source)))
+	tree := ts.parser_parse_string(parser, nil, source_c, u32(len(source)))
 	if tree == nil {
-		return source
+		ok = false
+		return minified, ok
 	}
 	defer ts.tree_delete(tree)
 
@@ -229,7 +254,8 @@ minify_css :: proc(source: string) -> string {
 
 	if ts.node_has_error(root) {
 		log.warnf("minify: CSS parse errors, skipping minification")
-		return source
+		ok = false
+		return minified, ok
 	}
 
 	comments: [dynamic]Range
@@ -270,8 +296,10 @@ minify_css :: proc(source: string) -> string {
 			// (last_written == 0), a doubled space (last_written == ' ',
 			// which happens when a stripped comment sits between two
 			// whitespace runs), or one adjacent to a delimiter.
-			if last_written != 0 && last_written != ' ' &&
+			if last_written != 0 &&
-			   !is_css_delim(last_written) && !is_css_delim(next) {
+			   last_written != ' ' &&
+			   !is_css_delim(last_written) &&
+			   !is_css_delim(next) {
 				strings.write_byte(&sb, ' ')
 				last_written = ' '
 			}
@@ -283,7 +311,8 @@ minify_css :: proc(source: string) -> string {
 		}
 	}
 
-	return strings.to_string(sb)
+	minified, ok = strings.to_string(sb), true
+	return minified, ok
 }
 
 collect_css_comments :: proc(node: ts.Node, comments: ^[dynamic]Range) {

minify_test.odin

@@ -1,59 +1,49 @@
 #+test
 package main
 
+import ts "treesitter"
+
 import "core:strings"
-import "core:sync"
 import "core:testing"
-import "treesitter"
 
-// The treesitter parser store is normally initialized by main() via
+// The treesitter registry is thread-safe and self-initializing (grammar() lazily
-// treesitter.init_persistent(); the test runner never calls main. init_once
+// binds it to the heap), and parsers are caller-owned. Each test opens its own
-// ensures it happens exactly once no matter which test thread arrives first,
+// parser(s) via ts.open_parser, passes them to minify_*, and frees the returned
-// and preloads the builtin parsers on that single thread — after which the
+// string — so the multi-threaded test runner needs no shared state or locking.
-// concurrent tests only *read* the cache (the unlocked writes in ensure_parser
+
-// are what corrupt it, not reads).
 @(private = "file")
-init_once: sync.Once
+open_parsers :: proc(t: ^testing.T) -> (html_parser, css_parser: ts.Parser) {
-
+	hg, hok := ts.grammar("html")
-setup_treesitter :: proc() {
+	cg, cok := ts.grammar("css")
-	sync.once_do(&init_once, proc() {
+	testing.expect(t, hok, "html grammar should load")
-		treesitter.init_persistent()
+	testing.expect(t, cok, "css grammar should load")
-		treesitter.ensure_parser("html")
+	return ts.open_parser(hg), ts.open_parser(cg)
-		treesitter.ensure_parser("css")
-	})
 }
 
-// The builtin parsers are cached and shared per-language, and tree-sitter
-// parsers are not reentrant — concurrent parser_parse_string corrupts their
-// internal subtree pool. The multi-threaded test runner would otherwise race,
-// so serialize parser use behind this mutex. (Production renders on a single
-// thread, so it needs no such lock.)
 @(private = "file")
-parse_mu: sync.Mutex
+expect_css :: proc(t: ^testing.T, parser: ts.Parser, input, want: string) {
-
+	got, ok := minify_css(parser, input)
-ts_minify_html :: proc(s: string) -> string {
+	testing.expect(t, ok)
-	sync.mutex_lock(&parse_mu)
+	defer delete(got)
-	defer sync.mutex_unlock(&parse_mu)
+	testing.expect_value(t, got, want)
-	return minify_html(s)
-}
-
-ts_minify_css :: proc(s: string) -> string {
-	sync.mutex_lock(&parse_mu)
-	defer sync.mutex_unlock(&parse_mu)
-	return minify_css(s)
 }
 
 // --- minify_html: inline <style> bodies get CSS-minified ---
 
 @(test)
 test_minify_html_inline_style :: proc(t: ^testing.T) {
-	setup_treesitter()
+	hp, cp := open_parsers(t)
+	defer ts.parser_delete(hp)
+	defer ts.parser_delete(cp)
+
 	input := `<html><head><style>
 body {
   color:  red;
 }
 </style></head><body><p>hi</p></body></html>`
-	result := ts_minify_html(input)
+	result, ok := minify_html(hp, cp, input)
+	testing.expect(t, ok)
+	defer delete(result)
 
 	// The CSS body is collapsed by minify_css (delimiters lose surrounding
 	// whitespace)...
@@ -65,10 +55,15 @@ body {
 
 @(test)
 test_minify_html_empty_style :: proc(t: ^testing.T) {
-	setup_treesitter()
+	hp, cp := open_parsers(t)
+	defer ts.parser_delete(hp)
+	defer ts.parser_delete(cp)
+
 	// An empty <style></style> has no raw_text child; must not crash.
 	input := `<html><head><style></style></head><body></body></html>`
-	result := ts_minify_html(input)
+	result, ok := minify_html(hp, cp, input)
+	testing.expect(t, ok)
+	defer delete(result)
 	testing.expect(t, strings.contains(result, "<style></style>"))
 }
 
@@ -76,19 +71,28 @@ test_minify_html_empty_style :: proc(t: ^testing.T) {
 
 @(test)
 test_minify_css_leading_and_comment :: proc(t: ^testing.T) {
-	setup_treesitter()
+	cg, cok := ts.grammar("css")
+	testing.expect(t, cok, "css grammar should load")
+	cp := ts.open_parser(cg)
+	defer ts.parser_delete(cp)
+
 	// Leading whitespace and a comment between two whitespace runs must not
 	// leave a leading space or a doubled space.
-	testing.expect_value(t, ts_minify_css("\n/* c */\na {}"), "a{}")
+	expect_css(t, cp, "\n/* c */\na {}", "a{}")
-	testing.expect_value(t, ts_minify_css("  a {}"), "a{}")
+	expect_css(t, cp, "  a {}", "a{}")
-	testing.expect_value(t, ts_minify_css("a {}\n/* trailing */"), "a{}")
+	expect_css(t, cp, "a {}\n/* trailing */", "a{}")
 }
 
 @(test)
 test_minify_html_script_preserved :: proc(t: ^testing.T) {
-	setup_treesitter()
+	hp, cp := open_parsers(t)
+	defer ts.parser_delete(hp)
+	defer ts.parser_delete(cp)
+
 	// <script> stays byte-for-byte (whitespace-significant JS).
 	input := "<html><body><script>\nconst s = \"a    b\";\n</script></body></html>"
-	result := ts_minify_html(input)
+	result, ok := minify_html(hp, cp, input)
+	testing.expect(t, ok)
+	defer delete(result)
 	testing.expect(t, strings.contains(result, "const s = \"a    b\";"))
 }

mustache/pipes.odin

@@ -347,7 +347,8 @@ apply_filter :: proc(
 		span := filter.end_pos - filter.op_pos
 		return take(value, filter.args[:], false, tmpl, filter.op_pos, span)
 	case .Rel_Url:
-		str, ok := reflect.as_string(value)
+		bv, _ := base_value(value)
+		str, ok := reflect.as_string(bv)
 		if !ok {
 			return value, Error_Body {
 				msg = "rel_url may only be used on strings",
@@ -617,3 +618,4 @@ take :: proc(
 		return any{new_clone(str[start:end], context.temp_allocator), typeid_of(string)}, warning
 	}
 }
+

render.odin

@@ -3,6 +3,7 @@ package main
 import "mustache"
 
 import diags "diagnostics"
+import ts "treesitter"
 
 import "core:encoding/json"
 import "core:fmt"
@@ -205,6 +206,24 @@ render_site :: proc(site: ^Site) {
 		}
 	}
 
+	// Parsers for minification, created once and reused across every page
+	// (render is single-threaded). Owned here; freed when render_site returns.
+	html_parser, css_parser: ts.Parser
+	if .Minify in site.features {
+		if g, gok := ts.grammar("html"); gok {
+			html_parser = ts.open_parser(g)
+		}
+		if g, gok := ts.grammar("css"); gok {
+			css_parser = ts.open_parser(g)
+		}
+	}
+	defer if html_parser != nil {
+		ts.parser_delete(html_parser)
+	}
+	defer if css_parser != nil {
+		ts.parser_delete(css_parser)
+	}
+
 	// Render individual content pages (skip all index pages)
 	for page in pages {
 		if page._is_index {
@@ -213,7 +232,9 @@ render_site :: proc(site: ^Site) {
 		tpl := get_template(&site.vfs, page.layout, &template_cache)
 		html := render_page_html(page, site, tpl, partials, ctx, &errors)
 		if .Minify in site.features {
-			html = minify_html(html)
+			if m, mok := minify_html(html_parser, css_parser, html); mok {
+				html = m
+			}
 		}
 		write_page(site.output_dir, page.permalink, html)
 	}
@@ -243,7 +264,9 @@ render_site :: proc(site: ^Site) {
 			&errors,
 		)
 		if .Minify in site.features {
-			html = minify_html(html)
+			if m, mok := minify_html(html_parser, css_parser, html); mok {
+				html = m
+			}
 		}
 		write_page(site.output_dir, fmt.aprintf("/%s/", section), html)
 	}
@@ -253,7 +276,9 @@ render_site :: proc(site: ^Site) {
 		home_tpl := get_template(&site.vfs, "home", &template_cache)
 		home_html := render_home_html(home, site, home_tpl, partials, ctx, &errors)
 		if .Minify in site.features {
-			home_html = minify_html(home_html)
+			if m, mok := minify_html(html_parser, css_parser, home_html); mok {
+				home_html = m
+			}
 		}
 		write_file(fmt.tprintf("%s/index.html", site.output_dir), home_html)
 	}
@@ -445,4 +470,3 @@ write_file :: proc(path: string, html: string) {
 		log.errorf("cannot write %s: %v", path, err)
 	}
 }
-

site/assets/css/main.css

@@ -0,0 +1,28 @@
+body>header {
+  background: var(--pico-primary-background);
+  position: sticky;
+  top: 0;
+
+  nav>ul li a {
+    color: var(--pico-primary-inverse);
+  }
+}
+
+/* TODO: Should probaby use a container for the page content like Odin does */
+.toc {
+  position: fixed;
+  height: calc(100vh - 116px);
+  width: 280px;
+  top: 116px;
+  left: 0;
+  overflow-y: auto;
+
+  & ul {
+    flex-direction: column;
+  }
+}
+
+main>article {
+  width: 60%;
+  margin: auto;
+}

site/content/docs.md

@@ -17,6 +17,8 @@ etc.
 TODO: Use consistent spacing with mustache tags
 -->
 
+[TOC]
+
 ## Introduction
 
 This guide assumes you have either read [The Guide](../guide), or have built a [Hugo](https://gohugo.io) site before. It also assumes you have a basic knowledge of HTML and CSS.
@@ -48,12 +50,36 @@ The highlight extension enables server-side syntax highlighting, powered by Tree
 
 If enabled, `minify` will perform simple whitespace removal on all your output `.html` files, and any `.css` files in your `assets` directories. Minifying JavaScript is not supported (yet).
 
-TODO: Do we minify inline css?
+Inline CSS will be minified.
 
 #### Sections
 
 The `sections` extension wraps each of your content sections in a `<section>` block. Sections are opened just before each heading, and closed before the next heading, or the end the document, whichever comes first. 
 
+```markdown
+<!-- Example -->
+## First Section
+
+content
+
+## Second Section
+
+content
+```
+
+Becomes:
+
+```html
+<section>
+  <h2>First Section</h2>
+  <p>content</p>
+</section>
+<section>
+  <h2>Second Section</h2>
+  <p>content</p>
+</section>
+```
+
 ### Opt-Out Features
 
 - emoji
@@ -272,6 +298,14 @@ When building your page(s), the following keys are accessible to your template f
 
 : The HTML rendered page content 
 
+`page.url`
+
+: The full url to the page. .e.g. `https://example.com/about`
+
+`page.rel_url`
+
+: The url to the page, minus the `site.base_url`, e.g. `/about`.
+
 TODO: write
 
 #### Site
@@ -366,6 +400,15 @@ To get around this, Thor extends mustache to include a handful of data filters i
 
 : Used to display a date in a particular format. See [DateTimes](#datetimes). If no format is given, the default will be used. 
 
+`rel_url`
+
+: Should only be used by theme designers or when building a site where the homepage is not the website root (i.e. https://example.com/my-cool-site/).
+Will take a url like `/about` and convert it to `https://example.com/my-cool-site/about`
+
+`tool`
+
+: Allows you to run your assets / content through any external tools you've configured. Most commonly used for css processors like Tailwind or Sass. See [Tools](#tools) for more  info.
+
 #### Chaining Pipes
 
 Thor allows you to chain two or more pipes, to allow complex data manipulation. However you are limited to no more than **8 pipes**[^pipes] for any given tag. If you believe you need more than 8 pipes, please [open an issue](../issues) with a **clear** and **concrete** example of the problem you are facing.
@@ -552,14 +595,73 @@ The following keys are available:
 
 Menu configuration for the site. Note that (unlike Hugo) if your configuration contains this key, frontmatter `menus` will be ignored. See [menus](#menus) for more info. 
 
+`tools`
 
+Allows you to configure external content/asset processing tools. Too... See [Tools](#tools) for more info.
 
+## Tools
+
+Configuring tools will allow you to create custom pipelines for your content and assets. For example, you could create a tailwindcss tool to pre-process your css files. You could also create a conversion tool to make all your images be a certain format or maximum size.
+
+Tools are configured in your config file and utilized in your template files:
+
+> [!DANGER]: You should never run a tool you have not configured yourself, as running arbetrary commands can be extremely dangerous.
+
+```mustache
+{{!layouts/partials/styles.css}}
+{{#params.stylesheets}}<link rel="stylesheet" href="{{. | tool tailwind}}">
+{{/params.stylesheets}}
+```
+
+Essentially, each tool is just a bash-style commandline string with a few special variables:
+
+<!-- TODO: It might be a terrible idea to combine mustache templates with shell scripts :/ -->
+
+```json5
+// thor.json
+{
+	"tools": {
+		// For demonstration purposes only, not recommended (see below).
+	    "tailwind": "tailwindcss -i {{in}} -o {{out}}",
+	}
+}
+```
+
+Both `{{in}}` and `{{out}}` exist to support tools that can't read  from `stdin` and/or can't write to `stdout`. If your tool can do those things, it is ***highly recommended*** to avoid `{{in}}` + `{{out}}`, as each requires thor to make **a full round-trip** to the file system, rather than keeping everything in memory.
+
+```json5
+// thor.json
+{
+	"tools": {
+		// The recommended way to configure Tailwind 4.3.3
+	    "tailwind": "tailwindcss {{#minify}}--minify{{/minify}} -i -",
+	}
+}
+```
+
+### Special Variables
+
+`in`
+
+:  The path to a file created by thor, containing the content that will be processed by your tool.
+
+`out`
+
+: The path to a file, created by your tool,  that will be represents the file output by the tool, that will be ingested by thor.
+
+`minify`
+
+: True if the `minify` config option has been set, or `--production` flag has been passed.
+
+- it is up to you to have the necessary binaries installed - thor won't do it for you.
+- every tool you want to use must be configured - there is no default <!-- TODO: Should there be? Perhaps we could include support for them in a future, interactive site builder setup TUI. -->
+ 
 ## Open Graph
 
 Thor supports [Open Graph](https://ogp.me/) tags, allowing you to customize how your site looks when linked to on social media or in apps like Discord. It is accessible through the `{{>opengraph}}` template, which should be automatically included in your header by your theme.
 
 ```html
-TODO: {{>opengraph shorcode here}}
+TODO: {{opengraph shorcode here}}
 ```
 
 The options are sourced [using the context stack], first from your page frontmatter, and then from your site config. Thor will attempt to infer as much information as possible, to keep you from needing to set them explictly.
@@ -591,9 +693,11 @@ TODO: Modules
 
 TODO: Themes
 
-## Zen
 
-> The guding principals behind Thor.
+<hgroup>
+  <h2 id="zen">Zen</h2>
+  <p>The guding principals behind Thor.</p>
+</hgroup>
 
 Simpler is Better
 

site/content/guide.md

@@ -17,7 +17,7 @@ At the heart of any good SSG is the content files. Thor supports markdown (`.md`
 
 Let's make a home page - copy the following text and place it into `./index.md`
 
-```md
+```markdown
 # Hello, World! 
 
 Welcome to your new site. 

site/thor.json

@@ -4,11 +4,27 @@
     "format": "January 2, 2006"
   },
   "markdown_extensions": {
-    "footnotes": true
+    "footnotes": true,
+    "highlight": true
+  },
+  "menus": {
+    "main": [
+      {
+        "name": "GitHub",
+        "url": "https://github.com/sbrow/thor",
+        "weight": 20
+      }
+    ]
   },
   "params": {
     "author": {
       "name": "Spencer Brower"
-    }
+    },
-  }
+    "stylesheets": [
+      "https://cdn.jsdelivr.net/npm/@picocss/pico@2/css/pico.blue.min.css",
+      "/css/main.css"
+    ]
+  },
+  "grammars": "~/.config/helix/runtime/grammars/",
+  "queries": "/nix/store/n9da8d007ygbgsx983jr3ar3wb1fsh6q-helix-25.07.1/lib/runtime/queries"
 }

treesitter/treesitter.odin

@@ -7,6 +7,7 @@ import "core:mem"
 import "core:os"
 import "core:strings"
 import "core:sync"
+import si "core:sys/info"
 import "core:thread"
 
 grammar_dir: string
@@ -120,17 +121,15 @@ foreign css_grammar {
 	tree_sitter_css :: proc() -> Language ---
 }
 
-// THREAD-SAFETY: a Grammar_Cache bundles shareable tree-sitter objects
+// A Grammar bundles only the immutable, shareable tree-sitter objects for a
-// (`language`, `query` — immutable, safe to read from many threads) with
+// language: `language` (from tree_sitter_x() or a dlopen'd .so) and `query` (the
-// per-thread ones (`parser`, `cursor` — a TSParser mutates its internal
+// compiled highlight query). Both are immutable after construction and safe to
-// subtree pool during a parse, and a TSQueryCursor is likewise stateful).
+// read from any number of threads. The stateful, single-thread-use objects —
-// A cached Grammar_Cache is therefore SINGLE-THREADED-USE: exactly one thread
+// parser and query cursor — are deliberately NOT stored here; the caller creates
-// may drive `parser`/`cursor` at a time. See GRAMMAR_CACHE_THREADING.md.
+// and owns those (see `open_parser`). See GRAMMAR_CACHE_THREADING.md.
-Grammar_Cache :: struct {
+Grammar :: struct {
 	language:     Language,
-	parser:       Parser,
 	query:        Query,
-	cursor:       Query_Cursor,
 	query_failed: bool,
 }
 
@@ -138,31 +137,98 @@ Get_Language_Proc :: #type proc() -> Language
 
 SPALL :: #config(SPALL, false)
 
-grammar_store: Grammar_Store
+// registry is the process-lifetime cache of immutable Grammars, keyed by language
-
+// name. It is the library's ONLY shared mutable state; `mu` serializes all access.
-// cache_mutex guards writes to grammar_store.cache. NOTE: today it is only
+// A nil value memoizes "tried and unavailable" so misses don't re-dlopen. Parsers
-// taken on the parallel preload path (grammar_worker), NOT in ensure_parser /
+// live in the caller, so the registry holds nothing single-thread-use.
-// load_grammar. Those lazy paths are intentionally unlocked because production
+Grammar_Registry :: struct {
-// only ever calls them single-threaded. See GRAMMAR_CACHE_THREADING.md before
+	mu:        sync.Mutex,
-// relying on this cache from multiple threads.
+	grammars:  map[string]^Grammar,
-cache_mutex: sync.Mutex
-
-Grammar_Store :: struct {
-	cache:     map[string]^Grammar_Cache,
 	allocator: mem.Allocator,
 }
 
-// KNOWN ISSUE (allocator lifetime): grammar_store is a process-lifetime global,
+registry: Grammar_Registry
-// but this captures whatever `context.allocator` is live at call time. In main
+
-// that is deliberately the heap (init_persistent runs before the site arena is
+// init_persistent is optional — grammar() lazily initializes the registry on first
-// installed), so it works. But it is fragile: called from any other context
+// use — but callers (main) may call it to bind the cache early. The allocator is
-// (e.g. a per-test tracking allocator, or lazily once the site arena is active)
+// pinned to the OS heap, NOT context.allocator, so the cache survives any transient
-// it would bind the cache to a transient allocator and leave it dangling once
+// allocator (per-build arena, per-test tracking) that happens to be live.
-// that allocator is torn down. The intended fix is to pin the heap allocator
-// explicitly (runtime.heap_allocator()). See GRAMMAR_CACHE_THREADING.md.
 init_persistent :: proc() {
-	grammar_store.allocator = context.allocator
+	sync.mutex_lock(&registry.mu)
-	grammar_store.cache = make(map[string]^Grammar_Cache, grammar_store.allocator)
+	defer sync.mutex_unlock(&registry.mu)
+	ensure_registry()
+}
+
+// ensure_registry binds the registry to the heap on first use. Caller must hold mu.
+@(private)
+ensure_registry :: proc() {
+	if registry.grammars == nil {
+		registry.allocator = os.heap_allocator()
+		registry.grammars = make(map[string]^Grammar, registry.allocator)
+	}
+}
+
+// grammar returns the immutable Grammar for `lang`, loading it on first use.
+// Thread-safe: concurrent callers serialize on `mu`. Returns (nil, false) when the
+// grammar cannot be loaded. The returned pointer is immutable — callers may cache
+// and read it (its `language`/`query`) without any further locking.
+grammar :: proc(lang: string) -> (^Grammar, bool) {
+	sync.mutex_lock(&registry.mu)
+	defer sync.mutex_unlock(&registry.mu)
+	ensure_registry()
+	if g, seen := registry.grammars[lang]; seen {
+		return g, g != nil
+	}
+	g := build_grammar(lang)
+	registry.grammars[lang] = g
+	return g, g != nil
+}
+
+// build_grammar loads a language and compiles its highlight query. It touches no
+// shared mutable state — only the set-once `registry.allocator` (for the heap
+// `new`), read-only globals, and the per-thread temp allocator — so it is safe to
+// run OFF the lock and concurrently, provided `ensure_registry` has already run
+// (see preload_grammars). Returns nil if the language itself cannot be loaded; a
+// grammar whose query is absent/failed is still returned (parsing works,
+// highlighting does not).
+@(private)
+build_grammar :: proc(lang: string) -> ^Grammar {
+	language, ok := load_language(lang)
+	if !ok {
+		return nil
+	}
+	g := new(Grammar, registry.allocator)
+	g.language = language
+	if query, qok := compile_query(lang, language); qok {
+		g.query = query
+	} else {
+		g.query_failed = true
+	}
+	return g
+}
+
+// open_parser creates a fresh parser bound to the grammar's language. The CALLER
+// owns the returned parser and must free it with parser_delete. This is the seam
+// for parser lifetime: today callers open one and reuse it across a batch of
+// parses on a single thread (render opens an html+css pair per render_site;
+// assets opens one per copy_assets_dir), which is safe because nothing is shared.
+// A future pooling or thread-local strategy can be dropped in here without
+// touching any call site. Returns nil on failure.
+open_parser :: proc(g: ^Grammar) -> Parser {
+	if g == nil {
+		return nil
+	}
+	parser := parser_new()
+	if parser == nil {
+		log.errorf("treesitter: cannot create parser")
+		return nil
+	}
+	if !parser_set_language(parser, g.language) {
+		log.errorf("treesitter: ABI mismatch setting parser language")
+		parser_delete(parser)
+		return nil
+	}
+	return parser
 }
 
 when SPALL {
@@ -173,6 +239,19 @@ when SPALL {
 		_thread_init = init
 		_thread_cleanup = cleanup
 	}
+
+	// Pool-shaped adapters: the thread pool calls these once per worker thread at
+	// startup/shutdown, so each worker's parses land in their own SPALL thread.
+	spall_pool_thread_init :: proc(t: ^thread.Thread, data: rawptr) {
+		if _thread_init != nil {
+			_thread_init()
+		}
+	}
+	spall_pool_thread_fini :: proc(t: ^thread.Thread, data: rawptr) {
+		if _thread_cleanup != nil {
+			_thread_cleanup()
+		}
+	}
 }
 
 builtin_language :: proc(lang: string) -> (language: Language, ok: bool) {
@@ -190,7 +269,7 @@ builtin_language :: proc(lang: string) -> (language: Language, ok: bool) {
 // load_query returns the highlight query source for a language. Builtin
 // languages (html/css) are baked into the binary via `#load`; all others are
 // read from the runtime `query_dir`. `path` is the on-disk location for
-// diagnostics ("(builtin)" for embedded queries). Mirrors `ensure_parser`.
+// diagnostics ("(builtin)" for embedded queries). Mirrors `load_language`.
 load_query :: proc(lang: string) -> (src: string, path: string, ok: bool) {
 	switch lang {
 	case "html":
@@ -239,50 +318,7 @@ load_language :: proc(lang: string) -> (language: Language, ok: bool) {
 	return
 }
 
-// THREAD-SAFETY: ensure_parser is NOT safe to call concurrently. It reads and
+compile_query :: proc(lang: string, language: Language) -> (query: Query, ok: bool) {
-// writes grammar_store.cache without taking cache_mutex, and it returns a shared
-// per-language Grammar_Cache whose `parser` is not reentrant. It is written for
-// single-threaded lazy use (minify, syntax highlighting). Callers that need
-// parsing on multiple threads must serialize with a lock or give each thread
-// its own parser. See GRAMMAR_CACHE_THREADING.md.
-ensure_parser :: proc(lang: string) -> ^Grammar_Cache {
-	if cached, ok := grammar_store.cache[lang]; ok {
-		return cached
-	}
-
-	grammar_store.cache[lang] = nil
-
-	language, ok := load_language(lang)
-	if !ok {
-		return nil
-	}
-
-	parser := parser_new()
-	if parser == nil {
-		log.errorf("treesitter: cannot create parser for %s", lang)
-		return nil
-	}
-	if !parser_set_language(parser, language) {
-		log.errorf("treesitter: ABI mismatch for %s grammar", lang)
-		parser_delete(parser)
-		return nil
-	}
-
-	gc := new(Grammar_Cache, grammar_store.allocator)
-	gc.language = language
-	gc.parser = parser
-	grammar_store.cache[lang] = gc
-	return gc
-}
-
-compile_query :: proc(
-	lang: string,
-	language: Language,
-) -> (
-	query: Query,
-	cursor: Query_Cursor,
-	ok: bool,
-) {
 	query_src, query_path, qok := load_query(lang)
 	if !qok {
 		return
@@ -344,113 +380,104 @@ compile_query :: proc(
 		return
 	}
 
-	cursor = query_cursor_new()
 	ok = true
 	return
 }
 
-load_grammar :: proc(lang: string) -> ^Grammar_Cache {
+// PRELOAD_MAX_WORKERS caps the pool when the CPU count is unknown or absurd.
-	gc := ensure_parser(lang)
+PRELOAD_MAX_WORKERS :: 4
-	if gc == nil {
-		return nil
-	}
-	if gc.query != nil {
-		return gc
-	}
-	if gc.query_failed {
-		return nil
-	}
 
-	query, cursor, ok := compile_query(lang, gc.language)
+// Preload_Task is the per-language work item handed to a pool worker.
-	if !ok {
+Preload_Task :: struct {
-		gc.query_failed = true
+	lang: string,
-		return nil
-	}
-
-	gc.query = query
-	gc.cursor = cursor
-	return gc
 }
 
-preload_grammar :: proc(lang: string) -> ^Grammar_Cache {
+// preload_worker builds one grammar off the lock (dlopen + query compile — the
-	language, ok := load_language(lang)
+// expensive, parallelizable part), then takes `mu` only to publish the finished
-	if !ok {
+// pointer. Publishing nil memoizes a failed load, matching grammar()/build_grammar.
-		return nil
+@(private)
-	}
+preload_worker :: proc(task: thread.Task) {
-
+	pt := cast(^Preload_Task)task.data
-	parser := parser_new()
+	g := build_grammar(pt.lang)
-	if parser == nil {
+	sync.mutex_lock(&registry.mu)
-		log.errorf("treesitter: cannot create parser for %s", lang)
+	registry.grammars[pt.lang] = g
-		return nil
+	sync.mutex_unlock(&registry.mu)
-	}
-	if !parser_set_language(parser, language) {
-		log.errorf("treesitter: ABI mismatch for %s grammar", lang)
-		parser_delete(parser)
-		return nil
-	}
-
-	gc := new(Grammar_Cache, grammar_store.allocator)
-	gc.language = language
-	gc.parser = parser
-
-	query, cursor, qok := compile_query(lang, language)
-	if !qok {
-		gc.query_failed = true
-		return gc
-	}
-
-	gc.query = query
-	gc.cursor = cursor
-	return gc
 }
 
+// preload_grammars warms the registry for the given languages ahead of the render
+// loop, so the first page that needs one doesn't pay the load cost inline. The
+// expensive per-language work (dlopen + compile_query) runs concurrently on a
+// bounded thread pool. Builtins are always available lazily and are skipped.
+//
+// INVARIANT: preload runs to completion before any concurrent lazy grammar() use
+// (render is single-threaded and runs afterward), so the registry is never read
+// while workers publish. Workers still take `mu` for the map insert, keeping the
+// "registry is only ever mutated under mu" discipline uniform with grammar().
 preload_grammars :: proc(languages: []string) {
-	if len(languages) == 0 {
+	// Snapshot (main thread, under mu): pin the registry allocator before any
-		return
+	// worker reads it, and collect the languages still needing a load. Dedupes
-	}
+	// against builtins and already-cached grammars (watch-mode reuse).
-
+	sync.mutex_lock(&registry.mu)
-	// Filter out already-loaded languages (watch mode reuse)
+	ensure_registry()
 	to_load := make([dynamic]string, 0, len(languages), context.temp_allocator)
 	for lang in languages {
-		if cached, ok := grammar_store.cache[lang]; ok && cached != nil {
+		if _, bok := builtin_language(lang); bok {
 			continue
 		}
-		if _, bok := builtin_language(lang); bok {
+		if _, seen := registry.grammars[lang]; seen {
 			continue
 		}
 		append(&to_load, lang)
 	}
+	sync.mutex_unlock(&registry.mu)
 
 	if len(to_load) == 0 {
 		return
 	}
 
-	threads := make([]^thread.Thread, len(to_load), context.temp_allocator)
+	// Bind the pool to the logical core count (load is dlopen/CPU-bound), but
-	for i in 0 ..< len(to_load) {
+	// never more workers than tasks, never fewer than one.
-		threads[i] = thread.create_and_start_with_poly_data(to_load[i], grammar_worker)
+	worker_count: int = ---
+	if _, logical, ok := si.cpu_core_count(); ok {
+		assert(logical > 0)
+		worker_count = min(logical, len(to_load))
+	} else {
+		worker_count = min(PRELOAD_MAX_WORKERS, len(to_load))
 	}
-	for t in threads {
-		thread.join(t)
-		thread.destroy(t)
-	}
-}
 
-grammar_worker :: proc(lang: string) {
+	// Serial fast path: a single worker gains nothing from the pool machinery.
+	if worker_count == 1 {
+		for lang in to_load {
+			g := build_grammar(lang)
+			sync.mutex_lock(&registry.mu)
+			registry.grammars[lang] = g
+			sync.mutex_unlock(&registry.mu)
+		}
+		return
+	}
+
+	// The pool appends to its own `tasks_done` from worker threads, so its
+	// bookkeeping allocator MUST be thread-safe — use the heap, not the (possibly
+	// arena) context allocator that is live during a build.
+	pool: thread.Pool
+	init_proc: thread.Thread_Init_Proc = nil
+	fini_proc: thread.Thread_Init_Proc = nil
 	when SPALL {
-		if _thread_init != nil {
+		init_proc = spall_pool_thread_init
-			_thread_init()
+		fini_proc = spall_pool_thread_fini
-		}
-		defer if _thread_cleanup != nil {
-			_thread_cleanup()
-		}
 	}
+	thread.pool_init(&pool, os.heap_allocator(), worker_count, init_proc, nil, fini_proc, nil)
+	defer thread.pool_destroy(&pool)
 
-	gc := preload_grammar(lang)
+	tasks := make([]Preload_Task, len(to_load), context.temp_allocator)
-	if gc != nil {
+	thread.pool_start(&pool)
-		sync.mutex_lock(&cache_mutex)
+	for lang, i in to_load {
-		grammar_store.cache[lang] = gc
+		tasks[i].lang = lang
-		sync.mutex_unlock(&cache_mutex)
+		// Per-task context allocator = heap, so any stray context.allocator use
+		// inside a worker is thread-safe.
+		thread.pool_add_task(&pool, os.heap_allocator(), preload_worker, &tasks[i], i)
 	}
+	thread.pool_finish(&pool) // barrier: all workers joined before we return
 }
 
 extract_query_token :: proc(src: []byte, offset: u32) -> string {
@@ -485,3 +512,4 @@ helix_version_from_path :: proc(path: string) -> string {
 	if end <= start do return ""
 	return path[start:end]
 }
+