Compare commits

...

10 Commits

Author SHA1 Message Date
Spencer Brower c7b42ce7aa refactor: Simplified content.odin. 2026-07-16 18:08:09 -04:00
Spencer Brower a4b16e1b0a chore: Updated AGENTS.md 2026-07-16 17:27:41 -04:00
Spencer Brower f70205876f fix: Removed need for has_date in templates. 2026-07-16 17:25:19 -04:00
Spencer Brower f78e3811e4 feat!: Added the ability to disable features / extensions.
BREAKING CHANGE: `sectionate` has been moved to `markdown_extensions.sections`
in the config.
2026-07-16 17:25:19 -04:00
Spencer Brower 3898925228 chore: Updated AGENTS.md. 2026-07-16 14:48:31 -04:00
Spencer Brower 323a1318cf perf: Added spall support. 2026-07-16 14:40:13 -04:00
Spencer Brower 06a3c4d82d fix: Fixed leaks. 2026-07-16 14:22:46 -04:00
Spencer Brower eb452b25fc fix(mustache): Renamed effective to base_value. 2026-07-16 13:42:15 -04:00
Spencer Brower 7e93c3468f fix(mustache): Got whitespace management to pass the tests. 2026-07-16 13:42:15 -04:00
Spencer Brower 1f44adcf33 chore: Updated TODOs. 2026-07-15 18:29:00 -04:00
19 changed files with 1105 additions and 724 deletions
+2
View File
@@ -1,3 +1,5 @@
.direnv
result
thor
*.spall
thor-prof
+144 -49
View File
@@ -5,10 +5,10 @@ Thor is a static site generator written in [Odin](https://odin-lang.org), replac
## Architecture
```
thor.json ← site config (title, base_url, author, params, sectionate)
thor.json ← site config (title, base_url, author, params)
content/ ← markdown and HTML content files
layouts/ ← Mustache templates + partials (including icons)
assets/ ← CSS (Tufte-based, no build step) and JS
assets/ ← CSS (Tufte-based), JS, fonts, images — copied/processed to public/
public/ ← build output (generated)
```
@@ -16,42 +16,71 @@ public/ ← build output (generated)
| File | Responsibility |
|---|---|
| `main.odin` | Entry point. Sets `context.logger`, calls `init_site`, `walk_content`, `render_site` |
| `site.odin` | `Site` struct (config + arena), `init_site`, `load_site_config`, `site_merge`, `site_allocator`, `destroy_site` |
| `main.odin` | Entry point. Sets `context.logger`, calls `init_site`, `walk_content`, `render_site`. Optional Spall profiling via `SPALL` config flag. |
| `site.odin` | `Flags` (CLI args), `Config_File` (from `thor.json`), `Site` (runtime state + arena), `Feature` + `Markdown_Extension` bit_set enums, `DEFAULT_MARKDOWN_EXTENSIONS`, 5-step `init_site` (defaults → flags → config → apply config → apply flags), `load_config_file`, `apply_config`, `apply_cli_flags`, `parse_extension_list`, `find_config` |
| `frontmatter.odin` | JSON frontmatter parser (`{ }` delimited) |
| `content.odin` | `Page` struct, content walker, page loader, cmark integration, full markdown pipeline |
| `content.odin` | `Page` struct, content walker, page loader, cmark integration, full markdown pipeline, `copy_assets_dir` (recursive copy with CSS minification) |
| `footnotes.odin` | Note definition stripping (pre-cmark) + sidenote/marginnote injection (post-cmark) |
| `alerts.odin` | GitHub alert post-processor (`> [!CAUTION]` → styled blockquote) |
| `emoji.odin` | Emoji shortcode expander (`:shrug:``¯\_(ツ)_/¯`), post-cmark |
| `highlight.odin` | Post-cmark Tree-sitter syntax highlighter; loads grammars/queries from Helix, caches loaded grammars, reports syntax errors with file/line |
| `tree_sitter.odin` | C FFI bindings for Tree-sitter (TSParser, TSQuery, TSQueryCursor, node traversal, dlopen/dlsym) |
| `highlight.odin` | Post-cmark Tree-sitter syntax highlighter; statically links HTML/CSS grammars, dlopen for others; caches loaded grammars, reports syntax errors |
| `tree_sitter.odin` | C FFI bindings for Tree-sitter (TSParser, TSQuery, TSQueryCursor, node traversal, dlopen/dlsym). Statically links `tree-sitter-html` and `tree-sitter-css` via Nix. |
| `sectionate.odin` | `wrap_sections` proc — splits HTML at `<h2` into `<section>` wrappers |
| `render.odin` | Mustache template rendering, all page types, RSS, sitemap, robots.txt, `load_partials` recursive scan |
| `feed.odin` | RSS feed + sitemap XML generation |
| `mustache/` | Vendored [odin-mustache](https://github.com/benjamindblock/odin-mustache) library |
| `render.odin` | Template rendering pipeline: pre-parses templates/partials once, struct-based data model (`Base_Data`/`Page_Data`/`Home_Data`/`Posts_Data`), `mustache.render()`, minification gate, RSS, sitemap, robots.txt |
| `minify.odin` | Tree-sitter-based HTML and CSS minification (`minify_html`, `minify_css`). Strips comments, collapses whitespace, removes inter-tag spaces. Preserves `<pre>`/`<code>`/`<script>`/`<style>` content. |
| `feed.odin` | RSS feed + sitemap XML generation. Uses `strings.Builder`. `format_rfc822` uses `core:time` for date parsing. |
| `mustache/` | Mustache template engine (spec-compliant, custom implementation) |
Icon SVGs live as HTML partials in `layouts/partials/icons/` (home, github, rss, chevron_up, star).
### Data flow
```
thor.json → init_site → Site (config + Dynamic_Arena)
thor.json → find_config → init_site:
1. set defaults (base_url, DEFAULT_MARKDOWN_EXTENSIONS)
2. parse CLI flags (Flags struct)
3. load config file (Config_File from thor.json)
4. apply_config — non-empty config fields override defaults
5. apply_cli_flags — flags override config, -ext/-no-ext adjust extensions
→ Site (config + Dynamic_Arena + features: bit_set + markdown_extensions: bit_set)
content/ → walk_content → []Page (with body_html from pipeline)
layouts/*.html → render_site (Mustache render_in_layout) → public/
layouts/*.html → parse() → mustache.Template (parsed once)
render_site → mustache.render(Page_Data, partials) → optional minify_html → public/
```
### Config system
`Site` has `params: json.Value` for arbitrary user-defined data from `thor.json`. Social links and other template-only data live under `"params"`. `sectionate: bool` controls automatic `<section>` wrapping at `<h2>` boundaries.
Config is split into three structs with a clear 5-step initialization flow:
- **`Flags`** — CLI args only. Parsed by `core:flags`. Includes path overrides, build-mode toggles, and `-ext`/`-no-ext` for markdown extension overrides.
- **`Config_File`** — parsed from `thor.json` via `json.unmarshal_string`. Holds title, paths, `markdown_extensions` (JSON), `params` (JSON).
- **`Site`** — runtime state: arena, resolved config values, `features: bit_set[Feature]`, `markdown_extensions: bit_set[Markdown_Extension]`.
**`Feature` enum** — build-mode toggles: `Drafts`, `Minify`, `Watch`. Checked with `.Minify in site.features`.
**`Markdown_Extension` enum** — content pipeline toggles: `Emoji`, `Sidenotes`, `Alerts`, `Highlight`, `Sections`. Default is `DEFAULT_MARKDOWN_EXTENSIONS` (currently `.Emoji, .Sidenotes, .Alerts`). Configurable via:
- `thor.json`: `"markdown_extensions": { "emoji": true, "highlight": false, ... }`
- CLI: `-ext:highlight,sections` (enable) / `-no-ext:emoji` (disable). Comma-separated, case-insensitive.
**`find_config`** — walks up from CWD looking for `thor.json`. Falls back to `./thor.json`.
Config precedence: `CLI flags > thor.json values > hardcoded defaults`.
```json
{
"title": "...",
"base_url": "...",
"author": "...",
"sectionate": true,
"markdown_extensions": {
"emoji": true,
"sidenotes": true,
"alerts": true,
"highlight": true,
"sections": true
},
"params": {
"social": [
{ "name": "github", "url": "...", "icon": "icons/github" }
@@ -60,37 +89,74 @@ layouts/*.html → render_site (Mustache render_in_layout) → public/
}
```
Templates access params via dotted keys: `{{#params.social}}`, `{{>* icon}}`.
### Template system
Config precedence: `CLI flags > thor.json values > hardcoded defaults`.
Templates use Mustache with template inheritance (`{{<base}}` / `{{$block}}`):
```html
<!-- base.html -->
<body>{{> nav}}{{$content}}{{/content}}{{> footer}}</body>
<!-- post.html -->
{{<base}}
{{$content}}
<main><article><h1>{{page_title}}</h1>{{&body}}</article></main>
{{/content}}
{{/base}}
```
Data is passed as **typed structs** (not `map[string]any`). Mustache resolves struct fields via Odin reflection, including `using`-embedded fields. Date presence is checked via string truthiness (`{{#date_iso}}`) — no separate `has_date` bool needed.
```odin
Base_Data :: struct {
now: datetime.DateTime,
body: string,
title: string,
// ...
}
Page_Data :: struct {
using base: Base_Data, // fields promoted via struct_get fallback
page_title: string,
// ...
}
```
`render_site` pre-parses all templates and partials once (via `mustache.parse`), then reuses them for every page render.
### Markdown pipeline (in content.odin `load_page`)
```
raw markdown
expand_emoji (pre-cmark: :shortcode: → unicode)
→ strip_definitions (pre-cmark: extract [^id]: definitions)
strip_definitions (pre-cmark: extract [^id]: definitions — only if .Sidenotes enabled)
→ cmark markdown_to_html (Unsafe mode for HTML passthrough)
→ expand_emoji (post-cmark: :shortcode: → unicode, avoids cmark escape issues)
→ inject_sidenotes (post-cmark: [^id] → <label><input><span> markup)
→ inject_alerts (post-cmark: [!TYPE] blockquotes → styled alerts)
→ highlight_code (post-cmark: tree-sitter per code block, with error reporting)
→ wrap_sections (post-cmark: if site.sectionate, wraps content in <section> at <h2>)
→ expand_emoji (post-cmark: :shortcode: → unicode — only if .Emoji enabled)
→ inject_notes (post-cmark: [^id] → <label><input><span> markup — only if .Sidenotes enabled)
→ inject_alerts (post-cmark: [!TYPE] blockquotes → styled alerts — only if .Alerts enabled)
→ highlight_code (post-cmark: tree-sitter per code block — only if .Highlight enabled)
→ wrap_sections (post-cmark: wraps content in <section> at <h2> — only if .Sections enabled)
```
`.html` content files skip cmark entirely — body is used as-is.
Each step is gated by `bit_set[Markdown_Extension]`. All `.html` content files skip the pipeline entirely — body is used as-is.
### Syntax highlighting
Build-time highlighting via Tree-sitter C FFI. No client-side JavaScript.
- Grammars loaded via `dlopen` from Helix's compiled `.so` files
- Highlight queries (`.scm`) loaded from Helix's runtime directory
- Paths hardcoded in `tree_sitter.odin` (Nix store paths, Helix-version-dependent)
- Capture names mapped to CSS classes: `keyword``.hl-keyword`, `constant.numeric.integer``.hl-constant-numeric-integer`, etc.
- Atom-one-dark color theme in `main.css`
- Failed grammar loads are cached (no retries) and logged via `log.warnf`
- Syntax errors detected via `ts_node_has_error`, reported with file path and line number relative to code block
- **HTML and CSS grammars** statically linked via Nix (`mkGrammarStaticLib` in `thor/flake.nix`). Always available, no dlopen.
- **Other grammars** (bash, odin, nu, etc.) loaded via `dlopen` from Helix's compiled `.so` files.
- Highlight queries (`.scm`) loaded from Helix's runtime directory.
- Paths hardcoded in `tree_sitter.odin` (Nix store paths, Helix-version-dependent).
- Grammar loading split: `ensure_parser` (parser only, used by minify) vs `load_grammar` (parser + query, used by highlight).
- Capture names mapped to CSS classes: `keyword``.hl-keyword`, etc.
- Atom-one-dark color theme in `main.css`.
### Minification
Optional, enabled with `-minify` flag (`.Minify` in `Feature` bit_set).
- **HTML** — tree-sitter parses output, strips comments, removes inter-tag whitespace, preserves `<pre>`/`<code>`/`<textarea>`/`<script>`/`<style>` content. Applied after template rendering.
- **CSS** — tree-sitter parses `.css` files in `assets/`, strips comments, collapses whitespace, trims around `{};:,`. Applied during `copy_assets_dir`.
- Non-CSS files in `assets/` copied verbatim.
### Memory management
@@ -100,7 +166,18 @@ Build-time highlighting via Tree-sitter C FFI. No client-side JavaScript.
- `site_allocator(site)` returns the arena allocator for callers
- `destroy_site` frees the arena
- `main.odin` sets `context.logger = log.create_console_logger()` — without this, all `log.*` calls are silently dropped
- **Not yet wired:** `context.allocator` is not set to the arena in `main.odin`, so rendering and content processing still use the heap allocator
- `context.allocator` is set to `site_allocator(&site)` in the main loop
### Spall profiling
Optional, compiled out by default. Enabled with `-define:SPALL=true`:
```bash
odin build . -define:SPALL=true -o:speed -out:thor-prof
./thor-prof -drafts # generates thor.spall
```
Uses `core:prof/spall` with `@(instrumentation_enter)`/`@(instrumentation_exit)` hooks — every function auto-instrumented, no manual annotation needed.
## Building
@@ -110,8 +187,7 @@ Build-time highlighting via Tree-sitter C FFI. No client-side JavaScript.
nix develop
# From blog root:
odin run ./thor -- -drafts
cp assets/css/main.css public/css/main.css
cp assets/js/main.js public/js/main.js
# Assets (CSS/JS/fonts) are copied/minified automatically by thor
caddy run # serves public/ on blog.localhost
```
@@ -120,44 +196,63 @@ No CSS build step — `main.css` is static Tufte-based CSS, no preprocessor or c
### Production build
```bash
nix build # runs thor + copies CSS/JS, outputs to ./result/
nix build # runs thor, outputs to ./result/
```
### Tests
```bash
cd thor
odin test . # site + mustache smoke tests
odin test mustache # mustache spec + targeted tests (37 total)
odin test . # site tests (config, frontmatter, footnotes)
odin test . -all-packages # includes mustache spec tests
```
## Vendored mustache patches
## Mustache engine
Four modifications to `mustache/mustache.odin`:
Spec-compliant Mustache implementation at `mustache/`. See `mustache/SPEC.md` for the implementation specification.
1. **`any` unwrapping** in `map_get` and `data_type` — when values come from `map[string]any`, the inner `any` wrapper is unwrapped so type detection works correctly for nested maps and lists.
### Files
2. **Layout partials**`layout_template.partials = tmpl.partials` added so partials (`{{> nav}}`, `{{> footer}}`) work inside the base layout template.
| File | Responsibility |
|---|---|
| `mustache.odin` | Public API (`parse`, `render`, `Template`), parser (`parse_section`), renderer (`render_nodes`), template inheritance (`merge_block_overrides`) |
| `tokenizer.odin` | Tokenizer (template string → `[]Token`), standalone whitespace detection |
| `data.odin` | Reflection-based data model: `effective` (union/distinct peeling), `lookup_in`, `resolve_name`, `is_truthy`, `any_to_string`, `list_info`, `write_value` |
| `spec_test.odin` | JSON spec test runner — loads `spec/specs/*.json`, runs each test case |
3. **Inline partial rendering** — replaced `template_insert_partial` (which injected tokens into the main token list, breaking section iteration) with `template_render_partial` (which lexes the partial, temporarily swaps `tmpl.lexer`, and recursively processes via `template_process_tokens`). Fixes partials inside sections. Standalone indentation applied to partial source before lexing.
### Architecture
4. **Dynamic Names**`{{>*key}}` support. When a partial token value starts with `*`, the remaining key is resolved from the data context stack, and the resolved string is used as the partial name. Enables per-item partial selection inside sections (e.g., `{{>* icon}}` resolves `icon` from each social link).
```
parse(source) → tokenize → trim_standalone_whitespace → parse_section → Template
render(tmpl, data, partials) → render_nodes (walks flat node array against context stack) → string
```
Extracted `template_process_tokens` from `template_eat_tokens` to separate ROOT initialization + skip pass from the core token loop, allowing partials to reuse the loop.
- **Two-phase API**: `parse()` produces a reusable `Template`, `render()` walks it against data. Templates parsed once, rendered many times.
- **Flat `[dynamic]Node` array** with `first_child`/`child_count` indices — pre-order layout.
- **Context stack**: `^[dynamic]any` with `append`/`pop` for section push/pop.
- **`effective(a)`** peels Named/Distinct/Union layers (including `json.Value`).
- **`lookup_in`** resolves keys on structs (via `reflect.struct_field_value_by_name` with `allow_using = true`) and maps (via runtime map internals).
- **Template inheritance**: `{{<parent}}` loads parent from partials, `{{$block}}` defines overridable sections. `merge_block_overrides` propagates overrides through multi-level chains.
- **Dynamic partial names**: `{{>*key}}` resolves partial name from data context at render time.
### Not implemented
- Lambdas (`~lambdas.json`)
- Set delimiters (`{{= =}}`, `delimiters.json`)
## Known limitations
- cmark allocates via C malloc, not the arena. HTML output leaks until process exit.
- CSS/JS cache busting uses manual `?v=N` query params instead of content hashing.
- `json.Value` params require 64-byte aligned arena (workaround for `dynamic_arena_allocator_proc` ignoring per-allocation alignment).
- Tree-sitter grammar/query paths hardcoded in `tree_sitter.odin` (Nix store hashes, Helix-version-dependent).
- `TSQueryCapture` needs explicit `_padding: u32` field for C ABI compatibility (40-byte sizeof).
- `{{&content}}` in layout template must have no leading whitespace — `template_insert_content_into_layout` indents all lines by preceding whitespace, which corrupts `<pre>` blocks.
- highlight.js removed; syntax highlighting is build-time only (no fallback if Tree-sitter fails).
- Tree-sitter grammar/query paths for dynamic grammars hardcoded in `tree_sitter.odin` (Nix store hashes, Helix-version-dependent). HTML/CSS are statically linked.
- `map[string]any` not fully supported by mustache `lookup_in` — thor uses structs instead.
- `format_f64` in mustache brute-forces shortest float representation.
## Design decisions
See `HUGO.md` for analysis of why thor doesn't need Hugo's shortcode context isolation.
See `mustache/PARTIAL_INDENT.md` for whitespace handling analysis.
See `mustache/SPEC.md` for the original implementation specification.
## TODO
+19 -15
View File
@@ -1,32 +1,31 @@
- README.md
- [ ] "Zero is beautiful"
- [ ] Content-hash fingerprinting for CSS and JS cache busting
- [ ] Enable configuration to opt-out of features, particular pre/post processors.
- [ ] proper date/time/now object.
- [ ] Footer isn't centered properly.
- [ ] Performance
- [ ] See if we can disable bounds checks in `write_indented` and elsewhere.
- [ ] Instead of loading the site fresh each time in watch mode, create a
`reload_site` proc, that just updates changed resources.
- [ ] mustache data keys for opengraph, etc.
- [ ] Infer page types / collections rather than having dedicated procs for
everything
- [ ] Block attributes on code fences (`{ #ex-1 }`) — hello-world.md
- [x] Emoji shortcodes (`:shrug:` etc.) — 2 instances
- [x] Backslash in shrug not visible.
- [ ] include-code shortcode (`{{< include-code ... >}}`) — i-ported-fd-to-odin
- [x] Nix build integration — main flake runs thor + tailwindcss instead of Hugo
- [x] OpenGraph meta tags
- [x] copy all files from `static` to `public`.
- [ ] ensure sidenote numbers render in display order and not in declaration order.
- [x] Search up for `thor.json` files.
- [ ] OpenGraph meta tags — verify all fields match production site
- [ ] Add Opt-in deflist support.
- [ ] We need to be able to do `Year_Section` in a non-magical, unpriviledged way.
- [ ] set opengraph tags / description automatically if unset. (Like hugo does)
- [ ] Table of contents support.
- [ ] Nav items should be active when the current page is selected.
- [ ] Theme selector for syntax highlighting.
- use http://github.com/helix-editor/helix/tree/master/runtime/themes) as a
guide
- [ ] Search for grammars in multiple places
- [ ] grammars
- [ ] Search in multiple places
- [ ] Download missing grammars.
- [ ] Add reflection support to mustache / consider rewriting from scratch.
- [ ] Syntax highlighting in production: CI (`nix build` on ubuntu-latest) has no grammar `.so`s and a machine-specific `QUERIES_PATH` nix store hash, so the deployed site renders unhighlighted. Provide grammars + queries as nix build inputs and pass paths to thor at runtime (env vars/flags).
- [ ] Durable highlight paths: read `GRAPHS_PATH`/`QUERIES_PATH` from env vars set by the flake instead of hardcoded nix store hashes, so they survive `nix flake update` and let the grammar/query version-mismatch detector fire automatically.
- [ ] CI
- [ ] Syntax highlighting in production: CI (`nix build` on ubuntu-latest) has no grammar `.so`s and a machine-specific `QUERIES_PATH` nix store hash, so the deployed site renders unhighlighted. Provide grammars + queries as nix build inputs and pass paths to thor at runtime (env vars/flags).
- [ ] Unit tests for highlighting helpers: `capture_name_to_css`, `escape_html`, `unescape_html`, `extract_query_token`, `helix_version_from_path`.
- [ ] `<pre><code>` blocks need to set background to theme background,
regardless of prefers-dark. (or use a different theme)
@@ -35,8 +34,11 @@
- [ ] filesystem poll loop
- [ ] event based
- [ ] Free cmark HTML output (`body_html`) — cmark allocates via C malloc, not the arena, so it leaks per iteration in watch mode
- [ ] Split `load_page` into frontmatter-parse + body-process phases so draft pages can skip the markdown pipeline entirely
- [ ] commands
- [ ] Add spall and find ways to reduce run time. (Getting close to 1/2sec here.)
- [ ] `build` alias of default
- [ ] `init` set up new project
- [ ] Use spall to find ways to reduce run time.
- [ ] warn/error when unknown key used in mustache.
- [ ] Review every file in thor
- [ ] Review alerts.odin
@@ -45,8 +47,10 @@
- [ ] Review feed.odin
- [ ] Review footnotes.odin
- [ ] Review frontmatter.odin
- [ ] Review highlight.odin
- [ ] Review main.odin
- [ ] Review main_test.odin
- [ ] Review mustache_test.odin
- [ ] Review minifiy.odin
- [ ] Review render.odin
- [ ] Review sectionate.odin
- [x] Review site.odin
- [ ] Review tree_sitter.odin
+59 -59
View File
@@ -8,7 +8,7 @@ import "core:os"
import "core:strings"
Page_Type :: enum {
Standalone,
Page,
Post,
Home,
}
@@ -23,68 +23,48 @@ Page :: struct {
draft: bool,
is_starred: bool,
menu: string,
body: string,
body_html: string,
bundle_dir: string,
}
// walk_content reads the content directory and returns all non-draft pages
// (or all pages if include_drafts is true).
//
// TODO: What is the lifetime of pages?
walk_content :: proc(site: ^Site) -> []Page {
// site_load_content reads the content directory and populates site.pages.
// Drafts are excluded unless .Drafts is enabled.
site_load_content :: proc(site: ^Site) {
site.pages = make([dynamic]Page, 0, 8, site_allocator(site))
load_homepage(site)
load_pages(site)
load_posts(site)
}
load_homepage :: proc(site: ^Site) {
content_path := site.content_dir
include_drafts := .Drafts in site.features
sectionate := .Sections in site.features
ext := site.markdown_extensions
allocator := site_allocator(site)
pages := make([dynamic]Page, allocator)
collect_home(&pages, content_path, sectionate)
collect_standalone(&pages, content_path, sectionate)
posts_path := fmt.tprintf("%s/posts", content_path)
if os.exists(posts_path) {
collect_posts(&pages, posts_path, sectionate)
}
if include_drafts {
return pages[:]
} else {
filtered := make([dynamic]Page, allocator)
for &page in pages {
if !page.draft {
append(&filtered, page)
}
}
delete(pages)
return filtered[:]
}
}
collect_home :: proc(pages: ^[dynamic]Page, content_path: string, sectionate: bool) {
html_path := fmt.tprintf("%s/index.html", content_path)
if os.exists(html_path) {
page, ok := load_page(html_path, .Home, "", sectionate)
if ok {
page, ok := load_page(html_path, .Home, "", ext)
if ok && (!page.draft || .Drafts in site.features) {
page.permalink = "/"
append(pages, page)
append(&site.pages, page)
}
return
}
md_path := fmt.tprintf("%s/index.md", content_path)
if os.exists(md_path) {
page, ok := load_page(md_path, .Home, "", sectionate)
if ok {
page, ok := load_page(md_path, .Home, "", ext)
if ok && (!page.draft || .Drafts in site.features) {
page.permalink = "/"
append(pages, page)
append(&site.pages, page)
}
}
}
collect_standalone :: proc(pages: ^[dynamic]Page, content_path: string, sectionate: bool) {
load_pages :: proc(site: ^Site) {
content_path := site.content_dir
ext := site.markdown_extensions
entries, err := os.read_all_directory_by_path(content_path, context.allocator)
if err != nil {
log.warnf("thor: cannot read %s: %v", content_path, err)
@@ -104,14 +84,20 @@ collect_standalone :: proc(pages: ^[dynamic]Page, content_path: string, sectiona
}
slug := strip_extension(entry.name)
page, ok := load_page(entry.fullpath, .Standalone, slug, sectionate)
if ok {
append(pages, page)
page, ok := load_page(entry.fullpath, .Page, slug, ext)
if ok && (!page.draft || .Drafts in site.features) {
append(&site.pages, page)
}
}
}
collect_posts :: proc(pages: ^[dynamic]Page, posts_path: string, sectionate: bool) {
load_posts :: proc(site: ^Site) {
posts_path := fmt.tprintf("%s/posts", site.content_dir)
if !os.exists(posts_path) {
return
}
ext := site.markdown_extensions
entries, err := os.read_all_directory_by_path(posts_path, context.allocator)
if err != nil {
log.warnf("thor: cannot read %s: %v", posts_path, err)
@@ -126,9 +112,9 @@ collect_posts :: proc(pages: ^[dynamic]Page, posts_path: string, sectionate: boo
continue
}
slug := strip_extension(entry.name)
page, ok := load_page(entry.fullpath, .Post, slug, sectionate)
if ok {
append(pages, page)
page, ok := load_page(entry.fullpath, .Post, slug, ext)
if ok && (!page.draft || .Drafts in site.features) {
append(&site.pages, page)
}
case .Directory:
index_path := fmt.tprintf("%s/index.html", entry.fullpath)
@@ -138,10 +124,10 @@ collect_posts :: proc(pages: ^[dynamic]Page, posts_path: string, sectionate: boo
if !os.exists(index_path) {
continue
}
page, ok := load_page(index_path, .Post, entry.name, sectionate)
if ok {
page, ok := load_page(index_path, .Post, entry.name, ext)
if ok && (!page.draft || .Drafts in site.features) {
page.bundle_dir = entry.fullpath
append(pages, page)
append(&site.pages, page)
}
case .Undetermined, .Symlink, .Named_Pipe, .Socket, .Block_Device, .Character_Device:
}
@@ -152,7 +138,7 @@ load_page :: proc(
file_path: string,
page_type: Page_Type,
slug: string,
sectionate: bool,
ext: bit_set[Markdown_Extension],
) -> (
page: Page,
ok: bool,
@@ -177,16 +163,30 @@ load_page :: proc(
page.draft = fm.draft
page.is_starred = fm.isStarred
page.menu = fm.menu
page.body = strings.clone(body)
if strings.has_suffix(file_path, ".html") {
page.body_html = strings.clone(body)
} else {
clean_body, sn_defs, mn_defs := strip_definitions(body)
sn_defs := make(map[string]string)
mn_defs := make(map[string]string)
clean_body := body
if .Sidenotes in ext {
clean_body, sn_defs, mn_defs = strip_definitions(body)
}
html := cm.markdown_to_html_from_string(clean_body, {.Unsafe})
if .Emoji in ext {
html = expand_emoji(html)
html = highlight_code(inject_alerts(inject_notes(html, sn_defs, mn_defs)), file_path)
if sectionate {
}
if .Sidenotes in ext {
html = inject_notes(html, sn_defs, mn_defs)
}
if .Alerts in ext {
html = inject_alerts(html)
}
if .Highlight in ext {
html = highlight_code(html, file_path)
}
if .Sections in ext {
html = wrap_sections(html)
}
page.body_html = html
@@ -197,7 +197,7 @@ load_page :: proc(
page.permalink = "/"
case .Post:
page.permalink = fmt.aprintf("/posts/%s/", slug)
case .Standalone:
case .Page:
page.permalink = fmt.aprintf("/%s/", slug)
}
+13 -13
View File
@@ -4,7 +4,7 @@ import "core:fmt"
import "core:strings"
import "core:time"
generate_rss :: proc(pages: []Page, config: Site) -> string {
generate_rss :: proc(site: ^Site) -> string {
sb := strings.builder_make()
strings.write_string(&sb, fmt.aprintf(
@@ -16,13 +16,13 @@ generate_rss :: proc(pages: []Page, config: Site) -> string {
<description>%s</description>
<language>en-us</language>
<atom:link href="%s/index.xml" rel="self" type="application/rss+xml"/>`,
xml_escape(config.title),
config.base_url,
xml_escape(config.description),
config.base_url,
xml_escape(site.title),
site.base_url,
xml_escape(site.description),
site.base_url,
))
for page in pages {
for page in site.pages {
if page.type == .Home {
continue
}
@@ -42,10 +42,10 @@ generate_rss :: proc(pages: []Page, config: Site) -> string {
</item>
`,
xml_escape(page.title),
config.base_url,
site.base_url,
page.permalink,
pub_date,
config.base_url,
site.base_url,
page.permalink,
xml_escape(page.body_html),
))
@@ -55,26 +55,26 @@ generate_rss :: proc(pages: []Page, config: Site) -> string {
return strings.to_string(sb)
}
generate_sitemap :: proc(pages: []Page, base_url: string) -> string {
generate_sitemap :: proc(site: ^Site) -> string {
sb := strings.builder_make()
strings.write_string(&sb, `<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml">
`)
for page in pages {
for page in site.pages {
lastmod := ""
if page.date != "" {
lastmod = fmt.aprintf("<lastmod>%s</lastmod>", page.date)
}
strings.write_string(&sb, fmt.aprintf(
"<url><loc>%s%s</loc>%s</url>\n", base_url, page.permalink, lastmod,
"<url><loc>%s%s</loc>%s</url>\n", site.base_url, page.permalink, lastmod,
))
}
// Posts list page
posts_lastmod := ""
for page in pages {
for page in site.pages {
if page.type == .Post && page.date > posts_lastmod {
posts_lastmod = page.date
}
@@ -84,7 +84,7 @@ generate_sitemap :: proc(pages: []Page, base_url: string) -> string {
posts_lm = fmt.aprintf("<lastmod>%s</lastmod>", posts_lastmod)
}
strings.write_string(&sb, fmt.aprintf(
"<url><loc>%s/posts/</loc>%s</url>\n", base_url, posts_lm,
"<url><loc>%s/posts/</loc>%s</url>\n", site.base_url, posts_lm,
))
strings.write_string(&sb, "</urlset>")
+19 -12
View File
@@ -14,11 +14,15 @@ Note_Kind :: enum {
// sidenotes, [*id]: text for marginnotes), removes them, and returns the cleaned
// text plus separate maps of id->definition for each kind.
// Handles multi-line definitions with indented continuation lines.
strip_definitions :: proc(body: string) -> (clean_body: string, sn_defs, mn_defs: map[string]string) {
sn_defs = make(map[string]string)
mn_defs = make(map[string]string)
strip_definitions :: proc(
body: string,
) -> (
clean_body: string,
sn_defs, mn_defs: map[string]string,
) {
// TODO: Should this be temp allocated?
lines := strings.split(body, "\n")
defer delete(lines)
output_lines: [dynamic]string
defer delete(output_lines)
@@ -119,8 +123,9 @@ inject_notes :: proc(html: string, sn_defs, mn_defs: map[string]string) -> strin
return html
}
parts: [dynamic]string
defer delete(parts)
parts: strings.Builder
strings.builder_init_len(&parts, 0) // TODO: Set a reasonable default
defer strings.builder_destroy(&parts)
remaining := html
@@ -134,16 +139,16 @@ inject_notes :: proc(html: string, sn_defs, mn_defs: map[string]string) -> strin
pos = mn_pos
}
if pos < 0 {
append(&parts, remaining)
strings.write_string(&parts, remaining)
break
}
// Append text before the reference
append(&parts, remaining[:pos])
strings.write_string(&parts, remaining[:pos])
close := strings.index(remaining[pos + 2:], "]")
if close < 0 {
append(&parts, remaining[pos:])
strings.write_string(&parts, remaining[pos:])
break
}
@@ -157,7 +162,7 @@ inject_notes :: proc(html: string, sn_defs, mn_defs: map[string]string) -> strin
def_text, found := defs[id]
if !found {
// No definition found, leave as literal text
append(&parts, remaining[pos:ref_end])
strings.write_string(&parts, remaining[pos:ref_end])
remaining = remaining[ref_end:]
continue
}
@@ -167,6 +172,7 @@ inject_notes :: proc(html: string, sn_defs, mn_defs: map[string]string) -> strin
def_html = strip_p_tags(def_html)
note: string
defer delete(note)
if is_margin {
note = fmt.aprintf(
`<label for="mn-%s" class="margin-toggle"></label><input type="checkbox" id="mn-%s" class="margin-toggle"><span class="marginnote">%s</span>`,
@@ -182,12 +188,12 @@ inject_notes :: proc(html: string, sn_defs, mn_defs: map[string]string) -> strin
def_html,
)
}
append(&parts, note)
strings.write_string(&parts, note)
remaining = remaining[ref_end:]
}
return strings.join(parts[:], "")
return strings.to_string(parts)
}
// strip_p_tags removes surrounding <p></p> if the HTML is a single paragraph.
@@ -201,3 +207,4 @@ strip_p_tags :: proc(html: string) -> string {
}
return s
}
+29 -5
View File
@@ -34,6 +34,13 @@ test_parse_def_line :: proc(t: ^testing.T) {
test_strip_definitions :: proc(t: ^testing.T) {
body := "Intro[^a] and [*b].\n\n[^a]: a side\n\n[*b]: a margin\n"
clean, sn_defs, mn_defs := strip_definitions(body)
defer {
delete(clean)
delete(sn_defs["a"])
delete(sn_defs)
delete(mn_defs["b"])
delete(mn_defs)
}
// references stay; definitions are removed
testing.expect(t, strings.contains(clean, "Intro[^a]"))
@@ -60,13 +67,22 @@ test_strip_definitions :: proc(t: ^testing.T) {
@(test)
test_inject_notes :: proc(t: ^testing.T) {
html := "Text[^a] more [*b] end."
sn_defs := map[string]string{"a" = "side note"}
mn_defs := map[string]string{"b" = "margin note"}
sn_defs := map[string]string {
"a" = "side note",
}
defer delete(sn_defs)
mn_defs := map[string]string {
"b" = "margin note",
}
defer delete(mn_defs)
out := inject_notes(html, sn_defs, mn_defs)
// sidenote: numbered, fn- prefix, .sidenote span, rendered text
testing.expect(t, strings.contains(out, `for="fn-a" class="margin-toggle sidenote-number"></label>`))
testing.expect(
t,
strings.contains(out, `for="fn-a" class="margin-toggle sidenote-number"></label>`),
)
testing.expect(t, strings.contains(out, `class="sidenote"`))
testing.expect(t, strings.contains(out, "side note"))
@@ -87,12 +103,20 @@ test_inject_notes_no_defs :: proc(t: ^testing.T) {
@(test)
test_inject_notes_missing_ref :: proc(t: ^testing.T) {
html := "Ref[^missing] and [*missing] end."
sn := map[string]string{"other" = "x"}
mn := map[string]string{"other2" = "y"}
sn := map[string]string {
"other" = "x",
}
defer delete_map(sn)
mn := map[string]string {
"other2" = "y",
}
defer delete_map(mn)
out := inject_notes(html, sn, mn)
// references with no matching definition are left as literal text
testing.expect(t, len(out) > 2)
testing.expect(t, strings.contains(out, "[^missing]"))
testing.expect(t, strings.contains(out, "[*missing]"))
}
+48 -2
View File
@@ -1,10 +1,38 @@
package main
import "base:runtime"
import "core:fmt"
import "core:log"
import "core:os"
import "core:prof/spall"
import "core:sync"
import "core:time"
SPALL :: #config(SPALL, false)
when SPALL {
spall_ctx: spall.Context
@(thread_local)
spall_buffer: spall.Buffer
}
main :: proc() {
when SPALL {
ctx, ok := spall.context_create_with_scale("thor.spall", false, 1.0)
if !ok {
fmt.eprintln("Failed to create spall trace file")
os.exit(1)
}
spall_ctx = ctx
defer spall.context_destroy(&spall_ctx)
spall_backing := make([]u8, spall.BUFFER_DEFAULT_SIZE)
defer delete(spall_backing)
spall_buffer = spall.buffer_create(spall_backing, u32(sync.current_thread_id()))
defer spall.buffer_destroy(&spall_ctx, &spall_buffer)
}
console_logger := log.create_console_logger()
context.logger = console_logger
defer log.destroy_console_logger(console_logger)
@@ -17,8 +45,8 @@ main :: proc() {
// TODO: Make it so this isn't necessary
context.allocator = site_allocator(&site)
pages := walk_content(&site)
render_site(pages, site)
site_load_content(&site)
render_site(&site)
if !(.Watch in site.features) {
break
@@ -27,3 +55,21 @@ main :: proc() {
}
}
when SPALL {
@(instrumentation_enter)
spall_enter :: proc "contextless" (
proc_address, call_site_return_address: rawptr,
loc: runtime.Source_Code_Location,
) {
spall._buffer_begin(&spall_ctx, &spall_buffer, "", "", loc)
}
@(instrumentation_exit)
spall_exit :: proc "contextless" (
proc_address, call_site_return_address: rawptr,
loc: runtime.Source_Code_Location,
) {
spall._buffer_end(&spall_ctx, &spall_buffer)
}
}
-137
View File
@@ -1,137 +0,0 @@
# Partial Indentation — Problem & Solutions
## Problem
When a partial tag `{{> name}}` is standalone (only non-whitespace on its line), the mustache spec requires that its leading whitespace be treated as indentation and **prepended to each line of the partial source before rendering**.
This is a source-level transformation, not output post-processing. The distinction matters when interpolated content contains newlines:
```
partial source: "|\n{{{content}}}\n|\n"
content value: "<\n->"
indent: " "
Expected output: " |\n <\n->\n |\n"
```
The line `->` gets NO indent — it comes from expanded content (`<\n->`), not from a partial source line. Post-processing the output would incorrectly indent it.
### Spec tests that require this
- **Standalone Without Previous Line** — indent at start of template
- **Standalone Without Newline** — indent at end of template
- **Standalone Indentation** — indent with multi-line interpolated content
3 of 14 tests in `partials.json`. The other 11 (basic lookup, context, recursion, nesting, inline usage, failed lookup, padding) work without indentation handling.
### thor's real usage
Thor's partials (`{{> nav}}`, `{{> footer}}`, `{{>* icon}}`) are typically standalone with indentation. Without indentation handling, HTML output has wrong indentation — ugly but functional since HTML ignores whitespace.
## Solution 1: Store source, re-parse with indent (Recommended)
Add `source: string` to `Template`. When rendering a standalone partial with non-empty indent:
1. Prepend indent to each line of `partial.source`
2. Re-tokenize + re-parse with `context.temp_allocator`
3. Render the re-parsed nodes
When indent is empty, render the pre-parsed nodes directly (no re-parse).
### Pros
- Correct by construction — exactly matches spec ("prepended to each line before rendering")
- ~20 lines of code
- Nested partials accumulate indentation naturally
- No line-start state tracking
### Cons
- Template gains a `source: string` field
- Standalone partials with indent get re-parsed at render time (negligible for small fragments)
### Implementation sketch
```odin
// tokenizer: capture indent during trim_standalone_whitespace
// for Partial tokens, before stripping left whitespace:
tokens[i].indent = text[nl+1:] // capture indentation
// mustache.odin: Template gains source field
Template :: struct {
nodes: [dynamic]Node,
source: string,
}
// parse: store source
tmpl.source = source
// renderer: re-parse if indent
case .Partial:
pt, found := partials[name]
if !found do break
if len(node.indent) > 0 {
indented := indent_lines(pt.source, node.indent)
reparse, err := parse(indented, context.temp_allocator, context.temp_allocator)
if err == nil {
defer delete(reparse.nodes)
render_nodes(reparse.nodes[:], reparse.nodes[:], ctx, partials, b)
}
} else {
render_nodes(pt.nodes[:], pt.nodes[:], ctx, partials, b)
}
```
## Solution 2: Modify text nodes during rendering
Track "at line start" state while rendering the partial's nodes. Insert indent before text-node content that begins a new line. Variables/sections render normally — their output is NOT indented.
### Pros
- No source storage
- No re-parsing
### Cons
- Complex: must track line-start state across nodes
- Variables producing multi-line output need careful handling (their newlines don't create indented lines)
- Trailing newline edge case (partial ending with `\n` shouldn't leave trailing indent)
- Nested partials with accumulated indentation need extra logic
- ~60+ lines of fiddly code
### Implementation sketch
```odin
render_partial :: proc(nodes, ctx, partials, b, indent) {
at_line_start := true
for node in nodes {
switch node.kind {
case .Text:
// Walk text, prepend indent at line starts
// Insert indent after each \n
// Don't indent after final \n of last text node
...
at_line_start = (text ends with \n)
case .Variable, .Unescaped:
// Render normally — no indent applied to output
write_value(b, val, ...)
at_line_start = false // can't know if output ends with \n
case .Section, .Inverted:
// Complex: nested text nodes need indent too?
...
}
}
}
```
The "can't know if variable output ends with \n" problem makes `at_line_start` unreliable, requiring heuristics or output buffering.
## Adjacent Tag Standalone Detection
**Problem:** When a block-type tag and its close tag are adjacent (no text between them), like `{{<include}}{{/include}}\n`, neither tag is individually detected as standalone. The current check looks at the immediately adjacent token — if it's another tag (not text), the check fails. So the trailing `\n` isn't consumed.
**Affected spec tests:**
- `~inheritance.json`: "Inherit", "Override parent with newlines"
- Potentially any `{{<name}}{{/name}}` or `{{#name}}{{/name}}` on its own line
**Solution:** When scanning left/right for the line boundary, skip over adjacent standalone-eligible tags (they're transparent). Two helper procs replace the current inline checks:
- `check_left(tokens, i)` — scans backwards through adjacent eligible tags until finding text or start of template
- `check_right(tokens, i)` — scans forwards through adjacent eligible tags until finding text or end of template
Both return `(ok: bool, text_idx: int)` where `text_idx` is the text token to trim (or -1 if none).
+5 -1
View File
@@ -1,2 +1,6 @@
- [ ] Initialize string builder size based on filesize?
- [ ] Review [whitespace handling code](./tokenizer.odin)
- [x] Review [whitespace handling code](./tokenizer.odin)
- [x] Add leak checks for public procs.
- [ ] Simplify `trim_standalone_whitespace` — four temp arrays (`li_buf`, `ri_buf`, `left_done`, `right_done`) could potentially be reduced to two by merging the done-tracking into the index buffers using sentinel values (e.g. `-2` = already trimmed).
- [ ] Adjacent standalone tags don't capture indent for the second tag (e.g. ` {{>a}}\n {{>b}}``{{>b}}` gets `indent=""` because its left text was consumed by `{{>a}}`'s right-trim). Blocks recover via `deindent_blocks`, but partials/parents have no fallback.
- [ ] lazy load partials ? (with mutex)
+77 -36
View File
@@ -6,22 +6,30 @@ import "core:reflect"
import "core:strconv"
import "core:strings"
// effective unwraps union variants and strips Named/Distinct layers,
// base_value unwraps union variants and strips Named/Distinct layers,
// returning the "peeled" any value and its base type info.
effective :: proc(a: any) -> (val: any, info: ^runtime.Type_Info) {
base_value :: proc(a: any) -> (val: any, info: ^runtime.Type_Info) {
if a == nil {
return
}
val = a
info = nil
if val == nil do return
ti := type_info_of(val.id)
if ti == nil do return
if ti == nil {
return
}
base := runtime.type_info_base(ti)
if _, ok := base.variant.(runtime.Type_Info_Union); ok {
variant := reflect.get_union_variant(val)
if variant == nil do return {}, nil
return effective(variant)
if variant == nil {
return {}, nil
} else {
return base_value(variant)
}
}
info = base
@@ -31,35 +39,36 @@ effective :: proc(a: any) -> (val: any, info: ^runtime.Type_Info) {
// lookup_in resolves a single key in a container (struct or map).
// Returns found=false if the key doesn't exist or the container is not a struct/map.
lookup_in :: proc(container: any, key: string) -> (result: any, found: bool) {
if container == nil do return nil, false
if container == nil {
return
}
val, info := effective(container)
if info == nil do return nil, false
val, info := base_value(container)
if info == nil {
return
}
#partial switch v in info.variant {
case runtime.Type_Info_Struct:
result = reflect.struct_field_value_by_name(val, key, allow_using = true)
found = result != nil
return
case runtime.Type_Info_Map:
mi := v
rm_ptr := (^runtime.Raw_Map)(val.data)
if rm_ptr.len == 0 do return nil, false
if rm_ptr.len == 0 {
return
}
k := key
seed := runtime.map_seed(rm_ptr^)
h := mi.map_info.key_hasher(&k, seed)
value_ptr := runtime.__dynamic_map_get(rm_ptr, mi.map_info, h, &k)
if value_ptr == nil do return nil, false
if value_ptr != nil {
result = any{value_ptr, mi.value.id}
found = true
return
case:
return nil, false
}
}
return
}
@@ -68,9 +77,12 @@ lookup_in :: proc(container: any, key: string) -> (result: any, found: bool) {
// resolve against the prior result only.
resolve_name :: proc(name: string, ctx: []any) -> any {
if name == "." {
if len(ctx) > 0 do return ctx[len(ctx) - 1]
if len(ctx) > 0 {
return ctx[len(ctx) - 1]
} else {
return nil
}
}
parts: [16]string
part_count := 0
@@ -89,22 +101,33 @@ resolve_name :: proc(name: string, ctx: []any) -> any {
part_count += 1
}
if part_count == 0 do return nil
if part_count == 0 {
return nil
}
dot_parts := parts[:part_count]
result: any = nil
found := false
for i := len(ctx) - 1; i >= 0; i -= 1 {
result, found = lookup_in(ctx[i], dot_parts[0])
if found do break
if found {
break
}
}
if !found do return nil
if len(dot_parts) == 1 do return result
if !found {
return nil
}
if len(dot_parts) == 1 {
return result
}
for i := 1; i < len(dot_parts); i += 1 {
result, found = lookup_in(result, dot_parts[i])
if !found do return nil
if !found {
return nil
}
}
return result
@@ -112,11 +135,17 @@ resolve_name :: proc(name: string, ctx: []any) -> any {
// is_truthy checks mustache truthiness.
is_truthy :: proc(a: any) -> bool {
if a == nil do return false
if a == nil {
return false
}
val, info := effective(a)
if info == nil do return false
if reflect.is_nil(val) do return false
val, info := base_value(a)
if info == nil {
return false
}
if reflect.is_nil(val) {
return false
}
#partial switch _ in info.variant {
case runtime.Type_Info_Slice, runtime.Type_Info_Dynamic_Array, runtime.Type_Info_Map:
@@ -129,8 +158,10 @@ is_truthy :: proc(a: any) -> bool {
// list_info returns element type info, count, and data pointer for a list value.
// Returns elem_info=nil if the value is not a list.
list_info :: proc(a: any) -> (elem_info: ^runtime.Type_Info, count: int, data: rawptr) {
val, info := effective(a)
if info == nil do return nil, 0, nil
val, info := base_value(a)
if info == nil {
return
}
#partial switch v in info.variant {
case runtime.Type_Info_Slice:
@@ -148,8 +179,10 @@ list_info :: proc(a: any) -> (elem_info: ^runtime.Type_Info, count: int, data: r
// any_to_string converts a scalar value to a string using the temp allocator.
any_to_string :: proc(a: any) -> string {
if a == nil do return ""
val, _ := effective(a)
if a == nil {
return ""
}
val, _ := base_value(a)
switch v in val {
case string:
@@ -174,14 +207,18 @@ format_f64 :: proc(v: f64) -> string {
buf: [64]byte
for prec in 1 ..= 17 {
s := strconv.write_float(buf[:], v, 'g', prec, 64)
if len(s) > 0 && s[0] == '+' do s = s[1:]
if len(s) > 0 && s[0] == '+' {
s = s[1:]
}
parsed, ok := strconv.parse_f64(s)
if ok && parsed == v {
return strings.clone(s, context.temp_allocator)
}
}
s := strconv.write_float(buf[:], v, 'g', -1, 64)
if len(s) > 0 && s[0] == '+' do s = s[1:]
if len(s) > 0 && s[0] == '+' {
s = s[1:]
}
return strings.clone(s, context.temp_allocator)
}
@@ -189,7 +226,9 @@ format_f64 :: proc(v: f64) -> string {
// optionally HTML-escaped.
write_value :: proc(b: ^strings.Builder, a: any, escape: bool) {
s := any_to_string(a)
if len(s) == 0 do return
if len(s) == 0 {
return
}
if !escape {
strings.write_string(b, s)
@@ -200,7 +239,9 @@ write_value :: proc(b: ^strings.Builder, a: any, escape: bool) {
for i in 0 ..< len(s) {
switch s[i] {
case '&', '<', '>', '"':
if i > start do strings.write_string(b, s[start:i])
if i > start {
strings.write_string(b, s[start:i])
}
switch s[i] {
case '&':
strings.write_string(b, "&amp;")
+245 -35
View File
@@ -50,8 +50,21 @@ Node :: struct {
child_count: int,
}
// node_span returns the number of flat-array entries a node occupies:
// 1 for leaf nodes, 1 + child_count for container nodes (whose children
// are stored contiguously after them in the array).
node_span :: proc(n: Node) -> int {
#partial switch n.kind {
case .Section, .Inverted, .Parent, .Block:
return 1 + n.child_count
case:
return 1
}
}
Template :: struct {
nodes: [dynamic]Node,
source: string,
}
Block_Override :: struct {
@@ -66,6 +79,13 @@ template_free :: proc(tmpl: ^Template) {
}
}
delete_partials :: proc(partials: map[string]Template) {
for _, &p in partials {
template_free(&p)
}
delete(partials)
}
// ---------------------------------------------------------------------------
// Public API
// ---------------------------------------------------------------------------
@@ -86,8 +106,11 @@ parse :: proc(
tmpl.nodes, err = parse_tokens(tokens[:], allocator)
if err != nil {
delete(tmpl.nodes)
return {}, err
}
return tmpl, err
tmpl.source = source
deindent_blocks(tmpl.nodes[:], 0, len(tmpl.nodes), allocator)
return tmpl, nil
}
render :: proc(
@@ -165,8 +188,7 @@ parse_section :: proc(
pos^ += 1
idx := len(nodes)
append(nodes, Node{kind = .Section, key = tok.value, first_child = -1})
err := parse_section(tokens, pos, nodes, tok.value)
if err != nil do return err
parse_section(tokens, pos, nodes, tok.value) or_return
nodes[idx].first_child = idx + 1
nodes[idx].child_count = len(nodes) - idx - 1
@@ -174,8 +196,7 @@ parse_section :: proc(
pos^ += 1
idx := len(nodes)
append(nodes, Node{kind = .Inverted, key = tok.value, first_child = -1})
err := parse_section(tokens, pos, nodes, tok.value)
if err != nil do return err
parse_section(tokens, pos, nodes, tok.value) or_return
nodes[idx].first_child = idx + 1
nodes[idx].child_count = len(nodes) - idx - 1
@@ -202,6 +223,7 @@ parse_section :: proc(
kind = .Partial,
key = tok.value,
is_dynamic = tok.is_dynamic,
indent = tok.indent,
first_child = -1,
},
)
@@ -210,18 +232,22 @@ parse_section :: proc(
case .Parent:
pos^ += 1
idx := len(nodes)
append(nodes, Node{kind = .Parent, key = tok.value, first_child = -1})
err := parse_section(tokens, pos, nodes, tok.value)
if err != nil do return err
append(
nodes,
Node{kind = .Parent, key = tok.value, indent = tok.indent, first_child = -1},
)
parse_section(tokens, pos, nodes, tok.value) or_return
nodes[idx].first_child = idx + 1
nodes[idx].child_count = len(nodes) - idx - 1
case .Block_Open:
pos^ += 1
idx := len(nodes)
append(nodes, Node{kind = .Block, key = tok.value, first_child = -1})
err := parse_section(tokens, pos, nodes, tok.value)
if err != nil do return err
append(
nodes,
Node{kind = .Block, key = tok.value, indent = tok.indent, first_child = -1},
)
parse_section(tokens, pos, nodes, tok.value) or_return
nodes[idx].first_child = idx + 1
nodes[idx].child_count = len(nodes) - idx - 1
}
@@ -233,6 +259,172 @@ parse_section :: proc(
return nil
}
// ---------------------------------------------------------------------------
// Post-parse: de-indent block content
// ---------------------------------------------------------------------------
deindent_blocks :: proc(all_nodes: []Node, start: int, end: int, allocator := context.allocator) {
i := start
for i < end {
#partial switch all_nodes[i].kind {
case .Block:
if all_nodes[i].child_count > 0 {
cs := all_nodes[i].first_child
ce := cs + all_nodes[i].child_count
deindent_blocks(all_nodes, cs, ce, allocator)
children := all_nodes[cs:ce]
common := find_common_indent(children)
if len(common) > 0 {
if len(all_nodes[i].indent) == 0 {
all_nodes[i].indent = common
}
for j := cs; j < ce; {
if all_nodes[j].kind == .Text && len(all_nodes[j].text) > 0 {
all_nodes[j].text = remove_line_indent(
all_nodes[j].text,
common,
allocator,
)
}
j += node_span(all_nodes[j])
}
}
}
case .Section, .Inverted, .Parent:
if all_nodes[i].child_count > 0 {
cs := all_nodes[i].first_child
ce := cs + all_nodes[i].child_count
deindent_blocks(all_nodes, cs, ce, allocator)
}
}
i += node_span(all_nodes[i])
}
}
find_common_indent :: proc(children: []Node) -> string {
common: string
found := false
i := 0
for i < len(children) {
if children[i].kind == .Text {
text := children[i].text
if len(text) > 0 {
line_start := 0
for j in 0 ..= len(text) {
if j == len(text) || text[j] == '\n' {
line := text[line_start:j]
if len(strings.trim_space(line)) > 0 {
ws := leading_whitespace(line)
if !found {
common = ws
found = true
} else if len(ws) < len(common) {
common = ws
}
}
line_start = j + 1
}
}
}
}
i += node_span(children[i])
}
if found {
return common
} else {
return ""
}
}
leading_whitespace :: proc(s: string) -> string {
for i in 0 ..< len(s) {
if s[i] != ' ' && s[i] != '\t' {
return s[:i]
}
}
return s
}
remove_line_indent :: proc(s: string, indent: string, allocator := context.allocator) -> string {
if len(indent) == 0 {
return s
}
buf := make([dynamic]u8, 0, len(s), allocator)
i := 0
at_line_start := true
for i < len(s) {
if at_line_start {
if i + len(indent) <= len(s) && s[i:i + len(indent)] == indent {
i += len(indent)
at_line_start = false
continue
}
at_line_start = false
}
append(&buf, s[i])
if s[i] == '\n' {
at_line_start = true
}
i += 1
}
return string(buf[:])
}
// ---------------------------------------------------------------------------
// Indentation helpers
// ---------------------------------------------------------------------------
indent_lines :: proc(source: string, indent: string) -> string {
if len(indent) == 0 || len(source) == 0 {
return source
}
b: strings.Builder
strings.builder_init(&b, context.temp_allocator)
write_indented(&b, indent, source)
return strings.to_string(b)
}
write_indented :: proc(b: ^strings.Builder, indent: string, content: string) {
if len(indent) == 0 || len(content) == 0 {
strings.write_string(b, content)
return
}
at_line_start := true
for i in 0 ..< len(content) {
if at_line_start {
strings.write_string(b, indent)
at_line_start = false
}
strings.write_byte(b, content[i])
if content[i] == '\n' {
at_line_start = true
}
}
}
render_template :: proc(
pt: Template,
ctx: ^[dynamic]any,
partials: map[string]Template,
b: ^strings.Builder,
blocks: map[string]Block_Override,
indent: string,
) -> Render_Error {
if len(indent) > 0 && len(pt.source) > 0 {
indented := indent_lines(pt.source, indent)
reparse := parse(indented, context.temp_allocator, context.temp_allocator) or_return
return render_nodes(reparse.nodes[:], reparse.nodes[:], ctx, partials, b, blocks)
}
return render_nodes(pt.nodes[:], pt.nodes[:], ctx, partials, b, blocks)
}
// ---------------------------------------------------------------------------
// Renderer — walk node array against context stack, write to builder
// ---------------------------------------------------------------------------
@@ -272,15 +464,13 @@ render_nodes :: proc(
for j in 0 ..< count {
elem_ptr := rawptr(uintptr(data) + uintptr(j) * uintptr(elem_info.size))
append(ctx, any{elem_ptr, elem_info.id})
err := render_nodes(all_nodes, children, ctx, partials, b, blocks)
pop(ctx)
if err != nil do return err
defer pop(ctx)
render_nodes(all_nodes, children, ctx, partials, b, blocks) or_return
}
} else {
append(ctx, val)
err := render_nodes(all_nodes, children, ctx, partials, b, blocks)
pop(ctx)
if err != nil do return err
defer pop(ctx)
render_nodes(all_nodes, children, ctx, partials, b, blocks) or_return
}
}
i += 1 + node.child_count
@@ -289,8 +479,7 @@ render_nodes :: proc(
val := resolve_name(node.key, ctx[:])
if !is_truthy(val) {
children := all_nodes[node.first_child:node.first_child + node.child_count]
err := render_nodes(all_nodes, children, ctx, partials, b, blocks)
if err != nil do return err
render_nodes(all_nodes, children, ctx, partials, b, blocks) or_return
}
i += 1 + node.child_count
@@ -302,25 +491,49 @@ render_nodes :: proc(
}
pt, found := partials[name]
if found {
err := render_nodes(pt.nodes[:], pt.nodes[:], ctx, partials, b)
if err != nil do return err
render_template(pt, ctx, partials, b, nil, node.indent) or_return
}
i += 1
case .Block:
rendered_override := false
content_nodes: []Node
content_pool: []Node
content_blocks := blocks
found_override := false
if blocks != nil {
if o, ok := blocks[node.key]; ok {
children := o.all_nodes[o.first:o.first + o.count]
err := render_nodes(o.all_nodes, children, ctx, partials, b)
if err != nil do return err
rendered_override = true
content_nodes = o.all_nodes[o.first:o.first + o.count]
content_pool = o.all_nodes
found_override = true
}
}
if !rendered_override {
children := all_nodes[node.first_child:node.first_child + node.child_count]
err := render_nodes(all_nodes, children, ctx, partials, b, blocks)
if err != nil do return err
if !found_override {
content_nodes = all_nodes[node.first_child:node.first_child + node.child_count]
content_pool = all_nodes
}
if len(node.indent) > 0 {
temp: strings.Builder
strings.builder_init(&temp, context.temp_allocator)
render_nodes(
content_pool,
content_nodes,
ctx,
partials,
&temp,
content_blocks,
) or_return
write_indented(b, node.indent, strings.to_string(temp))
} else {
render_nodes(
content_pool,
content_nodes,
ctx,
partials,
b,
content_blocks,
) or_return
}
i += 1 + node.child_count
@@ -329,8 +542,7 @@ render_nodes :: proc(
merged := merge_block_overrides(parent_children, all_nodes, blocks)
pt, found := partials[node.key]
if found {
err := render_nodes(pt.nodes[:], pt.nodes[:], ctx, partials, b, merged)
if err != nil do return err
render_template(pt, ctx, partials, b, merged, node.indent) or_return
}
i += 1 + node.child_count
}
@@ -360,10 +572,8 @@ merge_block_overrides :: proc(
count = child.child_count,
}
}
i += 1 + child.child_count
} else {
i += 1
}
i += node_span(child)
}
return result
+121
View File
@@ -0,0 +1,121 @@
#+test
package mustache
import "core:mem"
import "core:testing"
@(test)
leak_parse_free :: proc(t: ^testing.T) {
tmpl, err := parse("Hello {{name}}!")
testing.expect(t, err == nil)
defer template_free(&tmpl)
}
@(test)
leak_parse_free_tokens :: proc(t: ^testing.T) {
arena: mem.Dynamic_Arena
backing: [256]byte
mem.dynamic_arena_init(&arena)
defer mem.dynamic_arena_destroy(&arena)
tmpl, err := parse("Hello {{name}}!", context.allocator, mem.dynamic_arena_allocator(&arena))
testing.expect(t, err == nil)
defer template_free(&tmpl)
}
@(test)
leak_parse_error :: proc(t: ^testing.T) {
tmpl, err := parse("Hello {{name")
testing.expect(t, err != nil)
}
@(test)
leak_render :: proc(t: ^testing.T) {
Data :: struct {
name: string,
}
tmpl, err := parse("Hello {{name}}!")
testing.expect(t, err == nil)
defer template_free(&tmpl)
result, rerr := render(tmpl, Data{name = "World"})
testing.expect(t, rerr == nil)
defer delete(result)
}
@(test)
leak_render_partials :: proc(t: ^testing.T) {
partials := make_map(map[string]Template)
defer {
for _, &p in partials {
template_free(&p)
}
delete(partials)
}
pt, perr := parse("world")
testing.expect(t, perr == nil)
partials["name"] = pt
tmpl, err := parse("Hello {{> name}}!")
testing.expect(t, err == nil)
defer template_free(&tmpl)
Data :: struct {
name: string,
}
result, rerr := render(tmpl, Data{name = "World"}, partials)
testing.expect(t, rerr == nil)
defer delete(result)
}
@(test)
leak_render_sections :: proc(t: ^testing.T) {
tmpl, err := parse("{{#items}}{{.}}{{/items}}")
testing.expect(t, err == nil)
defer template_free(&tmpl)
Data :: struct {
items: [3]string,
}
result, rerr := render(tmpl, Data{items = {"a", "b", "c"}})
testing.expect(t, rerr == nil)
defer delete(result)
}
@(test)
leak_render_inheritance :: proc(t: ^testing.T) {
partials := make_map(map[string]Template)
defer delete_partials(partials)
layout_src := "{{$title}}default{{/title}}"
layout, lerr := parse(layout_src)
testing.expect(t, lerr == nil)
partials["layout"] = layout
tmpl_src := "{{<layout}}{{$title}}custom{{/title}}{{/layout}}"
tmpl, err := parse(tmpl_src)
testing.expect(t, err == nil)
defer template_free(&tmpl)
result, rerr := render(tmpl, {}, partials)
testing.expect(t, rerr == nil)
defer delete(result)
}
@(test)
leak_repeated_render :: proc(t: ^testing.T) {
tmpl, err := parse("Hello {{name}}!")
testing.expect(t, err == nil)
defer template_free(&tmpl)
Data :: struct {
name: string,
}
for _ in 0 ..< 3 {
result, rerr := render(tmpl, Data{name = "World"})
testing.expect(t, rerr == nil)
defer delete(result)
}
}
+3
View File
@@ -4,9 +4,12 @@ package mustache
import "core:encoding/json"
import "core:fmt"
import "core:os"
import "core:path/filepath"
import "core:testing"
run_spec_file :: proc(t: ^testing.T, path: string) {
dirs := [?]string{#directory, path}
path, _ := filepath.join(dirs[:], context.temp_allocator)
raw_data, err := os.read_entire_file(path, context.allocator)
if err != nil {
testing.expectf(t, false, "Failed to read %s", path)
+73 -13
View File
@@ -19,6 +19,7 @@ Token :: struct {
kind: Token_Kind,
value: string,
is_dynamic: bool,
indent: string,
pos: int,
}
@@ -141,24 +142,83 @@ tokenize :: proc(
// ---------------------------------------------------------------------------
trim_standalone_whitespace :: proc(tokens: ^[dynamic]Token) {
for i := 0; i < len(tokens); i += 1 {
if !should_trim_whitespace(tokens[i].kind) do continue
n := len(tokens)
left_ok, left_idx := check_left(tokens[:], i)
right_ok, right_idx := check_right(tokens[:], i)
// First pass: detect standalone status using original token values
li_buf := make([dynamic]int, n, context.temp_allocator)
defer delete(li_buf)
ri_buf := make([dynamic]int, n, context.temp_allocator)
defer delete(ri_buf)
if !left_ok || !right_ok do continue
if left_idx >= 0 {
text := tokens[left_idx].value
nl := strings.last_index_byte(text, '\n')
tokens[left_idx].value = text[:nl+1] if nl >= 0 else ""
for i := 0; i < n; i += 1 {
li_buf[i] = -1
ri_buf[i] = -1
}
if right_idx >= 0 {
text := tokens[right_idx].value
for i := 0; i < n; i += 1 {
should_trim_whitespace(tokens[i].kind) or_continue
left_ok, li := check_left(tokens[:], i)
right_ok, ri := check_right(tokens[:], i)
if left_ok && right_ok {
li_buf[i] = li
ri_buf[i] = ri
}
}
// Second pass: apply trims left-to-right
// Track which text tokens have been trimmed to avoid double-trimming
left_done := make([dynamic]bool, n, context.temp_allocator)
defer delete(left_done)
right_done := make([dynamic]bool, n, context.temp_allocator)
defer delete(right_done)
for i := 0; i < n; i += 1 {
should_trim_whitespace(tokens[i].kind) or_continue
li := li_buf[i]
ri := ri_buf[i]
if li < 0 && ri < 0 {
continue
}
if li >= 0 && !left_done[li] {
left_done[li] = true
text := tokens[li].value
nl := strings.last_index_byte(text, '\n')
if tokens[i].kind == .Partial ||
tokens[i].kind == .Parent ||
tokens[i].kind == .Block_Open {
tokens[i].indent = text[nl + 1:] if nl >= 0 else text
}
tokens[li].value = text[:nl + 1] if nl >= 0 else ""
}
if ri >= 0 && !right_done[ri] {
right_done[ri] = true
// When a Block_Open and its Section_Close are adjacent (e.g.
// {{$block}}{{/block}}\n), the close tag is the one that
// should consume the trailing newline — not the block open.
// Skip the right-trim if a Section_Close sits between this
// tag and its right text, or if this IS the Section_Close
// immediately following a Block_Open (the close handles it).
skip := false
if tokens[i].kind == .Block_Open {
for k := i + 1; k < ri; k += 1 {
if tokens[k].kind == .Section_Close {
skip = true
break
}
}
}
if tokens[i].kind == .Section_Close && i > 0 && tokens[i - 1].kind == .Block_Open {
skip = true
}
if !skip {
text := tokens[ri].value
nl := strings.index_byte(text, '\n')
tokens[right_idx].value = text[nl+1:] if nl >= 0 else ""
tokens[ri].value = text[nl + 1:] if nl >= 0 else ""
}
}
}
}
-111
View File
@@ -1,111 +0,0 @@
#+feature dynamic-literals
#+test
package main
import "core:testing"
import "mustache"
@(test)
test_simple_substitution :: proc(t: ^testing.T) {
data := map[string]string {
"name" = "World",
}
tpl, _ := mustache.parse("Hello, {{name}}!")
result, err := mustache.render(tpl, data)
testing.expect(t, err == nil)
testing.expect(t, result == "Hello, World!")
}
@(test)
test_bool_section :: proc(t: ^testing.T) {
data := map[string]bool {
"show" = true,
}
tpl, _ := mustache.parse("{{#show}}visible{{/show}}")
result, err := mustache.render(tpl, data)
testing.expect(t, err == nil)
testing.expect(t, result == "visible")
}
@(test)
test_inverted_section :: proc(t: ^testing.T) {
data := map[string]bool {
"show" = false,
}
tpl, _ := mustache.parse("{{^show}}hidden{{/show}}")
result, err := mustache.render(tpl, data)
testing.expect(t, err == nil)
testing.expect(t, result == "hidden")
}
@(test)
test_unescaped :: proc(t: ^testing.T) {
data := map[string]string {
"html" = "<b>bold</b>",
}
tpl, _ := mustache.parse("{{{html}}}")
result, err := mustache.render(tpl, data)
testing.expect(t, err == nil)
testing.expect(t, result == "<b>bold</b>")
}
@(test)
test_array_iteration :: proc(t: ^testing.T) {
items := make([dynamic]map[string]string)
defer delete(items)
append(&items, map[string]string{"name" = "Alice"})
append(&items, map[string]string{"name" = "Bob"})
data := map[string][dynamic]map[string]string {
"items" = items,
}
tpl, _ := mustache.parse("{{#items}}{{name}} {{/items}}")
result, err := mustache.render(tpl, data)
testing.expect(t, err == nil)
testing.expect(t, result == "Alice Bob ")
}
@(test)
test_partial :: proc(t: ^testing.T) {
data := map[string]string {
"name" = "Test",
}
greeting_tpl, _ := mustache.parse("Hello, {{name}}!")
partials := map[string]mustache.Template {
"greeting" = greeting_tpl,
}
main_tpl, _ := mustache.parse("{{>greeting}}")
result, err := mustache.render(main_tpl, data, partials)
testing.expect(t, err == nil)
testing.expect(t, result == "Hello, Test!")
}
@(test)
test_mixed_types :: proc(t: ^testing.T) {
data := map[string]any {
"site_title" = "One Idiot Developer",
"has_date" = true,
"date" = "17 Jul 2025",
}
tpl, _ := mustache.parse("{{site_title}}: {{#has_date}}{{date}}{{/has_date}}")
result, err := mustache.render(tpl, data)
testing.expect(t, err == nil)
testing.expect(t, result == "One Idiot Developer: 17 Jul 2025")
}
@(test)
test_nested_context :: proc(t: ^testing.T) {
page := map[string]string {
"title" = "My Post",
}
data := map[string]any {
"site_title" = "One Idiot Developer",
"page" = page,
}
tpl, _ := mustache.parse("{{site_title}}: {{#page}}{{title}}{{/page}}")
result, err := mustache.render(tpl, data)
testing.expect(t, err == nil)
testing.expect(t, result == "One Idiot Developer: My Post")
}
+41 -46
View File
@@ -15,7 +15,6 @@ Page_Context :: struct {
permalink: string,
title: string,
starred: bool,
has_date: bool,
date_iso: string,
date_display: string,
}
@@ -43,7 +42,6 @@ Base_Data :: struct {
Page_Data :: struct {
using base: Base_Data,
page_title: string,
has_date: bool,
date_iso: string,
date_display: string,
is_post: bool,
@@ -66,7 +64,6 @@ build_page_context :: proc(page: Page) -> Page_Context {
permalink = page.permalink,
title = page.title,
starred = page.is_starred,
has_date = page.date != "",
date_iso = page.date,
date_display = format_date(page.date),
}
@@ -129,13 +126,14 @@ render_template :: proc(
return result
}
render_site :: proc(pages: []Page, config: Site) {
render_site :: proc(site: ^Site) {
pages := site.pages[:]
sort_pages_by_date(pages)
// Load shared resources once
partials := load_partials(config.layouts_dir)
partials["base"] = load_template(config.layouts_dir, "base.html")
post_tpl := load_template(config.layouts_dir, "post.html")
partials := load_partials(site.layouts_dir)
partials["base"] = load_template(site.layouts_dir, "base.html")
post_tpl := load_template(site.layouts_dir, "post.html")
now, ok := time.time_to_datetime(time.now())
assert(ok)
@@ -143,11 +141,11 @@ render_site :: proc(pages: []Page, config: Site) {
// Build base data once
base := Base_Data {
now = now,
author = config.author,
params = config.params,
og_site_name = config.title,
og_description = config.description,
og_image = fmt.tprintf("%s/avatar.jpg", config.base_url),
author = site.author,
params = site.params,
og_site_name = site.title,
og_description = site.description,
og_image = fmt.tprintf("%s/avatar.jpg", site.base_url),
}
// Find home page
@@ -166,56 +164,56 @@ render_site :: proc(pages: []Page, config: Site) {
if page.type == .Home {
continue
}
html := render_page_html(page, config, post_tpl, partials, base)
if .Minify in config.features {
html := render_page_html(page, site, post_tpl, partials, base)
if .Minify in site.features {
html = minify_html(html)
}
write_page(config.output_dir, page.permalink, html)
write_page(site.output_dir, page.permalink, html)
}
// Render home page
if has_home {
home_tpl := load_template(config.layouts_dir, "home.html")
home_html := render_home_html(home, pages, config, home_tpl, partials, base)
if .Minify in config.features {
home_tpl := load_template(site.layouts_dir, "home.html")
home_html := render_home_html(home, site, home_tpl, partials, base)
if .Minify in site.features {
home_html = minify_html(home_html)
}
write_file(fmt.tprintf("%s/index.html", config.output_dir), home_html)
write_file(fmt.tprintf("%s/index.html", site.output_dir), home_html)
}
// Render posts list page
posts_tpl := load_template(config.layouts_dir, "posts_list.html")
posts_html := render_posts_html(pages, config, posts_tpl, partials, base)
if .Minify in config.features {
posts_tpl := load_template(site.layouts_dir, "posts_list.html")
posts_html := render_posts_html(site, posts_tpl, partials, base)
if .Minify in site.features {
posts_html = minify_html(posts_html)
}
write_page(config.output_dir, "/posts/", posts_html)
write_page(site.output_dir, "/posts/", posts_html)
// Generate RSS feed
rss := generate_rss(pages, config)
write_file(fmt.tprintf("%s/index.xml", config.output_dir), rss)
rss := generate_rss(site)
write_file(fmt.tprintf("%s/index.xml", site.output_dir), rss)
// Generate sitemap
sitemap := generate_sitemap(pages, config.base_url)
write_file(fmt.tprintf("%s/sitemap.xml", config.output_dir), sitemap)
sitemap := generate_sitemap(site)
write_file(fmt.tprintf("%s/sitemap.xml", site.output_dir), sitemap)
// Copy and optionally minify assets directory
copy_assets_dir(config.assets_dir, config.output_dir, config.features)
copy_assets_dir(site.assets_dir, site.output_dir, site.features)
// Generate robots.txt
robots := fmt.aprintf("User-agent: *\nAllow: /\nSitemap: %s/sitemap.xml\n", config.base_url)
write_file(fmt.tprintf("%s/robots.txt", config.output_dir), robots)
robots := fmt.aprintf("User-agent: *\nAllow: /\nSitemap: %s/sitemap.xml\n", site.base_url)
write_file(fmt.tprintf("%s/robots.txt", site.output_dir), robots)
total := len(pages) + 1
if !has_home {
total += 1
}
fmt.printfln("Rendered %d pages to %s", total, config.output_dir)
fmt.printfln("Rendered %d pages to %s", total, site.output_dir)
}
render_page_html :: proc(
page: Page,
config: Site,
site: ^Site,
content_tpl: mustache.Template,
partials: map[string]mustache.Template,
base: Base_Data,
@@ -224,14 +222,13 @@ render_page_html :: proc(
data := Page_Data {
base = base,
}
data.title = fmt.tprintf("%s | %s", page.title, config.title)
data.title = fmt.tprintf("%s | %s", page.title, site.title)
data.page_title = page.title
data.body = page.body_html
data.has_date = page.date != ""
data.date_iso = page.date
data.date_display = format_date(page.date)
data.is_post = is_article
data.og_url = fmt.tprintf("%s%s", config.base_url, page.permalink)
data.og_url = fmt.tprintf("%s%s", site.base_url, page.permalink)
data.og_title = strip_html_tags(page.title)
data.og_type = og_type(is_article)
data.is_article = is_article
@@ -242,15 +239,14 @@ render_page_html :: proc(
render_home_html :: proc(
home: Page,
pages: []Page,
config: Site,
site: ^Site,
content_tpl: mustache.Template,
partials: map[string]mustache.Template,
base: Base_Data,
) -> string {
list_pages := make([dynamic]Page_Context)
defer delete(list_pages)
for page in pages {
for page in site.pages {
if page.type == .Home {
continue
}
@@ -260,19 +256,18 @@ render_home_html :: proc(
data := Home_Data {
base = base,
}
data.title = config.title
data.title = site.title
data.body = home.body_html
data.list_pages = list_pages
data.og_url = fmt.tprintf("%s/", config.base_url)
data.og_title = config.title
data.og_url = fmt.tprintf("%s/", site.base_url)
data.og_title = site.title
data.og_type = "website"
data.is_article = false
return render_template(content_tpl, data, partials)
}
render_posts_html :: proc(
pages: []Page,
config: Site,
site: ^Site,
content_tpl: mustache.Template,
partials: map[string]mustache.Template,
base: Base_Data,
@@ -280,7 +275,7 @@ render_posts_html :: proc(
year_sections := make([dynamic]Year_Section)
defer delete(year_sections)
current_year := ""
for page in pages {
for page in site.pages {
if page.type != .Post {
continue
}
@@ -295,9 +290,9 @@ render_posts_html :: proc(
data := Posts_Data {
base = base,
}
data.title = fmt.tprintf("Posts | %s", config.title)
data.title = fmt.tprintf("Posts | %s", site.title)
data.year_sections = year_sections
data.og_url = fmt.tprintf("%s/posts/", config.base_url)
data.og_url = fmt.tprintf("%s/posts/", site.base_url)
data.og_title = "Posts"
data.og_type = "website"
data.is_article = false
+146 -89
View File
@@ -9,8 +9,10 @@ import "core:os"
import "core:strings"
// Site is the primary workhorse.
Site :: struct {
arena: mem.Dynamic_Arena,
pages: [dynamic]Page,
title: string,
description: string,
author: string,
@@ -20,38 +22,72 @@ Site :: struct {
assets_dir: string,
output_dir: string,
layouts_dir: string,
params: json.Value,
params: json.Object,
features: bit_set[Feature],
markdown_extensions: bit_set[Markdown_Extension],
}
Feature :: enum {
Sections,
Drafts,
Minify,
Watch,
}
Flags :: struct {
config_path: string `args:"name=config"`,
Markdown_Extension :: enum {
Emoji,
Sidenotes,
Alerts,
Highlight,
Sections,
}
DEFAULT_MARKDOWN_EXTENSIONS :: bit_set[Markdown_Extension] {
.Emoji,
.Sidenotes,
.Alerts,
// .Highlight,
// .Sections,
}
// Configuration loaded from `thor.json`. Gets folded in to Site before
// Flags
Config_File :: struct {
title: string,
description: string,
base_url: string `args:"name=base-url"`,
content_dir: string `args:"name=content"`,
assets_dir: string `args:"name=assets"`,
output_dir: string `args:"name=output"`,
layouts_dir: string,
base_url: string,
author: string,
content_dir: string,
assets_dir: string,
output_dir: string,
layouts_dir: string,
markdown_extensions: json.Value,
params: json.Value,
sectionate: bool `args:"name=sections"`,
drafts: bool `args:"name=drafts"`,
watch: bool,
minify: bool `args:"name=minify"`,
}
// Configuration loaded from command line arguments. Gets folded in to Site
// after Config_File
Flags :: struct {
config_path: string `args:"name=config" usage:"Path to thor.json config file"`,
base_url: string `args:"name=base-url" usage:"Site base URL (e.g. https://example.com)"`,
content_dir: string `args:"name=content" usage:"Path to content directory"`,
assets_dir: string `args:"name=assets" usage:"Path to assets directory (CSS, JS, fonts, images)"`,
output_dir: string `args:"name=output" usage:"Path to output directory (default: public/)"`,
drafts: bool `args:"name=drafts" usage:"Include draft pages in the build"`,
watch: bool `usage:"Rebuild on file changes (polls every 5 seconds)"`,
minify: bool `args:"name=minify" usage:"Minify HTML output and CSS assets"`,
md_enable: string `args:"name=ext" usage:"Enable markdown extensions (comma-separated: emoji,sidenotes,alerts,highlight,sections)"`,
md_disable: string `args:"name=no-ext" usage:"Disable markdown extensions (comma-separated: emoji,sidenotes,alerts,highlight,sections)"`,
}
init_site :: proc(site: ^Site, args: []string) {
_flags: Flags
mem.dynamic_arena_init(&site.arena, alignment = 64) // FIXME: This is a hack
mem.dynamic_arena_init(&site.arena, alignment = 64)
alloc := site_allocator(site)
// Set defaults
site.base_url = "http://localhost:8080"
site.markdown_extensions = DEFAULT_MARKDOWN_EXTENSIONS
_flags: Flags
flags.parse_or_exit(&_flags, args, .Odin, alloc)
path := _flags.config_path
@@ -65,106 +101,127 @@ init_site :: proc(site: ^Site, args: []string) {
}
}
cfg, cfg_ok := load_site_config(path, alloc)
if cfg_ok {
merge_flags(&cfg, _flags)
} else {
cfg = _flags
}
config: Config_File
config_loaded := load_config_file(&config, path, alloc)
site_apply_flags(site, cfg)
// Determine config file's directory for relative defaults
config_dir := "./"
if idx := strings.last_index(path, "/"); idx >= 0 {
config_dir = path[:idx]
}
// Hardcoded defaults (lowest precedence)
// TODO: Probably shouldn't use temp allocator here?
if site.content_dir == "" {
site.content_dir = fmt.tprintf("%s/content", config_dir)
}
if site.assets_dir == "" {
site.assets_dir = fmt.tprintf("%s/assets", config_dir)
}
if site.output_dir == "" {
site.output_dir = fmt.tprintf("%s/public", config_dir)
}
if site.layouts_dir == "" {
site.layouts_dir = fmt.tprintf("%s/layouts", config_dir)
}
if site.base_url == "" {
site.base_url = "http://localhost:8080"
}
if config_loaded {
site_apply_config(site, config, config_dir)
} else {
site_apply_path_defaults(site, config_dir)
}
load_site_config :: proc(
site_apply_cli_flags(site, _flags)
site.config_path = path
}
load_config_file :: proc(
config: ^Config_File,
path: string,
allocator := context.allocator,
) -> (
config: Flags,
ok: bool,
) {
) -> bool {
data, err := os.read_entire_file_from_path(path, allocator)
if err != nil {
return
return false
}
unmarshal_err := json.unmarshal_string(string(data), &config, allocator = allocator)
unmarshal_err := json.unmarshal_string(string(data), config, allocator = allocator)
if unmarshal_err != nil {
log.warnf("thor: failed to parse %s: %v", path, unmarshal_err)
return
return false
}
ok = true
return
return true
}
merge_flags :: proc(config: ^Flags, flags: Flags) {
if flags.base_url != "" {
config.base_url = flags.base_url
}
if flags.content_dir != "" {
config.content_dir = flags.content_dir
}
if flags.assets_dir != "" {
config.assets_dir = flags.assets_dir
}
if flags.output_dir != "" {
config.output_dir = flags.output_dir
}
if flags.drafts {
config.drafts = true
}
if flags.watch {
config.watch = true
}
if flags.sectionate {
config.sectionate = true
}
if flags.minify {
config.minify = true
}
config.config_path = flags.config_path
site_apply_config :: proc(site: ^Site, config: Config_File, config_dir: string) {
if config.title != "" do site.title = config.title
if config.description != "" do site.description = config.description
if config.author != "" do site.author = config.author
if config.base_url != "" do site.base_url = config.base_url
site.content_dir =
config.content_dir if config.content_dir != "" else fmt.tprintf("%s/content", config_dir)
site.assets_dir =
config.assets_dir if config.assets_dir != "" else fmt.tprintf("%s/assets", config_dir)
site.output_dir =
config.output_dir if config.output_dir != "" else fmt.tprintf("%s/public", config_dir)
site.layouts_dir =
config.layouts_dir if config.layouts_dir != "" else fmt.tprintf("%s/layouts", config_dir)
if params, ok := config.params.(json.Object); ok {
site.params = params
}
site_apply_flags :: proc(site: ^Site, flags: Flags) {
site.title = flags.title
site.description = flags.description
site.author = flags.author
site.base_url = flags.base_url
site.config_path = flags.config_path
site.content_dir = flags.content_dir
site.assets_dir = flags.assets_dir
site.output_dir = flags.output_dir
site.layouts_dir = flags.layouts_dir
site.params = flags.params
// Apply markdown extensions from config
if ext_obj, ok := config.markdown_extensions.(json.Object); ok {
apply_extension_config(&site.markdown_extensions, ext_obj)
}
}
site_apply_path_defaults :: proc(site: ^Site, config_dir: string) {
site.content_dir = fmt.tprintf("%s/content", config_dir)
site.assets_dir = fmt.tprintf("%s/assets", config_dir)
site.output_dir = fmt.tprintf("%s/public", config_dir)
site.layouts_dir = fmt.tprintf("%s/layouts", config_dir)
}
site_apply_cli_flags :: proc(site: ^Site, flags: Flags) {
if flags.base_url != "" do site.base_url = flags.base_url
if flags.content_dir != "" do site.content_dir = flags.content_dir
if flags.assets_dir != "" do site.assets_dir = flags.assets_dir
if flags.output_dir != "" do site.output_dir = flags.output_dir
if flags.sectionate {site.features += {.Sections}}
if flags.drafts {site.features += {.Drafts}}
if flags.watch {site.features += {.Watch}}
if flags.minify {site.features += {.Minify}}
site.markdown_extensions += parse_extension_list(flags.md_enable)
site.markdown_extensions -= parse_extension_list(flags.md_disable)
}
parse_extension_list :: proc(s: string) -> bit_set[Markdown_Extension] {
result: bit_set[Markdown_Extension]
if s == "" do return result
for part in strings.split(s, ",", allocator = context.temp_allocator) {
name := strings.to_lower(strings.trim_space(part), allocator = context.temp_allocator)
switch name {
case "emoji":
result += {.Emoji}
case "sidenotes":
result += {.Sidenotes}
case "alerts":
result += {.Alerts}
case "highlight":
result += {.Highlight}
case "sections":
result += {.Sections}
}
}
return result
}
apply_extension_config :: proc(ext: ^bit_set[Markdown_Extension], config: json.Object) {
for name, val in config {
enabled, is_bool := val.(json.Boolean)
if !is_bool do continue
switch name {
case "emoji":
if enabled {ext^ += {.Emoji}} else {ext^ -= {.Emoji}}
case "sidenotes":
if enabled {ext^ += {.Sidenotes}} else {ext^ -= {.Sidenotes}}
case "alerts":
if enabled {ext^ += {.Alerts}} else {ext^ -= {.Alerts}}
case "highlight":
if enabled {ext^ += {.Highlight}} else {ext^ -= {.Highlight}}
case "sections":
if enabled {ext^ += {.Sections}} else {ext^ -= {.Sections}}
}
}
}
site_allocator :: proc(site: ^Site) -> mem.Allocator {
+35 -75
View File
@@ -4,6 +4,7 @@ package main
import "core:encoding/json"
import "core:fmt"
import "core:log"
import "core:os"
import "core:testing"
@@ -17,7 +18,7 @@ write_temp_config :: proc(name: string, content: string) -> string {
}
@(test)
test_load_site_config :: proc(t: ^testing.T) {
test_load_config_file :: proc(t: ^testing.T) {
path := write_temp_config(
"valid",
`{
@@ -34,7 +35,8 @@ test_load_site_config :: proc(t: ^testing.T) {
)
defer os.remove(path)
cfg, ok := load_site_config(path, context.temp_allocator)
cfg: Config_File
ok := load_config_file(&cfg, path, context.temp_allocator)
testing.expect(t, ok)
testing.expect_value(t, cfg.title, "Test Site")
@@ -61,26 +63,33 @@ test_load_site_config :: proc(t: ^testing.T) {
}
@(test)
test_load_site_config_missing_file :: proc(t: ^testing.T) {
_, ok := load_site_config("./nonexistent_thor_test.json", context.temp_allocator)
test_load_config_file_missing :: proc(t: ^testing.T) {
cfg: Config_File
ok := load_config_file(&cfg, "./nonexistent_thor_test.json", context.temp_allocator)
testing.expect(t, !ok)
}
@(test)
test_load_site_config_invalid_json :: proc(t: ^testing.T) {
test_load_config_file_invalid_json :: proc(t: ^testing.T) {
path := write_temp_config("invalid", `{not valid json}`)
defer os.remove(path)
_, ok := load_site_config(path, context.temp_allocator)
cfg: Config_File
ok := false
{
context.logger = log.nil_logger()
ok = load_config_file(&cfg, path, context.temp_allocator)
}
testing.expect(t, !ok)
}
@(test)
test_load_site_config_partial :: proc(t: ^testing.T) {
test_load_config_file_partial :: proc(t: ^testing.T) {
path := write_temp_config("partial", `{"title":"Partial"}`)
defer os.remove(path)
cfg, ok := load_site_config(path, context.temp_allocator)
cfg: Config_File
ok := load_config_file(&cfg, path, context.temp_allocator)
testing.expect(t, ok)
testing.expect_value(t, cfg.title, "Partial")
@@ -89,73 +98,6 @@ test_load_site_config_partial :: proc(t: ^testing.T) {
testing.expect(t, cfg.params == nil)
}
@(test)
test_site_merge_overrides :: proc(t: ^testing.T) {
config := Flags {
base_url = "https://original.com",
content_dir = "./content",
}
flags := Flags {
base_url = "https://override.com",
}
merge_flags(&config, flags)
testing.expect_value(t, config.base_url, "https://override.com")
testing.expect_value(t, config.content_dir, "./content")
}
@(test)
test_site_merge_empty_flags_keep_config :: proc(t: ^testing.T) {
config := Flags {
base_url = "https://keep.com",
content_dir = "./keep",
}
flags := Flags{}
merge_flags(&config, flags)
testing.expect_value(t, config.base_url, "https://keep.com")
testing.expect_value(t, config.content_dir, "./keep")
}
@(test)
test_site_merge_drafts_true :: proc(t: ^testing.T) {
config := Flags {
drafts = false,
}
flags := Flags {
drafts = true,
}
merge_flags(&config, flags)
testing.expect(t, config.drafts)
}
@(test)
test_site_merge_drafts_false_preserves :: proc(t: ^testing.T) {
config := Flags {
drafts = false,
}
flags := Flags {
drafts = false,
}
merge_flags(&config, flags)
testing.expect(t, !config.drafts)
}
@(test)
test_site_merge_config_path :: proc(t: ^testing.T) {
config := Flags{}
flags := Flags {
config_path = "./custom/thor.json",
}
merge_flags(&config, flags)
testing.expect_value(t, config.config_path, "./custom/thor.json")
}
@(test)
test_init_site_defaults_no_config :: proc(t: ^testing.T) {
site: Site
@@ -164,9 +106,13 @@ test_init_site_defaults_no_config :: proc(t: ^testing.T) {
defer destroy_site(&site)
testing.expect_value(t, site.content_dir, "./content")
testing.expect_value(t, site.assets_dir, "./assets")
testing.expect_value(t, site.output_dir, "./public")
testing.expect_value(t, site.layouts_dir, "./layouts")
testing.expect_value(t, site.base_url, "http://localhost:8080")
testing.expect(t, .Emoji in site.markdown_extensions)
testing.expect(t, .Sidenotes in site.markdown_extensions)
testing.expect(t, .Alerts in site.markdown_extensions)
}
@(test)
@@ -177,6 +123,7 @@ test_init_site_config_dir_relative :: proc(t: ^testing.T) {
defer destroy_site(&site)
testing.expect_value(t, site.content_dir, "./sub/content")
testing.expect_value(t, site.assets_dir, "./sub/assets")
testing.expect_value(t, site.output_dir, "./sub/public")
testing.expect_value(t, site.layouts_dir, "./sub/layouts")
}
@@ -212,3 +159,16 @@ test_init_site_full_pipeline :: proc(t: ^testing.T) {
testing.expect_value(t, site.base_url, "https://config.com")
}
@(test)
test_init_site_md_enable_disable :: proc(t: ^testing.T) {
site: Site
args := []string{"thor", "-config:./nonexistent.json", "-ext:highlight,sections", "-no-ext:emoji"}
init_site(&site, args)
defer destroy_site(&site)
testing.expect(t, .Highlight in site.markdown_extensions)
testing.expect(t, .Sections in site.markdown_extensions)
testing.expect(t, !(.Emoji in site.markdown_extensions))
testing.expect(t, .Sidenotes in site.markdown_extensions)
}