thor/TODOS.md

TODOs

Priority Unclear (Evaluate Later)

• Decide the parallel-parsing strategy for the grammar cache (pool vs thread-local parsers) — deferred until the parallel usage shape is clear.
  • Resolved already: the registry (grammar()) is thread-safe and self- initializing, its allocator is pinned to the heap, and parsers/cursors are now caller-owned (open_parser) instead of cached — so the library holds no single-thread-use state and the multi-threaded tests pass without locks.
  • Remaining choice only matters once we actually parse in parallel: reuse parsers via a pool or per-thread storage, plugged in behind open_parser without touching call sites. A TSParser/TSQueryCursor is one-per-thread.
  • Full write-up: GRAMMAR_CACHE_THREADING.md.
• Restore parallel grammar preload (BEHAVIOUR CHANGE in edb2e64). preload_grammars used to dlopen+compile each grammar on its own thread; it is now serial (grammar() per lang on the calling thread). The startup dlopen cost is no longer overlapped.
  • Why serial for now: grammar() holds registry.mu across the whole dlopen+compile, so naively fanning out grammar() calls just queues every worker on that one lock — zero real parallelism.
  • To bring it back: move build_grammar (dlopen + compile_query, no shared mutable state) OUTSIDE the lock; take mu only to (1) init the registry + snapshot which langs are missing, and (2) briefly publish each finished ^Grammar. Restore the thread-based worker fan-out + SPALL init/cleanup hooks that edb2e64 deleted. Safe because glibc dlopen, the heap allocator, and each thread's own temp allocator are all thread-safe.
  • An RW_Mutex does NOT help the parallel-load side (writers stay exclusive); it only helps a future multi-threaded read path (concurrent cache-hit grammar() lookups during a parallel render). Consider it only if/when render itself goes multi-threaded — then pair rlock hits with off-lock build
    • a double-check-and-discard on the miss race (needs a free_grammar).
  • Assumption to preserve: preload completes before any concurrent grammar() use (render is single-threaded and runs after preload).
• instead of warnings: [dynamic]Error we should use warnings: [dynamic; 8]Error
  • when limit reached, the template fails and stops rendering.
• Brainstorm plugin/tool/(postccs/less/tailwind/sass) support.

General

Remember to polish existing features before moving on to new ones.

• Bake the default layouts into the binary at compile time via #load_directory so the binary is self-contained and doesn't read DEFAULTS_PATH from disk at runtime.
  • #load_directory is NOT recursive and its name is the base filename only, so defaults/layouts and defaults/layouts/partials must be loaded separately and mounted into the VFS with data set (not fs_path).
  • Removes the runtime dependency on THOR_DEFAULTS_PATH / #directory and the need to ship defaults/ alongside the binary (see flake.nix).
• build command alias of default
• new site command to set up new project
• Some render errors blank the entire page instead of scoping to the failing tag.
• We must remove all mention of posts from the odin code. At present, "posts" are a user-level construct defined as pages in a particular collection.
• ? Clean up the default layouts (no idea what this means)
• group_by currently requires a computed year field on the page.
  • We should replace this with {{ pages | group_by (date | "2006") }} or similar
• does it make sense for partials to be inside layouts?
  • current: layouts/partials
  • alt1: layouts, partials
  • alt2: templates/layouts, templates/partials
• add sort_by pipe
• pipes
  • first/last
    • write test for error when n == 0
    • warn when using negative numbers
    • warn when using default arg on string
    • add the above warnings to DIAGNOSTIC_TESTS.md
    • Instead of "clearer form", be exact: "last 5"
    • write tests for first 0 & last 0
    • update tests to assert diagnostics when appropriate
      • do we want to do this or just use DIAGNOSTIC_TESTS.md as the tests?
      • also update to use file logger so they don't clog up the output
    • narrow highlight in diagnostic to relevent part of line.
• page.url and page.permalink are confusing names. Switch to url + page_url
  • There is confusion over how to do this and whethor or not | rel_url should be needed to render these. Requires careful thought.

Documentation

• talk about the context stack (and its limit).
• highlight the differences in the way menus are handled.
• explain that html/css/js comments are rendered, except in minify mode.

Investigate

• running ./thor/thor still logs the debug message: using config /home/spencer/github.com/sbrow.github.io/thor.json
  • wrong cwd?

Diagnostics

All diagnostics (errors) should show

• What went wrong

• where (in the file)

• where (in the stack trace)

• how you can fix it (if applicable)

• try to make file paths clickable links.

• make sure the frontmatter parser has good diagnostics.

• add all known diagnostics to DIAGNOSTIC TODOS.

• fix the broken diagnostics in DIAGNOSTIC TODOS

• show stack trace for template partials?

• show where the typo occurred (config file path for apply_extension_config, full -ext:value for parse_extension_list)

• all diagnostics in DIAGNOSTIC TODOS must be tested.

• only report format errors once.

• only report missing partial errors once.

• better diagnostics for syntax errors in treesitter.

• Ensure diagnostics for MAX_CONTEXT_DEPTH are good.

• improve matching weights message.

• Test menu diagnostics

• Extend tag_error to all render-time errors, not just pipe errors. Currently only pipe errors (4 sites in render_nodes) get stamped with the correct template source/path. Other render errors still use the content template's source/path, which can point at the wrong file.

Analytics

Debug log stats:

• final Context_Stack cap
• highest PIPE args used
• longest slug length + name that generated it
• number of pages
• number of blocks
• enabled features / extensions
• etc

sbrow.github.io

• The home file was accidentally deleted in commit 9a9f731
• Ideas is now in the wrong spot. Date is wrong, and it is showing date when it shouldn't be.