mirror of
https://github.com/sbrow/envr.git
synced 2026-06-27 10:38:33 -04:00
Compare commits
13 Commits
9ebe789a67
...
7174377d21
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7174377d21 | ||
| 19d03ff71a | |||
| ce57009b92 | |||
| f51c0d6755 | |||
| cbab562d62 | |||
| 116ed6de4c | |||
| 3e5889d5c0 | |||
| 6fe61251ee | |||
| ba56748cc0 | |||
| 0b380c3674 | |||
| 9fcf09601e | |||
| c1e93b66e0 | |||
| 440f944b33 |
4
.gitignore
vendored
4
.gitignore
vendored
@@ -7,8 +7,12 @@ list.json
|
||||
man
|
||||
|
||||
# build artifacts
|
||||
*.spall
|
||||
builds
|
||||
envr
|
||||
envr-go
|
||||
findr/findr
|
||||
findr/findr-prof
|
||||
findr/bench-*.md
|
||||
result
|
||||
version.odin
|
||||
|
||||
17
CHANGELOG.md
17
CHANGELOG.md
@@ -1,5 +1,22 @@
|
||||
# Changelog
|
||||
|
||||
## [0.3.1](https://github.com/sbrow/envr/compare/v0.3.0...v0.3.1) (2026-06-17)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **find:** Ignored volatile dirs in benchmarks. ([6fe6125](https://github.com/sbrow/envr/commit/6fe61251ee2fe86bca3b50a5893699b5a71b1af8))
|
||||
|
||||
|
||||
### Performance Improvements
|
||||
|
||||
* **findr:** Added spall support. ([ce57009](https://github.com/sbrow/envr/commit/ce57009b9217926c9e171cb86043463706882bc3))
|
||||
* **findr:** Each thread gets its own buffer. ([cbab562](https://github.com/sbrow/envr/commit/cbab562d62d74595ce5578ebaba8d839affa4e7f))
|
||||
* **findr:** Improved performance of `join_path*` procedures. ([f51c0d6](https://github.com/sbrow/envr/commit/f51c0d6755a3ba6a54d9292c1a95e13d460df7b9))
|
||||
* **findr:** Replaced regex engine with glob. ([19d03ff](https://github.com/sbrow/envr/commit/19d03ff71a3ae0e2f8f1d4fe8e7e0cd11165405f))
|
||||
* **findr:** Use buffered writer. ([116ed6d](https://github.com/sbrow/envr/commit/116ed6de4c3e08909b25f35c458913022cafefdd))
|
||||
* Replaced `fd` with custom internals. ([440f944](https://github.com/sbrow/envr/commit/440f944b33b20cc5f86a7716af0ba6e793c6c094))
|
||||
|
||||
## [0.3.0](https://github.com/sbrow/envr/compare/v0.2.1...v0.3.0) (2026-06-16)
|
||||
|
||||
Version 0.3.0 represents a significant departure (and improvement) for envr.
|
||||
|
||||
2
TODOS.md
2
TODOS.md
@@ -32,8 +32,6 @@
|
||||
|
||||
18. 2 scan tests silently skip when fd isn't installed, tests pass without actually testing anything. These should use #assert to be sure that fd is in path.
|
||||
|
||||
19. Try to do all encryption / decryption in memory - only read / write encrypted data to disk.
|
||||
|
||||
20. add --format -f flag to commands that draw tables.
|
||||
|
||||
21. Replace `testing.expect` calls with `testing.expect_value` calls where appropriate.
|
||||
|
||||
34
findr/PERFORMANCE_IDEAS.md
Normal file
34
findr/PERFORMANCE_IDEAS.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# Performance Ideas
|
||||
|
||||
Current state after regex→glob migration. findr beats fd in 3/4 cases.
|
||||
|
||||
## Benchmark results (2026-06-17)
|
||||
|
||||
| Case | fd | findr | Ratio |
|
||||
|------|------|-------|-------|
|
||||
| 1 `-E .jj` | 172ms | 135ms | **1.27x faster** |
|
||||
| 2 `-H` | 1.184s | 1.097s | **1.08x faster** |
|
||||
| 3 `-HI` | 1.251s | 1.670s | **1.34x slower** |
|
||||
| 4 `-E .git` | 274ms | 202ms | **1.36x faster** |
|
||||
|
||||
Case 3 (`-HI`) skips gitignore entirely, so it's pure I/O + allocation. System time is 2x fd's (12.1s vs 5.5s), pointing to syscall/allocation overhead.
|
||||
|
||||
## Completed
|
||||
|
||||
1. **Per-thread result buffers** — each thread accumulates locally, merges once at exit. Eliminates per-result mutex contention.
|
||||
2. **Lean path join** — `join_path`/`join_path_dir` use stack buffer + `copy` + single alloc instead of `strings.Builder` + `fmt.sbprintf` + `clone`.
|
||||
3. **Regex→glob migration** — replaced regex NFA with backtracking glob matcher. Eliminated 27% of CPU spent on `add_thread`/`is_ignored`. Biggest win.
|
||||
|
||||
## Remaining ideas
|
||||
|
||||
1. **Larger getdents buffer** (8KB → 64KB+)
|
||||
Fewer syscalls per directory with many entries. Low effort.
|
||||
|
||||
2. **Eliminate entry name cloning**
|
||||
`strings.clone(name)` in `read_dir_entries` heap-allocates per dirent. Names are valid in the getdents buffer during `process_dir`, so the clone may be unnecessary. Low effort.
|
||||
|
||||
3. **Arena allocator per thread**
|
||||
Bump allocator for all transient strings, free once at exit. Bigger change, helps everywhere.
|
||||
|
||||
4. **Batched channel** (fd's approach)
|
||||
Replace global results array with buffered channel of batches. Enables streaming output and sorting like fd does.
|
||||
342
findr/PLAN.md
Normal file
342
findr/PLAN.md
Normal file
@@ -0,0 +1,342 @@
|
||||
# findr — Native Odin File Finder (fd Replacement)
|
||||
|
||||
## Overview
|
||||
|
||||
findr is a native Odin file finder that replaces `fd` in envr. It supports three ignore modes for A/B benchmarking against specific fd commands, plus a unique "emit ONLY gitignored files" mode that gives envr a single-pass advantage over fd's double-run-and-diff approach.
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
findr/
|
||||
findr.odin # main + CLI (hand-rolled arg parsing)
|
||||
walker.odin # parallel directory walker (getdents + thread pool)
|
||||
gitignore.odin # .gitignore parsing + glob→regex transpilation + matching
|
||||
test_env.odin # test harness: temp dir, mock filesystem, assert helpers
|
||||
findr_test.odin # integration tests
|
||||
gitignore_test.odin # transpilation + matching unit tests (22 tests)
|
||||
```
|
||||
|
||||
## CLI Interface
|
||||
|
||||
```
|
||||
findr [-I] [--ignored] [--no-hidden] [-E <glob>]... [pattern] [path]...
|
||||
```
|
||||
|
||||
Defaults: `include_hidden=true, ignore_mode=.Respected` (matches fd's `-H` behavior).
|
||||
|
||||
| fd command | findr equivalent |
|
||||
|---|---|
|
||||
| `fd -a \.env -E ... -HI ~/` | `findr -I -E ... \.env ~/` |
|
||||
| `fd -a \.env -E ... -H ~/` | `findr -E ... \.env ~/` |
|
||||
| `fd . -H ~/` | `findr ~/` |
|
||||
| `fd . -HI ~/` | `findr -I ~/` |
|
||||
| `fd . ~/` (no flags) | `findr --no-hidden ~/` |
|
||||
| *(findr original)* | `findr --ignored ~/` |
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
odin build findr -o:speed -out:findr/findr
|
||||
odin test findr
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
### Two Orthogonal Axes (matching fd's semantics)
|
||||
|
||||
1. **Hidden files** (`.` prefix): `include_hidden=true` includes them, `false` excludes them
|
||||
2. **Gitignore**: three modes (see `IgnoreMode` below)
|
||||
|
||||
### Types
|
||||
|
||||
```odin
|
||||
IgnoreMode :: enum {
|
||||
Respected, // skip gitignored, prune ignored dirs (fd -H default)
|
||||
All, // ignore .gitignore entirely, descend everywhere (fd -HI)
|
||||
Ignored, // emit ONLY gitignored files, prune ignored dirs (findr original)
|
||||
}
|
||||
|
||||
WalkOptions :: struct {
|
||||
pattern: string, // regex on basename; "" = match all
|
||||
excludes: []string, // glob patterns to skip entirely (fd -E)
|
||||
include_hidden: bool, // true = include dotfiles (fd -H)
|
||||
ignore_mode: IgnoreMode,
|
||||
}
|
||||
```
|
||||
|
||||
### process_dir Filtering Order Per Entry
|
||||
|
||||
Each directory traversal carries a `WorkItem` with the absolute path, a relative path from repo root, and a `^GIContext` linked list of gitignore contexts (one per ancestor directory with a `.gitignore`).
|
||||
|
||||
1. Skip `.git` directory
|
||||
2. **Load nested `.gitignore`**: If this directory has a `.gitignore`, push a new `GIContext` onto the chain (tracked in `pool.all_contexts` for cleanup)
|
||||
3. **Per entry**:
|
||||
- Skip non-regular files (symlinks, sockets, etc. — parity with `fd -t f`)
|
||||
- **Excludes**: if entry matches any exclude glob → skip entirely
|
||||
- **Hidden**: if `!include_hidden && name[0] == '.'` → skip entirely
|
||||
- **Gitignore status**: check `GIContext` chain deepest-to-root via `check_chain`, passing the **relative path** (not basename). First match wins (correct gitignore precedence). Nested negation overrides parent rules.
|
||||
- **Mode-based decision**:
|
||||
|
||||
| Mode | gitignored file | gitignored dir | normal file | normal dir |
|
||||
|---|---|---|---|---|
|
||||
| `.All` | emit if pattern matches | descend | emit if pattern matches | descend |
|
||||
| `.Respected` | skip | prune | emit if pattern matches | descend |
|
||||
| `.Ignored` | emit if pattern matches | prune | skip | descend |
|
||||
|
||||
**Nested repos**: When a directory contains `.git/`, the gitignore context chain is reset (new repo root). The relative path resets to `""`. Nested repos are always traversed to find deeper repos.
|
||||
|
||||
### Performance Architecture
|
||||
|
||||
- **Stat avoidance via `dirent.type`** — Uses `core:sys/linux` getdents directly, bypassing `core:os` which calls `openat` + `fstat` per entry.
|
||||
- **Prune ignored directories** — When a directory matches a gitignore/exclude pattern, it is not descended into.
|
||||
- **Parallel traversal** — Worker thread pool with shared LIFO queue and futex-based semaphore signaling. 5.4x speedup over serial on home directory.
|
||||
|
||||
## Decisions
|
||||
|
||||
- **Gitignore matching**: Transpile gitignore glob patterns to regex, then use `core:text/regex`. No dedicated glob matcher.
|
||||
- **Pattern matching**: Pattern is a regex (same as fd), matched against basename via `regex.match` (unanchored search).
|
||||
- **Excludes**: Glob patterns compiled via the same gitignore transpiler (`parse()`). Reuses tested transpilation logic.
|
||||
- **Nested gitignore**: Every `.gitignore` file within a repo is read, not just the root. Each directory's rules are scoped relative to that directory's path. Negation in a child overrides parent rules (correct gitignore precedence).
|
||||
- **Stat avoidance**: Use `core:sys/linux` getdents directly — read `dirent.type` from the kernel, never call stat. `DT_UNKNOWN` treated as regular file (correct for ext4/tmpfs; may miss dirs on XFS/BTRFS/FUSE — Phase 7 concern).
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- **In-process integration tests** — Tests call `walk()` directly (not via subprocess), build mock filesystems in temp dirs, and compare sorted output.
|
||||
- **Unit tests** — Pure-function tests for glob→regex transpilation and gitignore matching.
|
||||
- **Output sorting for determinism** — Always sort output lines before comparison.
|
||||
- **Memory tracking** — Odin's test runner reports leaks automatically.
|
||||
|
||||
### Test Coverage (findr_test.odin)
|
||||
|
||||
**`.Ignored` mode (original findr behavior):**
|
||||
|
||||
| Test | What it covers |
|
||||
|---|---|
|
||||
| `test_basic_gitignored` | Repo with `.gitignore`, gitignored files emitted, normal files skipped |
|
||||
| `test_non_repo_not_scanned` | Dirs without `.git/` produce no output |
|
||||
| `test_negation_pattern` | `!prod.env` un-ignores a file |
|
||||
| `test_dir_only_pattern` | `node_modules/` pattern doesn't emit file results |
|
||||
| `test_multiple_repos` | Multiple repos in one tree, each with its own `.gitignore` |
|
||||
| `test_nested_repos` | Repo inside a repo, both scanned independently |
|
||||
| `test_no_gitignore_file` | Repo with `.git/` but no `.gitignore` produces nothing |
|
||||
| `test_empty_gitignore` | Comments and blank lines only → no results |
|
||||
| `test_multiple_search_dirs` | Multiple top-level search dirs in one call |
|
||||
| `test_nested_gitignore_read` | Nested `.gitignore` rules applied (subdir patterns work) |
|
||||
| `test_nested_gitignore_negation` | Nested negation overrides parent pattern |
|
||||
| `test_multisegment_pattern` | `build/output.txt` matches relative path, not just basename |
|
||||
|
||||
**`.All` mode (fd -HI parity):**
|
||||
|
||||
| Test | What it covers |
|
||||
|---|---|
|
||||
| `test_all_mode_emits_all_files` | All files emitted regardless of gitignore |
|
||||
| `test_all_mode_descends_everywhere` | Gitignored dirs still descended |
|
||||
|
||||
**`.Respected` mode (fd -H parity):**
|
||||
|
||||
| Test | What it covers |
|
||||
|---|---|
|
||||
| `test_respected_mode_skips_gitignored` | Gitignored files skipped |
|
||||
| `test_respected_mode_prunes_ignored_dirs` | Gitignored dirs pruned |
|
||||
| `test_nested_gitignore_respected_mode` | Nested negation respected in `.Respected` mode |
|
||||
|
||||
**Filters:**
|
||||
|
||||
| Test | What it covers |
|
||||
|---|---|
|
||||
| `test_excludes_prune_dirs` | Excluded dirs not descended |
|
||||
| `test_pattern_filters_results` | Only pattern-matching files emitted |
|
||||
| `test_no_hidden_skips_dotfiles` | Hidden files skipped when include_hidden=false |
|
||||
|
||||
## Glob→Regex Transpilation Rules
|
||||
|
||||
| Gitignore pattern | Regex | Notes |
|
||||
|---|---|---|
|
||||
| `foo` | `(^|/)foo(/.*)?$` | matches at any depth |
|
||||
| `/foo` | `^foo(/.*)?$` | anchored to gitignore dir |
|
||||
| `foo/` | `(^|/)foo/.*$` | directory only |
|
||||
| `*.log` | `(^|/)[^/]*\.log$` | `*` = any chars except `/` |
|
||||
| `**/foo` | `(^|/)(.*/)?foo(/.*)?$` | `**` = any chars including `/` |
|
||||
| `foo/**/bar` | `(^|/)foo/(.*/)?bar(/.*)?$` | `**` between segments |
|
||||
| `!pattern` | (handled by layer) | negation flag, not regex |
|
||||
| `#comment` | (skipped) | |
|
||||
| `[abc]` | `[abc]` | same regex syntax |
|
||||
| `?` | `[^/]` | single char, no `/` |
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Gitignore Transpiler + Tests ✅
|
||||
|
||||
22 tests, all passing, zero leaks.
|
||||
|
||||
### Phase 2: findr Walker + Tests ✅
|
||||
|
||||
Parallel DFS using getdents with worker thread pool. 32 total tests pass, zero leaks.
|
||||
|
||||
### Phase 3: Parallel Traversal ✅
|
||||
|
||||
8-worker thread pool, shared LIFO queue, futex-based semaphore. 852ms vs 4.57s serial (5.4x speedup). Serial code removed — parallel is the only implementation.
|
||||
|
||||
### Phase 4: Benchmark ✅
|
||||
|
||||
findr found 227 gitignored files on `~` in 852ms. fd's double-run walked ~1.1M entries.
|
||||
|
||||
### Phase 5: fd-Parity API ✅
|
||||
|
||||
**Goal:** Make findr replicate specific fd commands for A/B benchmarking, plus keep the unique gitignored-only mode.
|
||||
|
||||
**Built:**
|
||||
- `IgnoreMode` enum (`.Respected`, `.All`, `.Ignored`) and `WalkOptions` struct
|
||||
- New `walk` signature: `walk(root, results, opts: WalkOptions, thread_count)`
|
||||
- Rewritten `process_dir` with centralized mode-based filtering
|
||||
- Pattern matching via `core:text/regex` on basenames
|
||||
- Exclude patterns compiled via existing `gitignore.parse()`
|
||||
- CLI arg parsing: `-I`, `--ignored`, `--no-hidden`, `-E <glob>`
|
||||
- 7 new integration tests (17 total) covering all three modes, excludes, pattern, and hidden filtering
|
||||
|
||||
**Result:** All tests pass (22 gitignore + 20 walker = 42), zero leaks.
|
||||
|
||||
### Phase 6: Parity (partially done)
|
||||
|
||||
**Goal:** Achieve file-count parity with fd. An invalid benchmark (different result sets) is useless.
|
||||
|
||||
#### Steps 1-2: Nested gitignore + relative path matching ✅
|
||||
|
||||
**What was done:**
|
||||
|
||||
1. **`Match` enum + `check_match`** in `gitignore.odin` — Tri-state return (`None`/`Ignored`/`Unignored`) so nested negation overrides work correctly. `is_ignored` wraps it as before.
|
||||
|
||||
2. **`GIContext` linked list** in `walker.odin` — Each context holds a `^Gitignore`, `base_rel` (relative path from repo root to this dir), and `parent: ^GIContext`. `process_dir` loads `.gitignore` in every directory within a repo (not just roots). `check_chain` walks deepest-to-root, first match wins (correct gitignore precedence).
|
||||
|
||||
3. **`WorkItem` struct** replaced plain `string` in the work queue:
|
||||
```odin
|
||||
WorkItem :: struct {
|
||||
path: string, // absolute directory path
|
||||
rel: string, // relative path from repo root ("" = root)
|
||||
gi_ctx: ^GIContext, // gitignore chain (nil = outside any repo)
|
||||
}
|
||||
```
|
||||
|
||||
4. **Relative path matching** — `check_chain` strips each context's `base_rel` prefix to get the locally-scoped relative path. Multi-segment patterns like `build/output.txt` now match correctly.
|
||||
|
||||
5. **Symlink filtering** — Only `DT_REG` and `DT_UNKNOWN` entries are emitted (matching `fd -t f`). Symlinks (`DT_LNK`) are skipped.
|
||||
|
||||
6. **`DT_UNKNOWN` handling** — Treated as regular files (no stat fallback). Correct for ext4/tmpfs; may miss directories on XFS/BTRFS/FUSE.
|
||||
|
||||
**Memory management:** All `GIContext` objects tracked in `pool.all_contexts` (mutex-protected append). Gitignore objects and context structs freed in bulk when `walk` completes.
|
||||
|
||||
**Parity achieved** (`~`, 5M+ files):
|
||||
|
||||
| Mode | findr | fd equivalent | diff |
|
||||
|---|---|---|---|
|
||||
| `.All` (-I) | 5,426,451 | `fd -HI -t f --exclude .git` | **0 (exact)** |
|
||||
| `.Respected` | 4,442,505 | `fd -H -t f --exclude .git` | +1,417 (0.03%) |
|
||||
| `--no-hidden` | 393,605 | `fd -t f --exclude .git` | +17 (0.004%) |
|
||||
|
||||
On the envr repo itself, all three modes are **exact match (0 diffs)**. The tiny residual diffs on `~` are likely from global gitignore (`~/.config/git/ignore`) and `.git/info/exclude` which fd reads but findr doesn't.
|
||||
|
||||
#### Step 3: DT_UNKNOWN stat fallback (TODO)
|
||||
|
||||
On XFS/BTRFS/FUSE filesystems, `dirent.type` returns `DT_UNKNOWN`. Currently findr treats these as regular files, which means directories may be missed (not descended into). Add a stat fallback in `read_dir_entries` when `d.type == .UNKNOWN` to determine the real type before proceeding. This is not needed for ext4/tmpfs (what tests and most Linux systems use).
|
||||
|
||||
### Phase 7: Performance Optimization (next)
|
||||
|
||||
**Goal:** Make findr competitive with or faster than fd across all modes. Current benchmark (`~`, hyperfine 5 runs):
|
||||
|
||||
| Command | Mean | vs fd equivalent |
|
||||
|---|---|---|
|
||||
| `findr --ignored` | 984ms | *(no fd equivalent)* |
|
||||
| `findr --no-hidden` | 542ms | 3.2x slower than `fd -t f` (170ms) |
|
||||
| `findr` (respected) | 4.134s | 2.4x slower than `fd -H -t f` (1.745s) |
|
||||
| `findr -I` (all) | 3.821s | 1.9x slower than `fd -HI -t f` (1.972s) |
|
||||
|
||||
**Bottleneck analysis:**
|
||||
|
||||
1. **Mutex contention on result collection** — Every file append goes through `sync.mutex_lock(&pool.results_mutex)` → `append` → `sync.mutex_unlock`. With 5M+ files across 16 threads, workers serialize on the mutex.
|
||||
|
||||
2. **`--ignored` regression** — Was 402ms before nested gitignore support, now 984ms. The overhead comes from loading `.gitignore` in every directory and checking the context chain per entry. Since `--ignored` mode prunes gitignored dirs, many of these `.gitignore` loads are wasted (the dir won't be descended into anyway). Optimization: skip loading `.gitignore` for directories that will be pruned.
|
||||
|
||||
3. **Per-string heap allocation** — Every path string is individually `strings.clone`'d and `delete`'d. Millions of alloc/free calls.
|
||||
|
||||
**Optimization plan:**
|
||||
|
||||
1. **Per-thread result buffers** — Each worker accumulates results in a thread-local `[dynamic]string`. Merge into shared array once at the end (single-threaded concat).
|
||||
|
||||
2. **Lazy gitignore loading for `.Ignored` mode** — Only load `.gitignore` when we need to decide whether to emit or descend. In `.Ignored` mode, we can check the parent context first and skip loading if the directory itself is already ignored.
|
||||
|
||||
3. **Arena allocator for paths** — Replace per-string `strings.clone` with a bump allocator. Free everything in one `arena_destroy` at the end.
|
||||
|
||||
4. **Larger getdents buffer** — Increase from 8KB to 64KB to reduce syscall count.
|
||||
|
||||
5. **BufWriter on stdout** — Batch `write` syscalls instead of per-line `fmt.println`.
|
||||
|
||||
**Success criteria:**
|
||||
- `.All` mode faster than `fd -HI -t f --exclude .git`
|
||||
- `.Respected` mode faster than `fd -H -t f --exclude .git`
|
||||
- `--ignored` mode faster than `fd -HI -t f --exclude .git` (restore pre-regression advantage)
|
||||
- Re-benchmark after each step using `findr/bench.sh`
|
||||
|
||||
### Phase 8: Integrate into envr
|
||||
|
||||
**Goal:** Replace ALL `fd` subprocess usage in envr with in-process findr calls. Remove `Feature.Fd` entirely.
|
||||
|
||||
#### Part A: Rewrite `scan_path` (`scan.odin`)
|
||||
|
||||
Replace the double-run-and-diff approach with a single `findr.walk` call using `.Ignored` mode:
|
||||
|
||||
```odin
|
||||
// Before: fd -HI + fd -H, then diff
|
||||
// After:
|
||||
findr.walk(search_path, &paths, WalkOptions{
|
||||
pattern = cfg.ScanConfig.Matcher,
|
||||
excludes = cfg.ScanConfig.Exclude[:],
|
||||
include_hidden = true,
|
||||
ignore_mode = .Ignored,
|
||||
}, thread_count)
|
||||
```
|
||||
|
||||
**Delete:** `build_fd_args`, `run_fd`, `next_fd_tmp_path`, `fd_counter`, `fd_seq`, `cant_scan`.
|
||||
|
||||
#### Part B: Add `find_repos` and rewrite `find_git_roots` (`config.odin`)
|
||||
|
||||
Add a `find_repos` proc to findr that walks a tree and collects directories containing `.git/`:
|
||||
|
||||
```odin
|
||||
find_repos :: proc(root: string, results: ^[dynamic]string, thread_count: int)
|
||||
```
|
||||
|
||||
- Reuses worker pool architecture
|
||||
- `process_dir` emits `dir_path` when `has_git == true`
|
||||
- Always descends into subdirs (except `.git`) to find nested repos
|
||||
- No gitignore/exclude/pattern processing
|
||||
|
||||
Replace `find_git_roots`'s `run_fd` call with `findr.find_repos`.
|
||||
|
||||
#### Part C: Remove `Feature.Fd` everywhere
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `features.odin` | Remove `Fd` from enum, remove fd binary check |
|
||||
| `cmd_scan.odin` | Remove feats/cant_scan guard + "install fd" error |
|
||||
| `cmd_check.odin` | Same removal |
|
||||
| `cmd_deps.odin` | Remove fd table row |
|
||||
| `db.odin` | Change check to `.Git not_in feats` only; update error message |
|
||||
| `scan_test.odin` | Remove `cant_scan` tests and assertions |
|
||||
|
||||
#### Part D: Verification
|
||||
|
||||
```bash
|
||||
odin build findr -o:speed -out:findr/findr
|
||||
odin test findr
|
||||
odin build . -o:speed -out:envr
|
||||
odin test .
|
||||
```
|
||||
|
||||
## Risks
|
||||
|
||||
| Risk | Mitigation |
|
||||
|---|---|
|
||||
| Gitignore edge cases (`**/foo`, `foo/**/bar`) | Comprehensive gitignore_test.odin with spec examples |
|
||||
| `DT_UNKNOWN` on XFS/BTRFS/FUSE | Phase 6 Step 3: stat fallback for unknown types |
|
||||
| Global gitignore (`~/.config/git/ignore`) and `.git/info/exclude` not read | Causes ~0.03% delta vs fd. Acceptable for envr's use case (finds `.env` files in repos). |
|
||||
| Thread safety of `regex.match` on shared `Regular_Expression` | Odin regex is read-only after compilation; `match` returns per-call `Captures` |
|
||||
71
findr/bench.sh
Executable file
71
findr/bench.sh
Executable file
@@ -0,0 +1,71 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
BENCH_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
TARGET="${1:-$HOME}"
|
||||
RESULTS_FILE="$BENCH_DIR/bench-results.md"
|
||||
FINDR="$BENCH_DIR/findr"
|
||||
|
||||
echo "=== findr benchmark suite ==="
|
||||
echo "Target: $TARGET"
|
||||
echo
|
||||
|
||||
# --- pre-flight checks ---
|
||||
if ! command -v fd &>/dev/null; then
|
||||
echo "ERROR: fd is not on PATH" >&2
|
||||
exit 1
|
||||
fi
|
||||
if ! command -v hyperfine &>/dev/null; then
|
||||
echo "ERROR: hyperfine is not on PATH" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- build findr if missing or stale ---
|
||||
NEEDS_BUILD=false
|
||||
if [[ ! -f "$BENCH_DIR/findr" ]]; then
|
||||
NEEDS_BUILD=true
|
||||
else
|
||||
# rebuild if any .odin source is newer than the binary
|
||||
if find "$BENCH_DIR" -name '*.odin' -newer "$BENCH_DIR/findr" | grep -q .; then
|
||||
NEEDS_BUILD=true
|
||||
fi
|
||||
fi
|
||||
if $NEEDS_BUILD; then
|
||||
echo "Building findr..."
|
||||
odin build "$BENCH_DIR" -o:speed -out:"$BENCH_DIR/findr"
|
||||
fi
|
||||
echo
|
||||
|
||||
# --- file counts ---
|
||||
echo "=== File counts ==="
|
||||
printf " fd -a -E .jj . : %8d\n" "$(fd -a -E .jj . "$TARGET" 2>/dev/null | wc -l)"
|
||||
printf " findr -E .jj : %8d\n" "$("$FINDR" -E .jj "$TARGET" 2>/dev/null | wc -l)"
|
||||
echo
|
||||
printf " fd -a -E .git -E .jj -H . : %8d\n" "$(fd -a -E .git -E .jj -H . "$TARGET" 2>/dev/null | wc -l)"
|
||||
printf " findr -E .git -E .jj -H : %8d\n" "$("$FINDR" -E .git -E .jj -H "$TARGET" 2>/dev/null | wc -l)"
|
||||
echo
|
||||
printf " fd -a -E .git -E .jj -HI . : %8d\n" "$(fd -a -E .git -E .jj -HI . "$TARGET" 2>/dev/null | wc -l)"
|
||||
printf " findr -E .git -E .jj -HI : %8d\n" "$("$FINDR" -E .git -E .jj -HI "$TARGET" 2>/dev/null | wc -l)"
|
||||
echo
|
||||
printf " fd -a -E .git -E .jj . : %8d\n" "$(fd -a -E .git -E .jj . "$TARGET" 2>/dev/null | wc -l)"
|
||||
printf " findr -E .git -E .jj : %8d\n" "$("$FINDR" -E .git -E .jj "$TARGET" 2>/dev/null | wc -l)"
|
||||
echo
|
||||
|
||||
# --- benchmarks ---
|
||||
echo "=== Benchmarks (hyperfine, 5 runs, 2 warmups) ==="
|
||||
echo
|
||||
hyperfine \
|
||||
--warmup 2 \
|
||||
--runs 5 \
|
||||
--export-markdown "$RESULTS_FILE" \
|
||||
"fd -a -E .jj . \"$TARGET\" > /dev/null" \
|
||||
"$FINDR -E .jj \"$TARGET\" > /dev/null" \
|
||||
"fd -a -E .git -E .jj -H . \"$TARGET\" > /dev/null" \
|
||||
"$FINDR -E .git -E .jj -H \"$TARGET\" > /dev/null" \
|
||||
"fd -a -E .git -E .jj -HI . \"$TARGET\" > /dev/null" \
|
||||
"$FINDR -E .git -E .jj -HI \"$TARGET\" > /dev/null" \
|
||||
"fd -a -E .git -E .jj . \"$TARGET\" > /dev/null" \
|
||||
"$FINDR -E .git -E .jj \"$TARGET\" > /dev/null"
|
||||
echo
|
||||
|
||||
echo "=== Results written to $RESULTS_FILE ==="
|
||||
27
findr/f.nu
Executable file
27
findr/f.nu
Executable file
@@ -0,0 +1,27 @@
|
||||
#!/usr/bin/env nu
|
||||
|
||||
def main [] {
|
||||
let all = (fd -HI -a .env . ~/ | lines | sort)
|
||||
let unignored = (fd -H -a .env ~/ | lines | sort)
|
||||
|
||||
$all | filter { |it| not ($it in $unignored) } | str join "\n"
|
||||
# sorted_list_intersect $all $unignored | str join "\n"
|
||||
}
|
||||
|
||||
def sorted_list_intersect [xs1: list, xs2: list] {
|
||||
let len1 = ($xs1 | length)
|
||||
let len2 = ($xs2 | length)
|
||||
mut i = 0
|
||||
mut j = 0
|
||||
while ($i < $len1 and $j < $len2) {
|
||||
if ($xs1 | get $i) < ($xs2 | get $j) {
|
||||
$i = $i + 1
|
||||
} else if ($xs2 | get $j) < ($xs1 | get $i) {
|
||||
$j = $j + 1
|
||||
} else {
|
||||
echo ($xs2 | get $j)
|
||||
$i = $i + 1
|
||||
$j = $j + 1
|
||||
}
|
||||
}
|
||||
}
|
||||
91
findr/findr.odin
Normal file
91
findr/findr.odin
Normal file
@@ -0,0 +1,91 @@
|
||||
package findr
|
||||
|
||||
import "core:bufio"
|
||||
import "core:os"
|
||||
import "core:strings"
|
||||
|
||||
main :: proc() {
|
||||
prof_init()
|
||||
defer prof_destroy()
|
||||
|
||||
args := os.args
|
||||
|
||||
opts: WalkOptions
|
||||
opts.include_hidden = false
|
||||
opts.ignore_mode = .Respected
|
||||
|
||||
excludes := make([dynamic]string)
|
||||
defer delete(excludes)
|
||||
|
||||
pattern := ""
|
||||
paths := make([dynamic]string)
|
||||
defer delete(paths)
|
||||
|
||||
i := 1
|
||||
for i < len(args) {
|
||||
arg := args[i]
|
||||
switch {
|
||||
case arg == "--ignored":
|
||||
opts.ignore_mode = .Ignored
|
||||
case arg == "-E":
|
||||
i += 1
|
||||
if i < len(args) {
|
||||
append(&excludes, args[i])
|
||||
}
|
||||
case strings.has_prefix(arg, "-E"):
|
||||
append(&excludes, arg[2:])
|
||||
case len(arg) > 1 && arg[0] == '-':
|
||||
for c, j in arg[1:] {
|
||||
switch c {
|
||||
case 'H':
|
||||
opts.include_hidden = true
|
||||
case 'I':
|
||||
opts.ignore_mode = .All
|
||||
case 'a':
|
||||
// no-op: accepted for fd compatibility
|
||||
}
|
||||
}
|
||||
case:
|
||||
if pattern == "" {
|
||||
pattern = arg
|
||||
} else {
|
||||
append(&paths, arg)
|
||||
}
|
||||
}
|
||||
i += 1
|
||||
}
|
||||
|
||||
if len(paths) == 0 && pattern != "" && os.exists(pattern) {
|
||||
append(&paths, pattern)
|
||||
pattern = ""
|
||||
}
|
||||
|
||||
opts.pattern = pattern
|
||||
if len(excludes) > 0 {
|
||||
opts.excludes = excludes[:]
|
||||
}
|
||||
|
||||
if len(paths) == 0 {
|
||||
append(&paths, ".")
|
||||
}
|
||||
|
||||
results := make([dynamic]string)
|
||||
defer {
|
||||
for r in results {delete(r)}
|
||||
delete(results)
|
||||
}
|
||||
|
||||
thread_count := os.get_processor_core_count()
|
||||
walk(paths[:], &results, opts, thread_count)
|
||||
|
||||
w: bufio.Writer
|
||||
bufio.writer_init(&w, os.to_stream(os.stdout), 1 << 13)
|
||||
defer bufio.writer_destroy(&w)
|
||||
|
||||
for r in results {
|
||||
bufio.writer_write_string(&w, r)
|
||||
bufio.writer_write_byte(&w, '\n')
|
||||
}
|
||||
bufio.writer_flush(&w)
|
||||
}
|
||||
|
||||
474
findr/findr_test.odin
Normal file
474
findr/findr_test.odin
Normal file
@@ -0,0 +1,474 @@
|
||||
package findr
|
||||
|
||||
import "core:os"
|
||||
import "core:sort"
|
||||
import "core:strings"
|
||||
import "core:sys/linux"
|
||||
import "core:testing"
|
||||
|
||||
// ============================================================================
|
||||
// .Ignored mode tests (original findr behavior — emit ONLY gitignored files)
|
||||
// ============================================================================
|
||||
|
||||
@(test)
|
||||
test_basic_gitignored :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.gitignore", "*.env\n")
|
||||
create_file(env, "repo/.env")
|
||||
create_file(env, "repo/secrets.env")
|
||||
create_file(env, "repo/normal.txt")
|
||||
|
||||
assert_output(t, env, nil, {include_hidden = true, ignore_mode = .Ignored}, {
|
||||
"repo/.env", "repo/secrets.env",
|
||||
})
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_non_repo_not_scanned :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_dir(env, "norepo")
|
||||
create_file(env, "norepo/.gitignore", "*.env\n")
|
||||
create_file(env, "norepo/.env")
|
||||
|
||||
assert_output_empty(t, env, nil, {include_hidden = true, ignore_mode = .Ignored})
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_negation_pattern :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.gitignore", "*.env\n!prod.env\n")
|
||||
create_file(env, "repo/.env")
|
||||
create_file(env, "repo/secrets.env")
|
||||
create_file(env, "repo/prod.env")
|
||||
|
||||
assert_output(t, env, nil, {include_hidden = true, ignore_mode = .Ignored}, {
|
||||
"repo/.env", "repo/secrets.env",
|
||||
})
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_dir_only_pattern :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.gitignore", "node_modules/\n")
|
||||
create_file(env, "repo/node_modules", "should not match (it's a file)")
|
||||
|
||||
create_dir(env, "repo/ignored_dir")
|
||||
create_file(env, "repo/.gitignore", "ignored_dir/\n")
|
||||
|
||||
assert_output(t, env, nil, {include_hidden = true, ignore_mode = .Ignored}, {
|
||||
"repo/ignored_dir/",
|
||||
})
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_multiple_repos :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo1")
|
||||
create_file(env, "repo1/.gitignore", "*.env\n")
|
||||
create_file(env, "repo1/a.env")
|
||||
|
||||
create_git_repo(env, "repo2")
|
||||
create_file(env, "repo2/.gitignore", "*.key\n")
|
||||
create_file(env, "repo2/secret.key")
|
||||
|
||||
assert_output(t, env, nil, {include_hidden = true, ignore_mode = .Ignored}, {
|
||||
"repo1/a.env", "repo2/secret.key",
|
||||
})
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_nested_repos :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "parent")
|
||||
create_file(env, "parent/.gitignore", "*.env\n")
|
||||
create_file(env, "parent/top.env")
|
||||
|
||||
create_git_repo(env, "parent/child")
|
||||
create_file(env, "parent/child/.gitignore", "*.key\n")
|
||||
create_file(env, "parent/child/api.key")
|
||||
|
||||
assert_output(t, env, nil, {include_hidden = true, ignore_mode = .Ignored}, {
|
||||
"parent/top.env", "parent/child/api.key",
|
||||
})
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_nested_gitignore_read :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.gitignore", "*.env\n")
|
||||
create_dir(env, "repo/sub")
|
||||
create_file(env, "repo/sub/.gitignore", "*.txt\n")
|
||||
create_file(env, "repo/sub/secret.txt")
|
||||
create_file(env, "repo/sub/.env")
|
||||
|
||||
// Both root and nested .gitignore are read.
|
||||
// secret.txt: ignored by sub/.gitignore (*.txt)
|
||||
// .env: ignored by root .gitignore (*.env)
|
||||
assert_output(t, env, nil, {include_hidden = true, ignore_mode = .Ignored}, {
|
||||
"repo/sub/secret.txt", "repo/sub/.env",
|
||||
})
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_nested_gitignore_negation :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.gitignore", "*.log\n")
|
||||
create_dir(env, "repo/sub")
|
||||
create_file(env, "repo/sub/.gitignore", "!important.log\n")
|
||||
create_file(env, "repo/sub/important.log")
|
||||
create_file(env, "repo/sub/debug.log")
|
||||
|
||||
// Nested negation overrides root pattern.
|
||||
// important.log: un-ignored by sub/.gitignore → NOT emitted in .Ignored mode
|
||||
// debug.log: still ignored by root → emitted
|
||||
assert_output(t, env, nil, {include_hidden = true, ignore_mode = .Ignored}, {
|
||||
"repo/sub/debug.log",
|
||||
})
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_nested_gitignore_respected_mode :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.gitignore", "*.log\n")
|
||||
create_dir(env, "repo/sub")
|
||||
create_file(env, "repo/sub/.gitignore", "!important.log\n")
|
||||
create_file(env, "repo/sub/important.log")
|
||||
create_file(env, "repo/sub/debug.log")
|
||||
|
||||
// In .Respected mode:
|
||||
// important.log: un-ignored by nested negation → emitted
|
||||
// debug.log: ignored by root → skipped
|
||||
assert_output(t, env, nil, {include_hidden = true, ignore_mode = .Respected}, {
|
||||
"repo/", "repo/.gitignore", "repo/sub/", "repo/sub/.gitignore", "repo/sub/important.log",
|
||||
})
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_multisegment_pattern :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.gitignore", "build/output.txt\n")
|
||||
create_dir(env, "repo/build")
|
||||
create_file(env, "repo/build/output.txt")
|
||||
create_file(env, "repo/build/other.txt")
|
||||
create_file(env, "repo/output.txt")
|
||||
|
||||
// Multi-segment pattern matches relative path, not just basename.
|
||||
// build/output.txt: matches → ignored
|
||||
// build/other.txt: doesn't match → not ignored
|
||||
// output.txt: doesn't match (needs build/ prefix) → not ignored
|
||||
assert_output(t, env, nil, {include_hidden = true, ignore_mode = .Ignored}, {
|
||||
"repo/build/output.txt",
|
||||
})
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_no_gitignore_file :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.env")
|
||||
|
||||
assert_output_empty(t, env, nil, {include_hidden = true, ignore_mode = .Ignored})
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_empty_gitignore :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.gitignore", "\n\n# comment\n\n")
|
||||
create_file(env, "repo/.env")
|
||||
|
||||
assert_output_empty(t, env, nil, {include_hidden = true, ignore_mode = .Ignored})
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_multiple_search_dirs :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "dir1/repo")
|
||||
create_file(env, "dir1/repo/.gitignore", "*.env\n")
|
||||
create_file(env, "dir1/repo/a.env")
|
||||
create_file(env, "dir1/repo/normal.txt")
|
||||
|
||||
create_git_repo(env, "dir2/repo")
|
||||
create_file(env, "dir2/repo/.gitignore", "*.env\n")
|
||||
create_file(env, "dir2/repo/b.env")
|
||||
|
||||
dir1 := join_path(env.temp_dir, "dir1")
|
||||
defer delete(dir1)
|
||||
dir2 := join_path(env.temp_dir, "dir2")
|
||||
defer delete(dir2)
|
||||
|
||||
results := make([dynamic]string)
|
||||
defer {
|
||||
for r in results {delete(r)}
|
||||
delete(results)
|
||||
}
|
||||
|
||||
opts := WalkOptions{include_hidden = true, ignore_mode = .Ignored}
|
||||
thread_count := os.get_processor_core_count()
|
||||
walk({dir1, dir2}, &results, opts, thread_count)
|
||||
|
||||
testing.expect_value(t, len(results), 2)
|
||||
|
||||
actual := make([dynamic]string, 0, len(results))
|
||||
for r in results {
|
||||
stripped := r
|
||||
if strings.has_prefix(stripped, env.temp_dir) {
|
||||
stripped = stripped[len(env.temp_dir):]
|
||||
if len(stripped) > 0 && stripped[0] == '/' {
|
||||
stripped = stripped[1:]
|
||||
}
|
||||
}
|
||||
append(&actual, stripped)
|
||||
}
|
||||
defer delete(actual)
|
||||
|
||||
expected := []string{"dir1/repo/a.env", "dir2/repo/b.env"}
|
||||
|
||||
sort.quick_sort(actual[:])
|
||||
sort.quick_sort(expected[:])
|
||||
|
||||
for i in 0 ..< len(expected) {
|
||||
testing.expect_value(t, actual[i], expected[i])
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// .All mode tests (fd -HI parity — ignore gitignore entirely)
|
||||
// ============================================================================
|
||||
|
||||
@(test)
|
||||
test_all_mode_emits_all_files :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.gitignore", "*.env\n")
|
||||
create_file(env, "repo/.env")
|
||||
create_file(env, "repo/secrets.env")
|
||||
create_file(env, "repo/normal.txt")
|
||||
|
||||
assert_output(t, env, nil, {include_hidden = true, ignore_mode = .All}, {
|
||||
"repo/", "repo/.env", "repo/.gitignore", "repo/secrets.env", "repo/normal.txt",
|
||||
})
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_all_mode_descends_everywhere :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.gitignore", "build/\n")
|
||||
create_dir(env, "repo/build")
|
||||
create_file(env, "repo/build/output.txt")
|
||||
|
||||
assert_output(t, env, nil, {include_hidden = true, ignore_mode = .All}, {
|
||||
"repo/", "repo/.gitignore", "repo/build/", "repo/build/output.txt",
|
||||
})
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// .Respected mode tests (fd -H parity — skip gitignored, prune ignored dirs)
|
||||
// ============================================================================
|
||||
|
||||
@(test)
|
||||
test_respected_mode_skips_gitignored :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.gitignore", "*.env\n")
|
||||
create_file(env, "repo/.env")
|
||||
create_file(env, "repo/secrets.env")
|
||||
create_file(env, "repo/normal.txt")
|
||||
|
||||
assert_output(t, env, nil, {include_hidden = true, ignore_mode = .Respected}, {
|
||||
"repo/", "repo/.gitignore", "repo/normal.txt",
|
||||
})
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_respected_mode_prunes_ignored_dirs :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.gitignore", "build/\n")
|
||||
create_dir(env, "repo/build")
|
||||
create_file(env, "repo/build/output.txt")
|
||||
create_file(env, "repo/main.txt")
|
||||
|
||||
assert_output(t, env, nil, {include_hidden = true, ignore_mode = .Respected}, {
|
||||
"repo/", "repo/.gitignore", "repo/main.txt",
|
||||
})
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Filter tests (excludes, pattern, hidden)
|
||||
// ============================================================================
|
||||
|
||||
@(test)
|
||||
test_excludes_prune_dirs :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.gitignore", "*.env\n")
|
||||
create_file(env, "repo/.env")
|
||||
create_dir(env, "repo/vendor")
|
||||
create_file(env, "repo/vendor/lib.env")
|
||||
|
||||
assert_output(t, env, nil,
|
||||
{include_hidden = true, ignore_mode = .Ignored, excludes = {"vendor"}},
|
||||
{"repo/.env"},
|
||||
)
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_pattern_filters_results :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.gitignore", "*.env\n*.key\n")
|
||||
create_file(env, "repo/.env")
|
||||
create_file(env, "repo/secrets.env")
|
||||
create_file(env, "repo/master.key")
|
||||
|
||||
assert_output(t, env, nil,
|
||||
{pattern = "\\.env$", include_hidden = true, ignore_mode = .Ignored},
|
||||
{"repo/.env", "repo/secrets.env"},
|
||||
)
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_no_hidden_skips_dotfiles :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.gitignore", "*.env\n")
|
||||
create_file(env, "repo/.env")
|
||||
create_file(env, "repo/secrets.env")
|
||||
create_file(env, "repo/.hidden.env")
|
||||
|
||||
assert_output(t, env, nil,
|
||||
{include_hidden = false, ignore_mode = .Ignored},
|
||||
{"repo/secrets.env"},
|
||||
)
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Special file type tests (SOCK, FIFO, CHR, BLK parity with fd)
|
||||
// ============================================================================
|
||||
|
||||
@(test)
|
||||
test_fifo_emitted :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.gitignore", "*.env\n")
|
||||
|
||||
fifo_path := join_path(env.temp_dir, "repo/test.fifo")
|
||||
defer delete(fifo_path)
|
||||
cpath := strings.clone_to_cstring(fifo_path)
|
||||
defer delete(cpath)
|
||||
linux.mknod(cpath, linux.S_IFIFO | linux.Mode{.IRUSR, .IWUSR}, 0)
|
||||
|
||||
assert_output(t, env, nil,
|
||||
{include_hidden = true, ignore_mode = .All},
|
||||
{"repo/", "repo/.gitignore", "repo/test.fifo"},
|
||||
)
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// in_repo propagation tests
|
||||
// ============================================================================
|
||||
|
||||
@(test)
|
||||
test_repo_without_root_gitignore :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_dir(env, "repo/sub")
|
||||
create_file(env, "repo/sub/.gitignore", "*.tmp\n")
|
||||
create_file(env, "repo/sub/file.tmp")
|
||||
create_file(env, "repo/sub/file.txt")
|
||||
|
||||
assert_output(t, env, nil,
|
||||
{include_hidden = true, ignore_mode = .Respected},
|
||||
{"repo/", "repo/sub/", "repo/sub/.gitignore", "repo/sub/file.txt"},
|
||||
)
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// .ignore file support tests (fd respects .ignore in addition to .gitignore)
|
||||
// ============================================================================
|
||||
|
||||
@(test)
|
||||
test_ignore_file_respected :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.ignore", "*.tmp\n")
|
||||
create_file(env, "repo/file.tmp")
|
||||
create_file(env, "repo/file.txt")
|
||||
|
||||
assert_output(t, env, nil,
|
||||
{include_hidden = true, ignore_mode = .Respected},
|
||||
{"repo/", "repo/.ignore", "repo/file.txt"},
|
||||
)
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_ignore_overrides_gitignore :: proc(t: ^testing.T) {
|
||||
env := create_test_env()
|
||||
defer destroy_test_env(&env)
|
||||
|
||||
create_git_repo(env, "repo")
|
||||
create_file(env, "repo/.gitignore", "*.log\n")
|
||||
create_file(env, "repo/.ignore", "important.log\n")
|
||||
create_file(env, "repo/debug.log")
|
||||
create_file(env, "repo/important.log")
|
||||
|
||||
assert_output(t, env, nil,
|
||||
{include_hidden = true, ignore_mode = .Respected},
|
||||
{"repo/", "repo/.gitignore", "repo/.ignore"},
|
||||
)
|
||||
}
|
||||
88
findr/gitignore.odin
Normal file
88
findr/gitignore.odin
Normal file
@@ -0,0 +1,88 @@
|
||||
package findr
|
||||
|
||||
import "core:strings"
|
||||
|
||||
Gitignore :: struct {
|
||||
rules: [dynamic]Rule,
|
||||
}
|
||||
|
||||
Rule :: struct {
|
||||
pattern: GlobPattern,
|
||||
negated: bool,
|
||||
dir_only: bool,
|
||||
}
|
||||
|
||||
Match :: enum {
|
||||
None,
|
||||
Ignored,
|
||||
Unignored,
|
||||
}
|
||||
|
||||
is_ignored :: proc(gi: ^Gitignore, path: string, is_dir: bool) -> bool {
|
||||
return check_match(gi, path, is_dir) == .Ignored
|
||||
}
|
||||
|
||||
check_match :: proc(gi: ^Gitignore, path: string, is_dir: bool) -> Match {
|
||||
result := Match.None
|
||||
for &rule in gi.rules {
|
||||
if rule.dir_only && !is_dir do continue
|
||||
if glob_match_compiled(&rule.pattern, path) {
|
||||
result = rule.negated ? .Unignored : .Ignored
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
parse :: proc(content: string) -> Gitignore {
|
||||
gi: Gitignore
|
||||
gi.rules = make([dynamic]Rule)
|
||||
|
||||
remaining := content
|
||||
for {
|
||||
line, ok := strings.split_lines_iterator(&remaining)
|
||||
if !ok do break
|
||||
|
||||
s := strings.trim_space(line)
|
||||
if len(s) == 0 do continue
|
||||
if s[0] == '#' do continue
|
||||
|
||||
negated := false
|
||||
if s[0] == '!' {
|
||||
negated = true
|
||||
s = s[1:]
|
||||
}
|
||||
|
||||
if len(s) > 0 && s[0] == '\\' {
|
||||
if len(s) > 1 && (s[1] == '#' || s[1] == '!') {
|
||||
s = s[1:]
|
||||
}
|
||||
}
|
||||
|
||||
dir_only := false
|
||||
if len(s) > 0 && s[len(s) - 1] == '/' {
|
||||
dir_only = true
|
||||
s = s[:len(s) - 1]
|
||||
}
|
||||
|
||||
anchored := false
|
||||
if len(s) > 0 && s[0] == '/' {
|
||||
anchored = true
|
||||
s = s[1:]
|
||||
}
|
||||
|
||||
if len(s) == 0 do continue
|
||||
|
||||
gp := glob_compile(s, anchored)
|
||||
append(&gi.rules, Rule{pattern = gp, negated = negated, dir_only = dir_only})
|
||||
}
|
||||
|
||||
return gi
|
||||
}
|
||||
|
||||
destroy :: proc(gi: ^Gitignore) {
|
||||
for &rule in gi.rules {
|
||||
glob_destroy(&rule.pattern)
|
||||
}
|
||||
delete(gi.rules)
|
||||
}
|
||||
|
||||
219
findr/gitignore_test.odin
Normal file
219
findr/gitignore_test.odin
Normal file
@@ -0,0 +1,219 @@
|
||||
package findr
|
||||
|
||||
import "core:testing"
|
||||
|
||||
@(test)
|
||||
test_glob_simple :: proc(t: ^testing.T) {
|
||||
testing.expect(t, glob_match("foo", "foo", false))
|
||||
testing.expect(t, glob_match("foo", "bar/foo", false))
|
||||
testing.expect(t, !glob_match("foo", "foobar", false))
|
||||
testing.expect(t, !glob_match("foo", "foo/bar", false))
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_glob_anchored :: proc(t: ^testing.T) {
|
||||
testing.expect(t, glob_match("foo", "foo", true))
|
||||
testing.expect(t, !glob_match("foo", "bar/foo", true))
|
||||
testing.expect(t, !glob_match("foo", "foobar", true))
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_glob_star :: proc(t: ^testing.T) {
|
||||
testing.expect(t, glob_match("*.log", "test.log", false))
|
||||
testing.expect(t, glob_match("*.log", ".log", false))
|
||||
testing.expect(t, !glob_match("*.log", "test.txt", false))
|
||||
testing.expect(t, !glob_match("*.log", "dir/test", false))
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_glob_question :: proc(t: ^testing.T) {
|
||||
testing.expect(t, glob_match("?.log", "a.log", false))
|
||||
testing.expect(t, !glob_match("?.log", "ab.log", false))
|
||||
testing.expect(t, !glob_match("?.log", ".log", false))
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_glob_char_class :: proc(t: ^testing.T) {
|
||||
testing.expect(t, glob_match("[abc].log", "a.log", false))
|
||||
testing.expect(t, glob_match("[abc].log", "b.log", false))
|
||||
testing.expect(t, !glob_match("[abc].log", "d.log", false))
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_glob_negated_class :: proc(t: ^testing.T) {
|
||||
testing.expect(t, glob_match("[!abc].log", "d.log", false))
|
||||
testing.expect(t, !glob_match("[!abc].log", "a.log", false))
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_glob_dot_literal :: proc(t: ^testing.T) {
|
||||
testing.expect(t, glob_match(".env", ".env", false))
|
||||
testing.expect(t, glob_match(".env", "dir/.env", false))
|
||||
testing.expect(t, !glob_match(".env", "env", false))
|
||||
testing.expect(t, !glob_match(".env", "x.env", false))
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_glob_globstar_prefix :: proc(t: ^testing.T) {
|
||||
testing.expect(t, glob_match("**/foo", "foo", false))
|
||||
testing.expect(t, glob_match("**/foo", "a/b/foo", false))
|
||||
testing.expect(t, !glob_match("**/foo", "foobar", false))
|
||||
testing.expect(t, !glob_match("**/foo", "a/foobar", false))
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_glob_globstar_suffix :: proc(t: ^testing.T) {
|
||||
testing.expect(t, glob_match("abc/**", "abc/x", false))
|
||||
testing.expect(t, glob_match("abc/**", "abc/x/y", false))
|
||||
testing.expect(t, !glob_match("abc/**", "abc", false))
|
||||
testing.expect(t, !glob_match("abc/**", "abcd/x", false))
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_glob_globstar_middle :: proc(t: ^testing.T) {
|
||||
testing.expect(t, glob_match("foo/**/bar", "foo/bar", false))
|
||||
testing.expect(t, glob_match("foo/**/bar", "foo/x/bar", false))
|
||||
testing.expect(t, !glob_match("foo/**/bar", "foo/barx", false))
|
||||
testing.expect(t, !glob_match("foo/**/bar", "foo/x/y/baz", false))
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_glob_backslash_escape :: proc(t: ^testing.T) {
|
||||
testing.expect(t, glob_match("\\!foo", "!foo", false))
|
||||
testing.expect(t, !glob_match("\\!foo", "foo", false))
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_glob_hash_literal :: proc(t: ^testing.T) {
|
||||
testing.expect(t, glob_match("#foo", "#foo", false))
|
||||
testing.expect(t, !glob_match("#foo", "foo", false))
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_glob_hash_pattern :: proc(t: ^testing.T) {
|
||||
testing.expect(t, glob_match("#*#", "#test#", false))
|
||||
testing.expect(t, glob_match("#*#", "##", false))
|
||||
testing.expect(t, !glob_match("#*#", "test", false))
|
||||
testing.expect(t, !glob_match("#*#", "#test", false))
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_glob_empty :: proc(t: ^testing.T) {
|
||||
testing.expect(t, glob_match("", "", false))
|
||||
testing.expect(t, !glob_match("", "foo", false))
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_is_ignored_basic :: proc(t: ^testing.T) {
|
||||
gi := parse("*.env\n")
|
||||
defer destroy(&gi)
|
||||
|
||||
testing.expect_value(t, is_ignored(&gi, ".env", false), true)
|
||||
testing.expect_value(t, is_ignored(&gi, "foo.env", false), true)
|
||||
testing.expect_value(t, is_ignored(&gi, ".env.local", false), false)
|
||||
testing.expect_value(t, is_ignored(&gi, "config.yaml", false), false)
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_is_ignored_negation :: proc(t: ^testing.T) {
|
||||
gi := parse("*.env\n!.env.production\n")
|
||||
defer destroy(&gi)
|
||||
|
||||
testing.expect_value(t, is_ignored(&gi, ".env", false), true)
|
||||
testing.expect_value(t, is_ignored(&gi, ".env.production", false), false)
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_is_ignored_dir_only :: proc(t: ^testing.T) {
|
||||
gi := parse("node_modules/\n")
|
||||
defer destroy(&gi)
|
||||
|
||||
testing.expect_value(t, is_ignored(&gi, "node_modules", true), true)
|
||||
testing.expect_value(t, is_ignored(&gi, "node_modules", false), false)
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_is_ignored_anchored :: proc(t: ^testing.T) {
|
||||
gi := parse("/secret.key\n")
|
||||
defer destroy(&gi)
|
||||
|
||||
testing.expect_value(t, is_ignored(&gi, "secret.key", false), true)
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_is_ignored_comments_skipped :: proc(t: ^testing.T) {
|
||||
gi := parse("# this is a comment\n#another\n*.tmp\n")
|
||||
defer destroy(&gi)
|
||||
|
||||
testing.expect_value(t, len(gi.rules), 1)
|
||||
testing.expect_value(t, is_ignored(&gi, "file.tmp", false), true)
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_is_ignored_blank_lines_skipped :: proc(t: ^testing.T) {
|
||||
gi := parse("\n\n \n*.log\n\n")
|
||||
defer destroy(&gi)
|
||||
|
||||
testing.expect_value(t, len(gi.rules), 1)
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_is_ignored_last_match_wins :: proc(t: ^testing.T) {
|
||||
gi := parse("*.env\n!*.env\n")
|
||||
defer destroy(&gi)
|
||||
|
||||
testing.expect_value(t, is_ignored(&gi, ".env", false), false)
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_is_ignored_no_rules :: proc(t: ^testing.T) {
|
||||
gi := parse("")
|
||||
defer destroy(&gi)
|
||||
|
||||
testing.expect_value(t, is_ignored(&gi, "anything", false), false)
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_is_ignored_env_pattern :: proc(t: ^testing.T) {
|
||||
gi := parse(".env*\n")
|
||||
defer destroy(&gi)
|
||||
|
||||
testing.expect_value(t, is_ignored(&gi, ".env", false), true)
|
||||
testing.expect_value(t, is_ignored(&gi, ".env.local", false), true)
|
||||
testing.expect_value(t, is_ignored(&gi, ".envrc", false), true)
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_is_ignored_globstar :: proc(t: ^testing.T) {
|
||||
gi := parse("**/cache\n")
|
||||
defer destroy(&gi)
|
||||
|
||||
testing.expect_value(t, is_ignored(&gi, "cache", false), true)
|
||||
testing.expect_value(t, is_ignored(&gi, "foo/cache", false), true)
|
||||
testing.expect_value(t, is_ignored(&gi, "foo/bar/cache", false), true)
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_star_negation_subpath :: proc(t: ^testing.T) {
|
||||
gi := parse("*\n!public/\n")
|
||||
defer destroy(&gi)
|
||||
|
||||
// public dir itself is un-ignored
|
||||
testing.expect_value(t, is_ignored(&gi, "public", true), false)
|
||||
// children of public/ should still be ignored by *
|
||||
testing.expect_value(t, is_ignored(&gi, "public/uuid-dir", true), true)
|
||||
testing.expect_value(t, is_ignored(&gi, "public/uuid-dir/file.txt", false), true)
|
||||
}
|
||||
|
||||
@(test)
|
||||
test_is_ignored_hash_pattern :: proc(t: ^testing.T) {
|
||||
gi := parse("\\#*\\#\n")
|
||||
defer destroy(&gi)
|
||||
|
||||
testing.expect_value(t, is_ignored(&gi, "#foo#", false), true)
|
||||
testing.expect_value(t, is_ignored(&gi, "#test#", false), true)
|
||||
testing.expect_value(t, is_ignored(&gi, "AUTHORS", false), false)
|
||||
testing.expect_value(t, is_ignored(&gi, "build.zig", false), false)
|
||||
testing.expect_value(t, is_ignored(&gi, "ChangeLog", false), false)
|
||||
}
|
||||
|
||||
210
findr/glob.odin
Normal file
210
findr/glob.odin
Normal file
@@ -0,0 +1,210 @@
|
||||
package findr
|
||||
|
||||
Range :: struct {
|
||||
lo: u8,
|
||||
hi: u8,
|
||||
}
|
||||
|
||||
Class_Data :: struct {
|
||||
negated: bool,
|
||||
ranges: [dynamic]Range,
|
||||
}
|
||||
|
||||
Token_Kind :: enum u8 { Char, Star, Globstar, Question, Class }
|
||||
|
||||
Token :: struct {
|
||||
kind: Token_Kind,
|
||||
byte: u8,
|
||||
class_idx: u16,
|
||||
}
|
||||
|
||||
GlobPattern :: struct {
|
||||
tokens: [dynamic]Token,
|
||||
classes: [dynamic]Class_Data,
|
||||
anchored: bool,
|
||||
}
|
||||
|
||||
glob_compile :: proc(pattern: string, anchored: bool) -> GlobPattern {
|
||||
gp: GlobPattern
|
||||
gp.tokens = make([dynamic]Token)
|
||||
gp.classes = make([dynamic]Class_Data)
|
||||
gp.anchored = anchored
|
||||
|
||||
i := 0
|
||||
for i < len(pattern) {
|
||||
c := pattern[i]
|
||||
|
||||
if c == '*' {
|
||||
if i + 1 < len(pattern) && pattern[i + 1] == '*' {
|
||||
prev_slash := i == 0 || pattern[i - 1] == '/'
|
||||
at_end := i + 2 >= len(pattern)
|
||||
next_slash := !at_end && pattern[i + 2] == '/'
|
||||
|
||||
if prev_slash && (next_slash || at_end) {
|
||||
append(&gp.tokens, Token{kind = .Globstar})
|
||||
if next_slash {
|
||||
i += 3
|
||||
} else {
|
||||
i += 2
|
||||
}
|
||||
} else {
|
||||
append(&gp.tokens, Token{kind = .Star})
|
||||
i += 2
|
||||
}
|
||||
} else {
|
||||
append(&gp.tokens, Token{kind = .Star})
|
||||
i += 1
|
||||
}
|
||||
} else if c == '?' {
|
||||
append(&gp.tokens, Token{kind = .Question})
|
||||
i += 1
|
||||
} else if c == '[' {
|
||||
i += 1
|
||||
negated := false
|
||||
if i < len(pattern) && pattern[i] == '!' {
|
||||
negated = true
|
||||
i += 1
|
||||
}
|
||||
|
||||
ranges := make([dynamic]Range)
|
||||
|
||||
if i < len(pattern) && pattern[i] == ']' {
|
||||
append(&ranges, Range{lo = ']', hi = ']'})
|
||||
i += 1
|
||||
}
|
||||
|
||||
for i < len(pattern) && pattern[i] != ']' {
|
||||
if i + 2 < len(pattern) && pattern[i + 1] == '-' && pattern[i + 2] != ']' {
|
||||
append(&ranges, Range{lo = pattern[i], hi = pattern[i + 2]})
|
||||
i += 3
|
||||
} else {
|
||||
append(&ranges, Range{lo = pattern[i], hi = pattern[i]})
|
||||
i += 1
|
||||
}
|
||||
}
|
||||
|
||||
if i < len(pattern) {
|
||||
i += 1
|
||||
}
|
||||
|
||||
class_idx := u16(len(gp.classes))
|
||||
append(&gp.classes, Class_Data{negated = negated, ranges = ranges})
|
||||
append(&gp.tokens, Token{kind = .Class, class_idx = class_idx})
|
||||
} else if c == '\\' {
|
||||
i += 1
|
||||
if i < len(pattern) {
|
||||
append(&gp.tokens, Token{kind = .Char, byte = pattern[i]})
|
||||
i += 1
|
||||
}
|
||||
} else {
|
||||
append(&gp.tokens, Token{kind = .Char, byte = c})
|
||||
i += 1
|
||||
}
|
||||
}
|
||||
|
||||
return gp
|
||||
}
|
||||
|
||||
match_tokens :: proc(tokens: []Token, classes: []Class_Data, ti: int, path: string, pi: int) -> bool {
|
||||
if ti >= len(tokens) {
|
||||
return pi == len(path)
|
||||
}
|
||||
|
||||
tok := tokens[ti]
|
||||
switch tok.kind {
|
||||
case .Char:
|
||||
if pi < len(path) && path[pi] == tok.byte {
|
||||
return match_tokens(tokens, classes, ti + 1, path, pi + 1)
|
||||
}
|
||||
return false
|
||||
|
||||
case .Question:
|
||||
if pi < len(path) && path[pi] != '/' {
|
||||
return match_tokens(tokens, classes, ti + 1, path, pi + 1)
|
||||
}
|
||||
return false
|
||||
|
||||
case .Star:
|
||||
max_end := pi
|
||||
for max_end < len(path) && path[max_end] != '/' {
|
||||
max_end += 1
|
||||
}
|
||||
for end := max_end; end >= pi; end -= 1 {
|
||||
if match_tokens(tokens, classes, ti + 1, path, end) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
|
||||
case .Globstar:
|
||||
if ti + 1 >= len(tokens) {
|
||||
return true
|
||||
}
|
||||
if match_tokens(tokens, classes, ti + 1, path, pi) {
|
||||
return true
|
||||
}
|
||||
for end := pi + 1; end <= len(path); end += 1 {
|
||||
if path[end - 1] == '/' {
|
||||
if match_tokens(tokens, classes, ti + 1, path, end) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
|
||||
case .Class:
|
||||
if pi >= len(path) {
|
||||
return false
|
||||
}
|
||||
cd := classes[tok.class_idx]
|
||||
ch := path[pi]
|
||||
in_range := false
|
||||
for r in cd.ranges {
|
||||
if ch >= r.lo && ch <= r.hi {
|
||||
in_range = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if in_range != cd.negated {
|
||||
return match_tokens(tokens, classes, ti + 1, path, pi + 1)
|
||||
}
|
||||
return false
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
glob_match_compiled :: proc(gp: ^GlobPattern, path: string) -> bool {
|
||||
tokens := gp.tokens[:]
|
||||
classes := gp.classes[:]
|
||||
|
||||
if gp.anchored {
|
||||
return match_tokens(tokens, classes, 0, path, 0)
|
||||
}
|
||||
|
||||
if match_tokens(tokens, classes, 0, path, 0) {
|
||||
return true
|
||||
}
|
||||
for i := 1; i < len(path); i += 1 {
|
||||
if path[i - 1] == '/' {
|
||||
if match_tokens(tokens, classes, 0, path, i) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
glob_destroy :: proc(gp: ^GlobPattern) {
|
||||
for &cd in gp.classes {
|
||||
delete(cd.ranges)
|
||||
}
|
||||
delete(gp.classes)
|
||||
delete(gp.tokens)
|
||||
}
|
||||
|
||||
glob_match :: proc(pattern: string, path: string, anchored: bool) -> bool {
|
||||
gp := glob_compile(pattern, anchored)
|
||||
result := glob_match_compiled(&gp, path)
|
||||
glob_destroy(&gp)
|
||||
return result
|
||||
}
|
||||
64
findr/prof.odin
Normal file
64
findr/prof.odin
Normal file
@@ -0,0 +1,64 @@
|
||||
package findr
|
||||
|
||||
import "base:runtime"
|
||||
import "core:prof/spall"
|
||||
import "core:sync"
|
||||
|
||||
SPALL_ENABLED :: #config(SPALL_ENABLED, ODIN_DEBUG)
|
||||
|
||||
spall_ctx: spall.Context
|
||||
|
||||
@(thread_local) spall_buffer: spall.Buffer
|
||||
@(thread_local) spall_backing: []u8
|
||||
|
||||
@(instrumentation_enter)
|
||||
spall_enter :: proc "contextless" (
|
||||
proc_address, call_site_return_address: rawptr,
|
||||
loc: runtime.Source_Code_Location,
|
||||
) {
|
||||
when SPALL_ENABLED {
|
||||
spall._buffer_begin(&spall_ctx, &spall_buffer, "", "", loc)
|
||||
}
|
||||
}
|
||||
|
||||
@(instrumentation_exit)
|
||||
spall_exit :: proc "contextless" (
|
||||
proc_address, call_site_return_address: rawptr,
|
||||
loc: runtime.Source_Code_Location,
|
||||
) {
|
||||
when SPALL_ENABLED {
|
||||
spall._buffer_end(&spall_ctx, &spall_buffer)
|
||||
}
|
||||
}
|
||||
|
||||
prof_init :: proc() {
|
||||
when SPALL_ENABLED {
|
||||
spall_ctx = spall.context_create_with_scale("findr.spall", false, 1.0)
|
||||
spall_backing = make([]u8, spall.BUFFER_DEFAULT_SIZE)
|
||||
spall_buffer = spall.buffer_create(spall_backing, u32(sync.current_thread_id()))
|
||||
spall._buffer_name_thread(&spall_ctx, &spall_buffer, "main")
|
||||
}
|
||||
}
|
||||
|
||||
prof_destroy :: proc() {
|
||||
when SPALL_ENABLED {
|
||||
spall.buffer_destroy(&spall_ctx, &spall_buffer)
|
||||
delete(spall_backing)
|
||||
spall.context_destroy(&spall_ctx)
|
||||
}
|
||||
}
|
||||
|
||||
prof_thread_init :: proc(name: string) {
|
||||
when SPALL_ENABLED {
|
||||
spall_backing = make([]u8, spall.BUFFER_DEFAULT_SIZE)
|
||||
spall_buffer = spall.buffer_create(spall_backing, u32(sync.current_thread_id()))
|
||||
spall._buffer_name_thread(&spall_ctx, &spall_buffer, name)
|
||||
}
|
||||
}
|
||||
|
||||
prof_thread_destroy :: proc() {
|
||||
when SPALL_ENABLED {
|
||||
spall.buffer_destroy(&spall_ctx, &spall_buffer)
|
||||
delete(spall_backing)
|
||||
}
|
||||
}
|
||||
13
findr/profile.sh
Executable file
13
findr/profile.sh
Executable file
@@ -0,0 +1,13 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
|
||||
echo "Building findr-prof..."
|
||||
odin build "$DIR" -debug -out:"$DIR/findr-prof"
|
||||
|
||||
echo "Running profiler..."
|
||||
"$DIR/findr-prof" -E .git -E .jj -HI ~/git.verticalaxion.com
|
||||
|
||||
echo
|
||||
echo "Spall trace: $DIR/findr.spall"
|
||||
152
findr/test_env.odin
Normal file
152
findr/test_env.odin
Normal file
@@ -0,0 +1,152 @@
|
||||
package findr
|
||||
|
||||
import "core:fmt"
|
||||
import "core:log"
|
||||
import "core:os"
|
||||
import "core:sort"
|
||||
import "core:strings"
|
||||
import "core:testing"
|
||||
|
||||
TestEnv :: struct {
|
||||
temp_dir: string,
|
||||
}
|
||||
|
||||
create_test_env :: proc() -> (env: TestEnv) {
|
||||
tmp, err := os.mkdir_temp("", "findr-test-*", context.allocator)
|
||||
if err != nil {
|
||||
log.error("Failed to create temp dir:", err)
|
||||
panic("Failed to create temp dir")
|
||||
}
|
||||
|
||||
env.temp_dir = tmp
|
||||
return
|
||||
}
|
||||
|
||||
destroy_test_env :: proc(env: ^TestEnv) {
|
||||
os.remove_all(env.temp_dir)
|
||||
delete(env.temp_dir)
|
||||
}
|
||||
|
||||
create_dir :: proc(env: TestEnv, path: string) {
|
||||
full := join_path(env.temp_dir, path)
|
||||
defer delete(full)
|
||||
os.mkdir_all(full, os.Permissions_Default_Directory)
|
||||
}
|
||||
|
||||
create_file :: proc(env: TestEnv, path: string, content: string = "") {
|
||||
full := join_path(env.temp_dir, path)
|
||||
defer delete(full)
|
||||
|
||||
dir_end := strings.last_index(full, "/")
|
||||
if dir_end >= 0 {
|
||||
dir_path := full[:dir_end]
|
||||
os.mkdir_all(dir_path, os.Permissions_Default_Directory)
|
||||
}
|
||||
|
||||
f, err := os.create(full)
|
||||
if err != nil {
|
||||
log.error("Failed to create file:", full, err)
|
||||
return
|
||||
}
|
||||
if len(content) > 0 {
|
||||
os.write_string(f, content)
|
||||
}
|
||||
os.close(f)
|
||||
}
|
||||
|
||||
create_git_repo :: proc(env: TestEnv, path: string) {
|
||||
sub := join_path(path, ".git")
|
||||
defer delete(sub)
|
||||
create_dir(env, sub)
|
||||
}
|
||||
|
||||
assert_output :: proc(
|
||||
t: ^testing.T,
|
||||
env: TestEnv,
|
||||
args: []string,
|
||||
opts: WalkOptions,
|
||||
expected: []string,
|
||||
) {
|
||||
results := collect_results(env, args, opts)
|
||||
defer {
|
||||
for r in results {delete(r)}
|
||||
delete(results)
|
||||
}
|
||||
|
||||
sorted_expected := make([dynamic]string, 0, len(expected))
|
||||
for e in expected {append(&sorted_expected, e)}
|
||||
defer delete(sorted_expected)
|
||||
|
||||
sorted_actual := make([dynamic]string, 0, len(results))
|
||||
for a in results {append(&sorted_actual, a)}
|
||||
defer delete(sorted_actual)
|
||||
|
||||
sort.quick_sort(sorted_expected[:])
|
||||
sort.quick_sort(sorted_actual[:])
|
||||
|
||||
if len(sorted_expected) != len(sorted_actual) {
|
||||
testing.fail(t)
|
||||
log.error(
|
||||
fmt.tprintf("Expected %d results, got %d", len(sorted_expected), len(sorted_actual)),
|
||||
)
|
||||
log.error("Expected:", sorted_expected[:])
|
||||
log.error("Actual: ", sorted_actual[:])
|
||||
return
|
||||
}
|
||||
|
||||
for i in 0 ..< len(sorted_expected) {
|
||||
if sorted_expected[i] != sorted_actual[i] {
|
||||
testing.fail(t)
|
||||
log.error(fmt.tprintf("Mismatch at index %d", i))
|
||||
log.error("Expected:", sorted_expected[:])
|
||||
log.error("Actual: ", sorted_actual[:])
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
assert_output_empty :: proc(
|
||||
t: ^testing.T,
|
||||
env: TestEnv,
|
||||
args: []string,
|
||||
opts: WalkOptions,
|
||||
) {
|
||||
results := collect_results(env, args, opts)
|
||||
defer {
|
||||
for r in results {delete(r)}
|
||||
delete(results)
|
||||
}
|
||||
if len(results) > 0 {
|
||||
testing.fail(t)
|
||||
log.error(fmt.tprintf("Expected no results, got %d:", len(results)))
|
||||
for r in results {
|
||||
log.error(" ", r)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
collect_results :: proc(env: TestEnv, args: []string, opts: WalkOptions) -> [dynamic]string {
|
||||
results := make([dynamic]string)
|
||||
|
||||
full_args := make([dynamic]string, 0, len(args) + 1, context.temp_allocator)
|
||||
append(&full_args, env.temp_dir)
|
||||
for a in args {append(&full_args, a)}
|
||||
|
||||
thread_count := os.get_processor_core_count()
|
||||
walk(full_args[:], &results, opts, thread_count)
|
||||
|
||||
for i in 0 ..< len(results) {
|
||||
r := results[i]
|
||||
if strings.has_prefix(r, env.temp_dir) {
|
||||
stripped := r[len(env.temp_dir):]
|
||||
if len(stripped) > 0 && stripped[0] == '/' {
|
||||
stripped = stripped[1:]
|
||||
}
|
||||
new_r, _ := strings.clone(stripped)
|
||||
delete(r)
|
||||
results[i] = new_r
|
||||
}
|
||||
}
|
||||
|
||||
return results
|
||||
}
|
||||
434
findr/walker.odin
Normal file
434
findr/walker.odin
Normal file
@@ -0,0 +1,434 @@
|
||||
package findr
|
||||
|
||||
import "core:fmt"
|
||||
import "core:os"
|
||||
import "core:strings"
|
||||
import "core:sync"
|
||||
import "core:sys/linux"
|
||||
import "core:text/regex"
|
||||
import "core:thread"
|
||||
|
||||
IgnoreMode :: enum {
|
||||
Respected, // skip gitignored, prune ignored dirs (fd -H default)
|
||||
All, // ignore .gitignore entirely, descend everywhere (fd -HI)
|
||||
Ignored, // emit ONLY gitignored files, prune ignored dirs (findr original)
|
||||
}
|
||||
|
||||
WalkOptions :: struct {
|
||||
pattern: string, // regex on basename; "" = match all
|
||||
excludes: []string, // glob patterns to skip entirely (fd -E)
|
||||
include_hidden: bool, // true = include dotfiles (fd -H)
|
||||
ignore_mode: IgnoreMode,
|
||||
}
|
||||
|
||||
RawEntry :: struct {
|
||||
name: string,
|
||||
type: linux.Dirent_Type,
|
||||
}
|
||||
|
||||
GIContext :: struct {
|
||||
gi: ^Gitignore, // nil if this dir had no .gitignore
|
||||
base_rel: string, // relative path from repo root to this dir
|
||||
parent: ^GIContext, // parent context (nil if repo root)
|
||||
}
|
||||
|
||||
WorkItem :: struct {
|
||||
path: string, // absolute directory path
|
||||
rel: string, // relative path from repo root ("" = root)
|
||||
gi_ctx: ^GIContext, // gitignore chain (nil = outside any repo)
|
||||
in_repo: bool, // true if inside a git repo
|
||||
}
|
||||
|
||||
WalkerPool :: struct {
|
||||
queue: [dynamic]WorkItem,
|
||||
queue_mutex: sync.Mutex,
|
||||
queue_sema: sync.Atomic_Sema,
|
||||
results: ^[dynamic]string,
|
||||
results_mutex: sync.Mutex,
|
||||
active: i64,
|
||||
done: sync.One_Shot_Event,
|
||||
threads: [dynamic]^thread.Thread,
|
||||
opts: WalkOptions,
|
||||
pattern_re: regex.Regular_Expression,
|
||||
has_pattern: bool,
|
||||
exclude_gi: ^Gitignore,
|
||||
all_contexts: [dynamic]^GIContext,
|
||||
contexts_lock: sync.Mutex,
|
||||
}
|
||||
|
||||
walk :: proc(roots: []string, results: ^[dynamic]string, opts: WalkOptions, thread_count: int) {
|
||||
if len(roots) == 0 do return
|
||||
|
||||
pool := new(WalkerPool)
|
||||
pool.queue = make([dynamic]WorkItem)
|
||||
pool.results = results
|
||||
pool.active = i64(len(roots))
|
||||
pool.threads = make([dynamic]^thread.Thread)
|
||||
pool.all_contexts = make([dynamic]^GIContext)
|
||||
pool.opts = opts
|
||||
pool.exclude_gi = nil
|
||||
pool.has_pattern = false
|
||||
|
||||
if len(opts.pattern) > 0 {
|
||||
re, err := regex.create(opts.pattern, {regex.Flag.No_Capture})
|
||||
if err == nil {
|
||||
pool.pattern_re = re
|
||||
pool.has_pattern = true
|
||||
}
|
||||
}
|
||||
|
||||
if len(opts.excludes) > 0 {
|
||||
sb: strings.Builder
|
||||
strings.builder_init(&sb)
|
||||
for ex in opts.excludes {
|
||||
fmt.sbprintf(&sb, "%s\n", ex)
|
||||
}
|
||||
content := strings.to_string(sb)
|
||||
pool.exclude_gi = new(Gitignore)
|
||||
pool.exclude_gi^ = parse(content)
|
||||
strings.builder_destroy(&sb)
|
||||
}
|
||||
|
||||
for root in roots {
|
||||
root_clone, _ := strings.clone(root)
|
||||
append(&pool.queue, WorkItem{path = root_clone})
|
||||
sync.atomic_sema_post(&pool.queue_sema)
|
||||
}
|
||||
|
||||
for i in 0 ..< thread_count {
|
||||
t := thread.create(walk_worker)
|
||||
t.data = rawptr(pool)
|
||||
t.init_context = context
|
||||
thread.start(t)
|
||||
append(&pool.threads, t)
|
||||
}
|
||||
|
||||
sync.one_shot_event_wait(&pool.done)
|
||||
|
||||
for _ in 0 ..< thread_count {
|
||||
sync.atomic_sema_post(&pool.queue_sema)
|
||||
}
|
||||
|
||||
for t in pool.threads {
|
||||
thread.destroy(t)
|
||||
}
|
||||
delete(pool.threads)
|
||||
for item in pool.queue {
|
||||
delete(item.path)
|
||||
if len(item.rel) > 0 {delete(item.rel)}
|
||||
}
|
||||
delete(pool.queue)
|
||||
|
||||
for ctx in pool.all_contexts {
|
||||
if ctx.gi != nil {
|
||||
destroy(ctx.gi)
|
||||
free(ctx.gi)
|
||||
}
|
||||
if len(ctx.base_rel) > 0 {
|
||||
delete(ctx.base_rel)
|
||||
}
|
||||
free(ctx)
|
||||
}
|
||||
delete(pool.all_contexts)
|
||||
|
||||
if pool.has_pattern {
|
||||
regex.destroy(pool.pattern_re)
|
||||
}
|
||||
if pool.exclude_gi != nil {
|
||||
destroy(pool.exclude_gi)
|
||||
free(pool.exclude_gi)
|
||||
}
|
||||
|
||||
free(pool)
|
||||
}
|
||||
|
||||
walk_worker :: proc(t: ^thread.Thread) {
|
||||
pool := cast(^WalkerPool)t.data
|
||||
|
||||
prof_thread_init("walker")
|
||||
defer prof_thread_destroy()
|
||||
|
||||
local_results := make([dynamic]string, 0, 256)
|
||||
defer delete(local_results)
|
||||
|
||||
for {
|
||||
sync.atomic_sema_wait(&pool.queue_sema)
|
||||
|
||||
sync.mutex_lock(&pool.queue_mutex)
|
||||
if len(pool.queue) == 0 {
|
||||
sync.mutex_unlock(&pool.queue_mutex)
|
||||
if sync.atomic_load_explicit(&pool.active, .Acquire) == 0 {
|
||||
sync.one_shot_event_signal(&pool.done)
|
||||
}
|
||||
break
|
||||
}
|
||||
last := len(pool.queue) - 1
|
||||
item := pool.queue[last]
|
||||
ordered_remove(&pool.queue, last)
|
||||
sync.mutex_unlock(&pool.queue_mutex)
|
||||
|
||||
process_dir(pool, item, &local_results)
|
||||
delete(item.path)
|
||||
if len(item.rel) > 0 {delete(item.rel)}
|
||||
|
||||
old := sync.atomic_sub_explicit(&pool.active, 1, .Release)
|
||||
if old == 1 {
|
||||
sync.one_shot_event_signal(&pool.done)
|
||||
}
|
||||
}
|
||||
|
||||
if len(local_results) > 0 {
|
||||
sync.mutex_lock(&pool.results_mutex)
|
||||
for res in local_results {
|
||||
append(pool.results, res)
|
||||
}
|
||||
sync.mutex_unlock(&pool.results_mutex)
|
||||
}
|
||||
}
|
||||
|
||||
process_dir :: proc(pool: ^WalkerPool, item: WorkItem, local_results: ^[dynamic]string) {
|
||||
dir_path := item.path
|
||||
has_git := false
|
||||
entries := read_dir_entries(dir_path, &has_git)
|
||||
defer free_entries(&entries)
|
||||
|
||||
gi_ctx := item.gi_ctx
|
||||
rel := item.rel
|
||||
|
||||
if has_git {
|
||||
gi_ctx = nil
|
||||
rel = ""
|
||||
}
|
||||
|
||||
child_in_repo := has_git || item.in_repo
|
||||
|
||||
gi := load_ignore_patterns(dir_path, child_in_repo)
|
||||
if gi != nil {
|
||||
new_ctx := new(GIContext)
|
||||
new_ctx.gi = gi
|
||||
if len(rel) > 0 {
|
||||
new_ctx.base_rel, _ = strings.clone(rel)
|
||||
}
|
||||
new_ctx.parent = gi_ctx
|
||||
|
||||
sync.mutex_lock(&pool.contexts_lock)
|
||||
append(&pool.all_contexts, new_ctx)
|
||||
sync.mutex_unlock(&pool.contexts_lock)
|
||||
|
||||
gi_ctx = new_ctx
|
||||
}
|
||||
|
||||
rel_buf: [4096]u8
|
||||
|
||||
for entry in entries {
|
||||
if entry.name == ".git" do continue
|
||||
|
||||
is_dir := entry.type == .DIR
|
||||
is_nondir := entry.type != .DIR
|
||||
|
||||
if pool.exclude_gi != nil && is_ignored(pool.exclude_gi, entry.name, is_dir) {
|
||||
continue
|
||||
}
|
||||
|
||||
if !pool.opts.include_hidden && len(entry.name) > 0 && entry.name[0] == '.' {
|
||||
continue
|
||||
}
|
||||
|
||||
entry_rel := build_rel(rel_buf[:], rel, entry.name)
|
||||
|
||||
ignored := false
|
||||
if gi_ctx != nil && pool.opts.ignore_mode != .All {
|
||||
ignored = check_chain(gi_ctx, entry_rel, is_dir)
|
||||
}
|
||||
|
||||
should_emit: bool
|
||||
if ignored {
|
||||
should_emit = pool.opts.ignore_mode == .Ignored
|
||||
} else {
|
||||
should_emit = pool.opts.ignore_mode != .Ignored
|
||||
}
|
||||
|
||||
if is_dir {
|
||||
if should_emit && matches_pattern(pool, entry.name) {
|
||||
dir_path_out := join_path_dir(dir_path, entry.name)
|
||||
append(local_results, dir_path_out)
|
||||
}
|
||||
if !ignored {
|
||||
child_rel, _ := strings.clone(entry_rel)
|
||||
child_path := join_path(dir_path, entry.name)
|
||||
push_work(
|
||||
pool,
|
||||
WorkItem {
|
||||
path = child_path,
|
||||
rel = child_rel,
|
||||
gi_ctx = gi_ctx,
|
||||
in_repo = child_in_repo,
|
||||
},
|
||||
)
|
||||
}
|
||||
} else if is_nondir {
|
||||
if should_emit && matches_pattern(pool, entry.name) {
|
||||
full_path := join_path(dir_path, entry.name)
|
||||
append(local_results, full_path)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
check_chain :: proc(ctx: ^GIContext, entry_rel: string, is_dir: bool) -> bool {
|
||||
c := ctx
|
||||
for c != nil {
|
||||
if c.gi != nil {
|
||||
rel := relative_to(entry_rel, c.base_rel)
|
||||
match := check_match(c.gi, rel, is_dir)
|
||||
if match != .None {
|
||||
return match == .Ignored
|
||||
}
|
||||
}
|
||||
c = c.parent
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
relative_to :: proc(entry_rel, base_rel: string) -> string {
|
||||
if len(base_rel) == 0 do return entry_rel
|
||||
prefix_len := len(base_rel)
|
||||
if len(entry_rel) > prefix_len &&
|
||||
entry_rel[prefix_len] == '/' &&
|
||||
strings.has_prefix(entry_rel, base_rel) {
|
||||
return entry_rel[prefix_len + 1:]
|
||||
}
|
||||
return entry_rel
|
||||
}
|
||||
|
||||
build_rel :: proc(buf: []u8, rel, name: string) -> string {
|
||||
if len(rel) == 0 do return name
|
||||
pos := copy(buf, rel)
|
||||
if pos < len(buf) {
|
||||
buf[pos] = '/'
|
||||
pos += 1
|
||||
pos += copy(buf[pos:], name)
|
||||
}
|
||||
return string(buf[:pos])
|
||||
}
|
||||
|
||||
matches_pattern :: proc(pool: ^WalkerPool, name: string) -> bool {
|
||||
if !pool.has_pattern do return true
|
||||
cap, ok := regex.match(pool.pattern_re, name)
|
||||
regex.destroy(cap)
|
||||
return ok
|
||||
}
|
||||
|
||||
push_work :: proc(pool: ^WalkerPool, item: WorkItem) {
|
||||
sync.atomic_add_explicit(&pool.active, 1, .Relaxed)
|
||||
sync.mutex_lock(&pool.queue_mutex)
|
||||
append(&pool.queue, item)
|
||||
sync.mutex_unlock(&pool.queue_mutex)
|
||||
sync.atomic_sema_post(&pool.queue_sema)
|
||||
}
|
||||
|
||||
read_dir_entries :: proc(dir_path: string, has_git: ^bool) -> [dynamic]RawEntry {
|
||||
entries := make([dynamic]RawEntry)
|
||||
|
||||
cpath := strings.clone_to_cstring(dir_path)
|
||||
if cpath == nil do return entries
|
||||
|
||||
fd, err := linux.open(cpath, {.DIRECTORY, .CLOEXEC})
|
||||
delete(cpath)
|
||||
if err != .NONE do return entries
|
||||
|
||||
buf: [8192]u8
|
||||
has_git^ = false
|
||||
|
||||
for {
|
||||
n, errno := linux.getdents(fd, buf[:])
|
||||
if n <= 0 || errno != .NONE do break
|
||||
|
||||
offs := 0
|
||||
for d in linux.dirent_iterate_buf(buf[:n], &offs) {
|
||||
name := linux.dirent_name(d)
|
||||
if name == "." || name == ".." do continue
|
||||
|
||||
if name == ".git" && d.type == .DIR {
|
||||
has_git^ = true
|
||||
}
|
||||
|
||||
cloned := strings.clone(name)
|
||||
append(&entries, RawEntry{name = cloned, type = d.type})
|
||||
}
|
||||
}
|
||||
|
||||
linux.close(fd)
|
||||
return entries
|
||||
}
|
||||
|
||||
free_entries :: proc(entries: ^[dynamic]RawEntry) {
|
||||
for &entry in entries {
|
||||
delete(entry.name)
|
||||
}
|
||||
delete(entries^)
|
||||
}
|
||||
|
||||
load_ignore_patterns :: proc(dir_path: string, in_repo: bool) -> ^Gitignore {
|
||||
has_patterns := false
|
||||
sb: strings.Builder
|
||||
strings.builder_init(&sb)
|
||||
defer strings.builder_destroy(&sb)
|
||||
|
||||
if in_repo {
|
||||
gi_path := join_path(dir_path, ".gitignore")
|
||||
data, err := os.read_entire_file_from_path(gi_path, context.allocator)
|
||||
delete(gi_path)
|
||||
if err == .NONE {
|
||||
fmt.sbprintf(&sb, "%s", string(data))
|
||||
delete(data)
|
||||
has_patterns = true
|
||||
}
|
||||
}
|
||||
|
||||
ig_path := join_path(dir_path, ".ignore")
|
||||
idata, ierr := os.read_entire_file_from_path(ig_path, context.allocator)
|
||||
delete(ig_path)
|
||||
if ierr == .NONE {
|
||||
fmt.sbprintf(&sb, "%s", string(idata))
|
||||
delete(idata)
|
||||
has_patterns = true
|
||||
}
|
||||
|
||||
if !has_patterns do return nil
|
||||
|
||||
content := strings.to_string(sb)
|
||||
gi := new(Gitignore)
|
||||
gi^ = parse(content)
|
||||
return gi
|
||||
}
|
||||
|
||||
join_path :: proc(parent, child: string) -> string {
|
||||
need_sep := len(parent) == 0 || parent[len(parent) - 1] != '/'
|
||||
total := len(parent) + len(child)
|
||||
if need_sep do total += 1
|
||||
buf := make([]u8, total, context.allocator)
|
||||
pos := copy(buf, parent)
|
||||
if need_sep {
|
||||
buf[pos] = '/'
|
||||
pos += 1
|
||||
}
|
||||
copy(buf[pos:], child)
|
||||
return string(buf)
|
||||
}
|
||||
|
||||
join_path_dir :: proc(parent, child: string) -> string {
|
||||
need_sep := len(parent) == 0 || parent[len(parent) - 1] != '/'
|
||||
total := len(parent) + len(child) + 1 // +1 for trailing '/'
|
||||
if need_sep do total += 1
|
||||
buf := make([]u8, total, context.allocator)
|
||||
pos := copy(buf, parent)
|
||||
if need_sep {
|
||||
buf[pos] = '/'
|
||||
pos += 1
|
||||
}
|
||||
pos += copy(buf[pos:], child)
|
||||
buf[pos] = '/'
|
||||
return string(buf)
|
||||
}
|
||||
|
||||
@@ -106,6 +106,10 @@
|
||||
# Build tools
|
||||
zip
|
||||
|
||||
# Helper tools
|
||||
delta
|
||||
hyperfine
|
||||
|
||||
# IDE
|
||||
unstable.helix
|
||||
typescript-language-server
|
||||
|
||||
@@ -1 +1 @@
|
||||
0.3.0
|
||||
0.3.1
|
||||
|
||||
Reference in New Issue
Block a user