2 Commits

Author SHA1 Message Date
Spencer Brower
f6cfb4a98d chore(dev): release 0.4.0 2026-06-16 23:33:11 -04:00
1fc5f8280e feat: Replaced fd with custom internals. 2026-06-16 23:32:48 -04:00
34 changed files with 1606 additions and 1740 deletions

3
.gitignore vendored
View File

@@ -7,12 +7,9 @@ list.json
man man
# build artifacts # build artifacts
*.spall
builds builds
envr envr
envr-go envr-go
findr/findr findr/findr
findr/findr-prof
findr/bench-*.md
result result
version.odin version.odin

View File

@@ -1,21 +1,11 @@
# Changelog # Changelog
## [0.4.0](https://github.com/sbrow/envr/compare/v0.3.0...v0.4.0) (2026-06-18) ## [0.4.0](https://github.com/sbrow/envr/compare/v0.3.0...v0.4.0) (2026-06-17)
### Features ### Features
* Removed runtime git dependency. ([12574e1](https://github.com/sbrow/envr/commit/12574e123bdedba3aca813143e906ec5e0b95719)) * Replaced `fd` with custom internals. ([1fc5f82](https://github.com/sbrow/envr/commit/1fc5f8280e4a67b67bc14c08eb50ec4334e09cda))
### Bug Fixes
* Fixed memory leaks in the db. ([5059572](https://github.com/sbrow/envr/commit/5059572951b3ec20b3d2027032a9c3be5cb14dba))
### Performance Improvements
* Replaced `fd` with custom internals. ([2ef733f](https://github.com/sbrow/envr/commit/2ef733fe58594b0a0b6e3ef85142b74af445ccb8))
## [0.3.0](https://github.com/sbrow/envr/compare/v0.2.1...v0.3.0) (2026-06-16) ## [0.3.0](https://github.com/sbrow/envr/compare/v0.2.1...v0.3.0) (2026-06-16)

View File

@@ -12,13 +12,14 @@ the tool [of your choosing](#backup-options).
## Features ## Features
- **Encrypted Storage**: All `.env` files are encrypted using your ssh key and - 🔐 **Encrypted Storage**: All `.env` files are encrypted using your ssh key and
[libsodium](https://github.com/jedisct1/libsodium). [libsodium](https://github.com/jedisct1/libsodium) encryption.
- **Automatic Sync**: Update the database with one command, which can easily - 🔄 **Automatic Sync**: Update the database with one command, which can easily
be run on a cron. be run on a cron.
- **Smart Scanning**: Automatically discover and import `.env` files in your - 🔍 **Smart Scanning**: Automatically discover and import `.env` files in your
home directory. home directory.
- **Rename Detection**: Automatically find and updates renamed/moved - **Interactive CLI**: User-friendly prompts for file selection and management.
- 🗂️ **Rename Detection**: Automatically finds and updates renamed/moved
repositories. repositories.
## TODOS ## TODOS
@@ -27,10 +28,14 @@ repositories.
- [x] Allow configuration of ssh key. - [x] Allow configuration of ssh key.
- [x] Allow multiple ssh keys. - [x] Allow multiple ssh keys.
## Installation ## Prerequisites
You will need an SSH key pair for encryption and decryption. You can generate one - An SSH key pair (for encryption/decryption)
with `ssh-keygen -t ed25519`. It will be saved to `~/.ssh/id_ed25519`. - The following binaries:
- [fd](https://github.com/sharkdp/fd)
- [git](https://git-scm.com)
## Installation
### With Odin ### With Odin
@@ -91,12 +96,7 @@ The configuration file is created during initialization:
], ],
"scan": { "scan": {
"matcher": "\\.env", "matcher": "\\.env",
"exclude": [ "exclude": "*.envrc",
"*\\.envrc",
"\\.local/",
"node_modules",
"vendor"
],
"include": "~" "include": "~"
} }
} }

View File

@@ -35,7 +35,13 @@ Stdout will be captured by redirecting `os.stdout` to a pipe.
## Hard to test (interactive / external deps) ## Hard to test (interactive / external deps)
### `cmd_deps` (cmd_deps.odin)
- Needs `git` and/or `fd` in PATH
- Test TTY and non-TTY paths
- Skip if dependencies not available (with `#assert` like TODO 28 suggests)
### `cmd_scan` (cmd_scan.odin) ### `cmd_scan` (cmd_scan.odin)
- Needs `fd` installed
- Test with fixture git repo containing `.env` files - Test with fixture git repo containing `.env` files
- Test `find_unbacked` integration (already partially tested in `cmd_check_test.odin`) - Test `find_unbacked` integration (already partially tested in `cmd_check_test.odin`)
- Non-TTY JSON output path - Non-TTY JSON output path
@@ -60,4 +66,5 @@ Stdout will be captured by redirecting `os.stdout` to a pipe.
- DB integration tests should use in-memory SQLite (`:memory:`) where possible. - DB integration tests should use in-memory SQLite (`:memory:`) where possible.
- Temp dir fixtures should follow the pattern in `scan_test.odin`. - Temp dir fixtures should follow the pattern in `scan_test.odin`.
- External dependency tests (`fd`, `git`) should use `#assert` to ensure the dependency is present rather than silently skipping (TODO 28).
- Tests that manipulate the `HOME` env var must use a mutex to prevent races with parallel test execution. - Tests that manipulate the `HOME` env var must use a mutex to prevent races with parallel test execution.

View File

@@ -2,13 +2,11 @@
1. Consider giving db its own allocator 1. Consider giving db its own allocator
27. Commands are still leaking. 2. **db.odin:324-327** — Map iteration (`remote_set`) is non-deterministic. Same file can produce different JSON on each backup, causing spurious DB diffs. Sort remotes before storing.
2. Generate md and man pages again. 3. **db.odin:135, 250** — String interpolation into SQL (`VACUUM INTO '%s'`, `ATTACH DATABASE '%s'`). Currently safe because input is controlled, but fragile.
3. **db.odin:324-327**Map iteration (`remote_set`) is non-deterministic. Same file can produce different JSON on each backup, causing spurious DB diffs. Sort remotes before storing. 4. **features.odin:30-41**`find_binary` uses `strings.join` instead of `filepath.join`, uses `os.stat` instead of checking executability, hardcodes `:` as PATH separator (wrong on Windows).
4. Make sure official path separators are used when appropriate, rather than '/'.
5. **cmd_restore.odin:20-30 & cmd_remove.odin:19-29** — Identical path-resolution block copy-pasted. `is_abs` guard is redundant since `filepath.abs` is a no-op on absolute paths. Extract a helper. 5. **cmd_restore.odin:20-30 & cmd_remove.odin:19-29** — Identical path-resolution block copy-pasted. `is_abs` guard is redundant since `filepath.abs` is a no-op on absolute paths. Extract a helper.
@@ -20,7 +18,7 @@
11. **db.odin:352-353**`hex.encode` error ignored. `string(hex_bytes)` aliases the byte slice. 11. **db.odin:352-353**`hex.encode` error ignored. `string(hex_bytes)` aliases the byte slice.
12. **cmd_sync.odin:80, cmd_list.odin:33**`make([]string, 2)` for table rows never freed. Leaks per row. Defer to memory pass. 12. **cmd_sync.odin:80, cmd_list.odin:33, cmd_deps.odin:9**`make([]string, 2)` for table rows never freed. Leaks per row. Defer to memory pass.
13. **cmd_list.odin** — Non-TTY branch builds `ListEntry` structs and marshals JSON separately. Now that `render_json_rows` (issue 1) accepts an `io.Writer` and uses `json.marshal`, unify both branches to use it. Note: will change JSON keys from `"directory"/"path"` to `"Directory"/"Path"`. 13. **cmd_list.odin** — Non-TTY branch builds `ListEntry` structs and marshals JSON separately. Now that `render_json_rows` (issue 1) accepts an `io.Writer` and uses `json.marshal`, unify both branches to use it. Note: will change JSON keys from `"directory"/"path"` to `"Directory"/"Path"`.
@@ -28,6 +26,8 @@
15. Add a text filter to the multi_select. 15. Add a text filter to the multi_select.
16. Create backup / fallback fd.
17. Add tests for untested commands. 17. Add tests for untested commands.
18. 2 scan tests silently skip when fd isn't installed, tests pass without actually testing anything. These should use #assert to be sure that fd is in path. 18. 2 scan tests silently skip when fd isn't installed, tests pass without actually testing anything. These should use #assert to be sure that fd is in path.
@@ -40,12 +40,6 @@
23. procedures should be ordered by use, main at the top, then in the order they are called from main. 23. procedures should be ordered by use, main at the top, then in the order they are called from main.
24. Shell completion
25. Bring back windows support / cross-compilation.
26. Test all cmds / terminal branches.
## Double-check AI output ## Double-check AI output
- [ ] cli.odin - [ ] cli.odin
@@ -53,6 +47,7 @@
- [x] cmd_backup.odin - [x] cmd_backup.odin
- [x] cmd_check.odin - [x] cmd_check.odin
- [ ] cmd_check_test.odin - [ ] cmd_check_test.odin
- [x] cmd_deps.odin
- [ ] cmd_edit_config.odin - [ ] cmd_edit_config.odin
- [x] cmd_init.odin - [x] cmd_init.odin
- [x] cmd_list.odin - [x] cmd_list.odin
@@ -71,6 +66,8 @@
- [ ] db.odin - [ ] db.odin
- [ ] db_integration_test.odin - [ ] db_integration_test.odin
- [ ] db_test.odin - [ ] db_test.odin
- [x] features.odin
- [x] features_test.odin
- [x] main.odin - [x] main.odin
- [x] prompt.odin - [x] prompt.odin
- [ ] scan.odin - [ ] scan.odin
@@ -81,11 +78,3 @@
- [ ] ssh_test.odin - [ ] ssh_test.odin
- [ ] table.odin - [ ] table.odin
- [ ] table_test.odin - [ ] table_test.odin
- [ ] findr/findr_test.odin
- [ ] findr/gitignore.odin
- [ ] findr/gitignore_test.odin
- [ ] findr/glob.odin
- [ ] findr/glob_test.odin
- [ ] findr/repos.odin
- [ ] findr/test_env.odin
- [ ] findr/walker.odin

92
WINDOWS.md Normal file
View File

@@ -0,0 +1,92 @@
# Windows Compatibility Guide
This document outlines Windows compatibility issues and solutions for the envr project.
## Critical Issues
### 1. Path Handling Bug (MUST FIX)
**File:** `app/env_file.go:209`
**Issue:** Uses `path.Join` instead of `filepath.Join`, which won't work correctly on Windows due to different path separators.
**Current code:**
```go
f.Path = path.Join(newDir, path.Base(f.Path))
```
**Fixed code:**
```go
f.Path = filepath.Join(newDir, filepath.Base(f.Path))
```
## External Dependencies
The application relies on external tools that need to be installed separately on Windows:
### Required Tools
1. **fd** - Fast file finder
- Install via: `winget install sharkdp.fd` or `choco install fd`
- Alternative: `scoop install fd`
2. **git** - Version control system
- Install via: `winget install Git.Git` or download from git-scm.com
- Usually already available on most development machines
## Minor Compatibility Notes
### File Permissions
- Unix file permissions (`0755`, `0644`) are used throughout the codebase
- These are safely ignored on Windows - no changes needed
### Editor Configuration
**File:** `cmd/edit_config.go:20-24`
**Issue:** Relies on `$EDITOR` environment variable which is less common on Windows.
**Current behavior:** Fails if `$EDITOR` is not set
**Recommended improvement:** Add fallback detection for Windows editors:
```go
editor := os.Getenv("EDITOR")
if editor == "" {
if runtime.GOOS == "windows" {
editor = "notepad.exe" // or "code.exe" for VS Code
} else {
fmt.Println("Error: $EDITOR environment variable is not set")
return
}
}
```
## Installation Instructions for Windows
1. Install required dependencies:
```powershell
winget install sharkdp.fd
winget install Git.Git
```
2. Fix the path handling bug in `app/env_file.go:209`
3. Build and run as normal:
```powershell
go build
.\envr.exe init
```
## Testing on Windows
After applying the critical path fix, the core functionality should work correctly on Windows. The application has been designed with cross-platform compatibility in mind, using:
- `filepath` package for path operations (mostly)
- `os.UserHomeDir()` for home directory detection
- Standard Go file operations
## Summary
- **1 critical bug** must be fixed for Windows compatibility
- **2 external tools** need to be installed
- **1 minor enhancement** recommended for better Windows UX
- Overall architecture is Windows-compatible

View File

@@ -43,6 +43,13 @@ key somewhere, otherwise your data could be lost forever.`,
{"list", "envr list", "View your tracked files", "", {}}, {"list", "envr list", "View your tracked files", "", {}},
{"remove", "envr remove <path>", "Remove a .env file from your database", "", {}}, {"remove", "envr remove <path>", "Remove a .env file from your database", "", {}},
{"check", "envr check [path]", "Check if files are backed up", "", {}}, {"check", "envr check [path]", "Check if files are backed up", "", {}},
{
"deps",
"envr deps",
"Check for missing binaries",
"envr relies on external binaries for certain functionality.\n\nThe check command reports on which binaries are available and which are not.",
{},
},
{"version", "envr version", "Show envr's version", "", {}}, {"version", "envr version", "Show envr's version", "", {}},
{"edit-config", "envr edit-config", "Edit your config with your default editor", "", {}}, {"edit-config", "envr edit-config", "Edit your config with your default editor", "", {}},
{ {
@@ -54,12 +61,20 @@ key somewhere, otherwise your data could be lost forever.`,
}, },
} }
delete_command :: proc(cmd: ^Command) {
delete(cmd.args)
delete(cmd.flags)
delete(cmd.bool_set)
bufio.writer_destroy(cmd.out_buf)
free(cmd.out_buf)
}
// Caller is responsible for calling delete_command(cmd). // Caller is responsible for calling delete_command(cmd).
// FIXME: Works in kinda a wonky and awkward way. // FIXME: Works in kinda a wonky and awkward way.
parse_args :: proc(args: []string, out: io.Stream, err: io.Stream) -> (cmd: Command, ok: bool) { parse_args :: proc(args: []string, out: io.Stream, err: io.Stream) -> (cmd: Command, ok: bool) {
{ {
cmd.out_buf = new(bufio.Writer) cmd.out_buf = new(bufio.Writer)
bufio.writer_init(cmd.out_buf, out, allocator = context.allocator) bufio.writer_init(cmd.out_buf, out)
cmd.out = bufio.writer_to_writer(cmd.out_buf) cmd.out = bufio.writer_to_writer(cmd.out_buf)
cmd.err = err cmd.err = err
} }
@@ -122,12 +137,27 @@ parse_args :: proc(args: []string, out: io.Stream, err: io.Stream) -> (cmd: Comm
return cmd, true return cmd, true
} }
print_command_help :: proc(cmd: ^Command) { has_flag :: proc(cmd: ^Command, name: string) -> bool {
ok := write_command_help(cmd.name, cmd.out) _, ok := cmd.flags[name]
if !ok { if ok {
fmt.wprintf(cmd.err, "Unknown command: %s\n", cmd.name) return true
write_usage(cmd.out)
} }
_, ok2 := cmd.bool_set[name]
return ok2
}
find_command :: proc(name: string) -> (CommandInfo, bool) {
for c in COMMANDS {
if c.name == name {
return c, true
}
for a in c.aliases {
if a == name {
return c, true
}
}
}
return CommandInfo{}, false
} }
write_command_help :: proc(name: string, w: io.Writer) -> bool { write_command_help :: proc(name: string, w: io.Writer) -> bool {
@@ -160,18 +190,12 @@ write_command_help :: proc(name: string, w: io.Writer) -> bool {
return true return true
} }
find_command :: proc(name: string) -> (CommandInfo, bool) { print_command_help :: proc(cmd: ^Command) {
for c in COMMANDS { ok := write_command_help(cmd.name, cmd.out)
if c.name == name { if !ok {
return c, true fmt.wprintf(cmd.err, "Unknown command: %s\n", cmd.name)
} write_usage(cmd.out)
for a in c.aliases {
if a == name {
return c, true
}
}
} }
return CommandInfo{}, false
} }
// TODO: command args should be shown in usage. // TODO: command args should be shown in usage.
@@ -246,21 +270,3 @@ Use "envr [command] --help" for more information about a command.
) )
} }
has_flag :: proc(cmd: ^Command, name: string) -> bool {
_, ok := cmd.flags[name]
if ok {
return true
}
_, ok2 := cmd.bool_set[name]
return ok2
}
delete_command :: proc(cmd: ^Command) {
bufio.writer_flush(cmd.out_buf)
delete(cmd.args)
delete(cmd.flags)
delete(cmd.bool_set)
bufio.writer_destroy(cmd.out_buf)
free(cmd.out_buf)
}

View File

@@ -5,6 +5,8 @@ import "core:os"
import "core:path/filepath" import "core:path/filepath"
cmd_check :: proc(cmd: ^Command) { cmd_check :: proc(cmd: ^Command) {
feats := check_features()
check_path: string check_path: string
if len(cmd.args) > 0 { if len(cmd.args) > 0 {
check_path = cmd.args[0] check_path = cmd.args[0]
@@ -40,6 +42,15 @@ cmd_check :: proc(cmd: ^Command) {
files_in_path: [dynamic]string files_in_path: [dynamic]string
if is_dir { if is_dir {
if cant_scan(feats) {
fmt.wprintln(
cmd.err,
"Error: please install fd to use the check command (https://github.com/sharkdp/fd)",
flush = false,
)
return
}
scanned, scan_ok := scan_path(abs_path, db.cfg) scanned, scan_ok := scan_path(abs_path, db.cfg)
if !scan_ok { if !scan_ok {
fmt.wprintln(cmd.err, "Error scanning directory for .env files", flush = false) fmt.wprintln(cmd.err, "Error scanning directory for .env files", flush = false)
@@ -54,8 +65,6 @@ cmd_check :: proc(cmd: ^Command) {
if !list_ok { if !list_ok {
return return
} }
defer delete(db_files)
defer for &file in db_files {delete_envfile(&file)}
not_backed := find_unbacked(files_in_path[:], db_files[:]) not_backed := find_unbacked(files_in_path[:], db_files[:])
@@ -63,23 +72,13 @@ cmd_check :: proc(cmd: ^Command) {
if len(files_in_path) == 0 { if len(files_in_path) == 0 {
fmt.wprintln(cmd.out, "No .env files found in the specified directory.", flush = false) fmt.wprintln(cmd.out, "No .env files found in the specified directory.", flush = false)
} else { } else {
fmt.wprintln( fmt.wprintln(cmd.out, "✓ All .env files in the directory are backed up.", flush = false)
cmd.out,
"✓ All .env files in the directory are backed up.",
flush = false,
)
} }
} else { } else {
fmt.wprintf( fmt.wprintf(cmd.out, "Found %d .env file(s) that are not backed up:\n", len(not_backed), flush = false)
cmd.out,
"Found %d .env file(s) that are not backed up:\n",
len(not_backed),
flush = false,
)
for file in not_backed { for file in not_backed {
fmt.wprintf(cmd.out, " %s\n", file, flush = false) fmt.wprintf(cmd.out, " %s\n", file, flush = false)
} }
fmt.wprintln(cmd.out, "\nRun 'envr sync' to back up these files.", flush = false) fmt.wprintln(cmd.out, "\nRun 'envr sync' to back up these files.", flush = false)
} }
} }

33
cmd_deps.odin Normal file
View File

@@ -0,0 +1,33 @@
package main
import "core:fmt"
import "core:os"
import "core:terminal"
// TODO: Improve table rendering
cmd_deps :: proc(cmd: ^Command) {
feats := check_features()
headers := []string{"Feature", "Status"}
rows: [dynamic][]string
if .Git in feats {
append(&rows, []string{"Git", "\u2713 Available"})
} else {
append(&rows, []string{"Git", "\u2717 Missing"})
}
if .Fd in feats {
append(&rows, []string{"fd", "\u2713 Available"})
} else {
append(&rows, []string{"fd", "\u2717 Missing"})
}
if terminal.is_terminal(os.stdout) {
render_table(cmd.out, headers, rows[:])
} else {
render_json_rows(cmd.out, headers, rows[:])
fmt.wprint(cmd.out, "\n", flush = false)
}
}

View File

@@ -26,7 +26,6 @@ cmd_list :: proc(cmd: ^Command) {
return return
} }
defer delete(rows) defer delete(rows)
defer for &row in rows {delete_envfile(&row)}
if terminal.is_terminal(os.stdout) { if terminal.is_terminal(os.stdout) {
headers := []string{"Directory", "Path"} headers := []string{"Directory", "Path"}
@@ -35,7 +34,7 @@ cmd_list :: proc(cmd: ^Command) {
for row in rows { for row in rows {
dir_str := strings.concatenate({row.Dir, "/"}, context.temp_allocator) dir_str := strings.concatenate({row.Dir, "/"}, context.temp_allocator)
filename := filepath.base(row.Path) filename := filepath.base(row.Path)
row_slice := make([]string, 2, context.temp_allocator) row_slice := make([]string, 2)
row_slice[0] = dir_str row_slice[0] = dir_str
row_slice[1] = filename row_slice[1] = filename
append(&table_rows, row_slice) append(&table_rows, row_slice)

View File

@@ -6,6 +6,16 @@ import "core:os"
import "core:terminal" import "core:terminal"
cmd_scan :: proc(cmd: ^Command) { cmd_scan :: proc(cmd: ^Command) {
feats := check_features()
if cant_scan(feats) {
fmt.wprintln(
cmd.err,
"Error: please install fd to use the scan command (https://github.com/sharkdp/fd)",
flush = false,
)
return
}
db, db_ok := db_open(cmd.config_path) db, db_ok := db_open(cmd.config_path)
if !db_ok { if !db_ok {
return return

View File

@@ -6,8 +6,6 @@ import "core:os"
import "core:path/filepath" import "core:path/filepath"
import "core:strings" import "core:strings"
import "findr"
SshKeyPair :: struct { SshKeyPair :: struct {
Private: string `json:"private"`, Private: string `json:"private"`,
Public: string `json:"public"`, Public: string `json:"public"`,
@@ -25,6 +23,14 @@ Config :: struct {
config_path: string `json:"-"`, config_path: string `json:"-"`,
} }
default_config_path :: proc(home: string, allocator := context.allocator) -> string {
path, err := filepath.join([]string{home, ".envr", "config.json"}, allocator)
if err != nil {
panic("Ran out of memory when building config path")
}
return path
}
load_config :: proc(config_path: string) -> (Config, bool) { load_config :: proc(config_path: string) -> (Config, bool) {
data, read_err := os.read_entire_file_from_path(config_path, context.allocator) data, read_err := os.read_entire_file_from_path(config_path, context.allocator)
if read_err != nil { if read_err != nil {
@@ -45,14 +51,6 @@ load_config :: proc(config_path: string) -> (Config, bool) {
return cfg, true return cfg, true
} }
default_config_path :: proc(home: string, allocator := context.allocator) -> string {
path, err := filepath.join([]string{home, ".envr", "config.json"}, allocator)
if err != nil {
panic("Ran out of memory when building config path")
}
return path
}
delete_config :: proc(cfg: ^Config) { delete_config :: proc(cfg: ^Config) {
for key in cfg.Keys { for key in cfg.Keys {
delete(key.Private) delete(key.Private)
@@ -73,73 +71,13 @@ delete_config :: proc(cfg: ^Config) {
delete(cfg.ScanConfig.Include) delete(cfg.ScanConfig.Include)
} }
save_config :: proc(cfg: Config, force: bool = false) -> bool { envr_dir :: proc(config_path: string) -> string {
config_dir := envr_dir(cfg.config_path) return filepath.dir(config_path)
if !os.exists(config_dir) {
mkdir_err := os.make_directory(config_dir)
if mkdir_err != nil {
fmt.printf("Error creating %s directory: %v\n", config_dir, mkdir_err)
return false
}
}
if os.exists(cfg.config_path) && !force {
info, stat_err := os.stat(cfg.config_path, context.allocator)
if stat_err == nil {
defer os.file_info_delete(info, context.allocator)
if info.size > 0 {
fmt.println("Config file already exists. Run again with --force to reinitialize.")
return false
}
}
}
data, marshal_err := json.marshal(cfg, {pretty = true, use_spaces = true, spaces = 2})
if marshal_err != nil {
fmt.printf("Error marshaling config: %v\n", marshal_err)
return false
}
defer delete(data)
write_err := os.write_entire_file(cfg.config_path, data)
if write_err != nil {
fmt.printf("Error writing config: %v\n", write_err)
return false
}
return true
} }
// Caller is responsible for calling delete_config() data_path :: proc(config_path: string) -> string {
new_config :: proc( path, _ := filepath.join([]string{envr_dir(config_path), "data.envr"})
private_key_paths: []string, return path
cfg_path: string = "~/.envr/config.json",
) -> Config {
keys := make([dynamic]SshKeyPair, 0, len(private_key_paths))
for priv in private_key_paths {
// TODO: Is this bad?
priv_key := strings.clone(priv)
pub, _ := strings.concatenate([]string{priv_key, ".pub"})
append(&keys, SshKeyPair{Private = priv_key, Public = pub})
}
exclude := make([dynamic]string, 0, 4)
append(&exclude, strings.clone("*\\.envrc"))
append(&exclude, strings.clone("\\.local/"))
append(&exclude, strings.clone("node_modules"))
append(&exclude, strings.clone("vendor"))
include := make([dynamic]string, 0, 1)
append(&include, strings.clone("~"))
scan_cfg := ScanConfig {
Matcher = strings.clone("\\.env"),
Exclude = exclude,
Include = include,
}
return Config{Keys = keys, ScanConfig = scan_cfg, config_path = cfg_path}
} }
find_ssh_private_keys :: proc() -> (keys: [dynamic]string, ok: bool) { find_ssh_private_keys :: proc() -> (keys: [dynamic]string, ok: bool) {
@@ -188,11 +126,73 @@ find_ssh_private_keys :: proc() -> (keys: [dynamic]string, ok: bool) {
return return
} }
find_git_roots :: proc(cfg: Config) -> (roots: [dynamic]string, ok: bool) { // Caller is responsible for calling delete_config()
paths := search_paths(cfg) new_config :: proc(
findr.find_repos(paths[:], &roots, os.get_processor_core_count()) private_key_paths: []string,
ok = true cfg_path: string = "~/.envr/config.json",
return ) -> Config {
keys := make([dynamic]SshKeyPair, 0, len(private_key_paths))
for priv in private_key_paths {
// TODO: Is this bad?
priv_key := strings.clone(priv)
pub, _ := strings.concatenate([]string{priv_key, ".pub"})
append(&keys, SshKeyPair{Private = priv_key, Public = pub})
}
exclude := make([dynamic]string, 0, 4)
append(&exclude, strings.clone("*\\.envrc"))
append(&exclude, strings.clone("\\.local/"))
append(&exclude, strings.clone("node_modules"))
append(&exclude, strings.clone("vendor"))
include := make([dynamic]string, 0, 1)
append(&include, strings.clone("~"))
scan_cfg := ScanConfig {
Matcher = strings.clone("\\.env"),
Exclude = exclude,
Include = include,
}
return Config{Keys = keys, ScanConfig = scan_cfg, config_path = cfg_path}
}
save_config :: proc(cfg: Config, force: bool = false) -> bool {
config_dir := envr_dir(cfg.config_path)
if !os.exists(config_dir) {
mkdir_err := os.make_directory(config_dir)
if mkdir_err != nil {
fmt.printf("Error creating %s directory: %v\n", config_dir, mkdir_err)
return false
}
}
if os.exists(cfg.config_path) && !force {
info, stat_err := os.stat(cfg.config_path, context.allocator)
if stat_err == nil {
defer os.file_info_delete(info, context.allocator)
if info.size > 0 {
fmt.println("Config file already exists. Run again with --force to reinitialize.")
return false
}
}
}
data, marshal_err := json.marshal(cfg, {pretty = true, use_spaces = true, spaces = 2})
if marshal_err != nil {
fmt.printf("Error marshaling config: %v\n", marshal_err)
return false
}
defer delete(data)
write_err := os.write_entire_file(cfg.config_path, data)
if write_err != nil {
fmt.printf("Error writing config: %v\n", write_err)
return false
}
return true
} }
search_paths :: proc(cfg: Config) -> (paths: [dynamic]string) { search_paths :: proc(cfg: Config) -> (paths: [dynamic]string) {
@@ -216,13 +216,25 @@ search_paths :: proc(cfg: Config) -> (paths: [dynamic]string) {
return return
} }
envr_dir :: proc(config_path: string) -> string { find_git_roots :: proc(cfg: Config) -> (roots: [dynamic]string, ok: bool) {
return filepath.dir(config_path) paths := search_paths(cfg)
}
for sp in paths {
// User is responsible for freeing the path args := []string{"fd", "-H", "-t", "d", "^\\.git$", sp}
data_path :: proc(config_path: string, allocator := context.allocator) -> string { lines, fd_ok := run_fd(args)
path, _ := filepath.join([]string{envr_dir(config_path), "data.envr"}, allocator) if !fd_ok {
return path return
}
for line in lines {
cleaned, _ := filepath.clean(line)
parent := filepath.dir(cleaned)
cloned, _ := strings.clone(parent)
append(&roots, cloned)
}
}
ok = true
return
} }

View File

@@ -2,7 +2,6 @@ package main
import "core:fmt" import "core:fmt"
import "core:mem" import "core:mem"
import "core:os"
MAGIC :: "ENVR" MAGIC :: "ENVR"
MAGIC_BYTES := [4]u8{u8('E'), u8('N'), u8('V'), u8('R')} MAGIC_BYTES := [4]u8{u8('E'), u8('N'), u8('V'), u8('R')}
@@ -21,19 +20,80 @@ RecipientEntry :: struct {
EncryptedKey: [CRYPTO_SECRETBOX_KEY_BYTES + CRYPTO_BOX_MAC_BYTES]u8, EncryptedKey: [CRYPTO_SECRETBOX_KEY_BYTES + CRYPTO_BOX_MAC_BYTES]u8,
} }
sodium_initialized: bool
ensure_sodium :: proc() -> bool {
if sodium_initialized {
return true
}
rc := sodium_init()
if rc < 0 {
fmt.println("Error: libsodium initialization failed")
return false
}
sodium_initialized = true
return true
}
X25519Keypair :: struct { X25519Keypair :: struct {
Public: [CRYPTO_BOX_PUBLICKEY_BYTES]u8, Public: [CRYPTO_BOX_PUBLICKEY_BYTES]u8,
Private: [CRYPTO_BOX_SECRETKEY_BYTES]u8, Private: [CRYPTO_BOX_SECRETKEY_BYTES]u8,
} }
@(init) ssh_to_x25519 :: proc(keys: []SshKeyPair) -> (pairs: []X25519Keypair, ok: bool) {
init_sodium :: proc "contextless" () { if len(keys) == 0 {
if sodium_init() < 0 { return
os.exit(1)
} }
pairs = make([]X25519Keypair, len(keys))
for i in 0 ..< len(keys) {
ssh_kp, parse_ok := parse_ssh_private_key(keys[i].Private)
if !parse_ok {
fmt.printf("Error: failed to parse SSH private key: %s\n", keys[i].Private)
delete(pairs)
return
}
ssh_pub, pub_ok := parse_ssh_public_key(keys[i].Public)
if !pub_ok {
fmt.printf("Error: failed to parse SSH public key: %s\n", keys[i].Public)
delete(pairs)
return
}
pk_rc := crypto_sign_ed25519_pk_to_curve25519(&pairs[i].Public[0], &ssh_pub[0])
if pk_rc != 0 {
fmt.println("Error: failed to convert ed25519 public key to curve25519")
delete(pairs)
return
}
ed25519_sk: [64]u8
for j in 0 ..< 32 {
ed25519_sk[j] = ssh_kp.Private[j]
}
for j in 0 ..< 32 {
ed25519_sk[32 + j] = ssh_kp.Public[j]
}
sk_rc := crypto_sign_ed25519_sk_to_curve25519(&pairs[i].Private[0], &ed25519_sk[0])
if sk_rc != 0 {
fmt.println("Error: failed to convert ed25519 private key to curve25519")
delete(pairs)
return
}
}
ok = true
return
} }
encrypt :: proc(plaintext: []u8, keys: []SshKeyPair) -> (ciphertext: []u8, ok: bool) { encrypt :: proc(plaintext: []u8, keys: []SshKeyPair) -> (ciphertext: []u8, ok: bool) {
if !ensure_sodium() {
return
}
x25519_pairs, pairs_ok := ssh_to_x25519(keys) x25519_pairs, pairs_ok := ssh_to_x25519(keys)
if !pairs_ok { if !pairs_ok {
return return
@@ -133,6 +193,10 @@ encrypt :: proc(plaintext: []u8, keys: []SshKeyPair) -> (ciphertext: []u8, ok: b
} }
decrypt :: proc(ciphertext: []u8, keys: []SshKeyPair) -> (plaintext: []u8, ok: bool) { decrypt :: proc(ciphertext: []u8, keys: []SshKeyPair) -> (plaintext: []u8, ok: bool) {
if !ensure_sodium() {
return
}
if len(ciphertext) < HEADER_SIZE { if len(ciphertext) < HEADER_SIZE {
fmt.println("Error: ciphertext too short (header)") fmt.println("Error: ciphertext too short (header)")
return return
@@ -272,52 +336,3 @@ decrypt :: proc(ciphertext: []u8, keys: []SshKeyPair) -> (plaintext: []u8, ok: b
return return
} }
ssh_to_x25519 :: proc(keys: []SshKeyPair) -> (pairs: []X25519Keypair, ok: bool) {
if len(keys) == 0 {
return
}
pairs = make([]X25519Keypair, len(keys))
for i in 0 ..< len(keys) {
ssh_kp, parse_ok := parse_ssh_private_key(keys[i].Private)
if !parse_ok {
fmt.printf("Error: failed to parse SSH private key: %s\n", keys[i].Private)
delete(pairs)
return
}
ssh_pub, pub_ok := parse_ssh_public_key(keys[i].Public)
if !pub_ok {
fmt.printf("Error: failed to parse SSH public key: %s\n", keys[i].Public)
delete(pairs)
return
}
pk_rc := crypto_sign_ed25519_pk_to_curve25519(&pairs[i].Public[0], &ssh_pub[0])
if pk_rc != 0 {
fmt.println("Error: failed to convert ed25519 public key to curve25519")
delete(pairs)
return
}
ed25519_sk: [64]u8
for j in 0 ..< 32 {
ed25519_sk[j] = ssh_kp.Private[j]
}
for j in 0 ..< 32 {
ed25519_sk[32 + j] = ssh_kp.Public[j]
}
sk_rc := crypto_sign_ed25519_sk_to_curve25519(&pairs[i].Private[0], &ed25519_sk[0])
if sk_rc != 0 {
fmt.println("Error: failed to convert ed25519 private key to curve25519")
delete(pairs)
return
}
}
ok = true
return
}

427
db.odin
View File

@@ -2,12 +2,12 @@ package main
import "core:crypto/hash" import "core:crypto/hash"
import "core:encoding/hex" import "core:encoding/hex"
import "core:encoding/ini"
import "core:encoding/json" import "core:encoding/json"
import "core:fmt" import "core:fmt"
import "core:os" import "core:os"
import "core:path/filepath" import "core:path/filepath"
import "core:strings" import "core:strings"
import "core:time"
import "sqlite" import "sqlite"
@@ -51,85 +51,51 @@ delete_envfile :: proc(f: ^EnvFile) {
delete(f.contents) delete(f.contents)
} }
db_open :: proc(cfg_path: string) -> (database: Db, ok: bool) { make_temp_path :: proc() -> string {
database.cfg = load_config(cfg_path) or_return ts := time.time_to_unix(time.now())
b: strings.Builder
{ strings.builder_init(&b)
db: ^rawptr defer strings.builder_destroy(&b)
rc := sqlite.db_open(":memory:", &db) fmt.sbprintf(&b, "/tmp/envr-%d-%d.db", os.get_pid(), ts)
if rc != sqlite.OK { return strings.to_string(b)
fmt.printf("Error opening in-memory database: %s\n", sqlite.db_errmsg(db))
return
}
create_sql: cstring = "CREATE TABLE IF NOT EXISTS envr_env_files (path TEXT PRIMARY KEY NOT NULL, remotes TEXT, sha256 TEXT NOT NULL, contents TEXT NOT NULL)"
rc = sqlite.db_exec(db, create_sql, nil, nil, nil)
if rc != sqlite.OK {
fmt.printf("Error creating table: %s\n", sqlite.db_errmsg(db))
sqlite.db_close(db)
return
}
database.db = db
}
// TODO: Use different allocators?
data_path := data_path(database.cfg.config_path, context.temp_allocator)
if os.exists(data_path) {
if ok = db_restore_from_encrypted(&database, data_path); !ok {
sqlite.db_close(database.db)
return
}
} else {
// DB was created
database.changed = true
}
return database, true
} }
db_restore_from_encrypted :: proc(db: ^Db, data_path: string) -> bool { db_open :: proc(cfg_path: string) -> (Db, bool) {
encrypted_data, read_err := os.read_entire_file_from_path(data_path, context.allocator) cfg, ok := load_config(cfg_path)
defer delete(encrypted_data) if !ok {
if read_err != nil { return Db{}, false
fmt.printf("Error reading encrypted database: %v\n", read_err)
return false
} }
plaintext, dec_ok := decrypt(encrypted_data, db.cfg.Keys[:]) data_path := data_path(cfg.config_path)
if !dec_ok { _, stat_err := os.stat(data_path, context.allocator)
fmt.println("Error: decryption failed")
return false
}
defer delete(plaintext)
n := i64(len(plaintext)) db: ^rawptr
buf := sqlite.malloc64(n) rc := sqlite.db_open(":memory:", &db)
if buf == nil {
fmt.println("Error: failed to allocate buffer for deserialization")
return false
}
copy(buf[:len(plaintext)], plaintext)
rc := sqlite.deserialize(
db.db,
"main",
buf,
n,
n,
sqlite.DESERIALIZE_FREEONCLOSE | sqlite.DESERIALIZE_RESIZEABLE,
)
if rc != sqlite.OK { if rc != sqlite.OK {
sqlite.free(buf) fmt.printf("Error opening in-memory database: %s\n", sqlite.db_errmsg(db))
fmt.printf("Error deserializing database: %s\n", sqlite.db_errmsg(db.db)) return Db{}, false
return false
} }
return true create_sql: cstring = "CREATE TABLE IF NOT EXISTS envr_env_files (path TEXT PRIMARY KEY NOT NULL, remotes TEXT, sha256 TEXT NOT NULL, contents TEXT NOT NULL)"
rc = sqlite.db_exec(db, create_sql, nil, nil, nil)
if rc != sqlite.OK {
fmt.printf("Error creating table: %s\n", sqlite.db_errmsg(db))
sqlite.db_close(db)
return Db{}, false
}
if stat_err == nil {
if !db_restore_from_encrypted(db, cfg) {
sqlite.db_close(db)
return Db{}, false
}
}
return Db{db = db, cfg = cfg, changed = stat_err != nil}, true
} }
db_close :: proc(d: ^Db) { db_close :: proc(d: ^Db) {
defer sqlite.db_close(d.db) defer sqlite.db_close(d.db)
defer delete_config(&d.cfg)
if d.changed { if d.changed {
rc := sqlite.db_exec(d.db, "VACUUM", nil, nil, nil) rc := sqlite.db_exec(d.db, "VACUUM", nil, nil, nil)
@@ -168,6 +134,13 @@ db_close :: proc(d: ^Db) {
} }
} }
// Caller is responsible for calling:
// ```odin
// delete(results)
// for &result in results {
// delete(&result)
// }
// ```
db_list :: proc(d: ^Db, allocator := context.allocator) -> (results: [dynamic]EnvFile, ok: bool) { db_list :: proc(d: ^Db, allocator := context.allocator) -> (results: [dynamic]EnvFile, ok: bool) {
stmt: ^rawptr stmt: ^rawptr
rc := sqlite.prepare_v2( rc := sqlite.prepare_v2(
@@ -216,6 +189,145 @@ db_list :: proc(d: ^Db, allocator := context.allocator) -> (results: [dynamic]En
return return
} }
db_restore_from_encrypted :: proc(db: ^rawptr, cfg: Config) -> bool {
encrypted_data, read_err := os.read_entire_file_from_path(
data_path(cfg.config_path),
context.allocator,
)
defer delete(encrypted_data)
if read_err != nil {
fmt.printf("Error reading encrypted database: %v\n", read_err)
return false
}
plaintext, dec_ok := decrypt(encrypted_data, cfg.Keys[:])
if !dec_ok {
fmt.println("Error: decryption failed")
return false
}
defer delete(plaintext)
n := i64(len(plaintext))
buf := sqlite.malloc64(n)
if buf == nil {
fmt.println("Error: failed to allocate buffer for deserialization")
return false
}
copy(buf[:len(plaintext)], plaintext)
rc := sqlite.deserialize(
db,
"main",
buf,
n,
n,
sqlite.DESERIALIZE_FREEONCLOSE | sqlite.DESERIALIZE_RESIZEABLE,
)
if rc != sqlite.OK {
sqlite.free(buf)
fmt.printf("Error deserializing database: %s\n", sqlite.db_errmsg(db))
return false
}
return true
}
get_git_remotes :: proc(dir: string) -> [dynamic]string {
remotes: [dynamic]string
remote_set: map[string]bool
b: strings.Builder
strings.builder_init(&b)
defer strings.builder_destroy(&b)
fmt.sbprintf(&b, "%s-git-remotes", make_temp_path())
tmp_path := strings.to_string(b)
tmp_file, tmp_err := os.open(tmp_path, os.O_CREATE | os.O_WRONLY | os.O_TRUNC)
if tmp_err != nil {
return remotes
}
args := []string{"git", "remote", "-v"}
desc := os.Process_Desc {
command = args,
stdout = tmp_file,
stderr = nil,
working_dir = dir,
}
p, start_err := os.process_start(desc)
os.close(tmp_file)
if start_err != nil {
os.remove(tmp_path)
return remotes
}
state, wait_err := os.process_wait(p)
if wait_err != nil || state.exit_code != 0 {
os.remove(tmp_path)
return remotes
}
data, read_err := os.read_entire_file_from_path(tmp_path, context.allocator)
defer delete(data)
os.remove(tmp_path)
if read_err != nil {
return remotes
}
lines := strings.split(string(data), "\n")
for &line in lines {
line = strings.trim_space(line)
if len(line) == 0 {
continue
}
parts := strings.fields(line)
if len(parts) >= 2 {
remote_set[parts[1]] = true
}
}
for remote, _ in remote_set {
cloned, _ := strings.clone(remote)
append(&remotes, cloned)
}
return remotes
}
new_env_file :: proc(path: string) -> (EnvFile, bool) {
abs_path, abs_err := filepath.abs(path)
if abs_err != nil {
fmt.printf("Error getting absolute path: %v\n", abs_err)
return EnvFile{}, false
}
dir := filepath.dir(abs_path)
remotes := get_git_remotes(dir)
data, read_err := os.read_entire_file_from_path(abs_path, context.allocator)
defer delete(data)
if read_err != nil {
fmt.printf("Error reading file %s: %v\n", abs_path, read_err)
return EnvFile{}, false
}
digest := hash.hash_bytes(hash.Algorithm.SHA256, data, context.temp_allocator)
// TODO: Handle error
hex_bytes, _ := hex.encode(digest)
return EnvFile {
Path = abs_path,
Dir = dir,
Remotes = remotes,
Sha256 = string(hex_bytes),
contents = string(data),
},
true
}
db_insert :: proc(d: ^Db, file: EnvFile) -> bool { db_insert :: proc(d: ^Db, file: EnvFile) -> bool {
remotes_json, marshal_err := json.marshal(file.Remotes) remotes_json, marshal_err := json.marshal(file.Remotes)
if marshal_err != nil { if marshal_err != nil {
@@ -355,36 +467,75 @@ db_delete :: proc(d: ^Db, path: string) -> bool {
return true return true
} }
new_env_file :: proc(path: string) -> (EnvFile, bool) { to_cstring :: proc {
abs_path, abs_err := filepath.abs(path) string_to_cstring,
if abs_err != nil { strings.to_cstring,
fmt.printf("Error getting absolute path: %v\n", abs_err) }
return EnvFile{}, false
string_to_cstring :: proc(s: string, allocator := context.allocator) -> cstring {
cs, err := strings.clone_to_cstring(s, allocator)
if err != nil {
fmt.printf("Failed to convert string to cstring: %v\n", err)
panic("Allocation Exception")
}
return cs
}
clone_cstring :: proc(c: cstring, allocator := context.allocator) -> string {
str, err := strings.clone_from_cstring(c, allocator)
if err != nil {
fmt.printf("Failed to convert string to cstring: %v\n", err)
delete(str)
panic("Allocation Exception")
} }
dir := filepath.dir(abs_path) return str
}
remotes := get_git_remotes(dir) db_update_required :: proc(status: SyncFlag) -> bool {
return .BackedUp in status || .DirUpdated in status
}
data, read_err := os.read_entire_file_from_path(abs_path, context.allocator) shares_remote :: proc(f: ^EnvFile, remotes: []string) -> bool {
defer delete(data) for r1 in f.Remotes {
if read_err != nil { for r2 in remotes {
fmt.printf("Error reading file %s: %v\n", abs_path, read_err) if r1 == r2 {
return EnvFile{}, false return true
}
}
}
return false
}
update_dir :: proc(f: ^EnvFile, new_dir: string) {
f.Dir = new_dir
base := filepath.base(f.Path)
new_path, _ := strings.concatenate({new_dir, "/", base})
f.Path = new_path
f.Remotes = get_git_remotes(new_dir)
}
find_moved_dirs :: proc(d: ^Db, f: ^EnvFile) -> ([dynamic]string, bool) {
feats := check_features()
if .Fd not_in feats || .Git not_in feats {
fmt.println("Error: fd and git are required for moved dir detection")
return {}, false
} }
digest := hash.hash_bytes(hash.Algorithm.SHA256, data, context.temp_allocator) roots, roots_ok := find_git_roots(d.cfg)
// TODO: Handle error if !roots_ok {
hex_bytes, _ := hex.encode(digest) return {}, false
}
return EnvFile { moved: [dynamic]string
Path = abs_path, for root in roots {
Dir = dir, remotes := get_git_remotes(root)
Remotes = remotes, if shares_remote(f, remotes[:]) {
Sha256 = string(hex_bytes), cloned, _ := strings.clone(root)
contents = string(data), append(&moved, cloned)
}, }
true }
return moved, true
} }
db_sync :: proc(d: ^Db, f: ^EnvFile) -> (SyncFlag, string) { db_sync :: proc(d: ^Db, f: ^EnvFile) -> (SyncFlag, string) {
@@ -463,31 +614,6 @@ env_file_sync :: proc(f: ^EnvFile, dir: SyncDirection, d: ^Db) -> (SyncFlag, str
return result, "" return result, ""
} }
find_moved_dirs :: proc(d: ^Db, f: ^EnvFile) -> ([dynamic]string, bool) {
roots, roots_ok := find_git_roots(d.cfg)
if !roots_ok {
return {}, false
}
moved: [dynamic]string
for root in roots {
remotes := get_git_remotes(root)
if shares_remote(f, remotes[:]) {
cloned, _ := strings.clone(root)
append(&moved, cloned)
}
}
return moved, true
}
update_dir :: proc(f: ^EnvFile, new_dir: string) {
f.Dir = new_dir
base := filepath.base(f.Path)
new_path, _ := strings.concatenate({new_dir, "/", base})
f.Path = new_path
f.Remotes = get_git_remotes(new_dir)
}
// Loads the contents of the the file at f.Path into f.contents // Loads the contents of the the file at f.Path into f.contents
// //
// Caller is responsible for calling delete on f.contents and f.Sha256 // Caller is responsible for calling delete on f.contents and f.Sha256
@@ -509,72 +635,3 @@ env_file_backup :: proc(f: ^EnvFile) -> bool {
return true return true
} }
shares_remote :: proc(f: ^EnvFile, remotes: []string) -> bool {
for r1 in f.Remotes {
for r2 in remotes {
if r1 == r2 {
return true
}
}
}
return false
}
get_git_remotes :: proc(dir: string) -> [dynamic]string {
remotes: [dynamic]string
remote_set: map[string]bool
defer delete(remote_set)
config_path, _ := filepath.join({dir, ".git", "config"}, context.temp_allocator)
m, _, ok := ini.load_map_from_path(config_path, context.allocator)
if !ok {
return remotes
}
defer ini.delete_map(m)
for section_name, section in m {
if strings.has_prefix(section_name, "remote ") {
if url, ok := section["url"]; ok {
remote_set[url] = true
}
}
}
for remote in remote_set {
cloned, _ := strings.clone(remote)
append(&remotes, cloned)
}
return remotes
}
db_update_required :: proc(status: SyncFlag) -> bool {
return .BackedUp in status || .DirUpdated in status
}
to_cstring :: proc {
string_to_cstring,
strings.to_cstring,
}
string_to_cstring :: proc(s: string, allocator := context.allocator) -> cstring {
cs, err := strings.clone_to_cstring(s, allocator)
if err != nil {
fmt.printf("Failed to convert string to cstring: %v\n", err)
panic("Allocation Exception")
}
return cs
}
// Caller is responsible for freeing the result
clone_cstring :: proc(c: cstring, allocator := context.allocator) -> string {
str, err := strings.clone_from_cstring(c, allocator)
if err != nil {
fmt.printf("Failed to convert string to cstring: %v\n", err)
delete(str)
panic("Allocation Exception")
}
return str
}

View File

@@ -319,85 +319,11 @@ test_shares_remote_both_empty :: proc(t: ^testing.T) {
testing.expect(t, !shares_remote(&f, remotes), "both empty should not share") testing.expect(t, !shares_remote(&f, remotes), "both empty should not share")
} }
delete_remotes :: proc(remotes: [dynamic]string) {
for &r in remotes {
delete(r)
}
delete(remotes)
}
@(test) @(test)
test_get_git_remotes_single :: proc(t: ^testing.T) { test_make_temp_path_format :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-remotes-%d", os.get_pid()) p := make_temp_path()
os.mkdir_all(base) testing.expect(t, strings.has_suffix(p, ".db"), "should end with .db")
defer os.remove_all(base) testing.expect(t, strings.contains(p, fmt.tprintf("%d", os.get_pid())), "should contain PID")
git_dir := fmt.tprintf("%s/.git", base)
os.mkdir_all(git_dir)
config_content := "[core]\n\trepositoryformatversion = 0\n[remote \"origin\"]\n\turl = git@github.com:user/repo.git\n\tfetch = +refs/heads/*:refs/remotes/origin/*\n"
config_path := fmt.tprintf("%s/config", git_dir)
err := os.write_entire_file(config_path, transmute([]u8)config_content)
testing.expect(t, err == nil, "should write .git/config")
remotes := get_git_remotes(base)
defer delete_remotes(remotes)
testing.expect(t, len(remotes) == 1, "should find 1 remote")
if len(remotes) != 1 do return
testing.expect_value(t, remotes[0], "git@github.com:user/repo.git")
}
@(test)
test_get_git_remotes_multiple :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-remotes-multi-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
git_dir := fmt.tprintf("%s/.git", base)
os.mkdir_all(git_dir)
config_content := "[remote \"origin\"]\n\turl = git@github.com:user/repo.git\n[remote \"upstream\"]\n\turl = https://gitlab.com/upstream/repo.git\n"
config_path := fmt.tprintf("%s/config", git_dir)
err := os.write_entire_file(config_path, transmute([]u8)config_content)
testing.expect(t, err == nil, "should write .git/config")
remotes := get_git_remotes(base)
defer delete_remotes(remotes)
testing.expect(t, len(remotes) == 2, "should find 2 remotes")
}
@(test)
test_get_git_remotes_no_config :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-remotes-none-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
remotes := get_git_remotes(base)
defer delete_remotes(remotes)
testing.expect(t, len(remotes) == 0, "should return empty when no .git/config")
}
@(test)
test_get_git_remotes_no_remotes :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-remotes-empty-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
git_dir := fmt.tprintf("%s/.git", base)
os.mkdir_all(git_dir)
config_content := "[core]\n\trepositoryformatversion = 0\n\tbare = false\n"
config_path := fmt.tprintf("%s/config", git_dir)
err := os.write_entire_file(config_path, transmute([]u8)config_content)
testing.expect(t, err == nil, "should write .git/config")
remotes := get_git_remotes(base)
defer delete_remotes(remotes)
testing.expect(t, len(remotes) == 0, "should return empty when no remote sections")
} }
@(test) @(test)
@@ -472,51 +398,3 @@ test_update_dir :: proc(t: ^testing.T) {
testing.expect_value(t, f.Path, "/new/location/.env") testing.expect_value(t, f.Path, "/new/location/.env")
} }
@(test)
test_closing_db_has_no_leaks :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-leak-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
cfg_path, err := filepath.join([]string{base, "config.json"}, context.temp_allocator)
testing.expect(t, err == nil, "cfgPath should build successfully")
cfg := new_config([]string{"fixtures/keys/insecure-test-key"}, cfg_path)
defer delete_config(&cfg)
testing.expect(t, save_config(cfg, force = true), "save should succeed")
db, ok := db_open(cfg_path)
testing.expect(t, ok, "db should open")
if !ok do return
db_close(&db)
}
@(test)
test_open_existing_db_has_no_leaks :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-leak-existing-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
cfg_path, err := filepath.join([]string{base, "config.json"}, context.temp_allocator)
testing.expect(t, err == nil, "cfgPath should build successfully")
cfg := new_config([]string{"fixtures/keys/insecure-test-key"}, cfg_path)
defer delete_config(&cfg)
testing.expect(t, save_config(cfg, force = true), "save should succeed")
// First open/close creates data.envr on disk
db, ok := db_open(cfg_path)
testing.expect(t, ok, "db should open")
if !ok do return
f := make_test_env_file("/project/.env", "abc123", "SECRET=value", []string{"git@github.com:user/repo.git"})
defer delete(f.Remotes)
testing.expect(t, db_insert(&db, f), "insert should succeed")
db_close(&db)
// Second open exercises db_restore_from_encrypted
db2, ok2 := db_open(cfg_path)
testing.expect(t, ok2, "db should open existing")
if !ok2 do return
db_close(&db2)
}

View File

@@ -45,6 +45,7 @@ at before, restore your backup with:
* [envr backup](envr_backup.md) - Import a .env file into envr * [envr backup](envr_backup.md) - Import a .env file into envr
* [envr check](envr_check.md) - check if files in the current directory are backed up * [envr check](envr_check.md) - check if files in the current directory are backed up
* [envr deps](envr_deps.md) - Check for missing binaries
* [envr edit-config](envr_edit-config.md) - Edit your config with your default editor * [envr edit-config](envr_edit-config.md) - Edit your config with your default editor
* [envr init](envr_init.md) - Set up envr * [envr init](envr_init.md) - Set up envr
* [envr list](envr_list.md) - View your tracked files * [envr list](envr_list.md) - View your tracked files

24
docs/cli/envr_deps.md Normal file
View File

@@ -0,0 +1,24 @@
## envr deps
Check for missing binaries
### Synopsis
envr relies on external binaries for certain functionality.
The check command reports on which binaries are available and which are not.
```
envr deps [flags]
```
### Options
```
-h, --help help for deps
```
### SEE ALSO
* [envr](envr.md) - Manage your .env files.

51
features.odin Normal file
View File

@@ -0,0 +1,51 @@
package main
import "base:runtime"
import "core:mem"
import "core:os"
import "core:strings"
Feature :: enum {
Git,
Fd,
}
AvailableFeatures :: bit_set[Feature]
check_features :: proc() -> AvailableFeatures {
feats: AvailableFeatures
s: mem.Scratch
mem.scratch_init(&s, 4 * mem.DEFAULT_PAGE_SIZE)
defer mem.scratch_destroy(&s)
context.temp_allocator = mem.scratch_allocator(&s)
path_env := os.get_env("PATH", context.temp_allocator)
paths := strings.split(path_env, ":", context.temp_allocator)
if find_binary(paths, "git") != "" {
feats += {.Git}
}
if find_binary(paths, "fd") != "" {
feats += {.Fd}
}
return feats
}
find_binary :: proc(
paths: []string,
name: string,
allocator: runtime.Allocator = context.temp_allocator,
) -> string {
for p in paths {
candidate := strings.join({strings.trim_right(p, "/"), name}, "/", allocator)
_, err := os.stat(candidate, allocator)
if err == nil {
return candidate
}
}
return ""
}

34
features_test.odin Normal file
View File

@@ -0,0 +1,34 @@
package main
import "core:os"
import "core:strings"
import "core:testing"
@(test)
test_find_binary_exists :: proc(t: ^testing.T) {
path := os.get_env("PATH", context.temp_allocator)
paths := strings.split(path, ":", context.temp_allocator)
result := find_binary(paths, "sh")
testing.expect(t, result != "", "sh should be found on PATH")
}
@(test)
test_find_binary_not_exists :: proc(t: ^testing.T) {
old_path := os.get_env("PATH", context.temp_allocator)
defer {
if old_path != "" {
os.set_env("PATH", old_path)
}
}
os.set_env("PATH", "/tmp/envr-nope")
path := os.get_env("PATH", context.temp_allocator)
paths := strings.split(path, ":", context.temp_allocator)
result := find_binary(paths, "no_such_binary_xyz")
testing.expect(t, result == "", "nonexistent binary should not be found")
}

177
findr/PLAN.md Normal file
View File

@@ -0,0 +1,177 @@
# findr — Gitignored File Finder
## Overview
findr is a native Odin tool that finds **gitignored files** within git repositories. It replaces envr's current approach of running `fd` twice (all files vs. unignored files) and diffing the results.
**Simplified scope:** findr does one thing — walks directories, finds git repos, reads each repo's `.gitignore`, and prints every gitignored file. No flags, no filtering, no pattern matching. envr handles result filtering itself.
## Current fd Usage in envr (being replaced)
1. **`scan.odin:13-43`** (`scan_path`) — runs `fd` twice per search path:
- Run 1: `fd -a <matcher> [-E <exclude>]... -HI <path>` → all files including gitignored
- Run 2: `fd -a <matcher> [-E <exclude>]... -H <path>` → hidden but NOT gitignored
- Diff = gitignored files only
2. Both go through `run_fd` (`scan.odin:68-118`), which spawns a subprocess and captures output via temp files.
After findr integration, `scan_path` calls `findr.walk(path)` directly — no subprocess, no double-run, no diff.
## Directory Structure
```
findr/
findr.odin # main + CLI (positional dir args only)
walker.odin # recursive directory walker using core:sys/linux getdents
gitignore.odin # .gitignore parsing + glob→regex transpilation + matching
test_env.odin # test harness: temp dir, mock filesystem, assert helpers
findr_test.odin # integration tests (10 tests)
gitignore_test.odin # transpilation + matching unit tests (22 tests)
```
## Decisions
- **Scope**: findr prints ALL gitignored files. No regex filtering, no exclude patterns, no type filters. envr post-processes the output.
- **Gitignore matching**: Transpile gitignore glob patterns to regex, then use `core:text/regex`. No dedicated glob matcher.
- **Stat avoidance**: Use `core:sys/linux` getdents directly — read `dirent.type` from the kernel, never call stat.
- **Architecture**: Separate directory with its own `main`. Core logic (`walk` proc + `gitignore` package) designed to be importable into envr later.
## CLI Interface
```
findr [dir1] [dir2] ...
```
No flags. Defaults to `.` if no dirs given. Prints absolute or relative paths (as given) to stdout, one per line.
## Build
```bash
odin build findr -o:speed -out:findr/findr
```
## How It Works
```
walk(dir):
entries = getdents(dir) # via core:sys/linux, zero stat calls
if entries contains ".git/":
gi = parse(.gitignore) # if present
for entry in entries:
if entry is gitignored file:
emit entry path
if entry is dir (not ignored):
walk(entry) # recurse to find nested repos
else:
for entry in entries:
if entry is dir:
walk(entry) # descend looking for repos
```
Key behaviors:
- **Nested repos**: When a repo is found, subdirectories are still traversed to find nested repos. Gitignored directories are pruned (not descended into).
- **Flat gitignore**: Only the root `.gitignore` is read. `.gitignore` files in subdirectories of a repo are ignored.
- **Non-repo dirs**: Traversed recursively to find repos. No gitignore rules apply.
## Performance Architecture
### Implemented
- **Stat avoidance via `dirent.type`** — Uses `core:sys/linux` getdents directly, bypassing `core:os` which calls `openat` + `fstat` per entry. File type comes free from the directory entry.
- **Prune ignored directories** — When a directory matches a gitignore pattern, it is not descended into. Skips potentially thousands of readdir calls.
### Future (if needed)
- Work-stealing parallel traversal (per-thread LIFO deques with batch stealing, like fd)
- BufWriter on stdout for large result sets
- Arena allocators for path strings
## Testing Strategy
- **In-process integration tests** — Tests call `walk()` directly (not via subprocess), build mock filesystems in temp dirs, and compare sorted output.
- **Unit tests** — Pure-function tests for glob→regex transpilation and gitignore matching.
- **Output sorting for determinism** — Always sort output lines before comparison.
- **Memory tracking** — Odin's test runner reports leaks automatically. All 32 tests pass with zero leaks.
### Test Coverage (findr_test.odin)
| Test | What it covers |
|---|---|
| `test_basic_gitignored` | Repo with `.gitignore`, gitignored files emitted, normal files skipped |
| `test_non_repo_not_scanned` | Dirs without `.git/` produce no output |
| `test_negation_pattern` | `!prod.env` un-ignores a file |
| `test_dir_only_pattern` | `node_modules/` pattern doesn't emit file results |
| `test_multiple_repos` | Multiple repos in one tree, each with its own `.gitignore` |
| `test_nested_repos` | Repo inside a repo, both scanned independently |
| `test_gitignore_in_subdir_ignored` | Subdirectory `.gitignore` files are not read |
| `test_no_gitignore_file` | Repo with `.git/` but no `.gitignore` produces nothing |
| `test_empty_gitignore` | Comments and blank lines only → no results |
| `test_multiple_search_dirs` | Multiple top-level search dirs in one call |
### Gitignore Unit Tests (gitignore_test.odin)
22 tests covering: simple/anchored patterns, `*`, `?`, `[abc]`, `[!abc]`, dot escaping, globstar variants, backslash escapes, empty patterns, basic matching, negation, dir-only, comments, blank lines, last-match-wins, env patterns.
## Glob→Regex Transpilation Rules
| Gitignore pattern | Regex | Notes |
|---|---|---|
| `foo` | `(^|/)foo(/.*)?$` | matches at any depth |
| `/foo` | `^foo(/.*)?$` | anchored to gitignore dir |
| `foo/` | `(^|/)foo/.*$` | directory only |
| `*.log` | `(^|/)[^/]*\.log$` | `*` = any chars except `/` |
| `**/foo` | `(^|/)(.*/)?foo(/.*)?$` | `**` = any chars including `/` |
| `foo/**/bar` | `(^|/)foo/(.*/)?bar(/.*)?$` | `**` between segments |
| `!pattern` | (handled by layer) | negation flag, not regex |
| `#comment` | (skipped) | |
| `[abc]` | `[abc]` | same regex syntax |
| `?` | `[^/]` | single char, no `/` |
## Implementation Phases
### Phase 1: Gitignore Transpiler + Tests ✅
**Goal:** Isolated, fully-tested glob→regex transpiler.
**Result:** 22 tests, all passing, zero leaks.
---
### Phase 2: findr Walker + Tests ✅
**Goal:** Working tool that finds gitignored files in git repos.
**Built:**
- `walker.odin` — Single-threaded DFS using `core:sys/linux` getdents. Finds repos, reads `.gitignore`, emits gitignored files, recurses into subdirs for nested repos.
- `findr.odin` — Minimal CLI: `findr [dirs...]`, no flags.
- `test_env.odin` — Test harness with temp dirs and mock filesystems.
- `findr_test.odin` — 10 integration tests.
**Result:** All 32 tests pass (22 gitignore + 10 walker), zero leaks.
---
### Phase 3: Parallel Traversal (future)
**Goal:** Parallelize directory descent for large trees.
---
### Phase 4: Benchmark (future)
**Goal:** Quantify performance vs fd on large directory trees.
---
### Phase 5: Integrate into envr (future)
**Goal:** Replace `run_fd` in `scan.odin`. `scan_path` calls `findr.walk()` directly instead of two subprocess runs + diff.
## Risks
| Risk | Mitigation |
|---|---|
| Single-threaded may be slow on huge trees | Add threading in Phase 3 after correctness |
| Gitignore edge cases (`**/foo`, `foo/**/bar`) | Comprehensive gitignore_test.odin with spec examples |
| dirent.type may be UNKNOWN on some filesystems | Fall back to stat only when type is UNKNOWN |
| Missing nested `.env` files in monorepos | Accepted limitation — flat gitignore model |
| Memory allocation churn from path strings | Use thread-local arena allocators in Phase 3 |

33
findr/findr.odin Normal file
View File

@@ -0,0 +1,33 @@
package findr
import "core:fmt"
import "core:os"
main :: proc() {
args := os.args
search_dirs := make([dynamic]string)
defer delete(search_dirs)
for i in 1..<len(args) {
append(&search_dirs, args[i])
}
if len(search_dirs) == 0 {
append(&search_dirs, ".")
}
results := make([dynamic]string)
defer {
for r in results { delete(r) }
delete(results)
}
for dir in search_dirs {
walk(dir, &results)
}
for r in results {
fmt.println(r)
}
}

View File

@@ -1,15 +1,7 @@
package findr package findr
import "core:os"
import "core:sort"
import "core:strings"
import "core:sys/linux"
import "core:testing" import "core:testing"
// ============================================================================
// Gitignored file emission tests (emit ONLY gitignored files, descend everywhere)
// ============================================================================
@(test) @(test)
test_basic_gitignored :: proc(t: ^testing.T) { test_basic_gitignored :: proc(t: ^testing.T) {
env := create_test_env() env := create_test_env()
@@ -21,9 +13,7 @@ test_basic_gitignored :: proc(t: ^testing.T) {
create_file(env, "repo/secrets.env") create_file(env, "repo/secrets.env")
create_file(env, "repo/normal.txt") create_file(env, "repo/normal.txt")
assert_output(t, env, nil, {}, { assert_output(t, env, nil, {"repo/.env", "repo/secrets.env"})
"repo/.env", "repo/secrets.env",
})
} }
@(test) @(test)
@@ -35,7 +25,7 @@ test_non_repo_not_scanned :: proc(t: ^testing.T) {
create_file(env, "norepo/.gitignore", "*.env\n") create_file(env, "norepo/.gitignore", "*.env\n")
create_file(env, "norepo/.env") create_file(env, "norepo/.env")
assert_output_empty(t, env, nil, {}) assert_output_empty(t, env, nil)
} }
@(test) @(test)
@@ -49,9 +39,23 @@ test_negation_pattern :: proc(t: ^testing.T) {
create_file(env, "repo/secrets.env") create_file(env, "repo/secrets.env")
create_file(env, "repo/prod.env") create_file(env, "repo/prod.env")
assert_output(t, env, nil, {}, { assert_output(t, env, nil, {"repo/.env", "repo/secrets.env"})
"repo/.env", "repo/secrets.env", }
})
@(test)
test_dir_only_pattern :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "node_modules/\n")
create_file(env, "repo/node_modules", "should not match (it's a file)")
create_dir(env, "repo/ignored_dir")
create_file(env, "repo/.gitignore", "ignored_dir/\n")
// dir-only patterns don't produce file results
assert_output(t, env, nil, {})
} }
@(test) @(test)
@@ -67,9 +71,7 @@ test_multiple_repos :: proc(t: ^testing.T) {
create_file(env, "repo2/.gitignore", "*.key\n") create_file(env, "repo2/.gitignore", "*.key\n")
create_file(env, "repo2/secret.key") create_file(env, "repo2/secret.key")
assert_output(t, env, nil, {}, { assert_output(t, env, nil, {"repo1/a.env", "repo2/secret.key"})
"repo1/a.env", "repo2/secret.key",
})
} }
@(test) @(test)
@@ -85,13 +87,11 @@ test_nested_repos :: proc(t: ^testing.T) {
create_file(env, "parent/child/.gitignore", "*.key\n") create_file(env, "parent/child/.gitignore", "*.key\n")
create_file(env, "parent/child/api.key") create_file(env, "parent/child/api.key")
assert_output(t, env, nil, {}, { assert_output(t, env, nil, {"parent/top.env", "parent/child/api.key"})
"parent/top.env", "parent/child/api.key",
})
} }
@(test) @(test)
test_nested_gitignore_read :: proc(t: ^testing.T) { test_gitignore_in_subdir_ignored :: proc(t: ^testing.T) {
env := create_test_env() env := create_test_env()
defer destroy_test_env(&env) defer destroy_test_env(&env)
@@ -102,43 +102,10 @@ test_nested_gitignore_read :: proc(t: ^testing.T) {
create_file(env, "repo/sub/secret.txt") create_file(env, "repo/sub/secret.txt")
create_file(env, "repo/sub/.env") create_file(env, "repo/sub/.env")
assert_output(t, env, nil, {}, { // .gitignore in subdir is not read (flat model).
"repo/sub/secret.txt", "repo/sub/.env", // secret.txt should NOT appear (subdir .gitignore ignored).
}) // .env should NOT appear (it's nested, not top-level of repo).
} assert_output(t, env, nil, {})
@(test)
test_nested_gitignore_negation :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.log\n")
create_dir(env, "repo/sub")
create_file(env, "repo/sub/.gitignore", "!important.log\n")
create_file(env, "repo/sub/important.log")
create_file(env, "repo/sub/debug.log")
assert_output(t, env, nil, {}, {
"repo/sub/debug.log",
})
}
@(test)
test_multisegment_pattern :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "build/output.txt\n")
create_dir(env, "repo/build")
create_file(env, "repo/build/output.txt")
create_file(env, "repo/build/other.txt")
create_file(env, "repo/output.txt")
assert_output(t, env, nil, {}, {
"repo/build/output.txt",
})
} }
@(test) @(test)
@@ -149,7 +116,7 @@ test_no_gitignore_file :: proc(t: ^testing.T) {
create_git_repo(env, "repo") create_git_repo(env, "repo")
create_file(env, "repo/.env") create_file(env, "repo/.env")
assert_output_empty(t, env, nil, {}) assert_output_empty(t, env, nil)
} }
@(test) @(test)
@@ -161,7 +128,7 @@ test_empty_gitignore :: proc(t: ^testing.T) {
create_file(env, "repo/.gitignore", "\n\n# comment\n\n") create_file(env, "repo/.gitignore", "\n\n# comment\n\n")
create_file(env, "repo/.env") create_file(env, "repo/.env")
assert_output_empty(t, env, nil, {}) assert_output_empty(t, env, nil)
} }
@(test) @(test)
@@ -172,7 +139,6 @@ test_multiple_search_dirs :: proc(t: ^testing.T) {
create_git_repo(env, "dir1/repo") create_git_repo(env, "dir1/repo")
create_file(env, "dir1/repo/.gitignore", "*.env\n") create_file(env, "dir1/repo/.gitignore", "*.env\n")
create_file(env, "dir1/repo/a.env") create_file(env, "dir1/repo/a.env")
create_file(env, "dir1/repo/normal.txt")
create_git_repo(env, "dir2/repo") create_git_repo(env, "dir2/repo")
create_file(env, "dir2/repo/.gitignore", "*.env\n") create_file(env, "dir2/repo/.gitignore", "*.env\n")
@@ -185,136 +151,10 @@ test_multiple_search_dirs :: proc(t: ^testing.T) {
results := make([dynamic]string) results := make([dynamic]string)
defer { defer {
for r in results {delete(r)} for r in results { delete(r) }
delete(results) delete(results)
} }
walk(dir1, &results)
opts := WalkOptions{} walk(dir2, &results)
thread_count := os.get_processor_core_count()
walk({dir1, dir2}, &results, opts, thread_count)
testing.expect_value(t, len(results), 2) testing.expect_value(t, len(results), 2)
actual := make([dynamic]string, 0, len(results))
for r in results {
stripped := r
if strings.has_prefix(stripped, env.temp_dir) {
stripped = stripped[len(env.temp_dir):]
if len(stripped) > 0 && stripped[0] == '/' {
stripped = stripped[1:]
}
}
append(&actual, stripped)
}
defer delete(actual)
expected := []string{"dir1/repo/a.env", "dir2/repo/b.env"}
sort.quick_sort(actual[:])
sort.quick_sort(expected[:])
for i in 0 ..< len(expected) {
testing.expect_value(t, actual[i], expected[i])
}
}
// ============================================================================
// Ignored directory recursion tests
// ============================================================================
@(test)
test_ignored_dir_descended :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "secrets/\n")
create_dir(env, "repo/secrets")
create_file(env, "repo/secrets/.env")
create_file(env, "repo/secrets/api.key")
// Ignored dir's contents are emitted AND descended into
assert_output(t, env, nil, {}, {
"repo/secrets/", "repo/secrets/.env", "repo/secrets/api.key",
})
}
@(test)
test_nested_ignored_dir :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "build/\n")
create_dir(env, "repo/build")
create_dir(env, "repo/build/sub")
create_file(env, "repo/build/output.txt")
create_file(env, "repo/build/sub/deep.env")
assert_output(t, env, nil, {}, {
"repo/build/", "repo/build/output.txt",
"repo/build/sub/", "repo/build/sub/deep.env",
})
}
// ============================================================================
// Filter tests (excludes, pattern)
// ============================================================================
@(test)
test_excludes_prune_dirs :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n")
create_file(env, "repo/.env")
create_dir(env, "repo/vendor")
create_file(env, "repo/vendor/lib.env")
assert_output(t, env, nil,
{excludes = {"vendor"}},
{"repo/.env"},
)
}
@(test)
test_pattern_filters_results :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n*.key\n")
create_file(env, "repo/.env")
create_file(env, "repo/secrets.env")
create_file(env, "repo/master.key")
assert_output(t, env, nil,
{pattern = "\\.env$"},
{"repo/.env", "repo/secrets.env"},
)
}
// ============================================================================
// Special file type tests
// ============================================================================
@(test)
test_fifo_emitted :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n*.fifo\n")
fifo_path := join_path(env.temp_dir, "repo/test.fifo")
defer delete(fifo_path)
cpath := strings.clone_to_cstring(fifo_path)
defer delete(cpath)
linux.mknod(cpath, linux.S_IFIFO | linux.Mode{.IRUSR, .IWUSR}, 0)
assert_output(t, env, nil,
{pattern = "\\.fifo$"},
{"repo/test.fifo"},
)
} }

View File

@@ -1,36 +1,110 @@
package findr package findr
import "core:fmt"
import "core:strings" import "core:strings"
import "core:text/regex"
Gitignore :: struct { is_regex_meta :: proc(c: u8) -> bool {
rules: [dynamic]Rule, switch c {
case '.', '+', '(', ')', '{', '}', '^', '$', '|':
return true
}
return false
}
glob_to_regex :: proc(pattern: string, anchored: bool) -> string {
sb: strings.Builder
strings.builder_init(&sb)
defer strings.builder_destroy(&sb)
if anchored {
fmt.sbprintf(&sb, "^")
} else {
fmt.sbprintf(&sb, "(^|/)")
}
i := 0
for i < len(pattern) {
c := pattern[i]
if c == '*' {
if i + 1 < len(pattern) && pattern[i + 1] == '*' {
prev_slash := i == 0 || pattern[i - 1] == '/'
at_end := i + 2 >= len(pattern)
next_slash := !at_end && pattern[i + 2] == '/'
if prev_slash && (next_slash || at_end) {
if next_slash {
i += 3
fmt.sbprintf(&sb, "(.*/)?")
} else {
i += 2
fmt.sbprintf(&sb, ".*")
}
} else {
fmt.sbprintf(&sb, "[^/]*")
i += 2
}
} else {
fmt.sbprintf(&sb, "[^/]*")
i += 1
}
} else if c == '?' {
fmt.sbprintf(&sb, "[^/]")
i += 1
} else if c == '[' {
append(&sb.buf, '[')
i += 1
if i < len(pattern) && pattern[i] == '!' {
append(&sb.buf, '^')
i += 1
}
if i < len(pattern) && pattern[i] == ']' {
append(&sb.buf, ']')
i += 1
}
for i < len(pattern) && pattern[i] != ']' {
append(&sb.buf, pattern[i])
i += 1
}
if i < len(pattern) {
append(&sb.buf, ']')
i += 1
}
} else if c == '\\' {
i += 1
if i < len(pattern) {
if is_regex_meta(pattern[i]) {
append(&sb.buf, '\\')
}
append(&sb.buf, pattern[i])
i += 1
}
} else if is_regex_meta(c) {
append(&sb.buf, '\\')
append(&sb.buf, c)
i += 1
} else {
append(&sb.buf, c)
i += 1
}
}
fmt.sbprintf(&sb, "(/.*)?$")
s := strings.to_string(sb)
result, _ := strings.clone(s)
return result
} }
Rule :: struct { Rule :: struct {
pattern: GlobPattern, regex: regex.Regular_Expression,
negated: bool, negated: bool,
dir_only: bool, dir_only: bool,
} }
Match :: enum { Gitignore :: struct {
None, rules: [dynamic]Rule,
Ignored,
Unignored,
}
is_ignored :: proc(gi: ^Gitignore, path: string, is_dir: bool) -> bool {
return check_match(gi, path, is_dir) == .Ignored
}
check_match :: proc(gi: ^Gitignore, path: string, is_dir: bool) -> Match {
result := Match.None
for &rule in gi.rules {
if rule.dir_only && !is_dir do continue
if glob_match_compiled(&rule.pattern, path) {
result = rule.negated ? .Unignored : .Ignored
}
}
return result
} }
parse :: proc(content: string) -> Gitignore { parse :: proc(content: string) -> Gitignore {
@@ -72,17 +146,37 @@ parse :: proc(content: string) -> Gitignore {
if len(s) == 0 do continue if len(s) == 0 do continue
gp := glob_compile(s, anchored) regex_str := glob_to_regex(s, anchored)
append(&gi.rules, Rule{pattern = gp, negated = negated, dir_only = dir_only}) re, err := regex.create(regex_str, {regex.Flag.No_Capture})
delete(regex_str)
if err != nil do continue
append(&gi.rules, Rule{
regex = re,
negated = negated,
dir_only = dir_only,
})
} }
return gi return gi
} }
is_ignored :: proc(gi: ^Gitignore, path: string, is_dir: bool) -> bool {
matched := false
for rule in gi.rules {
if rule.dir_only && !is_dir do continue
cap, ok := regex.match(rule.regex, path)
regex.destroy(cap)
if ok {
matched = !rule.negated
}
}
return matched
}
destroy :: proc(gi: ^Gitignore) { destroy :: proc(gi: ^Gitignore) {
for &rule in gi.rules { for rule in gi.rules {
glob_destroy(&rule.pattern) regex.destroy(rule.regex)
} }
delete(gi.rules) delete(gi.rules)
} }

View File

@@ -2,6 +2,90 @@ package findr
import "core:testing" import "core:testing"
@(test)
test_glob_simple :: proc(t: ^testing.T) {
result := glob_to_regex("foo", false)
defer delete(result)
testing.expect_value(t, result, "(^|/)foo(/.*)?$")
}
@(test)
test_glob_anchored :: proc(t: ^testing.T) {
result := glob_to_regex("foo", true)
defer delete(result)
testing.expect_value(t, result, "^foo(/.*)?$")
}
@(test)
test_glob_star :: proc(t: ^testing.T) {
result := glob_to_regex("*.log", false)
defer delete(result)
testing.expect_value(t, result, "(^|/)[^/]*\\.log(/.*)?$")
}
@(test)
test_glob_question :: proc(t: ^testing.T) {
result := glob_to_regex("?.log", false)
defer delete(result)
testing.expect_value(t, result, "(^|/)[^/]\\.log(/.*)?$")
}
@(test)
test_glob_char_class :: proc(t: ^testing.T) {
result := glob_to_regex("[abc].log", false)
defer delete(result)
testing.expect_value(t, result, "(^|/)[abc]\\.log(/.*)?$")
}
@(test)
test_glob_negated_class :: proc(t: ^testing.T) {
result := glob_to_regex("[!abc].log", false)
defer delete(result)
testing.expect_value(t, result, "(^|/)[^abc]\\.log(/.*)?$")
}
@(test)
test_glob_dot_escaped :: proc(t: ^testing.T) {
result := glob_to_regex(".env", false)
defer delete(result)
testing.expect_value(t, result, "(^|/)\\.env(/.*)?$")
}
@(test)
test_glob_globstar_prefix :: proc(t: ^testing.T) {
result := glob_to_regex("**/foo", false)
defer delete(result)
testing.expect_value(t, result, "(^|/)(.*/)?foo(/.*)?$")
}
@(test)
test_glob_globstar_suffix :: proc(t: ^testing.T) {
result := glob_to_regex("abc/**", false)
defer delete(result)
testing.expect_value(t, result, "(^|/)abc/.*(/.*)?$")
}
@(test)
test_glob_globstar_middle :: proc(t: ^testing.T) {
result := glob_to_regex("foo/**/bar", false)
defer delete(result)
testing.expect_value(t, result, "(^|/)foo/(.*/)?bar(/.*)?$")
}
@(test)
test_glob_backslash_escape :: proc(t: ^testing.T) {
result := glob_to_regex("\\!foo", false)
defer delete(result)
testing.expect_value(t, result, "(^|/)!foo(/.*)?$")
}
@(test)
test_glob_empty :: proc(t: ^testing.T) {
result := glob_to_regex("", false)
defer delete(result)
testing.expect_value(t, result, "(^|/)(/.*)?$")
}
@(test) @(test)
test_is_ignored_basic :: proc(t: ^testing.T) { test_is_ignored_basic :: proc(t: ^testing.T) {
gi := parse("*.env\n") gi := parse("*.env\n")
@@ -92,27 +176,3 @@ test_is_ignored_globstar :: proc(t: ^testing.T) {
testing.expect_value(t, is_ignored(&gi, "foo/bar/cache", false), true) testing.expect_value(t, is_ignored(&gi, "foo/bar/cache", false), true)
} }
@(test)
test_star_negation_subpath :: proc(t: ^testing.T) {
gi := parse("*\n!public/\n")
defer destroy(&gi)
// public dir itself is un-ignored
testing.expect_value(t, is_ignored(&gi, "public", true), false)
// children of public/ should still be ignored by *
testing.expect_value(t, is_ignored(&gi, "public/uuid-dir", true), true)
testing.expect_value(t, is_ignored(&gi, "public/uuid-dir/file.txt", false), true)
}
@(test)
test_is_ignored_hash_pattern :: proc(t: ^testing.T) {
gi := parse("\\#*\\#\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, "#foo#", false), true)
testing.expect_value(t, is_ignored(&gi, "#test#", false), true)
testing.expect_value(t, is_ignored(&gi, "AUTHORS", false), false)
testing.expect_value(t, is_ignored(&gi, "build.zig", false), false)
testing.expect_value(t, is_ignored(&gi, "ChangeLog", false), false)
}

View File

@@ -1,203 +0,0 @@
package findr
Range :: struct {
lo: u8,
hi: u8,
}
Class_Data :: struct {
negated: bool,
ranges: [dynamic]Range,
}
Token_Kind :: enum u8 { Char, Star, Globstar, Question, Class }
Token :: struct {
kind: Token_Kind,
byte: u8,
class_idx: u16,
}
GlobPattern :: struct {
tokens: [dynamic]Token,
classes: [dynamic]Class_Data,
anchored: bool,
}
glob_compile :: proc(pattern: string, anchored: bool) -> GlobPattern {
gp: GlobPattern
gp.tokens = make([dynamic]Token)
gp.classes = make([dynamic]Class_Data)
gp.anchored = anchored
i := 0
for i < len(pattern) {
c := pattern[i]
if c == '*' {
if i + 1 < len(pattern) && pattern[i + 1] == '*' {
prev_slash := i == 0 || pattern[i - 1] == '/'
at_end := i + 2 >= len(pattern)
next_slash := !at_end && pattern[i + 2] == '/'
if prev_slash && (next_slash || at_end) {
append(&gp.tokens, Token{kind = .Globstar})
if next_slash {
i += 3
} else {
i += 2
}
} else {
append(&gp.tokens, Token{kind = .Star})
i += 2
}
} else {
append(&gp.tokens, Token{kind = .Star})
i += 1
}
} else if c == '?' {
append(&gp.tokens, Token{kind = .Question})
i += 1
} else if c == '[' {
i += 1
negated := false
if i < len(pattern) && pattern[i] == '!' {
negated = true
i += 1
}
ranges := make([dynamic]Range)
if i < len(pattern) && pattern[i] == ']' {
append(&ranges, Range{lo = ']', hi = ']'})
i += 1
}
for i < len(pattern) && pattern[i] != ']' {
if i + 2 < len(pattern) && pattern[i + 1] == '-' && pattern[i + 2] != ']' {
append(&ranges, Range{lo = pattern[i], hi = pattern[i + 2]})
i += 3
} else {
append(&ranges, Range{lo = pattern[i], hi = pattern[i]})
i += 1
}
}
if i < len(pattern) {
i += 1
}
class_idx := u16(len(gp.classes))
append(&gp.classes, Class_Data{negated = negated, ranges = ranges})
append(&gp.tokens, Token{kind = .Class, class_idx = class_idx})
} else if c == '\\' {
i += 1
if i < len(pattern) {
append(&gp.tokens, Token{kind = .Char, byte = pattern[i]})
i += 1
}
} else {
append(&gp.tokens, Token{kind = .Char, byte = c})
i += 1
}
}
return gp
}
match_tokens :: proc(tokens: []Token, classes: []Class_Data, ti: int, path: string, pi: int) -> bool {
if ti >= len(tokens) {
return pi == len(path)
}
tok := tokens[ti]
switch tok.kind {
case .Char:
if pi < len(path) && path[pi] == tok.byte {
return match_tokens(tokens, classes, ti + 1, path, pi + 1)
}
return false
case .Question:
if pi < len(path) && path[pi] != '/' {
return match_tokens(tokens, classes, ti + 1, path, pi + 1)
}
return false
case .Star:
max_end := pi
for max_end < len(path) && path[max_end] != '/' {
max_end += 1
}
for end := max_end; end >= pi; end -= 1 {
if match_tokens(tokens, classes, ti + 1, path, end) {
return true
}
}
return false
case .Globstar:
if ti + 1 >= len(tokens) {
return true
}
if match_tokens(tokens, classes, ti + 1, path, pi) {
return true
}
for end := pi + 1; end <= len(path); end += 1 {
if path[end - 1] == '/' {
if match_tokens(tokens, classes, ti + 1, path, end) {
return true
}
}
}
return false
case .Class:
if pi >= len(path) {
return false
}
cd := classes[tok.class_idx]
ch := path[pi]
in_range := false
for r in cd.ranges {
if ch >= r.lo && ch <= r.hi {
in_range = true
break
}
}
if in_range != cd.negated {
return match_tokens(tokens, classes, ti + 1, path, pi + 1)
}
return false
}
return false
}
glob_match_compiled :: proc(gp: ^GlobPattern, path: string) -> bool {
tokens := gp.tokens[:]
classes := gp.classes[:]
if gp.anchored {
return match_tokens(tokens, classes, 0, path, 0)
}
if match_tokens(tokens, classes, 0, path, 0) {
return true
}
for i := 1; i < len(path); i += 1 {
if path[i - 1] == '/' {
if match_tokens(tokens, classes, 0, path, i) {
return true
}
}
}
return false
}
glob_destroy :: proc(gp: ^GlobPattern) {
for &cd in gp.classes {
delete(cd.ranges)
}
delete(gp.classes)
delete(gp.tokens)
}

View File

@@ -1,111 +0,0 @@
package findr
import "core:testing"
glob_match :: proc(pattern: string, path: string, anchored: bool) -> bool {
gp := glob_compile(pattern, anchored)
result := glob_match_compiled(&gp, path)
glob_destroy(&gp)
return result
}
@(test)
test_glob_simple :: proc(t: ^testing.T) {
testing.expect(t, glob_match("foo", "foo", false))
testing.expect(t, glob_match("foo", "bar/foo", false))
testing.expect(t, !glob_match("foo", "foobar", false))
testing.expect(t, !glob_match("foo", "foo/bar", false))
}
@(test)
test_glob_anchored :: proc(t: ^testing.T) {
testing.expect(t, glob_match("foo", "foo", true))
testing.expect(t, !glob_match("foo", "bar/foo", true))
testing.expect(t, !glob_match("foo", "foobar", true))
}
@(test)
test_glob_star :: proc(t: ^testing.T) {
testing.expect(t, glob_match("*.log", "test.log", false))
testing.expect(t, glob_match("*.log", ".log", false))
testing.expect(t, !glob_match("*.log", "test.txt", false))
testing.expect(t, !glob_match("*.log", "dir/test", false))
}
@(test)
test_glob_question :: proc(t: ^testing.T) {
testing.expect(t, glob_match("?.log", "a.log", false))
testing.expect(t, !glob_match("?.log", "ab.log", false))
testing.expect(t, !glob_match("?.log", ".log", false))
}
@(test)
test_glob_char_class :: proc(t: ^testing.T) {
testing.expect(t, glob_match("[abc].log", "a.log", false))
testing.expect(t, glob_match("[abc].log", "b.log", false))
testing.expect(t, !glob_match("[abc].log", "d.log", false))
}
@(test)
test_glob_negated_class :: proc(t: ^testing.T) {
testing.expect(t, glob_match("[!abc].log", "d.log", false))
testing.expect(t, !glob_match("[!abc].log", "a.log", false))
}
@(test)
test_glob_dot_literal :: proc(t: ^testing.T) {
testing.expect(t, glob_match(".env", ".env", false))
testing.expect(t, glob_match(".env", "dir/.env", false))
testing.expect(t, !glob_match(".env", "env", false))
testing.expect(t, !glob_match(".env", "x.env", false))
}
@(test)
test_glob_globstar_prefix :: proc(t: ^testing.T) {
testing.expect(t, glob_match("**/foo", "foo", false))
testing.expect(t, glob_match("**/foo", "a/b/foo", false))
testing.expect(t, !glob_match("**/foo", "foobar", false))
testing.expect(t, !glob_match("**/foo", "a/foobar", false))
}
@(test)
test_glob_globstar_suffix :: proc(t: ^testing.T) {
testing.expect(t, glob_match("abc/**", "abc/x", false))
testing.expect(t, glob_match("abc/**", "abc/x/y", false))
testing.expect(t, !glob_match("abc/**", "abc", false))
testing.expect(t, !glob_match("abc/**", "abcd/x", false))
}
@(test)
test_glob_globstar_middle :: proc(t: ^testing.T) {
testing.expect(t, glob_match("foo/**/bar", "foo/bar", false))
testing.expect(t, glob_match("foo/**/bar", "foo/x/bar", false))
testing.expect(t, !glob_match("foo/**/bar", "foo/barx", false))
testing.expect(t, !glob_match("foo/**/bar", "foo/x/y/baz", false))
}
@(test)
test_glob_backslash_escape :: proc(t: ^testing.T) {
testing.expect(t, glob_match("\\!foo", "!foo", false))
testing.expect(t, !glob_match("\\!foo", "foo", false))
}
@(test)
test_glob_hash_literal :: proc(t: ^testing.T) {
testing.expect(t, glob_match("#foo", "#foo", false))
testing.expect(t, !glob_match("#foo", "foo", false))
}
@(test)
test_glob_hash_pattern :: proc(t: ^testing.T) {
testing.expect(t, glob_match("#*#", "#test#", false))
testing.expect(t, glob_match("#*#", "##", false))
testing.expect(t, !glob_match("#*#", "test", false))
testing.expect(t, !glob_match("#*#", "#test", false))
}
@(test)
test_glob_empty :: proc(t: ^testing.T) {
testing.expect(t, glob_match("", "", false))
testing.expect(t, !glob_match("", "foo", false))
}

View File

@@ -1,128 +0,0 @@
package findr
import "core:strings"
import "core:sync"
import "core:sys/linux"
import "core:thread"
RepoPool :: struct {
queue: [dynamic]string,
queue_mutex: sync.Mutex,
queue_sema: sync.Atomic_Sema,
results: ^[dynamic]string,
results_lock: sync.Mutex,
active: i64,
done: sync.One_Shot_Event,
threads: []^thread.Thread,
}
find_repos :: proc(roots: []string, results: ^[dynamic]string, thread_count: int) {
if len(roots) == 0 do return
pool := new(RepoPool)
pool.queue = make([dynamic]string)
pool.results = results
pool.active = i64(len(roots))
pool.threads = make([]^thread.Thread, thread_count)
for root in roots {
root_clone, _ := strings.clone(root)
append(&pool.queue, root_clone)
sync.atomic_sema_post(&pool.queue_sema)
}
for i in 0 ..< thread_count {
t := thread.create(repo_worker)
t.data = rawptr(pool)
t.init_context = context
thread.start(t)
pool.threads[i] = t
}
sync.one_shot_event_wait(&pool.done)
for _ in 0 ..< thread_count {
sync.atomic_sema_post(&pool.queue_sema)
}
for t in pool.threads {
thread.destroy(t)
}
delete(pool.threads)
for path in pool.queue {
delete(path)
}
delete(pool.queue)
free(pool)
}
repo_worker :: proc(t: ^thread.Thread) {
pool := cast(^RepoPool)t.data
for {
sync.atomic_sema_wait(&pool.queue_sema)
sync.mutex_lock(&pool.queue_mutex)
if len(pool.queue) == 0 {
sync.mutex_unlock(&pool.queue_mutex)
if sync.atomic_load_explicit(&pool.active, .Acquire) == 0 {
sync.one_shot_event_signal(&pool.done)
}
break
}
last := len(pool.queue) - 1
dir_path := pool.queue[last]
ordered_remove(&pool.queue, last)
sync.mutex_unlock(&pool.queue_mutex)
process_repo_dir(pool, dir_path)
delete(dir_path)
old := sync.atomic_sub_explicit(&pool.active, 1, .Release)
if old == 1 {
sync.one_shot_event_signal(&pool.done)
}
}
}
process_repo_dir :: proc(pool: ^RepoPool, dir_path: string) {
cpath := strings.clone_to_cstring(dir_path)
if cpath == nil do return
defer delete(cpath)
fd, open_err := linux.open(cpath, {.DIRECTORY, .CLOEXEC})
if open_err != .NONE do return
defer linux.close(fd)
if has_git_dir(fd) {
cloned, _ := strings.clone(dir_path)
sync.mutex_lock(&pool.results_lock)
append(pool.results, cloned)
sync.mutex_unlock(&pool.results_lock)
}
buf: [32 * 1024]u8
for {
n, errno := linux.getdents(fd, buf[:])
if n <= 0 || errno != .NONE do break
offs := 0
for d in linux.dirent_iterate_buf(buf[:n], &offs) {
name := linux.dirent_name(d)
if name == "." || name == ".." do continue
if name == ".git" do continue
if d.type == .DIR {
child_path := join_path(dir_path, name)
sync.atomic_add_explicit(&pool.active, 1, .Relaxed)
sync.mutex_lock(&pool.queue_mutex)
append(&pool.queue, child_path)
sync.mutex_unlock(&pool.queue_mutex)
sync.atomic_sema_post(&pool.queue_sema)
}
}
}
}

View File

@@ -61,24 +61,23 @@ create_git_repo :: proc(env: TestEnv, path: string) {
} }
assert_output :: proc( assert_output :: proc(
t: ^testing.T, t: ^testing.T,
env: TestEnv, env: TestEnv,
args: []string, args: []string,
opts: WalkOptions,
expected: []string, expected: []string,
) { ) {
results := collect_results(env, args, opts) results := collect_results(env, args)
defer { defer {
for r in results {delete(r)} for r in results { delete(r) }
delete(results) delete(results)
} }
sorted_expected := make([dynamic]string, 0, len(expected)) sorted_expected := make([dynamic]string, 0, len(expected))
for e in expected {append(&sorted_expected, e)} for e in expected { append(&sorted_expected, e) }
defer delete(sorted_expected) defer delete(sorted_expected)
sorted_actual := make([dynamic]string, 0, len(results)) sorted_actual := make([dynamic]string, 0, len(results))
for a in results {append(&sorted_actual, a)} for a in results { append(&sorted_actual, a) }
defer delete(sorted_actual) defer delete(sorted_actual)
sort.quick_sort(sorted_expected[:]) sort.quick_sort(sorted_expected[:])
@@ -94,7 +93,7 @@ assert_output :: proc(
return return
} }
for i in 0 ..< len(sorted_expected) { for i in 0..<len(sorted_expected) {
if sorted_expected[i] != sorted_actual[i] { if sorted_expected[i] != sorted_actual[i] {
testing.fail(t) testing.fail(t)
log.error(fmt.tprintf("Mismatch at index %d", i)) log.error(fmt.tprintf("Mismatch at index %d", i))
@@ -105,15 +104,10 @@ assert_output :: proc(
} }
} }
assert_output_empty :: proc( assert_output_empty :: proc(t: ^testing.T, env: TestEnv, args: []string) {
t: ^testing.T, results := collect_results(env, args)
env: TestEnv,
args: []string,
opts: WalkOptions,
) {
results := collect_results(env, args, opts)
defer { defer {
for r in results {delete(r)} for r in results { delete(r) }
delete(results) delete(results)
} }
if len(results) > 0 { if len(results) > 0 {
@@ -125,17 +119,18 @@ assert_output_empty :: proc(
} }
} }
collect_results :: proc(env: TestEnv, args: []string, opts: WalkOptions) -> [dynamic]string { collect_results :: proc(env: TestEnv, args: []string) -> [dynamic]string {
results := make([dynamic]string) results := make([dynamic]string)
full_args := make([dynamic]string, 0, len(args) + 1, context.temp_allocator) full_args := make([dynamic]string, 0, len(args) + 1, context.temp_allocator)
append(&full_args, env.temp_dir) append(&full_args, env.temp_dir)
for a in args {append(&full_args, a)} for a in args { append(&full_args, a) }
thread_count := os.get_processor_core_count() for dir in full_args {
walk(full_args[:], &results, opts, thread_count) walk(dir, &results)
}
for i in 0 ..< len(results) { for i in 0..<len(results) {
r := results[i] r := results[i]
if strings.has_prefix(r, env.temp_dir) { if strings.has_prefix(r, env.temp_dir) {
stripped := r[len(env.temp_dir):] stripped := r[len(env.temp_dir):]

View File

@@ -1,458 +1,260 @@
package findr package findr
import "core:bytes"
import "core:fmt" import "core:fmt"
import "core:os" import "core:os"
import "core:strings" import "core:strings"
import "core:sync" import "core:sync"
import "core:sync/chan"
import "core:sys/linux" import "core:sys/linux"
import "core:text/regex"
import "core:thread" import "core:thread"
OUTPUT_BUF_SIZE :: 64 * 1024 FINDR_PARALLEL :: #config(FINDR_PARALLEL, false)
FINDR_THREADS :: #config(FINDR_THREADS, 8)
WalkOptions :: struct { RawEntry :: struct {
pattern: string, // regex on basename; "" = match all name: string,
excludes: []string, // glob patterns to skip entirely type: linux.Dirent_Type,
} }
GIContext :: struct { walk :: proc(root: string, results: ^[dynamic]string) {
gi: ^Gitignore, // nil if this dir had no .gitignore when FINDR_PARALLEL {
base_rel: string, // relative path from repo root to this dir walk_parallel(root, results)
parent: ^GIContext, // parent context (nil if repo root) } else {
walk_dir_serial(root, results)
}
} }
WorkItem :: struct { read_dir_entries :: proc(dir_path: string, has_git: ^bool) -> [dynamic]RawEntry {
path: string, // absolute directory path entries := make([dynamic]RawEntry)
rel: string, // relative path from repo root ("" = root)
gi_ctx: ^GIContext, // gitignore chain (nil = outside any repo) cpath := strings.clone_to_cstring(dir_path)
in_repo: bool, // true if inside a git repo if cpath == nil do return entries
in_ignored: bool, // true if inside a gitignored directory
fd, err := linux.open(cpath, {.DIRECTORY, .CLOEXEC})
delete(cpath)
if err != .NONE do return entries
buf: [8192]u8
has_git^ = false
for {
n, errno := linux.getdents(fd, buf[:])
if n <= 0 || errno != .NONE do break
offs := 0
for d in linux.dirent_iterate_buf(buf[:n], &offs) {
name := linux.dirent_name(d)
if name == "." || name == ".." do continue
if name == ".git" && d.type == .DIR {
has_git^ = true
}
cloned := strings.clone(name)
append(&entries, RawEntry{name = cloned, type = d.type})
}
}
linux.close(fd)
return entries
} }
WalkerPool :: struct { free_entries :: proc(entries: ^[dynamic]RawEntry) {
queue: [dynamic]WorkItem, for &entry in entries {
queue_mutex: sync.Mutex, delete(entry.name)
queue_sema: sync.Atomic_Sema, }
result_chan: chan.Chan([]u8), delete(entries^)
active: i64,
done: sync.One_Shot_Event,
threads: []^thread.Thread,
opts: WalkOptions,
pattern_re: regex.Regular_Expression,
has_pattern: bool,
exclude_gi: ^Gitignore,
all_contexts: [dynamic]^GIContext,
contexts_lock: sync.Mutex,
} }
Collector_Data :: struct { walk_dir_serial :: proc(dir_path: string, results: ^[dynamic]string) {
ch: chan.Chan([]u8), has_git := false
results: ^[dynamic]string, entries := read_dir_entries(dir_path, &has_git)
defer free_entries(&entries)
if has_git {
gi := load_gitignore(dir_path)
defer if gi != nil {
destroy(gi)
free(gi)
}
for entry in entries {
if entry.name == ".git" do continue
is_dir := entry.type == .DIR
if gi != nil && is_ignored(gi, entry.name, is_dir) {
if !is_dir {
full_path := join_path(dir_path, entry.name)
append(results, full_path)
}
continue
}
if is_dir {
child_path := join_path(dir_path, entry.name)
walk_dir_serial(child_path, results)
delete(child_path)
}
}
} else {
for entry in entries {
if entry.type == .DIR {
child_path := join_path(dir_path, entry.name)
walk_dir_serial(child_path, results)
delete(child_path)
}
}
}
} }
collect_worker :: proc(t: ^thread.Thread) { load_gitignore :: proc(dir_path: string) -> ^Gitignore {
data := cast(^Collector_Data)t.data gi_path := join_path(dir_path, ".gitignore")
for { defer delete(gi_path)
batch := chan.recv(data.ch) or_break
defer delete(batch)
start := 0 data, err := os.read_entire_file_from_path(gi_path, context.allocator)
for { if err != nil do return nil
remaining: []u8
#no_bounds_check {remaining = batch[start:]}
idx := bytes.index_byte(remaining, '\n') gi := new(Gitignore)
if idx < 0 do break gi^ = parse(string(data))
delete(data)
i := start + idx return gi
if i > start {
segment: []u8
#no_bounds_check {segment = batch[start:i]}
s, _ := strings.clone(string(segment))
append(data.results, s)
}
start = i + 1
}
}
}
walk :: proc(roots: []string, results: ^[dynamic]string, opts: WalkOptions, thread_count: int) {
if len(roots) == 0 do return
ch, _ := chan.create(chan.Chan([]u8), max(2 * thread_count, 2), context.allocator)
defer chan.destroy(ch)
data := new(Collector_Data)
data.ch = ch
data.results = results
defer free(data)
collector := thread.create(collect_worker)
collector.data = rawptr(data)
collector.init_context = context
thread.start(collector)
pool := new(WalkerPool)
pool.queue = make([dynamic]WorkItem)
pool.result_chan = ch
pool.active = i64(len(roots))
pool.threads = make([]^thread.Thread, thread_count)
pool.all_contexts = make([dynamic]^GIContext)
pool.opts = opts
pool.exclude_gi = nil
pool.has_pattern = false
if len(opts.pattern) > 0 {
re, err := regex.create(opts.pattern, {regex.Flag.No_Capture})
if err == nil {
pool.pattern_re = re
pool.has_pattern = true
}
}
if len(opts.excludes) > 0 {
sb: strings.Builder
strings.builder_init(&sb)
for ex in opts.excludes {
fmt.sbprintf(&sb, "%s\n", ex)
}
content := strings.to_string(sb)
pool.exclude_gi = new(Gitignore)
pool.exclude_gi^ = parse(content)
strings.builder_destroy(&sb)
}
for root in roots {
root_clone, _ := strings.clone(root)
append(&pool.queue, WorkItem{path = root_clone})
sync.atomic_sema_post(&pool.queue_sema)
}
for i in 0 ..< thread_count {
t := thread.create(walk_worker)
t.data = rawptr(pool)
t.init_context = context
thread.start(t)
pool.threads[i] = t
}
sync.one_shot_event_wait(&pool.done)
for _ in 0 ..< thread_count {
sync.atomic_sema_post(&pool.queue_sema)
}
for t in pool.threads {
thread.destroy(t)
}
delete(pool.threads)
for item in pool.queue {
delete(item.path)
if len(item.rel) > 0 {delete(item.rel)}
}
delete(pool.queue)
for ctx in pool.all_contexts {
if ctx.gi != nil {
destroy(ctx.gi)
free(ctx.gi)
}
if len(ctx.base_rel) > 0 {
delete(ctx.base_rel)
}
free(ctx)
}
delete(pool.all_contexts)
if pool.has_pattern {
regex.destroy(pool.pattern_re)
}
if pool.exclude_gi != nil {
destroy(pool.exclude_gi)
free(pool.exclude_gi)
}
free(pool)
chan.close(ch)
thread.join(collector)
thread.destroy(collector)
}
flush_buf :: proc(ch: chan.Chan([]u8), local: ^[dynamic]u8) {
if len(local) == 0 do return
batch := local[:]
local^ = make([dynamic]u8, 0, OUTPUT_BUF_SIZE)
chan.send(ch, batch)
}
append_path :: proc(buf: ^[dynamic]u8, parent, name: string, trailing_slash: bool) {
need_sep := len(parent) > 0 && parent[len(parent) - 1] != '/'
size := len(parent) + len(name) + 1
if need_sep do size += 1
if trailing_slash do size += 1
old_len := len(buf)
reserve(buf, old_len + size)
resize(buf, old_len + size)
pos := old_len
pos += copy(buf[pos:], parent)
if need_sep {buf[pos] = '/'; pos += 1}
pos += copy(buf[pos:], name)
if trailing_slash {buf[pos] = '/'; pos += 1}
buf[pos] = '\n'
}
walk_worker :: proc(t: ^thread.Thread) {
pool := cast(^WalkerPool)t.data
local_buf := make([dynamic]u8, 0, OUTPUT_BUF_SIZE)
defer {
if len(local_buf) > 0 {
flush_buf(pool.result_chan, &local_buf)
}
delete(local_buf)
}
for {
sync.atomic_sema_wait(&pool.queue_sema)
sync.mutex_lock(&pool.queue_mutex)
if len(pool.queue) == 0 {
sync.mutex_unlock(&pool.queue_mutex)
if sync.atomic_load_explicit(&pool.active, .Acquire) == 0 {
sync.one_shot_event_signal(&pool.done)
}
break
}
last := len(pool.queue) - 1
item := pool.queue[last]
ordered_remove(&pool.queue, last)
sync.mutex_unlock(&pool.queue_mutex)
process_dir(pool, item, &local_buf)
delete(item.path)
if len(item.rel) > 0 {delete(item.rel)}
if len(local_buf) >= OUTPUT_BUF_SIZE {
flush_buf(pool.result_chan, &local_buf)
}
old := sync.atomic_sub_explicit(&pool.active, 1, .Release)
if old == 1 {
sync.one_shot_event_signal(&pool.done)
}
}
}
process_dir :: proc(pool: ^WalkerPool, item: WorkItem, local_buf: ^[dynamic]u8) {
dir_path := item.path
cpath := strings.clone_to_cstring(dir_path)
if cpath == nil do return
defer delete(cpath)
fd, open_err := linux.open(cpath, {.DIRECTORY, .CLOEXEC})
if open_err != .NONE do return
defer linux.close(fd)
has_git := false
if !item.in_ignored {
has_git = has_git_dir(fd)
}
gi_ctx := item.gi_ctx
rel := item.rel
if has_git {
gi_ctx = nil
rel = ""
}
child_in_repo := has_git || item.in_repo
gi: ^Gitignore = nil
if !item.in_ignored {
gi = load_ignore_patterns(dir_path, child_in_repo)
}
if gi != nil {
new_ctx := new(GIContext)
new_ctx.gi = gi
if len(rel) > 0 {
new_ctx.base_rel, _ = strings.clone(rel)
}
new_ctx.parent = gi_ctx
sync.mutex_lock(&pool.contexts_lock)
append(&pool.all_contexts, new_ctx)
sync.mutex_unlock(&pool.contexts_lock)
gi_ctx = new_ctx
}
buf: [32 * 1024]u8
rel_buf: [4096]u8
for {
n, errno := linux.getdents(fd, buf[:])
if n <= 0 || errno != .NONE do break
offs := 0
for d in linux.dirent_iterate_buf(buf[:n], &offs) {
name := linux.dirent_name(d)
if name == "." || name == ".." do continue
if name == ".git" do continue
is_dir := d.type == .DIR
is_nondir := d.type != .DIR
if pool.exclude_gi != nil && is_ignored(pool.exclude_gi, name, is_dir) {
continue
}
entry_rel := build_rel(rel_buf[:], rel, name)
ignored := false
if item.in_ignored {
ignored = true
} else if gi_ctx != nil {
ignored = check_chain(gi_ctx, entry_rel, is_dir)
}
if is_dir {
if ignored && matches_pattern(pool, name) {
append_path(local_buf, dir_path, name, true)
}
child_rel, _ := strings.clone(entry_rel)
child_path := join_path(dir_path, name)
push_work(
pool,
WorkItem {
path = child_path,
rel = child_rel,
gi_ctx = gi_ctx,
in_repo = child_in_repo,
in_ignored = ignored,
},
)
} else if is_nondir {
if ignored && matches_pattern(pool, name) {
append_path(local_buf, dir_path, name, false)
}
}
}
}
}
check_chain :: proc(ctx: ^GIContext, entry_rel: string, is_dir: bool) -> bool {
c := ctx
for c != nil {
if c.gi != nil {
rel := relative_to(entry_rel, c.base_rel)
match := check_match(c.gi, rel, is_dir)
if match != .None {
return match == .Ignored
}
}
c = c.parent
}
return false
}
relative_to :: proc(entry_rel, base_rel: string) -> string {
if len(base_rel) == 0 do return entry_rel
prefix_len := len(base_rel)
if len(entry_rel) > prefix_len &&
entry_rel[prefix_len] == '/' &&
strings.has_prefix(entry_rel, base_rel) {
return entry_rel[prefix_len + 1:]
}
return entry_rel
}
build_rel :: proc(buf: []u8, rel, name: string) -> string {
if len(rel) == 0 do return name
pos := copy(buf, rel)
if pos < len(buf) {
buf[pos] = '/'
pos += 1
pos += copy(buf[pos:], name)
}
return string(buf[:pos])
}
matches_pattern :: proc(pool: ^WalkerPool, name: string) -> bool {
if !pool.has_pattern do return true
cap, ok := regex.match(pool.pattern_re, name)
regex.destroy(cap)
return ok
}
push_work :: proc(pool: ^WalkerPool, item: WorkItem) {
sync.atomic_add_explicit(&pool.active, 1, .Relaxed)
sync.mutex_lock(&pool.queue_mutex)
append(&pool.queue, item)
sync.mutex_unlock(&pool.queue_mutex)
sync.atomic_sema_post(&pool.queue_sema)
}
has_git_dir :: proc(fd: linux.Fd) -> bool {
git_fd, err := linux.openat(fd, ".git", {.DIRECTORY, .CLOEXEC})
if err == .NONE {
linux.close(git_fd)
return true
}
return false
}
load_ignore_patterns :: proc(dir_path: string, in_repo: bool) -> ^Gitignore {
has_patterns := false
sb: strings.Builder
strings.builder_init(&sb)
defer strings.builder_destroy(&sb)
if in_repo {
gi_path := join_path(dir_path, ".gitignore")
data, err := os.read_entire_file_from_path(gi_path, context.allocator)
delete(gi_path)
if err == .NONE {
fmt.sbprintf(&sb, "%s", string(data))
delete(data)
has_patterns = true
}
}
ig_path := join_path(dir_path, ".ignore")
idata, ierr := os.read_entire_file_from_path(ig_path, context.allocator)
delete(ig_path)
if ierr == .NONE {
fmt.sbprintf(&sb, "%s", string(idata))
delete(idata)
has_patterns = true
}
if !has_patterns do return nil
content := strings.to_string(sb)
gi := new(Gitignore)
gi^ = parse(content)
return gi
} }
join_path :: proc(parent, child: string) -> string { join_path :: proc(parent, child: string) -> string {
need_sep := len(parent) == 0 || parent[len(parent) - 1] != '/' b: strings.Builder
total := len(parent) + len(child) strings.builder_init(&b)
if need_sep do total += 1 defer strings.builder_destroy(&b)
buf := make([]u8, total, context.allocator)
pos := copy(buf, parent) fmt.sbprintf(&b, "%s", parent)
if need_sep { if len(parent) == 0 || parent[len(parent) - 1] != '/' {
buf[pos] = '/' fmt.sbprintf(&b, "/")
pos += 1 }
} fmt.sbprintf(&b, "%s", child)
copy(buf[pos:], child)
return string(buf) s := strings.to_string(b)
result, _ := strings.clone(s)
return result
} }
when FINDR_PARALLEL {
WalkerPool :: struct {
queue: [dynamic]string,
queue_mutex: sync.Mutex,
queue_sema: sync.Atomic_Sema,
results: ^[dynamic]string,
results_mutex: sync.Mutex,
active: i64,
done: sync.One_Shot_Event,
threads: [dynamic]^thread.Thread,
}
walk_parallel :: proc(root: string, results: ^[dynamic]string) {
pool := new(WalkerPool)
pool.queue = make([dynamic]string)
pool.results = results
pool.active = 1
pool.threads = make([dynamic]^thread.Thread)
root_clone, _ := strings.clone(root)
append(&pool.queue, root_clone)
sync.atomic_sema_post(&pool.queue_sema)
num_threads := FINDR_THREADS
for i in 0..<num_threads {
t := thread.create(walk_worker)
t.data = rawptr(pool)
t.init_context = context
thread.start(t)
append(&pool.threads, t)
}
sync.one_shot_event_wait(&pool.done)
for _ in 0..<num_threads {
sync.atomic_sema_post(&pool.queue_sema)
}
for t in pool.threads {
thread.destroy(t)
}
delete(pool.threads)
for path in pool.queue {
delete(path)
}
delete(pool.queue)
free(pool)
}
push_work :: proc(pool: ^WalkerPool, path: string) {
sync.atomic_add_explicit(&pool.active, 1, .Relaxed)
sync.mutex_lock(&pool.queue_mutex)
append(&pool.queue, path)
sync.mutex_unlock(&pool.queue_mutex)
sync.atomic_sema_post(&pool.queue_sema)
}
process_dir_parallel :: proc(pool: ^WalkerPool, dir_path: string) {
has_git := false
entries := read_dir_entries(dir_path, &has_git)
defer free_entries(&entries)
if has_git {
gi := load_gitignore(dir_path)
defer if gi != nil {
destroy(gi)
free(gi)
}
for entry in entries {
if entry.name == ".git" do continue
is_dir := entry.type == .DIR
if gi != nil && is_ignored(gi, entry.name, is_dir) {
if !is_dir {
full_path := join_path(dir_path, entry.name)
sync.mutex_lock(&pool.results_mutex)
append(pool.results, full_path)
sync.mutex_unlock(&pool.results_mutex)
}
continue
}
if is_dir {
child_path := join_path(dir_path, entry.name)
push_work(pool, child_path)
}
}
} else {
for entry in entries {
if entry.type == .DIR {
child_path := join_path(dir_path, entry.name)
push_work(pool, child_path)
}
}
}
}
walk_worker :: proc(t: ^thread.Thread) {
pool := cast(^WalkerPool) t.data
for {
sync.atomic_sema_wait(&pool.queue_sema)
sync.mutex_lock(&pool.queue_mutex)
if len(pool.queue) == 0 {
sync.mutex_unlock(&pool.queue_mutex)
if sync.atomic_load_explicit(&pool.active, .Acquire) == 0 {
sync.one_shot_event_signal(&pool.done)
}
break
}
last := len(pool.queue) - 1
dir_path := pool.queue[last]
ordered_remove(&pool.queue, last)
sync.mutex_unlock(&pool.queue_mutex)
process_dir_parallel(pool, dir_path)
delete(dir_path)
old := sync.atomic_sub_explicit(&pool.active, 1, .Release)
if old == 1 {
sync.one_shot_event_signal(&pool.done)
}
}
}
}

View File

@@ -95,6 +95,7 @@
devShells.default = pkgs.mkShell { devShells.default = pkgs.mkShell {
buildInputs = with pkgs; [ buildInputs = with pkgs; [
fd
nushell nushell
libsodium libsodium
@@ -106,7 +107,6 @@
zip zip
# Helper tools # Helper tools
delta
hyperfine hyperfine
# IDE # IDE

View File

@@ -1,31 +1,14 @@
package main package main
import "core:bufio"
import "core:fmt" import "core:fmt"
import "core:mem"
import "core:os" import "core:os"
main :: proc() { main :: proc() {
when ODIN_DEBUG {
heap_track: mem.Tracking_Allocator
mem.tracking_allocator_init(&heap_track, context.allocator)
defer mem.tracking_allocator_destroy(&heap_track)
defer if len(heap_track.allocation_map) > 0 {
for _, leak in heap_track.allocation_map {
fmt.eprintf("LEAK: %v leaked %m\n", leak.location, leak.size)
}
}
context.allocator = mem.tracking_allocator(&heap_track)
temp_track: mem.Tracking_Allocator
mem.tracking_allocator_init(&temp_track, context.temp_allocator)
defer mem.tracking_allocator_destroy(&temp_track)
context.temp_allocator = mem.tracking_allocator(&temp_track)
}
defer free_all(context.temp_allocator) defer free_all(context.temp_allocator)
cmd, ok := parse_args(os.args, os.to_writer(os.stdout), os.to_writer(os.stderr)) cmd, ok := parse_args(os.args, os.to_writer(os.stdout), os.to_writer(os.stderr))
defer delete_command(&cmd) // delete flushes automatically defer bufio.writer_flush(cmd.out_buf)
if !ok { if !ok {
return return
} }
@@ -35,6 +18,8 @@ main :: proc() {
cmd_init(&cmd) cmd_init(&cmd)
case "version": case "version":
cmd_version(&cmd) cmd_version(&cmd)
case "deps":
cmd_deps(&cmd)
case "list": case "list":
cmd_list(&cmd) cmd_list(&cmd)
case "backup", "add": case "backup", "add":

127
scan.odin
View File

@@ -1,21 +1,137 @@
package main package main
import "core:fmt"
import "core:os" import "core:os"
import "core:strings"
import "core:sync"
import "core:terminal"
import "findr" fd_counter: sync.Atomic_Mutex
fd_seq: int
// Caller is responsible for freeing paths // Caller is responsible for freeing paths
scan_path :: proc(search_path: string, cfg: Config) -> (paths: [dynamic]string, ok: bool) { scan_path :: proc(search_path: string, cfg: Config) -> (paths: [dynamic]string, ok: bool) {
opts := findr.WalkOptions { if terminal.is_terminal(os.stdout) {
pattern = cfg.ScanConfig.Matcher, fmt.printf("Searching for all files in \"%s\"...\n", search_path)
excludes = cfg.ScanConfig.Exclude[:],
} }
findr.walk({search_path}, &paths, opts, os.get_processor_core_count()) all_files, all_ok := run_fd(build_fd_args(search_path, cfg, true))
if !all_ok {
return
}
if terminal.is_terminal(os.stdout) {
fmt.printf("Search for unignored fies in \"%s\"...\n", search_path)
}
unignored_files, unignored_ok := run_fd(build_fd_args(search_path, cfg, false))
if !unignored_ok {
return
}
unignored_set := make(map[string]bool, len(unignored_files), context.temp_allocator)
for file in unignored_files {
unignored_set[file] = true
}
for file in all_files {
if !(file in unignored_set) {
append(&paths, file)
}
}
ok = true ok = true
return return
} }
@(private = "file")
build_fd_args :: proc(search_path: string, cfg: Config, include_ignored: bool) -> []string {
args_len := 3 + 2 * len(cfg.ScanConfig.Exclude) + 2
args := make([dynamic]string, 0, args_len, context.temp_allocator)
append(&args, "fd")
append(&args, "-a")
append(&args, cfg.ScanConfig.Matcher)
for exclude in cfg.ScanConfig.Exclude {
append(&args, "-E")
append(&args, exclude)
}
if include_ignored {
append(&args, "-HI")
} else {
append(&args, "-H")
}
append(&args, search_path)
return args[:]
}
run_fd :: proc(args: []string) -> (lines: []string, ok: bool) {
tmp_path := next_fd_tmp_path()
tmp_file, tmp_err := os.open(tmp_path, os.O_CREATE | os.O_WRONLY | os.O_TRUNC)
if tmp_err != nil {
// TODO: Log a message here
return
}
desc := os.Process_Desc {
command = args,
stdout = tmp_file,
stderr = nil,
}
p, start_err := os.process_start(desc)
os.close(tmp_file)
if start_err != nil {
os.remove(tmp_path)
return
}
state, wait_err := os.process_wait(p)
if wait_err != nil || state.exit_code != 0 {
os.remove(tmp_path)
return
}
data, read_err := os.read_entire_file_from_path(tmp_path, context.temp_allocator)
os.remove(tmp_path)
if read_err != nil {
return
}
output := string(data)
output = strings.trim_space(output)
if len(output) == 0 {
ok = true
return
}
raw_lines := strings.split(output, "\n", context.temp_allocator)
result := make([dynamic]string, 0, len(raw_lines), context.temp_allocator)
for line in raw_lines {
trimmed := strings.trim_space(line)
if len(trimmed) > 0 {
append(&result, trimmed)
}
}
return result[:], true
}
@(private = "file")
next_fd_tmp_path :: proc() -> string {
sync.atomic_mutex_lock(&fd_counter)
n := fd_seq
fd_seq += 1
sync.atomic_mutex_unlock(&fd_counter)
return fmt.tprintf("/tmp/envr-fd-%d-%d", os.get_pid(), n)
}
cant_scan :: proc(feats: AvailableFeatures) -> bool {
return Feature.Fd not_in feats
}
find_unbacked :: proc(local_files: []string, db_files: []EnvFile) -> []string { find_unbacked :: proc(local_files: []string, db_files: []EnvFile) -> []string {
// Lives until the end of the function
backed_set := make(map[string]bool, len(db_files), context.temp_allocator) backed_set := make(map[string]bool, len(db_files), context.temp_allocator)
for file in db_files { for file in db_files {
backed_set[file.Path] = true backed_set[file.Path] = true
@@ -29,3 +145,4 @@ find_unbacked :: proc(local_files: []string, db_files: []EnvFile) -> []string {
} }
return unbacked[:] return unbacked[:]
} }

View File

@@ -3,10 +3,14 @@ package main
import "core:fmt" import "core:fmt"
import "core:os" import "core:os"
import "core:path/filepath" import "core:path/filepath"
import "core:strings"
import "core:testing" import "core:testing"
@(test) @(test)
test_scan_path_finds_gitignored_env_files :: proc(t: ^testing.T) { test_scan_path_finds_gitignored_env_files :: proc(t: ^testing.T) {
feats := check_features()
testing.expect(t, cant_scan(feats) == false)
base := fmt.tprintf("/tmp/envr-scan-test-%d", os.get_pid()) base := fmt.tprintf("/tmp/envr-scan-test-%d", os.get_pid())
os.mkdir_all(base) os.mkdir_all(base)
defer os.remove_all(base) defer os.remove_all(base)
@@ -38,12 +42,7 @@ test_scan_path_finds_gitignored_env_files :: proc(t: ^testing.T) {
} }
results, ok := scan_path(base, cfg) results, ok := scan_path(base, cfg)
defer { defer delete(results)
for path in results {
delete(path)
}
delete(results)
}
testing.expect(t, ok, "scan_path should succeed") testing.expect(t, ok, "scan_path should succeed")
found_env := false found_env := false
@@ -70,6 +69,9 @@ test_scan_path_finds_gitignored_env_files :: proc(t: ^testing.T) {
@(test) @(test)
test_scan_path_empty_dir :: proc(t: ^testing.T) { test_scan_path_empty_dir :: proc(t: ^testing.T) {
feats := check_features()
testing.expect(t, cant_scan(feats) == false)
base := fmt.tprintf("/tmp/envr-scan-empty-%d", os.get_pid()) base := fmt.tprintf("/tmp/envr-scan-empty-%d", os.get_pid())
os.mkdir_all(base) os.mkdir_all(base)
defer os.remove_all(base) defer os.remove_all(base)
@@ -83,3 +85,12 @@ test_scan_path_empty_dir :: proc(t: ^testing.T) {
testing.expect(t, ok, "scan_path should succeed") testing.expect(t, ok, "scan_path should succeed")
testing.expect(t, len(results) == 0, fmt.tprintf("expected 0 results, got %d", len(results))) testing.expect(t, len(results) == 0, fmt.tprintf("expected 0 results, got %d", len(results)))
} }
@(test)
test_scan_meets_expectations :: proc(t: ^testing.T) {
testing.expect(t, cant_scan({}), "no features should mean can't scan")
testing.expect(t, cant_scan({.Git}), "Git alone should mean can't scan")
testing.expect(t, !cant_scan({.Fd}), "having Fd should mean can scan")
testing.expect(t, !cant_scan({.Fd, .Git}), "both Fd and Git should mean can scan")
}

View File

@@ -12,6 +12,24 @@ Ed25519Keypair :: struct {
Private: [32]u8, Private: [32]u8,
} }
read_wire_string :: proc(data: []u8, offset: ^int) -> (s: string, ok: bool) {
if offset^ + 4 > len(data) {
return
}
length := u32(data[offset^]) << 24 | u32(data[offset^ + 1]) << 16 |
u32(data[offset^ + 2]) << 8 | u32(data[offset^ + 3])
offset^ += 4
if offset^ + int(length) > len(data) {
return
}
s = string(data[offset^ : offset^ + int(length)])
offset^ += int(length)
ok = true
return
}
parse_ssh_public_key :: proc(pub_path: string) -> (pub: [32]u8, ok: bool) { parse_ssh_public_key :: proc(pub_path: string) -> (pub: [32]u8, ok: bool) {
data, err := os.read_entire_file_from_path(pub_path, context.temp_allocator) data, err := os.read_entire_file_from_path(pub_path, context.temp_allocator)
if err != nil { if err != nil {
@@ -235,21 +253,3 @@ is_encrypted_key :: proc(priv_path: string) -> bool {
return ciphername != "none" return ciphername != "none"
} }
read_wire_string :: proc(data: []u8, offset: ^int) -> (s: string, ok: bool) {
if offset^ + 4 > len(data) {
return
}
length := u32(data[offset^]) << 24 | u32(data[offset^ + 1]) << 16 |
u32(data[offset^ + 2]) << 8 | u32(data[offset^ + 3])
offset^ += 4
if offset^ + int(length) > len(data) {
return
}
s = string(data[offset^ : offset^ + int(length)])
offset^ += int(length)
ok = true
return
}