1 Commits

Author SHA1 Message Date
Spencer Brower
7df83b064e chore(odin): release 0.3.0 2026-06-16 11:39:29 -04:00
37 changed files with 1207 additions and 2522 deletions

View File

@@ -2,6 +2,8 @@ on:
push: push:
branches: branches:
- main - main
- dev
- odin
permissions: permissions:
contents: write contents: write

4
.gitignore vendored
View File

@@ -7,12 +7,8 @@ list.json
man man
# build artifacts # build artifacts
*.spall
builds builds
envr envr
envr-go envr-go
findr/findr
findr/findr-prof
findr/bench-*.md
result result
version.odin version.odin

View File

@@ -2,10 +2,6 @@
## [0.3.0](https://github.com/sbrow/envr/compare/v0.2.1...v0.3.0) (2026-06-16) ## [0.3.0](https://github.com/sbrow/envr/compare/v0.2.1...v0.3.0) (2026-06-16)
Version 0.3.0 represents a significant departure (and improvement) for envr.
The entire codebase was rewritten in [Odin](https://odin-lang.org/) (from Go).
This reduced the binary size from over 17MB to under 600k, improved performance,
and significantly reduced the number of project dependencies from 69 to just 2.
### ⚠ BREAKING CHANGES ### ⚠ BREAKING CHANGES

View File

@@ -12,13 +12,14 @@ the tool [of your choosing](#backup-options).
## Features ## Features
- **Encrypted Storage**: All `.env` files are encrypted using your ssh key and - 🔐 **Encrypted Storage**: All `.env` files are encrypted using your ssh key and
[libsodium](https://github.com/jedisct1/libsodium). [libsodium](https://github.com/jedisct1/libsodium) encryption.
- **Automatic Sync**: Update the database with one command, which can easily - 🔄 **Automatic Sync**: Update the database with one command, which can easily
be run on a cron. be run on a cron.
- **Smart Scanning**: Automatically discover and import `.env` files in your - 🔍 **Smart Scanning**: Automatically discover and import `.env` files in your
home directory. home directory.
- **Rename Detection**: Automatically find and updates renamed/moved - **Interactive CLI**: User-friendly prompts for file selection and management.
- 🗂️ **Rename Detection**: Automatically finds and updates renamed/moved
repositories. repositories.
## TODOS ## TODOS
@@ -27,10 +28,14 @@ repositories.
- [x] Allow configuration of ssh key. - [x] Allow configuration of ssh key.
- [x] Allow multiple ssh keys. - [x] Allow multiple ssh keys.
## Installation ## Prerequisites
You will need an SSH key pair for encryption and decryption. You can generate one - An SSH key pair (for encryption/decryption)
with `ssh-keygen -t ed25519`. It will be saved to `~/.ssh/id_ed25519`. - The following binaries:
- [fd](https://github.com/sharkdp/fd)
- [git](https://git-scm.com)
## Installation
### With Odin ### With Odin
@@ -91,12 +96,7 @@ The configuration file is created during initialization:
], ],
"scan": { "scan": {
"matcher": "\\.env", "matcher": "\\.env",
"exclude": [ "exclude": "*.envrc",
"*\\.envrc",
"\\.local/",
"node_modules",
"vendor"
],
"include": "~" "include": "~"
} }
} }

View File

@@ -35,7 +35,13 @@ Stdout will be captured by redirecting `os.stdout` to a pipe.
## Hard to test (interactive / external deps) ## Hard to test (interactive / external deps)
### `cmd_deps` (cmd_deps.odin)
- Needs `git` and/or `fd` in PATH
- Test TTY and non-TTY paths
- Skip if dependencies not available (with `#assert` like TODO 28 suggests)
### `cmd_scan` (cmd_scan.odin) ### `cmd_scan` (cmd_scan.odin)
- Needs `fd` installed
- Test with fixture git repo containing `.env` files - Test with fixture git repo containing `.env` files
- Test `find_unbacked` integration (already partially tested in `cmd_check_test.odin`) - Test `find_unbacked` integration (already partially tested in `cmd_check_test.odin`)
- Non-TTY JSON output path - Non-TTY JSON output path
@@ -60,4 +66,5 @@ Stdout will be captured by redirecting `os.stdout` to a pipe.
- DB integration tests should use in-memory SQLite (`:memory:`) where possible. - DB integration tests should use in-memory SQLite (`:memory:`) where possible.
- Temp dir fixtures should follow the pattern in `scan_test.odin`. - Temp dir fixtures should follow the pattern in `scan_test.odin`.
- External dependency tests (`fd`, `git`) should use `#assert` to ensure the dependency is present rather than silently skipping (TODO 28).
- Tests that manipulate the `HOME` env var must use a mutex to prevent races with parallel test execution. - Tests that manipulate the `HOME` env var must use a mutex to prevent races with parallel test execution.

View File

@@ -1,16 +1,13 @@
# TODOs # TODOs
1. Consider giving db its own allocator 1. Consider giving db its own allocator
27. Commands are still leaking. 2. **db.odin:324-327** — Map iteration (`remote_set`) is non-deterministic. Same file can produce different JSON on each backup, causing spurious DB diffs. Sort remotes before storing.
28. **db.odin** — Inconsistencies in how struct vs sqlite are named. 3. **db.odin:135, 250** — String interpolation into SQL (`VACUUM INTO '%s'`, `ATTACH DATABASE '%s'`). Currently safe because input is controlled, but fragile.
2. Generate md and man pages again. 4. **features.odin:30-41**`find_binary` uses `strings.join` instead of `filepath.join`, uses `os.stat` instead of checking executability, hardcodes `:` as PATH separator (wrong on Windows).
3. **db.odin:324-327** — Map iteration (`remote_set`) is non-deterministic. Same file can produce different JSON on each backup, causing spurious DB diffs. Sort remotes before storing.
4. Make sure official path separators are used when appropriate, rather than '/'.
5. **cmd_restore.odin:20-30 & cmd_remove.odin:19-29** — Identical path-resolution block copy-pasted. `is_abs` guard is redundant since `filepath.abs` is a no-op on absolute paths. Extract a helper. 5. **cmd_restore.odin:20-30 & cmd_remove.odin:19-29** — Identical path-resolution block copy-pasted. `is_abs` guard is redundant since `filepath.abs` is a no-op on absolute paths. Extract a helper.
@@ -22,7 +19,7 @@
11. **db.odin:352-353**`hex.encode` error ignored. `string(hex_bytes)` aliases the byte slice. 11. **db.odin:352-353**`hex.encode` error ignored. `string(hex_bytes)` aliases the byte slice.
12. **cmd_sync.odin:80, cmd_list.odin:33**`make([]string, 2)` for table rows never freed. Leaks per row. Defer to memory pass. 12. **cmd_sync.odin:80, cmd_list.odin:33, cmd_deps.odin:9**`make([]string, 2)` for table rows never freed. Leaks per row. Defer to memory pass.
13. **cmd_list.odin** — Non-TTY branch builds `ListEntry` structs and marshals JSON separately. Now that `render_json_rows` (issue 1) accepts an `io.Writer` and uses `json.marshal`, unify both branches to use it. Note: will change JSON keys from `"directory"/"path"` to `"Directory"/"Path"`. 13. **cmd_list.odin** — Non-TTY branch builds `ListEntry` structs and marshals JSON separately. Now that `render_json_rows` (issue 1) accepts an `io.Writer` and uses `json.marshal`, unify both branches to use it. Note: will change JSON keys from `"directory"/"path"` to `"Directory"/"Path"`.
@@ -30,10 +27,14 @@
15. Add a text filter to the multi_select. 15. Add a text filter to the multi_select.
16. Create backup / fallback fd.
17. Add tests for untested commands. 17. Add tests for untested commands.
18. 2 scan tests silently skip when fd isn't installed, tests pass without actually testing anything. These should use #assert to be sure that fd is in path. 18. 2 scan tests silently skip when fd isn't installed, tests pass without actually testing anything. These should use #assert to be sure that fd is in path.
19. Try to do all encryption / decryption in memory - only read / write encrypted data to disk.
20. add --format -f flag to commands that draw tables. 20. add --format -f flag to commands that draw tables.
21. Replace `testing.expect` calls with `testing.expect_value` calls where appropriate. 21. Replace `testing.expect` calls with `testing.expect_value` calls where appropriate.
@@ -42,12 +43,6 @@
23. procedures should be ordered by use, main at the top, then in the order they are called from main. 23. procedures should be ordered by use, main at the top, then in the order they are called from main.
24. Shell completion
25. Bring back windows support / cross-compilation.
26. Test all cmds / terminal branches.
## Double-check AI output ## Double-check AI output
- [ ] cli.odin - [ ] cli.odin
@@ -55,6 +50,7 @@
- [x] cmd_backup.odin - [x] cmd_backup.odin
- [x] cmd_check.odin - [x] cmd_check.odin
- [ ] cmd_check_test.odin - [ ] cmd_check_test.odin
- [x] cmd_deps.odin
- [ ] cmd_edit_config.odin - [ ] cmd_edit_config.odin
- [x] cmd_init.odin - [x] cmd_init.odin
- [x] cmd_list.odin - [x] cmd_list.odin
@@ -73,6 +69,8 @@
- [ ] db.odin - [ ] db.odin
- [ ] db_integration_test.odin - [ ] db_integration_test.odin
- [ ] db_test.odin - [ ] db_test.odin
- [x] features.odin
- [x] features_test.odin
- [x] main.odin - [x] main.odin
- [x] prompt.odin - [x] prompt.odin
- [ ] scan.odin - [ ] scan.odin
@@ -83,11 +81,3 @@
- [ ] ssh_test.odin - [ ] ssh_test.odin
- [ ] table.odin - [ ] table.odin
- [ ] table_test.odin - [ ] table_test.odin
- [ ] findr/findr_test.odin
- [ ] findr/gitignore.odin
- [ ] findr/gitignore_test.odin
- [ ] findr/glob.odin
- [ ] findr/glob_test.odin
- [ ] findr/repos.odin
- [ ] findr/test_env.odin
- [ ] findr/walker.odin

92
WINDOWS.md Normal file
View File

@@ -0,0 +1,92 @@
# Windows Compatibility Guide
This document outlines Windows compatibility issues and solutions for the envr project.
## Critical Issues
### 1. Path Handling Bug (MUST FIX)
**File:** `app/env_file.go:209`
**Issue:** Uses `path.Join` instead of `filepath.Join`, which won't work correctly on Windows due to different path separators.
**Current code:**
```go
f.Path = path.Join(newDir, path.Base(f.Path))
```
**Fixed code:**
```go
f.Path = filepath.Join(newDir, filepath.Base(f.Path))
```
## External Dependencies
The application relies on external tools that need to be installed separately on Windows:
### Required Tools
1. **fd** - Fast file finder
- Install via: `winget install sharkdp.fd` or `choco install fd`
- Alternative: `scoop install fd`
2. **git** - Version control system
- Install via: `winget install Git.Git` or download from git-scm.com
- Usually already available on most development machines
## Minor Compatibility Notes
### File Permissions
- Unix file permissions (`0755`, `0644`) are used throughout the codebase
- These are safely ignored on Windows - no changes needed
### Editor Configuration
**File:** `cmd/edit_config.go:20-24`
**Issue:** Relies on `$EDITOR` environment variable which is less common on Windows.
**Current behavior:** Fails if `$EDITOR` is not set
**Recommended improvement:** Add fallback detection for Windows editors:
```go
editor := os.Getenv("EDITOR")
if editor == "" {
if runtime.GOOS == "windows" {
editor = "notepad.exe" // or "code.exe" for VS Code
} else {
fmt.Println("Error: $EDITOR environment variable is not set")
return
}
}
```
## Installation Instructions for Windows
1. Install required dependencies:
```powershell
winget install sharkdp.fd
winget install Git.Git
```
2. Fix the path handling bug in `app/env_file.go:209`
3. Build and run as normal:
```powershell
go build
.\envr.exe init
```
## Testing on Windows
After applying the critical path fix, the core functionality should work correctly on Windows. The application has been designed with cross-platform compatibility in mind, using:
- `filepath` package for path operations (mostly)
- `os.UserHomeDir()` for home directory detection
- Standard Go file operations
## Summary
- **1 critical bug** must be fixed for Windows compatibility
- **2 external tools** need to be installed
- **1 minor enhancement** recommended for better Windows UX
- Overall architecture is Windows-compatible

View File

@@ -43,6 +43,13 @@ key somewhere, otherwise your data could be lost forever.`,
{"list", "envr list", "View your tracked files", "", {}}, {"list", "envr list", "View your tracked files", "", {}},
{"remove", "envr remove <path>", "Remove a .env file from your database", "", {}}, {"remove", "envr remove <path>", "Remove a .env file from your database", "", {}},
{"check", "envr check [path]", "Check if files are backed up", "", {}}, {"check", "envr check [path]", "Check if files are backed up", "", {}},
{
"deps",
"envr deps",
"Check for missing binaries",
"envr relies on external binaries for certain functionality.\n\nThe check command reports on which binaries are available and which are not.",
{},
},
{"version", "envr version", "Show envr's version", "", {}}, {"version", "envr version", "Show envr's version", "", {}},
{"edit-config", "envr edit-config", "Edit your config with your default editor", "", {}}, {"edit-config", "envr edit-config", "Edit your config with your default editor", "", {}},
{ {
@@ -54,12 +61,20 @@ key somewhere, otherwise your data could be lost forever.`,
}, },
} }
delete_command :: proc(cmd: ^Command) {
delete(cmd.args)
delete(cmd.flags)
delete(cmd.bool_set)
bufio.writer_destroy(cmd.out_buf)
free(cmd.out_buf)
}
// Caller is responsible for calling delete_command(cmd). // Caller is responsible for calling delete_command(cmd).
// FIXME: Works in kinda a wonky and awkward way. // FIXME: Works in kinda a wonky and awkward way.
parse_args :: proc(args: []string, out: io.Stream, err: io.Stream) -> (cmd: Command, ok: bool) { parse_args :: proc(args: []string, out: io.Stream, err: io.Stream) -> (cmd: Command, ok: bool) {
{ {
cmd.out_buf = new(bufio.Writer) cmd.out_buf = new(bufio.Writer)
bufio.writer_init(cmd.out_buf, out, allocator = context.allocator) bufio.writer_init(cmd.out_buf, out)
cmd.out = bufio.writer_to_writer(cmd.out_buf) cmd.out = bufio.writer_to_writer(cmd.out_buf)
cmd.err = err cmd.err = err
} }
@@ -122,12 +137,27 @@ parse_args :: proc(args: []string, out: io.Stream, err: io.Stream) -> (cmd: Comm
return cmd, true return cmd, true
} }
print_command_help :: proc(cmd: ^Command) { has_flag :: proc(cmd: ^Command, name: string) -> bool {
ok := write_command_help(cmd.name, cmd.out) _, ok := cmd.flags[name]
if !ok { if ok {
fmt.wprintf(cmd.err, "Unknown command: %s\n", cmd.name) return true
write_usage(cmd.out)
} }
_, ok2 := cmd.bool_set[name]
return ok2
}
find_command :: proc(name: string) -> (CommandInfo, bool) {
for c in COMMANDS {
if c.name == name {
return c, true
}
for a in c.aliases {
if a == name {
return c, true
}
}
}
return CommandInfo{}, false
} }
write_command_help :: proc(name: string, w: io.Writer) -> bool { write_command_help :: proc(name: string, w: io.Writer) -> bool {
@@ -160,18 +190,12 @@ write_command_help :: proc(name: string, w: io.Writer) -> bool {
return true return true
} }
find_command :: proc(name: string) -> (CommandInfo, bool) { print_command_help :: proc(cmd: ^Command) {
for c in COMMANDS { ok := write_command_help(cmd.name, cmd.out)
if c.name == name { if !ok {
return c, true fmt.wprintf(cmd.err, "Unknown command: %s\n", cmd.name)
write_usage(cmd.out)
} }
for a in c.aliases {
if a == name {
return c, true
}
}
}
return CommandInfo{}, false
} }
// TODO: command args should be shown in usage. // TODO: command args should be shown in usage.
@@ -246,21 +270,3 @@ Use "envr [command] --help" for more information about a command.
) )
} }
has_flag :: proc(cmd: ^Command, name: string) -> bool {
_, ok := cmd.flags[name]
if ok {
return true
}
_, ok2 := cmd.bool_set[name]
return ok2
}
delete_command :: proc(cmd: ^Command) {
bufio.writer_flush(cmd.out_buf)
delete(cmd.args)
delete(cmd.flags)
delete(cmd.bool_set)
bufio.writer_destroy(cmd.out_buf)
free(cmd.out_buf)
}

View File

@@ -5,6 +5,8 @@ import "core:os"
import "core:path/filepath" import "core:path/filepath"
cmd_check :: proc(cmd: ^Command) { cmd_check :: proc(cmd: ^Command) {
feats := check_features()
check_path: string check_path: string
if len(cmd.args) > 0 { if len(cmd.args) > 0 {
check_path = cmd.args[0] check_path = cmd.args[0]
@@ -40,6 +42,15 @@ cmd_check :: proc(cmd: ^Command) {
files_in_path: [dynamic]string files_in_path: [dynamic]string
if is_dir { if is_dir {
if cant_scan(feats) {
fmt.wprintln(
cmd.err,
"Error: please install fd to use the check command (https://github.com/sharkdp/fd)",
flush = false,
)
return
}
scanned, scan_ok := scan_path(abs_path, db.cfg) scanned, scan_ok := scan_path(abs_path, db.cfg)
if !scan_ok { if !scan_ok {
fmt.wprintln(cmd.err, "Error scanning directory for .env files", flush = false) fmt.wprintln(cmd.err, "Error scanning directory for .env files", flush = false)
@@ -61,23 +72,13 @@ cmd_check :: proc(cmd: ^Command) {
if len(files_in_path) == 0 { if len(files_in_path) == 0 {
fmt.wprintln(cmd.out, "No .env files found in the specified directory.", flush = false) fmt.wprintln(cmd.out, "No .env files found in the specified directory.", flush = false)
} else { } else {
fmt.wprintln( fmt.wprintln(cmd.out, "✓ All .env files in the directory are backed up.", flush = false)
cmd.out,
"✓ All .env files in the directory are backed up.",
flush = false,
)
} }
} else { } else {
fmt.wprintf( fmt.wprintf(cmd.out, "Found %d .env file(s) that are not backed up:\n", len(not_backed), flush = false)
cmd.out,
"Found %d .env file(s) that are not backed up:\n",
len(not_backed),
flush = false,
)
for file in not_backed { for file in not_backed {
fmt.wprintf(cmd.out, " %s\n", file, flush = false) fmt.wprintf(cmd.out, " %s\n", file, flush = false)
} }
fmt.wprintln(cmd.out, "\nRun 'envr sync' to back up these files.", flush = false) fmt.wprintln(cmd.out, "\nRun 'envr sync' to back up these files.", flush = false)
} }
} }

33
cmd_deps.odin Normal file
View File

@@ -0,0 +1,33 @@
package main
import "core:fmt"
import "core:os"
import "core:terminal"
// TODO: Improve table rendering
cmd_deps :: proc(cmd: ^Command) {
feats := check_features()
headers := []string{"Feature", "Status"}
rows: [dynamic][]string
if .Git in feats {
append(&rows, []string{"Git", "\u2713 Available"})
} else {
append(&rows, []string{"Git", "\u2717 Missing"})
}
if .Fd in feats {
append(&rows, []string{"fd", "\u2713 Available"})
} else {
append(&rows, []string{"fd", "\u2717 Missing"})
}
if terminal.is_terminal(os.stdout) {
render_table(cmd.out, headers, rows[:])
} else {
render_json_rows(cmd.out, headers, rows[:])
fmt.wprint(cmd.out, "\n", flush = false)
}
}

View File

@@ -12,7 +12,8 @@ cmd_edit_config :: proc(cmd: ^Command) {
config_path := cmd.config_path config_path := cmd.config_path
if !os.exists(config_path) { _, stat_err := os.stat(config_path, context.allocator)
if stat_err != nil {
fmt.wprintf( fmt.wprintf(
cmd.err, cmd.err,
"Config file does not exist at %s. Run 'envr init' first.\n", "Config file does not exist at %s. Run 'envr init' first.\n",

View File

@@ -25,6 +25,7 @@ cmd_list :: proc(cmd: ^Command) {
if !list_ok { if !list_ok {
return return
} }
defer delete(rows)
if terminal.is_terminal(os.stdout) { if terminal.is_terminal(os.stdout) {
headers := []string{"Directory", "Path"} headers := []string{"Directory", "Path"}
@@ -33,7 +34,7 @@ cmd_list :: proc(cmd: ^Command) {
for row in rows { for row in rows {
dir_str := strings.concatenate({row.Dir, "/"}, context.temp_allocator) dir_str := strings.concatenate({row.Dir, "/"}, context.temp_allocator)
filename := filepath.base(row.Path) filename := filepath.base(row.Path)
row_slice := make([]string, 2, context.temp_allocator) row_slice := make([]string, 2)
row_slice[0] = dir_str row_slice[0] = dir_str
row_slice[1] = filename row_slice[1] = filename
append(&table_rows, row_slice) append(&table_rows, row_slice)

View File

@@ -6,13 +6,23 @@ import "core:os"
import "core:terminal" import "core:terminal"
cmd_scan :: proc(cmd: ^Command) { cmd_scan :: proc(cmd: ^Command) {
feats := check_features()
if cant_scan(feats) {
fmt.wprintln(
cmd.err,
"Error: please install fd to use the scan command (https://github.com/sharkdp/fd)",
flush = false,
)
return
}
db, db_ok := db_open(cmd.config_path) db, db_ok := db_open(cmd.config_path)
if !db_ok { if !db_ok {
return return
} }
defer db_close(&db) defer db_close(&db)
search_dirs := search_paths(db.cfg, context.temp_allocator) search_dirs := search_paths(db.cfg)
if len(search_dirs) == 0 { if len(search_dirs) == 0 {
fmt.wprintln( fmt.wprintln(
cmd.err, cmd.err,

View File

@@ -24,42 +24,63 @@ cmd_sync :: proc(cmd: ^Command) {
if !list_ok { if !list_ok {
return return
} }
defer delete(files)
results := make([]SyncEntry, len(files), context.temp_allocator) // TODO: Set sane default size
results: [dynamic]SyncEntry
defer delete(results)
for &file, i in files { for &file in files {
result, err := db_sync(&db, &file) old_path: string
old_path, _ = strings.clone(file.Path, context.temp_allocator)
result, err_msg := db_sync(&db, &file)
status: string status: string
if err != .None { is_dir_updated := .DirUpdated in result
status = sync_error_message(err)
} else if .BackedUp in result { switch {
status = .DirUpdated in result ? "Moved & Backed Up" : "Backed Up" case .Error in result:
} else if .Restored in result { if len(err_msg) > 0 {
status = .DirUpdated in result ? "Moved & Restored" : "Restored" status = err_msg
} else if .DirUpdated in result {
status = "Moved"
} else { } else {
status = "error"
}
case .BackedUp in result:
status = "Backed Up"
case .Restored in result:
status = "Restored"
case .DirUpdated in result:
status = "Moved"
case:
status = "OK" status = "OK"
} }
// TODO: Handle errors if is_dir_updated {
path_str, _ := strings.clone(file.Path, context.temp_allocator) if !db_delete(&db, old_path) {
status_str, _ := strings.clone(status, context.temp_allocator) return
results[i] = SyncEntry {
Path = path_str,
Status = status_str,
} }
} }
if db_update_required(result) {
if !db_insert(&db, file) {
return
}
}
path_str, _ := strings.clone(file.Path)
status_str, _ := strings.clone(status)
append(&results, SyncEntry{Path = path_str, Status = status_str})
}
if terminal.is_terminal(os.stdout) { if terminal.is_terminal(os.stdout) {
headers := []string{"File", "Status"} headers := []string{"File", "Status"}
// TODO: Use [2]string instead of slice here table_rows := make([dynamic][]string, 0, len(results))
table_rows := make([dynamic][]string, 0, len(results), context.temp_allocator)
for res in results { for res in results {
row_slice := [2]string{res.Path, res.Status} row_slice := make([]string, 2)
append(&table_rows, row_slice[:]) row_slice[0] = res.Path
row_slice[1] = res.Status
append(&table_rows, row_slice)
} }
render_table(cmd.out, headers, table_rows[:]) render_table(cmd.out, headers, table_rows[:])
@@ -73,23 +94,3 @@ cmd_sync :: proc(cmd: ^Command) {
} }
} }
sync_error_message :: proc(e: SyncError) -> string {
switch e {
case .None:
return ""
case .DirMissing:
return "directory missing"
case .MultipleDirs:
return "multiple directories found"
case .GitRootFailed:
return "failed to find git roots"
case .WriteFailed:
return "failed to write file"
case .ReadFailed:
return "failed to read file"
case .DbFailed:
return "failed to update database"
}
return "unknown error"
}

View File

@@ -6,8 +6,6 @@ import "core:os"
import "core:path/filepath" import "core:path/filepath"
import "core:strings" import "core:strings"
import "findr"
SshKeyPair :: struct { SshKeyPair :: struct {
Private: string `json:"private"`, Private: string `json:"private"`,
Public: string `json:"public"`, Public: string `json:"public"`,
@@ -25,16 +23,25 @@ Config :: struct {
config_path: string `json:"-"`, config_path: string `json:"-"`,
} }
load_config :: proc(config_path: string, allocator := context.allocator) -> (Config, bool) { default_config_path :: proc(home: string, allocator := context.allocator) -> string {
// TODO: Should we use context.allocator + defer delete()? path, err := filepath.join([]string{home, ".envr", "config.json"}, allocator)
data, read_err := os.read_entire_file_from_path(config_path, context.temp_allocator) if err != nil {
panic("Ran out of memory when building config path")
}
return path
}
load_config :: proc(config_path: string) -> (Config, bool) {
data, read_err := os.read_entire_file_from_path(config_path, context.allocator)
if read_err != nil { if read_err != nil {
fmt.println("No config file found. Please run `envr init` to generate one.") fmt.println("No config file found. Please run `envr init` to generate one.")
return Config{}, false return Config{}, false
} }
defer delete(data)
cfg: Config cfg: Config
err := json.unmarshal(data, &cfg, .JSON5, allocator) // TODO: use json 5
err := json.unmarshal(data, &cfg)
if err != nil { if err != nil {
fmt.printf("Error parsing config: %v\n", err) fmt.printf("Error parsing config: %v\n", err)
return Config{}, false return Config{}, false
@@ -44,101 +51,33 @@ load_config :: proc(config_path: string, allocator := context.allocator) -> (Con
return cfg, true return cfg, true
} }
default_config_path :: proc(home: string, allocator := context.allocator) -> string { delete_config :: proc(cfg: ^Config) {
path, err := filepath.join([]string{home, ".envr", "config.json"}, allocator)
if err != nil {
panic("Ran out of memory when building config path")
}
return path
}
delete_config :: proc(cfg: ^Config, allocator := context.allocator) {
for key in cfg.Keys { for key in cfg.Keys {
delete(key.Private, allocator) delete(key.Private)
delete(key.Public, allocator) delete(key.Public)
} }
delete(cfg.Keys) delete(cfg.Keys)
delete(cfg.ScanConfig.Matcher, allocator) delete(cfg.ScanConfig.Matcher)
for exclude in cfg.ScanConfig.Exclude { for exclude in cfg.ScanConfig.Exclude {
delete(exclude, allocator) delete(exclude)
} }
delete(cfg.ScanConfig.Exclude) delete(cfg.ScanConfig.Exclude)
for include in cfg.ScanConfig.Include { for include in cfg.ScanConfig.Include {
delete(include, allocator) delete(include)
} }
delete(cfg.ScanConfig.Include) delete(cfg.ScanConfig.Include)
} }
save_config :: proc(cfg: Config, force: bool = false) -> bool { envr_dir :: proc(config_path: string) -> string {
config_dir := envr_dir(cfg.config_path) return filepath.dir(config_path)
if !os.exists(config_dir) {
mkdir_err := os.make_directory(config_dir)
if mkdir_err != nil {
fmt.printf("Error creating %s directory: %v\n", config_dir, mkdir_err)
return false
}
}
if os.exists(cfg.config_path) && !force {
info, stat_err := os.stat(cfg.config_path, context.allocator)
if stat_err == nil {
defer os.file_info_delete(info, context.allocator)
if info.size > 0 {
fmt.println("Config file already exists. Run again with --force to reinitialize.")
return false
}
}
}
data, marshal_err := json.marshal(cfg, {pretty = true, use_spaces = true, spaces = 2})
if marshal_err != nil {
fmt.printf("Error marshaling config: %v\n", marshal_err)
return false
}
defer delete(data)
write_err := os.write_entire_file(cfg.config_path, data)
if write_err != nil {
fmt.printf("Error writing config: %v\n", write_err)
return false
}
return true
} }
// Caller is responsible for calling delete_config() data_path :: proc(config_path: string) -> string {
new_config :: proc( path, _ := filepath.join([]string{envr_dir(config_path), "data.envr"})
private_key_paths: []string, return path
cfg_path: string = "~/.envr/config.json",
) -> Config {
keys := make([dynamic]SshKeyPair, 0, len(private_key_paths))
for priv in private_key_paths {
// TODO: Is this bad?
priv_key := strings.clone(priv)
pub, _ := strings.concatenate([]string{priv_key, ".pub"})
append(&keys, SshKeyPair{Private = priv_key, Public = pub})
}
exclude := make([dynamic]string, 0, 4)
append(&exclude, strings.clone("*\\.envrc"))
append(&exclude, strings.clone("\\.local/"))
append(&exclude, strings.clone("node_modules"))
append(&exclude, strings.clone("vendor"))
include := make([dynamic]string, 0, 1)
append(&include, strings.clone("~"))
scan_cfg := ScanConfig {
Matcher = strings.clone("\\.env"),
Exclude = exclude,
Include = include,
}
return Config{Keys = keys, ScanConfig = scan_cfg, config_path = cfg_path}
} }
find_ssh_private_keys :: proc() -> (keys: [dynamic]string, ok: bool) { find_ssh_private_keys :: proc() -> (keys: [dynamic]string, ok: bool) {
@@ -187,49 +126,115 @@ find_ssh_private_keys :: proc() -> (keys: [dynamic]string, ok: bool) {
return return
} }
find_git_roots :: proc( // Caller is responsible for calling delete_config()
cfg: Config, new_config :: proc(
allocator := context.temp_allocator, private_key_paths: []string,
) -> ( cfg_path: string = "~/.envr/config.json",
roots: [dynamic]string, ) -> Config {
ok: bool, keys := make([dynamic]SshKeyPair, 0, len(private_key_paths))
) { for priv in private_key_paths {
paths := search_paths(cfg, allocator) // TODO: Is this bad?
// TODO: Pass allocator to findr priv_key := strings.clone(priv)
// findr.find_repos(paths[:], &roots, os.get_processor_core_count(), allocator) pub, _ := strings.concatenate([]string{priv_key, ".pub"})
findr.find_repos(paths[:], &roots, os.get_processor_core_count()) append(&keys, SshKeyPair{Private = priv_key, Public = pub})
}
exclude := make([dynamic]string, 0, 4)
append(&exclude, strings.clone("*\\.envrc"))
append(&exclude, strings.clone("\\.local/"))
append(&exclude, strings.clone("node_modules"))
append(&exclude, strings.clone("vendor"))
include := make([dynamic]string, 0, 1)
append(&include, strings.clone("~"))
scan_cfg := ScanConfig {
Matcher = strings.clone("\\.env"),
Exclude = exclude,
Include = include,
}
return Config{Keys = keys, ScanConfig = scan_cfg, config_path = cfg_path}
}
save_config :: proc(cfg: Config, force: bool = false) -> bool {
config_dir := envr_dir(cfg.config_path)
if !os.exists(config_dir) {
mkdir_err := os.make_directory(config_dir)
if mkdir_err != nil {
fmt.printf("Error creating %s directory: %v\n", config_dir, mkdir_err)
return false
}
}
if os.exists(cfg.config_path) && !force {
info, stat_err := os.stat(cfg.config_path, context.allocator)
if stat_err == nil {
defer os.file_info_delete(info, context.allocator)
if info.size > 0 {
fmt.println("Config file already exists. Run again with --force to reinitialize.")
return false
}
}
}
data, marshal_err := json.marshal(cfg, {pretty = true, use_spaces = true, spaces = 2})
if marshal_err != nil {
fmt.printf("Error marshaling config: %v\n", marshal_err)
return false
}
defer delete(data)
write_err := os.write_entire_file(cfg.config_path, data)
if write_err != nil {
fmt.printf("Error writing config: %v\n", write_err)
return false
}
return true
}
search_paths :: proc(cfg: Config) -> (paths: [dynamic]string) {
// TODO: Is this okay?
// TODO: handle error
home, _ := os.user_home_dir(context.temp_allocator)
for include in cfg.ScanConfig.Include {
// TODO: Do we need to manually expand ~/ in odin?
expanded, _ := strings.replace(include, "~", home, 1)
if filepath.is_abs(expanded) {
append(&paths, expanded)
} else {
defer delete(expanded)
resolved, err := filepath.abs(expanded)
if err == nil {
append(&paths, resolved)
}
}
}
return
}
find_git_roots :: proc(cfg: Config) -> (roots: [dynamic]string, ok: bool) {
paths := search_paths(cfg)
for sp in paths {
args := []string{"fd", "-H", "-t", "d", "^\\.git$", sp}
lines, fd_ok := run_fd(args)
if !fd_ok {
return
}
for line in lines {
cleaned, _ := filepath.clean(line)
parent := filepath.dir(cleaned)
cloned, _ := strings.clone(parent)
append(&roots, cloned)
}
}
ok = true ok = true
return return
} }
search_paths :: proc(cfg: Config, allocator := context.allocator) -> [dynamic]string {
// TODO: handle error
home, _ := os.user_home_dir(context.temp_allocator)
paths, _ := new_clone(cfg.ScanConfig.Include, allocator)
for &include in paths {
// TODO: Do we need to manually expand ~/ in odin?
expanded, _ := strings.replace(include, "~", home, 1, allocator)
if filepath.is_abs(expanded) {
include = expanded
} else {
resolved, err := filepath.abs(expanded, allocator)
if err == nil {
include = resolved
}
}
}
return paths^
}
envr_dir :: proc(config_path: string) -> string {
return filepath.dir(config_path)
}
// User is responsible for freeing the path
data_path :: proc(config_path: string, allocator := context.allocator) -> string {
path, _ := filepath.join([]string{envr_dir(config_path), "data.envr"}, allocator)
return path
}

View File

@@ -187,10 +187,14 @@ test_search_paths_expands_tilde :: proc(t: ^testing.T) {
cfg := Config { cfg := Config {
ScanConfig = ScanConfig{Include = make([dynamic]string, 0, 1)}, ScanConfig = ScanConfig{Include = make([dynamic]string, 0, 1)},
} }
append(&cfg.ScanConfig.Include, "~")
defer delete(cfg.ScanConfig.Include) defer delete(cfg.ScanConfig.Include)
append(&cfg.ScanConfig.Include, "~")
paths := search_paths(cfg, context.temp_allocator) paths := search_paths(cfg)
defer delete(paths)
for path in paths {
defer delete(path)
}
testing.expect(t, len(paths) == 1, "should have 1 path") testing.expect(t, len(paths) == 1, "should have 1 path")
if len(paths) > 0 { if len(paths) > 0 {

View File

@@ -2,7 +2,6 @@ package main
import "core:fmt" import "core:fmt"
import "core:mem" import "core:mem"
import "core:os"
MAGIC :: "ENVR" MAGIC :: "ENVR"
MAGIC_BYTES := [4]u8{u8('E'), u8('N'), u8('V'), u8('R')} MAGIC_BYTES := [4]u8{u8('E'), u8('N'), u8('V'), u8('R')}
@@ -21,19 +20,80 @@ RecipientEntry :: struct {
EncryptedKey: [CRYPTO_SECRETBOX_KEY_BYTES + CRYPTO_BOX_MAC_BYTES]u8, EncryptedKey: [CRYPTO_SECRETBOX_KEY_BYTES + CRYPTO_BOX_MAC_BYTES]u8,
} }
sodium_initialized: bool
ensure_sodium :: proc() -> bool {
if sodium_initialized {
return true
}
rc := sodium_init()
if rc < 0 {
fmt.println("Error: libsodium initialization failed")
return false
}
sodium_initialized = true
return true
}
X25519Keypair :: struct { X25519Keypair :: struct {
Public: [CRYPTO_BOX_PUBLICKEY_BYTES]u8, Public: [CRYPTO_BOX_PUBLICKEY_BYTES]u8,
Private: [CRYPTO_BOX_SECRETKEY_BYTES]u8, Private: [CRYPTO_BOX_SECRETKEY_BYTES]u8,
} }
@(init) ssh_to_x25519 :: proc(keys: []SshKeyPair) -> (pairs: []X25519Keypair, ok: bool) {
init_sodium :: proc "contextless" () { if len(keys) == 0 {
if sodium_init() < 0 { return
os.exit(1)
} }
pairs = make([]X25519Keypair, len(keys))
for i in 0 ..< len(keys) {
ssh_kp, parse_ok := parse_ssh_private_key(keys[i].Private)
if !parse_ok {
fmt.printf("Error: failed to parse SSH private key: %s\n", keys[i].Private)
delete(pairs)
return
}
ssh_pub, pub_ok := parse_ssh_public_key(keys[i].Public)
if !pub_ok {
fmt.printf("Error: failed to parse SSH public key: %s\n", keys[i].Public)
delete(pairs)
return
}
pk_rc := crypto_sign_ed25519_pk_to_curve25519(&pairs[i].Public[0], &ssh_pub[0])
if pk_rc != 0 {
fmt.println("Error: failed to convert ed25519 public key to curve25519")
delete(pairs)
return
}
ed25519_sk: [64]u8
for j in 0 ..< 32 {
ed25519_sk[j] = ssh_kp.Private[j]
}
for j in 0 ..< 32 {
ed25519_sk[32 + j] = ssh_kp.Public[j]
}
sk_rc := crypto_sign_ed25519_sk_to_curve25519(&pairs[i].Private[0], &ed25519_sk[0])
if sk_rc != 0 {
fmt.println("Error: failed to convert ed25519 private key to curve25519")
delete(pairs)
return
}
}
ok = true
return
} }
encrypt :: proc(plaintext: []u8, keys: []SshKeyPair) -> (ciphertext: []u8, ok: bool) { encrypt :: proc(plaintext: []u8, keys: []SshKeyPair) -> (ciphertext: []u8, ok: bool) {
if !ensure_sodium() {
return
}
x25519_pairs, pairs_ok := ssh_to_x25519(keys) x25519_pairs, pairs_ok := ssh_to_x25519(keys)
if !pairs_ok { if !pairs_ok {
return return
@@ -133,6 +193,10 @@ encrypt :: proc(plaintext: []u8, keys: []SshKeyPair) -> (ciphertext: []u8, ok: b
} }
decrypt :: proc(ciphertext: []u8, keys: []SshKeyPair) -> (plaintext: []u8, ok: bool) { decrypt :: proc(ciphertext: []u8, keys: []SshKeyPair) -> (plaintext: []u8, ok: bool) {
if !ensure_sodium() {
return
}
if len(ciphertext) < HEADER_SIZE { if len(ciphertext) < HEADER_SIZE {
fmt.println("Error: ciphertext too short (header)") fmt.println("Error: ciphertext too short (header)")
return return
@@ -272,52 +336,3 @@ decrypt :: proc(ciphertext: []u8, keys: []SshKeyPair) -> (plaintext: []u8, ok: b
return return
} }
ssh_to_x25519 :: proc(keys: []SshKeyPair) -> (pairs: []X25519Keypair, ok: bool) {
if len(keys) == 0 {
return
}
pairs = make([]X25519Keypair, len(keys))
for i in 0 ..< len(keys) {
ssh_kp, parse_ok := parse_ssh_private_key(keys[i].Private)
if !parse_ok {
fmt.printf("Error: failed to parse SSH private key: %s\n", keys[i].Private)
delete(pairs)
return
}
ssh_pub, pub_ok := parse_ssh_public_key(keys[i].Public)
if !pub_ok {
fmt.printf("Error: failed to parse SSH public key: %s\n", keys[i].Public)
delete(pairs)
return
}
pk_rc := crypto_sign_ed25519_pk_to_curve25519(&pairs[i].Public[0], &ssh_pub[0])
if pk_rc != 0 {
fmt.println("Error: failed to convert ed25519 public key to curve25519")
delete(pairs)
return
}
ed25519_sk: [64]u8
for j in 0 ..< 32 {
ed25519_sk[j] = ssh_kp.Private[j]
}
for j in 0 ..< 32 {
ed25519_sk[32 + j] = ssh_kp.Public[j]
}
sk_rc := crypto_sign_ed25519_sk_to_curve25519(&pairs[i].Private[0], &ed25519_sk[0])
if sk_rc != 0 {
fmt.println("Error: failed to convert ed25519 private key to curve25519")
delete(pairs)
return
}
}
ok = true
return
}

604
db.odin
View File

@@ -2,32 +2,28 @@ package main
import "core:crypto/hash" import "core:crypto/hash"
import "core:encoding/hex" import "core:encoding/hex"
import "core:encoding/ini"
import "core:encoding/json" import "core:encoding/json"
import "core:fmt" import "core:fmt"
import "core:mem"
import "core:os" import "core:os"
import "core:path/filepath" import "core:path/filepath"
import "core:strings" import "core:strings"
import "core:time"
import "sqlite" import "sqlite"
SyncFlagEnum :: enum { SyncFlagEnum :: enum {
Noop,
DirUpdated, DirUpdated,
Restored, Restored,
BackedUp, BackedUp,
Error,
} }
SyncFlag :: bit_set[SyncFlagEnum] SyncFlag :: bit_set[SyncFlagEnum]
SyncError :: enum { SyncDirection :: enum {
None, TrustDatabase,
DirMissing, TrustFilesystem,
MultipleDirs,
GitRootFailed,
WriteFailed,
ReadFailed,
DbFailed,
} }
Db :: struct { Db :: struct {
@@ -35,7 +31,6 @@ Db :: struct {
db: ^rawptr, db: ^rawptr,
cfg: Config, cfg: Config,
changed: bool, changed: bool,
arena: mem.Dynamic_Arena,
} }
EnvFile :: struct { EnvFile :: struct {
@@ -46,7 +41,6 @@ EnvFile :: struct {
contents: string, contents: string,
} }
@(deprecated = "call db_close to clean up EnvFiles")
delete_envfile :: proc(f: ^EnvFile) { delete_envfile :: proc(f: ^EnvFile) {
delete(f.Path) delete(f.Path)
for &remote in f.Remotes { for &remote in f.Remotes {
@@ -57,33 +51,29 @@ delete_envfile :: proc(f: ^EnvFile) {
delete(f.contents) delete(f.contents)
} }
db_open :: proc(cfg_path: string) -> (database: Db, ok: bool) { make_temp_path :: proc() -> string {
database = db_init() or_return ts := time.time_to_unix(time.now())
database.cfg = load_config(cfg_path, db_allocator(&database)) or_return b: strings.Builder
strings.builder_init(&b)
// TODO: Use different allocators? defer strings.builder_destroy(&b)
data_path := data_path(database.cfg.config_path, context.temp_allocator) fmt.sbprintf(&b, "/tmp/envr-%d-%d.db", os.get_pid(), ts)
if os.exists(data_path) { return strings.to_string(b)
if ok = db_restore_from_encrypted(&database, data_path); !ok {
sqlite.db_close(database.db)
return database, false
}
} else {
// DB was created
database.changed = true
}
return database, true
} }
// Creates a database an allocator and fresh, empty table, with zero encryption. db_open :: proc(cfg_path: string) -> (Db, bool) {
// In production, you most likely want to use `db_open`. cfg, ok := load_config(cfg_path)
db_init :: proc() -> (database: Db, ok: bool) { if !ok {
return Db{}, false
}
data_path := data_path(cfg.config_path)
_, stat_err := os.stat(data_path, context.allocator)
db: ^rawptr db: ^rawptr
rc := sqlite.db_open(":memory:", &db) rc := sqlite.db_open(":memory:", &db)
if rc != sqlite.OK { if rc != sqlite.OK {
fmt.printf("Error opening in-memory database: %s\n", sqlite.db_errmsg(db)) fmt.printf("Error opening in-memory database: %s\n", sqlite.db_errmsg(db))
return return Db{}, false
} }
create_sql: cstring = "CREATE TABLE IF NOT EXISTS envr_env_files (path TEXT PRIMARY KEY NOT NULL, remotes TEXT, sha256 TEXT NOT NULL, contents TEXT NOT NULL)" create_sql: cstring = "CREATE TABLE IF NOT EXISTS envr_env_files (path TEXT PRIMARY KEY NOT NULL, remotes TEXT, sha256 TEXT NOT NULL, contents TEXT NOT NULL)"
@@ -91,69 +81,21 @@ db_init :: proc() -> (database: Db, ok: bool) {
if rc != sqlite.OK { if rc != sqlite.OK {
fmt.printf("Error creating table: %s\n", sqlite.db_errmsg(db)) fmt.printf("Error creating table: %s\n", sqlite.db_errmsg(db))
sqlite.db_close(db) sqlite.db_close(db)
return return Db{}, false
}
database.db = db
mem.dynamic_arena_init(&database.arena)
return database, true
}
db_allocator :: proc(db: ^Db) -> mem.Allocator {
return mem.dynamic_arena_allocator(&db.arena)
}
db_restore_from_encrypted :: proc(db: ^Db, data_path: string) -> bool {
encrypted_data, read_err := os.read_entire_file_from_path(data_path, context.temp_allocator)
if read_err != nil {
fmt.printf("Error reading encrypted database: %v\n", read_err)
return false
} }
// TODO: Use context.temp_allocator if stat_err == nil {
plaintext, dec_ok := decrypt(encrypted_data, db.cfg.Keys[:]) if !db_restore_from_encrypted(db, cfg) {
if !dec_ok { sqlite.db_close(db)
fmt.println("Error: decryption failed") return Db{}, false
return false
} }
defer delete(plaintext)
n := i64(len(plaintext))
buf := sqlite.malloc64(n)
if buf == nil {
fmt.println("Error: failed to allocate buffer for deserialization")
return false
}
copy(buf[:len(plaintext)], plaintext)
rc := sqlite.deserialize(
db.db,
"main",
buf,
n,
n,
sqlite.DESERIALIZE_FREEONCLOSE | sqlite.DESERIALIZE_RESIZEABLE,
)
if rc != sqlite.OK {
sqlite.free(buf)
fmt.printf("Error deserializing database: %s\n", sqlite.db_errmsg(db.db))
return false
} }
return true return Db{db = db, cfg = cfg, changed = stat_err != nil}, true
} }
db_close :: proc(d: ^Db) { db_close :: proc(d: ^Db) {
allocator := db_allocator(d) defer sqlite.db_close(d.db)
defer {
sqlite.db_close(d.db)
delete_config(&d.cfg, allocator)
mem.dynamic_arena_destroy(&d.arena)
}
if d.changed { if d.changed {
rc := sqlite.db_exec(d.db, "VACUUM", nil, nil, nil) rc := sqlite.db_exec(d.db, "VACUUM", nil, nil, nil)
@@ -171,14 +113,13 @@ db_close :: proc(d: ^Db) {
defer sqlite.free(data) defer sqlite.free(data)
sqlite_data := data[:sz] sqlite_data := data[:sz]
// TODO: PAss allocator chain
encrypted, enc_ok := encrypt(sqlite_data, d.cfg.Keys[:]) encrypted, enc_ok := encrypt(sqlite_data, d.cfg.Keys[:])
if !enc_ok { if !enc_ok {
fmt.println("Error: encryption failed") fmt.println("Error: encryption failed")
return return
} }
data_path := data_path(d.cfg.config_path, allocator) data_path := data_path(d.cfg.config_path)
envr_d := envr_dir(d.cfg.config_path) envr_d := envr_dir(d.cfg.config_path)
os.mkdir_all(envr_d) os.mkdir_all(envr_d)
@@ -193,8 +134,14 @@ db_close :: proc(d: ^Db) {
} }
} }
// Results will be freed when `db_close` is called. // Caller is responsible for calling:
db_list :: proc(d: ^Db) -> ([]EnvFile, bool) { // ```odin
// delete(results)
// for &result in results {
// delete(&result)
// }
// ```
db_list :: proc(d: ^Db, allocator := context.allocator) -> (results: [dynamic]EnvFile, ok: bool) {
stmt: ^rawptr stmt: ^rawptr
rc := sqlite.prepare_v2( rc := sqlite.prepare_v2(
d.db, d.db,
@@ -205,13 +152,10 @@ db_list :: proc(d: ^Db) -> ([]EnvFile, bool) {
) )
if rc != sqlite.OK { if rc != sqlite.OK {
fmt.printf("Error preparing query: %s\n", sqlite.db_errmsg(d.db)) fmt.printf("Error preparing query: %s\n", sqlite.db_errmsg(d.db))
return []EnvFile{}, false return
} }
defer sqlite.finalize(stmt) defer sqlite.finalize(stmt)
allocator := db_allocator(d)
results := make([dynamic]EnvFile, 0, 10, allocator)
for { for {
rc = sqlite.step(stmt) rc = sqlite.step(stmt)
if rc == sqlite.DONE { if rc == sqlite.DONE {
@@ -219,7 +163,7 @@ db_list :: proc(d: ^Db) -> ([]EnvFile, bool) {
} }
if rc != sqlite.ROW { if rc != sqlite.ROW {
fmt.printf("Error stepping query: %s\n", sqlite.db_errmsg(d.db)) fmt.printf("Error stepping query: %s\n", sqlite.db_errmsg(d.db))
#no_bounds_check return results[:], false return
} }
remotes_json := string(sqlite.column_text(stmt, 1)) remotes_json := string(sqlite.column_text(stmt, 1))
@@ -241,16 +185,156 @@ db_list :: proc(d: ^Db) -> ([]EnvFile, bool) {
) )
} }
#no_bounds_check return results[:], true ok = true
return
}
db_restore_from_encrypted :: proc(db: ^rawptr, cfg: Config) -> bool {
encrypted_data, read_err := os.read_entire_file_from_path(
data_path(cfg.config_path),
context.allocator,
)
defer delete(encrypted_data)
if read_err != nil {
fmt.printf("Error reading encrypted database: %v\n", read_err)
return false
}
plaintext, dec_ok := decrypt(encrypted_data, cfg.Keys[:])
if !dec_ok {
fmt.println("Error: decryption failed")
return false
}
defer delete(plaintext)
n := i64(len(plaintext))
buf := sqlite.malloc64(n)
if buf == nil {
fmt.println("Error: failed to allocate buffer for deserialization")
return false
}
copy(buf[:len(plaintext)], plaintext)
rc := sqlite.deserialize(
db,
"main",
buf,
n,
n,
sqlite.DESERIALIZE_FREEONCLOSE | sqlite.DESERIALIZE_RESIZEABLE,
)
if rc != sqlite.OK {
sqlite.free(buf)
fmt.printf("Error deserializing database: %s\n", sqlite.db_errmsg(db))
return false
}
return true
}
get_git_remotes :: proc(dir: string) -> [dynamic]string {
remotes: [dynamic]string
remote_set: map[string]bool
b: strings.Builder
strings.builder_init(&b)
defer strings.builder_destroy(&b)
fmt.sbprintf(&b, "%s-git-remotes", make_temp_path())
tmp_path := strings.to_string(b)
tmp_file, tmp_err := os.open(tmp_path, os.O_CREATE | os.O_WRONLY | os.O_TRUNC)
if tmp_err != nil {
return remotes
}
args := []string{"git", "remote", "-v"}
desc := os.Process_Desc {
command = args,
stdout = tmp_file,
stderr = nil,
working_dir = dir,
}
p, start_err := os.process_start(desc)
os.close(tmp_file)
if start_err != nil {
os.remove(tmp_path)
return remotes
}
state, wait_err := os.process_wait(p)
if wait_err != nil || state.exit_code != 0 {
os.remove(tmp_path)
return remotes
}
data, read_err := os.read_entire_file_from_path(tmp_path, context.allocator)
defer delete(data)
os.remove(tmp_path)
if read_err != nil {
return remotes
}
lines := strings.split(string(data), "\n")
for &line in lines {
line = strings.trim_space(line)
if len(line) == 0 {
continue
}
parts := strings.fields(line)
if len(parts) >= 2 {
remote_set[parts[1]] = true
}
}
for remote, _ in remote_set {
cloned, _ := strings.clone(remote)
append(&remotes, cloned)
}
return remotes
}
new_env_file :: proc(path: string) -> (EnvFile, bool) {
abs_path, abs_err := filepath.abs(path)
if abs_err != nil {
fmt.printf("Error getting absolute path: %v\n", abs_err)
return EnvFile{}, false
}
dir := filepath.dir(abs_path)
remotes := get_git_remotes(dir)
data, read_err := os.read_entire_file_from_path(abs_path, context.allocator)
defer delete(data)
if read_err != nil {
fmt.printf("Error reading file %s: %v\n", abs_path, read_err)
return EnvFile{}, false
}
digest := hash.hash_bytes(hash.Algorithm.SHA256, data, context.temp_allocator)
// TODO: Handle error
hex_bytes, _ := hex.encode(digest)
return EnvFile {
Path = abs_path,
Dir = dir,
Remotes = remotes,
Sha256 = string(hex_bytes),
contents = string(data),
},
true
} }
// TODO: Should we use context.temp_allocator for proc scoped lifetimes?
db_insert :: proc(d: ^Db, file: EnvFile) -> bool { db_insert :: proc(d: ^Db, file: EnvFile) -> bool {
remotes_json, marshal_err := json.marshal(file.Remotes, allocator = context.temp_allocator) remotes_json, marshal_err := json.marshal(file.Remotes)
if marshal_err != nil { if marshal_err != nil {
fmt.printf("Error marshaling remotes: %v\n", marshal_err) fmt.printf("Error marshaling remotes: %v\n", marshal_err)
return false return false
} }
defer delete(remotes_json)
sql: cstring = sql: cstring =
"INSERT OR REPLACE INTO " + "INSERT OR REPLACE INTO " +
@@ -306,8 +390,7 @@ db_insert :: proc(d: ^Db, file: EnvFile) -> bool {
return true return true
} }
// Result will be freed when `db_close` is called. db_fetch :: proc(d: ^Db, path: string, allocator := context.allocator) -> (EnvFile, bool) {
db_fetch :: proc(d: ^Db, path: string) -> (EnvFile, bool) {
sql: cstring = "SELECT path, remotes, sha256, contents FROM envr_env_files WHERE path = ?" sql: cstring = "SELECT path, remotes, sha256, contents FROM envr_env_files WHERE path = ?"
stmt: ^rawptr stmt: ^rawptr
rc := sqlite.prepare_v2(d.db, sql, -1, &stmt, nil) rc := sqlite.prepare_v2(d.db, sql, -1, &stmt, nil)
@@ -317,8 +400,6 @@ db_fetch :: proc(d: ^Db, path: string) -> (EnvFile, bool) {
} }
defer sqlite.finalize(stmt) defer sqlite.finalize(stmt)
allocator := db_allocator(d)
cpath := to_cstring(path, allocator) cpath := to_cstring(path, allocator)
defer delete(cpath, allocator) defer delete(cpath, allocator)
rc = sqlite.bind_text(stmt, 1, cpath, -1, nil) rc = sqlite.bind_text(stmt, 1, cpath, -1, nil)
@@ -342,7 +423,7 @@ db_fetch :: proc(d: ^Db, path: string) -> (EnvFile, bool) {
json.unmarshal_string(remotes_json, &remotes, allocator = allocator) json.unmarshal_string(remotes_json, &remotes, allocator = allocator)
} }
file_path := clone_cstring(sqlite.column_text(stmt, 0), allocator) file_path := clone_cstring(sqlite.column_text(stmt, 0))
return EnvFile { return EnvFile {
Path = file_path, Path = file_path,
@@ -386,175 +467,6 @@ db_delete :: proc(d: ^Db, path: string) -> bool {
return true return true
} }
new_env_file :: proc(path: string) -> (EnvFile, bool) {
abs_path, abs_err := filepath.abs(path)
if abs_err != nil {
fmt.printf("Error getting absolute path: %v\n", abs_err)
return EnvFile{}, false
}
dir := filepath.dir(abs_path)
// TODO: Should we use the db allocator here?
remotes := get_git_remotes(dir, context.allocator)
data, read_err := os.read_entire_file_from_path(abs_path, context.allocator)
defer delete(data)
if read_err != nil {
fmt.printf("Error reading file %s: %v\n", abs_path, read_err)
return EnvFile{}, false
}
digest := hash.hash_bytes(hash.Algorithm.SHA256, data, context.temp_allocator)
// TODO: Handle error
hex_bytes, _ := hex.encode(digest)
return EnvFile {
Path = abs_path,
Dir = dir,
Remotes = remotes,
Sha256 = string(hex_bytes),
contents = string(data),
},
true
}
// Reconciles `f` with the filesystem and persists changes to the database.
db_sync :: proc(d: ^Db, f: ^EnvFile) -> (SyncFlag, SyncError) {
allocator := db_allocator(d)
result: SyncFlag = {}
old_path := f.Path
if !os.exists(f.Dir) {
moved, err := try_move_dir(d, f, allocator)
if !moved {
return {}, err
}
result += {.DirUpdated}
}
if !os.exists(f.Path) {
write_err := os.write_entire_file(f.Path, f.contents)
if write_err != nil {
fmt.eprintf("db_sync: failed to write %s: %v\n", f.Path, write_err)
return result, .WriteFailed
}
if !db_persist(d, f, old_path) {
return result, .DbFailed
}
return result + {.Restored}, .None
}
data, read_err := os.read_entire_file_from_path(f.Path, allocator)
if read_err != nil {
fmt.eprintf("db_sync: failed to read %s: %v\n", f.Path, read_err)
return result, .ReadFailed
}
digest := hash.hash_bytes(hash.Algorithm.SHA256, data, context.temp_allocator)
hex_bytes, hex_err := hex.encode(digest, allocator)
if hex_err != nil {
fmt.eprintf("db_sync: failed to encode hash for %s: %v\n", f.Path, hex_err)
return result, .ReadFailed
}
current_sha := string(hex_bytes)
if current_sha == f.Sha256 {
if !db_persist(d, f, old_path) {
return result, .DbFailed
}
return result, .None
}
f.contents = string(data)
f.Sha256 = current_sha
if !db_persist(d, f, old_path) {
return result, .DbFailed
}
return result + {.BackedUp}, .None
}
db_persist :: proc(d: ^Db, f: ^EnvFile, old_path: string) -> bool {
if f.Path != old_path {
if !db_delete(d, old_path) {
return false
}
}
return db_insert(d, f^)
}
try_move_dir :: proc(d: ^Db, f: ^EnvFile, allocator: mem.Allocator) -> (bool, SyncError) {
// TODO: Should thread the allocator through, right?
roots, ok := find_git_roots(d.cfg, allocator)
if !ok {
return false, .GitRootFailed
}
match_count := 0
matched_dir: string
for root in roots {
remotes := get_git_remotes(root, context.temp_allocator)
if shares_remote(f, remotes[:]) {
match_count += 1
matched_dir = root
}
}
switch match_count {
case 0:
return false, .DirMissing
case 1:
f.Dir = matched_dir
base := filepath.base(f.Path)
new_path, _ := filepath.join({f.Dir, base}, allocator)
f.Path = new_path
f.Remotes = get_git_remotes(f.Dir, allocator)
return true, .None
case:
return false, .MultipleDirs
}
}
shares_remote :: proc(f: ^EnvFile, remotes: []string) -> bool {
for r1 in f.Remotes {
for r2 in remotes {
if r1 == r2 {
return true
}
}
}
return false
}
get_git_remotes :: proc(dir: string, allocator: mem.Allocator) -> [dynamic]string {
config_path, _ := filepath.join({dir, ".git", "config"}, context.temp_allocator)
// TODO: Handle error
m, _, ok := ini.load_map_from_path(config_path, context.temp_allocator)
if !ok {
return nil
}
remotes := make([dynamic]string, 0, 1, allocator)
for section_name, section in m {
if strings.has_prefix(section_name, "remote ") {
if url, ok := section["url"]; ok {
found := false
for r in remotes {
if r == url {found = true; break}
}
if !found {
cloned, _ := strings.clone(url, allocator)
append(&remotes, cloned)
}
}
}
}
return remotes
}
to_cstring :: proc { to_cstring :: proc {
string_to_cstring, string_to_cstring,
strings.to_cstring, strings.to_cstring,
@@ -569,7 +481,6 @@ string_to_cstring :: proc(s: string, allocator := context.allocator) -> cstring
return cs return cs
} }
// Unless an explicit allocator is passed, caller is responsible for freeing the result
clone_cstring :: proc(c: cstring, allocator := context.allocator) -> string { clone_cstring :: proc(c: cstring, allocator := context.allocator) -> string {
str, err := strings.clone_from_cstring(c, allocator) str, err := strings.clone_from_cstring(c, allocator)
if err != nil { if err != nil {
@@ -581,3 +492,146 @@ clone_cstring :: proc(c: cstring, allocator := context.allocator) -> string {
return str return str
} }
db_update_required :: proc(status: SyncFlag) -> bool {
return .BackedUp in status || .DirUpdated in status
}
shares_remote :: proc(f: ^EnvFile, remotes: []string) -> bool {
for r1 in f.Remotes {
for r2 in remotes {
if r1 == r2 {
return true
}
}
}
return false
}
update_dir :: proc(f: ^EnvFile, new_dir: string) {
f.Dir = new_dir
base := filepath.base(f.Path)
new_path, _ := strings.concatenate({new_dir, "/", base})
f.Path = new_path
f.Remotes = get_git_remotes(new_dir)
}
find_moved_dirs :: proc(d: ^Db, f: ^EnvFile) -> ([dynamic]string, bool) {
feats := check_features()
if .Fd not_in feats || .Git not_in feats {
fmt.println("Error: fd and git are required for moved dir detection")
return {}, false
}
roots, roots_ok := find_git_roots(d.cfg)
if !roots_ok {
return {}, false
}
moved: [dynamic]string
for root in roots {
remotes := get_git_remotes(root)
if shares_remote(f, remotes[:]) {
cloned, _ := strings.clone(root)
append(&moved, cloned)
}
}
return moved, true
}
db_sync :: proc(d: ^Db, f: ^EnvFile) -> (SyncFlag, string) {
return env_file_sync(f, .TrustFilesystem, d)
}
// If SyncFlag is .BackedUp, Caller is responsible for calling delete on f.contents and f.Sha256
env_file_sync :: proc(f: ^EnvFile, dir: SyncDirection, d: ^Db) -> (SyncFlag, string) {
result: SyncFlag = {}
_, stat_err := os.stat(f.Dir, context.allocator)
if stat_err != nil {
moved_dirs: [dynamic]string
if d != nil {
dirs, dirs_ok := find_moved_dirs(d, f)
if !dirs_ok {
return {.Error}, "failed to find moved dirs"
}
moved_dirs = dirs
}
if len(moved_dirs) == 0 {
return {.Error}, "directory missing"
} else if len(moved_dirs) == 1 {
update_dir(f, moved_dirs[0])
result = {.DirUpdated}
} else {
return {.Error}, "multiple directories found"
}
}
_, file_stat_err := os.stat(f.Path, context.allocator)
if file_stat_err != nil {
write_err := os.write_entire_file(f.Path, f.contents)
if write_err != nil {
msg, _ := strings.concatenate({"failed to write file: ", fmt.tprintf("%v", write_err)})
return {.Error}, msg
}
return result + {.Restored}, ""
}
data, read_err := os.read_entire_file_from_path(f.Path, context.allocator)
if read_err != nil {
msg, _ := strings.concatenate(
{"failed to read file for SHA comparison: ", fmt.tprintf("%v", read_err)},
)
return {.Error}, msg
}
digest := hash.hash_bytes(hash.Algorithm.SHA256, data)
// TODO: Handle error
hex_bytes, _ := hex.encode(digest)
current_sha := string(hex_bytes)
if current_sha == f.Sha256 {
return result, ""
}
switch dir {
case .TrustDatabase:
write_err := os.write_entire_file(f.Path, f.contents)
if write_err != nil {
msg, _ := strings.concatenate({"failed to write file: ", fmt.tprintf("%v", write_err)})
return {.Error}, msg
}
return result + {.Restored}, ""
case .TrustFilesystem:
if !env_file_backup(f) {
return {.Error}, "failed to backup file"
}
return result + {.BackedUp}, ""
}
return result, ""
}
// Loads the contents of the the file at f.Path into f.contents
//
// Caller is responsible for calling delete on f.contents and f.Sha256
env_file_backup :: proc(f: ^EnvFile) -> bool {
data, read_err := os.read_entire_file_from_path(f.Path, context.allocator)
if read_err != nil {
fmt.printf("Error reading file %s: %v\n", f.Path, read_err)
return false
}
f.contents = string(data)
digest := hash.hash_bytes(hash.Algorithm.SHA256, data, context.temp_allocator)
hex_bytes, alloc_err := hex.encode(digest)
if alloc_err != nil {
fmt.printf("Error generating hash for file %s: %v\n", f.Path, alloc_err)
return false
}
f.Sha256 = string(hex_bytes)
return true
}

View File

@@ -1,7 +1,5 @@
package main package main
import "core:crypto/hash"
import "core:encoding/hex"
import "core:fmt" import "core:fmt"
import "core:os" import "core:os"
import "core:path/filepath" import "core:path/filepath"
@@ -10,13 +8,30 @@ import "core:testing"
import "sqlite" import "sqlite"
make_test_db :: proc() -> (Db, bool) {
db: ^rawptr
rc := sqlite.db_open(":memory:", &db)
if rc != sqlite.OK {
return Db{}, false
}
create_sql: cstring = "CREATE TABLE IF NOT EXISTS envr_env_files (path TEXT PRIMARY KEY NOT NULL, remotes TEXT, sha256 TEXT NOT NULL, contents TEXT NOT NULL)"
rc = sqlite.db_exec(db, create_sql, nil, nil, nil)
if rc != sqlite.OK {
sqlite.db_close(db)
return Db{}, false
}
return Db{db = db}, true
}
make_test_env_file :: proc(path, sha, contents: string, remotes: []string = {}) -> EnvFile { make_test_env_file :: proc(path, sha, contents: string, remotes: []string = {}) -> EnvFile {
f := EnvFile { f := EnvFile {
Path = path, Path = path,
Dir = "", Dir = "",
Sha256 = sha, Sha256 = sha,
contents = contents, contents = contents,
Remotes = make([dynamic]string, 0, len(remotes), context.temp_allocator), Remotes = make([dynamic]string, 0, len(remotes)),
} }
for r in remotes { for r in remotes {
append(&f.Remotes, r) append(&f.Remotes, r)
@@ -26,10 +41,10 @@ make_test_env_file :: proc(path, sha, contents: string, remotes: []string = {})
@(test) @(test)
test_db_insert_and_fetch :: proc(t: ^testing.T) { test_db_insert_and_fetch :: proc(t: ^testing.T) {
d, ok := db_init() d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db") testing.expect(t, ok, "failed to create test db")
if !ok do return if !ok do return
defer db_close(&d) defer sqlite.db_close(d.db)
path := "/project/.env" path := "/project/.env"
sha := "abc123" sha := "abc123"
@@ -41,7 +56,7 @@ test_db_insert_and_fetch :: proc(t: ^testing.T) {
testing.expect(t, db_insert(&d, f), "insert should succeed") testing.expect(t, db_insert(&d, f), "insert should succeed")
fetched, fetch_ok := db_fetch(&d, "/project/.env") fetched, fetch_ok := db_fetch(&d, "/project/.env")
// defer delete_envfile(&fetched) defer delete_envfile(&fetched)
testing.expect(t, fetch_ok, "fetch should succeed") testing.expect(t, fetch_ok, "fetch should succeed")
if !fetch_ok do return if !fetch_ok do return
@@ -54,10 +69,10 @@ test_db_insert_and_fetch :: proc(t: ^testing.T) {
@(test) @(test)
test_db_fetch_missing :: proc(t: ^testing.T) { test_db_fetch_missing :: proc(t: ^testing.T) {
d, ok := db_init() d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db") testing.expect(t, ok, "failed to create test db")
if !ok do return if !ok do return
defer db_close(&d) defer sqlite.db_close(d.db)
_, fetch_ok := db_fetch(&d, "/nonexistent/.env") _, fetch_ok := db_fetch(&d, "/nonexistent/.env")
testing.expect(t, !fetch_ok, "fetch missing should return false") testing.expect(t, !fetch_ok, "fetch missing should return false")
@@ -65,9 +80,10 @@ test_db_fetch_missing :: proc(t: ^testing.T) {
@(test) @(test)
test_db_insert_or_replace :: proc(t: ^testing.T) { test_db_insert_or_replace :: proc(t: ^testing.T) {
d, ok := db_init() d, ok := make_test_db()
defer db_close(&d)
testing.expect(t, ok, "failed to create test db") testing.expect(t, ok, "failed to create test db")
if !ok do return
defer sqlite.db_close(d.db)
f1 := make_test_env_file("/project/.env", "sha1", "KEY=old") f1 := make_test_env_file("/project/.env", "sha1", "KEY=old")
defer delete(f1.Remotes) defer delete(f1.Remotes)
@@ -79,13 +95,18 @@ test_db_insert_or_replace :: proc(t: ^testing.T) {
results, list_ok := db_list(&d) results, list_ok := db_list(&d)
testing.expect(t, list_ok, "list should succeed") testing.expect(t, list_ok, "list should succeed")
if !list_ok do return
defer delete(results)
for &result in results {
defer delete_envfile(&result)
}
testing.expect(t, len(results) == 1, "should have 1 row, not 2") testing.expect(t, len(results) == 1, "should have 1 row, not 2")
fetched, fetch_ok := db_fetch(&d, "/project/.env") fetched, fetch_ok := db_fetch(&d, "/project/.env")
testing.expect(t, fetch_ok, "fetch should succeed") testing.expect(t, fetch_ok, "fetch should succeed")
if !fetch_ok do return if !fetch_ok do return
// defer delete_envfile(&fetched) defer delete_envfile(&fetched)
testing.expect_value(t, fetched.contents, "KEY=new") testing.expect_value(t, fetched.contents, "KEY=new")
testing.expect_value(t, fetched.Sha256, "sha2") testing.expect_value(t, fetched.Sha256, "sha2")
@@ -93,10 +114,10 @@ test_db_insert_or_replace :: proc(t: ^testing.T) {
@(test) @(test)
test_db_delete_existing :: proc(t: ^testing.T) { test_db_delete_existing :: proc(t: ^testing.T) {
d, ok := db_init() d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db") testing.expect(t, ok, "failed to create test db")
if !ok do return if !ok do return
defer db_close(&d) defer sqlite.db_close(d.db)
f := make_test_env_file("/project/.env", "sha", "KEY=val") f := make_test_env_file("/project/.env", "sha", "KEY=val")
defer delete(f.Remotes) defer delete(f.Remotes)
@@ -110,19 +131,20 @@ test_db_delete_existing :: proc(t: ^testing.T) {
@(test) @(test)
test_db_delete_missing :: proc(t: ^testing.T) { test_db_delete_missing :: proc(t: ^testing.T) {
d, ok := db_init() d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db") testing.expect(t, ok, "failed to create test db")
if !ok do return if !ok do return
defer db_close(&d) defer sqlite.db_close(d.db)
testing.expect(t, !db_delete(&d, "/nonexistent/.env"), "delete missing should return false") testing.expect(t, !db_delete(&d, "/nonexistent/.env"), "delete missing should return false")
} }
@(test) @(test)
test_db_list_multiple :: proc(t: ^testing.T) { test_db_list_multiple :: proc(t: ^testing.T) {
d, ok := db_init() d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db") testing.expect(t, ok, "failed to create test db")
defer db_close(&d) if !ok do return
defer sqlite.db_close(d.db)
f1 := make_test_env_file("/proj1/.env", "sha1", "A=1", []string{"git@github.com:a/repo.git"}) f1 := make_test_env_file("/proj1/.env", "sha1", "A=1", []string{"git@github.com:a/repo.git"})
defer delete(f1.Remotes) defer delete(f1.Remotes)
@@ -136,27 +158,36 @@ test_db_list_multiple :: proc(t: ^testing.T) {
results, list_ok := db_list(&d) results, list_ok := db_list(&d)
testing.expect(t, list_ok, "list should succeed") testing.expect(t, list_ok, "list should succeed")
if !list_ok do return
defer delete(results)
defer {
for &result in results {
delete_envfile(&result)
}
}
testing.expect_value(t, len(results), 3) testing.expect_value(t, len(results), 3)
} }
@(test) @(test)
test_db_list_empty :: proc(t: ^testing.T) { test_db_list_empty :: proc(t: ^testing.T) {
d, ok := db_init() d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db") testing.expect(t, ok, "failed to create test db")
defer db_close(&d) if !ok do return
defer sqlite.db_close(d.db)
results, list_ok := db_list(&d) results, list_ok := db_list(&d)
testing.expect(t, list_ok, "list should succeed on empty db") testing.expect(t, list_ok, "list should succeed on empty db")
testing.expect(t, len(results) == 0, "should have 0 rows") testing.expect(t, len(results) == 0, "should have 0 rows")
if list_ok do delete(results)
} }
@(test) @(test)
test_db_insert_sets_changed :: proc(t: ^testing.T) { test_db_insert_sets_changed :: proc(t: ^testing.T) {
d, ok := db_init() d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db") testing.expect(t, ok, "failed to create test db")
if !ok do return if !ok do return
defer db_close(&d) defer sqlite.db_close(d.db)
testing.expect(t, !d.changed, "changed should start false") testing.expect(t, !d.changed, "changed should start false")
@@ -169,10 +200,10 @@ test_db_insert_sets_changed :: proc(t: ^testing.T) {
@(test) @(test)
test_db_delete_sets_changed :: proc(t: ^testing.T) { test_db_delete_sets_changed :: proc(t: ^testing.T) {
d, ok := db_init() d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db") testing.expect(t, ok, "failed to create test db")
if !ok do return if !ok do return
defer db_close(&d) defer sqlite.db_close(d.db)
f := make_test_env_file("/project/.env", "sha", "KEY=val") f := make_test_env_file("/project/.env", "sha", "KEY=val")
defer delete(f.Remotes) defer delete(f.Remotes)
@@ -185,10 +216,10 @@ test_db_delete_sets_changed :: proc(t: ^testing.T) {
@(test) @(test)
test_db_serialize :: proc(t: ^testing.T) { test_db_serialize :: proc(t: ^testing.T) {
d, ok := db_init() d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db") testing.expect(t, ok, "failed to create test db")
if !ok do return if !ok do return
defer db_close(&d) defer sqlite.db_close(d.db)
f := make_test_env_file("/project/.env", "sha", "KEY=val") f := make_test_env_file("/project/.env", "sha", "KEY=val")
defer delete(f.Remotes) defer delete(f.Remotes)
@@ -203,6 +234,37 @@ test_db_serialize :: proc(t: ^testing.T) {
testing.expect(t, sz > 0, "serialized size should be > 0") testing.expect(t, sz > 0, "serialized size should be > 0")
} }
@(test)
test_db_update_required_noop :: proc(t: ^testing.T) {
testing.expect(t, !db_update_required({}), "Noop should not require update")
}
@(test)
test_db_update_required_backed_up :: proc(t: ^testing.T) {
testing.expect(t, db_update_required({.BackedUp}), "BackedUp should require update")
}
@(test)
test_db_update_required_dir_updated :: proc(t: ^testing.T) {
testing.expect(t, db_update_required({.DirUpdated}), "DirUpdated should require update")
}
@(test)
test_db_update_required_restored :: proc(t: ^testing.T) {
testing.expect(t, !db_update_required({.Restored}), "Restored alone should not require update")
}
@(test)
test_db_update_required_error :: proc(t: ^testing.T) {
testing.expect(t, !db_update_required({.Error}), "Error alone should not require update")
}
@(test)
test_db_update_required_combined :: proc(t: ^testing.T) {
combined := SyncFlag{.DirUpdated, .Restored}
testing.expect(t, db_update_required(combined), "DirUpdated|Restored should require update")
}
@(test) @(test)
test_shares_remote_overlap :: proc(t: ^testing.T) { test_shares_remote_overlap :: proc(t: ^testing.T) {
f := EnvFile { f := EnvFile {
@@ -257,81 +319,11 @@ test_shares_remote_both_empty :: proc(t: ^testing.T) {
testing.expect(t, !shares_remote(&f, remotes), "both empty should not share") testing.expect(t, !shares_remote(&f, remotes), "both empty should not share")
} }
delete_remotes :: proc(remotes: [dynamic]string) {
for &r in remotes {
delete(r)
}
delete(remotes)
}
@(test) @(test)
test_get_git_remotes_single :: proc(t: ^testing.T) { test_make_temp_path_format :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-remotes-%d", os.get_pid()) p := make_temp_path()
os.mkdir_all(base) testing.expect(t, strings.has_suffix(p, ".db"), "should end with .db")
defer os.remove_all(base) testing.expect(t, strings.contains(p, fmt.tprintf("%d", os.get_pid())), "should contain PID")
git_dir := fmt.tprintf("%s/.git", base)
os.mkdir_all(git_dir)
config_content := "[core]\n\trepositoryformatversion = 0\n[remote \"origin\"]\n\turl = git@github.com:user/repo.git\n\tfetch = +refs/heads/*:refs/remotes/origin/*\n"
config_path := fmt.tprintf("%s/config", git_dir)
err := os.write_entire_file(config_path, transmute([]u8)config_content)
testing.expect(t, err == nil, "should write .git/config")
remotes := get_git_remotes(base, context.temp_allocator)
testing.expect(t, len(remotes) == 1, "should find 1 remote")
if len(remotes) != 1 do return
testing.expect_value(t, remotes[0], "git@github.com:user/repo.git")
}
@(test)
test_get_git_remotes_multiple :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-remotes-multi-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
git_dir := fmt.tprintf("%s/.git", base)
os.mkdir_all(git_dir)
config_content := "[remote \"origin\"]\n\turl = git@github.com:user/repo.git\n[remote \"upstream\"]\n\turl = https://gitlab.com/upstream/repo.git\n"
config_path := fmt.tprintf("%s/config", git_dir)
err := os.write_entire_file(config_path, transmute([]u8)config_content)
testing.expect(t, err == nil, "should write .git/config")
remotes := get_git_remotes(base, context.temp_allocator)
testing.expect(t, len(remotes) == 2, "should find 2 remotes")
}
@(test)
test_get_git_remotes_no_config :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-remotes-none-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
remotes := get_git_remotes(base, context.temp_allocator)
testing.expect(t, len(remotes) == 0, "should return empty when no .git/config")
}
@(test)
test_get_git_remotes_no_remotes :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-remotes-empty-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
git_dir := fmt.tprintf("%s/.git", base)
os.mkdir_all(git_dir)
config_content := "[core]\n\trepositoryformatversion = 0\n\tbare = false\n"
config_path := fmt.tprintf("%s/config", git_dir)
err := os.write_entire_file(config_path, transmute([]u8)config_content)
testing.expect(t, err == nil, "should write .git/config")
remotes := get_git_remotes(base, context.temp_allocator)
testing.expect(t, len(remotes) == 0, "should return empty when no remote sections")
} }
@(test) @(test)
@@ -364,205 +356,45 @@ test_new_env_file_missing :: proc(t: ^testing.T) {
} }
@(test) @(test)
test_closing_db_has_no_leaks :: proc(t: ^testing.T) { test_env_file_backup :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-leak-%d", os.get_pid()) base := fmt.tprintf("/tmp/envr-test-backup-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
cfg_path, err := filepath.join([]string{base, "config.json"}, context.temp_allocator)
testing.expect(t, err == nil, "cfgPath should build successfully")
{
cfg := new_config([]string{"fixtures/keys/insecure-test-key"}, cfg_path)
testing.expect(t, save_config(cfg, force = true), "save should succeed")
delete_config(&cfg)
}
db, ok := db_open(cfg_path)
testing.expect(t, ok, "db should open")
db_close(&db)
}
@(test)
test_open_existing_db_has_no_leaks :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-leak-existing-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
cfg_path, err := filepath.join([]string{base, "config.json"}, context.temp_allocator)
testing.expect(t, err == nil, "cfgPath should build successfully")
{
cfg := new_config([]string{"fixtures/keys/insecure-test-key"}, cfg_path)
testing.expect(t, save_config(cfg, force = true), "save should succeed")
delete_config(&cfg)
}
// First open/close creates data.envr on disk
db, ok := db_open(cfg_path)
testing.expect(t, ok, "db should open")
if !ok do return
f := make_test_env_file(
"/project/.env",
"abc123",
"SECRET=value",
[]string{"git@github.com:user/repo.git"},
)
defer delete(f.Remotes)
testing.expect(t, db_insert(&db, f), "insert should succeed")
db_close(&db)
// Second open exercises db_restore_from_encrypted
db2, ok2 := db_open(cfg_path)
testing.expect(t, ok2, "db should open existing")
if !ok2 do return
db_close(&db2)
}
@(test)
test_db_sync_noop :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-sync-noop-%d", os.get_pid())
os.mkdir_all(base) os.mkdir_all(base)
defer os.remove_all(base) defer os.remove_all(base)
env_path := fmt.tprintf("%s/.env", base) env_path := fmt.tprintf("%s/.env", base)
content := "KEY=value\n" err := os.write_entire_file(env_path, "KEY=12345\n")
write_err := os.write_entire_file(env_path, transmute([]u8)content) testing.expect(t, err == nil, ".env file should exist")
testing.expect(t, write_err == nil, "should write .env file")
digest := hash.hash_bytes( f := EnvFile {
hash.Algorithm.SHA256, Path = env_path,
transmute([]u8)content,
context.temp_allocator,
)
hex_bytes, _ := hex.encode(digest, context.temp_allocator)
sha := string(hex_bytes)
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
defer db_close(&d)
f := make_test_env_file(env_path, sha, content)
f.Dir = base
db_insert(&d, f)
result, sync_err := db_sync(&d, &f)
testing.expect(t, sync_err == .None, "sync should not error")
testing.expect(t, result == {}, "should be noop")
}
@(test)
test_db_sync_backed_up :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-sync-backup-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
env_path := fmt.tprintf("%s/.env", base)
changed_content := "KEY=changed\n"
write_err := os.write_entire_file(env_path, transmute([]u8)changed_content)
testing.expect(t, write_err == nil, "should write .env file")
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
defer db_close(&d)
f := make_test_env_file(env_path, "old_sha", "KEY=original")
f.Dir = base
db_insert(&d, f)
result, sync_err := db_sync(&d, &f)
testing.expect(t, sync_err == .None, "sync should not error")
testing.expect(t, .BackedUp in result, "should be backed up")
}
@(test)
test_db_sync_restored :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-sync-restore-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
env_path := fmt.tprintf("%s/.env", base)
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
defer db_close(&d)
f := make_test_env_file(env_path, "some_sha", "SECRET=value")
f.Dir = base
defer delete(f.Remotes)
db_insert(&d, f)
result, err := db_sync(&d, &f)
testing.expect(t, err == .None, "sync should not error")
testing.expect(t, .Restored in result, "should be restored")
data, read_err := os.read_entire_file_from_path(env_path, context.temp_allocator)
testing.expect(t, read_err == nil, "file should exist after restore")
if read_err == nil {
testing.expect_value(t, string(data), "SECRET=value")
} }
defer delete(f.contents)
defer delete(f.Sha256)
testing.expect(t, env_file_backup(&f), "backup should succeed")
testing.expect_value(t, f.contents, "KEY=12345\n")
testing.expect_value(t, len(f.Sha256), 64)
} }
@(test) @(test)
test_db_sync_dir_missing :: proc(t: ^testing.T) { test_env_file_backup_missing :: proc(t: ^testing.T) {
d, ok := db_init() f := EnvFile {
testing.expect(t, ok, "failed to create test db") Path = "/tmp/envr-nonexistent-backup/.env",
defer db_close(&d)
f := make_test_env_file("/nonexistent/path/.env", "sha", "KEY=val")
db_insert(&d, f)
result, err := db_sync(&d, &f)
testing.expect(t, err == .DirMissing, "should return DirMissing error")
}
@(test)
test_db_sync_moved :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-sync-moved-%d", os.get_pid())
search_root := fmt.tprintf("%s/search", base)
repo_dir := fmt.tprintf("%s/myproject", search_root)
git_dir := fmt.tprintf("%s/.git", repo_dir)
defer os.remove_all(base)
os.mkdir_all(git_dir)
config_content := "[remote \"origin\"]\n\turl = git@github.com:user/repo.git\n"
config_path := fmt.tprintf("%s/config", git_dir)
write_err := os.write_entire_file(config_path, transmute([]u8)config_content)
testing.expect(t, write_err == nil, "should write .git/config")
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
defer db_close(&d)
d.cfg.ScanConfig.Include = make([dynamic]string, 0, 1, context.temp_allocator)
append(&d.cfg.ScanConfig.Include, search_root)
f := make_test_env_file(
"/old/nonexistent/path/.env",
"some_sha",
"SECRET=value",
[]string{"git@github.com:user/repo.git"},
)
testing.expect(t, db_insert(&d, f), "insert should succeed")
result, err := db_sync(&d, &f)
testing.expect(t, err == .None, "sync should not error")
if err != .None do return
testing.expect(t, .DirUpdated in result, "should have DirUpdated flag")
testing.expect(t, .Restored in result, "should have Restored flag")
expected_path := fmt.tprintf("%s/.env", repo_dir)
testing.expect_value(t, f.Path, expected_path)
testing.expect_value(t, f.Dir, repo_dir)
_, old_exists := db_fetch(&d, "/old/nonexistent/path/.env")
testing.expect(t, !old_exists, "old path should be deleted from db")
new_fetched, new_ok := db_fetch(&d, expected_path)
testing.expect(t, new_ok, "new path should exist in db")
if new_ok {
testing.expect_value(t, new_fetched.contents, "SECRET=value")
} }
testing.expect(t, !env_file_backup(&f), "missing file should return false")
}
@(test)
test_update_dir :: proc(t: ^testing.T) {
f := EnvFile {
Path = "/old/project/.env",
Dir = "/old/project",
Remotes = make([dynamic]string, 0),
}
defer delete_envfile(&f)
update_dir(&f, "/new/location")
testing.expect_value(t, f.Dir, "/new/location")
testing.expect_value(t, f.Path, "/new/location/.env")
} }

View File

@@ -45,6 +45,7 @@ at before, restore your backup with:
* [envr backup](envr_backup.md) - Import a .env file into envr * [envr backup](envr_backup.md) - Import a .env file into envr
* [envr check](envr_check.md) - check if files in the current directory are backed up * [envr check](envr_check.md) - check if files in the current directory are backed up
* [envr deps](envr_deps.md) - Check for missing binaries
* [envr edit-config](envr_edit-config.md) - Edit your config with your default editor * [envr edit-config](envr_edit-config.md) - Edit your config with your default editor
* [envr init](envr_init.md) - Set up envr * [envr init](envr_init.md) - Set up envr
* [envr list](envr_list.md) - View your tracked files * [envr list](envr_list.md) - View your tracked files

24
docs/cli/envr_deps.md Normal file
View File

@@ -0,0 +1,24 @@
## envr deps
Check for missing binaries
### Synopsis
envr relies on external binaries for certain functionality.
The check command reports on which binaries are available and which are not.
```
envr deps [flags]
```
### Options
```
-h, --help help for deps
```
### SEE ALSO
* [envr](envr.md) - Manage your .env files.

51
features.odin Normal file
View File

@@ -0,0 +1,51 @@
package main
import "base:runtime"
import "core:mem"
import "core:os"
import "core:strings"
Feature :: enum {
Git,
Fd,
}
AvailableFeatures :: bit_set[Feature]
check_features :: proc() -> AvailableFeatures {
feats: AvailableFeatures
s: mem.Scratch
mem.scratch_init(&s, 4 * mem.DEFAULT_PAGE_SIZE)
defer mem.scratch_destroy(&s)
context.temp_allocator = mem.scratch_allocator(&s)
path_env := os.get_env("PATH", context.temp_allocator)
paths := strings.split(path_env, ":", context.temp_allocator)
if find_binary(paths, "git") != "" {
feats += {.Git}
}
if find_binary(paths, "fd") != "" {
feats += {.Fd}
}
return feats
}
find_binary :: proc(
paths: []string,
name: string,
allocator: runtime.Allocator = context.temp_allocator,
) -> string {
for p in paths {
candidate := strings.join({strings.trim_right(p, "/"), name}, "/", allocator)
_, err := os.stat(candidate, allocator)
if err == nil {
return candidate
}
}
return ""
}

34
features_test.odin Normal file
View File

@@ -0,0 +1,34 @@
package main
import "core:os"
import "core:strings"
import "core:testing"
@(test)
test_find_binary_exists :: proc(t: ^testing.T) {
path := os.get_env("PATH", context.temp_allocator)
paths := strings.split(path, ":", context.temp_allocator)
result := find_binary(paths, "sh")
testing.expect(t, result != "", "sh should be found on PATH")
}
@(test)
test_find_binary_not_exists :: proc(t: ^testing.T) {
old_path := os.get_env("PATH", context.temp_allocator)
defer {
if old_path != "" {
os.set_env("PATH", old_path)
}
}
os.set_env("PATH", "/tmp/envr-nope")
path := os.get_env("PATH", context.temp_allocator)
paths := strings.split(path, ":", context.temp_allocator)
result := find_binary(paths, "no_such_binary_xyz")
testing.expect(t, result == "", "nonexistent binary should not be found")
}

View File

@@ -1,320 +0,0 @@
package findr
import "core:os"
import "core:sort"
import "core:strings"
import "core:sys/linux"
import "core:testing"
// ============================================================================
// Gitignored file emission tests (emit ONLY gitignored files, descend everywhere)
// ============================================================================
@(test)
test_basic_gitignored :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n")
create_file(env, "repo/.env")
create_file(env, "repo/secrets.env")
create_file(env, "repo/normal.txt")
assert_output(t, env, nil, {}, {
"repo/.env", "repo/secrets.env",
})
}
@(test)
test_non_repo_not_scanned :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_dir(env, "norepo")
create_file(env, "norepo/.gitignore", "*.env\n")
create_file(env, "norepo/.env")
assert_output_empty(t, env, nil, {})
}
@(test)
test_negation_pattern :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n!prod.env\n")
create_file(env, "repo/.env")
create_file(env, "repo/secrets.env")
create_file(env, "repo/prod.env")
assert_output(t, env, nil, {}, {
"repo/.env", "repo/secrets.env",
})
}
@(test)
test_multiple_repos :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo1")
create_file(env, "repo1/.gitignore", "*.env\n")
create_file(env, "repo1/a.env")
create_git_repo(env, "repo2")
create_file(env, "repo2/.gitignore", "*.key\n")
create_file(env, "repo2/secret.key")
assert_output(t, env, nil, {}, {
"repo1/a.env", "repo2/secret.key",
})
}
@(test)
test_nested_repos :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "parent")
create_file(env, "parent/.gitignore", "*.env\n")
create_file(env, "parent/top.env")
create_git_repo(env, "parent/child")
create_file(env, "parent/child/.gitignore", "*.key\n")
create_file(env, "parent/child/api.key")
assert_output(t, env, nil, {}, {
"parent/top.env", "parent/child/api.key",
})
}
@(test)
test_nested_gitignore_read :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n")
create_dir(env, "repo/sub")
create_file(env, "repo/sub/.gitignore", "*.txt\n")
create_file(env, "repo/sub/secret.txt")
create_file(env, "repo/sub/.env")
assert_output(t, env, nil, {}, {
"repo/sub/secret.txt", "repo/sub/.env",
})
}
@(test)
test_nested_gitignore_negation :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.log\n")
create_dir(env, "repo/sub")
create_file(env, "repo/sub/.gitignore", "!important.log\n")
create_file(env, "repo/sub/important.log")
create_file(env, "repo/sub/debug.log")
assert_output(t, env, nil, {}, {
"repo/sub/debug.log",
})
}
@(test)
test_multisegment_pattern :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "build/output.txt\n")
create_dir(env, "repo/build")
create_file(env, "repo/build/output.txt")
create_file(env, "repo/build/other.txt")
create_file(env, "repo/output.txt")
assert_output(t, env, nil, {}, {
"repo/build/output.txt",
})
}
@(test)
test_no_gitignore_file :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.env")
assert_output_empty(t, env, nil, {})
}
@(test)
test_empty_gitignore :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "\n\n# comment\n\n")
create_file(env, "repo/.env")
assert_output_empty(t, env, nil, {})
}
@(test)
test_multiple_search_dirs :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "dir1/repo")
create_file(env, "dir1/repo/.gitignore", "*.env\n")
create_file(env, "dir1/repo/a.env")
create_file(env, "dir1/repo/normal.txt")
create_git_repo(env, "dir2/repo")
create_file(env, "dir2/repo/.gitignore", "*.env\n")
create_file(env, "dir2/repo/b.env")
dir1 := join_path(env.temp_dir, "dir1")
defer delete(dir1)
dir2 := join_path(env.temp_dir, "dir2")
defer delete(dir2)
results := make([dynamic]string)
defer {
for r in results {delete(r)}
delete(results)
}
opts := WalkOptions{}
thread_count := os.get_processor_core_count()
walk({dir1, dir2}, &results, opts, thread_count)
testing.expect_value(t, len(results), 2)
actual := make([dynamic]string, 0, len(results))
for r in results {
stripped := r
if strings.has_prefix(stripped, env.temp_dir) {
stripped = stripped[len(env.temp_dir):]
if len(stripped) > 0 && stripped[0] == '/' {
stripped = stripped[1:]
}
}
append(&actual, stripped)
}
defer delete(actual)
expected := []string{"dir1/repo/a.env", "dir2/repo/b.env"}
sort.quick_sort(actual[:])
sort.quick_sort(expected[:])
for i in 0 ..< len(expected) {
testing.expect_value(t, actual[i], expected[i])
}
}
// ============================================================================
// Ignored directory recursion tests
// ============================================================================
@(test)
test_ignored_dir_descended :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "secrets/\n")
create_dir(env, "repo/secrets")
create_file(env, "repo/secrets/.env")
create_file(env, "repo/secrets/api.key")
// Ignored dir's contents are emitted AND descended into
assert_output(t, env, nil, {}, {
"repo/secrets/", "repo/secrets/.env", "repo/secrets/api.key",
})
}
@(test)
test_nested_ignored_dir :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "build/\n")
create_dir(env, "repo/build")
create_dir(env, "repo/build/sub")
create_file(env, "repo/build/output.txt")
create_file(env, "repo/build/sub/deep.env")
assert_output(t, env, nil, {}, {
"repo/build/", "repo/build/output.txt",
"repo/build/sub/", "repo/build/sub/deep.env",
})
}
// ============================================================================
// Filter tests (excludes, pattern)
// ============================================================================
@(test)
test_excludes_prune_dirs :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n")
create_file(env, "repo/.env")
create_dir(env, "repo/vendor")
create_file(env, "repo/vendor/lib.env")
assert_output(t, env, nil,
{excludes = {"vendor"}},
{"repo/.env"},
)
}
@(test)
test_pattern_filters_results :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n*.key\n")
create_file(env, "repo/.env")
create_file(env, "repo/secrets.env")
create_file(env, "repo/master.key")
assert_output(t, env, nil,
{pattern = "\\.env$"},
{"repo/.env", "repo/secrets.env"},
)
}
// ============================================================================
// Special file type tests
// ============================================================================
@(test)
test_fifo_emitted :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n*.fifo\n")
fifo_path := join_path(env.temp_dir, "repo/test.fifo")
defer delete(fifo_path)
cpath := strings.clone_to_cstring(fifo_path)
defer delete(cpath)
linux.mknod(cpath, linux.S_IFIFO | linux.Mode{.IRUSR, .IWUSR}, 0)
assert_output(t, env, nil,
{pattern = "\\.fifo$"},
{"repo/test.fifo"},
)
}

View File

@@ -1,88 +0,0 @@
package findr
import "core:strings"
Gitignore :: struct {
rules: [dynamic]Rule,
}
Rule :: struct {
pattern: GlobPattern,
negated: bool,
dir_only: bool,
}
Match :: enum {
None,
Ignored,
Unignored,
}
is_ignored :: proc(gi: ^Gitignore, path: string, is_dir: bool) -> bool {
return check_match(gi, path, is_dir) == .Ignored
}
check_match :: proc(gi: ^Gitignore, path: string, is_dir: bool) -> Match {
result := Match.None
for &rule in gi.rules {
if rule.dir_only && !is_dir do continue
if glob_match_compiled(&rule.pattern, path) {
result = rule.negated ? .Unignored : .Ignored
}
}
return result
}
parse :: proc(content: string) -> Gitignore {
gi: Gitignore
gi.rules = make([dynamic]Rule)
remaining := content
for {
line, ok := strings.split_lines_iterator(&remaining)
if !ok do break
s := strings.trim_space(line)
if len(s) == 0 do continue
if s[0] == '#' do continue
negated := false
if s[0] == '!' {
negated = true
s = s[1:]
}
if len(s) > 0 && s[0] == '\\' {
if len(s) > 1 && (s[1] == '#' || s[1] == '!') {
s = s[1:]
}
}
dir_only := false
if len(s) > 0 && s[len(s) - 1] == '/' {
dir_only = true
s = s[:len(s) - 1]
}
anchored := false
if len(s) > 0 && s[0] == '/' {
anchored = true
s = s[1:]
}
if len(s) == 0 do continue
gp := glob_compile(s, anchored)
append(&gi.rules, Rule{pattern = gp, negated = negated, dir_only = dir_only})
}
return gi
}
destroy :: proc(gi: ^Gitignore) {
for &rule in gi.rules {
glob_destroy(&rule.pattern)
}
delete(gi.rules)
}

View File

@@ -1,118 +0,0 @@
package findr
import "core:testing"
@(test)
test_is_ignored_basic :: proc(t: ^testing.T) {
gi := parse("*.env\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, ".env", false), true)
testing.expect_value(t, is_ignored(&gi, "foo.env", false), true)
testing.expect_value(t, is_ignored(&gi, ".env.local", false), false)
testing.expect_value(t, is_ignored(&gi, "config.yaml", false), false)
}
@(test)
test_is_ignored_negation :: proc(t: ^testing.T) {
gi := parse("*.env\n!.env.production\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, ".env", false), true)
testing.expect_value(t, is_ignored(&gi, ".env.production", false), false)
}
@(test)
test_is_ignored_dir_only :: proc(t: ^testing.T) {
gi := parse("node_modules/\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, "node_modules", true), true)
testing.expect_value(t, is_ignored(&gi, "node_modules", false), false)
}
@(test)
test_is_ignored_anchored :: proc(t: ^testing.T) {
gi := parse("/secret.key\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, "secret.key", false), true)
}
@(test)
test_is_ignored_comments_skipped :: proc(t: ^testing.T) {
gi := parse("# this is a comment\n#another\n*.tmp\n")
defer destroy(&gi)
testing.expect_value(t, len(gi.rules), 1)
testing.expect_value(t, is_ignored(&gi, "file.tmp", false), true)
}
@(test)
test_is_ignored_blank_lines_skipped :: proc(t: ^testing.T) {
gi := parse("\n\n \n*.log\n\n")
defer destroy(&gi)
testing.expect_value(t, len(gi.rules), 1)
}
@(test)
test_is_ignored_last_match_wins :: proc(t: ^testing.T) {
gi := parse("*.env\n!*.env\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, ".env", false), false)
}
@(test)
test_is_ignored_no_rules :: proc(t: ^testing.T) {
gi := parse("")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, "anything", false), false)
}
@(test)
test_is_ignored_env_pattern :: proc(t: ^testing.T) {
gi := parse(".env*\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, ".env", false), true)
testing.expect_value(t, is_ignored(&gi, ".env.local", false), true)
testing.expect_value(t, is_ignored(&gi, ".envrc", false), true)
}
@(test)
test_is_ignored_globstar :: proc(t: ^testing.T) {
gi := parse("**/cache\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, "cache", false), true)
testing.expect_value(t, is_ignored(&gi, "foo/cache", false), true)
testing.expect_value(t, is_ignored(&gi, "foo/bar/cache", false), true)
}
@(test)
test_star_negation_subpath :: proc(t: ^testing.T) {
gi := parse("*\n!public/\n")
defer destroy(&gi)
// public dir itself is un-ignored
testing.expect_value(t, is_ignored(&gi, "public", true), false)
// children of public/ should still be ignored by *
testing.expect_value(t, is_ignored(&gi, "public/uuid-dir", true), true)
testing.expect_value(t, is_ignored(&gi, "public/uuid-dir/file.txt", false), true)
}
@(test)
test_is_ignored_hash_pattern :: proc(t: ^testing.T) {
gi := parse("\\#*\\#\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, "#foo#", false), true)
testing.expect_value(t, is_ignored(&gi, "#test#", false), true)
testing.expect_value(t, is_ignored(&gi, "AUTHORS", false), false)
testing.expect_value(t, is_ignored(&gi, "build.zig", false), false)
testing.expect_value(t, is_ignored(&gi, "ChangeLog", false), false)
}

View File

@@ -1,203 +0,0 @@
package findr
Range :: struct {
lo: u8,
hi: u8,
}
Class_Data :: struct {
negated: bool,
ranges: [dynamic]Range,
}
Token_Kind :: enum u8 { Char, Star, Globstar, Question, Class }
Token :: struct {
kind: Token_Kind,
byte: u8,
class_idx: u16,
}
GlobPattern :: struct {
tokens: [dynamic]Token,
classes: [dynamic]Class_Data,
anchored: bool,
}
glob_compile :: proc(pattern: string, anchored: bool) -> GlobPattern {
gp: GlobPattern
gp.tokens = make([dynamic]Token)
gp.classes = make([dynamic]Class_Data)
gp.anchored = anchored
i := 0
for i < len(pattern) {
c := pattern[i]
if c == '*' {
if i + 1 < len(pattern) && pattern[i + 1] == '*' {
prev_slash := i == 0 || pattern[i - 1] == '/'
at_end := i + 2 >= len(pattern)
next_slash := !at_end && pattern[i + 2] == '/'
if prev_slash && (next_slash || at_end) {
append(&gp.tokens, Token{kind = .Globstar})
if next_slash {
i += 3
} else {
i += 2
}
} else {
append(&gp.tokens, Token{kind = .Star})
i += 2
}
} else {
append(&gp.tokens, Token{kind = .Star})
i += 1
}
} else if c == '?' {
append(&gp.tokens, Token{kind = .Question})
i += 1
} else if c == '[' {
i += 1
negated := false
if i < len(pattern) && pattern[i] == '!' {
negated = true
i += 1
}
ranges := make([dynamic]Range)
if i < len(pattern) && pattern[i] == ']' {
append(&ranges, Range{lo = ']', hi = ']'})
i += 1
}
for i < len(pattern) && pattern[i] != ']' {
if i + 2 < len(pattern) && pattern[i + 1] == '-' && pattern[i + 2] != ']' {
append(&ranges, Range{lo = pattern[i], hi = pattern[i + 2]})
i += 3
} else {
append(&ranges, Range{lo = pattern[i], hi = pattern[i]})
i += 1
}
}
if i < len(pattern) {
i += 1
}
class_idx := u16(len(gp.classes))
append(&gp.classes, Class_Data{negated = negated, ranges = ranges})
append(&gp.tokens, Token{kind = .Class, class_idx = class_idx})
} else if c == '\\' {
i += 1
if i < len(pattern) {
append(&gp.tokens, Token{kind = .Char, byte = pattern[i]})
i += 1
}
} else {
append(&gp.tokens, Token{kind = .Char, byte = c})
i += 1
}
}
return gp
}
match_tokens :: proc(tokens: []Token, classes: []Class_Data, ti: int, path: string, pi: int) -> bool {
if ti >= len(tokens) {
return pi == len(path)
}
tok := tokens[ti]
switch tok.kind {
case .Char:
if pi < len(path) && path[pi] == tok.byte {
return match_tokens(tokens, classes, ti + 1, path, pi + 1)
}
return false
case .Question:
if pi < len(path) && path[pi] != '/' {
return match_tokens(tokens, classes, ti + 1, path, pi + 1)
}
return false
case .Star:
max_end := pi
for max_end < len(path) && path[max_end] != '/' {
max_end += 1
}
for end := max_end; end >= pi; end -= 1 {
if match_tokens(tokens, classes, ti + 1, path, end) {
return true
}
}
return false
case .Globstar:
if ti + 1 >= len(tokens) {
return true
}
if match_tokens(tokens, classes, ti + 1, path, pi) {
return true
}
for end := pi + 1; end <= len(path); end += 1 {
if path[end - 1] == '/' {
if match_tokens(tokens, classes, ti + 1, path, end) {
return true
}
}
}
return false
case .Class:
if pi >= len(path) {
return false
}
cd := classes[tok.class_idx]
ch := path[pi]
in_range := false
for r in cd.ranges {
if ch >= r.lo && ch <= r.hi {
in_range = true
break
}
}
if in_range != cd.negated {
return match_tokens(tokens, classes, ti + 1, path, pi + 1)
}
return false
}
return false
}
glob_match_compiled :: proc(gp: ^GlobPattern, path: string) -> bool {
tokens := gp.tokens[:]
classes := gp.classes[:]
if gp.anchored {
return match_tokens(tokens, classes, 0, path, 0)
}
if match_tokens(tokens, classes, 0, path, 0) {
return true
}
for i := 1; i < len(path); i += 1 {
if path[i - 1] == '/' {
if match_tokens(tokens, classes, 0, path, i) {
return true
}
}
}
return false
}
glob_destroy :: proc(gp: ^GlobPattern) {
for &cd in gp.classes {
delete(cd.ranges)
}
delete(gp.classes)
delete(gp.tokens)
}

View File

@@ -1,111 +0,0 @@
package findr
import "core:testing"
glob_match :: proc(pattern: string, path: string, anchored: bool) -> bool {
gp := glob_compile(pattern, anchored)
result := glob_match_compiled(&gp, path)
glob_destroy(&gp)
return result
}
@(test)
test_glob_simple :: proc(t: ^testing.T) {
testing.expect(t, glob_match("foo", "foo", false))
testing.expect(t, glob_match("foo", "bar/foo", false))
testing.expect(t, !glob_match("foo", "foobar", false))
testing.expect(t, !glob_match("foo", "foo/bar", false))
}
@(test)
test_glob_anchored :: proc(t: ^testing.T) {
testing.expect(t, glob_match("foo", "foo", true))
testing.expect(t, !glob_match("foo", "bar/foo", true))
testing.expect(t, !glob_match("foo", "foobar", true))
}
@(test)
test_glob_star :: proc(t: ^testing.T) {
testing.expect(t, glob_match("*.log", "test.log", false))
testing.expect(t, glob_match("*.log", ".log", false))
testing.expect(t, !glob_match("*.log", "test.txt", false))
testing.expect(t, !glob_match("*.log", "dir/test", false))
}
@(test)
test_glob_question :: proc(t: ^testing.T) {
testing.expect(t, glob_match("?.log", "a.log", false))
testing.expect(t, !glob_match("?.log", "ab.log", false))
testing.expect(t, !glob_match("?.log", ".log", false))
}
@(test)
test_glob_char_class :: proc(t: ^testing.T) {
testing.expect(t, glob_match("[abc].log", "a.log", false))
testing.expect(t, glob_match("[abc].log", "b.log", false))
testing.expect(t, !glob_match("[abc].log", "d.log", false))
}
@(test)
test_glob_negated_class :: proc(t: ^testing.T) {
testing.expect(t, glob_match("[!abc].log", "d.log", false))
testing.expect(t, !glob_match("[!abc].log", "a.log", false))
}
@(test)
test_glob_dot_literal :: proc(t: ^testing.T) {
testing.expect(t, glob_match(".env", ".env", false))
testing.expect(t, glob_match(".env", "dir/.env", false))
testing.expect(t, !glob_match(".env", "env", false))
testing.expect(t, !glob_match(".env", "x.env", false))
}
@(test)
test_glob_globstar_prefix :: proc(t: ^testing.T) {
testing.expect(t, glob_match("**/foo", "foo", false))
testing.expect(t, glob_match("**/foo", "a/b/foo", false))
testing.expect(t, !glob_match("**/foo", "foobar", false))
testing.expect(t, !glob_match("**/foo", "a/foobar", false))
}
@(test)
test_glob_globstar_suffix :: proc(t: ^testing.T) {
testing.expect(t, glob_match("abc/**", "abc/x", false))
testing.expect(t, glob_match("abc/**", "abc/x/y", false))
testing.expect(t, !glob_match("abc/**", "abc", false))
testing.expect(t, !glob_match("abc/**", "abcd/x", false))
}
@(test)
test_glob_globstar_middle :: proc(t: ^testing.T) {
testing.expect(t, glob_match("foo/**/bar", "foo/bar", false))
testing.expect(t, glob_match("foo/**/bar", "foo/x/bar", false))
testing.expect(t, !glob_match("foo/**/bar", "foo/barx", false))
testing.expect(t, !glob_match("foo/**/bar", "foo/x/y/baz", false))
}
@(test)
test_glob_backslash_escape :: proc(t: ^testing.T) {
testing.expect(t, glob_match("\\!foo", "!foo", false))
testing.expect(t, !glob_match("\\!foo", "foo", false))
}
@(test)
test_glob_hash_literal :: proc(t: ^testing.T) {
testing.expect(t, glob_match("#foo", "#foo", false))
testing.expect(t, !glob_match("#foo", "foo", false))
}
@(test)
test_glob_hash_pattern :: proc(t: ^testing.T) {
testing.expect(t, glob_match("#*#", "#test#", false))
testing.expect(t, glob_match("#*#", "##", false))
testing.expect(t, !glob_match("#*#", "test", false))
testing.expect(t, !glob_match("#*#", "#test", false))
}
@(test)
test_glob_empty :: proc(t: ^testing.T) {
testing.expect(t, glob_match("", "", false))
testing.expect(t, !glob_match("", "foo", false))
}

View File

@@ -1,128 +0,0 @@
package findr
import "core:strings"
import "core:sync"
import "core:sys/linux"
import "core:thread"
RepoPool :: struct {
queue: [dynamic]string,
queue_mutex: sync.Mutex,
queue_sema: sync.Atomic_Sema,
results: ^[dynamic]string,
results_lock: sync.Mutex,
active: i64,
done: sync.One_Shot_Event,
threads: []^thread.Thread,
}
find_repos :: proc(roots: []string, results: ^[dynamic]string, thread_count: int) {
if len(roots) == 0 do return
pool := new(RepoPool)
pool.queue = make([dynamic]string)
pool.results = results
pool.active = i64(len(roots))
pool.threads = make([]^thread.Thread, thread_count)
for root in roots {
root_clone, _ := strings.clone(root)
append(&pool.queue, root_clone)
sync.atomic_sema_post(&pool.queue_sema)
}
for i in 0 ..< thread_count {
t := thread.create(repo_worker)
t.data = rawptr(pool)
t.init_context = context
thread.start(t)
pool.threads[i] = t
}
sync.one_shot_event_wait(&pool.done)
for _ in 0 ..< thread_count {
sync.atomic_sema_post(&pool.queue_sema)
}
for t in pool.threads {
thread.destroy(t)
}
delete(pool.threads)
for path in pool.queue {
delete(path)
}
delete(pool.queue)
free(pool)
}
repo_worker :: proc(t: ^thread.Thread) {
pool := cast(^RepoPool)t.data
for {
sync.atomic_sema_wait(&pool.queue_sema)
sync.mutex_lock(&pool.queue_mutex)
if len(pool.queue) == 0 {
sync.mutex_unlock(&pool.queue_mutex)
if sync.atomic_load_explicit(&pool.active, .Acquire) == 0 {
sync.one_shot_event_signal(&pool.done)
}
break
}
last := len(pool.queue) - 1
dir_path := pool.queue[last]
ordered_remove(&pool.queue, last)
sync.mutex_unlock(&pool.queue_mutex)
process_repo_dir(pool, dir_path)
delete(dir_path)
old := sync.atomic_sub_explicit(&pool.active, 1, .Release)
if old == 1 {
sync.one_shot_event_signal(&pool.done)
}
}
}
process_repo_dir :: proc(pool: ^RepoPool, dir_path: string) {
cpath := strings.clone_to_cstring(dir_path)
if cpath == nil do return
defer delete(cpath)
fd, open_err := linux.open(cpath, {.DIRECTORY, .CLOEXEC})
if open_err != .NONE do return
defer linux.close(fd)
if has_git_dir(fd) {
cloned, _ := strings.clone(dir_path)
sync.mutex_lock(&pool.results_lock)
append(pool.results, cloned)
sync.mutex_unlock(&pool.results_lock)
}
buf: [32 * 1024]u8
for {
n, errno := linux.getdents(fd, buf[:])
if n <= 0 || errno != .NONE do break
offs := 0
for d in linux.dirent_iterate_buf(buf[:n], &offs) {
name := linux.dirent_name(d)
if name == "." || name == ".." do continue
if name == ".git" do continue
if d.type == .DIR {
child_path := join_path(dir_path, name)
sync.atomic_add_explicit(&pool.active, 1, .Relaxed)
sync.mutex_lock(&pool.queue_mutex)
append(&pool.queue, child_path)
sync.mutex_unlock(&pool.queue_mutex)
sync.atomic_sema_post(&pool.queue_sema)
}
}
}
}

View File

@@ -1,152 +0,0 @@
package findr
import "core:fmt"
import "core:log"
import "core:os"
import "core:sort"
import "core:strings"
import "core:testing"
TestEnv :: struct {
temp_dir: string,
}
create_test_env :: proc() -> (env: TestEnv) {
tmp, err := os.mkdir_temp("", "findr-test-*", context.allocator)
if err != nil {
log.error("Failed to create temp dir:", err)
panic("Failed to create temp dir")
}
env.temp_dir = tmp
return
}
destroy_test_env :: proc(env: ^TestEnv) {
os.remove_all(env.temp_dir)
delete(env.temp_dir)
}
create_dir :: proc(env: TestEnv, path: string) {
full := join_path(env.temp_dir, path)
defer delete(full)
os.mkdir_all(full, os.Permissions_Default_Directory)
}
create_file :: proc(env: TestEnv, path: string, content: string = "") {
full := join_path(env.temp_dir, path)
defer delete(full)
dir_end := strings.last_index(full, "/")
if dir_end >= 0 {
dir_path := full[:dir_end]
os.mkdir_all(dir_path, os.Permissions_Default_Directory)
}
f, err := os.create(full)
if err != nil {
log.error("Failed to create file:", full, err)
return
}
if len(content) > 0 {
os.write_string(f, content)
}
os.close(f)
}
create_git_repo :: proc(env: TestEnv, path: string) {
sub := join_path(path, ".git")
defer delete(sub)
create_dir(env, sub)
}
assert_output :: proc(
t: ^testing.T,
env: TestEnv,
args: []string,
opts: WalkOptions,
expected: []string,
) {
results := collect_results(env, args, opts)
defer {
for r in results {delete(r)}
delete(results)
}
sorted_expected := make([dynamic]string, 0, len(expected))
for e in expected {append(&sorted_expected, e)}
defer delete(sorted_expected)
sorted_actual := make([dynamic]string, 0, len(results))
for a in results {append(&sorted_actual, a)}
defer delete(sorted_actual)
sort.quick_sort(sorted_expected[:])
sort.quick_sort(sorted_actual[:])
if len(sorted_expected) != len(sorted_actual) {
testing.fail(t)
log.error(
fmt.tprintf("Expected %d results, got %d", len(sorted_expected), len(sorted_actual)),
)
log.error("Expected:", sorted_expected[:])
log.error("Actual: ", sorted_actual[:])
return
}
for i in 0 ..< len(sorted_expected) {
if sorted_expected[i] != sorted_actual[i] {
testing.fail(t)
log.error(fmt.tprintf("Mismatch at index %d", i))
log.error("Expected:", sorted_expected[:])
log.error("Actual: ", sorted_actual[:])
return
}
}
}
assert_output_empty :: proc(
t: ^testing.T,
env: TestEnv,
args: []string,
opts: WalkOptions,
) {
results := collect_results(env, args, opts)
defer {
for r in results {delete(r)}
delete(results)
}
if len(results) > 0 {
testing.fail(t)
log.error(fmt.tprintf("Expected no results, got %d:", len(results)))
for r in results {
log.error(" ", r)
}
}
}
collect_results :: proc(env: TestEnv, args: []string, opts: WalkOptions) -> [dynamic]string {
results := make([dynamic]string)
full_args := make([dynamic]string, 0, len(args) + 1, context.temp_allocator)
append(&full_args, env.temp_dir)
for a in args {append(&full_args, a)}
thread_count := os.get_processor_core_count()
walk(full_args[:], &results, opts, thread_count)
for i in 0 ..< len(results) {
r := results[i]
if strings.has_prefix(r, env.temp_dir) {
stripped := r[len(env.temp_dir):]
if len(stripped) > 0 && stripped[0] == '/' {
stripped = stripped[1:]
}
new_r, _ := strings.clone(stripped)
delete(r)
results[i] = new_r
}
}
return results
}

View File

@@ -1,458 +0,0 @@
package findr
import "core:bytes"
import "core:fmt"
import "core:os"
import "core:strings"
import "core:sync"
import "core:sync/chan"
import "core:sys/linux"
import "core:text/regex"
import "core:thread"
OUTPUT_BUF_SIZE :: 64 * 1024
WalkOptions :: struct {
pattern: string, // regex on basename; "" = match all
excludes: []string, // glob patterns to skip entirely
}
GIContext :: struct {
gi: ^Gitignore, // nil if this dir had no .gitignore
base_rel: string, // relative path from repo root to this dir
parent: ^GIContext, // parent context (nil if repo root)
}
WorkItem :: struct {
path: string, // absolute directory path
rel: string, // relative path from repo root ("" = root)
gi_ctx: ^GIContext, // gitignore chain (nil = outside any repo)
in_repo: bool, // true if inside a git repo
in_ignored: bool, // true if inside a gitignored directory
}
WalkerPool :: struct {
queue: [dynamic]WorkItem,
queue_mutex: sync.Mutex,
queue_sema: sync.Atomic_Sema,
result_chan: chan.Chan([]u8),
active: i64,
done: sync.One_Shot_Event,
threads: []^thread.Thread,
opts: WalkOptions,
pattern_re: regex.Regular_Expression,
has_pattern: bool,
exclude_gi: ^Gitignore,
all_contexts: [dynamic]^GIContext,
contexts_lock: sync.Mutex,
}
Collector_Data :: struct {
ch: chan.Chan([]u8),
results: ^[dynamic]string,
}
collect_worker :: proc(t: ^thread.Thread) {
data := cast(^Collector_Data)t.data
for {
batch := chan.recv(data.ch) or_break
defer delete(batch)
start := 0
for {
remaining: []u8
#no_bounds_check {remaining = batch[start:]}
idx := bytes.index_byte(remaining, '\n')
if idx < 0 do break
i := start + idx
if i > start {
segment: []u8
#no_bounds_check {segment = batch[start:i]}
s, _ := strings.clone(string(segment))
append(data.results, s)
}
start = i + 1
}
}
}
walk :: proc(roots: []string, results: ^[dynamic]string, opts: WalkOptions, thread_count: int) {
if len(roots) == 0 do return
ch, _ := chan.create(chan.Chan([]u8), max(2 * thread_count, 2), context.allocator)
defer chan.destroy(ch)
data := new(Collector_Data)
data.ch = ch
data.results = results
defer free(data)
collector := thread.create(collect_worker)
collector.data = rawptr(data)
collector.init_context = context
thread.start(collector)
pool := new(WalkerPool)
pool.queue = make([dynamic]WorkItem)
pool.result_chan = ch
pool.active = i64(len(roots))
pool.threads = make([]^thread.Thread, thread_count)
pool.all_contexts = make([dynamic]^GIContext)
pool.opts = opts
pool.exclude_gi = nil
pool.has_pattern = false
if len(opts.pattern) > 0 {
re, err := regex.create(opts.pattern, {regex.Flag.No_Capture})
if err == nil {
pool.pattern_re = re
pool.has_pattern = true
}
}
if len(opts.excludes) > 0 {
sb: strings.Builder
strings.builder_init(&sb)
for ex in opts.excludes {
fmt.sbprintf(&sb, "%s\n", ex)
}
content := strings.to_string(sb)
pool.exclude_gi = new(Gitignore)
pool.exclude_gi^ = parse(content)
strings.builder_destroy(&sb)
}
for root in roots {
root_clone, _ := strings.clone(root)
append(&pool.queue, WorkItem{path = root_clone})
sync.atomic_sema_post(&pool.queue_sema)
}
for i in 0 ..< thread_count {
t := thread.create(walk_worker)
t.data = rawptr(pool)
t.init_context = context
thread.start(t)
pool.threads[i] = t
}
sync.one_shot_event_wait(&pool.done)
for _ in 0 ..< thread_count {
sync.atomic_sema_post(&pool.queue_sema)
}
for t in pool.threads {
thread.destroy(t)
}
delete(pool.threads)
for item in pool.queue {
delete(item.path)
if len(item.rel) > 0 {delete(item.rel)}
}
delete(pool.queue)
for ctx in pool.all_contexts {
if ctx.gi != nil {
destroy(ctx.gi)
free(ctx.gi)
}
if len(ctx.base_rel) > 0 {
delete(ctx.base_rel)
}
free(ctx)
}
delete(pool.all_contexts)
if pool.has_pattern {
regex.destroy(pool.pattern_re)
}
if pool.exclude_gi != nil {
destroy(pool.exclude_gi)
free(pool.exclude_gi)
}
free(pool)
chan.close(ch)
thread.join(collector)
thread.destroy(collector)
}
flush_buf :: proc(ch: chan.Chan([]u8), local: ^[dynamic]u8) {
if len(local) == 0 do return
batch := local[:]
local^ = make([dynamic]u8, 0, OUTPUT_BUF_SIZE)
chan.send(ch, batch)
}
append_path :: proc(buf: ^[dynamic]u8, parent, name: string, trailing_slash: bool) {
need_sep := len(parent) > 0 && parent[len(parent) - 1] != '/'
size := len(parent) + len(name) + 1
if need_sep do size += 1
if trailing_slash do size += 1
old_len := len(buf)
reserve(buf, old_len + size)
resize(buf, old_len + size)
pos := old_len
pos += copy(buf[pos:], parent)
if need_sep {buf[pos] = '/'; pos += 1}
pos += copy(buf[pos:], name)
if trailing_slash {buf[pos] = '/'; pos += 1}
buf[pos] = '\n'
}
walk_worker :: proc(t: ^thread.Thread) {
pool := cast(^WalkerPool)t.data
local_buf := make([dynamic]u8, 0, OUTPUT_BUF_SIZE)
defer {
if len(local_buf) > 0 {
flush_buf(pool.result_chan, &local_buf)
}
delete(local_buf)
}
for {
sync.atomic_sema_wait(&pool.queue_sema)
sync.mutex_lock(&pool.queue_mutex)
if len(pool.queue) == 0 {
sync.mutex_unlock(&pool.queue_mutex)
if sync.atomic_load_explicit(&pool.active, .Acquire) == 0 {
sync.one_shot_event_signal(&pool.done)
}
break
}
last := len(pool.queue) - 1
item := pool.queue[last]
ordered_remove(&pool.queue, last)
sync.mutex_unlock(&pool.queue_mutex)
process_dir(pool, item, &local_buf)
delete(item.path)
if len(item.rel) > 0 {delete(item.rel)}
if len(local_buf) >= OUTPUT_BUF_SIZE {
flush_buf(pool.result_chan, &local_buf)
}
old := sync.atomic_sub_explicit(&pool.active, 1, .Release)
if old == 1 {
sync.one_shot_event_signal(&pool.done)
}
}
}
process_dir :: proc(pool: ^WalkerPool, item: WorkItem, local_buf: ^[dynamic]u8) {
dir_path := item.path
cpath := strings.clone_to_cstring(dir_path)
if cpath == nil do return
defer delete(cpath)
fd, open_err := linux.open(cpath, {.DIRECTORY, .CLOEXEC})
if open_err != .NONE do return
defer linux.close(fd)
has_git := false
if !item.in_ignored {
has_git = has_git_dir(fd)
}
gi_ctx := item.gi_ctx
rel := item.rel
if has_git {
gi_ctx = nil
rel = ""
}
child_in_repo := has_git || item.in_repo
gi: ^Gitignore = nil
if !item.in_ignored {
gi = load_ignore_patterns(dir_path, child_in_repo)
}
if gi != nil {
new_ctx := new(GIContext)
new_ctx.gi = gi
if len(rel) > 0 {
new_ctx.base_rel, _ = strings.clone(rel)
}
new_ctx.parent = gi_ctx
sync.mutex_lock(&pool.contexts_lock)
append(&pool.all_contexts, new_ctx)
sync.mutex_unlock(&pool.contexts_lock)
gi_ctx = new_ctx
}
buf: [32 * 1024]u8
rel_buf: [4096]u8
for {
n, errno := linux.getdents(fd, buf[:])
if n <= 0 || errno != .NONE do break
offs := 0
for d in linux.dirent_iterate_buf(buf[:n], &offs) {
name := linux.dirent_name(d)
if name == "." || name == ".." do continue
if name == ".git" do continue
is_dir := d.type == .DIR
is_nondir := d.type != .DIR
if pool.exclude_gi != nil && is_ignored(pool.exclude_gi, name, is_dir) {
continue
}
entry_rel := build_rel(rel_buf[:], rel, name)
ignored := false
if item.in_ignored {
ignored = true
} else if gi_ctx != nil {
ignored = check_chain(gi_ctx, entry_rel, is_dir)
}
if is_dir {
if ignored && matches_pattern(pool, name) {
append_path(local_buf, dir_path, name, true)
}
child_rel, _ := strings.clone(entry_rel)
child_path := join_path(dir_path, name)
push_work(
pool,
WorkItem {
path = child_path,
rel = child_rel,
gi_ctx = gi_ctx,
in_repo = child_in_repo,
in_ignored = ignored,
},
)
} else if is_nondir {
if ignored && matches_pattern(pool, name) {
append_path(local_buf, dir_path, name, false)
}
}
}
}
}
check_chain :: proc(ctx: ^GIContext, entry_rel: string, is_dir: bool) -> bool {
c := ctx
for c != nil {
if c.gi != nil {
rel := relative_to(entry_rel, c.base_rel)
match := check_match(c.gi, rel, is_dir)
if match != .None {
return match == .Ignored
}
}
c = c.parent
}
return false
}
relative_to :: proc(entry_rel, base_rel: string) -> string {
if len(base_rel) == 0 do return entry_rel
prefix_len := len(base_rel)
if len(entry_rel) > prefix_len &&
entry_rel[prefix_len] == '/' &&
strings.has_prefix(entry_rel, base_rel) {
return entry_rel[prefix_len + 1:]
}
return entry_rel
}
build_rel :: proc(buf: []u8, rel, name: string) -> string {
if len(rel) == 0 do return name
pos := copy(buf, rel)
if pos < len(buf) {
buf[pos] = '/'
pos += 1
pos += copy(buf[pos:], name)
}
return string(buf[:pos])
}
matches_pattern :: proc(pool: ^WalkerPool, name: string) -> bool {
if !pool.has_pattern do return true
cap, ok := regex.match(pool.pattern_re, name)
regex.destroy(cap)
return ok
}
push_work :: proc(pool: ^WalkerPool, item: WorkItem) {
sync.atomic_add_explicit(&pool.active, 1, .Relaxed)
sync.mutex_lock(&pool.queue_mutex)
append(&pool.queue, item)
sync.mutex_unlock(&pool.queue_mutex)
sync.atomic_sema_post(&pool.queue_sema)
}
has_git_dir :: proc(fd: linux.Fd) -> bool {
git_fd, err := linux.openat(fd, ".git", {.DIRECTORY, .CLOEXEC})
if err == .NONE {
linux.close(git_fd)
return true
}
return false
}
load_ignore_patterns :: proc(dir_path: string, in_repo: bool) -> ^Gitignore {
has_patterns := false
sb: strings.Builder
strings.builder_init(&sb)
defer strings.builder_destroy(&sb)
if in_repo {
gi_path := join_path(dir_path, ".gitignore")
data, err := os.read_entire_file_from_path(gi_path, context.allocator)
delete(gi_path)
if err == .NONE {
fmt.sbprintf(&sb, "%s", string(data))
delete(data)
has_patterns = true
}
}
ig_path := join_path(dir_path, ".ignore")
idata, ierr := os.read_entire_file_from_path(ig_path, context.allocator)
delete(ig_path)
if ierr == .NONE {
fmt.sbprintf(&sb, "%s", string(idata))
delete(idata)
has_patterns = true
}
if !has_patterns do return nil
content := strings.to_string(sb)
gi := new(Gitignore)
gi^ = parse(content)
return gi
}
join_path :: proc(parent, child: string) -> string {
need_sep := len(parent) == 0 || parent[len(parent) - 1] != '/'
total := len(parent) + len(child)
if need_sep do total += 1
buf := make([]u8, total, context.allocator)
pos := copy(buf, parent)
if need_sep {
buf[pos] = '/'
pos += 1
}
copy(buf[pos:], child)
return string(buf)
}

View File

@@ -66,7 +66,7 @@
packages.default = pkgs.stdenv.mkDerivation rec { packages.default = pkgs.stdenv.mkDerivation rec {
pname = "envr"; pname = "envr";
version = "0.3.0"; version = "0.2.0";
src = ./.; src = ./.;
nativeBuildInputs = [ nativeBuildInputs = [
@@ -95,6 +95,7 @@
devShells.default = pkgs.mkShell { devShells.default = pkgs.mkShell {
buildInputs = with pkgs; [ buildInputs = with pkgs; [
fd
nushell nushell
libsodium libsodium
@@ -105,10 +106,6 @@
# Build tools # Build tools
zip zip
# Helper tools
delta
hyperfine
# IDE # IDE
unstable.helix unstable.helix
typescript-language-server typescript-language-server

View File

@@ -1,31 +1,14 @@
package main package main
import "core:bufio"
import "core:fmt" import "core:fmt"
import "core:mem"
import "core:os" import "core:os"
main :: proc() { main :: proc() {
when ODIN_DEBUG {
heap_track: mem.Tracking_Allocator
mem.tracking_allocator_init(&heap_track, context.allocator)
defer mem.tracking_allocator_destroy(&heap_track)
defer if len(heap_track.allocation_map) > 0 {
for _, leak in heap_track.allocation_map {
fmt.eprintf("LEAK: %v leaked %m\n", leak.location, leak.size)
}
}
context.allocator = mem.tracking_allocator(&heap_track)
temp_track: mem.Tracking_Allocator
mem.tracking_allocator_init(&temp_track, context.temp_allocator)
defer mem.tracking_allocator_destroy(&temp_track)
context.temp_allocator = mem.tracking_allocator(&temp_track)
}
defer free_all(context.temp_allocator) defer free_all(context.temp_allocator)
cmd, ok := parse_args(os.args, os.to_writer(os.stdout), os.to_writer(os.stderr)) cmd, ok := parse_args(os.args, os.to_writer(os.stdout), os.to_writer(os.stderr))
defer delete_command(&cmd) // delete flushes automatically defer bufio.writer_flush(cmd.out_buf)
if !ok { if !ok {
return return
} }
@@ -35,6 +18,8 @@ main :: proc() {
cmd_init(&cmd) cmd_init(&cmd)
case "version": case "version":
cmd_version(&cmd) cmd_version(&cmd)
case "deps":
cmd_deps(&cmd)
case "list": case "list":
cmd_list(&cmd) cmd_list(&cmd)
case "backup", "add": case "backup", "add":

127
scan.odin
View File

@@ -1,21 +1,137 @@
package main package main
import "core:fmt"
import "core:os" import "core:os"
import "core:strings"
import "core:sync"
import "core:terminal"
import "findr" fd_counter: sync.Atomic_Mutex
fd_seq: int
// Caller is responsible for freeing paths // Caller is responsible for freeing paths
scan_path :: proc(search_path: string, cfg: Config) -> (paths: [dynamic]string, ok: bool) { scan_path :: proc(search_path: string, cfg: Config) -> (paths: [dynamic]string, ok: bool) {
opts := findr.WalkOptions { if terminal.is_terminal(os.stdout) {
pattern = cfg.ScanConfig.Matcher, fmt.printf("Searching for all files in \"%s\"...\n", search_path)
excludes = cfg.ScanConfig.Exclude[:],
} }
findr.walk({search_path}, &paths, opts, os.get_processor_core_count()) all_files, all_ok := run_fd(build_fd_args(search_path, cfg, true))
if !all_ok {
return
}
if terminal.is_terminal(os.stdout) {
fmt.printf("Search for unignored fies in \"%s\"...\n", search_path)
}
unignored_files, unignored_ok := run_fd(build_fd_args(search_path, cfg, false))
if !unignored_ok {
return
}
unignored_set := make(map[string]bool, len(unignored_files), context.temp_allocator)
for file in unignored_files {
unignored_set[file] = true
}
for file in all_files {
if !(file in unignored_set) {
append(&paths, file)
}
}
ok = true ok = true
return return
} }
@(private = "file")
build_fd_args :: proc(search_path: string, cfg: Config, include_ignored: bool) -> []string {
args_len := 3 + 2 * len(cfg.ScanConfig.Exclude) + 2
args := make([dynamic]string, 0, args_len, context.temp_allocator)
append(&args, "fd")
append(&args, "-a")
append(&args, cfg.ScanConfig.Matcher)
for exclude in cfg.ScanConfig.Exclude {
append(&args, "-E")
append(&args, exclude)
}
if include_ignored {
append(&args, "-HI")
} else {
append(&args, "-H")
}
append(&args, search_path)
return args[:]
}
run_fd :: proc(args: []string) -> (lines: []string, ok: bool) {
tmp_path := next_fd_tmp_path()
tmp_file, tmp_err := os.open(tmp_path, os.O_CREATE | os.O_WRONLY | os.O_TRUNC)
if tmp_err != nil {
// TODO: Log a message here
return
}
desc := os.Process_Desc {
command = args,
stdout = tmp_file,
stderr = nil,
}
p, start_err := os.process_start(desc)
os.close(tmp_file)
if start_err != nil {
os.remove(tmp_path)
return
}
state, wait_err := os.process_wait(p)
if wait_err != nil || state.exit_code != 0 {
os.remove(tmp_path)
return
}
data, read_err := os.read_entire_file_from_path(tmp_path, context.temp_allocator)
os.remove(tmp_path)
if read_err != nil {
return
}
output := string(data)
output = strings.trim_space(output)
if len(output) == 0 {
ok = true
return
}
raw_lines := strings.split(output, "\n", context.temp_allocator)
result := make([dynamic]string, 0, len(raw_lines), context.temp_allocator)
for line in raw_lines {
trimmed := strings.trim_space(line)
if len(trimmed) > 0 {
append(&result, trimmed)
}
}
return result[:], true
}
@(private = "file")
next_fd_tmp_path :: proc() -> string {
sync.atomic_mutex_lock(&fd_counter)
n := fd_seq
fd_seq += 1
sync.atomic_mutex_unlock(&fd_counter)
return fmt.tprintf("/tmp/envr-fd-%d-%d", os.get_pid(), n)
}
cant_scan :: proc(feats: AvailableFeatures) -> bool {
return Feature.Fd not_in feats
}
find_unbacked :: proc(local_files: []string, db_files: []EnvFile) -> []string { find_unbacked :: proc(local_files: []string, db_files: []EnvFile) -> []string {
// Lives until the end of the function
backed_set := make(map[string]bool, len(db_files), context.temp_allocator) backed_set := make(map[string]bool, len(db_files), context.temp_allocator)
for file in db_files { for file in db_files {
backed_set[file.Path] = true backed_set[file.Path] = true
@@ -29,3 +145,4 @@ find_unbacked :: proc(local_files: []string, db_files: []EnvFile) -> []string {
} }
return unbacked[:] return unbacked[:]
} }

View File

@@ -3,10 +3,14 @@ package main
import "core:fmt" import "core:fmt"
import "core:os" import "core:os"
import "core:path/filepath" import "core:path/filepath"
import "core:strings"
import "core:testing" import "core:testing"
@(test) @(test)
test_scan_path_finds_gitignored_env_files :: proc(t: ^testing.T) { test_scan_path_finds_gitignored_env_files :: proc(t: ^testing.T) {
feats := check_features()
testing.expect(t, cant_scan(feats) == false)
base := fmt.tprintf("/tmp/envr-scan-test-%d", os.get_pid()) base := fmt.tprintf("/tmp/envr-scan-test-%d", os.get_pid())
os.mkdir_all(base) os.mkdir_all(base)
defer os.remove_all(base) defer os.remove_all(base)
@@ -38,12 +42,7 @@ test_scan_path_finds_gitignored_env_files :: proc(t: ^testing.T) {
} }
results, ok := scan_path(base, cfg) results, ok := scan_path(base, cfg)
defer { defer delete(results)
for path in results {
delete(path)
}
delete(results)
}
testing.expect(t, ok, "scan_path should succeed") testing.expect(t, ok, "scan_path should succeed")
found_env := false found_env := false
@@ -70,6 +69,9 @@ test_scan_path_finds_gitignored_env_files :: proc(t: ^testing.T) {
@(test) @(test)
test_scan_path_empty_dir :: proc(t: ^testing.T) { test_scan_path_empty_dir :: proc(t: ^testing.T) {
feats := check_features()
testing.expect(t, cant_scan(feats) == false)
base := fmt.tprintf("/tmp/envr-scan-empty-%d", os.get_pid()) base := fmt.tprintf("/tmp/envr-scan-empty-%d", os.get_pid())
os.mkdir_all(base) os.mkdir_all(base)
defer os.remove_all(base) defer os.remove_all(base)
@@ -83,3 +85,12 @@ test_scan_path_empty_dir :: proc(t: ^testing.T) {
testing.expect(t, ok, "scan_path should succeed") testing.expect(t, ok, "scan_path should succeed")
testing.expect(t, len(results) == 0, fmt.tprintf("expected 0 results, got %d", len(results))) testing.expect(t, len(results) == 0, fmt.tprintf("expected 0 results, got %d", len(results)))
} }
@(test)
test_scan_meets_expectations :: proc(t: ^testing.T) {
testing.expect(t, cant_scan({}), "no features should mean can't scan")
testing.expect(t, cant_scan({.Git}), "Git alone should mean can't scan")
testing.expect(t, !cant_scan({.Fd}), "having Fd should mean can scan")
testing.expect(t, !cant_scan({.Fd, .Git}), "both Fd and Git should mean can scan")
}

View File

@@ -12,6 +12,24 @@ Ed25519Keypair :: struct {
Private: [32]u8, Private: [32]u8,
} }
read_wire_string :: proc(data: []u8, offset: ^int) -> (s: string, ok: bool) {
if offset^ + 4 > len(data) {
return
}
length := u32(data[offset^]) << 24 | u32(data[offset^ + 1]) << 16 |
u32(data[offset^ + 2]) << 8 | u32(data[offset^ + 3])
offset^ += 4
if offset^ + int(length) > len(data) {
return
}
s = string(data[offset^ : offset^ + int(length)])
offset^ += int(length)
ok = true
return
}
parse_ssh_public_key :: proc(pub_path: string) -> (pub: [32]u8, ok: bool) { parse_ssh_public_key :: proc(pub_path: string) -> (pub: [32]u8, ok: bool) {
data, err := os.read_entire_file_from_path(pub_path, context.temp_allocator) data, err := os.read_entire_file_from_path(pub_path, context.temp_allocator)
if err != nil { if err != nil {
@@ -235,21 +253,3 @@ is_encrypted_key :: proc(priv_path: string) -> bool {
return ciphername != "none" return ciphername != "none"
} }
read_wire_string :: proc(data: []u8, offset: ^int) -> (s: string, ok: bool) {
if offset^ + 4 > len(data) {
return
}
length := u32(data[offset^]) << 24 | u32(data[offset^ + 1]) << 16 |
u32(data[offset^ + 2]) << 8 | u32(data[offset^ + 3])
offset^ += 4
if offset^ + int(length) > len(data) {
return
}
s = string(data[offset^ : offset^ + int(length)])
offset^ += int(length)
ok = true
return
}

View File

@@ -6,7 +6,7 @@ import "core:io"
import "core:strings" import "core:strings"
render_table :: proc(w: io.Writer, headers: []string, rows: [][]string) { render_table :: proc(w: io.Writer, headers: []string, rows: [][]string) {
col_widths := make([dynamic]int, 0, len(headers), context.temp_allocator) col_widths := make([dynamic]int, 0, len(headers))
for i in 0 ..< len(headers) { for i in 0 ..< len(headers) {
append(&col_widths, strings.rune_count(headers[i])) append(&col_widths, strings.rune_count(headers[i]))
} }
@@ -20,14 +20,11 @@ render_table :: proc(w: io.Writer, headers: []string, rows: [][]string) {
} }
b: strings.Builder b: strings.Builder
strings.builder_init(&b, context.temp_allocator) strings.builder_init(&b)
defer strings.builder_destroy(&b)
defer delete(col_widths)
hline :: proc( hline :: proc(w: io.Writer, b: ^strings.Builder, left, mid, right: string, widths: [dynamic]int) {
w: io.Writer,
b: ^strings.Builder,
left, mid, right: string,
widths: [dynamic]int,
) {
strings.write_string(b, left) strings.write_string(b, left)
for i in 0 ..< len(widths) { for i in 0 ..< len(widths) {
for _ in 0 ..< widths[i] + 2 { for _ in 0 ..< widths[i] + 2 {