17 Commits

Author SHA1 Message Date
e7da6f60e1 fix: Fixed leaks. 2026-06-19 15:12:12 -04:00
0083e4e0db fix(scan): Fixed a bug preventing TUI from working. 2026-06-19 14:39:53 -04:00
33cd7c4eda feat: Colorized console output. 2026-06-19 13:45:55 -04:00
a03d388a0c refactor: Allocations now use the temp_allocator more frequently. 2026-06-19 07:50:57 -04:00
84764d03a6 refactor: Cleaned up the sync and scan commands. 2026-06-19 07:29:51 -04:00
0523c09601 refactor: Gave db its own allocator. 2026-06-18 17:29:28 -04:00
f137fc79fc refactor: Fixed up env_file_sync. 2026-06-18 16:35:03 -04:00
8d5e50566b ci: Fixed release-please. 2026-06-18 10:43:47 -04:00
5059572951 fix: Fixed memory leaks in the db. 2026-06-18 10:35:21 -04:00
d2b84ac4c6 refactor(crypto): used a proper init procedure. 2026-06-18 08:18:09 -04:00
96bc218c46 style: Ordered procedures by usage, with main at the top. 2026-06-18 08:08:46 -04:00
3b32e365c9 chore: Updated TODOS.md 2026-06-18 07:45:38 -04:00
12574e123b feat: Removed runtime git dependency.
This also allowed us to drop the Features code.
2026-06-18 07:29:44 -04:00
bc464a3410 chore: Removed completed todo. 2026-06-17 18:03:54 -04:00
2ef733fe58 perf: Replaced fd with custom internals. 2026-06-17 17:56:31 -04:00
159ff91938 build: Incremented flake version number. 2026-06-16 11:56:10 -04:00
Spencer Brower
84550d4708 chore(main): release 0.3.0 2026-06-16 11:43:36 -04:00
42 changed files with 2764 additions and 1277 deletions

View File

@@ -2,8 +2,6 @@ on:
push:
branches:
- main
- dev
- odin
permissions:
contents: write

4
.gitignore vendored
View File

@@ -7,8 +7,12 @@ list.json
man
# build artifacts
*.spall
builds
envr
envr-go
findr/findr
findr/findr-prof
findr/bench-*.md
result
version.odin

View File

@@ -2,6 +2,10 @@
## [0.3.0](https://github.com/sbrow/envr/compare/v0.2.1...v0.3.0) (2026-06-16)
Version 0.3.0 represents a significant departure (and improvement) for envr.
The entire codebase was rewritten in [Odin](https://odin-lang.org/) (from Go).
This reduced the binary size from over 17MB to under 600k, improved performance,
and significantly reduced the number of project dependencies from 69 to just 2.
### ⚠ BREAKING CHANGES

View File

@@ -12,14 +12,13 @@ the tool [of your choosing](#backup-options).
## Features
- 🔐 **Encrypted Storage**: All `.env` files are encrypted using your ssh key and
[libsodium](https://github.com/jedisct1/libsodium) encryption.
- 🔄 **Automatic Sync**: Update the database with one command, which can easily
- **Encrypted Storage**: All `.env` files are encrypted using your ssh key and
[libsodium](https://github.com/jedisct1/libsodium).
- **Automatic Sync**: Update the database with one command, which can easily
be run on a cron.
- 🔍 **Smart Scanning**: Automatically discover and import `.env` files in your
- **Smart Scanning**: Automatically discover and import `.env` files in your
home directory.
- **Interactive CLI**: User-friendly prompts for file selection and management.
- 🗂️ **Rename Detection**: Automatically finds and updates renamed/moved
- **Rename Detection**: Automatically find and updates renamed/moved
repositories.
## TODOS
@@ -28,15 +27,11 @@ repositories.
- [x] Allow configuration of ssh key.
- [x] Allow multiple ssh keys.
## Prerequisites
- An SSH key pair (for encryption/decryption)
- The following binaries:
- [fd](https://github.com/sharkdp/fd)
- [git](https://git-scm.com)
## Installation
You will need an SSH key pair for encryption and decryption. You can generate one
with `ssh-keygen -t ed25519`. It will be saved to `~/.ssh/id_ed25519`.
### With Odin
If you already have `odin` installed:
@@ -96,7 +91,12 @@ The configuration file is created during initialization:
],
"scan": {
"matcher": "\\.env",
"exclude": "*.envrc",
"exclude": [
"*\\.envrc",
"\\.local/",
"node_modules",
"vendor"
],
"include": "~"
}
}

View File

@@ -35,13 +35,7 @@ Stdout will be captured by redirecting `os.stdout` to a pipe.
## Hard to test (interactive / external deps)
### `cmd_deps` (cmd_deps.odin)
- Needs `git` and/or `fd` in PATH
- Test TTY and non-TTY paths
- Skip if dependencies not available (with `#assert` like TODO 28 suggests)
### `cmd_scan` (cmd_scan.odin)
- Needs `fd` installed
- Test with fixture git repo containing `.env` files
- Test `find_unbacked` integration (already partially tested in `cmd_check_test.odin`)
- Non-TTY JSON output path
@@ -66,5 +60,4 @@ Stdout will be captured by redirecting `os.stdout` to a pipe.
- DB integration tests should use in-memory SQLite (`:memory:`) where possible.
- Temp dir fixtures should follow the pattern in `scan_test.odin`.
- External dependency tests (`fd`, `git`) should use `#assert` to ensure the dependency is present rather than silently skipping (TODO 28).
- Tests that manipulate the `HOME` env var must use a mutex to prevent races with parallel test execution.

View File

@@ -1,13 +1,18 @@
# TODOs
1. envr scan crashes when there are zero results.
1. Consider giving db its own allocator
27. Commands are still leaking.
2. **db.odin:324-327** — Map iteration (`remote_set`) is non-deterministic. Same file can produce different JSON on each backup, causing spurious DB diffs. Sort remotes before storing.
28. **db.odin** — Inconsistencies in how struct vs sqlite are named.
3. **db.odin:135, 250** — String interpolation into SQL (`VACUUM INTO '%s'`, `ATTACH DATABASE '%s'`). Currently safe because input is controlled, but fragile.
29. Add color flag and support non colored output.
4. **features.odin:30-41**`find_binary` uses `strings.join` instead of `filepath.join`, uses `os.stat` instead of checking executability, hardcodes `:` as PATH separator (wrong on Windows).
2. Generate md and man pages again.
3. **db.odin:324-327** — Map iteration (`remote_set`) is non-deterministic. Same file can produce different JSON on each backup, causing spurious DB diffs. Sort remotes before storing.
4. Make sure official path separators are used when appropriate, rather than '/'.
5. **cmd_restore.odin:20-30 & cmd_remove.odin:19-29** — Identical path-resolution block copy-pasted. `is_abs` guard is redundant since `filepath.abs` is a no-op on absolute paths. Extract a helper.
@@ -19,7 +24,7 @@
11. **db.odin:352-353**`hex.encode` error ignored. `string(hex_bytes)` aliases the byte slice.
12. **cmd_sync.odin:80, cmd_list.odin:33, cmd_deps.odin:9**`make([]string, 2)` for table rows never freed. Leaks per row. Defer to memory pass.
12. **cmd_sync.odin:80, cmd_list.odin:33**`make([]string, 2)` for table rows never freed. Leaks per row. Defer to memory pass.
13. **cmd_list.odin** — Non-TTY branch builds `ListEntry` structs and marshals JSON separately. Now that `render_json_rows` (issue 1) accepts an `io.Writer` and uses `json.marshal`, unify both branches to use it. Note: will change JSON keys from `"directory"/"path"` to `"Directory"/"Path"`.
@@ -27,14 +32,10 @@
15. Add a text filter to the multi_select.
16. Create backup / fallback fd.
17. Add tests for untested commands.
18. 2 scan tests silently skip when fd isn't installed, tests pass without actually testing anything. These should use #assert to be sure that fd is in path.
19. Try to do all encryption / decryption in memory - only read / write encrypted data to disk.
20. add --format -f flag to commands that draw tables.
21. Replace `testing.expect` calls with `testing.expect_value` calls where appropriate.
@@ -43,6 +44,12 @@
23. procedures should be ordered by use, main at the top, then in the order they are called from main.
24. Shell completion
25. Bring back windows support / cross-compilation.
26. Test all cmds / terminal branches.
## Double-check AI output
- [ ] cli.odin
@@ -50,7 +57,6 @@
- [x] cmd_backup.odin
- [x] cmd_check.odin
- [ ] cmd_check_test.odin
- [x] cmd_deps.odin
- [ ] cmd_edit_config.odin
- [x] cmd_init.odin
- [x] cmd_list.odin
@@ -69,8 +75,6 @@
- [ ] db.odin
- [ ] db_integration_test.odin
- [ ] db_test.odin
- [x] features.odin
- [x] features_test.odin
- [x] main.odin
- [x] prompt.odin
- [ ] scan.odin
@@ -81,3 +85,11 @@
- [ ] ssh_test.odin
- [ ] table.odin
- [ ] table_test.odin
- [ ] findr/findr_test.odin
- [ ] findr/gitignore.odin
- [ ] findr/gitignore_test.odin
- [ ] findr/glob.odin
- [ ] findr/glob_test.odin
- [ ] findr/repos.odin
- [ ] findr/test_env.odin
- [ ] findr/walker.odin

View File

@@ -1,92 +0,0 @@
# Windows Compatibility Guide
This document outlines Windows compatibility issues and solutions for the envr project.
## Critical Issues
### 1. Path Handling Bug (MUST FIX)
**File:** `app/env_file.go:209`
**Issue:** Uses `path.Join` instead of `filepath.Join`, which won't work correctly on Windows due to different path separators.
**Current code:**
```go
f.Path = path.Join(newDir, path.Base(f.Path))
```
**Fixed code:**
```go
f.Path = filepath.Join(newDir, filepath.Base(f.Path))
```
## External Dependencies
The application relies on external tools that need to be installed separately on Windows:
### Required Tools
1. **fd** - Fast file finder
- Install via: `winget install sharkdp.fd` or `choco install fd`
- Alternative: `scoop install fd`
2. **git** - Version control system
- Install via: `winget install Git.Git` or download from git-scm.com
- Usually already available on most development machines
## Minor Compatibility Notes
### File Permissions
- Unix file permissions (`0755`, `0644`) are used throughout the codebase
- These are safely ignored on Windows - no changes needed
### Editor Configuration
**File:** `cmd/edit_config.go:20-24`
**Issue:** Relies on `$EDITOR` environment variable which is less common on Windows.
**Current behavior:** Fails if `$EDITOR` is not set
**Recommended improvement:** Add fallback detection for Windows editors:
```go
editor := os.Getenv("EDITOR")
if editor == "" {
if runtime.GOOS == "windows" {
editor = "notepad.exe" // or "code.exe" for VS Code
} else {
fmt.Println("Error: $EDITOR environment variable is not set")
return
}
}
```
## Installation Instructions for Windows
1. Install required dependencies:
```powershell
winget install sharkdp.fd
winget install Git.Git
```
2. Fix the path handling bug in `app/env_file.go:209`
3. Build and run as normal:
```powershell
go build
.\envr.exe init
```
## Testing on Windows
After applying the critical path fix, the core functionality should work correctly on Windows. The application has been designed with cross-platform compatibility in mind, using:
- `filepath` package for path operations (mostly)
- `os.UserHomeDir()` for home directory detection
- Standard Go file operations
## Summary
- **1 critical bug** must be fixed for Windows compatibility
- **2 external tools** need to be installed
- **1 minor enhancement** recommended for better Windows UX
- Overall architecture is Windows-compatible

191
cli.odin
View File

@@ -43,13 +43,6 @@ key somewhere, otherwise your data could be lost forever.`,
{"list", "envr list", "View your tracked files", "", {}},
{"remove", "envr remove <path>", "Remove a .env file from your database", "", {}},
{"check", "envr check [path]", "Check if files are backed up", "", {}},
{
"deps",
"envr deps",
"Check for missing binaries",
"envr relies on external binaries for certain functionality.\n\nThe check command reports on which binaries are available and which are not.",
{},
},
{"version", "envr version", "Show envr's version", "", {}},
{"edit-config", "envr edit-config", "Edit your config with your default editor", "", {}},
{
@@ -61,20 +54,12 @@ key somewhere, otherwise your data could be lost forever.`,
},
}
delete_command :: proc(cmd: ^Command) {
delete(cmd.args)
delete(cmd.flags)
delete(cmd.bool_set)
bufio.writer_destroy(cmd.out_buf)
free(cmd.out_buf)
}
// Caller is responsible for calling delete_command(cmd).
// FIXME: Works in kinda a wonky and awkward way.
parse_args :: proc(args: []string, out: io.Stream, err: io.Stream) -> (cmd: Command, ok: bool) {
{
cmd.out_buf = new(bufio.Writer)
bufio.writer_init(cmd.out_buf, out)
bufio.writer_init(cmd.out_buf, out, allocator = context.allocator)
cmd.out = bufio.writer_to_writer(cmd.out_buf)
cmd.err = err
}
@@ -90,6 +75,7 @@ parse_args :: proc(args: []string, out: io.Stream, err: io.Stream) -> (cmd: Comm
cmd.flags = make(map[string]string)
cmd.bool_set = make(map[string]bool)
// TODO: Optimize loop?
i := 2
for i < len(args) {
arg := args[i]
@@ -137,13 +123,80 @@ parse_args :: proc(args: []string, out: io.Stream, err: io.Stream) -> (cmd: Comm
return cmd, true
}
has_flag :: proc(cmd: ^Command, name: string) -> bool {
_, ok := cmd.flags[name]
if ok {
return true
print_command_help :: proc(cmd: ^Command) {
ok := write_command_help(cmd.name, cmd.out)
if !ok {
fmt.wprintf(cmd.err, "Unknown command: %s\n", cmd.name)
write_usage(cmd.out)
}
_, ok2 := cmd.bool_set[name]
return ok2
}
write_command_help :: proc(name: string, w: io.Writer) -> bool {
info, found := find_command(name)
if !found {
return false
}
fmt.wprintf(
w,
"%s\n\n\n" +
COLOR_HEADINGS +
"Usage:" +
ANSI_RESET +
"\n\n " +
COLOR_FLAGS +
"%s" +
ANSI_RESET +
" [flags]\n\n",
info.short,
info.usage,
flush = false,
)
if len(info.aliases) > 0 {
fmt.wprintf(
w,
"\n" +
COLOR_HEADINGS +
"Aliases:" +
ANSI_RESET +
"\n\n " +
COLOR_COMMANDS +
"%s" +
ANSI_RESET,
info.name,
flush = false,
)
for a in info.aliases {
fmt.wprintf(w, ", " + COLOR_COMMANDS + "%s" + ANSI_RESET, a, flush = false)
}
fmt.wprintf(w, "\n", flush = false)
}
if len(info.long) > 0 {
fmt.wprintf(w, "\n%s\n", info.long, flush = false)
}
fmt.wprintf(
w,
"\n" +
COLOR_HEADINGS +
"Flags:" +
ANSI_RESET +
"\n\n " +
COLOR_FLAGS +
"-h, --help" +
ANSI_RESET +
" help for %s\n " +
COLOR_FLAGS +
"-c, --config-file" +
ANSI_RESET +
` <path> config file (default "~/.envr/config.json")
`,
info.name,
flush = false,
)
return true
}
find_command :: proc(name: string) -> (CommandInfo, bool) {
@@ -160,53 +213,15 @@ find_command :: proc(name: string) -> (CommandInfo, bool) {
return CommandInfo{}, false
}
write_command_help :: proc(name: string, w: io.Writer) -> bool {
info, found := find_command(name)
if !found {
return false
}
fmt.wprintf(w, "Usage: %s [flags]\n\n", info.usage, flush = false)
fmt.wprintf(w, "%s\n", info.short, flush = false)
if len(info.aliases) > 0 {
fmt.wprintf(w, "\nAliases:\n %s", info.name, flush = false)
for a in info.aliases {
fmt.wprintf(w, ", %s", a, flush = false)
}
fmt.wprintf(w, "\n", flush = false)
}
if len(info.long) > 0 {
fmt.wprintf(w, "\n%s\n", info.long, flush = false)
}
fmt.wprintf(
w,
"\nFlags:\n -h, --help help for %s\n -c, --config-file <path> config file (default \"~/.envr/config.json\")\n",
info.name,
flush = false,
)
return true
}
print_command_help :: proc(cmd: ^Command) {
ok := write_command_help(cmd.name, cmd.out)
if !ok {
fmt.wprintf(cmd.err, "Unknown command: %s\n", cmd.name)
write_usage(cmd.out)
}
}
// TODO: command args should be shown in usage.
write_usage :: proc(w: io.Writer) {
fmt.wprintf(
w,
`envr keeps your .env synced to a local, age encrypted database.
`envr keeps your .env synced to a local, encrypted database.
Is a safe and easy way to gather all your .env files in one place where they can
easily be backed by another tool such as restic or git.
All your data is stored in ~/data.age
All your data is stored in ~/.envr/data.envr
Getting started is easy:
@@ -233,21 +248,29 @@ at before, restore your backup with:
> envr restore ~/<path to repository>/.env
Usage:
envr [command]
%sUsage:%s
Available Commands:
%senvr%s [command]
%sAvailable Commands:%s
`,
COLOR_HEADINGS,
ANSI_RESET,
COLOR_FLAGS,
ANSI_RESET,
COLOR_HEADINGS,
ANSI_RESET,
flush = false,
)
for c in COMMANDS {
name_start := len(c.name)
fmt.wprintf(w, "%s", c.name, flush = false)
fmt.wprintf(w, " %s%s", COLOR_COMMANDS, c.name, flush = false)
for a in c.aliases {
fmt.wprintf(w, ", %s", a, flush = false)
name_start += len(a) + 2
}
fmt.wprint(w, ANSI_RESET)
padding := 20 - name_start
if padding > 0 {
for _ in 0 ..< padding {
@@ -259,14 +282,40 @@ Available Commands:
fmt.wprintf(
w,
`
Flags:
-h, --help help for envr
-c, --config-file <path> config file (default "~/.envr/config.json")
"\n" +
COLOR_HEADINGS +
"Flags:" +
ANSI_RESET +
"\n\n " +
COLOR_FLAGS +
"-h, --help" +
ANSI_RESET +
" help for envr\n" +
COLOR_FLAGS +
` -c, --config-file` +
ANSI_RESET +
` <path> config file (default "~/.envr/config.json")
Use "envr [command] --help" for more information about a command.
Use "` +
COLOR_FLAGS +
"envr" +
ANSI_RESET +
` [command] --help" for more information about a command.
`,
flush = false,
)
}
has_flag :: proc(cmd: ^Command, name: string) -> bool {
return name in cmd.flags || name in cmd.bool_set
}
delete_command :: proc(cmd: ^Command) {
bufio.writer_flush(cmd.out_buf)
delete(cmd.args)
delete(cmd.flags)
delete(cmd.bool_set)
bufio.writer_destroy(cmd.out_buf)
free(cmd.out_buf)
}

View File

@@ -57,7 +57,7 @@ test_usage_text_contains_flags_and_help_hint :: proc(t: ^testing.T) {
testing.expect(t, strings.contains(text, "Flags:"), "missing Flags section")
testing.expect(t, strings.contains(text, "--help"), "missing --help flag")
testing.expect(t, strings.contains(text, "[command] --help"), "missing help hint")
}
}
@(test)
test_command_help_backup :: proc(t: ^testing.T) {

View File

@@ -4,9 +4,9 @@ import "core:fmt"
import "core:os"
import "core:path/filepath"
// TODO: What happens if you pass a non existent path to cmd_check?
// TODO: UX could be improved, so "run envr add ." if file not exists.
cmd_check :: proc(cmd: ^Command) {
feats := check_features()
check_path: string
if len(cmd.args) > 0 {
check_path = cmd.args[0]
@@ -39,18 +39,10 @@ cmd_check :: proc(cmd: ^Command) {
is_dir := os.is_directory(abs_path)
files_in_path: [dynamic]string
// TODO: set a reasonable default
files_in_path := make([dynamic]string, context.temp_allocator)
if is_dir {
if cant_scan(feats) {
fmt.wprintln(
cmd.err,
"Error: please install fd to use the check command (https://github.com/sharkdp/fd)",
flush = false,
)
return
}
scanned, scan_ok := scan_path(abs_path, db.cfg)
if !scan_ok {
fmt.wprintln(cmd.err, "Error scanning directory for .env files", flush = false)
@@ -72,13 +64,23 @@ cmd_check :: proc(cmd: ^Command) {
if len(files_in_path) == 0 {
fmt.wprintln(cmd.out, "No .env files found in the specified directory.", flush = false)
} else {
fmt.wprintln(cmd.out, "✓ All .env files in the directory are backed up.", flush = false)
fmt.wprintln(
cmd.out,
"✓ All .env files in the directory are backed up.",
flush = false,
)
}
} else {
fmt.wprintf(cmd.out, "Found %d .env file(s) that are not backed up:\n", len(not_backed), flush = false)
fmt.wprintf(
cmd.out,
"Found %d .env file(s) that are not backed up:\n",
len(not_backed),
flush = false,
)
for file in not_backed {
fmt.wprintf(cmd.out, " %s\n", file, flush = false)
}
fmt.wprintln(cmd.out, "\nRun 'envr sync' to back up these files.", flush = false)
}
}

View File

@@ -1,33 +0,0 @@
package main
import "core:fmt"
import "core:os"
import "core:terminal"
// TODO: Improve table rendering
cmd_deps :: proc(cmd: ^Command) {
feats := check_features()
headers := []string{"Feature", "Status"}
rows: [dynamic][]string
if .Git in feats {
append(&rows, []string{"Git", "\u2713 Available"})
} else {
append(&rows, []string{"Git", "\u2717 Missing"})
}
if .Fd in feats {
append(&rows, []string{"fd", "\u2713 Available"})
} else {
append(&rows, []string{"fd", "\u2717 Missing"})
}
if terminal.is_terminal(os.stdout) {
render_table(cmd.out, headers, rows[:])
} else {
render_json_rows(cmd.out, headers, rows[:])
fmt.wprint(cmd.out, "\n", flush = false)
}
}

View File

@@ -12,8 +12,7 @@ cmd_edit_config :: proc(cmd: ^Command) {
config_path := cmd.config_path
_, stat_err := os.stat(config_path, context.allocator)
if stat_err != nil {
if !os.exists(config_path) {
fmt.wprintf(
cmd.err,
"Config file does not exist at %s. Run 'envr init' first.\n",

View File

@@ -25,7 +25,6 @@ cmd_list :: proc(cmd: ^Command) {
if !list_ok {
return
}
defer delete(rows)
if terminal.is_terminal(os.stdout) {
headers := []string{"Directory", "Path"}
@@ -34,7 +33,7 @@ cmd_list :: proc(cmd: ^Command) {
for row in rows {
dir_str := strings.concatenate({row.Dir, "/"}, context.temp_allocator)
filename := filepath.base(row.Path)
row_slice := make([]string, 2)
row_slice := make([]string, 2, context.temp_allocator)
row_slice[0] = dir_str
row_slice[1] = filename
append(&table_rows, row_slice)

View File

@@ -6,23 +6,13 @@ import "core:os"
import "core:terminal"
cmd_scan :: proc(cmd: ^Command) {
feats := check_features()
if cant_scan(feats) {
fmt.wprintln(
cmd.err,
"Error: please install fd to use the scan command (https://github.com/sharkdp/fd)",
flush = false,
)
return
}
db, db_ok := db_open(cmd.config_path)
if !db_ok {
return
}
defer db_close(&db)
search_dirs := search_paths(db.cfg)
search_dirs := search_paths(db.cfg, context.temp_allocator)
if len(search_dirs) == 0 {
fmt.wprintln(
cmd.err,
@@ -33,9 +23,15 @@ cmd_scan :: proc(cmd: ^Command) {
}
// TODO: Figure out a sane default
all_files: [dynamic]string
// Can't use temp allocator becuase strings inside are copied to context.allocator
all_files := make([dynamic]string)
defer {
for &f in all_files {delete(f)}
delete(all_files)
}
for dir in search_dirs {
found, scan_ok := scan_path(dir, db.cfg)
defer delete(found)
if !scan_ok {
fmt.wprintf(cmd.err, "Error scanning %s\n", dir, flush = false)
continue

View File

@@ -24,68 +24,55 @@ cmd_sync :: proc(cmd: ^Command) {
if !list_ok {
return
}
defer delete(files)
// TODO: Set sane default size
results: [dynamic]SyncEntry
defer delete(results)
// TODO: Can't use temp allocator becuase strings inside are copied to context.allocator
results := make([]SyncEntry, len(files))
defer {
for &e in results {
delete(e.Path)
delete(e.Status)
}
delete(results)
}
for &file in files {
old_path: string
old_path, _ = strings.clone(file.Path, context.temp_allocator)
result, err_msg := db_sync(&db, &file)
for &file, i in files {
result, err := db_sync(&db, &file)
status: string
is_dir_updated := .DirUpdated in result
switch {
case .Error in result:
if len(err_msg) > 0 {
status = err_msg
} else {
status = "error"
}
case .BackedUp in result:
status = "Backed Up"
case .Restored in result:
status = "Restored"
case .DirUpdated in result:
if err != .None {
status = sync_error_message(err)
} else if .BackedUp in result {
status = .DirUpdated in result ? "Moved & Backed Up" : "Backed Up"
} else if .Restored in result {
status = .DirUpdated in result ? "Moved & Restored" : "Restored"
} else if .DirUpdated in result {
status = "Moved"
case:
} else {
status = "OK"
}
if is_dir_updated {
if !db_delete(&db, old_path) {
return
// TODO: Handle errors
path_str, _ := strings.clone(file.Path, context.temp_allocator)
status_str, _ := strings.clone(status, context.temp_allocator)
results[i] = SyncEntry {
Path = path_str,
Status = status_str,
}
}
if db_update_required(result) {
if !db_insert(&db, file) {
return
}
}
path_str, _ := strings.clone(file.Path)
status_str, _ := strings.clone(status)
append(&results, SyncEntry{Path = path_str, Status = status_str})
}
if terminal.is_terminal(os.stdout) {
headers := []string{"File", "Status"}
table_rows := make([dynamic][]string, 0, len(results))
// TODO: Use [2]string instead of slice here
table_rows := make([dynamic][]string, 0, len(results), context.temp_allocator)
for res in results {
row_slice := make([]string, 2)
row_slice[0] = res.Path
row_slice[1] = res.Status
append(&table_rows, row_slice)
row_slice := [2]string{res.Path, res.Status}
append(&table_rows, row_slice[:])
}
render_table(cmd.out, headers, table_rows[:])
} else {
data, marshal_err := json.marshal(results[:])
data, marshal_err := json.marshal(results[:], allocator = context.temp_allocator)
if marshal_err != nil {
fmt.wprintf(cmd.err, "Error marshaling JSON: %v\n", marshal_err, flush = false)
return
@@ -94,3 +81,23 @@ cmd_sync :: proc(cmd: ^Command) {
}
}
sync_error_message :: proc(e: SyncError) -> string {
switch e {
case .None:
return ""
case .DirMissing:
return "directory missing"
case .MultipleDirs:
return "multiple directories found"
case .GitRootFailed:
return "failed to find git roots"
case .WriteFailed:
return "failed to write file"
case .ReadFailed:
return "failed to read file"
case .DbFailed:
return "failed to update database"
}
return "unknown error"
}

15
colors.odin Normal file
View File

@@ -0,0 +1,15 @@
package main
import "core:terminal/ansi"
COLOR_HEADINGS ::
ansi.CSI + ansi.FG_BRIGHT_GREEN + ";" + ansi.BOLD + ";" + ansi.UNDERLINE + ansi.SGR
COLOR_COMMANDS :: ansi.CSI + ansi.FG_BRIGHT_CYAN + ";" + ansi.BOLD + ansi.SGR
COLOR_EXAMPLE :: ansi.CSI + ansi.ITALIC + ansi.SGR
COLOR_FLAGS :: ansi.CSI + ansi.BOLD + ";" + ansi.FG_BRIGHT_WHITE + ansi.SGR
ANSI_RESET :: ansi.CSI + ansi.RESET + ansi.SGR

View File

@@ -6,6 +6,8 @@ import "core:os"
import "core:path/filepath"
import "core:strings"
import "findr"
SshKeyPair :: struct {
Private: string `json:"private"`,
Public: string `json:"public"`,
@@ -23,25 +25,16 @@ Config :: struct {
config_path: string `json:"-"`,
}
default_config_path :: proc(home: string, allocator := context.allocator) -> string {
path, err := filepath.join([]string{home, ".envr", "config.json"}, allocator)
if err != nil {
panic("Ran out of memory when building config path")
}
return path
}
load_config :: proc(config_path: string) -> (Config, bool) {
data, read_err := os.read_entire_file_from_path(config_path, context.allocator)
load_config :: proc(config_path: string, allocator := context.allocator) -> (Config, bool) {
// TODO: Should we use context.allocator + defer delete()?
data, read_err := os.read_entire_file_from_path(config_path, context.temp_allocator)
if read_err != nil {
fmt.println("No config file found. Please run `envr init` to generate one.")
return Config{}, false
}
defer delete(data)
cfg: Config
// TODO: use json 5
err := json.unmarshal(data, &cfg)
err := json.unmarshal(data, &cfg, .JSON5, allocator)
if err != nil {
fmt.printf("Error parsing config: %v\n", err)
return Config{}, false
@@ -51,33 +44,104 @@ load_config :: proc(config_path: string) -> (Config, bool) {
return cfg, true
}
delete_config :: proc(cfg: ^Config) {
default_config_path :: proc(home: string, allocator := context.allocator) -> string {
path, err := filepath.join([]string{home, ".envr", "config.json"}, allocator)
if err != nil {
panic("Ran out of memory when building config path")
}
return path
}
delete_config :: proc(cfg: ^Config, allocator := context.allocator) {
for key in cfg.Keys {
delete(key.Private)
delete(key.Public)
delete(key.Private, allocator)
delete(key.Public, allocator)
}
delete(cfg.Keys)
delete(cfg.ScanConfig.Matcher)
delete(cfg.ScanConfig.Matcher, allocator)
for exclude in cfg.ScanConfig.Exclude {
delete(exclude)
delete(exclude, allocator)
}
delete(cfg.ScanConfig.Exclude)
for include in cfg.ScanConfig.Include {
delete(include)
delete(include, allocator)
}
delete(cfg.ScanConfig.Include)
}
envr_dir :: proc(config_path: string) -> string {
return filepath.dir(config_path)
save_config :: proc(cfg: Config, force: bool = false) -> bool {
config_dir := envr_dir(cfg.config_path)
if !os.exists(config_dir) {
mkdir_err := os.make_directory(config_dir)
if mkdir_err != nil {
fmt.printf("Error creating %s directory: %v\n", config_dir, mkdir_err)
return false
}
}
if os.exists(cfg.config_path) && !force {
info, stat_err := os.stat(cfg.config_path, context.temp_allocator)
if stat_err == nil {
defer os.file_info_delete(info, context.temp_allocator)
if info.size > 0 {
fmt.println("Config file already exists. Run again with --force to reinitialize.")
return false
}
}
}
data, marshal_err := json.marshal(
cfg,
{pretty = true, use_spaces = true, spaces = 2},
context.temp_allocator,
)
if marshal_err != nil {
fmt.printf("Error marshaling config: %v\n", marshal_err)
return false
}
write_err := os.write_entire_file(cfg.config_path, data)
if write_err != nil {
fmt.printf("Error writing config: %v\n", write_err)
return false
}
return true
}
data_path :: proc(config_path: string) -> string {
path, _ := filepath.join([]string{envr_dir(config_path), "data.envr"})
return path
// Caller is responsible for calling delete_config()
new_config :: proc(
private_key_paths: []string,
cfg_path: string = "~/.envr/config.json",
) -> Config {
keys := make([dynamic]SshKeyPair, 0, len(private_key_paths))
for priv in private_key_paths {
// TODO: Is this bad?
priv_key := strings.clone(priv)
pub, _ := strings.concatenate([]string{priv_key, ".pub"})
append(&keys, SshKeyPair{Private = priv_key, Public = pub})
}
exclude := make([dynamic]string, 0, 4)
append(&exclude, strings.clone("*\\.envrc"))
append(&exclude, strings.clone("\\.local/"))
append(&exclude, strings.clone("node_modules"))
append(&exclude, strings.clone("vendor"))
include := make([dynamic]string, 0, 1)
append(&include, strings.clone("~"))
scan_cfg := ScanConfig {
Matcher = strings.clone("\\.env"),
Exclude = exclude,
Include = include,
}
return Config{Keys = keys, ScanConfig = scan_cfg, config_path = cfg_path}
}
find_ssh_private_keys :: proc() -> (keys: [dynamic]string, ok: bool) {
@@ -126,115 +190,49 @@ find_ssh_private_keys :: proc() -> (keys: [dynamic]string, ok: bool) {
return
}
// Caller is responsible for calling delete_config()
new_config :: proc(
private_key_paths: []string,
cfg_path: string = "~/.envr/config.json",
) -> Config {
keys := make([dynamic]SshKeyPair, 0, len(private_key_paths))
for priv in private_key_paths {
// TODO: Is this bad?
priv_key := strings.clone(priv)
pub, _ := strings.concatenate([]string{priv_key, ".pub"})
append(&keys, SshKeyPair{Private = priv_key, Public = pub})
}
exclude := make([dynamic]string, 0, 4)
append(&exclude, strings.clone("*\\.envrc"))
append(&exclude, strings.clone("\\.local/"))
append(&exclude, strings.clone("node_modules"))
append(&exclude, strings.clone("vendor"))
include := make([dynamic]string, 0, 1)
append(&include, strings.clone("~"))
scan_cfg := ScanConfig {
Matcher = strings.clone("\\.env"),
Exclude = exclude,
Include = include,
}
return Config{Keys = keys, ScanConfig = scan_cfg, config_path = cfg_path}
}
save_config :: proc(cfg: Config, force: bool = false) -> bool {
config_dir := envr_dir(cfg.config_path)
if !os.exists(config_dir) {
mkdir_err := os.make_directory(config_dir)
if mkdir_err != nil {
fmt.printf("Error creating %s directory: %v\n", config_dir, mkdir_err)
return false
}
}
if os.exists(cfg.config_path) && !force {
info, stat_err := os.stat(cfg.config_path, context.allocator)
if stat_err == nil {
defer os.file_info_delete(info, context.allocator)
if info.size > 0 {
fmt.println("Config file already exists. Run again with --force to reinitialize.")
return false
}
}
}
data, marshal_err := json.marshal(cfg, {pretty = true, use_spaces = true, spaces = 2})
if marshal_err != nil {
fmt.printf("Error marshaling config: %v\n", marshal_err)
return false
}
defer delete(data)
write_err := os.write_entire_file(cfg.config_path, data)
if write_err != nil {
fmt.printf("Error writing config: %v\n", write_err)
return false
}
return true
}
search_paths :: proc(cfg: Config) -> (paths: [dynamic]string) {
// TODO: Is this okay?
// TODO: handle error
home, _ := os.user_home_dir(context.temp_allocator)
for include in cfg.ScanConfig.Include {
// TODO: Do we need to manually expand ~/ in odin?
expanded, _ := strings.replace(include, "~", home, 1)
if filepath.is_abs(expanded) {
append(&paths, expanded)
} else {
defer delete(expanded)
resolved, err := filepath.abs(expanded)
if err == nil {
append(&paths, resolved)
}
}
}
return
}
find_git_roots :: proc(cfg: Config) -> (roots: [dynamic]string, ok: bool) {
paths := search_paths(cfg)
for sp in paths {
args := []string{"fd", "-H", "-t", "d", "^\\.git$", sp}
lines, fd_ok := run_fd(args)
if !fd_ok {
return
}
for line in lines {
cleaned, _ := filepath.clean(line)
parent := filepath.dir(cleaned)
cloned, _ := strings.clone(parent)
append(&roots, cloned)
}
}
find_git_roots :: proc(
cfg: Config,
allocator := context.temp_allocator,
) -> (
roots: [dynamic]string,
ok: bool,
) {
paths := search_paths(cfg, allocator)
// TODO: Pass allocator to findr
// findr.find_repos(paths[:], &roots, os.get_processor_core_count(), allocator)
findr.find_repos(paths[:], &roots, os.get_processor_core_count())
ok = true
return
}
search_paths :: proc(cfg: Config, allocator := context.allocator) -> [dynamic]string {
// TODO: handle error
home, _ := os.user_home_dir(context.temp_allocator)
paths, _ := new_clone(cfg.ScanConfig.Include, allocator)
for &include in paths {
// TODO: Do we need to manually expand ~/ in odin?
expanded, _ := strings.replace(include, "~", home, 1, allocator)
if filepath.is_abs(expanded) {
include = expanded
} else {
resolved, err := filepath.abs(expanded, allocator)
if err == nil {
include = resolved
}
}
}
return paths^
}
envr_dir :: proc(config_path: string) -> string {
return filepath.dir(config_path)
}
// User is responsible for freeing the path
data_path :: proc(config_path: string, allocator := context.allocator) -> string {
path, _ := filepath.join([]string{envr_dir(config_path), "data.envr"}, allocator)
return path
}

View File

@@ -187,14 +187,10 @@ test_search_paths_expands_tilde :: proc(t: ^testing.T) {
cfg := Config {
ScanConfig = ScanConfig{Include = make([dynamic]string, 0, 1)},
}
defer delete(cfg.ScanConfig.Include)
append(&cfg.ScanConfig.Include, "~")
defer delete(cfg.ScanConfig.Include)
paths := search_paths(cfg)
defer delete(paths)
for path in paths {
defer delete(path)
}
paths := search_paths(cfg, context.temp_allocator)
testing.expect(t, len(paths) == 1, "should have 1 path")
if len(paths) > 0 {

View File

@@ -2,6 +2,7 @@ package main
import "core:fmt"
import "core:mem"
import "core:os"
MAGIC :: "ENVR"
MAGIC_BYTES := [4]u8{u8('E'), u8('N'), u8('V'), u8('R')}
@@ -20,85 +21,24 @@ RecipientEntry :: struct {
EncryptedKey: [CRYPTO_SECRETBOX_KEY_BYTES + CRYPTO_BOX_MAC_BYTES]u8,
}
sodium_initialized: bool
ensure_sodium :: proc() -> bool {
if sodium_initialized {
return true
}
rc := sodium_init()
if rc < 0 {
fmt.println("Error: libsodium initialization failed")
return false
}
sodium_initialized = true
return true
}
X25519Keypair :: struct {
Public: [CRYPTO_BOX_PUBLICKEY_BYTES]u8,
Private: [CRYPTO_BOX_SECRETKEY_BYTES]u8,
}
ssh_to_x25519 :: proc(keys: []SshKeyPair) -> (pairs: []X25519Keypair, ok: bool) {
if len(keys) == 0 {
return
@(init)
init_sodium :: proc "contextless" () {
if sodium_init() < 0 {
os.exit(1)
}
pairs = make([]X25519Keypair, len(keys))
for i in 0 ..< len(keys) {
ssh_kp, parse_ok := parse_ssh_private_key(keys[i].Private)
if !parse_ok {
fmt.printf("Error: failed to parse SSH private key: %s\n", keys[i].Private)
delete(pairs)
return
}
ssh_pub, pub_ok := parse_ssh_public_key(keys[i].Public)
if !pub_ok {
fmt.printf("Error: failed to parse SSH public key: %s\n", keys[i].Public)
delete(pairs)
return
}
pk_rc := crypto_sign_ed25519_pk_to_curve25519(&pairs[i].Public[0], &ssh_pub[0])
if pk_rc != 0 {
fmt.println("Error: failed to convert ed25519 public key to curve25519")
delete(pairs)
return
}
ed25519_sk: [64]u8
for j in 0 ..< 32 {
ed25519_sk[j] = ssh_kp.Private[j]
}
for j in 0 ..< 32 {
ed25519_sk[32 + j] = ssh_kp.Public[j]
}
sk_rc := crypto_sign_ed25519_sk_to_curve25519(&pairs[i].Private[0], &ed25519_sk[0])
if sk_rc != 0 {
fmt.println("Error: failed to convert ed25519 private key to curve25519")
delete(pairs)
return
}
}
ok = true
return
}
// TODO: Optimize performance
encrypt :: proc(plaintext: []u8, keys: []SshKeyPair) -> (ciphertext: []u8, ok: bool) {
if !ensure_sodium() {
return
}
x25519_pairs, pairs_ok := ssh_to_x25519(keys)
x25519_pairs, pairs_ok := ssh_to_x25519(keys, context.temp_allocator)
if !pairs_ok {
return
}
defer delete(x25519_pairs)
sym_key: [CRYPTO_SECRETBOX_KEY_BYTES]u8
randombytes_buf(&sym_key[0], CRYPTO_SECRETBOX_KEY_BYTES)
@@ -107,7 +47,7 @@ encrypt :: proc(plaintext: []u8, keys: []SshKeyPair) -> (ciphertext: []u8, ok: b
randombytes_buf(&main_nonce[0], CRYPTO_SECRETBOX_NONCE_BYTES)
ct_len := len(plaintext) + CRYPTO_SECRETBOX_MAC_BYTES
secret_ct := make([]u8, ct_len)
secret_ct := make([]u8, ct_len, context.temp_allocator)
pt_ptr: [^]u8
if len(plaintext) > 0 {
pt_ptr = &plaintext[0]
@@ -126,7 +66,7 @@ encrypt :: proc(plaintext: []u8, keys: []SshKeyPair) -> (ciphertext: []u8, ok: b
}
num_recipients := u32(len(x25519_pairs))
entries := make([]RecipientEntry, num_recipients)
entries := make([]RecipientEntry, num_recipients, context.temp_allocator)
for i in 0 ..< len(x25519_pairs) {
for j in 0 ..< CRYPTO_BOX_PUBLICKEY_BYTES {
@@ -186,17 +126,11 @@ encrypt :: proc(plaintext: []u8, keys: []SshKeyPair) -> (ciphertext: []u8, ok: b
mem.copy(&ciphertext[pos], &secret_ct[0], ct_len)
delete(entries)
delete(secret_ct)
ok = true
return
}
decrypt :: proc(ciphertext: []u8, keys: []SshKeyPair) -> (plaintext: []u8, ok: bool) {
if !ensure_sodium() {
return
}
if len(ciphertext) < HEADER_SIZE {
fmt.println("Error: ciphertext too short (header)")
return
@@ -240,11 +174,10 @@ decrypt :: proc(ciphertext: []u8, keys: []SshKeyPair) -> (plaintext: []u8, ok: b
enc_nonce: [CRYPTO_BOX_NONCE_BYTES]u8
enc_pub: [CRYPTO_BOX_PUBLICKEY_BYTES]u8
x25519_pairs, pairs_ok := ssh_to_x25519(keys)
x25519_pairs, pairs_ok := ssh_to_x25519(keys, context.temp_allocator)
if !pairs_ok {
return
}
defer delete(x25519_pairs)
found := false
matched_pi := 0
@@ -336,3 +269,57 @@ decrypt :: proc(ciphertext: []u8, keys: []SshKeyPair) -> (plaintext: []u8, ok: b
return
}
ssh_to_x25519 :: proc(
keys: []SshKeyPair,
allocator := context.temp_allocator,
) -> (
[]X25519Keypair,
bool,
) {
if len(keys) == 0 {
return {}, false
}
pairs := make([]X25519Keypair, len(keys), allocator)
for i in 0 ..< len(keys) {
ssh_kp, parse_ok := parse_ssh_private_key(keys[i].Private)
if !parse_ok {
fmt.printf("Error: failed to parse SSH private key: %s\n", keys[i].Private)
delete(pairs)
return pairs, false
}
ssh_pub, pub_ok := parse_ssh_public_key(keys[i].Public)
if !pub_ok {
fmt.printf("Error: failed to parse SSH public key: %s\n", keys[i].Public)
delete(pairs)
return pairs, false
}
pk_rc := crypto_sign_ed25519_pk_to_curve25519(&pairs[i].Public[0], &ssh_pub[0])
if pk_rc != 0 {
fmt.println("Error: failed to convert ed25519 public key to curve25519")
delete(pairs)
return pairs, false
}
ed25519_sk: [64]u8
for j in 0 ..< 32 {
ed25519_sk[j] = ssh_kp.Private[j]
}
for j in 0 ..< 32 {
ed25519_sk[32 + j] = ssh_kp.Public[j]
}
sk_rc := crypto_sign_ed25519_sk_to_curve25519(&pairs[i].Private[0], &ed25519_sk[0])
if sk_rc != 0 {
fmt.println("Error: failed to convert ed25519 private key to curve25519")
delete(pairs)
return pairs, false
}
}
return pairs, true
}

606
db.odin
View File

@@ -2,28 +2,32 @@ package main
import "core:crypto/hash"
import "core:encoding/hex"
import "core:encoding/ini"
import "core:encoding/json"
import "core:fmt"
import "core:mem"
import "core:os"
import "core:path/filepath"
import "core:strings"
import "core:time"
import "sqlite"
SyncFlagEnum :: enum {
Noop,
DirUpdated,
Restored,
BackedUp,
Error,
}
SyncFlag :: bit_set[SyncFlagEnum]
SyncDirection :: enum {
TrustDatabase,
TrustFilesystem,
SyncError :: enum {
None,
DirMissing,
MultipleDirs,
GitRootFailed,
WriteFailed,
ReadFailed,
DbFailed,
}
Db :: struct {
@@ -31,6 +35,7 @@ Db :: struct {
db: ^rawptr,
cfg: Config,
changed: bool,
arena: mem.Dynamic_Arena,
}
EnvFile :: struct {
@@ -41,6 +46,7 @@ EnvFile :: struct {
contents: string,
}
@(deprecated = "call db_close to clean up EnvFiles")
delete_envfile :: proc(f: ^EnvFile) {
delete(f.Path)
for &remote in f.Remotes {
@@ -51,29 +57,33 @@ delete_envfile :: proc(f: ^EnvFile) {
delete(f.contents)
}
make_temp_path :: proc() -> string {
ts := time.time_to_unix(time.now())
b: strings.Builder
strings.builder_init(&b)
defer strings.builder_destroy(&b)
fmt.sbprintf(&b, "/tmp/envr-%d-%d.db", os.get_pid(), ts)
return strings.to_string(b)
}
db_open :: proc(cfg_path: string) -> (database: Db, ok: bool) {
database = db_init() or_return
database.cfg = load_config(cfg_path, db_allocator(&database)) or_return
db_open :: proc(cfg_path: string) -> (Db, bool) {
cfg, ok := load_config(cfg_path)
if !ok {
return Db{}, false
// TODO: Use different allocators?
data_path := data_path(database.cfg.config_path, context.temp_allocator)
if os.exists(data_path) {
if ok = db_restore_from_encrypted(&database, data_path); !ok {
sqlite.db_close(database.db)
return database, false
}
} else {
// DB was created
database.changed = true
}
data_path := data_path(cfg.config_path)
_, stat_err := os.stat(data_path, context.allocator)
return database, true
}
// Creates a database an allocator and fresh, empty table, with zero encryption.
// In production, you most likely want to use `db_open`.
db_init :: proc() -> (database: Db, ok: bool) {
db: ^rawptr
rc := sqlite.db_open(":memory:", &db)
if rc != sqlite.OK {
fmt.printf("Error opening in-memory database: %s\n", sqlite.db_errmsg(db))
return Db{}, false
return
}
create_sql: cstring = "CREATE TABLE IF NOT EXISTS envr_env_files (path TEXT PRIMARY KEY NOT NULL, remotes TEXT, sha256 TEXT NOT NULL, contents TEXT NOT NULL)"
@@ -81,21 +91,69 @@ db_open :: proc(cfg_path: string) -> (Db, bool) {
if rc != sqlite.OK {
fmt.printf("Error creating table: %s\n", sqlite.db_errmsg(db))
sqlite.db_close(db)
return Db{}, false
return
}
database.db = db
mem.dynamic_arena_init(&database.arena)
return database, true
}
db_allocator :: proc(db: ^Db) -> mem.Allocator {
return mem.dynamic_arena_allocator(&db.arena)
}
db_restore_from_encrypted :: proc(db: ^Db, data_path: string) -> bool {
encrypted_data, read_err := os.read_entire_file_from_path(data_path, context.temp_allocator)
if read_err != nil {
fmt.printf("Error reading encrypted database: %v\n", read_err)
return false
}
if stat_err == nil {
if !db_restore_from_encrypted(db, cfg) {
sqlite.db_close(db)
return Db{}, false
// TODO: Use context.temp_allocator
plaintext, dec_ok := decrypt(encrypted_data, db.cfg.Keys[:])
if !dec_ok {
fmt.println("Error: decryption failed")
return false
}
defer delete(plaintext)
n := i64(len(plaintext))
buf := sqlite.malloc64(n)
if buf == nil {
fmt.println("Error: failed to allocate buffer for deserialization")
return false
}
copy(buf[:len(plaintext)], plaintext)
rc := sqlite.deserialize(
db.db,
"main",
buf,
n,
n,
sqlite.DESERIALIZE_FREEONCLOSE | sqlite.DESERIALIZE_RESIZEABLE,
)
if rc != sqlite.OK {
sqlite.free(buf)
fmt.printf("Error deserializing database: %s\n", sqlite.db_errmsg(db.db))
return false
}
return Db{db = db, cfg = cfg, changed = stat_err != nil}, true
return true
}
db_close :: proc(d: ^Db) {
defer sqlite.db_close(d.db)
allocator := db_allocator(d)
defer {
sqlite.db_close(d.db)
delete_config(&d.cfg, allocator)
mem.dynamic_arena_destroy(&d.arena)
}
if d.changed {
rc := sqlite.db_exec(d.db, "VACUUM", nil, nil, nil)
@@ -113,13 +171,14 @@ db_close :: proc(d: ^Db) {
defer sqlite.free(data)
sqlite_data := data[:sz]
// TODO: PAss allocator chain
encrypted, enc_ok := encrypt(sqlite_data, d.cfg.Keys[:])
if !enc_ok {
fmt.println("Error: encryption failed")
return
}
data_path := data_path(d.cfg.config_path)
data_path := data_path(d.cfg.config_path, allocator)
envr_d := envr_dir(d.cfg.config_path)
os.mkdir_all(envr_d)
@@ -134,14 +193,8 @@ db_close :: proc(d: ^Db) {
}
}
// Caller is responsible for calling:
// ```odin
// delete(results)
// for &result in results {
// delete(&result)
// }
// ```
db_list :: proc(d: ^Db, allocator := context.allocator) -> (results: [dynamic]EnvFile, ok: bool) {
// Results will be freed when `db_close` is called.
db_list :: proc(d: ^Db) -> ([]EnvFile, bool) {
stmt: ^rawptr
rc := sqlite.prepare_v2(
d.db,
@@ -152,10 +205,13 @@ db_list :: proc(d: ^Db, allocator := context.allocator) -> (results: [dynamic]En
)
if rc != sqlite.OK {
fmt.printf("Error preparing query: %s\n", sqlite.db_errmsg(d.db))
return
return []EnvFile{}, false
}
defer sqlite.finalize(stmt)
allocator := db_allocator(d)
results := make([dynamic]EnvFile, 0, 10, allocator)
for {
rc = sqlite.step(stmt)
if rc == sqlite.DONE {
@@ -163,7 +219,7 @@ db_list :: proc(d: ^Db, allocator := context.allocator) -> (results: [dynamic]En
}
if rc != sqlite.ROW {
fmt.printf("Error stepping query: %s\n", sqlite.db_errmsg(d.db))
return
#no_bounds_check return results[:], false
}
remotes_json := string(sqlite.column_text(stmt, 1))
@@ -185,156 +241,16 @@ db_list :: proc(d: ^Db, allocator := context.allocator) -> (results: [dynamic]En
)
}
ok = true
return
}
db_restore_from_encrypted :: proc(db: ^rawptr, cfg: Config) -> bool {
encrypted_data, read_err := os.read_entire_file_from_path(
data_path(cfg.config_path),
context.allocator,
)
defer delete(encrypted_data)
if read_err != nil {
fmt.printf("Error reading encrypted database: %v\n", read_err)
return false
}
plaintext, dec_ok := decrypt(encrypted_data, cfg.Keys[:])
if !dec_ok {
fmt.println("Error: decryption failed")
return false
}
defer delete(plaintext)
n := i64(len(plaintext))
buf := sqlite.malloc64(n)
if buf == nil {
fmt.println("Error: failed to allocate buffer for deserialization")
return false
}
copy(buf[:len(plaintext)], plaintext)
rc := sqlite.deserialize(
db,
"main",
buf,
n,
n,
sqlite.DESERIALIZE_FREEONCLOSE | sqlite.DESERIALIZE_RESIZEABLE,
)
if rc != sqlite.OK {
sqlite.free(buf)
fmt.printf("Error deserializing database: %s\n", sqlite.db_errmsg(db))
return false
}
return true
}
get_git_remotes :: proc(dir: string) -> [dynamic]string {
remotes: [dynamic]string
remote_set: map[string]bool
b: strings.Builder
strings.builder_init(&b)
defer strings.builder_destroy(&b)
fmt.sbprintf(&b, "%s-git-remotes", make_temp_path())
tmp_path := strings.to_string(b)
tmp_file, tmp_err := os.open(tmp_path, os.O_CREATE | os.O_WRONLY | os.O_TRUNC)
if tmp_err != nil {
return remotes
}
args := []string{"git", "remote", "-v"}
desc := os.Process_Desc {
command = args,
stdout = tmp_file,
stderr = nil,
working_dir = dir,
}
p, start_err := os.process_start(desc)
os.close(tmp_file)
if start_err != nil {
os.remove(tmp_path)
return remotes
}
state, wait_err := os.process_wait(p)
if wait_err != nil || state.exit_code != 0 {
os.remove(tmp_path)
return remotes
}
data, read_err := os.read_entire_file_from_path(tmp_path, context.allocator)
defer delete(data)
os.remove(tmp_path)
if read_err != nil {
return remotes
}
lines := strings.split(string(data), "\n")
for &line in lines {
line = strings.trim_space(line)
if len(line) == 0 {
continue
}
parts := strings.fields(line)
if len(parts) >= 2 {
remote_set[parts[1]] = true
}
}
for remote, _ in remote_set {
cloned, _ := strings.clone(remote)
append(&remotes, cloned)
}
return remotes
}
new_env_file :: proc(path: string) -> (EnvFile, bool) {
abs_path, abs_err := filepath.abs(path)
if abs_err != nil {
fmt.printf("Error getting absolute path: %v\n", abs_err)
return EnvFile{}, false
}
dir := filepath.dir(abs_path)
remotes := get_git_remotes(dir)
data, read_err := os.read_entire_file_from_path(abs_path, context.allocator)
defer delete(data)
if read_err != nil {
fmt.printf("Error reading file %s: %v\n", abs_path, read_err)
return EnvFile{}, false
}
digest := hash.hash_bytes(hash.Algorithm.SHA256, data, context.temp_allocator)
// TODO: Handle error
hex_bytes, _ := hex.encode(digest)
return EnvFile {
Path = abs_path,
Dir = dir,
Remotes = remotes,
Sha256 = string(hex_bytes),
contents = string(data),
},
true
#no_bounds_check return results[:], true
}
// TODO: Should we use context.temp_allocator for proc scoped lifetimes?
db_insert :: proc(d: ^Db, file: EnvFile) -> bool {
remotes_json, marshal_err := json.marshal(file.Remotes)
remotes_json, marshal_err := json.marshal(file.Remotes, allocator = context.temp_allocator)
if marshal_err != nil {
fmt.printf("Error marshaling remotes: %v\n", marshal_err)
return false
}
defer delete(remotes_json)
sql: cstring =
"INSERT OR REPLACE INTO " +
@@ -390,7 +306,8 @@ db_insert :: proc(d: ^Db, file: EnvFile) -> bool {
return true
}
db_fetch :: proc(d: ^Db, path: string, allocator := context.allocator) -> (EnvFile, bool) {
// Result will be freed when `db_close` is called.
db_fetch :: proc(d: ^Db, path: string) -> (EnvFile, bool) {
sql: cstring = "SELECT path, remotes, sha256, contents FROM envr_env_files WHERE path = ?"
stmt: ^rawptr
rc := sqlite.prepare_v2(d.db, sql, -1, &stmt, nil)
@@ -400,6 +317,8 @@ db_fetch :: proc(d: ^Db, path: string, allocator := context.allocator) -> (EnvFi
}
defer sqlite.finalize(stmt)
allocator := db_allocator(d)
cpath := to_cstring(path, allocator)
defer delete(cpath, allocator)
rc = sqlite.bind_text(stmt, 1, cpath, -1, nil)
@@ -423,7 +342,7 @@ db_fetch :: proc(d: ^Db, path: string, allocator := context.allocator) -> (EnvFi
json.unmarshal_string(remotes_json, &remotes, allocator = allocator)
}
file_path := clone_cstring(sqlite.column_text(stmt, 0))
file_path := clone_cstring(sqlite.column_text(stmt, 0), allocator)
return EnvFile {
Path = file_path,
@@ -467,6 +386,181 @@ db_delete :: proc(d: ^Db, path: string) -> bool {
return true
}
new_env_file :: proc(path: string) -> (EnvFile, bool) {
abs_path, abs_err := filepath.abs(path)
if abs_err != nil {
fmt.printf("Error getting absolute path: %v\n", abs_err)
return EnvFile{}, false
}
dir := filepath.dir(abs_path)
// TODO: Should we use the db allocator here?
remotes := get_git_remotes(dir, context.allocator)
data, read_err := os.read_entire_file_from_path(abs_path, context.allocator)
defer delete(data)
if read_err != nil {
fmt.printf("Error reading file %s: %v\n", abs_path, read_err)
return EnvFile{}, false
}
digest := hash.hash_bytes(hash.Algorithm.SHA256, data, context.temp_allocator)
// TODO: Handle error
hex_bytes, _ := hex.encode(digest, context.temp_allocator)
return EnvFile {
Path = abs_path,
Dir = dir,
Remotes = remotes,
Sha256 = string(hex_bytes),
contents = string(data),
},
true
}
// Reconciles `f` with the filesystem and persists changes to the database.
db_sync :: proc(d: ^Db, f: ^EnvFile) -> (SyncFlag, SyncError) {
allocator := db_allocator(d)
result: SyncFlag = {}
old_path := f.Path
if !os.exists(f.Dir) {
moved, err := try_move_dir(d, f, allocator)
if !moved {
return {}, err
}
result += {.DirUpdated}
}
if !os.exists(f.Path) {
write_err := os.write_entire_file(f.Path, f.contents)
if write_err != nil {
fmt.eprintf("db_sync: failed to write %s: %v\n", f.Path, write_err)
return result, .WriteFailed
}
if !db_persist(d, f, old_path) {
return result, .DbFailed
}
return result + {.Restored}, .None
}
data, read_err := os.read_entire_file_from_path(f.Path, allocator)
if read_err != nil {
fmt.eprintf("db_sync: failed to read %s: %v\n", f.Path, read_err)
return result, .ReadFailed
}
digest := hash.hash_bytes(hash.Algorithm.SHA256, data, context.temp_allocator)
hex_bytes, hex_err := hex.encode(digest, allocator)
if hex_err != nil {
fmt.eprintf("db_sync: failed to encode hash for %s: %v\n", f.Path, hex_err)
return result, .ReadFailed
}
current_sha := string(hex_bytes)
if current_sha == f.Sha256 {
if !db_persist(d, f, old_path) {
return result, .DbFailed
}
return result, .None
}
f.contents = string(data)
f.Sha256 = current_sha
if !db_persist(d, f, old_path) {
return result, .DbFailed
}
return result + {.BackedUp}, .None
}
db_persist :: proc(d: ^Db, f: ^EnvFile, old_path: string) -> bool {
if f.Path != old_path {
if !db_delete(d, old_path) {
return false
}
}
return db_insert(d, f^)
}
try_move_dir :: proc(d: ^Db, f: ^EnvFile, allocator: mem.Allocator) -> (bool, SyncError) {
roots, ok := find_git_roots(d.cfg)
if !ok {
return false, .GitRootFailed
}
defer {
for root in roots {
delete(root)
}
delete(roots)
}
match_count := 0
matched_dir: string
for root in roots {
remotes := get_git_remotes(root, context.temp_allocator)
if shares_remote(f, remotes[:]) {
match_count += 1
matched_dir = root
}
}
switch match_count {
case 0:
return false, .DirMissing
case 1:
f.Dir, _ = strings.clone(matched_dir, allocator)
base := filepath.base(f.Path)
new_path, _ := filepath.join({f.Dir, base}, allocator)
f.Path = new_path
f.Remotes = get_git_remotes(f.Dir, allocator)
return true, .None
case:
return false, .MultipleDirs
}
}
shares_remote :: proc(f: ^EnvFile, remotes: []string) -> bool {
for r1 in f.Remotes {
for r2 in remotes {
if r1 == r2 {
return true
}
}
}
return false
}
get_git_remotes :: proc(dir: string, allocator: mem.Allocator) -> [dynamic]string {
config_path, _ := filepath.join({dir, ".git", "config"}, context.temp_allocator)
// TODO: Handle error
m, _, ok := ini.load_map_from_path(config_path, context.temp_allocator)
if !ok {
return nil
}
remotes := make([dynamic]string, 0, 1, allocator)
for section_name, section in m {
if strings.has_prefix(section_name, "remote ") {
if url, ok := section["url"]; ok {
found := false
for r in remotes {
if r == url {found = true; break}
}
if !found {
// FIXME: Currently leaks when adding a file with envr scan
cloned, _ := strings.clone(url, allocator)
append(&remotes, cloned)
}
}
}
}
return remotes
}
to_cstring :: proc {
string_to_cstring,
strings.to_cstring,
@@ -481,6 +575,7 @@ string_to_cstring :: proc(s: string, allocator := context.allocator) -> cstring
return cs
}
// Unless an explicit allocator is passed, caller is responsible for freeing the result
clone_cstring :: proc(c: cstring, allocator := context.allocator) -> string {
str, err := strings.clone_from_cstring(c, allocator)
if err != nil {
@@ -492,146 +587,3 @@ clone_cstring :: proc(c: cstring, allocator := context.allocator) -> string {
return str
}
db_update_required :: proc(status: SyncFlag) -> bool {
return .BackedUp in status || .DirUpdated in status
}
shares_remote :: proc(f: ^EnvFile, remotes: []string) -> bool {
for r1 in f.Remotes {
for r2 in remotes {
if r1 == r2 {
return true
}
}
}
return false
}
update_dir :: proc(f: ^EnvFile, new_dir: string) {
f.Dir = new_dir
base := filepath.base(f.Path)
new_path, _ := strings.concatenate({new_dir, "/", base})
f.Path = new_path
f.Remotes = get_git_remotes(new_dir)
}
find_moved_dirs :: proc(d: ^Db, f: ^EnvFile) -> ([dynamic]string, bool) {
feats := check_features()
if .Fd not_in feats || .Git not_in feats {
fmt.println("Error: fd and git are required for moved dir detection")
return {}, false
}
roots, roots_ok := find_git_roots(d.cfg)
if !roots_ok {
return {}, false
}
moved: [dynamic]string
for root in roots {
remotes := get_git_remotes(root)
if shares_remote(f, remotes[:]) {
cloned, _ := strings.clone(root)
append(&moved, cloned)
}
}
return moved, true
}
db_sync :: proc(d: ^Db, f: ^EnvFile) -> (SyncFlag, string) {
return env_file_sync(f, .TrustFilesystem, d)
}
// If SyncFlag is .BackedUp, Caller is responsible for calling delete on f.contents and f.Sha256
env_file_sync :: proc(f: ^EnvFile, dir: SyncDirection, d: ^Db) -> (SyncFlag, string) {
result: SyncFlag = {}
_, stat_err := os.stat(f.Dir, context.allocator)
if stat_err != nil {
moved_dirs: [dynamic]string
if d != nil {
dirs, dirs_ok := find_moved_dirs(d, f)
if !dirs_ok {
return {.Error}, "failed to find moved dirs"
}
moved_dirs = dirs
}
if len(moved_dirs) == 0 {
return {.Error}, "directory missing"
} else if len(moved_dirs) == 1 {
update_dir(f, moved_dirs[0])
result = {.DirUpdated}
} else {
return {.Error}, "multiple directories found"
}
}
_, file_stat_err := os.stat(f.Path, context.allocator)
if file_stat_err != nil {
write_err := os.write_entire_file(f.Path, f.contents)
if write_err != nil {
msg, _ := strings.concatenate({"failed to write file: ", fmt.tprintf("%v", write_err)})
return {.Error}, msg
}
return result + {.Restored}, ""
}
data, read_err := os.read_entire_file_from_path(f.Path, context.allocator)
if read_err != nil {
msg, _ := strings.concatenate(
{"failed to read file for SHA comparison: ", fmt.tprintf("%v", read_err)},
)
return {.Error}, msg
}
digest := hash.hash_bytes(hash.Algorithm.SHA256, data)
// TODO: Handle error
hex_bytes, _ := hex.encode(digest)
current_sha := string(hex_bytes)
if current_sha == f.Sha256 {
return result, ""
}
switch dir {
case .TrustDatabase:
write_err := os.write_entire_file(f.Path, f.contents)
if write_err != nil {
msg, _ := strings.concatenate({"failed to write file: ", fmt.tprintf("%v", write_err)})
return {.Error}, msg
}
return result + {.Restored}, ""
case .TrustFilesystem:
if !env_file_backup(f) {
return {.Error}, "failed to backup file"
}
return result + {.BackedUp}, ""
}
return result, ""
}
// Loads the contents of the the file at f.Path into f.contents
//
// Caller is responsible for calling delete on f.contents and f.Sha256
env_file_backup :: proc(f: ^EnvFile) -> bool {
data, read_err := os.read_entire_file_from_path(f.Path, context.allocator)
if read_err != nil {
fmt.printf("Error reading file %s: %v\n", f.Path, read_err)
return false
}
f.contents = string(data)
digest := hash.hash_bytes(hash.Algorithm.SHA256, data, context.temp_allocator)
hex_bytes, alloc_err := hex.encode(digest)
if alloc_err != nil {
fmt.printf("Error generating hash for file %s: %v\n", f.Path, alloc_err)
return false
}
f.Sha256 = string(hex_bytes)
return true
}

View File

@@ -309,7 +309,6 @@ test_ssh_key_parse_from_fixtures :: proc(t: ^testing.T) {
if !x_ok {
return
}
defer delete(x25519_pairs)
testing.expect(t, len(x25519_pairs) == 1, "should have 1 x25519 keypair")
}

View File

@@ -1,5 +1,7 @@
package main
import "core:crypto/hash"
import "core:encoding/hex"
import "core:fmt"
import "core:os"
import "core:path/filepath"
@@ -8,30 +10,13 @@ import "core:testing"
import "sqlite"
make_test_db :: proc() -> (Db, bool) {
db: ^rawptr
rc := sqlite.db_open(":memory:", &db)
if rc != sqlite.OK {
return Db{}, false
}
create_sql: cstring = "CREATE TABLE IF NOT EXISTS envr_env_files (path TEXT PRIMARY KEY NOT NULL, remotes TEXT, sha256 TEXT NOT NULL, contents TEXT NOT NULL)"
rc = sqlite.db_exec(db, create_sql, nil, nil, nil)
if rc != sqlite.OK {
sqlite.db_close(db)
return Db{}, false
}
return Db{db = db}, true
}
make_test_env_file :: proc(path, sha, contents: string, remotes: []string = {}) -> EnvFile {
f := EnvFile {
Path = path,
Dir = "",
Sha256 = sha,
contents = contents,
Remotes = make([dynamic]string, 0, len(remotes)),
Remotes = make([dynamic]string, 0, len(remotes), context.temp_allocator),
}
for r in remotes {
append(&f.Remotes, r)
@@ -41,10 +26,10 @@ make_test_env_file :: proc(path, sha, contents: string, remotes: []string = {})
@(test)
test_db_insert_and_fetch :: proc(t: ^testing.T) {
d, ok := make_test_db()
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
if !ok do return
defer sqlite.db_close(d.db)
defer db_close(&d)
path := "/project/.env"
sha := "abc123"
@@ -56,7 +41,7 @@ test_db_insert_and_fetch :: proc(t: ^testing.T) {
testing.expect(t, db_insert(&d, f), "insert should succeed")
fetched, fetch_ok := db_fetch(&d, "/project/.env")
defer delete_envfile(&fetched)
// defer delete_envfile(&fetched)
testing.expect(t, fetch_ok, "fetch should succeed")
if !fetch_ok do return
@@ -69,10 +54,10 @@ test_db_insert_and_fetch :: proc(t: ^testing.T) {
@(test)
test_db_fetch_missing :: proc(t: ^testing.T) {
d, ok := make_test_db()
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
if !ok do return
defer sqlite.db_close(d.db)
defer db_close(&d)
_, fetch_ok := db_fetch(&d, "/nonexistent/.env")
testing.expect(t, !fetch_ok, "fetch missing should return false")
@@ -80,10 +65,9 @@ test_db_fetch_missing :: proc(t: ^testing.T) {
@(test)
test_db_insert_or_replace :: proc(t: ^testing.T) {
d, ok := make_test_db()
d, ok := db_init()
defer db_close(&d)
testing.expect(t, ok, "failed to create test db")
if !ok do return
defer sqlite.db_close(d.db)
f1 := make_test_env_file("/project/.env", "sha1", "KEY=old")
defer delete(f1.Remotes)
@@ -95,18 +79,13 @@ test_db_insert_or_replace :: proc(t: ^testing.T) {
results, list_ok := db_list(&d)
testing.expect(t, list_ok, "list should succeed")
if !list_ok do return
defer delete(results)
for &result in results {
defer delete_envfile(&result)
}
testing.expect(t, len(results) == 1, "should have 1 row, not 2")
fetched, fetch_ok := db_fetch(&d, "/project/.env")
testing.expect(t, fetch_ok, "fetch should succeed")
if !fetch_ok do return
defer delete_envfile(&fetched)
// defer delete_envfile(&fetched)
testing.expect_value(t, fetched.contents, "KEY=new")
testing.expect_value(t, fetched.Sha256, "sha2")
@@ -114,10 +93,10 @@ test_db_insert_or_replace :: proc(t: ^testing.T) {
@(test)
test_db_delete_existing :: proc(t: ^testing.T) {
d, ok := make_test_db()
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
if !ok do return
defer sqlite.db_close(d.db)
defer db_close(&d)
f := make_test_env_file("/project/.env", "sha", "KEY=val")
defer delete(f.Remotes)
@@ -131,20 +110,19 @@ test_db_delete_existing :: proc(t: ^testing.T) {
@(test)
test_db_delete_missing :: proc(t: ^testing.T) {
d, ok := make_test_db()
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
if !ok do return
defer sqlite.db_close(d.db)
defer db_close(&d)
testing.expect(t, !db_delete(&d, "/nonexistent/.env"), "delete missing should return false")
}
@(test)
test_db_list_multiple :: proc(t: ^testing.T) {
d, ok := make_test_db()
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
if !ok do return
defer sqlite.db_close(d.db)
defer db_close(&d)
f1 := make_test_env_file("/proj1/.env", "sha1", "A=1", []string{"git@github.com:a/repo.git"})
defer delete(f1.Remotes)
@@ -158,36 +136,27 @@ test_db_list_multiple :: proc(t: ^testing.T) {
results, list_ok := db_list(&d)
testing.expect(t, list_ok, "list should succeed")
if !list_ok do return
defer delete(results)
defer {
for &result in results {
delete_envfile(&result)
}
}
testing.expect_value(t, len(results), 3)
}
@(test)
test_db_list_empty :: proc(t: ^testing.T) {
d, ok := make_test_db()
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
if !ok do return
defer sqlite.db_close(d.db)
defer db_close(&d)
results, list_ok := db_list(&d)
testing.expect(t, list_ok, "list should succeed on empty db")
testing.expect(t, len(results) == 0, "should have 0 rows")
if list_ok do delete(results)
}
@(test)
test_db_insert_sets_changed :: proc(t: ^testing.T) {
d, ok := make_test_db()
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
if !ok do return
defer sqlite.db_close(d.db)
defer db_close(&d)
testing.expect(t, !d.changed, "changed should start false")
@@ -200,10 +169,10 @@ test_db_insert_sets_changed :: proc(t: ^testing.T) {
@(test)
test_db_delete_sets_changed :: proc(t: ^testing.T) {
d, ok := make_test_db()
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
if !ok do return
defer sqlite.db_close(d.db)
defer db_close(&d)
f := make_test_env_file("/project/.env", "sha", "KEY=val")
defer delete(f.Remotes)
@@ -216,10 +185,10 @@ test_db_delete_sets_changed :: proc(t: ^testing.T) {
@(test)
test_db_serialize :: proc(t: ^testing.T) {
d, ok := make_test_db()
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
if !ok do return
defer sqlite.db_close(d.db)
defer db_close(&d)
f := make_test_env_file("/project/.env", "sha", "KEY=val")
defer delete(f.Remotes)
@@ -234,37 +203,6 @@ test_db_serialize :: proc(t: ^testing.T) {
testing.expect(t, sz > 0, "serialized size should be > 0")
}
@(test)
test_db_update_required_noop :: proc(t: ^testing.T) {
testing.expect(t, !db_update_required({}), "Noop should not require update")
}
@(test)
test_db_update_required_backed_up :: proc(t: ^testing.T) {
testing.expect(t, db_update_required({.BackedUp}), "BackedUp should require update")
}
@(test)
test_db_update_required_dir_updated :: proc(t: ^testing.T) {
testing.expect(t, db_update_required({.DirUpdated}), "DirUpdated should require update")
}
@(test)
test_db_update_required_restored :: proc(t: ^testing.T) {
testing.expect(t, !db_update_required({.Restored}), "Restored alone should not require update")
}
@(test)
test_db_update_required_error :: proc(t: ^testing.T) {
testing.expect(t, !db_update_required({.Error}), "Error alone should not require update")
}
@(test)
test_db_update_required_combined :: proc(t: ^testing.T) {
combined := SyncFlag{.DirUpdated, .Restored}
testing.expect(t, db_update_required(combined), "DirUpdated|Restored should require update")
}
@(test)
test_shares_remote_overlap :: proc(t: ^testing.T) {
f := EnvFile {
@@ -319,11 +257,81 @@ test_shares_remote_both_empty :: proc(t: ^testing.T) {
testing.expect(t, !shares_remote(&f, remotes), "both empty should not share")
}
delete_remotes :: proc(remotes: [dynamic]string) {
for &r in remotes {
delete(r)
}
delete(remotes)
}
@(test)
test_make_temp_path_format :: proc(t: ^testing.T) {
p := make_temp_path()
testing.expect(t, strings.has_suffix(p, ".db"), "should end with .db")
testing.expect(t, strings.contains(p, fmt.tprintf("%d", os.get_pid())), "should contain PID")
test_get_git_remotes_single :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-remotes-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
git_dir := fmt.tprintf("%s/.git", base)
os.mkdir_all(git_dir)
config_content := "[core]\n\trepositoryformatversion = 0\n[remote \"origin\"]\n\turl = git@github.com:user/repo.git\n\tfetch = +refs/heads/*:refs/remotes/origin/*\n"
config_path := fmt.tprintf("%s/config", git_dir)
err := os.write_entire_file(config_path, transmute([]u8)config_content)
testing.expect(t, err == nil, "should write .git/config")
remotes := get_git_remotes(base, context.temp_allocator)
testing.expect(t, len(remotes) == 1, "should find 1 remote")
if len(remotes) != 1 do return
testing.expect_value(t, remotes[0], "git@github.com:user/repo.git")
}
@(test)
test_get_git_remotes_multiple :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-remotes-multi-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
git_dir := fmt.tprintf("%s/.git", base)
os.mkdir_all(git_dir)
config_content := "[remote \"origin\"]\n\turl = git@github.com:user/repo.git\n[remote \"upstream\"]\n\turl = https://gitlab.com/upstream/repo.git\n"
config_path := fmt.tprintf("%s/config", git_dir)
err := os.write_entire_file(config_path, transmute([]u8)config_content)
testing.expect(t, err == nil, "should write .git/config")
remotes := get_git_remotes(base, context.temp_allocator)
testing.expect(t, len(remotes) == 2, "should find 2 remotes")
}
@(test)
test_get_git_remotes_no_config :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-remotes-none-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
remotes := get_git_remotes(base, context.temp_allocator)
testing.expect(t, len(remotes) == 0, "should return empty when no .git/config")
}
@(test)
test_get_git_remotes_no_remotes :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-remotes-empty-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
git_dir := fmt.tprintf("%s/.git", base)
os.mkdir_all(git_dir)
config_content := "[core]\n\trepositoryformatversion = 0\n\tbare = false\n"
config_path := fmt.tprintf("%s/config", git_dir)
err := os.write_entire_file(config_path, transmute([]u8)config_content)
testing.expect(t, err == nil, "should write .git/config")
remotes := get_git_remotes(base, context.temp_allocator)
testing.expect(t, len(remotes) == 0, "should return empty when no remote sections")
}
@(test)
@@ -356,45 +364,205 @@ test_new_env_file_missing :: proc(t: ^testing.T) {
}
@(test)
test_env_file_backup :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-backup-%d", os.get_pid())
test_closing_db_has_no_leaks :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-leak-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
cfg_path, err := filepath.join([]string{base, "config.json"}, context.temp_allocator)
testing.expect(t, err == nil, "cfgPath should build successfully")
{
cfg := new_config([]string{"fixtures/keys/insecure-test-key"}, cfg_path)
testing.expect(t, save_config(cfg, force = true), "save should succeed")
delete_config(&cfg)
}
db, ok := db_open(cfg_path)
testing.expect(t, ok, "db should open")
db_close(&db)
}
@(test)
test_open_existing_db_has_no_leaks :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-leak-existing-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
cfg_path, err := filepath.join([]string{base, "config.json"}, context.temp_allocator)
testing.expect(t, err == nil, "cfgPath should build successfully")
{
cfg := new_config([]string{"fixtures/keys/insecure-test-key"}, cfg_path)
testing.expect(t, save_config(cfg, force = true), "save should succeed")
delete_config(&cfg)
}
// First open/close creates data.envr on disk
db, ok := db_open(cfg_path)
testing.expect(t, ok, "db should open")
if !ok do return
f := make_test_env_file(
"/project/.env",
"abc123",
"SECRET=value",
[]string{"git@github.com:user/repo.git"},
)
defer delete(f.Remotes)
testing.expect(t, db_insert(&db, f), "insert should succeed")
db_close(&db)
// Second open exercises db_restore_from_encrypted
db2, ok2 := db_open(cfg_path)
testing.expect(t, ok2, "db should open existing")
if !ok2 do return
db_close(&db2)
}
@(test)
test_db_sync_noop :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-sync-noop-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
env_path := fmt.tprintf("%s/.env", base)
err := os.write_entire_file(env_path, "KEY=12345\n")
testing.expect(t, err == nil, ".env file should exist")
content := "KEY=value\n"
write_err := os.write_entire_file(env_path, transmute([]u8)content)
testing.expect(t, write_err == nil, "should write .env file")
f := EnvFile {
Path = env_path,
}
defer delete(f.contents)
defer delete(f.Sha256)
testing.expect(t, env_file_backup(&f), "backup should succeed")
testing.expect_value(t, f.contents, "KEY=12345\n")
testing.expect_value(t, len(f.Sha256), 64)
digest := hash.hash_bytes(
hash.Algorithm.SHA256,
transmute([]u8)content,
context.temp_allocator,
)
hex_bytes, _ := hex.encode(digest, context.temp_allocator)
sha := string(hex_bytes)
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
defer db_close(&d)
f := make_test_env_file(env_path, sha, content)
f.Dir = base
db_insert(&d, f)
result, sync_err := db_sync(&d, &f)
testing.expect(t, sync_err == .None, "sync should not error")
testing.expect(t, result == {}, "should be noop")
}
@(test)
test_env_file_backup_missing :: proc(t: ^testing.T) {
f := EnvFile {
Path = "/tmp/envr-nonexistent-backup/.env",
}
testing.expect(t, !env_file_backup(&f), "missing file should return false")
test_db_sync_backed_up :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-sync-backup-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
env_path := fmt.tprintf("%s/.env", base)
changed_content := "KEY=changed\n"
write_err := os.write_entire_file(env_path, transmute([]u8)changed_content)
testing.expect(t, write_err == nil, "should write .env file")
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
defer db_close(&d)
f := make_test_env_file(env_path, "old_sha", "KEY=original")
f.Dir = base
db_insert(&d, f)
result, sync_err := db_sync(&d, &f)
testing.expect(t, sync_err == .None, "sync should not error")
testing.expect(t, .BackedUp in result, "should be backed up")
}
@(test)
test_update_dir :: proc(t: ^testing.T) {
f := EnvFile {
Path = "/old/project/.env",
Dir = "/old/project",
Remotes = make([dynamic]string, 0),
test_db_sync_restored :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-sync-restore-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
env_path := fmt.tprintf("%s/.env", base)
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
defer db_close(&d)
f := make_test_env_file(env_path, "some_sha", "SECRET=value")
f.Dir = base
defer delete(f.Remotes)
db_insert(&d, f)
result, err := db_sync(&d, &f)
testing.expect(t, err == .None, "sync should not error")
testing.expect(t, .Restored in result, "should be restored")
data, read_err := os.read_entire_file_from_path(env_path, context.temp_allocator)
testing.expect(t, read_err == nil, "file should exist after restore")
if read_err == nil {
testing.expect_value(t, string(data), "SECRET=value")
}
}
@(test)
test_db_sync_dir_missing :: proc(t: ^testing.T) {
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
defer db_close(&d)
f := make_test_env_file("/nonexistent/path/.env", "sha", "KEY=val")
db_insert(&d, f)
result, err := db_sync(&d, &f)
testing.expect(t, err == .DirMissing, "should return DirMissing error")
}
@(test)
test_db_sync_moved :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-sync-moved-%d", os.get_pid())
search_root := fmt.tprintf("%s/search", base)
repo_dir := fmt.tprintf("%s/myproject", search_root)
git_dir := fmt.tprintf("%s/.git", repo_dir)
defer os.remove_all(base)
os.mkdir_all(git_dir)
config_content := "[remote \"origin\"]\n\turl = git@github.com:user/repo.git\n"
config_path := fmt.tprintf("%s/config", git_dir)
write_err := os.write_entire_file(config_path, transmute([]u8)config_content)
testing.expect(t, write_err == nil, "should write .git/config")
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
defer db_close(&d)
d.cfg.ScanConfig.Include = make([dynamic]string, 0, 1, context.temp_allocator)
append(&d.cfg.ScanConfig.Include, search_root)
f := make_test_env_file(
"/old/nonexistent/path/.env",
"some_sha",
"SECRET=value",
[]string{"git@github.com:user/repo.git"},
)
testing.expect(t, db_insert(&d, f), "insert should succeed")
result, err := db_sync(&d, &f)
testing.expect(t, err == .None, "sync should not error")
if err != .None do return
testing.expect(t, .DirUpdated in result, "should have DirUpdated flag")
testing.expect(t, .Restored in result, "should have Restored flag")
expected_path := fmt.tprintf("%s/.env", repo_dir)
testing.expect_value(t, f.Path, expected_path)
testing.expect_value(t, f.Dir, repo_dir)
_, old_exists := db_fetch(&d, "/old/nonexistent/path/.env")
testing.expect(t, !old_exists, "old path should be deleted from db")
new_fetched, new_ok := db_fetch(&d, expected_path)
testing.expect(t, new_ok, "new path should exist in db")
if new_ok {
testing.expect_value(t, new_fetched.contents, "SECRET=value")
}
defer delete_envfile(&f)
update_dir(&f, "/new/location")
testing.expect_value(t, f.Dir, "/new/location")
testing.expect_value(t, f.Path, "/new/location/.env")
}

View File

@@ -4,11 +4,11 @@ Manage your .env files.
### Synopsis
envr keeps your .env synced to a local, age encrypted database.
envr keeps your .env synced to a local, encrypted database.
Is a safe and eay way to gather all your .env files in one place where they can
easily be backed by another tool such as restic or git.
All your data is stored in ~/data.age
All your data is stored in ~/.envr/data.envr
Getting started is easy:
@@ -45,7 +45,6 @@ at before, restore your backup with:
* [envr backup](envr_backup.md) - Import a .env file into envr
* [envr check](envr_check.md) - check if files in the current directory are backed up
* [envr deps](envr_deps.md) - Check for missing binaries
* [envr edit-config](envr_edit-config.md) - Edit your config with your default editor
* [envr init](envr_init.md) - Set up envr
* [envr list](envr_list.md) - View your tracked files

View File

@@ -1,24 +0,0 @@
## envr deps
Check for missing binaries
### Synopsis
envr relies on external binaries for certain functionality.
The check command reports on which binaries are available and which are not.
```
envr deps [flags]
```
### Options
```
-h, --help help for deps
```
### SEE ALSO
* [envr](envr.md) - Manage your .env files.

View File

@@ -1,51 +0,0 @@
package main
import "base:runtime"
import "core:mem"
import "core:os"
import "core:strings"
Feature :: enum {
Git,
Fd,
}
AvailableFeatures :: bit_set[Feature]
check_features :: proc() -> AvailableFeatures {
feats: AvailableFeatures
s: mem.Scratch
mem.scratch_init(&s, 4 * mem.DEFAULT_PAGE_SIZE)
defer mem.scratch_destroy(&s)
context.temp_allocator = mem.scratch_allocator(&s)
path_env := os.get_env("PATH", context.temp_allocator)
paths := strings.split(path_env, ":", context.temp_allocator)
if find_binary(paths, "git") != "" {
feats += {.Git}
}
if find_binary(paths, "fd") != "" {
feats += {.Fd}
}
return feats
}
find_binary :: proc(
paths: []string,
name: string,
allocator: runtime.Allocator = context.temp_allocator,
) -> string {
for p in paths {
candidate := strings.join({strings.trim_right(p, "/"), name}, "/", allocator)
_, err := os.stat(candidate, allocator)
if err == nil {
return candidate
}
}
return ""
}

View File

@@ -1,34 +0,0 @@
package main
import "core:os"
import "core:strings"
import "core:testing"
@(test)
test_find_binary_exists :: proc(t: ^testing.T) {
path := os.get_env("PATH", context.temp_allocator)
paths := strings.split(path, ":", context.temp_allocator)
result := find_binary(paths, "sh")
testing.expect(t, result != "", "sh should be found on PATH")
}
@(test)
test_find_binary_not_exists :: proc(t: ^testing.T) {
old_path := os.get_env("PATH", context.temp_allocator)
defer {
if old_path != "" {
os.set_env("PATH", old_path)
}
}
os.set_env("PATH", "/tmp/envr-nope")
path := os.get_env("PATH", context.temp_allocator)
paths := strings.split(path, ":", context.temp_allocator)
result := find_binary(paths, "no_such_binary_xyz")
testing.expect(t, result == "", "nonexistent binary should not be found")
}

320
findr/findr_test.odin Normal file
View File

@@ -0,0 +1,320 @@
package findr
import "core:os"
import "core:sort"
import "core:strings"
import "core:sys/linux"
import "core:testing"
// ============================================================================
// Gitignored file emission tests (emit ONLY gitignored files, descend everywhere)
// ============================================================================
@(test)
test_basic_gitignored :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n")
create_file(env, "repo/.env")
create_file(env, "repo/secrets.env")
create_file(env, "repo/normal.txt")
assert_output(t, env, nil, {}, {
"repo/.env", "repo/secrets.env",
})
}
@(test)
test_non_repo_not_scanned :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_dir(env, "norepo")
create_file(env, "norepo/.gitignore", "*.env\n")
create_file(env, "norepo/.env")
assert_output_empty(t, env, nil, {})
}
@(test)
test_negation_pattern :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n!prod.env\n")
create_file(env, "repo/.env")
create_file(env, "repo/secrets.env")
create_file(env, "repo/prod.env")
assert_output(t, env, nil, {}, {
"repo/.env", "repo/secrets.env",
})
}
@(test)
test_multiple_repos :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo1")
create_file(env, "repo1/.gitignore", "*.env\n")
create_file(env, "repo1/a.env")
create_git_repo(env, "repo2")
create_file(env, "repo2/.gitignore", "*.key\n")
create_file(env, "repo2/secret.key")
assert_output(t, env, nil, {}, {
"repo1/a.env", "repo2/secret.key",
})
}
@(test)
test_nested_repos :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "parent")
create_file(env, "parent/.gitignore", "*.env\n")
create_file(env, "parent/top.env")
create_git_repo(env, "parent/child")
create_file(env, "parent/child/.gitignore", "*.key\n")
create_file(env, "parent/child/api.key")
assert_output(t, env, nil, {}, {
"parent/top.env", "parent/child/api.key",
})
}
@(test)
test_nested_gitignore_read :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n")
create_dir(env, "repo/sub")
create_file(env, "repo/sub/.gitignore", "*.txt\n")
create_file(env, "repo/sub/secret.txt")
create_file(env, "repo/sub/.env")
assert_output(t, env, nil, {}, {
"repo/sub/secret.txt", "repo/sub/.env",
})
}
@(test)
test_nested_gitignore_negation :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.log\n")
create_dir(env, "repo/sub")
create_file(env, "repo/sub/.gitignore", "!important.log\n")
create_file(env, "repo/sub/important.log")
create_file(env, "repo/sub/debug.log")
assert_output(t, env, nil, {}, {
"repo/sub/debug.log",
})
}
@(test)
test_multisegment_pattern :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "build/output.txt\n")
create_dir(env, "repo/build")
create_file(env, "repo/build/output.txt")
create_file(env, "repo/build/other.txt")
create_file(env, "repo/output.txt")
assert_output(t, env, nil, {}, {
"repo/build/output.txt",
})
}
@(test)
test_no_gitignore_file :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.env")
assert_output_empty(t, env, nil, {})
}
@(test)
test_empty_gitignore :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "\n\n# comment\n\n")
create_file(env, "repo/.env")
assert_output_empty(t, env, nil, {})
}
@(test)
test_multiple_search_dirs :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "dir1/repo")
create_file(env, "dir1/repo/.gitignore", "*.env\n")
create_file(env, "dir1/repo/a.env")
create_file(env, "dir1/repo/normal.txt")
create_git_repo(env, "dir2/repo")
create_file(env, "dir2/repo/.gitignore", "*.env\n")
create_file(env, "dir2/repo/b.env")
dir1 := join_path(env.temp_dir, "dir1")
defer delete(dir1)
dir2 := join_path(env.temp_dir, "dir2")
defer delete(dir2)
results := make([dynamic]string)
defer {
for r in results {delete(r)}
delete(results)
}
opts := WalkOptions{}
thread_count := os.get_processor_core_count()
walk({dir1, dir2}, &results, opts, thread_count)
testing.expect_value(t, len(results), 2)
actual := make([dynamic]string, 0, len(results))
for r in results {
stripped := r
if strings.has_prefix(stripped, env.temp_dir) {
stripped = stripped[len(env.temp_dir):]
if len(stripped) > 0 && stripped[0] == '/' {
stripped = stripped[1:]
}
}
append(&actual, stripped)
}
defer delete(actual)
expected := []string{"dir1/repo/a.env", "dir2/repo/b.env"}
sort.quick_sort(actual[:])
sort.quick_sort(expected[:])
for i in 0 ..< len(expected) {
testing.expect_value(t, actual[i], expected[i])
}
}
// ============================================================================
// Ignored directory recursion tests
// ============================================================================
@(test)
test_ignored_dir_descended :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "secrets/\n")
create_dir(env, "repo/secrets")
create_file(env, "repo/secrets/.env")
create_file(env, "repo/secrets/api.key")
// Ignored dir's contents are emitted AND descended into
assert_output(t, env, nil, {}, {
"repo/secrets/", "repo/secrets/.env", "repo/secrets/api.key",
})
}
@(test)
test_nested_ignored_dir :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "build/\n")
create_dir(env, "repo/build")
create_dir(env, "repo/build/sub")
create_file(env, "repo/build/output.txt")
create_file(env, "repo/build/sub/deep.env")
assert_output(t, env, nil, {}, {
"repo/build/", "repo/build/output.txt",
"repo/build/sub/", "repo/build/sub/deep.env",
})
}
// ============================================================================
// Filter tests (excludes, pattern)
// ============================================================================
@(test)
test_excludes_prune_dirs :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n")
create_file(env, "repo/.env")
create_dir(env, "repo/vendor")
create_file(env, "repo/vendor/lib.env")
assert_output(t, env, nil,
{excludes = {"vendor"}},
{"repo/.env"},
)
}
@(test)
test_pattern_filters_results :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n*.key\n")
create_file(env, "repo/.env")
create_file(env, "repo/secrets.env")
create_file(env, "repo/master.key")
assert_output(t, env, nil,
{pattern = "\\.env$"},
{"repo/.env", "repo/secrets.env"},
)
}
// ============================================================================
// Special file type tests
// ============================================================================
@(test)
test_fifo_emitted :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n*.fifo\n")
fifo_path := join_path(env.temp_dir, "repo/test.fifo")
defer delete(fifo_path)
cpath := strings.clone_to_cstring(fifo_path)
defer delete(cpath)
linux.mknod(cpath, linux.S_IFIFO | linux.Mode{.IRUSR, .IWUSR}, 0)
assert_output(t, env, nil,
{pattern = "\\.fifo$"},
{"repo/test.fifo"},
)
}

88
findr/gitignore.odin Normal file
View File

@@ -0,0 +1,88 @@
package findr
import "core:strings"
Gitignore :: struct {
rules: [dynamic]Rule,
}
Rule :: struct {
pattern: GlobPattern,
negated: bool,
dir_only: bool,
}
Match :: enum {
None,
Ignored,
Unignored,
}
is_ignored :: proc(gi: ^Gitignore, path: string, is_dir: bool) -> bool {
return check_match(gi, path, is_dir) == .Ignored
}
check_match :: proc(gi: ^Gitignore, path: string, is_dir: bool) -> Match {
result := Match.None
for &rule in gi.rules {
if rule.dir_only && !is_dir do continue
if glob_match_compiled(&rule.pattern, path) {
result = rule.negated ? .Unignored : .Ignored
}
}
return result
}
parse :: proc(content: string) -> Gitignore {
gi: Gitignore
gi.rules = make([dynamic]Rule)
remaining := content
for {
line, ok := strings.split_lines_iterator(&remaining)
if !ok do break
s := strings.trim_space(line)
if len(s) == 0 do continue
if s[0] == '#' do continue
negated := false
if s[0] == '!' {
negated = true
s = s[1:]
}
if len(s) > 0 && s[0] == '\\' {
if len(s) > 1 && (s[1] == '#' || s[1] == '!') {
s = s[1:]
}
}
dir_only := false
if len(s) > 0 && s[len(s) - 1] == '/' {
dir_only = true
s = s[:len(s) - 1]
}
anchored := false
if len(s) > 0 && s[0] == '/' {
anchored = true
s = s[1:]
}
if len(s) == 0 do continue
gp := glob_compile(s, anchored)
append(&gi.rules, Rule{pattern = gp, negated = negated, dir_only = dir_only})
}
return gi
}
destroy :: proc(gi: ^Gitignore) {
for &rule in gi.rules {
glob_destroy(&rule.pattern)
}
delete(gi.rules)
}

118
findr/gitignore_test.odin Normal file
View File

@@ -0,0 +1,118 @@
package findr
import "core:testing"
@(test)
test_is_ignored_basic :: proc(t: ^testing.T) {
gi := parse("*.env\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, ".env", false), true)
testing.expect_value(t, is_ignored(&gi, "foo.env", false), true)
testing.expect_value(t, is_ignored(&gi, ".env.local", false), false)
testing.expect_value(t, is_ignored(&gi, "config.yaml", false), false)
}
@(test)
test_is_ignored_negation :: proc(t: ^testing.T) {
gi := parse("*.env\n!.env.production\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, ".env", false), true)
testing.expect_value(t, is_ignored(&gi, ".env.production", false), false)
}
@(test)
test_is_ignored_dir_only :: proc(t: ^testing.T) {
gi := parse("node_modules/\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, "node_modules", true), true)
testing.expect_value(t, is_ignored(&gi, "node_modules", false), false)
}
@(test)
test_is_ignored_anchored :: proc(t: ^testing.T) {
gi := parse("/secret.key\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, "secret.key", false), true)
}
@(test)
test_is_ignored_comments_skipped :: proc(t: ^testing.T) {
gi := parse("# this is a comment\n#another\n*.tmp\n")
defer destroy(&gi)
testing.expect_value(t, len(gi.rules), 1)
testing.expect_value(t, is_ignored(&gi, "file.tmp", false), true)
}
@(test)
test_is_ignored_blank_lines_skipped :: proc(t: ^testing.T) {
gi := parse("\n\n \n*.log\n\n")
defer destroy(&gi)
testing.expect_value(t, len(gi.rules), 1)
}
@(test)
test_is_ignored_last_match_wins :: proc(t: ^testing.T) {
gi := parse("*.env\n!*.env\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, ".env", false), false)
}
@(test)
test_is_ignored_no_rules :: proc(t: ^testing.T) {
gi := parse("")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, "anything", false), false)
}
@(test)
test_is_ignored_env_pattern :: proc(t: ^testing.T) {
gi := parse(".env*\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, ".env", false), true)
testing.expect_value(t, is_ignored(&gi, ".env.local", false), true)
testing.expect_value(t, is_ignored(&gi, ".envrc", false), true)
}
@(test)
test_is_ignored_globstar :: proc(t: ^testing.T) {
gi := parse("**/cache\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, "cache", false), true)
testing.expect_value(t, is_ignored(&gi, "foo/cache", false), true)
testing.expect_value(t, is_ignored(&gi, "foo/bar/cache", false), true)
}
@(test)
test_star_negation_subpath :: proc(t: ^testing.T) {
gi := parse("*\n!public/\n")
defer destroy(&gi)
// public dir itself is un-ignored
testing.expect_value(t, is_ignored(&gi, "public", true), false)
// children of public/ should still be ignored by *
testing.expect_value(t, is_ignored(&gi, "public/uuid-dir", true), true)
testing.expect_value(t, is_ignored(&gi, "public/uuid-dir/file.txt", false), true)
}
@(test)
test_is_ignored_hash_pattern :: proc(t: ^testing.T) {
gi := parse("\\#*\\#\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, "#foo#", false), true)
testing.expect_value(t, is_ignored(&gi, "#test#", false), true)
testing.expect_value(t, is_ignored(&gi, "AUTHORS", false), false)
testing.expect_value(t, is_ignored(&gi, "build.zig", false), false)
testing.expect_value(t, is_ignored(&gi, "ChangeLog", false), false)
}

203
findr/glob.odin Normal file
View File

@@ -0,0 +1,203 @@
package findr
Range :: struct {
lo: u8,
hi: u8,
}
Class_Data :: struct {
negated: bool,
ranges: [dynamic]Range,
}
Token_Kind :: enum u8 { Char, Star, Globstar, Question, Class }
Token :: struct {
kind: Token_Kind,
byte: u8,
class_idx: u16,
}
GlobPattern :: struct {
tokens: [dynamic]Token,
classes: [dynamic]Class_Data,
anchored: bool,
}
glob_compile :: proc(pattern: string, anchored: bool) -> GlobPattern {
gp: GlobPattern
gp.tokens = make([dynamic]Token)
gp.classes = make([dynamic]Class_Data)
gp.anchored = anchored
i := 0
for i < len(pattern) {
c := pattern[i]
if c == '*' {
if i + 1 < len(pattern) && pattern[i + 1] == '*' {
prev_slash := i == 0 || pattern[i - 1] == '/'
at_end := i + 2 >= len(pattern)
next_slash := !at_end && pattern[i + 2] == '/'
if prev_slash && (next_slash || at_end) {
append(&gp.tokens, Token{kind = .Globstar})
if next_slash {
i += 3
} else {
i += 2
}
} else {
append(&gp.tokens, Token{kind = .Star})
i += 2
}
} else {
append(&gp.tokens, Token{kind = .Star})
i += 1
}
} else if c == '?' {
append(&gp.tokens, Token{kind = .Question})
i += 1
} else if c == '[' {
i += 1
negated := false
if i < len(pattern) && pattern[i] == '!' {
negated = true
i += 1
}
ranges := make([dynamic]Range)
if i < len(pattern) && pattern[i] == ']' {
append(&ranges, Range{lo = ']', hi = ']'})
i += 1
}
for i < len(pattern) && pattern[i] != ']' {
if i + 2 < len(pattern) && pattern[i + 1] == '-' && pattern[i + 2] != ']' {
append(&ranges, Range{lo = pattern[i], hi = pattern[i + 2]})
i += 3
} else {
append(&ranges, Range{lo = pattern[i], hi = pattern[i]})
i += 1
}
}
if i < len(pattern) {
i += 1
}
class_idx := u16(len(gp.classes))
append(&gp.classes, Class_Data{negated = negated, ranges = ranges})
append(&gp.tokens, Token{kind = .Class, class_idx = class_idx})
} else if c == '\\' {
i += 1
if i < len(pattern) {
append(&gp.tokens, Token{kind = .Char, byte = pattern[i]})
i += 1
}
} else {
append(&gp.tokens, Token{kind = .Char, byte = c})
i += 1
}
}
return gp
}
match_tokens :: proc(tokens: []Token, classes: []Class_Data, ti: int, path: string, pi: int) -> bool {
if ti >= len(tokens) {
return pi == len(path)
}
tok := tokens[ti]
switch tok.kind {
case .Char:
if pi < len(path) && path[pi] == tok.byte {
return match_tokens(tokens, classes, ti + 1, path, pi + 1)
}
return false
case .Question:
if pi < len(path) && path[pi] != '/' {
return match_tokens(tokens, classes, ti + 1, path, pi + 1)
}
return false
case .Star:
max_end := pi
for max_end < len(path) && path[max_end] != '/' {
max_end += 1
}
for end := max_end; end >= pi; end -= 1 {
if match_tokens(tokens, classes, ti + 1, path, end) {
return true
}
}
return false
case .Globstar:
if ti + 1 >= len(tokens) {
return true
}
if match_tokens(tokens, classes, ti + 1, path, pi) {
return true
}
for end := pi + 1; end <= len(path); end += 1 {
if path[end - 1] == '/' {
if match_tokens(tokens, classes, ti + 1, path, end) {
return true
}
}
}
return false
case .Class:
if pi >= len(path) {
return false
}
cd := classes[tok.class_idx]
ch := path[pi]
in_range := false
for r in cd.ranges {
if ch >= r.lo && ch <= r.hi {
in_range = true
break
}
}
if in_range != cd.negated {
return match_tokens(tokens, classes, ti + 1, path, pi + 1)
}
return false
}
return false
}
glob_match_compiled :: proc(gp: ^GlobPattern, path: string) -> bool {
tokens := gp.tokens[:]
classes := gp.classes[:]
if gp.anchored {
return match_tokens(tokens, classes, 0, path, 0)
}
if match_tokens(tokens, classes, 0, path, 0) {
return true
}
for i := 1; i < len(path); i += 1 {
if path[i - 1] == '/' {
if match_tokens(tokens, classes, 0, path, i) {
return true
}
}
}
return false
}
glob_destroy :: proc(gp: ^GlobPattern) {
for &cd in gp.classes {
delete(cd.ranges)
}
delete(gp.classes)
delete(gp.tokens)
}

111
findr/glob_test.odin Normal file
View File

@@ -0,0 +1,111 @@
package findr
import "core:testing"
glob_match :: proc(pattern: string, path: string, anchored: bool) -> bool {
gp := glob_compile(pattern, anchored)
result := glob_match_compiled(&gp, path)
glob_destroy(&gp)
return result
}
@(test)
test_glob_simple :: proc(t: ^testing.T) {
testing.expect(t, glob_match("foo", "foo", false))
testing.expect(t, glob_match("foo", "bar/foo", false))
testing.expect(t, !glob_match("foo", "foobar", false))
testing.expect(t, !glob_match("foo", "foo/bar", false))
}
@(test)
test_glob_anchored :: proc(t: ^testing.T) {
testing.expect(t, glob_match("foo", "foo", true))
testing.expect(t, !glob_match("foo", "bar/foo", true))
testing.expect(t, !glob_match("foo", "foobar", true))
}
@(test)
test_glob_star :: proc(t: ^testing.T) {
testing.expect(t, glob_match("*.log", "test.log", false))
testing.expect(t, glob_match("*.log", ".log", false))
testing.expect(t, !glob_match("*.log", "test.txt", false))
testing.expect(t, !glob_match("*.log", "dir/test", false))
}
@(test)
test_glob_question :: proc(t: ^testing.T) {
testing.expect(t, glob_match("?.log", "a.log", false))
testing.expect(t, !glob_match("?.log", "ab.log", false))
testing.expect(t, !glob_match("?.log", ".log", false))
}
@(test)
test_glob_char_class :: proc(t: ^testing.T) {
testing.expect(t, glob_match("[abc].log", "a.log", false))
testing.expect(t, glob_match("[abc].log", "b.log", false))
testing.expect(t, !glob_match("[abc].log", "d.log", false))
}
@(test)
test_glob_negated_class :: proc(t: ^testing.T) {
testing.expect(t, glob_match("[!abc].log", "d.log", false))
testing.expect(t, !glob_match("[!abc].log", "a.log", false))
}
@(test)
test_glob_dot_literal :: proc(t: ^testing.T) {
testing.expect(t, glob_match(".env", ".env", false))
testing.expect(t, glob_match(".env", "dir/.env", false))
testing.expect(t, !glob_match(".env", "env", false))
testing.expect(t, !glob_match(".env", "x.env", false))
}
@(test)
test_glob_globstar_prefix :: proc(t: ^testing.T) {
testing.expect(t, glob_match("**/foo", "foo", false))
testing.expect(t, glob_match("**/foo", "a/b/foo", false))
testing.expect(t, !glob_match("**/foo", "foobar", false))
testing.expect(t, !glob_match("**/foo", "a/foobar", false))
}
@(test)
test_glob_globstar_suffix :: proc(t: ^testing.T) {
testing.expect(t, glob_match("abc/**", "abc/x", false))
testing.expect(t, glob_match("abc/**", "abc/x/y", false))
testing.expect(t, !glob_match("abc/**", "abc", false))
testing.expect(t, !glob_match("abc/**", "abcd/x", false))
}
@(test)
test_glob_globstar_middle :: proc(t: ^testing.T) {
testing.expect(t, glob_match("foo/**/bar", "foo/bar", false))
testing.expect(t, glob_match("foo/**/bar", "foo/x/bar", false))
testing.expect(t, !glob_match("foo/**/bar", "foo/barx", false))
testing.expect(t, !glob_match("foo/**/bar", "foo/x/y/baz", false))
}
@(test)
test_glob_backslash_escape :: proc(t: ^testing.T) {
testing.expect(t, glob_match("\\!foo", "!foo", false))
testing.expect(t, !glob_match("\\!foo", "foo", false))
}
@(test)
test_glob_hash_literal :: proc(t: ^testing.T) {
testing.expect(t, glob_match("#foo", "#foo", false))
testing.expect(t, !glob_match("#foo", "foo", false))
}
@(test)
test_glob_hash_pattern :: proc(t: ^testing.T) {
testing.expect(t, glob_match("#*#", "#test#", false))
testing.expect(t, glob_match("#*#", "##", false))
testing.expect(t, !glob_match("#*#", "test", false))
testing.expect(t, !glob_match("#*#", "#test", false))
}
@(test)
test_glob_empty :: proc(t: ^testing.T) {
testing.expect(t, glob_match("", "", false))
testing.expect(t, !glob_match("", "foo", false))
}

128
findr/repos.odin Normal file
View File

@@ -0,0 +1,128 @@
package findr
import "core:strings"
import "core:sync"
import "core:sys/linux"
import "core:thread"
RepoPool :: struct {
queue: [dynamic]string,
queue_mutex: sync.Mutex,
queue_sema: sync.Atomic_Sema,
results: ^[dynamic]string,
results_lock: sync.Mutex,
active: i64,
done: sync.One_Shot_Event,
threads: []^thread.Thread,
}
find_repos :: proc(roots: []string, results: ^[dynamic]string, thread_count: int) {
if len(roots) == 0 do return
pool := new(RepoPool)
pool.queue = make([dynamic]string)
pool.results = results
pool.active = i64(len(roots))
pool.threads = make([]^thread.Thread, thread_count)
for root in roots {
root_clone, _ := strings.clone(root)
append(&pool.queue, root_clone)
sync.atomic_sema_post(&pool.queue_sema)
}
for i in 0 ..< thread_count {
t := thread.create(repo_worker)
t.data = rawptr(pool)
t.init_context = context
thread.start(t)
pool.threads[i] = t
}
sync.one_shot_event_wait(&pool.done)
for _ in 0 ..< thread_count {
sync.atomic_sema_post(&pool.queue_sema)
}
for t in pool.threads {
thread.destroy(t)
}
delete(pool.threads)
for path in pool.queue {
delete(path)
}
delete(pool.queue)
free(pool)
}
repo_worker :: proc(t: ^thread.Thread) {
pool := cast(^RepoPool)t.data
for {
sync.atomic_sema_wait(&pool.queue_sema)
sync.mutex_lock(&pool.queue_mutex)
if len(pool.queue) == 0 {
sync.mutex_unlock(&pool.queue_mutex)
if sync.atomic_load_explicit(&pool.active, .Acquire) == 0 {
sync.one_shot_event_signal(&pool.done)
}
break
}
last := len(pool.queue) - 1
dir_path := pool.queue[last]
ordered_remove(&pool.queue, last)
sync.mutex_unlock(&pool.queue_mutex)
process_repo_dir(pool, dir_path)
delete(dir_path)
old := sync.atomic_sub_explicit(&pool.active, 1, .Release)
if old == 1 {
sync.one_shot_event_signal(&pool.done)
}
}
}
process_repo_dir :: proc(pool: ^RepoPool, dir_path: string) {
cpath := strings.clone_to_cstring(dir_path)
if cpath == nil do return
defer delete(cpath)
fd, open_err := linux.open(cpath, {.DIRECTORY, .CLOEXEC})
if open_err != .NONE do return
defer linux.close(fd)
if has_git_dir(fd) {
cloned, _ := strings.clone(dir_path)
sync.mutex_lock(&pool.results_lock)
append(pool.results, cloned)
sync.mutex_unlock(&pool.results_lock)
}
buf: [32 * 1024]u8
for {
n, errno := linux.getdents(fd, buf[:])
if n <= 0 || errno != .NONE do break
offs := 0
for d in linux.dirent_iterate_buf(buf[:n], &offs) {
name := linux.dirent_name(d)
if name == "." || name == ".." do continue
if name == ".git" do continue
if d.type == .DIR {
child_path := join_path(dir_path, name)
sync.atomic_add_explicit(&pool.active, 1, .Relaxed)
sync.mutex_lock(&pool.queue_mutex)
append(&pool.queue, child_path)
sync.mutex_unlock(&pool.queue_mutex)
sync.atomic_sema_post(&pool.queue_sema)
}
}
}
}

152
findr/test_env.odin Normal file
View File

@@ -0,0 +1,152 @@
package findr
import "core:fmt"
import "core:log"
import "core:os"
import "core:sort"
import "core:strings"
import "core:testing"
TestEnv :: struct {
temp_dir: string,
}
create_test_env :: proc() -> (env: TestEnv) {
tmp, err := os.mkdir_temp("", "findr-test-*", context.allocator)
if err != nil {
log.error("Failed to create temp dir:", err)
panic("Failed to create temp dir")
}
env.temp_dir = tmp
return
}
destroy_test_env :: proc(env: ^TestEnv) {
os.remove_all(env.temp_dir)
delete(env.temp_dir)
}
create_dir :: proc(env: TestEnv, path: string) {
full := join_path(env.temp_dir, path)
defer delete(full)
os.mkdir_all(full, os.Permissions_Default_Directory)
}
create_file :: proc(env: TestEnv, path: string, content: string = "") {
full := join_path(env.temp_dir, path)
defer delete(full)
dir_end := strings.last_index(full, "/")
if dir_end >= 0 {
dir_path := full[:dir_end]
os.mkdir_all(dir_path, os.Permissions_Default_Directory)
}
f, err := os.create(full)
if err != nil {
log.error("Failed to create file:", full, err)
return
}
if len(content) > 0 {
os.write_string(f, content)
}
os.close(f)
}
create_git_repo :: proc(env: TestEnv, path: string) {
sub := join_path(path, ".git")
defer delete(sub)
create_dir(env, sub)
}
assert_output :: proc(
t: ^testing.T,
env: TestEnv,
args: []string,
opts: WalkOptions,
expected: []string,
) {
results := collect_results(env, args, opts)
defer {
for r in results {delete(r)}
delete(results)
}
sorted_expected := make([dynamic]string, 0, len(expected))
for e in expected {append(&sorted_expected, e)}
defer delete(sorted_expected)
sorted_actual := make([dynamic]string, 0, len(results))
for a in results {append(&sorted_actual, a)}
defer delete(sorted_actual)
sort.quick_sort(sorted_expected[:])
sort.quick_sort(sorted_actual[:])
if len(sorted_expected) != len(sorted_actual) {
testing.fail(t)
log.error(
fmt.tprintf("Expected %d results, got %d", len(sorted_expected), len(sorted_actual)),
)
log.error("Expected:", sorted_expected[:])
log.error("Actual: ", sorted_actual[:])
return
}
for i in 0 ..< len(sorted_expected) {
if sorted_expected[i] != sorted_actual[i] {
testing.fail(t)
log.error(fmt.tprintf("Mismatch at index %d", i))
log.error("Expected:", sorted_expected[:])
log.error("Actual: ", sorted_actual[:])
return
}
}
}
assert_output_empty :: proc(
t: ^testing.T,
env: TestEnv,
args: []string,
opts: WalkOptions,
) {
results := collect_results(env, args, opts)
defer {
for r in results {delete(r)}
delete(results)
}
if len(results) > 0 {
testing.fail(t)
log.error(fmt.tprintf("Expected no results, got %d:", len(results)))
for r in results {
log.error(" ", r)
}
}
}
collect_results :: proc(env: TestEnv, args: []string, opts: WalkOptions) -> [dynamic]string {
results := make([dynamic]string)
full_args := make([dynamic]string, 0, len(args) + 1, context.temp_allocator)
append(&full_args, env.temp_dir)
for a in args {append(&full_args, a)}
thread_count := os.get_processor_core_count()
walk(full_args[:], &results, opts, thread_count)
for i in 0 ..< len(results) {
r := results[i]
if strings.has_prefix(r, env.temp_dir) {
stripped := r[len(env.temp_dir):]
if len(stripped) > 0 && stripped[0] == '/' {
stripped = stripped[1:]
}
new_r, _ := strings.clone(stripped)
delete(r)
results[i] = new_r
}
}
return results
}

458
findr/walker.odin Normal file
View File

@@ -0,0 +1,458 @@
package findr
import "core:bytes"
import "core:fmt"
import "core:os"
import "core:strings"
import "core:sync"
import "core:sync/chan"
import "core:sys/linux"
import "core:text/regex"
import "core:thread"
OUTPUT_BUF_SIZE :: 64 * 1024
WalkOptions :: struct {
pattern: string, // regex on basename; "" = match all
excludes: []string, // glob patterns to skip entirely
}
GIContext :: struct {
gi: ^Gitignore, // nil if this dir had no .gitignore
base_rel: string, // relative path from repo root to this dir
parent: ^GIContext, // parent context (nil if repo root)
}
WorkItem :: struct {
path: string, // absolute directory path
rel: string, // relative path from repo root ("" = root)
gi_ctx: ^GIContext, // gitignore chain (nil = outside any repo)
in_repo: bool, // true if inside a git repo
in_ignored: bool, // true if inside a gitignored directory
}
WalkerPool :: struct {
queue: [dynamic]WorkItem,
queue_mutex: sync.Mutex,
queue_sema: sync.Atomic_Sema,
result_chan: chan.Chan([]u8),
active: i64,
done: sync.One_Shot_Event,
threads: []^thread.Thread,
opts: WalkOptions,
pattern_re: regex.Regular_Expression,
has_pattern: bool,
exclude_gi: ^Gitignore,
all_contexts: [dynamic]^GIContext,
contexts_lock: sync.Mutex,
}
Collector_Data :: struct {
ch: chan.Chan([]u8),
results: ^[dynamic]string,
}
collect_worker :: proc(t: ^thread.Thread) {
data := cast(^Collector_Data)t.data
for {
batch := chan.recv(data.ch) or_break
defer delete(batch)
start := 0
for {
remaining: []u8
#no_bounds_check {remaining = batch[start:]}
idx := bytes.index_byte(remaining, '\n')
if idx < 0 do break
i := start + idx
if i > start {
segment: []u8
#no_bounds_check {segment = batch[start:i]}
s, _ := strings.clone(string(segment))
append(data.results, s)
}
start = i + 1
}
}
}
walk :: proc(roots: []string, results: ^[dynamic]string, opts: WalkOptions, thread_count: int) {
if len(roots) == 0 do return
ch, _ := chan.create(chan.Chan([]u8), max(2 * thread_count, 2), context.allocator)
defer chan.destroy(ch)
data := new(Collector_Data)
data.ch = ch
data.results = results
defer free(data)
collector := thread.create(collect_worker)
collector.data = rawptr(data)
collector.init_context = context
thread.start(collector)
pool := new(WalkerPool)
pool.queue = make([dynamic]WorkItem)
pool.result_chan = ch
pool.active = i64(len(roots))
pool.threads = make([]^thread.Thread, thread_count)
pool.all_contexts = make([dynamic]^GIContext)
pool.opts = opts
pool.exclude_gi = nil
pool.has_pattern = false
if len(opts.pattern) > 0 {
re, err := regex.create(opts.pattern, {regex.Flag.No_Capture})
if err == nil {
pool.pattern_re = re
pool.has_pattern = true
}
}
if len(opts.excludes) > 0 {
sb: strings.Builder
strings.builder_init(&sb)
for ex in opts.excludes {
fmt.sbprintf(&sb, "%s\n", ex)
}
content := strings.to_string(sb)
pool.exclude_gi = new(Gitignore)
pool.exclude_gi^ = parse(content)
strings.builder_destroy(&sb)
}
for root in roots {
root_clone, _ := strings.clone(root)
append(&pool.queue, WorkItem{path = root_clone})
sync.atomic_sema_post(&pool.queue_sema)
}
for i in 0 ..< thread_count {
t := thread.create(walk_worker)
t.data = rawptr(pool)
t.init_context = context
thread.start(t)
pool.threads[i] = t
}
sync.one_shot_event_wait(&pool.done)
for _ in 0 ..< thread_count {
sync.atomic_sema_post(&pool.queue_sema)
}
for t in pool.threads {
thread.destroy(t)
}
delete(pool.threads)
for item in pool.queue {
delete(item.path)
if len(item.rel) > 0 {delete(item.rel)}
}
delete(pool.queue)
for ctx in pool.all_contexts {
if ctx.gi != nil {
destroy(ctx.gi)
free(ctx.gi)
}
if len(ctx.base_rel) > 0 {
delete(ctx.base_rel)
}
free(ctx)
}
delete(pool.all_contexts)
if pool.has_pattern {
regex.destroy(pool.pattern_re)
}
if pool.exclude_gi != nil {
destroy(pool.exclude_gi)
free(pool.exclude_gi)
}
free(pool)
chan.close(ch)
thread.join(collector)
thread.destroy(collector)
}
flush_buf :: proc(ch: chan.Chan([]u8), local: ^[dynamic]u8) {
if len(local) == 0 do return
batch := local[:]
local^ = make([dynamic]u8, 0, OUTPUT_BUF_SIZE)
chan.send(ch, batch)
}
append_path :: proc(buf: ^[dynamic]u8, parent, name: string, trailing_slash: bool) {
need_sep := len(parent) > 0 && parent[len(parent) - 1] != '/'
size := len(parent) + len(name) + 1
if need_sep do size += 1
if trailing_slash do size += 1
old_len := len(buf)
reserve(buf, old_len + size)
resize(buf, old_len + size)
pos := old_len
pos += copy(buf[pos:], parent)
if need_sep {buf[pos] = '/'; pos += 1}
pos += copy(buf[pos:], name)
if trailing_slash {buf[pos] = '/'; pos += 1}
buf[pos] = '\n'
}
walk_worker :: proc(t: ^thread.Thread) {
pool := cast(^WalkerPool)t.data
local_buf := make([dynamic]u8, 0, OUTPUT_BUF_SIZE)
defer {
if len(local_buf) > 0 {
flush_buf(pool.result_chan, &local_buf)
}
delete(local_buf)
}
for {
sync.atomic_sema_wait(&pool.queue_sema)
sync.mutex_lock(&pool.queue_mutex)
if len(pool.queue) == 0 {
sync.mutex_unlock(&pool.queue_mutex)
if sync.atomic_load_explicit(&pool.active, .Acquire) == 0 {
sync.one_shot_event_signal(&pool.done)
}
break
}
last := len(pool.queue) - 1
item := pool.queue[last]
ordered_remove(&pool.queue, last)
sync.mutex_unlock(&pool.queue_mutex)
process_dir(pool, item, &local_buf)
delete(item.path)
if len(item.rel) > 0 {delete(item.rel)}
if len(local_buf) >= OUTPUT_BUF_SIZE {
flush_buf(pool.result_chan, &local_buf)
}
old := sync.atomic_sub_explicit(&pool.active, 1, .Release)
if old == 1 {
sync.one_shot_event_signal(&pool.done)
}
}
}
process_dir :: proc(pool: ^WalkerPool, item: WorkItem, local_buf: ^[dynamic]u8) {
dir_path := item.path
cpath := strings.clone_to_cstring(dir_path)
if cpath == nil do return
defer delete(cpath)
fd, open_err := linux.open(cpath, {.DIRECTORY, .CLOEXEC})
if open_err != .NONE do return
defer linux.close(fd)
has_git := false
if !item.in_ignored {
has_git = has_git_dir(fd)
}
gi_ctx := item.gi_ctx
rel := item.rel
if has_git {
gi_ctx = nil
rel = ""
}
child_in_repo := has_git || item.in_repo
gi: ^Gitignore = nil
if !item.in_ignored {
gi = load_ignore_patterns(dir_path, child_in_repo)
}
if gi != nil {
new_ctx := new(GIContext)
new_ctx.gi = gi
if len(rel) > 0 {
new_ctx.base_rel, _ = strings.clone(rel)
}
new_ctx.parent = gi_ctx
sync.mutex_lock(&pool.contexts_lock)
append(&pool.all_contexts, new_ctx)
sync.mutex_unlock(&pool.contexts_lock)
gi_ctx = new_ctx
}
buf: [32 * 1024]u8
rel_buf: [4096]u8
for {
n, errno := linux.getdents(fd, buf[:])
if n <= 0 || errno != .NONE do break
offs := 0
for d in linux.dirent_iterate_buf(buf[:n], &offs) {
name := linux.dirent_name(d)
if name == "." || name == ".." do continue
if name == ".git" do continue
is_dir := d.type == .DIR
is_nondir := d.type != .DIR
if pool.exclude_gi != nil && is_ignored(pool.exclude_gi, name, is_dir) {
continue
}
entry_rel := build_rel(rel_buf[:], rel, name)
ignored := false
if item.in_ignored {
ignored = true
} else if gi_ctx != nil {
ignored = check_chain(gi_ctx, entry_rel, is_dir)
}
if is_dir {
if ignored && matches_pattern(pool, name) {
append_path(local_buf, dir_path, name, true)
}
child_rel, _ := strings.clone(entry_rel)
child_path := join_path(dir_path, name)
push_work(
pool,
WorkItem {
path = child_path,
rel = child_rel,
gi_ctx = gi_ctx,
in_repo = child_in_repo,
in_ignored = ignored,
},
)
} else if is_nondir {
if ignored && matches_pattern(pool, name) {
append_path(local_buf, dir_path, name, false)
}
}
}
}
}
check_chain :: proc(ctx: ^GIContext, entry_rel: string, is_dir: bool) -> bool {
c := ctx
for c != nil {
if c.gi != nil {
rel := relative_to(entry_rel, c.base_rel)
match := check_match(c.gi, rel, is_dir)
if match != .None {
return match == .Ignored
}
}
c = c.parent
}
return false
}
relative_to :: proc(entry_rel, base_rel: string) -> string {
if len(base_rel) == 0 do return entry_rel
prefix_len := len(base_rel)
if len(entry_rel) > prefix_len &&
entry_rel[prefix_len] == '/' &&
strings.has_prefix(entry_rel, base_rel) {
return entry_rel[prefix_len + 1:]
}
return entry_rel
}
build_rel :: proc(buf: []u8, rel, name: string) -> string {
if len(rel) == 0 do return name
pos := copy(buf, rel)
if pos < len(buf) {
buf[pos] = '/'
pos += 1
pos += copy(buf[pos:], name)
}
return string(buf[:pos])
}
matches_pattern :: proc(pool: ^WalkerPool, name: string) -> bool {
if !pool.has_pattern do return true
cap, ok := regex.match(pool.pattern_re, name)
regex.destroy(cap)
return ok
}
push_work :: proc(pool: ^WalkerPool, item: WorkItem) {
sync.atomic_add_explicit(&pool.active, 1, .Relaxed)
sync.mutex_lock(&pool.queue_mutex)
append(&pool.queue, item)
sync.mutex_unlock(&pool.queue_mutex)
sync.atomic_sema_post(&pool.queue_sema)
}
has_git_dir :: proc(fd: linux.Fd) -> bool {
git_fd, err := linux.openat(fd, ".git", {.DIRECTORY, .CLOEXEC})
if err == .NONE {
linux.close(git_fd)
return true
}
return false
}
load_ignore_patterns :: proc(dir_path: string, in_repo: bool) -> ^Gitignore {
has_patterns := false
sb: strings.Builder
strings.builder_init(&sb)
defer strings.builder_destroy(&sb)
if in_repo {
gi_path := join_path(dir_path, ".gitignore")
data, err := os.read_entire_file_from_path(gi_path, context.allocator)
delete(gi_path)
if err == .NONE {
fmt.sbprintf(&sb, "%s", string(data))
delete(data)
has_patterns = true
}
}
ig_path := join_path(dir_path, ".ignore")
idata, ierr := os.read_entire_file_from_path(ig_path, context.allocator)
delete(ig_path)
if ierr == .NONE {
fmt.sbprintf(&sb, "%s", string(idata))
delete(idata)
has_patterns = true
}
if !has_patterns do return nil
content := strings.to_string(sb)
gi := new(Gitignore)
gi^ = parse(content)
return gi
}
join_path :: proc(parent, child: string) -> string {
need_sep := len(parent) == 0 || parent[len(parent) - 1] != '/'
total := len(parent) + len(child)
if need_sep do total += 1
buf := make([]u8, total, context.allocator)
pos := copy(buf, parent)
if need_sep {
buf[pos] = '/'
pos += 1
}
copy(buf[pos:], child)
return string(buf)
}

View File

@@ -66,7 +66,7 @@
packages.default = pkgs.stdenv.mkDerivation rec {
pname = "envr";
version = "0.2.0";
version = "0.3.0";
src = ./.;
nativeBuildInputs = [
@@ -79,6 +79,13 @@
mysqlite
];
doCheck = true;
checkPhase = ''
runHook preCheck
odin test . -all-packages
runHook postCheck
'';
buildPhase = ''
runHook preBuild
echo '${version}' > version.txt
@@ -95,7 +102,6 @@
devShells.default = pkgs.mkShell {
buildInputs = with pkgs; [
fd
nushell
libsodium
@@ -106,6 +112,10 @@
# Build tools
zip
# Helper tools
delta
hyperfine
# IDE
unstable.helix
typescript-language-server

View File

@@ -1,14 +1,31 @@
package main
import "core:bufio"
import "core:fmt"
import "core:mem"
import "core:os"
main :: proc() {
when ODIN_DEBUG {
heap_track: mem.Tracking_Allocator
mem.tracking_allocator_init(&heap_track, context.allocator)
defer mem.tracking_allocator_destroy(&heap_track)
defer if len(heap_track.allocation_map) > 0 {
for _, leak in heap_track.allocation_map {
fmt.eprintf("LEAK: %v leaked %m\n", leak.location, leak.size)
}
}
context.allocator = mem.tracking_allocator(&heap_track)
temp_track: mem.Tracking_Allocator
mem.tracking_allocator_init(&temp_track, context.temp_allocator)
defer mem.tracking_allocator_destroy(&temp_track)
context.temp_allocator = mem.tracking_allocator(&temp_track)
}
defer free_all(context.temp_allocator)
cmd, ok := parse_args(os.args, os.to_writer(os.stdout), os.to_writer(os.stderr))
defer bufio.writer_flush(cmd.out_buf)
defer delete_command(&cmd) // delete flushes automatically
if !ok {
return
}
@@ -18,8 +35,6 @@ main :: proc() {
cmd_init(&cmd)
case "version":
cmd_version(&cmd)
case "deps":
cmd_deps(&cmd)
case "list":
cmd_list(&cmd)
case "backup", "add":

View File

@@ -36,7 +36,7 @@ multi_select :: proc(
return
}
selected = make([dynamic]bool, 0, len(options))
selected = make([dynamic]bool, len(options))
cursor: int = 0
scroll_offset: int = 0

127
scan.odin
View File

@@ -1,137 +1,21 @@
package main
import "core:fmt"
import "core:os"
import "core:strings"
import "core:sync"
import "core:terminal"
fd_counter: sync.Atomic_Mutex
fd_seq: int
import "findr"
// Caller is responsible for freeing paths
scan_path :: proc(search_path: string, cfg: Config) -> (paths: [dynamic]string, ok: bool) {
if terminal.is_terminal(os.stdout) {
fmt.printf("Searching for all files in \"%s\"...\n", search_path)
opts := findr.WalkOptions {
pattern = cfg.ScanConfig.Matcher,
excludes = cfg.ScanConfig.Exclude[:],
}
all_files, all_ok := run_fd(build_fd_args(search_path, cfg, true))
if !all_ok {
return
}
if terminal.is_terminal(os.stdout) {
fmt.printf("Search for unignored fies in \"%s\"...\n", search_path)
}
unignored_files, unignored_ok := run_fd(build_fd_args(search_path, cfg, false))
if !unignored_ok {
return
}
unignored_set := make(map[string]bool, len(unignored_files), context.temp_allocator)
for file in unignored_files {
unignored_set[file] = true
}
for file in all_files {
if !(file in unignored_set) {
append(&paths, file)
}
}
findr.walk({search_path}, &paths, opts, os.get_processor_core_count())
ok = true
return
}
@(private = "file")
build_fd_args :: proc(search_path: string, cfg: Config, include_ignored: bool) -> []string {
args_len := 3 + 2 * len(cfg.ScanConfig.Exclude) + 2
args := make([dynamic]string, 0, args_len, context.temp_allocator)
append(&args, "fd")
append(&args, "-a")
append(&args, cfg.ScanConfig.Matcher)
for exclude in cfg.ScanConfig.Exclude {
append(&args, "-E")
append(&args, exclude)
}
if include_ignored {
append(&args, "-HI")
} else {
append(&args, "-H")
}
append(&args, search_path)
return args[:]
}
run_fd :: proc(args: []string) -> (lines: []string, ok: bool) {
tmp_path := next_fd_tmp_path()
tmp_file, tmp_err := os.open(tmp_path, os.O_CREATE | os.O_WRONLY | os.O_TRUNC)
if tmp_err != nil {
// TODO: Log a message here
return
}
desc := os.Process_Desc {
command = args,
stdout = tmp_file,
stderr = nil,
}
p, start_err := os.process_start(desc)
os.close(tmp_file)
if start_err != nil {
os.remove(tmp_path)
return
}
state, wait_err := os.process_wait(p)
if wait_err != nil || state.exit_code != 0 {
os.remove(tmp_path)
return
}
data, read_err := os.read_entire_file_from_path(tmp_path, context.temp_allocator)
os.remove(tmp_path)
if read_err != nil {
return
}
output := string(data)
output = strings.trim_space(output)
if len(output) == 0 {
ok = true
return
}
raw_lines := strings.split(output, "\n", context.temp_allocator)
result := make([dynamic]string, 0, len(raw_lines), context.temp_allocator)
for line in raw_lines {
trimmed := strings.trim_space(line)
if len(trimmed) > 0 {
append(&result, trimmed)
}
}
return result[:], true
}
@(private = "file")
next_fd_tmp_path :: proc() -> string {
sync.atomic_mutex_lock(&fd_counter)
n := fd_seq
fd_seq += 1
sync.atomic_mutex_unlock(&fd_counter)
return fmt.tprintf("/tmp/envr-fd-%d-%d", os.get_pid(), n)
}
cant_scan :: proc(feats: AvailableFeatures) -> bool {
return Feature.Fd not_in feats
}
find_unbacked :: proc(local_files: []string, db_files: []EnvFile) -> []string {
// Lives until the end of the function
backed_set := make(map[string]bool, len(db_files), context.temp_allocator)
for file in db_files {
backed_set[file.Path] = true
@@ -145,4 +29,3 @@ find_unbacked :: proc(local_files: []string, db_files: []EnvFile) -> []string {
}
return unbacked[:]
}

View File

@@ -3,14 +3,10 @@ package main
import "core:fmt"
import "core:os"
import "core:path/filepath"
import "core:strings"
import "core:testing"
@(test)
test_scan_path_finds_gitignored_env_files :: proc(t: ^testing.T) {
feats := check_features()
testing.expect(t, cant_scan(feats) == false)
base := fmt.tprintf("/tmp/envr-scan-test-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
@@ -42,7 +38,12 @@ test_scan_path_finds_gitignored_env_files :: proc(t: ^testing.T) {
}
results, ok := scan_path(base, cfg)
defer delete(results)
defer {
for path in results {
delete(path)
}
delete(results)
}
testing.expect(t, ok, "scan_path should succeed")
found_env := false
@@ -69,9 +70,6 @@ test_scan_path_finds_gitignored_env_files :: proc(t: ^testing.T) {
@(test)
test_scan_path_empty_dir :: proc(t: ^testing.T) {
feats := check_features()
testing.expect(t, cant_scan(feats) == false)
base := fmt.tprintf("/tmp/envr-scan-empty-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
@@ -85,12 +83,3 @@ test_scan_path_empty_dir :: proc(t: ^testing.T) {
testing.expect(t, ok, "scan_path should succeed")
testing.expect(t, len(results) == 0, fmt.tprintf("expected 0 results, got %d", len(results)))
}
@(test)
test_scan_meets_expectations :: proc(t: ^testing.T) {
testing.expect(t, cant_scan({}), "no features should mean can't scan")
testing.expect(t, cant_scan({.Git}), "Git alone should mean can't scan")
testing.expect(t, !cant_scan({.Fd}), "having Fd should mean can scan")
testing.expect(t, !cant_scan({.Fd, .Git}), "both Fd and Git should mean can scan")
}

View File

@@ -12,24 +12,6 @@ Ed25519Keypair :: struct {
Private: [32]u8,
}
read_wire_string :: proc(data: []u8, offset: ^int) -> (s: string, ok: bool) {
if offset^ + 4 > len(data) {
return
}
length := u32(data[offset^]) << 24 | u32(data[offset^ + 1]) << 16 |
u32(data[offset^ + 2]) << 8 | u32(data[offset^ + 3])
offset^ += 4
if offset^ + int(length) > len(data) {
return
}
s = string(data[offset^ : offset^ + int(length)])
offset^ += int(length)
ok = true
return
}
parse_ssh_public_key :: proc(pub_path: string) -> (pub: [32]u8, ok: bool) {
data, err := os.read_entire_file_from_path(pub_path, context.temp_allocator)
if err != nil {
@@ -253,3 +235,21 @@ is_encrypted_key :: proc(priv_path: string) -> bool {
return ciphername != "none"
}
read_wire_string :: proc(data: []u8, offset: ^int) -> (s: string, ok: bool) {
if offset^ + 4 > len(data) {
return
}
length := u32(data[offset^]) << 24 | u32(data[offset^ + 1]) << 16 |
u32(data[offset^ + 2]) << 8 | u32(data[offset^ + 3])
offset^ += 4
if offset^ + int(length) > len(data) {
return
}
s = string(data[offset^ : offset^ + int(length)])
offset^ += int(length)
ok = true
return
}

View File

@@ -4,9 +4,10 @@ import "core:encoding/json"
import "core:fmt"
import "core:io"
import "core:strings"
import "core:terminal/ansi"
render_table :: proc(w: io.Writer, headers: []string, rows: [][]string) {
col_widths := make([dynamic]int, 0, len(headers))
col_widths := make([dynamic]int, 0, len(headers), context.temp_allocator)
for i in 0 ..< len(headers) {
append(&col_widths, strings.rune_count(headers[i]))
}
@@ -20,11 +21,14 @@ render_table :: proc(w: io.Writer, headers: []string, rows: [][]string) {
}
b: strings.Builder
strings.builder_init(&b)
defer strings.builder_destroy(&b)
defer delete(col_widths)
strings.builder_init(&b, context.temp_allocator)
hline :: proc(w: io.Writer, b: ^strings.Builder, left, mid, right: string, widths: [dynamic]int) {
hline :: proc(
w: io.Writer,
b: ^strings.Builder,
left, mid, right: string,
widths: [dynamic]int,
) {
strings.write_string(b, left)
for i in 0 ..< len(widths) {
for _ in 0 ..< widths[i] + 2 {
@@ -42,14 +46,38 @@ render_table :: proc(w: io.Writer, headers: []string, rows: [][]string) {
hline(w, &b, "\u250c", "\u252c", "\u2510", col_widths)
cell :: proc(b: ^strings.Builder, s: string, width: int) {
extra := len(s) - strings.rune_count(s)
fmt.sbprintf(b, " %-*s \u2502", width + extra, s)
cell :: proc(b: ^strings.Builder, s: string, width: int, color: string = "", center := false) {
before: int
after: int
total_pad := width - strings.rune_count(s)
if center {
before = total_pad / 2
after = total_pad - before
} else {
before = 0
after = total_pad
}
fmt.sbprintf(
b,
" %s%s%s%*s%s%*s%s \u2502",
ansi.CSI,
color,
ansi.SGR,
before,
"",
s,
after,
"",
ansi.CSI + ansi.RESET + ansi.SGR,
)
}
strings.write_string(&b, "\u2502")
for i in 0 ..< len(headers) {
cell(&b, headers[i], col_widths[i])
cell(&b, headers[i], col_widths[i], ansi.FG_BRIGHT_GREEN, true)
}
fmt.wprintf(w, "%s\n", strings.to_string(b), flush = false)
strings.builder_reset(&b)

View File

@@ -3,6 +3,7 @@ package main
import "core:encoding/json"
import "core:fmt"
import "core:strings"
import "core:terminal/ansi"
import "core:testing"
@(test)
@@ -116,13 +117,30 @@ test_render_table_normal :: proc(t: ^testing.T) {
output := strings.to_string(b)
expected := `┌──────┬─────────────────────────┐
│ Name │ Path │
├──────┼─────────────────────────┤
│ foo │ /home/user/.env │
│ bar │ /home/user/project/.env │
───────────────────────────────
`
g := ansi.CSI + ansi.FG_BRIGHT_GREEN + ansi.SGR
r := ANSI_RESET
n := ansi.CSI + ansi.SGR
expected := fmt.tprintf(
"┌───────────────────────────────┐\n" +
"│ %sName%s │ %s Path %s │\n" +
"├──────┼─────────────────────────┤\n" +
"│ %sfoo %s │ %s/home/user/.env %s │\n" +
"│ %sbar %s │ %s/home/user/project/.env%s │\n" +
"└──────┴─────────────────────────┘\n",
g,
r,
g,
r,
n,
r,
n,
r,
n,
r,
n,
r,
)
testing.expect(
t,
output == expected,
@@ -148,11 +166,17 @@ test_render_table_empty :: proc(t: ^testing.T) {
output := strings.to_string(b)
expected := `┌──────┐
│ Name │
├──────┤
└──────┘
`
g := ansi.CSI + ansi.FG_BRIGHT_GREEN + ansi.SGR
r := ANSI_RESET
expected := fmt.tprintf(
"┌──────┐\n" +
"│ %sName%s │\n" +
"├──────┤\n" +
"└──────┘\n",
g,
r,
)
testing.expect(
t,
output == expected,
@@ -178,13 +202,30 @@ test_render_table_unicode :: proc(t: ^testing.T) {
output := strings.to_string(b)
expected := `┌─────────────┬────────┐
│ Status │ Detail │
├─────────────┼────────┤
│ ✓ Available │ ok │
│ ✗ Missing │ fail │
─────────────────────
`
g := ansi.CSI + ansi.FG_BRIGHT_GREEN + ansi.SGR
r := ANSI_RESET
n := ansi.CSI + ansi.SGR
expected := fmt.tprintf(
"┌─────────────────────┐\n" +
"│ %s Status %s │ %sDetail%s │\n" +
"├─────────────┼────────┤\n" +
"│ %s✓ Available%s │ %sok %s │\n" +
"│ %s✗ Missing %s │ %sfail %s │\n" +
"└─────────────┴────────┘\n",
g,
r,
g,
r,
n,
r,
n,
r,
n,
r,
n,
r,
)
testing.expect(
t,
output == expected,